Monthly Archives: November 2018

Obfuscated Command Line Detection Using Machine Learning

This blog post presents a machine learning (ML) approach to solving an emerging security problem: detecting obfuscated Windows command line invocations on endpoints. We start out with an introduction to this relatively new threat capability, and then discuss how such problems have traditionally been handled. We then describe a machine learning approach to solving this problem and point out how ML vastly simplifies development and maintenance of a robust obfuscation detector. Finally, we present the results obtained using two different ML techniques and compare the benefits of each.

Introduction

Malicious actors are increasingly “living off the land,” using built-in utilities such as PowerShell and the Windows Command Processor (cmd.exe) as part of their infection workflow in an effort to minimize the chance of detection and bypass whitelisting defense strategies. The release of new obfuscation tools makes detection of these threats even more difficult by adding a layer of indirection between the visible syntax and the final behavior of the command. For example, Invoke-Obfuscation and Invoke-DOSfuscation are two recently released tools that automate the obfuscation of Powershell and Windows command lines respectively.

The traditional pattern matching and rule-based approaches for detecting obfuscation are difficult to develop and generalize, and can pose a huge maintenance headache for defenders. We will show how using ML techniques can address this problem.

Detecting obfuscated command lines is a very useful technique because it allows defenders to reduce the data they must review by providing a strong filter for possibly malicious activity. While there are some examples of “legitimate” obfuscation in the wild, in the overwhelming majority of cases, the presence of obfuscation generally serves as a signal for malicious intent.

Background

There has been a long history of obfuscation being employed to hide the presence of malware, ranging from encryption of malicious payloads (starting with the Cascade virus) and obfuscation of strings, to JavaScript obfuscation. The purpose of obfuscation is two-fold:

  • Make it harder to find patterns in executable code, strings or scripts that can easily be detected by defensive software.
  • Make it harder for reverse engineers and analysts to decipher and fully understand what the malware is doing.

In that sense, command line obfuscation is not a new problem – it is just that the target of obfuscation (the Windows Command Processor) is relatively new. The recent release of tools such as Invoke-Obfuscation (for PowerShell) and Invoke-DOSfuscation (for cmd.exe) have demonstrated just how flexible these commands are, and how even incredibly complex obfuscation will still run commands effectively.

There are two categorical axes in the space of obfuscated vs. non-obfuscated command lines: simple/complex and clear/obfuscated (see Figure 1 and Figure 2). For this discussion “simple” means generally short and relatively uncomplicated, but can still contain obfuscation, while “complex” means long, complicated strings that may or may not be obfuscated. Thus, the simple/complex axis is orthogonal to obfuscated/unobfuscated. The interplay of these two axes produce many boundary cases where simple heuristics to detect if a script is obfuscated (e.g. length of a command) will produce false positives on unobfuscated samples. The flexibility of the command line processor makes classification a difficult task from an ML perspective.


Figure 1: Dimensions of obfuscation


Figure 2: Examples of weak and strong obfuscation

Traditional Obfuscation Detection

Traditional obfuscation detection can be split into three approaches. One approach is to write a large number of complex regular expressions to match the most commonly abused syntax of the Windows command line. Figure 3 shows one such regular expression that attempts to match ampersand chaining with a call command, a common pattern seen in obfuscation. Figure 4 shows an example command sequence this regex is designed to detect.


Figure 3: A common obfuscation pattern captured as a regular expression


Figure 4: A common obfuscation pattern (calling echo in obfuscated fashion in this example)

There are two problems with this approach. First, it is virtually impossible to develop regular expressions to cover every possible abuse of the command line. The flexibility of the command line results in a non-regular language, which is feasible yet impractical to express using regular expressions. A second issue with this approach is that even if a regular expression exists for the technique a malicious sample is using, a determined attacker can make minor modifications to avoid the regular expression. Figure 5 shows a minor modification to the sequence in Figure 4, which avoids the regex detection.


Figure 5: A minor change (extra carets) to an obfuscated command line that breaks the regular expression in Figure 3

The second approach, which is closer to an ML approach, involves writing complex if-then rules. However, these rules are hard to derive, are complex to verify, and pose a significant maintenance burden as authors evolve to escape detection by such rules. Figure 6 shows one such if-then rule.


Figure 6: An if-then rule that *may* indicate obfuscation (notice how loose this rule is, and how false positives are likely)

A third approach is to combine regular expressions and if-then rules. This greatly complicates the development and maintenance burden, and still suffers from the same weaknesses that make the first two approaches fragile. Figure 7 shows an example of an if-then rule with regular expressions. Clearly, it is easy to appreciate how burdensome it is to generate, test, maintain and determine the efficacy of such rules.


Figure 7: A combination of an if-then rule with regular expressions to detect obfuscation (a real hand-built obfuscation detector would consist of tens or hundreds of rules and still have gaps in its detection)

The ML Approach – Moving Beyond Pattern Matching and Rules

Using ML simplifies the solution to these problems. We will illustrate two ML approaches: a feature-based approach and a feature-less end-to-end approach.

There are some ML techniques that can work with any kind of raw data (provided it is numeric), and neural networks are a prime example. Most other ML algorithms require the modeler to extract pertinent information, called features, from raw data before they are fed into the algorithm. Some examples of this latter type are tree-based algorithms, which we will also look at in this blog (we described the structure and uses of Tree-Based algorithms in a previous blog post, where we used a Gradient-Boosted Tree-Based Model).

ML Basics – Neural Networks

Neural networks are a type of ML algorithm that have recently become very popular and consist of a series of elements called neurons. A neuron is essentially an element that takes a set of inputs, computes a weighted sum of these inputs, and then feeds the sum into a non-linear function. It has been shown that a relatively shallow network of neurons can approximate any continuous mapping between input and output. The specific type of neural network we used for this research is what is called a Convolutional Neural Network (CNN), which was developed primarily for computer vision applications, but has also found success in other domains including natural language processing. One of the main benefits of a neural network is that it can be trained without having to manually engineer features.

Featureless ML

While neural networks can be used with feature data, one of the attractions of this approach is that it can work with raw data (converted into numeric form) without doing any feature design or extraction. The first step in the model is converting text data into numeric form. We used a character-based encoding where each character type was encoded by a real valued number. The value was automatically derived during training and conveys semantic information about the relationships between characters as they apply to cmd.exe syntax.

Feature-Based ML

We also experimented with hand-engineered features and a Gradient Boosted Decision Tree algorithm. The features developed for this model were largely statistical in nature – derived from the presence and frequency of character sets and keywords. For example, the presence of dozens of ‘%’ characters or long, contiguous strings might contribute to detecting potential obfuscation. While any single feature will not perfectly separate the two classes, a combination of features as present in a tree-based model can learn flexible patterns in the data. The expectation is that those patterns are robust and can generalize to future obfuscation variants.

Data and Experiments

To develop our models, we collected non-obfuscated data from tens of thousands of endpoint events and generated obfuscated data using a variety of methods in Invoke-DOSfuscation. We developed our models using roughly 80 percent of the data as training data, and tested them on the remaining 20 percent. We ensured that our train-test split was stratified. For featureless ML (i.e. neural networks), we simply input Unicode code points into the first layer of the CNN model. The first layer converts the code point into semantically meaningful numerical representations (called embeddings) before feeding it into the rest of the neural network.

For the Gradient Boosted Tree method, we generated a number of features from the raw command lines. The following are some of them:

  • Length of the command line
  • The number of carets in the command line
  • The count of pipe symbols
  • The fraction of white space in the command line
  • The fraction of special characters
  • Entropy of the string
  • The frequency of the strings “cmd” and “power” in the command line

While each of these features individually is a weak signal and could not possibly be a good discriminator on its own, a flexible classifier such as a Gradient Boosted Tree – trained on sufficient data with these features – is able to classify obfuscated and non-obfuscated command lines in spite of the aforementioned difficulties.

Results

Evaluated against our test set, we were able to get nearly identical results from our Gradient Boosted Tree and neural network models.

The results for the GBT model were near perfect with metrics such as F1-score, precision, and recall all being close to 1.0. The CNN model was slightly less accurate.

While we certainly do not expect perfect results in a real-world scenario, these lab results were nonetheless encouraging. Recall that all of our obfuscated examples were generated by one source, namely the Invoke-DOSfuscation tool. While Invoke-DOSfuscation generates a wide variety of obfuscated samples, in the real world we expect to see at least some samples that are quite dissimilar from any that Invoke-DOSfuscation generates. We are currently collecting real world obfuscated command lines to get a more accurate picture of the generalizability of this model on obfuscated samples from actual malicious actors. We expect that command obfuscation, similar to PowerShell obfuscation before it, will continue to emerge in new malware families.

As an additional test we asked Daniel Bohannon (author of Invoke-DOSfuscation, the Windows command line obfuscation tool) to come up with obfuscated samples that in his experience would be difficult for a traditional obfuscation detector. In every case, our ML detector was still able to detect obfuscation. Some examples are shown in Figure 8.


Figure 8: Some examples of obfuscated text used to test and attempt to defeat the ML obfuscation detector (all were correctly identified as obfuscated text)

We also created very cryptic looking texts that, although valid Windows command lines and non-obfuscated, appear slightly obfuscated to a human observer. This was done to test efficacy of the detector with boundary examples. The detector was correctly able to classify the text as non-obfuscated in this case as well. Figure 9 shows one such example.


Figure 9: An example that appears on first glance to be obfuscated, but isn't really and would likely fool a non-ML solution (however, the ML obfuscation detector currently identifies it as non-obfuscated)

Finally, Figure 10 shows a complicated yet non-obfuscated command line that is correctly classified by our obfuscation detector, but would likely fool a non-ML detector based on statistical features (for example a rule-based detector with a hand-crafted weighing scheme and a threshold, using features such as the proportion of special characters, length of the command line or entropy of the command line).


Figure 10: An example that would likely be misclassified by an ML detector that uses simplistic statistical features; however, our ML obfuscation detector currently identifies it as non-obfuscated

CNN vs. GBT Results

We compared the results of a heavily tuned GBT classifier built using carefully selected features to those of a CNN trained with raw data (featureless ML). While the CNN architecture was not heavily tuned, it is interesting to note that with samples such as those in Figure 10, the GBT classifier confidently predicted non-obfuscated with a score of 19.7 percent (the complement of the measure of the classifier’s confidence in non-obfuscation). Meanwhile, the CNN classifier predicted non-obfuscated with a confidence probability of 50 percent – right at the boundary between obfuscated and non-obfuscated. The number of misclassifications of the CNN model was also more than that of the Gradient Boosted Tree model. Both of these are most likely the result of inadequate tuning of the CNN, and not a fundamental shortcoming of the featureless approach.

Conclusion

In this blog post we described an ML approach to detecting obfuscated Windows command lines, which can be used as a signal to help identify malicious command line usage. Using ML techniques, we demonstrated a highly accurate mechanism for detecting such command lines without resorting to the often inadequate and costly technique of maintaining complex if-then rules and regular expressions. The more comprehensive ML approach is flexible enough to catch new variations in obfuscation, and when gaps are detected, it can usually be handled by adding some well-chosen evader samples to the training set and retraining the model.

This successful application of ML is yet another demonstration of the usefulness of ML in replacing complex manual or programmatic approaches to problems in computer security. In the years to come, we anticipate ML to take an increasingly important role both at FireEye and in the rest of the cyber security industry.

The Origin of the Term Indicators of Compromise (IOCs)

I am an historian. I practice digital security, but I earned a bachelor's of science degree in history from the United States Air Force Academy. (1)

Historians create products by analyzing artifacts, among which the most significant is the written word.

In my last post, I talked about IOCs, or indicators of compromise. Do you know the origin of the term? I thought I did, but I wanted to rely on my historian's methodology to invalidate or confirm my understanding.

I became aware of the term "indicator" as an element of indications and warning (I&W), when I attended Air Force Intelligence Officer's school in 1996-1997. I will return to this shortly, but I did not encounter the term "indicator" in a digital security context until I encountered the work of Kevin Mandia.

In August 2001, shortly after its publication, I read Incident Response: Investigating Computer Crime, by Kevin Mandia, Chris Prosise, and Matt Pepe (Osborne/McGraw-Hill). I was so impressed by this work that I managed to secure a job with their company, Foundstone, by April 2002. I joined the Foundstone incident response team, which was led by Kevin and consisted of Matt Pepe, Keith Jones, Julie Darmstadt, and me.

I Tweeted earlier today that Kevin invented the term "indicator" (in the IR context) in that 2001 edition, but a quick review of the hard copy in my library does not show its usage, at least not prominently. I believe we were using the term in the office but that it had not appeared in the 2001 book. Documentation would seem to confirm that, as Kevin was working on the second edition of the IR book (to which I contributed), and that version, published in 2003, features the term "indicator" in multiple locations.

In fact, the earliest use of the term "indicators of compromise," appearing in print in a digital security context, appears on page 280 in Incident Response & Computer Forensics, 2nd Edition.


From other uses of the term "indicators" in that IR book, you can observe that IOC wasn't a formal, independent concept at this point, in 2003. In the same excerpt above you see "indicators of attack" mentioned.

The first citation of the term "indicators" in the 2003 book shows it is meant as an investigative lead or tip:


Did I just give up my search at this point? Of course not.

If you do time-limited Google searches for "indicators of compromise," after weeding out patent filings that reference later work (from FireEye, in 2013), you might find this document, which concludes with this statement:

Indicators of compromise are from Lynn Fischer, Lynn, "Looking for the Unexpected," Security Awareness Bulletin, 3-96, 1996. Richmond, VA: DoD Security Institute.

Here the context is the compromise of a person with a security clearance.

In the same spirit, the earliest reference to "indicator" in a security-specific, detection-oriented context appears in the patent Method and system for reducing the rate of infection of a communications network by a software worm (6 Dec 2002). Stuart Staniford is the lead author; he was later chief scientist at FireEye, although he left before FireEye acquired Mandiant (and me).

While Kevin, et al were publishing the second edition of their IR book in 2003, I was writing my first book, The Tao of Network Security Monitoring. I began chapter two with a discussion of indicators, inspired by my Air Force intelligence officer training in I&W and Kevin's use of the term at Foundstone.

You can find chapter two in its entirety online. In the chapter I also used the term "indicators of compromise," in the spirit Kevin used it; but again, it was not yet a formal, independent term.

My book was published in 2004, followed by two more in rapid succession.

The term "indicators" didn't really make a splash until 2009, when Mike Cloppert published a series on threat intelligence and the cyber kill chain. The most impactful in my opinion was Security Intelligence: Attacking the Cyber Kill Chain. Mike wrote:


I remember very much enjoying these posts, but the Cyber Kill Chain was the aspect that had the biggest impact on the security community. Mike does not say "IOC" in the post. Where he does say "compromise," he's using it to describe a victimized computer.

The stage is now set for seeing indicators of compromise in a modern context. Drum roll, please!

The first documented appearance of the term indicators of compromise, or IOCs, in the modern context, appears in basically two places simultaneously, with ultimate credit going to the same organziation: Mandiant.

The first Mandiant M-Trends report, published on 25 Jan 2010, provides the following description of IOCs on page 9:


The next day, 26 Jan 2010, Matt Frazier published Combat the APT by Sharing Indicators of Compromise to the Mandiant blog. Matt wrote to introduce an XML-based instantiation of IOCs, which could be read and created using free Mandiant tools.


Note how complicated Matt's IOC example is. It's not a file hash (alone), or a file name (alone), or an IP address, etc. It's a Boolean expression of many elements. You can read in the text that this original IOC definition rejects what some commonly consider "IOCs" to be. Matt wrote:

Historically, compromise data has been exchanged in CSV or PDFs laden with tables of "known bad" malware information - name, size, MD5 hash values and paragraphs of imprecise descriptions... (emphasis added)

On a related note, I looked for early citations of work on defining IOCs, and found a paper by Simson Garfinkel, well-respected forensic analyst. He gave credit to Matt Frazier and Mandiant, writing in 2011:

Frazier (2010) of MANDIANT developed Indicators of Compromise (IOCs), an XML-based language designed to express signatures of malware such as files with a particular MD5 hash value, file length, or the existence of particular registry entries. There is a free editor for manipulating the XML. MANDIANT has a tool that can use these IOCs to scan for malware and the so-called “Advanced Persistent Threat.”

Starting in 2010, the debate was initially about the format for IOCs, and how to produce and consume them. We can see in this written evidence from 2010, however, a definition of indicators of compromise and IOCs that contains all the elements that would be recognized in current usage.

tl;dr Mandiant invented the term indicators of compromise, or IOCs, in 2010, building off the term "indicator," introduced widely in a detection context by Kevin Mandia, no later than his 2003 incident response book.

(1) Yes, a BS, not a BA -- thank you USAFA for 14 mandatory STEM classes.

Even More on Threat Hunting

In response to my post More on Threat Hunting, Rob Lee asked:

[D]o you consider detection through ID’ing/“matching” TTPs not hunting?

To answer this question, we must begin by clarifying "TTPs." Most readers know TTPs to mean tactics, techniques and procedures, defined by David Bianco in his Pyramid of Pain post as:

How the adversary goes about accomplishing their mission, from reconnaissance all the way through data exfiltration and at every step in between.

In case you've forgotten David's pyramid, it looks like this.


It's important to recognize that the pyramid consists of indicators of compromise (IOCs). David uses the term "indicator" in his original post, but his follow-up post from his time at Sqrrl makes this clear:

There are a wide variety of IoCs ranging from basic file hashes to hacking Tactics, Techniques and Procedures (TTPs). Sqrrl Security Architect, David Bianco, uses a concept called the Pyramid of Pain to categorize IoCs. 

At this point it should be clear that I consider TTPs to be one form of IOC.

In The Practice of Network Security Monitoring, I included the following workflow:

You can see in the second column that I define hunting as "IOC-free analysis." On page 193 of the book I wrote:

Analysis is the process of identifying and validating normal, suspicious, and malicious activity. IOCs expedite this process. Formally, IOCs are manifestations of observable or discernible adversary actions. Informally, IOCs are ways to codify adversary activity so that technical systems can find intruders in digital evidence...

I refer to relying on IOCs to find intruders as IOC-centric analysis, or matching. Analysts match IOCs to evidence to identify suspicious or malicious activity, and then validate their findings.

Matching is not the only way to find intruders. More advanced NSM operations also pursue IOC-free analysis, or hunting. In the mid-2000s, the US Air Force popularized the term hunter-killer in the digital world. Security experts performed friendly force projection on their networks, examining data and sometimes occupying the systems themselves in order to find advanced threats. 

Today, NSM professionals like David Bianco and Aaron Wade promote network “hunting trips,” during which a senior investigator with a novel way to detect intruders guides junior analysts through data and systems looking for signs of the adversary. 

Upon validating the technique (and responding to any enemy actions), the hunters incorporate the new detection method into a CIRT’s IOC-centric operations. (emphasis added)

Let's consider Chris Sanders' blog post titled Threat Hunting for HTTP User Agents as an example of my definition of hunting. 

I will build a "hunting profile" via excerpts (in italics) from his post:

Assumption: "Attackers frequently use HTTP to facilitate malicious network communication."

Hypothesis: If I find an unusual user agent string in HTTP traffic, I may have discovered an attacker.

Question: “Did any system on my network communicate over HTTP using a suspicious or unknown user agent?”

Method: "This question can be answered with a simple aggregation wherein the user agent field in all HTTP traffic for a set time is analyzed. I’ve done this using Sqrrl Query Language here:

SELECT COUNT(*),user_agent FROM HTTPProxy GROUP BY user_agent ORDER BY COUNT(*) ASC LIMIT 20

This query selects the user_agent field from the HTTPProxy data source and groups and counts all unique entries for that field. The results are sorted by the count, with the least frequent occurrences at the top."

Results: Chris offers advice on how to interpret the various user agent strings produced by the query.

This is the critical part: Chris did not say "look for *this user agent*. He offered the reader an assumption, a hypothesis, a question, and a method. It is up to the defender to investigate the results. This, for me, is true hunting.

If Chris had instead referred users to this list of malware user agents (for example) and said look for "Mazilla/4.0", then I consider that manual (human) matching. If I created a Snort or Suricata rule to look for that user agent, then I consider that automated (machine) matching.

This is where my threat hunting definition likely diverges from modern practice. Analyst Z sees the results of Chris' hunt and thinks "Chris found user agent XXXX to be malicious, so I should go look for it." Analyst Z queries his or her data and does or does not find evidence of user agent XXXX.

I do not consider analyst Z's actions to be hunting. I consider it matching. There is nothing wrong with this. In fact, one of the purposes of hunting is to provide new inputs to the matching process, so that future hunting trips can explore new assumptions, hypotheses, questions, and methods, and let the machines do the matching on IOCs already found to be suggestive of adversary activity. This is why I wrote in my 2013 book "Upon validating the technique (and responding to any enemy actions), the hunters incorporate the new detection method into a CIRT’s IOC-centric operations."

The term "hunting" is a victim of its own success, with emotional baggage. We defenders have finally found a way to make "blue team" work appealing to the wider security community. Vendors love this new way to market their products. "If you're not hunting, are you doing anything useful?" one might ask.

Compared to "I'm threat hunting!" (insert chest beating), the alternative, "I'm matching!" (womp womp), seems sad. 

Nevertheless, we must remember that threat hunting methodologies were invented to find adversary activity for which there were no IOCs. Hunting was IOC-free analysis because we didn't know what to look for. Once you know what to look for, you are matching. Both forms of detection require analysis to validate adversary activity, of course. Let's not forget that.

I'm also very thankful, however it's defined or packaged, that people are excited to search for adversary activity in their environment, whether via matching or hunting. It's a big step from the mindset of 10 years ago, which had a "prevention works" milieu.

tl;dr Because TTPs are a form of IOC, then detection via matching IOCs is a form of matching, and not hunting.

More on Threat Hunting

Earlier this week hellor00t asked via Twitter:

Where would you place your security researchers/hunt team?

I replied:

For me, "hunt" is just a form of detection. I don't see the need to build a "hunt" team. IR teams detect intruders using two major modes: matching and hunting. Junior people spend more time matching. Senior people spend more time hunting. Both can and should do both functions.

This inspired Rob Lee to blog a response, from which I extract his core argument:

[Hunting] really isn’t, to me, about detecting threats...

Hunting is a hypothesis-led approach to testing your environment for threats. The purpose, to me, is not in finding threats but in determining what gaps you have in your ability to detect and respond to them...

In short, hunting, to me, is a way to assess your security (people, process, and technology) against threats while extending your automation footprint to better be prepared in the future. Or simply stated, it’s incident response without the incident that’s done with a purpose and contributes something. 

As background for my answer, I recommend my March 2017 post The Origin of Threat Hunting, which cites my article "Become a Hunter," published in the July-August 2011 issue of Information Security Magazine. I wrote it in the spring of 2011, when I was director of incident response for GE-CIRT.

For the term "hunting," I give credit to briefers from the Air Force and NSA who, in the mid-2000s briefed "hunter-killer" missions to the Red Team/Blue Team Symposium at the Johns Hopkins University Applied Physics Lab in Laurel, MD.

As a comment to that post, Tony Sager, who ran NSA VAO at the time I was briefed at ReBl, described hunting thus:

[Hunting] was an active and sustained search for Attackers...

For us, "Hunt" meant a very planned and sustained search, taking advantage of the existing infrastructure of Red/Blue Teams and COMSEC Monitoring, as well as intelligence information to guide the search. 

For the practice of hunting, as I experienced it, I give credit to our GE-CIRT incident handlers -- David Bianco,  Ken Bradley, Tim Crothers, Tyler Hudak, Bamm Visscher, and Aaron Wade -- who took junior analysts on "hunting trips," starting in 2008-2009.

It is very clear, to me, that hunting has always been associated with detecting an adversary, not "determining what gaps you have in your ability to detect and respond to them," as characterized by Rob.

For me, Rob is describing the job of an enterprise visibility architect, which I described in a 2007 post:

[W]e are stuck with numerous platforms, operating systems, applications, and data (POAD) for which we have zero visibility. 

I suggest that enterprises consider hiring or assigning a new role -- Enterprise Visibility Architect. The role of the EVA is to identify visibility deficiencies in existing and future POAD and design solutions to instrument these resources.

A primary reason to hire an enterprise visibility architect is to build visibility in, which I described in several posts, including this one from 2009 titled Build Visibility In. As a proponent of the "monitor first" school, I will always agree that it is important to identify and address visibility gaps.

So where do we go from here?

Tony Sager, as one of my wise men, offers sage advice at the conclusion of his comment:

"Hunt" emerged as part of a unifying mission model for my Group in the Information Assurance Directorate at NSA (the defensive mission) in the mid-late 2000's. But it was also a way to unify the relationship between IA and the SIGINT mission - intelligence as the driver for Hunting. The marketplace, of course, has now brought its own meaning to the term, but I just wanted to share some history. 

In my younger days I might have expressed much more energy and emotion when encountering a different viewpoint. At this point in my career, I'm more comfortable with other points of view, so long as they do not result in harm, or a waste of my taxpayer dollars, or other clearly negative consequences. I also appreciate the kind words Rob offered toward my point of view.

tl;dr I believe the definition and practice of hunting has always been tied to adversaries, and that Rob describes the work of an enterprise visibility architect when he focuses on visibility gaps rather than adversary activity.

Update 1: If in the course of conducting a hunt you identify a visibility or resistance deficiency, that is indeed beneficial. The benefit, however, is derivative. You hunt to find adversaries. Identifying gaps is secondary although welcome.

The same would be true of hunting and discovering misconfigured systems, or previously unidentified assets, or unpatched software, or any of the other myriad facts on the ground that manifest when one applies Clausewitz's directed telescope towards their computing environment.

#TweetBlog: APT29, Phishing and the Challenges of Attribution

FireEye researchers, analysts and incident responders frequently share information and engage with the security community on Twitter and other social media platforms. Sometimes this information adds so much to ongoing discussions that we feel it is important to share on our blogs.

Recently, we detected intrusion attempts against multiple industries as part of a phishing campaign that we suspect is being carried out by APT29. Following the release of our blog post, one of the authors and the head skeptic, Andrew Thompson (@QW5kcmV3), took to Twitter to discuss various attribution possibilities that were considered along the way, as well as some of the challenges that come with attribution.

Cyber Security 101 for the C-Suite – Active Directory Security is Paramount

Folks,

Today's post is for all executives worldwide who comprise the C-Suite at thousands of organizations worldwide.


I pen today's post with profound respect for all executives worldwide, because I understand first-hand just how important the nature of their responsibilities is, how valuable their time is, and how far-reaching the consequences of their decisions are.

A quick footnote for all C*Os : In case you're wondering who I am to be penning this, I'm former Microsoft Program Manager for Active Directory Security. Relevance? Microsoft's Active Directory is the foundation of your entire organization's cyber security. Finally, like you, I also happen to be the CEO of a $ Billion+ company.

Today's post is in the form of a simple letter, that follows (below.)

Thanks,
Sanjay


<Begin Letter>

Subject - Cyber Security 101 for the C-Suite


To: Chairmen, CEOs and CFOs Worldwide



Dear C*O,

Hi, I'm Sanjay, former Microsoft Program Manager for Active Directory Security, but more importantly a sincere well-wisher who cares deeply about cyber security, and who just happens to know a thing or two about the very technology that lies at the very foundation of cyber security of your ($ Billion to $ Trillion) organization, and those of 85% of all organizations worldwide.

I write to you to bring to your attention a matter of paramount importance to your organization's foundational security.



Context - Foundational Security

Today we all engage in business in what is essentially a global digital village, wherein just about just every aspect of business, whether it be production, marketing, sales, customer-service, collaboration, finance etc. etc. substantially relies on technology.


Within our respective organizations, it is our IT infrastructure that enables and empowers our workforce to engage in business.

For instance, we all (including us C*Os) log on to a computer every day, send and receive email, and create, share and access digital assets (e.g. documents, applications, services etc.) all of which are securely stored on our organizational computers.

It is only logical then that ensuring the security of the very IT infrastructure that enables and empowers our entire workforce to engage in business digitally, and the security of our digital assets is vital. In other words, cyber security is very important.

Now, if I told you that at the very foundation of your entire IT infrastructure, and consequently at the very foundation of the security of all your digital assets lay a single high-value asset, then I think you'd agree that its security would be paramount.

At the very foundation of your organization's IT infrastructure and that of its cyber security, and by corollary the cyber security of the entirety of all your digital assets (e.g. thousands of computers, thousands of employee user accounts and passwords, every single organizational email sent and received every minute of every day, all your applications, services, Intranet portals, Internet facing applications etc.) as well as the entirety of your organization's data, lies a single technology - Microsoft Active Directory.


Most simply put, Active Directory is the database that contains, stores and protects the entirety of your organization's building blocks of cyber security - each one of thousands of user accounts and their passwords, each one of thousands of computer accounts (for all laptops, desktops, servers etc.), each one of thousands of security groups that protect all your data etc. etc.

If your organization's Active Directory were compromised, everything would immediately be exposed to the risk of compromise.

Thus as you'll hopefully agree, ensuring the security of your organization's foundational Active Directory is well, paramount.



A Provable Concern - Inadequate Protection

Now, you might most likely be thinking - Well, if that's the case, I'm sure that our CIO, our CISO and their world-class IT and Cyber Security teams know all this, and have it adequately taken care of, so why should I be concerned ?


Here's why you should be concerned - In all likelihood, not only may your world-class IT and Cyber Security teams not have this adequately covered, they may have yet to realize just how very important, and in fact paramount Active Directory security is.

Further, they likely may not know what it actually takes to adequately secure your organization's  foundational Active Directory.

Now, as incredulous as that may sound, you have to trust me on this, not because I'm asking you to do so as a concerned well-wisher, but because I'm asking you to do so as arguably the world's #1 subject matter expert on Active Directory Security.

You see, prior to doing what I currently do, I was Microsoft's subject matter expert for Active Directory Security on Microsoft's Windows Server Development team. In case you're curious as to what I do currently do with all this knowledge, well, its this.

As the world's leading subject matter expert on Active Directory Security, I would highly encourage you to ask your IT and Cyber Security leadership, specifically your CIO and your CISO, just how secure they think your organization's Active Directory is.



Simple Proof - You Just Have to Ask

When you ask them about it, please do request specific answers, and here are 7 simple questions you can ask them, the answers to which will give you an indication of just how secure your organization's Active Directory actually is today -


  1. Is the security of our foundational Active Directory deployment a top cyber security priority today?

  2. Do we know exactly what the Top-5 security risks to our foundational Active Directory are?

  3. Do our Active Directory Admins know what Active Directory Effective Permissions are?

  4. Do we know exactly who possesses what level of privileged access in our Active Directory?

  5. Do we know exactly who can control/manage each one of our Active Directory privileged accounts and groups?

  6. Do we know exactly who can run Mimikatz DCSync against our Active Directory today?

  7. Can you tell me exactly who can reset my domain user account's password to then be able to login as me?

I could suggest 50 such elemental cyber security questions, but for now these 7 simple, precise questions will suffice as there are only 2 possibilities here - either your IT and cyber security leadership have exact answers to these questions, or they don't.


If they can't give you exact answers to these questions, your organization's Active Directory is not secure - its as simple as that.


They might tell you that this is complicated or that they have a good approximation, or that this is very difficult to do, or that they have many other latest buzzword measures like Active Directory Auditing, Privileged Access Management, ATA, Just-in-Time Administration etc. in place, but none of that matters, because the truth is simple - they either have exact answers, or they don't.

(These questions are paramount to cyber security, and today there exists technology that can enable every organization in the world to answer them precisely, but because Microsoft likely forgot to adequately educate its customers, your IT personnel may likely not even know the importance of these paramount questions, let alone knowing what it takes to correctly answer them.)

If a $Billion+ organization doesn't even know exactly who has what privileged access in their Active Directory, as well as exactly who can manage each one of their privileged accounts and groups, how could their Active Directory possibly be secure?

If an organization's foundational Active Directory is not secure, how can the entirety of the organization's digital (IT) assets be secure, and if that's not case, how could an organization possibly be considered secure from a cyber security perspective?



Driving Change

As a member of the C-Suite, you not only have the privilege of being able to impact vital change in your organization, you also have the responsibility and the authority to demand and ensure the cyber security of the very foundation of your organization.


As a C*O, one of the most important responsibilities you shoulder is ensuring that your organization is secure, and ensuring that the very foundation of your organization's IT infrastructure and cyber security are always adequately protected, is paramount.




The Likely Reason (Optional Reading)

Here's the likely reason for why such a common-sense yet paramount matter may not be on your CIO's and CISO's radar yet.

You see, your CIO and CISO shoulder great responsibility. Unfortunately, amongst many other things, they're likely also being guided by inputs from a 1000 cyber security companies, who unfortunately may not be the best source of objective guidance.

For instance, consider CyberArk, a highly respected $ Billion+ cyber security company, that claims that over 50% of the Fortune 100's CISOs rely on them. As a subject matter expert, I can tell you that CyberArk itself may not know how to correctly assess privileged access in an Active Directory, so you see, unfortunately your CIO and CISO may not be getting the best guidance.

CyberArk is absolutely correct that "Privilege is Everywhere." However, those who know Windows Security will tell you that in a Windows network powered by Active Directory, the majority of all privileged access (delegated & unrestricted) lies inside Active Directory, but CyberArk doesn't seem to have the capability to correctly audit privileged access inside Active Directory.


The majority of all Privileged Access,including the "Keys to the Kingdom", resides inside Active Directory

CyberArk isn't alone. As unbelievable as it may sound, today even Microsoft doesn't seem to know what it takes to do so, let alone possessing the capability to help its customers correctly do so. In fact, most of the world's top IT Consulting, Audit, Cloud and Cyber Security companies also operate on Active Directory, and they too likely have neither a clue nor the capability to accurately determine exactly who has what privileged access in their own foundational Active Directory deployments.

You may find this hard to believe, but of the 1000+ cyber security companies exhibiting or presenting at the upcoming RSA Conference 2019, not a single one of them can help your organization's IT personnel fulfill such a fundamental yet paramount cyber security need - finding out exactly who has what privileged access in your organization's foundational Active Directory.

In their defense, I'll say this - if it were easy, they would've all done it by now. Unfortunately, as paramount as it is, its not easy.

Thus, I know what your CIO and CISO may perhaps not yet know, or understand the paramount importance of, which is that of all the things that need to be secured, none could possibly be more important than securing your organization's foundational Active Directory, so I thought I'd share this with you, because as a member of the C-Suite, you could provide them strategic guidance and the executive support that their teams need to accomplish this paramount objective for your organization.



In Conclusion

I only wrote this letter because we're all in this together, and I care deeply about foundational cyber security, as hopefully do you, and I felt that I could perhaps help bridge the gap between those tasked with the great responsibility of securing Active Directory (i.e. your IT personnel) and those whose executive support they need to be able to do so (i.e. you, the C-Suite.)

If any of what I shared above made sense, I would encourage you to embrace my suggestions earnestly, and act upon them, and if needed, I can prove and demonstrate every thing I've shared above, and you should feel free to take me up on this.

As for myself, all I can say is that today my work and knowledge silently help secure and defend so many of the world's most important organizations across six continents worldwide.

Thank you for your time.

Respectfully,
Sanjay Tandon.

Chairman and CEO,
Paramount Defenses



PS: Please know that I am also doing my bit to help Microsoft and the World better Understand Active Directory Security



<End Letter>

Cybersecurity and Class M Planets

I was considering another debate about appropriate cybersecurity measures and I had the following thought: not all networks are the same. Profound, right? This is so obvious, yet so obviously forgotten.

Too often when confronting a proposed defensive measure, an audience approaches the concept from their own preconceived notion of what assets need to be protected.

Some think about an information technology enterprise organization with endpoints, servers, and infrastructure. Others think about an industrial organization with manufacturing equipment. Others imagine an environment with no network at all, where constituents access cloud-hosted resources. Still others think in terms of being that cloud hosting environment itself.

Beyond those elements, we need to consider the number of assets, their geographic diversity, their relative value, and many other aspects that you can no doubt imagine.

This made me wonder if we need some sort of easy reference term to capture the essential nature of these sorts of diverse environments. I thought immediately of the term "class M planet," from Star Trek. From the Wikipedia entry:

[An] Earth-like planet, the Class M designation is similar to the real-world astronomical theory of life-supporting planets within the habitable zone... Class M planets are said to possess an atmosphere composed of nitrogen and oxygen as well as an abundance of liquid water necessary for carbon-based life to exist. Extensive plant and animal life often flourishes; often, a sentient race is also present. 

In contrast, consider a class Y planet:

Class Y planets are referred to as "demon" worlds, where surface conditions do not fall into any other recognized category. Such worlds are usually hostile and lethal to humanoid life. If life forms develop on these worlds they usually take on many bizarre forms, like living crystal or rock, liquid or gaseous physical states, or incorporeal, dimensional, or energy-based states. 

Given their work providing names for various offensive security activities in ATT&CK, I wonder if MITRE might consider creating a naming scheme to capture this idea? For example, a "class M" network might be an enterprise organization with endpoints, servers, and infrastructure, of a certain size. Or perhaps M1 might be "small," M2 "medium," and M3 "large," where each is associated with a user count.

Perhaps an environment with no network at all, where constituents access cloud-hosted resources, would be a class C network. (I'm not sure "network" is even the right term, if there is no "network" for which the organization is responsible.)

With such a scheme in place, we could begin a cybersecurity discussion by asking, "given a class M network, what defensive processes, people, or technology are appropriate," versus "given a class C network, what defensive processes, people, or technology are appropriate."

This is only an idea, and I'd be happy if something was already created to address this problem. Comments below are welcome (pending moderation to repel trolls and spammers.) Alternatively, reply to my announcement of this post via @taosecurity on Twitter.