Category Archives: Freeware

How Will I Fill This Web Historian-Shaped Hole in My Heart?

With the recent integration of Mandiant Web Historian™ into Mandiant Redline™, you may be asking "How do I review my Web History using Redline?" If so, then follow along as I explain how to collect and review web history data in Redline - with a focus on areas where the workflow and features differ from that of Web Historian.

For those of you unfamiliar with Redline, it is Mandiant's premier free tool, providing host investigative capabilities to help users find signs of malicious activity through memory and file analysis and the development of a threat assessment profile.

Configuring Web History Data Collection

Web Historian provided three options for choosing how to find web history data that you want to analyze: scan my local system, scan a profile folder, and parse an individual history file. Redline allows you to accomplish all three of these operations using a single option, Analyze this Computer, which is found under the Main Menu in the upper left corner. Specifying the path to a profile folder or a history file will require some additional configuration:

Figure 1: Finding your web history data Web Historian (Left) vs. Redline (Right)

Click on Analyze this Computer to begin configuring your analysis session. To ensure that Redline collects your desired web history data, click the link to Edit your script . On the View and Edit Your Script window are several options; take a look around and turn on any and all data that might interest you. For our purposes, we will be focusing on the Browser History options underneath the Network tab. This section contains all of the familiar options from Web Historian; simply select the boxes corresponding to the data you wish to collect.

One difference you may notice is the absence of an option to specify the browser(s) you would like to target. You can now find that option by selecting Show Advanced Parameters from the upper right corner of the window. Once advanced parameters are enabled, simply type the name of any browser(s) you would like to target, each on its own separate line in the Target Browser field. To have Redline collect any web history data regardless of browser, just leave this field empty.

You may also notice that enabling advanced parameters activates a field for History Files Location. As you may have guessed, this is where you can specify a path to a profile folder or history file to analyze directly, as you were able to do in Web Historian.

Figure 2: Configuring Redline to Collect Browser History Data

Now that you have finished configuring your script, choose a location to save your analysis session and then hit OK . Redline will run the script, which will require Administrator privileges and may trigger a UAC prompt before running depending on how your system is configured. After a brief collecting and processing time, your web history data will be ready for review.

Reviewing your Data

For the actual review of your web history data, you should feel right at home in Redline. Just like Web Historian, Redline uses a sortable, searchable, configurable table view for each of the individual categories of web history data.

Figure 3: Displaying your web history data for review in both Web Historian (behind) and Redline (front)

Although similar, Redline does have a few minor differences in how it visualizes your data:

  • Redline does not break the data into pages; instead it will discretely page in large data sets (25k+ rows) automatically as you scroll down through the list.
  • To configure the table view, you will need to manipulate the column headers for ordering and resizing, and right-click on a column header to show and hide columns - as opposed to using the column configuration menu in Web Historian.
  • Searching and simple filtering is done in each individual table view and is not applied globally. To access the find options, either press the magnifying glass in the bottom right corner, or press Ctrl-f while a table view is selected.
  • To export your data to a CSV (comma separated values) format file, click on export in the bottom right corner. Like Web Historian, Redline will only export data currently in the table view. If you applied any filtering or tags, those will affect the data as it is exported.

In addition to the features that have always been available in Web Historian, Redline also allows you to review your web history with its full suite of analytical capabilities and investigative tools. Check out the Redline user guide for a full description of these capabilities. Here are just a few of the most popular:

  • Timeline provides a chronological listing of all web-based events (e.g., URL last browsed to, File Download Started, etc.) in a single heterogeneous display. You can employ this to follow the activities of a user or attacker as they played out on the system. You can also quickly reduce your target investigative scope using the Timeline's powerful filtering capabilities.
  • Use tags and comments to mark-up your findings as you perform your investigation, making it easier to keep track of what you have seen while moving forward. You can then go back and export those results into your favorite reporting solution.
  • Use Indicators of Compromise (IOCs) as a quick way to determine if your system contains any potential security breaches or other evidence of compromise. Visit http://www.openioc.org/ to learn more about IOCs.
  • Last but not least, Redline gives you the ability to examine an entire system worth of metadata. With Redline, you are not simply restricted to Web History related data; you can investigate security incidents with the scope and context of the full system.

If your favorite feature from Web Historian has not yet been included in Redline (Graphing, Complex Filtering, etc.), feel free to make a request using one of the contact methods specified below. We will be taking feedback into consideration when choosing what the Redline team works on in the future.

As always, feel free to contact us with send any additional questions. And just in case you do not already have it, the latest version of Redline (v1.10 as of the time of this writing) can always be found here.

Redline: Answering Your Questions

Those of you who attended the "Tools of Engagement: Redline™ - We've Got the Tool, If You've Got the Time" webinar last month by David Ross and myself will recall that we ran short on time while answering all of your questions. The webinar covered the latest updates to Redline, Mandiant's free tool for investigating hosts for signs of malicious activity through memory and file analysis, and subsequently developing a threat assessment profile.

If your question was one of those we did not get to, don't worry. We are going to cover all of those unanswered questions in this post, as well as retread some of those which were covered during the Q&A for people who were unable to catch it live. 

Without further ado, following are your answers in no particular order:

Does Redline support disk images for collection and analysis?

At the moment, Redline only works on memory images and live hosts. We are currently focusing on providing the best possible set of analysis tools for incident response.

Does Redline work for Macs or Mac Memory images? Does it work on Linux?

Redline officially supports data collected with Mandiant Intelligent Response® (MIR), Mandiant Memoryze™, or a Redline Collector.

Unfortunately, all of those currently only support collection on the various Windows platforms. However, I have heard of people having success getting audits collected with Memoryze™ for the Mac to at least import into Redline. Be careful to note that if attempting this, the MRI scores and other analyses may be incorrect or invalid, as the scoring in Redline assumes it is operating against data collected from a Windows host.

Is there a specific audit you need to run in order to do Timeline Analysis?

Short answer: no. The timeline analysis will parse any and all data with timestamps available for collection. The comprehensive collector option in Redline is the recommended starting place if timeline analysis is your goal as the standard collector collects very little in the way of timestamps.

How valuable would Redline be against a virtual machine created from a forensic image?

As long as you can log in to the virtual machine with administrator rights to run the collection, Redline should have no problem importing and analyzing the data (provided it is one of the supported operating systems).

Is Redline free when used on an enterprise environment?

Redline is free to use in any sized environment, although the collection aspect of Redline quickly becomes challenging with large scale and globally distributed networks. This leads to the next question...

Can Redline collection be run on a remote machine?

Redline does not itself support remote collection. We recommend purchasing Mandiant Intelligent Response® if you would like centralized remote collection of your hosts' data over an enterprise sized network.

Can you demonstrate how to use TimeWrinkles™ for events that occur over multiple days and put them all into one view?

The easiest way to view timeline windows that are separated by greater than an hour is to create multiple manual TimeWrinkles around the points of time you are specifically interested in.

Using item based TimeWrinkles you can also potentially see time entries that occur over multiple days. For instance, you could see the actions that happened around the creating of a file, as well as when that same file was last accessed a few days later all in one view, just by creating a TimeWrinkle around that file.

How do you get a TimeWrinkle based on a file?

If you select any row within the Timeline and right click on it, Redline will give you the option to create a TimeWrinkle based on that item. In this case, you would just need to find the file in question within your timeline, select it, and choose "Add a New TimeWrinkle" from its right click menu.

Can Redline be used to pull strings from a memory image? We would like to pull info from the csrss process to see what commands might have run on a box.

Redline can be configured to collect strings using the process listing audit against both a memory image and a live machine. You can collect strings from files with the File listing audit, but this option is only available against a live machine.

We do recommend restricting string collection to a single process or file at a time though, as turning strings collection on for a full process or file listing will significantly increase the amount of data returned and the time it takes to collect it.

Can I export the data to a file?

Copy and Pasting from any of the list views (including Timeline), will place up to 20k selected rows onto your clipboard in CSV format. Using the right-click menu's copy options also allows you to specify if you would like to include a header row in your data or not. Full list CSV export directly to a file will be available in the next release of Redline.

Is the timeline feature available in Mandiant Intelligent Response ® (MIR ® )?

Timeline as it exists in Redline is not available in MIR. But using the "open with..." feature in the MIR Console on any audit result will allow you to import your data you would like to timeline directly into Redline for analysis.

Is there a way to get external data sources in to Redline that are not host-based? (ex. IDS, flow, etc.)

At the moment Redline only supports analysis of the xml data which is collected by the various Mandiant products listed above. Full schema definitions for those formats can be found here.

How much alteration is being done to the suspect system by Redline?

Depending on if the collector is being run from the host's hard disk as opposed to an external drive, the collection and log files have the potential to overwrite some amount disk slack space. Also if the "Preserve Timestamps" option is not configured on your collector, some audits may modify the timestamps for files they touch. You can find the "Preserve Timestamps" option at Main Menu -> Redline Options->Default Script Options->General->Preserve Timestamps". Redline defaults this option on.

Prior to re-image, what is a good Redline collector that can quickly get information to sift through later?

If the next immediate action is to re-image the box, I tend to err on the side of collecting as much information as time permits, since there will be no second chance to go back and recollect additional data. But for a little bit faster collection time, I suggest starting with the comprehensive collector and scaling back or removing the larger audits: files, registry, and processes.

While the collector run times depend heavily on machine in question, it is not unheard of for the comprehensive collector to run 1-2 hours. By limiting the files audit to a specific base path like the Windows directory or the System32 directory, and limiting your registry audit to a few specific keys (i.e. HKEY_LOCAL_MACHINESoftwareMicrosoftWindowsCurrentVersionRun) you can significantly reduce the time that your collection will take to run. If you need to collect even quicker, consider turning off the various hashes like MD5.

Can I run this tool via command line?

The collector runs as a batch script via the command line, but the analysis and visualization portions are only available through the graphical user interface.

Are you limited to the types of variants/intrusions you can scan for?

The Malware Risk Index (MRI) scoring configuration is limited in the types of things it allows you to look for, but within those confines you can add or tweak to your heart's content. Indicators of Compromise (IOC) provide a much more flexible definition format to describe what malware you would like to search for. The first Tools of Engagement: Redline webinar walks through an example of creating a new MRI rule and an Indicator of Compromise in the course of performing the investigation and applying them to a Redline analysis.

Can you tell if the attacker has manipulated the time stamps of the files?

Redline does not automatically detect timestamp manipulation. But often an experienced eye can pick it out by looking for things such as every potentially suspect files encountered having "00" for their seconds place, or similar statistically improbable occurrences.

Is there any known malware that targets Redline? Have you had any difficulties with attackers running their tools inside rootkit protection?

We have yet to encounter any malware that has specifically attempted to avoid collection or detection by Redline and its various analysis techniques. As for general rootkit protection, Redline uses raw disk access by default where possible to avoid being subverted by rootkits.

What metadata shows how many times an executable has executed?

Prefetch files (.pf) are windows specific cache files to improve application startup performance. They contain the first and last run time as well as how many executions have occurred in total. These files are parsed and their relevant data captured by the Prefetch audit available in the Redline Collector setup.

That wraps up all of unanswered (as well as answered) questions! And just in case you do not already have it, the latest version of Redline (1.7 as of the time of this writing) can always be found here.

 

 

 

Highlighter Super Users Series: Post 2

Back in November I published the first interview from the Highlighter™ Super Users blog series. My goal with this series is to shed some light on all the great things that can be achieved using this freeware tool. In part 2, I interviewed toolsmith author and HolisticInfoSec.org webmaster, Russ McRee.

Super User Interview #2: Russ McRee

Russ McRee is the author of ISSA Journal's toolsmith series and runs HolisticInfoSec.org. In October 2011 Russ contacted me to discuss Highlighter in that month's issue of the ISSA Journal, and later for the nomination of Highlighter for the 2011 Toolsmith Tool of the Year. As someone who has analyzed Highlighter's effectiveness as a forensics tool for his own articles, I asked him to answer a few questions based on his experience with the freeware tool.

  1. Name
    Russ McRee
  2. Realm of work
    Security Analytics (security incident management, security monitoring, attack and penetration testing).
  3. How did you hear of Highlighter?
    I watch the websites and check for tool updates.
  4. Do you know of any other tools that do what Highlighter does?
    Log Parser, Log Parser Lizard, Log Parser Studio, Splunk
  5. How do you normally use Highlighter?
    I mainly use Highlighter for Log analysis, forensic investigations, demonstrations and research (see http://www.youtube.com/watch?v=w0uOCOINrWY and https://www.sans.org/reading_room/whitepapers/logging/evil-lens-web-logs_33950)
  6. Can you describe one scenario in which Highlighter helped you find evil and/or solve crime?
    I had a recent mysterious case of core utility files and binaries gone missing from very important infrastructure management servers that initially looked malicious and intentional. Using Highlighter for analysis of Windows event logs led to the discovery of a sync job gone awry (misconfiguration) in the Application log via time stamp matching and keyword highlights.
  7. On a scale from 1 (worst) to 5 (best), how well does Highlighter address your use case(s)?
    4
  8. What is missing from Highlighter for your use case(s)?
    Word wrap option
  9. What is one Highlighter feature addition that would serve the Information Security community best?
    Potential DB support
  10. Are you aware of, or have you used, any of the following features:
    • Activity Over Time feature that lets you view log data as a function of Entries Per Day
      No, I was not aware.
    • Hotkeys feature
      Yes, I was aware of this feature.
    • Ability to change basic font settings for your output
      Yes, I was aware of this feature.
  11. Have you ever seen Highlighter used in such a way that your eyeballs melted from all the Awesome?
    My eyeballs melted from the awesome when I stuffed Highlighter with a 2.44GB Swatch log file during large file testing while writing October 2011's toolsmith. It took a little time to load and format (to be expected), but it handled 24,502,412 log entries admirably (no choking). I threw a query for a specific inode at it and Highlighter tagged 1930 hits across 25 million+ lines in ten minutes.

Keep an eye out for the final post in the Highlighter Super Users Series. If you're interested in sharing your own experiences with this tool, please let me know by commenting below.

A Look Back at 2012: The Armory

 As we are mere hours away from celebrating 2013, we'd like to focus today on M-Unition's Armory channel. The Armory is the place to be if you want to be the first to find out about the latest releases, free tools and of course, our ever popular M-Trends report. The most popular posts in this category are listed below for your reading pleasure.

New Product Offering: Mandiant Cloud Alert

This past year we made several product announcements, but this one was especially rewarding. When you deal with cybersecurity risks on a daily basis you need tools to help you see activity in real time. At MIRcon ™ 2012, we announced our newest product offering: Mandiant Cloud Alert™. Mandiant Cloud Alert is a powerful tool, enabling organizations to identify malicious communication, audit existing security measures, monitor how the organization is trending over time, and track incidents in their network.

Unibody Memory Analysis - Introducing Memoryze™ for the Mac 1.0

Memoryze™ for the Mac 1.0, which brings memory imaging and analysis to the Mac, joins a growing list of freeware tools Mandiant provided this past year.

Memoryze for the Mac 1.0 brings many of the features of Memoryze™ to the Apple Macintosh platform. This new tool enables acquisition of memory images via the command-line or a simple GUI. In addition, Memoryze for the Mac 1.0 can perform offline analysis against memory images or live analysis on a running system.

Leveraging the Application Compatibility Cache in Forensic Investigations

Freeware tool, Shim Cache Parser™, was developed in the course of our incident response investigations, according to Mandiant's Andrew Davis.

During keyword searches of compromised systems, the Mandiant team discovered known malicious file names in the Windows Registry. Further research showed the cache data was generated by the Windows Application Compatibility Database. Along with these file names, other types of file metadata can be recovered such as file size, file last modified times, and last execution time, depending on the operating system version. This data can be very useful during an incident response. It helps identify which systems an attacker may have executed malware on and can also provide information about the time that it may have occurred.

Shim Cache Parser is the proof-of-concept tool we developed to extract this useful forensic evidence. You can download it here.

Mandiant Introduces Reverse-Proxy Open Source Tool

Mandiant's Sean Cunningham and Mark Thomas discuss the availability of a highly efficient reverse HTTP(S) proxy called simply 'RProxy™'. Mandiant released RProxy as an open sources tool to encourage the general community to participate in its evolution. You can download the tool here.

M-Trends: The One Threat Report You Need to Read

Each year Mandiant takes a look back at engagements we've responded to and puts together trends that help you fight back against targeted threats. On March 6th, we released our latest M-Trends report, An Evolving Threat, which revealed key insights, statistics and case studies illustrating how the tools and tactics of targeted attackers, including the Advanced Persistent Threat (APT), have evolved over the last year. We're currently working on the 2013 edition of M-Trends and plan to release it at RSA 2013.

Freeware Release: Redline 1.7

Redline™ is Mandiant's free tool for investigating hosts for signs of malicious activity through memory and file analysis, and subsequently developing a threat assessment profile. It combines configurable collection of Mandiant's full range of forensic artifacts (the same set available to our enterprise product, Mandiant Intelligent Response®, guided analysis, Mandiant's Custom Malware Risk Index (MRI) scoring, and Indicator of Compromise (IOC) matching) to provide you with the tools needed to rapidly triage a potentially compromised host.

Recently, we released Redline version 1.7. To make this the most compelling release to date, we focused on two of the most frequently requested features: Timeline and Search.

Timeline

With Timeline, we wanted to do more than just slap together all date related items into a staggering list that buries you under more data than is humanly possible to review. Even on infrequently used systems, hundreds of thousands of dated artifacts are collected using the Redline Comprehensive Collector. To help narrow a complete timeline down to a manageable subset of data we have provided you with a trio of powerful filtering capabilities: Field Filters, TimeWrinkle™ and TimeCrunch™.

You can use these filters to shave down your timeline to a manageable number of events . We have made it simple to quickly scan your list for suspicious activity by providing you with a summary column that highlights valuable information about any given event. But do not fear, if you need to investigate any item in further detail, you can select that event and choose the "Show Details" button to view everything we have collected.

Figure 1: The new Timeline view in Redline

Field Filters

Field filters are a straightforward means of excluding or including entire categories of time related events from your view by simply checking those that you care about. For example, we often find that File Accessed events tend to be very noisy. With field filters you can remove them from view until they are absolutely necessary.

Even when you use field filters, we realized that there was still too much data for manual review. Naturally, we turned to our expert professional services consultants to see how they comb through large quantities of time-oriented data. Unable to find a good solution that met their needs, they developed the concepts of TimeWrinkle and TimeCrunch. We worked closely with them to bring these tools to Redline's new Timeline capabilities.

TimeWrinkle™

TimeWrinkle provides you with the means to filter your Timeline view to display only the events that occurred in a set of configurable windows of time. TimeWrinkle comes in two varieties: custom and item-based. Choosing which variety to use and when simply depends on the type of investigative lead you start with.

If you know the general time when suspected malicious activity occurred (whether from a user IT ticket, an IDS log, or some other similar means) you use a custom TimeWrinkle to restrict your timeline to only events that took place around that region of time. If the default five minute window radius is not sufficient, you can adjust it anywhere between 0 and 60 minutes to better suit your needs.

However, if you know something more specific about the suspected malicious activity (such as a file name or MD5 checksum) you can use an item-based TimeWrinkle. Creating an item-based TimeWrinkle will take a selected item (e.g. File, Registry Key, Process, etc.) and narrow your timeline to events that take place around any of the of the associated timestamps for that item. To create an item-based TimeWrinkle, right click on any item in the Timeline and select "Add New TimeWrinkle". This will use default settings to generate a TimeWrinkle around the selected item which can then be edited to exclude, include, or adjust any of the individual field windows within the TimeWrinkle.

Figure 2: Timeline Configuration

TimeCrunch™

There can still be times where you have too much data to review manually even after you have trimmed your timeline to a narrow window using TimeWrinkles. To further aid you in reducing your data, we also provide the ability to trim out a minute worth of events for a specific field by using a TimeCrunch. TimeCrunches are useful when Field Filters (being applied so broadly) are detrimental to your investigation, and instead you need to remove specific event types in a more surgical manner.

The most common example of this is when an antivirus scan updates the file accessed timestamp on a very large number of files in a very short amount of time. When this occurs, the file accessed timestamp will become too noisy to be of investigative use for the window in which the antivirus scan ran. Applying a TimeCrunch can quickly exclude a minute worth of this cluttersome data without losing potentially relevant file accessed timestamps elsewhere in your timeline as with Field Filters.

Figure 3: TimeCrunch

Search

Now you are probably saying to yourself, "Timeline sounds awesome, but what do I do if my investigative lead is not something that is time based?" For example, what do you do if your lead is a potentially compromised credit card number. No worries, we have considered you as well.

Now standard on every list shaped view in Redline is a full featured search capability. For example, by using the search feature you can quickly search the entire list of strings from all processes in memory to determine if the suspect credit card number was present. If any matches are found, we will highlight and scroll to each of them. And if your investigative lead is less specific (e.g. you suspect that some unknown credit card numbers may have been stolen) we also allow you to specify your search criteria as a Regular Expression.

For our current Redline users, upgrade to this latest version of the freeware tool to take advantage of the new features. For new users, don't wait another minute to download Redline and get your hands on this great set of analysis tools. Please be sure to check out the updated user documentation for more details and drop by the Redline section of the Mandiant forums to give us feedback on your experience and post any questions you may have.

Lastly, be sure to tune in to our upcoming This entry was posted in Free tools, Freeware, Indicator of Compromise, IoC, Redline, Ted Wilson on by .

Freeware Release: IOC Editor (IOCe) v2.2.0

The updated release of IOC Editor has been a long time coming, but it is well worth the upgrade. There have always been grumblings about IOC Editor, but lately those grumblings have been growing louder. The noise eventually got so loud that even my noise canceling headphones couldn't silence it. I could have just turned up the music a little more, but instead I decided to grab the code and fix some of the issues once and for all.

Some of the changes to IOC Editor were minor (Ctrl-X no longer exits the program), some were major (check out the new Properties panel where commenting per IndicatorItem is now available), and of course I discovered some bugs that I didn't even know existed (No, I didn't cause them with my other changes). Long story short, there are a lot of new features and improvements in this version.

Make sure you pay attention to what's different and any messages that pop-up as the wording and options may have changed.

In this blog post, I am going to touch on a couple of new features and improvements. If you would like to see the full list, please check-out the release notes. Fair warning, there is a lot to read as I did get a bit fix happy.

Some of the New:

Options Dialog

This is still a bit of a work in progress, but there is now a place to set some of the defaults for IOC Editor.

Figure 1: Option Dialog

Figure 1 shows the preferences that are currently available in the options dialog. You can set the default author (we typically use the e-mail address to identify the author), and set whether you want to warn on Deleting or Pruning.

Now before we go too far, let me quickly discuss the difference between deleting and pruning:

Deleting will only delete the selected item, whether it's an individual item, or a logic item (AND, OR). If you delete a logic item, the items that are below it will be bumped up a level.

Figure 2: Before DeleteFigure 3: After Delete

Figure 2 shows what the Indicator of Compromise (IOC) looks like before deleting the AND. Figure 3 shows what the IOC looks like after deleting the AND. Notice that the items that were under the AND were moved up a level and are now under the top OR.

There is no problem with this IOC, but what happens if you were to delete the AND in the following IOC?

Figure 4: Before Delete

If you delete the AND that is shown in Figure 4, you will end up with the "File Name is not good_file.exe" under the top OR. If you were to search your enterprise for all files that were not "good_file.exe", that would be very, very bad.

Now, onto pruning. The Prune option is only available for logic items (ANDs and ORs) and will remove the entire logic branch.

Going back to the example shown in Figure 4, if you Prune the selected AND, the AND, in addition to all items under it, will be deleted.

Figure 5: After Prune

You can see in Figure 5 that the only items that remain were directly under the top OR.

Now that we have that cleared up, let's continue with the rest of the cool new stuff in IOC Editor.

Keyboard Shortcuts

Everybody loves keyboard shortcuts; the less we have to type-out, the better. With that in mind, I've gone through and added shortcuts where appropriate. Below is a list of the shortcuts currently available in the editor:

Shortcuts

Add Buttons (AND, OR, Item)

The buttons to add an AND, OR, or Item have been moved to a menu bar above the definition area. This takes up less space and allows for future expansion.

Figure 6: New Add Buttons

If you just click the button, the "Item" button remembers what you recently added. It also has a drop down with a list of terms (same as right-clicking in the definition area and selecting "Add Item.")

Properties Panel

The properties panel is an area to the right of the definition area that will show all pertinent details of the selected item. By default, the properties panel is not shown. To display it, either click the button or use the Ctrl+P keyboard shortcut.

For those who have used Visual Studio before, the properties panel will look very familiar.

Figure 7: Properties

Figure 7 shows the details that are given for a selected item. Not all of the items are editable from here, but it at least gives you a view into what's going on behind-the-scenes.

You can also edit/add comments for the item. With this upgrade, you can add a comment to individual items, such as an MD5, that lets someone know what file it actually goes with.

Improvements:

In addition to the new features, there were a number of improvements (aka bug fixes).

Unsaved Changes Warning

An important change was the rewording of the unsaved changes dialog box that better aligns with other standard Windows applications.

NOTE: Read this carefully as this has changed from the previous version of IOC Editor.

Figure 8: Unsaved changes

Figure 8 shows the new wording presented when exiting IOC Editor with unsaved changes.

  • Yes - saves changes and exits
  • No - does not save changes and exits
  • Cancel - cancels the exit process and does not save changes

Description Improvements

The Description field has been tweaked a bit, and now allows for proper display of carriage returns and tabs. You now don't have to deal with descriptions that look like this:

"This malware is evil⌷Why would anyone have this⌷Report Immediately"

Too Many More

As I stated earlier in this post, I went a little bug fix crazy. If you're interested in a full list of bug fixes, please click here.

The Future:

There are still plenty of other features and enhancements that I would like to make to IOC Editor, such as ioc_lint integration, additional options, etc. I'm also looking forward to your feedback on the current updates/additions.

Highlighter Super Users Series: Post 1

The Highlighter™ Super Users series is a little something I've put together to reach out to the Highlighter community. As a user of this freeware tool from Mandiant, I want you to know there are many users out there who can help you get through your log analysis paralysis. This series is meant to highlight (see what I did there?) how some users have solved a various range of problems using Highlighter. These interviews will provide insight into the benefits and pitfalls of using Highlighter, some features you may not be aware of, and a few use cases you may not have considered.

Super User Interview #1: Ken Johnson

Ken Johnson is one of Highlighter's Twitter-friendly users. He is a malware analyst and incident responder extraordinaire; fighting evil one keyword search at a time. Known as @patories on Twitter, I reached out to him and asked some questions about his experience using Highlighter.

  1. Name
    Ken Johnson
  2. Realm of work
    My primary work is focused on malware analysis and incident response. Occasionally I also do some forensics work.
  3. How did you hear about Highlighter?
    I first saw Highlighter when I was familiarizing myself with free tools. I have used Memoryze™ previously.
  4. Do you know of any other tools that do what Highlighter does?
    Highlighter is the only tool I know of, and it does what I need so I haven't looked for others.
  5. How do you normally use Highlighter?
    I use Highlighter to trim out known good traffic from proxy logs. This helps get to the unknown stuff quicker. When logs can be multiple gigabytes this is a time saver.
  6. Can you describe one scenario in which Highlighter helped you find evil and/or solve crime?
    On more than one occasion I have used Highlighter to narrow down proxy log traffic to find connections that are malicious. There was an instance about 2 months ago where users fell for a Phish. We used Highlighter to find the C&C IP's that machines kept calling home to, by filtering out what was normal and analyzing what was left. Highlighter helped find almost 50 IP/URLS that were malicious.
  7. On a scale from 1 (worst) to 5 (best), how well does Highlighter address your use case(s)?
    I would have to give Highlighter a 4.
  8. What is missing from Highlighter for your use case(s)?
    I would like to have the ability to whitelist traffic so I do not have to manually keep removing internal hosts that we see. This may be in the program and I have not found it.
  9. What is one Highlighter feature addition that would serve the Information Security community best?
    I think the ability to whitelist hostnames would be a nice addition.
  10. Are you aware of, or have you used, any of the following features:
    • Activity Over Time feature that lets you view log data as a function of Entries Per Day
      No, I was not aware of this one.
    • Ability to change basic font settings for your output
      I know it is there, but for my use this is never used.
  11. Have you ever seen Highlighter used in such a way that your eyeballs melted from all the Awesome?
    I have only seen myself use it, but I have seen my co-workers eyeballs melt when I show them the awesomeness that they can do. Some are still stuck in the grep world...

Keep an eye out for the second post in the Highlighter Super Users Series featuring Russ McRee, author of ISSA Journal's toolsmith series and mastermind behind www.holisticinfosec.org. If you're interested in sharing your own experiences with this tool, please let me know by commenting below.

Memoryze for the Mac: Support Added for OS X Mountain Lion (10.8)

Earlier this year, Mandiant launched a new freeware tool: Memoryze for the Mac™. The tool brings many of the features of Memoryze to the Apple® Macintosh platform, enabling acquisition of memory images via the command-line or a simple GUI. We are excited to announce it now fully supports OS X 10.6-10.8.

Recently, OS X Mountain Lion added kernel Address Space Layout Randomization. It is a welcome security feature, raising the bar for kernel exploitation. This feature adds an extra step into the memory analysis tool. Previously, we could depend on the paging table IdlePML4 and IdlePDPT addresses being at the same physical memory location. With 10.8 and KASLR the physical memory addresses of IdlePML4 and IdlePDPT became BootPML4 and BootPDPT, while IdlePML4 and IdlePDPT are now randomized with ASLR. The boot paging tables do not contain the full kernel virtual memory layout. Since Memoryze for Mac does not depend on any symbol information, we developed a mechanism to uniformly discover the randomized location of the kernel paging tables.

Once again, Mountain Lion has changed the memory location of nsysent. Prior to the change, it was located directly after the sysent table itself. As documented in several locations on the web, this made automated discovery and verification of the table size convenient. Unfortunately, Apple decided to move the location of nsysent, causing us to develop a new sysent size discovery mechanism.

We have a growing list of cool new features to add to Memoryze for Mac, but it may be until after the new year before we are able to dev the features.

M-Unition Podcast: Mandiant’s Redline Tool Makes Incident Response Easy for Experts and Beginners

On today's podcast, Kristen Cooper talks with Lucas Zaichkowsky on the latest version of Redline, a free tool from Mandiant.

The podcast will explain in detail what Redline is capable of, highlighting two features that set it apart from other tools. First, the tool is intuitive enough to be used by novice incident responders, without compromising capabilities that advanced incident responders utilize in the tool. Secondly, the tool is capable of applying Indicators of Compromise (IOC) to data that it collects. This allows Redline to detect evidence of attacks, even though there may be no evidence of active malware on additional computers.

Listen along as Lucas details the product demonstration he performed at Black Hat 2012 that really showcases Redline's unique value.

To listen to the full podcast and learn more about Redline click here.

New Open Source Tool: Audit Parser

Mandiant RedlineTM and IOC Finder TM collect and parse a huge body of evidence from a running system. In fact, they're based on the same agent software as our flagship Mandiant Intelligent Response® product. During the course of their "audits", these tools conduct comprehensive analysis of the file system (including hashing, time stamps, parsing of PE file structures, and digital signature checks), registry hives, processes in memory, event logs, active network connections,DNS cache contents,web browser history, system restore points, scheduled tasks, prefetch entries, persistence mechanisms, and much more.

Once this data is collected, Redline and IOCFinder currently allow you to do one of two things:

  • Review the contents of memory through a visual workflow in Redline
  • Search for Indicators of Compromise (IOCs) and generate a report of "hits"

But what if you want to analyze all of the raw evidence - not just memory or IOC hits - and do traditional forensics and timeline analysis? That's where Audit Parser steps in. It's the newest addition to Mandiant's portfolio of free software.

Audit Parser is simple:it takes the complex XML data produced by Redline or IOCFinder and converts it into human-readable tab-delimited text. You can then easily review the output in Excel, use a dedicated CSV file viewer (we're fans of "CSVed" and"CSVFileView"), import it into a database, or grep / manipulate it to your heart's content.

When paired with Redline's new start-up workflow to build a "collector" script, Audit Parser gives you a complete(and free)live response analysis toolkit. You can customize the Redline collector to gather as much or as little evidence as desired, run it on your target system, and then easily review all of the results following a quick conversion with Audit Parser.

The screen capture below shows Audit Parser's options - it's pretty straightforward to use:

Tabular data in Excel doesn't make for the most exciting screen shots, but we wanted to give you a glimpse into what the output looks like and the extent of evidence available for filtering, sorting, and analysis:

  • A filtered view of a file system audit, showing complete file metadata for all PE files within %SYSTEMROOT% created between 2011-2012 that are not digitally signed.

  • A portion of a prefetch audit, showing how the contents of .PF files are automatically parsed to provide last time executed, # of times executed, and original file path metadata.

  • A portion of a full registry dump, showing review of Active Setup Installed Components registry keys - the data includes all key value / data pairs and last modified dates.

  • A portion of the parsed Windows event logs, showing review of process auditing events including event log source, time generated, event ID, and full event message contents.

The default "comprehensive collector" script in Redline collects all of the artifacts listed above, as well as many more.

But wait - that's not all! Audit Parser also contains timeline generation functionality. Just specify a time & date range, and it will build a sorted timeline of all file system, registry, and event log events that occurred within that period. Future releases will add more audit types and customizability to this feature.

Audit Parser is written in Python and is distributed under the Apache License. It requires the lxml (http://lxml.de/) library. We're also distributing a Windows EXE built with Py2EXE for users that may not have a Python environment set up. You can download the tool and documentation on GitHub at: https://github.com/mandiant/AuditParser

If you have any questions or comments, feel free to leave them below, e-mail me (ryan [dot] kazanciyan [at] mandiant.com), or DM me on Twitter at @ryankaz42. I'll also be at Black Hat USA next week teaching Mandiant's Incident Response course where we'll be going through an in-depth live response analysis lab using Redline, Audit Parser, and other forensic tools. I was on a recent M-Unition podcast discussing the class and how it is completely revamped for 2012. You can listen to the podcast here. Hope to see you there!

Unibody Memory Analysis — Introducing Memoryze™ for the Mac 1.0

Today, Mandiant is introducing a new free tool, Memoryze™ for the Mac 1.0, which brings memory imaging and analysis to the Mac. It joins a growing list of freeware tools Mandiant currently provides.

Memoryze™ for the Mac 1.0 brings many of the features of Memoryze to the Apple Macintosh platform. This new tool enables acquisition of memory images via the command-line or a simple GUI. In addition, Memoryze™ for the Mac 1.0 can perform offline analysis against memory images or live analysis on a running system.

The tool supports the following features:

  • Imaging the full range of system memory
  • Acquiring individual processes memory regions
  • Enumerating all running processes
    • Including those hidden by rootkits

For each process, Memoryze™ for the Mac 1.0 can:

  • Report all open file handles in a process (e.g. all files,sockets, pipes, etc.)
  • List the virtual address space of a process including loaded libraries and allocated portions of heap and execution stack
  • List network connections
    • Active and listening
  • Enumerate
    • All loaded kernel extensions including those hidden by rootkits
    • The System Call Table and Mach Trap Table
    • All running Mach Tasks

Okay, enough of the marketing. Memoryze™ for the Mac 1.0 can be downloaded here. To help get you started we'll present a few of the features in this blog post.

For offline analysis your first step is going to be acquiring memory. The Mac Memory Dumper App makes this process as simple as pushing a button. To begin the acquisition, Memoryze™ for the Mac 1.0 will require you to authenticate so that the application can load a memory dumping driver. After selecting the location to store the image and authenticating, Memoryze™ for the Mac 1.0 will begin the acquisition process. The tool will provide you with a progress bar and an image size monitor for each of your acquisitions (fancy, huh?).

Note that the final size of the dumped image may exceed the size of your physical RAM. If the system has 8GB of physical RAM installed the dump may be 10GB. You may ask yourself, "Self, why is the dumped image bigger than my actual memory size?" There are regions that are physically addressable but are not part of actual DRAM, pesky memory-mapped devices. These regions are written to the image file as 0x0-bytes to help preserve the correct offsets within the image.

Once Memoryze™ for the Mac 1.0 finishes the acquisition; we can use it to perform memory analysis(note Memoryze™ for the Mac 1.0 can also perform analysis on images acquired by other tools). With our data ready, let's run through several of the process analysis features we mentioned above.

We'll start by performing a basic process listing based on the memory image we just created. Execute the command below:

macmemoryze proclist -f ~/Documents/my.mem 2>err.txt

Memoryze™ for the Mac 1.0 will open "my.mem" for analysis and detect two critical pieces of information about the image: the operating system version and whether the system is running 32 or 64-bit kernel. Armed with this information the proclist analysis module locates the operating system data structure that maintains the list of running processes.

If you take a look at the output below, you can see that Memoryze™ for the Mac 1.0 extracts the PADDR, or physical address, of each process (this is also the offset of the process in the acquired memory image file). You can use the PADDR to quickly locate the process in question in an offline tool such as a hex editor. Memoryze™ for the Mac 1.0 also extracts other standard identifying information such as the process NAME, PID and parent PID (PPID). In addition, the tool provides the start time for each process in UTC. Finally, Memoryze™ for the Mac 1.0 extracts each process' associated USERNAME, effective userid (EUID), and real userid (RUID).

Based on the process listing above, we may be interested in getting a more complete understanding of what a particular process may be doing. We can list file descriptors and memory sections for all of the processes in the listing, but this would get pretty lengthy and present too much information at once. Using filters we can limit display to a smaller subset or a single process. We can use the "-n" option to filter processes by NAME or the "-p" to filter processes by PID.Execute the following command:

macmemoryze proclist -w -p 14 ~/Documents/my.mem 2>err.txt

In the image above, we show a snippet of a file descriptor listing for PID 14 using the command-line switch "-p 14". This shows us all open file descriptors for the given process. This includes files, UNIX domain sockets, networking sockets (such as TCP), and so on. For several of the types/subtypes, Memoryze™ for the Mac 1.0 will provide a value associated with the item type. The value for a descriptor of type FILE is the filename associated with the file descriptor while the value for a type SOCKET subtype TCP is the source IP address, destination IP address, and associated ports.

Now that we've completed some filtering, let's dig a little deeper and perform detailed analysis of the "notifyd" process. The image below contains a snippet of its memory sections listing. Memoryze™ for the Mac 1.0 shows us the start and end virtual addresses and a human-readable size for each section. For some memory sections, Memoryze™ for the Mac 1.0 also provides a type, such as MALLOC or IOKIT. These types provide insight into the purpose of the memory section. For other memory sections, Memoryze™ for the Mac 1.0 displays the filename that is located at (or was used to initialize) that particular memory region. Please execute:

macmemoryze proclist -s -p 14 ~/Documents/my.mem 2>err.txt

Neat, huh? So now we've analyzed the standard operating system (OS) process list structure. If it's good enough for the OS, it's good enough for us, right? Not really. It's fairly trivial for malware to unlink itself from the process list that the OS maintains. In light of this, Memoryze™ for the Mac 1.0 provides a process carving feature that allows it to enumerate and analyze processes based on their signature in memory. This means that Memoryze™ for the Mac 1.0 does not have to depend on the OS to provide a list of processes. This enables Memoryze™ for the Mac 1.0 to discover processes that have been hidden from standard OS listings. This same carving feature extends to the kextlist and syscalllist Memoryze™ for the Mac 1.0 features, allowing you to discover other hidden data within the OS.

Now don't forget, Memoryze™ for the Mac 1.0 can be run offline using an acquired memory image, or live, analyzing the running system in real-time. Below we show a system call table listing using the live memory analysis feature of Memoryze™ for the Mac 1.0. Notice the missing "-f" option. We can use this listing to check for system call table hooking. System call table hooking allows attackers to surreptitiously monitor or filter user-level programs interactions with the OS kernel. This is commonly used to hide files and network connections from user-level programs. The syscalllist feature also supports discovery and listing of the Mach Trap table. Mach Trap, what? The mach trap table is analogous to the system call table in the BSD portion of OS X, but within the Mach portion of OS X. So we want to ensure we can check for possible hooking within that table also. To perform a live listing execute:

sudo macmemoryze syscalllist -s

In order to hook the syscall table we would probably want to use a driver. Have no fear! Memoryze™ for the Mac 1.0 can carve loaded kexts from memory. A decent malware author would probably want to hide itself from the OS by unlinking from internal data structures. To combat this, Memoryze™ for the Mac 1.0 will parse live memory or a dead file to find loaded drivers, as shown in the screenshot below.

Memoryze™ for the Mac 1.0 supports an XML output option ("-x") that will create an XML file with an extended version of the console output, only in XML format. "Extended version," you ask? There is only so much console real estate. We must make decision about what information to display. So therefore, stuff gets left out. For example, we parse the full file path of the executing process and the process arguments. These are not displayed in the console output, but are accessible if using the XML output option. These XML files can also be loaded into Mandiant Redline™ (currently windows only, boo!) for viewing in a GUI.

There are other features of Memoryze™ for the Mac 1.0 that we have not detailed here, but we don't want to give you all the answers. What's the fun in that? We really want you to use the tool and provide us with some feedback on features, interface, usages, and so on.

Memoryze™ for the Mac 1.0 currently supports:

  • Mac OS X Snow Leopard (10.6) 32/64-bit
  • Mac OS X Lion (10.7) 32/64-bit

We hope to continue to improve the state of Mac memory analysis for incident responders and security professionals.

"Mac" is a trademark of The Apple Corporation. Mandiant is not affiliated with or endorsed by The Apple Corporation.

Exploring Symbol Type Information with PdbXtract

Mandiant is introducing a new free tool today, PdbXtract™, which allows you to browse and search PDB-type information.

PdbXtract allows you to explore symbolic type information as extracted from Microsoft PDB files. This tool is primarily designed for reverse engineering Windows-based applications and for exploring the internals of Windows kernel components. You can download PdbXtract.

A programming database (PDB) file is a binary file containing program debug information in a Microsoft-proprietary format. This file is produced by the compiler/linker when a program is built. The information it contains is used by debuggers to debug a program and can greatly assist a developer in debugging program issues by resolving function pointers to symbolic names, for example.

Perhaps the most useful and richest source of debugging information contained in PDBs is type data which holds detailed information about data structures, constants, and other named symbols. While this information is primarily used to debug program components, it can also be used to gain insight into how core operating system components work by observing both the format of the data structure and how the structure is used.

PdbXtract is not a pure PDB parser. It only extracts type information using Microsoft's DebugInterface Access (DIA) COM. If you are interested in just parsing/dumping raw PDB information, there are a few alternatives out there to DIA, including Volatility's open source pdbparse (http://code.google.com/p/pdbparse/) or the PDB utility that comes with the Undocumented Windows 2000 Secrets book. However, most of the practical tools I have seen that operate on PDB's use DIA, including Microsoft's own Dia2dump, this one http://www.codeproject.com/Articles/37456/How-To-Inspect-the-Content-of-a-Program-Database-P and this one http://www.ishani.org/web/articles/obsolete/pdb-cracking-tool/, to name a few. To reiterate, PdbXtract does not parse or capture the wealth of other information available in a PDB, including: functions, debug streams, modules, publics, globals, files, section information, injected sources, source files, OEM specific types, compilands, and others.

The tools mentioned above are fine for inspecting the contents of a single PDB. However, often times as part of my job in R&D, we have to use knowledge of type information across all supported Windows operating systems to implement features. For example, if you are dealing with partially undocumented or "opaque" types (example: you need to walk the PEB's InInitializationOrderModuleList to obtain a list of loaded modules in a process) or have full source type information but do not want to tie your program to a specific version of those types as implemented in the headers of the SDK you are compiling against, you probably want to just use static offsets such as:

PVOID NextMod=(PVOID ((DWORD_PTR)PebPtr+InInitOrderModList_Offset+Flink_Offset);

The problem has always been: how do I get the value of InInitOrderModList_Offset for all platforms we support, taking into account 32-bit/64-bit variations? The answer has always been: useWinDbg (or if you are interested in possibly-correct kernel symbols only, you might use Matt Suiche's Moonsols library (http://msdn.moonsols.com )). Launch a VM for each OS you want to support, attach with a debugger, and use the power of WinDbg to extract the type information. Well, WinDbg's magical "dt" command just relies on the PDB information for the corresponding binary (after retrieving the necessary symbol files from your local symbol store and optionally the Microsoft public symbol server), so it stands to reason that we should be able to do the same. The end goal is to make a searchable database for all the exported types of OS binaries we care about, so that we don't have to constantly relive the tedium of doing this in WinDbg.

Features

PdbXtract has two main features: exploring a single PDB (PDB Explorer) and searching a library of PDBs for one or more operating systems. PDB Explorer opens the PDB, parses type information using DIA, and displays a list of all structs, unions and enums. If you click on one of the types, a C-style struct (with offsets) definition will be displayed in the text area to the right, as shown below for the type IMAGE_FILE_HEADER.

The library tab allows you to create and search a library of PDB type information. I have created a library for *most* of the operating systems we support for the following important system binaries: kernel (ntoskrnl.exe, ntkrpamp, ntkrnlpa, ntkrnlmp),ndis.sys, win32k.sys and hal (hal.dll, halaacpi.dll, halmacpi.dll). You might ask why other system DLLs were not included, such askernel32, user32, advapi32, etc.The answer to that being the corresponding PDBs for those binaries you get off the symbol server are stripped of type information. Why? Because Microsoft expects you to use their headers when you compile your application, and thus your program'sPDB will have the necessary type information.

The library included with PdbXtract covers several of Microsoft's major operating system releases, but you can easily add more symbols. PdbXtract includes a utility, called PdbFetch, that simply runs Microsoft's symchk utility to grab the symbols for the file names you supply (usage: pdbfetch, where is a text file that contains a list of full paths to system binaries you want to retrieve symbols for). Pdbfetch creates a "PDB set" which consists ofthe directory structure with containing PDBs as created by symchkplus a manifest.xml file which summarizes the OSplatform information. To use a PDB set in PdbXtract, go to the library tab and click "new" if you want to create a new library from the PDB set or "add" if you want to add them to an existing library. Once you create/add a PDB set to a library you can delete them - the only thing that matters is the sqlite .pdbx database that's created.

Perhaps someone out there will find this useful and maybe create a searchable web front end with the resulting SQLite database? The sky is the limit. Let me know if you do by commenting below.

As a final note, you might wonder why you can't just download the entire symbol packages from Microsoft, which include every symbol file on the MS Symbol server, and create a ginormous library. Why is there a requirement to acquire the PDBs using pdbfetch? The answer is - you could do that - but it is data overload(several GB of PDBs) when you will not care about 99% of them. Plus it is easier to capture OS platform/build info at run time rather than guessing at it from the name of the symbol package installer (PDBs give no indication of what OS the corresponding binary originated from).

The Latest Version of Redline Finds Indicators of Compromise and More

We are on a roll with our freeware. The latest version of Redline is now available! For those who are not familiar with Redline - you may be asking, what is it? Simply put, Redline brings together analysis tools which help you perform a guided investigation of a potentially compromised system. And did we mention that it is free?

This latest and greatest version of Redline includes some awesome new features, courtesy of recommendations from our strong and growing user base and input from internal users here at Mandiant. For those who have been loyal Redline users, you will find that it is no longer just a memory forensics tool! It has grown into a multi-purpose product for creating Indicators of Compromise (IOC) and matching them across all types of host data, while maintaining all the traditional memory forensics capabilities that you're used to.

Get the data that matters, and do it faster

  • With Redline, you can now include and search for Indicators of Compromise and create a searchable report detailing any suspicious activity found matching those IOCs. Need more on what IOCs are? Click here for more information.
  • Specify a set of IOCsbefore collection and Redline will now help tailor the configuration to provide meaningful search resultsand ensure that all the data required by the chosen IOCs is collected, speeding up your time to completion.
  • Not sure if the IOCs you have chosen are the ones you want? Not to worry! When choosing indicators to search for, there is now a handy preview window to see the detailed information of each indicator.
  • You are no longer limited to just memory data. Redline now enables you to configure and collect a much broader range of data about the target host, such as event logs and file listings. This data will in turn be searchable using the new Indicator of Compromise search options, providing you with better overall search results.

Multi-task with the best

  • With Redline you can now perform investigations while searching for indicators - at the same time! For example, while the session is still matching IOCs, you can start diving into the Malware Risk Indicator (MRI) Scores and start anew investigation or even continue an existing investigation.
  • Now there's no guessing where you are in the process. You can check the progress of your investigation at any time via "Background Tasks" in the main menu. You will also receive a notification when one of your background tasks has been scheduled.

For our current users, be sure to upgrade to this latest version of Redline to take advantage of the new features. For new users, don't wait another minute to download Redline and get your hands on this great set of analysis tools.