Category Archives: Free tools

Highlighter Super Users Series: Post 2

Back in November I published the first interview from the Highlighter™ Super Users blog series. My goal with this series is to shed some light on all the great things that can be achieved using this freeware tool. In part 2, I interviewed toolsmith author and HolisticInfoSec.org webmaster, Russ McRee.

Super User Interview #2: Russ McRee

Russ McRee is the author of ISSA Journal's toolsmith series and runs HolisticInfoSec.org. In October 2011 Russ contacted me to discuss Highlighter in that month's issue of the ISSA Journal, and later for the nomination of Highlighter for the 2011 Toolsmith Tool of the Year. As someone who has analyzed Highlighter's effectiveness as a forensics tool for his own articles, I asked him to answer a few questions based on his experience with the freeware tool.

  1. Name
    Russ McRee
  2. Realm of work
    Security Analytics (security incident management, security monitoring, attack and penetration testing).
  3. How did you hear of Highlighter?
    I watch the websites and check for tool updates.
  4. Do you know of any other tools that do what Highlighter does?
    Log Parser, Log Parser Lizard, Log Parser Studio, Splunk
  5. How do you normally use Highlighter?
    I mainly use Highlighter for Log analysis, forensic investigations, demonstrations and research (see http://www.youtube.com/watch?v=w0uOCOINrWY and https://www.sans.org/reading_room/whitepapers/logging/evil-lens-web-logs_33950)
  6. Can you describe one scenario in which Highlighter helped you find evil and/or solve crime?
    I had a recent mysterious case of core utility files and binaries gone missing from very important infrastructure management servers that initially looked malicious and intentional. Using Highlighter for analysis of Windows event logs led to the discovery of a sync job gone awry (misconfiguration) in the Application log via time stamp matching and keyword highlights.
  7. On a scale from 1 (worst) to 5 (best), how well does Highlighter address your use case(s)?
    4
  8. What is missing from Highlighter for your use case(s)?
    Word wrap option
  9. What is one Highlighter feature addition that would serve the Information Security community best?
    Potential DB support
  10. Are you aware of, or have you used, any of the following features:
    • Activity Over Time feature that lets you view log data as a function of Entries Per Day
      No, I was not aware.
    • Hotkeys feature
      Yes, I was aware of this feature.
    • Ability to change basic font settings for your output
      Yes, I was aware of this feature.
  11. Have you ever seen Highlighter used in such a way that your eyeballs melted from all the Awesome?
    My eyeballs melted from the awesome when I stuffed Highlighter with a 2.44GB Swatch log file during large file testing while writing October 2011's toolsmith. It took a little time to load and format (to be expected), but it handled 24,502,412 log entries admirably (no choking). I threw a query for a specific inode at it and Highlighter tagged 1930 hits across 25 million+ lines in ten minutes.

Keep an eye out for the final post in the Highlighter Super Users Series. If you're interested in sharing your own experiences with this tool, please let me know by commenting below.

Freeware Release: Redline 1.7

Redline™ is Mandiant's free tool for investigating hosts for signs of malicious activity through memory and file analysis, and subsequently developing a threat assessment profile. It combines configurable collection of Mandiant's full range of forensic artifacts (the same set available to our enterprise product, Mandiant Intelligent Response®, guided analysis, Mandiant's Custom Malware Risk Index (MRI) scoring, and Indicator of Compromise (IOC) matching) to provide you with the tools needed to rapidly triage a potentially compromised host.

Recently, we released Redline version 1.7. To make this the most compelling release to date, we focused on two of the most frequently requested features: Timeline and Search.

Timeline

With Timeline, we wanted to do more than just slap together all date related items into a staggering list that buries you under more data than is humanly possible to review. Even on infrequently used systems, hundreds of thousands of dated artifacts are collected using the Redline Comprehensive Collector. To help narrow a complete timeline down to a manageable subset of data we have provided you with a trio of powerful filtering capabilities: Field Filters, TimeWrinkle™ and TimeCrunch™.

You can use these filters to shave down your timeline to a manageable number of events . We have made it simple to quickly scan your list for suspicious activity by providing you with a summary column that highlights valuable information about any given event. But do not fear, if you need to investigate any item in further detail, you can select that event and choose the "Show Details" button to view everything we have collected.

Figure 1: The new Timeline view in Redline

Field Filters

Field filters are a straightforward means of excluding or including entire categories of time related events from your view by simply checking those that you care about. For example, we often find that File Accessed events tend to be very noisy. With field filters you can remove them from view until they are absolutely necessary.

Even when you use field filters, we realized that there was still too much data for manual review. Naturally, we turned to our expert professional services consultants to see how they comb through large quantities of time-oriented data. Unable to find a good solution that met their needs, they developed the concepts of TimeWrinkle and TimeCrunch. We worked closely with them to bring these tools to Redline's new Timeline capabilities.

TimeWrinkle™

TimeWrinkle provides you with the means to filter your Timeline view to display only the events that occurred in a set of configurable windows of time. TimeWrinkle comes in two varieties: custom and item-based. Choosing which variety to use and when simply depends on the type of investigative lead you start with.

If you know the general time when suspected malicious activity occurred (whether from a user IT ticket, an IDS log, or some other similar means) you use a custom TimeWrinkle to restrict your timeline to only events that took place around that region of time. If the default five minute window radius is not sufficient, you can adjust it anywhere between 0 and 60 minutes to better suit your needs.

However, if you know something more specific about the suspected malicious activity (such as a file name or MD5 checksum) you can use an item-based TimeWrinkle. Creating an item-based TimeWrinkle will take a selected item (e.g. File, Registry Key, Process, etc.) and narrow your timeline to events that take place around any of the of the associated timestamps for that item. To create an item-based TimeWrinkle, right click on any item in the Timeline and select "Add New TimeWrinkle". This will use default settings to generate a TimeWrinkle around the selected item which can then be edited to exclude, include, or adjust any of the individual field windows within the TimeWrinkle.

Figure 2: Timeline Configuration

TimeCrunch™

There can still be times where you have too much data to review manually even after you have trimmed your timeline to a narrow window using TimeWrinkles. To further aid you in reducing your data, we also provide the ability to trim out a minute worth of events for a specific field by using a TimeCrunch. TimeCrunches are useful when Field Filters (being applied so broadly) are detrimental to your investigation, and instead you need to remove specific event types in a more surgical manner.

The most common example of this is when an antivirus scan updates the file accessed timestamp on a very large number of files in a very short amount of time. When this occurs, the file accessed timestamp will become too noisy to be of investigative use for the window in which the antivirus scan ran. Applying a TimeCrunch can quickly exclude a minute worth of this cluttersome data without losing potentially relevant file accessed timestamps elsewhere in your timeline as with Field Filters.

Figure 3: TimeCrunch

Search

Now you are probably saying to yourself, "Timeline sounds awesome, but what do I do if my investigative lead is not something that is time based?" For example, what do you do if your lead is a potentially compromised credit card number. No worries, we have considered you as well.

Now standard on every list shaped view in Redline is a full featured search capability. For example, by using the search feature you can quickly search the entire list of strings from all processes in memory to determine if the suspect credit card number was present. If any matches are found, we will highlight and scroll to each of them. And if your investigative lead is less specific (e.g. you suspect that some unknown credit card numbers may have been stolen) we also allow you to specify your search criteria as a Regular Expression.

For our current Redline users, upgrade to this latest version of the freeware tool to take advantage of the new features. For new users, don't wait another minute to download Redline and get your hands on this great set of analysis tools. Please be sure to check out the updated user documentation for more details and drop by the Redline section of the Mandiant forums to give us feedback on your experience and post any questions you may have.

Lastly, be sure to tune in to our upcoming This entry was posted in Free tools, Freeware, Indicator of Compromise, IoC, Redline, Ted Wilson on by .

Freeware Release: IOC Editor (IOCe) v2.2.0

The updated release of IOC Editor has been a long time coming, but it is well worth the upgrade. There have always been grumblings about IOC Editor, but lately those grumblings have been growing louder. The noise eventually got so loud that even my noise canceling headphones couldn't silence it. I could have just turned up the music a little more, but instead I decided to grab the code and fix some of the issues once and for all.

Some of the changes to IOC Editor were minor (Ctrl-X no longer exits the program), some were major (check out the new Properties panel where commenting per IndicatorItem is now available), and of course I discovered some bugs that I didn't even know existed (No, I didn't cause them with my other changes). Long story short, there are a lot of new features and improvements in this version.

Make sure you pay attention to what's different and any messages that pop-up as the wording and options may have changed.

In this blog post, I am going to touch on a couple of new features and improvements. If you would like to see the full list, please check-out the release notes. Fair warning, there is a lot to read as I did get a bit fix happy.

Some of the New:

Options Dialog

This is still a bit of a work in progress, but there is now a place to set some of the defaults for IOC Editor.

Figure 1: Option Dialog

Figure 1 shows the preferences that are currently available in the options dialog. You can set the default author (we typically use the e-mail address to identify the author), and set whether you want to warn on Deleting or Pruning.

Now before we go too far, let me quickly discuss the difference between deleting and pruning:

Deleting will only delete the selected item, whether it's an individual item, or a logic item (AND, OR). If you delete a logic item, the items that are below it will be bumped up a level.

Figure 2: Before DeleteFigure 3: After Delete

Figure 2 shows what the Indicator of Compromise (IOC) looks like before deleting the AND. Figure 3 shows what the IOC looks like after deleting the AND. Notice that the items that were under the AND were moved up a level and are now under the top OR.

There is no problem with this IOC, but what happens if you were to delete the AND in the following IOC?

Figure 4: Before Delete

If you delete the AND that is shown in Figure 4, you will end up with the "File Name is not good_file.exe" under the top OR. If you were to search your enterprise for all files that were not "good_file.exe", that would be very, very bad.

Now, onto pruning. The Prune option is only available for logic items (ANDs and ORs) and will remove the entire logic branch.

Going back to the example shown in Figure 4, if you Prune the selected AND, the AND, in addition to all items under it, will be deleted.

Figure 5: After Prune

You can see in Figure 5 that the only items that remain were directly under the top OR.

Now that we have that cleared up, let's continue with the rest of the cool new stuff in IOC Editor.

Keyboard Shortcuts

Everybody loves keyboard shortcuts; the less we have to type-out, the better. With that in mind, I've gone through and added shortcuts where appropriate. Below is a list of the shortcuts currently available in the editor:

Shortcuts

Add Buttons (AND, OR, Item)

The buttons to add an AND, OR, or Item have been moved to a menu bar above the definition area. This takes up less space and allows for future expansion.

Figure 6: New Add Buttons

If you just click the button, the "Item" button remembers what you recently added. It also has a drop down with a list of terms (same as right-clicking in the definition area and selecting "Add Item.")

Properties Panel

The properties panel is an area to the right of the definition area that will show all pertinent details of the selected item. By default, the properties panel is not shown. To display it, either click the button or use the Ctrl+P keyboard shortcut.

For those who have used Visual Studio before, the properties panel will look very familiar.

Figure 7: Properties

Figure 7 shows the details that are given for a selected item. Not all of the items are editable from here, but it at least gives you a view into what's going on behind-the-scenes.

You can also edit/add comments for the item. With this upgrade, you can add a comment to individual items, such as an MD5, that lets someone know what file it actually goes with.

Improvements:

In addition to the new features, there were a number of improvements (aka bug fixes).

Unsaved Changes Warning

An important change was the rewording of the unsaved changes dialog box that better aligns with other standard Windows applications.

NOTE: Read this carefully as this has changed from the previous version of IOC Editor.

Figure 8: Unsaved changes

Figure 8 shows the new wording presented when exiting IOC Editor with unsaved changes.

  • Yes - saves changes and exits
  • No - does not save changes and exits
  • Cancel - cancels the exit process and does not save changes

Description Improvements

The Description field has been tweaked a bit, and now allows for proper display of carriage returns and tabs. You now don't have to deal with descriptions that look like this:

"This malware is evil⌷Why would anyone have this⌷Report Immediately"

Too Many More

As I stated earlier in this post, I went a little bug fix crazy. If you're interested in a full list of bug fixes, please click here.

The Future:

There are still plenty of other features and enhancements that I would like to make to IOC Editor, such as ioc_lint integration, additional options, etc. I'm also looking forward to your feedback on the current updates/additions.

Highlighter Super Users Series: Post 1

The Highlighter™ Super Users series is a little something I've put together to reach out to the Highlighter community. As a user of this freeware tool from Mandiant, I want you to know there are many users out there who can help you get through your log analysis paralysis. This series is meant to highlight (see what I did there?) how some users have solved a various range of problems using Highlighter. These interviews will provide insight into the benefits and pitfalls of using Highlighter, some features you may not be aware of, and a few use cases you may not have considered.

Super User Interview #1: Ken Johnson

Ken Johnson is one of Highlighter's Twitter-friendly users. He is a malware analyst and incident responder extraordinaire; fighting evil one keyword search at a time. Known as @patories on Twitter, I reached out to him and asked some questions about his experience using Highlighter.

  1. Name
    Ken Johnson
  2. Realm of work
    My primary work is focused on malware analysis and incident response. Occasionally I also do some forensics work.
  3. How did you hear about Highlighter?
    I first saw Highlighter when I was familiarizing myself with free tools. I have used Memoryze™ previously.
  4. Do you know of any other tools that do what Highlighter does?
    Highlighter is the only tool I know of, and it does what I need so I haven't looked for others.
  5. How do you normally use Highlighter?
    I use Highlighter to trim out known good traffic from proxy logs. This helps get to the unknown stuff quicker. When logs can be multiple gigabytes this is a time saver.
  6. Can you describe one scenario in which Highlighter helped you find evil and/or solve crime?
    On more than one occasion I have used Highlighter to narrow down proxy log traffic to find connections that are malicious. There was an instance about 2 months ago where users fell for a Phish. We used Highlighter to find the C&C IP's that machines kept calling home to, by filtering out what was normal and analyzing what was left. Highlighter helped find almost 50 IP/URLS that were malicious.
  7. On a scale from 1 (worst) to 5 (best), how well does Highlighter address your use case(s)?
    I would have to give Highlighter a 4.
  8. What is missing from Highlighter for your use case(s)?
    I would like to have the ability to whitelist traffic so I do not have to manually keep removing internal hosts that we see. This may be in the program and I have not found it.
  9. What is one Highlighter feature addition that would serve the Information Security community best?
    I think the ability to whitelist hostnames would be a nice addition.
  10. Are you aware of, or have you used, any of the following features:
    • Activity Over Time feature that lets you view log data as a function of Entries Per Day
      No, I was not aware of this one.
    • Ability to change basic font settings for your output
      I know it is there, but for my use this is never used.
  11. Have you ever seen Highlighter used in such a way that your eyeballs melted from all the Awesome?
    I have only seen myself use it, but I have seen my co-workers eyeballs melt when I show them the awesomeness that they can do. Some are still stuck in the grep world...

Keep an eye out for the second post in the Highlighter Super Users Series featuring Russ McRee, author of ISSA Journal's toolsmith series and mastermind behind www.holisticinfosec.org. If you're interested in sharing your own experiences with this tool, please let me know by commenting below.

Memoryze for the Mac: Support Added for OS X Mountain Lion (10.8)

Earlier this year, Mandiant launched a new freeware tool: Memoryze for the Mac™. The tool brings many of the features of Memoryze to the Apple® Macintosh platform, enabling acquisition of memory images via the command-line or a simple GUI. We are excited to announce it now fully supports OS X 10.6-10.8.

Recently, OS X Mountain Lion added kernel Address Space Layout Randomization. It is a welcome security feature, raising the bar for kernel exploitation. This feature adds an extra step into the memory analysis tool. Previously, we could depend on the paging table IdlePML4 and IdlePDPT addresses being at the same physical memory location. With 10.8 and KASLR the physical memory addresses of IdlePML4 and IdlePDPT became BootPML4 and BootPDPT, while IdlePML4 and IdlePDPT are now randomized with ASLR. The boot paging tables do not contain the full kernel virtual memory layout. Since Memoryze for Mac does not depend on any symbol information, we developed a mechanism to uniformly discover the randomized location of the kernel paging tables.

Once again, Mountain Lion has changed the memory location of nsysent. Prior to the change, it was located directly after the sysent table itself. As documented in several locations on the web, this made automated discovery and verification of the table size convenient. Unfortunately, Apple decided to move the location of nsysent, causing us to develop a new sysent size discovery mechanism.

We have a growing list of cool new features to add to Memoryze for Mac, but it may be until after the new year before we are able to dev the features.

Using AuditParser to Process and Analyze Large Volumes of Data Collected with Redline

In this blog post, I am going to show you some ways to review data that have been collected with the Mandiant Redline™ tool, without using the Redline interface. I will be using Mandiant's AuditParser™ tool in order to transform the Redline audit XML into tab separated data. This will let you take data and view it in different ways, as well as perform timeline analysis on data you have collected. I will focus on using data collected by a Redline Portable collector, which was configured to perform a Comprehensive Collection. The AuditParser tool will also work with audit data that have been collected with the MIR Agent, or with IOCFinder™.

Timeline analysis can be an important technique when performing an investigation on a system. It involves taking data from several sources, in this case, Redline audit data, and sorting it chronologically. This allows an analyst to see numerous data sources in a chronological fashion, allowing multiple, discrete events to be correlated together. I will be demonstrating timeline analysis, with AuditParser, on a system that is infected with malware.

The first item I want to show is how AuditParser converts audit data to the tab separated files. Once you've got the AuditParser tool, you can run it, and point to the directory of audit data that you have collected with Redline. There is also a Windows executable version of this tool available, if you do not have access to Python.

C:Documents and SettingsUserDesktopAuditParser>python AuditParser.py -i ..LR_Collection -o ..Processed_Audits
Parsing input file: ..LR_Collectionmir.w32apifiles.1176360b.xml
Parsing input file: ..LR_Collectionmir.w32eventlogs.771e480e.xml
Parsing input file: ..LR_Collectionmir.w32registryapi.3e2e3e42.xml
Parsing input file: ..LR_Collectionmir.w32scripting-persistence.27091e25.xml
...

This will output a set of files in ..Processed_Audits, which are the tab separated files, one for each audit in the input.

C:Documents and SettingsUserDesktopAuditParser>dir ..Processed_Audits
Directory of C:Documents and SettingsUserDesktopIndividual_Audits
10/19/2012 08:56 PM
20,385,250 mir.w32apifiles.1176360b.xml.txt
10/19/2012 08:56 PM
592,829 mir.w32eventlogs.771e480e.xml.txt
10/19/2012 09:00 PM
48,625,256 mir.w32registryapi.3e2e3e42.xml.txt
10/19/2012 09:01 PM
301,325 mir.w32scripting-persistence.27091e25.xml.txt
...

Now that you have these files parsed, you can start performing analysis with any software that supports processing tab separated files, such as Microsoft Excel or Apache OpenOffice. This lets you quickly sort and filter data you have collected and quickly look for anomalies. Below is a screenshot of the output of the Persistence audit. The persistence audit combines the File, Registry and Services audit to look for executable files that will persistently run on a system. Each of the columns - denoting information collected from the Registry, Filesystem and Services - can be filtered in order to perform an investigative analysis.

Click to enlarge image

If you sort by files that do not have digital signatures (Column: Signature Exists, select 'False'), one entry is found. This is a serviceDLL entry. This indicates that there is an unsigned service running on this system. The Persistence audit gives you some information you can use to further analysis. It gives the location to the file on the file system, the file owner, a hash of the file, Registry path & value information. The file full path, "c:WINDOWSimewmimachine2.dll" is not a typical location for a service DLL, so this item is definitely something worth investigating.

Click to enlarge image

You could continue reviewing data in individual audits using some of the information presented in the Persistence audit module. In fact, there is some additional information contained in the XML for the persistence item, pertaining to the file 'wmimachine2.dll.' Continuing with the individual audits would be familiar sorting and filtering operations, so that will be skipped for now. Instead, what I'm going to show next is how to timeline the audit data that we have already collected in order to identify other items of interest which have occurred around the creation of our malicious serviceDLL. In order to do this, I will rerun the AuditParser tool with the timeline switches.

C:Documents and SettingsUserDesktopAuditParser>python AuditParser.py -i ..LR_Collection -o ..Processed_Audits
ed -t --starttime 2012-10-18T00:00:00Z --endtime 2012-10-21T00:00:00Z

This will rerun AuditParser, regenerating the .txt files you already have processed and creating a new file, timeline.txt. This is a tab separated file that contains a timeline of File, Registry, Event Log, Url History, Process Item and Prefetch audit data. I picked a date range of 2012-10-18 through 2012-10-20 to timeline.

Here is a screenshot of timelined data:

Click to enlarge image

You can see that we timeline on multiple timestamps for each item. For instance, a file can have up to eight different timestamps (API + NTFS timestamps), and the AuditParser tool will create an entry for each of those timestamps.

From the Persistence audit data, you can see the file created date is "2012-10-20T01:06:55Z". You can start by narrowing down the date range of items displayed by filtering the date with the string '2012-10-20T01'. This will show you all of the activity on 2012-10-20 between 01:00:00Z and 01:59:59Z. Below is a screenshot of some activity around the time of interest.

Click to enlarge image

If you look through the data, you can see that there was a file staged in C:Recycler, named 'scvhost.exe'. This was likely executed with a command shell, cmd.exe. After scvhost.exe was executed, you can see the Eventlog record the service start for the ".NET Runtime Optimization Service v2.086521.BackUp_X86" service. Further down, you can see artifacts in the registry where the '6to4' service has been hijacked and used to host this new, malicious service. Here you can see the how the registry has been timelined, and shows the Paths and Values.

Click to enlarge image

This does not necessarily finish the investigation of this host. Malware analysis may need to be performed on the malware 'scvhost.exe' and 'wmimachine2.dll' in order to identify additional host-based and network-based indicators of compromise. The source of 'scvhost.exe' has not been identified, so that would require further investigation.

Hopefully, this demonstration of using AuditParser to process Redline audit data has been helpful, showing a new way to process and analyze the large volume of data that can be collected with Redline. The timeline capability of AuditParser can be a powerful analysis tool. If they are not already, Redline and AuditParser should become part of your investigative toolkit. You can download each tool here (Redline) and here (AuditParser).

Unibody Memory Analysis — Introducing Memoryze™ for the Mac 1.0

Today, Mandiant is introducing a new free tool, Memoryze™ for the Mac 1.0, which brings memory imaging and analysis to the Mac. It joins a growing list of freeware tools Mandiant currently provides.

Memoryze™ for the Mac 1.0 brings many of the features of Memoryze to the Apple Macintosh platform. This new tool enables acquisition of memory images via the command-line or a simple GUI. In addition, Memoryze™ for the Mac 1.0 can perform offline analysis against memory images or live analysis on a running system.

The tool supports the following features:

  • Imaging the full range of system memory
  • Acquiring individual processes memory regions
  • Enumerating all running processes
    • Including those hidden by rootkits

For each process, Memoryze™ for the Mac 1.0 can:

  • Report all open file handles in a process (e.g. all files,sockets, pipes, etc.)
  • List the virtual address space of a process including loaded libraries and allocated portions of heap and execution stack
  • List network connections
    • Active and listening
  • Enumerate
    • All loaded kernel extensions including those hidden by rootkits
    • The System Call Table and Mach Trap Table
    • All running Mach Tasks

Okay, enough of the marketing. Memoryze™ for the Mac 1.0 can be downloaded here. To help get you started we'll present a few of the features in this blog post.

For offline analysis your first step is going to be acquiring memory. The Mac Memory Dumper App makes this process as simple as pushing a button. To begin the acquisition, Memoryze™ for the Mac 1.0 will require you to authenticate so that the application can load a memory dumping driver. After selecting the location to store the image and authenticating, Memoryze™ for the Mac 1.0 will begin the acquisition process. The tool will provide you with a progress bar and an image size monitor for each of your acquisitions (fancy, huh?).

Note that the final size of the dumped image may exceed the size of your physical RAM. If the system has 8GB of physical RAM installed the dump may be 10GB. You may ask yourself, "Self, why is the dumped image bigger than my actual memory size?" There are regions that are physically addressable but are not part of actual DRAM, pesky memory-mapped devices. These regions are written to the image file as 0x0-bytes to help preserve the correct offsets within the image.

Once Memoryze™ for the Mac 1.0 finishes the acquisition; we can use it to perform memory analysis(note Memoryze™ for the Mac 1.0 can also perform analysis on images acquired by other tools). With our data ready, let's run through several of the process analysis features we mentioned above.

We'll start by performing a basic process listing based on the memory image we just created. Execute the command below:

macmemoryze proclist -f ~/Documents/my.mem 2>err.txt

Memoryze™ for the Mac 1.0 will open "my.mem" for analysis and detect two critical pieces of information about the image: the operating system version and whether the system is running 32 or 64-bit kernel. Armed with this information the proclist analysis module locates the operating system data structure that maintains the list of running processes.

If you take a look at the output below, you can see that Memoryze™ for the Mac 1.0 extracts the PADDR, or physical address, of each process (this is also the offset of the process in the acquired memory image file). You can use the PADDR to quickly locate the process in question in an offline tool such as a hex editor. Memoryze™ for the Mac 1.0 also extracts other standard identifying information such as the process NAME, PID and parent PID (PPID). In addition, the tool provides the start time for each process in UTC. Finally, Memoryze™ for the Mac 1.0 extracts each process' associated USERNAME, effective userid (EUID), and real userid (RUID).

Based on the process listing above, we may be interested in getting a more complete understanding of what a particular process may be doing. We can list file descriptors and memory sections for all of the processes in the listing, but this would get pretty lengthy and present too much information at once. Using filters we can limit display to a smaller subset or a single process. We can use the "-n" option to filter processes by NAME or the "-p" to filter processes by PID.Execute the following command:

macmemoryze proclist -w -p 14 ~/Documents/my.mem 2>err.txt

In the image above, we show a snippet of a file descriptor listing for PID 14 using the command-line switch "-p 14". This shows us all open file descriptors for the given process. This includes files, UNIX domain sockets, networking sockets (such as TCP), and so on. For several of the types/subtypes, Memoryze™ for the Mac 1.0 will provide a value associated with the item type. The value for a descriptor of type FILE is the filename associated with the file descriptor while the value for a type SOCKET subtype TCP is the source IP address, destination IP address, and associated ports.

Now that we've completed some filtering, let's dig a little deeper and perform detailed analysis of the "notifyd" process. The image below contains a snippet of its memory sections listing. Memoryze™ for the Mac 1.0 shows us the start and end virtual addresses and a human-readable size for each section. For some memory sections, Memoryze™ for the Mac 1.0 also provides a type, such as MALLOC or IOKIT. These types provide insight into the purpose of the memory section. For other memory sections, Memoryze™ for the Mac 1.0 displays the filename that is located at (or was used to initialize) that particular memory region. Please execute:

macmemoryze proclist -s -p 14 ~/Documents/my.mem 2>err.txt

Neat, huh? So now we've analyzed the standard operating system (OS) process list structure. If it's good enough for the OS, it's good enough for us, right? Not really. It's fairly trivial for malware to unlink itself from the process list that the OS maintains. In light of this, Memoryze™ for the Mac 1.0 provides a process carving feature that allows it to enumerate and analyze processes based on their signature in memory. This means that Memoryze™ for the Mac 1.0 does not have to depend on the OS to provide a list of processes. This enables Memoryze™ for the Mac 1.0 to discover processes that have been hidden from standard OS listings. This same carving feature extends to the kextlist and syscalllist Memoryze™ for the Mac 1.0 features, allowing you to discover other hidden data within the OS.

Now don't forget, Memoryze™ for the Mac 1.0 can be run offline using an acquired memory image, or live, analyzing the running system in real-time. Below we show a system call table listing using the live memory analysis feature of Memoryze™ for the Mac 1.0. Notice the missing "-f" option. We can use this listing to check for system call table hooking. System call table hooking allows attackers to surreptitiously monitor or filter user-level programs interactions with the OS kernel. This is commonly used to hide files and network connections from user-level programs. The syscalllist feature also supports discovery and listing of the Mach Trap table. Mach Trap, what? The mach trap table is analogous to the system call table in the BSD portion of OS X, but within the Mach portion of OS X. So we want to ensure we can check for possible hooking within that table also. To perform a live listing execute:

sudo macmemoryze syscalllist -s

In order to hook the syscall table we would probably want to use a driver. Have no fear! Memoryze™ for the Mac 1.0 can carve loaded kexts from memory. A decent malware author would probably want to hide itself from the OS by unlinking from internal data structures. To combat this, Memoryze™ for the Mac 1.0 will parse live memory or a dead file to find loaded drivers, as shown in the screenshot below.

Memoryze™ for the Mac 1.0 supports an XML output option ("-x") that will create an XML file with an extended version of the console output, only in XML format. "Extended version," you ask? There is only so much console real estate. We must make decision about what information to display. So therefore, stuff gets left out. For example, we parse the full file path of the executing process and the process arguments. These are not displayed in the console output, but are accessible if using the XML output option. These XML files can also be loaded into Mandiant Redline™ (currently windows only, boo!) for viewing in a GUI.

There are other features of Memoryze™ for the Mac 1.0 that we have not detailed here, but we don't want to give you all the answers. What's the fun in that? We really want you to use the tool and provide us with some feedback on features, interface, usages, and so on.

Memoryze™ for the Mac 1.0 currently supports:

  • Mac OS X Snow Leopard (10.6) 32/64-bit
  • Mac OS X Lion (10.7) 32/64-bit

We hope to continue to improve the state of Mac memory analysis for incident responders and security professionals.

"Mac" is a trademark of The Apple Corporation. Mandiant is not affiliated with or endorsed by The Apple Corporation.

Exploring Symbol Type Information with PdbXtract

Mandiant is introducing a new free tool today, PdbXtract™, which allows you to browse and search PDB-type information.

PdbXtract allows you to explore symbolic type information as extracted from Microsoft PDB files. This tool is primarily designed for reverse engineering Windows-based applications and for exploring the internals of Windows kernel components. You can download PdbXtract.

A programming database (PDB) file is a binary file containing program debug information in a Microsoft-proprietary format. This file is produced by the compiler/linker when a program is built. The information it contains is used by debuggers to debug a program and can greatly assist a developer in debugging program issues by resolving function pointers to symbolic names, for example.

Perhaps the most useful and richest source of debugging information contained in PDBs is type data which holds detailed information about data structures, constants, and other named symbols. While this information is primarily used to debug program components, it can also be used to gain insight into how core operating system components work by observing both the format of the data structure and how the structure is used.

PdbXtract is not a pure PDB parser. It only extracts type information using Microsoft's DebugInterface Access (DIA) COM. If you are interested in just parsing/dumping raw PDB information, there are a few alternatives out there to DIA, including Volatility's open source pdbparse (http://code.google.com/p/pdbparse/) or the PDB utility that comes with the Undocumented Windows 2000 Secrets book. However, most of the practical tools I have seen that operate on PDB's use DIA, including Microsoft's own Dia2dump, this one http://www.codeproject.com/Articles/37456/How-To-Inspect-the-Content-of-a-Program-Database-P and this one http://www.ishani.org/web/articles/obsolete/pdb-cracking-tool/, to name a few. To reiterate, PdbXtract does not parse or capture the wealth of other information available in a PDB, including: functions, debug streams, modules, publics, globals, files, section information, injected sources, source files, OEM specific types, compilands, and others.

The tools mentioned above are fine for inspecting the contents of a single PDB. However, often times as part of my job in R&D, we have to use knowledge of type information across all supported Windows operating systems to implement features. For example, if you are dealing with partially undocumented or "opaque" types (example: you need to walk the PEB's InInitializationOrderModuleList to obtain a list of loaded modules in a process) or have full source type information but do not want to tie your program to a specific version of those types as implemented in the headers of the SDK you are compiling against, you probably want to just use static offsets such as:

PVOID NextMod=(PVOID ((DWORD_PTR)PebPtr+InInitOrderModList_Offset+Flink_Offset);

The problem has always been: how do I get the value of InInitOrderModList_Offset for all platforms we support, taking into account 32-bit/64-bit variations? The answer has always been: useWinDbg (or if you are interested in possibly-correct kernel symbols only, you might use Matt Suiche's Moonsols library (http://msdn.moonsols.com )). Launch a VM for each OS you want to support, attach with a debugger, and use the power of WinDbg to extract the type information. Well, WinDbg's magical "dt" command just relies on the PDB information for the corresponding binary (after retrieving the necessary symbol files from your local symbol store and optionally the Microsoft public symbol server), so it stands to reason that we should be able to do the same. The end goal is to make a searchable database for all the exported types of OS binaries we care about, so that we don't have to constantly relive the tedium of doing this in WinDbg.

Features

PdbXtract has two main features: exploring a single PDB (PDB Explorer) and searching a library of PDBs for one or more operating systems. PDB Explorer opens the PDB, parses type information using DIA, and displays a list of all structs, unions and enums. If you click on one of the types, a C-style struct (with offsets) definition will be displayed in the text area to the right, as shown below for the type IMAGE_FILE_HEADER.

The library tab allows you to create and search a library of PDB type information. I have created a library for *most* of the operating systems we support for the following important system binaries: kernel (ntoskrnl.exe, ntkrpamp, ntkrnlpa, ntkrnlmp),ndis.sys, win32k.sys and hal (hal.dll, halaacpi.dll, halmacpi.dll). You might ask why other system DLLs were not included, such askernel32, user32, advapi32, etc.The answer to that being the corresponding PDBs for those binaries you get off the symbol server are stripped of type information. Why? Because Microsoft expects you to use their headers when you compile your application, and thus your program'sPDB will have the necessary type information.

The library included with PdbXtract covers several of Microsoft's major operating system releases, but you can easily add more symbols. PdbXtract includes a utility, called PdbFetch, that simply runs Microsoft's symchk utility to grab the symbols for the file names you supply (usage: pdbfetch, where is a text file that contains a list of full paths to system binaries you want to retrieve symbols for). Pdbfetch creates a "PDB set" which consists ofthe directory structure with containing PDBs as created by symchkplus a manifest.xml file which summarizes the OSplatform information. To use a PDB set in PdbXtract, go to the library tab and click "new" if you want to create a new library from the PDB set or "add" if you want to add them to an existing library. Once you create/add a PDB set to a library you can delete them - the only thing that matters is the sqlite .pdbx database that's created.

Perhaps someone out there will find this useful and maybe create a searchable web front end with the resulting SQLite database? The sky is the limit. Let me know if you do by commenting below.

As a final note, you might wonder why you can't just download the entire symbol packages from Microsoft, which include every symbol file on the MS Symbol server, and create a ginormous library. Why is there a requirement to acquire the PDBs using pdbfetch? The answer is - you could do that - but it is data overload(several GB of PDBs) when you will not care about 99% of them. Plus it is easier to capture OS platform/build info at run time rather than guessing at it from the name of the symbol package installer (PDBs give no indication of what OS the corresponding binary originated from).

The Latest Version of Redline Finds Indicators of Compromise and More

We are on a roll with our freeware. The latest version of Redline is now available! For those who are not familiar with Redline - you may be asking, what is it? Simply put, Redline brings together analysis tools which help you perform a guided investigation of a potentially compromised system. And did we mention that it is free?

This latest and greatest version of Redline includes some awesome new features, courtesy of recommendations from our strong and growing user base and input from internal users here at Mandiant. For those who have been loyal Redline users, you will find that it is no longer just a memory forensics tool! It has grown into a multi-purpose product for creating Indicators of Compromise (IOC) and matching them across all types of host data, while maintaining all the traditional memory forensics capabilities that you're used to.

Get the data that matters, and do it faster

  • With Redline, you can now include and search for Indicators of Compromise and create a searchable report detailing any suspicious activity found matching those IOCs. Need more on what IOCs are? Click here for more information.
  • Specify a set of IOCsbefore collection and Redline will now help tailor the configuration to provide meaningful search resultsand ensure that all the data required by the chosen IOCs is collected, speeding up your time to completion.
  • Not sure if the IOCs you have chosen are the ones you want? Not to worry! When choosing indicators to search for, there is now a handy preview window to see the detailed information of each indicator.
  • You are no longer limited to just memory data. Redline now enables you to configure and collect a much broader range of data about the target host, such as event logs and file listings. This data will in turn be searchable using the new Indicator of Compromise search options, providing you with better overall search results.

Multi-task with the best

  • With Redline you can now perform investigations while searching for indicators - at the same time! For example, while the session is still matching IOCs, you can start diving into the Malware Risk Indicator (MRI) Scores and start anew investigation or even continue an existing investigation.
  • Now there's no guessing where you are in the process. You can check the progress of your investigation at any time via "Background Tasks" in the main menu. You will also receive a notification when one of your background tasks has been scheduled.

For our current users, be sure to upgrade to this latest version of Redline to take advantage of the new features. For new users, don't wait another minute to download Redline and get your hands on this great set of analysis tools.

Research Tool Release: ApateDNS

Here at Mandiant we deal with our fair share of malicious code. Being able to quickly identify specific information about a piece of malware is imperative. More specifically, knowing which domains a piece of malware uses for command and control (C2) communication is important to on-site incident responders.

To aid analysts in DNS identification, I have written ApateDNS. It is a simple tool that acts as a phony DNS server that can log or manipulate DNS requests being made to it. Malware analysts typically use this to redirect beacon traffic from a guest virtual machine to the host system (or another virtual machine) to monitor beacon and/or communication channels using Netcat or a custom written C2 script. Forensic analysts typically use this tool to quickly extract DNS names from malware samples.

ApateDNS automatically sets up your Windows network configurations by attempting to determine the default route or current DNS settings. This is most useful when in a guest virtual machine since the default route is typically the host machine. As shown in the figure below, ApateDNS has found the default route in my virtual machine (192.168.239.1) and uses this IP address for any DNS request on my virtual host. The user may override this by specifying an IP address for DNS Reply IP.

Malware often uses multiple C2 domains. To catch this, ApateDNS allows a user to specify a number of non-existent DNS (NXDOMAIN) replies for any possible DNS lookup. As seen in the figure below, the malware returns a single, non-existent domain for each DNS request (since a "1" is entered for "# of NXDOMAIN's"). The example malware beacons and detects if a valid IP address has been resolved from a DNS request, if not, it will continue to walk down its C2 domain list. By using the NXDOMAIN functionality, we see three different DNS requests made by the malware: evil1.example.com, evil2.example.com and evil3.example.com.

ApateDNS gives malware analysts an easy way to control DNS on their machine and forensic analysts a way to monitor DNS requests made by malware. Of course, not all malware utilizes DNS and some may not beacon without a specific set of conditions being satisfied. ApateDNS's use cases are not limited just to malware. It can be used for any purpose where a user may want to monitor outbound DNS requests or traffic.

Feel free to check out ApateDNS here.

Highlighter v1.1.2 Released

Hey, guess what?! MANDIANT has just released Highlighter v1.1.2 in response to your feedback - a fix for one particularly nagging issue with highlights and removals not updating the view immediately, and a few extra items thrown in to make Highlighter a little nicer to use.

Wipe the cheesy poofs off your fingers and go here to the download page to check out the updates.

We have listened to your suggestions on how to improve this tool and have worked hard to make it a prime source for rapid review of logs and other structured text files.

New Feature:
  • Ability to change the display font. (Look in the menu under File -> Font.)

Improvements:

  • Selecting text in the display will now more accurately line up with the mouse pointer.
  • The display will now remain at the same point in the file after removing or restoring lines.

Fixes:

  • Display refresh issues in Windows 7.
  • In some cases, state files did not properly store and restore state.