Author Archives: BrianKrebs

International Action Targets Emotet Crimeware

Authorities across Europe on Tuesday said they’d seized control over Emotet, a prolific malware strain and cybercrime-as-service operation. Investigators say the action could help quarantine more than a million Microsoft Windows systems currently compromised with malware tied to Emotet infections.

First surfacing in 2014, Emotet began as a banking trojan, but over the years it has evolved into one of the more aggressive platforms for spreading malware that lays the groundwork for ransomware attacks.

In a statement published Wednesday morning on an action dubbed “Operation Ladybird,” the European police agency Europol said the investigation involved authorities in the Netherlands, Germany, United States, the United Kingdom, France, Lithuania, Canada and Ukraine.

“The EMOTET infrastructure essentially acted as a primary door opener for computer systems on a global scale,” Europol said. “Once this unauthorised access was established, these were sold to other top-level criminal groups to deploy further illicit activities such data theft and extortion through ransomware.”

Experts say Emotet is a pay-per-install botnet that is used by several distinct cybercrime groups to deploy secondary malware — most notably the ransomware strain Ryuk and Trickbot, a powerful banking trojan. It propagates mainly via malicious links and attachments sent through compromised email accounts, blasting out tens of thousands of malware-laced missives daily.

Emotet relies on several hierarchical tiers of control servers that communicate with infected systems. Those controllers coordinate the dissemination of second-stage malware and the theft of passwords and other data, and their distributed nature is designed to make the crimeware infrastructure more difficult to dismantle or commandeer.

In a separate statement on the malware takeover, the Dutch National police said two of the three primary servers were located in the Netherlands.

“A software update is placed on the Dutch central servers for all infected computer systems,” the Dutch authorities wrote. “All infected computer systems will automatically retrieve the update there, after which the Emotet infection will be quarantined. Simultaneous action in all the countries concerned was necessary to be able to effectively dismantle the network and thwart any reconstruction.”

A statement from the German Federal Criminal Police Office about their participation in Operation Ladybird said prosecutors seized 17 servers in Germany that acted as Emotet controllers.

“As part of this investigation, various servers were initially identified in Germany with which the malicious software is distributed and the victim systems are monitored and controlled using encrypted communication,” the German police said.

Sources close to the investigation told KrebsOnSecurity the law enforcement action included the arrest of several suspects in Europe thought to be connected to the crimeware gang, although that could not be confirmed. The core group of criminals behind Emotet are widely considered to be operating out of Russia.

A video released to YouTube by the National Police of Ukraine this morning shows authorities there raiding a residence, seizing cash and computer equipment, and what appear to be numerous large bars made of gold.

Police in the Netherlands seized huge volumes of data stolen by Emotet infections, including email addresses, usernames and passwords. A tool on the Dutch police website lets users learn if their email address has been compromised by Emotet.

But because Emotet is typically used to install additional malware that gets its hooks deeply into infected systems, cleaning up after it is going to be far more complicated and may require a complete rebuild of compromised computers.

The U.S. Cybersecurity & Infrastructure Security Agency has labeled Emotet “one of the most prevalent ongoing threats” that is difficult to combat because of its ‘worm-like’ features that enable network-wide infections.” Hence, a single Emotet infection can often lead to multiple systems on the same network getting compromised.

It is too soon to say how effective this operation has been in fully wresting control over Emotet, but a takedown of this size is a significant action.

In October, Microsoft used trademark law to disrupt the Trickbot botnet. Around the same time, the U.S. Cyber Command also took aim at Trickbot. However, neither of those actions completely dismantled the crimeware network, which remains in operation today.

Roman Hüssy, a Swiss information technology expert who maintains Feodotracker — a site that lists the location of major botnet controllers — told KrebsOnSecurity that prior to January 25, some 98 Emotet control servers were active. The site now lists 20 Emotet controllers online, although it is unclear if any of those remaining servers have been commandeered as part of the quarantine effort.

A current list of Emotet control servers online. Source: Feodotracker.abuse.ch

DDoS-Guard To Forfeit Internet Space Occupied by Parler

Parler, the beleaguered social network advertised as a “free speech” alternative to Facebook and Twitter, has had a tough month. Apple and Google removed the Parler app from its stores, and Amazon blocked the platform from using its hosting services. Parler has since found a home in DDoS-Guard, a Russian digital infrastructure company. But now it appears DDoS-Guard is about to be relieved of more than two-thirds of the Internet address space the company leases to clients — including the Internet addresses currently occupied by Parler.

The pending disruption for DDoS-Guard and Parler comes compliments of Ron Guilmette, a researcher who has made it something of a personal mission to de-platform conspiracy theorist and far-right groups.

In October, a phone call from Guilmette to an Internet provider in Oregon was all it took to briefly sideline a vast network of sites tied to 8chan/8kun — a controversial online image board linked to several mass shootings — and QAnon, the far-right conspiracy theory which holds that a cabal of Satanic pedophiles is running a global child sex-trafficking ring and plotting against President Donald Trump. As a result, those QAnon and 8chan sites also ultimately ended up in the arms of DDoS-Guard.

Much like Internet infrastructure firm CloudFlare, DDoS-Guard typically doesn’t host sites directly but instead acts as a go-between to simultaneously keep the real Internet addresses of its clients confidential and to protect them from crippling Distributed Denial-of-Service (DDoS) attacks.

The majority of DDoS-Guard’s employees are based in Russia, but the company is actually incorporated in two other places: As “Cognitive Cloud LLP” in Scotland, and as DDoS-Guard Corp. based in Belize.  However, none of the company’s employees are listed as based in Belize, and DDoS-Guard makes no mention of the Latin American region in its map of global operations.

In studying the more than 11,000 Internet addresses assigned to those two companies, Guilmette found that approximately 66 percent of them were doled out to the Belize entity by LACNIC, the regional Internet registry for the Latin American and Caribbean regions.

Suspecting that DDoS-Guard incorporated in Belize on paper just to get a huge swaths of IP addresses that are supposed to be given only to entities with a physical presence in the region, Guilmette filed a complaint with the Internet registry about his suspicions back in November.

Guilmette said LACNIC told him it would investigate, and that any adjudication on the matter could take up to three months. But earlier this week, LACNIC published a notice on its website that it intends to revoke 8,192 IPv4 addresses from DDoS-Guard — including the Internet address currently assigned to Parler[.]com.

A notice of revocation posted by LACNIC.

LACNIC has not yet responded to requests for comment. The notice on its site says the Internet addresses are set to be revoked on Feb. 24.

DDoS-Guard CEO Evgeniy Marchenko maintains the company has done nothing wrong, and that DDoS-Guard does indeed have a presence in Belize.

“They were used strongly according [to] all LACNIC policies by [a] company legally substituted in LACNIC region,” Marchenko said in an email to KrebsOnSecurity. “There is nothing illegal or extremist. We have employers and representatives in different countries around the world because we are global service. And Latin America region is not an exception.”

Guilmette said DDoS-Guard could respond by simply moving Parler and other sites sitting in those address ranges to another part of its network. But he considers it a victory nonetheless that a regional Internet registry took his concerns seriously.

“It appeared to me that it was more probable than not that they got these 8,000+ IPv4 addresses by simply creating an arguably fraudulent shell company in Belize and then going cap in hand to LACNIC, claiming that they had a real presence in the Latin & South American region, and then asking for 8,000+ IPv4 addresses,” he said. “So I reported my suspicions to the LACNIC authorities in early November, and as I have only just recently learned, the LACNIC authorities followed up diligently on my report and, it seems, verified my suspicions.”

In October, KrebsOnSecurity covered another revelation by Guilmette about the same group of QAnon and 8chan-related sites that moved to DDoS-Guard: The companies that provided the Internet address space used by the sites were defunct businesses in the eyes of their respective U.S. state regulators. In other words, the American Registry for Internet Numbers (ARIN) — the non-profit which administers IP addresses for entities based in North America — was well within its contract rights to revoke the IP space.

Guilmette brought his findings to ARIN, which declined to act on the complaint and instead referred the matter to state investigatory agencies.

Still, Guilmette’s gadfly efforts to stir things up in the RIR community sometimes do pay off. For example, he spent nearly three years documenting how $50 million worth of the increasingly scarce IPv4 addresses were misappropriated from African companies to dodgy Internet marketing firms.

His complaints about those findings to the African Network Information Centre (AFRINIC) resulted in an investigation that led to the termination of a top AFRINIC executive, who was found to have quietly sold many of the address blocks for personal gain to marketers based in Europe, Asia and elsewhere.

And this week, AFRINIC took the unusual step of officially documenting the extent of the damage wrought by its former employee, and revoking discrete chunks of address space currently being used by marketing firms.

In a detailed report released today (PDF), AFRNIC said its investigation revealed more than 2.3 million IPv4 addresses were “without any lawful authority, misappropriated from AFRINIC’s pool of resources and attributed to organizations without any justification.”

AFRINIC said it began its inquiry in earnest back in March 2019, when it received an application by the U.S. Federal Bureau of Investigation (FBI) about “certain suspicious activities regarding several IPv4 address blocks which it held.” So far, AFRNINIC said it has reclaimed roughly half of the wayward IP address blocks, with the remainder “yet to be reclaimed due to ongoing due diligence.”

New Charges Derail COVID Release for Hacker Who Aided ISIS

A hacker serving a 20-year sentence for stealing personal data on 1,300 U.S. military and government employees and giving it to an Islamic State hacker group in 2015 has been charged once again with fraud and identity theft. The new charges have derailed plans to deport him under compassionate release because of the COVID-19 pandemic.

Ardit Ferizi, a 25-year-old citizen of Kosovo, was slated to be sent home earlier this month after a federal judge signed an order commuting his sentence to time served. The release was granted in part due to Ferizi’s 2018 diagnosis if asthma, as well as a COVID outbreak at the facility where he was housed in 2020.

But while Ferizi was in quarantine awaiting deportation the Justice Department unsealed new charges against him, saying he’d conspired from prison with associates on the outside to access stolen data and launder the bitcoin proceeds of his previous crimes.

In the years leading up to his arrest, Ferizi was the administrator of a cybercrime forum called Pentagon Crew. He also served as the leader of an ethnic Albanian group of hackers from Kosovo known as Kosova Hacker’s Security (KHS), which focused on compromising government and private websites in Israel, Serbia, Greece, Ukraine and the United States.

The Pentagon Crew forum founded by Ferizi.

In December 2015, Ferizi was apprehended in Malaysia and extradited to the United States. In January 2016, Ferizi pleaded guilty to providing material support to a terrorist group and to unauthorized access. He admitted to hacking a U.S.-based e-commerce company, stealing personal and financial data on 1,300 government employees, and providing the data to an Islamic State hacking group.

Ferizi give the purloined data to Junaid “Trick” Hussain, a 21-year-old hacker and recruiter for ISIS who published it in August 2015 as part of a directive that ISIS supporters kill the named U.S. military members and government employees. Later that month, Hussain was reportedly killed by a drone strike in Syria.

The government says Ferizi and his associates made money by hacking PayPal and other financial accounts, and through pornography sites he allegedly set up mainly to steal personal and financial data from visitors.

Junaid Hussain’s Twitter profile photo.

Between 2015 and 2019, Ferizi was imprisoned at a facility in Illinois that housed several other notable convicts. For example, prosecutors allege that Ferizi was an associate of Mahmud “Red” Abouhalima, who was serving a 240 year sentence at the prison for his role in the 1993 World Trade Center bombing.

Another inmate incarcerated at the same facility was Shawn Bridges, a former U.S. Secret Service agent serving almost eight years for stealing $820,000 worth of bitcoin from online drug dealers while investigating the hidden underground website Silk Road. Prosecutors say Ferizi and Bridges discussed ways to hide their bitcoin.

The information about Ferizi’s inmate friends came via a tip from another convict, who told the FBI that Ferizi was allegedly using his access to the prison’s email system to share email and bitcoin account passwords with family members back home.

The Justice Department said subpoenas served on Ferizi’s email accounts and interviews with his associates show Ferizi’s brother in Kosovo used the information to “liquidate the proceeds of Ferizi’s previous criminal hacking activities.”

[Side note: It may be little more than a coincidence, but my PayPal account was hacked in Dec. 2015 by criminals who social engineered PayPal employees over the phone into changing my password and bypassing multi-factor authentication. The hackers attempted to send my balance to an account tied to Hussain, but the transfer never went through.]

Ferizi is being tried in California, but has not yet had an initial appearance in court. He’s charged with one count of aggravated identity theft and one count of wire fraud. If convicted of wire fraud, he faces a maximum penalty of 20 years in prison and a fine of $250,000. If convicted of aggravated identity theft, he faces a mandatory penalty of 2 years in prison in addition to the punishment imposed for a wire fraud conviction.

Joker’s Stash Carding Market to Call it Quits

Joker’s Stash, by some accounts the largest underground shop for selling stolen credit card and identity data, says it’s closing up shop effective mid-February 2021. The announcement came on the heels of a turbulent year for the major cybercrime store, and just weeks after U.S. and European authorities seized a number of its servers.

A farewell message posted by Joker’s Stash admin on Jan. 15, 2021.

The Russian and English language carding store first opened in October 2014, and quickly became a major source of “dumps” — information stolen from compromised payment cards that thieves can buy and use to create physical counterfeit copies of the cards.

But 2020 turned out to be a tough year for Joker’s Stash. As cyber intelligence firm Intel 471 notes, the curator of the store announced in October that he’d contracted COVID-19, spending a week in the hospital. Around that time, Intel 471 says many of Joker’s loyal customers started complaining that the shop’s payment card data quality was increasingly poor.

“The condition impacted the site’s forums, inventory replenishments and other operations,” Intel 471 said.

Image: Gemini Advisory

That COVID diagnosis may have affected the shop owner’s ability to maintain fresh and valid inventory on his site. Gemini Advisory, a New York City-based company that monitors underground carding shops, tracked a “severe decline” in the volume of compromised payment card accounts for sale on Joker’s Stash over the past six months.

“Joker’s Stash has received numerous user complaints alleging that card data validity is low, which even prompted the administrator to upload proof of validity through a card-testing service,” Gemini wrote in a blog post about the planned shutdown.

Image: Gemini Advisory

Then on Dec. 16, 2020, several of Joker’s long-held domains began displaying notices that the sites had been seized by the U.S. Department of Justice and Interpol. The crime shop quickly recovered, moving to new infrastructure and assuring the underground community that it would continue to operate normally.

Gemini estimates that Joker’s Stash generated more than a billion dollars in revenue over the past several years. Much of that revenue came from high-profile breaches, including tens of millions of payment card records stolen from major merchants including Saks Fifth Avenue, Lord and TaylorBebe StoresHilton HotelsJason’s DeliWhole FoodsChipotle, Wawa, Sonic Drive-In, the Hy-Vee supermarket chain, Buca Di Beppo, and Dickey’s BBQ.

Joker’s Stash routinely teased big breaches days or weeks in advance of selling payment card records stolen from those companies, and periodically linked to this site and other media outlets as proof of his shop’s prowess and authenticity.

Like many other top cybercrime bazaars, Joker’s Stash was a frequent target of phishers looking to rip off unwary or unsophisticated thieves. In 2018, KrebsOnSecurity detailed a vast network of fake Joker’s Stash sites set up to steal login credentials and bitcoin. The phony sites all traced back to the owners of a Pakistani web site design firm. Many of those fake sites are still active (e.g. jokersstash[.]su).

As noted here in 2016, Joker’s Stash attracted an impressive number of customers who kept five and six-digit balances at the shop, and who were granted early access to new breaches as well as steep discounts for bulk buys. Those “partner” customers will be given the opportunity to cash out their accounts. But the majority of Stash customers do not enjoy this status, and will have to spend their balances by Feb. 15 or forfeit those funds.

The dashboard for a Joker’s Stash customer who’s spent over $10,000 buying stolen credit cards from the site.

Gemini said another event that may have contributed to this threat actor shutting down their marketplace is the recent spike in the value of Bitcoin. A year ago, one bitcoin was worth about $9,000. Today a single bitcoin is valued at more than $35,000.

“JokerStash was an early advocate of Bitcoin and claims to keep all proceeds in this cryptocurrency,” Gemini observed in a blog post. “This actor was already likely to be among the wealthiest cybercriminals, and the spike may have multiplied their fortune, earning them enough money to retire. However, the true reason behind this shutdown remains unclear.”

If the bitcoin price theory holds, that would be fairly rich considering the parting lines in the closure notice posted to Joker’s Stash.

“We are also want to wish all young and mature ones cyber-gangsters not to lose themselves in the pursuit of easy money,” the site administrator(s) advised. “Remember, that even all the money in the world will never make you happy and that all the most truly valuable things in this life are free.”

Regardless, the impending shutdown is unlikely to have much of an impact on the overall underground carding industry, Gemini notes.

“Given Joker’s Stash’s high profile, it relied on a robust network of criminal vendors who offered their stolen records on this marketplace, among others,” the company wrote. “Gemini assesses with a high level of confidence that these vendors are very likely to fully transition to other large, top-tier dark web marketplaces.”

Microsoft Patch Tuesday, January 2021 Edition

Microsoft today released updates to plug more than 80 security holes in its Windows operating systems and other software, including one that is actively being exploited and another which was disclosed prior to today. Ten of the flaws earned Microsoft’s most-dire “critical” rating, meaning they could be exploited by malware or miscreants to seize remote control over unpatched systems with little or no interaction from Windows users.

Most concerning of this month’s batch is probably a critical bug (CVE-2021-1647) in Microsoft’s default anti-malware suite — Windows Defender — that is seeing active exploitation. Microsoft recently stopped providing a great deal of detail in their vulnerability advisories, so it’s not entirely clear how this is being exploited.

But Kevin Breen, director of research at Immersive Labs, says depending on the vector the flaw could be trivial to exploit.

“It could be as simple as sending a file,” he said. “The user doesn’t need to interact with anything, as Defender will access it as soon as it is placed on the system.”

Fortunately, this bug is probably already patched by Microsoft on end-user systems, as the company continuously updates Defender outside of the normal monthly patch cycle.

Breen called attention to another critical vulnerability this month — CVE-2020-1660 — which is a remote code execution flaw in nearly every version of Windows that earned a CVSS score of 8.8 (10 is the most dangerous).

“They classify this vulnerability as ‘low’ in complexity, meaning an attack could be easy to reproduce,” Breen said. “However, they also note that it’s ‘less likely’ to be exploited, which seems counterintuitive. Without full context of this vulnerability, we have to rely on Microsoft to make the decision for us.”

CVE-2020-1660 is actually just one of five bugs in a core Microsoft service called Remote Procedure Call (RPC), which is responsible for a lot of heavy lifting in Windows. Some of the more memorable computer worms of the last decade spread automatically by exploiting RPC vulnerabilities.

Allan Liska, senior security architect at Recorded Future, said while it is concerning that so many vulnerabilities around the same component were released simultaneously, two previous vulnerabilities in RPC — CVE-2019-1409 and CVE-2018-8514 — were not widely exploited.

The remaining 70 or so flaws patched this month earned Microsoft’s less-dire “important” ratings, which is not to say they’re much less of a security concern. Case in point: CVE-2021-1709, which is an “elevation of privilege” flaw in Windows 8 through 10 and Windows Server 2008 through 2019.

“Unfortunately, this type of vulnerability is often quickly exploited by attackers,” Liska said. “For example, CVE-2019-1458 was announced on December 10th of 2019, and by December 19th an attacker was seen selling an exploit for the vulnerability on underground markets. So, while CVE-2021-1709 is only rated as [an information exposure flaw] by Microsoft it should be prioritized for patching.”

Trend Micro’s ZDI Initiative pointed out another flaw marked “important” — CVE-2021-1648, an elevation of privilege bug in Windows 8, 10 and some Windows Server 2012 and 2019 that was publicly disclosed by ZDI prior to today.

“It was also discovered by Google likely because this patch corrects a bug introduced by a previous patch,” ZDI’s Dustin Childs said. “The previous CVE was being exploited in the wild, so it’s within reason to think this CVE will be actively exploited as well.”

Separately, Adobe released security updates to tackle at least eight vulnerabilities across a range of products, including Adobe Photoshop and Illustrator. There are no Flash Player updates because Adobe retired the browser plugin in December (hallelujah!), and Microsoft’s update cycle from last month removed the program from Microsoft’s browsers.

Windows 10 users should be aware that the operating system will download updates and install them all at once on its own schedule, closing out active programs and rebooting the system. If you wish to ensure Windows has been set to pause updating so you have ample opportunity to back up your files and/or system, see this guide.

Please back up your system before applying any of these updates. Windows 10 even has some built-in tools to help you do that, either on a per-file/folder basis or by making a complete and bootable copy of your hard drive all at once. You never know when a patch roll-up will bork your system or possibly damage important files. For those seeking more flexible and full-featured backup options (including incremental backups), Acronis and Macrium are two that I’ve used previously and are worth a look.

That said, there don’t appear to be any major issues cropping up yet with this month’s update batch. But before you apply updates consider paying a visit to AskWoody.com, which usually has the skinny on any reports about problematic patches.

As always, if you experience glitches or issues installing any of these patches this month, please consider leaving a comment about it below; there’s a better-than-even chance other readers have experienced the same and may chime in here with some helpful tips.

SolarWinds: What Hit Us Could Hit Others

New research into the malware that set the stage for the megabreach at IT vendor SolarWinds shows the perpetrators spent months inside the company’s software development labs honing their attack before inserting malicious code into updates that SolarWinds then shipped to thousands of customers. More worrisome, the research suggests the insidious methods used by the intruders to subvert the company’s software development pipeline could be repurposed against many other major software providers.

In a blog post published Jan. 11, SolarWinds said the attackers first compromised its development environment on Sept. 4, 2019. Soon after, the attackers began testing code designed to surreptitiously inject backdoors into Orion, a suite of tools used by many Fortune 500 firms and a broad swath of the federal government to manage their internal networks.

Image: SolarWinds.

According to SolarWinds and a technical analysis from CrowdStrike, the intruders were trying to work out whether their “Sunspot” malware — designed specifically for use in undermining SolarWinds’ software development process — could successfully insert their malicious “Sunburst” backdoor into Orion products without tripping any alarms or alerting Orion developers.

In October 2019, SolarWinds pushed an update to their Orion customers that contained the modified test code. By February 2020, the intruders had used Sunspot to inject the Sunburst backdoor into the Orion source code, which was then digitally signed by the company and propagated to customers via SolarWinds’ software update process.

Crowdstrike said Sunspot was written to be able to detect when it was installed on a SolarWinds developer system, and to lie in wait until specific Orion source code files were accessed by developers. This allowed the intruders to “replace source code files during the build process, before compilation,” Crowdstrike wrote.

The attackers also included safeguards to prevent the backdoor code lines from appearing in Orion software build logs, and checks to ensure that such tampering wouldn’t cause build errors.

“The design of SUNSPOT suggests [the malware] developers invested a lot of effort to ensure the code was properly inserted and remained undetected, and prioritized operational security to avoid revealing their presence in the build environment to SolarWinds developers,” CrowdStrike wrote.

A third malware strain — dubbed “Teardrop” by FireEye, the company that first disclosed the SolarWinds attack in December — was installed via the backdoored Orion updates on networks that the SolarWinds attackers wanted to plunder more deeply.

So far, the Teardrop malware has been found on several government networks, including the Commerce, Energy and Treasury departments, the Department of Justice and the Administrative Office of the U.S. Courts.

SolarWinds emphasized that while the Sunspot code was specifically designed to compromise the integrity of its software development process, that same process is likely common across the software industry.

“Our concern is that right now similar processes may exist in software development environments at other companies throughout the world,” said SolarWinds CEO Sudhakar Ramakrishna. “The severity and complexity of this attack has taught us that more effectively combatting similar attacks in the future will require an industry-wide approach as well as public-private partnerships that leverage the skills, insight, knowledge, and resources of all constituents.”

Ubiquiti: Change Your Password, Enable 2FA

Ubiquiti, a major vendor of cloud-enabled Internet of Things (IoT) devices such as routers, network video recorders, security cameras and access control systems, is urging customers to change their passwords and enable multi-factor authentication. The company says an incident at a third-party cloud provider may have exposed customer account information and credentials used to remotely manage Ubiquiti gear.

In an email sent to customers today, Ubiquiti Inc. [NYSE: UI] said it recently became aware of “unauthorized access to certain of our information technology systems hosted by a third party cloud provider,” although it declined to name that provider.

The statement continues:

“We are not currently aware of evidence of access to any databases that host user data, but we cannot be certain that user data has not been exposed. This data may include your name, email address, and the one-way encrypted password to your account (in technical terms, the passwords are hashed and salted). The data may also include your address and phone number if you have provided that to us.”

Ubiquiti has not yet responded to requests for more information, but the notice was confirmed as official in a post on the company’s user support forum.

The warning from Ubiquiti carries particular significance because the company has made it fairly difficult for customers using the latest Ubiquiti firmware to interact with their devices without first authenticating through the company’s cloud-based systems.

This has become a sticking point for many Ubiquiti customers, as evidenced by numerous threads on the topic in the company’s user support forums over the past few months.

“While I and others do appreciate the convenience and option of using hosted accounts, this incident clearly highlights the problem with relying on your infrastructure for authenticating access to our devices,” wrote one Ubiquiti customer today whose sentiment was immediately echoed by other users. “A lot us cannot take your process for granted and need to keep our devices offline during setup and make direct connections by IP/Hostname using our Mobile Apps.”

To manage your security settings on a Ubiquiti device, visit https://account.ui.com and log in. Click on ‘Security’ from the left-hand menu.

1. Change your password
2. Set a session timeout value
3. Enable 2FA

Image: twitter.com/crosstalksol/

According to Ubiquiti’s investment literature, the company has shipped more than 85 million devices that play a key role in networking infrastructure in over 200 countries and territories worldwide.

This is a developing story that may be updated throughout the day.

Sealed U.S. Court Records Exposed in SolarWinds Breach

The ongoing breach affecting thousands of organizations that relied on backdoored products by network software firm SolarWinds may have jeopardized the privacy of countless sealed court documents on file with the U.S. federal court system, according to a memo released Wednesday by the Administrative Office (AO) of the U.S. Courts.

The judicial branch agency said it will be deploying more stringent controls for receiving and storing sensitive documents filed with the federal courts, following a discovery that its own systems were compromised as part of the SolarWinds supply chain attack. That intrusion involved malicious code being surreptitiously inserted into updates shipped by SolarWinds for some 18,000 users of its Orion network management software as far back as March 2020.

“The AO is working with the Department of Homeland Security on a security audit relating to vulnerabilities in the Judiciary’s Case Management/Electronic Case Files system (CM/ECF) that greatly risk compromising highly sensitive non-public documents stored on CM/ECF, particularly sealed filings,” the agency said in a statement published Jan. 6.

“An apparent compromise of the confidentiality of the CM/ECF system due to these discovered vulnerabilities currently is under investigation,” the statement continues. “Due to the nature of the attacks, the review of this matter and its impact is ongoing.”

The AO declined to comment on specific questions about their breach disclosure. But a source close to the investigation told KrebsOnSecurity that the federal court document system was “hit hard,” by the SolarWinds attackers, which multiple U.S. intelligence and law enforcement agencies have attributed as “likely Russian in origin.”

The source said the intruders behind the SolarWinds compromise seeded the AO’s network with a second stage “Teardrop” malware that went beyond the “Sunburst” malicious software update that was opportunistically pushed out to all 18,000 customers using the compromised Orion software. This suggests the attackers were targeting the agency for deeper access to its networks and communications.

The AO’s court document system powers a publicly searchable database called PACER, and the vast majority of the files in PACER are not restricted and are available to anyone willing to pay for the records.

But experts say many other documents stored in the AO’s system are sealed — either temporarily or indefinitely by the courts or parties to a legal matter — and may contain highly sensitive information, including intellectual property and trade secrets, or even the identities of confidential informants.

Nicholas Weaver, a lecturer at the computer science department at University of California, Berkeley, said the court document system doesn’t hold documents that are classified for national security reasons. But he said the system is full of sensitive sealed filings — such as subpoenas for email records and so-called “trap and trace” requests that law enforcement officials use to determine with whom a suspect is communicating via phone, when and for how long.

“This would be a treasure trove for the Russians knowing about a lot of ongoing criminal investigations,” Weaver said. “If the FBI has indicted someone but hasn’t arrested them yet, that’s all under seal. A lot of the investigative tools that get protected under seal are filed very early on in the process, often with gag orders that prevent [the subpoenaed party] from disclosing the request.”

The acknowledgement from the AO comes hours after the U.S. Justice Department said it also was a victim of the SolarWinds intruders, who took control over the department’s Office 365 system and accessed email sent or received from about three percent of DOJ accounts (the department has more than 100,000 employees).

The SolarWinds hack also reportedly jeopardized email systems used by top Treasury Department officials, and granted the attackers access to networks inside the Energy, Commerce and Homeland Security departments.

The New York Times on Wednesday reported that investigators are examining whether a breach at another software provider — JetBrains — may have precipitated the attack on SolarWinds. The company, which was founded by three Russian engineers in the Czech Republic, makes a tool called TeamCity that helps developers test and manage software code. TeamCity is used by developers at 300,000 organizations, including SolarWinds and 79 of the Fortune 100 companies.

“Officials are investigating whether the company, founded by three Russian engineers in the Czech Republic with research labs in Russia, was breached and used as a pathway for hackers to insert back doors into the software of an untold number of technology companies,” The Times said. “Security experts warn that the monthslong intrusion could be the biggest breach of United States networks in history.”

Under the AO’s new procedures, highly sensitive court documents filed with federal courts will be accepted for filing in paper form or via a secure electronic device, such as a thumb drive, and stored in a secure stand-alone computer system. These sealed documents will not be uploaded to CM/ECF.

“This new practice will not change current policies regarding public access to court records, since sealed records are confidential and currently are not available to the public,” the AO said.

James Lewis, senior vice president at the Center for Strategic and International Studies, said it’s too soon to tell the true impact of the breach at the court system, but the fact that they were apparently targeted is a “a very big deal.”

“We don’t know what the Russians took, but the fact that they had access to this system means they had access to a lot of great stuff, because federal cases tend to involve fairly high profile targets,” he said.

All Aboard the Pequod!

Like countless others, I frittered away the better part of Jan. 6 doomscrolling and watching television coverage of the horrifying events unfolding in our nation’s capital, where a mob of President Trump supporters and QAnon conspiracy theorists was incited to lay siege to the U.S. Capitol. For those trying to draw meaning from the experience, might I suggest consulting the literary classic Moby Dick, which simultaneously holds clues about QAnon’s origins and offers an apt allegory about a modern-day Captain Ahab and his ill-fated obsessions.

Many have speculated that Jim Watkins, the administrator of the online message board 8chan (a.k.a. 8kun), and/or his son Ron are in fact “Q,” the anonymous persona behind the QAnon conspiracy theory, which holds that President Trump is secretly working to save the world from a satanic cult of pedophiles and cannibals.

Last year, as I was scrutinizing the computer networks that kept QAnon online, researcher Ron Guilmette pointed out a tantalizing utterance from Watkins the younger which adds tenuous credence to the notion that one or both of them is Q.

We’ll get to how the Great White Whale (the Capitol?) fits into this tale in a moment. But first, a bit of background. A person identified only as “Q” has for years built an impressive following for the far-right conspiracy movement by leaving periodic “Q drops,” cryptic messages that QAnon adherents spend much time and effort trying to decipher and relate to current events.

Researchers who have studied more than 5,000 Q drops are convinced that there are two distinct authors of these coded utterances. The leading theory is that those identities corresponded to the aforementioned father-and-son team responsible for operating 8chan.

Jim Watkins, 56, is the current owner of 8chan, a community perhaps now best known as a forum for violent extremists and mass shooters. Watkins is an American pig farmer based in the Philippines; Ron reportedly resides in Japan.

In the aftermath of back-to-back mass shootings on Aug. 3 and Aug. 4, 2019 in which a manifesto justifying one of the attacks was uploaded to 8chan, Cloudflare stopped providing their content delivery network to 8chan. Several other providers quickly followed suit, leaving 8chan offline for months before it found a haven at a notorious bulletproof hosting facility in Russia.

One reason Q watchers believe Ron and Jim Watkins may share authorship over the Q drops is that while 8chan was offline, the messages from Q ceased. The drops reappeared only months later when 8chan rebranded as 8kun.

CALL ME ISHMAEL

Here’s where the admittedly “Qonspiratorial” clue about the Watkins’ connection to Q comes in. On Aug. 5, 2019, Ron Watkins posted a Twitter message about 8chan’s ostracization which compared the community’s fate to that of the Pequod, the name of the doomed whaling ship in the Herman Melville classic “Moby Dick.”

“If we are still down in a few hours then maybe 8chan will just go clearnet and we can brave DDOS attacks like Ishmael on the Pequod,” Watkins the younger wrote.

Ishmael, the first-person narrator in the novel, is a somewhat disaffected American sailor who decides to try his hand at a whaling ship. Ishmael is a bit of a minor character in the book; very soon into the novel we are introduced to a much more interesting and enigmatic figure — a Polynesian harpooner by the name of Queequeg.

Apart from being a cannibal from the Pacific islands who has devoured many people, Queequeg is a pretty nice guy and shows Ismael the ropes of whaling life. Queequeg is covered head to toe in tattoos, which are described by the narrator as the work of a departed prophet and seer from the cannibal’s home island.

Like so many Q drops, Queequeg’s tattoos tell a mysterious tale, but we never quite learn what that full story is. Indeed, the artist who etched them into Queequeg’s body is long dead, and the cannibal himself can’t seem to explain what it all means.

Ishmael describes Queequeg’s mysterious markings in this passage:

“…a complete theory of the heavens and earth, and a mystical treatise on the art of attaining truth; so that Queequeg in his own proper person was a riddle to unfold; a wondrous work in one volume; but whose mysteries not even himself could read, though his own live heart beat against them; and these mysteries were therefore destined in the end to moulder away with the living parchment whereon they were inscribed, and so be unsolved to the last.”

THE GREAT WHITE WHALE

It’s perhaps fitting then that one of the most recognizable figures from the mob that stormed the U.S. Capitol on Wednesday was a heavily-tattooed, spear-wielding QAnon leader who goes by the name “Q Shaman” (a.k.a. Jake Angeli).

“Q Shaman,” a.k.a. Jake Angeli, at a Black Lives Matter event in Arizona (left) and Wednesday, confronted by U.S. Capitol Police. Image: Twitter, @KelemenCari.

“Angeli’s presence at the riot, along with others wearing QAnon paraphernalia, comes as the conspiracy-theory movement has been responsible for the popularization of Trump’s voter-fraud conspiracy theories,” writes Rachel E. Greenspan for Yahoo! News.

“As Q has become increasingly hands-off, giving fewer and fewer messages to his devotees, QAnon leaders like Angeli have gained fame and power in the movement,” Greenspan wrote.

If somehow Moby Dick was indeed the inspiration for the “Q” identity in QAnon, yesterday’s events at The Capitol were the inexorable denouement of a presidential term that increasingly came to be defined by conspiracy theories. In a somewhat prescient Hartford Courant op-ed published in 2018, author Steven Almond observed that Trump’s presidency could be best understood through the lens of the Pequod’s Captain Ahab. To wit:

“Melville is offering a mythic account of how one man’s virile bombast ensnares everyone and everything it encounters. The setting is nautical, the language epic. But the tale, stripped to its ribs, is about the seductive power of the wounded male ego, how naturally a ship steered by men might tack to its vengeful course.”

“Trump’s presidency has been, in its way, a retelling of this epic. Whether we cast him as agent or principal hardly matters. What matters is that Americans have joined the quest. In rapture or disgust, we’ve turned away from the compass of self-governance and toward the mesmerizing drama of aggression on display, the masculine id unchained and all that it unchains within us. With every vitriolic tweet storm and demeaning comment, Trump strikes through the mask.”

EPILOGUE

If all of the above theorizing reads like yet another crackpot QAnon conspiracy, that may be the inevitable consequence of my spending far too much time going down this particular rabbit hole (and re-reading Moby Dick in the process!).

In any case, none of this is likely to matter to the diehard QAnon conspiracy theorists themselves, says Mike Rothschild, a writer who specializes in researching and debunking conspiracy theories.

“Even if Jim Watkins was revealed as owning the board or making the posts, it wouldn’t matter,” Rothschild said. “Anything that happens that disconfirms Q being an official in the military industrial complex is going to help fuel their persecution complex.”

Rothschild has been working hard on finishing his next book, “The Storm is Upon Us: How QAnon Became a Movement, Cult, and Conspiracy Theory of Everything,” which is due to be published in October 2021. Who’s printing the book? Ten points if you guessed Melville House, an independent publisher named after Herman Melville.

Hamas May Be Threat to 8chan, QAnon Online

In October 2020, KrebsOnSecurity looked at how a web of sites connected to conspiracy theory movements QAnon and 8chan were being kept online by DDoS-Guard, a dodgy Russian firm that also hosts the official site for the terrorist group Hamas. New research shows DDoS-Guard relies on data centers provided by a U.S.-based publicly traded company, which experts say could be exposed to civil and criminal liabilities as a result of DDoS-Guard’s business with Hamas.

Many of the IP address ranges in in this map of QAnon and 8Chan-related sites — are assigned to VanwaTech. Source: twitter.com/Redrum_of_Crows

Last year’s story examined how a phone call to Oregon-based CNServers was all it took to briefly sideline multiple websites related to 8chan/8kun — a controversial online image board linked to several mass shootings — and QAnon, the far-right conspiracy theory which holds that a cabal of Satanic pedophiles is running a global child sex-trafficking ring and plotting against President Donald Trump.

From that piece:

A large number of 8kun and QAnon-related sites (see map above) are connected to the Web via a single Internet provider in Vancouver, Wash. called VanwaTech (a.k.a. “OrcaTech“). Previous appeals to VanwaTech to disconnect these sites have fallen on deaf ears, as the company’s owner Nick Lim reportedly has been working with 8kun’s administrators to keep the sites online in the name of protecting free speech.

After that story, CNServers and a U.K.-based hosting firm called SpartanHost both cut ties with VanwaTech. Following a brief disconnection, the sites came back online with the help of DDoS-Guard, an Internet company based in Russia. DDoS-Guard is now VanwaTech’s sole connection to the larger Internet.

A review of the several thousand websites hosted by DDoS-Guard is revelatory, as it includes a vast number of phishing sites and domains tied to cybercrime services or forums online.

Replying to requests for comment from a CBSNews reporter following up on my Oct. 2020 story, DDoS-Guard issued a statement saying, “We observe network neutrality and are convinced that any activity not prohibited by law in our country has the right to exist.”

But experts say DDoS-Guard’s business arrangement with a Denver-based publicly traded data center firm could create legal headaches for the latter thanks to the Russian company’s support of Hamas.

In a press release issued in late 2019, DDoS-Guard said its services rely in part on a traffic-scrubbing facility in Los Angeles owned by CoreSite [NYSE:COR], a real estate investment trust which invests in “carrier-neutral data centers and provides colocation and peering services.”

This facilities map published by DDoS-Guard suggests the company’s network actually has at least two points of presence in the United States.

Hamas has long been named by the U.S. Treasury and State departments as a Specially Designated Global Terrorist (SDGT) organization. Under such a designation, any U.S. person or organization that provides money, goods or services to an SDGT entity could face civil and/or criminal prosecution and hefty fines ranging from $250,000 to $1 million per violation.

Sean Buckley, a former Justice Department prosecutor with the law firm Kobre & Kim, said U.S. persons and companies within the United States “are prohibited from any transaction or dealing in property or interests in property blocked pursuant to an entity’s designation as a SDGT, including but not limited to the making or receiving of any contribution of funds, goods, or services to or for the benefit of individuals or entities so designated.”

CoreSite did not respond to multiple requests for comment. But Buckley said companies can incur fines and prosecution for violating SDGT sanctions even when they don’t know that they are doing so.

In 2019, for example, a U.S. based cosmetics company was fined $1 million after investigators determined its eyelash kits were sourcing materials from North Korea, even though the supplier in that case told the cosmetics firm the materials had come from China.

“U.S. persons or companies found to willfully violate these regulations can be subject to criminal penalties under the International Emergency Economic Powers Act,” Buckley said. “However, even in the case that they are unaware they’re violating these regulations, or if the transaction isn’t directly with the sanctioned entity, these companies still run a risk of facing substantial civil and monetary penalties by the Department of Treasury’s Office of Foreign Asset Control if the sanctioned entity stands to benefit from such a transaction.”

DDoS-Guard said its partnership with CoreSite will help its stable of websites load more quickly and reliably for people visiting them from the United States. It is possible that when and if CoreSite decides it’s too risky to continue doing business with DDoS-Guard, sites like those affiliated with Hamas, QAnon and 8Chan may become more difficult to reach.

Meanwhile, DDoS-Guard customer VanwaTech continues to host a slew of sites promoting the conspiracy theory that the U.S. 2020 presidential election was stolen from President Donald Trump via widespread voting fraud and hacked voting machines, including maga[.]host, donaldsarmy[.]us, and donaldwon[.]com.

These sites are being used to help coordinate a protest rally in Washington, D.C. on January 6, 2021, the same day the U.S. Congress is slated to count electoral votes certified by the Electoral College, which in December elected Joseph R. Biden as the 46th president of The United States.

In a tweet late last year, President Trump urged his supporters to attend the Jan. 6 protest, saying the event “will be wild.”

8chan, which has rebranded as 8kun, has been linked to white supremacism, neo-Nazism, antisemitism, multiple mass shootings, and child pornography. The FBI in 2019 identified QAnon as a potential domestic terror threat, noting that some of its followers have been linked to violent incidents motivated by fringe beliefs.

Happy 11th Birthday, KrebsOnSecurity!

Today marks the 11th anniversary of KrebsOnSecurity! Thank you, Dear Readers, for your continued encouragement and support!

With the ongoing disruption to life and livelihood wrought by the Covid-19 pandemic, 2020 has been a fairly horrid year by most accounts. And it’s perhaps fitting that this was also a leap year, piling on an extra day to a solar rotation that most of us probably can’t wait to see in the rearview mirror.

But it was hardly a dull one for computer security news junkies. In almost every category — from epic breaches and ransomware to cybercrime justice and increasingly aggressive phishing and social engineering scams — 2020 was a year that truly went to eleven.

Almost 150 stories here this past year generated nearly 9,000 responses from readers (although about 6 percent of those were on just one story). Thank you all for your thoughtful engagement, wisdom, news tips and support.

I’d like to reprise a note from last year’s anniversary post concerning ads. A good chunk of the loyal readers here are understandably security- and privacy-conscious, and many block advertisements by default — including the ads displayed here.

KrebsOnSecurity does not run third-party ads and has no plans to change that; all of the creatives you see on this site are hosted in-house, are purely image-based, and are vetted first by Yours Truly. Love them or hate ’em, these ads help keep the content at KrebsOnSecurity free to any and all readers. If you’re currently blocking ads here, please consider making an exception for this site.

In case you missed them, some of the most popular feature/enterprise stories on the site this year (in no particular order) included:

The Joys of Owning an ‘OG’ Email Account
Confessions of an ID Theft Kingpin (Part II)
Why and Where You Should Plant Your Flag
Thinking of a Career in Cybersecurity? Read This
Turn on MFA Before Crooks Do it for You
Romanian Skimmer Gang in Mexico Outed by KrebsOnSecurity Stole $1.2 Billion
Who’s Behind the ‘Web Listings’ Mail Scam?
When in Doubt: Hang Up, Look Up, & Call Back
Riding the State Unemployment Fraud Wave
Would You Have Fallen for this Phone Scam?

VMware Flaw a Vector in SolarWinds Breach?

U.S. government cybersecurity agencies warned this week that the attackers behind the widespread hacking spree stemming from the compromise at network software firm SolarWinds used weaknesses in other, non-SolarWinds products to attack high-value targets. According to sources, among those was a flaw in software virtualization platform VMware, which the U.S. National Security Agency (NSA) warned on Dec. 7 was being used by Russian hackers to impersonate authorized users on victim networks.

On Dec. 7, 2020, the NSA said “Russian state-sponsored malicious cyber actors are exploiting a vulnerability in VMware Access and VMware Identity Manager products, allowing the actors access to protected data and abusing federated authentication.”

VMware released a software update to plug the security hole (CVE-2020-4006) on Dec. 3, and said it learned about the flaw from the NSA.

The NSA advisory (PDF) came less than 24 hours before cyber incident response firm FireEye said it discovered attackers had broken into its networks and stolen more than 300 proprietary software tools the company developed to help customers secure their networks.

On Dec. 13, FireEye disclosed that the incident was the result of the SolarWinds compromise, which involved malicious code being surreptitiously inserted into updates shipped by SolarWinds for users of its Orion network management software as far back as March 2020.

In its advisory on the VMware vulnerability, the NSA urged patching it “as soon as possible,” specifically encouraging the National Security System, Department of Defense, and defense contractors to make doing so a high priority.

The NSA said that in order to exploit this particular flaw, hackers would already need to have access to a vulnerable VMware device’s management interface — i.e., they would need to be on the target’s internal network (provided the vulnerable VMware interface was not accessible from the Internet). However, the SolarWinds compromise would have provided that internal access nicely.

In response to questions from KrebsOnSecurity, VMware said it has “received no notification or indication that the CVE 2020-4006 was used in conjunction with the SolarWinds supply chain compromise.”

VMware added that while some of its own networks used the vulnerable SolarWinds Orion software, an investigation has so far revealed no evidence of exploitation.

“While we have identified limited instances of the vulnerable SolarWinds Orion software in our environment, our own internal investigation has not revealed any indication of exploitation,” the company said in a statement. “This has also been confirmed by SolarWinds own investigations to date.”

On Dec. 17, DHS’s Cybersecurity and Infrastructure Security Agency (CISA) released a sobering alert on the SolarWinds attack, noting that CISA had evidence of additional access vectors other than the SolarWinds Orion platform.

CISA’s advisory specifically noted that “one of the principal ways the adversary is accomplishing this objective is by compromising the Security Assertion Markup Language (SAML) signing certificate using their escalated Active Directory privileges. Once this is accomplished, the adversary creates unauthorized but valid tokens and presents them to services that trust SAML tokens from the environment. These tokens can then be used to access resources in hosted environments, such as email, for data exfiltration via authorized application programming interfaces (APIs).”

Indeed, the NSA’s Dec. 7 advisory said the hacking activity it saw involving the VMware vulnerability “led to the installation of a web shell and follow-on malicious activity where credentials in the form of SAML authentication assertions were generated and sent to Microsoft Active Directory Federation Services (ADFS), which in turn granted the actors access to protected data.”

Also on Dec. 17, the NSA released a far more detailed advisory explaining how it has seen the VMware vulnerability being used to forge SAML tokens, this time specifically referencing the SolarWinds compromise.

Asked about the potential connection, the NSA said only that “if malicious cyber actors gain initial access to networks through the SolarWinds compromise, the TTPs [tactics, techniques and procedures] noted in our December 17 advisory may be used to forge credentials and maintain persistent access.”

“Our guidance in this advisory helps detect and mitigate against this, no matter the initial access method,” the NSA said.

CISA’s analysis suggested the crooks behind the SolarWinds intrusion were heavily focused on impersonating trusted personnel on targeted networks, and that they’d devised clever ways to bypass multi-factor authentication (MFA) systems protecting networks they targeted.

The bulletin references research released earlier this week by security firm Volexity, which described encountering the same attackers using a novel technique to bypass MFA protections provided by Duo for Microsoft Outlook Web App (OWA) users.

Duo’s parent Cisco Systems Inc. responded that the attack described by Volexity didn’t target any specific vulnerability in its products. As Ars Technica explained, the bypass involving Duo’s protections could have just as easily involved any of Duo’s competitors.

“MFA threat modeling generally doesn’t include a complete system compromise of an OWA server,” Ars’ Dan Goodin wrote. “The level of access the hacker achieved was enough to neuter just about any defense.”

Several media outlets, including The New York Times and The Washington Post, have cited anonymous government sources saying the group behind the SolarWinds hacks was known as APT29 or “Cozy Bear,” an advanced threat group believed to be part of the Russian Federal Security Service (FSB).

SolarWinds has said almost 18,000 customers may have received the backdoored Orion software updates. So far, only a handful of customers targeted by the suspected Russian hackers behind the SolarWinds compromise have been made public — including the U.S. Commerce, Energy and Treasury departments, and the DHS.

No doubt we will hear about new victims in the public and private sector in the coming days and weeks. In the meantime, thousands of organizations are facing incredibly costly, disruptive and time-intensive work in determining whether they were compromised and if so what to do about it.

The CISA advisory notes the attackers behind the SolarWinds compromises targeted key personnel at victim firms — including cyber incident response staff, and IT email accounts. The warning suggests organizations that suspect they were victims should assume their email communications and internal network traffic are compromised, and rely upon or build out-of-band systems for discussing internally how they will proceed to clean up the mess.

“If the adversary has compromised administrative level credentials in an environment—or if organizations identify SAML abuse in the environment, simply mitigating individual issues, systems, servers, or specific user accounts will likely not lead to the adversary’s removal from the network,” CISA warned. “In such cases, organizations should consider the entire identity trust store as compromised. In the event of a total identity compromise, a full reconstitution of identity and trust services is required to successfully remediate. In this reconstitution, it bears repeating that this threat actor is among the most capable, and in many cases, a full rebuild of the environment is the safest action.”

Malicious Domain in SolarWinds Hack Turned into ‘Killswitch’

A key malicious domain name used to control potentially thousands of computer systems compromised via the months-long breach at network monitoring software vendor SolarWinds was commandeered by security experts and used as a “killswitch” designed to turn the sprawling cybercrime operation against itself, KrebsOnSecurity has learned.

Austin, Texas-based SolarWinds disclosed this week that a compromise of its software update servers earlier this year may have resulted in malicious code being pushed to nearly 18,000 customers of its Orion platform. Many U.S. federal agencies and Fortune 500 firms use(d) Orion to monitor the health of their IT networks.

On Dec. 13, cyber incident response firm FireEye published a detailed writeup on the malware infrastructure used in the SolarWinds compromise, presenting evidence that the Orion software was first compromised back in March 2020. FireEye said hacked networks were seen communicating with a malicious domain name — avsvmcloud[.]com — one of several domains the attackers had set up to control affected systems.

As first reported here on Tuesday, there were signs over the past few days that control over the domain had been transferred to Microsoft. Asked about the changeover, Microsoft referred questions to FireEye and to GoDaddy, the current domain name registrar for the malicious site.

Today, FireEye responded that the domain seizure was part of a collaborative effort to prevent networks that may have been affected by the compromised SolarWinds software update from communicating with the attackers. What’s more, the company said the domain was reconfigured to act as a “killswitch” that would prevent the malware from continuing to operate in some circumstances.

“SUNBURST is the malware that was distributed through SolarWinds software,” FireEye said in a statement shared with KrebsOnSecurity. “As part of FireEye’s analysis of SUNBURST, we identified a killswitch that would prevent SUNBURST from continuing to operate.”

The statement continues:

“Depending on the IP address returned when the malware resolves avsvmcloud[.]com, under certain conditions, the malware would terminate itself and prevent further execution. FireEye collaborated with GoDaddy and Microsoft to deactivate SUNBURST infections.”

“This killswitch will affect new and previous SUNBURST infections by disabling SUNBURST deployments that are still beaconing to avsvmcloud[.]com. However, in the intrusions FireEye has seen, this actor moved quickly to establish additional persistent mechanisms to access to victim networks beyond the SUNBURST backdoor.

This killswitch will not remove the actor from victim networks where they have established other backdoors. However, it will make it more difficult to for the actor to leverage the previously distributed versions of SUNBURST.”

It is likely that given their visibility into and control over the malicious domain, Microsoft, FireEye, GoDaddy and others now have a decent idea which companies may still be struggling with SUNBURST infections.

The killswitch revelations came as security researchers said they’d made progress in decoding SUNBURST’s obfuscated communications methods. Chinese cybersecurity firm RedDrip Team published their findings on Github, saying its decoder tool had identified nearly a hundred suspected victims of the SolarWinds/Orion breach, including universities, governments and high tech companies.

Meanwhile, the potential legal fallout for SolarWinds in the wake of this breach continues to worsen. The Washington Post reported Tuesday that top investors in SolarWinds sold millions of dollars in stock in the days before the intrusion was revealed. SolarWinds’s stock price has fallen more than 20 percent in the past few days. The Post cited former enforcement officials at the U.S. Securities and Exchange Commission (SEC) saying the sales were likely to prompt an insider trading investigation.

SolarWinds Hack Could Affect 18K Customers

The still-unfolding breach at network management software firm SolarWinds may have resulted in malicious code being pushed to nearly 18,000 customers, the company said in a legal filing on Monday. Meanwhile, Microsoft should soon have some idea which and how many SolarWinds customers were affected, as it recently took possession of a key domain name used by the intruders to control infected systems.

On Dec. 13, SolarWinds acknowledged that hackers had inserted malware into a service that provided software updates for its Orion platform, a suite of products broadly used across the U.S. federal government and Fortune 500 firms to monitor the health of their IT networks.

In a Dec. 14 filing with the U.S. Securities and Exchange Commission (SEC), SolarWinds said roughly 33,000 of its more than 300,000 customers were Orion customers, and that fewer than 18,000 customers may have had an installation of the Orion product that contained the malicious code. SolarWinds said the intrusion also compromised its Microsoft Office 365 accounts.

The initial breach disclosure from SolarWinds came five days after cybersecurity incident response firm FireEye announced it had suffered an intrusion that resulted in the theft of some 300 proprietary software tools the company provides to clients to help secure their IT operations.

On Dec. 13, FireEye published a detailed writeup on the malware infrastructure used in the SolarWinds compromise, presenting evidence that the Orion software was first compromised back in March 2020. FireEye didn’t explicitly say its own intrusion was the result of the SolarWinds hack, but the company confirmed as much to KrebsOnSecurity earlier today.

Also on Dec. 13, news broke that the SolarWinds hack resulted in attackers reading the email communications at the U.S. Treasury and Commerce departments.

On Dec. 14, Reuters reported the SolarWinds intrusion also had been used to infiltrate computer networks at the U.S. Department of Homeland Security (DHS). That disclosure came less than 24 hours after DHS’s Cybersecurity and Infrastructure Security Agency (CISA) took the unusual step of issuing an emergency directive ordering all federal agencies to immediately disconnect the affected Orion products from their networks.

ANALYSIS

Security experts have been speculating as to the extent of the damage from the SolarWinds hack, combing through details in the FireEye analysis and elsewhere for clues about how many other organizations may have been hit.

And it seems that Microsoft may now be in perhaps the best position to take stock of the carnage. That’s because sometime on Dec. 14, the software giant took control over a key domain name — avsvmcloud[.]com — that was used by the SolarWinds hackers to communicate with systems compromised by the backdoored Orion product updates.

Armed with that access, Microsoft should be able to tell which organizations have IT systems that are still trying to ping the malicious domain. However, because many Internet service providers and affected companies are already blocking systems from accessing that malicious control domain or have disconnected the vulnerable Orion services, Microsoft’s visibility may be somewhat limited.

Microsoft has a long history of working with federal investigators and the U.S. courts to seize control over domains involved in global malware menaces, particularly when those sites are being used primarily to attack Microsoft Windows customers.

Microsoft dodged direct questions about its visibility into the malware control domain, suggesting those queries would be better put to FireEye or GoDaddy (the current domain registrar for the malware control server). But in a response on Twitter, Microsoft spokesperson Jeff Jones seemed to confirm that control of the malicious domain had changed hands.

“We worked closely with FireEye, Microsoft and others to help keep the internet safe and secure,” GoDaddy said in a written statement. “Due to an ongoing investigation and our customer privacy policy, we can’t comment further at this time.”

FireEye declined to answer questions about exactly when it learned of its own intrusion via the Orion compromise, or approximately when attackers first started offloading sensitive tools from FireEye’s network. But the question is an interesting one because its answer may speak to the motivations and priorities of the hackers.

Based on the timeline known so far, the perpetrators of this elaborate hack would have had a fairly good idea back in March which of SolarWinds’ 18,000 Orion customers were worth targeting, and perhaps even in what order.

Alan Paller, director of research for the SANS Institute, a security education and training company based in Maryland, said the attackers likely chose to prioritize their targets based on some calculation of risk versus reward.

Paller said the bad guys probably sought to balance the perceived strategic value of compromising each target with the relative likelihood that exploiting them might result in the entire operation being found out and dismantled.

“The way this probably played out is the guy running the cybercrime team asked his people to build a spreadsheet where they ranked targets by the value of what they could get from each victim,” Paller said. “And then next to that they likely put a score for how good the malware hunters are at the targets, and said let’s first go after the highest priority ones that have a hunter score of less than a certain amount.”

The breach at SolarWinds could well turn into an existential event for the company, depending on how customers react and how SolarWinds is able to weather the lawsuits that will almost certainly ensue.

“The lawsuits are coming, and I hope they have a good general counsel,” said James Lewis, senior vice president at the Center for Strategic and International Studies. “Now that the government is telling people to turn off [the SolarWinds] software, the question is will anyone turn it back on?”

According to its SEC filing, total revenue from the Orion products across all customers — including those who may have had an installation of the Orion products that contained the malicious update — was approximately $343 million, or roughly 45 percent of the firm’s total revenue. SolarWinds’ stock price has fallen 25 percent since news of the breach first broke.

Some of the legal and regulatory fallout may hinge on what SolarWinds knew or should have known about the incident, when, and how it responded. For example, Vinoth Kumar, a cybersecurity “bug hunter” who has earned cash bounties and recognition from multiple companies for reporting security flaws in their products and services, posted on Twitter that he notified SolarWinds in November 2019 that the company’s software download website was protected by a simple password that was published in the clear on SolarWinds’ code repository at Github.

Andrew Morris, founder of the security firm GreyNoise Intelligence, on said that as of Tuesday evening SolarWinds still hadn’t removed the compromised Orion software updates from its distribution server.

Another open question is how or whether the incoming U.S. Congress and presidential administration will react to this apparently broad cybersecurity event. CSIS’s Lewis says he doubts lawmakers will be able to agree on any legislative response, but he said it’s likely the Biden administration will do something.

“It will be a good new focus for DHS, and the administration can issue an executive order that says federal agencies with regulatory authority need to manage these things better,” Lewis said. “But whoever did this couldn’t have picked a better time to cause a problem, because their timing almost guarantees a fumbled U.S. response.”