Posted by Felix Groebert, Information Security Engineering
Today, we’re excited to announce a yearly Google Cloud Platform (GCP) VRP Prize to promote security research of GCP. A prize of $100,000.00 will be paid to the reporter of the best vulnerability affecting GCP reported through our Vulnerability Reward Program (g.co/vulnz) and having a public write-up (nominations will be received here).
We’ve received vulnerability reports for various application security flaws in GCP over the years, but we felt research of our Cloud platform has been under-represented in our Vulnerability Reward Program. So, with the GCP VRP Prize, we hope to encourage even more researchers to focus on GCP products and help us identify even more security vulnerabilities.
Note that we will continue to pay hundreds of thousands of dollars to our top bug hunters through our Vulnerability Research Grants Program even when no bugs are found, and to reward up to tens of thousands of dollars per bug to the most impactful findings. This prize is meant to create an additional incentive for more people to focus on public, open security research on GCP who would otherwise not participate in the reward program.
This competition draws on our previous contests, such as Pwnium and the Project Zero Prize, and rather than focusing bug hunters on collecting vulnerabilities for complex bug chains, we are attempting a slightly different twist and selecting a single winner out of all vulnerabilities we receive. That said, this approach comes with its own challenges, such as: defining the right incentives for bug hunters (both in terms of research as well as their communications with our team when reporting vulnerabilities); or ensuring there are no conflicting incentives, either when our own team is looking for similar vulnerabilities (since we aren't eligible for collecting the prize).
For the rest of the year, we will be seeking feedback from our top bug hunters and the security community to help define what vulnerabilities are the most significant, and we hope we can work together to find the best way to incentivize, recognize, and reward open security research. To further incentivize research in 2019, we will be issuing GCP VRP grants summing up to $100,000 to our top 2018 researchers.
Head over here for the full details on the contest. Note that if you have budget constraints for access to testing environments, you can use the free tier of GCP.
We look forward to our Vulnerability Rewards Programs resulting in even more GCP customer protection in the following years thanks to the hard work of the security research community. Follow us on @GoogleVRP.
Avaya is the second largest VOIP solution provider (source) with an install base covering 90% of the Fortune 100 companies (source), with products targeting a wide spectrum of customers, from small business and midmarket, to large corporations. As part of the ongoing McAfee Advanced Threat Research effort into researching critical vulnerabilities in widely deployed software and hardware, we decided to have a look at the Avaya 9600 series IP Deskphone. We were able to find the presence of a Remote Code Execution (RCE) vulnerability in a piece of open source software that Avaya likely copied and modified 10 years ago, and then failed to apply subsequent security patches to. The bug affecting the open source software was reported in 2009, yet its presence in the phone’s firmware remained unnoticed until now. Only the H.323 software stack is affected (as opposed to the SIP stack that can also be used with these phones), and the Avaya Security Advisory (ASA) can be found here ASA-2019-128.
The video below demonstrates how an attacker can leverage this bug to take over the normal operation of the phone, exfiltrate audio from its speaker phone, and potentially “bug” the phone. The current attack is conducted with the phone directly connected to an attacker’s laptop but would also work via a connection to the same network as a vulnerable phone. The full technical details can be found here, while the rest of this article will give a high-level overview on how this bug was found and some consideration regarding its resolution. The firmware image Avaya published on June 25th resolves the issue and can be found here. As a user, you can verify if your Deskphone is vulnerable: first determine if you have one of the affected models (9600 Series, J100 Series or B189), then you can find which firmware version your phone is using in the “About Avaya IP Deskphone” screen under the Home menu, version 6.8.1 and earlier are vulnerable when using a H.323 firmware (SIP versions are not affected).
What are Researchers Looking for?
When studying the security of embedded and IoT devices, researchers generally have a couple of goals in mind to help kickstart their research. In most cases, two of the main targets are recovering the files on the system so as to study how the device functions, and then finding a way to interact directly with the system in a privileged fashion (beyond what a normal user should be able to do). The two can be intertwined, for instance getting a privileged access to the system can enable a researcher to recover the files stored on it, while recovering the files first can show how to enable a privileged access.
In this case, recovering the files was straightforward, but gaining a privileged access required a little more patience.
Recovering the Files From the Phone
When we say recovering the files from the phone, we mean looking for the operating system and the various pieces of software running on it. User files, e.g. contacts, settings and call logs, are usually not of interest to a security researcher and will not be covered here. To recover the files, the easiest approach is to look for firmware updates for the device. If we are lucky, they will be freely available and not encrypted. In most cases, an encrypted firmware does not increase the security of the system but rather raises the barrier of entry for security researchers and attackers alike. In this case, we are in luck, Avaya’s website serves firmware updates for its various phone product lines and anyone can download them. The download contains multiple tar files (a type of archive file format). We can then run a tool called binwalk on the extracted files. Binwalk is a large dictionary of patterns that represents known file formats; given an unknown firmware file, it will look for any known pattern and, upon finding potential matches, will attempt to process them accordingly. For instance, if it finds what looks like a .zip file inside the firmware, it will try to unzip it. Running this tool is always a good first step when facing an unknown firmware file as, in most cases, it will identify useful items for you.
When processing the phone’s firmware, extracting the files and running binwalk on them gave us the program the phone runs at startup (the bootloader), the Linux kernel used by the phone, and a JFFS filesystem that contains all the phone’s binaries and configuration files. This is a great start, as from there we can start understanding the inner workings of the device and look for bugs. At this stage however, we are limited to performing a static analysis: we can look at the files and peek at the assembly instructions of various binaries, but we cannot execute them. To make life easier, there are usually two options. The first one is to emulate the whole phone, or at least some region of interest, while the other is to get a privileged access to the system, to inspect what is running on it as well as run debugging tools. Best results come when you mix and match all these options appropriately. For the sake of simplicity, we will only cover the latter, but both were used in various ways to help us in our research.
Getting the Privileged Access
In most cases, when talking about gaining privileged access to an IoT/embedded device, security researchers are on the lookout for an administrative interface called a root shell that lets them execute any code they want with the highest level of privilege. Sometimes, one is readily available for maintenance purposes; other times more effort is required to gain access to it, assuming one is present in the first place. This is when hardware hacking comes into play; security researchers love to rip open devices and void warranties, looking for potential debug ports, gatekeepers of the sought-after privileged access.
Close up of the phone’s circuit board. UART ports in Red and the EEPROM in blue
In the picture above, we can see two debug ports labeled UART0 and UART1. This type of test point, where the copper is directly exposed, is commonly used during the manufacturing process to program the device or verify everything is working properly. UART stands for Universal Asynchronous Receiver-Transmitter and is meant for two-way communication. This is the most likely place where we can find the administrative access we are looking for. By buying a $15 cable that converts UART to USB and soldering wires onto the test pads, we can see debug information being printed on screen when the phone boots up, but soon the flow of debug information dries up. This is a curious behavior—why stop the debug messages?—so we need to investigate more. By using a disassembler to convert raw bytes into computer instructions, we can peek into the code of the bootloader recovered earlier and find out that during the boot process the phone fetches settings from external memory to decide whether the full set of debug features should be enabled on the serial console. The external memory is called an EEPROM and is easily identifiable on the board, first by its shape and then by the label printed on it. Labels on electronic components are used to identify them and to retrieve their associated datasheet, the technical documentation describing how to use the chip from an electrical engineering standpoint. Soldering wires directly to the chip under a microscope, and connecting it to a programmer (a $30 gizmo called a buspirate), allows us to change the configuration stored on it and enable the debug capabilities of the phone.
EEPROM ready to be re-programmed
Rebooting the phones gives us much more debug information and, eventually, we are greeted with the root shell we were after.
Confirmation we have a root shell. Unrelated debug messages are being printed while we are invoking the “whoami” command
The approach described above is fairly lengthy and is only interesting to security researchers in a similar situation. A more generic technique would be to directly modify the filesystem by altering the flash storage (a NAND Flash on the back of the circuit board) as we did for previous research, and then automatically start an SSH server or a remote shell. Another common technique is to tamper with the NAND flash while the filesystem is loading in memory, to get the bootloader in an exception state that will then allow the researcher to modify the boot arguments of the Linux kernel. Otherwise, to get remote shell access, using an older firmware with known RCE vulnerabilities is probably the easiest method to consider; it can be a good starting point for security researchers and is not threatening to regular users as they should already have the most up-to-date software. All things considered, these methods are not a risk to end-users and are more of a stepping stone for security researchers to conduct their research.
In Search of Vulnerabilities
After gaining access to a root shell and the ability to reverse engineer the files on the phone, we are faced with the open-ended task to look for potentially vulnerable software. As the phone runs Linux, the usual command line utilities people use for administering Linux systems are readily available to us. It is natural to look at the list of processes running, find the ones having network connection and so forth. While poking around, it becomes clear that one of the utilities, dhclient, is of great interest. It is already running on the system and handles network configuration (the so-called DHCP requests to configure the phone’s IP address). If we invoke it in the command line, the following is printed:
Showing a detailed help screen describing its expected arguments is normal behavior, but a 2004-2007 copyright is a big red flag. A quick search confirms that the 4.0.0 version is more than 10 years old and, even worse, an exploit targeting it is publicly available. Dhclient code is open source, so finding the differences between two successive version is straightforward. Studying the exploit code and how the bug was patched helps us to narrow down which part of the code could be vulnerable. By once again using a disassembler, we confirm the phone’s version of dhclient is indeed vulnerable to the bug reported in 2009. Converting the original exploit to make it work on the phone requires a day or two of work, while building the proof of concept demonstrated in the above video is a matter of mere hours. Indeed, all the tools to stream audio from the phone to a separate machine are already present on the system, which greatly reduces the effort to create this demo. We did not push the exploitation further than the Proof of Concept shown in the above video, but we can assume that at this point, building a weaponized version able to threaten private networks is more of a software engineering task and a skilled attacker might only need a few weeks, if not days, to put one together.
Upon finding the flaw, we immediately notified Avaya with detailed instructions on how to reproduce the bug and suggested fixes. They were able to fix, test and release a patched firmware image in approximately two months. At the time of publication, the fix will have been out for more than 30 days, leaving IT administrators ample time to deploy the new image. In a large enterprise setting, it is pretty common to first have a testing phase where a new image is being deployed to selected devices to ensure no conflict arises from the deployment. This explains why the timeline from the patch release to deployment to the whole fleet may take longer than what is typical in consumer grade software.
IoT and embedded devices tend to blend into our environment, in some cases not warranting a second thought about the security and privacy risks they pose. In this case, with a minimal hardware investment and free software, we were able to uncover a critical bug that remained out-of-sight for more than a decade. Avaya was prompt to fix the problem and the threat this bug poses is now mitigated, but it is important to realize this is not an isolated case and many devices across multiple industries still run legacy code more than a decade old. From a system administration perspective, it is important to consider all these networked devices as tiny black-box computers running unmanaged code which should be isolated and monitored accordingly. The McAfee Network Security Platform (NSP) detects this attack as “DHCP: Subnet Mask Option Length Overflow” (signature ID: 0x42601100), ensuring our customers remain protected. Finally, for the technology enthusiasts reading this, the barrier of entry to hardware hacking has never been this low, with plenty of online resources and cheap hardware to get started. Looking for this type of vulnerability is a great entry point to information security and will help make the embedded world a safer place.
During her briefing with Kelly Shortridge, vice president of product strategy at Capsule8, Dr. Nicole Forsgren, research and strategy at Google, did a beautiful job of adding imagery to the story she told of the attendee reactions during the now-famous talk Paul Hammond and John Allspaw gave at Velocity in 2009. If you're not familiar, the title of said talk was, "10 Deploys Per Day: Dev & Ops Cooperation at Flickr."
Forsgren recalled that, "The room was split. At the end of this process, large pieces of code would be deployed and, basically, lit everyone on fire. Half the room was amazed and it was changing the world. Half of the room said they were monsters and how dare they light people on fire 10 times per day." Forsgren concluded that "DevOps has crossed the chasm - the business benefits are too striking. We see most of the industry doing this. There is no turning the ship around."
Indeed, DevOps has long moved beyond the conceptual and has become a widely adopted practice in software development and delivery. It gave birth to the InfoSec equivalent of DevSecOps and the concept of "shifting security left." From where I sit within Veracode, I see the ways that many security solutions providers are doing their best to provide developers with the tools they need to embed security into their workflow, yet it’s clear that there is still more to be done to get InfoSec professionals on board.
"James Wickett has said the ratio of engineers in development, operations, and InfoSec in a typical technology organization is 100:10:1. If we integrate [InfoSec professionals] earlier to have input, the shift left can build a more collaborative culture, contribute to amazing outcomes - like stability, reliability, and resiliency," Forsgren said. "We need to build secure systems, and we will find ways to do this. We know this is super important, and security is the next frontier. Security can contribute to this and join DevOps. Or you can stand aside as DevOps figures this out and carves its own path. I would love to see InfoSec contributing the expertise we just don't have."
Forsgren was clearly echoing the sentiment Dino Dai Zovi expressed in his conference keynote. Certainly, the concept of being lit on fire 10 times per day would create a fight-or-flight response, and it is much easier to go to no than to go to yes. Yet, when Forsgren spoke about the benefits of this type of work, she explained that what InfoSec pros would face would be mini-fires with a smaller blast radius. She argues that it is time for InfoSec to say, "no, and…"
The Security of Chaos
It appeared that Shortridge couldn't have agreed more.
"The real DevOps will be held accountable for security fixes," said Shortridge. "So what should goals and outcomes become? Why should InfoSec and DevOps goals diverge? InfoSec should support innovation in the face of change - not add friction. InfoSec has arguably failed, so 'this is how we've always done it' is invalid. The greatest advances in security are rarely spawned by the security industry."
In other words, it's time to start jumping out of the proverbial planes in order to face our fears and start doing things differently in security. Shortridge reminded us that it is inevitable that things will fail and things will be pwned, which is why she is a proponent of adopting chaos engineering. Chaos engineering is the discipline of experimenting on a software system in production to provide your organization with a level of confidence in the system's capability to withstand turbulent and unexpected conditions, while still creating adequate quality of service (resiliency) during difficult times.
The concept of chaos engineering was created while Greg Orzell was overseeing Netflix's migration to the cloud in 2011. He wanted to address the lack of adequate resilience by creating a tool that would cause breakdowns in their production environment - the one used by Netflix customers. In doing this, the team could move from a development model that assumed no breakdowns to one where they were considered inevitable. This encouraged developers to build resilience into their software from the start. By regularly "killing" random instances of software service, they could test redundant architecture to make sure that a server failure wouldn't noticeably impact the customer experience.
"Expect your security controls will fail and prepare accordingly. System architectures must be designed assuming the controls and users will fail," she said. "Users very rarely follow the ideal behaviors. Don’t try to avoid incidents. Embrace your ability to respond to them. Ensure that your systems are resilient enough to handle incidents gracefully. Pivot toward realistic resilience."
If your team can plan for nothing but the chaos factor, then you should understand that there are true benefits to applying chaos resilience, including lower remediation costs, decreased stress levels during real incidents, and less burnout.
"Incidents are a problem with known processes, rather than fear and uncertainty. It creates feedback loops to foster understanding of systemic risk. Chaos engineering does this to help us continuously refine security strategy - essentially all the time red teaming. You have the ability to automate the toil, or the manual, repetitive, tactical work that doesn't provide enduring value," she said.
How to Marry DevOps and Security
At the end of the talk, Forsgren offered these tenants for a scalable love between DevOps and Security:
Sit in on early design decisions and demos – but say “No, and…” vs. “No.”
Provide input on tests so every testing suite has InfoSec’s stamp on it.
By the last “no” gate in the delivery process, nearly all issues will be fixed.
InfoSec should focus on outcomes that are aligned with business goals.
Time To Remediate (TTR) should become the preliminary anchor of your security metrics.
Security- and performance-related gamedays can’t be separate species.
Cultivate buy-in together for resilience and chaos engineering.
Visibility/observability: collecting system information is essential.
Your DevOps colleagues are likely already collecting the data you need - work with them to collect it.
Changing culture: change what people do, not what they think.
Forsgren and Shortridge made the case that security cannot force itself into DevOps, it must marry it - and have an equal partnership. Chaos/resilience are natural homes for InfoSec and represent its future, and InfoSec will need to evolve to unify responsibility and accountability.
"If not, InfoSec will sit at the kids’ table until it is uninvited from the business," Shortridge said. "Giving up control isn’t a harbinger of doom. Resilience is a beacon of hope."
Posted byElie Bursztein, Security & Anti-abuse Research Lead, Daniela Oliveira, Professor at the University of Florida
Phishing attacks continue to be one of the common forms of account compromise threats. Every day, Gmail blocks more than 100 million phishing emails and Google Safe Browsing helps protect more than 4 billion devices against dangerous sites.
As part of our ongoing efforts to further protect users from phishing, we’re partnering with Daniela Oliveira from the University of Florida during a talk at Black Hat 2019 to explore the reasons why social engineering attacks remain effective phishing tactics, even though they have been around for decades.
Overall, the research finds there are a few key factors that make phishing an effective attack vector:
Phishing is constantly evolving: 68% of the phishing emails blocked by Gmail today are new variations that were never seen before. This fast pace adversarial evolution requires humans and machines to adapt very quickly to prevent them.
Phishing is targeted: Many of the campaigns targeting Gmail end-users and enterprise consumers only target a few dozen individuals. Enterprise users being 4.8x more targeted than end-users.
Phishers are persuasion experts: As highlighted by Daniela’s research with Natalie Ebner et al. at the University of Florida, phishers have mastered the use of persuasion techniques, emotional salience and gain or loss framing to trick users into reacting to phishing emails.
45% of users don’t understand what phishing is: After surveying Internet users, we found that 45% of them do not understand what phishing is or the risk associated with it. This lack of awareness increases the risk of being phished and potentially hinders the adoption of 2-step verification.
Protecting users against phishing requires a layered defense approach that includes:
Educating users about phishing so they understand what it is, how to detect it and how to protect themselves.
Leveraging the recent advances in AI to build robust phishing detections that can keep pace with fast evolving phishing campaigns.
Displaying actionable phishing warnings that are easy to understand by users so they know how to react when they see them.
Using strong two factor authentication makes it more difficult for phishers to compromise accounts. Two-factor technologies, as visible in the graph above, can be effective against the various forms of phishing, which highlights the importance of driving awareness and adoption among users.
While technologies to help mitigate phishing exist, such as FIDO standard security keys, there is still work to be done to help users increase awareness understand how to protect themselves against phishing.
You’ve probably heard of CafePress, a custom T-shirt and merchandise company allowing users to create their own unique apparel and gifts. With a plethora of users looking to make their own creative swag, it’s no surprise that the company was recently targeted in a cybercriminal ploy. According to Forbes, CafePress experienced a data breach back in February that exposed over 23 million records including unique email addresses, names, physical addresses, phone numbers, and passwords.
How exactly did this breach occur? While this information is still a bit unclear, security researcher Jim Scott stated that approximately half of the breached passwords had been exposed through gaps in an encryption method called base64 SHA1. As a result, the breach database service HaveIBeenPwned sent out an email notification to those affected letting them know that their information had been compromised. According to Engadget, about 77% of the email addresses in the breach have shown up in previous breach alerts on HaveIBeenPwned.
Scott stated that those who used CafePress through third-party applications like Facebook or Amazon did not have their passwords compromised. And even though third-party platform users are safe from this breach, this isn’t always the case. With data breaches becoming more common, it’s important for users to protect their information as best as they can. Check out the following tips to help users defend their data:
Check to see if you’ve been affected. If you know you’ve made purchases through CafePress recently, use this tool to check if you could have been potentially affected.
Place a fraud alert. If you suspect that your data might have been compromised, place a fraud alert on your credit. This not only ensures that any new or recent requests undergo scrutiny, but also allows you to have extra copies of your credit report so you can check for suspicious activity.
Consider using identity theft protection. A solution like McAfee Identify Theft Protection will help you to monitor your accounts and alert you of any suspicious activity.
And, of course, stay on top of the latest consumer and mobile security threats by following meand @McAfee_Homeon Twitter, listen to our podcast Hackable?, and ‘Like’ us on Facebook.
When your organization is young and growing, you may find yourself overwhelmed with a never-ending to-do list. It can be easy to overlook security when you’re hiring new employees, finding infrastructure, and adopting policies. Without a proper cybersecurity strategy, however, the business that you’ve put your heart and soul into, or the brilliant idea that you’ve spent years bringing to life, are on the line. Every year, businesses face significant financial, brand, and reputational damage resulting from a data breach, and many small businesses don’t ever recover.
Not only that, but as you grow you may be looking to gain investors or strategic partners. Many of these firms are not willing to give organizations that don’t take security seriously a chance. A strong security stance can be your differentiator among your customers and within the Venture Capital landscape.
One thing’s for sure: you’ve spent a great deal of time creating a business of your own, so why throw it all away by neglecting your security? You can begin building your own cybersecurity strategy by following these steps:
1. Start by identifying your greatest business needs.
This understanding is critical when determining how your vulnerabilities could affect your organization. Possible business needs could include manufacturing, developing software, or gaining new customers. Make a list of your most important business priorities.
2. Conduct a third-party security assessment to identify and remediate the greatest vulnerabilities to your business needs.
The assessment should evaluate your organization’s overall security posture, as well as the security of your partners and contractors.
Once you understand the greatest risks to your business needs, you can prioritize your efforts and budget based on ways to remediate these.
3. Engage a Network Specialist to set-up a secure network or review your existing network.
A properly designed and configured network can help prevent unwanted users from getting into your environment and is a bare necessity when protecting your sensitive data.
Don’t have a set office space? If you and your team are working from home or communal office spaces, be sure to never conduct sensitive business on a shared network.
4. Implement onboarding (and offboarding) policies to combat insider threat, including a third-party vendor risk management assessment.
Your team is your first line of defense, but as you grow, managing the risk of bringing on more employees can be challenging. Whether attempting to maliciously steal data or clicking a bad link unknowingly, employees pose great threats to organizations.
As part of your onboarding policy, be sure to conduct thorough background checks and monitor users’ access privileges. This goes for your employees, as well as any third parties and contractors you bring on.
5. Implement a security awareness training program and take steps to make security awareness part of your company culture.
Make sure your training program includes topics such as password best practices, phishing identification and secure travel training. Keep in mind, though, that company-wide security awareness should be more than once-a-year training. Instead, focus on fostering a culture of cybersecurity awareness.
6. Set-up multi-factor authentication and anti-phishing measures.
Technology should simplify your security initiatives, not complicate them. Reduce the number of administrative notifications to only what is necessary and consider improvements that don’t necessarily require memorizing more passwords, such as password managers and multi-factor authentication for access to business-critical data.
7. Monitor your data and endpoints continuously with a Managed Security Services Provider.
As you grow, so does the amount of endpoints you have to manage and data you have to protect. One of the best ways to truly ensure this data is protected is to have analysts monitoring your data at all hours. A managed security services provider will monitor your data through a 24/7 security operations center, keeping eyes out for any suspicious activity such as: phishing emails, malicious sites, and any unusual network activity.
You’re not done yet: revisit your security strategy as you evolve.
It’s important to remember that effective cybersecurity strategies vary among organizations. As you grow, you’ll want to consider performing regular penetration testing and implementing an Incident Response Plan.
And, as your business changes, you must continually reassess your security strategy and threat landscape.