Posted under: Research and Analysis
As we discussed in the first post of our Making an Impact with Security Awareness Training series, organizations need to architect training programs around a clear definition of success, both to determine the most appropriate content to deliver, and also to manage management expectations. The definition of success for any security initiative is measurable risk reduction, and that applies just as much to security awareness training.
We also covered the limitations of existing training approaches – including weak generic content, and a lack of instrumentation & integration, to determine the extent of risk reduction. To overcome these limitations we introduced the concept of Continuous, Contextual Content (3C) as the cornerstone of the kind of training program which can achieve security initiatives.
We described 3C as:
“It’s giving employees the necessary training, understanding they won’t retain everything. Not the first time anyway. Learning requires repetition, but why repeat training to someone that already gets it? That’s a waste of time. Thus to follow up and focus on retention, you want to deliver appropriate content to the employee when they need it. That means refreshing the employee about phishing, not at a random time, but after they’ve clicked on a phishing message.”
Now we can dig in to understand how to move your training program toward 3C.
Start with Users
Any focus on risk reduction requires first identifying employees who present the most risk to the organization. Don’t overcomplicate your categorization process, or you won’t be able to keep it current. We suggest 4-6 groups categorized by their access to critical information.
- Senior Management: These individuals have the proverbial keys to the kingdom, so they tend to be targeted by whaling and other adversary campaigns. They also tend to resist extensive training given their other responsibilities. That said, if you cannot get senior management to lead by example and receive extensive training, you have a low likelihood of success with the program overall.
- Finance: This team has almost the same risk profile as senior management. They access financial reporting systems and the flow of money. Stealing money is the objective of many campaigns, so these folks need a bit more love to prepare for the inevitable attacks.
- HR and Customer Service: Attackers target Human Resources and Customer Service frequently as well, mostly because they provide the easiest path into the organization; attackers then continue toward their ultimate goal. Interacting with the outside world makes up a significant part these groups’ job functions, so they need to be well-versed in email attacks and safe web browsing.
- Everyone else: We could define another dozen categories, but that would quickly pass the point of diminishing returns. The key for this group is to ensure that everyone has a baseline understanding of security, which they can apply when they see attacks.
Once you have defined your categories you design a curriculum for each group. There will be a base level of knowledge, for the everyone else group. Then you extend the more advanced curricula to address the most significant risks to each specific group, by building a quick threat model and focusing training to address it. For example senior management needs a deep understanding of whaling tactics they are likely to face.
Keep in mind that the frequency of formal training varies by group. If the program calls for intensive training during on-boarding and semi-annual refreshers, you’ll want more frequent training for HR and Customer Service. Given how quickly attack tactics change, updating training for those groups every quarter seems reasonable to keep them current.
Just as we finish saying you need to define the frequency for your different user groups, the first “C” is continuous. What gives? A security training program encompasses both formal training and ad-hoc lessons as needed. Attackers don’t seem to take days off, and the threat landscape changes almost daily. Your program needs to reflect the dynamic nature of security and implement triggers to initiate additional training.
You stay current by analyzing threat intelligence looking for significant new attacks that warrant additional training. Ransomware provides a timely example of this need. A few years ago when the first ransomware attack hit, most employees were not prepared to defend against the attack and they certainly didn’t know what to do once the ransomware locked their devices. For these new attack vectors, you may need to put together a quick video explaining the attack and what to do in the event the employee sees it. To be clear, speed matters here so don’t worry about your training video being perfect, just get something out there to prepare your employees for an imminent attack. Soon enough your security training vendor will update existing training and will introduce new material based on emerging attacks, so make sure you pay attention to available updates within the training platform.
Continuous training also involves evaluating not just potential attacks identified via threat intel but also changes in the risk profile of an employee. Keep on top of the employee’s risk profile, integrate with other security tools, including email security gateways, web security proxies and services, web/DNS security tools, DLP, and other content inspection technologies, security analytics including user behavior analytics (UBA), etc. These integrations set the stage for contextual training.
If any of the integrated security monitors or controls detects an attack on a specific user, or determines a user did something which violates policy, it provides an opportunity to deliver ad hoc training on that particular attack. The best time to train an employee and have the knowledge stick remains when they are conscious of its relevance. People have different learning styles, and their receptivity varies, but they should be much more receptive right after making a mistake. Then their fresh experience which puts the training in context.
Similar to teaching a child not to touch a hot stove after they’ve burnt their hand, showing an employee how to detect a phishing message is more impactful right after they clicked on a phishing message. We’ll dig in with a detailed example in our next post.
To wrap up our earlier frequency discussion, you have a few different options for training delivery:
- Scheduled: As described above, you provide materials during onboarding and as part of the ongoing training program. Periodic refreshers and updated training on new attacks are likely the bare minimum to meet your compliance requirements.
- Preemptive: In this model you deliver training when triggered by threat intel or a change in risk profile, as determined by security analytics/UBA. The emergence of a new ransomware variant is an example of a likely trigger for preemptive training.
- Reactive: This model triggers delivery of training when an employee makes a mistake. For example, train on how to protect customer data after the DLP system blocks an outgoing email with a customer’s social security number in the body.
Assuming risk reduction is the overall objective of your security awareness training program, you need a way to assess its effectiveness. How can you measure your security training program? It starts by defining a baseline of security effectiveness. We all understand that assessing security goes well beyond training, but you need to understand your current security posture before starting a new training program.
That means tracking attacks against the organization, particularly the types of attacks most impacted by security training – including phishing, drive-by downloads, customer data leakage, etc. Obtain this information via integration with your email and web security tools and your SIEM or UBA system. If you cannot establish a baseline before the program starts, we recommend you initiate data collection immediately. It’s decidedly suboptimal, but you can trend improvement over time from the start of your program.
As far as metrics to track, you can use these buckets to get started:
- Micro: Here you monitor employee-specific risk, such as how many times an employee clicks on a phishing simulation and how many times you’ve had to clean up the employee’s device after malware outbreaks.
- Macro: These indicators include benchmark data from organizations of similar size and sector. You’ll want to know how many successful attacks hit your peers. Your training vendor likely has benchmark data you can use, and we increasingly see this kind of information in training dashboards and reports to provide insight into effectiveness.
- Organizational: Based on micro and the macro data, how does your organization stack up? Here you’ll want to make an overall assessment of the organization, based on results from tests and other risk metrics/analytics.
- Qualitative: You’ll also want to understand what employees think of your training program. We recommend organizations perform 360° evaluations via employee surveys to gauge the effectiveness of training content, and for a sense of their general understanding of security.
For each of these metrics/assessments, you should be able to access the data quickly and easily via both a dashboard and results. The dashboard should clear reflect both the micro and macro effectiveness of your efforts. Which employees need additional training because they make the same mistake over and over again? Which employees can’t seem to find the time to complete scheduled training? Are the number of bad clicks during phishing simulations trending in the right direction?
The documentation from the program will substantiate (or not) your training efforts, which will make the difference between expanding the program or sending it to the dustbin. We’ll wrap up this series in our next post, working through a detailed example of setting up the program – and, more importantly, adapting it as you learn what works and doesn’t.
- Mike Rothman
Subscribe to our daily email digest