Awareness Training - How Risk Scoring Works

This article describes in detail how Mimecast Awareness Training's risk scoring works, and is intended for Administrators.

Also see Reporting & Insights.

Introduction

One of the biggest problems facing security professionals is how to effectively follow a user's digital footprint and accurately assign risk.

Mimecast Awareness Training helps security professionals optimize real time user training data to assign timely, accurate, and relevant user and organizational risk scores, with the ratings updated daily.

Mimecast Awareness Training's risk scoring helps you:

• Expose potentially high risk users, and help predict who the riskiest user may be in the future.
• Continuously assess information security risks and assess where improvements can be made.
• Make better business decisions using objective, verifiable, and actionable data.
• Understand how your organization's risk compares to industry peers.

The following risk scores are available in the Mimecast Awareness Training platform:

• Organizational Risk Score: This is displayed on the Awareness Training Dashboard.
• Individual Risk Score: Each user's risk score is displayed via the Mimecast Awareness Training User Risk reporting, accessed via Reporting & Insights | Risk Center.

How Risk Scores are Calculated

The Mimecast Awareness Training platform uses the SAFE Score User Risk metric for measuring organizational and user risk. This is made up of various components that analyze millions of data points captured from users' interaction with the platform and actual behavior. It passes this data through an algorithm that:

• Analyzes the historical (covering 2.5 years) and current data for severity, frequency, duration, and confidence.
• Combines the scores with peer and industry comparisons to generate both an organizational and individual SAFE Score User Risk grade ranging from A to F, with A being the highest grade and F the lowest.
• Back end calculations for company and user level scores are run once per hour, every hour, seven days a week.

The SAFE Score User Risk metric is arrived at by looking at user:

• Sentiment: This comprises the results from the Mimecast Awareness Training survey sent to all users every six months which asks:
  • How the user feels about security.
  • How their sentiment has improved over time.
• Engagement: This is compiled by looking at:
  • Whether the user has outstanding modules.
  • How quickly the user viewed the Mimecast Awareness Training videos once they receive a notification.
  • Did the user follow the module through to completion, including the acknowledgment if enabled.
• Knowledge: This is compiled by looking at:
  • How is the user performing on the training modules, including how promptly the question was answered after the training was assigned.
  • Has the user answered the post module questions correctly.
• Human Error: This is compiled by looking at whether the user clicks on Mimecast Awareness Training simulated phishing email links and real phishing email TTP URL rewritten links.
  • If the user has a low grade due to clicking on phishing links, not clicking on phishing links will improve this component grade over time.

For Mimecast Awareness Training Email Security Cloud Gateway customers, the Human Error score includes phishing simulation data and real Targeted Threat Protection URL click data.
For Mimecast Awareness Training Email Security Cloud Integrated customers, the Human Error score is comprises phishing simulation data only.  

The weighting used to calculate the SAFE Score User Risk is dependent on the:

• Scores from the components listed below.
• Products configured on your account.

Configuration	Human Error	Sentiment	Engagement	Knowledge
Awareness Training and TTP URL Protect	55% (27.5% TTP URL links and 27.5% Awareness Training phishing links)	5%	20%	20%
Awareness Training only	55%	5%	20%	20%
TTP URL Protect only	100%	0%	0%	0%
Neither Awareness Training and TTP URL Protect	No score is displayed.

At the start of June 2021 we changed the weighting configurations as follows.

AT Only Customer Configuration	Human Error	Sentiment	Engagement	Knowledge
Pre June 2021	8.3%	25%	33.3%	33.3%
Post June 2021	55%	5%	20%	20%

AT & TTP URL Customer Configuration	Human Error	Sentiment	Engagement	Knowledge
Pre June 2021	55% (40% TTP URL links and 15% Awareness Training phishing links)	5%	20%	20%
Post June 2021	55% (27.5% TTP URL links and 27.5% Awareness Training phishing links)	No Change	No Change	No Change

The SAFE Score Grade is arrived at by the user's overall percentage score:

SAFE Score Grade	Lower Bound & (Inclusive)	Upper Bound % (Inclusive)
A	90	100
B	80	89
C	70	79
D	60	69
F	0	59

User Risk Scores

Mimecast Awareness Training's user risk score is similar to consumer credit scores, with a low rating indicating a potentially high risk user and a higher rating indicating a potentially less risky user. Specifically, it analyzes various elements from three dimensions derived from user training: sentiment, engagement, and knowledge to assess a user's security risk profile over time. Risk scores take into account historical performance as well as accounting for recent trends.

Organization Risk Score

Mimecast Awareness Training takes the underlying raw data in the user risk score and runs it through a proprietary algorithm. This analyzes the data across industry baselines and creates an overall, contextual, organization risk score.

On new accounts with no Mimecast Awareness Training activity, a default SAFE Score user risk score of D (65%) is applied.

How Risk Scores are Used

The risk scores can be leveraged for multiple use cases, including benchmarking, user risk management, and cyber insurance.

Benchmarking

Here are a few ways organizations can use Mimecast Awareness Training's Risk Score for benchmarking:

• To see how the organization is generally performing, and how effective the current security programs are.
• To easily see if your organization's risk score is more or less advanced than industry peers. With this information, organizations can make better decisions on how to efficiently allocate resources for their security program.
• Organization executives and leadership teams are increasingly concerned with cybersecurity performance. Mimecast Awareness Training's risk scores are an effective, accepted way to communicate security performance with an organization's board.

User Risk Mitigation

Traditional tools fail to provide useful information about an organization's largest security risk; human error. Mimecast Awareness Training's risk scores provide a solution, helping security administrators:

• Quickly identify and prioritize high risk users.
• Rapidly understand the areas of the greatest vulnerability within the human element.
• Integrate risk scores with existing SIEMs and UEBA analytical platforms and solutions, to gain visibility into human risk across the organization.

Cyber Insurance

Mimecast Awareness Training's risk scores can be used by underwriters to prepare models for their business plans. Significant data breaches have impacted the cyber/data breach insurance market. The combination of these breaches, insurance regulatory scrutiny, and the underwriters' desire to maintain profitability have led to fewer insurers providing cyber/data breach cover.

To better understand this exposure, underwriters are now aggressively seeking supplemental data. Mimecast Awareness Training meets this need by offering underwriters unique insight into an organization's most exposed risk source; its users.