Content Examination - Fuzzy Hashes

This article contains information on configuring a Content Examination definition, including fuzzy hashing, scanning options, policy overrides, notification settings, and examples to help administrators manage and secure email content effectively.

The task of analyzing software applications' malicious behaviors is challenging due to the enormous number of code samples received into email environments, with a limited amount of analyst time and resources. This detection work usually starts with an analyst classifying samples into known and unknown malware.

Fuzzy hashing is a type of compression function for calculating the similarity between digital files. It attempts to automate the process of grouping similar malware. Fuzzy hash functions hold a certain tolerance for changes, and can tell how different two files are by comparing the similarity of their outputs.

This property is desirable for clustering malware campaigns, as they often use multiple variants from the same family that perform the exact same set of behaviors, but have different cryptographic hashes. The comparison function should provide some usable metric or distance by which we can decide whether the inputs are similar or not.

Configuring a Content Examination definition allows you to utilize this fuzzy hash function. You can:

• Upload a control document. This must be created before adding to the definition.
• Use fuzzy hashing to compare its content with the content inside attachments.
• Determine the level of similarity expressed as a percentage.

Fuzzy hashes can be used in conjunction with other search terms (e.g. regular expressions, words, or phrases). See the Phrase Match Examples page for further details.

Configuring a Fuzzy Hash Control Document

To configure a fuzzy hash control document to use with a Content Examination definition:

1. Log in to the Mimecast Administration Console.
2. Navigate to Golicies | Gateway Policies.
3. Hover over the Definitions button.
4. Select the Content Definitions definition type from the list.
5. Click on the Fuzzy Hash Definitions button.

6. Click the Generate Fuzzy Hash button.
7. Complete the Fuzzy Hash Generation section as follows:

Field / Option	Description
Description	Enter a description of the file. The description is visible to administrators when viewing the definition or selecting entries from the list of previously generated hash values.
Fuzzy Hash Type	Specify the type of fuzzy hash you would like to generate. The options are: Mimecast Fuzzy Hash (MFH): This ignores any images in an attachment, basing its similarity score on the attachment's text. With this option: The control file must be a minimum file size of 4 MB. All images should be removed from the control document to reduce the time taken to generate the fuzzy hash. SSDEEP: This uses the entire attachment (including text and images) to determine how similar one file is to another. Both: Both MFH and SSDEEP are used.
New File Upload	Click the Browse button to select the control document file. Only one file can be selected.

8. Click on the Generate button.

Adding a Fuzzy Hash to a Content Examination Definition

Once you've created a fuzzy hash definition, you can add it to a Content Examination definition. This enables you to define the criteria that must be met before your configured actions take effect.

1. Log in to the Mimecast Administration Console.
2. Navigate to Policies | Gateway Policies.
3. Hover over the Definitions button.
4. Select the Content Definitions definition type from the list.
5. Select a Folder in the hierarchy. Definitions cannot be placed in the "Root" folder.
6. Either click on the:

• • Definition to be changed.
  • New Content Definition button to create a definition.

7. To enter the fuzzy hash in the Word / Phrase Match List field:

• • Click on the Insert | Fuzzy Hash menu item.
  • Complete the Policy Definition dialog as follows:

Field / Option	Description
Fuzzy Hash Definition	Click on the Lookup button to display a list of all fuzzy hash files. Click on the Select link to the left of the fuzzy hash you wish to use.
Line Score	Specify a value to assign to the fuzzy hash. This is measured against the definition's activation score.
Append	This controls where a fuzzy hash is placed in the Word / Phrase Match List. If selected, the fuzzy hash is added to the bottom of the list. If disabled, the fuzzy hash is added to the top of the list.

• • Click the Save and Exit button. The fuzzy hash and line score are displayed in the Word / Phrase Match List.

8. Click on the Fuzzy Hash Setting field to specify a similarity percentage value. This is applied to all the fuzzy hashes defined in the Word / Phrase Match List.
9. Click on the Save and Exit button.

See Also...

• Email Security Cloud Gateway - Content Examination - Definitions
• Email Security Cloud Gateway - Content Examination Policy