API & Integrations - Microsoft Sentinel Integration

This article contains information about an outdated version of the Mimecast for Microsoft Sentinel integration. It uses version 1.0 of Mimecast's API and is not recommended for new installations. For instructions on installing the latest version of Mimecast for Microsoft Sentinel, which utilizes Mimecast's 2.0 API, please refer to the following article: API Integrations: Microsoft Sentinel Integration.

We strongly recommend using this version for any new installations.

You will be able to view information on configuring Mimecast Audit Logs for Microsoft Sentinel, covering prerequisites, deployment steps, common issues, and solutions. It is intended for Administrators.

The same steps can also be followed for:

• Mimecast Cloud Gateway MTA 
• Mimecast Cloud Gateway Targeted Threat Protection
• Mimecast Cloud Gateway Threat Intel Regional Feed

Steps relating to Microsoft Azure Portal may differ, as the workflow may change over time.

Prerequisites

• You have an Administrator role within the Mimecast Administration Console.
• You have login access to the Microsoft Azure Portal.
• You have a license for Microsoft Sentinel.
• You have set up the required Mimecast API 1.0 integrations. See Managing API 1.0 for Cloud Gateway.
• Enhanced Logging checkboxes are selected in your Account Settings.
• The Threat Intelligence package is enabled on your Account (Customer Support will need to check and enable this).

Deployment

Create an App Registration and a Client Secret in Azure

Only one App Registration needs to be created, and it can be used for each of the Data Connectors.

1. Log in to the Azure portal (https://portal.azure.com/#home) using the MS Azure credentials.
2. In Azure, search for App Registrations and click on it:

3. Click on New Registration in the top-left:
   
   

4. Give the Registration an appropriate name, and click on Register. Everything else can be left as default.

 

5. Click on the New Registration. Under the Overview menu, record the Application (Client) ID:
   
   

6. Navigate to Certificates & Secrets, and select New Client Secret:

7. Enter a user-friendly description, choose an appropriate expiry period, and click on Add. Immediately copy and record the Value field (not the ID) of the new secret, as this will be obfuscated on future visits to the page - if it is lost, it will need to be replaced.
8.  In the API permissions section grant Microsoft Graph ‘ThreatIndicators.ReadWrite.OwnedBy’ permissions (Type Application) and then select ‘Grant admin consent for Default Directory’.

Locate and Record the Object ID of the User Creating the Application

9. Search for the Microsoft Entra ID in the Azure portal search bar and select Microsoft Entra ID.

10. In the Users menu, locate your user. This is the user that will be performing this integration.

 

11. Click on your display name, and locate the Object ID in the Overview tab. Copy and record this value.
    
    

12. You should now have:

• • Mimecast App ID: The Application ID of the 1.0 application from Mimecast.
  • Mimecast App Key: The Application Key of the 1.0 application from Mimecast.
  • Mimecast Access Key: The Access Key of the 1.0 application from Mimecast.
  • Mimecast Secret Key: The Secret Key of the 1.0 application from Mimecast.
  • App Registration Client ID: The Application (Client) ID of the App Registration.
  • App Registration Client Secret: The Client Secret of the App Registration.
  • User Object ID: The Object ID of the user creating the Sentinel integration.
  • API Base URL: The Mimecast API Base URL appropriate to your region api-overview/global-base-urls/ (E.g. https://us-api.mimecast.com) for US-A grid customers.
  • Mimecast API Service Account Email Address: The email address of the service account that you used to generate API keys in Mimecast.
  • Mimecast API Service Account Password: The password of that email address.

Deploying the Data Connector

1. Search for the Microsoft Sentinel service in the Azure portal search bar and select Microsoft Sentinel service.

2. Choose an existing workspace, or create one if needed, and click into the workspace.
3. In the left panel, scroll down and select Content Hub on the chosen workspace.

4. Search for Mimecast, locate, and select the connector you want to add. Check the box next to it, and click on Install in the top right menu.

5. Wait for the installation to complete, select the option again and click on Manage.

6. Go to the Content-Type: Data Connector and click on it.

7. Select the connector and click on Open Connector Page.

8. On the next page, select Deploy to Azure.

9. It will redirect to the deployment page. Provide input of details per the instructions Microsoft outlined in the previous step of where to find each if needed. (keep note of the provided App Name).

• • Subscription: Choose the Azure subscription that will be used for the application.
  • Resource Group: Choose the Resource Group tied to the Sentinel workspace that the setup was initiated from.
  • Region: Choose the region appropriate to you. For E.g. customers on the US Eastern Seaboard should choose the US East.
  • App Name: This must be a globally unique name for each connector E.g.  'mimecastaudit', and can only contain letters and numbers. 
  • Object ID: The Object ID of the user creating the app.
  • Storage Account Type: Leave as default.
  • Location: Leave as default.
  • App Insights Location: Set this to the same location chosen in the Region above. E.g. for US East, this would be "eastus" and UK South would be "uksouth". More regions.<https://azuretracks.com/2021/04/current-azure-region-names-reference/>.
  • Mimecast Email: The email address of the service account used to generate API keys in Mimecast.
  • Mimecast Password: The password of the service account. This field does not hide data, and will be visible to anyone who can see your screen.
  • Mimecast App ID: The Application ID from the Mimecast API integration.
  • Mimecast App Key: The app key. This field does not hide data, and will be visible to anyone who can see your screen.
  • Mimecast Access Key: The access key. This field does not hide data and will be visible to anyone who can see your screen.
  • Mimecast Secret Key: The secret key. This field does not hide data and will be visible to anyone who can see your screen.
  • Mimecast Base URL: The API URL appropriate to your region, e.g. https://us-api.mimecast.com for US-A grid customers.
  • Active Directory App Secret: The Client Secret Value (not Secret ID) from the App Registration (This field does not hide data and will be visible to anyone who can see your screen).
  • Workspace ID/Workspace Key : This can be viewed in the previous browser tab or located in Log Analytics Workspaces under Agents | Log Analytics agent instructions.
  • App Insights Workspace Resource ID: This can be located in Log Analytics Workspaces under Properties.

Important

See the below table that specifies different checkpoint file requirements for other connectors.

10. Search for Storage Accounts and select Storage Account to create a blank checkpoint file for the integration.

11. Select the Storage Account named the same as the app name.

12. Select the Container from the left panel, select the Blob Containers, and locate the relevant checkpoints folder (folder names for the different connector types are described in the table at the bottom of this section):

Important

See the below table that specifies different checkpoint file requirements for other connectors.

13. Create an empty checkpoint.txt file locally and upload it to storage, as shown below. Important please see below that specifies different checkpoint file requirements for other connectors.

 

You should then see the file appear in the folder with size 0 Bytes. This file is used to store the checkpoint token to keep track of log progress, so it generally shouldn't be touched (otherwise duplicate logs will appear in the SIEM). In the same stroke, deleting this file can potentially be a useful troubleshooting step if the integration stopped pulling logs at some point in the past. The folder name under Blob containers in the Storage Browser of this section differs based on connector type. The filename should always be “checkpoint.txt”. Please reference this table if you’re unsure which folder to place the file in:

Connector Checkpoint File Requirements
SIEM Logs	SIEM-checkpoints folder Files Needed: checkpoint.txt dlp-checkpoint.txt
Audit Logs	Audit-checkpoints folder Files Needed: checkpoint.txt
TTP Logs	TTP-checkpoints folder Files Needed: attachment-checkpoint.txt impersonation-checkpoint.txt url-checkpoint.txt
Threat Intel	TIR-checkpoint folder Files Needed: checkpoint.txt

Once all of the above steps have been completed, the integration should be up and running. The remaining tasks are to add the Workbook and Analytics Rule, both of which are technically optional but still recommended.

For instructions on how to integrate any threat intelligence third party vendor such as the Microsoft Threat Intelligence Platform Connector, please refer to the external Microsoft Azure Sentinel article below and navigate to the Enable the Threat Intelligence Platforms data connector in Microsoft Sentinel

https://learn.microsoft.com/en-us/azure/sentinel/connect-threat-intelligence-tip

 

Deploying and Adding the Workbook

1. Open Azure Sentinel and click into the New Workspace that was created per the above steps.
2. In the workspace in the left panel select Content Hub.

3. Search for the Mimecast, locate, and select the connector you want to add. Check the box next to it, and click on Install in the menu that pops up on the right:

4. Once the installation is complete, select the option again, and click on Manage:

5. In the Manage screen, click on the option with Content-type: Workbook, then select Save in the bottom right, choose your region, and click on Yes:

The workbook will then appear in the Sentinel workspace.

​​​​​​​Deploying and Adding the Analytics Rule

1. Enter the Azure Sentinel workspace, and locate the installed Mimecast application in the content hub.
2. Click on the option with Content-type: Analytics rule, select the rule in the next screen, and click on Create Rule:

3. Click on Incident Settings and set Limit the group to alerts created within the selected time frame to 1 day.

​​Common Issues

Additional items created having set up the Microsoft Sentinel Integration

In the App Name field, nothing is accepted, it just gives me a blank validation failure no matter what I put in the field, with no details about what the problem could be.

This is usually the result of missing Resource Providers. The following list of Resource Providers must be registered with the chosen subscription:

Microsoft.Web

Microsoft.Storage

Microsoft.KeyVault

Microsoft.Insights

Microsoft.OperationalInsights

Microsoft.OperationsManagement

How to Register Resource Providers

1. In Azure, search for Subscriptions.

2. Click on Subscriptions and locate the subscription being used to create the integration.

3. Click the subscription and find Resource Providers in the left-hand panel under Settings:

4. Search for each of the providers listed above, and ensure the Status is Registered, with a green checkmark. If not, select the provider, and click on Register at the top:

​​​​​​​

Error seen when trying to create a new Analytic Rule

When trying to install the Analytics Rule, Set rule logic step, you may see an error similar to:

'where' operator: Failed to resolve table or column expression named 'MimecastAudit_CL'

This step requires a query to run against logs in the workspace, and the log table won't be initialized until logs have begun to ingest. Usually, this will just be a matter of waiting a few minutes until some logs are available. 

Additionally, if logs are ingesting but a particular log type has yet to appear - for example, an AV rejection - you won’t be able to add the analytics rule, as the search must be validated first.

Lastly, if it’s been roughly an hour and you’re still getting a “Failed to resolve table” error, take a look at the next point here - the connector might not be working.

When setting up the MTA connector, the Function App doesn’t seem to do anything. When watching the log stream for several minutes, it shows no indication that any code is being executed, just occasional “Draining…” messages or other unrelated things. What gives?

When the function app is rolled out, code is pulled in from a location specified in the setup template. By default, this is set to:

https://github.com/mimecast/Azure-Sentinel/blob/mimecast_seg_connector/Solutions/MimecastSEG/Data%20Connectors/MimecastSEGSentinelConn.zip.

However, if you carefully inspect the template for the Audit connector, you’ll note that it points to an aka.ms URL instead:

https://aka.ms/sentinel-MimecastAudit-functionapp.

If you follow that first GitHub link, you’ll note that it doesn’t point directly to a .zip file download, but rather a webpage that itself contains a link to download the .zip. As such, the code doesn’t get inserted into the function app - presumably either nothing happens or a .html page gets rolled in instead. To correct this, we need to change the MTA template to pull from the appropriate source - for MTA, that is:

https://aka.ms/sentinel-MimecastSEG-functionapp.

Opening this link in a browser will download the code that the function app runs if you’re interested in inspecting it for yourself.

The steps to fix this are as follows:

1. First, if you created a connector already without modifying the template and it’s just sitting around twiddling its thumbs, go ahead and delete any related resources (meaning the function app, Sentinel workspace, and storage account).
2. Run through the connector setup as normal, following the “Setup Guide” section of this article, until you reach steps 5-9 (inputting the info into the data connector). On the page where all the data fields are presented, select “Edit Template” at the top:

4. This will bring you into an editor where you can manually modify the .json for the template. Scroll down and locate line 209, “WEBSITE_RUN_FROM_PACKAGE”:

5. Modify the URL in this field to reflect the one mentioned above instead - that being: https://aka.ms/sentinel-MimecastSEG-functionapp

How to Check the Status of the Connector

A quick way to check the connector's health is by taking a look at the Log Stream for the Function App.

To locate this:

1. In Azure, search for and click on Function App:

6. Click into the Function App with the same name that you entered in the App Name field when setting up the connector. In the left menu, scroll down and locate Log Stream, and wait for the log stream to initialize. You'll see real-time logs for the connector, including any errors, as well as Mimecast request IDs that can be provided to Support when troubleshooting related issues. Please leave this screen open for at least 30 minutes to ensure that a collection run is captured. Afterward, click on Copy in the toolbar at the top of this section and paste the copied logs into a text file to provide to Support.

Do I need Separate App Registrations for each Log Connector Type (Audit, SIEM, TTP, Threat Intel)?

All connectors can use the same App Registration, and there’s no reason to make separate registrations for each.

See Also...

• https://learn.microsoft.com/en-us/azure/azure-functions
• https://learn.microsoft.com/en-us/azure/sentinel/detect-threats-custom
• https://learn.microsoft.com/en-us/azure/sentinel/monitor-your-data​​​​​​​