Security Operations email parsing

Generate new Security Operations records from external detection systems using Email Parsing. This feature provides a method for integrating information from external tools such as malware detection, vulnerability detection, firewalls, threat intelligence, and more.

How emails are parsed

Any system that can send an email, can create Security Operations records, for example, security incidents, requests, vulnerable items, vulnerabilities, security incident observables, attack methods, and more.

All Security Operations plugins (Security Incident Response, Threat Intelligence, and Vulnerability Response) have a property (email_to ) that defines the email address where external integrations should send emails so they can be processed by the email parsers. See Email Processing > Properties for more information.

Email sent to any of the Security Operations email addresses is stored in an email events table. These emails are processed to determine whether they match any email parser.

Emails that have a match are flagged and the transform and duplication rules create or update a Security Operations record. The email is linked to that record and flagged as matched.

Emails that do not match are listed in Unmatched Emails as a Security Operations record. These emails can be reviewed to help build email parsers to handle them. A Reprocess action allows you to run the unmatched email through the parsers again, if necessary. The original email log is linked to each record.

The duplication rules for the email transform manage multiple emails relating to the same issue. These rules define what makes a duplicate record and can prevent duplicate emails from being created. When a duplicate is detected, the rule specifies what action to take:

  • no action (do not create a new record)
  • create the record as a child record of the existing record
  • update the existing record

When updating, the duplicate rule specifies which fields in the existing record are updated. By default, email events are deleted after 30 days.

Multiple records

External detection systems (malware detectors, vulnerability, and so on) can send emails that report on multiple items at one time. The email parser supports separators within the email.

For example, a malware detector could send you an email report about all the systems within your network infected by a malware.

Malicious email example
In this example, when the Record Separator is set within the Email Transform as =================, the separator splits the email into four sections that are evaluated individually. The separator setting causes the transform to then create a Security Incident for each of the three affected systems.
Note: The header section is detected but does not have any affected systems, so it is used in all three records and does not create a fourth.

Field Transforms

Field Transforms pull in data from each section. For example, if something in the header or footer of an email applies to all records -- such as the Malware Hash, Malware Name, and Type -- the field transform for them should set Search for value to one that examines an email, either At the start of a line in the email body or Anywhere in the email body.

Field Transforms must be set to search At the start of a line within the record section or Sec for data that is defined within each section, such as System, IP address, or Status. The record section options are only available for defined email transforms that include a record separator.

When parsing an email with a defined separator, records are only created for sections with at least one piece of section-specific data.

In the above example, three records are created, even though there are four sections defined in the transform. The first section is a header, and it lacks anything specific to a single system. If any of the fields within the first section are completed (System, IP, or Status), then a record is created for that section.