Configure threat intelligence sources in Splunk Enterprise Security

Configure threat intelligence sources to get intelligence data in Splunk Enterprise Security. The configuration process for intelligence sources is different depending on the threat intelligence system you have access to.

To configure sources for the threat intelligence management system, see Configure sources for threat intelligence management.

To configure sources for the threat intelligence management (cloud) system, see Configure sources for threat intelligence management (cloud).

Configure sources for threat intelligence management

Splunk Enterprise Security includes several intelligence sources that retrieve information across the internet. You must turn on intelligence source integrations to begin ingesting the intelligence data and using it in your security investigations.

The following threat intelligence sources are turned on by default:

• Mozilla Public Suffix List
• MITRE ATT&CK Framework
• ICANN Top-level Domains List

Turn on intelligence sources

To turn on intelligence source integrations for threat intelligence management, complete the following steps:

Prerequisites

• Your Splunk Enterprise deployment must be connected to the internet. If your deployment is not connected to the internet, turn off these sources or source them in an alternate way.
• To set up firewall rules for these sources, you might want to use a proxy server to collect the intelligence before forwarding it to Splunk Enterprise Security and allow the IP address for the proxy server to access Splunk Enterprise Security. The IP addresses for these sources can change. See Configure proxy server settings.

Steps

1. In Splunk Enterprise Security, select Configure and then Intelligence.
2. In the Threat intelligence management section, select Threat intelligence sources.
3. Toggle the Status switch to On.
4. Review the Description field for all defined intelligence sources to learn more about the types of information or threat indicators that can be correlated with your events.
5. Configure the intelligence sources that are turned on and fit your security use cases using the URLs to the source websites to review the source provider's documentation. Each source website provides suggestions for polling intervals and other configuration requirements separate from Splunk Enterprise Security.

Splunk Enterprise Security expects all intelligence sources to provide properly-formatted data and valuable intelligence information. Feed providers are responsible for malformed data or false positives that might be identified in your environment as a result.

To see a reference table of the available sources, see Available threat intelligence and generic intelligence sources included in Splunk Enterprise Security.

If you determine that your Splunk Enterprise Security installation is retrieving data from unexpected IP addresses, perform a WHOIS or nslookup to determine if the IP address matches that of one of the intelligence sources configured in your environment.

Add a custom URL-based intelligence source

Splunk Enterprise Security can periodically download an intelligence feed available from the internet and store it in the $SPLUNK_DB/modinput/threatlist directory. You can then use the inputintelligence search command to use the intelligence in reports, searches, or dashboards.

Steps

1. (Optional) Configure a proxy for retrieving intelligence. See Configure proxy server settings in Splunk Enterprise Security.
2. In Splunk Enterprise Security, select Configure and then Intelligence.
3. In the Threat intelligence management section, select Threat intelligence sources.
4. Select New to add a new intelligence source.
5. Type a Name for the download. The name can only contain alphanumeric characters, hyphens, and underscores. The name cannot contain spaces.
6. Do not select the check box for Sinkhole.
7. Deselect the check box for Is Threat Intelligence.
8. Type a Type for the download. The type identifies the type of information that the feed contains.
9. Type a Description. Describe the information in the feed.
10. Leave the default Weight because the field does not matter for the generic intelligence source.
11. (Optional) Change the default download Interval for the feed. Defaults to 43200 seconds, or every 12 hours.
12. (Optional) Type POST arguments for the feed. You can use POST arguments to retrieve user credentials from Credential Management. Use the format key=$user:<username>$ or key=$user:<username>,realm:<realm>$ to specify a username and realm.
13. Do not use the Maximum age setting.
14. (Optional) If you need to specify a custom User agent string to bypass network security controls in your environment, type it in the format <user-agent>/<version>. For example, Mozilla/5.0 or AppleWebKit/602.3.12. The value in this field must match this regex: ([A-Za-z0-9_.-]+)/([A-Za-z0-9_.-]+). Check with your security device administrator to ensure the string you type here is accepted by your network security controls.
15. Fill out the Parsing options fields to make sure that your list parses successfully. You must fill out either a delimiting regular expression or an extracting regular expression. You cannot leave both fields blank.

    Field	Description	Example
    Delimiting regular expression	A regular expression string used to split, or delimit, lines in an intelligence source. For complex delimiters, use an extracting regular expression. For parsing options, you can either use a delimiting regular expression or an extracting regular expression, but not both.	, or : or \t
    Extracting regular expression	A regular expression used to extract fields from individual lines of an intelligence source document. Use to extract values in the intelligence source. For parsing options, you can either use a delimiting regular expression or an extracting regular expression, but not both.	^(\S+)\t+(\S+)\t+\S+\t+\S+\t*(\S*)
    Fields	Required if your document is line-delimited. Comma-separated list of fields to be extracted from the intelligence list. Can also be used to rename or combine fields. Description is a required field. Additional acceptable fields are the fields in the corresponding KV Store collection for the threat intelligence, visible in the local lookup files or the DA-ESS-ThreatIntelligence/collections.conf file. Defaults to description:$1,ip:$2.	<fieldname>:$<number>,<field name>.$<number> ip:$1,description:domain_blocklist
    Ignoring regular expression	A regular expression used to ignore lines in an intelligence source. Defaults to ignoring blank lines and comments that begin with #.	^\s*$)
    Skip header lines	The number of header lines to skip when processing the intelligence source.	0
    Intelligence file encoding	If the file encoding is something other than ASCII or UTF8, specify the encoding here. Leave blank otherwise.	latin1

16. (Optional) Change the Download Options fields to make sure that your list downloads successfully.

    Field	Description	Example
    Retry interval	Number of seconds to wait between download retry attempts. Review the recommended poll interval of the intelligence source provider before changing the retry interval.	60
    Remote site user	If the threat feed requires authentication, type the user name to use in remote authentication, if required. The user name you add in this field must match the name of a credential on the Credentials page.	buttercup
    Remote site user realm	If the threat feed requires authentication, type the user name to use in remote authentication, if required. The realm you add in this field must match the realm of a credential on the Credentials page.	paddock
    Retries	The maximum number of retry attempts.	3
    Timeout	Number of seconds to wait before marking a download attempt as failed.	30

17. (Optional) If you are using a proxy server, fill out the Proxy options for the feed. See Modify proxy and parser settings in Splunk Enterprise Security.
18. Save your changes.

Verify your threat intelligence sources

After you add new intelligence sources or configure included intelligence sources using the threat intelligence management system, verify that the intelligence is being parsed successfully and that threat indicators are being added to the threat intelligence KV Store collections. The modular input responsible for parsing intelligence runs every 12 hours. This verification procedure is relevant only for URL-based sources and TAXII feeds.

Follow these steps to verify that intelligence source data is being parsed successfully:

1. In Splunk Enterprise Security, select Security analytics then Audit and then Threat intelligence audit.
2. Find the intelligence source and confirm that the download_status column states threat list downloaded.
   For TAXII feeds, the UI states Retrieved document from TAXII feed.
3. Review the Intelligence audit events to see if there are errors associated with the lookup name.

If the download fails, attempt the download directly from the terminal of the Splunk server using a curl or wget utility. If the intelligence source can be successfully downloaded using one of these utilities, but is not being downloaded successfully in Splunk Enterprise Security, ask your system administrator whether you need to specify a custom user-agent string to bypass network security controls in your environment.

Follow these steps to verify that threat indicators are being added to KV store collections:

1. Select Security analytics then Security intelligence then Threat Intelligence and then Threat artifacts.
2. Search for the threat source name in the Intel Source ID field.
3. Confirm that threat indicators exist for the threat source.

Configure parse modifier settings

When threat intelligence data is ingested, fields are often embedded within each other. By configuring threat list settings you can separate the fields. Extraction of field and their corresponding values is based on when threat documents are processed and written to their respective threat collections. Configure parse modifier settings to extract fields from the threat intelligence data.

Steps

1. In Splunk Enterprise Security, select Configure and then Intelligence.
2. In the Threat intelligence management section, select Proxy and parser settings.
3. You have the option to turn on any of the following parse modifier settings:
   
   • Certificate attribute breakout
   • IDNA encode domains
   • Parse domain from URL
4. Turn on the parse modifier setting based on your requirements. Turn on Certificate attribute breakout to parse fields in the certificate_issuer and the certificate_subject fields.
   For example: A raw certificate issuer field might be a single string as follows:
   C = US, ST = CA, L = San Francisco, O = The Company Name, OU = The Organizational Unit Name, CN = The common name, emailAddress = theemailaddress@email.gov, STREET=123 main street
   Multiple other potential fields may exist within this single string. When you parse fields in the certificate_issuer fields by activating the Certificate attribute breakout parse modifier, all extra fields are parsed from the raw certificate_issuer field and stored into their own fields in the collection as follows:
   • 'certificate_issuer_common_name': 'The common name',
   • 'certificate_issuer_email': 'theemailaddress@email.gov',
   • 'certificate_issuer_locality': 'San Francisco',
   • 'certificate_issuer_organization': 'The Company Name',
   • 'certificate_issuer_state': 'CA',
   • 'certificate_issuer_street': '123 main street',
   • 'certificate_issuer_unit': 'The Organizational Unit Name'
   When you parse fields in the the certificate_subject field fields by activating the Certificate attribute breakout parse modifier, parsing occurs as follows:
   • 'certificate_subject_common_name': 'The common name',
   • 'certificate_subject_email': 'theemailaddress@email.gov',
   • 'certificate_subject_locality': 'San Francisco',
   • 'certificate_subject_organization': 'The Company Name',
   • 'certificate_subject_state': 'CA',
   • 'certificate_subject_street': '123 main street',
   • 'certificate_subject_unit': 'The Organizational Unit Name'
   If you want to transform the names written in non-ASCII characters to their ASCII-based representation, turn on IDNA encode domains. Turn on IDNA encode domains to include both the IDNA and the international encoding for applicable domains in the domain field.
   If you want to extract a hostname from a URL, turn on Parse domain from URL. Turn on the Parse domain from URL to parse the domain field from the url field.

Configure cloud-hosted threat intelligence data source integrations

Activating the data source integrations imports data into the threat intelligence management (cloud) system. After you import data into the cloud system, you can use it in threat-matching searches and also to enrich investigations.

Follow these steps to activate the cloud-hosted threat intelligence data sources:

1. In Splunk Enterprise Security, select the Configure page and then Threat intelligence.
2. Select Threat intelligence sources.
3. (Optional) To apply a filter, such as Type or Status, to the sources table, select the column header of the field you want to filter by. Not all fields are filterable. You can see sorting and filtering options for a field by selecting the down arrow icon ( ) in the column header. Fields that aren't filterable don't have a filter menu with check boxes.

   Many of the intelligence sources are available immediately upon activation, but certain paid and proprietary intelligence sources are only available after validation of API keys and credentials.

4. For an open intelligence source, select Activate. See Available open intelligence sources for Splunk Enterprise Security for a listing of all free OSINT sources available through the cloud threat intelligence system and the types of indicators they report about.
5. For a premium intelligence source, select Activate.
   1. Enter the required credentials. To find the requirements for each available premium intelligence source, see Available premium intelligence sources for Splunk Enterprise Security.
   2. Select Yes, confirm to confirm your credentials.
6. Repeat the process for all the threat intelligence sources that you want to activate.
7. (Optional) To deactivate a source, select the source you want, and then select Deactivate.

If an intelligence source indicates "Activation failed", check for expired API credentials or for an overdue subscription payment. You might need to deactivate the source, enter new credentials, and then activate the source again.

After you activate sources, you can configure threat lists for threat-matching automation and investigation enrichment. See Configure threat lists in Splunk Enterprise Security.