Splunk® Enterprise Security

Use Splunk Enterprise Security

Acrobat logo Download manual as PDF


This documentation does not apply to the most recent version of ES. Click here for the latest version.
Acrobat logo Download topic as PDF

Configure threat intelligence sources

Splunk Enterprise Security provides threat intelligence and context for security monitoring and investigation by comparing the contents of multiple threat sources to indexed events. Enterprise Security supports multiple threat intelligence collection sources, and includes a selection of threat intelligence sources.

Each source represents a collection of knowledge about a specific type of security threat or activity. Splunk administrators can add new sources to ES if those sources are available as a structured data file or stream.

Supported threat collection methods

Any content that is available as a structured text file can be added to ES as a threat source. This includes support for:

  • STIX documents and TAXII feeds
  • OpenIOC documents
  • Content collected via API or URL
  • File-based content
  • Database-hosted content.

Supported threat intelligence groups

Supported IOC data types Threat Collection Details
X509 Certificates certificate_intel Threat collections are defined in collections.conf, and stored in KVStore. To provide for manually entered threat data, each threat collection also has an associated lookup file. To manually add threat intelligence data open Configure > Data Enrichment and select Lists and Lookups on the Enterprise Security menu bar.
Email email_intel
Files names/hashes file_intel
HTTP http_intel
IP/Domains ip_intel
Processes process_intel
Registry entries registry_intel
Services service_intel
Users user_intel

Review the provided threat sources

Splunk Enterprise Security includes pre-configured threat intelligence sources. Each threat source site offers suggestions for polling intervals and other configuration requirements independent of Enterprise Security. Use the source site links to review the provider's documentation for a threat source.

  1. On the Enterprise Security menu bar, open Configure > Data Enrichment > Threat Intelligence Downloads and review the Description field in all defined threat intelligence sources.
  2. Use the Threat Intelligence Download Settings page to enable and configure threatlist settings.
  3. Use the Threat Intelligence Audit dashboard to review the status of all threat download sources.
  4. Use the Threat Artifacts dashboard to find or query on individual threat elements.

Threat sources included with ES

Threat Source Threat Provider Source site
Alexa Top 1 Million Sites Alexa Internet http://s3.amazonaws.com/alexa-static/
Emerging Threats compromised IPs blocklist Emerging Threats http://rules.emergingthreats.net/blockrules
Emerging Threats fwip rules Emerging Threats http://rules.emergingthreats.net/fwrules
Malware domain host list Hail a TAXII.com http://hailataxii.com
iblocklist Logmein I-Blocklist http://list.iblocklist.com
iblocklist Piratebay I-Blocklist http://list.iblocklist.com
iblocklist Proxy I-Blocklist http://list.iblocklist.com
iblocklist Rapidshare I-Blocklist http://list.iblocklist.com
iblocklist Spyware I-Blocklist http://list.iblocklist.com
iblocklist Tor I-Blocklist http://list.iblocklist.com
iblocklist Web attacker I-Blocklist http://list.iblocklist.com
ICANN Top-level Domains List IANA http://data.iana.org
Malware Domain Blocklist Malware Domains http://mirror1.malwaredomains.com
MaxMind GeoIP ASN IPv4 Database MaxMind http://download.maxmind.com
MaxMind GeoIP ASN IPv6 Database MaxMind http://download.maxmind.com
Mozilla Public Suffix List Mozilla https://publicsuffix.org
abuse.ch Palevo C&C IP Blocklist abuse.ch https://palevotracker.abuse.ch
Phishtank Database Phishtank http://data.phishtank.com
SANS blocklist SANS http://isc.sans.edu
abuse.ch ZeuS blocklist (bad IPs only) abuse.ch https://zeustracker.abuse.ch
abuse.ch ZeuS blocklist (standard) abuse.ch https://zeustracker.abuse.ch

Threat Intelligence Download Settings

As an analyst, use the Threat Intelligence Download page to implement, configure, and enable threat intelligence sources that require an API or URL connection, or are lookup based. On the Enterprise Security menu bar, open Configure > Data Enrichment and select Threat Intelligence Downloads.

ES33 Threat Intel Download.png

Select the threat source name field to view the settings.

Threat Intelligence Download Setting
  • Type: The name or type of threat source. (Required)
  • Description: Details about the source. (Required)
  • URL: A link to the source download. Use the base URL defined by the feed provider. (Required)
  • Weight: The threat source weight multiplier used when making Risk Analysis calculations. (Required) A higher number implies an increased relevance. See Assigning a risk score through search for an example.
  • Interval; The download interval.
  • POST Arguments: Optional arguments that can be passed to the URL for threat source downloading.
  • Maximum age: The retention period for this threat source. If the last change time exceeds the maximum age, the threat source data will be purged from the collection. To enable this feature, see "Configure threat source retention" in this topic.
Parsing Options
  • Delimiting regular expression: A delimiter used to split lines in a threat source document.
  • Extracting regular expression: A regular expression used to extract fields from individual lines of a threat source document.
  • Fields: A transforms.conf-style expression used to rename or combine fields. (Required)
  • Ignoring regular expression: A regular expression used to IGNORE lines in a threat source.
  • Skip header lines: The number of header lines to skip when processing the threat source. Set to "1" for lookup tables.
Download Options
  • Retry interval: Number of seconds to wait between download retry attempts. Review the recommended poll interval of the threat source provider before changing the retry interval. (Required)
  • Remote site user: A username to use in remote authentication, if required. The user must correspond to the name of a Splunk secure stored credential in Credential Management
  • Retries: The maximum number of retry attempts.
  • Timeout: Number of seconds to wait before marking a download attempt as failed. (Required)
Proxy Options: (Optional)
  • Proxy Server: The proxy server address.
  • Proxy Port: The proxy server port.
  • Proxy User: The proxy server user credential. Supported authentication methods are Basic and Digest. The user must correspond to the name of a Splunk secure stored credential in Credential Management
Important: If you remove an existing proxy user and password in the Threat Intelligence Download Setting editor, the download process will no longer reference the stored credentials. Removing the referenced to credential does not delete the stored credentials from Credential Management.

Edit a threat source

Use the Threat Intelligence Download Settings editor to modify information about an existing threat source.

To edit a threat source:

  1. Go to Configure > Data Enrichment > Threat Intelligence Downloads.
  2. Select the name of the threat source you want to edit, which opens the Threat Intelligence Download Settings editor.
  3. Make changes to the fields as required.
  4. Save changes.

Note: To allow non-admin users to edit threat sources, see "Adding capabilities to a role" in the Installation and Upgrade manual.

Configure threat source retention

To remove threat source data from a threat collection based on date, the threat source must have a Maximum age setting and a retention tracking search enabled.

  1. On the Enterprise Security menu bar, open Configure > Data Enrichment and select Threat Intelligence Downloads.
  2. Select a threat source.
  3. Change the Maximum age setting using a relative time specifier. Example: -7d, -30d. TAXII feeds do not support this setting.
  4. Enable the retention search for the collection. On the Enterprise Security menu bar, open Settings and select Content Management.
  5. Search for "retention" using the search filter.
  6. Enable the retention search for the collection that hosts the threat source. All retention searches are disabled by default.

Add an OpenIOC file

All file-based threat sources must be uploaded to the monitored path on the Enterprise Security search head, or made available on a mounted network share. An OpenIOC file must have a .ioc extension to be read and parsed.

Default settings

The modular input da_ess_threat_local is configured to monitor the $SPLUNK_HOME/etc/apps/DA-ESS-ThreatIntelligence/local/data/threat_intel folder, ingest any .ioc files found, and delete the documents after processing.

To review or modify the da_ess_threat_local inputs settings:

1. On the Enterprise Security menu bar, open Settings > Data Inputs and select Threat Intelligence Manager.
2. Select the da_ess_threat_local modular input.
3. Review or modify the settings as required.

Note: Do not modify the default da_ess_threat_default, or sa_threat_local inputs.

Add a STIX file

All file-based threat sources must be uploaded to the monitored path on the Enterprise Security search head, or made available on a mounted network share. A STIX file must have a .xml extension to be read and parsed.

Default settings

The modular input da_ess_threat_local is configured to monitor the $SPLUNK_HOME/etc/apps/DA-ESS-ThreatIntelligence/local/data/threat_intel folder, ingest any .xml files found, and delete the documents after processing.

To review or modify the da_ess_threat_local inputs settings:

  1. On the Enterprise Security menu bar, open Settings > Data Inputs and open Threat Intelligence Manager
  2. Select the da_ess_threat_local modular input.
  3. Review or modify the settings as required.

Note: Do not modify the default da_ess_threat_default, or sa_threat_local inputs.

Add a TAXII feed

  1. On the Enterprise Security menu bar, open Configure > Data Enrichment and choose Threat Intelligence Downloads.
  2. Click New to open the Threat Intelligence Download Settings editor.
  3. Enter a Type of taxii.
  4. Enter taxii-specific POST arguments in line, space delimited.
    1. collection: The name of the data collection from a TAXII feed. Example: collection="A_TAXII_Feed_Name"
    2. earliest: The earliest threat data to pull from the TAXII feed. Example: earliest="-1y"
    3. (Optional) taxii_username: An optional method to provide a TAXII feed username. Example: taxii_username="user"
    4. (Optional) taxii_password: An optional method to provide a TAXII feed password. If username is set without a providing a password, the input will attempt to find the password in Credential Management. Example: taxii_password="password"
    5. (Optional) cert_file: Add the certificate file name if the TAXII feed uses certificate authentication. The file name must match exactly and is case sensitive. Example: cert_file="cert.crt"
    6. (Optional) key_file: Add the key file name for the certificate if the TAXII feed uses certificate authentication. The file name must match exactly and is case sensitive. Example: key_file="cert.key"
  5. Save the changes.
  6. Review the "Threat Intelligence Audit" dashboard to validate that you properly configured the TAXII feed and Splunk Enterprise Security is downloading it.

Add a TAXII feed with certificate authentication

In a Splunk Cloud deployment, customers must work with Splunk Support to add or change files on cloud-based nodes. Add the certificate and keys to the same app directory that you define the TAXII feed in, for example, DA-ESS-ThreatIntelligence.

  1. Follow the steps for adding a TAXII feed to Splunk Enterprise Security.
  2. Add the certificate to the $SPLUNK_HOME/etc/apps/<app_name>/auth directory.
  3. Add the private key for the certificate to the same /auth directory.
  4. Add the TAXII feed, using the cert_file and key_file arguments to specify the file names of the certificate and private key file.

Add a URL-based threat source

  1. On the Enterprise Security menu bar, open Configure > Data Enrichment and choose Threat Intelligence Downloads.
  2. Click New to open the Threat Intelligence Download Settings editor.
  3. Enter the information about the threat source, filling in all required fields. For examples, use the pre-configured threat sources provided in Enterprise Security.
  4. Save the changes when you are done.
  5. Validate the source is properly configured and downloading by reviewing the "Threat Intelligence Audit" dashboard.

Adding a custom source

Use a lookup file to add a local threat source to the Enterprise Security threat collections. A lookup based threat source can add data to any of the "supported threat intelligence groups". To add a new lookup table and enable it as a threat source:

1. Build a lookup file with the required fields.
  • For a list of fields required in a threat collection, open Configure > Data Enrichment on the menu bar and select Lists and Lookups.
  • Find the pre-configured lookup that matches the threat collection data you are providing.
Lookup name Details
Local Certificate Intel Threat intelligence pertaining to SSL certificates
Local Domain Intel Threat intelligence pertaining to domains
Local Email Intel Threat intelligence pertaining to email
Local File Intel Threat intelligence pertaining to files
Local HTTP Intel Threat intelligence pertaining to HTTP
Local IP Intel Threat intelligence pertaining to IPs
Local Process Intel Threat intelligence pertaining to processes
Local Registry Intel Threat intelligence pertaining to the Windows registry
Local Service Intel Threat intelligence pertaining to services
Local User Intel Threat intelligence pertaining to users
  • Select the lookup name in the Lists and Lookups UI to view the required fields.
  • Create a new .csv file with a header row containing the required fields.
  • Add the data to the .csv file.
2. Create and upload a lookup table file.
  • On the Splunk Enterprise menu, open Settings > Lookups and open Lookup table files.
  • Click Add New.
  • Select a Destination App. For example, SA-ThreatIntelligence.
  • Select the lookup file to upload. The file must be in .csv format.
  • Provide the destination file name. Enter the name that this lookup table file will have on the Splunk server. When uploading a plaintext csv file, the filename should end in .csv. Example, threatening_stuff.csv.
  • Save.
3. Set the permissions for the lookup table.
  • In Lookup Table Files, find the new lookup and select Permissions.
  • Set Object should appear in to All apps.
  • Set Read access for Everyone.
  • Set Write access for admin. Optionally, add other roles with lookup editor permissions. See "Adding capabilities to a role" in the Installation and Upgrade manual.
  • Save.
4. Add a lookup definition.
  • Browse to Settings > Lookups.
  • Select Add New in the Actions column for Lookup definitions.
  • Select the Destination App. For example, SA-ThreatIntelligence.
  • Provide a name for the lookup threat source. The name defined here is the name used in the Threatlist input stanza definition. For example: threatening_stuff_list.
  • Select a Type: of File based
  • Select the Lookup File: that you added in step one.
  • Save.
5. Set the permissions on the lookup definition.
  • In Lookup definitions, find the new definition and select Permissions.
  • Set Object should appear in to All apps.
  • Set Read access for Everyone.
  • Set Write access for admin. Optionally, add other roles with lookup editor permissions. See "Adding capabilities to a role" in the Installation and Upgrade manual.
  • Save.
6. Add a threat source input stanza by creating or cloning an input.
  • Browse to Configure > Data Enrichment > Threat Intelligence Downloads.
  • Determine which of the threat source input is the best match for the new content.
  • If one of the pre-configured inputs is a match for your threat intel source, select the Clone option under the Actions column.
    • Fill out the required fields in the newly cloned threat source input.
      • Name The new threat source name. Example: threatening stuff list
      • Description A descriptive identifier.
      • URL Add a reference to the lookup definition. Example: lookup://threatening_stuff_list
  • If one of the pre-configured inputs isn't a match for your threat intel source, select New. Fill out the required fields in the newly created threat source input.
7. Verify the new lookup-based threat source was imported.
  • Browse to Audit > Threat Intelligence Audit and verify the threat source is enabled. Use the panel Threat Intelligence Audit Events to find errors associated with the lookup name.
  • Browse to Advanced Threat > Threat Artifacts and verify an object in the threat source can be found.

Add database-hosted content

For an example of how to add database-hosted content, see this blog post discussing custom threat feed integration with Enterprise Security (http://blogs.splunk.com/2014/03/10/custom-threat-feed-integration-with-enterprise-security).

Add threat data manually

Each threat collection has a pre-configured lookup file to accept manually entered threat data.

  1. On the Enterprise Security menu bar, open Configure > Data Enrichment and select Lists and Lookups.
  2. Find the pre-configured lookup that matches the threat collection data you are providing. Example: Local Certificate intel.
  3. Select the lookup to open the lookup editor.
  4. Update the lookup, populating all of the fields for each row (right-click on the row, and choose Insert Row Below).
  5. Click Save when finished.

Configure a custom folder for threat sources

Enterprise Security provides inputs configured to monitor folders and ingest threat intelligence sources. Use the Threat Intelligence Manager page to review, add, or change inputs for monitoring folders containing threat sources.

Default settings

The modular input da_ess_threat_local is configured to monitor the $SPLUNK_HOME/etc/apps/DA-ESS-ThreatIntelligence/local/data/threat_intel folder, ingest any .csv, .ioc and .xml files found, and delete the documents after processing.

To review or modify the da_ess_threat_local inputs settings:

  1. Navigate to Settings > Data Input > Threat Intelligence Manager
  2. Select the da_ess_threat_local modular input.
  3. Review or modify the settings as required.

Note: Do not modify the default da_ess_threat_default, or sa_threat_local inputs.

Create a new input monitor for threat sources

  1. Navigate to Settings > Data Input > Threat Intelligence Manager
  2. Select New
  3. Provide a descriptive name for the modular input.
  4. Provide a path to the file repository.
  5. (Optional): Provide a maximum file size in bytes.
  6. (Optional): Set the sinkhole option. The default is to retain all documents.
  7. (Optional): Set the default weight for all threat intelligence documents consumed from this directory.
Last modified on 28 March, 2017
PREVIOUS
Threat Intelligence dashboards
  NEXT
Advanced Threat dashboards

This documentation applies to the following versions of Splunk® Enterprise Security: 4.0.1, 4.0.2, 4.0.3, 4.0.4, 4.0.5, 4.0.6


Was this documentation topic helpful?


You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters