Splunk® Enterprise Security

Administer Splunk Enterprise Security

Add new threat intelligence sources in Splunk Enterprise Security

Administrators can add new threat intelligence sources to Splunk Enterprise Security by downloading a feed from the internet, uploading a structured file, or inserting the threat intelligence directly from events in Splunk Enterprise Security.

Add new intelligence sources using any of the following methods:

Add a URL-based intelligence source

Splunk Enterprise Security can periodically download an intelligence feed available from the internet and store it in the $SPLUNK_DB/modinput/threatlist directory. You can then use the inputintelligence search command to use the intelligence in reports, searches, or dashboards.

Steps

  1. In Splunk Enterprise Security, select Configure and then Threat intelligence.
  2. In the Threat intelligence management section, select Threat intelligence sources.
  3. Select New to add a new intelligence source.
  4. Enter a Name for the download. The name can only contain alphanumeric characters, hyphens, and underscores. The name cannot contain spaces.
  5. Do not select the check box for Sinkhole.
  6. Deselect the check box for Is Threat Intelligence.
  7. Enter a Type for the download. The type identifies the type of information that the feed contains.
  8. Enter a Description. Describe the information in the feed.
  9. Leave the default Weight because the field does not matter for the generic intelligence source.
  10. (Optional) Change the default download Interval for the feed. Defaults to 43200 seconds, or every 12 hours.
  11. (Optional) Enter POST arguments for the feed. You can use POST arguments to retrieve user credentials from Credential Management. Use the format key=$user:<username>$ or key=$user:<username>,realm:<realm>$ to specify a username and realm.
  12. Do not use the Maximum age setting.
  13. (Optional) If you need to specify a custom User agent string to bypass network security controls in your environment, enter it in the format <user-agent>/<version>. For example, Mozilla/5.0 or AppleWebKit/602.3.12. The value in this field must match this regex: ([A-Za-z0-9_.-]+)/([A-Za-z0-9_.-]+). Check with your security device administrator to ensure the string you enter here is accepted by your network security controls.
  14. Fill out the Parsing options fields to make sure that your list parses successfully. You must fill out either a delimiting regular expression or an extracting regular expression. You cannot leave both fields blank.
    Field Description Example
    Delimiting regular expression A regular expression string used to split, or delimit, lines in an intelligence source. For complex delimiters, use an extracting regular expression.

    For parsing options, you can either use a delimiting regular expression or an extracting regular expression, but not both.

    		 , or : or \t
    Extracting regular expression A regular expression used to extract fields from individual lines of an intelligence source document. Use to extract values in the intelligence source.

    For parsing options, you can either use a delimiting regular expression or an extracting regular expression, but not both.

    		 ^(\S+)\t+(\S+)\t+\S+\t+\S+\t*(\S*)
    Fields Required if your document is line-delimited. Comma-separated list of fields to be extracted from the intelligence list. Can also be used to rename or combine fields. Description is a required field. Additional acceptable fields are the fields in the corresponding KV Store collection for the threat intelligence, visible in the local lookup files or the DA-ESS-ThreatIntelligence/collections.conf file. Defaults to description:$1,ip:$2. <fieldname>:$<number>,<field name>.$<number>
    ip:$1,description:domain_blocklist
    Ignoring regular expression A regular expression used to ignore lines in an intelligence source. Defaults to ignoring blank lines and comments that begin with #. ^\s*$)
    Skip header lines The number of header lines to skip when processing the intelligence source. 0
    Intelligence file encoding If the file encoding is something other than ASCII or UTF8, specify the encoding here. Leave blank otherwise. latin1
  15. (Optional) Change the Download Options fields to make sure that your list downloads successfully.
    Field Description Example
    Retry interval Number of seconds to wait between download retry attempts. Review the recommended poll interval of the intelligence source provider before changing the retry interval. 60
    Remote site user If the threat feed requires authentication, enter the user name to use in remote authentication, if required. The user name you add in this field must match the name of a credential on the Credentials page. buttercup
    Remote site user realm If the threat feed requires authentication, enter the user name to use in remote authentication, if required. The realm you add in this field must match the realm of a credential on the Credentials page. paddock
    Retries The maximum number of retry attempts. 3
    Timeout Number of seconds to wait before marking a download attempt as failed. 30
  16. (Optional) If you are using a proxy server, fill out the Proxy options for the feed. See Configure proxy server settings in Splunk Enterprise Security.
  17. Save your changes.

Add a TAXII feed

Add threat intelligence provided as a TAXII feed to Splunk Enterprise Security.

Current versions of Splunk Enterprise Security only support TAXII version 1.0 and TAXII version 1.1.

Prerequisite

Determine whether the TAXII feed requires certificate authentication. If it does, add the certificate and keys to the same app directory in which you define the TAXII feed. For example, DA-ESS-ThreatIntelligence.

  1. Follow the steps to add a new certificate to Splunk Enterprise Security to add both the certificate and the private key files.
  2. Follow the steps for adding a TAXII feed to Splunk Enterprise Security, using the cert_file and key_file POST arguments to specify the file names of the certificate and private key file.

Steps

  1. On the Enterprise Security menu bar, select Configure and then Threat intelligence.
  2. Select New to add a new TAXII feed.
  3. Enter a Name for the threat intelligence feed.
  4. Enter a Description and URL for the threat intelligence field.
  5. Verify that the check box for Is Threat Intelligence is selected.
  6. (Optional) Select or deselect the check box for Sinkhole. Select the check box to delete the downloaded file after processing. The sinkhole option works for anything in the pickup directory that has been processed. The pickup directories follow: 
    $SPLUNK_HOME/etc/apps/SA-ThreatIntelligence/local/data/threat_intel
$SPLUNK_HOME/etc/apps/DA-ESS-ThreatIntelligence/default/data/threat_intel
$SPLUNK_HOME/etc/apps/DA-ESS-ThreatIntelligence/local/data/threat_intel
$SPLUNK_HOME/etc/apps/<custom>
  7. Enter a Type of taxii.
  8. Enter a Description for the threat intelligence feed.
  9. Enter a URL to use to download the TAXII feed.
  10. (Optional) Change the default Weight for the threat intelligence feed. Increase the weight if the threats on the threat feed are high-confidence and malicious threats that should increase the risk score for assets and identities that interact with the indicators from the threat source.
  11. (Optional) Adjust the interval at which to download the threat intelligence. Defaults to 43200 seconds, or twice a day.
  12. Enter TAXII-specific space-delimited POST arguments for the threat intelligence feed.
    <POST argument>="<POST argument value>"
    Example POST argument Description Example
    collection Name of the data collection from a TAXII feed. collection="A_TAXII_Feed_Name"
    earliest The earliest threat data to pull from the TAXII feed.

    You can use the "earliest" POST argument only when the modular input runs for the first time. All subsequent runs of the modular input use the timestamp of the last modular input as the "earliest" POST argument.

    		 earliest="-1y"
    taxii_username An optional method to provide a TAXII feed username. taxii_username="user"
    taxii_password An optional method to provide a TAXII feed password. If you provide a username without providing a password, the threat intelligence modular input attempts to find the password in Credential Management. taxii_password="password"
    taxii_username_realm An optional method to provide a realm for the TAXII feed username. Used with the taxii_username to locate the user credential password in Credential Management. taxii_username_realm="realm"
    cert_file Add the certificate file name if the TAXII feed uses certificate authentication. The file name must match exactly and is case sensitive. cert_file="cert.crt"
    key_file Add the key file name for the certificate if the TAXII feed uses certificate authentication. The file name must match exactly and is case sensitive. key_file="cert.key"
  13. TAXII feeds do not use the Maximum age setting.
  14. TAXII feeds do not use the User agent setting.
  15. TAXII feeds do not use the Parsing Options settings.
  16. (Optional) Change the Download Options.
  17. (Optional) Change the Proxy Options.
  18. Save the changes.

You cannot use an authenticated proxy with a TAXII feed because the libtaxii library used by Enterprise Security does not support authenticated proxies. If possible, use an unauthenticated proxy instead.

Upload a STIX or OpenIOC file

Splunk Enterprise Security supports adding the following file types directly in the Splunk Enterprise Security interface:

  • OpenIOC 1.0 and 1.1
  • STIX 1.0, 2.0, and 2.1
  • CSV

Parsing STIX documents of version 2.0 and version 2.1 parses STIX observable objects such as type: "observed-data" from the threat intelligence document as outlined in the collections.conf configuration file. The STIX pattern syntax used in STIX "indicator" objects and elsewhere is not currently supported.

To add a file in the Splunk Enterprise Security interface, complete the following steps:

  1. On the Enterprise Security menu bar, select Configure > Data Enrichment > Threat Intelligence Management.
  2. Select New.
  3. Select IOC/STIX/STIX 2 from the drop down menu.
    This opens the Add Intelligence Document dialog.
  4. Enter the information for the threat intelligence document that you want to upload.
  5. Select the General tab and enter a Weight for the threat intelligence file.
  6. Select the Threat intelligence checkbox if you want to classify the intelligence document as threat intelligence. Classifying an intelligence document as threat intelligence triggers specific workloads.

    Use the tooltips provided in the UI to populate the remaining fields based on the intelligence document that you plan to upload.

  7. (Optional) Select the Advanced tab and select the Sinkhole check box. This deletes the file after the intelligence from the file is processed.
  8. Select Save.


Upload a custom CSV file

You can add a custom file of threat intelligence to Splunk Enterprise Security. Adding threat intelligence enhances the analysts' security monitoring capabilities and adds context to their investigations. Splunk Enterprise Security supports multiple types of threat intelligence so that you can add your own threat intelligence.

How to format threat intelligence files

You can format the custom CSV file by adding headers for each type of intelligence in the file. The custom file can contain multiple types of intelligence, but you can include headers for each column in the CSV file.

Alternatively, for threat intelligence sources without headers such as "iblocklist_tor", you can use Parsing Options fields in Splunk Enterprise Security to ensure that the CSV file parses successfully.

If you upload a threat intel CSV file, where the headers on the CSV do not map to the headers in the collections.conf configuration file for various threat collections such as email_intel, ip_intel, certificate_intel, add transforms.conf-style field settings to the Fields field in the Parsing tab,
For example, for the following CSV file: 

foo,bar,baz
alpha,bravo,charlie

If the Fields setting is certificate_version:$1,certificate_serial:$3,certificate_subject_unit:$2, then the resulting data from the certificate_intel collection is as follows: 

certificate_version | certificate_serial | certificate_subject_unit
--------------------+--------------------+--------------------------
alpha               | charlie            | bravo

You must select fields that map to fields in the transforms.conf configuration file for the various threat collections.

Add the custom file to Splunk Enterprise Security

  1. On the Enterprise Security menu bar, select Configure and then Threat intelligence.
  2. Enter a file name for the file you want to upload. The file name you enter becomes the name of the file saved to $SPLUNK_HOME/etc/apps/SA-ThreatIntelligence/lookups. The file name cannot include spaces or special characters and is saved in $SPLUNK_HOME/etc/apps/SA-ThreatIntelligence/lookups to ensure that all the search heads in a cluster are synchronized.
  3. Upload the CSV-formatted file.
  4. Enter a Weight for the threat list. The weight of a threat file increases the risk score of objects associated with threat intelligence on this list.
  5. (Optional) Select the Overwrite check box. If you have previously uploaded a file with the same file name, select this check box to overwrite the previous version of the file.
  6. (Optional) In the Advanced tab, select the Sinkhole check box. This deletes the file after the intelligence from the file is processed.
  7. Select Save.

Add threat intelligence from Splunk events

You can add threat intelligence from Splunk events to the local threat intelligence lookups.

  1. Write a search that produces threat indicators.
  2. Add | outputlookup local_<threat intelligence type>_intel append=t to the end of the search.

The local_<threat intelligence type>_intel lookup files do not automatically prune themselves. Using append=t in a scheduled search adds to the file until the file is pruned either by some other scheduled search or manually.

If you run a scheduled search at an interval to populate this file for ingestion into the threat intelligence framework, append=f results in the lookup being overwritten each time the scheduled search is run so that you do not have to prune the file manually. Ensure that your scheduled run time is greater than your threat intelligence data source interval if this occurs.

Follow these guidelines to construct the search and leverage the local threat intelligence lookups:

  • Identify the local lookups that serve as threat intelligence documents.
    Navigate to Data Enrichment > Threat Intelligence Management > Sources.
    This lists the available local lookups such as local_ip_intel, local_http_intel, local_file_intel.
  • Edit the fields in the local CSV lookup using Edit Intelligence Document > Fields.
    To identify the fields supported by the lookup, navigate to the collections.conf configuration file: Settings > Lookups > Lookup Definitions and search for the ip_intel lookup.
    All fields supported by the ip_intel lookup are listed in Supported Fields for ip_intel.
  • Alternatively, you can also map the fields in the local CSV lookup to the fields in the ip_intel in the collections.conf file. For example: The following field names are supported by the ip_intel lookup in the collections.conf file:
    • ip
    • domain
    • description
    • address
    • city
    • country
    • postal_code
    • state_prov
    • organization_name
    • organization_id
    • registration_time
    However, the fields in your local CSV lookup are as follows and don't match the fields for ip_intel in the collections.conf file:
    • ip_address
    • domain_name
    • address
    Then, you can map the fields in the local CSV lookup to the format specified for the field names in the collections.conf file as follows: ip:$1, domain:$2, description:$3

You can also, write a search that produces a list of IP addresses that are testing a web server for vulnerabilities and add them to the local_ip_intel lookup to be processed by the modular input and added to the ip_intel KV Store collection.

Add and maintain threat intelligence locally

Each threat collection has a local lookup file that you can use to manually add threat intelligence.

  1. On the Enterprise Security menu bar, select Configure > Content > Content Management.
  2. Find the local lookup that matches the type of threat indicator you want to add. For example, Local Certificate intel to add information about malicious or spoofed certificates.
  3. Select the lookup name to edit the lookup.
  4. Add indicators to the lookup. Right-click to select Insert Row Below to add new rows as needed.
  5. (Optional) Enter a numeric Weight to change the risk score for objects associated with indicators on this threat intelligence source.
  6. Select Save.

Add threat intelligence with a custom lookup file

You can add threat intelligence to Splunk Enterprise Security as a custom lookup file. Add a custom lookup file in this way if you want to edit the lookup file in Splunk Enterprise Security. If you want to add a lookup file to have the intelligence in it extracted once, upload the CSV file instead. A lookup-based threat source can add data to any of the supported threat intelligence types, such as file or IP intelligence.

Prerequisite

Create the custom CSV file. The custom file can contain multiple types of intelligence, but you must include headers for each column in the CSV file.


Steps

First, add the lookup to Splunk Enterprise Security.

  1. Select Configure > Content > Content Management.
  2. Select Create New Content > Managed Lookup.
  3. Select Create New.
  4. Select the lookup file to upload.
  5. Select an App of SA-ThreatIntelligence.
  6. (Optional) Modify the file name. For example, enter threatindicatorszerodayattack.csv.
  7. (Optional) Modify the definition name. For example, zero_day_attack_threat_indicators_list.
  8. Leave the default lookup type of Manual editing.
  9. Enter a label for the lookup. The label appears as the name for the lookup on the Content Management page. For example, Zero Day Threat Indicators.
  10. Enter a description for the lookup. For example, File-based threat indicators from zero day malware.
  11. Save.

Next, add a threat source input stanza that corresponds to the lookup file so that ES can parse the threat intelligence.

  1. Select Configure" and then Threat intelligence.
  2. Select New.
  3. Enter a Name. The name cannot include spaces. For example, zero_day_attack_threat_indicators.
  4. Enter a Type. For example, zero_day_IOCs.
  5. Enter a Description. For example, File-based threat indicators from zero day malware.
  6. Enter a URL that references the lookup definition you created. For example, lookup://zero_day_attack_threat_indicators_list
  7. (Optional) Change the default Weight for the threat data.
  8. (Optional) Change the default Retry interval for the lookup.
  9. If your lookup contains multiple types of threat intelligence, enter the headers in the Fields section.
  10. Select Save.

Upload threat intelligence using REST API

The Splunk Enterprise Security REST API supports uploading threat intelligence files in OpenIOC, STIX, or CSV format. See Threat Intelligence API reference.

Last modified on 20 December, 2024
Configure threat intelligence sources in Splunk Enterprise Security   Configure threat lists in Splunk Enterprise Security

This documentation applies to the following versions of Splunk® Enterprise Security: 8.0.0, 8.0.1

Acrobat logo Download manual
Acrobat logo Download this page

Was this topic useful?







You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters