Splunk® Enterprise Security

User Manual

Download manual as PDF

This documentation does not apply to the most recent version of ES. Click here for the latest version.
Download topic as PDF

Configure lists and lookups

To configure or edit the lists or lookup files used with the Splunk Enterprise Security, select Configure > Data Enrichment > Lists and Lookups. Use Lists and Lookups to view and edit the default lists and lookups in Enterprise Security.

Es lookup files 3-0.png

Click the name of a list to view or edit it. Click Export to export a copy of the file in CSV format.

Internal lookups

Splunk Enterprise Security maintains internal lookups to provide information for dashboards or to create notable events. See Available internal lookups for more details on the lookups included with Splunk Enterprise Security.

These lookups are created in three ways.

  • Populated by a static lookup table.
  • Populated internally by search commands, called a search-driven lookup.
  • Populated with information from the Internet.

The internal lookups populated with information from the Internet are used by some correlation searches to identify hosts that are recognized as malicious or suspicious according to various online sources, such as the SANS Institute. If Splunk Enterprise Security is not connected to the Internet, the lookup files are not updated and the correlation searches that rely on the lookups may not function correctly. Most of the internal lookups populated by the Internet are threat intelligence sources. See Threat sources included with ES in this manual.

Edit lists and lookups

From the ES menu bar, select Configure > Data Enrichment > Lists and Lookups to view the list of current lookup files. Click a file name to open that lookup file in the lookup editor.

ES listsandlookups editor.png

The name of the CSV file is shown in the upper left-hand corner of the panel, assets.csv in this example. The lookup fields are shown at the top of the table, the values for the fields are displayed in the rows below that. Positive numbers are in green, negative numbers are shown in red. The priority values in this file are color-coded. Each CSV file looks slightly different depending on the fields it contains. Lookups do not accept regular expressions.

Only users with appropriate permissions can edit lookups. See Permissions in this manual to edit permissions for a user role.

Edit lookup content

  1. From the ES menu bar, select Configure > Data Enrichment > Lists and Lookups to view the list of current lookup files.
  2. Click a file name to open that lookup file in the lookup editor.
  3. Change a value in a cell by selecting the cell and typing the new value.
  4. Right-click the table to open a context menu that you can use to add columns or rows to the file.
  5. Click Save to save your changes or Cancel to return to the list of lookups without saving.

Note: You cannot save a lookup file that contains empty header fields.

To review the last time a lookup file was edited and by whom, use a search. For example

index=_internal uri_path="/splunk-es/en-US/app/SplunkEnterpriseSecuritySuite/ess_lookups_edit"

Add new lookup files

An admin may add new CSV files to support new functions and data enrichment in the application. CSV files used as lookups must be created with Unix-style line endings (\n).

Note: CSV files used as lookups must be created with Unix-style line endings ("\n"). Splunk will not correctly read lookup files saved using Macintosh ("\r") or Windows line endings ("\r\n").

  1. From the Splunk platform system menu, select Settings > Lookups.
  2. Next to Lookup table files, click Add New.
  3. Verify that the Destination app is set to SplunkEnterpriseSecuritySuite.
  4. Upload a lookup file.
  5. Type a Destination filename to be displayed in the lookup list.
  6. Click Save.

By default, lookups are saved as Private. To share the information with other users, searches, and upgrade events, change the file permissions.

  1. Click Permissions next to the newly imported CSV file.
  2. Select the appropriate level and type of permissions for this file. Set access to This app only to limit access to Splunk Enterprise Security or select All apps to allow all apps in this Splunk instance to access the lookup.
  3. Click Save.

ES lookup permissions 3-0.png

Verify lookup files

Confirm that you added a lookup file successfully by using the inputlookup search command to display the list.

inputlookup append=T application_protocol_lookup

Search-driven lookups

A search-driven lookup allows you to create a lookup based on the results of a search that runs at regular scheduled intervals. Create a search-driven lookup if you want to know when something new happens in your environment, or need to consistently update a lookup based on changing information from a data model or another lookup. The search can only run against data stored in data models or in an existing lookup. Lookups created as search-driven lookups are excluded from bundle replication and are not sent to the indexers.

The search-driven lookup collects and stores information from data models or other lookups. The data stored in the lookup represents a historical summary of selected fields gathered from events. You can view changes on a dashboard or use a correlation search to compare data from the search-driven lookup with new events, and alert if there is a match. For example, if you want to find out when a new user logs in to a web server.

  1. Search for user data in the Authentication data model and filter by the web server host name with the where command.
  2. Verify the search results match the known hosts and users in your environment.
  3. Create a guided search-driven lookup to collect and store information on a recurring schedule about users logging in to the web servers.
  4. Create a correlation search that alerts you when a user logs in to one of the web servers that has not in the past, based on the historical information in the search-driven lookup.

Create a search-driven lookup

Create a new search-driven lookup. In this example (also included with Splunk Enterprise Security), you want to track attacks identified by your intrusion detection system (IDS). You can then be notified of new attacks with a correlation search, or determine whether an attack is new to your environment or not. The Intrusion Center dashboard uses this search-driven lookup for the New Attacks - Last 30 Days panel. See Intrusion Center dashboard.

  1. From the Splunk Enterprise Security menu bar, select Configure > Content Management.
  2. Click Create New Content and select Search-Driven Lookup.
  3. (Optional) Select an App. The default app is SplunkEnterpriseSecuritySuite. You can create the lookup in a specific app, such as SA-NetworkProtection, or a custom app. You cannot change the app after you save the search-driven lookup.
  4. Type a description for the search.
    For example, "Maintains a list of attacks identified by an IDS and the first and last time that the attacks were seen."
  5. Type a label for the lookup. This is the name of the search-driven lookup that appears on Content Management.
    For example, IDS Attack Tracker.
  6. Name the lookup. Set a name that is unique and describes the content of the lookup. After you save the lookup, the name cannot be changed.
    For example, ids_attack_tracker.
  7. Type a cron schedule to define how often you want the search to run.
    For example, 25 * * * * to run the search at 25 minutes every hour every day.
  8. Select real-time or continuous scheduling for the search. Real-time scheduling prioritizes search performance, while continuous scheduling prioritizes data integrity.
    For this example, select Continuous Schedule because the lookup must track all data points.
  9. Type a Search Name to set the name of the saved search. After you save the lookup, the name cannot be changed.
    For example, Network - IDS Attack Tracker - Lookup Gen.
  10. Select guided mode to use the guided search editor to create the search.
  11. Click Open guided search editor to start creating the search.
  12. Select the data source for the search.
    For example, select Data Model.
  13. Fill out the rest of the required fields.
    For example, select a data model of Intrusion_Detection and a data model object of IDS_Attacks. Select Yes for the summaries only field to perform the search against only the data in the accelerated data model.
  14. Select a time range for the search.
    For example, select a relative time range that begins with an earliest time of 70 minutes ago, starting at the beginning of the minute, and ends now.
  15. Click Next.
  16. (Optional) Type a where clause to filter the data and click Next.
    For example, use a where clause to filter the data from the Intrusion_Detection data model to only the data from a specific IDS vendor.
  17. Add an aggregate value to track specific statistics about the data and store that information in the lookup. At least one aggregate is required.
    For example, to track the first time that an IDS attack was seen in your environment, add a new aggregate with a function of min and a field of _time and save it as firstTime. Track the last time an attack was seen by adding another aggregate with a max function and a field of _time and saving it as lastTime. This creates two columns in the lookup, firstTime and lastTime.
  18. Add split-by clauses to track additional data points in the lookup.
    For example, add a new split-by clause of IDS_Attacks.ids_type and rename it as ids_type to monitor the IDS type in the lookup. Add additional split-by clauses to similarly rename IDS_Attacks.signature and IDS_Attacks.vendor_product. All three clauses appear as columns in the lookup.
  19. Click Next.
  20. Set a retention period that defines the age of the data to be stored in the lookup. For guidance on the time format, see Date and time format variables in the Search Reference manual.
    For example, you want to keep 5 years of IDS attack evidence stored in this lookup. Select a time field of lastTime to base the retention on the last time an attack was identified by the IDS. Type an earliest time of -5y and indicate the format of the time value that you entered: %s.
  21. Click Next.
  22. Review the search created by the wizard and click Done to save the search.

Modify a search-driven lookup

  1. From the Splunk Enterprise Security menu bar, select Configure > Content Management.
  2. Select a Type of Search-Driven Lookup.
  3. Click the lookup that you want to edit.
  4. Make any changes and click Save.

Existing search-driven lookups

ES system searches that end with Lookup Gen can be edited as guided search-driven lookups. The following search-driven lookups have already been migrated to guided search-driven lookups and have a default retention period of 5 years as of Splunk Enterprise Security version 4.2.0.

  • Access - Access App Tracker - Lookup Gen
  • Endpoint - Listening Ports Tracker - Lookup Gen
  • Endpoint - Malware Tracker - Lookup Gen
  • Endpoint - System Version Tracker - Lookup Gen
  • Network - IDS Attack Tracker - Lookup Gen
  • Network - IDS Category Tracker - Lookup Gen
  • Network - Port And Protocol Tracker - Lookup Gen

Any locally-defined search-driven lookups can also be converted to guided mode, making editing easier, or kept in manual mode, requiring that you edit the search directly. Open a locally-defined search-driven lookup in Content Management to convert it to guided mode.

Enable or disable the search populating a search-driven lookup

You can enable or disable the search of a search-driven lookup to prevent the search from updating the lookup. If you disable the search that populates a search-driven lookup, the search stops updating the lookup and the data in the lookup will stop being updated. Correlation searches or dashboards that rely on the data inside the lookup will be out-of-date.

  1. Select Configure > Content Management.
  2. Filter on a type of search-driven lookup and open the search-driven lookup that you want to enable or disable.
  3. Locate the Search name of the search-driven lookup.
  4. From the Splunk platform menu bar, select Settings > Searches, reports, alerts.
  5. Locate the search and enable or disable it.

Available internal lookups

The following lookups are available by default in Splunk Enterprise Security. Select Configure > Data Enrichment > Lists and Lookups to view the internal lookups.

Application Protocols

The Application Protocols list is a list of port/protocol combinations and their approval status in the organization. This list is used by the Port & Protocol Tracker dashboard. See Port & Protocol Tracker dashboard.

The following fields are available in this file.

Field Description
dest_port The destination port number (must be 0-65535)
transport The protocol of the network traffic (icmp, tcp, udp).
app application name
status The approval status of the port (approved, pending, unapproved). By default, the port is considered approved.

Assets

The Assets lookup contains information about the assets in your environment. This list of assets is matched to incoming events. See Add asset and identity data to Splunk Enterprise Security.

Categories

The category list can contain any set of categories you choose for organizing an asset or an identity. A category is logical classification or grouping used for assets and identities. Common choices for assets include compliance and security standards such as PCI, or functional categories such as server and web_farm. Common choices for identities include titles and roles. For additional examples, see Asset and identity lookup header and field reference.

Note: To enrich events with category information in asset and identity correlation, you must maintain the category field in the asset and identity lists rather than the Categories list. See Asset and identity lookup header and field reference.

There are two ways to maintain the Categories list.

Run a saved search to maintain a list of categories

Splunk Enterprise Security includes a saved search that takes categories defined in the asset and identity lists and adds them to the Asset/Identity Categories list. The search is not scheduled by default.

  1. From the Splunk platform menu bar, select Settings > Searches, reports, alerts.
  2. Enable the Identity - Make Categories - Lookup Gen saved search.

Manually maintain a list of categories

Maintain the Categories list manually by adding categories to the lookup directly. By default, you must maintain the list manually.

  1. Select Configure > Data Enrichment > Lists and Lookups.
  2. Click the Asset/Identity Categories list.
  3. Add new categories to the list.
  4. Save your changes.

Expected Views

The Expected Views list specifies Splunk Enterprise Security views that are monitored on a regular basis. The View Audit dashboard uses this lookup. See View Audit for more about the dashboard.

The following table shows the fields in this file.

Field Description
app The application that contains the view (SplunkEnterpriseSecuritySuite)
is_expected Either "true" or "false". If not specified, Splunk Enterprise Security assumes by default that activity is not expected.
view The name of the view. Available in the URL.

To find the name of a view:

  1. Navigate to the view in Enterprise Security
  2. Look at the last segment of the URL to find the view name

For example, the view in the URL below is named incident_review: Ess-incidentReviewURL.png

Identities

The Identities lookup contains a list of identities that are matched to incoming events. See Add asset and identity data to Splunk Enterprise Security in this manual.

Interesting Ports

Interesting Ports contains a list of TCP and UDP ports determined to be required, prohibited, or insecure in your deployment. Administrators can set a policy defining the allowed and disallowed ports and modify the lookup to match that policy. To get alerts when those ports are seen in your environment, enable the correlation search that triggers an alert for those ports, such as Prohibited Port Activity Detected.

If you open the lookup file interesting_ports.csv in the lookup editor, the header of the file describes the fields in the file and also described in this table.

Field Description Example
app The application or service name Win32Time
dest The destination host for the network service. Accepts a wildcard. DARTH*, 10.10.1.100, my_host, etc. Using just a wildcard * will match all hosts.
dest_pci_domain An optional PCI Domain. Accepts a wildcard. trust, untrust, etc.
dest_port The destination port number. Accepts a wildcard. 443, 3389, 5900, etc.
transport The transport protocol. Accepts a wildcard. tcp or udp
is_required Is the service required to be running? Alert if not present. true or false
is_prohibited Is the service/traffic/port prohibited from running? Alert if present. true or false
is_secure Is the service traffic encrypted? true or false
note A brief description of the service and use-case Unencrypted telnet services are insecure.

Interesting Processes

Interesting Processes contains a list of processes. This list is used to determine whether a process is required, prohibited, and/or secure. Use the List and Lookup editor to modify or add to this list. The Interesting Processes lookup is named interesting_processes.csv.

The following table shows the fields in this file.

Column Description
app application name
dest destination of process
dest_pci_domain PCI domain, if available
is_required true or false
is_prohibited true or false
is_secure true or false
note Any additional information about this process

Interesting Services

Interesting Services contains a list of services in your deployment. This list is used to determine whether a service is required, prohibited, and/or secure. Use the List and Lookup editor to modify or add to this list. The Interesting Services is named interesting_services.csv.

The following table shows the fields in this file.

Column Description
app application name
dest destination of process
dest_pci_domain PCI domain, if available
is_required true or false
is_prohibited true or false
is_secure true or false
note Any additional information about this process

Primary Functions

Primary Functions contains a list of primary processes and services, and their function in your deployment. Use this list to designate which services are primary and the port and transport to use. The Primary Functions lookup file is named primary_functions.csv.

The following table shows the fields in this file.

Column Description
process name of process
service name of service
dest_pci_domain PCI domain, if available
transport tcp or udp
port port number
is_primary true or false
function function of this process (for example, Proxy, Authentication, Database, Domain Name Service (DNS), Web, Mail)

Prohibited Traffic

Prohibited Traffic lists processes that will generate an alert if they are detected. This list is used by the System Center dashboard and is useful for detecting software that has been prohibited by the security policy (such as IRC or data destruction tools) or for software that is known to be malicious (such as malware that was recently implicated in an outbreak).

The Prohibited Traffic file is named prohibited_traffic.csv.

The following table shows the fields in this file.

Field Description
app The name of the process (such as echo, chargen, etc.)
is_prohibited Either "true" or "false"
note A text description of why the process is rejected

Urgency Levels

Urgency Levels contains the combinations of priority and severity that dictate the urgency of notable events. For more information, see Notable Event Urgency assignment in this manual.

PREVIOUS
Content Management
  NEXT
Dashboard Troubleshooting

This documentation applies to the following versions of Splunk® Enterprise Security: 4.5.0, 4.5.1


Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters