Configure lists and lookups

To configure or edit the lists or lookup files used with the Splunk Enterprise Security, select Configure > Data Enrichment > Lists and Lookups. Use Lists and Lookups to view and edit the default lists and lookups in Enterprise Security.

Click the name of a list to view or edit it. Click Export to export a copy of the file in CSV format.

Internal lookups

Splunk Enterprise Security maintains internal lookups to provide information for dashboards or to create notable events. See Available internal lookups for more details on the lookups included with Splunk Enterprise Security.

These lookups are created in three ways.

• Populated by a static lookup table.
• Populated internally by search commands, called a search-driven lookup.
• Populated with information from the Internet.

The internal lookups populated with information from the Internet are used by some correlation searches to identify hosts that are recognized as malicious or suspicious according to various online sources, such as the SANS Institute. If Splunk Enterprise Security is not connected to the Internet, the lookup files are not updated and the correlation searches that rely on the lookups might not function correctly. Most of the internal lookups populated by the Internet are threat intelligence sources. See Threat intelligence sources included with Splunk Enterprise Security in this manual.

Edit lists and lookups

From the ES menu bar, select Configure > Data Enrichment > Lists and Lookups to view the list of current lookup files. Click a file name to open that lookup file in the lookup editor.

The name of the CSV file is shown in the upper left-hand corner of the panel, assets.csv in this example. The lookup fields are shown at the top of the table, the values for the fields are displayed in the rows below that. Positive numbers are in green, negative numbers are shown in red. The priority values in this file are color-coded. Each CSV file looks slightly different depending on the fields it contains. Lookups do not accept regular expressions.

Only users with appropriate permissions can edit lookups. See Permissions in this manual to edit permissions for a user role.

Edit lookup content

1. From the ES menu bar, select Configure > Data Enrichment > Lists and Lookups to view the list of current lookup files.
2. Click a file name to open that lookup file in the lookup editor.
3. Change a value in a cell by selecting the cell and typing the new value.
4. Right-click the table to open a context menu that you can use to add columns or rows to the file.
5. Click Save to save your changes or Cancel to return to the list of lookups without saving.

Note: You cannot save a lookup file that contains empty header fields.

To review the last time a lookup file was edited and by whom, use a search. For example

index=_internal uri_path="/splunk-es/en-US/app/SplunkEnterpriseSecuritySuite/ess_lookups_edit"

Add new lookup files

An admin might add new CSV files to support new functions and data enrichment in the application. CSV files used as lookups must be created with Unix-style line endings (\n).

Note: CSV files used as lookups must be created with Unix-style line endings ("\n"). Splunk will not correctly read lookup files saved using Macintosh ("\r") or Windows line endings ("\r\n").

1. From the Splunk platform system menu, select Settings > Lookups.
2. Next to Lookup table files, click Add New.
3. Verify that SplunkEnterpriseSecuritySuite is selected as the Destination app.
4. Upload a lookup file.
5. Type a Destination filename to be displayed in the lookup list.
6. Click Save.

By default, lookups are saved as Private. To share the information with other users, searches, and upgrade events, change the file permissions.

1. Click Permissions next to the newly imported CSV file.
2. Select the appropriate level and type of permissions for this file. Set access to This app only to limit access to Splunk Enterprise Security or select All apps to allow all apps in this Splunk instance to access the lookup.
3. Click Save.

Verify lookup files

Confirm that you added a lookup file successfully by using the inputlookup search command to display the list.

inputlookup append=T application_protocol_lookup

Search-driven lookups

A search-driven lookup lets you create a lookup based on the results of a search that runs at regular scheduled intervals. Create a search-driven lookup if you want to know when something new happens in your environment, or need to consistently update a lookup based on changing information from a data model or another lookup. The search can run only against data stored in data models or in an existing lookup. Lookups created as search-driven lookups are excluded from bundle replication and are not sent to the indexers.

The search-driven lookup collects and stores information from data models or other lookups. The data stored in the lookup represents a historical summary of selected fields gathered from events. You can view changes on a dashboard or use a correlation search to compare data from the search-driven lookup with new events, and alert if there is a match. For example, to find out when a new user logs in to a web server.

1. Search for user data in the Authentication data model and filter by the web server host name with the where command.
2. Verify the search results match the known hosts and users in your environment.
3. Create a guided search-driven lookup to collect and store information on a recurring schedule about users logging in to the web servers.
4. Create a correlation search that alerts you when a user logs in to one of the web servers that has not in the past, based on the historical information in the search-driven lookup.

Create a search-driven lookup

Create a search-driven lookup.

1. From the Splunk Enterprise Security menu bar, select Configure > Content Management.
2. Click Create New Content and select Search-Driven Lookup.
3. (Optional) Select an App. The default app is SplunkEnterpriseSecuritySuite. You can create the lookup in a specific app, such as SA-NetworkProtection, or a custom app. You cannot change the app after you save the search-driven lookup.
4. (Optional) Type a description for the search.
5. Type a label for the lookup. This is the name of the search-driven lookup that appears on Content Management.
6. Type a name for the lookup. After you save the lookup, the name cannot be changed.
7. Type a cron schedule to define how often you want the search to run.
8. Select real-time or continuous scheduling for the search. Real-time scheduling prioritizes search performance, while continuous scheduling prioritizes data integrity.
9. Type a Search Name to define the name of the saved search. After you save the lookup, the name cannot be changed.
10. Select a mode of Guided to create a search without having to write the search syntax yourself, or select Manual to write your own search. See the example for help building a search with the guided search editor.
11. If you create a search in manual mode, type a search.
12. Click Save to save the search.

In this example search-driven lookup included with Splunk Enterprise Security, you want to track attacks identified by your intrusion detection system (IDS). You can then be notified of new attacks with a correlation search, or determine whether an attack is new to your environment or not. The Intrusion Center dashboard uses this search-driven lookup for the New Attacks - Last 30 Days panel. See Intrusion Center dashboard.

1. From the Splunk Enterprise Security menu bar, select Configure > Content Management.
2. Click Create New Content and select Search-Driven Lookup.
3. (Optional) Select an App of SA-NetworkProtection. You cannot change the app after you save the search-driven lookup.
4. Type a description of "Maintains a list of attacks identified by an IDS and the first and last time that the attacks were seen."
5. Type a label of IDS Attack Tracker Example for the lookup. This is the name of the search-driven lookup that appears on Content Management.
6. Type a unique and descriptive name for the lookup of ids_attack_tracker_example. After you save the lookup, the name cannot be changed.
7. Type a cron schedule to define how often you want the search to run. If your IDS collects data often, type a cron schedule of 25 * * * * to run the search at 25 minutes every hour every day.
8. Select a Continuous Schedule because the lookup must track all data points.
9. Type a Search Name of Network - IDS Attack Tracker - Example Lookup Gen.
10. Select guided mode to use the guided search editor to create the search.
11. Click Open guided search editor to start creating the search.
12. Select a data source of Data Model because the IDS Attack data is stored in a data model.
13. Select a data model of Intrusion_Detection and a data model dataset of IDS_Attacks.
14. Select Yes for the summaries only field to run the search against only the data in the accelerated data model.
15. Select a time range that uses Relative time that begins with an earliest time of 70 minutes ago, starting at the beginning of the minute, and ends now. Click Apply to save the time range.
16. Click Next.
17. (Optional) Type a where clause to filter the data from the data model to only the data from a specific IDS vendor and click Next.
18. Add aggregate values to track specific statistics about the data and store that information in the lookup. At least one aggregate is required.
    1. To track the first time that an IDS attack was seen in your environment, add a new aggregate with a function of min and a field of _time and save it as firstTime.
    2. Track the last time an attack was seen by adding another aggregate with a max function and a field of _time and saving it as lastTime. This creates two columns in the lookup, firstTime and lastTime.
19. Add split-by clauses to track more data points in the lookup. All split-by clauses appear as columns in the lookup.
    1. Add a split-by clause of IDS_Attacks.ids_type and rename it as ids_type to monitor the IDS type in the lookup.
    2. Add a split-by clause to rename IDS_Attacks.signature as signature.
    3. Add a split-by clause to rename IDS_Attacks.vendor_product as vendor_product.
20. Click Next.
21. Select a retention period that defines the age of the data to be stored in the lookup. For example, you want to keep 5 years of IDS attack evidence stored in this lookup. Select a time field of lastTime to base the retention on the last time an attack was identified by the IDS. Type an earliest time of -5y and indicate the format of the time value that you entered: %s. You can find guidance on the time format in the Splunk platform documentation.
• For Splunk Enterprise, see Date and time format variables in the Splunk Enterprise Search Reference manual.
• For Splunk Cloud, see Date and time format variables in the Splunk Cloud Search Reference manual.




22. Click Next.
23. Review the search created by the wizard and click Done to finish using the guided search editor.
24. Click Save to save the search.

Modify a search-driven lookup

1. From the Splunk Enterprise Security menu bar, select Configure > Content Management.
2. Select a Type of Search-Driven Lookup.
3. Click the lookup that you want to edit.
4. Make changes and click Save.

Existing search-driven lookups

ES system searches that end with Lookup Gen can be edited as guided search-driven lookups. As of Splunk Enterprise Security version 4.2.0, the following search-driven lookups were migrated to guided search-driven lookups and have a default retention period of 5 years.

• Access - Access App Tracker - Lookup Gen
• Endpoint - Listening Ports Tracker - Lookup Gen
• Endpoint - Malware Tracker - Lookup Gen
• Endpoint - System Version Tracker - Lookup Gen
• Network - IDS Attack Tracker - Lookup Gen
• Network - IDS Category Tracker - Lookup Gen
• Network - Port And Protocol Tracker - Lookup Gen

Any locally-defined search-driven lookups can also be converted to guided mode, making editing easier, or kept in manual mode, requiring that you edit the search directly. Open a locally-defined search-driven lookup in Content Management to convert it to guided mode.

Enable or disable the search populating a search-driven lookup

You can enable or disable the search of a search-driven lookup to prevent the search from updating the lookup. If you disable the search that populates a search-driven lookup, the search stops updating the lookup and the data in the lookup will stop being updated. Correlation searches or dashboards that rely on the data inside the lookup will be out-of-date.

1. Select Configure > Content Management.
2. Filter on a type of search-driven lookup and open the search-driven lookup that you want to enable or disable.
3. Find the Search name of the search-driven lookup.
4. From the Splunk platform menu bar, select Settings > Searches, reports, alerts.
5. Find the search and enable or disable it.

Available internal lookups

The following lookups are available by default in Splunk Enterprise Security. Select Configure > Data Enrichment > Lists and Lookups to view the internal lookups.

Application Protocols

The Application Protocols list is a list of port/protocol combinations and their approval status in the organization. This list is used by the Port & Protocol Tracker dashboard. See Port & Protocol Tracker dashboard.

The following fields are available in this file.

Field	Description
dest_port	The destination port number (must be 0-65535)
transport	The protocol of the network traffic (icmp, tcp, udp).
app	application name
status	The approval status of the port (approved, pending, unapproved). By default, the port is considered approved.

Assets

The Assets lookup contains information about the assets in your environment. This list of assets is matched to incoming events. See Add asset and identity data to Splunk Enterprise Security.

Categories

The category list can contain any set of categories you choose for organizing an asset or an identity. A category is logical classification or grouping used for assets and identities. Common choices for assets include compliance and security standards such as PCI, or functional categories such as server and web_farm. Common choices for identities include titles and roles. For more examples, see Asset and identity lookup header and field reference.

Note: To enrich events with category information in asset and identity correlation, you must maintain the category field in the asset and identity lists rather than the Categories list. See Asset and identity lookup header and field reference.

There are two ways to maintain the Categories list.

Run a saved search to maintain a list of categories

Splunk Enterprise Security includes a saved search that takes categories defined in the asset and identity lists and adds them to the Asset/Identity Categories list. The search is not scheduled by default.

1. From the Splunk platform menu bar, select Settings > Searches, reports, alerts.
2. Enable the Identity - Make Categories - Lookup Gen saved search.

Manually maintain a list of categories

Maintain the Categories list manually by adding categories to the lookup directly. By default, you must maintain the list manually.

1. Select Configure > Data Enrichment > Lists and Lookups.
2. Click the Asset/Identity Categories list.
3. Add new categories to the list.
4. Save your changes.

Expected Views

The Expected Views list specifies Splunk Enterprise Security views that are monitored on a regular basis. The View Audit dashboard uses this lookup. See View Audit for more about the dashboard.

The following table shows the fields in this file.

Field	Description
app	The application that contains the view (SplunkEnterpriseSecuritySuite)
is_expected	Either "true" or "false". If not specified, Splunk Enterprise Security assumes by default that activity is not expected.
view	The name of the view. Available in the URL.

To find the name of a view:

1. Navigate to the view in Enterprise Security
2. Look at the last segment of the URL to find the view name

For example, the view in the URL below is named incident_review:

Identities

The Identities lookup contains a list of identities that are matched to incoming events. See Add asset and identity data to Splunk Enterprise Security in this manual.

Interesting Ports

Interesting Ports contains a list of TCP and UDP ports determined to be required, prohibited, or insecure in your deployment. Administrators can set a policy defining the allowed and disallowed ports and modify the lookup to match that policy. To get alerts when those ports are seen in your environment, enable the correlation search that triggers an alert for those ports, such as Prohibited Port Activity Detected.

If you open the lookup file interesting_ports.csv in the lookup editor, the header of the file describes the fields in the file and also described in this table.

Field	Description	Example
app	The application or service name	Win32Time
dest	The destination host for the network service. Accepts a wildcard.	DARTH*, 10.10.1.100, my_host, etc. Using just a wildcard * will match all hosts.
dest_pci_domain	An optional PCI Domain. Accepts a wildcard.	trust, untrust, etc.
dest_port	The destination port number. Accepts a wildcard.	443, 3389, 5900, etc.
transport	The transport protocol. Accepts a wildcard.	tcp or udp
is_required	Is the service required to be running? Alert if not present.	true or false
is_prohibited	Is the service/traffic/port prohibited from running? Alert if present.	true or false
is_secure	Is the service traffic encrypted?	true or false
note	A brief description of the service and use-case	Unencrypted telnet services are insecure.

Interesting Processes

Interesting Processes contains a list of processes. This list is used to determine whether a process is required, prohibited, and/or secure. Use the List and Lookup editor to modify or add to this list. The Interesting Processes lookup is named interesting_processes.csv.

The following table shows the fields in this file.

Column	Description
app	application name
dest	destination of process
dest_pci_domain	PCI domain, if available
is_required	true or false
is_prohibited	true or false
is_secure	true or false
note	Any additional information about this process

Interesting Services

Interesting Services contains a list of services in your deployment. This list is used to determine whether a service is required, prohibited, and/or secure. Use the List and Lookup editor to modify or add to this list. The Interesting Services is named interesting_services.csv.

The following table shows the fields in this file.

Column	Description
app	application name
dest	destination of process
dest_pci_domain	PCI domain, if available
is_required	true or false
is_prohibited	true or false
is_secure	true or false
note	Any additional information about this process

Primary Functions

Primary Functions contains a list of primary processes and services, and their function in your deployment. Use this list to designate which services are primary and the port and transport to use. The Primary Functions lookup file is named primary_functions.csv.

The following table shows the fields in this file.

Column	Description
process	name of process
service	name of service
dest_pci_domain	PCI domain, if available
transport	tcp or udp
port	port number
is_primary	true or false
function	function of this process (for example, Proxy, Authentication, Database, Domain Name Service (DNS), Web, Mail)

Prohibited Traffic

Prohibited Traffic lists processes that will generate an alert if they are detected. This list is used by the System Center dashboard and is useful for detecting software that is prohibited by the security policy, such as IRC or data destruction tools, or for software that is known to be malicious, such as malware that was recently implicated in an outbreak.

The Prohibited Traffic file is named prohibited_traffic.csv.

The following table shows the fields in this file.

Field	Description
app	The name of the process (such as echo, chargen, etc.)
is_prohibited	Either "true" or "false"
note	A text description of why the process is rejected

Urgency Levels

Urgency Levels contains the combinations of priority and severity that dictate the urgency of notable events. See Notable Event Urgency assignment in this manual.