Create and manage search-driven lookups in Splunk Enterprise Security

A search-driven lookup lets you create a lookup based on the results of a search that runs at regular scheduled intervals. The search can run only against data stored in data models or in an existing lookup. Lookups created as search-driven lookups are excluded from bundle replication and are not sent to the indexers.

When to use search-driven lookups

Create a search-driven lookup if you want to know when something new happens in your environment, or need to consistently update a lookup based on changing information from a data model or another lookup.

The search-driven lookup collects and stores information from data models or other lookups. The data stored in the lookup represents a historical summary of selected fields gathered from events. You can view changes on a dashboard or use a correlation search to compare data from the search-driven lookup with new events, and alert if there is a match. For example, to find out when a new user logs in to a web server.

1. Search for user data in the Authentication data model and filter by the web server host name with the where command.
2. Verify the search results match the known hosts and users in your environment.
3. Create a guided search-driven lookup to collect and store information on a recurring schedule about users logging in to the web servers.
4. Create a correlation search that alerts you when a user logs in to one of the web servers that he or she has not accessed in the past, based on the historical information in the search-driven lookup.

Create a search-driven lookup

Create a search-driven lookup.

1. From the Splunk Enterprise Security menu bar, select Configure > Content> Content Management.
2. Click Create New Content and select Search-Driven Lookup.
3. (Optional) Select an App. The default app is SplunkEnterpriseSecuritySuite. You can create the lookup in a specific app, such as SA-NetworkProtection, or a custom app. You cannot change the app after you save the search-driven lookup.
4. (Optional) Type a description for the search.
5. Type a label for the lookup. This is the name of the search-driven lookup that appears on Content Management.
6. Type a name for the lookup. After you save the lookup, the name cannot be changed.
7. Type a cron schedule to define how often you want the search to run.
8. Select real-time or continuous scheduling for the search. Real-time scheduling prioritizes search performance, while continuous scheduling prioritizes data integrity.
9. Type a Search Name to define the name of the saved search. After you save the lookup, the name cannot be changed.
10. Select a mode of Guided to create a search without having to write the search syntax yourself, or select Manual to write your own search. See the example for help building a search with the guided search editor.
11. If you create a search in manual mode, type a search.
12. Click Save to save the search.

Example search-driven lookup

In this example search-driven lookup included with Splunk Enterprise Security, you want to track attacks identified by your intrusion detection system (IDS). You can then be notified of new attacks with a correlation search, or determine whether an attack is new to your environment or not. The Intrusion Center dashboard uses this search-driven lookup for the New Attacks - Last 30 Days panel. See Intrusion Center dashboard.

1. From the Splunk Enterprise Security menu bar, select Configure > Content > Content Management.
2. Click Create New Content and select Search-Driven Lookup.
3. (Optional) Select an App of SA-NetworkProtection. You cannot change the app after you save the search-driven lookup.
4. Type a description of "Maintains a list of attacks identified by an IDS and the first and last time that the attacks were seen."
5. Type a label of IDS Attack Tracker Example for the lookup. This is the name of the search-driven lookup that appears on Content Management.
6. Type a unique and descriptive name for the lookup of ids_attack_tracker_example. After you save the lookup, the name cannot be changed.
7. Type a cron schedule to define how often you want the search to run. If your IDS collects data often, type a cron schedule of 25 * * * * to run the search at 25 minutes every hour every day.
8. Select a Continuous Schedule because the lookup must track all data points.
9. Type a Search Name of Network - IDS Attack Tracker - Example Lookup Gen.
10. Select guided mode to use the guided search editor to create the search.
11. Click Open guided search editor to start creating the search.
12. Select a data source of Data Model because the IDS Attack data is stored in a data model.
13. Select a data model of Intrusion_Detection and a data model dataset of IDS_Attacks.
14. Select Yes for the summaries only field to run the search against only the data in the accelerated data model.
15. Select a time range that uses Relative time that begins with an earliest time of 70 minutes ago, starting at the beginning of the minute, and ends now. Click Apply to save the time range.
16. Click Next.
17. (Optional) Type a where clause to filter the data from the data model to only the data from a specific IDS vendor and click Next.
18. Add aggregate values to track specific statistics about the data and store that information in the lookup. At least one aggregate is required.
    1. To track the first time that an IDS attack was seen in your environment, add a new aggregate with a function of min and a field of _time and save it as firstTime.
    2. Track the last time an attack was seen by adding another aggregate with a max function and a field of _time and saving it as lastTime. This creates two columns in the lookup, firstTime and lastTime.
19. Add split-by clauses to track more data points in the lookup. All split-by clauses appear as columns in the lookup.
    1. Add a split-by clause of IDS_Attacks.ids_type and rename it as ids_type to monitor the IDS type in the lookup.
    2. Add a split-by clause to rename IDS_Attacks.signature as signature.
    3. Add a split-by clause to rename IDS_Attacks.vendor_product as vendor_product.
20. Click Next.
21. Select a retention period that defines the age of the data to be stored in the lookup. For example, you want to keep 5 years of IDS attack evidence stored in this lookup. Select a time field of lastTime to base the retention on the last time an attack was identified by the IDS. Type an earliest time of -5y and indicate the format of the time value that you entered: %s. You can find guidance on the time format in the Splunk platform documentation.
• For Splunk Enterprise, see Date and time format variables in the Splunk Enterprise Search Reference manual.
• For Splunk Cloud, see Date and time format variables in the Splunk Cloud Search Reference manual.




22. Click Next.
23. Review the search created by the wizard and click Done to finish using the guided search editor.
24. Click Save to save the search.

Modify a search-driven lookup

1. From the Splunk Enterprise Security menu bar, select Configure > Content > Content Management.
2. Select a Type of Search-Driven Lookup.
3. Click the lookup that you want to edit.
4. Make changes and click Save.

Enable or disable the search populating a search-driven lookup

You can enable or disable the search of a search-driven lookup to prevent the search from updating the lookup. If you disable the search that populates a search-driven lookup, the search stops updating the lookup and the data in the lookup will stop being updated. Correlation searches or dashboards that rely on the data inside the lookup will be out-of-date.

1. Select Configure > Content > Content Management.
2. Filter on a type of search-driven lookup and open the search-driven lookup that you want to enable or disable.
3. Find the Search name of the search-driven lookup.
4. From the Splunk platform menu bar, select Settings > Searches, reports, alerts.
5. Find the search and enable or disable it.