Write a search for a Flow Model

The Search in the Flow Model scans the event logs, transforms or extracts events based on the specifications of the search, and returns the results. Depending on the structure of your data and the complexity of the process you want to track you might need to combine data sources and create a new field to make a single Step field in your search.

What you need to know before you write a search

Consider the following before you write a search for a Flow Model.

Knowledge of SPL

You must have knowledge of SPL, the Splunk Search Processing Language. If you are not familiar with SPL, see the Search Tutorial.

Understand the process you want to track

Familiarize yourself with your data and determine what field names correspond to the process you want to track. You need to know your Flow Model components: the field names that represent the Correlation IDs, Step, and Attributes in your data. If you do not know the Correlation IDs, Step, and Attributes for your data, see Identify your Flow Model components.

Determine if you need to combine data sources or create a single Step field

Consider the complexity of the process you want to track. If the process spans multiple data sources, you need to write a search that captures the complete process across all data sources. In Splunk Business Flow, there are two ways to combine data sources. You can write a search for a single Flow Model and use a multisearch command to combine data sources. Alternatively, you can create individual Flow models for each data source and use the Combine Flow Models feature to generate a search. For more information, see Combine Flow Models.

The following tutorials walk through how to combine data sources using the multisearch command. The multisearch command runs multiple streaming searches at the same time. The SPL join command does not behave the same way the join command works in SQL. For more information on multisearch syntax, see Multisearch in the Search Reference manual.

Depending on how your data is structured, you might need to rename or create fields. The Step corresponds to the series of actions an item or person takes in the process that you want to track. You can only select one field name for the Step. If you want to track multiple field names in your data, then you need to make a single Step field. The Step field has to be the same field name across all data sources. You might also decide to change field names for Correlation IDs, or Attributes. For example, if you have a string of numbers as a field name, consider renaming the field to something clearer. Use the eval command to create a field.

Write a search

After you familiarize yourself with your data and identify your Flow Model components you can write search.

1. Log into Splunk Enterprise.
2. Click the Splunk Enterprise logo.
3. Open the Search & Reporting app.
4. Specify the index, source or source type, and time range in your search.
   The fewer events you need to process, the more efficient the search. For more information on search optimization, see Quick tips for optimization in the Splunk Enterprise Search Manual.
5. In the Search bar, type the index on which you uploaded the data.
6. In the Search bar, type the source type after the index.
7. (Optional) Combine data sources. ]
   Use a multisearch command to combine data sources. The multisearch command is a generating command that runs multiple streaming searches at the same time. It requires at least two subsearches and allows only streaming operations in each subsearch. The multisearch command must be the first command in the search. Generating commands start with a pipe. The events from each subsearch are interleaved. The results are not appended. The multisearch command does not operate like a join command in SQL. The multisearch command has the following synatx:

   | multisearch [search sourcetype = ....] [search sourcetype = ....]

   For more on the multisearch command, see Multisearch in the Splunk Enterprise Search Reference.
8. (Optional) Create a single Step field.
   If you want to track multiple field names in your data, then you need to make a single Step field. Use the eval command to create a field.

Tutorials

The following tutorials use data from the fictitious Buttercup Games Store. The Buttercup Games Store dataset has three data sources: web-6.txt, order.txt, and call-center.txt. The Getting Started Tutorial uses the same data.

Prerequisite If you did not complete the Getting Started Tutorial, download the Game_store.zip file. Do not uncompress the file. To upload the Game Store data into the Splunk platform, see Upload the tutorial data in the the Getting Started Tutorial.

Write a search for a single data source

Suppose you are a business analyst at the fictitious Buttercup Games Store. The Buttercup Games company launched a coupon campaign for a flash sale. You want to analyze the effectiveness of the coupon campaign and what customers purchase by geographical region. The data spans from July 31 to August 2, 2018.

After familiarizing yourself with the data, you decide that you want to track the actions the customer took on the website and sort events into journeys by individual customer. The customer_id is the Correlation ID. The customer_id is the field name that contains the the unique field values that correspond to each customer. The field name action contains the actions each user executed, such as: Add To Cart, Apply Coupon, and Purchase Game. Therefore, action is the Step. You can select multiple Attributes. For the Attributes, you can use country and product to compare product sales by geographical region. All of these fields are contained in the web-6 source type.

Index	Source type	Correlation ID	Step	Attributes	Time range
tutorial	web-6	customer_id	action	country, product	July 31 to August 2, 2018.

Steps

1. Log into Splunk Enterprise.
2. Click the Splunk Enterprise logo.
3. Open the Search & Reporting app.
4. In the time range picker, click Date Range.
5. Select Between.
6. Type 07/31/2018 and 08/02/2018.
7. In the Search bar, type the index on which you uploaded the data.
   

   index=tutorial

8. In the Search bar, type the source type after the index .

   index=tutorial sourcetype=web-6

   You are only interested in the data on the web-6.txt. The process you want to track does not span multiple data sources, so you do not need to combine data sources with a multisearch command. You have one field name action which corresponds to Step. You do not need to make a single Step field. This search returns all events from the weblogs data source that occurred between July 31st to August 2nd 2018.

Combine multiple data sources and create fields

Continuing with the previous tutorial, suppose you are a business analyst at the Buttercup Games Store. After you analyze the weblog data, you decide to investigate the effect of the flash sale on the call center. You want to discover the roadblocks in the purchase process, track when customers called into support, and examine the results by geographical region.

In this case, you are interested in two source types the: web-6 and call-center. The index is the same for both source types. The time range is from July 31 to August 2, 2018. You want to group events into journeys by individual customer. In both the web-6 and the call-center data the unique identifier for the customer is customer_id. Therefore, the Correlation ID is customer_id. The field name that captures all the steps the customer took on the website in the web-6 weblog data is action. In the call center data the Step field is queue. Select agent_id and country as Attributes to group journeys by geographic location and call center agent. This process spans two data sources. Use a multisearch command to combine data sources. This process has two field names that correspond to Steps: action, and queue. The Step field has to be the same field name across all data sources. You need to create a field for the combined Step.

Index	Source type	Correlation ID	Step	Attributes	Time range
tutorial	web-6, call-center	customer_id	action, queue	agent_id, country	July 31 to August 2, 2018.

Steps

1. Click the Splunk Enterprise logo button.
2. Open the Splunk Search and Reporting app.
3. In the time range picker, click Date Range.
4. Select Between.
5. Type 07/31/2018 and 08/02/2018.
6. To combine the call center and weblog data sources start with a multisearch.
   1. Type the following In the Search bar:
      

      | multisearch

   2. In the Search bar, type the subsearches and indexes after the multisearch command.
      Define your subsearches. Since you have two data sources, you have two subsearches: web-6 and call_center.Use the tutorial index for both data sources.

      | multisearch [index=tutorial sourcetype=web-6][search index=tutorial sourcetype=call_center]

7. Make a single Step field.
   You can have multiple Correlation IDs and Attributes, but only one Step. In this tutorial, you have two field names that correspond to Step: action and queue.The following two sub steps walk through two different ways to create a single Step field. In option A, you create a field with the same name as the original. In option b, you create a field with and rename the field. Choose either option A or B to complete this tutorial.
   1. Use the eval command to make the names of the Step fields the same for the two source types. In this tutorial, the eval command creates a new field called action for the call center values. The values from the queue field are copied into the new action field. The actions that the customer took on the website and in the call center, such as Add-To-Cart and Call Dropped, correspond to the field name action. For more information, see Eval in the Splunk Enterprise Search Reference.

      Command	Search
      eval	| multisearch [search index = tutorial sourcetype = web-6] [search index = tutorial sourcetype = call_center | eval action = queue]

      As a result of this search, the values in the queue field, such as Call Dropped, Call Queued, and Inbound Call, are copied into the action field. All of the values for both the web-6 and call_center are in a field called action. You can now use the action field as the Step.
      

   2. Use the eval command to create a new field and copy the values from the queue and action fields into this new field. The coalesce function groups the field values into the new field.

      Commands	Search
      eval and coalesce	| multisearch[search index=tutorial sourcetype=web-6] [search index=tutorial sourcetype=call_center] | eval newStep=coalesce(action, queue)

      As a result of this search, the field name newStep contains field values from both queue and action.

8. Using the method you selected in the previous step, type the search in the Search bar. The search returns the weblog and call center events that occurred between July 31 to August 2, 2018. The only difference between these searches is the name of the Step field.