Splunk® IT Service Intelligence

Administration Manual

Acrobat logo Download manual as PDF

Splunk IT Service Intelligence version 4.4.x will no longer be supported as of October 22, 2021. See the Splunk Software Support Policy for details. For information about upgrading to a supported version, see Before you upgrade IT Service Intelligence.
This documentation does not apply to the most recent version of Splunk® IT Service Intelligence. Click here for the latest version.
Acrobat logo Download topic as PDF

Create a custom aggregation policy in ITSI

You can create one or more customized notable event aggregation policies. By default, users assigned the itoa_admin role or itoa_team_admin role can create, modify, and delete aggregation policies. Users assigned the itoa_analyst role can view notable event aggregation policies but cannot modify or delete them.

If you want to use Smart Mode in a custom aggregation policy, you must first create and save the custom aggregation policy. After you save the policy, re-open it to enable and configure Smart Mode.

What aggregation policies do

Notable event aggregation polices help you group notable events into episodes to organize them in Episode Review in the following ways:

  1. Group notable events into episodes based on the following criteria:
      • A field matches a string
      • A field does not match a string
      • A field is greater than a number
      • A field is greater than or equal to a number
      • A field is less than a number
      • A field is less than or equal to a number
  2. Split events into multiple episodes by one or more fields, such as source or host.
  3. Stop grouping when certain criteria are met, such as when a certain number of events is reached, the episode has existed for a certain amount of time, or when a certain event occurs, such as Severity matches Normal.
  4. Take automatic actions on an episode when certain trigger conditions are met. Note that not all actions may be available if role-based permissions are set. For information about episode actions, see Take action on an episode in ITSI.

Create an aggregation policy

When creating a new policy, be mindful of the policies that have already been created. Multiple aggregation policies can be created with filtering criteria that capture the same set or subset of events. If the same notable events are captured by more than one policy, the action rules from each policy are applied to the notable events.

To create an aggregation policy:

  1. Click Configure > Notable Event Aggregation Policies. The default policy and any other policies that have been created are listed.
  2. Click Create Notable Event Aggregation Policy. The Filtering Criteria step opens.

Filtering Criteria

You can create rules for determining which notable events are included in the group based on multiple AND/OR conditions. You can use wildcard characters in fields. You must specify at least one rule.

The field names in original events that conflict in the itsi_tracked_alerts index are renamed with the prefix orig_*. To filter or split by fields in original events, append orig_ to the field name. For example, use orig_sourcetype instead of sourcetype. sourcetype means the sourcetype of generated notable events in the itsi_tracked_alerts index, rather than the sourcetype of original events.

Field Description
Include the events if Determine when an event is added to this episode. For example, you could specify severity matches Critical to include all critical events in the episode. To include events based on a saved search, type search_name matches <name of saved search>.

You can match on any fields in the event. The syntax for the field value does not follow standard SPL. The value must exactly match the field value in the event, including capitalization and spaces. You can use asterisks (*) as wildcards (for example, location does not match webserver*

Click + Add Rule (AND) to add another rule to the rule block, or + Add Rule (OR) to start a new block of rules that is separated from the previous block by an OR condition. For example, if there are three rules in the first block and two rules in the second block, the total clause would read "(Rule1 AND Rule2 AND Rule3) OR (Rule1 AND Rule2)".

Split events by field Split events into separate groups based on one or more raw notable event field names. Each field is considered separately. For example, if you split by name and norm_severity and there are three values for each field, nine episodes are created. A preview shows the events that meet the filtering criteria grouped by the specified fields.
Break episode Add breaking conditions to stop grouping. If the conditions are met, the current episode ends and a new episode is started. Use wildcards to search for a string in a field.

For example, you might enter Break episode if the following event occurs: message matches *status Normal. The breaking event must be included in the episode by meeting the filtering criteria for the episode.

The maximum value for The flow of events into the episode paused for is 86400 seconds (24 hours).

When an episode is closed, either in Episode Review or through an aggregation policy, it automatically breaks.

Episode Information Specify how you want episode information to appear in Episode Review. This information is at the episode level and is separate from the information for the underlying notable events in the episode.
  • Same as the first event sets the episode fields to the values from the first event in the episode.
  • Same as the last event continually updates the episode fields as each new event enters the episode.
  • Static value lets you provide specific field values. If you select Static value for Episode Title or Episode Description, you can use a token such as %title% or %description% to insert the value of a field.

If there are multiple events in an episode with the same timestamp, the policy sorts them by milliseconds to determine the first and last events in an episode.

Changing a policy's episode information fields breaks all active episodes associated with that aggregation policy and starts a new episode. Make sure to carefully plan the timing of changes to these fields to mitigate potential risks associated with this behavior.

Action Rules

Add action rules to take specific automated actions on each episode created by the aggregation policy. Action rules are optional. You can define more than one action rule per aggregation policy.

  • Use the If pane to specify the trigger conditions for the action(s).
  • Use the Then pane to specify the actions to take if the trigger conditions are met.

For example, if you want to close the episode and change the severity level to Info when a clearing event comes in, you could specify the following rule:

If the following event occurs: severity matches Normal, change severity to Info for the episode, add a comment Don't worry for the episode, and change status to Closed for the episode.

Choose from the following default trigger conditions:

Trigger condition Notes
The following event occurs Trigger an action when an event containing specific field values is added to the episode. For example, when severity matches Critical. You can match on any fields in the event.

The syntax for the field value does not follow standard SPL. The value must exactly match the field value in the event, including capitalization and spaces. You can use asterisks (*) as wildcards (for example, service_name does not match BG_*

The episode existed for Trigger an action if the episode has existed for a given amount of time, in seconds. You might close an episode if it's existed for 86400 seconds (one day).
The number of events in the episode is Trigger an action if the number of events in the episode exceeds or does not reach a certain limit. By default, an episode breaks when it reaches 10,000 events, and a new episode is created.
The flow of events into the episode paused for Trigger an action if the episode hasn't received any events for a given amount of time, in seconds.

The maximum value for this field is 86400 seconds (24 hours).

Choose from the following default actions:

Action Notes
Change severity For example, change the severity to Critical when the number of events in the episode is greater than 100.
Change status For descriptions of each status, see Update the status of an episode.

Changing an episode's status to Closed through an aggregation policy action rule also breaks the episode so it no longer receives events.

Change owner Episodes are unassigned by default.
Add a comment Does not accept token replacement.
Ping a host Provide the event field that contains the host that you want to ping in the Host field. For example, %server%.
Send an email For information on configuring an email, see Send an email in the ITSI User Manual.
Run a script Provide the file name of the script stored in $SPLUNK_HOME/bin/scripts. For more information, see Configure scripted alerts in the Splunk Enterprise Alerting Manual.

Note: The run a script functionality is officially deprecated and will be removed in a future release. It will be replaced with custom alert actions as a more scalable and robust framework for integrating custom actions. To learn how to migrate existing alert action scripts to the custom alert action framework, see Convert a script alert action to a custom alert action.

Create an incident in ServiceNow Requires the Splunk Add-on for ServiceNow. For configuration information, see Create a ticket in ServiceNow or Remedy
Create an incident in Remedy Requires the Splunk Add-on for Remedy. For configuration information, see Create a ticket in ServiceNow or Remedy.
Create an incident in VictorOps Requires VictorOps For Splunk. For configuration information, see Create a ticket in VictorOps.

Policy Info

Specify a title and description for your notable event aggregation policy. The policy is enabled by default and immediately takes effect. Disable it if you do not want it to take effect yet.

Click Next and you will see the message that the policy has been successfully created. After the policy has been created, new incoming notable events are grouped in Episode Review according to the criteria in the policy. Note that notable events that existed before the policy was created are not grouped by the policy.

Set aggregation policy permissions

After you create a notable event aggregation policy, set read/write permissions for it to control who can modify or read it.

  1. Click Configure > Notable Event Aggregation Policies.
  2. Locate the policy and click Edit > Edit Permissions.
  3. Assign read or write permissions to roles as desired.
  4. You cannot assign write permissions unless the role possesses the write_itsi_notable_aggregation_policy capability.

  5. Click Save.
Last modified on 08 April, 2020
About the default aggregation policy in ITSI
Group similar events with Smart Mode in ITSI

This documentation applies to the following versions of Splunk® IT Service Intelligence: 4.4.1, 4.4.2, 4.4.3, 4.4.4, 4.4.5

Was this documentation topic helpful?

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters