Splunk® IT Service Intelligence

Event Analytics Manual

Normalize alerts with correlation search templates in ITSI

IT Service Intelligence (ITSI) ships with several predefined correlation search templates to help you normalize alerts from common third-party systems. Leverage these searches when creating a correlation search to bring third-party alerts into ITSI and normalize them as notable events. For more information about correlation searches, see Overview of correlation searches in ITSI.

Prerequisites

Requirement Description
ITSI role You must have the write_itsi_correlation_search capability to create a correlation search. The itoa_admin and itoa_team_admin ITSI roles have this capabilities by default.
Ingest third-party data You must be ingesting data from the corresponding third-party alerting system into Splunk Enterprise in order to normalize it in ITSI. Optionally, you can install the related Splunk add-on for that system. The table below lists the add-ons related to each search, if available.

Access correlation search templates

All third-party search templates are available within the correlation search creation workflow. To leverage a template, perform the following steps:

  1. From the ITSI main menu, click Configuration > Correlation Searches.
  2. Click Create New Search > Create Correlation Search.
  3. Provide a name and description for the search.
  4. For Search Type, choose Predefined.
  5. Click Select a Search and choose from one of the predefined search templates described below.
  6. Click Select an index and choose an index to use for the search.
  7. Configure the rest of the correlation search to normalize the third-party alert fields. For instructions, see Ingest third-party alerts into ITSI.

Available correlation search templates

Choose from the following correlation search templates to bring third-party alerts into ITSI:

Search name Search Description
BMC TrueSight Events

index=* | eval itsi_host=alias_host | eval tmp_entity=alias_host | eval itsi_eventtype=alias_parameter | eval itsi_class=CLASS | eval itsi_message=Msg | eval itsi_incident=itsm_incident_id | eval itsi_ip=mc_host_address | eval itsi_support=support_group | eval itsi_severity=case(mc_original_severity="CRITICAL", 6, mc_original_severity="OK", 2, 1=1,1) | dedup alias_host alias_parameter Msg

BMC Truesight (patrol, msend) stateful events. Deduplicated by alias_host, alias_parameter, Msg.
MuleSoft Events

index=* sourcetype=httpevent severity!=INFO | dedup source | eval tmp_entity=source | eval itsi_severity=case(severity=="WARN", 3, severity=="ERROR", 5, 1=1,1) | eval itsi_message=message | eval itsi_eventtype=logger

MuleSoft stateful related events, filtering out severity=INFO, deduplicated by source.
Nagios Events

index=* sourcetype=nagiosserviceperf | eval norm_severity=case(severity=="CRITICAL",6,severity=="WARNING",4, severity="OK", 2) | dedup consecutive=true src_host severity name | eval tmp_entity=host | eval host=src_host

Nagios stateful performance events. Filtering by sourcetype=nagiosserviceperf, deduplicated by consecutive, src_host, severity, name.

Add-on: Splunk Add-on for Nagios Core

Netcool Events

index=netcool | eval itsi_host=NODE | eval itsi_eventID=ALERTID | eval itsi_alertKey = ALERTKEY | dedup consecutive=true itsi_host itsi_alertID itsi_alertKey | eval itsi_tally = TALLY | eval itsi_orig_severity = SEVERITY | eval itsi_severity = case(SEVERITY==0,2, SEVERITY==1,2, SEVERITY==2,3,SEVERITY==3, 4, SEVERITY==4, 5, SEVERITY==5, 6, 1=1, 1) | eval itsi_message = SUMMARY | eval tmp_entity = APPLICATION | eval itsi_length=len(_raw) | eval itsi_identifier = IDENTIFIER + LASTOCCURRENCE + STATECHANGE | eval itsi_time=strftime(LASTOCCURRENCE,"%F %T")

Netcool stateful performance events. Deduplicated by consecutive, itsi_host , itsi_alertID , itsi_alertKey.
NewRelic Events

index=* sourcetype=newrelic* | eval tmp_entity=norm_instance | eval itsi_severity=case(health_status=="green", 2, health_status=="red", 6) | search itsi_severity!=2 | dedup transaction_name health_status

New Relic stateful events. Filtering by sourcetype=newrelic*, deduplicated by transaction_name, health_status.

Add-on: Splunk Add-on for New Relic

ScienceLogic em7

index=* | dedup em7_var_evententityname em7_var_alertid | eval itsi_category = em7_var_categoryname | eval itsi_message = em7_var_eventmessage | eval itsi_orig_severity = em7_var_eventseveritytext | eval itsi_host = em7_var_evententityname | eval tmp_entity = em7_var_evententityname | eval itsi_url = em7_var_eventurllink | eval itsi_ip = em7_var_ipaddress | eval itsi_location = em7_var_slsystemname | eval itsi_class = class | eval itsi_eventID = em7_var_alertid | eval itsi_supportGroup = em7_var_support_group | eval itsi_backLink = em7_var_device_back_link | eval itsi_severity=case(em7_var_eventseveritytext=="NOTICE", 3, em7_var_eventseveritytext=="MINOR", 4, em7_var_eventseveritytext=="MAJOR", 5, em7_var_eventseveritytext=="CRITICAL", 6, 1=1,1)

ScienceLogic em7 stateful events. Deduplicated by em7_var_evententityname, em7_var_alertid (used by notable event identifier fields).
SolarWinds Events

index=* | eval itsi_host=NodeName | eval tmp_entity=NodeName | eval itsi_orig_severity=Severity | eval itsi_severity=Severity | eval itsi_alert_type=eventtype | eval itsi_NodeDescription=NodeDescription | eval itsi_vendor=Vendor | eval itsi_summary=StatusDescription | eval itsi_category=tag | eval itsi_location=Location | eval itsi_severity=case(Severity<=500, 2, Severity<=4000, 3, Severity<=9000, 4, Severity<=12000, 5 , Severity>=15000, 6, 1=1, 1) | dedup NodeName eventtype StatusDescription

SolarWinds stateful events, not performance metrics. Deduplicated by NodeName, eventtype, StatusDescription.

Add-on: SolarWinds Add-on for Splunk

Unix or Linux Events

index=* status=Stopped | dedup host status Description | eval itsi_severity=status | eval itsi_host=host | eval itsi_eventtype=eventtype | eval tmp_entity=host | eval itsi_severity=case(status=="Stopped", 4, 1=1,1)

Unix and Linux-based stateful events using the field Status as severity. If clearing events (Up) are being ingested, remove the filter for status=Stopped (clearing events can be used to automatically clear notable events). Deduplicated by host, status, and Description.

Add-on: Splunk Add-on for Unix and Linux

WinEvent:System or WinEvent:Application

index=* sourcetype=WinEventLog severity!=informational | rename event_id AS orig_event_id | eval tmp_entity=host |eval itsi_eventtype=eventtype | eval itsi_message=Message | eval itsi_ip=Source_Network_Address | eval itsi_fqdn=ComputerName | eval itsi_event_category=category | eval itsi_severity=case(severity=="medium", 4, severity=="high", 5, 1=1,1) | dedup host Message orig_event_id

Windows-based stateful events from winevents:system and winevents:application. Filtering out informational events and deduplicated on Message, host, and orig_event_id.

Add-on: Splunk Add-on for Microsoft Windows

AppDynamics

Search name Search Description
Events

index=* source=application_events | eval itsi_application=application_name | eval itsi_customer_name=account_name | eval itsi_application_id=application_id | rename application_events{}.severity as itsi_raw_severity | spath output=itsi_affectedEntities path=application_events{}.affectedEntities{}.name | spath output=itsi_triggeredEntity path=application_events{}.triggeredEntity.name | spath output=itsi_subType path=application_events{}.subType | spath output=itsi_summary path=application_events{}.summary | eval itsi_severity=if(itsi_raw_severity=="MINOR","4",if(itsi_raw_severity=="WARN","3",if(itsi_raw_severity=="NORMAL","2",if(itsi_raw_severity=="ERROR","5","6")))) | dedup itsi_triggeredEntity itsi_application itsi_subType

AppDynamics stateful events based on the ingest events from AppDynamics, using spath to expand the key-value pairs into single fields. Deduplicated by itsi_triggeredEntity, itsi_application, and itsi_subType.

Add-on: Splunk Add-on for AppDynamics

Health Rule Violations

index=* source=healthrule_violations | dedup healthrule_violations{}.affectedEntityDefinition.entityId healthrule_violations{}.deepLinkUrl | spath output=itsi_ruleViolation path=healthrule_violations{}.name | spath output=itsi_affectedEntityDefinition path=healthrule_violations{}.affectedEntityDefinition.name | eval tmp_entity=itsi_affectedEntityDefinition | spath output=event_description path=healthrule_violations{}.description | spath output=severity path=healthrule_violations{}.severity | spath output=itsi_triggered_entity_name path=healthrule_violations{}.triggeredEntityDefinition.name | spath output=itsi_entity_type path=healthrule_violations{}.triggeredEntityDefinition.entityType | eval eventsource="APPD" | eval svc_entity=itsi_affectedEntityDefinition | eval itsi_severity=case(severity == "CRITICAL", 5,severity == "WARNING", 3) | eval tmp_entity=svc_entity

AppDynamics health rule stateful violations based on the ingest of health rule events from AppDynamics, using spath to expand the key-value pairs into single fields. Deduplicated on healthrule_violations{}.affectedEntityDefinition.entityId and healthrule_violations{}.deepLinkUrl.

Add-on: Splunk Add-on for AppDynamics

Last modified on 28 April, 2023
Ingest third-party alerts into ITSI   Ingest SNMP traps into ITSI

This documentation applies to the following versions of Splunk® IT Service Intelligence: 4.11.0, 4.11.1, 4.11.2, 4.11.3, 4.11.4, 4.11.5, 4.11.6, 4.12.0 Cloud only, 4.12.1 Cloud only, 4.12.2 Cloud only, 4.13.0, 4.13.1, 4.13.2, 4.13.3, 4.14.0 Cloud only, 4.14.1 Cloud only, 4.14.2 Cloud only, 4.15.0, 4.15.1, 4.15.2, 4.15.3, 4.16.0 Cloud only, 4.17.0, 4.17.1, 4.18.0, 4.18.1, 4.19.0, 4.19.1


Was this topic useful?







You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters