Ingest and normalize third-party alerts into ITSI

Onboard alerts from Splunk Cloud Platform or other third-party sources by creating a data integration connection in ITSI. A data integration connection normalizes raw event data in order to bring that data into ITSI Event Analytics. Follow the assisted workflow to configure and create a data integration connection.

Prerequisites

You must have the correct permissions to create a data integration connection:

Role	Permissions
itoa_admin, itoa_team_admin	Create, update, and delete a connection Activate and deactivate a connection Preview results for the field mappings defined for a connection View all the connections available for a data integration source (i.e. Nagios) View list of data integration connections on the Deployed Integrations section
itoa_analyst	View all the connections available for a data integration source (i.e. Nagios) View list of data integration connections on the Deployed Integrations section

Create a data integration connection

1. From the ITSI navigation menu, select Configuration then Data Integrations.
2. Select one of the data integration categorized as an Alerts integration.
3. Select Add connection to add a new data connection for that specific data source.
4. Follow the guided steps in the workflow to map your field data to the correct values and generate a data integration connection.

Select data ingest method

1. Set a title for your data integration connection. This title can't be changed after you create the connection.
2. Select a data Ingest method depending on where your data is stored:
   • Indexed data (Splunk, add-on): raw alerts exist in a Splunk index.
   • HTTP request (Webhook): alerts exist in a third-party source and will be ingested using the relevant Splunk add-on.
3. If you select indexed data, set a lookback period to provide a time range for the search to find data. The search must return at least one result.

Ingest data into Splunk

You can ingest your data using the HTTP Event Collector in Splunk. For more information, see Set up and use HTTP Event Collector in Splunk Web. Format your payload using the universal correlation search format:

{
  "event": {
    "title": "<title>",
    "description": "<description>",
    "owner": "<owner>",
    "status": "<status>",
    "app": "<app>",
    "signature": "<signature>",
    "src": "<source>",
    "subcomponent": "<subcomponent>",
    "vendor_severity": "<vendor_severity>",
    "itsiDrilldownEarliestOffset": "<itsiDrilldownEarliestOffset>",
    "itsiDrilldownLatestOffset": "<itsiDrilldownLatestOffset>",
    "itsiDrilldownSearch": "<itsiDrilldownSearch>",
    "itsiDrilldownWebName": "<itsiDrilldownWebName>",
    "itsiDrilldownWebURL": "<itsiDrilldownWebURL>",
    "itsi_instruction": "<itsi_instruction>"
  }
}

Map data fields for ingest and configuration

1. Input the fields from your data integration source that will map to the Splunk Common Information Model (CIM). CIM is a shared semantic model focused on extracting value from data, and ensures that your data is normalized and can integrate smoothly with ITSI Event Analytics.
   Select one of the following transformation options for each required field:
   • Composition: input multiple fields to map to the property. Select one or more fields and/or one or more text strings.
   • Mapping rule: select either the value case mapping, or the coalesce option. Value case mapping sets conditional rules for field mapping. Coalesce normalizes field names with the same value and takes the first non-null value to combine. You must set a default value to apply to the field in case these mapping rules don't return any values.
   • Regex: apply a regex expression to extract data from the field.

   Note: The default field map configuration is populated from the itsi_data_integration_template.conf file.

2. Set a transformation option and value for each of the following fields:

   Field	Description
   Src	The host or source of your alert data.
   Vendor severity	The original vendor-specific severity or health status string for this alert. For example, critical or warning.
   Severity ID	The numeric or vendor-specific severity indicator corresponding to the event severity. For ITSI, severity_id is one of the following values: 1 = Info or Unknown 2 = Normal or Cleared 3 = Low 4 = Medium 5 = High 6 = Critical
   Title	The title of the notable event in Episode Review. For example, mysql-01 server cpu Load %.
   Owner	The ITSI role to which the notable event is assigned in Episode Review. If using advanced mode, the value must resolve to a username in the system.
   Status	The triage status to display in Episode Review. For example, New.

3. (Optional) Configure the additional field values. For descriptions for each field value, refer to the ITSI Normalization documentation.

Schedule

1. Configure the schedule for the data integration connection:
   • Basic: Schedule searches to run at regular intervals. Configure the search interval in the Run Every menu.
   • Cron: Schedule searches to run periodically at fixed times, dates, or intervals. Enter a schedule in Cron Schedule. For more information, see Use cron expressions for scheduling in the Alerting Manual.
2. Identify the service(s) impacted by the alerts generated from this data integration, and set the entity lookup field used to look up corresponding entities. For example, host.

   Field	Description	Defaults
   Service	Select one or more ITSI services to which this correlation search applies. You can only select services that belong to teams for which you have read access.	None
   Entity Lookup Field	The field in the data retrieved by the correlation search that is used to look up corresponding entities. For example, host.	None

Throttling

A data integration connection search can create multiple alerts for the same condition. In most cases, it is best to have a single alert for the same condition for a specific duration. Throttling blocks the search from creating duplicate alerts for the same issue each time the search runs.

Configure the following fields to suppress alerts:

Field	Description
Fields to group by	Fields to compare to identify similar events. For example, cpu_load_percent. If a field matches this field during the suppress period, a new alert won't be created. You can define multiple fields. The fields available depend on the search fields that the correlation search returns.
Suppress period	During the suppress period, any additional event that matches any of the Fields to group by does not create a new alert. After the suppress period passes, the next matching event creates a new alert, and throttling conditions resume. Enter a relative time range in seconds. For example, 60s (60 seconds).

Preview and save data integration connection

1. Select Preview results in the Transformed field section to preview the results of your field mappings, and view the notable events created based on your mapping rules. Additionally, you can search for fields in the Fields section.
2. When you finish configuring your data integration connection, select Save.

View data integration connections

To see a list of all configured data integration connections in ITSI, select the Deployed integrations tab from the Data Integrations page. Alternatively, select the specific data source from the page to view the connections specific to that data source. You can deactivate, clone, or delete your connection after you create it.

You can also view your new data integration connection listed on the Searches, Reports, and Alerts page. These saved searches are prefixed with DATA_INTEGRATION_CS-.

Note: When you update a data connection from ITSI, the corresponding saved search is automatically updated. However, changes made to the saved search from the Searches, Reports, and Alerts page won't be reflected.

Next steps

You can filter by events on the ITSI Episode Review page based on your data connection settings. Additionally, when you select an episode you will see the impacted services and KPIs based on your configuration settings. View the other common fields configured for each episode on the Common Fields tab on the Episode Review page.