Splunk® Content Packs for ITSI and IT Essentials Work

Splunk Content Packs for ITSI and IT Essentials Work

Acrobat logo Download manual as PDF


Acrobat logo Download topic as PDF

Configure the Content Pack for Alert Routing

To configure the Content Pack for Alert Routing follow steps one through four. To optionally adjust the default alert configurations of the content pack, go to Customize the default configurations.

Step 1: Install prerequisite apps

Some of the supported actions in this content pack depend on the installation and configuration of specific apps available on Splunkbase. Install the prerequisite apps for the actions you want to take before configuring this content pack.

Step 2: Configure the search macro

Provide the base URI for the Splunk search head that runs IT Essentials Work for the within the itew_get_splunk_base_uri search macro. This allows for quick navigation from the alert to the entity health in IT Essentials Work. You have to update this macro with the correct base URI. Follow these steps to do so:

  1. In Splunk Enterprise go to Settings > Advanced Search > Search macros.
  2. In the App dropdown select IT Essentials Work Alerting Content (DA-ITSI-CP-itew-alerting-content).
  3. Select the itew_get_splunk_base_uri search macro.
  4. Enter the base URI for the Splunk search head that runs IT Essentials Work in the Definition field. The URI has to be surrounded by double quotes as in the example.
    "Screenshot of alert routing macro description field where base URI for the Splunk search head is specified."

Step 3: Configure alert routing rules on the entity

The content pack determines the appropriate external alert actions from several new information fields you define on each entity. To begin configuring alert actions follow these steps:

  1. Log in to Splunk as an admin.
  2. In the IT Essentials Work app, go to Configuration > Entities.
  3. Select the entity you want to configure. If you want to add information fields to multiple entities, select each entity and then select Bulk Action > Edit selected.
  4. Add necessary info fields to configure the alert routing.
    "Screenshot of entity info fields where alter routing fields are specified"
    Info field name Supported values Required Single-value example Multi-value example
    alert_routing Email, On-Call, ServiceNow, Custom Yes Email Email, ServiceNow
    alert_email Valid email address Yes, when Email is set for alert routing pony@buttercup.com pony@buttercup.com, horse@buttercup.com
    alert_oncall_routing_key Valid Splunk On-Call routing key Yes, when On-Call is set for alert_routing splunk-team NA
    alert_snow_assignment_group Valid ServiceNow assignment group Yes, when ServiceNow is set for alert_routing splunk-team NA
    alert_custom_params Any valid value supported by your custom action No


Step 4: Enable alert searches

The final step is to enable one or more IT Essentials Work Alert Action Generator searches. Follow these steps to do so:

  1. In Splunk Enterprise go to Settings > Searches, reports, and alerts.
  2. Select Type: Alert and App: All. Then enter "IT Essentials Work" in the filter field.
  3. Select Edit > Enable for each alert action search you want to enable. You only need to enable the searches that correspond to the alert_routing actions you configured on the entities.
    "Screenshot of alert action generator searches on the Searches, reports, and alerts page."
  4. If you enable the IT Essentials Work - Splunk OnCall Alert Action Generator search and intend to send alerts to Splunk On-Call, you have to edit the alert action within that search to provide the appropriate Splunk On-Call API key.
    "Screenshot of the Splunk On-Call (VictorOps) section of Edit Alert screen highlighting the API Key that has to be selected for Splunk On-Call alerts."
  5. If you enable the IT Essentials Work - ServiceNow Alert Action Generator search and intend to send incidents to ServiceNow, you have to edit the alert action within that search to provide your ServiceNow Account.
    "Screenshot of the ServiceNow section of Edit Alert screen highlighting the account dropdown that has to be selected for ServiceNow alerts."

Customize the default configurations

Configure alternate alert routing for a specific vital metric alert

By default, the alert routing configured on the entity is used when an alert is triggered for that entity; however, you might want to configure an alternate alert route for a specific alert. For example, you might want to route all alerts for a disk-space vital metric alert to the storage team regardless of the entity that triggered the alert.

To configure alternate alert routing for a specific vital metric alert, follow these steps from the IT Essentials Work menu:

  1. Go to Configuration > Entities.
  2. Select the Entity types tab and edit the entity type for the alert you want to modify.
  3. Open the Vital Metrics section.
  4. Select the vital metric you want to modify.
  5. At the end of the vital metric search, use Splunk eval commands to set a new alert_routing value and any additional routing configuration fields as shown in the example.

    Example eval command:
    | eval alert_routing="Email",alert_email="storage-team@buttercup.com
    

    "Screenshot of an eval command added to the vital metric search on an entity type"

Configure alternate alert routing based on other conditions

By default, the alert routing configured on the entity is used when an alert is triggered for that entity however, you might want to configure alternate alert routes based on more complex conditions. For example, you might want to alert via email if the severity is warning level, and alert via email and create a ServiceNow ticket if the severity is critical.

To configure alternate alert routing based on other conditions, you have to modify or enhance the SPL of the alert search in the content pack. So, you have to have a basic understanding of SPL syntax.

To modify the search macro, follow these steps:

  1. In Splunk Enterprise go to Settings > Advanced Search > Search macros.
  2. In the App dropdown select IT Essentials Work Alerting Content (DA-ITSI-CP-itew-alerting-content).
  3. Select the lookup_entity_contact_details search macro..
  4. Add an eval command to the end of the existing SPL to set the alert_routing configurations you want. In the example below, the alert_routing is changed to email only when the alert severity isn't critical.

    Example eval command:
    | eval alert_routing=if(severity<6, "Email", alert_routing)
    


    "Screenshot of an eval command added to the lookup search macro"

Configure alternate alert action parameters

The content pack includes a variety of default parameters including scheduling frequency, throttling behavior, external action text, and more. While these parameters were carefully chosen to provide a solid implementation without any changes, you might find it useful or necessary to modify the alert action configuration to suit your organization's needs. You can do this safely because any changes you make are stored in the local folder so subsequent content pack upgrades won't overwrite local changes.

To alter the default alert action configurations and parameters, follow these steps:

  1. In Splunk Enterprise go to Settings > Searches, reports, and alerts.
  2. Select Type: Alert and App: All. Then enter "IT Essentials Work" in the filter field.
    "Screenshot of Searches, Reports, and Alerts page that shows the alert action generator searches"
  3. Find the IT Essentials Work - Alert Action Generator search for the action you want to change and edit it accordingly. For example, you might want to change the ServiceNow Correlation ID parameter to better suit your ServiceNow environment.
    "Screenshot of Edit alert screen with the ServiceNow Correlation ID field highlighted"
Last modified on 04 October, 2021
PREVIOUS
Install the Content Pack for Alert Routing
  NEXT
About the Content Pack for Amazon Web Services Dashboards and Reports

This documentation applies to the following versions of Splunk® Content Packs for ITSI and IT Essentials Work: current


Was this documentation topic helpful?

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters