Install and configure the Content Pack for Monitoring Splunk as a Service

The Splunk App for Content Packs allows you to access content packs, preview their contents, and install them in your environment. The Splunk App for Content Packs includes the Content Pack for Monitoring Splunk as a Service provided you are using ITSI version 4.9.0 or higher. If you are using ITSI version 4.8.x or lower, then you must install the content pack using backup and restore functionality provided by ITSI.

For a full list of the objects shipped in this content pack, see About the the Content Pack for Monitoring Splunk as a Service.

Installation and configuration overview

Follow these high-level steps to configure the Content Pack for Monitoring Splunk as a Service:

1. Install the Content Pack for Monitoring Splunk as a Service.
2. Remove unnecessary services.
3. Import or update entities.
4. Test the configuration.
5. (Optional) Configure alerting.
6. Tune KPI thresholds.

Installation considerations

The Content Pack for Monitoring Splunk as a Service doesn't have team restrictions and is therefore available to the Global team. If you want to limit visibility to the content pack, create a new ITSI team and associate the services to the newly created team. For more information about teams, see Overview of teams in ITSI in the Service Insights manual.

All services and service templates in this content pack begin with the name Splunk and base searches are prefaced with SPLK for ease of filtering.

Prerequisites

• Install and configure the IT Service Intelligence (ITSI). See About Splunk ITSI in the Install and Upgrade Manual.
• Take a full backup of your ITSI environment in case you need to uninstall the content pack later. For more information, see Create a full backup of ITSI in the Administration Manual.

Install the Content Pack for Monitoring Splunk as a Service

You have two options for installing and configuring the Content Pack for Monitoring Splunk as a Service;

• One option is to install the content pack from the Splunk App for Content Packs. The Content Pack for Monitoring Splunk as a Service is included in the Splunk App for Content Packs if you are using ITSI version 4.9.x.
• Your second option is to install the content pack using backup and restore functionality provided by ITSI. You must choose this option if you are using ITSI version 4.8.x or lower.

Install the content pack from the Splunk App for Content Packs

To install the Content Pack for Monitoring Splunk as a Service, you must install the Splunk App for Content Packs. To install the Splunk App for Content Packs in your environment, see the Splunk App for Content Pack installation instructions.

After you install the Splunk App for Content Packs, follow these steps to configure the Content Pack for Monitoring Splunk as a Service:

1. From the ITSI main navigation bar, click Configuration > Data Integrations.
2. Select Add content packs or Add structure to your data depending on your version of ITSI.
3. Select the Monitoring Splunk as a Service content pack.
4. Review what's included in the content pack and click Proceed.
5. Configure the content pack settings. Refer to the table that follows these steps for setting descriptions.
6. When you've completed your selections, click Install selected.
7. Click Install to confirm the installation. When the installation completes you can view all objects that were successfully installed in your environment. A green checkmark on the Data Integrations page shows any other content packs you have installed.

Refer to the following table for content pack setting option descriptions:

Content pack setting	Description
Choose which objects to install	For a first-time installation, select the items you want to install and deselect any you do not want to install. For an upgrade installation, the installer identifies which objects from the latest version of the content pack are new and which objects are currently in your environment. You can choose which objects to install from the new version.
Choose a conflict resolution rule for the objects you install: Install as new	For upgrades or subsequent installs, decide what happens to duplicate objects introduced from the content pack. When you install as new, objects are installed and any existing identical objects in your environment remain intact.
Choose a conflict resolution rule for the objects you install: Replace existing	For upgrades or subsequent installs, decide what happens to duplicate objects introduced from the content pack. When you replace existing, any existing identical objects are replaced with those from the new installation. Any changes you previously made to these objects are overwritten.
Import as enabled	Select whether to install objects as enabled or to leave them in their original state. Importing objects as disabled ensures your environment is not impacted from the addition of new content. This setting only applies to services, correlation searches, and aggregation policies. All other objects such as KPI base searches and saved searches are installed in their original state regardless of the option you choose.
Add a prefix to new objects	Optionally, append a custom prefix to each object installed from the content pack. For example, you can prefix your objects with CP- to indicate they came from a content pack. This option can help you locate and manage the objects after installation.
Backfill service KPIs	Optionally backfill your ITSI environment with the previous seven days of KPI data. Enable backfill if you want to configure adaptive thresholding and predictive analytics for the new services. This setting only applies to KPIs, not service health scores.

Install the content pack using backup and restore functionality provided by ITSI

If you are using ITSI version 4.8.x or lower, follow these steps to install the Content Pack for Monitoring Splunk as a Service. For instructions on restoring a backup, see Restore from a backup zip in the Administration Manual.

1. Download the following ITSI backup file: BACKUP-CP-SPLUNK-2.0.1.zip.
2. On your ITSI search head, create a restore job and upload the backup file. Give the job the same name as the backup file you downloaded. For example, BACKUP-CP-SPLUNK-2.0.1
3. After the restore job completes, confirm that the objects included in the content pack are restored to your environment. For a full list of the objects shipped in this content pack, see About the Content Pack for Monitoring Splunk as a Service.

Remove unnecessary services

Not all deployed environments have the full assortment of services represented in this content pack. After you install the content pack, go to Configuration > Services within ITSI and delete any services that don't exist in your environment.

Import or update entities

To model your Splunk architecture, you need to import each piece of your Splunk infrastructure into ITSI as an individual entity. All services included in this content pack rely on the corresponding entity having an entity information field called splunk_role. You need to include this field when you import your Splunk infrastructure. If your entities already exist, you need to add this field by re-importing them.

To create your Splunk entities, go to Configuratiion > Entities from the ITSI main menu. You can create entities manually for a smaller environment, or bulk import them from a CSV or Splunk search in a larger environment.

If using the bulk import method, perform the following steps:

1. Create a Splunk table or a CSV file with the columns host and splunk_role.
2. For each Splunk host you want to import, assign a role in the splunk_role column for the the component of your Splunk architecture it corresponds to. This table lists the the splunk_role value needed for each type of host in order for it to be mapped to a service in the service tree.

   Instance type	splunk_role
   ITSI instances	itsi
   Indexers	indexer
   Cluster managers	indexer_cluster_master
   Search heads	search_head
   Clustered search heads	search_head_cluster
   Deployment servers	deployment_server
   License masters	license_master
   Enterprise Security instances	enterprise_security
   Heavy forwarders	heavy_forwarder
   Management Console	mgmt_console

3. From the ITSI main menu, click Configuration > Entities.
4. Click Create entity and import your entities using one of the following methods:
   • Import entities from a CSV file in ITSI
   • Import entities from a search in ITSI

   In the Import Column As step of the import, configure the following column types:

   Column	Column type
   host	Entity Title
   splunk_role	Entity Information Field

Test the configuration

After you install the content pack and import your entities, make sure everything is set up correctly. Click Service Analyzer from the ITSI main menu and confirm you see a service tree similar to the following:

(Optional) Configure alerting

The Content Pack for Monitoring Splunk as a Service is designed to integrate with the Content Pack for Monitoring and Alerting. It's a best practice to use the Content Pack for Monitoring and Alerting to receive proactive notifications on service, entity, and KPI degradations. For installation and configuration steps, see Install and configure the Content Pack for Monitoring and Alerting.

Tune KPI thresholds

You need to configure thresholds for the KPIs included in this content pack based on your Splunk environment. Some KPIs, such as basic performance counters like CPU utilization, have universal best practices for threshold configuration. Others, like the number of forwarders, are very specific to your deployment. For instructions to configure KPI thresholds, see Configure KPI thresholds in ITSI in the Service Insights manual.

Consider leveraging machine learning algorithms to determine environment-specific thresholds. For instructions to configure time-based or adaptive KPI thresholds, see Overview of advanced thresholding in ITSI in the Service Insights manual. If you perform advanced thresholding on your KPIs, make sure to revisit them after a week to make sure the learning period is representative of a typical healthy week.