Install the Splunk App for Infrastructure in a distributed deployment

To complete this task, you must be an administrator familiar with clustered environments on Splunk Enterprise.

You can deploy the Splunk App for Infrastructure in your distributed deployment of Splunk Enterprise. To do so, you need to complete these steps:

• Install SAI on the search head tier.
• Install the Splunk Add-on for Infrastructure on the indexer tier.
• Configure an HTTP Event Collector (HEC) and set up a load balancer to send HTTP traffic from each system to the indexer tier.
• If you're collecting AWS data, deploy or configure a heavy forwarder to handle AWS data collection.
• Set up dependencies for sending and receiving data.

After you complete these steps, start collecting data from systems to monitor in the app. You can run the easy install script, or set up data collection manually. For more information, see the Administer Splunk App for Infrastructure guide.

What the distributed deployment looks like

This diagram describes a distributed environment that is ingesting data from a Windows system, a Mac system, a Linux system, and a heavy forwarder for AWS data collection. Each system sends S2S traffic from a universal forwarder directly to an indexer cluster and HTTP traffic from collectd to a third-party load balancer. The load balancer forwards traffic to HECs in the indexer cluster.

Where to install the App and other dependencies

The following table describes the required locations for installing the Splunk App for Infrastructure and other dependencies in your distributed deployment environment.

Component	Search heads	Indexers	Heavy forwarder	Description
Splunk App for Infrastructure	Required		Required*	*Only when you deploy a heavy forwarder for AWS data collection.
Splunk Add-on for Infrastructure		Required	Required*	You must install the add-on on each indexer to provide props and transforms for data types. *Only when you deploy a heavy forwarder for AWS data collection or use a heavy forwarder as an intermediary before you send data to an indexer.
Splunk Add-on for Amazon Web Services			Required	You must install the add-on if you are collecting data from AWS. Version 4.5 is supported. Version 4.6 is not supported.
HTTP Event Collector		Required*		*If you are collecting metrics from a *nix host, this is required. Collectd, which collects metrics data from *nix hosts, sends data to a HEC.
TCP input		Required*		*If you are collecting *nix and Windows logs and Windows metrics, configure a TCP input. You need to configure a port to receive data from a universal forwarder.

Steps

Follow these steps to set up the Splunk App for Infrastructure in a distributed Splunk Enterprise deployment.

1. Install the Splunk App for Infrastructure on search heads

Install the Splunk App for Infrastructure on every search head in the cluster. For more details about this task, see Deploy a configuration bundle in the Splunk Enterprise Distributed Search manual.

1. Download the Splunk App for Infrastructure from Splunkbase.
2. On the machine that runs the search head cluster's deployer, copy the Splunk_App_Infrastructure directory to the $SPLUNK_HOME/etc/shcluster/apps directory.
3. Push the Splunk App for Infrastructure to every search head in the cluster:

   $SPLUNK_HOME/bin/splunk apply shcluster-bundle     -target <any_cluster_member_mgmt_url:mgmt_port> -auth <username:passwd>
   

2. Install the Splunk Add-on for Infrastructure on indexers

Install the Splunk Add-on for Infrastructure on the indexers. When you install the add-on, it creates the em_metrics and infra_alerts indexes, and handles props and transforms for all data types. For more information about the source types and components that the add-on configures, see Source types and components for the Splunk Add-on for Infrastructure in the Use Splunk Add-on for Infrastructure manual.

For more information about installing the add-on across an indexer cluster, see Update common peer configurations and apps in the Managing Indexers and Clusters of Indexers guide.

1. Download the Splunk Add-on for Infrastructure from Splunkbase.
2. On the machine that runs the indexer cluster master node, copy the Splunk_TA_Infrastructure directory to the $SPLUNK_HOME/etc/master-apps directory.
3. In Splunk Web, open the cluster master and go to Settings > Indexer Clustering.
4. Click Edit from the drop-down list.
5. Click Configuration Bundle Actions.
6. Click Validate and Check Restart. Verify that the process was successful.
7. Click Push. This pushes the Splunk Add-on for Infrastructure to all the Indexers.

3. Configure inputs.conf for the indexing tier

Enable receiving on the TCP port for logs and perform metrics for Windows data collection for every indexer in the cluster. To do this, open a receiving port for the indexing tier. For more information about opening a receiving port, see inputs.conf in the Splunk Enterprise Admin Manual.

If you are collecting metrics data from a *nix host, also configure an HEC token. When you configure an HEC token, set the source type to em_metrics, and specify the metrics index you want to use. By default, the metrics index is em_metrics. For more information about configuring an HEC token, see Create an Event Collector token in the Getting Data In guide.

1. On the machine that runs the indexer cluster master node or search head master node, go to the $SPLUNK_HOME/etc/system/local directory.
2. open the inputs.conf file with a text editor.
3. Add a [splunktcp] stanza:

   [splunktcp://<port>]
   disabled = 0
   

   where <port> is the port that you want to use to receive data from your host machines. The recommended value is 9997.

   For more information about configuring inputs.conf, see inputs.conf in the Admin Manual.

4. If you are collecting metrics data from a *nix host, add an HEC token stanza:

   [http://<token_name>]
   disabled = 0
   index = em_metrics
   indexes = em_metrics
   sourcetype = em_metrics
   token = <string>
   

   where <token_name> is the name of the token and <string> is a unique identifier for the token value.
5. If you have not already enabled global HEC settings, enable HEC now in an http stanza:

   [http]
   disabled = 0
   

6. Copy the inputs.conf file to the $SPLUNK_HOME/etc/master-apps/_cluster/local directory.

4. Push the indexer cluster master node's configuration bundle to the indexer cluster

Push the Splunk_TA_Infrastructure directory and inputs.conf file to every indexer in the indexer cluster. For more information, see Update common peer configurations and apps.

On the machine that is running the indexer cluster master node, apply the configuration bundle to every indexer in the cluster:

$SPLUNK_HOME/bin/splunk apply cluster-bundle

5. (Optional) Configure a heavy forwarder to collect AWS data

If you have not already deployed a heavy forwarder that can handle receiving AWS data, first deploy a heavy forwarder. For more information, see Deploy a heavy forwarder in the Forwarding Data guide.

Install these apps and add-ons on the heavy forwarder:

• Splunk App for Infrastructure
• Splunk Add-on for Infrastructure
• Splunk Add-on for AWS

For information about installing apps and add-ons, see Where to get more apps and add-ons in the Splunk Enterprise Admin Manual.

6. Configure data collection

Configure data collection for the Splunk App for Infrastructure using the easy install script under the Add Data tab. You can collect data from Linux, Mac OS X, and Windows hosts. if you installed and configured the Splunk Add-on for AWS on a heavy forwarder, you can also collect data from your AWS accounts.

The easy install script in the Add Data tab cannot set up data forwarding to multiple indexers or a load balancer. If you are sending data to multiple indexers or a load balancer, manually configure data collection.

For information about configuring data collection, see How to add data to Splunk App for Infrastructure in the Administer Splunk App for Infrastructure manual.