Configure the HTTP Event Collector to receive metrics data for SAI

Use an HTTP Event Collector (HEC) to collect metrics from collectd and fluentd. Whether you run the easy install script or set up integrations manually, you have to configure HEC for metrics you collect with collectd and fluentd. To use HEC to collect metrics, configure an HEC token for the Splunk App for Infrastructure (SAI). Collectd and fluentd send metrics data to the index you specify in the HEC token configuration.

em_metrics is the default metrics index to send data you receive from HEC. If you want to use another metrics index, specify it when you create the HEC token instead. If you use another index, you have to update the sai_metrics_indexes macro, too. For more information about using another index, see Use custom metric indexes in Splunk App for Infrastructure.

These integrations use collectd and fluentd:

Collection agent	Integration
collectd	Linux Unix Mac OS X
fluentd	Kubernetes OpenShift

Prerequisites

• You plan to collect data from an integration that requires HEC.
• You have an index or multiple indexes you want to send metrics data to.
• You enabled HEC. See Enable HTTP Event Collector in the Splunk Enterprise Getting Data In guide.

Steps

Follow these steps to configure an HEC token for SAI data collection. You can configure an HEC token in Splunk Web or with .conf files.

Configure an HEC token in Splunk Web

These steps show you how to set up an HEC token in Splunk Web to collect metrics data from collectd and fluentd in SAI. For more information about configuring an HEC token in Splunk Web, see Set up and use HTTP Event Collector in Splunk Web in the Splunk Enterprise Getting Data In guide.

1. In Splunk Web, log in as an administrator.
2. Go to Settings > Data inputs, select HTTP Event Collector, and click Global Settings. Ensure that All Tokens is set to Enabled. Also take note of the HTTP Port Number because you will need it later when you start adding data. When you're done, click Save.
3. Click New Token.
4. For Select Source, don't check Enable indexer acknowledgement.
5. For Input Settings, these are the required settings for SAI. If you use another metrics index, specify it instead of em_metrics. You can also include multiple allowed indexes.

   Setting	Value
   Source type	em_metrics
   App context	Splunk_TA_Infrastructure
   Select Allowed Indexes	em_metrics Collectd and fluentd send metrics data to the default index only.
   Default Index	em_metrics

6. Review the settings and then generate the HEC Token to send data over HEC to the Splunk Enterprise instance.
7. Confirm the token was created and copy the Token Value. You need to provide this when you configure an integration that uses fluentd or collectd. You can also return to this page to view it later.

Configure an HEC token from inputs.conf

These steps show you how to set up an HEC token with .conf files to collect metrics data from collectd and fluentd in SAI. For more information about configuring an HEC token with .conf files, see Set up and use HTTP Event Collector with configuration files in the Splunk Enterprise Getting Data In guide.

1. Go to the $SPLUNK_HOME/etc/system/local directory.
2. Open the inputs.conf file with a text editor. If it doesn't exist yet, create it.
3. Enter this HEC token stanza. These are the default values. If you use another metrics index, specify it instead of em_metrics. You can also include multiple allowed indexes.

   [http://<token_name>]
   disabled = 0
   index = em_metrics
   indexes = em_metrics
   sourcetype = em_metrics
   token = <string>
   

4. Save your changes and close the file.
5. Restart splunkd:

   $SPLUNK_HOME/bin/splunk restart