Scale HTTP Event Collector with distributed deployments

You can use the HTTP event collector (HEC) as part of a distributed Splunk platform deployment. The Splunk software processes HEC data in the same way as it does any other input.

You can configure HEC on Splunk Cloud Platform deployments. When you configure a Splunk Cloud Platform distributed deployment, the indexers in that deployment automatically have HEC configured.

If you want to scale HEC on a Splunk Enterprise distributed deployment, familiarize yourself with distributed Splunk Enterprise deployments before you proceed. See the following related links:

• For more information about distributed deployments, see Scale your deployment with Splunk Enterprise components in the Distributed Deployment Manual and Components of a Splunk Enterprise deployment in the Capacity Planning Manual.
• For more information about deployment server, see About deployment server and forwarder management in the Updating Splunk Enterprise Instances manual.

Where to place HEC

HEC can be placed on indexers or heavy forwarders. HEC, like any data input configuration, must reside on the component when the data enters the system. As a best practice, place HEC on an indexer. HEC is not supported on universal forwarders. As with any data input configuration, you can also place HEC directly on a clustered or non-clustered indexer, if necessary.

The deployment server distributes any app that contains your HEC configurations. This configuration includes the following information:

• HTTP Event Collector default values (port, SSL, source type, index)
• SSL settings
• HTTP Event Collector tokens

Each HEC input entry must contain a valid universally unique identifier (UUID) for the token. HEC stores its configurations in the $SPLUNK_HOME/etc/apps/splunk_httpinput/ directory or %SPLUNK_HOME%\etc\apps\splunk_httpinput\ on Windows.

Place and distribute HEC on non-clustered indexers

Use HEC with non-clustered indexers in one of two ways:

• Place HEC on heavy forwarders that forward to the indexers by using a deployment server to distribute configurations to the HEC.
• Place HEC directly on the indexers by using a deployment server to distribute configurations to the HEC.

See About deployment server and forwarder management in the Updating Splunk Enterprise Instances manual to learn more.

For more information about configuring deployment clients, see Configure deployment clients in the Updating Splunk Enterprise Instances manual.

Place and distribute HEC on indexer cluster peer nodes

Use HEC in a distributed Splunk platform deployment that uses indexer clustering.

Use the configuration bundle method to distribute HEC configurations to the peer nodes. Using your HEC port number, preferred protocol (HTTP or HTTPS), SSL settings, and HTTP Event Collector tokens, connect the HECs on your deployment forwarders with the peer nodes of your deployment's indexer cluster. Tokens are managed centrally on the Splunk Enterprise instance running the cluster manager node.

See Use forwarders to get data into the indexer cluster and Update common peer configurations and apps in the Managing Indexers and Clusters of Indexers manual.

Place and distribute HEC on heavy forwarders

Use HEC in a distributed Splunk platform deployment that uses forwarders. Use a deployment server to distribute HEC configurations to the heavy forwarders in your Splunk platform deployment.

If you plan to distribute HEC configurations through the deployment server, set the useDeploymentServer option in the [http] stanza of inputs.conf on the deployment server to 1. When this option is set to 1 and you make UI-based HEC changes on the deployment server, those changes are placed directly in the $SPLUNK_HOME/etc/deployment-apps/splunk_httpinput/ folder, rather than in $SPLUNK_HOME/etc/apps/splunk_httpinput/. See inputs.conf for further information.

See Deploy a heavy forwarder in the Forwarding Data manual to learn more.

Example serverclass.conf file

A server class is a group of deployment clients that you can manage as a single unit. You assign the deployment clients you want to use in your HEC deployment to one common server class. Later, when you distribute HEC settings to the deployment clients, only members of that server class receive the configuration settings. The following example serverclass.conf file defines a server class FWD2Local for HEC.

[global]
whitelist.0=*
restartSplunkd=true
stateOnClient = enabled
 
[serverClass:FWD2Local]
whitelist.0=*
[serverClass:FWD2Local:app:splunk_httpinput]

For the purposes of deploying HEC settings, you can think of HEC as an app called splunk_httpinput. Within the stanzas, you can set client filtering attributes and several non-filtering attributes.

For more information about available client filtering attributes, see Define filters through serverclass.conf in the Updating Splunk Enterprise Instances Manual. To learn more about available non-filtering attributes, see Use serverclass.conf to define server classes in the Updating Splunk Enterprise Instances Manual.

See also

For more information about distributed deployment, see the Overview of Splunk Enterprise distributed deployments chapter in the Distributed Deployment Manual and Components of a Splunk Enterprise deployment in the Capacity Planning Manual.

For more information about distributed deployment, including advanced configuration options and general examples, see the Updating Splunk Instances Manual.