Set up and use HTTP Event Collector with configuration files
HTTP Event Collector (HEC) stores its settings on a Splunk Enterprise instance in two configuration files: inputs.conf and outputs.conf. These files are not accessible on Splunk Cloud instances, and you must manage configurations on Splunk Cloud instances through Splunk Web.
Configuring HEC inputs with a configuration file is a slightly different process than configuring other data inputs. In many cases, you edit the files in the $SPLUNK_HOME/etc/system/local directory. For HEC, you edit the files in the $SPLUNK_HOME/etc/apps/splunk_httpinput/local/ directory.
No matter how many inputs.conf files a Splunk Enterprise instance has and where they reside, Splunk Enterprise combines all their settings, using the rules of location precedence. See Configuration file precedence in the Admin Manual.
To set up HEC with configuration files, follow these steps:
- In the $SPLUNK_HOME/etc/apps/splunk_httpinput directory, create a
localdirectory, if it does not exist.
- Change to the $SPLUNK_HOME/etc/apps/splunk_httpinput/local directory.
- Create an inputs.conf file if it does not exist.
- Use a text editor to open inputs.conf for editing.
- Specify global and token settings as described in Token-related settings later in this topic.
- Save the file and close it.
- Restart Splunk Enterprise for the changes to take effect.
The HEC token must be a Globally Unique IDentifier (GUID).
HEC stores settings related to token management in the inputs.conf configuration file.
You can specify whether settings apply globally to all tokens or only to specific tokens:
[http]stanza contains global settings that apply to all tokens.
token_nameindicates the token name as assigned by the user, apply to individual tokens. Settings specified here override settings specified within the
The inputs.conf file contains basic explanatory information about each setting.
[http] stanza contains global settings that apply to all tokens.
|dedicatedIoThreads||The number of dispatcher threads on the HTTP Event Collector server. The default value is 2. Do not alter this setting unless you are requested to do so by Splunk Support. The value of this parameter must never be more than the number of physical CPU cores on your Splunk Enterprise server.|
|disabled||Whether tokens are disabled. 1 indicates true, and 0 indicates false. The default value is 1. When set to 1 in the [http] stanza, this parameter disables all tokens.|
|enableSSL||Whether the HTTP Event Collector server protocol is HTTP or HTTPS. 1 indicates HTTPS is enabled; 0 indicates HTTP. The default value is 1. HTTP Event Collector shares SSL settings with the Splunk Enterprise instance and can't have |
|index||The global default index. This parameter can be overridden when set in an individual token stanza, or when the header of event data contains an |
|maxSockets||The number of HTTP Event Collector connections, expressed as an integer, that Splunk Enterprise accepts simultaneously. You can limit this number to constrain resource usage. When set to 0, Splunk Enterprise automatically sets it to one-third of the maximum allowable open files on the host. If this number is less than 50, it is set to 50. If this number is greater than 400000, it is set to 400000. If set to a negative number, no limit is enforced. Defaults to 0.|
|maxThreads||The number of threads, expressed as an integer, that can be used by active HTTP transactions. You can limit this number to constrain resource usage. When set to 0, Splunk Enterprise automatically sets the limit to one-third of the maximum allowable threads on the host. If this number is less than 20, it is set to 20. If this number is greater than 150000, it is set to 150000. If |
|outputgroup||The global default output group. An output group is a group of indexers set up by the Splunk Enterprise administrator to index the data. If there is no output group specified, event data goes to the local indexer. If the given output group is invalid, the data is dropped, and an error message is logged to splunkd.log. For more information about specifying output groups, see Output group-related settings later in this topic.|
|port||The HTTP Event Collector server port. The default value is 8088. This port number must not already be in use.|
|sourcetype||The global default source type. This parameter can be overridden either when set in an individual token's stanza or by event data whose header contains a |
|useDeploymentServer||Whether to use the Deployment Server. When set to 1 (true), writes to the location specified by |
http://<token_name> stanzas, where
<token_name> indicates the token name as assigned by the user, apply to individual tokens. Settings specified here override settings specified within the
The type of default host for the token. This parameter can be set to any of the following literal values:
|disabled||Whether the token is disabled. 1 indicates true; 0 indicates false. The default value is 0.|
|index||The token's default index. This parameter can be overridden by event data whose header contains an |
|indexes||A list of allowable indexes to which the data can be indexed. See global settings for including lists.|
|persistentQueueSize||The maximum size of the persistent queue. The value of this parameter is in the form |
|source||The token's default source. This parameter can be overridden by event data whose header contains a |
|queueSize||The maximum size of the input queue in memory. The value of this parameter is in the form <integer>[KB|MB|GB]. The default value is |
|sourcetype||The token's default source type. This parameter can be overridden by event data whose header contains a |
|token||The HTTP Event Collector token. The token must be a unique GUID.|
Settings that apply to forwarding and load balancing are stored in outputs.conf, including settings for specifying HTTP Event Collector output groups. These settings are the same ones that Splunk Enterprise admins use to manage forwarding and load balancing among indexers.
Specify global settings in the
[tcpout] stanza, and specify per-output group settings in the
The outputs.conf file contains basic explanatory information about each setting. For more information, see About forwarding and receiving in the Forwarding Data manual and Configure forwarders with outputs.conf in the Splunk Universal Forwarder Forwarder Manual.
[tcpout] stanza defines the output groups to which the data is forwarded.
|defaultGroup||A comma-separated list of one or more target output group names in the form |
Per-output group settings
[tcpout:target_group] stanza defines the configuration of the target output group indicated by
<target_group>. You can have as many target groups as you want. If more than one target group is specified, the forwarder clones the data to each target group.
|blockWarnThreshold||The output pipeline's send failure count threshold. The default value is 100. After the threshold is met, a failure message is displayed as a banner in Splunk Web. To effectively disable this warning, set this to a very large value, like 2000000.|
|server||<server name>]:<port>, [<ip>|<server name>]:<port>, .... For each mentioned system, you must include the port number, and the IP address or server name.|
Set up and use HTTP Event Collector in Splunk Web
Set up and use HTTP Event Collector from the CLI
This documentation applies to the following versions of Splunk® Enterprise: 6.5.0, 6.5.1, 6.5.2, 6.5.3, 6.5.4, 6.5.5, 6.5.6, 6.5.7, 6.5.8, 6.5.9, 6.5.10, 6.6.0, 6.6.1, 6.6.2, 6.6.3, 6.6.4, 6.6.5, 6.6.6, 6.6.7, 6.6.8, 6.6.9, 6.6.10, 6.6.11, 6.6.12, 7.0.0, 7.0.1, 7.0.2, 7.0.3, 7.0.4, 7.0.5, 7.0.6, 7.0.7, 7.0.8, 7.0.9, 7.0.10, 7.0.11, 7.0.13, 7.1.0, 7.1.1, 7.1.2, 7.1.3, 7.1.4, 7.1.5, 7.1.6, 7.1.7, 7.1.8, 7.1.9, 7.1.10, 7.2.0, 7.2.1, 7.2.2, 7.2.3, 7.2.4, 7.2.5, 7.2.6, 7.2.7, 7.2.8, 7.2.9, 7.2.10, 7.3.0, 7.3.1, 7.3.2, 7.3.3, 7.3.4, 7.3.5, 7.3.6, 7.3.7, 7.3.8, 7.3.9, 8.0.0, 8.0.1, 8.0.2, 8.0.3, 8.0.4, 8.0.5, 8.0.6, 8.0.7, 8.0.8, 8.0.9, 8.1.0, 8.1.1, 8.1.2, 8.1.3, 8.1.4, 8.1.5, 8.2.0, 8.2.1