Related Answers
Download topic as PDF
Set up and use HTTP Event Collector with configuration files
HTTP Event Collector stores its settings on your Splunk Enterprise instance in two .conf files: inputs.conf and outputs.conf. These files are not accessible on Splunk Cloud instances, and configurations must be managed through Splunk Web.
Configuring HEC inputs with a configuration file is a slightly different process than configuring other data inputs. In many cases, you would edit inputs.conf in the $SPLUNK_HOME/etc/system/local directory. For HEC, you should edit $SPLUNK_HOME/etc/apps/splunk_httpinput/local/inputs.conf.
No matter how many inputs.conf files a Splunk Enterprise instance has and where they reside, Splunk Enterprise combines all their settings, using the rules of location precedence, as described in configuration file precedence.
- In the
$SPLUNK_HOME/etc/apps/splunk_httpinputdirectory, create a directorylocal, if it does not exist. - Change to the
$SPLUNK_HOME/etc/apps/splunk_httpinput/localdirectory. - Create an
inputs.conffile if it does not exist. - Open
inputs.conffor editing. - Specify global and token settings as described later in this topic.
- The HEC token must be a unique GUID.
- Save the file and close it.
- Restart Splunk Enterprise for the changes to take effect.
HEC stores settings related to token management in the inputs.conf file.
You can specify whether settings apply globally (to all tokens) or to specific tokens:
- The
[http]stanza contains global settings that apply to all tokens. - The
[token_name]stanzas, wheretoken_nameindicates the token name as assigned by the user, apply to individual tokens. Settings specified here override settings specified within the[http]stanza.
The inputs.conf file contains basic explanatory information about each setting.
Global settings
The [http] stanza contains global settings that apply to all tokens.
| Parameter | Description |
|---|---|
dedicatedIoThreads | The number of dispatcher threads on the HTTP Event Collector server. The default value is 2. This setting should not be altered unless you have been requested to do so by Splunk Support. The value of this parameter should never be more than the number of physical CPU cores on your Splunk Enterprise server. |
disabled | Whether tokens are disabled. 1 indicates true; 0 indicates false. The default value is 0. When set to 1 in the [http] stanza, this parameter disables all tokens. |
enableSSL | Whether the HTTP Event Collector server protocol is HTTP or HTTPS. 1 indicates HTTPS is enabled; 0 indicates HTTP. The default value is 1. HTTP Event Collector shares SSL settings with the Splunk management server and cannot have enableSSL settings that differ from the settings on the Splunk management server. |
index | The global default index. This parameter can be overridden when set in an individual token's stanza, or by event data whose header contains an index parameter set to a different value. You can limit the set of allowed values for this parameter on a per-token basis by using the indexes parameter.
|
maxSockets | The number of HTTP Event Collector connections (expressed as an integer) that Splunk Enterprise accepts simultaneously. You can limit this number to constrain resource usage. When set to 0, Splunk Enterprise automatically sets it to one third of the maximum allowable open files on the host. If this number is less than 50, it will be set to 50. If this number is greater than 400000, it will be set to 400000. If set to a negative number, no limit will be enforced. Defaults to 0. |
maxThreads | The number of threads (expressed as an integer) that can be used by active HTTP transactions. You can limit this number to constrain resource usage. When set to 0, Splunk Enterprise automatically sets the limit to one third of the maximum allowable threads on the host. If this number is less than 20, it will be set to 20. If this number is greater than 150000, it will be set to 150000. If maxSockets is not negative and maxThreads is greater than maxSockets, then Splunk Enterprise sets maxThreads to be equal to maxSockets. If set to a negative number, no limit will be enforced. Defaults to 0. |
outputGroup | The global default output group. An output group is a group of indexers set up by the Splunk Enterprise administrator to index the data. If there is no output group specified, event data will go to the local indexer. If the given output group is invalid, the data will be dropped and an error message will be logged to splunkd.log. For more information about specifying output groups, see Output group-related settings later in this topic. |
port | The HTTP Event Collector server port. The default value is 8088. This port number must not already be in use. |
sourcetype | The global default sourcetype. This parameter can be overridden when set in an individual token's stanza, or by event data whose header contains a sourcetype parameter set to a different value. |
useDeploymentServer | Whether to use Deployment Server. When set to true (1), writes to the location specified by repositoryLocation property in serverclass.conf. Defaults to 0 and writes to $SPLUNK_HOME/etc/apps. |
Per-token settings
The < token_name> stanzas, where <token_name> indicates the token name as assigned by the user, apply to individual tokens. Settings specified here override settings specified within the [http] stanza.
| Parameter | Description |
|---|---|
connection_host | The type of default host for the token. This parameter can be set to any of the following literal values:
|
disabled | Whether the token is disabled. 1 indicates true; 0 indicates false. The default value is 0. |
index | The token's default index. This parameter can be overridden by event data whose header contains an index parameter set to a different value. |
indexes | A list of allowable indexes to which the data can be indexed. |
persistentQueueSize | The maximum size of the persistent queue. The value of this parameter is in the form <integer>[KB|MB|GB]. The default value is 0, which indicates there is no persistent queue. Persistent queues can help prevent loss of transient data by saving data in an input queue to disk. When set, the value of the persistentQueueSize parameter must be more than the value of the queueSize parameter. For more information about persistent queues, see Use persistent queues to help prevent data loss. |
source | The token's default source. This parameter can be overridden by event data whose header contains a source parameter set to a different value. |
queueSize | The maximum size of the input queue in memory. The value of this parameter is in the form <integer>[KB|MB|GB]. The default value is 500KB. |
sourcetype | The token's default sourcetype. This parameter can be overridden by event data whose header contains a sourcetype parameter set to a different value. |
token | The HTTP Event Collector token. The token must be a unique GUID. |
Settings that apply to forwarding and load balancing are stored in outputs.conf, including settings for specifying HTTP Event Collector output groups. These settings are the same ones that Splunk Enterprise admins use to manage forwarding and load balancing among indexers.
- The
[tcpout]stanza defines the output groups to which the data will be forwarded. - The
[tcpout:<target_group>]stanza defines the configuration of the target output group indicated by<target_group>. You can have as many target groups as you want. If more than one target group is specified, the forwarder clones the data to each target group.
The outputs.conf file contains basic explanatory information about each setting. For more information, see about forwarding and receiving and configure forwarders with outputs.conf
Global settings
The [tcpout] stanza defines the output groups to which the data will be forwarded.
| Parameter | Description |
|---|---|
defaultGroup | A comma-separated list of one or more target output group names in the form <target_group>, <target_group>, ... The names of the output groups are specified later in the outputs.conf file in the [tcpout:<target_group>] stanzas. Data will be sent to the specified groups. |
Per-output group settings
The [tcpout:target_group] stanza defines the configuration of the target output group indicated by <target_group>. You can have as many target groups as you want. If more than one target group is specified, the forwarder clones the data to each target group.
| Parameter | Description |
|---|---|
blockWarnThreshold | The output pipeline's send failure count threshold. The default value is 100. After the threshold is met, a failure message will be displayed as a banner in the Splunk Enterprise UI. To effectively disable UI warnings, set this to a very large value (for example, 2000000). |
server | Required. A comma-separated list of one or more Splunk servers to which to send data. The list is in the form [<ip>|<server name>]:<port>, [<ip>|<servername>]:<port>, ... For each mentioned system, the following are required:
|
|
PREVIOUS Set up and use HTTP Event Collector in Splunk Web |
NEXT Set up and use HTTP Event Collector from the CLI |
This documentation applies to the following versions of Splunk® Enterprise: 6.5.0, 6.5.1, 6.5.2, 6.5.3, 6.5.4, 6.5.5, 6.5.6, 6.5.7, 6.5.8, 6.5.9, 6.5.10, 6.6.0, 6.6.1, 6.6.2, 6.6.3, 6.6.4, 6.6.5, 6.6.6, 6.6.7, 6.6.8, 6.6.9, 6.6.10, 6.6.11, 6.6.12, 7.0.0, 7.0.1, 7.0.2, 7.0.3, 7.0.4, 7.0.5, 7.0.6, 7.0.7, 7.0.8, 7.0.9, 7.0.10, 7.0.11, 7.0.13, 7.1.0, 7.1.1, 7.1.2, 7.1.3, 7.1.4, 7.1.5, 7.1.6, 7.1.7, 7.1.8, 7.1.9, 7.1.10, 7.2.0, 7.2.1, 7.2.2, 7.2.3, 7.2.4, 7.2.5, 7.2.6, 7.2.7, 7.2.8, 7.2.9, 7.3.0, 7.3.1, 7.3.2, 7.3.3, 8.0.0, 8.0.1
Comments
Hi Hmaldonado,
What is shown here is incorrect. EnableSSL is 0 for HTTP, and 1 for HTTPS. I will fix this error. Apologies for the incorrect information and thank you for the heads up.
Hmaldonado, thank you for the comment. The information in this topic is based on the information in the Splunk Enterprise inputs.conf.spec file. We've asked engineers to look into whether the information in inputs.conf.spec is incorrect.
An example of the per token settings would also be nice.
ie. [http://<token_name] as it isn't clear that what is should be.
This document says that enableSSL=1 means HTTP. It say that enableSSL=0 is the default and means HTTPS. This looks wrong to me.
If this is true:
enableSSL Whether the HTTP Event Collector server protocol is HTTP or HTTPS. 1 indicates HTTPS is enabled; 0 indicates HTTP. The default value is 1. HTTP Event Collector shares SSL settings with the Splunk management server and cannot have enableSSL settings that differ from the settings on the Splunk management server.
The the certificate exposed on 8088 should be the same one exposed on 8000, and it isn't. can you update this page to demonstrate how to enable specific certificates for the HEC.
thanks.