Configure HTTP event collection
Configure the HTTP event collector (HEC) on a single-instance Splunk Enterprise deployment to ingest data using the Splunk Add-on for Amazon Kinesis Firehose.
- Install the Splunk Add-on for Amazon Kinesis Firehose on a single-instance Splunk Enterprise deployment
- For optimal performance, set
ackIdleCleanupto true in
$SPLUNK_HOME/etc/apps/splunk_httpinput/local/inputs.conffor *nix users and
%SPLUNK_HOME%\etc\apps\splunk_httpinput\local\inputs.conffor Windows users.
- Decide what index you want to use to collect your Amazon Kinesis Firehose data. Ensure that this index is enabled and active. Sending data to a disabled or deleted index results in dropped events.
- Go to Settings > Data inputs > HTTP Event Collector click Global Settings.
- Check the box next to Enable SSL, then click Save.
- Create an HTTP event collector token with indexer acknowledgments enabled. During the configuration:
- Specify a Source type for your incoming data.
- Select an Index to which Amazon Kinesis Firehose will send data.
- Check the box next to Enable indexer acknowledgement.
Configure timestamp extraction
You can configure your add-on to send timestamped events to HTTP Event Collector when auto_extract_timestamp is set to "true" in the /event URL.
To configure this, enable one of the following endpoints:
services/collector/event/1.0: Provides timestamps for event data events when auto_extract_timestamp is set to "true" in the /event URL
services/collector/raw/1.0: Provides timestamps for raw data events when auto_extract_timestamp is set to "true" in the /event URL
When one or both of these endpoints are enabled, the add-on extracts timestamps as follows:
* If there is no timestamp in the event's JSON envelope, extraction is performed by leverage pipeline. * If there is a timestamp, Splunk honors it. * If "time=xxx" is used in the /event URL then auto_extract_timestamp is disabled.
Download and install Trumpet
Trumpet is a configuration tool that leverages AWS CloudFormation to set up AWS infrastructure. This infrastructure pushes data to your Splunk platform instance using the HTTP Event Collector (HEC).
To install and configure Trumpet, see the README file on Github.
Configure Amazon Kinesis Firehose to send data to the Splunk platform
This documentation applies to the following versions of Splunk® Enterprise: 7.2.0, 7.2.1, 7.2.2, 7.2.3, 7.2.4, 7.2.5, 7.2.6, 7.2.7, 7.2.8, 7.2.9, 7.2.10, 7.3.0, 7.3.1, 7.3.2, 7.3.3, 7.3.4, 7.3.5, 7.3.6, 7.3.7, 7.3.8, 7.3.9, 8.0.0, 8.0.1, 8.0.2, 8.0.3, 8.0.4, 8.0.5, 8.0.6, 8.0.7, 8.0.8