Splunk® Enterprise

Add AWS CloudTrail data with Kinesis Firehose: Distributed deployment with indexer clustering

Splunk Enterprise version 8.0 is no longer supported as of October 22, 2021. See the Splunk Software Support Policy for details. For information about upgrading to a supported version, see How to upgrade Splunk Enterprise.
This documentation does not apply to the most recent version of Splunk® Enterprise. For documentation on the most recent version, go to the latest release.

Configure HTTP event collection

Configure the HTTP event collector (HEC) on a single-instance Splunk Enterprise deployment to ingest data using the Splunk Add-on for Amazon Kinesis Firehose.

Prerequisite

  • Install the Splunk Add-on for Amazon Kinesis Firehose on a single-instance Splunk Enterprise deployment
  • For optimal performance, set ackIdleCleanup to true in inputs.conf located in $SPLUNK_HOME/etc/apps/splunk_httpinput/local/inputs.conf for *nix users and %SPLUNK_HOME%\etc\apps\splunk_httpinput\local\inputs.conf for Windows users.

Steps

  1. Decide what index you want to use to collect your Amazon Kinesis Firehose data. Ensure that this index is enabled and active. Sending data to a disabled or deleted index results in dropped events.
  2. Go to Settings > Data inputs > HTTP Event Collector click Global Settings.
  3. Check the box next to Enable SSL, then click Save.
  4. Create an HTTP event collector token with indexer acknowledgments enabled. During the configuration:
  5. Specify a Source type for your incoming data.
  6. Select an Index to which Amazon Kinesis Firehose will send data.
  7. Check the box next to Enable indexer acknowledgement.
  • Save the token that Splunk Web provides. You need this token when you configure Amazon Kinesis Firehose.
  • Repeat steps 4 and 5 for each additional source type from which you want to collect data. Each source type requires a unique HTTP event collector token.


    • Configure timestamp extraction

    You can configure your add-on to send timestamped events to HTTP Event Collector when auto_extract_timestamp is set to "true" in the /event URL.

    To configure this, enable one of the following endpoints:

    • services/collector/event/1.0: Provides timestamps for event data events when auto_extract_timestamp is set to "true" in the /event URL
    • services/collector/raw/1.0: Provides timestamps for raw data events when auto_extract_timestamp is set to "true" in the /event URL

    When one or both of these endpoints are enabled, the add-on extracts timestamps as follows: 

    * If there is no timestamp in the event's JSON envelope, extraction is performed by leverage pipeline.
* If there is a timestamp, Splunk honors it.
* If "time=xxx" is used in the /event URL then auto_extract_timestamp is disabled.

    https://docs.splunk.com/Documentation/Splunk/1/SimplerGDI/HECEndpoints#HEC_Endpoints

    Last modified on 16 March, 2020
    Install the Splunk Add-on for Amazon Web Services   Distribute the HTTP Event Collector settings

    This documentation applies to the following versions of Splunk® Enterprise: 7.2.0, 7.2.1, 7.2.2, 7.2.3, 7.2.4, 7.2.5, 7.2.6, 7.2.7, 7.2.8, 7.2.9, 7.2.10, 7.3.0, 7.3.1, 7.3.2, 7.3.3, 7.3.4, 7.3.5, 7.3.6, 7.3.7, 7.3.8, 7.3.9, 8.0.0, 8.0.1, 8.0.2, 8.0.3, 8.0.4, 8.0.5, 8.0.6, 8.0.7, 8.0.8, 8.0.9, 8.0.10

    Acrobat logo Download manual
    Acrobat logo Download this page

    Was this topic useful?







    You must be logged into splunk.com in order to post comments. Log in now.

    Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

    0 out of 1000 Characters