Splunk® Enterprise

Add AWS Config data with Kinesis Firehose: Distributed deployment with indexer clustering

Splunk Enterprise version 8.0 is no longer supported as of October 22, 2021. See the Splunk Software Support Policy for details. For information about upgrading to a supported version, see How to upgrade Splunk Enterprise.
This documentation does not apply to the most recent version of Splunk® Enterprise. For documentation on the most recent version, go to the latest release.

Configure Amazon Kinesis Firehose to send data to the Splunk platform

Prerequisites

  • Before you configure Amazon Kinesis Firehose to send data to the Splunk platform, you need to prepare the Splunk platform to receive AWS data.
  • Go to the AWS Management Console to configure Amazon Kinesis Firehose to send data to the Splunk platform. See Choose Splunk for Your Destination in the AWS documentation for step-by-step instructions. Repeat this process for each token that you configured in the HTTP event collector, or that Splunk Support configured for you.

When prompted during the configuration, enter the following information:

Field in Amazon Kinesis Firehose configuration page Value
Destination Select Splunk.
Splunk cluster endpoint If you are on a distributed Splunk Enterprise deployment, enter the URL and port of your data receiver node.
For example, if you have an ELB that proxies traffic to your indexers with DNS name example-test-123456789.us-east-1.elb.amazonaws.com and port 443, enter https://example-test-123456789.us-east-1.elb.amazonaws.com:443.
If you want to send data directly to multiple Splunk indexers acting as your data collection nodes, you need a URL that resolves to multiple IP addresses (one for each node) with the port enabled for HTTP event collector on those nodes.
For example, if the hostname that resolves to your indexers is inputs.example-deployment.com, enter https://inputs.example-deployment.com:8088.
Splunk endpoint type Select raw unless you are using an AWS Lambda function to format your events for the HTTP event collector event endpoint, in which case you should choose event. For more information about preprocessing events, see Event formatting requirements.
Authentication token Enter your HTTP event collector token that you configured or received from Splunk Support.
S3 backup mode Best practice: Backup all events to S3 until you have validated that events are fully processed by the Splunk platform and available in Splunk searches. You can adjust this setting after you have verified data is searchable in the Splunk platform.

After you configure Amazon Kinesis Firehose to send data to the Splunk platform, go to the Splunk search page and search for the source types of the data you are collecting. See Source types for the Splunk Add-on for Amazon Kinesis Firehose for a list of source types that this add-on applies to your Firehose data. Validate that the data is searchable in the Splunk platform before you adjust the S3 backup mode setting in the AWS Management Console.

If you are unable to see your data in the Splunk platform, see troubleshoot the Splunk Add-on for Amazon Kinesis Firehose.

Last modified on 12 June, 2019
Configure Elastic Load Balancer settings   Validate your data

This documentation applies to the following versions of Splunk® Enterprise: 7.2.0, 7.2.1, 7.2.2, 7.2.3, 7.2.4, 7.2.5, 7.2.6, 7.2.7, 7.3.0, 7.3.1, 7.3.2, 7.3.3, 7.3.4, 7.3.5, 7.3.6, 7.3.7, 7.3.8, 7.3.9, 8.0.0, 8.0.1, 8.0.2, 8.0.3, 8.0.4, 8.0.5, 8.0.6, 8.0.7, 8.0.8, 8.0.9, 8.0.10


Was this topic useful?







You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters