Splunk® Supported Add-ons

Splunk Add-on for Amazon Kinesis Firehose

Download manual as PDF

Download topic as PDF

Hardware and software requirements for the Splunk Add-on for Amazon Kinesis Firehose

Splunk platform requirements

The Splunk Add-on for Amazon Kinesis Firehose requires Splunk platform version 6.6.X or later.

This add-on is supported in the following Splunk platform environments:

  • managed Splunk Cloud
  • Splunk Enterprise running in an AWS VPC
  • Splunk Enterprise running in your own internal network with SSL certificates installed on each indexer. Self-signed certificates are not supported, you must use a CA-signed certificate.

This add-on is not supported in self-service Splunk Cloud at this time.

If you plan to run this add-on entirely in managed Splunk Cloud, there are no additional Splunk platform requirements.

If you plan to run this add-on in a self-managed AWS instance, see Splunk Enterprise, self-managed in the Cloud for sizing considerations specific to AWS.

If you are running this add-on on Splunk Enterprise in your own internal network, see System Requirements in the Splunk Enterprise Installation Manual.

SSL requirements

Amazon Kinesis Firehose requires HTTP Event Collector (HEC) endpoint to be terminated with a valid CA-signed certificate matching the DNS hostname used to connect to your HEC endpoint.

You must use a trusted CA-signed certificate. Self-signed certificates are not supported.

If you are sending data directly into Splunk Enterprise indexers in your own internal network or AWS VPC, a CA-signed certificate must be installed to each of the indexers. If you are using an ELB to send data, you must install a CA-signed certificate on the load balancer.

Managed Splunk Cloud users are provided an ELB with a proper CA-signed certificate and a hostname for each stack. For ELB users on distributed Splunk Enterprise deployments, see Configure an Elastic Load Balancer for the Splunk Add-on for Amazon Kinesis Firehose for information on how to configure an ELB with proper SSL certifications.

Splunk admin requirements

To install the Splunk Add-on for Amazon Kinesis Firehose, you must be a member of the admin or sc_admin role.

Amazon Kinesis Firehose requirements

The Splunk Add-on for Amazon Kinesis Firehose requires specific configuration in Amazon Kinesis Firehose. See What is Amazon Kinesis Firehose? in the AWS documentation.

Event formatting requirements

The Splunk Add-on for Amazon Kinesis Firehose supports data collection using either of the two HTTP Event Collector endpoint types: raw and event. If you collect data using the raw endpoint, no special formatting is required for most source types. The aws:cloudwatchlogs:vpcflow contains a nested events JSON array that cannot be parsed by the HTTP Event Collector. Prepare this data for the Splunk platform using an AWS Lambda function that extracts the nested JSON events correctly into a newline-delimited set of events. All other source types can be sent directly to the raw endpoint without any preprocessing.

If you collect data using the event endpoint, format your events into the JSON format expected by HTTP Event Collector before sending them from Amazon Kinesis Firehose to the Splunk platform. You can apply an AWS Lambda blueprint to preprocess your events into the JSON structure and set event-specific fields, which allows you greater control over how your events are handled by the Splunk platform. For example, you can create and apply a Lambda blueprint that sends data from the same Firehose stream to different indexes depending on event type.

For information about the required JSON structure, see Format events for HTTP Event Collector on the Splunk developer portal.

PREVIOUS
Release history for the Splunk Add-on for Amazon Kinesis Firehose
  NEXT
Installation and configuration overview for the Splunk Add-on for Amazon Kinesis Firehose

This documentation applies to the following versions of Splunk® Supported Add-ons: released


Comments

@Svarathan,

There are a few Lambda blueprints that Kinesis Firehose provides. You can read more about them in the Firehose documentation here: https://docs.aws.amazon.com/firehose/latest/dev/data-transformation.html or in the Lambda blueprints UI of your AWS console.

Myu splunk, Splunker
January 10, 2018

Is there Sample Lambda code which could be put in the docs and/or a template which could be followed?

Svarathan splunk, Splunker
January 10, 2018

Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters