Splunk® Connect for Kafka

Install and Use Splunk Connect for Kafka

Download manual as PDF

Download topic as PDF

Load balancing

Splunk Connect for Kafka supports the following types of load balancing configurations:

  • A comma-separated list of HTTP Event Collectors (HEC) endpoints for internal load balancing.
  • A preconfigured hardware or software load balancer forwarding requests to HEC endpoints.

Include a load balancer in front of the HECs that are enabled on your Splunk platform indexer cluster or collection of Splunk platform heavy forwarders. Load balancing is supported when HEC token acknowledgment is enabled.

If you are using Splunk Cloud, submit a case on the Splunk Support Portal to create or modify a load balancer for use with Splunk Connect for Kafka. If you are ingesting Kinesis Firehose events, you can reuse the Elastic Load Balancing (ELB) URL that you configured for the Splunk Add-on for Amazon Kinesis Firehose.

If your deployment is configured with load balancing, and HEC acknowledgment is enabled (splunk.hec.ack.enabled:true), follow these steps to ensure data ingests correctly.

  1. Enable sticky sessions on your load balancer with the maximum configurable cookie timeout period.
  2. Set HEC channels (splunk.hec.total.channels) to multiple HEC endpoints (for example, multiple indexers or 2 * indexers behind your load balancer). This will ensure the data flow is evenly balanced across the Splunk platform indexers.
  3. Save your changes.

Data duplication might occur even with sticky sessions when requests are offloaded to a different endpoint under load.

See the Configuration examples page to see examples of load balancing with a list of HEC enabled endpoints, and load balancing with a preconfigured load balancer.

PREVIOUS
Security configurations
  NEXT
Index routing

This documentation applies to the following versions of Splunk® Connect for Kafka: 1.1.0


Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters