Load balancing
Include a load balancer in front of the HTTP Event Collectors (HEC) that are enabled on your Splunk platform indexer cluster or collection of Splunk platform heavy forwarders. Load balancing is supported when HEC token acknowledgment is enabled.
If you are using Splunk Cloud, submit a case on the Splunk Support Portal to create or modify a load balancer for use with Splunk Connect for Kafka. If you are ingesting Kinesis Firehose events, you can reuse the Elastic Load Balancing (ELB) URL that you configured for the Splunk Add-on for Amazon Kinesis Firehose.
If your deployment is configured with load balancing, and HEC acknowledgment is enabled (splunk.hec.ack.enabled:true
), follow these steps to ensure data ingests correctly.
- Enable sticky sessions on your load balancer with the maximum configurable cookie timeout period.
- Set HEC channels (
splunk.hec.total.channels
) to multiple HEC endpoints (for example, multiple indexers or 2 * indexers behind your load balancer). This will ensure the data flow is evenly balanced across the Splunk platform indexers. - Save your changes.
Data duplication might occur even with sticky sessions when requests are offloaded to a different endpoint under load.
Security configurations | Index routing |
This documentation applies to the following versions of Splunk® Connect for Kafka: 1.0.0
Feedback submitted, thanks!