System requirements

To install Splunk Connect for Kafka, you must meet the following system requirements:

• A Kafka Connect environment running Kafka version 0.10.0 or later.
• Java 8 or later.
• Splunk platform environment of version 6.5 or later.
• Configured and valid HTTP Event Collector (HEC) tokens.

Architecture requirements

Splunk Connect for Kafka supports two types of architectures:

• Set up a heavy forwarder layer in front of a Splunk platform indexer cluster to offload the data injection load to your Splunk platform indexer cluster. Setting up a heavy forwarder layer has performance benefits for the Splunk search app. For example:

A Kafka Connect Cluster (in containers, virtual machines, or physical machines) -> Heavy Forwarders (HEC) -> Splunk Indexer Cluster

• Directly inject data to a Splunk platform indexer cluster. For example:

A Kafka Connect Cluster (in containers or virtual machines or physical machines) -> Splunk Indexer Cluster (HEC)

HTTP Event Collector (HEC) requirements

• HEC token settings must be the same on all Splunk platform data injection nodes in your environment, including indexers and heavy forwarders.
• (Optional) When creating a HEC token, enable indexer acknowledgment in order to prevent potential data loss.

If indexer acknowledgment is enabled, set ackIdleCleanup to true in inputs.conf

See About the Splunk HTTP Event Collector in the Splunk Enterprise manual for more information.
See the Indexer acknowledgment section of the Splunk developer portal for more information.