Hardware and software requirements for the Splunk Add-on for Kafka

Splunk admin requirements

To install and configure the Splunk Add-on for Kafka, you must be member of the admin or sc_admin role.

Kafka requirements

The Splunk Add-on for Kafka collects data from machines running Apache Kafka version 0.8.2.

Dependencies

Splunk Add-on for JMX

To collect performance metrics from your Kafka clusters, this add-on relies on the Splunk Add-on for Java Management Extensions. Install the Splunk Add-on for JMX to your search heads and to at least one heavy forwarder.

Snappy binding

If you have the Snappy compression method enabled when injecting data into Kafka, install a Snappy binding to allow this add-on to support Snappy Kafka messages. See Kafka data injected with Snappy compression enabled.

Sizing guidelines

There are no sizing guidelines for the monitor inputs.

For the JMX input, dedicate one heavy forwarder for the JMX input. A single heavy forwarder can handle the data for all six JMX collection templates included with this add-on.

For the modular inputs that collect Kafka topic messages, your sizing requirements depend on how much data per second producers push to the Kafka topics from which you want to collect data. The Splunk Add-on for Kafka can handle millions of records per minute, but indexing depends on splunkd capacity. If splunkd can process N records per second, and there are M records per second flowing into the Kafka topics, you need a capacity of M divided by N in your heavy forwarders.

Splunk platform requirements

Because this add-on runs on the Splunk platform, all of the system requirements apply for the Splunk software that you use to run this add-on.

• For Splunk Enterprise system requirements: see System Requirements in the Splunk Enterprise Installation Manual.
• For Splunk Light system requirements: see System Requirements in the Splunk Light Installation Manual.
• If you are managing on-premises forwarders to get data into Splunk Cloud, see System Requirements in the Splunk Enterprise Installation Manual, which includes information about forwarders.

For information about installation locations and environments, see Install the Splunk Add-on for Kafka.