HDFS-About HDFS Connector:1

About HDFS Connector

The Splunk HDFS Connector app provides a connector for sending data from Splunk to the Hadoop HDFS file system. It includes a dashboard that allows you to view status information about the connector on your Splunk indexer.

You can use the HDFS Connector in the following ways:

  • Index data on Splunk, then forward the data to HDFS
  • Index data and forward some subset of it to HDFS

How does it work?

The HDFS Connector app runs on a Splunk instance, typically an indexer. It forwards data to HDFS by way of a connector. The connector sends data to Hadoop in either of two ways:

  • It can connect directly with HDFS.
  • It can connect to HDFS through Flume.

The connector also sends status data back to Splunk, which can be viewed through a dashboard included with the app.

This manual assumes your Splunk instance is a single indexer, but you can also use the connector with a heavy forwarder or in a scaled-out, multi-indexer environment.

For a detailed description, see "What an HDFS Connector deployment looks like".

How to get HDFS Connector

The HDFS Connector app is available on Splunkbase.