Hunk User Manual

Add or edit an HDFS provider in the user interface

Before you add a new virtual index, you set up the provider for that index. You can set up multiple providers with multiple indexes for one provider. When you add a virtual index, you need to have the following information at hand:

  • The host name and port for the NameNode of the Hadoop cluster.
  • The host name and port for the JobTracker of the Hadoop cluster.
  • Installation directories of Hadoop command line libraries and Java installation.
  • Path to a writable directory on the DataNode/TaskTracker *nix filesystem, the one for which the Hadoop user account has read and write permission.
  • Path to a writable directory in HDFS that can be used exclusively by Splunk on this search head.

You can also add HDFS providers and virtual indexes by editing indexes.conf. See "Set up a virtual index" in this manual for instructions on setting up virtual indexes in the configuration file.

Add a provider

1. In the top menu, select Settings > Virtual Indexes.

2. Select the Providers tab in the Virtual Indexes page and click New Provider or the name of the provider you want to edit.

3. The Add New/Edit Provider page, give your provider a Name.

4. Select the Provider Family in the drop down list (note that this field cannot be edited).

5. Provide the following Environment Variables:

  • Java Home: provide the path to your Java instance.
  • Hadoop Home: Provide the path to your Hadoop client directory.

6. Provide the following Hadoop Cluster Information:

  • Hadoop Version: Tell Hunk which version of Hadoop the cluster is running one of: Hadoop 1.0, Hadoop 2.0 with MRv1 or Hadoop 2.0 with Yarn
  • JobTracker: Provide the path to the Job Tracker.
  • File System: Provide the path to the default file system.

7. Provide the following Splunk Settings:

  • HDFS working directory: This is a path in HDFS (or whatever the default file system is) that you want Hunk to use as a working directory.
  • Job queue: This is job queue where you want the MapReduce jobs for this provider to be submitted to.

8. Click Add Secure Cluster to configure security for the cluster and provide your Kerberos Server configuration. For more about configuring Kerberos for your cluster, see "Configure Kerberos authentication" in this manual. AddSecureCluster.png

9. The Additional Settings fields specify your provider configuration variables. Hunk populates these preset configuration variables for each provider you create. You can leave the preset variables in place or edit them as needed. If you want to learn more about these settings, see Provider Configuration Variables in the reference section of this manual.

Note: If you are configuring Hunk to work with YARN, you must add new settings. See "Required configuration variables for YARN" in this manual.

9. Click Save.

Last modified on 21 February, 2016
About the Hunk user interface   Add or edit a virtual index in the user interface

This documentation applies to the following versions of Hunk®(Legacy): 6.0, 6.0.1, 6.0.2, 6.0.3, 6.1, 6.1.1, 6.1.2, 6.1.3, 6.2, 6.2.1, 6.2.2, 6.2.3, 6.2.4, 6.2.5, 6.2.6, 6.2.7, 6.2.8, 6.2.9, 6.2.10, 6.2.11, 6.2.12, 6.2.13, 6.3.0, 6.3.1, 6.3.2, 6.3.3, 6.3.4, 6.3.5, 6.3.6, 6.3.7, 6.3.8, 6.3.9, 6.3.10, 6.3.11, 6.3.12, 6.3.13, 6.4.0, 6.4.1, 6.4.2, 6.4.3, 6.4.4, 6.4.5, 6.4.6, 6.4.7, 6.4.8, 6.4.9, 6.4.10, 6.4.11

Was this topic useful?

You must be logged into in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters