Add or edit an HDFS provider in the user interface
Before you add a new virtual index, you set up the provider for that index. You can set up multiple providers with multiple indexes for one provider. When you add a virtual index, you need to have the following information at hand:
- The host name and port for the NameNode of the Hadoop cluster.
- The host name and port for the JobTracker of the Hadoop cluster.
- Installation directories of Hadoop command line libraries and Java installation.
- Path to a writable directory on the DataNode/TaskTracker *nix filesystem, the one for which the Hadoop user account has read and write permission.
- Path to a writable directory in HDFS that can be used exclusively by Splunk on this search head.
Add a provider
1. In the top menu, select Settings > Virtual Indexes.
2. Select the Providers tab in the Virtual Indexes page and click New Provider or the name of the provider you want to edit.
3. The Add New/Edit Provider page, give your provider a Name.
4. Select the Provider Family in the drop down list (note that this field cannot be edited).
5. Provide the following Environment Variables:
- Java Home: provide the path to your Java instance.
- Hadoop Home: Provide the path to your Hadoop client directory.
6. Provide the following Hadoop Cluster Information:
- Hadoop Version: Tell Hunk which version of Hadoop the cluster is running one of: Hadoop 1.0, Hadoop 2.0 with MRv1 or Hadoop 2.0 with Yarn
- JobTracker: Provide the path to the Job Tracker.
- File System: Provide the path to the default file system.
7. Provide the following Splunk Settings:
- HDFS working directory: This is a path in HDFS (or whatever the default file system is) that you want Hunk to use as a working directory.
- Job queue: This is job queue where you want the MapReduce jobs for this provider to be submitted to.
8. Click Add Secure Cluster to configure security for the cluster and provide your Kerberos Server configuration. For more about configuring Kerberos for your cluster, see "Configure Kerberos authentication" in this manual.
9. The Additional Settings fields specify your provider configuration variables. Hunk populates these preset configuration variables for each provider you create. You can leave the preset variables in place or edit them as needed. If you want to learn more about these settings, see Provider Configuration Variables in the reference section of this manual.
Note: If you are configuring Hunk to work with YARN, you must add new settings. See "Required configuration variables for YARN" in this manual.
9. Click Save.
About the Hunk user interface
Add or edit a virtual index in the user interface
This documentation applies to the following versions of Hunk®(Legacy): 6.0, 6.0.1, 6.0.2, 6.0.3, 6.1, 6.1.1, 6.1.2, 6.1.3, 6.2, 6.2.1, 6.2.2, 6.2.3, 6.2.4, 6.2.5, 6.2.6, 6.2.7, 6.2.8, 6.2.9, 6.2.10, 6.2.11, 6.2.12, 6.2.13, 6.3.0, 6.3.1, 6.3.2, 6.3.3, 6.3.4, 6.3.5, 6.3.6, 6.3.7, 6.3.8, 6.3.9, 6.3.10, 6.3.11, 6.3.12, 6.3.13, 6.4.0, 6.4.1, 6.4.2, 6.4.3, 6.4.4, 6.4.5, 6.4.6, 6.4.7, 6.4.8, 6.4.9, 6.4.10, 6.4.11