Hunk Tutorial

Step 5: Configure an HDFS provider

We will now give Hunk the information it needs to work with your Hadoop directory. For more detailed information about setting up a provider, see Add or edit an HDFS provider in this manual.

Note: Before you set up a provider, make sure that Hunk has the following permissions:

  • Read-only access to the HDFS directory where your virtual index data resides.
  • Read-write access to the HDFS directory where your Splunk instance is installed. (This is usually your splunkMR directory, for example: User/hue/splunk_mr/dispatch).
  • Read-write access to the Datanode where your /tmp directory resides. This is the temp directory that the vix.splunk.home.datanode value points to in your Provider settings.

1. In Hunk Web, select Manager > Virtual Indexes in the menu bar.


2. Click the Provider tab (if it is not already selected) and click New Provider. NewProvidersButton.png

The New Provider page appears:


3. Give your provider a Name. For our example, the name is "PonyProvider".

4. Select a Provider Family, if you are using our example, the option is "Hadoop".

5. In the Java Home field, provide the path to your Java instance. The path in our example is /usr/java/jdk1.6.0_31. (You can run ls /usr/java to verify your version for this example.)

6. In the Hadoop Home field, provide the path to your Hadoop client directory. In our example we are using /usr/lib/hadoop.

7. In the Hadoop Version field, tell Hunk which version of Hadoop you are using. Our example uses Hadoop 2.x, (MRv1). Hunk currently supports MRv1 and MRv2. If you are using YARN, make sure you select it as your type.

8. In the JobTracker field, provide that host and port to where the JobTracker resides. In our example, this is localhost:8021. If you are using YARN, leave this field blank.

9. In the File System field, provide the URI to the default file system. In our example, this URI is: hdfs://localhost:8020.

10. For HDFS working directory provide the path in HDFS that you want Hunk to use as a working directory. For our example, this is /user/root/splunkmr.

11. If you are using YARN (for example, with Hortonworks for Sandbox 2.0), you must add the following new settings to the Additional Settings section. These are:

  • yarn
  • vix.yarn.resourcemanager.address= <namenode>:8050
  • vix.yarn.resourcemanager.scheduler.address= <namenode>:8030

12. Click Save.

Last modified on 28 January, 2014
Step 4: Install and license Hunk   Step 6: Set up a Virtual Index

This documentation applies to the following versions of Hunk®(Legacy): 6.0, 6.0.1, 6.0.2, 6.0.3, 6.1, 6.1.1, 6.1.2, 6.1.3, 6.2, 6.2.1, 6.2.2, 6.2.3, 6.2.4, 6.2.5, 6.2.6, 6.2.7, 6.2.8, 6.2.9, 6.2.10, 6.2.11, 6.2.12, 6.2.13, 6.3.0, 6.3.1, 6.3.2, 6.3.3, 6.3.4, 6.3.5, 6.3.6, 6.4.0, 6.4.1, 6.4.2, 6.4.3, 6.4.4, 6.4.5, 6.4.6, 6.4.7, 6.4.8, 6.4.9, 6.4.10, 6.4.11

Was this topic useful?

You must be logged into in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters