Step 5: Configure an HDFS provider
We will now give Hunk the information it needs to work with your Hadoop directory. For more detailed information about setting up a provider, see Add or edit an HDFS provider in this manual.
Note: Before you set up a provider, make sure that Hunk has the following permissions:
- Read-only access to the HDFS directory where your virtual index data resides.
- Read-write access to the HDFS directory where your Splunk instance is installed. (This is usually your
splunkMRdirectory, for example:User/hue/splunk_mr/dispatch). - Read-write access to the Datanode where your
/tmpdirectory resides. This is the temp directory that thevix.splunk.home.datanodevalue points to in your Provider settings.
1. In Hunk Web, select Manager > Virtual Indexes in the menu bar.
2. Click the Provider tab (if it is not already selected) and click New Provider.
The New Provider page appears:
3. Give your provider a Name. For our example, the name is "PonyProvider".
4. Select a Provider Family, if you are using our example, the option is "Hadoop".
5. In the Java Home field, provide the path to your Java instance. The path in our example is /usr/java/jdk1.6.0_31. (You can run ls /usr/java to verify your version for this example.)
6. In the Hadoop Home field, provide the path to your Hadoop client directory. In our example we are using /usr/lib/hadoop.
7. In the Hadoop Version field, tell Hunk which version of Hadoop you are using. Our example uses Hadoop 2.x, (MRv1). Hunk currently supports MRv1 and MRv2. If you are using YARN, make sure you select it as your type.
8. In the JobTracker field, provide that host and port to where the JobTracker resides. In our example, this is localhost:8021. If you are using YARN, leave this field blank.
9. In the File System field, provide the URI to the default file system. In our example, this URI is: hdfs://localhost:8020.
10. For HDFS working directory provide the path in HDFS that you want Hunk to use as a working directory. For our example, this is /user/root/splunkmr.
11. If you are using YARN (for example, with Hortonworks for Sandbox 2.0), you must add the following new settings to the Additional Settings section. These are:
- vix.mapreduce.framework.name= yarn
- vix.yarn.resourcemanager.address= <namenode>:8050
- vix.yarn.resourcemanager.scheduler.address= <namenode>:8030
12. Click Save.
| Step 4: Install and license Hunk | Step 6: Set up a Virtual Index |
This documentation applies to the following versions of Hunk®(Legacy): 6.0, 6.0.1, 6.0.2, 6.0.3, 6.1, 6.1.1, 6.1.2, 6.1.3, 6.2, 6.2.1, 6.2.2, 6.2.3, 6.2.4, 6.2.5, 6.2.6, 6.2.7, 6.2.8, 6.2.9, 6.2.10, 6.2.11, 6.2.12, 6.2.13, 6.3.0, 6.3.1, 6.3.2, 6.3.3, 6.3.4, 6.3.5, 6.3.6, 6.4.0, 6.4.1, 6.4.2, 6.4.3, 6.4.4, 6.4.5, 6.4.6, 6.4.7, 6.4.8, 6.4.9, 6.4.10, 6.4.11
Download manual
Feedback submitted, thanks!