Hunk User Manual

Configure Hunk to use a streaming resource library

To use a streaming resource library you modify indexes.conf to add a provider family and one or more providers.

The [provider-family] stanza encapsulates the common configurations for a family of ERPs. It determines all of the connection information for the type of system your ERPs are running on. The [provider] stanza then provides more specific information such as the path to your Java installation, the path to your Hadoop library, and other MapReduce configurations that you want to use when running searches against that cluster.

Edit Indexes.conf

Edit indexes.conf to establish a virtual index. This is where you tell Splunk about your Hadoop cluster and about the data you want to access via virtual indexes.

If you do not already have a local copy of indexes.conf, create a copy and place it into your local directory. In this example we are using:


Note: The following changes to indexes.conf become effective at search time, no restart is necessary.

Create a provider family

The [provider-family] stanza encapsulates the connection information for the type of system streaming resource libraries run on.

You can configure multiple streaming resource libraries that share the same executable and certain basic environment variables (defined in the provider family) but then differ in the cluster details (defined in the provider), such as the JobTracker host/port or paths to data.

You must configure the provider family first. You may configure multiple providers for a provider family. Note that provider configurations can override the family.

Tell Hunk about your system by adding the following to a [provider-family] stanza in indexes.conf:

[provider-family:<your provider type, for example: mongodb>]
vix.command = java
vix.command.arg.1 = -Xmx512m
vix.command.arg.2 = -classpath

Create providers

For each different cluster in your system (provider family), you create a separate [provider] stanza.

When you create your providers, you can use and/or override the information configured for the provider family. In Hunk, the attributes in the provider stanza are merged with the family stanza, which it inherits from. The vix. prefix is stripped from each attribute and the values are passed to the MapReduce job configuration.

[provider:MyMongoProvider] <the family configured in provider family, in this example, Mongodb> = localhost:<xxxx>

Set provider configuration variables

Hunk also provides preset configuration variables for each provider you create. You can leave the preset variables in place or edit them as needed. If you want to edit them, see Provider Configuration Variables in the reference section of this manual.

Note: If you are configuring Hunk to work with YARN, you must add new settings. See "Required configuration variables for YARN" in this manual.

Set up your indexes

Once your provider family and providers are configured, you can configure one or more virtual indexes for each provider. You can implement varying degrees of filtering for virtual indexes that use streaming libraries, keeping in mind that some event fields could be generated via: lookups, calculated fields, field aliases, eventtypes, tags etc are specific to Splunk indexing, which the external system might not know how to generate.

For more information about creating virtual indexes, see the following topics:

Last modified on 10 September, 2016
About streaming resource libraries   About pass-through authentication

This documentation applies to the following versions of Hunk®(Legacy): 6.1, 6.1.1, 6.1.2, 6.1.3, 6.2, 6.2.1, 6.2.2, 6.2.3, 6.2.4, 6.2.5, 6.2.6, 6.2.7, 6.2.8, 6.2.9, 6.2.10, 6.2.11, 6.2.12, 6.2.13, 6.3.0, 6.3.1, 6.3.2, 6.3.3, 6.3.4, 6.3.5, 6.3.6, 6.3.7, 6.3.8, 6.3.9, 6.3.10, 6.3.11, 6.3.12, 6.3.13, 6.4.0, 6.4.1, 6.4.2, 6.4.3, 6.4.4, 6.4.5, 6.4.6, 6.4.7, 6.4.8, 6.4.9, 6.4.10, 6.4.11

Was this topic useful?

You must be logged into in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters