Required configuration variables for YARN
Splunk Analytics for Hadoop reaches End of Life on January 31, 2025.
If you are using YARN, you must add resource manager settings to the configuration variable settings:
- vix.mapreduce.framework.name = yarn
- vix.yarn.resourcemanager.address= <namenode>:<port>
- vix.yarn.resourcemanager.scheduler.address= <namenode>:<port>
If you installed Hortonworks for Sandbox 2.0, add the following settings/port:
- vix.yarn.resourcemanager.address= <namenode>:8050
- vix.yarn.resourcemanager.scheduler.address= <namenode>:8030
If you are using the Cloudera VM for Yarn, add the following settings/port:
- vix.yarn.resourcemanager.address = <your namenode>:8032
- vix.yarn.resourcemanager.scheduler.address = <your namenode>:8030
If your cluster does not use the default configuration values you must add your configurations to the search head. (For example, if the value of property yarn.application.classpath
in yarn-site.xml
is different than the value in yarn-default.xml
on the cluster.)
To update your search head, either:
- Set the value in
yarn-site.xml
of the Hadoop CLI on the search head - Set the value in provider to
vix.yarn.application.classpath
variable
YARN for High Availabilty
For YARN using HA (High Availability), you must add the following variables:
- HA - YARN [CDH5.3 YARN(MR2 Included)]
Setting: | Use it to: |
---|---|
vix.yarn.resourcemanager.ha.rm-ids
|
comma separated list of resource manager ids, in this example: rm57,rm40
|
vix.yarn.resourcemanager.address.rm57
|
The address of the applications manager interface in the ResourceManager. In this example test-piv-cent65x64-003.sv.splunk.com:8032
|
vix.yarn.resourcemanager.address.rm40
|
The address of the applications manager interface in the ResourceManager, for rm40: In this example, #test-piv-cent65x64-004.sv.splunk.com:8032
|
vix.yarn.resourcemanager.scheduler.address.rm57
|
The address of the scheduler interface in the ResourceManagerfor rm57. In this example, test-piv-cent65x64-003.sv.splunk.com:8030
|
vix.yarn.resourcemanager.scheduler.address.rm40
|
The address of the scheduler interface in the ResourceManager rm40. In this example: test-piv-cent65x64-004.sv.splunk.com:8030
|
vix.yarn.resourcemanager.ha.enabled
|
Set to true |
vix.yarn.resourcemanager.cluster-id
|
Cluster ID used when ResourceManager is Highly Available.> # yarnRM |
vix.yarn.application.classpath
|
$HADOOP_CLIENT_CONF_DIR,
$HADOOP_CONF_DIR, $HADOOP_COMMON_HOME/*, $HADOOP_COMMON_HOME/lib/*, $HADOOP_HDFS_HOME/*, $HADOOP_HDFS_HOME/lib/*, $HADOOP_YARN_HOME/*, $HADOOP_YARN_HOME/lib/* |
vix.mapreduce.application.classpath
|
= $HADOOP_MAPRED_HOME/*,$HADOOP_MAPRED_HOME/lib/* |
Virtual archive index configuration variables | Providers |
This documentation applies to the following versions of Splunk® Enterprise: 7.0.0, 7.0.1, 7.0.2, 7.0.3, 7.0.4, 7.0.5, 7.0.6, 7.0.7, 7.0.8, 7.0.9, 7.0.10, 7.0.11, 7.0.13, 7.1.0, 7.1.1, 7.1.2, 7.1.3, 7.1.4, 7.1.5, 7.1.6, 7.1.7, 7.1.8, 7.1.9, 7.1.10, 7.2.0, 7.2.1, 7.2.2, 7.2.3, 7.2.4, 7.2.5, 7.2.6, 7.2.7, 7.2.8, 7.2.9, 7.2.10, 7.3.0, 7.3.1, 7.3.2, 7.3.3, 7.3.4, 7.3.5, 7.3.6, 7.3.7, 7.3.8, 7.3.9, 8.0.0, 8.0.1, 8.0.2, 8.0.3, 8.0.4, 8.0.5, 8.0.6, 8.0.7, 8.0.8, 8.0.9, 8.0.10, 8.1.0, 8.1.1, 8.1.2, 8.1.3, 8.1.4, 8.1.5, 8.1.6, 8.1.7, 8.1.8, 8.1.9, 8.1.10, 8.1.11, 8.1.12, 8.1.13, 8.1.14, 8.2.0, 8.2.1, 8.2.2, 8.2.3, 8.2.4, 8.2.5, 8.2.6, 8.2.7, 8.2.8, 8.2.9, 8.2.10, 8.2.11, 8.2.12, 9.0.0, 9.0.1, 9.0.2, 9.0.3, 9.0.4, 9.0.5, 9.0.6, 9.0.7, 9.0.8, 9.0.9, 9.0.10, 9.1.0, 9.1.1, 9.1.2, 9.1.3, 9.1.4, 9.1.5, 9.2.0, 9.2.1, 9.2.2, 9.3.0
Feedback submitted, thanks!