Configuration file directories

A Splunk Enterprise installation can have multiple versions of a configuration file located across several directories. For example, you might have the same configuration file with different settings located in each of the default, local, and app directories. Splunk Enterprise uses a layering scheme and rules to evaluate overlapping configurations and prioritize them.

When you need to override a setting that's been defined as a default, you can place a customized configuration file in a different folder path under the Splunk Enterprise installation. For a description and examples of how precedence is determined, see Configuration file precedence.

A detailed list of settings for each configuration file is provided in the .spec file named for that configuration file. You can find the latest version of the .spec and .example files in the $SPLUNK_HOME/etc/system/README folder of your Splunk Enterprise installation, or in the documentation at the configuration file reference.

About the default files

The default directory contains preconfigured versions of the configuration files with default settings. The location of the default directory in a Splunk Enterprise installation is $SPLUNK_HOME/etc/system/default.

"all these worlds are yours, except /default - attempt no editing there" -- duckfez, 2010

You should never change a configuration file that's located in the $SPLUNK_HOME/etc/system/default directory. The Splunk Enterprise upgrade process overwrites the contents in that folder automatically, which will remove any changes. If you want to retain a setting you've changed through an upgrade, place your configuration file into a local folder path such as $SPLUNK_HOME/etc/system/local or $SPLUNK_HOME/etc/apps/$app_name/local as described below.

The upgrade process also inspects the content in the $SPLUNK_HOME/etc/system/local folder path. An upgrade usually does not make changes to the local configuration files, but if changes are made they are noted in the configuration file or in the migration log. You can choose to preview the changes to your customized configuration files as part of the upgrade process before any changes are made.

Where you can place (or find) your modified configuration files

To change the settings in a particular configuration file, you must first create a new file of the same name in a non-default directory, and add the required settings and changed values to your new configuration file. A setting with a new value defined in a non-default directory will take precedence over a setting defined in the default directory.

When changing a default setting using a new configuration file, you only need to define the stanza category, the setting, and update the value. Do not make a complete copy of the configuration file from the default directory into another folder, as the settings in that copy will take precedence and override changes made during an upgrade.

The following is the configuration directory structure in $SPLUNK_HOME/etc:

$SPLUNK_HOME/etc/system/local

Local changes on a site-wide basis go here; for example, settings you want to make available to all apps. If the configuration file you're looking for doesn't already exist in this directory, create it and verify the service account has permissions to it.

$SPLUNK_HOME/etc/slave-apps/[_cluster|<app_name>]/[local|default]

For indexer cluster peer nodes only.

The subdirectories under $SPLUNK_HOME/etc/slave-apps contain configuration files that are common across all peer nodes.

DO NOT change the content of these subdirectories on the cluster peer itself. Instead, use the cluster manager node to distribute any new or modified files to them.

The _cluster directory contains configuration files that are not part of real apps but that still need to be identical across all peers. A typical example is the indexes.conf file.

For more information, see Update common peer configurations in the Managing Indexers and Clusters manual.

$SPLUNK_HOME/etc/apps/<app_name>/[local|default]

If you're in an app when a configuration change is made, the setting goes into a configuration file in the app's /local directory. For example, edits for search-time settings in the Search app go here: $SPLUNK_HOME/etc/apps/search/local/.

If you want to edit a configuration file so that the change only applies to a certain app, copy the file to the app's /local directory, verify permissions, and make your changes there.

$SPLUNK_HOME/etc/users

User-specific configuration changes go here.

$SPLUNK_HOME/etc/system/README

This directory contains supporting reference documentation. For most configuration files, there are two reference files: .spec and .example; for example, inputs.conf.spec and inputs.conf.example. The .spec file specifies the syntax, including a list of available attributes and variables. The .example file contains examples of real-world usage.