Deploy secure passwords across multiple servers

At initial startup, Splunk Enterprise creates a file $SPLUNK_HOME/etc/auth/splunk.secret. This file contains a key used to encrypt some of your authentication information in configuration files. Each of the following files can be encrypted across a deployment using splunk.secret. Note that the passwords and encryption methods used for each file are not necessarily interchangeable.

• web.conf: Your SSL passwords on every instance.
• authentication.conf: Your LDAP passwords, if you have any.
• inputs.conf: Your SSL passwords, if you use splunktcp-ssl to set up ssl for data distribution.
• outputs.conf:: Your SSL passwords, if you use splunktcp-ssl to configure splunktcp-ssl in inputs.conf to set up ssl for data distribution.
• server.conf: pass4symmkey, if you have one.
• passwords.conf: Your password for a given app.

When Splunk software starts, if it detects a clear-text password, in one of these settings, it will create or overwrite the configuration in the equivalent local folder with the encrypted password.

Note: If the pass4symmkey or SSLPassword is specified in a default apps file, the password is obfuscated in the local version of the file upon restart. The default version of the file remains in clear text. However, if the file is listed using curl or a splunkd endpoint, the passwords appear encrypted.

When you deploy Splunk software on multiple servers, you must encrypt the passwords and ensure that they are consistent across your deployment. Splunk recommends that you use an encryption tool such as Vault's "secret/*" to secure your passwords as you distribute them.

You should perform these steps at initial deployment and also any time you need to deploy a new password for your instances:

1. Using your encryption tool, deploy the passwords to all servers.

2. On each server, place the password, in clear text, in the relevant file.

3. Immediately start/restart to encrypt all the passwords under the server's unique secret.

In a search head cluster, the captain replicates its splunk.secret file to all other cluster members during initial deployment of the cluster, so you do not need to copy it manually. As part of its normal operation, the cluster also automatically replicates any credentials that are stored by apps for their own use.