Configure Splunk forwarding to use your own SSL certificates

You can send data from forwarders to indexers using SSL certificates that you procure, rather than the ones that Splunk provides.

You can either sign these certificates by yourself or use a third party to sign them. Using your own certificates to secure Splunk communications involves the following procedures:

• Obtaining the certificates.
• Preparing the certificates for use.
• Configuring indexers to use a new signed certificate.
• Configuring forwarders to use a new signed certificate.

Prerequisites for configuring SSL certificates for forwarding

Before you can secure communications between Splunk indexers and forwarders, you must procure and prepare the certificates. You must satisfy the following conditions to properly secure communication between Splunk indexers and forwarders:

• The certificates that you procure are in Privacy-Enhanced Mail format, and the format of those files conforms with the x.509 public key certificate standard.
• The certificate key must be in RSA security format.

You can also create multiple certificates that are signed by the same certificate authority (CA) with different common names and distribute those certificates to your indexers for added security.

A common name is the name of the server that the certificate protects. When you give the forwarder the CA public key, the forwarder trusts the CA, verifies the certificate of the CA, and matches one of the SSL common names or alternative names, as you can configure using either the sslCommonNameToCheck or sslAltNameToCheck settings in the forwarder configuration file.

If you need help on creating and preparing your own certificates, see the following topics for more information:

• About securing data from forwarders
• About securing inter-Splunk communication

Configure indexers to use a signed SSL certificate

1. Open a shell or command prompt.
2. Using this prompt or your file system management tools, copy the server certificate and CA public certificate into an accessible directory on the indexer that you want to configure. For example, you can use a destination directory of $SPLUNK_HOME/etc/auth/mycerts/.
   
3. On the indexer where you want to install the certificate, use a text editor to open the $SPLUNK_HOME/etc/system/local/inputs.conf configuration file for editing.
4. In the inputs.conf file, configure the indexer to use the new server certificate. Add the following stanzas and settings to the file.

   Setting/stanza name	Data type	Description
   [splunktcp-ssl:<port>]	n/a	Defines an input to receive data over SSL on the port you specify.
   [SSL]	n/a	Defines the SSL settings for all inputs you define for this instance.
   serverCert	string	The location of the server certificate on the machine. This is the certificate that the machine uses to support inbound connections over SSL. You can use either the absolute path, such as /opt/splunk/etc/auth/mycerts or you can use a relative path, such as etc/auth/mycerts and the instance uses the Splunk Enterprise installation directory.
   sslPassword (Optional)	string	The password that you entered when you created the certificate, if you created a password.
   requireClientCert	Boolean	Whether or not Splunk Enterprise requires that a connecting client present a valid SSL certificate before the connection can succeed. A value of "true" means that the receiving instance must see a valid certificate to let the client authenticate. A value of "false" means that clients can connect without presenting a certificate. Configure this setting to "true" if you want your receivers to require authentication with certificates.
   sslVersions (Optional)	comma-separated list	The list of SSL versions that the receiver supports.
   cipherSuite (Optional)	string	The list of cipher suite strings that the input processors are to use.
   sslCommonNameToCheck (Optional except in certain circumstances)	comma-separated list	A list of one or more common names upon which the Splunk Enterprise receiving instance checks for a match in the certificate that the client presents upon connecting. This setting is only valid if you have configured the 'requireClientCert' setting with a value of "true". If none of the common names in this setting value matches the common name in the certificate of the connecting client, the instance declines the connection as not authorized.
   sslAltNameToCheck (Optional except in certain circumstances)	comma-separated list	A list of one or more alternate names upon which the Splunk Enterprise receiving instance checks for a match n the certificate that the client presents upon connecting. This setting is only valid if you have configured the 'requireClientCert' setting with a value of "true". If none of the alternate names in this setting value matches the alternate name in the certificate of the connecting client, the instance declines the connection as not authorized.

5. Save the inputs.conf file and close it.
6. On indexers that do not run on Windows, open the $SPLUNK_HOME/etc/system/local/server.conf configuration file for editing.
7. Add the following text to establish the location of the certificate authority certificate.
   

   sslRootCAPath = <Absolute path to the CA certificate. The default value is 
   $SPLUNK_HOME/etc/auth/cacert.pem>
   

8. Save the server.conf file and close it.
9. Using the CLI, restart the splunkd process:

   # $SPLUNK_HOME/bin/splunk restart splunkd
   

Configuration file examples for configuring SSL certificates on receiving indexers

Following is an example of an inputs.conf configuration file on a receiving indexer.

[splunktcp-ssl:9997]
disabled=0

[SSL]
serverCert = /opt/splunk/etc/auth/mycerts/myServerCert.pem
sslPassword = myCertificatePassword
requireClientCert = true
sslVersions *,-ssl2
sslCommonNameToCheck = indexer1.mycompany.com,indexer2.mycompany.com

When you make edits to the $SPLUNK_HOME/etc/system/local/inputs.conf configuration file to install certificates, if you supply a password for your server certificate, Splunk Enterprise encrypts that password from cleartext when you restart Splunk Enterprise. This encryption does not occur in directories that are outside of $SPLUNK_HOME/etc/system/local.

The server.conf configuration file establishes and references the location of the certificate authority certificate:

[sslConfig]
sslRootCAPath = /opt/splunk/etc/auth/mycerts/myCACertificate.pem

Configure forwarders to use a signed SSL certificate

1. Generate a new certificate. For instructions on how to create a certificate, see How to self-sign certificates.
2. Copy the new certificate and the CA public certificate myCACertificate.pem into an accessible folder on the forwarders you want to configure. For example, you can use a destination folder of $SPLUNK_HOME/etc/auth/mycerts/.
   

   If you configure inputs.conf or outputs.conf in an app directory, the forwarder does not encrypt the passwords and the clear-text value remains in the file. You might want to create different certificates (signed by the same root CA) to use when configuring SSL in app directories.

3. Define the [tcpout] stanza in $SPLUNK_HOME/etc/system/local/outputs.conf (or in the appropriate directory of any app you use to distribute your forwarding configuration):

   Setting/stanza name	Data type	Description
   [tcpout:<name>]	n/a	Defines an output group to send data to a receiver.
   server	string	The hostname or IP address and port on which to connect securely to forward data.
   clientCert	string	The location of the client certificate on the machine. This is the certificate that the forwarder uses to connect to the receiving indexer over SSL. You can use either the absolute path, such as /opt/splunk/etc/auth/mycerts or you can use a relative path, such as etc/auth/mycerts and the instance uses the forwarder installation directory.
   useClientSSLCompression	Boolean	Whether or not the client performs SSL compression when connecting and communicating with servers. The default value of "true" means that the client uses SSL compression. A value of "false" means the client doesn't use compression. Disabling compression, particularly with transport layer security (TLS) can cause problems with bandwidth.
   sslPassword (Optional)	string	Same as the setting in the inputs.conf configuration file
   sslVerifyServerCert (Optional)	Boolean	Whether or not, upon connection to a receiver, the forwarder confirms that the receiver has a valid SSL server certificate. A value of "true" means that the forwarder checks for a valid server certificate upon connection, then checks the common or alternate names against the names in the server certificate against the names in the values for the 'sslCommonNameToCheck' and 'sslAltNameToCheck' settings on the forwarder. If there is no match against the common or alternate names, the client aborts the connection to the receiver as not authorized.
   sslCommonNameToCheck (Optional)	comma-separated list	Same as the setting in the inputs.conf configuration file, except that you must give the 'sslVerifyServerCert' setting a value of "true" in the outputs.conf configuration file and the client does the certificate verification.
   sslAltNameToCheck (Optional)	comma-separated list	Same as the setting in the inputs.conf configuration file, except that you must give the 'sslVerifyServerCert' setting a value of "true" in the outputs.conf configuration file and the client does the certificate verification.
   cipherSuite (Optional)	comma-separated list	Same as the setting in the inputs.conf configuration file.

4. Save the outputs.conf file and close it.
5. On forwarders that do not run on Windows, open the server.conf configuration file for editing.
6. Add the following stanza and settings to the file:
   

   [sslConfig]
   sslRootCAPath = <absolute path to the certificate authority certificate. For example, the default value 
   is $SPLUNK_HOME/etc/auth/cacert.pem>
   

7. Save the server.conf file and close it.
8. Restart the splunkd process.
   

   $SPLUNK_HOME/bin/splunk restart splunkd
   

Configuration file examples for configuring SSL certificates on forwarders

Following is an example of an outputs.conf configuration file on a receiving indexer.

[tcpout:group1]
server=10.1.1.197:9997
disabled = 0
clientCert = /opt/splunk/etc/auth/mycerts/myClientCert.pem
useClientSSLCompression = true
sslPassword = myCertificatePassword
sslCommonNameToCheck = indexer1.mycompany.com,indexer2.mycompany.com
sslAltNameToCheck = indexer3.mycompany.com
sslVerifyServerCert = true 

When you make edits to the $SPLUNK_HOME/etc/system/local/outputs.conf configuration file to install certificates, if you supply a password for your server certificate, Splunk Enterprise encrypts that password from cleartext when you restart Splunk Enterprise. This encryption does not occur in directories that are outside of $SPLUNK_HOME/etc/system/local.

The server.conf configuration file establishes and references the location of the certificate authority certificate. You must configure this on the forwarder even though it is the client:

[sslConfig]
sslRootCAPath = /opt/splunk/etc/auth/mycerts/myCACertificate.pem

Forward data over SSL to more than one indexer

If you need to forward data securely to multiple indexers, complete the following procedure:

1. On the forwarder where you want to send data to multiple indexers, use the "Configure forwarders to use a signed certificate" procedure to open and make changes to the outputs.conf configuration file.
2. In the target output group definition stanza for the forwarder, add a host:port line for each indexer to which you want to send data over SSL. Separate multiple entries with commas.
3. Save the outputs.conf file and close it.
4. Restart the forwarder.

The following example outputs.conf file uses the same certificate for the indexer and the forwarders:

[tcpout]

[tcpout:group1]
server = 10.1.12.112:9997,10.1.12.111:9999
# multiple servers: 10.1.12.112:9997, 10.1.12.111:9999
disabled = 0
clientCert = $SPLUNK_HOME/etc/auth/client.pem
useClientSSLCompression = true 
# Defaults to the value set in the useClientSSLCompression 
# setting set in server.conf.
sslPassword = <password for the client certificate>
sslCommonNameToCheck = indexercn.example.org
sslVerifyServerCert = true

Forward data over SSL to multiple indexers using certificates with different common names

You can create and configure a server certificate for each indexer by configuring the outputs.conf file on the forwarder with an [SSLConfig] stanza for each indexer.

If you have created one server certificate for each indexer and you have set a unique sslCommonNameToCheck or sslAltNameToCheck in each indexer certificate to be checked by the forwarders, you must configure one [tcpout-server://host:port] configuration stanza for each indexer in outputs.conf. This action lets you specify which name to check for each indexer.

Next steps

Check the forwarder-indexer configuration to make sure it works. See Validate your configuration.