Steps for securing your Splunk Enterprise deployment with TLS

The general workflow to secure your Splunk platform deployment with TLS follows:

1. Decide how you want to secure your Splunk platform deployment. This determines how much securing work you actually do.
   • If you use Splunk Cloud Platform, your Splunk Cloud Platform infrastructure already has certificates that protect it. Splunk provides and maintains these certificates, including the certificates that protect connection between forwarders and SCP.. You can consider the following options for the data collection and forwarding infrastructure that you manage and that sends data to your Splunk Cloud Platform instance.
   • You can choose to secure communications between Splunk Web and your browser, or Splunk Web and the Splunk platform deployment
   • You can also secure communications between individual Splunk platform instances. This is similar to securing Splunk Web, but has a slightly different procedure
   • You can secure both of these types of communication. This provides the best level of security but takes additional time and requires a better understanding of your Splunk platform deployment and its position in the network.
2. Obtain the TLS certificates that you need to secure the deployment in the way you want
   • You can get the certificates from a third party, or
   • You can create the certificates yourself
3. Verify that the certificates are valid
4. Install the certificates on each Splunk platform instance
5. Configure each Splunk platform instance to use the certificates
6. If necessary, configure your domain name service (DNS) registry to account for the information that the certificates contain
7. Test and troubleshoot

This topic is a high-level workflow for obtaining and installing certificates for Splunk Platform instances that you manage. It does not contain specifics on configurations, certificate store locations, or certificate composition requirements. See the links in each section that follows for specific details on that section, or read through the topic and proceed to the Next Steps section to continue into each section to begin installing and configuring certificates.

1. Decide how you want to secure your Splunk platform deployment

This first step in the workflow is the most challenging because it is the most open-ended and changes based on your specific situation. As a Splunk administrator, it's your responsibility to ensure that the data that your Splunk Enterprise or Splunk Cloud Platform supporting infrastructure is protected. How you protect that data depends on the topology of your network, the number of instances in your Splunk platform infrastructure, and the sensitivity of your data. While Splunk best practice is to secure all parts of your Splunk platform infrastructure, your use case might limit the ability to do so.

When you decide how to secure your Splunk platform infrastructure, you should take into account the following:

• Does any of your infrastructure use the internet to communicate with other parts of your infrastructure, even to connect with Splunk Cloud Platform?
• Similarly, do you have external users that connect to your infrastructure over the internet to use the Splunk platform?
• Do you need to securely connect your forwarding tier infrastructure to Splunk Cloud Platform?

Answers to these questions determine what you secure in your infrastructure, and where. As a reminder, Splunk best practice is to secure all contact points of your Splunk platform deployment wherever you can, even if you protect that deployment with a firewall.

There are two basic areas in Splunk platform infrastructure that you can secure:

Secure Inter-Splunk communications

You can use TLS certificates to secure communications that involve communication between individual nodes in your Splunk infrastructure. This involves almost any kind of node that does not use the Splunk Web interface. It's also known as "Splunk-to-Splunk" communication. This type of exchange involves communication between things like

• Forwarders and indexers
• Search heads and search peers
• Any kind of clustered instance, like an indexer or search head cluster, or a search head deployer
• A deployment server and a deployment client
• License managers and license peers
• The Splunk CLI and any Splunk instance you want to connect to using the CLI
• The Splunk App Key Value Store service
• Python modules within the Splunk platform installation

Secure Splunk Web communications

You secure Splunk Web communications similarly to, but slightly different than, you do communications between Splunk platform instances. There is a configuration for connecting your browser with Splunk Web, and another for connecting Splunk Web with search heads and search head clusters.

2. Obtain the certificates that you need to secure your Splunk platform deployment

After you determine how you want to protect the infrastructure, the next step is to get the TLS certificates that perform this protection.

With Splunk Cloud Platform, it's straightforward. Splunk protects the contact points in your Splunk Cloud Platform instance with TLS technology, updates the certificates for you as needed, and provides the Universal Forwarder Credentials Package which you can use to connect forwarders to send data to Splunk Cloud Platform.

On Splunk Enterprise and for connections between instances in your Splunk Cloud Platform-supporting infrastructure, this work is your responsibility. You must obtain, install, and configure certificates to protect data communication between Splunk platform instances.

Obtain certificates from a third party

You can contact a third party certificate authority (CA) and obtain a signed certificate from them to install into your Splunk platform infrastructure. This is the most secure option that is available because third party CAs are known entities.

Caveats to using this method include a delay in receiving the certificates and likely a cost for obtaining them, depending on the certificate authority you use and how you obtain the certificates.

See the following topics for specifics:

• How to obtain third-party certificates for inter-Splunk communication
• How to obtain third-party certificates for Splunk Web

Create and sign certificates yourself

In this scenario, you are the certificate authority.

If you have experience creating and signing TLS certificates, you can generate, sign, and install the certificates into your Splunk platform deployment, then configure the deployment to use the certificates. You can use any method you are comfortable with to generate the certificates, as long as the certificates you generate are in the correct file format and meet the requirements for the level of security you want to use them.

See the following topic for specifics:

How to create and sign your own TLS certificates

3. Verify that the certificates are valid

After you get the certificates, and before you install them, you need to verify that the certificates you received are valid.

• The certificates must be in the correct file format, usually the privacy-enhanced mail (PEM) format. If they aren't, you have to convert them to this format
• The certificate must contain the correct information as the certificate authority provided it
• The certificate must contain the information that you provided as part of setting up the certificate. For example, it must contain information about the domains, or Common Names, that you want the certificate to protect. If it is a wildcard certificate, which is one that protects multiple domains, it must also contain any subdomains that you want protected.
• It must be valid for the dates that you want the protection to be in force. You can't extend a certificate that's expired, instead you must replace it with a new one. Splunk Enterprise won't run with a certificate that hasn't yet come into force.

See the following topic for specifics:

• How to prepare TLS certificates for use with the Splunk platform

4. Install the certificates on each Splunk platform instance

After you obtain the certificates, you can install and configure the instances to see and use them.

Generally, you can install a certificate directly into the certificate store on your Splunk platform instance by creating a directory within that certificate store and placing the files you downloaded there.

Depending on the complexity of your deployment, and the interaction with any existing certificates you might have in the deployment, you might need to perform additional steps to make the certificates work in the deployment:

• You might need to convert the format of your certificate if it isn't in the required format.
• You might need to concatenate certificates, especially if your environment uses multiple certificates or certificate chains as part of a securement strategy that supersedes your Splunk platform deployment. Splunk platform instances must see a complete certificate chain to operate properly.

See the following topics for specifics:

• Configure TLS certificates for Splunk indexing and forwarding
• Configure TLS certificates for inter-Splunk communication
• Configure TLS certificates for Splunk Web

5. Configure each Splunk platform instance to use the certificates

After you have installed the certificates into your Splunk platform installation, your Splunk platform deployment needs to know where to find and use them. In nearly all cases, you perform this configuration using Splunk configuration files.

The configuration files you use, and the location within each configuration file where you do the configuration, depends on several factors:

• The Splunk platform instance function: indexer, search head, cluster node, and so on
• The Splunk service whose contact points you want to protect. For example, the Splunk daemon, the CLI, App Key Value Store, Python modules, and so on

When you perform configurations, you follow the same rules for configuring the location and usage of certificates that you would when you set up other parts of a configuration file. Configuration file precedence and other rules apply.

See the following topics for specifics:

• Configure TLS certificates for Splunk indexing and forwarding
• Configure TLS certificates for inter-Splunk communication
• Configure TLS certificates for Splunk Web

6. Configure your Domain Name Services (DNS) registry properly

DNS is an integral part of securing your Splunk platform infrastructure.

While you can obtain a certificate for one or more IP addresses from a public CA, the addresses must be public, and you must demonstrate that you own them. If you can't, they won't issue that kind of certificate to you. They also won't issue certificates for private IP addresses.

You must ensure your DNS is properly configured, even if you host the deployment on a completely private network. All machines that run Splunk deployment infrastructure must have properly configured DNS.

This is because secure communications using certificates must leverage either a Common Name or a Subject Alternative Name that is present in the certificates that you obtain and deploy. While it's possible to use IP addresses, even private ones, in certificates, using fully-qualified domain names (FQDN) and having DNS perform proper resolution eases the certificate configuration process significantly.

1. Review your DNS configuration. Ensure that DNS servers run the latest versions of DNS software.
2. Confirm that DNS records and references are properly configured.
3. Test DNS on each Splunk platform instance. At a minimum, the machine should be able to resolve its own fully qualified domain name and the names of machines to which it connects to an IP address.

7. Test and troubleshoot your certificates after installation

After you install the certificates, you can test the security of your Splunk deployment by reviewing the Splunk daemon logs on each of the instances to confirm that there are no errors in the configuration.

1. Start the Splunk platform instances.
2. Review the Splunk daemon logs, either by performing a search within Splunk Web or by reviewing the log file on the instance directly.
3. Search for errors that contain references to TLS and SSL. If you don't find any, your Splunk platform instances are properly configured.
4. If you do find errors, review the logs to determine what caused the errors, then review your certificates to ensure they meet the requirements for your specific application, taking into account the types of certificates, the domains that the certificates protect, interaction with other certificates, and other factors.

Post-setup: Configure certificate requirements and host name validation

Version 8.2.2202 and higher of Splunk Cloud Platform and 9.0.0 and higher of Splunk Enterprise let you configure your Splunk platform instances so that they verify the host names in certificates that they receive when they connect to other Splunk platform instances. Certificate requirements, in addition to certificate host name validation, work to further improve security within your Splunk platform deployments.

See Enable TLS certificate host name validation for additional information and procedures.

Next Steps

If you have a Splunk Cloud Platform deployment with external infrastructure that forwards data to it, see the following topic to configure the universal forwarder credentials package on that forwarding infrastructure:

• Install and configure the Splunk Cloud Platform universal forwarder credentials package

If you have a Splunk Enterprise deployment, read on to understand the next steps for securing the infrastructure with certificates.

Now that you understand the overall procedure for using TLS certificates with your Splunk platform deployment, you need some certificates to work with if you don't already have them. Choose from one of the following links for specific instructions on getting or creating the certificates.

• If you want to obtain a certificate from a third-party CA to protect communication between Splunk platform instances, see How to obtain certificates from a third party for inter-Splunk communication.
• If you want to obtain a certificate from a third-party CA to protect communications that involve Splunk Web, see How to obtain certificates from a third party for Splunk Web.
• If you want to create and sign your own certificates to protect any part of Splunk Enterprise, see How to create and sign your own certificates for use with the Splunk platform.
• If you have already obtained certificates and just need to know how to configure them to use with the Splunk platform, see How to prepare TLS certificates for use with the Splunk platform.