Configure Splunk Enterprise for Common Criteria
After you install the Splunk Enterprise instance and SELinux policy, as described in Install the Common Criteria-compliant Splunk Enterprise and SELinux policy packages, you can then configure the instance.
Set the splunk user
You must perform the tasks in this topic as the "splunk" user. The "splunk" user is is the user under which your Splunk Enterprise application runs. If you create or modify any files in the installation as the root user or any other user, the splunk user cannot access Splunk Enterprise, which can cause unexpected behavior.
Run the following commands to become the "splunk" user:
su - splunk export SPLUNK_HOME=/opt/splunk export SPLUNK_ETC=/etc/opt/splunk
Generate or obtain Common Criteria-compliant security certificates
Splunk Enterprise in Common Criteria mode does not generate any cryptographic keys or certificates. Use OpenSSL or any other key or certificate generation tool to generate self-signed certificates. These certificates must be Federal Information Process Standards (FIPS)-compliant. You can also get certificates issued by certificate authorities (CAs) such as Verisign/GlobalSign. The certificates must be in privacy-enhanced mail (PEM) format.
If you use the Splunk-generated default certificates, Splunk Enterprise will not be able to communicate over the network. The CLI, as well Splunk Web, will not function. Splunk Enterprise logs any errors in splunkd.log
.
List of certificates and keys
Provide certificates and keys for Splunk Enterprise to work in Common Criteria mode. Some of these certificates are optional, depending on whether you need the functionality. The details of these attributes can be found in /etc/opt/splunk/system/README/*.conf.spec
.
Configuration file | Configuration stanza name | Setting name | Notes |
---|---|---|---|
server.conf | [sslConfig] | serverCert | |
server.conf | [sslConfig] | sslRootCAPath | |
server.conf | [kvstore] | serverCert | |
web.conf | [settings] | serverCert | |
audit.conf | [auditTrail] | privateKey | Provided in the installation step before installing the SELinux policy package |
audit.conf | [auditTrail] | privateKey | Provided in the installation step before installing the SELinux policy package |
audit.conf | [auditTrail] | publicKey | Provided in the installation step before installing the SELinux policy package |
distsearch.conf | [tokenExchKeys] | publicKey | |
distsearch.conf | [tokenExchKeys] | privateKey | |
inputs.conf | [SSL] | serverCert | Needed only if using splunktcp-ssl for getting input from forwarders |
outputs.conf | [SSL] | serverCert | Needed if this is a forwarder configuration |
outputs.conf | [tcpout] | clientCert | CRLs: must store CRL files under /etc/opt/splunk/auth/crl directory. Look at README in that directory |
Update Splunk Enterprise configuration files with Common Criteria-compliant settings
Update or create the /etc/opt/splunk/system/local.conf files
with these settings. The paths shown in these samples are for illustration and can be different if desired.
server.conf
[general] requireBootPassphrase = true allowRemoteLogin = never [sslConfig] cipherSuite = ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384 # Note: ECDHE-ECDSA-AES256-SHA384 equates to ECDHE-ECDSA-AES256-CBC-SHA384 as defined in the Security Target. sendStrictTransportSecurityHeader = true serverCert = <absolute_path_to_server_certificate> sslAltNameToCheck = <comma separated list of SSL alternate names> sslCommonNameList = <comma separated list of Common Namess> # On RHEL 6.5, this will be typically '/etc/pki/tls/certs/ca-bundle.crt'. # For any additional CAs that need to be trusted, append them to this file. sslRootCAPath = <path to OS root cert store> sslVerifyServerCert = true sslVersions = tls1.2 sslVersionsForClient = tls1.2 [kvstore] serverCert = <absolute path to App Key Value Store certificate> [applicationsManagement] allowInternetAccess = false
web.conf
[settings] cipherSuite= ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384 enableSplunkWebSSL = 1 privKeyPath = <absolute path to encrypted private key> serverCert = <absolute path to public certificate> sslVersions = tls1.2
authentication.conf
[secrets] disabled = false
alert_actions.conf
cipherSuite = ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384 pdf.html_image_rendering = false sslAltNameToCheck = <comma separated list_of SSL alternate names> sslCommonNameToCheck = <comma separated list of common names> sslVerifyServerCert = true sslVersions = tls1.2 use_tls = 1
inputs.conf
#Use only if configuring Splunk Enterprise as an Indexer, which can receive data from forwarders. [SSL] cipherSuite = ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384 requireClientCert = true serverCert = <absolute path to server certificate> sslAltNameToCheck = <comma separated list of SSL alternate names> sslCommonNameToCheck = <comma separated list of common names> sslVersions = tls1.2
outputs.conf
[tcpout] defaultGroup = group1 cipherSuite = ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384 clientCert = <absolute path to client certificate> sslAltNameToCheck = <comma separated list of SSL alternate names> sslCommonNameToCheck = <comma separated list of common names> sslVerifyServerCert = true sslVersions = tls1.2 useClientSSLCompression = true
Enable Common Criteria mode for Splunk Enterprise
To activate Common Criteria mode for Splunk Enterprise, modify the /etc/opt/splunk/splunk-launch.conf
configuration file.
SPLUNK_COMMON_CRITERIA=1 SPLUNK_FIPS=1 # Do not generate python byte code PYTHONDONTWRITEBYTECODE=1
Create a Splunk Enterprise administrative account
When Splunk Enterprise first starts, it prompts you to create an administrative account. You must create this account, or your cannot log in.
This appears to be your first time running this version of Splunk. Splunk software must create an administrator account during startup. Otherwise, you cannot log in. Create credentials for the administrator account. Characters do not appear on the screen when you type in credentials. Please enter an administrator username:
- Type in the username that you want the software to create, for example
admin
. - The software then prompts:
Password must contain at least: * 8 total printable ASCII character(s). Please enter a new password:
Type in a password that meets the shown password requirements. - Re-enter the password you chose in the previous step. The software creates the account and continues start-up.
Simplify environment setup with .bashrc inclusion
Include these lines in the /home/splunk/.bashrc
shell configuration file so that your Common Criteria-compliant environment is setup properly when using the Splunk CLI.
export SPLUNK_ETC=/etc/opt/splunk export DBUS_SESSION_BUS_ADDRESS=$(awk '{ print $1}' /tmp/dbus-address) export DBUS_SESSION_BUS_PID=$(awk '{ print $2}' /tmp/dbus-address) export PATH=/usr/bin:$PATH . /opt/splunk/bin/setSplunkEnv
Generate a Common Criteria-Compliant "splunk.secret" file
The Splunk Enterprise deterministic random bit generator (DRBG) must be seeded with sufficient entropy. Use the rdrand-gen
(RdRand) tool to generate a seed file that Splunk Enterprise can use to generate a Common Criteria-compliant splunk.secret file.
Run the following commands:
rdrand-gen -v -n 1M -m reseed_delay -o $SPLUNK_HOME/rdrand.bin runcon -u system_u -t splunk_t -r system_r splunk gen-cc-splunk-secret --rand-path $SPLUNK_HOME/rdrand.bin --bytes-to-read 384
Initialize Secret Storage
-
Before starting Splunk Enterprise, start the dbus process. The dbus daemon communicates with GNOME keyring.
/home/splunk/run_dbus.sh
- Check that the the dbus process is running with the splunk_dbusd_t SELinux context
ps auxZ | grep dbus You will see an output similar to: unconfined_u:system_r:splunk_dbusd_t:s0 splunk 28563 0.0 0.0 31680 872 ? Ssl 14:36 0:00 dbus-daemon --session --print-pid --print-address --fork
- Source the
.bashrc
so that the environment variables for dbus are are set up.source /home/splunk/.bashrc
- Initialize the secret storage password:
runcon -u system_u -t splunk_t -r system_r splunk secret-storage --unlock
- To see list of keys available for secret storage:
runcon -u system_u -t splunk_t -r system_r splunk secret-storage --spec
Add secrets to Secret Storage
Run the following command to add secrets to the GNOME keyring:
runcon -u system_u -t splunk_t -r system_r splunk secret-storage --write --no-prompt <conf-file> <stanza-name> <setting-name> <passphrase>
where:
<conf-file> is the configuration file (for example, server.conf)
<stanza-name> is the name of a stanza in the file (for example, sslConfig
)
<setting-name> is the name of a setting (for example, sslKeysfilePassword
)
<passphrase> is the passphrase to use
List of Secrets
<conf-file>, <stanza-name>, <attribute-name> alert_actions.conf, [email], auth_password audit.conf, [auditTrail], privateKeyPassphrase distsearch.conf, [tokenExchKeys], privateKeyPassphrase inputs.conf, [SSL], sslPassword outputs.conf, [tcpout], sslPassword server.conf, [sslConfig], sslPassword server.conf, [kvstore], sslPassword web.conf, [settings], sslPassword
An example: [splunk@qa-cc-rhel65-03 ~]$ runcon -u system_u -t splunk_t -r system_r splunk secret-storage --write --no-prompt server sslConfig sslKeysfilePassword password
Start Splunk Enterprise and validate your configuration
- Start Splunk Enterprise.
/home/splunk/run_splunk.sh
- Confirm that Splunk Enterprise is running with the splunk_t SELinux context.
ps auxZ | grep splunk
- Review the
/opt/splunk/var/log/splunk/splunkd.log
file. Look for a message similar to the following that indicates that Splunk Enterprise is running in common criteria mode:ServerConfig - Splunk is starting in Common Criteria Mode.
Both splunkd and splunkweb should work normally in Common Criteria mode. - Confirm that you have a valid Splunk Enterprise license installed. See Types of Splunk Enterprise licenses.
Using Splunk Enterprise in Common Criteria Mode
- Run Splunk CLI commands as the 'splunk' user, When you run the commands, prepend the SELinux
run-on
command to set the proper context within SELinux.runcon -u system_u -t splunk_t -r system_r splunk <cli_cmd>
- To stop Splunk Enterprise, use the provided stop_splunk.sh script:
/home/splunk/stop_splunk.sh
- If you need to stop dbus, use the provided stop_dbus.sh script and run the following commands:
/home/splunk/stop_dbus.sh pkill gnome-keyring rm /tmp/dbus-address
Update certificate revocation list information
Splunk Enterprise expects to find the certificate revocation lists (CRLs) for revocation-checking in the $SPLUNK_ETC/auth/crl
directory. It expects these lists to be in privacy enhanced mail (PEM) format. Splunk provides a script as an example of how you can automate the update of CRLs. Any other mechanism that downloads the CRL files under the designated location will work as well. The example script expects the user to provide a list of URLs, one per line, which are CRLs for the certificates Splunk Enterprise will use.
An example file follows:
$ cat crl.txt http://pki.google.com/GIAG2.crl http://g.symcb.com/crls/gtglobal.crl
The following bash script reads through the crl.txt file, download the CRL files into the $SPLUNK_ETC/auth/crl
directory, and converts it into PEM format if necessary.
#!/bin/bash # NOTE: Only applicable for Splunk version 6.4.x and higher, while running in Common Criteria mode. # This script is provided as an example for downloading the CRL files in a location # Splunk expects it to be. Any other mechanism which updates CRL files should work. # The user can run the script one time OR setup a cron job to run it periodically (say every 30 min). # The script cleans out ALL existing CRL files (*.crl, *.pem) and then downloads the new versions. # Example invocation: /home/splunk/update_crl.sh /home/splunk/crl.txt /etc/opt/splunk/auth/crl if [ "$#" -ne 2 ]; then echo "Usage: $0 <crllist_file_absolute_path> <crl_download_location_absolute_path>" exit 1 fi PWD=`pwd` filename=$1 crl_dir=$2 if [ ! -f "$filename" ] || [ ! -d "$crl_dir" ] || [[ "$filename" != /* ]] || [[ $crl_dir != /* ]]; then echo "Both the crllist_file and crl_download_location must exist and be specified as absolute paths." exit 2 fi # go to $crl_dir cd $crl_dir # remove older CRL files if present rm -rf *.crl *.pem while read -r line || [[ -n "$line" ]]; do url=$line wget $url if [ "$?" -ne 0 ]; then echo "Failed to download CRL file: $url" fi done < "$filename" # For each file except README in this dir, check if the file is in DER format. # If yes, then convert to PEM and remove the corresponding CRL file. for f in ./* do if [ $f != "./README" ];then # use openssl from the OS itself openssl crl -in $f -text -noout &> /dev/null if [ "$?" -ne 0 ]; then #DER format, must convert to PEM openssl crl -inform der -in $f -out $f.pem if [ "$?" -ne 0 ]; then echo "Failed to convert DER format CRL file ($f) into PEM format. Splunk will not use this CRL file" fi rm $f fi fi done #revert to old pwd cd $PWD
Let's say, you have saved this script under /home/splunk/update_crl.sh
. Set the appropriate SELinux context and file-permissions on this file.
chown splunk:splunk update_crl.sh crl.txt chcon -u system_u -r object_r -t initrc_exec_t update_crl.sh chmod 755 update_crl.sh
You can set up a cron job to execute this script periodically and update CRL files which Splunk Enterprise uses. Run the following command as the 'splunk' user,
$ crontab -e
This opens a cron configuration file for 'splunk' user. Add this line to this file to update CRL info every 15 minutes.
*/15 * * * * /home/splunk/update_crl.sh /home/splunk/crl.txt /etc/opt/splunk/auth/crl &> /dev/null
This updates the CRL files. To load the updated CRL info into Splunk Enterprise, you can simply run a search in Splunk Web: | rest /services/server/security/crl/_reload
To automate, save this search as the 'admin' user. You can update the /etc/opt/splunk/users/admin/search/local/savedsearches.conf
file to add the search, similar to the following:
[Reload CRL information] display.general.type = statistics display.page.search.tab = statistics display.visualizations.show = 0 request.ui_dispatch_app = search request.ui_dispatch_view = search enableSched = 1 #update every minute (change if necessary) cron_schedule=* * * * * search = | rest /services/server/security/crl/_reload
You can check that CRL info is updating in Splunk Enterprise on a regular basis by enabling 'DEBUG' logging for the 'X509' component. Messages similar to the following appear when CRL info is reloaded in splunkd.log:
09-08-2016 15:50:14.452 -0700 DEBUG X509 - Successfully added (/etc/opt/splunk/auth/crl/GIAG2.crl.pem) to the revocation store. 09-08-2016 15:50:14.452 -0700 DEBUG X509 - Successfully added (/etc/opt/splunk/auth/crl/gtglobal.crl.pem) to the revocation store. 09-08-2016 15:50:14.452 -0700 DEBUG X509 - Added 2 CRL files from the CRL directory(/etc/opt/splunk/auth/crl) to the revocation store.
Install the Common Criteria-compliant Splunk Enterprise and SELinux policy packages | Add custom policies |
This documentation applies to the following versions of Splunk® Enterprise: 7.3.3, 7.3.4, 8.1.1, 9.0.4, 9.0.5, 9.0.6, 9.0.7, 9.0.8, 9.0.9, 9.0.10, 9.1.0, 9.1.1, 9.1.2, 9.1.3, 9.1.4, 9.1.5, 9.1.6, 9.1.7, 9.2.0, 9.2.1, 9.2.2, 9.2.3, 9.2.4
Feedback submitted, thanks!