Send Splunk UBA anomalies and threats to Splunk ES as notable events
This image summarizes the steps you need to take to push anomalies and threats from Splunk User Behavior Analytics (UBA) to Splunk Enterprise Security (ES) as notable events.
- Verify that the Splunk Add-on for UEBA is installed and enabled.
- Enable SA-UEBA so that Splunk ES can retrieve data from Splunk UBA.
- Configure the Splunk platform to receive data from the Splunk UBA output connector.
- Add an output connector in Splunk UBA.
- (Optional) Learn How threats and notables are synchronized.
Verify that the Splunk Add-on for UEBA is installed and enabled
Perform the following steps to verify that the Splunk Add-on for UEBA is installed.
- Log in to the Splunk search head with Splunk Enterprise Security installed.
- In Splunk Web, select Apps > Manage Apps.
- Search for ueba and verify that Splunk_TA_ueba is installed and enabled.
- If the add-on is not enabled, click Enable to enable it.
Enable SA-UEBA so that Splunk ES can retrieve data from Splunk UBA
Enable SA-UEBA so that dashboards and knowledge objects are visible to users in Splunk Enterprise Security. This task is required to retrieve data from Splunk UBA to display on the Session Center dashboard in Splunk Enterprise Security, to model data sent from Splunk UBA with the UEBA data model, and to use correlation searches to analyze threats and anomalies sent from Splunk UBA.
- Log in to the Splunk search head with Splunk Enterprise Security installed.
- In Splunk Web, select Apps > Manage Apps.
- Locate the SA-UEBA add-on.
- Click Enable to enable the add-on.
Configure the Splunk platform to receive data from the Splunk UBA output connector
The connection between Splunk UBA and the Splunk platform uses TCP-SSL by default. Set up the Splunk platform to accept the encrypted connection so that the Splunk platform can receive data from the Splunk UBA output connector.
Splunk Cloud Platform customers must work with Splunk Cloud Platform Support to set up this connection.
The following procedure uses the Splunk default certificates and the global
[SSL] stanza in the inputs.conf file. For better security, consider using your own certificates, or commercially signed certificates from a trusted certificate authority.
- See About securing Splunk Enterprise with SSL in the Splunk Enterprise Securing the Splunk Platform manual.
- See TCP: in the Splunk Enterprise Admin Manual for more information about configuring tcp-ssl using inputs.conf.
Perform the following steps on Splunk Enterprise:
- Create a
$SPLUNK_HOME/etc/apps/Splunk_TA_ueba. For example:
cd /opt/splunk/etc/apps/Splunk_TA_ueba mkdir local cd local
- Create a file called inputs.conf and add the following stanza:
[tcp-ssl:10008] listenOnIPv6 = no index = ueba sourcetype = ueba serverCert = $SPLUNK_HOME/etc/auth/server.pem sslPassword = password
- In distributed deployments, deploy the changes to the inputs.conf file across all peers in your indexer cluster. See Manage common configurations across all peers in the Splunk Enterprise Managing Indexers and Clusters of Indexers manual.
- Restart Splunk Enterprise.
- In Splunk Web, select System > Server controls.
- Click Restart Splunk.
- Verify that SSL is enabled for port 10008 in
$SPLUNK_HOME/var/log/splunk/splunkd.log. For example:
11-07-2019 15:07:42.661 -0800 INFO TcpInputProc - Creating raw Acceptor for IPv4 port 10008 with SSL
- Copy the root CA certificate from
/opt/splunk/etc/auth/cacert.pemon the Splunk Enterprise instance to
/home/caspidaon the Splunk UBA management server.
Perform the following tasks on the Splunk UBA management server:
- Log in to the Splunk UBA management server as the caspida user.
- Ensure that
$JAVA_HOMEis set correctly on your system. Run the
CaspidaCommonEnv.shscript to set this environment variable:
- Import the rootCA certificate to the Java certificate store.
On RHEL systems, use the following command:
sudo keytool -import -alias "splunk es" -keystore $JAVA_HOME/lib/security/cacerts -file ~/cacert.pem
On other Linux systems, use the following command:
sudo keytool -import -alias "splunk es" -keystore $JAVA_HOME/jre/lib/security/cacerts -file ~/cacert.pem
- When prompted, type the keystore password and trust the certificate. The default keystore password is
- From the command line of the Splunk UBA management server, edit the
/etc/caspida/local/conf/uba-site.propertiesfile to change the following parameter from "false" to "true" as shown:
- Restart Splunk UBA. Run the following commands on the Splunk UBA management server:
/opt/caspida/bin/Caspida stop-all /opt/caspida/bin/Caspida start-all
Add an output connector in Splunk UBA
Set up Splunk UBA to send data to the Splunk search head or a Splunk forwarder. Splunk UBA uses the output connector to send anomalies and threats to Splunk Enterprise Security.
- Make sure the Splunk Add-on for Splunk UBA is properly deployed and enabled. See Deploy the Splunk Add-on for Splunk UBA.
- Make sure the SA-UEBA add-on is enabled. See Enable SA-UEBA so that Splunk ES can retrieve data from Splunk UBA.
- Log in to Splunk UBA as an admin user.
- Select Manage > Output Connectors.
- Click New Output Connector.
- Select an output connector of SplunkES and click Next.
- Type a Name for the output connector.
- The Forwarder Host is the IP address of the Splunk search head or forwarder with the TCP data input enabled. For a search head cluster, choose a search head to integrate with Splunk UBA because Splunk UBA does not support integrating with multiple search heads.
- The Forwarder Port is TCP port 10008 that is open for receiving data on the Splunk search head or forwarder. Make sure this port is open behind your firewall.
- Specify the Splunk ES API URL in the following format:
https://<forwarder-host>:<mgmt-port>Splunk UBA uses this connection to synchronize the status of its threats with notable events in Splunk ES. The management port is usually 8089.
- Specify a username and password. See Splunk Enterprise and Splunk ES requirements for the requirements of this user account.
- Select how you want this output connector to process anomalies and threats.
Option Description Process Threats Select Process Threats to enable Splunk UBA to export threats based on any configured output connectors, such as sending email or sending threats to Splunk ES. If this is not selected, no threats are exported from Splunk UBA, even if Auto Process is selected. Process Anomalies Select Process Anomalies to enable Splunk UBA to export anomalies based on any configured output connectors, such as sending email or sending anomalies to Splunk ES. If this is not selected, no anomalies are exported from Splunk UBA, even if Auto Process is selected. Auto Process Select Auto Process to automatically forward anomalies and threats as they are created to Splunk ES. If this is not selected, you must forward each anomaly or threat individually during threat review with the Export to Splunk ES action from the Anomaly Details or Threat Details page. See Review Current Threats in Use Splunk User Behavior Analytics.
Selecting Auto Process in conjunction with Process Anomalies can cause a significant increase in traffic. Review the output connector logs to verify the health and performance of your system. See the Output Connector Service health monitor status codes. If you are monitoring Splunk UBA using the Splunk UBA Monitoring app, see About the Splunk UBA Monitoring app in theSplunk UBA Monitoring App manual.
- Click OK to save the new output connector.
If the output connector does not work, verify that you configured the host server correctly in the uba-site.properties file. The host identified in the file populates the host field for events sent to Splunk Enterprise Security or the Splunk platform. See Connect Splunk UBA to Splunk Enterprise to view an anomaly's raw events in Install and Upgrade Splunk User Behavior Analytics.
How threats and notable events are synchronized
Once the output connector is configured, Splunk UBA attempts to send threats to Splunk ES every 5 minutes with no limits on the number of retries. Any issues with the connection mean that new threats are not sent to Splunk ES until the connection issues are resolved. Any connection issues between the output connector and Splunk ES also affect other output connectors that are configured, such as email and ServiceNow. If the connection issues persist for more than one hour, alerts are generated in the health monitor in Splunk UBA. See Monitor the health of your Splunk UBA deployment in Administer Splunk User Behavior Analytics.
Perform the following steps to change the retry interval:
- On the Splunk UBA management node, log in as the caspida user.
- Edit the
etc/caspida/local/conf/uba-site.propertiesfile and modify the
uba.splunkes.retry.delay.minutesproperty. For example, to set the retry interval to 3 mintues:
uba.splunkes.retry.delay.minutes = 3
- In distributed deployments, synchronize the cluster:
/opt/caspida/bin/Caspida sync-cluster /etc/caspida/local/conf
Work with Splunk UBA threats as notable events in Splunk ES
When Splunk UBA and Splunk ES are integrated using an output connector, Splunk UBA creates a new custom status on Splunk ES called Closed in Uba as shown in the following image.
The status of threats in Splunk UBA and their corresponding notable events in Splunk ES are synchronized.
What happens when a threat is closed in Splunk UBA
Threats in Splunk UBA can be closed by the user or closed by the system:
- Threats in Splunk UBA are considered closed by the user if a user clicks Not a Threat in Splunk UBA.
- In all other cases, the threat in Splunk UBA is considered to be closed by the system.
When a threat is closed in Splunk UBA, Splunk UBA checks the status of the corresponding notable event in Splunk ES. If the notable event is not already closed in Splunk ES, Splunk UBA closes the notable event by setting the end status to
Closed in Uba.
If the notable event is reopened in Splunk ES, a threat closed by the user in Splunk UBA is reopened. A threat closed by the system remains closed in Splunk UBA. The threat can still be viewed, but no action can be taken on the threat. This workflow is illustrated in the following diagram:
What happens when a threat is reopened in Splunk UBA
A threat in Splunk UBA can be reopened in the following cases:
- Threat computation causes a threat to be reopened.
- An anomaly action rule affects anomalies that cause a threat to be reopened.
- A threat rule is modified, causing a threat to be reopened.
When a threat is reopened, Splunk UBA checks to see if the notable event in Splunk ES has an end status of
ClosedInUba and if yes, the notable event is also reopened.
No action is taken if the notable event is already open in Splunk ES, or if it has an end status other than
Splunk UBA queries for the status of notable events in Splunk ES
Splunk UBA queries Splunk ES in 5-minute intervals to synchronize the status of threats in Splunk UBA and notable events in Splunk ES.
When the query detects that a notable event is closed, Splunk UBA checks to see if the corresponding threat is also closed. If not, the threat is closed with a status of
ClosedbyUser as shown in the following diagram:
When the query detects that a notable event is not closed, Splunk UBA checks to see if the corresponding threat has a status of
ClosedbyUser. If so, the threat is reopened with a status of
Active as shown in the following diagram:
What happens if the output connector is unable to send threats to Splunk ES?
If the output connector is unable to send threats to Splunk ES, due to a network issue or Splunk ES being temporarily unavailable, the output connector makes another attempt every 5 minutes. After 1 hour, if the connection is not resolved, the output connector raises an error in the health monitor. See OCS-11 in Administer Splunk User Behavior Analytics.
Integrate Splunk ES and Splunk UBA with the Splunk Add-on for Splunk UBA
Pull notable events from Splunk ES to Splunk UBA
This documentation applies to the following versions of Splunk® User Behavior Analytics: 5.1.0, 18.104.22.168