Select which data sources to use with behavioral analytics service

Building on our previous example where a heavy forwarder is sending a copy of all data to both Splunk Cloud Platform and behavioral analytics service, you can also configure the heavy forwarder to send only data from selected data sources to behavioral analytics service. Sending data from specific data sources gives you more control over what you are sending and can also help alleviate network bandwidth and memory usage.

This example configures the existing heavy forwarder that is already sending data to Splunk Cloud Platform to clone specific source types supported by behavioral analytics service and send events from that source type to Splunk Cloud Platform and behavioral analytics service. See Forward data to third-party systems in the Splunk Enterprise Forwarding Data manual for additional examples.

In this example, we are starting with a heavy forwarder that is configured to send data to splunkcloud by default as shown in the following tcpout stanza in the outputs.conf file:

[tcpout]
defaultGroup=splunkcloud
[tcpout:splunkcloud]
[tcpout:splunk_ba]

Perform the following steps to send data from specific source types to Splunk Cloud Platform and behavioral analytics service:

1. Create an app on the heavy forwarder to store the new configuration.

   mkdir -p etc/apps/splunk_ba_forwarding etc/apps/splunk_ba_forwarding/local

2. Create a props.conf and transforms.conf inside the local folder:

   touch local/props.conf
   touch local/transforms.conf
   

3. Create the following new stanza inside the transforms.conf file. The TCP_ROUTING parameter changes the tcpout group where events are sent. In the example, each event processed by the splunkAndBA stanza is sent to both tcpout groups splunkcloud and splunk_ba.

   [splunkAndBA]
   REGEX=.
   DEST_KEY=_TCP_ROUTING
   FORMAT=splunkcloud,splunk_ba
   

4. Update the props.conf file and specify the source types you want to be processed. For example:

   [WinEventLog]
   TRANSFORMS-routing=splunkAndBA
   [webgateway]
   TRANSFORMS-routing=splunkAndBA
   [cisco:asa]
   TRANSFORMS-routing=splunkAndBA
   [windows_snare_syslog]
   TRANSFORMS-routing=splunkAndBA
   [dhcp]
   TRANSFORMS-routing=splunkAndBA
   [WinEventLog]
   TRANSFORMS-routing=splunkAndBA
   [XmlWinEventLog]
   TRANSFORMS-routing=splunkAndBA
   [pan:traffic]
   TRANSFORMS-routing=splunkAndBA
   [bit9:carbonblack:json]
   TRANSFORMS-routing=splunkAndBA
   [o365:management:activity]
   TRANSFORMS-routing=splunkAndBA
   

5. Use the btool command-line tool to ensure the desired settings are present. First, check the props.conf file:

   $ splunk btool props list WinEventLog --debug | grep -v m/d | grep -v s/d
   /opt/splunk/etc/apps/splunk_ba_forwarding/local/props.conf [WinEventLog]
   /opt/splunk/etc/apps/splunk_ba_forwarding/local/props.conf TRANSFORMS-routing = splunkAndBA
   

   Run the following command to check the transforms.conf file:

   $ splunk btool transforms list splunkAndBA --debug | grep -v m/d
   /opt/splunk/etc/apps/splunk_ba_forwarding/local/transforms.conf [splunkAndBA]
   /opt/splunk/etc/apps/splunk_ba_forwarding/local/transforms.conf DEST_KEY = _TCP_ROUTING
   /opt/splunk/etc/apps/splunk_ba_forwarding/local/transforms.conf FORMAT = splunkcloud,splunk_ba
   /opt/splunk/etc/apps/splunk_ba_forwarding/local/transforms.conf REGEX = .
   

   See Use btool to troubleshoot configurations in the Splunk Enterprise Troubleshooting Manual for more information about the btool command-line tool.
6. Changes to source type ingestion require you to restart the heavy forwarder.
   1. Log into Splunk Web as an admin role.
   2. In Splunk Web, go to Settings > Server controls.
   3. Select Restart Splunk.
7. Search for enriched events to verify that the data you want is being ingested by behavioral analytics service and also being properly parsed. See Search for enriched events from Splunk Mission Control.