Monitor the quality of data sent from the Splunk platform
When you create a data source with Splunk Direct, you can keep track of the quality of data added from the Splunk platform. One important metric is whether the value of a particular field meets Splunk UBA requirements. When an event contains a field value that is not accepted by Splunk UBA, the event might not be parsed. This can cause data models to not work as expected. You can monitor these mismatches by setting up the following configuration:
- Edit
/etc/caspida/local/conf/uba-site.properties
and add following line at the end of the file:
splunk.direct.enum.normalize=true
- (Optional) Add additional customization properties.
- Add properties for monitoring interval, number of mismatch values to keep, and indicator thresholds.
#the period in milliseconds of persisting the stats continuously collected per Splunk Direct datasource in the database. Also the period of time the Splunk Direct Data Source Enum Check indicator analyzes every time it checks going back from the check time. And finally the indicator's check period splunk.direct.enum.monitor.interval = 3600000 #number of most frequent mismatch values to keep per enum field splunk.direct.enum.monitor.top.count = 50 # ratio=mismatch_count/total_event_count # if ratio is between 0.1 and 0.2, indicator goes to warn ubaMonitor.etl.enumMonitor.warn.threshold = 0.1 # if ratio is above 0.2, indicator goes to bad ubaMonitor.etl.enumMonitor.bad.threshold = 0.2
- Modify the values as needed to suit the needs of your own environment.
- Run the following command to have your changes take effect.
sudo service caspida-sysmon restart
- Add properties for monitoring interval, number of mismatch values to keep, and indicator thresholds.
- Normalization rules are defined in
/etc/caspida/conf/normalize.rules
. Perform the following tasks to update the rule:- Make a copy of this file and place it in
/etc/caspida/local/conf/normalize.rules
. - Rules belonging to the same field are grouped together. In the following example, all matches with "allow" or "allowed" will be set to "allowed". If all the rules are parsed and no match is found, a mismatched value is recorded.
For example, given the following event:
{"action": "foo", "severity": 6}
And the following rules:
[action] REGEX: (?i)allow(ed)?= allowed REGEX: (?i)denied=blocked REGEX: (?i)drop(ped)?= dropped REGEX: (?i)teardown = Alerted [severity] RANGE: [6-7] = high
The event becomes the following after it is normalized:
Since the action "foo" is not defined in the example, it is recorded as a mismatched value.{"action": "foo", "severity": "high"}
- Make a copy of this file and place it in
- Synchronize the changes across all Splunk UBA nodes in your cluster.
/opt/caspida/bin/Caspida sync-cluster /etc/caspida/local/conf
- Restart Splunk UBA for the changes to take effect.
/opt/caspida/bin/Caspida stop /opt/caspida/bin/Caspida start
Health monitor status codes are available to provide visibility into the status of events containing mismatched values compared to total events. See Health Monitor status code reference.
Verify that you successfully added the data source | Review and edit existing data sources in Splunk UBA |
This documentation applies to the following versions of Splunk® User Behavior Analytics: 5.0.0, 5.0.1, 5.0.2, 5.0.3, 5.0.4, 5.0.4.1, 5.0.5, 5.0.5.1, 5.1.0, 5.1.0.1, 5.2.0, 5.2.1, 5.3.0, 5.4.0, 5.4.1
Feedback submitted, thanks!