Monitor the quality of data sent from the Splunk platform

When you create a data source with Splunk Direct, you can keep track of the quality of data added from the Splunk platform. One important metric is whether the value of a particular field meets Splunk UBA requirements. When an event contains a field value that is not accepted by Splunk UBA, the event might not be parsed. This can cause data models to not work as expected. You can monitor these mismatches by setting up the following configuration:

1. Edit /etc/caspida/local/conf/uba-site.properties and add following line at the end of the file:
   

   splunk.direct.enum.normalize=true

2. (Optional) Add additional customization properties.
   1. Add properties for monitoring interval, number of mismatch values to keep, and indicator thresholds.

      #the period in milliseconds of persisting the stats continuously collected per Splunk Direct datasource in the database. Also the period of time the Splunk Direct Data Source Enum Check indicator analyzes every time it checks going back from the check time. And finally the indicator's check period
      splunk.direct.enum.monitor.interval = 3600000
      #number of most frequent mismatch values to keep per enum field
      splunk.direct.enum.monitor.top.count = 50
      # ratio=mismatch_count/total_event_count
      # if ratio is between 0.1 and 0.2, indicator goes to warn
      ubaMonitor.etl.enumMonitor.warn.threshold = 0.1
      # if ratio is above 0.2, indicator goes to bad
      ubaMonitor.etl.enumMonitor.bad.threshold = 0.2
      

   2. Modify the values as needed to suit the needs of your own environment.
   3. Run the following command to have your changes take effect.

      sudo service caspida-sysmon restart

3. Normalization rules are defined in /etc/caspida/conf/normalize.rules. Perform the following tasks to update the rule:
   1. Make a copy of this file and place it in /etc/caspida/local/conf/normalize.rules.
   2. Rules belonging to the same field are grouped together. In the following example, all matches with "allow" or "allowed" will be set to "allowed". If all the rules are parsed and no match is found, a mismatched value is recorded. For example, given the following event:

      {"action": "foo", "severity": 6}

      And the following rules:

      [action]
      REGEX: (?i)allow(ed)?= allowed
      REGEX: (?i)denied=blocked
      REGEX: (?i)drop(ped)?= dropped
      REGEX: (?i)teardown = Alerted
      
      [severity]
      RANGE: [6-7] = high
      

      The event becomes the following after it is normalized:

      {"action": "foo", "severity": "high"}

      Since the action "foo" is not defined in the example, it is recorded as a mismatched value.
4. Synchronize the changes across all Splunk UBA nodes in your cluster.

   /opt/caspida/bin/Caspida sync-cluster  /etc/caspida/local/conf

5. Restart Splunk UBA for the changes to take effect.

   /opt/caspida/bin/Caspida stop
   /opt/caspida/bin/Caspida start
   

Health monitor status codes are available to provide visibility into the status of events containing mismatched values compared to total events. See Health Monitor status code reference.