Troubleshoot missing notable events in Splunk Enterprise Security
If you have a Correlation Search that isn't generating notable events when you think it should, you can check the following potential causes and solutions.
|The notable events are being suppressed.
|Check to see if the notable index contains notable events. Search in Splunk Web against the notable index to determine if the notable event exists but is being excluded from Incident Review:
Suppressions filter notable events from appearing in Incident Review. If you see your notable event in the index, then make sure that no suppressions are preventing the notable event from appearing in Incident Review.
|The entire correlation search doesn't match, but part of it does.
|Run the correlation search manually over the given timeframe and see if it matches the events. If it doesn't match, remove parts of the search until you isolate the part of the search that doesn't match.
|The notable alert action isn't triggered.
|Check the notable alert action logs. These logs indicate if the notable alert action is triggered to make a notable event. Search in Splunk Web to view these logs:
|Splunk Enterprise cannot parse the stash file.
|Verify that the search output doesn't include any unnecessary output. Make sure that the correlation search only outputs the fields you really need, and that the fields don't include extra content such as XML or excessive amounts of text. Extra content can make it difficult for Splunk to parse the stash file. If the stash file can't be parsed, then your notable events may not be generated correctly.
|The correlation search schedule is incorrect, not running, or suppressed.
|Check the search scheduler logs. Search in Splunk Web to view the scheduler logs:
Look for the following:
|If you are using a distributed architecture, you may have missed creating the notable index on your cluster.
|See Configure and deploy indexes in the Installation and Upgrade Manual.
Troubleshoot lookups in Splunk Enterprise Security
Turn on debug logging in Splunk Enterprise Security
This documentation applies to the following versions of Splunk® Enterprise Security: 7.0.1, 7.0.2, 7.1.0, 7.1.1, 7.1.2, 7.2.0, 7.3.0