Troubleshooting common errors

ESCU does not generate its own logs because the app leverages core features of the Splunk Platform.

Following are some common log files that you can use to identify issues that might appear to be ESCU-related but usually exist in the Splunk Platform:

• splunkd.log
• scheduler.log
• splunkd_access.log

For more information on troubleshooting, see the product documentation:

• If you use Splunk Enterprise Security, see the Splunk Enterprise Security Troubleshooting manual for additional troubleshooting information.
• For search related issues on Splunk Enterprise, see the Search Troubleshooting Guide.
• To confirm if there is data present for a search or if you must troubleshoot data model issues, see the Troubleshooting Datamodel manual.

Troubleshoot a specific threat detection use case

Follow these steps to troubleshoot a specific threat detection use case in Splunk Enterprise Security:

1. Get a high-level understanding of what the detection is trying to accomplish.
   Each ESCU detection targets a specific detection use case. In some troubleshooting scenarios, this metadata can be useful. For more information, see the following resources in Splunk Documentation and Github repository.
   • Splunk Enterprise Security Content Update documentation
   • STRT detections
   • Security content Github repository
   • Detections in the Github repository. For example, https://github.com/splunk/security_content/blob/develop/detections/endpoint/7zip_commandline_to_smb_share_path.yml
2. Confirm that the data source contains events by using search on data models.
   Each ESCU detection searches events in a data source.

   | from datamodel Endpoint.Processes | search *

3. Ensure that events in a data source are correctly indexed by the Splunk Platform.
   Verify this by identifying the SPL for the detection and minimizing it to its simplest form to confirm that events exist. Certain ESCU detections rely on data models. For example, you can confirm if events are created in the following detection by reducing the search to the minimum time and removing every condition.

   | tstats `security_content_summariesonly` count min(_time) as firstTime max(_time) as lastTime from datamodel=Endpoint.Processes where (Processes.process_name ="7z.exe" OR Processes.process_name = "7za.exe" OR Processes.original_file_name = "7z.exe" OR Processes.original_file_name = "7za.exe") AND (Processes.process="*\\C$\\*" OR Processes.process="*\\Admin$\\*" OR Processes.process="*\\IPC$\\*") by Processes.original_file_name Processes.parent_process_name Processes.parent_process Processes.process_name Processes.process Processes.parent_process_id Processes.process_id Processes.dest Processes.user | `drop_dm_object_name(Processes)` | `security_content_ctime(firstTime)` | `security_content_ctime(lastTime)` | `7zip_commandline_to_smb_share_path_filter`

   as follows:

   | tstats `security_content_summariesonly` count min(_time) as firstTime max(_time) as lastTime from datamodel=Endpoint.Processes

   Not every detection uses data models but you can identify the conditions and remove them from the SPL search query.

4. Review the configuration of the detection.
   Each ESCU detection is initially configured to run on a schedule to identify on-going threats and troubleshoot mis-configured or disabled alerts. You can find specific details about the detection such as name, the SPL query, the last time it was run, the next time it runs, the frequency of the schedule, and so on by going to Searches, Reports, and Alerts in the Splunk UI.