Data source planning for Splunk Enterprise Security

The volume, type, and number of data sources influences the overall Splunk platform architecture, the number and placement of forwarders, estimated load, and impact on network resources.

Splunk Enterprise Security requires that all data sources comply with the Splunk Common Information Model (CIM). Splunk Enterprise Security is designed to leverage the CIM standardized data models both when searching data to populate dashboard panels and views, and when providing data for detections.

Map add-ons to data sources

The add-ons included with Splunk Enterprise Security are designed to parse and categorize known data sources and other technologies for CIM compliance.

For each data source:

1. Identify the add-on: Identify the technology and determine the corresponding add-on. The primary sources for add-ons are the Technology-specific add-ons provided with Splunk Enterprise Security and the CIM-compatible content available on Splunkbase. If the add-on you want to use is not already compatible with the CIM, modify it to support CIM data schemas. For an example, see Use the CIM to normalize data at search time in the Common Information Model Add-on Manual.
2. Install the add-on: Install the add-on on the Splunk Enterprise Security search head. Install add-ons that perform index-time processing on each indexer. If the forwarder architecture includes sending data through a parsing or heavy forwarder, the add-on might be needed on the heavy forwarder. Splunk Cloud Platform customers must work with Splunk Support to install add-ons on search heads and indexers, but are responsible for on-premises forwarders.
3. Configure the server, device, or technology where necessary: Enable logging or data collection for the device or application and/or configure the output for collection by a Splunk instance. Consult the vendor documentation for implementation steps.
4. Customize the add-on where necessary: An add-on might require customization, such as setting the location or source of the data, choosing whether the data is located in a file or in a database, or other unique settings.
5. Set up a Splunk data input and confirm the source type settings: The README file of the add-on includes information about the source type setting associated with the data, and might include customization notes about configuring the input.

Considerations for data inputs

Splunk platform instances provide several types of input configurations to ingest data. Depending on the technology or source being collected, choose the input method that matches the infrastructure requirements based on the performance impact, ease of data access, stability, minimizing source latency, and maintainability.

• Monitoring files: Deploy a Splunk forwarder on each system hosting the files, and set the source type on the forwarder using an input configuration. If you have a large number of systems with identical files, use the Splunk Enterprise deployment server to set up standardized file inputs across large groups of forwarders.

• Monitoring network ports: Use standard tools such as a syslog server, or create listener ports on a forwarder. Sending multiple network sources to the same port or file complicates source typing. For more information, see the Splunk platform documentation.

• For Splunk Enterprise, see Get data from TCP and UDP ports in Splunk Enterprise Getting Data In.
• For Splunk Cloud Platform, see Get data from TCP and UDP ports in Splunk Cloud Platform Getting Data In.

• Monitoring Windows data: A forwarder can obtain information from Windows hosts using a variety of configuration options. For more information, see the Splunk platform documentation.

• For Splunk Enterprise, see How to get Windows data into Splunk Enterprise in Splunk Enterprise Getting Data In.
• For Splunk Cloud Platform, see Monitoring Windows data with Splunk Enterprise in Splunk Cloud Platform Getting Data In.

• Monitoring network wire data: Splunk Stream supports the capture of real-time wire data. See About Splunk Stream in the Splunk Stream Installation and Configuration Manual.

• Scripted inputs: Use scripted inputs to get data from an API or other remote data interfaces and message queues. Configure the forwarder to call shell scripts, python scripts, Windows batch files, PowerShell, or any other utility that can format and stream the data that you want to index. You can also write the data polled by any script to a file for direct monitoring by a forwarder. For more information, see the Splunk platform documentation.

• For Splunk Enterprise, see Get data from APIs and other remote data interfaces through scripted inputs in Splunk Enterprise Getting Data In.
• For Splunk Cloud Platform, see Get data from APIs and other remote data interfaces through scripted inputs in Splunk Cloud Platform Getting Data In.

Collect asset and identity information

Splunk Enterprise Security compares asset and identity data with events in Splunk platform to provide data enrichment and additional context for analysis. Collect and add your asset and identity information to Splunk Enterprise Security to take advantage of the data enrichment. See Add asset and identity data to Splunk Enterprise Security in Administer Splunk Enterprise Security.