Splunk® Enterprise Security

Data Source Integration Manual

Acrobat logo Download manual as PDF


This documentation does not apply to the most recent version of Splunk® Enterprise Security. For documentation on the most recent version, go to the latest release.
Acrobat logo Download topic as PDF

Overview

The Splunk App for Enterprise Security gives the security practitioner visibility into security-relevant data that is captured and indexed within Splunk.

The reports and correlation searches of the Splunk App for Enterprise Security, present a unified view of security across heterogeneous vendor data formats. Splunk does this based on search-time mappings to a common set of field names and tags that can be defined at any time after the data is captured, indexed, and available for an immediate search.

This means that you do not need to write parsers before you can start collecting and searching the data. However, you need to define the field extractions and tags for each data format before the Enterprise Security reports and correlation searches will work on that data. These tags and field extractions for data formats are defined in add-ons. The Splunk App for Enterprise Security ships with an initial set of these add-ons. This guide explains how to create your own.

Add-ons contain the Splunk "knowledge" (field extractions, tags, and source types) that is necessary to extract and normalize information from the data sources at search time and make the resulting information available for reporting. By creating your own add-ons, you can add new or custom types of data and fully integrate them with the existing dashboards and reports within the Splunk App for Enterprise Security.

After you create an add-on, you can add it to your Enterprise Security deployment or post it to Splunkbase to share with others.

What is an add-on?

An add-on is a Splunk app that extracts knowledge from IT data so that it can be processed by Enterprise Security, as well as other apps that leverage the "Common Information Model" (CIM). The add-on may pull data into Splunk or map data that is coming in. Add-ons might conflict with or duplicate other Splunk apps that are pulling in the same sort of data if they conflict on the source type. The difference between an add-on and another Splunk app is compliance with the CIM.

Note: The add-on does not require a user interface because reporting is handled by existing dashboards, centers, and searches in Enterprise Security.

What add-ons do

Each add-on contains search time knowledge mapping specific to a the data format generated by a specific technology:

  • When data is added to Splunk, Splunk assigns a source type based on the data format and the originating technology. The Splunk App for Enterprise Security uses this source type information, along with the search time knowledge mapping in the add-on. For some classes of data sources, Splunk and the Splunk App for Enterprise Security can determine the source type; for other classes of data sources, you must explicitly specify the source type when you create the Splunk data input.
  • A minority of add-ons deal with with complex or ambiguous data. In this case configuration to enhance Splunk's universal indexing, extracting correct timestamps from incomplete or non-standard time formats, determining event boundaries for multi-line data, and optimizing the data representation in the index for search speed and performance is supplied by add-ons.
  • During searches Splunk uses field extractions and tagging configuration from add-ons to know how to normalize the data, including tagging data (for example, firewall data vs. malware data) and identifying fields used by the dashboards and searches (for example, identifying the source and destination of an action).

Add-ons rely heavily on the Splunk source type that has been assigned to the data, so during installation it is important to ensure that this is set up correctly.

Define a source type for the data

By default, Splunk sets a source type for a given data input. Each add-on has at least one source type defined for the data that is captured and indexed within Splunk. This action requires an override of the automatic source type that Splunk attempts to assign to the data source, because the primary source type must be set in the add-on in order to apply the correct field extractions used by Enterprise Security. An add-on can extrapolate key data within the raw text of logs to extract "fields" that are fully compliant with the "Common Information Model".

An add-on performs the following functions:

  • Capture and index the data If necessary, the add-on can import and source type the data into Splunk. This action is not required if the data is in Splunk and source-typed correctly.
  • Identify the relevant events that should be visible for security purposes, such as a successful login to a server.
  • Extract fields and aliases that match the CIM so that notable events are generated and dashboards function correctly.
  • Create tags to categorize the data. For example, tag all data indicating network communication with the tags "network" and "communicate."
  • Create additional required fields that are not in the original data source, such as fields that describe the vendor or product.
  • Normalize field values to a common standard, such as changing "Accepted public key" or "Success Audit" to "action=success."

Each add-on is designed for a specific data format, such as a particular vendor's firewall or router. After the add-on is created, data sources need to be assigned the corresponding source type for the add-on to begin processing the data.

Tasks you need to build an add-on

See the "Knowledge Manager Manual" in the core Splunk product documentation for information about the following tasks.

See "Out-of-the-box source types" in this document for a list of tags and source types that are available with the Splunk App for Enterprise Security.

Available add-ons

Each Enterprise Security add-on is specific to a single technology or portion of a technology that provides the Splunk knowledge necessary to incorporate that technology into the Splunk App for Enterprise Security. You can use prepackaged add-ons when they are available.

Add-ons for a number of common source types are bundled with the Splunk App for Enterprise Security. You might need to configure some of these add-ons for your environment. Each add-on contains a README file that describes the required configurations.

Last modified on 09 September, 2015
  NEXT
Create an add-on

This documentation applies to the following versions of Splunk® Enterprise Security: 3.1, 3.1.1, 3.2, 3.2.1, 3.2.2, 3.3.0, 3.3.1, 3.3.2, 3.3.3


Was this documentation topic helpful?


You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters