Automatically input data with Netflow proprietary configurations

Splunk Stream lets you automatically pull proprietary Netflow configuration data from configuration apps. To do this, you install a configuration app that Splunk Stream uses to automatically configure Netflow proprietary elements.

If you configure traffic with VLAN-ID, you must configure for both traffic directions. Otherwise you may experience a mismatch of VLANs in the TCP streams, which can cause queue overflow errors and traffic loss.

How Splunk Stream updates configurations

When you install a configuration app on your search head, Splunk Stream automatically collects the configuration data from your third party application using scripted inputs and then seamlessly provides decoding of Netflow/IPFIX proprietary data.

To accomplish this, Splunk Stream runs a scripted input that searches for data from your configuration apps. This input fetches the data from the configuration app, validates the configurations, and pushes the configurations into Splunk Stream. The input also checks for upgraded, modified, or deleted apps. Note that this process is only relevant for NetFlow data at this time.

This scripted input executes every 120 seconds and processes data from configuration apps that use the syntax splunk_app_stream_ipfix_appname. See Create configuration apps.

Once the configurations are successfully incorporated in Splunk Stream, the input behaves like any other built-in configuration.

Create configuration apps

A configuration app consists of the following files packaged as a tar file for upload with the syntax splunk_app_stream_ipfix_appname:

• Vocabulary.xml file (netflow.xml)
• Vocabulary mapping file (ipfixmap.conf)
• Stream file (netflow.json)

You create or obtain these files and place them in the default folder of the app for installation. The following resources can help you create or obtain a configuration app:

• Detailed instructions for creating an app can be found at https://dev.splunk.com/enterprise/docs/developapps/createapps/.
• Example configuration app files are provided at (internal link) in this topic.
• Consult your vendor-specific documentation for more about vendor-specific netflow proprietary elements.
• For help creating the configuration files you can reach out to Splunk support.
• If your Splunk Stream instance is in a cloud environment, your configuration app must be approved by Splunk. See https://docs.splunk.com/Documentation/SplunkCloud/8.2.2203/Admin/PrivateApps.

Configuration app file precedence and behavior

Modifying configuration apps can affect how Splunk Stream processes your configuration app, keep the following in mind when you modify a configuration app:

• When any configuration file is updated, the new changes are validated and pushed into the app. If the new changes cannot be validated, Stream forwarder keeps the previous unchanged version.
• If the vocabulary file is deleted, both dependent files (the Stream file and the ipfixmap.conf file) are deleted from Stream App configurations.
• If the Stream file or Conf file is deleted, the configuration in Splunk Stream is deleted.
• If the configuration app is deleted, all relevant configurations are deleted from Splunk Stream.
• Splunk layers files with the same name in both local and default folders. This means that if the same file is available in the default folder and the local folder, then both files get merged. If the same content is available in both the files then content which is present in the local file gets higher priority.

Creating configuration app files

These examples show one possible way to create the files required for configuration apps. For more about creating these files, see your third-party product documentation or reach out to support. If you are running Splunk Stream in a cloud environment, your app must be approved by dev ops: https://docs.splunk.com/Documentation/SplunkCloud/8.2.2203/Admin/PrivateApps.

Create a vocabulary file

The following is an example of how to create a vocabulary.xml using the netflow.xml file.

<Term id="netflow.httpRequestMethod"> <Type>string</Type> <Comment>HTTP request method associated with a flow</Comment> </Term>

Create a vocabulary mapping file

Splunk Stream supports mapping of IPFIX proprietary elements to Stream forwarder vocabulary terms. This lets you add and specify proprietary flow elements as fields in NetFlow protocol stream configurations that you create in the Configure Streams UI. Ifyou need assistance creating this file, contact your Splunk support representative.

By creating a mapping file, you map Netflow proprietary elements to Stream vocabulary terms: For example:

netflowElement.0.id = 459
netflowElement.0.termid = netflow.httpRequestMethod

.

Create a stream definition

The following is an example of how you can add a new vocabulary term to the Netflow Stream:

   {
     "aggType": "value", 
     "desc": "HTTP request method",
     "enabled": true, 
     "name": "netflow_http_req_method",
     "term": "netflow.httpRequestMethod"
   }