Use Splunk Stream to ingest Netflow and IPFIX data

You can use Splunk Stream to ingest Netflow and IPFIX data. Splunk Stream supports flow data sent over the UDP protocol.

Configure indexers

Enable the Http_input receiver on your Splunk platform deployment's indexers:

1. Navigate to your splunk_httpinput directory (for example, $SPLUNK_HOME/etc/apps/splunk_httpinput/local/ for a single instance deployment, and $SPLUNK_HOME/etc/master-app/splunk_httpinput/local/ for a distributed deployment), and create an inputs.conf file, if one does not already exist.
2. Open inputs.conf and add stanzas to enable receiving. For example:

   [http] 
   disabled = 0 
   port = 8088 
   dedicatedIoThreads = 8 
   
   [http://streamfwd] 
   disabled = 0
   index=main
   token = 152B6F2B-8A21-4850-A444-CB0646FD88BE 
   indexes=_internal,main
   

   The HEC token is automatically populated when the user creates a HEC token in Splunk Web. If you are working in a Managed Cloud deployment, contact your account team.
3. Save your changes, and exit.
4. Restart your Splunk platform deployment.

(Optional) Modify Splunk_TA_stream and push to clustered indexers

For Splunk platform deployments that use indexer clustering, make the following changes to the Splunk_TA_stream app:

1. Navigate to Splunk_TA_stream/default on your Splunk platform deployment.
2. Remove the following files, if they are present: inputs.conf, inputs.conf.spec.
3. Push the modified Splunk_TA_stream to all indexers in your Splunk platform deployment.

Configure the independent Stream forwarder for NetFlow

Configure the independent Stream forwarder to work with NetFlow.

1. On your deployment's independent Stream forwarder, navigate to streamfwd.conf.
2. Open streamfwd.conf and enable forwarding. For example:

   [streamfwd] 
   httpEventCollectorToken = 152B6F2B-8A21-4850-A444-CB0646FD88BE 
   #(Match this with the token in the indexers) 
   ipAddr = 0.0.0.0 
   processingThreads = 4

3. Edit your deployment's Netflow configurations. For example:

   [streamfwd]
   
   httpEventCollectorToken = <GUID>
   
   indexer.0.uri= <HEC VIP>
   netflowReceiver.0.port = 9996
   netflowReceiver.0.decoder = netflow
   netflowReceiver.0.ip = 172.18.1.4
   netflowReceiver.0.decodingThreads = 16
   

4. Save your changes.
5. Navigate to your server's /etc/sysctl.conf directory.
6. Adjust your kernel settings to increase buffer sizes for high-volume packet capture. For example:

   sysctl -w net.core.rmem_default = 33554432
   sysctl -w net.core.rmem_max = 33554432
   sysctl -w net.core.netdev_max_backlog = 10000
   

7. Reload the settings:

   /sbin/sysctl -p
   

8. Set the minimum system ulimits from your command line interface:

   ulimit -n 64000
   ulimit -u 16000
   

9. Save your changes.
10. Restart the streamfwd service:

    service streamfwd restart
    

Configure search heads

1. Log in to the search head where the Splunk App for Stream is installed.
2. Navigate to the Splunk App for Stream, then click Configuration > Distributed Forwarder Management.
3. Click Create New Group.
4. Enter a name. For example, INFRA_NETFLOW.
5. Enter a description.
6. Click Next.
7. Enter INFRA_NETFLOW as the rule and click Next.
8. Do not select any options. Click Finish.
9. Navigate to the Splunk App for Stream, then click Configuration > Configure Streams.
10. Click New Stream > Metadata.
11. Enter Name as INFRA_NETFLOW.
12. Select NetFlow as the protocol.

Selecting NetFlow works for NetFlow, sFlow, jFlow, and IPFIX protocols.

13. Enter a description then click Next.
14. Select No in the Aggregation box then click Next.
15. (Optional) Deselect any fields that do not apply to your use case then click Next.
16. (Optional) Develop filters to reduce noise from high traffic devices then click Next.
17. Select the index for this collection and click enable then click Next.
18. Select only the Infra_netflow group and Create_Stream.
19. Configure your NetFlow generator to send records to the new streamfwd.
20. Validate your results by searching the configured index on your Splunk platform deployment.