Splunk Stream

Installation and Configuration Manual

Download manual as PDF

Download topic as PDF

Use Splunk Stream to ingest Netflow and IPFIX data

Use Splunk Stream to ingest Netflow and IPFIX data.

Configure indexers

Enable the Http_input receiver on your Splunk platform deployment's indexers:

  1. Navigate to your splunk_httpinput directory (for example, $SPLUNK_HOME/etc/apps/splunk_httpinput/local/ for a single instance deployment, and $SPLUNK_HOME/etc/master-app/splunk_httpinput/local/ for a distributed deployment), and create an inputs.conf file, if one does not already exist.
  2. Open inputs.conf and add stanzas to enable receiving. For example:
    [http] 
    disabled = 0 
    port = 8088 
    dedicatedIoThreads = 8 
    
    [http://streamfwd] 
    disabled = 0
    index=main
    token = 152B6F2B-8A21-4850-A444-CB0646FD88BE 
    indexes=_internal,main
    
  3. Save your changes, and exit.
  4. Restart your Splunk platform deployment.

(Optional) Modify Splunk_TA_stream and push to clustered indexers

For Splunk platform deployments that use indexer clustering, make the following changes to the Splunk_TA_stream app:

  1. Navigate to Splunk_TA_stream/default on your Splunk platform deployment.
  2. Remove the following files, if they are present: inputs.conf, inputs.conf.spec.
  3. Push the modified Splunk_TA_stream to all indexers in your Splunk platform deployment.

Configure the independent Stream forwarder

Configure the independent Stream forwarder on your Splunk platform deployment.

  1. On your deployment's independent Stream forwarder, navigate to streamfwd.conf.
  2. Open streamfwd.conf and enable forwarding. For example:
    [streamfwd] 
    httpEventCollectorToken = 152B6F2B-8A21-4850-A444-CB0646FD88BE 
    #(Match this with the token in the indexers) 
    ipAddr = 0.0.0.0 
    processingThreads = 4
    
    #Set the minimum system Ulimits
    	Ulimit -n 64000
    	Ulimit -u 16000
    
  3. Save your changes.
  4. Navigate to your independent Stream forwarder's etc/sysctl.conf directory.
  5. Adjust your kernel settings to increase buffer sizes for high-volume packet capture. For example:
    Sysctl -w net.core.rmem_default = 33554432
    Sysctl -w net.core.rmem_max = 33554432
    Sysctl -w net.core.netdev_max_backlog = 10000
    
  6. Reload the settings:
    /sbin/sysctl -p
    
  7. Restart the streamfwd service:
    Service streamfwd restart
    
  8. Navigate to your deployment's streamfwd.conf file, and add your deployment's Netflow configurations. For example:
    [streamfwd]
    
    httpEventCollectorToken = <GUID>
    
    indexer.0.uri= <HEC VIP>
    netflowReceiver.0.port = 9996
    netflowReceiver.0.decoder = netflow
    netflowReceiver.0.ip = 172.18.1.4
    netflowReceiver.0.decodingThreads = 16
    
  9. Save your changes.
  10. Restart your Splunk platform deployment.

Configure search heads

  1. Log in to the search head where the Splunk App for Stream is installed.
  2. Navigate to the Splunk App for Stream, then click Configuration > Distributed Forwarder Management.
  3. Click Create New Group.
  4. Enter a name. For example, INFRA_NETFLOW.
  5. Enter a description.
  6. Click Next.
  7. Enter INFRA_NETFLOW as the rule and click Next.
  8. Do not select any options. Click Finish.
  9. Navigate to the Splunk App for Stream, then click Configuration > Configure Streams.
  10. Click New Stream > Metadata.
  11. Enter Name as INFRA_NETFLOW.
  12. Select NetFlow as the protocol.
  13. Selecting NetFlow works for NetFlow, sFlow, jFlow, and IPFIX protocols.

  14. Enter a description then click Next.
  15. Select No in the Aggregation box then click Next.
  16. (Optional) Deselect any fields that do not apply to your use case then click Next.
  17. (Optional) Develop filters to reduce noise from high traffic devices then click Next.
  18. Select the index for this collection and click enable then click Next.
  19. Select only the Infra_netflow group and Create_Stream.
  20. Configure your NetFlow generator to send records to the new streamfwd.
  21. Validate your results by searching the configured index on your Splunk platform deployment.
Last modified on 01 April, 2020
PREVIOUS
Ingest pcap files
  NEXT
Use Stream configuration templates

This documentation applies to the following versions of Splunk Stream: 7.1.2, 7.1.3, 7.2.0


Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters