Splunk Stream

Installation and Configuration Manual

Acrobat logo Download manual as PDF

This documentation does not apply to the most recent version of StreamApp. Click here for the latest version.
Acrobat logo Download topic as PDF

Use Splunk Stream to ingest Netflow and IPFIX data

You can use Splunk Stream to ingest Netflow and IPFIX data. Splunk Stream supports flow data sent over the UDP protocol.

Configure indexers

Enable the Http_input receiver on your Splunk platform deployment's indexers:

  1. Navigate to your splunk_httpinput directory (for example, $SPLUNK_HOME/etc/apps/splunk_httpinput/local/ for a single instance deployment, and $SPLUNK_HOME/etc/master-app/splunk_httpinput/local/ for a distributed deployment), and create an inputs.conf file, if one does not already exist.
  2. Open inputs.conf and add stanzas to enable receiving. For example:
    [http] 
    disabled = 0 
    port = 8088 
    dedicatedIoThreads = 8 
    
    [http://streamfwd] 
    disabled = 0
    index=main
    token = 152B6F2B-8A21-4850-A444-CB0646FD88BE 
    indexes=_internal,main
    
    The HEC token is automatically populated when the user creates a HEC token in Splunk Web. If you are working in a Managed Cloud deployment, contact your account team.
  3. Save your changes, and exit.
  4. Restart your Splunk platform deployment.

(Optional) Modify Splunk_TA_stream and push to clustered indexers

For Splunk platform deployments that use indexer clustering, make the following changes to the Splunk_TA_stream app:

  1. Navigate to Splunk_TA_stream/default on your Splunk platform deployment.
  2. Remove the following files, if they are present: inputs.conf, inputs.conf.spec.
  3. Push the modified Splunk_TA_stream to all indexers in your Splunk platform deployment.

Configure the independent Stream forwarder for NetFlow

Configure the independent Stream forwarder to work with NetFlow.

  1. On your deployment's independent Stream forwarder, navigate to streamfwd.conf.
  2. Open streamfwd.conf and enable forwarding. For example:
    [streamfwd] 
    httpEventCollectorToken = 152B6F2B-8A21-4850-A444-CB0646FD88BE 
    #(Match this with the token in the indexers) 
    ipAddr = 0.0.0.0 
    processingThreads = 4
  3. Edit your deployment's Netflow configurations. For example:
    [streamfwd]
    
    httpEventCollectorToken = <GUID>
    
    indexer.0.uri= <HEC VIP>
    netflowReceiver.0.port = 9996
    netflowReceiver.0.decoder = netflow
    netflowReceiver.0.ip = 172.18.1.4
    netflowReceiver.0.decodingThreads = 16
    
  4. Save your changes.
  5. Navigate to your server's /etc/sysctl.conf directory.
  6. Adjust your kernel settings to increase buffer sizes for high-volume packet capture. For example:
    sysctl -w net.core.rmem_default = 33554432
    sysctl -w net.core.rmem_max = 33554432
    sysctl -w net.core.netdev_max_backlog = 10000
    
  7. Reload the settings:
    /sbin/sysctl -p
    
  8. Set the minimum system ulimits from your command line interface:
    ulimit -n 64000
    ulimit -u 16000
    
  9. Save your changes.
  10. Restart the streamfwd service:
    service streamfwd restart
    

Configure search heads

  1. Log in to the search head where the Splunk App for Stream is installed.
  2. Navigate to the Splunk App for Stream, then click Configuration > Distributed Forwarder Management.
  3. Click Create New Group.
  4. Enter a name. For example, INFRA_NETFLOW.
  5. Enter a description.
  6. Click Next.
  7. Enter INFRA_NETFLOW as the rule and click Next.
  8. Do not select any options. Click Finish.
  9. Navigate to the Splunk App for Stream, then click Configuration > Configure Streams.
  10. Click New Stream > Metadata.
  11. Enter Name as INFRA_NETFLOW.
  12. Select NetFlow as the protocol.
  13. Selecting NetFlow works for NetFlow, sFlow, jFlow, and IPFIX protocols.

  14. Enter a description then click Next.
  15. Select No in the Aggregation box then click Next.
  16. (Optional) Deselect any fields that do not apply to your use case then click Next.
  17. (Optional) Develop filters to reduce noise from high traffic devices then click Next.
  18. Select the index for this collection and click enable then click Next.
  19. Select only the Infra_netflow group and Create_Stream.
  20. Configure your NetFlow generator to send records to the new streamfwd.
  21. Validate your results by searching the configured index on your Splunk platform deployment.
Last modified on 15 September, 2020
PREVIOUS
Ingest pcap files
  NEXT
Use Stream configuration templates

This documentation applies to the following versions of Splunk Stream: 7.1.2, 7.1.3, 7.2.0


Was this documentation topic helpful?

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters