Use Splunk Stream to ingest Netflow and IPFIX data
You can use Splunk Stream to ingest Netflow and IPFIX data. Splunk Stream supports flow data sent over the UDP protocol.
Configure indexers
Enable the Http_input
receiver on your Splunk platform deployment's indexers:
- Navigate to your
splunk_httpinput
directory (for example,$SPLUNK_HOME/etc/apps/splunk_httpinput/local/
for a single instance deployment, and$SPLUNK_HOME/etc/master-app/splunk_httpinput/local/
for a distributed deployment), and create aninputs.conf
file, if one does not already exist. - Open
inputs.conf
and add stanzas to enable receiving. For example:The HEC token is automatically populated when the user creates a HEC token in Splunk Web. If you are working in a Managed Cloud deployment, contact your account team.[http] disabled = 0 port = 8088 dedicatedIoThreads = 8 [http://streamfwd] disabled = 0 index=main token = 152B6F2B-8A21-4850-A444-CB0646FD88BE indexes=_internal,main
- Save your changes, and exit.
- Restart your Splunk platform deployment.
(Optional) Modify Splunk_TA_stream and push to clustered indexers
For Splunk platform deployments that use indexer clustering, make the following changes to the Splunk_TA_stream
app:
- Navigate to
Splunk_TA_stream/default
on your Splunk platform deployment. - Remove the following files, if they are present:
inputs.conf
,inputs.conf.spec
. - Push the modified
Splunk_TA_stream
to all indexers in your Splunk platform deployment.
Configure the independent Stream forwarder for NetFlow
Configure the independent Stream forwarder to work with NetFlow.
- On your deployment's independent Stream forwarder, navigate to
streamfwd.conf
. - Open
streamfwd.conf
and enable forwarding. For example:[streamfwd] httpEventCollectorToken = 152B6F2B-8A21-4850-A444-CB0646FD88BE #(Match this with the token in the indexers) ipAddr = 0.0.0.0 processingThreads = 4
- Edit your deployment's Netflow configurations. For example:
[streamfwd] httpEventCollectorToken = <GUID> indexer.0.uri= <HEC VIP> netflowReceiver.0.port = 9996 netflowReceiver.0.decoder = netflow netflowReceiver.0.ip = 172.18.1.4 netflowReceiver.0.decodingThreads = 16
- Save your changes.
- Navigate to your server's
/etc/sysctl.conf
directory. - Adjust your kernel settings to increase buffer sizes for high-volume packet capture. For example:
sysctl -w net.core.rmem_default = 33554432 sysctl -w net.core.rmem_max = 33554432 sysctl -w net.core.netdev_max_backlog = 10000
- Reload the settings:
/sbin/sysctl -p
- Set the minimum system ulimits from your command line interface:
ulimit -n 64000 ulimit -u 16000
- Save your changes.
- Restart the
streamfwd
service:service streamfwd restart
Configure search heads
- Log in to the search head where the Splunk App for Stream is installed.
- Navigate to the Splunk App for Stream, then click Configuration > Distributed Forwarder Management.
- Click Create New Group.
- Enter a name. For example, INFRA_NETFLOW.
- Enter a description.
- Click Next.
- Enter INFRA_NETFLOW as the rule and click Next.
- Do not select any options. Click Finish.
- Navigate to the Splunk App for Stream, then click Configuration > Configure Streams.
- Click New Stream > Metadata.
- Enter Name as INFRA_NETFLOW.
- Select NetFlow as the protocol.
- Enter a description then click Next.
- Select No in the Aggregation box then click Next.
- (Optional) Deselect any fields that do not apply to your use case then click Next.
- (Optional) Develop filters to reduce noise from high traffic devices then click Next.
- Select the index for this collection and click enable then click Next.
- Select only the
Infra_netflow group
andCreate_Stream
. - Configure your NetFlow generator to send records to the new
streamfwd
. - Validate your results by searching the configured index on your Splunk platform deployment.
Selecting NetFlow works for NetFlow, sFlow, jFlow, and IPFIX protocols.
Ingest pcap files | Use Stream configuration templates |
This documentation applies to the following versions of Splunk Stream™: 7.1.2, 7.1.3, 7.2.0
Feedback submitted, thanks!