Command line options for the Independent Stream Forwarder
Independent Stream Forwarder includes command line options that let you read PCAP file data, send PCAP file data to indexers, identify network interfaces, manage SSL keys, and perform other configuration tasks.
The streamfwd
binary is located in $SPLUNK_HOME/etc/apps/Splunk_TA_stream/<OS_arch>/bin
or, for independent streamfwd
deployment, /opt/streamfwd/bin
.
To view all streamfwd
command line options, specify the -h
option. For example:
[root@myserver bin]# ./streamfwd -h Usage: streamfwd [-r FILE1]... [--pcapdir DIR1]... [pcap_options] [options] [output_option] Process live or saved network traffic and forward the resulting events. streamfwd COMMAND Run command and exit. Options: -D Run as daemon (or Windows service). -v Verbose. (See also streamfwdlog.conf.) --PARAM VALUE Same as setting PARAM = VALUE in streamfwd.conf or inputs.conf. Only applies to non-nested params. Examples: --index my_index --processingThreads 6 But not: --streamfwdcapture.2.repeat false -r FILE Read network traffic from a pcap file. --pcapdir DIR Read network traffic from pcap files in a directory. Relative files or directories are relative to cwd. If no PCAP files or directories are specified, live network traffic is captured. pcap_options: -b BITS_PER_SECOND Restricts bit rate (approximately). Applies to all pcap files. Default: 100 Mbps (10 Mbps with --repeat) --systime Use system time instead of times from pcap file. Applies to all pcap files. --repeat Repeat pcap files forever. Applies to and only to all files specified with -r. --afteringest ACTION Action to take after ingesting a pcap file from a directory. Applies to and only to all directories specified with --pcapdir. Default: --afteringest move actions (for --afteringest): delete Delete the file. move [SUBDIR] Move the file to a subdirectory. [default: finished_pcaps] Directory will be created if necessary. ignore Leave the file but mark it as already processed. repeat Continue to re-ingest all pcap files in rotation. stop Leave the file. After processing each directory once, stop monitoring. output_options: --modinput Run as a modular input. Output to stdout. (Also, input from stdin.) -s SERVER Send output to another instance of streamfwd. (SERVER = [https://]HOST[:PORT]) The output behavior is determined by both the directory structure containing <code>streamfwd</code> and <code>output_option</.code> (if specified). Rules in priority order: 1) If the directory looks like an independent agent, output is sent via HTTP to splunk. 2) If <code>output_option = --modinput</code>, the <code>streamfwd</code> runs as a modular input. 3) If directory looks like a TA and there are no command line arguments, <code>streamfwd</code> runs as a modular input. 4) If <code>output_option = -s SERVER</code>, the output is sent to SERVER. 5) Otherwise, output is sent to <code>localhost:8889</code>. commands: -h, --help Show this message and exit. --version Show version and build and exit. --scheme Show modular input scheme and exit. --validate-arguments Validate modular input XML config and exit. --iflist List network interfaces and exit. --sslkeylist List SSL keys. --addsslkey KEY_NAME PEM_FILE [PASSWORD] Add specified SSL key. --deletesslkey KEY_NAME Delete specified SSL key. -c [TEMPLATE_NAME] Activate specified product template. -c Deactivate any active product template. --listtemplates List installed product templates.
streamfwd
command line options override the streamfwd.conf
configuration file, which by default captures data from all network devices. streamfwd
command line options also override any specific capture locations specified by the streamfwdcapture
parameter in streamfwd.conf
.
Note: You do not need root privileges to run streamfwd
commands.
About streamfwd output behavior
The output behavior of the streamfwd
command differs depending on whether you are running the streamfwd
binary as an independent deployment or as part of Splunk_TA_stream
.
- If you are using an Independent Stream Forwarder
streamfwd
deployment, the output is sent to indexers by HTTP event collector. See Deploy Independent Stream Forwarder in this manual.
- If you are using
Splunk_TA_stream
, the output is sent throughlocalhost:8889
to the Splunk TA for Stream Wire Data, which forwards recieved events along with the events it generates itself. To confirm that the Splunk TA for Stream Wire Data is runnning:
- Click on Settings > Data inputs > Wire data.
- If the modular input status indicates disabled, click Enable.
Locating streamfwd.conf
streamfwd
looks for streamfwd.conf
in these directories:
- For
Splunk_TA_stream
:
$SPLUNK_HOME/etc/apps/Splunk_TA_stream/default $SPLUNK_HOME/etc/apps/Splunk_TA_stream/local
- For
streamfwd
independent deployment:
$STREAMFWD_PATH/default $STREAMFWD_PATH/local
where $STREAMFWD_PATH
is /opt/streamfwd
by default.
For information on the correct usage of default
and local
directories, see About configuration files in the Splunk Enterprise Admin Manual.
Examples
List network interfaces
Use the --iflist
option to view all network interfaces on Windows or Linux machines.
For example, on a Windows machine:
C:\Splunk_Home\etc\apps\Splunk_TA_stream\windows_x86_64\bin>streamfwd.exe --iflist <Sniffer> <Interface> <Name>\Device\NPF_{D6995D00-B75C-48DB-99AA-69F0150126BC}</Name> <Alias>Local Area Connection</Alias> <Description>Intel(R) PRO/1000 MT Network Connection</Description> </Interface> </Sniffer>
Read PCAP files
Use the -r <PCAP_FILE>
option to read the contents of a PCAP file.
For example:
[root@myserver bin] ./streamfwd -r my.pcap 16:35:30.094 INFO stream.CaptureServer - Found DataDirectory: /opt/splunk/etc/apps/Splunk_TA_stream/data 16:35:30.113 INFO stream.CaptureServer - Found UIDirectory: /opt/splunk/etc/apps/Splunk_TA_stream/ui 16:35:30.917 INFO stream.StreamSender - Successfully pinged server: d35f1088-eec6-4a9e-baf3-de0b0ad6870c 16:35:30.917 INFO stream.CaptureServer - Default configuration directory: /opt/splunk/etc/apps/Splunk_TA_stream/default 16:35:30.921 INFO stream.CaptureServer - Start sending pcap data 16:35:31.005 INFO stream.StreamSender - Successfully pinged server: d35f1088-eec9-4a9e-baf1-de0b0ad6021c 16:35:31.248 INFO stream.CaptureServer - Configuring offline capture with pcap file /root/network_data.pcap 16:35:31.266 INFO stream.CaptureServer - Starting data capture 16:35:31.267 INFO stream.SnifferReactor - Starting network capture: sniffer 16:35:31.318 INFO stream.main - streamfwd has started successfully (version 7.0.0 build 99) 16:35:31.331 INFO stream.SnifferReactor - Finished reading pcap file: /root/network_data.pcap
You can use the -r
option multiple times to specify multiple PCAP files to read in parallel. The -r
option is implied if one of your arguments is a valid PCAP file name. The following is functionally equivalent to the command in the above example:
[root@myserver bin] ./streamfwd my.pcap
If you provide a PCAP file without an -s
option, streamfwd
assumes "-s localhost:8889". Both of these examples send the data that the PCAP file contains to the Splunk TA for Stream Wire Data running on the server.
Note: Stream does not support .pcapng
file format on Windows. To use .pcapng
files on Windows, you must first convert them to .pcap
file format.
For more information, see Ingest pcap files in this manual.
Repeat PCAP files
Use the --repeat
option to cause streamfwd
to continuously repeat PCAP files until it is terminated.
For example, to continuously repeat two PCAP files at the rate of 1 Mbps each (2 Mbps total):
./streamfwd -r my.pcap -r your.pcap -b 1048576 --repeat
Get streamfwd
version
Use the --version
option to get the current streamfwd
version.
For example:
[root@myserver bin] ./streamfwd --version streamfwd version 7.0.0 build 99
Get modular input scheme
Use the --scheme
option to print the modular input scheme.
For example:
[root@myserver bin] ./streamfwd --scheme <scheme><title>Wire data</title><description>Passively capture wire data from network traffic.</description> <use_external_validation>true</use_external_validation><use_single_instance>true</use_single_instance> <streaming_mode>xml</streaming_mode><endpoint><args><arg name="splunk_stream_app_location"><title>Splunk App for Stream Location</title><description>URI including full path to splunk_app_stream installation (i.e. http://localhost:8000/en-us/custom/splunk_app_stream/) </description><validation>validate(match('splunk_stream_app_location', '^https?://.+'), 'Location must start with http:// or https://')</validation></arg <arg name="stream_forwarder_id"><title>Stream Forwarder Identifier</title><description>A string identifier for Stream forwarder</description> </arg><arg name="sslVerifyServerCert"><title>Verify Server Certificate</title><description>If true, Stream forwarder will make sure that the server that its connecting to has a valid SSL certificate. Defaults to false </description><validation>validate(is_bool('sslVerifyServerCert'),'Verify Server Certificate must be either true or false')</validation></arg><arg name="rootCA"><title>Root CA File</title><description>The path to the root certificate authority file. This value is used only if Verify Server Certificate is set to true. </description></arg><arg name="sslCommonNameToCheck"><title>Common Name of Server Certificate</title><description>By default, Stream forwarder uses host name to match the server certificate common name. Override this value to change that behavior. This value is used only if Verify Server Certificate is set to true.</description></arg></args></endpoint></scheme>
Install an Independent Stream Forwarder | streamfwd.conf |
This documentation applies to the following versions of Splunk Stream™: 7.3.0, 7.4.0, 8.0.0
Feedback submitted, thanks!