Command line options for the Independent Stream Forwarder

Independent Stream Forwarder includes command line options that let you read PCAP file data, send PCAP file data to indexers, identify network interfaces, manage SSL keys, and perform other configuration tasks.

The streamfwd binary is located in $SPLUNK_HOME/etc/apps/Splunk_TA_stream/<OS_arch>/bin or, for independent streamfwd deployment, /opt/streamfwd/bin.

To view all streamfwd command line options, specify the -h option. For example:

[root@myserver bin]# ./streamfwd -h

Usage:
  streamfwd [-r FILE1]... [--pcapdir DIR1]... [pcap_options] [options] [output_option]
      Process live or saved network traffic and forward the resulting events.

  streamfwd COMMAND
      Run command and exit.

Options:
  -D                           Run as daemon (or Windows service).
  -v                           Verbose.  (See also streamfwdlog.conf.)
  --PARAM VALUE                Same as setting PARAM = VALUE in streamfwd.conf or inputs.conf.
                               Only applies to non-nested params.
                               Examples:
                                 --index my_index
                                 --processingThreads 6
                               But not:
                                 --streamfwdcapture.2.repeat false
  -r FILE                      Read network traffic from a pcap file.
  --pcapdir DIR                Read network traffic from pcap files in a directory.

Relative files or directories are relative to cwd.
If no PCAP files or directories are specified, live network traffic is captured.

pcap_options:
  -b BITS_PER_SECOND           Restricts bit rate (approximately).  Applies to all pcap files.
                               Default: 100 Mbps (10 Mbps with --repeat)
  --systime                    Use system time instead of times from pcap file.  Applies to all pcap files.
  --repeat                     Repeat pcap files forever.  Applies to and only to all files specified with -r.
  --afteringest ACTION         Action to take after ingesting a pcap file from a directory.
                               Applies to and only to all directories specified with --pcapdir.
                               Default: --afteringest move

actions (for --afteringest):
  delete                       Delete the file.
  move [SUBDIR]                Move the file to a subdirectory.  [default: finished_pcaps]
                               Directory will be created if necessary.
  ignore                       Leave the file but mark it as already processed.
  repeat                       Continue to re-ingest all pcap files in rotation.
  stop                         Leave the file.  After processing each directory once, stop monitoring.

output_options:
  --modinput                   Run as a modular input.  Output to stdout.  (Also, input from stdin.)
  -s SERVER                    Send output to another instance of streamfwd.  (SERVER = [https://]HOST[:PORT])

The output behavior is determined by both the directory structure containing <code>streamfwd</code> and <code>output_option</.code> (if specified).
Rules in priority order:
1) If the directory looks like an independent agent, output is sent via HTTP to splunk.
2) If <code>output_option = --modinput</code>, the <code>streamfwd</code> runs as a modular input.
3) If directory looks like a TA and there are no command line arguments, <code>streamfwd</code> runs as a modular input.
4) If <code>output_option = -s SERVER</code>, the output is sent to SERVER.
5) Otherwise, output is sent to <code>localhost:8889</code>.

commands:
  -h, --help                   Show this message and exit.
  --version                    Show version and build and exit.
  --scheme                     Show modular input scheme and exit.
  --validate-arguments         Validate modular input XML config and exit.
  --iflist                     List network interfaces and exit.
  --sslkeylist                 List SSL keys.
  --addsslkey KEY_NAME PEM_FILE [PASSWORD]
                               Add specified SSL key.
  --deletesslkey KEY_NAME      Delete specified SSL key.
  -c [TEMPLATE_NAME]           Activate specified product template.
  -c                           Deactivate any active product template.
  --listtemplates              List installed product templates.

streamfwd command line options override the streamfwd.conf configuration file, which by default captures data from all network devices. streamfwd command line options also override any specific capture locations specified by the streamfwdcapture parameter in streamfwd.conf.

Note: You do not need root privileges to run streamfwd commands.

About streamfwd output behavior

The output behavior of the streamfwd command differs depending on whether you are running the streamfwd binary as an independent deployment or as part of Splunk_TA_stream.

• If you are using an Independent Stream Forwarder streamfwd deployment, the output is sent to indexers by HTTP event collector. See Deploy Independent Stream Forwarder in this manual.

• If you are using Splunk_TA_stream, the output is sent through localhost:8889 to the Splunk TA for Stream Wire Data, which forwards recieved events along with the events it generates itself. To confirm that the Splunk TA for Stream Wire Data is runnning:

1. Click on Settings > Data inputs > Wire data.
2. If the modular input status indicates disabled, click Enable.

Locating streamfwd.conf

streamfwd looks for streamfwd.conf in these directories:

• For Splunk_TA_stream:

$SPLUNK_HOME/etc/apps/Splunk_TA_stream/default
$SPLUNK_HOME/etc/apps/Splunk_TA_stream/local

• For streamfwd independent deployment:

$STREAMFWD_PATH/default
$STREAMFWD_PATH/local

where $STREAMFWD_PATH is /opt/streamfwd by default.

For information on the correct usage of default and local directories, see About configuration files in the Splunk Enterprise Admin Manual.

Examples

List network interfaces

Use the --iflist option to view all network interfaces on Windows or Linux machines.

For example, on a Windows machine:

C:\Splunk_Home\etc\apps\Splunk_TA_stream\windows_x86_64\bin>streamfwd.exe --iflist
<Sniffer>
  <Interface>
    <Name>\Device\NPF_{D6995D00-B75C-48DB-99AA-69F0150126BC}</Name>
    <Alias>Local Area Connection</Alias>
    <Description>Intel(R) PRO/1000 MT Network Connection</Description>
  </Interface>
</Sniffer>

Read PCAP files

Use the -r <PCAP_FILE> option to read the contents of a PCAP file.

For example:

[root@myserver bin] ./streamfwd -r my.pcap
16:35:30.094 INFO  stream.CaptureServer - Found DataDirectory: /opt/splunk/etc/apps/Splunk_TA_stream/data
16:35:30.113 INFO  stream.CaptureServer - Found UIDirectory: /opt/splunk/etc/apps/Splunk_TA_stream/ui
16:35:30.917 INFO  stream.StreamSender - Successfully pinged server: d35f1088-eec6-4a9e-baf3-de0b0ad6870c
16:35:30.917 INFO  stream.CaptureServer - Default configuration directory: /opt/splunk/etc/apps/Splunk_TA_stream/default
16:35:30.921 INFO  stream.CaptureServer - Start sending pcap data
16:35:31.005 INFO  stream.StreamSender - Successfully pinged server: d35f1088-eec9-4a9e-baf1-de0b0ad6021c
16:35:31.248 INFO  stream.CaptureServer - Configuring offline capture with pcap file /root/network_data.pcap
16:35:31.266 INFO  stream.CaptureServer - Starting data capture
16:35:31.267 INFO  stream.SnifferReactor - Starting network capture: sniffer
16:35:31.318 INFO  stream.main - streamfwd has started successfully (version 7.0.0 build 99)
16:35:31.331 INFO  stream.SnifferReactor - Finished reading pcap file: /root/network_data.pcap

You can use the -r option multiple times to specify multiple PCAP files to read in parallel. The -r option is implied if one of your arguments is a valid PCAP file name. The following is functionally equivalent to the command in the above example:

[root@myserver bin] ./streamfwd my.pcap

If you provide a PCAP file without an -s option, streamfwd assumes "-s localhost:8889". Both of these examples send the data that the PCAP file contains to the Splunk TA for Stream Wire Data running on the server.

Note: Stream does not support .pcapng file format on Windows. To use .pcapng files on Windows, you must first convert them to .pcap file format.

For more information, see Ingest pcap files in this manual.

Repeat PCAP files

Use the --repeat option to cause streamfwd to continuously repeat PCAP files until it is terminated.

For example, to continuously repeat two PCAP files at the rate of 1 Mbps each (2 Mbps total):

./streamfwd -r my.pcap -r your.pcap -b 1048576 --repeat

Get streamfwd version

Use the --version option to get the current streamfwd version.

For example:

[root@myserver bin] ./streamfwd --version
streamfwd version 7.0.0 build 99

Get modular input scheme

Use the --scheme option to print the modular input scheme.

For example:

[root@myserver bin] ./streamfwd --scheme
<scheme><title>Wire data</title><description>Passively capture wire data from network traffic.</description>
<use_external_validation>true</use_external_validation><use_single_instance>true</use_single_instance>
<streaming_mode>xml</streaming_mode><endpoint><args><arg name="splunk_stream_app_location"><title>Splunk App for Stream 
Location</title><description>URI including full path to splunk_app_stream installation (i.e. http://localhost:8000/en-us/custom/splunk_app_stream/)
</description><validation>validate(match('splunk_stream_app_location', '^https?://.+'), 'Location must start with http:// or https://')</validation></arg
<arg name="stream_forwarder_id"><title>Stream Forwarder Identifier</title><description>A string identifier for Stream forwarder</description>
</arg><arg name="sslVerifyServerCert"><title>Verify Server Certificate</title><description>If true, Stream forwarder will make sure that the server 
that its connecting to has a valid SSL certificate. Defaults to false </description><validation>validate(is_bool('sslVerifyServerCert'),'Verify Server 
Certificate must be either true or false')</validation></arg><arg name="rootCA"><title>Root CA File</title><description>The path to the root certificate authority file. This value is used only if Verify Server Certificate is set to true. </description></arg><arg 
name="sslCommonNameToCheck"><title>Common Name of Server Certificate</title><description>By default, Stream forwarder uses host name to 
match the server certificate common name. Override this value to change that behavior. This value is used only if Verify Server Certificate is set to 
true.</description></arg></args></endpoint></scheme>