Ingest PCAP files

Packet Capture (PCAP) is an application interface you can use with Splunk Stream to capture network traffic. Splunk Stream provides several methods for ingesting PCAP file data:

• Upload PCAP data in Splunk Web
• Ingest PCAP files using command line options
• Ingest PCAP files using an Independent Stream Forwarder

Supported file formats

Splunk Stream supports .pcap and .pcapng file formats on Linux and Mac.

Splunk Stream does not support .pcapng file format on Windows. To use .pcapng files on Windows, you must first convert them to .pcap file format.

Upload and index PCAP data in Splunk Web

To upload and index PCAP data in Splunk Web, you must have two Splunk Stream packages installed on your search head.

• Splunk App for Stream splunk_app_stream
• Splunk Add-on for for Stream Forwarder Splunk_TA_stream

1. In Splunk Web, go to Settings > Data Inputs.
2. Click on PCAP Files > New.
3. Configure the modular input:

   Field	Description
   Name	The name of the PCAP file modular input.
   PCAP File	Click Choose File and select the PCAP file you want to upload.
   System Time	Uses the system time clock as timestamp for each packet read.
   Repeat	Continuously repeats the PCAP file until the streamfwd process is terminated.
   Host field value	The name of the host that will appear in PCAP events.
   Index	The destination index for the PCAP file data.

4. Click Next.
   The PCAP file data is uploaded and sent to the specified index.

Ingest PCAP files using command line options

To read PCAP file data and send that data to Splunk indexers, you must have Splunk_TA_stream installed.

streamfwd [-r FILE1]... [--pcapdir DIR1]... [pcap_options] [options] [output_option]

Note: Relative files or directories are relative to the current working directory.

For detailed streamfwd command line options and usage information, see streamfwd command line options in this manual.

Read PCAP files

Use the -r option to read individual PCAP files.

./streamfwd -r my.pcap

Ingest PCAP files from a directory

Use the --pcapdir DIR option to monitor and index PCAP files in a directory.

./streamfwd --pcapdir ~/test_pcap_dir --afteringest repeat

The output behavior behavior of the command depends on the type of Stream forwarder you use in your configuration.

• When you use an Independent Stream Forwarder, the output is sent to indexers by the HTTP event collector.
• When you use the Splunk Add-on for Stream Forwarders, output is forwarded to indexers using the Splunk Add-on for Stream Wire Data.

For more information, see streamfwd command line options.

Note: When ingesting PCAP files from directories, make sure the PCAP is complete before moving the file to the directory so that file data is not truncated. You can also use a different file extension name (such as .temp) until the PCAP file is complete, then change the extension name to .pcap.

Ingest PCAP files using an Independent Stream Forwarder

You can configure an Independent Stream Forwarder to:

• Ingest individual PCAP files
• Monitor and ingest PCAP files from specific directories,
• Monitor live network traffic,
• Any combination of these simultaneously.

To configure the independent Stream Forwarder to use PCAP, you add streamfwdcapture parameters to streamfwd.conf.

Parameter	Description
streamfwdcapture.<N>.interface	Specify a network interface name or a path to a PCAP file or a directory of PCAP files.
streamfwdcapture.<N>.interfaceRegex	A regular expression specifying which network interfaces to capture.
streamfwdcapture.<N>.offline	*True means the Independent Forwarder uses PCAP files: the interface must be a PCAP file or a directory to monitor for PCAP files. False indicates that the interface is a network device name. The default value is false.
streamfwdcapture.<N>.repeat	*Only applies if interface is a PCAP file. True indicates that the Independent Forwarder plays back the PCAP file repeatedly for continuous load.
streamfwdcapture.<N>.afterIngest	*Only applies if interface is a directory. Specifies action to take after ingesting a PCAP file from the directory. Possible values: delete: Delete the file. move <subdir>: Move the file to a subdirectory (which will be created if needed). Default is finished_pcaps. ignore: Leave the file but mark it as already processed. repeat: Continue to re-ingest all PCAP files in rotation. stop: Leave the file. After processing each directory once, stop monitoring. *The default value is move.
streamfwdcapture.<N>.sysTime	*Only applies ifstreamfwdcapture.<N>.offline is set to true. True directs the Independent Forwarder to use the system time for packet timestamps instead of timestamps from the PCAP file. Default value is false.
streamfwdcapture.<N>.bitsPerSecond	*Only applies if offline is true. *Rate limiter: if undefined, defaults to 10 Mbps if <Repeat> is true, else 100 Mbps.

Note: The streamfwdcapture.<N>.interface parameter supports both absolute and relative directories. Relative directories are relative to either Splunk_TA_stream/default (Splunk App for Stream) or streamfwd/default (Independent Stream Forwarder).

Examples

The following examples show streamfwd.conf configurations using the streamfwdcapture parameter to ingest PCAP files. These examples apply to Splunk Add-on for Stream Forwarders and Independent Stream Forwarders.

Ingest a single PCAP file

To ingest a PCAP file /tmp/server1.pcap and repeat indefinitely, add the following parameters:

[streamfwd]
streamfwdcapture.0.offline = true
streamfwdcapture.0.interface = /tmp/server1.pcap
streamfwdcapture.0.repeat = true

Where streamfwdcapture.0.offline = true enables PCAP ingestion.

Monitor a single directory

To monitor and ingest PCAP files in a single directory, edit /tmp/test_pcap_dir to add the following parameters:

[streamfwd]
streamfwdcapture.0.offline = true
streamfwdcapture.0.interface = /tmp/test_pcap_dir

In this example streamfwdcapture.0.offline = true enables PCAP ingestion.

Note: If an afterIngest parameter is not specified, the move option is used by default. This automatically moves the PCAP to the ./finished_pcap subdirectory after PCAP ingestion.

Monitor multiple directories

To monitor and ingest PCAP files from two or more directories, use multiple streamfwdcapture.<N> groups. Each directory can have different options:

[streamfwd]
streamfwdcapture.0.offline = true
streamfwdcapture.0.interface = C:\temp\pcap_dir_1
streamfwdcapture.0.sysTime = true

streamfwdcapture.1.offline = true
streamfwdcapture.1.interface = C:\temp\pcap_dir_2
streamfwdcapture.1.afterIngest = delete

In this example:

• pcap files from C:\temp\pcap_dir_2 are processed using their original timestamps because streamfwdcapture.0.sysTime applies only to C:\temp\pcap_dir_1.
• Files from C:\temp\pcap_dir_2 are deleted after ingestion.
• Files from C:\temp\pcap_dir_1 are moved to C:\temp\pcap_dir_1\finished_pcaps.

Monitor both network interface and directory

To capture traffic from a live interface while simultaneously monitoring and ingesting pcap files from a specified directory, use multiple streamfwdcapture.<N> groups. For example:

streamfwdcapture.0.offline = false

streamfwdcapture.1.offline = true
streamfwdcapture.1.interface = /tmp/test_pcap_dir

In this example:

• streamfwdcapture.0.offline = false enables monitoring on all available network interfaces because streamfwdcapture.0.interface is not specified
• streamfwdcapture.1.offline = true enables pcap ingestion from /tmp/test_pcap_dir directory.

For more information about specifying network interfaces, see Use streamfwdcapture to specify network interfaces in this manual.