Configure 10Gbps network capture
To ensure efficient data capture with minimal packet loss over high-volume 10Gb network interfaces, Splunk App for Stream lets you deploy an alternate Stream forwarder, which supports 10Gbps data capture on compatible network interfaces.
This page shows you how to optimize your Linux environment and configure Stream forwarder to enable 10Gbps data capture on compatible devices.
Operating system requirements
Dedicated 10Gb capture mode is supported on 64-bit Linux platforms (kernel version 2.6.32 or later). A variety of network devices are supported.
Note: Dedicated 10Gb capture mode has been tested on RHEL/CentOS only.
Optimize Linux environment
For best results with dedicated capture mode, update your kernel boot parameters for hugepages, as follows:
- Edit
/etc/grub.conf
. - Add these parameters to your kernel boot line:
default_hugepagesz=1G hugepagesz=1G hugepages=4
. For example, after adding these parameters, your kernel boot line might look like this:kernel /vmlinuz-2.6.32-573.3.1.el6.x86_64 ro root=/dev/mapper/vg_cmload02-lv_root rd_LVM_LV=vg_cmload02/lv_root rd_NO_LUKS LANG=en_US.UTF-8 rd_NO_MD SYSFONT=latarcyrheb-sun16 crashkernel=auto rd_LVM_LV=vg_cmload02/lv_swap KEYBOARDTYPE=pc KEYTABLE=us rd_NO_DM default_hugepagesz=1G hugepagesz=1G hugepages=4 rhgb quiet
- Reboot your linux machine.
Note: Updating kernel boot parameters for dedicated capture mode is optional, though highly recommended.
Configure dedicated 10Gb capture mode
The following configuration steps apply to independent stream forwarder (streamfwd
) deployments only. You must be a root user to run streamfwd
in dedicated capture mode.
Step 1: Select streamfwd executable that supports 10Gb capture mode
Splunk App for Stream provides two different streamfwd
executables: one which supports dedicated 10Gb capture mode and one which does not. The streamfwd
executable that does not support dedicated 10Gb capture mode is chosen by default.
To select the streamfwd
executable that supports dedicated capture mode, run set_capturemode.sh dedicated
in the appropriate directory, depending on your stream forwarder deployment, as shown:
Splunk_TA_stream deployment
cd $SPLUNK_HOME/etc/apps/Splunk_TA_stream/ sudo ./set_capturemode.sh dedicated
Indpendent Stream forwarder deployment
cd /opt/streamfwd/scripts sudo ./set_capturemode.sh dedicated
Step 2: Enable dedicated capture mode in streamfwd.conf
To enable dedicated capture mode, add dedicatedCaptureMode = 1
to your local/streamfwd.conf
. For example:
[streamfwd] port = 8889 ipAddr = 127.0.0.1 dedicatedCaptureMode = 1
Step 3: Identify compatible interfaces
You can select the dedicated capture mode compatible interfaces on which you want to capture 10Gbps traffic. To identify compatible interfaces, run bin/streamfwd --iflist
. For example:
# ./bin/streamfwd --iflist Dedicated capture mode compatible devices ========================================= 0000:04:00.0 driver=uio_pci_generic if= 0000:04:00.1 driver=uio_pci_generic if= 0000:05:00.0 driver=uio_pci_generic if= 0000:05:00.1 driver=uio_pci_generic if= Dedicated capture mode non-compatible devices ============================================= 0000:02:00.0 driver=tg3 if=eth4 *Active* 0000:02:00.1 driver=tg3 if=eth5 0000:02:00.2 driver=tg3 if=eth6 0000:02:00.3 driver=tg3 if=eth7
You can capture packets at 10Gbps on any interface listed under Dedicated capture mode compatible devices
.
Note: Dedicated capture mode compatible network interfaces use PCI bus addressing format.
Step 4: Specify network address in streamfwd.conf
In local/streamfwd.conf
, specify the network address on which you want 10 Gbps capture to occur. For example:
[streamfwd] port = 8889 ipAddr = 127.0.0.1 dedicatedCaptureMode = 1 streamfwdcapture.0.interface = 0000:04:00.0
Configure Stream forwarder | Configure universal forwarder for Splunk_TA_stream |
This documentation applies to the following versions of Splunk Stream™: 6.6.0, 6.6.1, 6.6.2
Feedback submitted, thanks!