Splunk Stream

Installation and Configuration Manual

Download manual as PDF

This documentation does not apply to the most recent version of StreamApp. Click here for the latest version.
Download topic as PDF

Configure 10Gbps network capture

To ensure efficient data capture with minimal packet loss over high-volume 10Gb network interfaces, Splunk Stream lets you deploy an alternate Stream forwarder, which supports 10Gbps data capture on compatible network interfaces.

This page shows you how to optimize your Linux environment and configure Stream forwarder to enable 10Gbps data capture on compatible devices.

Operating system requirements

Dedicated 10Gb capture mode is supported on 64-bit Linux platforms (kernel version 2.6.32 or later). A variety of network devices are supported.

Note: Dedicated 10Gb capture mode has been tested on RHEL/CentOS only.

Optimize Linux environment

For best results with dedicated capture mode, update your kernel boot parameters for hugepages, as follows:

  1. Edit /etc/grub.conf.
  2. Add these parameters to your kernel boot line: default_hugepagesz=1G hugepagesz=1G hugepages=4. For example, after adding these parameters, your kernel boot line might look like this:
    kernel /vmlinuz-2.6.32-573.3.1.el6.x86_64 ro root=/dev/mapper/vg_cmload02-lv_root rd_LVM_LV=vg_cmload02/lv_root rd_NO_LUKS LANG=en_US.UTF-8 rd_NO_MD SYSFONT=latarcyrheb-sun16 crashkernel=auto rd_LVM_LV=vg_cmload02/lv_swap  KEYBOARDTYPE=pc KEYTABLE=us rd_NO_DM default_hugepagesz=1G hugepagesz=1G hugepages=4 rhgb quiet
    
  3. Reboot your linux machine.

Note: Updating kernel boot parameters for dedicated capture mode is optional, though highly recommended.

Configure dedicated 10Gb capture mode

The following configuration steps apply to independent stream forwarder (streamfwd) deployments only. You must be a root user to run streamfwd in dedicated capture mode.

Step 1: Enable dedicated capture mode in streamfwd.conf

  1. Edit local/streamfwd.conf.
  2. Add dedicatedCaptureMode = 1. For example:
    [streamfwd]
    port = 8889
    ipAddr = 127.0.0.1
    dedicatedCaptureMode = 1
    
  3. Restart Splunk.

Step 2: Identify compatible interfaces

Use the streamfwd --iflist command to identity the interface on which you want to capture 10Gbps traffic. You can capture packets at 10Gbps on any interface listed under Dedicated capture mode compatible devices. For example:

# ./linux_x86_64/bin/streamfwd --iflist
Dedicated capture mode compatible devices
=========================================
0000:04:00.0 driver=uio_pci_generic if=
0000:04:00.1 driver=uio_pci_generic if=
0000:05:00.0 driver=uio_pci_generic if=
0000:05:00.1 driver=uio_pci_generic if=

Dedicated capture mode non-compatible devices
=============================================
0000:02:00.0 driver=tg3 if=eth4 *Active*
0000:02:00.1 driver=tg3 if=eth5
0000:02:00.2 driver=tg3 if=eth6
0000:02:00.3 driver=tg3 if=eth7

Step 3: Specify network address in streamfwd.conf

  1. Edit local/streamfwd.conf.
  2. Specify the network address of the 10Gbps-compatible device. For example:
    [streamfwd]
    port = 8889
    ipAddr = 127.0.0.1
    dedicatedCaptureMode = 1
    streamfwdcapture.0.interface = 0000:04:00.0
    

    Dedicated capture mode requires specifying network device(s) using the PCI bus address notation.

  3. Restart Splunk.
PREVIOUS
Configure Flow collector
  NEXT
Use SSL keys for decryption

This documentation applies to the following versions of Splunk Stream: 7.0.0


Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters