Forward data to third-party systems

Splunk forwarders can forward raw data to non-Splunk systems over a plain TCP socket or packaged in standard syslog. Because they are forwarding to a non-Splunk system, they can send only raw data.

By editing outputs.conf, props.conf, and transforms.conf, you can configure a heavy forwarder to route data conditionally to third-party systems, in the same way that it routes data conditionally to other Splunk instances. You can filter the data by host, source, or source type. You can also use regular expressions to further qualify the data.

Data forwarding to third-party systems is one of several search result export methods that Splunk software offers. For information about the other export methods available to you, see Export search results in the Search Manual.

TCP data

You can use any kind of forwarder, such as a universal forwarder, to forward TCP data to a third-party system:

1. Configure the third party receiving host to expect incoming data on a TCP port.

2. Edit outputs.conf to specify the receiving host and port.

To route the data, you must use a heavy forwarder, which has the ability to parse data.

3. Edit props.conf to determine what data to route.

4. Edit transforms.conf to determine where to route the data based on what you configured in props.conf.

Edit configuration files

To forward data, edit outputs.conf:

• Specify target groups for the receiving servers.
• Specify the IP address and TCP port for each receiving server.
• Set sendCookedData to false, so that the forwarder sends raw data.

To route and filter the data on heavy forwarders only, also edit props.conf and transforms.conf:

• In props.conf, specify the host, source, or sourcetype of your data stream. Specify a transform to perform on the input.
• In transforms.conf, define the transform and specify _TCP_ROUTING. You can also use regular expressions to further filter the data.

Forward all data

This example shows how to send all the data from a forwarder to a third-party system. Since you are sending all the data, you only need to edit outputs.conf:

[tcpout]

[tcpout:fastlane]
server = 10.1.1.35:6996
sendCookedData = false

Forward a subset of data

This example shows how to use a heavy forwarder to filter a subset of data and send the subset to a third-party system. Light and universal forwarders cannot route or filter data.

1. Edit props.conf and transforms.conf to specify the filtering criteria.

In props.conf, apply the bigmoney transform to all host names beginning with nyc:

[host::nyc*]
TRANSFORMS-nyc = bigmoney

In transforms.conf, configure the bigmoney transform to specify TCP_ROUTING as the DEST_KEY and the bigmoneyreader target group as the FORMAT:

[bigmoney]
REGEX = .
DEST_KEY=_TCP_ROUTING
FORMAT=bigmoneyreader

2. In outputs.conf, define both a bigmoneyreader target group for the non-Splunk server and a default target group to receive any other data:

[tcpout]
defaultGroup = default-clone-group-192_168_1_104_9997

[tcpout:default-clone-group-192_168_1_104_9997]
server = 192.168.1.104:9997

[tcpout:bigmoneyreader]
server=10.1.1.197:7999
sendCookedData=false

The forwarder will send all data from host names beginning with nyc to the non-Splunk server specified in the bigmoneyreader target group. It will send data from all other hosts to the server specified in the default-clone-group-192_168_1_104_9997 target group.

Note: If you want to forward only the data specifically identified in props.conf and transforms.conf, set defaultGroup=nothing.

Syslog data

You can configure a heavy forwarder to send data in standard syslog format. The forwarder sends the data through a separate output processor. The syslog output processor is not available for universal or light forwarders.

The syslog output processor sends RFC 3164-compliant events to a TCP/UDP-based server and port, making the payload of any non-compliant data RFC 3164-compliant.

By default, Splunk software does not change the content of an event to make its character set compliant with the third-party server. You can specify a SEDCMD configuration in props.conf to address data that contains characters that the third-party server cannot process. This option is useful for removing newline characters from Windows Event Log events. See Anonymize data through a sed script in Getting Data In.

You can also filter the data with props.conf and transforms.conf. When you do so, you need to specify _SYSLOG_ROUTING as the DEST_KEY.

Forward syslog data to a third-party host

1. Identify the third-party receiving host.

2. On the forwarder that is to send data to the third-party host, open $SPLUNK_HOME/etc/system/local/outputs.conf for editing.

3. In the outputs.conf file, add a stanza that specifies the receiving host in a syslog target group.

[syslog]
defaultGroup=syslogGroup

[syslog:syslogGroup]
server = 10.1.1.197:514

If you define multiple event types for syslog data, you must include the string "syslog" in all of the event type names.

Forward syslog data

In outputs.conf, specify the syslog target group:

[syslog:<target_group>]
<attribute1> = <val1>
<attribute2> = <val2>
...

The target group stanza requires this attribute:

Required Attribute	Default	Value
server	n/a	This must be in the format <hostname_or_ ipaddress>:<port>. This is a combination of the IP address or servername of the syslog server and the port on which the syslog server is listening. Note that syslog servers use port 514 by default.

These attributes are optional:

Optional Attribute	Default	Value
type	udp	The transport protocol. Must be set to "tcp" or "udp".
priority	<13> - this signifies a facility of 1 ("user") and a severity of 5 ("notice")	Syslog priority. This must be an integer 1 to 3 digits in length, surrounded by angle brackets; for example: <34>. This value will appear in the syslog header. Mimics the number passed via syslog interface call; see outputs.conf for more information. Compute the priority value as (<facility> * 8) + <severity>. If facility is 4 (security/authorization messages) and severity is 2 (critical conditions), priority value will be: (4 * 8) + 2 = 34, which you specify in the conf file as <34>.
syslogSourceType	n/a	This must be in the format sourcetype::syslog, the source type for syslog messages.
timestampformat	""	The format used when adding a timestamp to the header. This must be in the format: <%b %e %H:%M:%S>. See "Configure timestamps" in the Getting Data In manual for details.

Send a subset of data to a syslog server

This example shows how to configure a heavy forwarder to forward data from hosts whose names begin with "nyc" to a syslog server named "loghost.example.com" over port 514:

1. Edit props.conf and transforms.conf to specify the filtering criteria.

In props.conf, apply the send_to_syslog transform to all host names beginning with nyc:

[host::nyc*]
TRANSFORMS-nyc = send_to_syslog

In transforms.conf, configure the send_to_syslog transform to specify _SYSLOG_ROUTING as the DEST_KEY and the my_syslog_group target group as the FORMAT:

[send_to_syslog]
REGEX = .
DEST_KEY = _SYSLOG_ROUTING
FORMAT = my_syslog_group

2. In outputs.conf, define the my_syslog_group target group for the non-Splunk server:

[syslog:my_syslog_group]
server = loghost.example.com:514