Splunk® Enterprise

Distributed Deployment Manual

Download manual as PDF

Splunk version 4.x reached its End of Life on October 1, 2013. Please see the migration information.
This documentation does not apply to the most recent version of Splunk. Click here for the latest version.
Download topic as PDF

Forward data to third-party systems

Splunk forwarders can forward raw data to non-Splunk systems. They can send the data over a plain TCP socket or packaged in standard syslog. Because they are forwarding to a non-Splunk system, they can send only raw data.

By editing outputs.conf, props.conf, and transforms.conf, you can configure a heavy forwarder to route data conditionally to third-party systems, in the same way that it routes data conditionally to other Splunk instances. You can filter the data by host, source, or source type. You can also use regex to further qualify the data.

TCP data

To forward TCP data to a third-party system, edit the forwarder's outputs.conf file to specify the receiving server and port. You must also configure the receiving server to expect the incoming data stream on that port. You can use any kind of forwarder, such as a universal forwarder, to perform this type of forwarding.

To route the data, you need to use a heavy forwarder, which has the ability to parse data. Edit the forwarder's props.conf and transforms.conf files as well as outputs.conf.

Edit the configuration files

To simply forward data, edit outputs.conf:

  • Specify target groups for the receiving servers.
  • Specify the IP address and TCP port for each receiving server.
  • Set sendCookedData to false, so that the forwarder sends raw data.

To route and filter the data (heavy forwarders only), also edit props.conf and transforms.conf:

  • In props.conf, specify the host, source, or sourcetype of your data stream. Specify a transform to perform on the input.
  • In transforms.conf, define the transform and specify _TCP_ROUTING. You can also use regex to further filter the data.

Forward all data

This example shows how to send all the data from a universal forwarder to a third-party system. Since you are sending all the data, you only need to edit outputs.conf:

[tcpout]

[tcpout:fastlane]
server = 10.1.1.35:6996
sendCookedData = false

Forward a subset of data

This example shows how to use a heavy forwarder to filter a subset of data and send the subset to a third-party system:

1. Edit props.conf and transforms.conf to specify the filtering criteria.

In props.conf, apply the bigmoney transform to all host names beginning with nyc:

[host::nyc*]
TRANSFORMS-nyc = bigmoney

In transforms.conf, configure the bigmoney transform to specify TCP_ROUTING as the DEST_KEY and the bigmoneyreader target group as the FORMAT:

[bigmoney]
REGEX = .
DEST_KEY=_TCP_ROUTING
FORMAT=bigmoneyreader

2. In outputs.conf, define both a bigmoneyreader target group for the non-Splunk server and a default target group to receive any other data:

[tcpout]
defaultGroup = default-clone-group-192_168_1_104_9997

[tcpout:default-clone-group-192_168_1_104_9997]
server = 192.168.1.104:9997

[tcpout:bigmoneyreader]
server=10.1.1.197:7999
sendCookedData=false

The forwarder will send all data from host names beginning with nyc to the non-Splunk server specified in the bigmoneyreader target group. It will send data from all other hosts to the server specified in the default-clone-group-192_168_1_104_9997 target group.

Note: If you want to forward only the data specifically identified in props.conf and transforms.conf, set defaultGroup=nothing.

Syslog data

You can configure a heavy forwarder to send data in standard syslog format. The forwarder sends the data through a separate output processor. You can also filter the data with props.conf and transforms.conf. You'll need to specify _SYSLOG_ROUTING as the DEST_KEY.

Note: The syslog output processor is not available for universal or light forwarders.

The syslog output processor sends RFC 3164 compliant events to a TCP/UDP-based server and port, making the payload of any non-compliant data RFC 3164 compliant. Yes, that means Windows event logs!

To forward syslog data, identify the third-party receiving server and specify it in a syslog target group in the forwarder's outputs.conf file.

Note: If you have defined multiple event types for syslog data, the event type names must all include the string "syslog".

Forward syslog data

In outputs.conf, specify the syslog target group:

[syslog:<target_group>]
<attribute1> = <val1>
<attribute2> = <val2>
...

The target group stanza requires this attribute:

Required Attribute Default Value
server n/a This must be in the format <ipaddress_or_servername>:<port>. This is a combination of the IP address or servername of the syslog server and the port on which the syslog server is listening. Note that syslog servers use port 514 by default.


These attributes are optional:

Optional Attribute Default Value
type udp The transport protocol. Must be set to "tcp" or "udp".
priority <13> - this signifies a facility of 1 ("user") and a severity of 5 ("notice") Syslog priority. This must be an integer 1 to 3 digits in length, surrounded by angle brackets; for example: <34>. This value will appear in the syslog header.

Mimics the number passed via syslog interface call; see outputs.conf for more information.

Compute the priority value as (<facility> * 8) + <severity>. If facility is 4 (security/authorization messages) and severity is 2 (critical conditions), priority value will be: (4 * 8) + 2 = 34, which you specify in the conf file as <34>.

syslogSourceType n/a This must be in the format sourcetype::syslog, the source type for syslog messages.
timestampformat "" The format used when adding a timestamp to the header. This must be in the format: <%b %e %H:%M:%S>. See "Configure timestamps" in the Getting Data In manual for details.


Send a subset of data to a syslog server

This example shows how to configure a heavy forwarder to forward data from hosts whose names begin with "nyc" to a syslog server named "loghost.example.com" over port 514:

1. Edit props.conf and transforms.conf to specify the filtering criteria.

In props.conf, apply the send_to_syslog transform to all host names beginning with nyc:

[host::nyc*]
TRANSFORMS-nyc = send_to_syslog

In transforms.conf, configure the send_to_syslog transform to specify _SYSLOG_ROUTING as the DEST_KEY and the my_syslog_group target group as the FORMAT:

[send_to_syslog]
REGEX = .
DEST_KEY = _SYSLOG_ROUTING
FORMAT = my_syslog_group

2. In outputs.conf, define the my_syslog_group target group for the non-Splunk server:

[syslog:my_syslog_group]
server = loghost.example.com:514
PREVIOUS
Route and filter data
  NEXT
Introducing the universal forwarder

This documentation applies to the following versions of Splunk® Enterprise: 4.3, 4.3.1, 4.3.2, 4.3.3, 4.3.4, 4.3.5, 4.3.6, 4.3.7, 5.0, 5.0.1, 5.0.2, 5.0.3, 5.0.4, 5.0.5, 5.0.6, 5.0.7, 5.0.8, 5.0.9, 5.0.10, 5.0.11, 5.0.12, 5.0.13, 5.0.14, 5.0.15, 5.0.16, 5.0.17, 5.0.18


Comments

in "Send a subset of data to a syslog server"<br />to send all the events with the syslog sourcetype to the third party system<br /><br />[syslog]<br />TRANSFORMS-fwdsyslog = send_to_syslog

Ykherian, Splunker
October 3, 2012

Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters