Forward data to third-party systems
Splunk forwarders can forward raw data to non-Splunk systems over a plain TCP socket or packaged in standard syslog. Because they are forwarding to a non-Splunk system, they can send only raw data.
By editing outputs.conf
, props.conf
, and transforms.conf,
you can configure a heavy forwarder to route data conditionally to third-party systems, in the same way that it routes data conditionally to other Splunk instances. You can filter the data by host, source, or source type. You can also use regular expressions to further qualify the data.
Data forwarding to third-party systems is one of several search result export methods that Splunk software offers. For information about the other export methods available to you, see Export search results in the Search Manual.
TCP data
You can use any kind of forwarder, such as a universal forwarder, to forward TCP data to a third-party system:
1. Configure the third party receiving host to expect incoming data on a TCP port.
2. Edit outputs.conf to specify the receiving host and port.
To route the data, you must use a heavy forwarder, which has the ability to parse data.
3. Edit props.conf to determine what data to route.
4. Edit transforms.conf to determine where to route the data based on what you configured in props.conf
.
Edit configuration files
To forward data, edit outputs.conf
:
- Specify target groups for the receiving servers.
- Specify the IP address and TCP port for each receiving server.
- Set
sendCookedData
tofalse
, so that the forwarder sends raw data.
To route and filter the data on heavy forwarders only, also edit props.conf
and transforms.conf
:
- In
props.conf
, specify the host, source, or sourcetype of your data stream. Specify a transform to perform on the input. - In
transforms.conf
, define the transform and specify_TCP_ROUTING
. You can also use regular expressions to further filter the data.
Forward all data
This example shows how to send all the data from a forwarder to a third-party system. Since you are sending all the data, you only need to edit outputs.conf
:
[tcpout] [tcpout:fastlane] server = 10.1.1.35:6996 sendCookedData = false
Forward a subset of data
This example shows how to use a heavy forwarder to filter a subset of data and send the subset to a third-party system. Light and universal forwarders cannot route or filter data.
1. Edit props.conf
and transforms.conf
to specify the filtering criteria.
In props.conf
, apply the bigmoney
transform to all host names beginning with nyc
:
[host::nyc*] TRANSFORMS-nyc = bigmoney
In transforms.conf
, configure the bigmoney
transform to specify TCP_ROUTING
as the DEST_KEY
and the bigmoneyreader
target group as the FORMAT
:
[bigmoney] REGEX = . DEST_KEY=_TCP_ROUTING FORMAT=bigmoneyreader
2. In outputs.conf
, define both a bigmoneyreader
target group for the non-Splunk server and a default target group to receive any other data:
[tcpout] defaultGroup = default-clone-group-192_168_1_104_9997 [tcpout:default-clone-group-192_168_1_104_9997] server = 192.168.1.104:9997 [tcpout:bigmoneyreader] server=10.1.1.197:7999 sendCookedData=false
The forwarder will send all data from host names beginning with nyc
to the non-Splunk server specified in the bigmoneyreader
target group. It will send data from all other hosts to the server specified in the default-clone-group-192_168_1_104_9997
target group.
Note: If you want to forward only the data specifically identified in props.conf
and transforms.conf
, set defaultGroup=nothing
.
Syslog data
You can configure a heavy forwarder to send data in standard syslog format. The forwarder sends the data through a separate output processor. The syslog output processor is not available for universal or light forwarders.
The syslog output processor sends RFC 3164-compliant events to a TCP/UDP-based server and port, making the payload of any non-compliant data RFC 3164-compliant.
By default, Splunk software does not change the content of an event to make its character set compliant with the third-party server. You can specify a SEDCMD
configuration in props.conf
to address data that contains characters that the third-party server cannot process. This option is useful for removing newline characters from Windows Event Log events. See Anonymize data through a sed script in Getting Data In.
You can also filter the data with props.conf
and transforms.conf
. When you do so, you need to specify _SYSLOG_ROUTING as the DEST_KEY.
Forward syslog data to a third-party host
1. Identify the third-party receiving host.
2. On the forwarder that is to send data to the third-party host, open $SPLUNK_HOME/etc/system/local/outputs.conf
for editing.
3. In the outputs.conf
file, add a stanza that specifies the receiving host in a syslog
target group.
[syslog] defaultGroup=syslogGroup [syslog:syslogGroup] server = 10.1.1.197:514
If you define multiple event types for syslog data, you must include the string "syslog" in all of the event type names.
Forward syslog data
In outputs.conf
, specify the syslog
target group:
[syslog:<target_group>] <attribute1> = <val1> <attribute2> = <val2> ...
The target group stanza requires this attribute:
Required Attribute | Default | Value |
---|---|---|
server | n/a | This must be in the format <hostname_or_ ipaddress>:<port> . This is a combination of the IP address or servername of the syslog server and the port on which the syslog server is listening. Note that syslog servers use port 514 by default.
|
These attributes are optional:
Optional Attribute | Default | Value |
---|---|---|
type | udp | The transport protocol. Must be set to "tcp" or "udp". |
priority | <13> - this signifies a facility of 1 ("user") and a severity of 5 ("notice") | Syslog priority. This must be an integer 1 to 3 digits in length, surrounded by angle brackets; for example: <34>. This value will appear in the syslog header.
Mimics the number passed via syslog interface call; see outputs.conf for more information. Compute the priority value as (<facility> * 8) + <severity>. If facility is 4 (security/authorization messages) and severity is 2 (critical conditions), priority value will be: (4 * 8) + 2 = 34, which you specify in the conf file as <34>. |
syslogSourceType | n/a | This must be in the format sourcetype::syslog , the source type for syslog messages.
|
timestampformat | "" | The format used when adding a timestamp to the header. This must be in the format: <%b %e %H:%M:%S>. See "Configure timestamps" in the Getting Data In manual for details. |
Send a subset of data to a syslog server
This example shows how to configure a heavy forwarder to forward data from hosts whose names begin with "nyc" to a syslog server named "loghost.example.com" over port 514:
1. Edit props.conf
and transforms.conf
to specify the filtering criteria.
In props.conf
, apply the send_to_syslog
transform to all host names beginning with nyc
:
[host::nyc*] TRANSFORMS-nyc = send_to_syslog
In transforms.conf
, configure the send_to_syslog
transform to specify _SYSLOG_ROUTING
as the DEST_KEY
and the my_syslog_group
target group as the FORMAT
:
[send_to_syslog] REGEX = . DEST_KEY = _SYSLOG_ROUTING FORMAT = my_syslog_group
2. In outputs.conf
, define the my_syslog_group
target group for the non-Splunk server:
[syslog:my_syslog_group] server = loghost.example.com:514
Route and filter data | Troubleshoot forwarder/receiver connection |
This documentation applies to the following versions of Splunk® Enterprise: 7.0.0, 7.0.1, 7.0.2, 7.0.3, 7.0.4, 7.0.5, 7.0.6, 7.0.7, 7.0.8, 7.0.9, 7.0.10, 7.0.11, 7.0.13, 7.1.0, 7.1.1, 7.1.2, 7.1.3, 7.1.4, 7.1.5, 7.1.6, 7.1.7, 7.1.8, 7.1.9, 7.1.10, 7.2.0, 7.2.1, 7.2.2, 7.2.3, 7.2.4, 7.2.5, 7.2.6, 7.2.7, 7.2.8, 7.2.9, 7.2.10, 7.3.0, 7.3.1, 7.3.2, 7.3.3, 7.3.4, 7.3.5, 7.3.6, 7.3.7, 7.3.8, 7.3.9, 8.0.0, 8.0.1, 8.0.2, 8.0.3, 8.0.4, 8.0.5, 8.0.6, 8.0.7, 8.0.8, 8.0.9, 8.0.10, 8.1.1, 8.1.2, 8.1.3, 8.1.4, 8.1.5, 8.1.6, 8.1.7, 8.1.8, 8.1.9, 8.1.13, 8.1.14, 8.2.0, 8.2.1, 8.2.2, 8.2.3, 8.2.4, 8.2.5, 8.2.6, 8.2.7, 8.2.8, 8.2.9, 8.2.10, 8.2.11, 8.2.12, 9.0.0, 9.0.1, 9.0.2, 9.0.3, 9.0.4, 9.0.5, 9.0.6, 9.0.7, 9.0.8, 9.0.9, 9.0.10, 9.1.0, 9.1.1, 9.1.2, 9.1.3, 9.1.4, 9.1.5, 9.1.6, 9.2.0, 9.2.1, 9.2.2, 9.2.3, 9.3.0, 9.3.1, 8.1.0, 8.1.10, 8.1.11, 8.1.12
Feedback submitted, thanks!