Route data to different places based on content

Route data to different places based on content

Enable routing to forward data from one Splunk instance to another based on content. For example, you can route data to systems based on sourcetype, a field you've extracted at index time, or the content of the raw event. Routing allows you to specifically distribute events to any system.

Configure routing

To set up routing:

• First, decide which events to route to which servers.
• Then, edit the props.conf, transforms.conf, and outputs.conf files on the forwarding servers.

Edit props.conf

Edit $SPLUNK_HOME/etc/system/local/props.conf and set a TRANSFORMS-routing= attribute:

[<spec>]
TRANSFORMS-routing=$UNIQUE_STANZA_NAME

<spec> can be:

• <sourcetype>, the sourcetype of an event
• host::<host>, where <host> is the host for an event
• source::<source>, where <source> is the source for an event

Use the $UNIQUE_STANZA_NAME when creating your entry in transforms.conf (below).

Edit transforms.conf

Edit $SPLUNK_HOME/etc/system/local/transforms.conf and set rules to match your props.conf stanza:

[$UNIQUE_STANZA_NAME]
REGEX=$YOUR_REGEX
DEST_KEY=_TCP_ROUTING
FORMAT=$UNIQUE_GROUP_NAME

• $UNIQUE_STANZA_NAME must match the name you created in props.conf.
• Enter the regex rules in $YOUR_REGEX to determine which events get conditionally routed.
• DEST_KEY should be set to _TCP_ROUTING to send events via TCP
• Set FORMAT to $UNIQUE_GROUP_NAME. This should match the group name you create in outputs.conf

Edit outputs.conf

Edit $SPLUNK_HOME/etc/system/local/outputs.conf and set which tcpout outputs go to which servers or groups:

[tcpout:$UNIQUE_GROUP_NAME]
server=$IP:$PORT

• Set $UNIQUE_GROUP_NAME to match the name you created in transforms.conf.
• Set the IP address and port to match the receiving server.

Basic routing example

The following example sends all events with sourcetype="syslog" to one target group, all events that contain the word error to another target group, and everything else to a third target group.

1. Edit $SPLUNK_HOME/etc/system/local/props.conf and set a TRANSFORMS-routing= attribute:

[default]
TRANSFORMS-routing=errorRouting

[syslog]
TRANSFORMS-routing=syslogRouting

2. Edit $SPLUNK_HOME/etc/system/local/transforms.conf and set errorRouting and syslogRouting rules:

[errorRouting]
REGEX=error
DEST_KEY=_TCP_ROUTING
FORMAT=errorGroup

[syslogRouting]
REGEX=.
DEST_KEY=_TCP_ROUTING
FORMAT=syslogGroup

3. Edit $SPLUNK_HOME/etc/system/local/outputs.conf and set which tcpout outputs go to with servers or groups:

[tcpout]
defaultGroup=everythingElseGroup

[tcpout:syslogGroup]
server=10.1.1.197:9997

[tcpout:errorGroup]
server=10.1.1.200:9999

[tcpout:everythingElseGroup]
server=10.1.1.250:6666

Advanced routing example

This examples combines routing, data balancing and target group specific parameters. This outputs.conf sends all events with sourcetype="syslog" to one balanced target group, all events that contain the word error to a different target group, and clones everything else to two target groups. The heartbeat frequency for all target groups is dialed down to 10 seconds.

Note: Steps 1 and 2, props.conf and transforms.conf, are the same as the example above.

3. Edit $SPLUNK_HOME/etc/system/local/outputs.conf and set which tcpout outputs go to with servers or groups:

[tcpout]
defaultGroup=everythingElseGroup1, everthingElseGroup2
heartbeatFrequency=10

[tcpout:syslogGroup]
server=10.1.1.197:9997, 10.1.1.198:7777

[tcpout:errorGroup]
server=10.1.1.200:9999

[tcpout:everythingElseGroup1]
server=10.1.1.240:6666

[tcpout:everythingElseGroup2]
server=10.1.1.245:5555

Indexing and routing

In order to only route certain data while still indexing locally you'll need to make sure to configure your outputs.conf as follows:

[tcpout]
indexAndForward=true

This will ensure that your Splunk instance will keep of copy of the data it routes.

• Was this topic useful?
• Post a comment