Documentation > Splunk > Admin Manual > Conditional Routing Example

Admin Manual

 


Overview
Setting Up Data Inputs
Authentication
Changing the Way Splunk Processes Your Data
Creating and Using Configuration Bundles
Configuring Your Sources
Splunk-2-Splunk Management
Integration
Managing Your Filesystem
How-To's

Conditional Routing Example

This documentation does not apply to the most recent version of Splunk. Click here for the latest version.

Contents

Conditional Routing Example

A real-world example.


Requirement

Access & Activity logs must be routed to the "ACCESS" Splunk Server. Error logs must be routed to the "ERROR" Splunk Server. Splunk will be installed on each log source (Unix Servers) in an online J2E\


E Stack to "tail-and-forward" events up to the "ACCESS" or "ERROR" servers. This configuration is to be affected on each server to be delivered by a software deployment tool (Opsware) .


Splunk Configuration Tasks

Module Configuration

In this case, a module is used to define pipelines that cause Splunk to process incoming events based on configuration settings. There will be one pipeline for each destination where events should flow\ 


to.


A pipeline has "global" settings. In our example below, we've given them each arbitrary names to represent their purpose. "name="errorOutboundPipeline" and "accessOutboundPipeline". The pipeline TYPE \


is set to "startup" causing the pipeline to become active when the Splunk server is started.


A pipeline can have several processors. In the each pipeline in our scenario, there is "queue input processor" (QIP). In the first processor's QIP we've created a "config" entry that defines a "queueN\


ame" which we've called "errorQueue" (relating any events that should be routed to the "ERROR" Splunk server). This setting creates a queue for data to be sent to and tells the pipeline that anything s\


ent to this queue shall be processed according to this module's settings. In the second processor, our queueName is "accessQueue". (relating any events that should be routed to the "ACCESS" Splunk serv\


er.). Directing traffic to one or more queues is done later in this document in the "Bundle Configuration" section.


The second processor in each pipeline, the "thruputProcessor" retains its default settings and does not need to be edited.


The third processor in each pipeline, the "tcpoutprocessor" is responsible for sending data out over TCP to a designated IP and port. In the first pipeline, our "tcpoutprocessor" needs a config entry t\


hat contains the "URI" -- or destination host and port where data should be sent. In this scenario, the first pipeline needs to send data to the "ERROR" Splunk server, so the IP address and port (10.1.\


  1. 84:9997) are set accordingly. The second pipeline's "tcpoutprocessor" responsible for routing to the "ACCESS" Splunk server has a configuration entry for URI that has its IP address and port (10.1.1.\

85:9997).


Creating the Module

In /opt/splunk/etc/modules, create a directory for your module to route log event traffic over TCP. For this scenario we'll call it "outboundTCP" (/opt/splunk/etc/modules/outboundTCP). Create a file c\


alled "config.xml" in the "outboundTCP" directory. In that file start with <module></module> and then for every pipeline create a <pipeline></pipeline> block. For this deployment we will use two pipel\


ines using the configuration settings below. 


/opt/splunk/etc/modules/outboundTCP/config.xml
 <module>
<!--  The first pipeline will route messages to the "ERROR" Splunk Server -->
<pipeline name="errorOutboundPipeline" type="startup">
            <processor name="outboundIn" plugin="queueinputprocessor">
                <config>
                    <queueName>errorQueue</queueName>
                </config>
            </processor>
            <processor name="thruput" plugin="thruputprocessor">
                <config>
                    <key>_raw</key>
                    <timeIntervalSecs>30</timeIntervalSecs>
                </config>
            </processor>
            <processor name="tcpout" plugin="tcpoutputprocessor">
                <config>
                    <uri>10.1.1.84:9997</uri>
                    <maxNumberOfRetriesAtHighestBackoff>0</maxNumberOfRetriesAtHighestBackoff>
                    <initialBackoff>1</initialBackoff>
                    <maxBackoff>20</maxBackoff>
                    <backoffAtStartup>5</backoffAtStartup>
                </config>
            </processor>
        </pipeline>
<!--  The second pipeline will route messages to the "ACCESS" Splunk Server -->
<pipeline name="accessOutboundPipeline" type="startup">
            <processor name="outboundIn" plugin="queueinputprocessor">
                <config>
                    <queueName>accessQueue</queueName>
                </config>
            </processor>
            <processor name="thruput" plugin="thruputprocessor">
                <config>
                    <key>_raw</key>
                    <timeIntervalSecs>30</timeIntervalSecs>
                </config>
            </processor>
            <processor name="tcpout" plugin="tcpoutputprocessor">
                <config>
                    <uri>10.1.1.85:9997</uri>
                    <maxNumberOfRetriesAtHighestBackoff>0</maxNumberOfRetriesAtHighestBackoff>
                    <initialBackoff>1</initialBackoff>
                    <maxBackoff>20</maxBackoff>
                    <backoffAtStartup>5</backoffAtStartup>
                </config>
            </processor>
        </pipeline>
</module>

Bundle Configuration

Splunk 2.1 introduced the concept of "bundles", which are a set of one or more Splunk server configuration files that can be created, saved and installed on another Splunk server. A bundle might contai\


n settings for log sources, event type configuration, regular expression filters or saved / Live Splunks. A Splunk server has a main bundle where most of the configuration files go. That bundle is cal\


led "local" (located in /opt/splunk/etc/bundles/local).


For this scenario, we will create and install a new bundle with our conditional routing scenario on every one of our servers that has a log source. We created a new directory to house our bundle in /op\


t/splunk/etc/bundles/fooCorp


The configuration of our bundle will contain three files, "inputs.conf", "props.conf" and "regexes.conf".


INPUTS.CONF

In this scenario, "inputs.conf" will contain all of the direct log sources that should be captured on this Splunk server. An easy way to configure inputs.conf is to use the GUI, which saves all inputs \


in the "/opt/splunk/bundles/local/inputs.conf" file. Open that file, copy the stanzas representing your inputs and paste in to copy this bundle's inputs.conf file. In our case we will be tailing "/var\


/fooCorplogs" and any files in it, which happen to be "fooCorperror.log" and "fooCorpaccess.log"


/opt/splunk/etc/bundles/fooCorp/inputs.conf 


[tail:///var/fooCorplogs]
disabled = false
host = localhost

PROPS.CONF 


In this scenario, "props.conf" will map a source to a REGEX for processing.  "props.conf" can have many different settings as detailed in the props.conf.spec file located in /opt/splunk/etc/bundles.  O\


ur props.conf directs any file in any directory that has the phrase "fooCorperror.log" in it to use the "fwd2error" regex; it also directs any file in any directory that has the phrase "fooCorpaccess.lo\


g" in it to use the "fwd2access" regex.


The regexes referenced in "props.conf" for the fooCorp bundle are contained in the "regexes.conf" file located in the same directory. Below is the configured "props.conf" file:


/opt/splunk/etc/bundles/fooCorp/props.conf 


[source::...fooCorperror.log...]
REGEXES=fwd2error
[source::...fooCorpaccess.log...]
REGEXES=fwd2access

REGEXES.CONF

The "regexes.conf" file contained within the "FooCorp" bundle will action to occur when the regular expression (regex) matches. "regexes.conf" can have many different settings as detailed in the props.\


conf.spec file located in /opt/splunk/etc/bundles. Earlier in the example, we created a module that had two pipelines, each having its own queue and TCP output destination -- "errorQueue" and "accessQue\


ue". "regexes.conf" will tie it all together.


A stanza is added called "[fwd2error]" which contains the regex, destination key and format. In the In our scenario, we want all events from our "fooCorperror.log" source (referenced in "props.conf" to\ 


be sent directly to the "errorQueue" and then processed by the outboundTCP module accordingly.


A stanza is added called "[fwd2access]" which contains the regex, destination key and format. In the In our scenario, we want all events from our "fooCorpaccess.log" source (referenced in "props.conf" \


to be sent directly to the "accessQueue" and then processed by the outboundTCP module accordingly. A regex with just a dot "." is set that matches each case and routes to the appropriate queue. Since\ 


regular expression matching is used, a very complex set of filtering can occur if you need it.


/opt/splunk/etc/bundles/fooCorp/regexes.conf 


[fwd2error]
REGEX=.
DEST_KEY=queue
FORMAT=errorQueue
[fwd2access]
REGEX=.
DEST_KEY=queue
FORMAT=accessQueue

By copying the "outboundTCP module" directory, and the fooCorp bundle to any server running Splunk server software, and restarting Splunk (via the GUI) or by running "splunk restart" from the CLI, simpl\


e set of configuration changes cause a powerful series of events to occur.

Retrieved from "http://docs.splunk.com/Documentation/Splunk/2.2.1/Admin/ConditionalRoutingExample"

This documentation applies to the following versions of Splunk: 2.1 , 2.2 , 2.2.1 , 2.2.3 , 2.2.6 View the Article History for its revisions.


You must be logged into splunk.com in order to post comments. Log in now.

Was this documentation topic helpful?

If you'd like to hear back from us, please provide your email address:

We'd love to hear what you think about this topic or the documentation as a whole. Feedback you enter here will be delivered to the documentation team.

Feedback submitted, thanks!