Using a Bundle with a Module

Using a Bundle with a Module

You can create a bundle to configure your module, just as you can use bundles with Splunk standard modules. Below is an example using a bundle to only send certain events, 404 messages from an apache log file, to the custom module.

Create the Processor

This example uses the sample C++ module described on the previous page, here referred to as "sampleProcessor".

Create the Module

Make a new directory under $SPLUNK_HOME/etc/modules. For this example, it is called "testModule" and the sampleProcessor executable file goes there.

Next, create a new config.xml with a new pipeline and the desired processors. This pipeline calls "readerin", "sampleProcessor" and "sendOut". readerin is configured to get data from the queue "testQueue", sampleProcessor appends the text "{xyzzy}" to the raw event and sendOut passes it to indexQueue to continue to the indexer.

Note You can also specify the full path instead of using "$$SPLUNK_HOME]]"

<module>
  <pipeline name="test-pipeline" type="startup">
    <processor name="readerIn" plugin="queueinputprocessor">
      <config>
        <queueName>testQueue</queueName>
      </config>
    </processor>
    <processor name="myProcessor" plugin="extcmdprocessor">
      <config>
        <command>$$SPLUNK_HOME]]/etc/modules/testModule/sampleProcessor</command>
        <addToRaw>xyzzy</addToRaw>
      </config>
    </processor>
    <processor name="sendOut" plugin="queueoutputprocessor">
      <config>
        <queueName>indexQueue</queueName>
      </config>
    </processor>
  </pipeline>
</module>

Create the Bundle

Make a new directory under $SPLUNK_HOME/etc/bundles, here also called testModule. In it, create three new files:

Note Replace testbox.splunk.com with your hostname and the tail path with your desired path.

inputs.conf specifies the files we want to tail and assigns a sourcetype.

host = testbox.splunk.com
[tail:///var/log/httpd]
disabled = false
host = testbox.splunk.com
sourcetype = access_log

props.conf specifies that events with sourcetype access_log should use the regular expression configured in regexes.conf for test-pipeline. "REGEXES-test" specifies the class ("-test") to identify a particular configuration stanza. Class names must be unique for each Splunk instance to avoid overriding existing behavior.

[access_log]
REGEXES-test = test-pipeline

regexes.conf defines test-pipeline to look for the specified regular expression and send those events to a queue named "testQueue".

[test-pipeline]
REGEX = \s404\s
DEST_KEY = queue
FORMAT = testQueue

The complete bundle looks like this:

testbox [root]:/opt/splunk/etc/bundles$ ls testModule
inputs.conf     props.conf      regexes.conf

Restart splunk to load the new pipeline. The 404 events should now show the specified text appended:

127.0.0.1 - - [27/Feb/2007:16:02:31 -0800] "GET /foo.html HTTP/1.1" 404 305 {xyzzy}

Add an Additional Processor

You can configure several processors in the same pipeline. For this example, to add a second instance of sampleProcessor, add the XML for a second processor with a different name. It can reference the same executable as myProcessor.

    <processor name="myOtherProcessor" plugin="extcmdprocessor">
      <config>
        <command>$$SPLUNK_HOME]]/etc/modules/testModule/sampleProcessor</command>
        <addToRaw>You are in a maze of twisty little passages, all alike</addToRaw>
      </config>
    </processor>

Then, the text of the event would look like this:

127.0.0.1 - - [27/Feb/2007:16:02:31 -0800] "GET /foo.html HTTP/1.1" 404 305 {xyzzy} {You are in a maze of twisty little passages, all alike}

• Was this topic useful?
• Post a comment