Modular inputs basic example

This topic shows the steps necessary to create a modular input. It uses a trivial Hello, World style script that lets you concentrate on the basic framework and structure of modular inputs. It omits details of an actual script you might use to index a stream of data. It also omits advanced configuration data you might use to fine tune the operation of the modular input.

The example uses Python as a scripting language. However, you can use a scripting language of your choice to create the script. The script should contain the same functional parts that the example Python script illustrates. The Splunk Developer Portal contains modular input examples for each of the Splunk SDKs.

Basic implementation requirements

A modular input is essentially a Splunk add-on. You place the modular input implementation in the same location you place apps and add-ons.

Directory	Description
bin	Required. Contains the script for the modular input.
README	Required. Contains inputs.conf.spec to register the modular input scheme.
default	Optional. Contains app.conf to configure the modular input as an add-on.
metadata	Optional. Contains default.meta to set permissions to share the script.

Script modes

A script for a modular input typically runs in three modes: introspection, execution, and validation.

Script mode	Description
Introspection	Defines the endpoints and behavior of the script. A modular input script must provide an introspection routine, even if it is a trivial routine that exits with a return code of zero. The script must define the command line argument, --scheme, to access the introspection routine.
Execution	Streams data for indexing.
Validation	Optional. Validates input data. If present, this routine guarantees that the script only accepts valid data. When implementing validation define the command line argument, --validate-arguments, to access the validation routine.

Essential Python script and configuration file for modular inputs

This minimal modular input contains a Python script file that creates a source type based on user inputs. The script contains an empty introspection routine and an empty validation routine. hello_mi is the name of the add-on that implements the modular input.

Python script file

This script has been made cross-compatible with Python 2 and Python 3 using python-future.

# $SPLUNK_HOME/etc/apps/hello_mi/bin/hello.py
from __future__ import print_function
from builtins import str
import sys
import xml.dom.minidom, xml.sax.saxutils

# Empty introspection routine
def do_scheme(): 
    pass

# Empty validation routine. This routine is optional.
def validate_arguments(): 
    pass

# Routine to get the value of an input
def get_who(): 
    try:
        # read everything from stdin
        config_str = sys.stdin.read()

        # parse the config XML
        doc = xml.dom.minidom.parseString(config_str)
        root = doc.documentElement
        conf_node = root.getElementsByTagName("configuration")[0]
        if conf_node:
            stanza = conf_node.getElementsByTagName("stanza")[0]
            if stanza:
                stanza_name = stanza.getAttribute("name")
                if stanza_name:
                    params = stanza.getElementsByTagName("param")
                    for param in params:
                        param_name = param.getAttribute("name")
                        if param_name and param.firstChild and \
                           param.firstChild.nodeType == param.firstChild.TEXT_NODE and \
                           param_name == "who":
                            return param.firstChild.data
    except Exception as e:
        raise Exception("Error getting Splunk configuration via STDIN: %s" % str(e))

    return ""

# Routine to index data
def run_script(): 
    print("hello world, %s!" % get_who())

# Script must implement these args: scheme, validate-arguments
if __name__ == '__main__':
    if len(sys.argv) > 1:
        if sys.argv[1] == "--scheme":
            do_scheme()
        elif sys.argv[1] == "--validate-arguments":
            validate_arguments()
        else:
            pass
    else:
        run_script()

    sys.exit(0)

Configuration file for modular inputs

inputs.conf.spec defines the default scheme for the modular input. The configuration file must contain at least one stanza referencing the input. Each stanza must contain one or more parameters. The values for the parameters in the configuration file are not used.

*$SPLUNK_HOME/etc/apps/hello_mi/README/inputs.conf.spec

[hello://<default>]
*Set up the hello scheme defaults.

who = <value>

• Note: Avoid adding the start_by_shell parameter to inputs.conf.spec. This parameter should only be used in inputs.conf. See Override default run behavior for modular input scripts for more information.

Access the modular input from Splunk Web

After creating the modular input, you can access it various ways from Splunk Web, and also from the Splunk Enterprise management port.

Note: Screen captures for this topic are from Splunk Enterprise 6. The layout from earlier versions may differ.

Data inputs

Navigate to Settings > Data inputs to view the input under Local inputs.

Click Add new to add additional data for your input.

Search page

After creating the modular input and adding some data, create the following search from the Search page to see event listings from your modular input.

Click the sourcetype link to view details of the source types you created.

Splunk Enterprise management port

You can access the REST endpoint for the modular input from the Splunk Enterprise management port. This example uses the default settings to access the REST endpoint:

https://localhost:8089/servicesNS/admin/

Add introspection and validation routines

To enhance the basic implementation you can add introspection and validation routines. The examples in Modular inputs examples provide details on introspection and validation.