Splunk® Enterprise

Developing Views and Apps for Splunk Web

Download manual as PDF

This documentation does not apply to the most recent version of Splunk. Click here for the latest version.
Download topic as PDF

Modular inputs overview

About modular inputs

A modular input is a Splunk Enterprise app or add-on that extends the Splunk Enterprise framework to define a custom input capability. Splunk Enterprise treats your custom input definitions as if they were Splunk Enterprise native inputs. The inputs appear automatically on the Settings > Data Inputs page. From a Splunk Web perspective, your users interactively create and update your custom inputs using Settings, just as they do for Splunk Enterprise native inputs.

Splunk Enterprise data sources

Splunk Enterprise has various ways to input data:

  • Monitor files and directories
  • Listen on TCP or UDP ports for network events
  • Read the output from a script

The following are typical use cases for scripts. You can use traditional scripted inputs or modular inputs for these use cases.

  • Stream results from a command, such as vmstat and iostat.
  • Query a database, web service, or API
  • Reformat complex data
  • Handle sensitive information more securely
  • Handle special characters in inputs

Modular input features

Modular inputs provide the following features:

  • Splunk Web automatically provides access to your custom defined inputs.
  • You can provide validation for the inputs.
  • You can package platform-specific versions of a script. For example you can include a Windows version, a Linux version, and an Apple (Darwin) version in your package.
  • You can stream data as XML data, which allows you to annotate the script output. This gives you greater control of how Splunk Enterprise processes the data.
  • You can use Splunk Enterprise REST endpoints to access your modular input scripts
  • You can set permissions for these endpoints using Splunk Enterprise capabilities.
  • You can define whether to launch a single instance or multiple instances. Single instance mode is useful when running in a single-threaded environment.

Modular inputs vs. scripted inputs

Modular inputs are ideal for packaging and sharing technology-specific apps or any app that includes a scripted input. Modular inputs presented in Splunk Enterprise Settings are easier for users to use and understand. You can capture key information without resorting to editing config files. Additionally, modular inputs provide runtime controls and allows you to stream XML to specify per event index-time settings.

The following table highlights the differences between modular inputs and scripted inputs:

Feature Scripted Inputs Modular Inputs
Configuration Inline arguments

Separate, non-Splunk Enterprise configuration

Parameters defined in inputs.conf

Splunk Web fields treated as native inputs in Settings

Validation support

Specify event boundaries Yes

But with additional complexity in your script

Yes

XML streaming simplifies specifying event boundaries

Single instance mode Yes

Requires manual implementation

Yes
Multi-platform support No Yes

You can package your script to include versions for separate platforms.

Checkpointing Yes

Requires manual implementation.

Yes
Run as Splunk Entrprise user Yes

You can specify which Splunk Enterprise user can run the script.

No

All modular input scripts are run as Splunk Enterprise system user.

Custom REST endpoints No Yes

Modular inputs can be accessed using REST.

Endpoint permissions N/A Access implemented using Splunk Enterprise capabilities

Implement modular inputs

To implement modular inputs, you specify a custom input stream and configuration specifications. It begins with creating the script that streams data for indexing. There are several requirements for your script to implement modular inputs. There are also optional procedures you can include in the script to enhance your implementation. You also have to create an input spec file for your script.

The Modular inputs basic example provides a basic, Hello World style, introduction to modular inputs. Modular inputs examples provides robust examples that detail advanced features.

Basic steps to create modular inputs

Here are the basic steps to create a modular input, with links to the documentation for each step:

Advanced features

Here are some of the more advanced features you can implement for modular inputs:

Developer tools and troubleshooting

Splunk provides some developer tools and troubleshooting tips to assist you in creating modular input scripts:

Modular input examples

The Modular inputs basic example provides a basic, Hello World style, introduction to modular inputs.

Modular inputs examples provides two examples that detail advanced features.

  • Twitter example
    This example streams JSON data from a Twitter source to Splunk for indexing.
  • Amazon S3 online storage example
    This example shows how to use modular inputs to index data from the Amazon S3 online storage web service.

The section Modular inputs examples in this manual provides a complete listing for the examples. The examples are also available for download from Splunk Apps.

These examples use Python for the scripting language. However, you can use various other scripting languages to implement modular inputs.

Note: Splunk Universal Forwarder, unlike other Splunk instances, does not provide a Python interpreter. In this case, to run these examples install Python on the server if one is not already available.


Creating modular inputs with Splunk SDKs

Developers can use Splunk SDKs to create modular inputs in Python, Java, JavaScript, and C#. For more information, see the following resources on the Splunk developer portal.

PREVIOUS
Example script that polls a database
  NEXT
Modular inputs basic example

This documentation applies to the following versions of Splunk® Enterprise: 6.0, 6.0.1, 6.0.2, 6.0.3, 6.0.4, 6.0.5, 6.0.6, 6.0.7, 6.0.8, 6.0.9, 6.0.10, 6.0.11, 6.0.12, 6.0.13, 6.0.14, 6.1, 6.1.1, 6.1.2, 6.1.3, 6.1.4, 6.1.5, 6.1.6, 6.1.7, 6.1.8, 6.1.9, 6.1.10, 6.1.11, 6.1.12, 6.1.13, 6.2.0, 6.2.1, 6.2.2, 6.2.3, 6.2.4, 6.2.5, 6.2.6, 6.2.7, 6.2.8, 6.2.9, 6.2.10, 6.2.11, 6.2.12, 6.2.13


Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters