Modular inputs overview

Learn how to build a modular input to work with unique data sources, formats, or data input use cases.

Working with modular inputs

Use modular inputs to define a custom input capability. Users can select and configure them like any other Splunk input.

Data sources

The Splunk platform has the following data input options.

• Monitor files and directories.
• Listen on TCP or UDP ports for network events.
• Read the output from a script.

Modular input use cases

Unique use cases might require a modular or scripted input. The following are some typical examples.

• Stream results from a command, such as vmstat and iostat.
• Query a database, web service, or API.
• Reformat complex data.
• Handle sensitive information more securely.
• Handle special characters in inputs.

Modular input features

The modular input API provides the following features.

Feature	Description
Splunk Web access	To view installed modular inputs in Splunk Web, click Settings > Data Inputs.
Validation	Developers can provide validation for modular inputs.
Platform-specific scripts	Package platform-specific versions of the modular input script. For example, you can include a Windows version, a Linux version, and an Apple (Darwin) version in one package.
Stream XML data	Streaming data in XML format lets you annotate the script output and manage how the data is processed.
REST API access	Use Splunk platform REST endpoints to access modular input scripts. You can use capability settings to manage endpoint permissioning.
Single or multiple instance modes	Developers can opt to launch a single instance or multiple instances.

Comparing modular inputs to scripted inputs

Use modular inputs for packaging and sharing technology-specific apps or any app that includes a scripted input.

The following table compares modular inputs and scripted inputs.

Feature	Scripted Inputs	Modular Inputs
Configuration	Inline arguments Separate configuration outside of the Splunk platform.	Parameters defined in inputs.conf Users can configure inputs using Splunk Web input Settings fields. Validation support
Specify event boundaries	Available, but requires additional script complexity.	Yes XML streaming simplifies specifying event boundaries.
Single instance mode	Yes, but requires manual implementation	Yes
Multi-platform support	No	Yes Developers can package a modular input script to include versions for separate platforms.
Checkpointing	Yes, but requires manual implementation.	Yes
Run as user	Yes You can specify which user can run the script.	No All modular input scripts are run as the user using the Splunk instance.
Custom REST endpoints	No	Yes Access modular inputs using REST
REST endpoint authorization	N/A	Yes Use capabilities to control access.

Implementation overview

Start building a modular input by creating a script that streams data for indexing. Some modular input script components are required. There are also optional procedures that you can include to expand functionality.

In addition to the script, an input spec file is also required.

Create a modular input

Here are the steps for creating a modular input.

• Create a modular input script
• Define a scheme for introspection
  • Introspection scheme details
  • Set up streaming (simple or XML)
  • Specify single or multiple instances of a script
• Set up logging
• Set up external validation
• Create a modular input spec file

Add advanced features

Here are some of the more advanced features that you can include in a modular input.

• Enable, disable, and update modular input scripts
• Override default script running behavior
• Specify permissions for modular input scripts
• Implement data checkpoints
• Understand how Splunk software reads the XML configuration
  • Use the modular inputs configuration utility when developing scripts
• Configuration layering for modular inputs
• Create a custom user interface

Developer tools and troubleshooting

The Splunk platform provides some developer tools and troubleshooting tips to assist you in creating modular input scripts:

• REST API access
• Modular inputs configuration utility
• Inputs status endpoint
• Track a modular input script

Modular input examples

The Modular inputs basic example provides an introduction to modular inputs.

Modular inputs examples show more advanced features, including the following.

• Twitter modular input: Stream JSON data from a Twitter source to the Splunk platform for indexing.
• Amazon S3 online storage: Use modular inputs to index data from the Amazon S3 online storage web service.

These examples use Python, but developers can use other languages to write modular input scripts.

Note: The Splunk universal forwarder does not provide a Python interpreter. If you intend to run the examples on a forwarder host, verify that a Python interpreter is installed on the host or, if necessary, install one.

Creating modular inputs with Splunk SDKs

Developers can use Splunk SDKs to create modular inputs in Python, Java, JavaScript, and C#. For more information, see the following resources on the Splunk developer portal.

• How to create modular inputs using Splunk SDK for Python
• How to create modular inputs in the Splunk SDK for Java
• How to work with modular inputs in the Splunk SDK for JavaScript
• How to create modular inputs in Splunk SDK for C#