Scripted inputs overview
This documentation does not apply to the most recent version of Splunk. Click here for the latest version.
Scripted inputs overview
Splunk understands many types of data and can immediately index these data sources to make the data available for searching. See What Splunk can index in the Getting Data In manual.
Splunk uses line termination characters and timestamps to parse the data into events. It then extracts fields which each event shares, such as host, source, sourcetype, eventtype, timestamp, linecount and others. Splunk also extracts custom per-event fields, such as username and transactionId.
However, there are times when you want to use scripts to feed data to Splunk for indexing, or prepare data from a non-standard source so Splunk can properly parse events and extract fields. You can use shell scripts, python scripts, Windows batch files, PowerShell, or any other utility that can format and stream the data that you want Splunk to index. You can stream the data to Splunk or write the data from a script to a file.
Streaming data to Splunk In the streaming model, Splunk starts the script at a specified interval. Splunk indexes the stdout data stream from the script. Before Splunk starts a script, it checks to see if the script is already running. If the script is running Splunk does not restart the script.
Writing data to a file for Splunk to index In this model, you configure a script to write to a log file. Then configure Splunk to monitor and index the log file. This scenario is basically file input into Splunk. However, you can configure Splunk to launch the program at specific intervals, rather than configure an external method (such as cron or Windows scheduled task) for launching the script.
Get data from APIs and other remote data interfaces through scripted inputs in the Getting Data In manual details how to add a scripted input using Splunk Web and how to manually edit the
inputs.conf file to add a scripted input. This section focuses on the structure of a script, and provides tips and examples to help you create your own scripts.
Use cases for scripted inputs
Typical use cases for scripted inputs are:
- Whenever data is not available as an ordinary file, or the data cannot be sent using TCP or UDP.
- Stream data from command-line tools, such as vmstat and iostat.
- Poll a database, web service, or API for specific data, and process the results with Splunk.
- Reformat complex data so Splunk can more easily parse the data into events and fields.
- Maintain data sources with slow or resource-intensive startup procedures.
- Provide special or complex handling for transient or unstable inputs.
- Scripts that manage passwords and credentials.
- Wrapper scripts for command line inputs that contains special characters (see "Using a wrapper script" in the Getting Data In manual)