Scripted inputs overview
Splunk Enterprise understands many types of data and can immediately index these data sources to make the data available for searching. See What Splunk can index in the Getting Data In manual.
Line termination characters and timestamps are used to parse the data into events. Fields are then extracted that each event shares, such as host, source, sourcetype, eventtype, timestamp, linecount and others. Custom per-event fields, such as username and transactionId, are also extracted.
However, there are times when you want to use scripts to feed data for indexing, or to prepare data from a non-standard source so events and extracted fields can be properly parsed. You can use shell scripts, python scripts, Windows batch files, PowerShell, or any other utility that can format and stream the data that you want to index. You can stream the data or write the data from a script to a file.
Streaming data In the streaming model, Splunk starts the script at a specified interval. Splunk indexes the stdout data stream from the script. Before Splunk starts a script, it checks to see if the script is already running. If the script is running Splunk does not restart the script.
Writing data to a file for indexing In this model, you configure a script to write to a log file. Then configure Splunk to monitor and index the log file. This scenario is basically file input into Splunk. However, you can configure Splunk to launch the program at specific intervals, rather than configure an external method (such as cron or Windows scheduled task) for launching the script.
Get data from APIs and other remote data interfaces through scripted inputs in the Getting Data In manual details how to add a scripted input using Splunk Web and how to manually edit the
inputs.conf file to add a scripted input. This section focuses on the structure of a script, and provides tips and examples to help you create your own scripts.
Use cases for scripted inputs
Typical use cases for scripted inputs are:
- Access data that is not available as an ordinary file.
- Access data that cannot be sent using TCP or UDP.
- Stream data from command-line tools, such as vmstat and iostat.
- Poll a database, web service, or API for specific data, and process the results.
- Reformat complex data so you can more easily parse the data into events and fields.
- Maintain data sources with slow or resource-intensive startup procedures.
- Provide special or complex handling for transient or unstable inputs.
- Scripts that manage passwords and credentials.
- Wrapper scripts for command line inputs that contains special characters (see "Using a wrapper script" in the Getting Data In manual)
How to restrict your users to one app
Setting up a scripted input
This documentation applies to the following versions of Splunk® Enterprise: 6.0, 6.0.1, 6.0.2, 6.0.3, 6.0.4, 6.0.5, 6.0.6, 6.0.7, 6.0.8, 6.0.9, 6.0.10, 6.0.11, 6.0.12, 6.0.13, 6.0.14, 6.1, 6.1.1, 6.1.2, 6.1.3, 6.1.4, 6.1.5, 6.1.6, 6.1.7, 6.1.8, 6.1.9, 6.1.10, 6.1.11, 6.1.12, 6.1.13, 6.2.0, 6.2.1, 6.2.2, 6.2.3, 6.2.4, 6.2.5, 6.2.6, 6.2.7, 6.2.8, 6.2.9, 6.2.10, 6.2.11, 6.2.12, 6.2.13