Download topic as PDF
Scripted inputs overview
During indexing, Splunk software uses line termination characters and timestamps to parse incoming data into events. Fields common to all events, such as host, source, sourcetype, eventtype, timestamp, linecount, are then extracted. Custom per-event fields, such as username and transactionId, are also extracted.
You might want to use scripts to send data for indexing, or to prepare data from a non-standard source so that events and extracted fields can be properly parsed. You can use shell scripts, python scripts, Windows batch files, PowerShell, or any other utility that can format and stream the data that you want to index.
You can use a script to stream data or to write the data from a script to a file.
- Streaming data
- In this scenario, the script is started at a specified interval. The platform indexes the
stdoutdata stream from the script.
- Prior to starting the script, the system checks to see If the script is already running. If it is currently running, the script is not restarted.
- Writing data to a file for indexing
- This scenario works like a file input. You create a script to write to a log file and then configure your Splunk deployment to monitor and index this log file.
- You can configure your Splunk deployment to launch the program at specific intervals, rather than configuring an external method (such as cron or Windows scheduled task) to launch the script.
Use cases for scripted inputs
Typical use cases for scripted inputs include the following.
- Access data that is not available as an ordinary file.
- Access data that cannot be sent using TCP or UDP.
- Stream data from command-line tools, such as
vmstatandiostat. - Poll a database, web service, or API for specific data and process the results.
- Reformat complex data to more easily parse the data into events and fields.
- Maintain data sources with slow or resource-intensive startup procedures.
- Provide special or complex handling for transient or unstable inputs.
- Scripts that manage passwords and credentials
- Wrapper scripts for command line inputs that contain special characters (see Using a wrapper script in Getting Data In )
Additional resources
Get data from APIs and other remote data interfaces through scripted inputs in the Getting Data In manual details how to add a scripted input using Splunk Web and how to manually edit the inputs.conf file to add a scripted input. This section focuses on script structure, and provides tips and examples to help you create your own scripts.
For information about working with external lookups to add fields from external sources to events, see Configure external lookups in the Knowledge Manager Manual.
For more information on the data that you can index, see What Splunk software can index in the Getting Data In manual.
|
PREVIOUS Modular inputs examples |
NEXT Setting up a scripted input |
This documentation applies to the following versions of Splunk® Enterprise: 7.0.0, 7.0.1, 7.0.2, 7.0.3, 7.0.4, 7.0.5, 7.0.6, 7.0.7, 7.0.8, 7.0.9, 7.0.10, 7.0.11, 7.0.13, 7.1.0, 7.1.1, 7.1.2, 7.1.3, 7.1.4, 7.1.5, 7.1.6, 7.1.7, 7.1.8, 7.1.9, 7.1.10, 7.2.0, 7.2.1, 7.2.2, 7.2.3, 7.2.4, 7.2.5, 7.2.6, 7.2.7, 7.2.8, 7.2.9, 7.2.10, 7.3.0, 7.3.1, 7.3.2, 7.3.3, 7.3.4, 7.3.5, 7.3.6, 7.3.7, 7.3.8, 7.3.9, 8.0.0, 8.0.1, 8.0.2, 8.0.3, 8.0.4, 8.0.5, 8.0.6, 8.0.7, 8.0.8, 8.0.9, 8.0.10, 8.1.0, 8.1.1, 8.1.2, 8.1.3, 8.1.4, 8.1.5, 8.2.0, 8.2.1, 8.2.2
Feedback submitted, thanks!