Splunk® Enterprise

Developing Views and Apps for Splunk Web

Download manual as PDF

Download topic as PDF

Scripted inputs overview

During indexing, Splunk software uses line termination characters and timestamps to parse incoming data into events. Fields common to all events, such as host, source, sourcetype, eventtype, timestamp, linecount, are then extracted. Custom per-event fields, such as username and transactionId, are also extracted.

You might want to use scripts to send data for indexing, or to prepare data from a non-standard source so that events and extracted fields can be properly parsed. You can use shell scripts, python scripts, Windows batch files, PowerShell, or any other utility that can format and stream the data that you want to index.

You can use a script to stream data or to write the data from a script to a file.

Streaming data
In this scenario, the script is started at a specified interval. The platform indexes the stdout data stream from the script.
Prior to starting the script, the system checks to see If the script is already running. If it is currently running, the script is not restarted.
Writing data to a file for indexing
This scenario works like a file input. You create a script to write to a log file and then configure your Splunk deployment to monitor and index this log file.
You can configure your Splunk deployment to launch the program at specific intervals, rather than configuring an external method (such as cron or Windows scheduled task) to launch the script.


Use cases for scripted inputs

Typical use cases for scripted inputs include the following.

  • Access data that is not available as an ordinary file.
  • Access data that cannot be sent using TCP or UDP.
  • Stream data from command-line tools, such as vmstat and iostat.
  • Poll a database, web service, or API for specific data and process the results.
  • Reformat complex data to more easily parse the data into events and fields.
  • Maintain data sources with slow or resource-intensive startup procedures.
  • Provide special or complex handling for transient or unstable inputs.
  • Scripts that manage passwords and credentials
  • Wrapper scripts for command line inputs that contain special characters (see Using a wrapper script in Getting Data In )

Additional resources

Get data from APIs and other remote data interfaces through scripted inputs in the Getting Data In manual details how to add a scripted input using Splunk Web and how to manually edit the inputs.conf file to add a scripted input. This section focuses on script structure, and provides tips and examples to help you create your own scripts.

For information about working with external lookups to add fields from external sources to events, see Configure external lookups in the Knowledge Manager Manual.

For more information on the data that you can index, see What Splunk software can index in the Getting Data In manual.

PREVIOUS
Modular inputs examples
  NEXT
Setting up a scripted input

This documentation applies to the following versions of Splunk® Enterprise: 6.3.0, 6.3.1, 6.3.2, 6.3.3, 6.3.4, 6.3.5, 6.3.6, 6.3.7, 6.3.8, 6.3.9, 6.3.10, 6.3.11, 6.3.12, 6.4.0, 6.4.1, 6.4.2, 6.4.3, 6.4.4, 6.4.5, 6.4.6, 6.4.7, 6.4.8, 6.4.9, 6.5.0, 6.5.1, 6.5.1612 (Splunk Cloud only), 6.5.2, 6.5.3, 6.5.4, 6.5.5, 6.5.6, 6.6.0, 6.6.1, 6.6.2, 6.6.3, 6.6.4, 7.0.0


Comments

Hi Woodcock,
I'll add this link. Thanks for the suggestion.

Frobinson splunk, Splunker
July 14, 2016

It makes sense to add a "see also" link for scripted lookups at the bottom.

Woodcock
July 3, 2016

Hi Pyro wood,
Thanks for this suggestion--I'll look into it. In the meantime, I'd suggest reaching out to other developers in our Splunk Answers community forum: https://answers.splunk.com/

You might also find this topic in our "Getting Data in" manual helpful. http://docs.splunk.com/Documentation/Splunk/latest/Data/Getdatafromscriptedinputs

Frobinson splunk, Splunker
May 26, 2016

Is there a possibility you could add a scripted input example for querying a rest-api?
Thanks!

Pyro wood
May 24, 2016

Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters