Splunk® Enterprise

Developing Views and Apps for Splunk Web

Download manual as PDF

This documentation does not apply to the most recent version of Splunk. Click here for the latest version.
Download topic as PDF

Scripted inputs overview

Splunk Enterprise understands many types of data and can immediately index these data sources to make the data available for searching. See What Splunk can index in the Getting Data In manual.

Line termination characters and timestamps are used to parse the data into events. Fields are then extracted that each event shares, such as host, source, sourcetype, eventtype, timestamp, linecount and others. Custom per-event fields, such as username and transactionId, are also extracted.

However, there are times when you want to use scripts to feed data for indexing, or to prepare data from a non-standard source so events and extracted fields can be properly parsed. You can use shell scripts, python scripts, Windows batch files, PowerShell, or any other utility that can format and stream the data that you want to index. You can stream the data or write the data from a script to a file.

Streaming data In the streaming model, Splunk starts the script at a specified interval. Splunk indexes the stdout data stream from the script. Before Splunk starts a script, it checks to see if the script is already running. If the script is running Splunk does not restart the script.

Writing data to a file for indexing In this model, you configure a script to write to a log file. Then configure Splunk to monitor and index the log file. This scenario is basically file input into Splunk. However, you can configure Splunk to launch the program at specific intervals, rather than configure an external method (such as cron or Windows scheduled task) for launching the script.

Get data from APIs and other remote data interfaces through scripted inputs in the Getting Data In manual details how to add a scripted input using Splunk Web and how to manually edit the inputs.conf file to add a scripted input. This section focuses on the structure of a script, and provides tips and examples to help you create your own scripts.

Use cases for scripted inputs

Typical use cases for scripted inputs are:

  • Access data that is not available as an ordinary file.
  • Access data that cannot be sent using TCP or UDP.
  • Stream data from command-line tools, such as vmstat and iostat.
  • Poll a database, web service, or API for specific data, and process the results.
  • Reformat complex data so you can more easily parse the data into events and fields.
  • Maintain data sources with slow or resource-intensive startup procedures.
  • Provide special or complex handling for transient or unstable inputs.
  • Scripts that manage passwords and credentials.
  • Wrapper scripts for command line inputs that contains special characters (see "Using a wrapper script" in the Getting Data In manual)
PREVIOUS
How to restrict your users to one app
  NEXT
Setting up a scripted input

This documentation applies to the following versions of Splunk® Enterprise: 6.0, 6.0.1, 6.0.2, 6.0.3, 6.0.4, 6.0.5, 6.0.6, 6.0.7, 6.0.8, 6.0.9, 6.0.10, 6.0.11, 6.0.12, 6.0.13, 6.0.14, 6.1, 6.1.1, 6.1.2, 6.1.3, 6.1.4, 6.1.5, 6.1.6, 6.1.7, 6.1.8, 6.1.9, 6.1.10, 6.1.11, 6.1.12, 6.1.13, 6.2.0, 6.2.1, 6.2.2, 6.2.3, 6.2.4, 6.2.5, 6.2.6, 6.2.7, 6.2.8, 6.2.9, 6.2.10, 6.2.11, 6.2.12, 6.2.13


Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters