Set up custom (scripted) inputs

Set up custom (scripted) inputs

Splunk can accept events from scripts that you provide. Scripted input is useful in conjunction with command-line tools, such as vmstat, iostat, netstat, top, etc. You can use scripted input to get data from APIs and other remote data interfaces and message queues. You can then use that data to generate metrics and status data through commands like vmstat, iostat, etc.

Lots of apps on Splunkbase provide scripted inputs for specific applications. -- You can find them on the Browse more apps tab in the Launcher.

You configure custom scripted inputs from Splunk Manager or by editing inputs.conf.

Note: On Windows platforms, you can enable text-based scripts, such those in perl and python, with an intermediary Windows batch (.bat) file.

Caution: Scripts launched through scripted input inherit Splunk's environment, so be sure to clear environment variables that can affect your script's operation. The only environment variable that's likely to cause problems is the library path (most commonly known as LD_LIBRARY_PATH on linux/solaris/freebsd).

Add a scripted input in Splunk Web

To add a scripted input in Splunk Web:

1. Click Manager in the upper right-hand corner of Splunk Web.

2. Under System configurations, click Data Inputs.

3. Click Scripts.

4. Click Add new to add an input.

5. In the Command text box, specify the script command, including the path to the script.

6. In Interval, specify the interval in seconds between script runtimes. The default is 60 (seconds).

7. Enter a new Source name to override the default source value, if necessary.

Important: Consult Splunk support before changing this value.

8. Change the Host value, if necessary.

9. Set the Source type.

Source type is a default field added to events. Source type is used to determine processing characteristics such as timestamps and event boundaries. Choose:

• From List. Select one of the predefined source types from the drop-down list.
• Manual. Label your own source type in the text box.

10. Set the Index. Leave the value as "default" unless you have defined multiple indexes to handle different types of events. In addition to indexes meant for user data, Splunk has a number of utility indexes, which show up in the dropdown box.

11. Click Save.

Add a scripted input via inputs.conf

Configure inputs.conf using the following attributes:

[script://$SCRIPT] 
interval = <integer>|<cron schedule>
index = <index>
sourcetype = <iostat, vmstat, etc>  OPTIONAL
source = <iostat, vmstat, etc> OPTIONAL
disabled = <true | false>

• script is the fully-qualified path to the location of the script.
  • As a best practice, put your script in the bin/ directory nearest the inputs.conf where your script is specified. So if you are configuring $SPLUNK_HOME/etc/system/local/inputs.conf, place your script in $SPLUNK_HOME/etc/system/bin/. If you're working on an application in $SPLUNK_HOME/etc/apps/$APPLICATION/, put your script in $SPLUNK_HOME/etc/apps/$APPLICATION/bin/.
• interval indicates how often to execute the specified command. Specify either an integer value representing seconds or a valid cron schedule.
  • Defaults to 60 seconds.
  • When a cron schedule is specified, the script is not executed on start up.
  • Splunk keeps one invocation of a script per instance. Intervals are based on when the script completes. So if you have a script configured to run every 10 minutes and the script takes 20 minutes to complete, the next run will occur 30 minutes after the first run.
  • For constant data streams, enter 1 (or a value smaller than the script's interval).
  • For one-shot data streams, enter -1. Setting interval to -1 will cause the script to run each time the splunk daemon restarts.
• index can be any index in your Splunk instance.
  • Default is main.
• disabled is a boolean value that can be set to true if you want to disable the input.
  • Defaults to false.
• sourcetype and source can be any value you'd like.
  • The value you specify is appended to data coming from your script in the sourcetype= or source= fields.
  • These are optional settings.

If you want the script to run continuously, write the script to never exit and set it on a short interval. This helps to ensure that if there is a problem the script gets restarted. Splunk keeps track of scripts it has spawned and will shut them down upon exit.

Example using inputs.conf

This example shows the use of the UNIX top command as a data input source.

• Start by creating a new application directory. This example uses scripts/:

$ mkdir $SPLUNK_HOME/etc/apps/scripts

• All scripts should be run out of a bin/ directory inside your application directory:
• $ mkdir $SPLUNK_HOME/etc/apps/scripts/bin
• This example uses a small shell script top.sh:

$ #!/bin/sh
 top -bn 1  # linux only - different OSes have different paramaters

• Make sure the script is executable:

chmod +x $SPLUNK_HOME/etc/apps/scripts/bin/top.sh

• Test that the script works by running it via the shell:

$SPLUNK_HOME/etc/apps/scripts/bin/top.sh

• The script should have sent one top output.
• Add the script entry to inputs.conf in $SPLUNK_HOME/etc/apps/scripts/default/:

[script:///opt/splunk/etc/apps/scripts/bin/top.sh]
interval = 5                # run every 5 seconds
sourcetype = top        # set sourcetype to top
source = script://./bin/top.sh   # set source to name of script

props.conf

You may need to modify props.conf:

• By default Splunk breaks the single top entry into multiple events.
• The easiest way to fix this problem is to tell the Splunk server to break only before something that does not exist in the output.

For example, adding the following to $SPLUNK_HOME/etc/apps/scripts/default/props.conf forces all lines into a single event:

[top]
BREAK_ONLY_BEFORE = <stuff>

Since there is no timestamp in the top output we need to tell Splunk to use the current time. This is done in props.conf by setting:

DATETIME_CONFIG = CURRENT

• Was this topic useful?
• Post a comment