Admin Manual

 


Send SNMP traps to other systems

This documentation does not apply to the most recent version of Splunk. Click here for the latest version.

Send SNMP traps to other systems

You can use Splunk as a monitoring tool to send SNMP alerts to other systems such as a Network Systems Management console.

If you're interested in sending SNMP traps on Windows, check this Community Wiki topic.

Configuration

Requirements

Create shell script

#!/usr/bin/perl
#
# sendsnmptrap.pl: A script to for Splunk alerts to send an SNMP trap.
#
# Modify the following as necessary for your local environment
#
$hostPortSNMP = "qa-tm1:162"; # Host:Port of snmpd or other SNMP trap handler
$snmpTrapCmd = "/usr/bin/snmptrap"; # Path to snmptrap, from http://www.net-snmp.org
$TRAPOID = "1.3.6.1.4.1.27389.1.2"; # Object IDentifier for traps/notifications 
$OID = "1.3.6.1.4.1.27389.1.1"; # Object IDentifier for objects, Splunk Enterprise OID is 27389
# Parameters passed in from the alert.
# $1-$9 is the positional parameter list. $ARGV[0] starts at $1 in Perl.
$searchCount = $ARGV[0]; # $1 - Number of events returned
$searchTerms = $ARGV[1]; # $2 - Search terms
$searchQuery = $ARGV[2]; # $3 - Fully qualified query string
$searchName = $ARGV[3]; # $4 - Name of saved search
$searchReason = $ARGV[4]; # $5 - Reason saved search triggered
$searchURL = $ARGV[5]; # $6 - URL/Permalink of saved search
$searchTags = $ARGV[6]; # $7 - Always empty as of 4.1
$searchPath = $ARGV[7]; # $8 - Path to raw saved results in Splunk instance (advanced)

# Send trap, with the the parameter list above mapping down into the OID.
$cmd = qq/$snmpTrapCmd -v 2c -c public $hostPortSNMP '' $TRAPOID 
$OID.1 i $searchCount $OID.2 s "$searchTerms" $OID.3 s "$searchQuery" $OID.4 s 
"$searchName" $OID.5 s "$searchReason" $OID.6 s "$searchURL" $OID.7 s 
"$searchTags" $OID.8 s "$searchPath"/;
 system($cmd);

Configure your alert to call a shell script

Shellscript.png

Here is an example of the script running, including what it returns:

[root@qa-tm1 ~]# snmptrapd -f -Lo
2007-08-13 16:13:07 NET-SNMP version 5.2.1.2 Started.
2007-08-13 16:14:03 qa-el4.splunk.com [172.16.0.121] (via UDP: [172.16.0.121]:32883) TRAP, SNMP v1, community public
        SNMPv2-SMI::enterprises.27389.1 Warm Start Trap (0) Uptime: 96 days, 20:45:08.35
        SNMPv2-SMI::enterprises.27389.1.1 = INTEGER: 7 SNMPv2-
SMI::enterprises.27389.1.2 = STRING: "sourcetype::syslog" SNMPv2-
SMI::enterprises.27389.1.3 = STRING: "search sourcetype::syslog starttime:12/31
/1969:16:00:00 endtime::08/13/2007:16:14:01" SNMPv2-SMI::enterprises.27389.1.4 
= STRING: "SyslogEventsLast24" SNMPv2-SMI::enterprises.27389.1.5 = STRING: 
"Saved Search [SyslogEventsLast24]: The number of hosts(7) was greater than 1" 
SNMPv2-SMI::enterprises.27389.1.6 = STRING: "http://qa-el4:18000/?q=sourcetype
%3a%3asyslog%20starttimeu%3a%3a0%20endtimeu%3a%3a1187046841" SNMPv2-
SMI::enterprises.27389.1.7 = STRING: "/home/tet/inst/splunk/var/run/splunk
/SyslogEventsLast24"
2007-08-13 16:14:15 NET-SNMP version 5.2.1.2 Stopped.

This documentation applies to the following versions of Splunk: 4.1 , 4.1.1 , 4.1.2 , 4.1.3 , 4.1.4 , 4.1.5 , 4.1.6 , 4.1.7 , 4.1.8 View the Article History for its revisions.


Comments

The SNMP scripts above won't work if you have complex search terms.
Lets say your saved search looks like this:

source=mylog/* server instance |rex Instance (?.*?)

This command won't execute and the SNMP trap will silently fail to be sent. This of course means that when something went totally wrong in your network you won't be alerted !

The quick fugly work around is to not send the search terms in the trap. I made the following change:

#$searchTerms = $ARGV[1]; # $2 - Search terms
#$searchQuery = $ARGV[2]; # $3 - Fully qualified query string
$searchTerms = "N/A"; # $2 - Search terms
$searchQuery = "N/A"; # $3 - Fully qualified query string

Totally ugly but it keeps traps from going into the black hole. the real fix should be to properly escape the content. As well as do things like length checks to make sure you don't piss off net-snmp.

Zscgeek
February 27, 2011

You must be logged into splunk.com in order to post comments. Log in now.

Was this documentation topic helpful?

If you'd like to hear back from us, please provide your email address:

We'd love to hear what you think about this topic or the documentation as a whole. Feedback you enter here will be delivered to the documentation team.

Feedback submitted, thanks!