Splunk® App for Unix and Linux

Install and Use the Splunk App for Unix and Linux

Download manual as PDF

Download topic as PDF

Saved searches

The Splunk App for Unix and Linux includes a number of saved searches that it uses to populate the Home, Metrics, Hosts and Alerts dashboards. This topic lists the searches by category and provides a description of what the searches do.

CPU searches

Saved search Intended purpose
Percent CPU by Host (UNIX - CPU) Returns per-host CPU usage percentage events. Uses the `Percent_CPU_by_Host(*)` macro.
Percent Load by Host (UNIX - CPU) Returns per-host CPU load average events. Uses the `Percent_Load_by_Host(*)` macro.
Top 5 CPU Processes by Host (UNIX - CPU) Returns the top five processes, based on CPU usage, per host. Uses the `Top_5_CPU_Processes_by_Host(*)` macro.
Number of Threads by Host (UNIX - CPU) Returns the number of threads in use for each host. Uses the `Number_Threads_by_Host(*)` macro.
Number of Processes by Host (UNIX - CPU) Returns the number of active processes on each host. Uses the `Number_Processes_by_Host(*)` macro.
CPU Usage by Command (UNIX - CPU) Returns per-command CPU usage events for a single host. Uses the `CPU_Usage_by_Command_for_Host(*)` macro.
CPU Usage by User (UNIX - CPU) Returns per-user CPU usage for a single host. Uses the `CPU_Usage_by_User_for_Host(*)` macro.
Usage by State (UNIX - CPU) Returns CPU usage by state for a single host. Uses the `CPU_Usage_by_State_for_Host(*)` macro.
Top CPU Processes for Host (UNIX - CPU) Returns the top processes based on CPU usage for a single host. Uses the `Top_CPU_Processes_for_Host(*)` macro.
Consumption by User Last Hour (UNIX - CPU) Returns the amount of CPU used by each user within the last hour. Uses the `os_index` macro and the "ps" source
Top Users by Consumption Last Hour (UNIX - CPU) Returns the amount of CPU time used by each user within the last hour. Uses the `os_index` macro and the "ps" source.
10 Most Popular Executables Last Hour (UNIX - CPU) Returns the top 10 processes by name in the last hour. Uses the `os_index` macro and the "lsof" source.

Memory searches

Saved search Intended purpose
Mem Usage for Host (UNIX - MEM) Returns per-host memory usage, per host. Uses the `Mem_Usage_for_Host(*)` macro.
Mem Usage by Command for Host (UNIX - MEM) Returns per-host memory usage by command, per host. Uses the `Mem_Usage_by_Command_for_Host(*)` macro.
Top Mem Usage Commands for Host (UNIX - MEM) Returns the top processes, based on memory usage, per host. Uses the `Top_Mem_Command_for_Host(*)` macro.
Top 10 Users by Resident Memory Last Hour (UNIX - MEM) Returns the top 10 users, based on memory usage, per host. Uses the `Top_Users_of_VM_for_Host(*)` macro.
Mem Usage by host Returns the amount of memory used for each host. Uses the `Percent_MEM_by_Host(1)` macro.
Top Commands by Memory and Host (UNIX - MEM) Returns the top 10 commands, based on memory usage, per host. Uses the `Top_Mem_Processes_by_Host(*)` macro.
Physical Memory by Host (UNIX - MEM) Returns the amount of physical memory installed, per host. Uses the `Memory_Hardware_by_Host(*)` macro.
Top_Memory_Users_by_

Command_by_Host

Returns the top memory users, by command, per host. Uses the `Top_Memory_Users_by_Command_by_Host(*)` macro.


Disk Searches

Saved search Intended purpose
Percent Disk Used by Volume and Host (UNIX - Disk) Returns the amount of disk used by each accessible volume, per host. Uses the `Disk_Used_Pct_by_Host(*)` macro.
Files Opened by Command (UNIX - Disk) Returns the number of files opened per command. Uses the `Open_Files_by_Command_and_Host(*)` macro.
Files Opened by Type (UNIX - Disk) Returns the number of files opened, by type. Uses the `Open_Files_by_Type_and_Host(*)` macro.


Sources

Saved search Intended purpose
vmstat Retrieves virtual memory states. Relies on the `os_index` and `memory_sourcetype` macros.
ps Retrieves information about executing processes. Relies on the `os_index` and `ps_sourcetype` macros.
top Retrieves events from the "top" process. Relies on the `os_index` and `top_sourcetype` macros.
hardware Retrieves information about the hardware installed in a host. Relies on the `os_index` and `hardware_sourcetype` macros.
iostat Retrieves information from the "iostat" process. Relies on the `os_index` and `iostat_sourcetype` macros.
netstat Retrieves information from the "netstat" process. Relies on the `os_index` and `netstat_sourcetype` macros.
protocol Retrieves information about network protocols installed on the system. Relies on the `os_index` and `protocol_sourcetype` macros.
openPorts Retrieves information about the open network ports on a system. Relies on the `os_index` and `open_ports_sourcetype` macros.
time Retrieves information about the system time. Relies on the `os_index` and `time_sourcetype` macros.
lsof Retrieves information about all open files on the system. Relies on the `os_index` and `lsof_sourcetype` macros.
df Retrieves information about disk usage on the system. Uses the `os_index` and `df_sourcetype` macros.
who Retrieves information from the "who" command. Uses the `os_index` and `who_sourcetype` macros.
usersWithLoginPrivs Retrives information on users who can log into the host. Uses the `os_index` and `users_with_login_privs_sourcetype` macros.
lastlog Retrieves information on who has last logged into the system. Uses the `os_index` and `lastlog_sourcetype` macros.
interfaces Gathers information on the network interfaces on the system. Uses the `os_index` and `interfaces_sourcetype` macros.
cpu Gathers information about the system's CPU. Uses the `os_index` and `cpu_sourcetype` macros.
rlog Gathers information from the "rlog" command. Uses the `os_index` and `rlog_sourcetype` macros.
package Gathers information about the software packages that the system has installed on it. Uses the `os_index` and `package_sourcetype` macros.

User Searches

Saved search Intended purpose
User Sessions Total number of user sessions, per host. Uses the `User_Sessions_by_Host(*)` macro.
Failed Logins Total number of failed logins, per host. Uses the `Failed_Logins_by_Host(*)` macro.
User Add Total number of user adds for a host. Uses the `os_index` and `user_add` macros.
User Delete Total number of user deletes for a host. Uses the `os_index` and `user_del` macros.
Group Add Total number of group adds for a host. Uses the `os_index` and `group_add` macros.
Group Delete Total number of group deletes for a host. Uses the `os_index` and `group_del` macros.
Password Change Total number of password changes for a host. Uses the `os_index` and `password_change` macros.
Password Change Failed Total number of failed password changes for a host. Uses the `os_index` and `password_change_failed` macros.
Failed Attempts at SU Total number of times where a user attempted and failed to become the superuser. Uses the `os_index` and `su_failed` macros.

Network Searches

Saved search Intended purpose
Thruput by Interface and Host (UNIX - NET) The amount of network throughput, by interface and host. Uses the `Thruput_by_Interface_by_Host(*)` macro.
Frequently Opened Ports (UNIX - NET) A list of the most frequently opened network ports. Uses the `Frequently_Open_Ports_by_Host(*)` macro.
Top Inet Addresses by Host (UNIX - NET) Uses the `Top_Inet_Addresses_by_Host(*)` macro.
Open Ports (UNIX - NET) Uses the `Open_Ports_by_Host(*)` macro.
Addresses Connected To (UNIX - NET) Uses the `Addresses_by_Host(*)` macro.
Sockets by State (UNIX - NET) Uses the `Sockets_by_State_by_Host(*)` macro.
Top 10 Users by Virtual Memory Last Hour (UNIX - MEM) The top 10 users, by virtual memory usage, in the last hour. Uses the `os_index` and `ps_sourcetype` macros.
Virtual Memory Subsystem Stats (UNIX - MEM) Information about a system's memory usage. Uses the `os_index` and `memory_sourcetype` macros.
Memory Usage over Last 3 Hours (UNIX - MEM)] Uses the `os_index` and `memory_sourcetype` macros.
Avg Resident Memory by Process Last 3 Hours (UNIX - MEM) Uses the `os_index` and `ps_sourcetype` macros.
Avg Virtual Memory by Process Last 3 Hours (UNIX - MEM) Uses the `os_index` and `ps_sourcetype` macros.


Package Searches

Saved search Intended purpose
Latest Packages by Host A list of the installed packages, per host. Uses the `os_index` and `package_sourcetype` macros.
Hardware Configurations by Host A detailed list of hardware configurations, per host. Uses the `os_index` `hardware_sourcetype` macros.

Utility Saved Searches

Saved search Intended purpose
UNIX - All Logs Gathers all available *nix logs that have been indexed. Uses the `os_index` macro.
UNIX - All Configs Returns all *nix configuration events. Uses the `os_index` macro.
UNIX - Timechart Errors Or Critical Returns a chart of all 'critical' or 'error' level messages. Uses the `os_index` and `unix_errors` macros.
UNIX - Timechart Config Changes Returns a chart of all *nix configuration changes. Uses the "nix_configs" event type.


Alerts

These alerts come with the Splunk App for Unix and Linux. You can also create additional custom alerts.

Saved search Intended purpose
Alert - syslog errors last hour Returns syslog events of type 'error'. Uses the `syslog_sourcetype` and `syslog_errors` macros. Runs once an hour by default.
Memory_Exceeds_MB_by_Process Triggers when memory usage for processes exceeds a certain level. Returns events per process. Uses the `Memory_Exceeds_MB_by_Process` macro. Runs every 5 minutes.
Memory_Exceeds_Percent_by_Host Triggers when per-host memory usage exceeds a certain perfentage. Returns events per host. Uses the `Memory_Exceeds_Percent_by_Host` macro. Runs every 5 minutes.
Memory_Exceeds_MB_by_Host Triggers when per-host memory usage exceeds a certain level. Returns events per host. Uses the `Memory_Exceeds_Percent_by_Host` macro. Runs every 5 minutes.
CPU_Exceeds_Percent_by_Host Triggers when per-host CPU usage exceeds a certain percentage. Returns events per host. Uses the `CPU_Exceeds_Percent_by_Host` macro. Runs every 5 minutes.
CPU_Under_Percent_by_Host Triggers when per-host CPU usage remains below a certain percentage. Returns events per host. Uses the `CPU_Under_Percent_by_Host` macro. Runs every 5 minutes.
Load_Exceeds_by_Host Triggers when per-host load averages exceed a certain level. Returns events per host. Uses the `Load_Exceeds_by_Host` macro. Runs every 5 minutes.
Threads_Exceeds_by_Host Triggers when per-host thread counts exceed a certain level. Returns events per host. Uses the `Threads_Exceeds_by_Host` macro. Runs every 5 minutes.
Processes_Exceeds_by_Host Triggers when per-host process counts exceed a certain level. Returns events per host. Uses the `Processes_Exceeds_by_Host` macro. Runs every 5 minutes.
Disk_Used_Exceeds_Perc_by_Host Triggers when per-host disk usage exceeds a certain percentage. Returns events per host. Uses the `Disk_Used_Exceeds_Percent_by_Host` macro. Runs every 5 minutes.
Open_Files_Exceeds_by_Process Triggers when per-process open file counts exceed a certain level. Returns events per process. Uses the `Open_Files_Exceeds_by_Process` macro. Runs every 5 minutes.
IO_Wait_Exceeds_Threshold Triggers when the amount of system I/O wait time exceeds a certain level. Returns events per host. Uses the `IO_Wait_Exceeds_Threshold` macro. Runs every 5 minutes.
IO_Utilization_Exceeds_Threshold Triggers when the amount of system I/O utilization exceeds a certain level. Returns events per host. Uses the `IO_Utilization_Exceeds_Threshold` macro. Runs every 5 minutes.

Home screen (regular and full screen)

The following searches power the Home screens with information about categories and groups that you have defined in the configuration settings.

Saved search Intended purpose
Dropdown Lookup - Dimension Populates the Category drop-down list. Uses the dropdowns.csv lookup table.
Dropdown Lookup - Group Populates the Group drop-down list based on the Category you have selected. Uses the dropdowns.csv lookup table.

Metrics screen

Saved search Intended purpose
Metrics Selectable Lookup Populates the Metrics viewer page with categories, groups, and host information. Uses the dropdowns.csv lookup table.

Lookups

Saved search Intended purpose
__generate_lookup_dropdowns Creates the dropdowns.csv lookup table by searching collected data for the top 50 hosts (by index time).
__safeguard_generate_lookup_

dropdowns

PREVIOUS
Create custom alerts
  NEXT
Search macros

This documentation applies to the following versions of Splunk® App for Unix and Linux: 5.2.3, 5.2.4, 5.2.5


Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters