Operating System Module configurations

To configure the ITSI Operating System Module:

1. Install a universal forwarder on any host that you want to send data to your ITSI deployment.
2. Install and configure either the Splunk Add-on for Windows or Splunk Add-on for Unix and Linux, depending on the operating system that runs on the host. See below for instructions.
3. Configure either the Splunk Add-on for Windows or Splunk Add-on for Unix and Linux by enabling the data inputs shown below, depending on the type of host.
4. Enable entity discovery for the module to automatically discover entities for which relevant data has been collected. See Enable the automatic entity discovery search.
5. Change the macro definition to include all the indexes that you're using for data collection.

See About forwarding and receiving in the Forwarding Data manual to learn how to install and configure universal forwarders.

Module entity roles

The Operating System Module assigns roles to entities.

ITSI Module	ITSI Role
ITSI Operating System Module	operating_system_host

Install supported technologies

The ITSI Operating System Module supports data from the following add-ons:

• The Splunk Add-on for Unix and Linux for Unix or Linux systems.
• The Splunk Add-on for Microsoft Windows for Microsoft Windows systems.

Use the following table as reference to install supported technologies onto your deployment:

Technology Name	Installation link	Search Heads	Indexers	Forwarders
Splunk Add-on for Microsoft Windows	Installation guide	x	x	x
Splunk Add-on for Unix and Linux	Installation guide	x	x	x

For *nix systems, install the sysstat package to collect operating system data.

See About installing Splunk add-ons to learn how to install a Splunk add-on in the following deployment scenarios.

• Single-instance Splunk Enterprise
• Distributed Splunk Enterprise
• Splunk Cloud Platform
• Splunk Light

See the What data the Splunk Add-on for Unix and Linux collects section of the Splunk Add-on for Unix and Linux manual for a reference of scripted and file inputs.

Configure supported technologies to collect data and send to your Splunk deployment

The ITSI Operating System Module displays host level metrics including CPU, Memory, Storage, and Network. Install the supported technologies that are relevant to your ITSI deployment.

Configure the Splunk Add-on for Unix and Linux to collect data and send to your Splunk deployment

1. Download the Splunk Add-on for Unix and Linux from Splunkbase.
2. From a shell, place the Splunk Add-on for Unix and Linux in the $SPLUNK_HOME/etc/apps directory.
3. Create an inputs.conf file in your $SPLUNK_HOME/etc/apps/Splunk_TA_nix/local/ directory.
4. Refer to the Informational Metrics table below. This table represents the inputs that you must enable for the add-on to provide KPIs to ITSI.
5. Using a text editor, edit $SPLUNK_HOME/etc/apps/Splunk_TA_nix/local/inputs.conf to include the stanzas that the table references.
6. Enable them by specifying disabled = 0 in each stanza.
7. Save the file and close it.
8. Restart your universal forwarder.
9. On the receiving indexer, use the Search and Reporting app to confirm that you see incoming data from the host that you configured.

Informational Metrics

Metric	Splunk Add-on for Unix and Linux
CPU Utilization % (all and per CPU core / VCPU)	[script://./bin/cpu.sh]
CPU Utilization	[script://./bin/cpu.sh]
CPU Utilization Interrupts	[script://./bin/vmstat.sh]
CPU Utilization System Threads	[script://./bin/vmstat.sh]
Disk Space Available %	[script://./bin/df.sh]
Disk Total IOPS	[script://./bin/iostat.sh]
Disk Read IOPS	[script://./bin/iostat.sh]
Disk Write IOPS	[script://./bin/iostat.sh]
Memory Available %	[script://./bin/vmstat.sh]
Memory Available - MB	[script://./bin/vmstat.sh]
Memory Operations: Paging	[script://./bin/vmstat.sh]
Memory Used: MB System	[script://./bin/vmstat.sh]
Network Utilization (Bytes total / sec)	[script://./bin/bandwidth.sh]
Processor Queuing / Load Average	[script://./bin/vmstat.sh]
Total Network Packets/Second	[script://./bin/bandwidth.sh]

Sample configuration file for use with the Splunk Add-on for Unix and Linux

The following sample configuration file collects the data and metrics needed to generate the KPIs for the Operating System Module. Copy and paste them into an inputs.conf file within the Splunk Add-on for Unix and Linux (Splunk_TA_Nix) on the host from which to collect data.

By default, the indexes are commented out. Uncomment the index line for each stanza and set it to the index you want to use.

[script://./bin/vmstat.sh]
interval = 60
sourcetype = vmstat
source = vmstat
# index = os
disabled = 0

[script://./bin/iostat.sh]
interval = 60
sourcetype = iostat
source = iostat
# index = os
disabled = 0

[script://./bin/nfsiostat.sh]
interval = 60
sourcetype = nfsiostat
source = nfsiostat
# index = os
disabled = 0

[script://./bin/ps.sh]
interval = 30
sourcetype = ps
source = ps
# index = os
disabled = 0

[script://./bin/bandwidth.sh]
interval = 60
sourcetype = bandwidth
source = bandwidth
# index = os
disabled = 0

[script://./bin/df.sh]
interval = 300
sourcetype = df
source = df
# index = os
disabled = 0

[script://./bin/cpu.sh]
sourcetype = cpu
source = cpu
interval = 30
# index = os
disabled = 0

[script://./bin/hardware.sh]
sourcetype = hardware
source = hardware
interval = 36000
# index = os
disabled = 0

[script://./bin/version.sh]
disabled = false
# index = os
interval = 86400
source = Unix:Version
sourcetype = Unix:Version

Configure the Splunk Add-on for Microsoft Windows to collect data and send to your Splunk deployment

1. Download the Splunk Add-on for Windows from Splunkbase.
2. From a PowerShell prompt, place the Splunk Add-on for Windows in the %SPLUNK_HOME%\etc\apps directory.
3. Create an inputs.conf file in your %SPLUNK_HOME%\etc\apps\Splunk_TA_windows\local\ directory.
4. Refer to the Informational Metrics table below. This table represents the inputs that you must enable for the add-on to provide KPIs to ITSI.
5. Using a text editor, edit %SPLUNK_HOME%\etc\apps\Splunk_TA_windows\local\inputs.conf to include the stanzas that the table references.
6. Enable the stanza by entering disabled = 0 in each stanza.
7. Save the file and close it.
8. Restart your universal forwarder.
9. On the receiving indexer, use the Search and Reporting app to confirm that you see incoming data from the host that you configured.

Informational Metrics

Metric	Splunk Add-on for Microsoft Windows
CPU Utilization % (all and per CPU core / VCPU)	[perfmon://CPU]
CPU Utilization	[perfmon://CPU]
CPU Utilization Interrupts	[perfmon://CPU]
CPU Utilization System Threads	[perfmon://System]
Disk Space Available %	[perfmon://LogicalDisk]
Disk Total IOPS	[perfmon://LogicalDisk]
Disk Read IOPS	[perfmon://LogicalDisk]
Disk Write IOPS	[perfmon://LogicalDisk]
Memory Available %	[WinHostMon://OperatingSystem]
Memory Operations: Paging	[perfmon://Memory]
Network Utilization (Bytes total / sec)	[perfmon://Network]
Processor Queuing / Load Average	[perfmon://System]
Total Network Packets/Second	[perfmon://Network]

Sample configuration file for use with the Splunk Add-on for Microsoft Windows

The following sample configuration file collects the data and metrics needed to generate the KPIs needed for the Operating System Module. Copy and paste them into an inputs.conf file within the Splunk Add-on for Microsoft Windows (Splunk_TA_Windows) on the host from which to collect data.

By default, the indexes are commented out. Uncomment the index line for each stanza and set it to the index you want to use.

[WinHostMon://Processor]
interval = 600
disabled = 0
type = Processor
# index = windows

[WinHostMon://OperatingSystem]
interval = 600
disabled = 0
type = OperatingSystem
# index = windows

[WinHostMon://Disk]
interval = 600
disabled = 0
type = Disk
# index = windows

###### Splunk 5.0+ Performance Counters ######
## CPU
[perfmon://CPU]
counters = % Processor Time; % User Time; % Privileged Time; Interrupts/sec; % DPC Time; % Interrupt Time; DPCs Queued/sec; DPC Rate; % Idle Time; % C1 Time; % C2 Time; % C3 Time; C1 Transitions/sec; C2 Transitions/sec; C3 Transitions/sec
disabled = 0
instances = *
interval = 60
object = Processor
useEnglishOnly=true
# index = perfmon

## Logical Disk
[perfmon://LogicalDisk]
counters = % Free Space; Free Megabytes; Current Disk Queue Length; % Disk Time; Avg. Disk Queue Length; % Disk Read Time; Avg. Disk Read Queue Length; % Disk Write Time; Avg. Disk Write Queue Length; Avg. Disk sec/Transfer; Avg. Disk sec/Read; Avg. Disk sec/Write; Disk Transfers/sec; Disk Reads/sec; Disk Writes/sec; Disk Bytes/sec; Disk Read Bytes/sec; Disk Write Bytes/sec; Avg. Disk Bytes/Transfer; Avg. Disk Bytes/Read; Avg. Disk Bytes/Write; % Idle Time; Split IO/Sec
disabled = 0
instances = *
interval = 60
object = LogicalDisk
useEnglishOnly=true
# index = perfmon

## Physical Disk
[perfmon://PhysicalDisk]
counters = Current Disk Queue Length; % Disk Time; Avg. Disk Queue Length; % Disk Read Time; Avg. Disk Read Queue Length; % Disk Write Time; Avg. Disk Write Queue Length; Avg. Disk sec/Transfer; Avg. Disk sec/Read; Avg. Disk sec/Write; Disk Transfers/sec; Disk Reads/sec; Disk Writes/sec; Disk Bytes/sec; Disk Read Bytes/sec; Disk Write Bytes/sec; Avg. Disk Bytes/Transfer; Avg. Disk Bytes/Read; Avg. Disk Bytes/Write; % Idle Time; Split IO/Sec
disabled = 0
instances = *
interval = 60
object = PhysicalDisk
useEnglishOnly=true
# index = perfmon

## Memory
[perfmon://Memory]
counters = Page Faults/sec; Available Bytes; Committed Bytes; Commit Limit; Write Copies/sec; Transition Faults/sec; Cache Faults/sec; Demand Zero Faults/sec; Pages/sec; Pages Input/sec; Page Reads/sec; Pages Output/sec; Pool Paged Bytes; Pool Nonpaged Bytes; Page Writes/sec; Pool Paged Allocs; Pool Nonpaged Allocs; Free System Page Table Entries; Cache Bytes; Cache Bytes Peak; Pool Paged Resident Bytes; System Code Total Bytes; System Code Resident Bytes; System Driver Total Bytes; System Driver Resident Bytes; System Cache Resident Bytes; % Committed Bytes In Use; Available KBytes; Available MBytes; Transition Pages RePurposed/sec; Free & Zero Page List Bytes; Modified Page List Bytes; Standby Cache Reserve Bytes; Standby Cache Normal Priority Bytes; Standby Cache Core Bytes; Long-Term Average Standby Cache Lifetime (s)
disabled = 0
interval = 60
object = Memory
useEnglishOnly=true
# index = perfmon

## Network
[perfmon://Network]
counters = Bytes Total/sec; Packets/sec; Packets Received/sec; Packets Sent/sec; Current Bandwidth; Bytes Received/sec; Packets Received Unicast/sec; Packets Received Non-Unicast/sec; Packets Received Discarded; Packets Received Errors; Packets Received Unknown; Bytes Sent/sec; Packets Sent Unicast/sec; Packets Sent Non-Unicast/sec; Packets Outbound Discarded; Packets Outbound Errors; Output Queue Length; Offloaded Connections; TCP Active RSC Connections; TCP RSC Coalesced Packets/sec; TCP RSC Exceptions/sec; TCP RSC Average Packet Size  
disabled = 0
instances = *
interval = 60
object = Network Interface
useEnglishOnly=true
# index = perfmon

## Process
[perfmon://Process]
counters = % Processor Time; % User Time; % Privileged Time; Virtual Bytes Peak; Virtual Bytes; Page Faults/sec; Working Set Peak; Working Set; Page File Bytes Peak; Page File Bytes; Private Bytes; Thread Count; Priority Base; Elapsed Time; ID Process; Creating Process ID; Pool Paged Bytes; Pool Nonpaged Bytes; Handle Count; IO Read Operations/sec; IO Write Operations/sec; IO Data Operations/sec; IO Other Operations/sec; IO Read Bytes/sec; IO Write Bytes/sec; IO Data Bytes/sec; IO Other Bytes/sec; Working Set - Private
disabled = 0
instances = *
interval = 60
object = Process
useEnglishOnly=true
useWinApiProcStats = 1
# index = perfmon

## System
[perfmon://System]
counters = File Read Operations/sec; File Write Operations/sec; File Control Operations/sec; File Read Bytes/sec; File Write Bytes/sec; File Control Bytes/sec; Context Switches/sec; System Calls/sec; File Data Operations/sec; System Up Time; Processor Queue Length; Processes; Threads; Alignment Fixups/sec; Exception Dispatches/sec; Floating Emulations/sec; % Registry Quota In Use
disabled = 0
instances = *
interval = 60
object = System
useEnglishOnly=true
# index = perfmon

Change the macro definition for indexes

If you're not collecting data in the default indexes given by any of the Splunk add-ons (Windows or Unix/Linux), you need to change the macro definition by adding the default indexes that you're using for data collection.

Steps

1. From Splunk Web, click Settings > Advanced Search > Search Macros.
2. In the filter bar, search for itsi_os_module_indexes.
3. Select the itsi_os_module_indexes macro.
4. In the Definition, add all of the indexes that you're using for data collection from add-ons combined with OR operators.
   For example:

   (index=windows OR index=perfmon OR index=os OR index=<index-name>)
   

If you're performing a fresh installation of the latest Splunk Add-on for Windows version 5.0.0 or the Splunk Add-on for Unix and Linux version 6.0.0 or higher, you need to add the default main index.