Operating System Module troubleshooting

This topic is intended as a first step in diagnosing your Operating System (OS) Module problem yourself.

Entity information not populating

If entity information is not populating in the OS Module dashboard for a Linux or Unix entity, try changing the input interval for the following two scripts in inputs.conf for the Unix and Linux add-on on the forwarder where the add-on is configured.

Location: $SPLUNK_HOME$/etc/apps/Splunk_TA_nix/local/inputs.conf

[script://./bin/hardware.sh]
interval = 18000

[script://./bin/version.sh]
interval = 18000

For the full configuration file, see the Sample configuration file for use with the Splunk Add-on for Unix and Linux.

If you are still unable to locate operating system entities, see the ITSI module troubleshooting section on entities.

Metrics not collected

If you are unable to view your data, run the data model audit to make sure your data models are processing your data.

If you are running into issues seeing your KPI data, see ITSI module troubleshooting section on KPIs.

Troubleshooting hosts

If you run into a host related issue, make sure you:

• Install a universal forwarder on all hosts that you want to send KPIs and EAs to the ITSI application. You must then
• Install and configure either the Splunk Add-on for Windows or Splunk Add-on for Unix and Linux, depending on the operating system that runs on the host. Finally, you must then
• Configure the add-on by enabling the data inputs shown above depending on the type of host.
• Install the necessary modules on *nix or windows hosts. For example, the Splunk Add-on for Unix and Linux might work properly only if the sar and mpstat modules have been installed on your *nix host.

Before performing an installation, see Install the universal forwarder software in the Splunk Forwarder Manual to learn how to install and configure universal forwarders.

Install and configure a universal forwarder on a Windows host

To configure a Windows host for delivery of performance metrics used by KPIs and entity attributes:

1. Download the universal forwarder onto the Windows host for which you want metrics.
2. Install the universal forwarder, either by using the installation GUI or a PowerShell prompt.
3. During the installation, specify the receiving indexer that the forwarder should send data to. Normally, this is the same instance that hosts the ITSI application, but can differ based on your deployment strategy.
4. Confirm that the install completes and that the SplunkForwarder service starts.

Install and configure the Splunk Add-on for Windows

Navigate to the Configure the Splunk Add-on for Microsoft Windows to collect data and send to your Splunk deployment section of the Operating System Module configurations section of this manual.

Install and configure a universal forwarder on a *nix host

1. Download the universal forwarder on the *nix host for which you want metrics.
2. From a shell, install the universal forwarder by unpacking the installation archive.
3. Start the universal forwarder.
4. Specify the receiving indexer that the forwarder should send data to by using the 'splunk add forward-server <host>:<port>' command.

Install and configure the Splunk Add-on for Unix and Linux

Navigate to the configure the Splunk Add-on for Unix and Linux to collect data and send to your Splunk deployment section of the Operating System Module configurations section of this manual.

Instructions for multiple forwarders

If you have many hosts that you want metrics for ITSI, consider using a deployment server to deliver the apps and configurations to all of your universal forwarders.

1. Download the Splunk software and the Splunk add-on for your host type (Splunk Add-on for Windows for Windows hosts, Splunk Add-on for Unix and Linux for *nix hosts.)
2. Install a full Splunk Enterprise instance.
3. If you want to deploy to Windows hosts, copy the Splunk Add-on for Windows into $SPLUNK_HOME/etc/deployment_apps on the instance you installed.
4. If you want to deploy to *nix hosts, copy the Splunk Add-on for Unix and Linux into $SPLUNK_HOME/etc/deployment_apps on the same instance.
5. Refer to the "Required KPI" table above. This table represents the minimum number of inputs that you must enable for the add-ons to send the KPI data.
6. Edit inputs.conf within each of the add-ons to enable the stanzas that the table references.
7. (Optional) Refer to the "Informational KPIs" table and enable the stanzas that the table references for each of those KPIs.
8. Save and close the inputs.conf files.
9. Restart the Splunk Enterprise instance. At this point, it becomes a deployment server.
10. (Optional) On the instance, define server classes that differentiate Windows hosts from *nix hosts.
11. (Optional) Assign the Splunk Add-on for Windows to the Windows host server class and assign the Splunk Add-on for Unix and Linux to the *nix host server class.
12. Install universal forwarders on your hosts. On Windows hosts, specify the deployment server during the installation process.
13. On *nix hosts, specify the deployment server after the forwarder has been installed.

Troubleshooting permissions

If a user encounters a permission-related obstacle, the issue could be related to their assigned role. ITSI permissions are determined by the role that each user has. Each role offers a different set of permissions.

ITSI access is broken down by the following roles:

User

Can use ITSI to view services, glass tables, and deep dives. Can create private glass tables or deep dives.

Analyst

User permissions, plus can own notable events.

Admin

Analyst permissions, plus can administer the entire ITSI system.

When a user clicks on a service from the module visualization page, the user's role determines what the user can do and view. By default, admin and analyst roles allow you to create, edit and delete services. Admin level access permissions are required to access the service configuration page.

Learn about adding navigation to a Splunk app.

Role value not importing on saved search or manually triggered entity search

If your entities are able to utilize the KPIs but the role field is either blank or was not being assigned to the entities, follow the steps below to make the role field visible.

1. SSH into search head with ITSI.
2. Navigate to $SPLUNK_HOME/etc/apps/DA-ITSI-OS/.
3. Check your /local folder to see if inputs.conf exists. Remove from /local/ if found.
4. Navigate to /DA-ITSI-OS/default/.
5. Open inputs.conf and navigate to the [itsi_csv_import://DA-ITSI-OS-OS_Hosts_Import] stanza.
6. Under the [itsi_csv_import://DA-ITSI-OS-OS_Hosts_Import] stanza, remove dest from the values in "entity_identifier_fields = host,dest".
7. Save and restart your search head.
8. Navigate to ITSI > Configure > Entities > Create New Entity > Import from Search.
9. Select Modules.
10. Select ITSI Module for Operating Systems, and OS Hosts Search.
11. Click the search icon at the end of the search string input if the search does not begin automatically.

Forecast tab panel is not populating

The Forecast tab of your ITSI Operating System Module deployment displays the following notification.

command="predict", Too few data points: 1. Need at least 2

In this case, the Forecast tab does not have enough data points to complete the predict command. You can get enough data by extending the time range window. The larger the time window, the more accurate the forecasting model.

Sample configuration files

See the following links for sample configuration files that collect the appropriate data and metrics to generate the KPIs needed for the Operating System Module. Copy and paste them into an inputs.conf file within the appropriate add-on on the host that you want to collect data from.

• Sample configuration file for use with the Splunk Add-on for Unix and Linux (Splunk_TA_Nix).
• Sample configuration file for use with the Splunk Add-on for Microsoft Windows (Splunk_TA_Windows).