Operating System Module troubleshooting
This topic is intended as a first step in diagnosing your Operating System (OS) Module problem yourself.
Entity information not populating
If entity information is not populating in the OS Module dashboard for a Linux or Unix entity, try changing the input interval for the following two scripts in
inputs.conf for the Unix and Linux add-on on the forwarder where the add-on is configured.
[script://./bin/hardware.sh] interval = 18000 [script://./bin/version.sh] interval = 18000
For the full configuration file, see the Sample configuration file for use with the Splunk Add-on for Unix and Linux.
If you are still unable to locate operating system entities, see the ITSI module troubleshooting section on entities.
Metrics not collected
If you are unable to view your data, run the data model audit to make sure your data models are processing your data.
If you are running into issues seeing your KPI data, see ITSI module troubleshooting section on KPIs.
If you run into a host related issue, make sure you:
- Install a universal forwarder on all hosts that you want to send KPIs and EAs to the ITSI application. You must then
- Install and configure either the Splunk Add-on for Windows or Splunk Add-on for Unix and Linux, depending on the operating system that runs on the host. Finally, you must then
- Configure the add-on by enabling the data inputs shown above depending on the type of host.
- Install the necessary modules on *nix or windows hosts. For example, the Splunk Add-on for Unix and Linux might work properly only if the
mpstatmodules have been installed on your *nix host.
Before performing an installation, see Install the universal forwarder software in the Splunk Forwarder Manual to learn how to install and configure universal forwarders.
Install and configure a universal forwarder on a Windows host
To configure a Windows host for delivery of performance metrics used by KPIs and entity attributes:
- Download the universal forwarder onto the Windows host for which you want metrics.
- Install the universal forwarder, either by using the installation GUI or a PowerShell prompt.
- During the installation, specify the receiving indexer that the forwarder should send data to. Normally, this is the same instance that hosts the ITSI application, but can differ based on your deployment strategy.
- Confirm that the install completes and that the SplunkForwarder service starts.
Install and configure the Splunk Add-on for Windows
Navigate to the Configure the Splunk Add-on for Microsoft Windows to collect data and send to your Splunk deployment section of the Operating System Module configurations section of this manual.
Install and configure a universal forwarder on a *nix host
- Download the universal forwarder on the *nix host for which you want metrics.
- From a shell, install the universal forwarder by unpacking the installation archive.
- Start the universal forwarder.
- Specify the receiving indexer that the forwarder should send data to by using the 'splunk add forward-server <host>:<port>' command.
Install and configure the Splunk Add-on for Unix and Linux
Navigate to the configure the Splunk Add-on for Unix and Linux to collect data and send to your Splunk deployment section of the Operating System Module configurations section of this manual.
Instructions for multiple forwarders
If you have many hosts that you want metrics for ITSI, consider using a deployment server to deliver the apps and configurations to all of your universal forwarders.
- Download the Splunk software and the Splunk add-on for your host type (Splunk Add-on for Windows for Windows hosts, Splunk Add-on for Unix and Linux for *nix hosts.)
- Install a full Splunk Enterprise instance.
- If you want to deploy to Windows hosts, copy the Splunk Add-on for Windows into $SPLUNK_HOME/etc/deployment_apps on the instance you installed.
- If you want to deploy to *nix hosts, copy the Splunk Add-on for Unix and Linux into $SPLUNK_HOME/etc/deployment_apps on the same instance.
- Refer to the "Required KPI" table above. This table represents the minimum number of inputs that you must enable for the add-ons to send the KPI data.
- Edit inputs.conf within each of the add-ons to enable the stanzas that the table references.
- (Optional) Refer to the "Informational KPIs" table and enable the stanzas that the table references for each of those KPIs.
- Save and close the inputs.conf files.
- Restart the Splunk Enterprise instance. At this point, it becomes a deployment server.
- (Optional) On the instance, define server classes that differentiate Windows hosts from *nix hosts.
- (Optional) Assign the Splunk Add-on for Windows to the Windows host server class and assign the Splunk Add-on for Unix and Linux to the *nix host server class.
- Install universal forwarders on your hosts. On Windows hosts, specify the deployment server during the installation process.
- On *nix hosts, specify the deployment server after the forwarder has been installed.
If a user encounters a permission-related obstacle, the issue could be related to their assigned role. ITSI permissions are determined by the role that each user has. Each role offers a different set of permissions.
ITSI access is broken down by the following roles:
- Can use ITSI to view services, glass tables, and deep dives. Can create private glass tables or deep dives.
- User permissions, plus can own notable events.
- Analyst permissions, plus can administer the entire ITSI system.
When a user clicks on a service from the module visualization page, the user's role determines what the user can do and view. By default, admin and analyst roles allow you to create, edit and delete services. Admin level access permissions are required to access the service configuration page.
To locate an existing user or role in Splunk Web, click Settings > Access Controls. Select Users or Roles and use the filter bar to search for an existing user or role.
Learn about adding navigation to a Splunk app.
Role value not importing on saved search or manually triggered entity search
If your entities are able to utilize the KPIs but the role field is either blank or was not being assigned to the entities, follow the steps below to make the role field visible.
- SSH into search head with ITSI
- Navigate to $SPLUNK_HOME/etc/apps/DA-ITSI-OS/
- Check your
/localfolder to see if inputs.conf exists. Remove from
- Navigate to
- Open inputs.conf and navigate to the
- Under the
destfrom the values in
"entity_identifier_fields = host,dest"
- Save and restart your search head.
- Navigate to ITSI > Configure > Entities > Create New Entity > Import from Search.
- Select Modules.
- Select ITSI Module for Operating Systems, and OS Hosts Search
- Click the search icon at the end of the search string input if the search does not begin automatically.
If this does not produce desired results, install the most recent version of Splunk IT Service Intelligence on your Splunk platform deployment.
Forecast tab panel is not populating
The Forecast tab of your ITSI Operating System Module deployment displays the following notification.
command="predict", Too few data points: 1. Need at least 2
In this case, the Forecast tab does not have enough data points to complete the predict command. You can get enough data by extending the time range window. The larger the time window, the more accurate the forecasting model.
Sample configuration files
See the following links for sample configuration files that collect the appropriate data and metrics to generate the KPIs needed for the Operating System Module. Copy and paste them into an
inputs.conf file within the appropriate add-on on the host that you want to collect data from.
Operating System Module data model reference table
About the ITSI Module for Storage Array Monitoring
This documentation applies to the following versions of Splunk® IT Service Intelligence: 3.0.0, 3.0.1, 3.0.2, 3.1.0, 3.1.1, 3.1.2, 3.1.3, 3.1.4, 4.0.0, 4.0.1, 4.0.2, 4.0.3, 4.0.4, 4.1.0, 4.1.1, 4.1.2, 4.1.5, 4.2.0, 4.2.1, 4.2.2, 4.2.3, 4.3.0, 4.3.1, 4.4.0, 4.4.1