Monitor data through Windows Management Instrumentation (WMI)

The Splunk platform supports the use of Windows Management Instrumentation (WMI) providers for access to Windows performance and event log data on remote Windows machines without the need to install software on those machines.

To get WMI data into Splunk Cloud Platform, you can install a universal or heavy forwarder on a Windows machine and configure that forwarder to use the WMI data input to collect data remotely from other Windows machines, then forward that data to your Splunk Cloud Platform instance. Splunk Cloud Platform cannot connect directly to a Windows machine using WMI, so a universal or heavy forwarder is the only option.

WMI data inputs can connect to multiple WMI providers. The input runs on the forwarder as a separate process called splunk-wmi.exe. It is a scripted input.

If possible, when you need to collect Windows data remotely, install a universal forwarder directly onto the machine where you want to collect the Windows data, rather than use WMI. The resource load of WMI can exceed that of installing a universal forwarder in many cases. Use a forwarder if you collect multiple event logs or performance counters from each machine, or from very busy machines like domain controllers. See Considerations for deciding how to monitor remote Windows data.

What do you need to monitor WMI-based data?

Here are the minimum requirements to monitor WMI-based data. You might need additional permissions based on the event logs or performance counters you want to monitor.

For additional details on what you need to monitor WMI-based data, see Security and remote access considerations in this topic.

Activity	Required permissions
Monitor remote event logs over WMI	Splunk Cloud Platform must receive data from a forwarder that runs on Windows. Splunk Enterprise can collect WMI data directly if it runs on a Windows machine. The forwarder must run as a Windows domain user with at least read access to WMI. The forwarder must run as a domain user with appropriate access to the desired event logs. For example, a user who is a member of the Event Log Readers group has appropriate access.
Monitor remote performance monitor counters over WMI	Splunk Cloud Platform must receive data from a forwarder that runs on Windows. Splunk Enterprise can collect performance data over WMI if it runs on a Windows machine. The forwarder must run as a domain user with at least read access to WMI. The forwarder must run as a domain user with appropriate access to the Windows Performance Data Helper libraries.

If you run Splunk Enterprise, you can collect data over WMI if the machine that you install the instance on runs Windows, or the forwarders that you use to send data to the instance run Windows.

Security and remote access considerations

Both of the Splunk platform instances that get WMI data and your Windows network must be correctly configured for data access over WMI. Whether the instance that indexes your data is a Splunk Cloud Platform or a Splunk Enterprise instance, review the following prerequisites before attempting to use the platform to get data over WMI.

Before the Splunk platform can get WMI-based data:

• The instance that gets the data must be installed with a Windows user that has permissions to perform remote network connections.
• The Windows user that the instance runs as must be a member of an Active Directory (AD) domain or forest and must have appropriate privileges to query WMI providers.
• The instance user must also be a member of the local Administrators group on the machine that runs the instance.
• The machine that runs the instance must be able to connect to the remote machine and must have permissions to get the data from the remote machine after it has connected.
• Both the Splunk platform instance and the target machines must be part of the same AD domain or forest.

The user that the Splunk platform instance runs as does not need to be a member of the Domain Admins group, and for security reasons, should not be. However, you must have domain administrator privileges to configure access for the user. If you don't have domain administrator access, find someone who can either configure Splunk user access or give domain administrator rights to you.

If you install the instance as the Local System Windows user, remote authentication over WMI does not work. The Local System user has no access to other machines on the network. It is not possible to grant privileges to a Local System account for access to another machine.

You can give the instance user access to WMI providers by doing one of the following:

• Assigning least-permissive rights, as detailed in "Configure WMI for least permissive access" later in this topic. This is the recommended option.
• Adding the user to the local Administrators group on each Windows member machine you want to poll. For security reasons this is not recommended.
• Adding the user to the Domain Admins global group. For security reasons, this is not recommended.

Group memberships and resource access control lists (ACLs)

To maintain security integrity, place the Windows users that Splunk platform instances run as into a domain global group and assign permissions on Windows machines and resource ACLs to that group, rather than assigning permissions directly to the user. Assignment of permissions directly to users is a security risk, and can cause problems during security audits or future changes.

Configure WMI for least permissive access

If the Windows user that you configured the Splunk platform to run as is not a domain administrator, you must configure WMI to provide access to this user. Grant least-permissive access to all Windows resources, including WMI, by following this checklist. For additional information and step-by-step instructions, see Prepare your Windows network for a Splunk Enterprise installation in the Installation manual. These instructions are also valid for universal forwarders.

You must grant several levels of access to the user the Splunk platform runs as for the instance to collect data over WMI using the least-permissive method.

• Local Security Policy Permissions. The Splunk platform Windows user needs the following Local Security Policy user rights assignments defined on each machine you poll for WMI-based data:
  • Access this Computer from the Network
  • Act as part of the operating system
  • Log on as a batch job
  • Log on as a service
  • Profile System Performance
  • Replace a process level token

• Distributed Component Object Model (DCOM) configuration and permissions. You must enable DCOM on every Windows machine you want to monitor. In addition, the Splunk platform Windows user must have access to DCOM. There are many methods available to do this, but the best is to nest the Distributed COM Users domain global group into the Distributed COM Users local group on each Windows machine you want to monitor, then add the Splunk platform Windows user to the Distributed COM Users domain global group. Search the Microsoft documentation website for "Securing a Remote WMI Connection" for specific details and instructions.

• Performance Monitor configuration and permissions. The Splunk platform Windows user must be a member of the Performance Log Users local group for the Splunk platform to access remote performance objects over WMI. The best way to do this is to put the Performance Log Users domain global group inside of the Performance Log Users local group on each Windows machine, and then assign the Windows user to the Performance Log Users group.

• WMI namespace security. The WMI namespace that the Splunk platform accesses must have proper permissions. In nearly all cases, the Root\CIMV2 namespace is acceptable for collecting WMI data. You must set namespace permissions manually on each Windows machine in your enterprise, as there is no global security for WMI. You must assign these rights must be assigned to the Root namespace and all subnamespaces below it. Search the Microsoft documentation website for "Managing WMI security".Enable the following permissions on the WMI tree for each host at the Root namespace for the Splunk user:
  • Execute Methods
  • Enable Account
  • Remote Enable
  • Read Security

There is no standard policy to deploy WMI security settings remotely to multiple machines at once using Group Policy. However, there is a blog on the Microsoft documentation website that describes using a startup script to achieve this task. Search the Microsoft documentation website for "Set WMI namespace security by using GPO (script)".

• Firewall configuration. If you have an active firewall, you must configure it to allow access for WMI. If you use the Windows Firewall included with recent versions of Windows, an exceptions list explicitly includes WMI. You must set this exception for both the originating and the target Windows machines. Search the Microsoft documentation website for "Connecting to WMI Remotely".

• User access control (UAC) configuration. If you run Windows Vista and higher, or the Windows Server 2012 and higher families, UAC affects how Windows assigns permissions. Search the Microsoft documentation website for "User Account Control and WMI".

Test access to WMI providers

After you configure WMI and set up the Splunk platform instance user for access to your domain, test access to the remote machine.

This procedure includes steps to temporarily change the Splunk platform instance data store directory (the location SPLUNK_DB points to). You must do this before testing access to WMI. Failure to do so can result in missing WMI events. This is because the splunk-wmi.exe process updates the WMI checkpoint file every time it runs.

If you attempt to log into a domain controller, you might have to change your domain controller security policy to assign the Allow log on locally policy for the designated user.

1. Log into the Windows machine that the Splunk platform runs on as the Splunk platform Windows user.
2. Open a command prompt (click Start > Run and type cmd).
3. Go to the bin subdirectory under your Splunk platform installation (for example, cd c:\Program Files\Splunk\bin).
4. Determine where the platform instance currently stores its data by running the following command:
   > splunk show datastore-dir

   Remember where the platform instance stores its data. You will recall it later.

5. Run the following command to change where the Splunk platform stores its data temporarily:
   > splunk set datastore-dir %TEMP%

   This example sets the data store directory to the directory that the TEMP environment variable specifies. If you want to set it to a different directory, you can do so, but the directory must already exist.

6. Restart the Splunk platform instance:
   > splunk restart

   It might take a while for the Splunk platform instance to restart.

7. Once The Splunk platform instance has restarted, test access to WMI providers, replacing <host> with the name of the remote machine:
   > splunk cmd splunk-wmi -wql "select * from win32_service" -namespace \\<host>\root\cimv2
   • If you see data stream back and no error messages, then the instance was able to connect to the WMI provider and query successfully.
   • If there is an error, a message with a reason on what caused the error appears. Look for the error="<msg>" string in the output for clues on how to correct the problem.

After testing WMI access, point the Splunk platform instance back to the correct database directory by running the following command, and then restarting the instance:

> splunk set datastore-dir <directory shown from Step 4>

Configure WMI-based inputs

All remote data collection in the Splunk platform on Windows happens through either WMI providers or a forwarder.

If you are using Splunk Cloud Platform, you must use a forwarder to collect WMI data and send it to your Splunk Cloud Platform instance.

If you are using Splunk Enterprise, you can configure WMI-based inputs using either Splunk Web or a configuration file. More configuration options are available when using a file.

Configure WMI-based inputs with Splunk Web

Splunk Cloud Platform isn't able to collect WMI data natively, but you can use Splunk Web on a Splunk Enterprise instance that you have configured as a heavy forwarder to collect the data and send it to your Splunk Cloud Platform instance.

To add WMI-based inputs in Splunk Web, use the Remote event log monitoring and Remote Performance monitoring data inputs. See the following topics for instructions:

• Configure remote Windows performance monitoring with Splunk Web.
• Configure remote Windows event log monitoring

Configure WMI-based inputs with configuration files

Splunk Cloud Platform isn't able to collect WMI data natively, so if you do not want to use a heavy forwarder, you must use a universal forwarder and modify configuration files.

The wmi.conf configuration file handles remote data collection through WMI. Review this file to see the default values for WMI-based inputs. If you want to make changes to the default values, add the settings and values you want to change to a separate version of the file, in the %SPLUNK_HOME%\etc\system\local\ directory on the universal or heavy forwarder that collects the WMI data. Set values only for the settings you want to change for a given type of data input.

See the wmi.conf file in the Admin Manual to learn about the default settings and values for the file, and About configuration files for information on configuring files.

The wmi.conf file contains several stanzas:

• The [settings] stanza, which specifies global WMI settings.
• One or more input-specific stanzas, which define how to connect to WMI providers to get data from the remote Windows machine.

Global settings

The [settings] stanza specifies global WMI parameters. The entire stanza and every setting within it are optional. If the stanza is not present, the Splunk platform uses the default system values.

The following settings control how the Splunk platform reconnects to a given WMI provider when an error occurs.

Setting	Description	Default value
initial_backoff	How long, in seconds, to wait the first time after an error occurs before trying to reconnect to the WMI provider. If connection errors continue to occur, the Splunk platform doubles the wait time until it reaches the value specified in max_backoff.	5
max_backoff	How long, in seconds, to wait between connection attempts, before invoking max_retries_at_max_backoff.	20
max_retries_at_max_backoff	If the wait time between connection attempts reaches max_backoff, how many times to try to reconnect to the provider, every max_backoff seconds. If the Splunk platform continues to encounter errors, it gives up, and won't attempt to connect to the problem provider again until you restart. It will continue to log errors such as the example shown above.	2
checkpoint_sync_interval	How long, in seconds, to wait for state data (event log checkpoint) to be written to disk.	2

Input-specific settings

Input-specific stanzas configure the Splunk platform to connect with WMI providers. They are defined by one of two settings that specify the type of data the Splunk platform should gather. The stanza name can be anything, but usually begins with WMI:, for example:

[WMI:AppAndSys]

When you configure WMI-based inputs in Splunk Web, Splunk uses this naming convention for input-specific stanza headers.

You can specify one of two types of data inputs in an input-specific stanza:

Data input	Description
Event log	The event_log_file setting configures the Splunk platform to expect event log data from the sources that the stanza defines.
Windows Query Language (WQL)	The wql setting configures the Splunk platform to expect data from a WMI provider. You must also specify a valid WQL statement. You must use this setting when you collect performance data.

Do not define both of these settings in one stanza. Use only one or the other. Otherwise, the input defined by the stanza will not run.

The common settings for both types of inputs are:

Setting	Description	Default value
server	A comma-separated list of Windows machines from which to get data. If this setting is missing, the Splunk platform attempts to connect to the local machine.	The local host
interval	How often, in seconds, to poll the WMI provider for new data. If this setting is not present and defined, the input that the stanza defines will not run.	Not applicable
disabled	Whether this input is enabled or disabled. Set this setting to 1 to disable the input, and 0 to enable it.	0 (enabled)

The event log-specific parameters are:

Setting	Description	Default value
event_log_file	A comma-separated list of event log channels to monitor.	Not applicable
current_only	Whether or not to collect events that occur only when the Splunk platform runs. If events are generated when the Splunk platform is stopped, it will not attempt to index those events when it is started again. Set to 1 to collect events that occur only when it is running, and 0 to collect all events.	0 (gather all events)
disable_hostname_normalization	Do not normalize the host name that is retrieved from a WMI event. By default, the Splunk platform normalizes host names by producing a single name for the host through identifying various equivalent host names for the local system. Set this parameter to 1 to disable host name normalization in events, and 0 to normalize host names in events	0 (normalize host names for WMI events)

The WQL-specific parameters are:

Setting	Description	Default value
wql	A valid WQL statement.	Not applicable
namespace	(Optional) Specifies the path to the WMI provider. The local machine must be able to connect to the remote machine using delegated authentication. If you do not specify a path to a remote machine, the Splunk platform connects to the default local namespace (\Root\CIMV2). This default namespace is where most of the providers that you can to query reside.	\\<local server>\Root\CIMV2
current_only	Whether or not an event notification query is expected. See "WQL query types: event notification versus standard" in this topic for additional information. Set this setting to 1 to tell the Splunk platform to expect an event notification query, and 0 to expect a standard query.	0 (expect a standard query)

WQL query types: event notification versus standard

The current_only setting in WQL stanzas determines the type of query the stanza expects to use to collect WMI-based data. When you configure the setting to 1, the stanza expects event notification data. Event notification data is data that alerts you of an incoming event. To get event notification data, you must use an event notification query.

For example, to find out when a remote host spawns processes, you must use an event notification query. Standard queries have no facilities for notifying you when an event has occurred, and can only return results on information that already exists.

Conversely, if you want to know what already-running processes on your system begin with the word "splunk", you must use a standard query. Event notification queries cannot tell you about static, preexisting information.

Event notification queries require that the WQL statement defined for the stanza be structurally and syntactically correct. Improperly formatted WQL will cause the input defined by the stanza to not run. Review the wmi.conf configuration file reference for specific details and examples.

WQL query stanzas do not update the WMI checkpoint file

When you use a WQL query stanza to gather data through WMI, the Splunk platform does not update the WMI checkpoint file, which is the file that determines if WMI data has been indexed. A WQL query of any type returns dynamic data and a context for saving a checkpoint for the data produced cannot be built. So, the Splunk platform indexes WMI data that it collects through WQL query stanzas as fresh data each time the stanza runs. This process can result in the indexing of duplicate events and possibly impact license volume.

If you need to index data regularly, such as event logs, use the appropriate monitor on a universal forwarder. If you must use WMI, use a standard WMI query type.

Examples of wmi.conf

The following is an example of a wmi.conf file:

[settings]
initial_backoff = 5
max_backoff = 20
max_retries_at_max_backoff = 2
checkpoint_sync_interval = 2

[WMI:AppAndSys]
server = foo, bar
interval = 10
event_log_file = Application, System, Directory Service
disabled = 0

[WMI:LocalSplunkWmiProcess]
interval = 5
wql = select * from Win32_PerfFormattedData_PerfProc_Process where Name = "splunk-wmi"
disabled = 0

# Listen from three event log channels, capturing log events that occur only
# while the Splunk platform runs.  Gather data from three machines.
[WMI:TailApplicationLogs]
interval = 10
event_log_file = Application, Security, System
server = srv1, srv2, srv3
disabled = 0
current_only = 1

# Listen for process-creation events on a remote machine
[WMI:ProcessCreation]
interval = 1
server = remote-machine
wql = select * from __InstanceCreationEvent within 1 where TargetInstance isa 'Win32_Process'
disabled = 0
current_only = 1

# Receive events whenever someone plugs/unplugs a USB device to/from the computer
[WMI:USBChanges]
interval = 1
wql = select * from __InstanceOperationEvent within 1 where TargetInstance ISA 'Win32_PnPEntity' and TargetInstance.Description='USB Mass Storage Device'
disabled = 0
current_only = 1

Fields for WMI data

When the Splunk platform indexes data from WMI-based inputs, it sets the originating host from the data it receives. it sets the source for received events to wmi. It sets the source type of the incoming events based on the following conditions:

• For event log data, the Splunk platform sets the source type to WinEventLog:<name of log file>. For example, WinEventLog:Application.
• For WQL data, the Splunk platform sets the source type to the name of the stanza that defines the input. For example, for a stanza named [WMI:LocalSplunkdProcess], Splunk sets the source type to WMI:LocalSplunkdProcess.

WMI and event transformations

WMI events are not available for transformation at index time. You cannot modify or extract WMI events as the Splunk platform indexes them. This is because WMI events arrive as a single source (a scripted input), which means they can be matched only as a single source.

You can modify and extract WMI events at search time. You can also address WMI-based inputs at parse time by specifying the sourcetype [wmi].

For information on how to transform events as they arrive in the Splunk platform, see About indexed field extraction in this manual.

Troubleshoot WMI inputs

When the Splunk platform cannot connect to a defined WMI provider, it generates an error like the following:

05-12-2011 02:39:40.632 -0700 ERROR ExecProcessor - message from ""C:\Program Files\Splunk\bin\splunk-wmi.exe"" WMI - Unable to connect to WMI namespace "\\w2k3m1\root\cimv2" (attempt to connect took 42.06 seconds) (error="The RPC server is unavailable." HRESULT=800706BA)

When this occurs, try the following tips:

• Confirm that the machine that is polling the instance has network access to the machine.
• Confirm that you have configured the instance for least permissive access to the WMI provider and have completed all of the security requirements.
• Confirm that the Splunk platform Windows user has access to WMI

If you encounter problems receiving events through WMI providers or are not getting the results you expect, see Common Issues with Splunk and WMI in the Troubleshooting Manual.