Considerations for deciding how to monitor remote Windows data

Considerations for deciding how to monitor remote Windows data

Splunk is capable of powerful indexing, searching and reporting of all of your Windows data. However, as with any software, getting the best out of Splunk requires careful planning and execution. This is particularly important when configuring and deploying it in your enterprise for remote data collection - the goal is to get the data you need, using the least amount of resource overhead and minimum permissions required. This topic helps you decide how and where to deploy Splunk for maximum efficiency when indexing data remotely from many Windows hosts.

Authentication

Windows requires authentication for just about every kind of remote activity it performs. While this is well understood by many system administrators, it's a particularly important concept to grasp when using Splunk in a Windows environment. Failure to understand how Splunk accesses Windows data can lead to suboptimal search results, or no results at all. This section attempts to explain the various security-related factors surrounding getting data into Splunk on Windows.

When you install Splunk, you have the choice of installing it as the "Local System" user, or as another user. More specific information about the installation ramifications can be found in the Installation Manual, but there are consequences for getting data in as well.

The user you tell Splunk to run as has a direct impact on the data returned by a search against remote machines. In order to get the data you want, you'll need to give this user the permissions required to access remote data sources.

The easiest way to do this is to make the user that Splunk runs as a member of the Administrators (or Domain Admins) groups. While this effectively resolves many permissions-related problems, you raise a number of security concerns and, depending on how your security topology is arranged, you may not even have permission to do this.

In most cases, you'll need to configure the Splunk user account to have the minimum level of permissions required to access the data sources. This entails a combination of adding the user to various domain security groups, and making changes to the access control lists of various Active Directory objects, depending on the data sources you need to access. While this method is more time-consuming than granting Domain Admininstrators access, it is much more secure, and provides for finer control of access.

If your Active Directory domain security policy is configured to enforce password changes regularly, you'll also need to make sure that either the Splunk account's password is set to never expire, or that you manually change the password before it expires as defined by the policy. Once you've changed the password; you'll need to restart any Splunk processes that run as that account.

It's also a good idea to assign the Splunk account the "Deny log on locally" user rights assignment in Local Security Policy to prevent the user from logging in interactively.

Individual pages dealing with remote access to Windows machines will have additional information and recommendations on how to configure the user Splunk runs as for least-permissive security. Review the "Security and remote access considerations" topic on those pages for specifics.

I/O Bandwidth

I/O bandwidth is more important when performing Splunk indexing on local machines, but is something that should be considered for remote data collection - and on forwarders - as well.

Disk bandwidth is always a concern, primarily when data is transformed on the machine running the forwarder. Anti-virus scanner drivers and drivers that intermediate between Splunk and the operating system should always be configured to ignore the Splunk directory and processes.

Network bandwidth usage should be monitored closely, especially in networks with thin WAN links.

Forwarders versus remote collection through WMI

The easiest way to get data off of a Windows host is with a local Splunk forwarder. Using a local forwarder offers the most types of data sources, minimizes network overhead and reduces operational risk and complexity, particularly where authentication is concerned.

However, there are circumstances – from organizational boundaries to local performance considerations – where remote collection is preferred, or even required. For these situations, Splunk supports using the native Windows Management Instrumentation (WMI) interface to collect event logs and performance data.

This table offers a list of data sources and their respective trade-offs for you to consider.

Data Source Considerations

Data Source	Local Forwarder	Remote Polling
Event logs	Yes	Yes*
Performance	Yes	Yes
Registry	Yes	No
Log files	Yes	Yes**
Crawl	Yes	No

* For remote event log collection, you must know the name of the Event Log you wish to collect. You are also subject to security considerations, particularly when indexing Security event logs. On local forwarders, you have the option to collect all logs, regardless of name.

** Remote log file collection using the "\\SERVERNAME\SHARE" syntax is supported, however you must use CIFS (Common Internet File System, or Server Message Block) as your application layer file access protocol and Splunk must have at least read access to both the share and the underlying file system.

Tradeoffs

Performance

For identical collecting of local Event Logs and flat log files, a local forwarder requires less CPU and performs basic pre-compression of the data in an effort to reduce network overhead. It is more memory intensive, however, mostly owing to the additional data source input options available.

If the forwarder is configured to run as the Local System user, the authentication requirements for the local machine are eliminated, as that account has full access to the local machine.

WMI remote polling is more CPU intensive on the target machine for the same set of data (either remote Event Logs or remote performance data), and is more network intensive overall. It also requires that Splunk runs as a user with explicit access to these data sources.

On highly audited hosts, such as domain controllers and operations masters, remote polling may not be able to keep up with the amount of data that is generated, particularly if you are monitoring Active Directory-related event logs or schema changes. This is because remote polling is best-effort by design of the WMI API, and is additionally throttled by Splunk to prevent unintentional denial of service attacks.

Deployment

Local forwarders are easier to deploy where you have control of the base OS build, and/or if you have many data sources, especially if transformation of data is required.

Remote polling works well when you want a limited set of data from a large number of hosts (for example, just process CPU data for usage billing). Remote polling may be your only option where you don’t have either build control or local administrator privileges on the target host.

A common usage scenario is to first test using remote polling, then add successful or useful polls to your local forwarding configuration later, or at mass deployment time.

Management

Both mechanisms offer logging and, potentially, alerting to let you know if a host is coming on or offline or is no longer connected. However, to prevent an unintentional denial of service attack, the WMI polling service in Splunk will start to poll less frequently over time if it is unable to contact a host for a period of time, and will eventually stop polling unreachable hosts altogether. Therefore remote polling is not advised for machines that are frequently offline, such as laptops or dynamically provisioned virtual machines.

Search Windows data on a non-Windows Splunk

You can index and search your Windows data on a non-Windows instance of Splunk, but you must first use a Windows instance of Splunk to gather the Windows data. You can easily do this by means of a Splunk forwarder running on Windows, configured to gather Windows inputs and then forward the data to the non-Windows instance of Splunk where searching and indexing will take place.

There are two main ways to proceed:

• Set up forwarders locally on each Windows machine that's generating data. These forwarders can send the Windows data to the non-Windows receiving instance of Splunk.

• Set up a forwarder on a separate Windows machine. The forwarder can perform remote polling on all the Windows machines in the environment and then forward the combined data to a non-Windows receiving instance of Splunk.

You must specially configure the non-Windows Splunk to handle the Windows data. For details, see "Searching data received from a forwarder running on a different operating system".

For information on setting up forwarders, see "Set up forwarding and receiving".

• Was this topic useful?
• Post a comment