Best practices guide

This topic discusses some best practices that we have put together in the course of developing and using the Splunk App for Windows Infrastructure. Expect continued updates to this guide as we update the app with feedback from our customers and partners.

General

Synchronize clocks on all servers

To ensure that the Splunk App for Windows Infrastructure sees all data coming in from the hosts in your Windows environment, make sure that those hosts have their clocks synchronized.

• On Windows servers, use the Windows Time service to synchronize with an available Network Time Protocol (NTP) server.
• On *nix servers (if you use *nix servers to host the Splunk App for Windows Infrastructure), use the ntpd client to synchronize with an available NTP server.

Note: The Windows Time service is not a full-fledged NTP client and Microsoft neither guarantees nor supports the accuracy of the service.

Active Directory

Below are some best practices for tuning Active Directory monitoring operations for the Splunk App for Windows Infrastructure. Note the following guidelines:

• You must make these changes inside the universal forwarders that you have installed on the AD domain controllers in your environment.
• You must deploy the appropriate TA-DomainController add-on for your Windows Server version.
  • For Windows Server 2003, use TA-DomainController-NT5.
  • For Windows Server 2008 and Server 2008 R2, use TA-DomainController-NT6.
  • For Windows Server 2012 R2 and later, use TA-DomainController-2012r2.
• You must define the changes within the TA-DomainController-<version> add-on that you install into the universal forwarder.
• If you use a Splunk Enterprise deployment server, create server classes which deploy the add-ons with these updated configurations. Otherwise, make these changes after you have deployed the add-ons into the universal forwarders on the domain controllers.
• Once you update configurations, you must restart the universal forwarders on each domain controller for the new changes to take effect.

Consider not including a baseline for Active Directory data collection

You don't need to collect a baseline - or dump - of your Active Directory schema to use with the Splunk App for Windows Infrastructure. In fact, doing so can significantly increase the memory usage footprint on your domain controllers. Unless you specifically need a baseline of your AD schema, consider turning it off.

To turn off baselining, edit %SPLUNK_HOME%\etc\apps\TA-DomainController-<version>\local\inputs.conf and modify the main admon stanza:

[admon://NearestDC]
disabled = 0
baseline = 0

Consider disabling the Active Directory monitoring input on all but a select group of domain controllers

When you collect Active Directory data for the Splunk App for Windows Infrastructure, it is not necessary to enable the Active Directory monitoring input (admon) on every domain controller in your Windows environment. If you have a number of domain controllers, select one (or two to three for redundancy) and enabling the admon inputs only on those hosts.

You should still install the TA-DomainController-<version> add-on into each domain controller. You should also install the Splunk Add-on for Windows (Splunk_TA_Windows) onto the host to get all other Windows data for the host into the Splunk App for Windows Infrastructure.

To configure active directory monitoring on a specific domain controller, edit %SPLUNK_HOME%\etc\apps\TA-DomainController-<version>\local\inputs.conf and disable the main admon stanza:

[admon://NearestDC]
disabled = 1

Next, create a new Active Directory monitoring stanza and set the targetDc attribute to the NetBIOS name of the controller on which you want to run admon. For example, if the server on which you want to run admon is named SF-DC2, configure a new admon stanza like the following:

[admon://ADMonitoring]
targetDc = SF-DC2
monitorSubtree = 1
baseline = 0
index = msad
disabled = 0

Consider specifying a domain controller for Security Event Log Security ID (SID) translations

The Splunk Enterprise event log monitor translates security identifiers (SIDs) by default for the Security Event Log. Translation turns SIDs (the very long string that begins with S-1-5-21 and ends with a long jumble of numbers) into friendly account names.

The Splunk App for Windows Infrastructure does not need SID translation in the Security Event Log. This is because Active Directory events already contain this information. To reduce the amount of memory that domain controllers use to perform SID translation, configure the Splunk Add-on for Windows (Splunk_TA_Windows) to disable SID translation.

To disable SID translation, edit %SPLUNK_HOME%\etc\apps\Splunk_TA_Windows\local\inputs.conf and modify the Security Event Log stanza:

[WinEventLog://Security]
evt_resolve_ad_obj = 0

If you require SID translation, you can limit both its scope and where it occurs by setting the current_only and evt_dc_name attributes:

[WinEventLog://Security]
evt_dc_name = SF-DC2 # only use SF-DC2 to translate SIDs
current_only = 1  # only translate SIDs for events that come in for new events

Consider limiting AD object access events to reduce impact on license usage

When you enable auditing on your AD domain controllers, the DCs create Security Event Code 4662 events each time a user access any kind of AD object. (On Windows Server 2003 and 2003 R2, the event code is 566). This can greatly impact license volume and potentially cause violations.

To address the problem, limit the indexing of these event codes by blacklisting some of the events which contain them (the app uses the events for Group Policy monitoring but no other purpose.)

Important: This procedure requires that you use Splunk universal forwarder version 6.2 or later. If you cannot use this version of the universal forwarder, then this strategy does not apply to you.

Follow these steps to limit the indexing of these event codes:

On new installations:

1. Follow the installation instructions to deploy the Splunk App for Windows Infrastructure. When deploying the universal forwarders, install version 6.2 or later.

2. Add a configuration stanza to inputs.conf in the Splunk Add-on for Windows (in %SPLUNK_HOME%\etc\apps\Splunk_TA_Windows\local) on each domain controller:

[WinEventLog://Security]
blacklist1=EventCode=4662 Message=”Object Type:\s+(?!groupPolicyContainer)”
### remove the comment from the following line if you run 
### Windows Server 2003 domain controllers.
# blacklist2=EventCode=566 Message=”Object Type:\s+(?!groupPolicyContainer)”

Note: A deployment server significantly reduces the amount of time required to update the universal forwarders on the DCs in your environment. Consider deploying one and configuring all of your universal forwarders as deployment clients. See "Set up a deployment server and create a server class" for additional information on how to configure deployment server.

3. Complete deployment by restarting the universal forwarders on all DCs.

4. On the indexer(s) that you configured the universal forwarders to send Security Event Log data to, log into Splunk Enteprise to confirm that the DCs only forward Security Event Code 4662/566 for groupPolicyContainer objects.

On existing installations:

1. Upgrade all universal forwarders in the deployment to version 6.2 or later.

Important: This best practice requires that all universal forwarders be version 6.1 or later. If you have already deployed forwarders earlier than version 6.2 to your DCs, then upgrade the forwarders to version 6.2 or later.

2. On each universal forwarder installed on a domain controller, add a configuration stanza to inputs.conf in the Splunk Add-on for Windows (in %SPLUNK_HOME%\etc\apps\Splunk_TA_Windows\local) on each domain controller:

[WinEventLog://Security]
blacklist1=EventCode=4662 Message=”Object Type:\s+(?!groupPolicyContainer)”
### remove the comment from the following line if you run 
### Windows Server 2003 domain controllers.
# blacklist2=EventCode=566 Message=”Object Type:\s+(?!groupPolicyContainer)”

Important: If you have not installed the Splunk Add-on for Windows on a DC universal forwarder, do so.

3. Restart the universal forwarders on the DCs for the changes to take effect.

4. On the indexer(s) that you configured the universal forwarders to send Security Event Log data to, log into Splunk Enteprise to confirm that the DCs only forward Security Event Code 4662/566 for groupPolicyContainer objects.

Note: A deployment server significantly reduces the amount of time required to update the universal forwarders on the DCs in your environment. Consider deploying one and configuring all of your universal forwarders as deployment clients. Read "Set up a deployment server and create a server class" for additional information on how to configure deployment server.