Troubleshoot the Splunk App for Windows Infrastructure
This topic discusses how you can troubleshoot your Splunk App for Windows Infrastructure deployment if you aren't seeing the data that you expect.
Is the central Splunk App for Windows Infrastructure instance correctly configured?
The first thing to check when Splunk App for Windows Infrastructure data is incomplete or incorrect is to confirm that the central Splunk instance is properly configured and is receiving data.
1. Confirm that every indexer in the central Splunk instance is configured to receive data.
Note: Review "Enable a receiver" in the core Splunk documentation for instructions.
2. If present, confirm that every search head in the central Splunk instance is configured properly. Search heads must search all available indexers to get all indexed AD data.
- Review "Distributed search deployment overview" and "Configure distributed search" in the core Splunk Enterprise documentation for specific instructions on configuring search heads and search peers (indexers).
- Review "How to deploy the Splunk App for Windows Infrastructure" for specific instructions on what specifically to install on the central Splunk instance's search heads.
3. Confirm that the Splunk App for Windows Infrastructure is installed properly.
- The Splunk App for Windows Infrastructure must reside on all indexers and search heads in the central Splunk instance.
- SA-ldapsearch must reside on all search heads in the central Splunk instance.
- Indexes that the Splunk App for Windows Infrastructure requires must be present on all indexers:
msad: for AD health metrics
winevents: for Directory Service, Replication Service, DNS server event logs
perfmon: for performance metrics
- Review "About managing indexes" in the core Splunk Enterprise documentation for instructions on how to confirm that the proper indexes exist.
%SPLUNK_HOME%\etc\apps\splunk_app_windows_infrastructure\local) must be configured with the proper indexes for the defined event types.
- Check for typos in the configuration file.
- Review "Install the app onto the central Splunk instance" for specific configuration instructions.
%SPLUNK_HOME%\etc\apps\SA-ldapsearch\local) must be configured with the proper credentials to bind to Active Directory:
- Check for typos in the configuration file.
- Confirm that the credentials are valid, and that the account is not locked out, does not have an expired password, and is allowed full access to the AD schema which you are trying to monitor.
- Review the table below for specific credential troubleshooting instructions.
4. Confirm that the app's lookup tables have been properly created.
- From the "Tools and Settings" menu, select "Build lookups".
5. Confirm that PowerShell is installed and enabled on all domain controllers and DNS servers, and that GPOs are in place to enable local script execution.
- Review "Enable auditing and local PowerShell script execution on Active Directory servers" for specific instructions.
6. Confirm that the attribute names in the base Distinguished Name have the proper capitalization.
Troubleshoot issues with ldapsearch
When the Splunk App for Windows Infrastructure cannot complete a search using the
SA-ldapsearch supporting add-on, it notifies you by displaying an error message in Manager's status bar (at the top of your browser window), as follows:
External search command 'ldapsearch' returned error code 1.
ERROR: com.unboundid.ldap.sdk.LDAPException: 80090308: LdapErr: DSID-0C090334, comment: AcceptSecurityContext error, data 525, vece
The Splunk App for Windows Infrastructure also writes a message to
$SPLUNK_HOME/var/log/splunk/SA-ldapsearch.log, similar to the following:
2012-08-10 14:58:34.108 -0700 pid=877 com.splunk.program.LDAPSearch:main#-1 ERROR Exception com.unboundid.ldap.sdk.LDAPException thrown: 80090308: LdapErr: DSID-0C090334, comment: AcceptSecurityContext error, data 525, vece^@
If you see an error message similar to this when performing a search, use the following table to decode the
data value and figure out how to resolve the error.
|Data value||What it means||What you should do|
|255||Either the domain was not found or there was a syntax error in the search command.||Confirm that the domain that you want to monitor exists and is configured properly, or that your search string is properly formatted and syntactically correct.|
|525||The username provided in
|52E||The password provided in
|530||The user account provided is not allowed to log into Active Directory at this time.||Remove the user's log on time restrictions from within Active Directory, then try again.|
|531||The user account provided is not allowed to log into Active Directory from the current server.||Modify the local security policy of the server from which the specified user is trying to log in to Active Directory, then try again.|
|532||The user account provided has an expired password.||Change the user's password or set the "Password never expires" bit from within Active Directory, then try again.|
|533||The user account provided is disabled.||Re-enable the user account from within Active Directory, then try again.|
|701||The user account provided has expired.||Re-enable the user account from within Active Directory, then try again.|
|773||The user account provided has the "User must reset password at next logon" bit set.||Un-set the "User must reset password at next logon" bit for the user account from within Active Directory, then try again.|
|775||The user account provided is locked because an incorrect password has been entered too many times.||Re-enable the user account from within Active Directory and change the password to a known good one, then try again.|
Windows event log or performance events from universal forwarders go to the 'main' index
When you install a universal forwarder on a Windows server, you must not select any options in the Enable Inputs screen of the installer. Doing so enables the scripted inputs that come with the forwarder by default. Those inputs send data to the "default" index as specified in their configuration files, which, on a standard Splunk installation, is
After you install the universal forwarder onto your Windows server, you must then install the appropriate technology add-ons included in the Splunk App for Windows Infrastructure's installation package. You must place the TAs in
%SPLUNK_HOME%\etc\apps within the universal forwarder. Review "Deploy configurations for all server roles" for specific instructions.
Dashboards fail to load after upgrading the app or Splunk Enterprise
If you experience an issue where some dashboard panels or menus fail to load after you upgrade either Splunk Enterprise or the Splunk App for Windows Infrastructure, clear your web browser's cache, log out of Splunk Enterprise, then log back in. This should fix the problem.
No data types found after upgrade
If you experience a problem where the first-time run process detects no data after an upgrade to 1.0.3, make sure you have added the new
winfra-admin role to the user that runs the app.
1. In the upper right hand of the Splunk Web page, click Settings.
2. In the window that pops up, under "Users and Authentication", click Access Controls. Splunk loads the "Access Controls" page.
3. Click Users. Splunk loads the "Users" page.
4. In the Username column, click the name of the user that runs the Splunk App for Windows Infrastructure. Splunk Enterprise loads the settings page for that user.
5. Scroll down to Assign to roles.
6. Add the
winfra-admin role to the Selected roles pane by clicking on its entry in the Available roles pane.
7. Click Save. Splunk saves the changes and returns you to the Users page.
Lookup table reference
Best practices guide
This documentation applies to the following versions of Splunk® App for Windows Infrastructure: 1.0.3, 1.0.4