Troubleshoot the Splunk Supporting Add-on for Active Directory

When an app that uses the Splunk Supporting Add-on for Active Directory cannot complete a search, it notifies you by displaying an error message in the Splunk status bar (at the top of your browser window), as follows:

External search command 'ldapsearch' returned error code 1.
ERROR "Invalid credentials for the user with binddn="<user>". Please 
correct and test your SA-ldapsearch credentials in the context of domain="<domain>"

It also writes a message to $SPLUNK_HOME/var/log/splunk/SA-ldapsearch.log, similar to the following:

2014-10-10 13:45:31,052, Level=ERROR, Pid=3950, File=search_command.py, Line=278, Abnormal exit: "Invalid credentials for the user with binddn="<user>". Please correct and test your SA-ldapsearch credentials in the context of domain="<domain>"

Configuration page not saving

if you get no response when saving or trying to test the connection in your domain configurations, upgrade your Splunk Enterprise deployment to version 7.0 or above.

LDAP commands exit with 'undefined domain' error

If you configure or reference an invalid domain in ldap.conf, the ldapfilter, ldapfetch, and ldapgroup commands in a subsequent search exit immediately with an error similar to the following:

External search command 'ldapfilter' returned error code 1. Script output = 
" ERROR Undefined domain name: <domain>. "

The commands immediately stop execution at that point and do not search further, even if the query source has additional entries with valid domains.

To fix the problem, confirm that you have defined all domains that the add-on must connect to in ldap.conf.

LDAP commands exit with 'No key or prefix' error

If you do not configure the default domain in ldap.conf the ldapfilter, ldapfetch, and ldapgroup commands in a subsequent search exit immediately with an error similar to the following:

 External search command 'ldapgroup' returned error code 1. Script output = 
" ERROR "KeyError at ""/Applications/Splunk/etc/apps/SA-ldapsearch/bin/packages/splunklib/data.py"", 
line 245 : u'No key or prefix: $text.'" "

To prevent this error, confirm that you have configured the default domain in the add-on configuration page.

LDAP commands exit with 'password is mandatory in simple bind' error

If test connection isn't successful when configuring the domain, you'll receive the following error when running any LDAP command.

External search command 'ldapsearch' returned error code 1. Script output = "error_message=password is mandatory in simple bind ".

To fix this issue, make sure that provided domain and the default domain in the LDAP command are configured properly and that the test connection is successful. See Configure the Splunk Supporting Add-on for Active Directory to learn more.

Other Issues

Authentication fails despite a successful connectivity test after configuration

If you encounter a problem where queries with SA-LDAPsearch fail despite successfully testing a connection that you set up on the configuration page, make sure that the user that you log into Splunk Enterprise as has the list_storage_passwords capability. This capability must be present because the configuration page saves passwords as storage passwords, and only this capability allows users to read storage passwords.

If you cannot grant the list_storage_passwords capability, as a workaround, you can use a clear-text password and obfuscate that password with base-64 encoding. In this case, however, you can not use the configuration page to save the password nor can you test the connection. This is because the configuration page moves any clear-text passwords to storage passwords when you save the configuration.

You must edit ldap.conf with a text editor, and for each LDAP domain you configure, add the password to its respective stanza in the following format:
password = <password>

Prepend the base-64 encoded password with {64}. For example:
password = {64}aAbBcCdDd=

Note that all passwords must be configured using the same method. That is, you cannot configure the password for one domain using the configuration page, while another using the ldap.conf. Use the ldaptestconnection command to test the configuration once completed.

To push a base-64 encoded password to all search heads in a distributed environment, see Configure the add-on to push a base-64 encoded password to all search-heads in a distributed environment.

From v3.1.0 onwards, base64 encoded or a clear password are not supported.

Domain name is case-sensitive when user configures ldapsearch

Windows forwarders log domain names as uppercase. Avoid configuration issues by configuring all domain names in uppercase on ldapsearch's settings page.

Ldapfetch, ldapfilter, and ldapgroup commands always query the default domain

The ldapfetch, ldapfilter, and ldapgroup commands expand the domain option based on input record values. This means that you cannot always use the domain option value to query for schema before processing records. The commands rely on the default domain when querying for schema instead. One reason for this is that they must know what column names to add to the output record stream for every domain they might query.

To fix this problem, configure the default domain so that it connects to the global catalog server for your Active Directory Domain Services (AD DS) forest. A global catalog server is a domain controller that stores a full copy of all objects in the directory for its host domain and a partial, read-only copy of all objects for all other domains in the forest. Global catalog servers respond to global catalog queries. They have a copy of the schema that applies to every domain in your forest.

For more information on Global Catalog, see "Understanding the Global Catalog (https://technet.microsoft.com/en-us/library/cc730749.aspx) on MS TechNet.

Some attributes are not available, and associated searches do not return results

Some attributes are constructed. This means that they get their values as a result of computation from other attributes.

Constructed attributes cannot be used in LDAP filters. Exceptions to this rule are the attributes createTimeStamp,modifyTimeStamp>, objectClass, and structuralObjectClass. Constructed attributes cannot be returned as value data in an LDAP search request unless you specify scope=base which means that the LDAP client accesses only a single object. It is not possible, for example, to search for all objects in a container and specify a constructed attribute as one of the requested attributes in that search.

Slow performance on queries to Active Directory with large number of users

To improve performance on queries against ADs with large numbers of users, select only the query attributes you need to complete your analysis. For example, if you need just two attributes, distinguishedName and sAMAccountName, say so. Use this command:

| ldapsearch search="(objectClass=user)" attrs="distinguishedName,sAMAccountName"

instead of:

| ldapsearch search="(objectClass=user)"

The former, more specific command reduces the total volume of data that it retrieves from Active Directory and passes through the search pipeline dramatically. This strategy significantly improves performance and reliability.

See the following pages for more information:

• Chapter 4: Active Directory Schema (https://www.safaribooksonline.com/library/view/active-directory-4th/9780596155179/ch04.html) on Safari Books.
• Object Attributes of Category: Constructed (http://www.selfadsi.org/deep-inside/attribute-constructed.htm) on SelfADSI.

Unable to query Active Directory anonymously

By default, Active Directory disables anonymous binding. Until you enable it, errors like this occur:

dc-1.msapps.local: Could not access the directory service at ldaps://dc-
1.msapps.local:3269: 000004DC: LdapErr: DSID-0C09072B, comment: In order to 
perform this operation a successful bind must be completed on the connection., 
data 0, v2580"

To enable anonymous binding you must change the seventh bit of the dsHeuristics attribute on the CN=Directory Service,CN=Windows NT,CN=Services,CN=Configuration,Root domain in forest directory object.

Valid values for the dsHeuristics attribute are 0 and 2. By default, the dsHeuristics attribute is not set, which is the same as if it were set to 0. If you set the seventh character to 2, anonymous clients can then perform any operation that the access control list (ACL) permits. If the attribute has been set, do not modify any bits in the dsHeuristics string other than the seventh. If the value has not been set, make sure that you provide the leading zeros up to the seventh bit. You can use the Active Directory Service Interface (ADSI) Editor Microsoft Management Console snap-in (adsiedit.msc) to make the change to the dsHeuristics attribute.

After you set the attribute, if you want anonymous users to be able to query Active Directory, you can enable anonymous access to specific directory objects. Users gain anonymous access to Active Directory objects through Anonymous Logon, which is a special security identifier (SID) that AD uses to represent anonymous network callers that perform an LDAP bind with NULL credentials.

See the following pages for more information:

• How Active Directory Searches Work (https://technet.microsoft.com/en-us/library/cc755809%28v=ws.10%29.aspx) on MS Technet.
• DSHeuristics Attribute (Windows) (https://msdn.microsoft.com/en-us/library/ms675656(v=vs.85).aspx) on MSDN.

SA-LDAPsearch does not return custom AD attributes

Active Directory does not replicate new attributes to the Global Catalog by default. If you run a query against a Global Catalog server and specify an attribute that has not been replicated to the GC, that attribute will not appear in a result of a query initiated by SA-LDAPsearch. To fix the problem, use the Active Directory Schema MMC snap-in to edit the attribute and enable replication to the Global Catalog. Next, restart the computer or Active Directory and attempt the query in SA-LDAPsearch again.

SA-LDAPsearch generates "The default configuration stanza for ldap.conf is missing" errors in a distributed Splunk Enterprise or Splunk Cloud environment

See "Workaround for default configuration stanza errors in distributed environments" in this manual.

Learn more about LDAP queries against Active Directory

Microsoft TechNet provides a number of useful documentation resources on querying Active Directory with LDAP. Here are a few recommendations:

• Active Directory Searches Technical Reference
• How Active Directory Searches Work
• Active Directory: LDAP Syntax Filters
• Global Catalog and LDAP Searches