Splunk® Enterprise

Release Notes

Acrobat logo Download manual as PDF

Splunk Enterprise version 7.2 will no longer be supported as of April 30, 2021. See the Splunk Software Support Policy for details. For information about upgrading to a supported version, see How to upgrade Splunk Enterprise.
This documentation does not apply to the most recent version of Splunk. Click here for the latest version.
Acrobat logo Download topic as PDF

Known issues

The following are issues and workarounds for this version of Splunk Enterprise.

Issues are listed in all relevant sections. Some issues appear more than once. To check for additional security issues related to this release, visit the Splunk Security Portal.

Refer to System requirements in the Installation Manual for a list of supported platforms and architectures.

For a list of deprecated features and platforms, refer to Deprecated features in this manual.

Upgrade issues

Date filed Issue number Description
2018-04-13 SPL-153403 After running the "clean userdata" command, admin is unable to login with msg "No users exist. Please set up a new user."

Workaround:
Create a $SPLUNK_HOME/etc/system/local/user-seed.conf and restart Splunk

[user_info]
PASSWORD = <yourpassword>


2017-05-23 SPL-141964 Older 6.0 and 6.1 maintenance release forwarders unable to forward events to 6.6.x and later indexers via splunktcp-ssl.

Workaround:
This affects communication between Splunk 6.6.x and later indexers and:
  • 6.0.0 to 6.0.6 forwarders
  • 6.1.0 to 6.1.4 forwarders

Upgrade your older forwarders to the latest maintenance releases or on your 6.6.x Indexer, add to inputs.conf:

[SSL]
sslVersions = *,-ssl2
cipherSuite = ALL:!aNULL:!eNULL:!LOW:!EXP:RC4+RSA:+HIGH:+MEDIUM


2017-05-23 SPL-141961 Older 6.0, 6.1, 6.2, 6.3 maintenance release versions unable to connect to 6.6.x and later via management port

Workaround:
This applies to License Master/Slave, Deployment Server/Client, Cluster Master/Peers, Search Head/Peers and affects Splunk 6.6.x and the following versions:
  • 6.0.0 to 6.0.6
  • 6.1.0 to 6.1.4
  • 6.2.0 to 6.2.6
  • 6.3.0 to 6.3.1
  • 6.3.1511.1

Upgrade your older instances to the latest maintenance releases or on your 6.6.x Splunk instances. Add the following stanza to server.conf:

[sslConfig]
sslVersions = *,-ssl2
sslVersionsForClient = *,-ssl2
cipherSuite = TLSv1+HIGH:TLSv1.2+HIGH:@STRENGTH


2017-03-20 SPL-139019 Possible compatibility issues between Python / SDK clients and new 6.6 and later default sslVersions, cipherSuites

Workaround:
Users can do either of the following:

1. Overwrite the new Splunk 6.6 server.conf [sslConfig] sslVersions, cipherSuites with your own settings that are compatible with your version of OpenSSL, e.g. the previous defaults from 6.5.x are compatible with OpenSSL 0.9.8 on Mac OSX:

[sslConfig]
sslVersions = *,-ssl2
sslVersionsForClient = *,-ssl2
cipherSuite = TLSv1+HIGH:TLSv1.2+HIGH:@STRENGTH

2. For some more up-to-date clients, it is possible to enforce TLS1.2 (e.g. --tlsv1.2 for curl) in order to connect successfully.

3. Upgrade OpenSSL on your platform and link it with your client (e.g. Python, curl, etc..). For example, OpenSSL 1.0.2 is currently available on Mac OSX via Homebrew (see https://brew.sh) and is compatible with the new Splunk 6.6 default sslVersions, cipherSuites.

2017-03-13 SPL-138647 Possible compatibility issues between new 6.6 and later default sslVersions, cipherSuites and external services, e.g. e-mail, LDAP

Workaround:
If security is not a significant concern, simply revert back to the 6.5.x SSL/TLS defaults, e.g. for e-mail, add to $SPLUNK_HOME/etc/system/local/alert_actions.conf

[email]
sslVersions = *,-ssl2
cipherSuite = TLSv1+HIGH:TLSv1.2+HIGH:@STRENGTH


To configure LDAP with the same settings used by e-mail alerts: $SPLUNK_HOME/etc/openldap/ldap.conf

TLS_PROTOCOL_MIN 3.1
TLS_CIPHER_SUITE TLSv1+HIGH:TLSv1.2+HIGH:@STRENGTH


To completely revert the LDAP configuration to the 6.5.x SSL/TLS defaults, comment out TLS_PROTOCOL_MIN and TLS_CIPHER_SUITE


If you would like to retain the more secure 6.6.x defaults, but prefer to add an exception for your less secure external services, follow the procedure below:

1. To determine what sslVersions and cipherSuites are supported by a server, run splunk cmd openssl s_client -connect hostname:port | awk '/Protocol/ || /Cipher/ || /Verify/'.

The example below is for a Postfix SMTP server:

eserv@indexer01:~$ splunk cmd openssl s_client -connect smtp-server01:465 | awk '/Protocol/ || /Cipher/ || /Verify/'
depth=1 C = US, O = Example Customer, OU = IT, CN = Example Customer IT CA, emailAddress = customer@example.org verify error:num=19:self signed certificate in certificate chain New, TLSv1/SSLv3, Cipher is DHE-RSA-AES256-SHA

   Protocol : TLSv1 
   Cipher : DHE-RSA-AES256-SHA 
   Verify return code: 19 (self signed certificate in certificate chain)

2. Check the OpenSSL output for Protocol and Cipher. In the example above, Protocol = TLSv1 and Cipher = DHE-RSA-AES256-SHA

3. Update Splunk's relevant sslVersions and/or cipherSuite. In the example above, sslVersions should be set to tls (allows TLSv1, TLSv1.1, TLSv1.2) and DHE-RSA-AES256-SHA should be appended to the end of the default cipherSuites definition, e.g. add
$SPLUNK_HOME/etc/system/local/alert_actions.conf
:

[email]
sslVersions = tls

cipherSuites = ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA

2014-08-20 SPL-89640 When running Splunk on Linux as non-root user and using RPM to upgrade, the RPM writes $SPLUNK_HOME/var/log/introspection as root, causing errors upon restarts

Workaround:
Chown the $SPLUNK_HOME/var/log/introspection directory to the user Splunk Enterprise runs as after upgrading and before restarting Splunk Enterprise.
2013-08-19 SPL-73386 Users are not allowed to run historical scheduled search

Workaround:
1. Create a special power/admin user who can run scheduled searches.

2. Assign this user ownership of the scheduled searches.

3. Share the searches at the app level and grant read/write permission to the correct set of users.

Data input issues

Date filed Issue number Description
2019-11-04 SPL-178914, SPL-171961 The datetime.xml timestamp recognition file does not recognize two-year dates after 2019 or Unix epoch-time seconds higher than 1599999999 (12:26:39 UTC 13 Sep 2020)
2017-08-17 SPL-144130 After data input and click 'start searching' will redirect to Launcher app
2017-07-19 SPL-143236 Custom sourcetype is not displayed on sourcetype menu

Workaround:
Set a filter and the sourcetype will display.
2015-11-12 SPL-109362 When the disk runs out of space for the limit set in the server.conf, add data workflow gets stuck with "Uploading file" message modal in the review stage
2015-05-22 SPL-101981 Field extractions do not work when sourcetypes use quotes in the Getting Data In interface.
2015-03-17 SPL-98163 INDEXED_EXTRACTIONS=W3C is truncating field cs_uri_stem when spaces are present in URL

Workaround:
Create a separate extraction in props.conf where defined w3c extraction method:

EXTRACT-cs_uri_stem1 = (GET|POST) (?<cs_uri_stem1>[^-]++)

Search issues

Date filed Issue number Description
2020-05-11 SPL-188941, SPL-180672 Join command with one or more field(s) not in common between search results set returns nonzero events
2020-04-07 SPL-185956, SPL-186131 replace_table_with_fields optimizer doesn't guarantee field order for searches where this matters , for example: <non-transforming search> | table | transpose

Workaround:
Add this to the search if field ordering for the first table command matters:
| noop search_optimization.replace_table_with_fields=f

Or, if you can restructure the search, so if you would have something with a transforming command first:

index=_internal 
| stats latest(_time) AS _time BY host index  
| table host _time index 
| transpose 2

Or run the search in VERBOSE mode.

2020-03-05 SPL-184348, SPL-185394, SPL-184601, SPL-185393 Splunk returns no results after adding field extractions without capturing group in REGEX when using FORMAT field::value

Workaround:
Add a capturing group to the REGEX.

REGEX = (.)

Example of configuration that would show this issue: props.conf:

[splunkd]
REPORT-Whatever = this-breaks-searching

and transforms.conf:

[this-breaks-searching]
REGEX = .
FORMAT = myfield::myvalue
2020-02-12 SPL-183259 When generating LISPY for field values that are numbers (""), the values aren't deduplicated, which can cause slowdowns in certain scenarios

Workaround:
Dedup values in search before, for example:

instead of

index="field_test" [search index="field_test" globalCallID_callId=1234* | fields globalCallID_callId]

add a stats or dedup in the subsearch:

index="field_test" [search index="field_test" globalCallID_callId=123* | stats values(globalCallID_callId) AS globalCallID_callId | mvexpand globalCallID_callId ]

If that list is still large and you're seeing the slowdown, consider moving the filtering to a | where after the initial search, for example:

index="field_test" globalCallID_callId=* | where [search index="field_test" globalCallID_callId=123* | stats values(globalCallID_callId) AS globalCallID_callId | mvexpand globalCallID_callId ]
2020-02-05 SPL-182842 Issue with maps viz, geostats in combination with |append or |inputlookup append=t, some pie chart not showing on map

Workaround:
Avoid "| append" or "|inputlookup append=t" if you can in combination with geostats
2020-01-30 SPL-182511, SPL-183265 split() on an empty string results in typeof(field) = Invalid and a "| mvexpand" will then not return that event

Workaround:
For searches that look like this:
| makeresults 
| eval a="" 
| eval a=split(a,"z"), b="junk" 
| foreach * 
    [| eval typeof_<<FIELD>>=typeof(<<FIELD>>)]
| mvexpand a

Add an eval before mvexpand to handle this for example:

...
| eval a=if(tostring(typeof(a))="Invalid","",a) 
| mvexpand a


2020-01-10 SPL-181573 geostats provides incorrect results for lower zoom levels when split BY has a higher cardinality than globallimit.

Workaround:
- Increase globallimit to the value of "unique values" number mentioned in the warning message:

"The split by field <field> has a large number of unique values <number>. Chart column set will be trimmed to 10. Use globallimit argument to control column count."

- Use very high globallimit in geostats and post process after if needed

- Don't use BY in geostats

- Use lower cardinality BY and/or higher globallimit in geostats

2020-01-09 SPL-181525, SPL-182841, SPL-182404, SPL-182843 Issue with maps viz, geostats in combination with |append or |inputlookup append=t, some pie chart not showing on map

Workaround:
Avoid "| append" or "|inputlookup append=t" if you can in combination with geostats
2019-12-18 SPL-181152, SPL-177255 Searching for lookup default_match value includes default_match value in lispy

Workaround:
For each lookup field <FIELD> that's causing issues, add to fields.conf:

[<FIELD>] INDEXED_VALUE=false

2019-09-11 SPL-176333, SPL-178302, SPL-178303, SPL-195181, SPL-178301 Lookups may return incorrect results due to internal caching

Workaround:
Add

allow_caching=f to the lookup command:

| lookup <name> allow_caching=f ... 

On 7.3+: Add allow_caching=f to the lookup definition on the search head

transforms.conf:
[<lookup name>]
allow_caching = f

To check if you might be running into this issue, you'll need to enable debug on the search in question by adding:

| noop log_DEBUG=CachedProvider
<pre>
If you have hits for the cached lookup, like in the sample log below, you can hit this issue.

<pre>
DEBUG CachedProvider - Cached provider metrics: lookup=<lookup name> hits=67064 misses=321 total=67385
<\pre><br/> 
|- 
| 2019-09-05||SPL-176008, SPL-166728||Alert email spacing issue 
|- 
| 2019-09-05||SPL-176009, SPL-166728||Alert email spacing issue 
|- 
| 2019-07-29||SPL-174005, SPL-182156, SPL-175325||Search crashes on indexer in ChunkedCSVLineReader::initReader due to empty kvstore lookup folder in the bundle<br/><br/>Workaround:<br/>Two options:
1) disable kvstore collection replication to the indexers (changes on the SH side):
in collections.conf under collections stanza set replicate=false

2) enable old lookups code (changes on the Indexers side):
in limits.conf set the below
[lookup]
use_lookups_v2 = false<br/> 
|- 
| 2019-07-25||SPL-173781||Transaction command not merging common values in multivalue fields after 7.2<br/><br/>Workaround:<br/>From 7.2 onward when events with multivalue fields used for a transaction don't overlap completely, they are considered to be different transactions.


Example:
In Splunk 7.1 and below this would create one transaction, but in 7.2 onward it will create two:
| makeresults 
| eval foo="a,b,c" 
| append 
[ makeresults 
| eval foo="c,d,e" ] 
| eval foo=split(foo,",") 
| transaction foo


If you append connected=f to the transaction command it will merge the events into one transaction if at least one value of the multivalue field overlaps:

| makeresults 
| eval foo="a,b,c" 
| append 
[ makeresults 
| eval foo="c,d,e" ] 
| eval foo=split(foo,",") 
| transaction foo connected=f<br/> 
|- 
| 2019-07-24||SPL-173708||After Upgrade Fields not displaying unless order of fields match fields_list in transforms.conf<br/><br/>Workaround:<br/>Current workaround employed by the Customer was to modify their script to have the exact same output as configured in fields_list inside of transforms.conf.<br/> 
|- 
| 2019-06-21||SPL-172299, SPL-168859||Any transformational commands will not include the base fields from transforms.conf when performing search in SMART mode resulting in required field not been included 
|- 
| 2019-06-20||SPL-172237, SPL-168112||Crashing thread: dispatch-  __assert_perror_fail and phase_1-StatsDatum8setValue 
|- 
| 2019-04-29||SPL-169655, SPL-173709, SPL-171705, SPL-176220, SPL-175476, SPL-175474||After Upgrade Fields not displaying unless order of fields match fields_list in transforms.conf<br/><br/>Workaround:<br/>Current workaround employed by the Customer was to modify their script to have the exact same output as configured in fields_list inside of transforms.conf.<br/> 
|- 
| 2019-03-22||SPL-168112, SPL-175126, SPL-170575, SPL-172237||Crashing thread: dispatch-  __assert_perror_fail and phase_1-StatsDatum8setValue 
|- 
| 2019-02-25||SPL-166901, SPL-170551, SPL-170761||Search intermittently hangs when using custom python command and chunked protocol on Windows 
|- 
| 2019-02-05||SPL-166001||16MB+ events are not displayed on the search results, but they will be listed on the fields sidebar and in the timeline. search.log message: "SRSSerializer - max str len exceeded - probably corrupt"<br/><br/>Workaround:<br/>Make sure fields are under 16777216 characters (or 16MB, usually _raw is the biggest)

OR

Revert back to the old serialization format (CSV), however, this applies to all searches, so you won't be getting the (performance) benefits of the new format.

$SPLUNK_HOME/etc/system/local/limits.conf:
[search]
results_serial_format=csv<br/> 
|- 
| 2019-01-28||SPL-165608, SPL-166627, SPL-172449, SPL-172453||column order is misaligned in alert email compared to the results showing in Splunk. 
|- 
| 2019-01-16||SPL-165046, SPL-165326||Search crashes due to missing name in EVAL- in props.conf 
|- 
| 2019-01-14||SPL-164879, SPL-163361||mvexpand on  7.1+ consumes more memory than expected 
|- 
| 2019-01-10||SPL-164718, SPL-165363, SPL-166562||limits.conf "phased_execution_mode = singlethreaded"  causes issue with field ordering, for example _time showing in legend for a | timechart<br/><br/>Workaround:<br/>Set the value of "phased_execution_mode" to "multithreaded" to restore the correct field order.<br/> 
|- 
| 2018-12-18||SPL-164107, SPL-169524||the mstats rate(x) function does not work with wildcards in the enhanced syntax<br/><br/>Workaround:<br/>The rate(x) function is designed to be applied to a single counter metric. Use an explicit projection field name or use "_value" syntax with wildcard and group by.  <br/> 
|- 
| 2018-12-13||SPL-163932, SPL-164894||Disabling case_sensitive_match in transforms.conf not working for WILDCARD type lookups<br/><br/>Workaround:<br/>You can normalise the data in the lookup (| eval field=lower(field)) before populating, and doing the same before looking it up. 
If you need the denormalised version, you can create a different field for the lookup instead to still have access to the original. 

Create lookup: 
... | eval field=lower(field) | outputlookup 

Use lookup: 
... | eval matchfield=lower(field) | lookup <lookup> matchfield ...<br/> 
|- 
| 2018-11-29||SPL-163319, ITSI-3045, SPL-163454||Despite the forceCsvResults parameter not existing in the configuration for a saved search with summary indexing enabled, the summarized data is improperly populated with this parameter.<br/><br/>Workaround:<br/>add below to $SPLUNK_HOME/etc/system/local/alert_actions.conf:

[summary_index]
command = summaryindex spool=t uselb=t addtime=t index="$action.summary_index._name{required=yes}$" file="$name_hash$_$#random$.stash_new" name="$name$" marker="$action.summary_index*{format=$KEY=\\\"$VAL\\\", key_regex="action.summary_index.(?!(?:command|inline|forceCsvResults|maxresults|maxtime|ttl|track_alert|(?:_.*))$)(.*)"}$"<br/> 
|- 
| 2018-11-07||SPL-162655, SPL-164505||Add ASNEW keyword to FIELDALIAS to support non-overriding version of aliasing<br/><br/>Workaround:<br/>Avoid applying the same alias field name to multiple original field names.

If you must do this, set it up as a calculated field (an EVAL-* statement) that uses the 'coalesce' function to create a new field that takes the value of one or more existing fields. This method lets you be explicit about ordering of input field values in the case of NULL fields. For example: EVAL-ip = coalesce(clientip,ipaddress)<br/> 
|- 
| 2018-11-01||SPL-162448, SPL-154678, SPL-163564|||metadata search error - Failed to apply deletes to some metadata 
|- 
| 2018-10-29||SPL-162339||Duplicate fields defined in |table or |fields command causes incorrect data to be assigned to a field in the Statistics tab<br/><br/>Workaround:<br/>Example search that shows this:
|stats count|eval test=1,test2=3,test3=4|table test test2 test test3
workaround: don't duplicate fields in the table or field command:
|stats count|eval test=1,test2=3,test3=4|table test test2 test3<br/> 
|- 
| 2018-10-24||SPL-162166, SPL-162548||splunkd: /opt/splunk/src/search/processors/lookup/IndexedCsvDataProvider.cpp:165: virtual void IndexedCsvDataProvider::lookupBatch(UnpackedResults&, const SearchResultsInfo&, const LookupDefinition&): Assertion `!_parse_only' failed.<br/><br/>Workaround:<br/>On the crashing peer (could be SH, Indexer or both) set the below in limits.conf:
max_memtable_bytes = 2*<size of the largest lookup>

example search to find the biggest lookups:
index=_* sourcetype=audittrail path=*lookups* size=* 
| stats max(size) AS size BY host, path 
| append 
[| rest services/server/introspection/kvstore/collectionstats 
| mvexpand data 
| table splunk_server title data 
| spath input=data 
| fields splunk_server size ns ] 
| eval host=coalesce(host,splunk_server) 
| fields host path ns size
| sort size | head 1<br/> 
|- 
| 2018-06-15||SPL-155648, SPL-169611, SPL-169612, SPL-185656||New phased_execution_mode is spawning extra processes for custom search commands<br/><br/>Workaround:<br/>If the custom search needs to run only once, disable the multithread for all searches.

$SPLUNK/etc/system/local stopped the issue from occurring.
[search]
phased_execution_mode = auto

Apply this workaround especially for deployment using ITSI, as the bug causes double backfill of the ITSI Episodes.

Beware, the workaround will cause a separate search issue SPL-165363, for splunk versions 7.0, 7.1 and 7.2 until the fix in 7.2.4
<br/> 
|- 
| 2017-08-23||SPL-144350||Archived Index is created without error when the splunk index is invalid 
|- 
| 2017-07-13||SPL-143111||"Splunkd daemon is not responding" when edit local windows event log collection 
|- 
| 2017-04-04||SPL-140765||Splunk having problems extracting json file consisting of 68k plus key-value pairs 
|- 
| 2017-03-21||SPL-140175||Aborted delete searches may result in stale lock files being left behind<br/><br/>Workaround:<br/>Delete stale lock files.

<br/> 
|- 
| 2016-11-29||SPL-133182||When two datasets have identical names but one is local (private) while the other is global, attempts to view or extend the global dataset use results from the local dataset instead. 
|- 
| 2016-04-27||SPL-118911||In SHC, referenced saved real-time searches in a dashboard do not stream results.<br/><br/>Workaround:<br/>See [http://docs.splunk.com/Documentation/Splunk/6.5.0/Viz/Savedsearches#Troubleshoot_referenced_real-time_searches_in_search_head_clusters Troubleshoot referenced real-time searches] for workaround details.

<br/> 
|- 
| 2015-08-10||SPL-105061, SOLNESS-7274||Broken module prevents splunkweb from starting 
|- 
| 2015-06-17||SPL-103247||Filtering on _time uses different semantics for the "=" operator on microseconds depending on whether the value is quoted. 
|- 
| 2015-04-23||SPL-100170||Automatic Lookups limitation: No results returned in Smart Mode when there are nested lookups and the intermediate field is not mentioned in the search. 
|- 
| 2014-12-22||SPL-94910||The replace function does not apply to fields names with an underscore in them.<br/><br/>Workaround:<br/>Rename the fields before the replace.
... | rename *_* AS *-* | replace "something" by "somethingelse"<br/> 
|- 
| 2014-11-13||SPL-93039||The relevancy search command does not work, always returning 0 or -inf. 
|- 
| 2014-10-02||SPL-91638, SPL-107375||For scheduled searches in a search head cluster, empty search jobs may appear in the job inspector for a cluster member. 
|- 
| 2014-09-15||SPL-90861, SPL-90396, SPL-90886||If search encounters invalid offsets or invalid rawdata at TSIDX offsets, it skips reading any number of events from that bucket. No message is displayed, though the information is added to search.log. 
|- 
| 2014-04-16||SPL-83129||Eval function strptime does not return results when 1970 date is used. 
|- 
| 2014-04-04||SPL-82650||A report created and scheduled by admin cannot be embedded by a power user. 
|- 
| 2014-03-27||SPL-82357||The splunk clean all -f CLI command doesn't remove data from the main index on Windows systems. 
|- 
| 2014-03-15||SPL-81934||For clusters, may be unable to open search results output file for search results in a cluster.<br/><br/>Workaround:<br/>Write to a temp file and rename to the target file.<br/> 
|- 
| 2014-02-21||SPL-80942||Flashtimeline: 500 Internal Server Error when pasting long URL into panel name. 
|- 
| 2013-12-18||SPL-78179||REST /saved/searches App names with special characters have invalid links. 
|- 
| 2013-08-19||SPL-73386||Users are not allowed to run historical scheduled search<br/><br/>Workaround:<br/>1. Create a special power/admin user who can run scheduled searches.

2. Assign this user ownership of the scheduled searches.

3. Share the searches at the app level and grant read/write permission to the correct set of users. <br/>
|}
== Saved search, alerting, scheduling, and job management issues ==

{| {{table}}
!Date filed 
!Issue number 
!Description 
|- 
| 2019-09-05||SPL-176008, SPL-166728||Alert email spacing issue 
|- 
| 2019-09-05||SPL-176009, SPL-166728||Alert email spacing issue 
|- 
| 2019-07-18||SPL-173414, SPL-179491, SPL-179612, SPL-179627||Splunk unable to load defined Saved Searches in a conf file if a bad/malformed cron_schedule value is present/set<br/><br/>Workaround:<br/>If this behavior is noticed (savedsearches - alerts/searches/reports) not showing up in the GUI, verify cron jobs are valid, if they're not, then correct them. 

A quick check would be to create a duplicate savedsearches.conf and removing all cron_schedule definitions and rebooting splunk with that config to identify if it is this issue.<br/> 
|- 
| 2019-05-22||SPL-170857, SPL-162249||The filter function of <splunk-search-dropdown> UI component is not working on on Splunk Enterprise 7.1 and later. 
|- 
| 2019-03-22||SPL-168109, SPL-164733||tstats searches do not run on datamodels that contain only a streamable BaseSearch object 
|- 
| 2018-11-29||SPL-163315, SPL-163882||Alert action not being fired with permission denied on reading search results after upgrade to 7.2.1<br/><br/>Workaround:<br/>Roll back to original version 
Or admin_all_objects is assigned<br/> 
|- 
| 2018-11-29||SPL-163319, ITSI-3045, SPL-163454||Despite the forceCsvResults parameter not existing in the configuration for a saved search with summary indexing enabled, the summarized data is improperly populated with this parameter.<br/><br/>Workaround:<br/>add below to $SPLUNK_HOME/etc/system/local/alert_actions.conf:

[summary_index]
command = summaryindex spool=t uselb=t addtime=t index="$action.summary_index._name{required=yes}$" file="$name_hash$_$#random$.stash_new" name="$name$" marker="$action.summary_index*{format=$KEY=\\\"$VAL\\\", key_regex="action.summary_index.(?!(?:command|inline|forceCsvResults|maxresults|maxtime|ttl|track_alert|(?:_.*))$)(.*)"}$"<br/> 
|- 
| 2018-09-19||SPL-160286||The data preview for the Add Data workflow does not display for Log to Metrics source types 
|- 
| 2017-11-29||SPL-146802||Distributed environment requires index defined on search head for log event alerts 
|- 
| 2016-09-23||SPL-129285||The search scheduler (SavedSplunker) has scaling problems with high disabled user count and external auth systems (SAML & LDAP) 
|- 
| 2015-04-09||SPL-99421||Long name of app causes accelerated search to not complete normally and shows invalid results on Windows 2008 R2<br/><br/>Workaround:<br/>Reduce length of name of the app and report acceleration searches will run properly within the context of the app.<br/> 
|- 
| 2014-08-15||SPL-89332||Report acceleration summaries do not show in Settings when you have hundreds of reports accelerated. 
|- 
| 2014-08-05||SPL-88396||After configuring a client name for a deployment client, the name is not shown in the Forwarder Management UI<br/><br/>Workaround:<br/>Create a server class, where you can see the client name, and use that group when you add data.<br/> 
|- 
| 2014-05-01||SPL-83686||Data Model Pivot: Extra NULL column displays in Pivot with big data and Numbered Attribute in Split Columns.<br/><br/>Workaround:<br/>The workaround is to add filter status=*, or make a more refined Data Model that has an object for events with status.<br/> 
|- 
| 2014-03-24||SPL-82262, SPL-82241||Pivot search command fails for an admin trying to pivot on a Private Data Model created by a User. 
|- 
| 2014-03-20||SPL-82164||Migrating invalid data models from 6.0 to 6.x fails. 
|- 
| 2014-03-19||SPL-82133||Data model allows users to upload a JSON file which has Field names with spaces but will not validate it. 
|- 
| 2014-03-10||SPL-81637||Splunkd preview runs indefinitely on any file preview with "DATETIME_CONFIG=none". 
|- 
| 2014-03-10||SPL-81645||Creating data model with root transaction name starting with root event name fails 
|- 
| 2013-11-26||SPL-77054, SPL-77055||Data model objects that have names starting with an underscore character ("_") do not work correctly and cannot be used in Pivot.
|}
== Charting, reporting, and visualization issues ==

{| {{table}}
!Date filed 
!Issue number 
!Description 
|- 
| 2020-02-09||SPL-183078, SPL-181033||_time is shown as GMT on Visualization when Time zone is set to default on Windows<br/><br/>Workaround:<br/>Set up a proper Timezone for a user on Preference.<br/> 
|- 
| 2020-02-09||SPL-183077, SPL-181033||_time is shown as GMT on Visualization when Time zone is set to default on Windows<br/><br/>Workaround:<br/>Set up a proper Timezone for a user on Preference.<br/> 
|- 
| 2020-02-05||SPL-182842||Issue with maps viz, geostats in combination with |append or |inputlookup append=t, some pie chart not showing on map<br/><br/>Workaround:<br/>Avoid "| append" or "|inputlookup append=t" if you can in combination with geostats<br/> 
|- 
| 2020-01-09||SPL-181525, SPL-182841, SPL-182404, SPL-182843||Issue with maps viz, geostats in combination with |append or |inputlookup append=t, some pie chart not showing on map<br/><br/>Workaround:<br/>Avoid "| append" or "|inputlookup append=t" if you can in combination with geostats<br/> 
|- 
| 2019-02-21||SPL-166770, SPL-168636, SPL-168637||URI malformed error in dashboard if search string contains % 
|- 
| 2019-01-15||SPL-164920, SPL-166952, SPL-167850, SPL-169010, SPL-169011||Dashboard issue: Multiselect URL retains single value after Hide Filters selected 
|- 
| 2018-05-02||SPL-154054, SPL-163446, SPL-164721||Dashboard Editor in de-DE locale CSS error in Format visualization modal for Stats Table/Line/Bar Charts 
|- 
| 2017-12-06||SPL-147115||Drilldown search fails when a timeformat is specified<br/><br/>Workaround:<br/>Remove the timeformat specification from the drilldown search or manually remove the search from the URL and run it in a new window.<br/> 
|- 
| 2016-09-15||SPL-128819, SPL-130243, SPL-130245||Editing panel in dashboard removes charting.legend.masterlegend option<br/><br/>Workaround:<br/>Use <option name="charting.legend.masterLegend">null</option><br/> 
|- 
| 2016-04-27||SPL-118911||In SHC, referenced saved real-time searches in a dashboard do not stream results.<br/><br/>Workaround:<br/>See [http://docs.splunk.com/Documentation/Splunk/6.5.0/Viz/Savedsearches#Troubleshoot_referenced_real-time_searches_in_search_head_clusters Troubleshoot referenced real-time searches] for workaround details.

<br/> 
|- 
| 2015-02-23||SPL-97193||The initial value for Multiselect input does not display properly in Visualizations Editor if input has empty string.
|}
== Data model and pivot issues ==

{| {{table}}
!Date filed 
!Issue number 
!Description 
|- 
| 2019-03-22||SPL-168109, SPL-164733||tstats searches do not run on datamodels that contain only a streamable BaseSearch object 
|- 
| 2014-12-08||SPL-94047, SPL-98628||While creating a Pivot and using the _time column as a Split column, the table columns aren't formatted in a human readable way, but displayed with the epoc timestamp.It works when using _time as a 'Split Row' column. 
|- 
| 2014-05-01||SPL-83686||Data Model Pivot: Extra NULL column displays in Pivot with big data and Numbered Attribute in Split Columns.<br/><br/>Workaround:<br/>The workaround is to add filter status=*, or make a more refined Data Model that has an object for events with status.<br/> 
|- 
| 2014-03-24||SPL-82262, SPL-82241||Pivot search command fails for an admin trying to pivot on a Private Data Model created by a User. 
|- 
| 2014-03-20||SPL-82164||Migrating invalid data models from 6.0 to 6.x fails. 
|- 
| 2014-03-19||SPL-82133||Data model allows users to upload a JSON file which has Field names with spaces but will not validate it. 
|- 
| 2014-03-11||SPL-81701||Data Model Pivot, "Legend Position" and "Stack Mode" change to default settings if you change the X/Y-Axis more than once. 
|- 
| 2014-03-10||SPL-81645||Creating data model with root transaction name starting with root event name fails 
|- 
| 2014-03-07||SPL-81538||When using Pivot, stack mode is lost when "Scatter Chart" is selected. 
|- 
| 2013-11-26||SPL-77054, SPL-77055||Data model objects that have names starting with an underscore character ("_") do not work correctly and cannot be used in Pivot.
|}
== Indexer and indexer clustering issues ==

{| {{table}}
!Date filed 
!Issue number 
!Description 
|- 
| 2019-03-21||SPL-168072, SPL-168054||Peer flaps from Up to BatchAdding when handleBucketsNotificationBatch is rejected during Master un-initialized 
|- 
| 2019-03-13||SPL-167708, SPL-170943, SPL-170937, SPL-170938||Apply cluster bundle does not apply bundle to any indexers which are in progress of adding to cluster<br/><br/>Workaround:<br/>restart affected indexer(s)<br/> 
|- 
| 2018-11-12||SPL-162802, SPL-161301||For a multisite cluster, splunk is not reaping prior search-buckets manifests after new generation<br/><br/>Workaround:<br/>Do manual cleanup of 
$SPLUNK_HOME/var/run/splunk/cluster/search-buckets
leaving the gen0 and 10 of the latest files per site as minimum

To automate this you can do something like this in cron once you're happy with the manual run, you just need to add the delete flag for find:

find $SPLUNK_HOME/var/run/splunk/cluster/search-buckets -regextype posix-extended -regex '.+_gen([0-9]{2,}|[1-9])\.csv\.gz' -mtime +2 

<br/> 
|- 
| 2018-10-23||SPL-161815||Thawed buckets in a indexer cluster are sporadically unsearchable upon restart 
|- 
| 2018-03-15||SPL-152168||Batch-mode retry can return more or less events than it should due to reordering from thread pool processing.<br/><br/>Workaround:<br/>Before you initiate searchable rolling restart or rolling upgrade, make sure the <code>search_retry</code> attribute in the <code>[search]</code> stanza of <code>limits.conf</code> is set to <code>false</code> (the default).<br>
<br>
If you have scheduled searches that must complete, either increase the value of <code>decommission_search_jobs_wait_secs</code> (default=180s) in <code>server.conf</code>, or do not run searchable rolling restart or rolling upgrade during the search's timeframe.<br/> 
|- 
| 2017-03-16||SPL-138846||In multisite clustering, deletion of events in hot buckets is not pushed to other sites 
|- 
| 2016-08-25||SPL-127353||Data rebalance finishes early when one peer is the source for all buckets<br/><br/>Workaround:<br/>when only one indexer in a cluster indexed data (has all the searchable copies), rebalance once before adding the new indexer, and then rebalance a second time<br/> 
|- 
| 2015-05-08||SPL-101184||Rolling restart in an Indexer Cluster may not be successful on a peer if a oneshot command is also running on that peer. Perform a manual restart to revive the peer. 
|- 
| 2014-10-13||SPL-91861||On Windows indexer on an ec2 instance, splunk-optimize main thread can crash on buckets on the temporary drive z:\>. 
|- 
| 2014-09-29||SPL-91432||On Windows when the master is down, the CLI command splunk offlinehangs when run from one of the streaming target peers. 
|- 
| 2014-09-08||SPL-90630||On a multisite cluster, no warning is given when search head names are the same. 
|- 
| 2014-08-29||SPL-90331||Multi-site indexer cluster doesn't meet replication factor/search head factor due to bucket issue.<br/><br/>Workaround:<br/>From the endpoint, add the buckets missing RF/SF to the to_fix list.

endpoint:
https://[host]:[port]/services/cluster/master/buckets/{bucket_id}/fix<br/> 
|- 
| 2014-07-29||SPL-87816||When implementing an indexer cluster or search head cluster, you cannot set pass4SymmKey in the general stanza. The system default values in the clustering and shclustering stanzas override any user-provided values in the general stanza.<br/><br/>Workaround:<br/>Set the value in the [clustering] or [shclustering] stanza, depending on the type of cluster you're implementing.<br/> 
|- 
| 2014-07-14||SPL-86799||After adding a new license to the clustering search head, splunkd on restart cannot be reached by splunkweb. 
|- 
| 2014-04-29||SPL-83636||When configuring a multi-site cluster using cluster-config, the error messages are incorrect if the SF/RF was previously set. 
|- 
| 2014-03-18||SPL-82038||Cluster-config does not work if a parameter value includes a space character. 
|- 
| 2014-03-17||SPL-81955||Multisite: Peer takes approximately 6 minutes to restart when its site configuration is changed. 
|- 
| 2014-01-06||SPL-78688||Peer is able to change to an invalid (empty) replication port 
|- 
| 2013-08-06||SPL-72484||You cannot use the CLI to delete an index with a capital letter in its name.
|}
== Distributed search and search head clustering issues ==

{| {{table}}
!Date filed 
!Issue number 
!Description 
|- 
| 2019-07-11||SPL-173029, SPL-184166, SPL-184164, SPL-184165||KV store backup/restore - large collection hangs at "Busy" status when trying to restore from a backup<br/><br/>Workaround:<br/>To restore from the full kvstore folder backup, if available.

Contact support for an alternative script to restore backup (restorekv.py)<br/> 
|- 
| 2019-05-06||SPL-169951, SPL-167421||the scheduled search "Bucket Copy Trigger" (aka Hadoop Data Roll) has stopped working properly. 
|- 
| 2019-05-06||SPL-169952, SPL-167421||the scheduled search "Bucket Copy Trigger" (aka Hadoop Data Roll) has stopped working properly. 
|- 
| 2019-04-10||SPL-169046, SPL-170862, SPL-171283, SPL-171341, SPL-171367||in SHC, several copies of the same scheduled index-time realtime search are running on distinct SHC instances - impacting ITSI itsi_event_grouping<br/><br/>Workaround:<br/>The following is a w/a that has been used by one customer to temporarily get back to 1 itsi_event_grouping running:

IMPORTANT: Follow these steps explicitly:

# Go to the SH UI, disable the scheduled search "itsi_event_grouping" from the SA-ITOA app. Wait for a minute for it to stop. If any copies are still running, stop them from the Activity-->Jobs, SA-ITOA --> itsi_event_grouping.
# On all search-heads, find the java processes, and terminate them
Connect to each Search-Heads and find the java processes (ps –ef|grep java)
# Go back to the UI, and re-enable the search itsi_event_grouping. Go back to the Jobs Manager and confirm that you see only 1 copy. If there are more than 1, press the STOP button (DO NOT DO DELETE) on other ones to ensure that there is ONLY 1 left running. You may need to refresh a couple of times to confirm.
 <br/> 
|- 
| 2018-03-14||SPL-152148||KV store replication fails on the upgrade search head during SHC member-by-member upgrade.<br/><br/>Workaround:<br/>To ensure there is no kvstore activity during upgrade, perform an offline upgrade as follows:

<ol>
  <li>Shutdown all cluster members.</li>
  <li>Upgrade all members.</li>
  <li> Start the member</li>
</ol>
<br/> 
|- 
| 2017-11-29||SPL-146802||Distributed environment requires index defined on search head for log event alerts 
|- 
| 2017-03-13||SPL-138654||Splunk searches fail when filepath gets too long on Windows 
|- 
| 2016-07-12||SPL-124085||On Search Head Cluster It is not possible to remove an App from the SHs once it has been disabled. 
|- 
| 2015-09-23||SPL-106978||Failed SHC captain election causes unnecessary change in server.conf 
|- 
| 2015-02-26||SPL-97385||$SPLUNK_HOME/var/run/splunk/snapshot contains large tarballs in the presence of large ES lookup table files.<br/><br/>Workaround:<br/>The allowable size of the download can be increased by setting the following in server.conf. 

[httpServer] 
max_content_length = 1500MB 

The other option is to disable the search which controls the generation of the large lookup file.  In this case, the search is:

[Endpoint - Local Processes Tracker - Lookup Gen]
<br/> 
|- 
| 2014-08-25||SPL-90028||Using "inputcsv dispatch=true" to read a CSV from a dispatch directory may not work on search head cluster members that have a replica of the desired artifact. 
|- 
| 2014-08-14||SPL-89131||In a search head cluster, the search Job management page on cluster member doesn't immediately reflect 'isSaved' state after you click Save. 
|- 
| 2014-08-02||SPL-88228||When user clicks on the RSS feed for an alert, search pool information is not displayed. Individual pool member information is displayed, however.
|}
== Universal forwarder issues ==

{| {{table}}
!Date filed 
!Issue number 
!Description 
|- 
| 2019-01-28||SPL-165635, SPL-191773, SPL-189789||splunk not reading file after log rotation 
|- 
| 2018-04-10||SPL-153251||Universal Forwarder txz package cannot be installed on FreeBSD 11.1<br/><br/>Workaround:<br/>1. Use pkg install instead of pkg add
OR
2. Install package by untarring tgz file to /opt/splunkforwarder<br/> 
|- 
| 2015-04-14||SPL-99687, SPL-129637||Splunk universal forwarder is 7-10 days behind recent Windows Security and system log events.<br/><br/>Workaround:<br/>To mitigate this, edit the following stanza in inputs.conf: [WinEventLog://Security] evt_resolve_ad_obj = 0.<br/> 
|- 
| 2015-04-07||SPL-99316||Universal Forwarders stop sending data repeatedly throughout the day<br/><br/>Workaround:<br/>In limits.conf, try changing file_tracking_db_threshold_mb in the [inputproc] stanza to a lower value.<br/> 
|- 
| 2014-08-05||SPL-88396||After configuring a client name for a deployment client, the name is not shown in the Forwarder Management UI<br/><br/>Workaround:<br/>Create a server class, where you can see the client name, and use that group when you add data.<br/> 
|- 
| 2013-09-18||SPL-74427, SPL-74448||The Splunk universal forwarder installer for Solaris 10 does not add the splunk user when you attempt to install it using the pkgadd command. This results in the script generating lots of errors.<br/><br/>Workaround:<br/>To work around this issue, create a splunk user on your system before attempting to run the installer.<br/>
|}
== Distributed deployment, forwarder, deployment server issues ==

{| {{table}}
!Date filed 
!Issue number 
!Description 
|- 
| 2019-01-31||SPL-165827, SPL-166188||Setting DC phoneHomeIntervalInSecs under  stanza is ineffective<br/><br/>Workaround:<br/>In the deploymentclient.conf, place the phoneHomeIntervalInSecs setting under a [deployment-client] stanza. <br/> 
|- 
| 2018-11-29||SPL-163320, SPL-162350||Ingest-time log-to-metrics conversion of structured log data is broken for 7.2.x Universal Forwarders 
|- 
| 2014-10-02||SPL-91648, SPL-91358||Forwarder unable to push scripted inputs to a Linux deployment client from a Windows deployment server. 
|- 
| 2014-08-15||SPL-89333||Using client filtering in forwarder management interface when the deployment server is servicing a large numbers of deployment clients (over approximately 5000) can cause a temporary spike in memory usage. 
|- 
| 2014-06-20||SPL-85739||When running a high number of deployment clients for a server, memory growth may be excessive.<br/><br/>Workaround:<br/>To mitigate this, set forceHttp10=always.<br/>
|}
== Monitoring Console issues ==

{| {{table}}
!Date filed 
!Issue number 
!Description 
|- 
| 2018-11-25||SPL-163189, SPL-161714||Saving edit to server roles in DMC configure view results in 409 Conflict error response code 
|- 
| 2017-08-18||SPL-144193||Bundle validation errors prevent future app deployment to indexer cluster 
|- 
| 2017-08-14||SPL-143981||Uninstall app dialog does not show the app name correctly when the app doesn't have the label 
|- 
| 2017-08-04||SPL-143664||Uploaded apps page makes two calls to packages endpoint 
|- 
| 2017-05-24||SPL-141982||Upload modal should use size=large File element 
|- 
| 2017-04-19||SPL-141274||Clicking Install multiple times in Install dialog causes error 
|- 
| 2017-04-19||SPL-141273||Task endpoint fetch once even when there's no last deploy task id 
|- 
| 2017-03-07||SPL-138351, SPL-172626||The role change of DMC via UI does not reflect to distsearch.conf<br/><br/>Workaround:<br/>As a workaround can the customer manually modify the distsearch.conf. <br/> 
|- 
| 2016-11-14||SPL-132151||XML error when trying to download uninstalled app
|}
== Splunk Web and interface issues ==

{| {{table}}
!Date filed 
!Issue number 
!Description 
|- 
| 2020-02-09||SPL-183078, SPL-181033||_time is shown as GMT on Visualization when Time zone is set to default on Windows<br/><br/>Workaround:<br/>Set up a proper Timezone for a user on Preference.<br/> 
|- 
| 2020-02-09||SPL-183077, SPL-181033||_time is shown as GMT on Visualization when Time zone is set to default on Windows<br/><br/>Workaround:<br/>Set up a proper Timezone for a user on Preference.<br/> 
|- 
| 2019-09-05||SPL-176008, SPL-166728||Alert email spacing issue 
|- 
| 2019-09-05||SPL-176009, SPL-166728||Alert email spacing issue 
|- 
| 2019-07-11||SPL-173061||UI exposes a nonfunctional option for modifying permissions on custom search commands 
|- 
| 2019-05-22||SPL-170857, SPL-162249||The filter function of <splunk-search-dropdown> UI component is not working on on Splunk Enterprise 7.1 and later. 
|- 
| 2019-01-28||SPL-165608, SPL-166627, SPL-172449, SPL-172453||column order is misaligned in alert email compared to the results showing in Splunk. 
|- 
| 2017-08-23||SPL-144350||Archived Index is created without error when the splunk index is invalid 
|- 
| 2017-07-13||SPL-143111||"Splunkd daemon is not responding" when edit local windows event log collection 
|- 
| 2016-11-14||SPL-132133||App Browser filtering of the apps does not work 
|- 
| 2015-11-09||SPL-109165||Interactive Field Extractor hangs when using "^" as delimiter.<br/><br/>Workaround:<br/>Use props and transforms to specify the delimiter of your choice. <br/> 
|- 
| 2015-08-10||SPL-105061, SOLNESS-7274||Broken module prevents splunkweb from starting 
|- 
| 2015-06-30||SPL-103701||Actions links should be removed for "Apps Browser" 
|- 
| 2014-07-16||SPL-87015||chart count by source and *| cluster showcount=t | table cluster_count _raw) no metadata/ result is available when user drills down on Count and Percent columns. 
|- 
| 2014-04-04||SPL-82650||A report created and scheduled by admin cannot be embedded by a power user. 
|- 
| 2014-02-26||SPL-81103||Username surrounded by dollar signs cannot create saved searches. 
|- 
| 2013-11-20||SPL-76798||Time range picker is not customizable via times.conf the same as version 5 or as suggested by docs. 
|- 
| 2013-08-19||SPL-73386||Users are not allowed to run historical scheduled search<br/><br/>Workaround:<br/>1. Create a special power/admin user who can run scheduled searches.

2. Assign this user ownership of the scheduled searches.

3. Share the searches at the app level and grant read/write permission to the correct set of users. <br/>
|}
== Windows-specific issues ==

{| {{table}}
!Date filed 
!Issue number 
!Description 
|- 
| 2019-04-18||SPL-169288, SPL-155149||Registry changes under SYSTEM\CurrentControlSet are not being read by WinRegMon<br/><br/>Workaround:<br/>Monitor SYSTEM\\ControlSet\d+ instead.<br/> 
|- 
| 2015-11-13||SPL-109430||In Windows only, inheritance is broken for folders created by splunkd. Files created are accessible only to the user as whom splunkd is running. 
|- 
| 2015-04-14||SPL-99687, SPL-129637||Splunk universal forwarder is 7-10 days behind recent Windows Security and system log events.<br/><br/>Workaround:<br/>To mitigate this, edit the following stanza in inputs.conf: [WinEventLog://Security] evt_resolve_ad_obj = 0.<br/> 
|- 
| 2015-04-01||SPL-98978||On differing versions of Splunk Enterprise indexer (5.0.1) and universal forwarder (6.2.2), collection of the Security Event log can take increasingly longer over time.<br/><br/>Workaround:<br/>To fix the problem, restart Windows on the forwarder.
<br/> 
|- 
| 2014-09-25||SPL-91279||Splunk Universal Forwarder on Windows (specifically, the splunk-perfmon.exe process) does not release key handles.<br/><br/>Workaround:<br/>See "Handle leak when an application collects performance data in Windows Vista, in Windows 7, in Windows Server 2008 or in Windows Server 2008 R2" on the Microsoft Support website for a hotfix download.<br/> 
|- 
| 2013-10-11||SPL-75116||The UI does not show configured items of some newly converted windows modular inputs that contain the name "default" in the stanza<br/><br/>Workaround:<br/>Edit inputs.conf: in stanzas that contain WinRegMon://default, replace "default" with something else, then restart splunk. <br/>
|}
== REST, Simple XML, and Advanced XML issues ==

{| {{table}}
!Date filed 
!Issue number 
!Description 
|- 
| 2017-07-13||SPL-143111||"Splunkd daemon is not responding" when edit local windows event log collection 
|- 
| 2016-10-31||SPL-131072||Datamodel backend allows invalid time values 
|- 
| 2013-05-15||SPL-67453||When sending the following XML data as a GET or POST param to a custom splunkd endpoint: <dashboard>&lt;foo&gt;</dashboard>, the endpoint actually receives:<dashboard><foo></dashboard>.
|}
== Authentication and authorization issues ==

{| {{table}}
!Date filed 
!Issue number 
!Description 
|- 
| 2018-06-27||SPL-156375, SPL-164872, SPL-166196, SPL-166197||Capability to Schedule Saved Searches restricted after upgrade to 7.x.<br/><br/>Workaround:<br/>edit_search_schedule_window capability needs to be added to the affected role.<br/> 
|- 
| 2018-04-13||SPL-153403||After running the "clean userdata" command, admin is unable to login with msg "No users exist. Please set up a new user."<br/><br/>Workaround:<br/>Create a $SPLUNK_HOME/etc/system/local/user-seed.conf and restart Splunk

<div class="samplecode"> 
[user_info]<br>
PASSWORD = <yourpassword><br>
</div><br/> 
|- 
| 2016-07-26||SPL-125052||Sole Admin can demote his/herself to Power without path of recovery in GUI<br/><br/>Workaround:<br/>Through the command line, you can open notepad and modify the password file to regain 'Admin' status.<br/> 
|- 
| 2015-11-13||SPL-109427||LDAP SSL no longer working in Splunk 6.3 (and later) for Windows 2003<br/><br/>Workaround:<br/>

The workaround is to 
1) obtain Ciphers configured on Windows AD 2003 server. 
2) tweak TLS_CIPHER_SUITE command in etc/openldap/ldap.conf to match it.
The following is a working TLS_CIPHER_SUITE for one of the customers:
{noformat}
TLS_CIPHER_SUITE HIGH:MEDIUM:@STRENGTH:+3DES:+RC4:!aNULL:!MD5:!SRP:!PSK:!aDSS:!kECDH:!kDH:!SEED,!IDEA:!RC2:!RC5
{noformat}
<br/> 
|- 
| 2012-02-22||SPL-48342||LDAP strategy host field cannot work with ipv6 format address but computer name is okay
|}
== PDF issues ==

{| {{table}}
!Date filed 
!Issue number 
!Description 
|- 
| 2016-11-23||SPL-132925||Table data rows generated with the addcoltotals command do not show up in PDF<br/><br/>Workaround:<br/>If you are using <code>addcoltotals</code> to generate a totals data row, renaming the <code>_time</code> field can cause PDF generation issues.
Remove the label and <code>labelfield</code> or change the label to a number to generate the PDF as expected.
<br/> 
|- 
| 2015-03-31||SPL-98890||Maps printed from Report page do not honor custom zoom and center. 
|- 
| 2014-06-16||SPL-85497||Unable to save generated PDFs using Chrome internal PDF viewer.<br/><br/>Workaround:<br/>Workaround: Enable Adobe Acrobat or Acrobat Reader as the default PDF viewer in Chrome. For more information, seehttps://support.google.com/chrome/answer/142056.
<br/>
|}
== Admin and CLI issues ==

{| {{table}}
!Date filed 
!Issue number 
!Description 
|- 
| 2019-09-09||SPL-176179, SPL-159600||Clone dialog in Searches, Reports, and Alerts manager page is listing internal apps as target 
|- 
| 2019-07-18||SPL-173414, SPL-179491, SPL-179612, SPL-179627||Splunk unable to load defined Saved Searches in a conf file if a bad/malformed cron_schedule value is present/set<br/><br/>Workaround:<br/>If this behavior is noticed (savedsearches - alerts/searches/reports) not showing up in the GUI, verify cron jobs are valid, if they're not, then correct them. 

A quick check would be to create a duplicate savedsearches.conf and removing all cron_schedule definitions and rebooting splunk with that config to identify if it is this issue.<br/> 
|- 
| 2019-02-17||SPL-166620, SPL-167267, SPL-175046||btool dumps : CountAccounter::CountAccounter(bool): Assertion `main_thread_created' failed (LDAP) 
|- 
| 2018-11-29||SPL-163320, SPL-162350||Ingest-time log-to-metrics conversion of structured log data is broken for 7.2.x Universal Forwarders 
|- 
| 2018-11-07||SPL-162655, SPL-164505||Add ASNEW keyword to FIELDALIAS to support non-overriding version of aliasing<br/><br/>Workaround:<br/>Avoid applying the same alias field name to multiple original field names.

If you must do this, set it up as a calculated field (an EVAL-* statement) that uses the 'coalesce' function to create a new field that takes the value of one or more existing fields. This method lets you be explicit about ordering of input field values in the case of NULL fields. For example: EVAL-ip = coalesce(clientip,ipaddress)<br/> 
|- 
| 2017-11-29||SPL-146820||Unable to access some settings/manager pages (data model editor) if starting from the setup page of a non-visible app<br/><br/>Workaround:<br/>Navigate to a visible app, such as the search and reporting app, and access the Splunk settings pages from that app context.<br/> 
|- 
| 2017-11-07||SPL-146255||limits.conf enable_clipping cloropleth setting is app/user tunable rather than global like the rest of limits.conf 
|- 
| 2017-04-11||SPL-141051||When LINE_BREAKER is defined for a sourcetype, UI forces SHOULD_LINEMERGE to true<br/><br/>Workaround:<br/>None in Splunk Cloud.
For on-prem, manually edit the props.conf file to set SHOULD_LINEMERGE to 'false'.<br/> 
|- 
| 2017-04-03||SPL-140747||SSL connection in Python when using new ciphers may be slow. 
|- 
| 2016-11-09||SPL-131880||Reports/Alerts owned by the deleted user cannot be found in the Orphaned filter for the Reassign Knowledge Objects page 
|- 
| 2015-09-23||SPL-106978||Failed SHC captain election causes unnecessary change in server.conf 
|- 
| 2015-03-11||SPL-97942||Capability defined in an app does not take effect when assigned to a role<br/><br/>Workaround:<br/>The workaround is to change the ui-prefs in ./etc/users/username/local/ui-prefs.conf to look like this:
[search]                                                                                    
display.events.fields = ["description","except_extract_1","except_extract_2","except_extract_3","sap_order_status","sourcetype","source","status","request_mode","request_id","request_status_id","object_id","BillToCity_","Airline_","BillToName_","BillToCountry_","City_"]
display.events.type = table<br/> 
|- 
| 2014-04-07||SPL-82699||SSO: Acceleration icon fails to display in Searches, Reports, and Alerts page. 
|- 
| 2013-05-25||SPL-68010||The error thrown when your Splunk instance cannot connect to splunkbase/.../checkforupdate is not an ERROR, should be lowered to INFO.<br/><br/>Workaround:<br/>Set server.conf [applicationsManager] allowInternetAccess = false<br/> 
|- 
| 2013-05-02||SPL-66511||If $SPLUNK_HOME/etc is located on a case-insensitive filesystem, creating a new view with the same name as an existing view but with different case (capital letters vs lowercase, etc) silently overwrites the existing view.
|}
== Uncategorized issues ==

{| {{table}}
!Date filed 
!Issue number 
!Description 
|- 
| 2019-11-26||SPL-180195, SPL-177752||Deadlock in splunk when using pstacks action 
|- 
| 2019-10-16||SPL-178057, SPL-174960||(PinkiePie)- Customer Spin Off for code fix - SmartStore - 0 bytes receipt.json Upload 
|- 
| 2019-10-16||SPL-178058, SPL-174960||Customer Spin Off for code fix - SmartStore - 0 bytes receipt.json Upload 
|- 
| 2019-10-01||SPL-177347||Collectd data via HEC blocks Heavy Forwarder queues when 'useACK = true' to send data to Indexers. 
|- 
| 2019-09-10||SPL-176230, SPL-177924, SPL-177925, SPL-177926, SPL-177927, SPL-177928||HealthReporter threads deadlock resulting in stuck _reload, blocked ingestion, eventually causing a crash<br/><br/>Workaround:<br/>You can work around the problem by setting the config full_health_log_interval to a very high value (default being 30s)

In etc/system/local/health.conf:
<pre>
[health_reporter]
full_health_log_interval = 1000000
2019-09-03 SPL-175882 Splunk fails to read file upon log rotation of log4j log file

Workaround:
By restarting the forwarder, the file would get pickup again
2019-07-11 SPL-173038 Deprecated Feature SH Pooling has several functional problems in versions 7.1.x and above

Workaround:
Customers are strongly advised to use Search Head Clustering instead.
2019-06-05 SPL-171553, SPL-171647 Smartstore: S3 GET is being done before S3 PUT for the receipt.json causing 404 errors (Source peer should not check if the bucket/receipt exists during uploads)
2019-05-23 SPL-170998, SPL-169775 "parsing" thread crashing on bad data when it is in a tar.gz2 format
2019-05-23 SPL-170999, SPL-169775 "parsing" thread crashing on bad data when it is in a tar.gz2 format
2019-04-26 SPL-169607, SPL-162658 Editing Summary Indexing not working when Search contains a tstats
2019-04-25 SPL-169562, SPL-170421, SPL-172722, SPL-173212, SPL-173213 EXTRACT with REGEX capture groups are not extracting fields without specifying FORMAT.
2019-04-25 SPL-169589, SPL-169940, SPL-172397 Carriage Returns are added to csv files when users export Windows Eventlog
2019-04-02 SPL-168649, SPL-167902 KV Store migration during Splunk rolling upgrade from version 7.0.X or older to 7.2.X may fail in some cases
2019-03-28 SPL-168459, SPL-176508, SPL-177366, SPL-180432 Splunkd Server Memory growth over time on ES search head
2019-03-13 SPL-167655, SPL-166228 Splunk crashes in _mongoc_openssl_ctx_new on shutdown
2019-03-12 SPL-167631, SPL-171280, SPL-174529 ERROR HttpInputDataHandler - Parsing error : Incorrect index
2019-03-05 SPL-167347, SPL-165968 Frequent searches with outputlookup may trigger highly increased KV Store storage usage or in some cases crash of the mongod process
2019-02-05 SPL-166011, SPL-167057 Field Extractor (IFX) returns no regex 'rules' in http response
2019-01-14 SPL-164837 'enable boot-start' should handle group ownership with systemd

Workaround:
Manually update /etc/systemd/system/Splunkd.service with the correct group after running 'enable boot-start'.
2019-01-14 SPL-164839, SPL-180783, SPL-181017, SPL-181018 systemd warnings about Splunkd.service being executable and world-inaccessible
2019-01-14 SPL-164859, SPL-167178, SPL-167179 Error in 'summaryindex' command: You have insufficient privileges to run this command.
2018-12-04 SPL-163474, SPL-158779 Indexer crash with thread: BatchSearch

Workaround:
2 possible workarounds:

1) On the SHs disable phased execution:

  1. limits.conf
[search]

phased_execution_mode = singlethreaded

OR

2) On the indexers - set batch_search_max_pipeline to default value:

  1. limits.conf

[search] batch_search_max_pipeline = 1

2018-11-16 SPL-162969, SPL-165614, SPL-164661 Splunk upgrade failures from pre 7.2 to 7.2.X due to kv store migration issues
2018-11-01 SPL-162469, SPL-163577, SPL-162764 After upgrade to 7.2 Splunk is unable to start - KVStoreConfigurationThread crash

Workaround:
https://www.suse.com/support/kb/doc/?id=7022289

export LD_LIBRARY_PATH=/lib64/noelision/:$LD_LIBRARY_PATH

2018-10-17 SPL-161632 customer can't install RPM Splunk 7.2 file in Red Hat EL5
2018-09-13 SPL-160029 Downgrading to Splunk Enterprise version 7.1 from version 7.2 causes an error in the mongod.log file.

Workaround:
Before downgrading from Splunk Enterprise version 7.2 to 7.1, resync the KV store with the following command:

curl -u username:password -XPOST https://localhost:8089/services/kvstore/resync/resync?featureCompatibilityVersion=3.4

If you use this command and and then restart Splunk before downgrading, run this command again before downgrading.

2018-09-04 SPL-159598 mongo 3.4 to 3.6 upgrade sometimes misses fcv document
2018-08-28 SPL-159413, SPL-162105, SPL-163566, SPL-162723, SPL-162670, SPL-162676, SPL-162724 "Failed to localize" due to "ERROR CacheManagerHandler"..."not an owner"..."and the bucket is draining"
2018-04-18 SPL-153555, SPL-152283 mongod errors out on distros with older glibc (2.7 and below) with " Invalid access at address: 0x10"
2018-03-20 SPL-152330, SPL-151992 After installing Splunk on Windows using msiexec and the "GENRANDOMPASSWORD=1" option (and if generated password ends with backslash) admin is unable to login with msg "No users exist. Please set up a new user."

Workaround:
Create a $SPLUNK_HOME/etc/system/local/user-seed.conf and restart Splunk

[user_info]
PASSWORD = <yourpassword>


2018-03-14 SPL-152095 Edit Summary Indexing - Index List empty/incomplete for User with Power role after upgrading to 6.6.0+

Workaround:
add indexes_edit and dispatch_rest_to_indexers capability to the Power role for all indexes to be listed
2018-01-25 SPL-148514 Splunk not starting on Linux kernel version 4.13.0-31

Workaround:
Do not upgrade kernel to version 4.13.0-31. Use either an older release or 4.13.0-32.35+
2017-05-09 SPL-141693 DataModel Editor - when child object has same name as inherited field, inherited field does not show in the inherited fields list.
2017-03-27 SPL-140442, SOLNESS-11786 In Splunk Enterprise 6.6.0 and later, with Enterprise Security 4.5.2 and 4.6.0, roles without "edit_roles" capability cannot perform operations on notable event review statuses.

Workaround:
If users cannot perform operations on notable event review statuses or have issues viewing "Edit all selected" links on Incident Review, user roles must be provided with the "edit_roles" capability.
2017-03-14 SPL-138731 New 6.6 and later default SHA256/2048-bit key certificates are not compatible with previous versions SHA1/1024-bit key certificates if cert verification is enabled

Workaround:
Users can do any of the following:

1. Disable certificate verification - the same root certificate is available with every Splunk download so enabling certificate verification while using the default certificates provides very little additional security.

2. Generate new SHA256/2048-bit key certificates using the new 6.6 root certificate and distribute to older versions of Splunk

3. Generate SHA1/1024-bit key certificates using the old root certificate to use with your new 6.6 instance. For convenience, the old root certificate is included in 6.6 in $SPLUNK_HOME/etc/auth/prev_release/

2017-01-18 SPL-135260 Documentation for Search formatting keyboard shortcut for non-English languages
2017-01-06 SPL-134707 Splunk restart does not create missing server.pem certificate on Windows

Workaround:
Use bin/splunk createssl server-cert -d etc/auth/ -n server to generate a new certificate.
2016-11-21 SPL-132670 Mac OS 10.11: disable boot-start doesn't remove the file /Library/LaunchAgents//com.splunk.plist by enabling boot-start in prior Splunk/UF
2016-08-31 SPL-127800 Opting in to data sharing on a monitoring console produces duplicate data.
2016-06-21 SPL-123174 JSON indexed_extractions doesn't work for TCP inputs
2015-10-07 SPL-107606 Inconsistency between summary and datamodel_summary files.
2015-06-18 SPL-103302 Files ownership are failed to be changed when using debian package to install splunk and $SPLUNK_HOME is a symlink

Workaround:
Run a recursive chown from the command line on $SPLUNK_HOME manually, post install.
2015-06-10 SPL-103010 Indexing throughput on a forwarder with four pipelinesets drops 30% compared to a forwarder with two pipelinesets.
2015-05-24 SPL-102008 On Internet Explorer, a warning message does not display when you cannot log in due to a time zone difference.
2015-05-11 SPL-101289 When the number of indexing pipeline sets is greater than four, indexing throughput decreases.
2015-05-06 SPL-100980 Single indexer does not scale when receiving parsed data from multiple PipelineSets.
2015-05-04 SPL-100792 There are multiple group=thruput metrics lines in metrics.log. Searches that do not differentiate among them may get falsely high totals.

Workaround:
Searches that key off these lines need to select their desired name=x category in order to see a single thruput value.
2015-04-24 SPL-100322 A view gets stuck with "loading" due to problematic navigation (default.xml)

Workaround:
Workaround is to use label attribute for collection element.

<collection label="Others">

           <view source="unclassified" match="Dashboard"/>
     </collection>  
2015-03-26 SPL-98700 splunkd Indexer crashes in IndexerTPoolWorker due to duplicated bucket id.

Workaround:
The workaround is to remove the duplicated bucket.
2015-03-25 SPL-98594 Routing events to two different groups not working as expected.

Workaround:
1 On the original UF, instead of configuring 1 s2s and 1 syslog group, configure 2 s2s groups.

2 Setup a proxy UF which takes input from the original UF and send input out syslog server. This solution only requires config change and no patch release is required.

2015-02-26 SPL-97389 When using timechart command, the embedded report shows different time format than the original report.
2015-01-08 SPL-95144, SPL-142789, SPL-101986, SPL-101987, SPL-106884, SPL-107317 Indexed message for Windows security event logs shows "FormatMessage error"

Workaround:
Splunk believes this was introduced in a Microsoft Windows patch. The workaround is to configure a delayed start of the Splunk service(s) so that it starts after the Windows Event Log service.
2014-11-10 SPL-92831 A mismatch of versions between the license-master and the license-slave is generating Warning messages like "WARN LMDirective - directive cmd=D_set_feature_state args='Acceleration,ENABLED' failed: reason='feature='Acceleration' is invalid' ."

Workaround:
The warnings can be ignored, the workaround is use same major versions (all on 6.2 or all on 6.1).


2014-10-31 SPL-92596 After upgrade from Splunk Enterprise 6.1 or earlier to 6.4.x on Windows, splunkweb service does not start automatically. Attempts to start it manually show "Error 1053: The service did not respond to the start or control request in a timely fashion."

Workaround:
This is expected behavior. See the Splunk Answers post: http://answers.splunk.com/answers/177187/why-is-the-splunk-web-service-not-running-after-an.html
2014-10-24 SPL-92432, SPL-99583 Chart in dashboard panel does not honor interval settings.

Workaround:
In the panel XML, specify a larger height to use the correct interval settings.
2014-10-17 SPL-92162 Writing large amounts of data (> 20 GB) to KV store collections using outputlookup can result in high memory usage on the machine.
2014-09-11 SPL-90738 Monitoring a directory with an unknown sourcetype produces indexing errors.
2014-08-26 SPL-90139 <timestamp> does not display in the Patterns tab when searches are run in fast mode.
2014-04-22 SPL-83365 Splunk Enterprise on Windows does not show an error message when a user without the edit_license capability tries to add a license through the CLI.
2014-04-14 SPL-83068 Default index can be set to random index.
2014-04-01 SPL-82517 Paper Size and Layout in PDF Schedule dialog do not respect Paper Size and Layout in Email Settings.
2014-03-23 SPL-82238 Datamodel fails to drill down further when the same attribute for Split Rows and Split Columns are selected.
2014-03-13 SPL-81856 Show all lines does not work in data model editor preview.
2014-03-12 SPL-81810 Licensing - license pool warning at license master keeps coming back after deleting it.

Workaround:
Delete the warnings on the peers first, then the License Manager.
2014-03-12 SPL-81781 In the Data Model Manager, "Acceleration Status" and "Access Count" fail to update when you click "Update".
2014-02-13 SPL-80568 Highcharts determines Y-axis values based on first point outside visible range.
2014-02-07 SPL-80285 In the Data Model Editor, the Edit Lookup page is blank if Lookup is shared only in Lookup Definitions.

Workaround:
For more information, see Add lookup files to Splunk.
2014-02-06 SPL-80187 In the Data Model Editor, lookup pages open with options displayed for other Lookup when the data model definition is private but the file is app or globally shared.

Workaround:
Share the definition. For more information, see Add lookup files to Splunk.
2014-01-31 SPL-79842 On Windows, Indexer doesn't accept new connections on splunktcpin port after queue blockage is resolved
2013-11-27 SPL-77139 Licenser pool usage gets reflected only after restarting splunkd.
2013-10-29 SPL-75764 Forwarder forwards duplicate data after props.conf is in place for cross platform scenario/when the forwarder is on Solaris and the indexer is on Linux.
2013-09-13 SPL-74337, BETA-496 You cannot specify a destination folder when installing on OSX.
2013-09-10 SPL-74209, SPL-74167 Persistent queues are not created on Windows for stanzas that contain unusual characters (such as < and >).

Workaround:
Specify the persistentQueue explicitly in the input definition.
2013-08-28 SPL-73826 Windows: hostname override not working properly
2013-06-13 SPL-69304 If license slaves are running <6.0 version, they do not have the idx field and in theLicense Usage view, the split by index field will show a field named UNKNOWN.
2013-04-30 SPL-66213 PDF server app is not working with latest Xvfb
2010-10-08 SPL-34347 wmi input default fields - with value including newlines doesn't search properly becasue of \r\n issue

Splunk Analytics for Hadoop

Date filed Issue number Description
2019-01-30 ERP-2149, ERP-2159, ERP-2160 HUNK queries to HADOOP HDFS log archive failing with: ChunkedOutputStreamReader: Invalid transport header line=""
2017-04-04 ERP-2040 Splunk archiving fails for large block sizes (buckets) due to HDFS write crashes for Hadoop version 2.8, 2.7.x

Workaround:
Upgrade Hadoop to 2.8.2 or higher.
2015-09-09 ERP-1650 timestamp data type not properly deserialized.
2015-08-05 ERP-1619 Searching on a newly created archive index before the bucket copy saved search is run causes a filenotfound exception.

Workaround:
Reenable the bucket copy saved search and let it run, or force the archiving to happen via | archivebuckets force=1 and then rerun the search.
2015-07-07 ERP-1598 minsplit rampup - splits generation takes too long.

Workaround:
Set minsplits=maxsplits
2015-05-12 ERP-1502 Non-accelerated pivot search on Pivot UI page waits for a long time to return result.
2015-01-08 ERP-1343, SPL-95174 Splunk Analytics for Hadoop searches fail on corrupted journal.gz files, although Splunk searches run without error.

Workaround:
Add the journal.gz to the input path's blacklist (vix.input.1.ignore = ....)
2014-10-27 ERP-1216 Data Explorer preview does not honor existing sourcetypes for big5/sjis files.
2014-10-03 ERP-1164 Report acceleration summary gets deleted when two Splunk Analytics for Hadoop instances point to the same Splunk working directory.

Workaround:
To mitigate this issue, make sure that vix.splunk.home.hdfs (or Working directory in the UI) is unique on both search heads that are not in a pool. To keep your instances in the same working directory, configure vix.splunk.search.cache.path to be unique on both search heads.
Last modified on 06 November, 2020
PREVIOUS
Welcome to Splunk Enterprise 7.2
  NEXT
Splunk Enterprise and anti-virus products

This documentation applies to the following versions of Splunk® Enterprise: 7.2.3


Was this documentation topic helpful?

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters