Splunk® Enterprise

Release Notes

Splunk Enterprise version 7.0 is no longer supported as of October 23, 2019. See the Splunk Software Support Policy for details. For information about upgrading to a supported version, see How to upgrade Splunk Enterprise.
This documentation does not apply to the most recent version of Splunk® Enterprise. For documentation on the most recent version, go to the latest release.

Known issues

The following are issues and workarounds for this version of Splunk Enterprise.

Issues are listed in all relevant sections. Some issues appear more than once. To check for additional security issues related to this release, visit the Splunk Security Portal.

Refer to System requirements in the Installation Manual for a list of supported platforms and architectures.

For a list of deprecated features and platforms, refer to Deprecated features in this manual.

Highlighted issues

Date filed Issue number Description
2018-02-08 SPL-148969, SPL-148600 Indexer may crash during hot bucket rolling following a streaming failure
2017-10-30 SPL-146088, SPL-151973, SPL-151110, SPL-151111 Clustering creates extra copies of buckets erroneously

Workaround:
Use the excess bucket removal functionality at regular intervals.

Upgrade issues

Date filed Issue number Description
2017-05-23 SPL-141961 Older 6.0, 6.1, 6.2, 6.3 maintenance release versions unable to connect to 6.6.x and later via management port.

Workaround:
This applies to License Master/Slave, Deployment Server/Client, Cluster Master/Peers, Search Head/Peers and affects Splunk 6.6.x and the following versions:
  • 6.0.0 to 6.0.6
  • 6.1.0 to 6.1.4
  • 6.2.0 to 6.2.6
  • 6.3.0 to 6.3.1
  • 6.3.1511.1

Upgrade your older instances to the latest maintenance releases or on your 6.6.x Splunk instances. Add the following stanza to server.conf:

[sslConfig]
sslVersions = *,-ssl2
sslVersionsForClient = *,-ssl2
cipherSuite = TLSv1+HIGH:TLSv1.2+HIGH:@STRENGTH


2017-03-20 SPL-139019 Possible compatibility issues between Python / SDK clients and new 6.6 and later default sslVersions, cipherSuites

Workaround:
Users can do either of the following:

1. Overwrite the new Splunk 6.6 server.conf [sslConfig] sslVersions, cipherSuites with your own settings that are compatible with your version of OpenSSL, e.g. the previous defaults from 6.5.x are compatible with OpenSSL 0.9.8 on Mac OSX:

[sslConfig]
sslVersions = *,-ssl2
sslVersionsForClient = *,-ssl2
cipherSuite = TLSv1+HIGH:TLSv1.2+HIGH:@STRENGTH

2. For some more up-to-date clients, it is possible to enforce TLS1.2 (e.g. --tlsv1.2 for curl) in order to connect successfully.

3. Upgrade OpenSSL on your platform and link it with your client (e.g. Python, curl, etc..). For example, OpenSSL 1.0.2 is currently available on Mac OSX via Homebrew (see https://brew.sh) and is compatible with the new Splunk 6.6 default sslVersions, cipherSuites.

2017-03-13 SPL-138647 Possible compatibility issues between new 6.6 and later default sslVersions, cipherSuites and external services, e.g. e-mail, LDAP

Workaround:
If security is not a significant concern, simply revert back to the 6.5.x SSL/TLS defaults, e.g. for e-mail, add to $SPLUNK_HOME/etc/system/local/alert_actions.conf

[email]
sslVersions = *,-ssl2
cipherSuite = TLSv1+HIGH:TLSv1.2+HIGH:@STRENGTH


To configure LDAP with the same settings used by e-mail alerts: $SPLUNK_HOME/etc/openldap/ldap.conf

TLS_PROTOCOL_MIN 3.1
TLS_CIPHER_SUITE TLSv1+HIGH:TLSv1.2+HIGH:@STRENGTH


To completely revert the LDAP configuration to the 6.5.x SSL/TLS defaults, comment out TLS_PROTOCOL_MIN and TLS_CIPHER_SUITE


If you would like to retain the more secure 6.6.x defaults, but prefer to add an exception for your less secure external services, follow the procedure below:

1. To determine what sslVersions and cipherSuites are supported by a server, run splunk cmd openssl s_client -connect hostname:port | awk '/Protocol/ || /Cipher/ || /Verify/'.

The example below is for a Postfix SMTP server:

eserv@indexer01:~$ splunk cmd openssl s_client -connect smtp-server01:465 | awk '/Protocol/ || /Cipher/ || /Verify/'
depth=1 C = US, O = Example Customer, OU = IT, CN = Example Customer IT CA, emailAddress = customer@example.org verify error:num=19:self signed certificate in certificate chain New, TLSv1/SSLv3, Cipher is DHE-RSA-AES256-SHA

   Protocol : TLSv1 
   Cipher : DHE-RSA-AES256-SHA 
   Verify return code: 19 (self signed certificate in certificate chain)

2. Check the OpenSSL output for Protocol and Cipher. In the example above, Protocol = TLSv1 and Cipher = DHE-RSA-AES256-SHA

3. Update Splunk's relevant sslVersions and/or cipherSuite. In the example above, sslVersions should be set to tls (allows TLSv1, TLSv1.1, TLSv1.2) and DHE-RSA-AES256-SHA should be appended to the end of the default cipherSuites definition, e.g. add
$SPLUNK_HOME/etc/system/local/alert_actions.conf
:

[email]
sslVersions = tls

cipherSuites = ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA

2014-08-20 SPL-89640 When running Splunk on Linux as non-root user and using RPM to upgrade, the RPM writes $SPLUNK_HOME/var/log/introspection as root, causing errors upon restarts

Workaround:
Chown the $SPLUNK_HOME/var/log/introspection directory to the user Splunk Enterprise runs as after upgrading and before restarting Splunk Enterprise.
2013-08-19 SPL-73386 Users are not allowed to run historical scheduled search

Workaround:
1. Create a special power/admin user who can run scheduled searches.

2. Assign this user ownership of the scheduled searches.

3. Share the searches at the app level and grant read/write permission to the correct set of users.

Data input issues

Date filed Issue number Description
2019-11-04 SPL-178912, SPL-171961 The datetime.xml timestamp recognition file does not recognize two-year dates after 2019 or Unix epoch-time seconds higher than 1599999999 (12:26:39 UTC 13 Sep 2020)
2018-07-15 SPL-157319, SPL-156315 After upgrade to 7.x, HEC events greater than 512KB are dropped with parsing errors, resulting in degrade of indexing throughput
2018-07-05 SPL-156817 HEC json file give "Invalid data format" on 7.x versions with event sizes greater than 512kb
2018-04-19 SPL-153591, SPL-155066, SPL-155067, SPL-155069 high delay on events from UF after upgrade to (6.6.x)
2018-04-17 SPL-153517, SPL-148346 UF ignores files that are currently being written to
2018-04-09 SPL-153180, SPL-148346 UF ignores files that are currently being written to
2018-03-27 SPL-152628 PREAMBLE_REGEX doesn't work on 7.0.2 but OK with 7.0.0
2018-02-08 SPL-148978, SPL-114085 When a *.tgz file was read, the result was "finished reading" but the percent still showing 0%.
2017-12-28 SPL-147638, SPL-157922, SPL-157923 Splunkd crashes when HEC inputs configuration contains duplicated tokens
2017-09-11 SPL-144797, SPL-133461 Compressed files are deleted from sinkhole even if decompression fails
2017-07-19 SPL-143236 Custom sourcetype is not displayed on sourcetype menu

Workaround:
Set a filter and the sourcetype will display.
2015-11-12 SPL-109362 When the disk runs out of space for the limit set in the server.conf, add data workflow gets stuck with "Uploading file" message modal in the review stage
2015-05-22 SPL-101981 Field extractions do not work when sourcetypes use quotes in the Getting Data In interface.
2015-03-17 SPL-98163 INDEXED_EXTRACTIONS=W3C is truncating field cs_uri_stem when spaces are present in URL

Workaround:
Create a separate extraction in props.conf where defined w3c extraction method:

EXTRACT-cs_uri_stem1 = (GET|POST) (?<cs_uri_stem1>[^-]++)

Search issues

Date filed Issue number Description
2019-07-08 SPL-172836, SPL-171270 dedup's sortby not working as expected when using head/transaction
2018-12-18 SPL-164112 Characters with accents not substituting properly with sed mode
2018-07-23 SPL-157725, SPL-144000 Can't search for indexed fields included in summary index since fields.conf "INDEXED = true" since 6.6

Workaround:
You could rename the field before summary indexing to use a different field name in the summary indexes (and then alias it back?).
2018-07-23 SPL-157727, SPL-144000 Can't search for indexed fields included in summary index since fields.conf "INDEXED = true" since 6.6

Workaround:
You could rename the field before summary indexing to use a different field name in the summary indexes (and then alias it back?).
2018-07-03 SPL-156712, SPL-148606 Inconsistent Search Results Against _audit Index.
2018-06-22 SPL-156141, SPL-146147 Search crashes when using lookup tables that are frequently updated

Workaround:
On the crashing peer (could be SH, Indexer or both) set the below in limits.conf:

max_memtable_bytes = 2*<size of the largest lookup>

example search to find the biggest lookups:

index=_* sourcetype=audittrail path=*lookups* size=* | stats max(size) AS size BY host, path | append [| rest services/server/introspection/kvstore/collectionstats | mvexpand data | table splunk_server title data | spath input=data | fields splunk_server size ns ] | eval host=coalesce(host,splunk_server) | fields host path ns size | sort size | head 1

2018-06-04 SPL-155106, SPL-155412, SPL-155413 splunkd process consuming large amount of memory in 7
2018-04-17 SPL-153464, SPL-157516, SPL-158568, SPL-158570 Job Progress Status goes from 0 to 100 back to 0
2018-04-12 SPL-153349, SPL-154301, SPL-154302, SPL-154303 Scheduling Alerts - Apply Time Range Of Initial Search Not Reflecting when Saving as Alert
2018-03-27 SPL-152616 Searches, Reports, and Alerts page - Unable to filter with savedsearches with ">" OR "<" in the name

Workaround:
Enclose the savedsearch name in quotes.
2018-02-20 SPL-149404, SPL-153486, SPL-157068, SPL-160449 Search.log error message asks user to consider increasing match limit for a Regex without a reason
2018-01-10 SPL-148042, SPL-148047, SPL-148048, SPL-148049 datamodel command flat search does not work properly
2018-01-02 SPL-147702, SPL-148925, SPL-151635 Vertical tab and 255 other characters in a savedsearches.conf can break the savedsearches REST endpoint
2017-12-12 SPL-147249 Inputlookup for lookup with space in the filename fails with "Invalid argument: ..." with search optimization enabled

Workaround:
Don't use spaces in your inputlookup filename
2017-10-17 SPL-145727, SPL-146668 append search command causes search parser to incorrectly identify a circular dependency and reject the search string

Workaround:
Append "| noop search_optimization=false" to turn off search optimization:

| inputlookup test1.csv | append [| inputlookup test2.csv] | table f1 | append [| inputlookup test1.csv | append [| inputlookup test2.csv] | table f1] | noop search_optimization=false

2017-10-11 SPL-145602, SPL-148349, SPL-153732 REGEX flag (?J) "duplicate group names" causes splunk to crash
2017-09-27 SPL-145252, SPL-146174, SPL-146175 Column sorting does not work on search and report page if the field contains a whitespace
2017-09-03 SPL-144601, SPL-142754 When limits are enforced and a new search request would be over the limit, the server returns the wrong HTTP response code of 500 — should be 503.
2017-08-23 SPL-144350 Archived Index is created without error when the splunk index is invalid
2017-08-03 SPL-143607 Searches ordered like this returns false results: "search ... | eventstats count | delta _time as d" , because it's being run in batch mode when it shouldn't

Workaround:
Place the delta command before eventstats in the search pipeline:

... | delta _time as d | eventstats count

2017-07-13 SPL-143111 "Splunkd daemon is not responding" when edit local windows event log collection
2017-04-04 SPL-140765 Splunk having problems extracting json file consisting of 68k plus key-value pairs
2017-03-21 SPL-140175 Aborted delete searches may result in stale lock files being left behind

Workaround:
Delete stale lock files.
2016-11-29 SPL-133182 When two datasets have identical names but one is local (private) while the other is global, attempts to view or extend the global dataset use results from the local dataset instead.
2016-06-17 SPL-122984 Searching renamed sourcetype is case-sensitive
2015-08-10 SPL-105061, SOLNESS-7274 Broken module prevents splunkweb from starting
2015-06-17 SPL-103247 Filtering on _time uses different semantics for the "=" operator on microseconds depending on whether the value is quoted.
2015-04-23 SPL-100170 Automatic Lookups limitation: No results returned in Smart Mode when there are nested lookups and the intermediate field is not mentioned in the search.
2014-12-22 SPL-94910 The replace function does not apply to fields names with an underscore in them.

Workaround:
Rename the fields before the replace.

... | rename *_* AS *-* | replace "something" by "somethingelse"

2014-11-13 SPL-93039 The relevancy search command does not work, always returning 0 or -inf.
2014-10-02 SPL-91638, SPL-107375 For scheduled searches in a search head cluster, empty search jobs may appear in the job inspector for a cluster member.
2014-09-15 SPL-90861, SPL-90396, SPL-90886 If search encounters invalid offsets or invalid rawdata at TSIDX offsets, it skips reading any number of events from that bucket. No message is displayed, though the information is added to search.log.
2014-04-16 SPL-83129 Eval function strptime does not return results when 1970 date is used.
2014-04-04 SPL-82650 A report created and scheduled by admin cannot be embedded by a power user.
2014-03-27 SPL-82357 The splunk clean all -f CLI command doesn't remove data from the main index on Windows systems.
2014-03-15 SPL-81934 For clusters, may be unable to open search results output file for search results in a cluster.

Workaround:
Write to a temp file and rename to the target file.
2014-02-21 SPL-80942 Flashtimeline: 500 Internal Server Error when pasting long URL into panel name.
2013-12-18 SPL-78179 REST /saved/searches App names with special characters have invalid links.
2013-08-19 SPL-73386 Users are not allowed to run historical scheduled search

Workaround:
1. Create a special power/admin user who can run scheduled searches.

2. Assign this user ownership of the scheduled searches.

3. Share the searches at the app level and grant read/write permission to the correct set of users.

Saved search, alerting, scheduling, and job management issues

Date filed Issue number Description
2018-09-04 SPL-159604, SPL-159053 Trigger Time format in alert emails without AM/PM designators and no Timezone information
2018-09-04 SPL-159602, SPL-159053 Trigger Time format in alert emails without AM/PM designators and no Timezone information.
2018-05-08 SPL-154302, SPL-153349 Scheduling Alerts - Apply Time Range Of Initial Search Not Reflecting when Saving as Alert
2018-04-21 SPL-153649, SPL-156991, SPL-157792, SPL-157793 Search scheduler shifts earliest_time and latest_time based on the skew, when using allow_skew

Workaround:
Don't use allow_skew for searches where this behaviour is a problem.
2018-01-10 SPL-148009, SPL-128919 The returning of sendalert command doesn't honor owner options
2017-12-13 SPL-147319, SPL-154403, SPL-154405 SHC AuthenticationManagerLDAP complains "Could not find user="system"" flooding splunkd.log

Workaround:
+ creating a local user called "system" would clear the INFO logging

+ or customer can turn off INFO logging by setting logging level to NOTICE or above: splunk set log-level AuthenticationManagerLDAP -level NOTICE

2017-11-29 SPL-146802 Distributed environment requires index defined on search head for log event alerts
2017-10-31 SPL-146104, SPL-143337 Possible false logging? -- reason="The maximum number of concurrent real-time scheduled searches on this cluster has been reached" concurrency_limit=1
2015-11-15 SPL-109471 For Real Time Scheduled Search in search head cluster, alerts are triggered twice when members cannot HB to captain
2015-04-09 SPL-99421 Long name of app causes accelerated search to not complete normally and shows invalid results on Windows 2008 R2

Workaround:
Reduce length of name of the app and report acceleration searches will run properly within the context of the app.
2014-08-15 SPL-89332 Report acceleration summaries do not show in Settings when you have hundreds of reports accelerated.
2014-08-05 SPL-88396 After configuring a client name for a deployment client, the name is not shown in the Forwarder Management UI

Workaround:
Create a server class, where you can see the client name, and use that group when you add data.
2014-05-01 SPL-83686 Data Model Pivot: Extra NULL column displays in Pivot with big data and Numbered Attribute in Split Columns.

Workaround:
The workaround is to add filter status=*, or make a more refined Data Model that has an object for events with status.
2014-03-24 SPL-82262, SPL-82241 Pivot search command fails for an admin trying to pivot on a Private Data Model created by a User.
2014-03-20 SPL-82164 Migrating invalid data models from 6.0 to 6.x fails.
2014-03-19 SPL-82133 Data model allows users to upload a JSON file which has Field names with spaces but will not validate it.
2014-03-10 SPL-81637 Splunkd preview runs indefinitely on any file preview with "DATETIME_CONFIG=none".
2014-03-10 SPL-81645 Creating data model with root transaction name starting with root event name fails
2013-11-26 SPL-77054, SPL-77055 Data model objects that have names starting with an underscore character ("_") do not work correctly and cannot be used in Pivot.

Charting, reporting, and visualization issues

Date filed Issue number Description
2018-07-27 SPL-158006, SPL-156227 Dashboard drilldown link to search is not displaying entire search string
2018-02-06 SPL-148880 Bar chart is not displayed correctly when data filtered down to a single bar
2017-10-03 SPL-145375, SPL-145145 Custom visualization cannot be saved in dashboard.
2016-09-15 SPL-128819, SPL-130243, SPL-130245 Editing panel in dashboard removes charting.legend.masterlegend option

Workaround:
Use <option name="charting.legend.masterLegend">null</option>
2016-04-27 SPL-118911 In SHC, referenced saved real-time searches in a dashboard do not stream results.

Workaround:
See Troubleshoot referenced real-time searches for workaround details.


2015-02-23 SPL-97193 The initial value for Multiselect input does not display properly in Visualizations Editor if input has empty string.

Distributed search and search head clustering issues

Date filed Issue number Description
2019-10-25 SPL-178412, SPL-155281 Indexer Clustering Search Performance - search manifest updates should be locked per site+genid
2019-10-18 SPL-178171, SPL-160828 DistributedPeerManager::handleConflicts needs to be improved
2019-09-06 SPL-176036, SPL-160828 DistributedPeerManager::handleConflicts needs to be improved
2018-12-14 SPL-164011, SPL-164677, SPL-164731, SPL-164732 SHC: when captain node is in AutomaticDetention status, all alerts (scheduled searches) appear to have stopped as well.
2018-10-26 SPL-162318, SPL-162906, SPL-163487, SPL-163488, SPL-163751 DispatchReaper fails to reap artifacts from fill_summary_index.py in SH Cluster

Workaround:
** Manual deletion **

To get sidlist.txt . splunk search '|REST /services/search/jobs label=searchname | table sid' --maxout 0 --preview

To delete dispatch dirs. for i in `cat sidlist.txt` do rm -rf $i done

2018-05-23 SPL-154829, SPL-141363 Indexers report "Unknown search command" for external search commands even though the indexers contain the search bundle with the external command

Workaround:
Use any of the following 3 workarounds:

1. Transform the "| command" part of the search into "| script command" 2. Transform the "| command" part of the search into "| localop | command" 3. Distribute the app to the indexers via the CM.

2018-05-01 SPL-154032, SPL-154067, SPL-154926, SPL-156192 SHC bundle rejected at push-time because of built-in apps warning is still created and picked up by SHC members

Workaround:
* Remove the bundle on SHC deployer, e.g. $SPLUNK_HOME/var/run/splunk/deploy/apps/search-0f00e250ca395564de84b53b3ae644617d2d3860.bundle
  • If the bad bundle was deployed to shcluster members, then apply shcluster-bundle on the deployer.
2018-04-03 SPL-152935, SPL-154616, SPL-154617, SPL-154618 KVStore Replication Error: replSetReconfig got BadValue _id field value of 256 is out of range
2018-01-12 SPL-148106, SPL-153831, SPL-153832, SPL-153833 Crashing thread: TcpChannelThread, Assertion `_slave != __null ClusteringMgr::_slave_writeBucketsToSearch.

Workaround:
The workaround for this crash for the time being is to make sure no searches are run directly on the cluster peers, including scheduled searches.

Or

Remove distsearch.conf definitions from cluster peers that point to search heads.

2017-11-29 SPL-146802 Distributed environment requires index defined on search head for log event alerts
2017-03-13 SPL-138654 Splunk searches fail when filepath gets too long on Windows
2016-07-12 SPL-124085 On Search Head Cluster It is not possible to remove an App from the SHs once it has been disabled.
2015-11-15 SPL-109471 For Real Time Scheduled Search in search head cluster, alerts are triggered twice when members cannot HB to captain
2015-09-23 SPL-106978 Failed SHC captain election causes unnecessary change in server.conf
2015-02-26 SPL-97385 $SPLUNK_HOME/var/run/splunk/snapshot contains large tarballs in the presence of large ES lookup table files.

Workaround:
The allowable size of the download can be increased by setting the following in server.conf.

[httpServer] max_content_length = 1500MB

The other option is to disable the search which controls the generation of the large lookup file. In this case, the search is:

[Endpoint - Local Processes Tracker - Lookup Gen]

2014-08-25 SPL-90028 Using "inputcsv dispatch=true" to read a CSV from a dispatch directory may not work on search head cluster members that have a replica of the desired artifact.
2014-08-14 SPL-89131 In a search head cluster, the search Job management page on cluster member doesn't immediately reflect 'isSaved' state after you click Save.
2014-08-02 SPL-88228 When user clicks on the RSS feed for an alert, search pool information is not displayed. Individual pool member information is displayed, however.

Data model and pivot issues

Date filed Issue number Description
2018-08-06 SPL-158340, SPL-152600 Save the pivot table as a Report or Dashboard: Pivot Table Error - Error in PivotRowCol
2018-06-25 SPL-156254, SPL-152600 Save the pivot table as a Report or Dashboard: Pivot Table Error - Error in PivotRowCol
2017-12-13 SPL-147319, SPL-154403, SPL-154405 SHC AuthenticationManagerLDAP complains "Could not find user="system"" flooding splunkd.log

Workaround:
+ creating a local user called "system" would clear the INFO logging

+ or customer can turn off INFO logging by setting logging level to NOTICE or above: splunk set log-level AuthenticationManagerLDAP -level NOTICE

2014-12-08 SPL-94047, SPL-98628 While creating a Pivot and using the _time column as a Split column, the table columns aren't formatted in a human readable way, but displayed with the epoc timestamp.It works when using _time as a 'Split Row' column.
2014-05-01 SPL-83686 Data Model Pivot: Extra NULL column displays in Pivot with big data and Numbered Attribute in Split Columns.

Workaround:
The workaround is to add filter status=*, or make a more refined Data Model that has an object for events with status.
2014-03-24 SPL-82262, SPL-82241 Pivot search command fails for an admin trying to pivot on a Private Data Model created by a User.
2014-03-20 SPL-82164 Migrating invalid data models from 6.0 to 6.x fails.
2014-03-19 SPL-82133 Data model allows users to upload a JSON file which has Field names with spaces but will not validate it.
2014-03-11 SPL-81701 Data Model Pivot, "Legend Position" and "Stack Mode" change to default settings if you change the X/Y-Axis more than once.
2014-03-10 SPL-81645 Creating data model with root transaction name starting with root event name fails
2014-03-07 SPL-81538 When using Pivot, stack mode is lost when "Scatter Chart" is selected.
2013-11-26 SPL-77054, SPL-77055 Data model objects that have names starting with an underscore character ("_") do not work correctly and cannot be used in Pivot.

Indexer and indexer clustering issues

Date filed Issue number Description
2019-10-25 SPL-178413, SPL-155281 Indexer Clustering Search Performance - search manifest updates should be locked per site+genid
2019-10-25 SPL-178414, SPL-155281, SPL-179523 Indexer Clustering Search Performance - search manifest updates should be locked per site+genid
2019-10-25 SPL-178412, SPL-155281 Indexer Clustering Search Performance - search manifest updates should be locked per site+genid
2019-05-29 SPL-171257, SPL-171303 Index Cluster Bundle Status stuck in "Bundle validation is in progress"

Workaround:
If the CM cluster-bundle-status gets stuck indefinitely in "Bundle validation is in progress"

1.) cancel the bundle push operation curl -k -u admin:pass https://host:mPort/services/cluster/master/control/default/cancel_bundle_push -X POST 2.) rollback to previous bundle or push a new bundle rollback: curl -k -u admin:pass https://host:mPort/services/cluster/master/control/default/rollback -X POST push bundle: curl -k -u admin:pass https://host:mPort/services/cluster/master/control/default/apply -X POST 3.) Restart the CM or peer if the above does not result in all peers on the same bundle


Also consider lowering the number of peers that download the bundle from the CM simultaneously: By default, the master pushes the bundle to all peers simultaneously. The max_peers_to_download_bundle setting in server.conf provides a means to limit the number of peers that receive the bundle simultaneously.

ie: on CM server.conf [clustering] max_peers_to_download_bundle = 3

2018-11-12 SPL-162802, SPL-161301 For a multisite cluster, splunk is not reaping prior search-buckets manifests after new generation

Workaround:
Do manual cleanup of

$SPLUNK_HOME/var/run/splunk/cluster/search-buckets leaving the gen0 and 10 of the latest files per site as minimum

To automate this you can do something like this in cron once you're happy with the manual run, you just need to add the delete flag for find:

find $SPLUNK_HOME/var/run/splunk/cluster/search-buckets -regextype posix-extended -regex '.+_gen([0-9]{2,}|[1-9])\.csv\.gz' -mtime +2


2018-10-23 SPL-161815 Thawed buckets in a indexer cluster are sporadically unsearchable upon restart
2018-04-05 SPL-153051, SPL-152821 Contention on DatabaseManager::_mux and CMIndexId mutex impacting search performance and indexer cluster stability.
2018-03-27 SPL-152596 peers frequently marked Down by Cluster Master after upgrade to 7.x
2018-03-22 SPL-152465, SPL-153596, SPL-153597, SPL-154595, SPL-154647, SPL-154648 Clustering - when a peer is in detention, we will make excess copies

Workaround:
If any indexers are in detention run `splunk remove excess-buckets` periodically.
2018-03-15 SPL-152212, SPL-144824 Replicating a bucket to an index that isReadOnly=true causes DatabaseDirectoryManager assertion "addReplicatedBucket request for readonly instance" resulting in target crash

Workaround:
Don't try to replicate the read-only index by adding "repFactor = 0" to indexes.conf for the index(es) in question.

[<indexname>] ... isReadOnly = true repFactor = 0

2018-02-27 SPL-151296, SPL-148100 Clustering : unnecessary file IO reads when spawning a search process
2018-02-23 SPL-151111, SPL-146088, SPL-151374 Clustering creates extra copies of buckets erroneously.
2018-02-08 SPL-148969, SPL-148600 Indexer may crash during hot bucket rolling following a streaming failure
2018-01-12 SPL-148121, SPL-148204, SPL-148362, SPL-114215 Incorrect value for generation_poll_interval in spec file
2017-11-24 SPL-146685, SPL-146214 Search returns the following error "Could not read event: cd=(n/a). Results may be incomplete ! (logging only the first such error; enable DEBUG to see the rest)"
2017-11-20 SPL-146575, SPL-154353, SPL-147996, SPL-154305, SPL-154354 RF and SF not being met on CM after adding new Indexes and rolling restart
2017-11-09 SPL-146335, SPL-151811, SPL-151813 DispatchReaper not cleaning up remote-bundle files on CM
2017-10-30 SPL-146088, SPL-151973, SPL-151110, SPL-151111 Clustering creates extra copies of buckets erroneously

Workaround:
Use the excess bucket removal functionality at regular intervals.
2017-10-26 SPL-145961, SPL-144862 Make cluster stop attempting to replicate thawed buckets but keep them searchable
2017-08-29 SPL-144482, SPL-143402 Fsck processes are stuck leading to fixup tasks not completing .
2017-03-16 SPL-138846 In multisite clustering, deletion of events in hot buckets is not pushed to other sites
2016-08-25 SPL-127353 Data rebalance finishes early when one peer is the source for all buckets

Workaround:
when only one indexer in a cluster indexed data (has all the searchable copies), rebalance once before adding the new indexer, and then rebalance a second time
2015-05-08 SPL-101184 Rolling restart in an Indexer Cluster may not be successful on a peer if a oneshot command is also running on that peer. Perform a manual restart to revive the peer.
2014-10-13 SPL-91861 On Windows indexer on an ec2 instance, splunk-optimize main thread can crash on buckets on the temporary drive z:\>.
2014-09-29 SPL-91432 On Windows when the master is down, the CLI command splunk offlinehangs when run from one of the streaming target peers.
2014-09-08 SPL-90630 On a multisite cluster, no warning is given when search head names are the same.
2014-07-29 SPL-87816 When implementing an indexer cluster or search head cluster, you cannot set pass4SymmKey in the general stanza. The system default values in the clustering and shclustering stanzas override any user-provided values in the general stanza.

Workaround:
Set the value in the [clustering] or [shclustering] stanza, depending on the type of cluster you're implementing.
2014-07-14 SPL-86799 After adding a new license to the clustering search head, splunkd on restart cannot be reached by splunkweb.
2014-04-29 SPL-83636 When configuring a multi-site cluster using cluster-config, the error messages are incorrect if the SF/RF was previously set.
2014-03-18 SPL-82038 Cluster-config does not work if a parameter value includes a space character.
2014-03-17 SPL-81955 Multisite: Peer takes approximately 6 minutes to restart when its site configuration is changed.
2014-01-06 SPL-78688 Peer is able to change to an invalid (empty) replication port
2013-08-06 SPL-72484 You cannot use the CLI to delete an index with a capital letter in its name.

Universal forwarder issues

Date filed Issue number Description
2021-08-16 SPL-210384, SPL-211917 Rolling restart causes forwarders to block
2019-01-28 SPL-165635, SPL-191773, SPL-189789 splunk not reading file after log rotation
2018-08-27 SPL-159337, SPL-163271, SPL-163272, SPL-163273 Splunk UF crashing due to invalid EVENT_BREAKER
2018-07-02 SPL-156698, SPL-158816, SPL-160530, SPL-160531 splunk-netmon consumes additional 2GB memory every day on Universal Forwarder.
2018-04-10 SPL-153251 Universal Forwarder txz package cannot be installed on FreeBSD 11.1

Workaround:
1. Use pkg install instead of pkg add

OR 2. Install package by untarring tgz file to /opt/splunkforwarder

2018-03-15 SPL-152201, SPL-144080 Splunk Forwarder crashes if EVENT_BREAKER_ENABLE is specified for a WMI input
2017-12-04 SPL-146940, SPL-147898, SPL-147899, SPL-148483, SPL-154028, SPL-154029 TcpOutputProc randomly drops indexers from the server list
2017-05-23 SPL-141961 Older 6.0, 6.1, 6.2, 6.3 maintenance release versions unable to connect to 6.6.x and later via management port.

Workaround:
This applies to License Master/Slave, Deployment Server/Client, Cluster Master/Peers, Search Head/Peers and affects Splunk 6.6.x and the following versions:
  • 6.0.0 to 6.0.6
  • 6.1.0 to 6.1.4
  • 6.2.0 to 6.2.6
  • 6.3.0 to 6.3.1
  • 6.3.1511.1

Upgrade your older instances to the latest maintenance releases or on your 6.6.x Splunk instances. Add the following stanza to server.conf:

[sslConfig]
sslVersions = *,-ssl2
sslVersionsForClient = *,-ssl2
cipherSuite = TLSv1+HIGH:TLSv1.2+HIGH:@STRENGTH


2017-03-20 SPL-139019 Possible compatibility issues between Python / SDK clients and new 6.6 and later default sslVersions, cipherSuites

Workaround:
Users can do either of the following:

1. Overwrite the new Splunk 6.6 server.conf [sslConfig] sslVersions, cipherSuites with your own settings that are compatible with your version of OpenSSL, e.g. the previous defaults from 6.5.x are compatible with OpenSSL 0.9.8 on Mac OSX:

[sslConfig]
sslVersions = *,-ssl2
sslVersionsForClient = *,-ssl2
cipherSuite = TLSv1+HIGH:TLSv1.2+HIGH:@STRENGTH

2. For some more up-to-date clients, it is possible to enforce TLS1.2 (e.g. --tlsv1.2 for curl) in order to connect successfully.

3. Upgrade OpenSSL on your platform and link it with your client (e.g. Python, curl, etc..). For example, OpenSSL 1.0.2 is currently available on Mac OSX via Homebrew (see https://brew.sh) and is compatible with the new Splunk 6.6 default sslVersions, cipherSuites.

2017-03-14 SPL-138731 New 6.6 and later default SHA256/2048-bit key certificates are not compatible with previous versions SHA1/1024-bit key certificates if cert verification is enabled

Workaround:
Users can do any of the following:

1. Disable certificate verification - the same root certificate is available with every Splunk download so enabling certificate verification while using the default certificates provides very little additional security.

2. Generate new SHA256/2048-bit key certificates using the new 6.6 root certificate and distribute to older versions of Splunk

3. Generate SHA1/1024-bit key certificates using the old root certificate to use with your new 6.6 instance. For convenience, the old root certificate is included in 6.6 in $SPLUNK_HOME/etc/auth/prev_release/

2015-06-10 SPL-103010 Indexing throughput on a forwarder with four pipelinesets drops 30% compared to a forwarder with two pipelinesets.
2015-04-14 SPL-99687, SPL-129637 Splunk universal forwarder is 7-10 days behind recent Windows Security and system log events.

Workaround:
To mitigate this, edit the following stanza in inputs.conf: [WinEventLog://Security] evt_resolve_ad_obj = 0.
2015-04-07 SPL-99316 Universal Forwarders stop sending data repeatedly throughout the day

Workaround:
In limits.conf, try changing file_tracking_db_threshold_mb in the [inputproc] stanza to a lower value.
2015-03-25 SPL-98594 Routing events to two different groups not working as expected.

Workaround:
1 On the original UF, instead of configuring 1 s2s and 1 syslog group, configure 2 s2s groups.

2 Setup a proxy UF which takes input from the original UF and send input out syslog server. This solution only requires config change and no patch release is required.

2014-08-05 SPL-88396 After configuring a client name for a deployment client, the name is not shown in the Forwarder Management UI

Workaround:
Create a server class, where you can see the client name, and use that group when you add data.
2013-09-18 SPL-74427, SPL-74448 The Splunk universal forwarder installer for Solaris 10 does not add the splunk user when you attempt to install it using the pkgadd command. This results in the script generating lots of errors.

Workaround:
To work around this issue, create a splunk user on your system before attempting to run the installer.

Distributed deployment, forwarder, deployment server issues

Date filed Issue number Description
2018-10-08 SPL-161043, SPL-141772 App deployment fails sporadically on Windows
2018-04-11 SPL-153261, SPL-155010, SPL-155009 Slow Performance in the Deployment Server UI and sometime crash the browser
2018-02-05 SPL-148851, SPL-151413, SPL-154007, SPL-154008 Application bundle cache (by default under $SPLUNK_HOME/var/run/tmp/) *never* gets cleaned up on Deployment server even server class no longer exists

Workaround:
manually delete the no longer existing serverclass cache.
2014-10-02 SPL-91648, SPL-91358 Forwarder unable to push scripted inputs to a Linux deployment client from a Windows deployment server.
2014-08-15 SPL-89333 Using client filtering in forwarder management interface when the deployment server is servicing a large numbers of deployment clients (over approximately 5000) can cause a temporary spike in memory usage.
2014-06-20 SPL-85739 When running a high number of deployment clients for a server, memory growth may be excessive.

Workaround:
To mitigate this, set forceHttp10=always.

Monitoring Console issues

Date filed Issue number Description
2019-01-24 SPL-165397, SPL-160335 No custom checklist item examples in checklist.conf.spec
2019-01-23 SPL-165338, SPL-160335 No custom checklist item examples in checklist.conf.spec
2017-11-07 SPL-146244, SPL-146097 Typo in split by dropdown of Monitoring Console's License usage dashboard
2017-08-31 SPL-144555, SPL-146585 App "Set up" links are missing on Splunk Cloud with DMC
2017-08-18 SPL-144193 Bundle validation errors prevent future app deployment to indexer cluster
2017-08-14 SPL-143981 Uninstall app dialog does not show the app name correctly when the app doesn't have the label
2017-08-04 SPL-143664 Uploaded apps page makes two calls to packages endpoint
2017-05-24 SPL-141982 Upload modal should use size=large File element
2017-04-19 SPL-141274 Clicking Install multiple times in Install dialog causes error
2017-04-19 SPL-141273 Task endpoint fetch once even when there's no last deploy task id
2017-03-30 SPL-140654, SPL-178056 wrong integrity check alert for file etc/users/users.ini
2017-03-07 SPL-138351, SPL-172626 The role change of DMC via UI does not reflect to distsearch.conf

Workaround:
As a workaround can the customer manually modify the distsearch.conf.
2016-11-14 SPL-132151 XML error when trying to download uninstalled app

Splunk Web and interface issues

Date filed Issue number Description
2019-07-11 SPL-173061 UI exposes a nonfunctional option for modifying permissions on custom search commands
2019-01-22 SPL-165253, SPL-166047, SPL-166776, SPL-166777 Using "%" in dashboard XML can cause infinite 'Loading...' loop for dashboards with no error reported.

Workaround:
Do manual URI encoding. For example, this would load just fine:


<html>

This is the <a href="http://%25%25problem%25%25/index.html">problem.</a>

</html>


2018-10-16 SPL-161441, SPL-145546, CAUTO-1588 When assigning indexes to roles, indexes defined on the indexer tier are not displayed

Workaround:
Replace the "$SPLUNK_HOME/etc/apps/search/default/data/ui/manager/authentication_roles.xml" file on the search head with a version from any Splunk Enterprise 6.6.x release. Refresh the configuration on the search head by calling a debug refresh (http[s]://[splunkweb hostname]:[splunkweb port]/debug/refresh) using a supported web browser.

2018-07-16 SPL-157354, SPL-152481 Setting "tools.sessions.forceSecure = True" in web.conf doesn't set the secure flag on session_id_* cookies
2018-04-12 SPL-153349, SPL-154301, SPL-154302, SPL-154303 Scheduling Alerts - Apply Time Range Of Initial Search Not Reflecting when Saving as Alert
2018-03-27 SPL-152616 Searches, Reports, and Alerts page - Unable to filter with savedsearches with ">" OR "<" in the name

Workaround:
Enclose the savedsearch name in quotes.
2018-01-29 SPL-148609, SPL-139017 The messages.po file contains French translations of css object when it shouldn't

Workaround:
The workaround is

==

Backup the files

  • cp $SPLUNK_HOME/lib/python2.7/site-packages/splunk/appserver/mrsparkle/locale/fr_FR/LC_MESSAGES/messages.po <somewhere_outside_splunk_fielsystem>

Copy the manifest file yours may be called something different but in ends in *-manifest.

  • cp $SPLUNK_HOME/splunk-6.5.2-67571ef4b87d-darwin-64-manifest <somewhere_outside_splunk_fielsystem>

Step 2

==

Make your edits to the $SPLUNK_HOME/lib/python2.7/site-packages/splunk/appserver/mrsparkle/locale/fr_FR/LC_MESSAGES/messages.po replacing the msgstr with the correct string value.

Step 3

==

Find the shasum 256 checksum of the newly edited file

On MAC shasum -a 256 $SPLUNK_HOME/lib/python2.7/site-packages/splunk/appserver/mrsparkle/locale/fr_FR/LC_MESSAGES/messages.po

On Centos sha256sum ...

Take this check sum and edit the manifest file replacing the old check sum with the new and restarting. This should remove the warning. Also if the manifest file is renamed or move out of the $SPLUNK_HOME directory it will no longer complain.

2017-10-09 SPL-145546, SPL-154871, SPL-161441, SPL-161442, SPL-161629, SPL-162435, SPL-156316 When assigning indexes to roles, indexes defined on the indexer tier are not displayed

Workaround:
Replace the "$SPLUNK_HOME/etc/apps/search/default/data/ui/manager/authentication_roles.xml" file on the search head with a version from any Splunk Enterprise 6.6.x release. Refresh the configuration on the search head by calling a debug refresh (http[s]://[splunkweb hostname]:[splunkweb port]/debug/refresh) using a supported web browser.

2017-09-27 SPL-145252, SPL-146174, SPL-146175 Column sorting does not work on search and report page if the field contains a whitespace
2017-08-23 SPL-144350 Archived Index is created without error when the splunk index is invalid
2017-07-13 SPL-143111 "Splunkd daemon is not responding" when edit local windows event log collection
2016-11-14 SPL-132133 App Browser filtering of the apps does not work
2015-11-09 SPL-109165 Interactive Field Extractor hangs when using "^" as delimiter.

Workaround:
Use props and transforms to specify the delimiter of your choice.
2015-08-10 SPL-105061, SOLNESS-7274 Broken module prevents splunkweb from starting
2015-06-30 SPL-103701 Actions links should be removed for "Apps Browser"
2014-07-16 SPL-87015 chart count by source and *| cluster showcount=t | table cluster_count _raw) no metadata/ result is available when user drills down on Count and Percent columns.
2014-04-04 SPL-82650 A report created and scheduled by admin cannot be embedded by a power user.
2014-02-26 SPL-81103 Username surrounded by dollar signs cannot create saved searches.
2013-11-20 SPL-76798 Time range picker is not customizable via times.conf the same as version 5 or as suggested by docs.
2013-08-19 SPL-73386 Users are not allowed to run historical scheduled search

Workaround:
1. Create a special power/admin user who can run scheduled searches.

2. Assign this user ownership of the scheduled searches.

3. Share the searches at the app level and grant read/write permission to the correct set of users.

Windows-specific issues

Date filed Issue number Description
2018-10-29 SPL-162352, SPL-158197 splunk-regmon - failed to start the driver due to permission issue
2018-07-02 SPL-156698, SPL-158816, SPL-160530, SPL-160531 splunk-netmon consumes additional 2GB memory every day on Universal Forwarder.
2018-06-05 SPL-155149, SPL-169287, SPL-169289, SPL-155603, SPL-169288 Registry changes under SYSTEM\CurrentControlSet are not being read by WinRegMon

Workaround:
Monitor SYSTEM\\ControlSet\d+ instead.
2018-01-25 SPL-148511, SPL-146092 Powershell "schedule" in cron format staggers
2017-09-18 SPL-144998, SPL-142005 Monitoring Windows Event Log files within archives may result in fields going missing
2015-11-13 SPL-109430 In Windows only, inheritance is broken for folders created by splunkd. Files created are accessible only to the user as whom splunkd is running.
2015-04-14 SPL-99687, SPL-129637 Splunk universal forwarder is 7-10 days behind recent Windows Security and system log events.

Workaround:
To mitigate this, edit the following stanza in inputs.conf: [WinEventLog://Security] evt_resolve_ad_obj = 0.
2015-04-01 SPL-98978 On differing versions of Splunk Enterprise indexer (5.0.1) and universal forwarder (6.2.2), collection of the Security Event log can take increasingly longer over time.

Workaround:
To fix the problem, restart Windows on the forwarder.


2014-10-31 SPL-92596 After upgrade from Splunk Enterprise 6.1 or earlier to 6.4.x on Windows, splunkweb service does not start automatically. Attempts to start it manually show "Error 1053: The service did not respond to the start or control request in a timely fashion."

Workaround:
This is expected behavior. See the Splunk Answers post: http://answers.splunk.com/answers/177187/why-is-the-splunk-web-service-not-running-after-an.html
2014-09-25 SPL-91279 Splunk Universal Forwarder on Windows (specifically, the splunk-perfmon.exe process) does not release key handles.

Workaround:
See "Handle leak when an application collects performance data in Windows Vista, in Windows 7, in Windows Server 2008 or in Windows Server 2008 R2" on the Microsoft Support website for a hotfix download.
2013-10-11 SPL-75116 The UI does not show configured items of some newly converted windows modular inputs that contain the name "default" in the stanza

Workaround:
Edit inputs.conf: in stanzas that contain WinRegMon://default, replace "default" with something else, then restart splunk.

REST, Simple XML, and Advanced XML issues

Date filed Issue number Description
2017-07-13 SPL-143111 "Splunkd daemon is not responding" when edit local windows event log collection
2016-10-31 SPL-131072 Datamodel backend allows invalid time values
2013-05-15 SPL-67453 When sending the following XML data as a GET or POST param to a custom splunkd endpoint: <dashboard>&lt;foo&gt;</dashboard>, the endpoint actually receives:<dashboard><foo></dashboard>.

PDF issues

Date filed Issue number Description
2019-06-04 SPL-171418, SPL-146805 PDF x-axis labels overlapping on line&column chart
2019-06-04 SPL-171419, SPL-146805 PDF x-axis labels overlapping on line&column chart
2016-11-23 SPL-132925 Table data rows generated with the addcoltotals command do not show up in PDF

Workaround:
If you are using addcoltotals to generate a totals data row, renaming the _time field can cause PDF generation issues.

Remove the label and labelfield or change the label to a number to generate the PDF as expected.

2015-03-31 SPL-98890 Maps printed from Report page do not honor custom zoom and center.
2014-06-16 SPL-85497 Unable to save generated PDFs using Chrome internal PDF viewer.

Workaround:
Workaround: Enable Adobe Acrobat or Acrobat Reader as the default PDF viewer in Chrome. For more information, seehttps://support.google.com/chrome/answer/142056.


Admin and CLI issues

Date filed Issue number Description
2018-11-01 SPL-162465, SPL-142345 SHOULD_LINEMERGE always shows true on UI when there is a LINE_BREAKER setting in sourcetype
2018-10-09 SPL-161134, SPL-142345 SHOULD_LINEMERGE always shows true on UI when there is a LINE_BREAKER setting in sourcetype
2018-05-01 SPL-154020, SPL-153543 Splunkd http proxy configuration `no_proxy` variables in server.conf – imprecise hostname/domain matching
2018-04-20 SPL-153625, SPL-154021, SPL-154022 leading and trailing comma validation should be robust for http proxy configuration
2018-04-20 SPL-153624, SPL-154857, SPL-155429 savedsearches.conf configuration is_visible needs clarification
2018-03-15 SPL-152206, SPL-145579 chkconfig directive missing for AWS with enable boot-start
2018-02-06 SPL-148877, SPL-145579 chkconfig directive missing for AWS with enable boot-start
2017-12-13 SPL-147286, SPL-152846, SPL-152848, SPL-152849 Setting DATETIME_CONFIG as filename does not update props.conf
2017-12-04 SPL-146928, SPL-141771 Starting Splunk via the CLI may fail or cause problems if service runs as a domain user and some storage is on a remote share

Workaround:
Use SCM (e.g., sc.exe) rather than CLI.


2017-12-04 SPL-146927, SPL-141771 Starting Splunk via the CLI may fail or cause problems if service runs as a domain user and some storage is on a remote share

Workaround:
Use SCM (e.g., sc.exe) rather than CLI.


2017-11-29 SPL-146820 Unable to access some settings/manager pages (data model editor) if starting from the setup page of a non-visible app

Workaround:
Navigate to a visible app, such as the search and reporting app, and access the Splunk settings pages from that app context.
2017-11-07 SPL-146255 limits.conf enable_clipping cloropleth setting is app/user tunable rather than global like the rest of limits.conf
2017-10-20 SPL-145827, SPL-156137, SPL-165766, SPL-165767 Capability rtsearch is enabling for power user after being remove when running CLI cmd and restarting splunk
2017-04-11 SPL-141051 When LINE_BREAKER is defined for a sourcetype, UI forces SHOULD_LINEMERGE to true

Workaround:
None in Splunk Cloud.

For on-prem, manually edit the props.conf file to set SHOULD_LINEMERGE to 'false'.

2017-04-03 SPL-140747 SSL connection in Python when using new ciphers may be slow.
2016-11-09 SPL-131880 Reports/Alerts owned by the deleted user cannot be found in the Orphaned filter for the Reassign Knowledge Objects page
2016-08-31 SPL-136475 cloud index manager page does not show accurate dates of latest events
2015-09-23 SPL-106978 Failed SHC captain election causes unnecessary change in server.conf
2015-03-11 SPL-97942 Capability defined in an app does not take effect when assigned to a role

Workaround:
The workaround is to change the ui-prefs in ./etc/users/username/local/ui-prefs.conf to look like this:

[search] display.events.fields = ["description","except_extract_1","except_extract_2","except_extract_3","sap_order_status","sourcetype","source","status","request_mode","request_id","request_status_id","object_id","BillToCity_","Airline_","BillToName_","BillToCountry_","City_"] display.events.type = table

2014-04-07 SPL-82699 SSO: Acceleration icon fails to display in Searches, Reports, and Alerts page.
2013-05-25 SPL-68010 The error thrown when your Splunk instance cannot connect to splunkbase/.../checkforupdate is not an ERROR, should be lowered to INFO.

Workaround:
Set server.conf [applicationsManager] allowInternetAccess = false
2013-05-02 SPL-66511 If $SPLUNK_HOME/etc is located on a case-insensitive filesystem, creating a new view with the same name as an existing view but with different case (capital letters vs lowercase, etc) silently overwrites the existing view.

Uncategorized issues

Date filed Issue number Description
2019-11-08 SPL-179256, SPL-179703, SPL-180148, SPL-180149 kvstore inputlookup with large 'where' filter fails silently when hitting 300 second timeout

Workaround:
Change logic of your search,

do filtering later in | search

2019-05-22 SPL-170880, SPL-169429 Do not evict bucket contents from target indexers after S3 upload
2019-03-20 SPL-168023, SPL-167635 Failed to localize because of CacheManager inconsistent bucket state after a truncate

Workaround:
https://confluence.splunk.com/display/PROD/Fixing+Failed+to+localize+errors+due+to+a+truncate
2019-03-20 SPL-168025, SPL-167635 Failed to localize because of CacheManager inconsistent bucket state after a truncate

Workaround:
https://confluence.splunk.com/display/PROD/Fixing+Failed+to+localize+errors+due+to+a+truncate
2019-03-20 SPL-168026, SPL-167635 Failed to localize because of CacheManager inconsistent bucket state after a truncate

Workaround:
https://confluence.splunk.com/display/PROD/Fixing+Failed+to+localize+errors+due+to+a+truncate
2019-02-27 SPL-167013, SPL-143275 Bucket rebuild fails with reason: Failed to process delete journals
2019-02-08 SPL-166228, SPL-166798, SPL-167655 Splunk crashes in _mongoc_openssl_ctx_new on shutdown
2019-02-06 SPL-166106, SPL-165008 MaxMind GeoIP DB needs to be updated for Jan 2019
2019-02-06 SPL-166107, SPL-165008 MaxMind GeoIP DB needs to be updated for Jan 2019
2019-01-28 SPL-165640, SPL-156891 Sometimes streamstats result is not drawn on certain chart types

Workaround:
Set stack mode as "stacked" along with multiseries mode
2019-01-16 SPL-165011, SPL-165008 MaxMind GeoIP DB needs to be updated for Jan 2019
2019-01-15 SPL-164979, SPL-166184, SPL-166510, SPL-166511 Search deadlock in StateStoreWorkerScheduler when executing kvstore lookup
2019-01-15 SPL-164976, SPL-164862 After migration, Splunk Cloud customer seeing unexpected large increase in outbound network bandwidth from forwarders
2018-10-10 SPL-161224, SPL-153371 S2 - Search of a frozen bucket returns with a "failed to localize" error.
2018-09-10 SPL-159813, SPL-163056, SPL-164724, SPL-164725 Post 6.6 / 7.0 upgrade, power user role cannot edit alert.expires from UI
2018-08-22 SPL-159203, SPL-159254, SPL-163561 splunk_archiver app is included in the Splunk Cloud package in error, needs to be excluded at packaging time
2018-08-15 SPL-158875, SPL-159174, SPL-159613, SPL-159614, SPL-159644 splunk shipped python in *nix doesn't work with iso2022_jp
2018-07-13 SPL-157243, SPL-158583, SPL-158584 Inability to disable UI warnings in messages.conf renders disabling the scheduler impractical.
2018-06-27 SPL-156375, SPL-164872, SPL-166196, SPL-166197 Capability to Schedule Saved Searches restricted after upgrade to 7.x.

Workaround:
edit_search_schedule_window capability needs to be added to the affected role.
2018-05-18 SPL-154616, SPL-152935 KVStore Replication Error: replSetReconfig got BadValue _id field value of 256 is out of range
2018-05-17 SPL-154593 Chunks of summary index data are routed to the wrong index when queues are blocked
2018-05-10 SPL-154382, SPL-166025, SPL-167034, SPL-167035, SPL-167036 Role Capability To See Indexes for Summary Indexing Gives Role Index Edit Ability

Workaround:
Enable indexes_edit and dispatch_rest_to_indexers capabilities for the Power role for all indexes to be listed
2018-05-08 SPL-154263 Splunk diag fails on files with modification time before 1970, "error: integer out of range for 'l' format code".

Workaround:
Change the timestamps of any files under SPLUNK_HOME dated prior to 1970.
2018-04-20 SPL-153629 Exiting embedded report url becomes invalid after upgrading to splunk 7.0.x
2018-04-09 SPL-153174, SPL-156193, SPL-156899 Request for better messaging for "Duplicated License situation happen on peer ..."
2018-04-02 SPL-152888, SPL-154243, SPL-155000, SPL-155019, SPL-155451 Chunks of summary index data are routed to the wrong index when queues are blocked
2018-03-29 SPL-152763, SPL-151501 Enabling/Disabling acceleration for a data model creates an unnecessary copy of the data model JSON in <appname>/local/data/models/<model>.json
2018-03-29 SPL-152761, SPL-151501 Enabling/Disabling acceleration for a data model creates an unnecessary copy of the data model JSON in <appname>/local/data/models/<model>.json
2018-03-29 SPL-152762, SPL-151501 Enabling/Disabling acceleration for a data model creates an unnecessary copy of the data model JSON in <appname>/local/data/models/<model>.json
2018-03-14 SPL-152095 Edit Summary Indexing - Index List empty/incomplete for User with Power role after upgrading to 6.6.0+

Workaround:
add indexes_edit and dispatch_rest_to_indexers capability to the Power role for all indexes to be listed
2018-03-14 SPL-152084, SPL-153333, SPL-153334, SPL-159597 S2S: clientCert required in outputs.conf on SSL client although requireClientCent=false set on SSL server
2018-02-21 SPL-149436, SPL-148448 Crash in BSONArrayBuilder::done() while trying to add large item into a collection via REST endpoint
2018-02-14 SPL-149190, SPL-141808 (Windows Only) Support sslRootCAPath on Windows
2018-02-14 SPL-149189, SPL-141808 (Windows Only) Support sslRootCAPath on Windows
2018-02-13 SPL-149157, SPL-178289, SPL-178767, SPL-178768 Missing some meta keys extracted by INDEXED_EXTRACTION after changing _raw content and adding index-time field extraction
2018-01-09 SPL-147956, SPL-152814, SPL-153081 mstats not returning results if tmp folder does not exist.
2017-12-11 SPL-147224, SPL-127642 Powershell log file "splunk-powershell.ps1.log" never rolls
2017-12-10 SPL-147210, SPL-145825 Rendering issues for stacked area graph with Null Values = Gaps

Workaround:
Reload the whole page

Switch back and forth between Null Value Modes works as well

2017-10-13 SPL-145671, SPL-146141, SPL-146142 Indexing Queue blocked after upgrading to 7.0
2017-10-12 SPL-145625 Setting colorPalette by token from search sometimes work (doesn't set color on some html elements)

Workaround:
use a static color map based on the value of the data instead, if that would fit actual use case:

<format type="color" field="Status"> <colorPalette type="map">{"Open":#65A637,"Closed":#555555}</colorPalette> </format>

http://docs.splunk.com/Documentation/SplunkCloud/6.6.3/Viz/TableFormatsXML#Table_format_source_code_example

2017-10-03 SPL-145365, SPL-145599, SPL-145600 Crash in IdataDO_Collector on shutdown
2017-09-23 SPL-145141, SPL-145487 mstats and mcatalog search all indexes rather than a default index
2017-09-18 SPL-144996, SPL-143312 Universal Forwarder Installer lies about the Event Logs it monitors by default

Workaround:
Click advanced install and select monitors you want. or Copy in Windows TA from older install.
2017-09-12 SPL-144856 The mstats command does not work when using parentheses around a single metric_name argument
2017-09-06 SPL-144654, SPL-140755 Missing events in RT search results (or any search if tsidx reduction is enabled) when using negation (NOT or !=)

Workaround:
1. In case of RT searches:

Since the issue here is pre-filtering - we can disable it in limits.conf: [realtime] indexfilter = 0

2. In case of tsidx reduction - the only workaround is to disable it i think :(

2017-09-06 SPL-144653, SPL-140755 Missing events in RT search results (or any search if tsidx reduction is enabled) when using negation (NOT or !=)

Workaround:
1. In case of RT searches:

Since the issue here is pre-filtering - we can disable it in limits.conf: [realtime] indexfilter = 0

2. In case of tsidx reduction - the only workaround is to disable it i think :(

2017-08-23 SPL-144346, SPL-148577, SPL-145047 The metadata command does not work with metrics indexes
2017-08-08 SPL-143777 Chart: missing time axis label near daylight savings boundary
2017-05-09 SPL-141693 DataModel Editor - when child object has same name as inherited field, inherited field does not show in the inherited fields list.
2017-03-27 SPL-140442, SOLNESS-11786 In Splunk Enterprise 6.6.0 and later, with Enterprise Security 4.5.2 and 4.6.0, roles without "edit_roles" capability cannot perform operations on notable event review statuses.

Workaround:
If users cannot perform operations on notable event review statuses or have issues viewing "Edit all selected" links on Incident Review, user roles must be provided with the "edit_roles" capability.
2017-02-13 SPL-136709 Chart retains legend and title after enabling trellis layout in splunk.js
2017-01-18 SPL-135260 Documentation for Search formatting keyboard shortcut for non-English languages
2017-01-06 SPL-134707 Splunk restart does not create missing server.pem certificate on Windows

Workaround:
Use bin/splunk createssl server-cert -d etc/auth/ -n server to generate a new certificate.
2016-11-21 SPL-132670 Mac OS 10.11: disable boot-start doesn't remove the file /Library/LaunchAgents//com.splunk.plist by enabling boot-start in prior Splunk/UF
2016-08-31 SPL-127800 Opting in to data sharing on a monitoring console produces duplicate data
2016-07-26 SPL-125052 Sole Admin can demote themself to Power without path of recovery in GUI.

Workaround:
Through the command line, you can open notepad and modify the password file to regain 'Admin' status.
2016-07-11 SPL-124026, SPL-122942 Relative paths should not be allowed under volume's path=file:// on remote storages
2016-06-22 SPL-123301, SPL-95164, SPL-167968 Aggressive calls to LDAP for non-existent/inactive users causes slow logins, performance issues/ skipped searches/ indexing pause
2016-06-21 SPL-123174 JSON indexed_extractions doesn't work for TCP inputs
2015-10-07 SPL-107606 Inconsistency between summary and datamodel_summary files.
2015-06-18 SPL-103302 Files ownership are failed to be changed when using debian package to install splunk and $SPLUNK_HOME is a symlink

Workaround:
Run a recursive chown from the command line on $SPLUNK_HOME manually, post install.
2015-05-24 SPL-102008 On Internet Explorer, a warning message does not display when you cannot log in due to a time zone difference.
2015-05-11 SPL-101289 When the number of indexing pipeline sets is greater than four, indexing throughput decreases.
2015-05-06 SPL-100980 Single indexer does not scale when receiving parsed data from multiple PipelineSets.
2015-05-04 SPL-100792 There are multiple group=thruput metrics lines in metrics.log. Searches that do not differentiate among them may get falsely high totals.

Workaround:
Searches that key off these lines need to select their desired name=x category in order to see a single thruput value.
2015-04-24 SPL-100322 A view gets stuck with "loading" due to problematic navigation (default.xml)

Workaround:
Workaround is to use label attribute for collection element.

<collection label="Others">

           <view source="unclassified" match="Dashboard"/>
     </collection>  
2015-03-26 SPL-98700 splunkd Indexer crashes in IndexerTPoolWorker due to duplicated bucket id.

Workaround:
The workaround is to remove the duplicated bucket.
2015-02-26 SPL-97389 When using timechart command, the embedded report shows different time format than the original report.
2015-01-08 SPL-95144, SPL-101986, SPL-101987, SPL-106884, SPL-107317, SPL-142789 Indexed message for Windows security event logs shows "FormatMessage error"

Workaround:
Splunk believes this was introduced in a Microsoft Windows patch. The workaround is to configure a delayed start of the Splunk service(s) so that it starts after the Windows Event Log service.
2014-11-10 SPL-92831 A mismatch of versions between the license-master and the license-slave is generating Warning messages like "WARN LMDirective - directive cmd=D_set_feature_state args='Acceleration,ENABLED' failed: reason='feature='Acceleration' is invalid' ."

Workaround:
The warnings can be ignored, the workaround is use same major versions (all on 6.2 or all on 6.1).


2014-10-24 SPL-92432, SPL-99583 Chart in dashboard panel does not honor interval settings.

Workaround:
In the panel XML, specify a larger height to use the correct interval settings.
2014-10-17 SPL-92162 Writing large amounts of data (> 20 GB) to KV store collections using outputlookup can result in high memory usage on the machine.
2014-09-11 SPL-90738 Monitoring a directory with an unknown sourcetype produces indexing errors.
2014-08-26 SPL-90139 <timestamp> does not display in the Patterns tab when searches are run in fast mode.
2014-04-22 SPL-83365 Splunk Enterprise on Windows does not show an error message when a user without the edit_license capability tries to add a license through the CLI.
2014-04-14 SPL-83068 Default index can be set to random index.
2014-04-01 SPL-82517, SPL-208875 Paper Size and Layout in PDF Schedule dialog do not respect Paper Size and Layout in Email Settings.
2014-03-23 SPL-82238 Datamodel fails to drill down further when the same attribute for Split Rows and Split Columns are selected.
2014-03-13 SPL-81856 Show all lines does not work in data model editor preview.
2014-03-12 SPL-81810 Licensing - license pool warning at license master keeps coming back after deleting it.

Workaround:
Delete the warnings on the peers first, then the License Manager.
2014-03-12 SPL-81781 In the Data Model Manager, "Acceleration Status" and "Access Count" fail to update when you click "Update".
2014-02-13 SPL-80568 Highcharts determines Y-axis values based on first point outside visible range.
2014-02-07 SPL-80285 In the Data Model Editor, the Edit Lookup page is blank if Lookup is shared only in Lookup Definitions.

Workaround:
For more information, see Add lookup files to Splunk.
2014-02-06 SPL-80187 In the Data Model Editor, lookup pages open with options displayed for other Lookup when the data model definition is private but the file is app or globally shared.

Workaround:
Share the definition. For more information, see Add lookup files to Splunk.
2014-01-31 SPL-79842 On Windows, Indexer doesnt accept new connections on splunktcpin port after queue blockage is resolved
2013-11-27 SPL-77139 Licenser pool usage gets reflected only after restarting splunkd.
2013-10-29 SPL-75764 Forwarder forwards duplicate data after props.conf is in place for cross platform scenario/when the forwarder is on Solaris and the indexer is on Linux.
2013-09-13 SPL-74337, BETA-496 You cannot specify a destination folder when installing on OSX.
2013-09-10 SPL-74209, SPL-74167 Persistent queues are not created on Windows for stanzas that contain unusual characters (such as < and >).

Workaround:
Specify the persistentQueue explicitly in the input definition.
2013-08-28 SPL-73826 Windows: hostname override not working properly
2013-06-13 SPL-69304 If license slaves are running <6.0 version, they do not have the idx field and in theLicense Usage view, the split by index field will show a field named UNKNOWN.
2013-04-30 SPL-66213 PDF server app is not working with latest Xvfb
2012-02-22 SPL-48342 LDAP strategy host field cannot work with ipv6 format address but computer name is okay
2010-10-08 SPL-34347 wmi input default fields - with value including newlines doesn't search properly becasue of \r\n issue

Splunk Analytics for Hadoop

Date filed Issue number Description
2017-09-12 ERP-2104, ERP-2079 Please make splunk_archiver.log roll
2017-04-04 ERP-2040 Splunk archiving fails for large block sizes (buckets) due to HDFS write crashes for Hadoop version 2.8, 2.7.x

Workaround:
Upgrade Hadoop to 2.8.2 or higher.
2015-09-09 ERP-1650 timestamp data type not properly deserialized.
2015-08-05 ERP-1619 Searching on a newly created archive index before the bucket copy saved search is run causes a filenotfound exception.

Workaround:
Reenable the bucket copy saved search and let it run, or force the archiving to happen via | archivebuckets force=1 and then rerun the search.
2015-07-07 ERP-1598 minsplit rampup - splits generation takes too long.

Workaround:
Set minsplits=maxsplits
2015-05-12 ERP-1502 Non-accelerated pivot search on Pivot UI page waits for a long time to return result.
2015-01-08 ERP-1343, SPL-95174 Splunk Analytics for Hadoop searches fail on corrupted journal.gz files, although Splunk searches run without error.

Workaround:
Add the journal.gz to the input path's blacklist (vix.input.1.ignore = ....)
2014-10-27 ERP-1216 Data Explorer preview does not honor existing sourcetypes for big5/sjis files.
2014-10-03 ERP-1164 Report acceleration summary gets deleted when two Splunk Analytics for Hadoop instances point to the same Splunk working directory.

Workaround:
To mitigate this issue, make sure that vix.splunk.home.hdfs (or Working directory in the UI) is unique on both search heads that are not in a pool. To keep your instances in the same working directory, configure vix.splunk.search.cache.path to be unique on both search heads.
Last modified on 29 October, 2024
Welcome to Splunk Enterprise 7.0   Splunk Enterprise and anti-virus products

This documentation applies to the following versions of Splunk® Enterprise: 7.0.0


Was this topic useful?







You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters