Splunk® Enterprise

Release Notes

Download manual as PDF

This documentation does not apply to the most recent version of Splunk. Click here for the latest version.
Download topic as PDF

Known issues

The following are issues and workarounds for this version of Splunk Enterprise.

Issues are listed in all relevant sections. Some issues appear more than once.

Refer to System requirements in the Installation Manual for a list of supported platforms and architectures.

For a list of deprecated features and platforms, refer to Deprecated features in this manual.

Upgrade issues

This section lists issues that customers have reported when upgrading from an earlier version of Splunk Enterprise. If you are considering an upgrade, please read "How to upgrade Splunk Enterprise" in the Installation Manual.

Date filed Issue number Description
2017-05-23 SPL-141964 Older 6.0 and 6.1 maintenance release forwarders unable to forward events to 6.6.x indexers via splunktcp-ssl

Workaround:
This affects communication between Splunk 6.6.x indexers and:
  • 6.0.0 to 6.0.6 forwarders
  • 6.1.0 to 6.1.4 forwarders

Upgrade your older forwarders to the latest maintenance releases or on your 6.6.x Indexer, add to inputs.conf:

[SSL]
sslVersions = *,-ssl2
cipherSuite = ALL:!aNULL:!eNULL:!LOW:!EXP:RC4+RSA:+HIGH:+MEDIUM


2017-05-23 SPL-141961 Older 6.0, 6.1, 6.2, 6.3 maintenance release versions unable to connect to 6.6.x via management port

Workaround:
This applies to License Master/Slave, Deployment Server/Client, Cluster Master/Peers, Search Head/Peers and affects Splunk 6.6.x and the following versions:
  • 6.0.0 to 6.0.6
  • 6.1.0 to 6.1.4
  • 6.2.0 to 6.2.6
  • 6.3.0 to 6.3.1
  • 6.3.1511.1

Upgrade your older instances to the latest maintenance releases or on your 6.6.x Splunk instances. Add the following stanza to server.conf:

[sslConfig]
sslVersions = *,-ssl2
sslVersionsForClient = *,-ssl2
cipherSuite = TLSv1+HIGH:TLSv1.2+HIGH:@STRENGTH


2017-03-20 SPL-139019 Possible compatibility issues between Python / SDK clients and new 6.6 default sslVersions, cipherSuites

Workaround:
Users can do either of the following:

1. Overwrite the new Splunk 6.6 server.conf [sslConfig] sslVersions, cipherSuites with your own settings that are compatible with your version of OpenSSL, e.g. the previous defaults from 6.5.x are compatible with OpenSSL 0.9.8 on Mac OSX:

[sslConfig]
sslVersions = *,-ssl2
sslVersionsForClient = *,-ssl2
cipherSuite = TLSv1+HIGH:TLSv1.2+HIGH:@STRENGTH

2. For some more up-to-date clients, it is possible to enforce TLS1.2 (e.g. --tlsv1.2 for curl) in order to connect successfully.

3. Upgrade OpenSSL on your platform and link it with your client (e.g. Python, curl, etc..). For example, OpenSSL 1.0.2 is currently available on Mac OSX via Homebrew (see https://brew.sh) and is compatible with the new Splunk 6.6 default sslVersions, cipherSuites.

2017-03-13 SPL-138647 Possible compatibility issues between new 6.6 default sslVersions, cipherSuites and external services, e.g. e-mail, LDAP

Workaround:
If security is not a significant concern, simply revert back to the 6.5.x SSL/TLS defaults, e.g. for e-mail, add to $SPLUNK_HOME/etc/system/local/alert_actions.conf

[email]
sslVersions = *,-ssl2
cipherSuite = TLSv1+HIGH:TLSv1.2+HIGH:@STRENGTH


If you would like to retain the more secure 6.6.x defaults, but prefer to add an exception for your less secure external services, follow the procedure below:

1. To determine what sslVersions and cipherSuites are supported by a server, run splunk cmd openssl s_client -connect hostname:port | awk '/Protocol/ || /Cipher/ || /Verify/'.

The example below is for a Postfix SMTP server:

eserv@indexer01:~$ splunk cmd openssl s_client -connect smtp-server01:465 | awk '/Protocol/ || /Cipher/ || /Verify/'
depth=1 C = US, O = Example Customer, OU = IT, CN = Example Customer IT CA, emailAddress = customer@example.org verify error:num=19:self signed certificate in certificate chain New, TLSv1/SSLv3, Cipher is DHE-RSA-AES256-SHA

   Protocol : TLSv1 
   Cipher : DHE-RSA-AES256-SHA 
   Verify return code: 19 (self signed certificate in certificate chain)

2. Check the OpenSSL output for Protocol and Cipher. In the example above, Protocol = TLSv1 and Cipher = DHE-RSA-AES256-SHA

3. Update Splunk's relevant sslVersions and/or cipherSuite. In the example above, sslVersions should be set to tls (allows TLSv1, TLSv1.1, TLSv1.2) and DHE-RSA-AES256-SHA should be appended to the end of the default cipherSuites definition, e.g. add $SPLUNK_HOME/etc/system/local/alert_actions.conf:

[email]
sslVersions = tls

cipherSuites = ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA

2017-02-16 SPL-136920 Exclusive use of Forward Secret Ciphers in server.conf prevents KV Store from working

Workaround:
The only workaround is to add non DH ciphers to the list, for example:

cipherSuite = AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256

First cipher on the list was added, KV store will be able to run.

2014-08-20 SPL-89640 When running Splunk on Linux as non-root user and using RPM to upgrade, the RPM writes $SPLUNK_HOME/var/log/introspection as root, causing errors upon restarts.

Workaround:
Chown the $SPLUNK_HOME/var/log/introspection directory to the user Splunk Enterprise runs as after upgrading and before restarting Splunk Enterprise.
2013-08-19 SPL-73386 Users are not allowed to run historical scheduled search

Workaround:
1. Create a special power/admin user who can run scheduled searches.

2. Assign this user ownership of the scheduled searches.

3. Share the searches at the app level and grant read/write permission to the correct set of users.

Data input issues

Date filed Issue number Description
2015-11-12 SPL-109362 When the disk runs out of space for the limit set in the server.conf, add data workflow gets stuck with "Uploading file" message modal in the review stage
2015-05-22 SPL-101981 Field extractions do not work when sourcetypes use quotes in the Getting Data In interface.
2015-03-17 SPL-98163 INDEXED_EXTRACTIONS=W3C is truncating field cs_uri_stem when spaces are present in URL

Workaround:
Create a separate extraction in props.conf where defined w3c extraction method:

EXTRACT-cs_uri_stem1 = (GET|POST) (?<cs_uri_stem1>[^-]++)

2014-03-10 SPL-81637 Splunkd preview runs indefinitely on any file preview with "DATETIME_CONFIG=none".
2013-10-29 SPL-75764 Forwarder forwards duplicate data after props.conf is in place for cross platform scenario/when the forwarder is on Solaris and the indexer is on Linux.
2013-10-11 SPL-75116 The UI does not show configured items of some newly converted windows modular inputs that contain the name "default" in the stanza

Workaround:
Edit inputs.conf: in stanzas that contain WinRegMon://default, replace "default" with something else, then restart splunk.
2013-09-10 SPL-74209, SPL-74167 Persistent queues are not created on Windows for stanzas that contain unusual characters (such as < and >).

Workaround:
Specify the persistentQueue explicitly in the input definition.

Search issues

Date filed Issue number Description
2017-06-19 SPL-142497, SPL-143374 Consecutive calls to random() return repeating values in a predictable pattern.
2017-04-04 SPL-140765 Splunk having problems extracting json file consisting of 68k plus key-value pairs
2017-03-21 SPL-140175 Aborted delete searches may result in stale lock files being left behind

Workaround:
Delete stale lock files.


2016-11-29 SPL-133182 When two datasets have identical names but one is local (private) while the other is global, attempts to view or extend the global dataset use results from the local dataset instead.
2016-10-13 SPL-130257 Dataset doesn't work if user has name with non-ascii characters
2016-09-15 SPL-128845, SPL-131106, SPL-131108 Distributed Search: Deployment -- Time to Reap : Dispatch Directory ... max < average time
2016-08-09 SPL-126101, SPL-141299 The median function is incorrect if the number of values is even. Documentation implied "perc_method=interpolated" was default, but actually defaults to nearest-rank
2016-04-27 SPL-118911 In SHC, referenced saved real-time searches in a dashboard do not stream results.

Workaround:
See Troubleshoot referenced real-time searches for workaround details.


2016-03-17 SPL-116082 Custom search commands that are defined for only a specific user will no longer run.
2015-08-10 SPL-105061, SOLNESS-7274 Broken module prevents splunkweb from starting
2015-06-17 SPL-103247 Filtering on _time uses different semantics for the "=" operator on microseconds depending on whether the value is quoted.
2015-04-23 SPL-100170 Automatic Lookups limitation: No results returned in Smart Mode when there are nested lookups and the intermediate field is not mentioned in the search.
2015-04-03 SPL-99110 Distributed search fails intermittently to a subset of peers with an unhelpful "Unknown error".

Workaround:
To work around this, edit /etc/sysctl.conf to modify the following lines:

net.core.rmem_max = 134217728 net.core.wmem_max = 134217728 net.ipv4.tcp_rmem = 16384 87380 67108864 net.ipv4.tcp_wmem = 16384 87380 67108864 Restart splunkd. Repeat above steps for SH and the Indexer. Run sysctl -p for changes to take effect. Sometimes sysctl -p may not work due to caching and rebooting is a better option. Restart indexer.

2014-12-22 SPL-94910 The replace function does not apply to fields names with an underscore in them.

Workaround:
Rename the fields before the replace.

... | rename *_* AS *-* | replace "something" by "somethingelse"

2014-11-13 SPL-93039 The relevancy search command does not work, always returning 0 or -inf.
2014-10-02 SPL-91638, SPL-107375 For scheduled searches in a search head cluster, empty search jobs may appear in the job inspector for a cluster member.
2014-09-15 SPL-90861, SPL-90396, SPL-90886 If search encounters invalid offsets or invalid rawdata at TSIDX offsets, it skips reading any number of events from that bucket. No message is displayed, though the information is added to search.log.
2014-04-16 SPL-83129 Eval function strptime does not return results when 1970 date is used.
2014-04-04 SPL-82650 A report created and scheduled by admin cannot be embedded by a power user.
2014-03-27 SPL-82357 The splunk clean all -f CLI command doesn't remove data from the main index on Windows systems.
2014-03-15 SPL-81934 For clusters, may be unable to open search results output file for search results in a cluster.

Workaround:
Write to a temp file and rename to the target file.
2014-02-21 SPL-80942 Flashtimeline: 500 Internal Server Error when pasting long URL into panel name.
2013-12-18 SPL-78179 REST /saved/searches App names with special characters have invalid links.
2013-08-19 SPL-73386 Users are not allowed to run historical scheduled search

Workaround:
1. Create a special power/admin user who can run scheduled searches.

2. Assign this user ownership of the scheduled searches.

3. Share the searches at the app level and grant read/write permission to the correct set of users.

Saved search, alerting, scheduling, and job management issues

Date filed Issue number Description
2016-11-14 SPL-132078 Running jobs should not be marked as expired
2016-09-23 SPL-129285 The search scheduler (SavedSplunker) has scaling problems with high disabled user count and external auth systems (SAML & LDAP)
2015-04-09 SPL-99421 Long name of app causes accelerated search to not complete normally and shows invalid results on Windows 2008 R2

Workaround:
Reduce length of name of the app and report acceleration searches will run properly within the context of the app.
2014-08-15 SPL-89332 Report acceleration summaries do not show in Settings when you have hundreds of reports accelerated.
2014-08-05 SPL-88396 After configuring a client name for a deployment client, the name is not shown in the Forwarder Management UI

Workaround:
Create a server class, where you can see the client name, and use that group when you add data.
2014-05-01 SPL-83686 Data Model Pivot: Extra NULL column displays in Pivot with big data and Numbered Attribute in Split Columns.

Workaround:
The workaround is to add filter status=*, or make a more refined Data Model that has an object for events with status.
2014-03-24 SPL-82262, SPL-82241 Pivot search command fails for an admin trying to pivot on a Private Data Model created by a User.
2014-03-20 SPL-82164 Migrating invalid data models from 6.0 to 6.x fails.
2014-03-19 SPL-82133 Data model allows users to upload a JSON file which has Field names with spaces but will not validate it.
2014-03-10 SPL-81645 Data model exhibits sticky UI when "transaction group by object" name has a single (x) character.
2013-11-26 SPL-77054, SPL-77055 Data model objects that have names starting with an underscore character ("_") do not work correctly and cannot be used in Pivot.

Charting, reporting, and visualization issues

Date filed Issue number Description
2017-02-13 SPL-136709 Chart retains legend and title after enabling trellis layout in splunk.js
2017-02-13 SPL-136707, SPL-134891 The trellis.enabled option is not supported for single value visualizations in splunk.js
2016-11-29 SPL-133182 When two datasets have identical names but one is local (private) while the other is global, attempts to view or extend the global dataset use results from the local dataset instead.
2016-09-15 SPL-128819, SPL-130243, SPL-130245 Editing panel in dashboard removes charting.legend.masterlegend option

Workaround:
Use <option name="charting.legend.masterLegend">null</option>
2016-04-27 SPL-118911 In SHC, referenced saved real-time searches in a dashboard do not stream results.

Workaround:
See Troubleshoot referenced real-time searches for workaround details.


2015-03-31 SPL-98890 Maps printed from Report page do not honor custom zoom and center.
2015-02-23 SPL-97193 The initial value for Multiselect input does not display properly in Visualizations Editor if input has empty string.
2014-10-24 SPL-92432, SPL-99583 Chart in dashboard panel does not honor interval settings.

Workaround:
In the panel XML, specify a larger height to use the correct interval settings.
2014-02-13 SPL-80568 Highcharts determines Y-axis values based on first point outside visible range.
2014-01-03 SPL-78612, SPL-143311 Deleting a dashboard with a scheduled PDF does not also delete the scheduled view.
2013-10-11 SPL-75116 The UI does not show configured items of some newly converted windows modular inputs that contain the name "default" in the stanza

Workaround:
Edit inputs.conf: in stanzas that contain WinRegMon://default, replace "default" with something else, then restart splunk.
2013-08-22 SPL-73569 Pie maps do not have legend labels.

Data model and pivot issues

Date filed Issue number Description
2017-02-22 SPL-137274, SPL-138967, SPL-138968 Pivot: area/line/bar/column charts showing multiple column values when clicking browser back button
2014-12-08 SPL-94047, SPL-98628 While creating a Pivot and using the _time column as a Split column, the table columns aren't formatted in a human readable way, but displayed with the epoc timestamp.It works when using _time as a 'Split Row' column.
2014-05-01 SPL-83686 Data Model Pivot: Extra NULL column displays in Pivot with big data and Numbered Attribute in Split Columns.

Workaround:
The workaround is to add filter status=*, or make a more refined Data Model that has an object for events with status.
2014-03-24 SPL-82262, SPL-82241 Pivot search command fails for an admin trying to pivot on a Private Data Model created by a User.
2014-03-20 SPL-82164 Migrating invalid data models from 6.0 to 6.x fails.
2014-03-19 SPL-82133 Data model allows users to upload a JSON file which has Field names with spaces but will not validate it.
2014-03-11 SPL-81701 Data Model Pivot, "Legend Position" and "Stack Mode" change to default settings if you change the X/Y-Axis more than once.
2014-03-10 SPL-81645 Data model exhibits sticky UI when "transaction group by object" name has a single (x) character.
2014-03-07 SPL-81538 When using Pivot, stack mode is lost when "Scatter Chart" is selected.
2013-11-26 SPL-77054, SPL-77055 Data model objects that have names starting with an underscore character ("_") do not work correctly and cannot be used in Pivot.

Indexer and indexer clustering issues

Date filed Issue number Description
2017-07-26 SPL-143402, SPL-143757, SPL-144482 Fsck processes are stuck leading to fixup tasks not completing, ever.
2017-03-10 SPL-138537, SPL-140926 After an upgrade to 6.5, indexers in an indexer cluster have a significant performance reduction, with index queues filling to 100% and blocking. The replication factor in the indexer cluster was equal to the number of indexers in the cluster.

Workaround:
In outputs.conf on the forwarder, set autoLBFrequency=15 (down from the default of 30).

autoLBFrequency=15

In server.conf on the indexer, increase the size of the index queue. [queue=indexQueue] maxSize = 6MB

2016-08-25 SPL-127353 Data rebalance finishes early when one peer is the source for all buckets

Workaround:
when only one indexer in a cluster indexed data (has all the searchable copies), rebalance once before adding the new indexer, and then rebalance a second time
2016-06-21 SPL-123174 JSON indexed_extractions doesn't work for TCP inputs
2015-05-11 SPL-101289 When the number of indexing pipeline sets is greater than four, indexing throughput decreases.
2015-05-08 SPL-101184 Rolling restart in an Indexer Cluster may not be successful on a peer if a oneshot command is also running on that peer. Perform a manual restart to revive the peer.
2015-05-06 SPL-100980 Single indexer does not scale when receiving parsed data from multiple PipelineSets.
2015-05-04 SPL-100792 There are multiple group=thruput metrics lines in metrics.log. Searches that do not differentiate among them may get falsely high totals.

Workaround:
Searches that key off these lines need to select their desired name=x category in order to see a single thruput value.
2015-03-26 SPL-98700 splunkd Indexer crashes in IndexerTPoolWorker due to duplicated bucket id.

Workaround:
The workaround is to remove the duplicated bucket.
2014-10-13 SPL-91861 On Windows indexer on an ec2 instance, splunk-optimize main thread can crash on buckets on the temporary drive z:\>.
2014-09-29 SPL-91432 On Windows when the master is down, the CLI command splunk offlinehangs when run from one of the streaming target peers.
2014-09-08 SPL-90630 On a multisite cluster, no warning is given when search head names are the same.
2014-08-29 SPL-90331 Multi-site indexer cluster doesn't meet replication factor/search head factor due to bucket issue.

Workaround:
From the endpoint, add the buckets missing RF/SF to the to_fix list.

endpoint: https://[host]:[port]/services/cluster/master/buckets/{bucket_id}/fix

2014-07-29 SPL-87816 When implementing an indexer cluster or search head cluster, you cannot set pass4SymmKey in the general stanza. The system default values in the clustering and shclustering stanzas override any user-provided values in the general stanza.

Workaround:
Set the value in the [clustering] or [shclustering] stanza, depending on the type of cluster you're implementing.
2014-07-14 SPL-86799 After adding a new license to the clustering search head, splunkd on restart cannot be reached by splunkweb.
2014-04-29 SPL-83636 If you first configure a master with default RF/SF and then give the misconfiguration command, you get an error message that is wrong.
2014-04-17 SPL-83169 on Windows, if peers' Windows explorer not closed for long enough time, adding a new index still requres a peer restart, not reload
2014-04-14 SPL-83068 Default index can be set to random index.
2014-03-18 SPL-82038 Cluster-config does not work if a parameter value includes a space character.
2014-03-17 SPL-81955 Multisite: Peer takes approximately 6 minutes to restart when its site configuration is changed.
2014-01-31 SPL-79842 Indexer doesn't accept new connections on splunktcpin port after queue blockage is resolved
2014-01-06 SPL-78688 Peer is able to change to an invalid (empty) replication port
2013-08-28 SPL-73826 Windows: hostname override not working properly
2013-08-06 SPL-72484 You cannot use the CLI to delete an index with a capital letter in its name.
2010-10-08 SPL-34347 wmi input default fields - with value including newlines doesn't search properly becasue of \r\n issue

Distributed search and search head clustering issues

Date filed Issue number Description
2017-07-04 SPL-142888, SPL-140831 Splunk not cleaning up $SPLUNK_HOME/var/run/searchpeers of .delta files and matching directories whose only non-empty subdirectory has the .index extension

Workaround:
Increase max_memtable_bytes under [lookup] inside limits.conf so that the largest lookup won't get indexed.
2017-03-13 SPL-138654 Splunk searches fail when filepath gets too long on Windows
2016-07-12 SPL-124085 On Search Head Cluster It is not possible to remove an App from the SHs once it has been disabled.
2015-09-23 SPL-106978 Failed SHC captain election causes unnecessary change in server.conf
2015-02-26 SPL-97385 $SPLUNK_HOME/var/run/splunk/snapshot contains large tarballs in the presence of large ES lookup table files.

Workaround:
The allowable size of the download can be increased by setting the following in server.conf.

[httpServer] max_content_length = 1500MB

The other option is to disable the search which controls the generation of the large lookup file. In this case, the search is:

[Endpoint - Local Processes Tracker - Lookup Gen]

2014-08-25 SPL-90028 Using "inputcsv dispatch=true" to read a CSV from a dispatch directory may not work on search head cluster members that have a replica of the desired artifact.
2014-08-14 SPL-89131 In a search head cluster, the search Job management page on cluster member doesn't immediately reflect 'isSaved' state after you click Save.
2014-08-02 SPL-88228 When user clicks on the RSS feed for an alert, search pool information is not displayed. Individual pool member information is displayed, however.

Universal forwarder issues

Date filed Issue number Description
2015-04-14 SPL-99687, SPL-129637 Splunk universal forwarder is 7-10 days behind recent Windows Security and system log events.

Workaround:
To mitigate this, edit the following stanza in inputs.conf: [WinEventLog://Security] evt_resolve_ad_obj = 0.
2015-04-07 SPL-99316 Universal Forwarders stop sending data repeatedly throughout the day

Workaround:
In limits.conf, try changing file_tracking_db_threshold_mb in the [inputproc] stanza to a lower value.
2014-08-05 SPL-88396 After configuring a client name for a deployment client, the name is not shown in the Forwarder Management UI

Workaround:
Create a server class, where you can see the client name, and use that group when you add data.
2013-09-18 SPL-74427, SPL-74448 The Splunk universal forwarder installer for Solaris 10 does not add the splunk user when you attempt to install it using the pkgadd command. This results in the script generating lots of errors.

Workaround:
To work around this issue, create a splunk user on your system before attempting to run the installer.

Distributed deployment, forwarder, deployment server issues

Date filed Issue number Description
2014-10-02 SPL-91648, SPL-91358 Forwarder unable to push scripted inputs to a Linux deployment client from a Windows deployment server.
2014-08-15 SPL-89333 Using client filtering in forwarder management interface when the deployment server is servicing a large numbers of deployment clients (over approximately 5000) can cause a temporary spike in memory usage.
2014-06-20 SPL-85739 When running a high number of deployment clients for a server, memory growth may be excessive.

Workaround:
To mitigate this, set forceHttp10=always.

Data Management Console Issues

Date filed Issue number Description
2017-03-07 SPL-138351 The role change of DMC via UI does not reflect to distsearch.conf

Workaround:
As a workaround can the customer manually modify the distsearch.conf.
2017-03-06 SPL-138223 Install log not showing operations until explicit refresh of page
2017-01-12 SPL-135005 Datamodel Editor: Empties out non-visible internal field (i.e. comment)
2016-11-14 SPL-132151 XML error when trying to download uninstalled app
2016-11-09 SPL-131880 Reports/Alerts owned by the deleted user can not be found in orphaned tab
2016-09-15 SPL-128845, SPL-131106, SPL-131108 Distributed Search: Deployment -- Time to Reap : Dispatch Directory ... max < average time
2014-04-07 SPL-82699 SSO: Acceleration icon fails to display in Searches, Reports, and Alerts page.

Splunk Web and interface issues

Date filed Issue number Description
2017-07-21 SPL-143281, SPL-143431 Broken @go URL "The view you requested could not be found"
2017-03-07 SPL-138331, SPL-138939, SPL-138940 Search syntax highlighting stops working event if one command does not have syntax property in searchbnf.conf
2017-01-18 SPL-135260 Search formatting keyboard shortcut does not work for French language
2016-11-14 SPL-132133 App Browser filtering of the apps does not work
2015-11-09 SPL-109165 Interactive Field Extractor hangs when using "^" as delimiter.

Workaround:
Use props and transforms to specify the delimiter of your choice.
2015-08-10 SPL-105061, SOLNESS-7274 Broken module prevents splunkweb from starting
2015-06-30 SPL-103701 Actions links should be removed for "Apps Browser"
2015-04-24 SPL-100322 A view gets stuck with "loading" due to problematic navigation (default.xml)

Workaround:
Workaround is to use label attribute for collection element.

<collection label="Others">

           <view source="unclassified" match="Dashboard"/>
     </collection>  
2014-07-16 SPL-87015 chart count by source and *| cluster showcount=t | table cluster_count _raw) no metadata/ result is available when user drills down on Count and Percent columns.
2014-04-04 SPL-82650 A report created and scheduled by admin cannot be embedded by a power user.
2014-02-26 SPL-81103 Username surrounded by dollar signs cannot create saved searches.
2013-11-20 SPL-76798 Time range picker is not customizable via times.conf the same as version 5 or as suggested by docs.
2013-08-19 SPL-73386 Users are not allowed to run historical scheduled search

Workaround:
1. Create a special power/admin user who can run scheduled searches.

2. Assign this user ownership of the scheduled searches.

3. Share the searches at the app level and grant read/write permission to the correct set of users.

Windows-specific issues

Date filed Issue number Description
2015-11-13 SPL-109430 In Windows only, inheritance is broken for folders created by splunkd. Files created are accessible only to the user as whom splunkd is running.
2015-04-14 SPL-99687, SPL-129637 Splunk universal forwarder is 7-10 days behind recent Windows Security and system log events.

Workaround:
To mitigate this, edit the following stanza in inputs.conf: [WinEventLog://Security] evt_resolve_ad_obj = 0.
2015-04-01 SPL-98978 On differing versions of Splunk Enterprise indexer (5.0.1) and universal forwarder (6.2.2), collection of the Security Event log can take increasingly longer over time.

Workaround:
To fix the problem, restart Windows on the forwarder.


2014-09-25 SPL-91279 Splunk Universal Forwarder on Windows (specifically, the splunk-perfmon.exe process) does not release key handles.

Workaround:
See "Handle leak when an application collects performance data in Windows Vista, in Windows 7, in Windows Server 2008 or in Windows Server 2008 R2" on the Microsoft Support website for a hotfix download.
2013-10-11 SPL-75116 The UI does not show configured items of some newly converted windows modular inputs that contain the name "default" in the stanza

Workaround:
Edit inputs.conf: in stanzas that contain WinRegMon://default, replace "default" with something else, then restart splunk.

Rest, Simple XML, and Advanced XML issues

Date filed Issue number Description
2016-10-31 SPL-131072 Datamodel backend allows invalid time values
2013-05-15 SPL-67453 When sending the following XML data as a GET or POST param to a custom splunkd endpoint: <dashboard>&lt;foo&gt;</dashboard>, the endpoint actually receives:<dashboard><foo></dashboard>.

Authentication and Authorization issues

For a list of security issues, please see the Security Advisory. A list of all recent advisories can be found in the Security Portal.

Date filed Issue number Description
2017-02-16 SPL-136920 Exclusive use of Forward Secret Ciphers in server.conf prevents KV Store from working

Workaround:
The only workaround is to add non DH ciphers to the list, for example:

cipherSuite = AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256

First cipher on the list was added, KV store will be able to run.

PDF issues

Date filed Issue number Description
2016-11-23 SPL-132925 Table data rows generated with the addcoltotals command do not show up in PDF

Workaround:
If you are using addcoltotals to generate a totals data row, renaming the _time field can cause PDF generation issues.

Remove the label and labelfield or change the label to a number to generate the PDF as expected.

2015-03-31 SPL-98890 Maps printed from Report page do not honor custom zoom and center.
2014-06-16 SPL-85497 Unable to save generated PDFs using Chrome internal PDF viewer.

Workaround:
Workaround: Enable Adobe Acrobat or Acrobat Reader as the default PDF viewer in Chrome. For more information, seehttps://support.google.com/chrome/answer/142056.


2013-05-16 SPL-67491 PDF report: Events format settings like List, Table, MaxLines, and Wrapping don't apply to PDF report
2012-11-26 SPL-58744 Area chart is not filled if the points are unconnected

Admin and CLI issues

Date filed Issue number Description
2017-04-03 SPL-140747 SSL connection in Python when using new ciphers may be slow.
2015-09-23 SPL-106978 Failed SHC captain election causes unnecessary change in server.conf
2015-03-11 SPL-97942 Capability defined in an app does not take affect when assigned to a role.

Workaround:
The workaround is to change the ui-prefs in ./etc/users/username/local/ui-prefs.conf to look like this:

[search] display.events.fields = ["description","except_extract_1","except_extract_2","except_extract_3","sap_order_status","sourcetype","source","status","request_mode","request_id","request_status_id","object_id","BillToCity_","Airline_","BillToName_","BillToCountry_","City_"] display.events.type = table

2013-05-02 SPL-66511 If $SPLUNK_HOME/etc is located on a case-insensitive filesystem, creating a new view with the same name as an existing view but with different case (capital letters vs lowercase, etc) silently overwrites the existing view.

Unsorted issues

Date filed Issue number Description
2017-08-03 SPL-143610 splunk_httpinput not honoring inputs.conf http stanza in UI
2017-05-23 SPL-141961 Older 6.0, 6.1, 6.2, 6.3 maintenance release versions unable to connect to 6.6.x via management port

Workaround:
This applies to License Master/Slave, Deployment Server/Client, Cluster Master/Peers, Search Head/Peers and affects Splunk 6.6.x and the following versions:
  • 6.0.0 to 6.0.6
  • 6.1.0 to 6.1.4
  • 6.2.0 to 6.2.6
  • 6.3.0 to 6.3.1
  • 6.3.1511.1

Upgrade your older instances to the latest maintenance releases or on your 6.6.x Splunk instances. Add the following stanza to server.conf:

[sslConfig]
sslVersions = *,-ssl2
sslVersionsForClient = *,-ssl2
cipherSuite = TLSv1+HIGH:TLSv1.2+HIGH:@STRENGTH


2017-05-23 SPL-141964 Older 6.0 and 6.1 maintenance release forwarders unable to forward events to 6.6.x indexers via splunktcp-ssl

Workaround:
This affects communication between Splunk 6.6.x indexers and:
  • 6.0.0 to 6.0.6 forwarders
  • 6.1.0 to 6.1.4 forwarders

Upgrade your older forwarders to the latest maintenance releases or on your 6.6.x Indexer, add to inputs.conf:

[SSL]
sslVersions = *,-ssl2
cipherSuite = ALL:!aNULL:!eNULL:!LOW:!EXP:RC4+RSA:+HIGH:+MEDIUM


2017-03-27 SPL-140442, SOLNESS-11786 In Splunk Enterprise 6.6.0 and later, with Enterprise Security 4.5.2 and 4.6.0, roles without "edit_roles" capability cannot perform operations on notable event review statuses.

Workaround:
If users cannot perform operations on notable event review statuses or have issues viewing "Edit all selected" links on Incident Review, user roles must be provided with the "edit_roles" capability.
2017-03-20 SPL-139019 Possible compatibility issues between Python / SDK clients and new 6.6 default sslVersions, cipherSuites

Workaround:
Users can do either of the following:

1. Overwrite the new Splunk 6.6 server.conf [sslConfig] sslVersions, cipherSuites with your own settings that are compatible with your version of OpenSSL, e.g. the previous defaults from 6.5.x are compatible with OpenSSL 0.9.8 on Mac OSX:

[sslConfig]
sslVersions = *,-ssl2
sslVersionsForClient = *,-ssl2
cipherSuite = TLSv1+HIGH:TLSv1.2+HIGH:@STRENGTH

2. For some more up-to-date clients, it is possible to enforce TLS1.2 (e.g. --tlsv1.2 for curl) in order to connect successfully.

3. Upgrade OpenSSL on your platform and link it with your client (e.g. Python, curl, etc..). For example, OpenSSL 1.0.2 is currently available on Mac OSX via Homebrew (see https://brew.sh) and is compatible with the new Splunk 6.6 default sslVersions, cipherSuites.

2017-03-14 SPL-138731 New 6.6 default SHA256/2048-bit key certificates are not compatible with previous versions SHA1/1024-bit key certificates if cert verification is enabled

Workaround:
Users can do any of the following:

1. Disable certificate verification - the same root certificate is available with every Splunk download so enabling certificate verification while using the default certificates provides very little additional security.

2. Generate new SHA256/2048-bit key certificates using the new 6.6 root certificate and distribute to older versions of Splunk

3. Generate SHA1/1024-bit key certificates using the old root certificate to use with your new 6.6 instance. For convenience, the old root certificate is included in 6.6 in $SPLUNK_HOME/etc/auth/prev_release/

2017-03-13 SPL-138647 Possible compatibility issues between new 6.6 default sslVersions, cipherSuites and external services, e.g. e-mail, LDAP

Workaround:
If security is not a significant concern, simply revert back to the 6.5.x SSL/TLS defaults, e.g. for e-mail, add to $SPLUNK_HOME/etc/system/local/alert_actions.conf

[email]
sslVersions = *,-ssl2
cipherSuite = TLSv1+HIGH:TLSv1.2+HIGH:@STRENGTH


If you would like to retain the more secure 6.6.x defaults, but prefer to add an exception for your less secure external services, follow the procedure below:

1. To determine what sslVersions and cipherSuites are supported by a server, run splunk cmd openssl s_client -connect hostname:port | awk '/Protocol/ || /Cipher/ || /Verify/'.

The example below is for a Postfix SMTP server:

eserv@indexer01:~$ splunk cmd openssl s_client -connect smtp-server01:465 | awk '/Protocol/ || /Cipher/ || /Verify/'
depth=1 C = US, O = Example Customer, OU = IT, CN = Example Customer IT CA, emailAddress = customer@example.org verify error:num=19:self signed certificate in certificate chain New, TLSv1/SSLv3, Cipher is DHE-RSA-AES256-SHA

   Protocol : TLSv1 
   Cipher : DHE-RSA-AES256-SHA 
   Verify return code: 19 (self signed certificate in certificate chain)

2. Check the OpenSSL output for Protocol and Cipher. In the example above, Protocol = TLSv1 and Cipher = DHE-RSA-AES256-SHA

3. Update Splunk's relevant sslVersions and/or cipherSuite. In the example above, sslVersions should be set to tls (allows TLSv1, TLSv1.1, TLSv1.2) and DHE-RSA-AES256-SHA should be appended to the end of the default cipherSuites definition, e.g. add $SPLUNK_HOME/etc/system/local/alert_actions.conf:

[email]
sslVersions = tls

cipherSuites = ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA

2017-02-16 SPL-136920 Exclusive use of Forward Secret Ciphers in server.conf prevents KV Store from working

Workaround:
The only workaround is to add non DH ciphers to the list, for example:

cipherSuite = AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256

First cipher on the list was added, KV store will be able to run.

2017-01-05 SPL-134638, SPL-143382, SPL-143398, SPL-143400, SPL-144110 Slow license master response times after upgrade to 6.5

Workaround:
Maybe: http://stackoverflow.com/a/17697733
2016-11-21 SPL-132670 Mac OS 10.11: disable boot-start doesn't remove the file /Library/LaunchAgents//com.splunk.plist by enabling boot-start in prior Splunk/UF
2016-08-31 SPL-136475 cloud index manager page does not show accurate dates of latest events
2016-06-29 SPL-138490 the index creation manager returns an invalid error that the index was not created because it already exists
2015-06-18 SPL-103302 Files ownership are failed to be changed when using debian package to install splunk and $SPLUNK_HOME is a symlink

Workaround:
Run a recursive chown from the command line on $SPLUNK_HOME manually, post install.
2015-06-01 SPL-102362 Dynamic indexer discovery only supports one input.
2015-05-24 SPL-102008 On Internet Explorer, a warning message does not display when you cannot log in due to a time zone difference.
2015-03-25 SPL-98594 Routing events to two different groups not working as expected.

Workaround:
1 On the original UF, instead of configuring 1 s2s and 1 syslog group, configure 2 s2s groups.

2 Setup a proxy UF which takes input from the original UF and send input out syslog server. This solution only requires config change and no patch release is required.

2014-11-10 SPL-92831 A mismatch of versions between the license-master and the license-slave is generating Warning messages like "WARN LMDirective - directive cmd=D_set_feature_state args='Acceleration,ENABLED' failed: reason='feature='Acceleration' is invalid' ."

Workaround:
The warnings can be ignored, the workaround is use same major versions (all on 6.2 or all on 6.1).


2014-10-17 SPL-92162 Writing large amounts of data (> 20 GB) to KV store collections using outputlookup can result in high memory usage on the machine.
2014-08-20 SPL-89640 When running Splunk on Linux as non-root user and using RPM to upgrade, the RPM writes $SPLUNK_HOME/var/log/introspection as root, causing errors upon restarts.

Workaround:
Chown the $SPLUNK_HOME/var/log/introspection directory to the user Splunk Enterprise runs as after upgrading and before restarting Splunk Enterprise.
2014-04-22 SPL-83365 Splunk Enterprise on Windows does not show an error message when a user without the edit_license capability tries to add a license through the CLI.
2014-03-12 SPL-81810 Licensing - license pool warning at license master keeps coming back after deleting it.

Workaround:
Delete the warnings on the peers first, then the License Manager.
2013-11-27 SPL-77139 Licenser pool usage gets reflected only after restarting splunkd.
2013-09-18 SPL-74427, SPL-74448 The Splunk universal forwarder installer for Solaris 10 does not add the splunk user when you attempt to install it using the pkgadd command. This results in the script generating lots of errors.

Workaround:
To work around this issue, create a splunk user on your system before attempting to run the installer.
2013-06-13 SPL-69304 If license slaves are running <6.0 version, they do not have the idx field and in theLicense Usage view, the split by index field will show a field named UNKNOWN.
2013-05-25 SPL-68010 The error thrown when your Splunk instance cannot connect to splunkbase/.../checkforupdate is not an ERROR, should be lowered to INFO.

Uncategorized issues

Date filed Issue number Description
2017-07-25 SPL-143331, SPL-144063, SPL-144366 default_match is not honoured when lookup matches is 0 (using a kvstore collection)
2017-06-27 SPL-142724, SPL-143409 Exact floating point calculations in eval may produce incorrect arithmetic groupings with search optimization enabled.

Workaround:
Workaround is to order the maths in the eval differently or disable search optimization with "| noop search_optimization=f"
2017-05-09 SPL-141693 DataModel Editor - when child object has same name as inherited field, inherited field does not show in the inherited fields list.
2017-02-27 SPL-137673, SPL-138969, SPL-138970 When running Health check on 6.5.2 Some linux items check on window some are skipped
2017-01-06 SPL-134707 Splunk restart does not create missing server.pem certificate on Windows

Workaround:
Use Template:Bin/splunk createssl server-cert -d etc/auth/ -n server to generate a new certificate.
2016-08-31 SPL-127800 opting in to data sharing on a monitoring console produces duplicate data
2016-07-26 SPL-125052 Sole Admin can demote his/herself to Power without path of recovery in GUI

Workaround:
Through the command line, you can open notepad and modify the password file to regain 'Admin' status.
2016-07-11 SPL-124026, SPL-122942 Relative paths should not be allowed under volume's path=file:// on remote storages
2015-11-13 SPL-109427 LDAP SSL no longer working in Splunk 6.3 (and later) for Windows 2003

Workaround:

The workaround is to 1) obtain Ciphers configured on Windows AD 2003 server. 2) tweak TLS_CIPHER_SUITE command in etc/openldap/ldap.conf to match it. The following is a working TLS_CIPHER_SUITE for one of the customers: {noformat} TLS_CIPHER_SUITE HIGH:MEDIUM:@STRENGTH:+3DES:+RC4:!aNULL:!MD5:!SRP:!PSK:!aDSS:!kECDH:!kDH:!SEED,!IDEA:!RC2:!RC5 {noformat}

2015-10-07 SPL-107606 Inconsistency between summary and datamodel_summary files.
2015-06-10 SPL-103010 Indexing throughput on a forwarder with four pipelinesets drops 30% compared to a forwarder with two pipelinesets.
2015-02-26 SPL-97389 When using timechart command, the embedded report shows different time format than the original report.
2015-01-08 SPL-95144, SPL-142789, SPL-101986, SPL-101987, SPL-106884, SPL-107317 Indexed message for Windows security event logs shows "FormatMessage error"

Workaround:
Splunk believes this was introduced in a Microsoft Windows patch. The workaround is to configure a delayed start of the Splunk service(s) so that it starts after the Windows Event Log service.
2014-10-31 SPL-92596 After upgrade from Splunk Enterprise 6.1 or earlier to 6.4.x on Windows, splunkweb service does not start automatically. Attempts to start it manually show "Error 1053: The service did not respond to the start or control request in a timely fashion."

Workaround:
This is expected behavior. See the Splunk Answers post: http://answers.splunk.com/answers/177187/why-is-the-splunk-web-service-not-running-after-an.html
2014-09-11 SPL-90738 Monitoring a directory with an unknown sourcetype produces indexing errors.
2014-08-26 SPL-90139 <timestamp> does not display in the Patterns tab when searches are run in fast mode.
2014-04-01 SPL-82517 Paper Size and Layout in PDF Schedule dialog do not respect Paper Size and Layout in Email Settings.
2014-03-23 SPL-82238 Datamodel fails to drill down further when the same attribute for Split Rows and Split Columns are selected.
2014-03-13 SPL-81856 Show all lines does not work in data model editor preview.
2014-03-12 SPL-81781 In the Data Model Manager, "Acceleration Status" and "Access Count" fail to update when you click "Update".
2014-02-07 SPL-80285 In the Data Model Editor, the Edit Lookup page is blank if Lookup is shared only in Lookup Definitions.

Workaround:
For more information, see Add lookup files to Splunk.
2014-02-06 SPL-80187 In the Data Model Editor, lookup pages open with options displayed for other Lookup when the data model definition is private but the file is app or globally shared.

Workaround:
Share the definition. For more information, see Add lookup files to Splunk.
2013-09-13 SPL-74337, BETA-496 You cannot specify a destination folder when installing on OSX.
2013-04-30 SPL-66213 PDF server app is not working with latest Xvfb
2012-02-22 SPL-48342 LDAP strategy host field cannot work with ipv6 format address but computer name is okay

Splunk Analytics for Hadoop

Date filed Issue number Description
2017-04-04 ERP-2040 Splunk archiving fails for large block sizes (buckets) due to HDFS write crashes.
2015-09-09 ERP-1650 timestamp data type not properly deserialized.
2015-08-05 ERP-1619 Searching on a newly created archive index before the bucket copy saved search is run causes a filenotfound exception.

Workaround:
Reenable the bucket copy saved search and let it run, or force the archiving to happen via | archivebuckets force=1 and then rerun the search.
2015-07-07 ERP-1598 minsplit rampup - splits generation takes too long.

Workaround:
Set minsplits=maxsplits
2015-05-12 ERP-1502 Non-accelerated pivot search on Pivot UI page waits for a long time to return result.
2015-01-08 ERP-1343, SPL-95174 Splunk Analytics for Hadoop searches fail on corrupted journal.gz files, although Splunk searches run without error.

Workaround:
Add the journal.gz to the input path's blacklist (vix.input.1.ignore = ....)
2014-10-27 ERP-1216 Data Explorer preview does not honor existing sourcetypes for big5/sjis files.
2014-10-03 ERP-1164 Report acceleration summary gets deleted when two Splunk Analytics for Hadoop instances point to the same Splunk working directory

Workaround:
To mitigate this issue, make sure that vix.splunk.home.hdfs (or Working directory in the UI) is unique on both search heads that are not in a pool. To keep your instances in the same working directory, configure vix.splunk.search.cache.path to be unique on both search heads.
PREVIOUS
Welcome to Splunk Enterprise 6.6
  NEXT
Splunk Enterprise and anti-virus products

This documentation applies to the following versions of Splunk® Enterprise: 6.6.2


Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters