Splunk Enterprise version 7.1 is no longer supported as of October 31, 2020. See the Splunk Software Support Policy for details. For information about upgrading to a supported version, see How to upgrade Splunk Enterprise.
This documentation does not apply to the most recent version of Splunk® Enterprise.
For documentation on the most recent version, go to the latest release.
Known issues
The following are issues and workarounds for this version of Splunk Enterprise.
Issues are listed in all relevant sections. Some issues appear more than once. To check for additional security issues related to this release, visit the Splunk Security Portal.
Refer to System requirements in the Installation Manual for a list of supported platforms and architectures.
For a list of deprecated features and platforms, refer to Deprecated features in this manual.
Highlighted issues
| Date filed | Issue number | Description |
|---|---|---|
| 2018-08-29 | SPL-159442, SPL-156444 | Searches may take considerably more memory than with 7.0.x or earlier. This applies particularly to searches that search and/or return a large result set. Due to search speed performance improvements some memory usage increase is expected with 7.1.x even after this issue is fixed.
Splunk recommends upgrading to version 7.1.4 or later. |
| 2018-06-14 | SPL-155560, SPL-155219 | DMA accelerating too much data when acceleration.backfill_time unset, resulting in heavy indexer load Workaround: acceleration.backfill_time needs to be set for all DMA searches in datamodels.conf. Otherwise the DMA will attempt to accelerate for "all time". |
| 2018-05-04 | SPL-154138, SPL-154542, SPL-154544, FAST-9662 | Searches with multikv extraction use too much memory: potentially orders of magnitude more than previous versions. |
Highlighted issues
| Date filed | Issue number | Description |
|---|---|---|
| 2018-12-07 | SPL-163753, SPL-162781 | KV store is unavailable after upgrading a search head cluster with eight or more members from 7.0.x or lower to 7.1.x or 7.2.x. Workaround: To avoid this issue, perform the following steps 1, 2 and 3 before upgrading using steps 4 and 5.
If you have already upgraded without performing the previous procedures, you can identify this issue as follows:
If you encounter this issue, contact Splunk support for recovery steps. |
| 2018-11-10 | SPL-162781, SPL-163104, SPL-163753, SPL-162810, SPL-163103 | KV store is unavailable after upgrading a search head cluster with eight or more members from 7.0.x or lower to 7.1.x or 7.2.x. Workaround: To avoid this issue, perform the following steps 1, 2 and 3 before upgrading using steps 4 and 5.
If you have already upgraded without performing the previous procedures, you can identify this issue as follows:
If you encounter this issue, contact Splunk support for recovery steps. |
Upgrade issues
| Date filed | Issue number | Description |
|---|---|---|
| 2018-04-13 | SPL-153403 | After running the "clean userdata" command, admin is unable to login with msg "No users exist. Please set up a new user." Workaround: Create a $SPLUNK_HOME/etc/system/local/user-seed.conf and restart Splunk [user_info] |
| 2017-05-23 | SPL-141961 | Older 6.0, 6.1, 6.2, 6.3 maintenance release versions unable to connect to 6.6.x and later via management port. Workaround: This applies to License Master/Slave, Deployment Server/Client, Cluster Master/Peers, Search Head/Peers and affects Splunk 6.6.x and the following versions:
Upgrade your older instances to the latest maintenance releases or on your 6.6.x Splunk instances. Add the following stanza to server.conf: [sslConfig]
|
| 2017-03-20 | SPL-139019 | Possible compatibility issues between Python / SDK clients and new 6.6 and later default sslVersions, cipherSuites Workaround: Users can do either of the following: 1. Overwrite the new Splunk 6.6 server.conf [sslConfig] sslVersions, cipherSuites with your own settings that are compatible with your version of OpenSSL, e.g. the previous defaults from 6.5.x are compatible with OpenSSL 0.9.8 on Mac OSX: [sslConfig] 2. For some more up-to-date clients, it is possible to enforce TLS1.2 (e.g. --tlsv1.2 for curl) in order to connect successfully. 3. Upgrade OpenSSL on your platform and link it with your client (e.g. Python, curl, etc..). For example, OpenSSL 1.0.2 is currently available on Mac OSX via Homebrew (see https://brew.sh) and is compatible with the new Splunk 6.6 default sslVersions, cipherSuites. |
| 2017-03-13 | SPL-138647 | Possible compatibility issues between new 6.6 and later default sslVersions, cipherSuites and external services, e.g. e-mail, LDAP Workaround: If security is not a significant concern, simply revert back to the 6.5.x SSL/TLS defaults, e.g. for e-mail, add to $SPLUNK_HOME/etc/system/local/alert_actions.conf
[email]
TLS_PROTOCOL_MIN 3.1
The example below is for a Postfix SMTP server: eserv@indexer01:~$ splunk cmd openssl s_client -connect smtp-server01:465 | awk '/Protocol/ || /Cipher/ || /Verify/' Protocol : TLSv1 Cipher : DHE-RSA-AES256-SHA Verify return code: 19 (self signed certificate in certificate chain) 2. Check the OpenSSL output for Protocol and Cipher. In the example above, Protocol = TLSv1 and Cipher = DHE-RSA-AES256-SHA 3. Update Splunk's relevant sslVersions and/or cipherSuite. In the example above, sslVersions should be set to tls (allows TLSv1, TLSv1.1, TLSv1.2) and DHE-RSA-AES256-SHA should be appended to the end of the defaultcipherSuites definition, e.g. add $SPLUNK_HOME/etc/system/local/alert_actions.conf :
[email] |
| 2014-08-20 | SPL-89640 | When running Splunk on Linux as non-root user and using RPM to upgrade, the RPM writes $SPLUNK_HOME/var/log/introspection as root, causing errors upon restarts Workaround: Chown the $SPLUNK_HOME/var/log/introspection directory to the user Splunk Enterprise runs as after upgrading and before restarting Splunk Enterprise. |
| 2013-08-19 | SPL-73386 | Users are not allowed to run historical scheduled search Workaround: 1. Create a special power/admin user who can run scheduled searches. 2. Assign this user ownership of the scheduled searches. 3. Share the searches at the app level and grant read/write permission to the correct set of users. |
Data input issues
| Date filed | Issue number | Description |
|---|---|---|
| 2019-11-04 | SPL-178913, SPL-171961 | The datetime.xml timestamp recognition file does not recognize two-year dates after 2019 or Unix epoch-time seconds higher than 1599999999 (12:26:39 UTC 13 Sep 2020) |
| 2018-07-26 | SPL-157923, SPL-147638 | Splunkd crashes when HEC inputs configuration contains duplicated tokens |
| 2018-07-05 | SPL-156817 | HEC json file give "Invalid data format" on 7.x versions with event sizes greater than 512kb |
| 2018-04-19 | SPL-153591, SPL-155066, SPL-155067, SPL-155069 | high delay on events from UF after upgrade to (6.6.x) |
| 2018-03-27 | SPL-152628 | PREAMBLE_REGEX doesn't work on 7.0.2 but OK with 7.0.0 |
| 2018-03-15 | SPL-152197, SPL-147327 | Error message for corrupted FSChangeMonitor database is not actionable. |
| 2017-07-19 | SPL-143236 | Custom sourcetype is not displayed on sourcetype menu Workaround: Set a filter and the sourcetype will display. |
| 2015-11-12 | SPL-109362 | When the disk runs out of space for the limit set in the server.conf, add data workflow gets stuck with "Uploading file" message modal in the review stage |
| 2015-05-22 | SPL-101981 | Field extractions do not work when sourcetypes use quotes in the Getting Data In interface. |
| 2015-03-17 | SPL-98163 | INDEXED_EXTRACTIONS=W3C is truncating field cs_uri_stem when spaces are present in URL Workaround: Create a separate extraction in props.conf where defined w3c extraction method: EXTRACT-cs_uri_stem1 = (GET|POST) (?<cs_uri_stem1>[^-]++) |
Search issues
| Date filed | Issue number | Description |
|---|---|---|
| 2020-01-02 | SPL-181330, SPL-181303 | Rex mode sed - 7.1.0+ - Sed with caret (^) is giving an incorrect result/not functioning as expected Workaround: - To remove the global flag at the end of the sed command i.e. Instead of: rex mode=sed field=test "s/^/\"/g" Do:
rex mode=sed field=test "s/^/\"/" |
| 2020-01-02 | SPL-181332, SPL-181303 | Rex mode sed - 7.1.0+ - Sed with caret (^) is giving an incorrect result/not functioning as expected Workaround: - To remove the global flag at the end of the sed command i.e. Instead of: rex mode=sed field=test "s/^/\"/g" Do:
rex mode=sed field=test "s/^/\"/" |
| 2020-01-02 | SPL-181331, SPL-181303 | Rex mode sed - 7.1.0+ - Sed with caret (^) is giving an incorrect result/not functioning as expected Workaround: - To remove the global flag at the end of the sed command i.e. Instead of: rex mode=sed field=test "s/^/\"/g" Do:
rex mode=sed field=test "s/^/\"/" |
| 2019-07-26 | SPL-173898, SPL-173452 | search time increases exponentially or factorially with number of subsearches |
| 2019-07-08 | SPL-172836, SPL-171270 | dedup's sortby not working as expected when using head/transaction |
| 2019-06-21 | SPL-172299, SPL-168859 | Any transformational commands will not include the base fields from transforms.conf when performing search in SMART mode resulting in required field not been included |
| 2019-04-05 | SPL-168797, SPL-166413 | info_max/min_time provides incorrect values when search is real time. |
| 2019-02-26 | SPL-166969, SPL-158113, SPL-166657 | Lookups elide empty string literals |
| 2019-02-25 | SPL-166897, UBA-11952 | After splunk restart, the first threat or anomaly does not get sent to ES Workaround: The first threat send from UBA to Splunk after the latter is restarted, will be missed. Please manually export the missed threat to Splunk ES again. Or
Always restart OCS after Splunk gets restarted. |
| 2019-01-16 | SPL-165046, SPL-165326 | Search crashes due to missing name in EVAL- in props.conf |
| 2018-12-18 | SPL-164112 | Characters with accents not substituting properly with sed mode |
| 2018-11-30 | SPL-163361, SPL-164567, SPL-164879 | mvexpand consumes more memory than expected, error: command.mvexpand: output will be truncated at <low number> results due to excessive memory usage. |
| 2018-11-07 | SPL-162658, SPL-168783, SPL-169607, SPL-170159 | Editing Summary Indexing not working when Search contains a tstats |
| 2018-11-01 | SPL-162447, SPL-154678 | |metadata search error - Failed to apply deletes to some metadata |
| 2018-10-05 | SPL-161000 | CPU stalls occurs which results in splunkd core dumping Workaround: No workaround is found |
| 2018-10-01 | SPL-160881, SPL-161173, SPL-161174, SPL-170371, SPL-170372 | eventstats on an event search can create duplicate events in some scenarios |
| 2018-09-24 | SPL-160449, SPL-149404 | Search.log error message asks user to consider increasing match limit for a Regex without a reason |
| 2018-09-12 | SPL-159979, SPL-161169, SPL-163063 | Crashing Thread: TcpChannelThread - Post process search using stats fields with null crashes Splunk Workaround: exclude the nulls |
| 2018-08-28 | SPL-159414, SPL-159182 | Memory growth with transactions and keeporphans |
| 2018-08-26 | SPL-159318, SPL-159666, SPL-160523 | search process crash in AST due to subsearch in a saved search Workaround: Two things needed: 1. in limits.conf - disable phased execution: [search] phased_execution_mode = singlethreaded 2. Disable search optimizations - add the below to the search (or via limits.conf) | noop search_optimization=false |
| 2018-08-16 | SPL-158934, SPL-159726, SPL-159751 | Post 7.1.1 upgrade issue: stats aggregating in additional empty records with mix of prestats and event data |
| 2018-08-10 | SPL-158581, SPL-157433 | lookup OUTPUTNEW commands mistakenly cause optimizer to remove preceding search commands resulting in missing field values. Workaround: Following 4 workarounds are known: 0. configure following in limits.conf of all SHs affected: [search_optimization::projection_elimination] cmds_black_list = lookup This is the most accurate workaround to use. 1. | inputlookup bugtest_lookupstats.csv | rex field=threat_collection_key "\|(?<threat_value>.+)" | search threat_value="0.0.0.0" | table threat_value | lookup bugtest_lookup.csv IP as threat_value OUTPUTNEW time | lookup bugtest_lookup.csv DOMAIN as threat_value OUTPUTNEW time | table threat_value time | noop search_optimization=false This explicitly disables the search optimisation. 2. | inputlookup bugtest_lookupstats.csv | rex field=threat_collection_key "\|(?<threat_value>.+)" | search threat_value="0.0.0.0" | table threat_value | lookup bugtest_lookup.csv DOMAIN as threat_value OUTPUTNEW time | lookup bugtest_lookup.csv IP as threat_value OUTPUTNEW time | table threat_value time This variant simply inverts the 2 lookup commands. 3. | inputlookup bugtest_lookupstats.csv | rex field=threat_collection_key "\|(?<threat_value>.+)" | search threat_value="0.0.0.0" | table threat_value | lookup bugtest_lookup.csv IP as threat_value OUTPUTNEW time | lookup bugtest_lookup.csv DOMAIN as threat_value OUTPUTNEW time | table threat_value time* This variant uses *time* * instead of *time*. |
| 2018-07-31 | SPL-158113, SPL-166657, SPL-166967, SPL-166969 | Explicit empty strings in lookups being returned as null since 7.1.0, don't show as part of a multivalue field on repeated matches. |
| 2018-07-09 | SPL-160683, SPL-164645, SPL-164880 | Error message on ES App's IR Dashboard when editing notable events due to inconsistent availableCount caused by Timeliner failure to write the events to disk |
| 2018-06-22 | SPL-156141, SPL-146147 | Search crashes when using lookup tables that are frequently updated Workaround: On the crashing peer (could be SH, Indexer or both) set the below in limits.conf: max_memtable_bytes = 2*<size of the largest lookup> example search to find the biggest lookups: index=_* sourcetype=audittrail path=*lookups* size=*
| stats max(size) AS size BY host, path
| append
[| rest services/server/introspection/kvstore/collectionstats
| mvexpand data
| table splunk_server title data
| spath input=data
| fields splunk_server size ns ]
| eval host=coalesce(host,splunk_server)
| fields host path ns size
| sort size | head 1 |
| 2018-06-15 | SPL-155648, SPL-169611, SPL-169612, SPL-185656 | New phased_execution_mode is spawning extra processes for custom search commands Workaround: If the custom search needs to run only once, disable the multithread for all searches. $SPLUNK/etc/system/local stopped the issue from occurring. [search] phased_execution_mode = auto Apply this workaround especially for deployment using ITSI, as the bug causes double backfill of the ITSI Episodes. Beware, the workaround will cause a separate search issue SPL-165363, for splunk versions 7.0, 7.1 and 7.2 until the fix in 7.2.4 |
| 2018-06-04 | SPL-155106, SPL-155412, SPL-155413 | splunkd process consuming large amount of memory in 7 |
| 2018-05-30 | SPL-154973, SPL-155773, SPL-158832 | timeline preview shows random events, but not the ones based on the selected timeline segment Workaround: Appending following to the SPL seems to do the trick: | sort - _time |
| 2018-05-27 | SPL-154920, SPL-156245, SPL-159249 | Search Removal With Case Insensitive Capability Workaround: To use the same letter case as the search function. |
| 2018-05-14 | SPL-154459, SPL-153958 | The mcollect and meventcollect commands fail to verify permissions for the indexes that they write to. |
| 2018-05-14 | SPL-154463, SPL-154931 | When eventstats is the last command in a reporting search in Splunk 7.1.0 the stats tab truncates all results past a certain number of results. |
| 2018-05-11 | SPL-154420, SPL-153686 | invalid regex range causing splunk to crash |
| 2018-05-09 | SPL-154314, SPL-156264, SPL-160168 | Timechart time modifiers not respected by time picker Workaround: [search] phased_execution_mode = singlethreaded
|
| 2018-05-04 | SPL-154138, SPL-154542, SPL-154544, FAST-9662 | Searches with multikv extraction use too much memory: potentially orders of magnitude more than previous versions. |
| 2018-04-27 | SPL-157120, SPL-158035 | Customer upgrade to splunk 7.1 and this broke his HUNK Archive index. |
| 2018-04-25 | SPL-153745, SPL-153724 | The mcollect and meventcollect commands erroneously count against licensing. |
| 2017-08-23 | SPL-144350 | Archived Index is created without error when the splunk index is invalid |
| 2017-08-03 | SPL-143607 | Searches ordered like this returns false results: "search ... | eventstats count | delta _time as d" , because it's being run in batch mode when it shouldn't Workaround: Place the delta command before eventstats in the search pipeline:
|
| 2017-07-13 | SPL-143111 | "Splunkd daemon is not responding" when edit local windows event log collection |
| 2017-04-04 | SPL-140765 | Splunk having problems extracting json file consisting of 68k plus key-value pairs |
| 2017-03-21 | SPL-140175 | Aborted delete searches may result in stale lock files being left behind Workaround: Delete stale lock files. |
| 2016-11-29 | SPL-133182 | When two datasets have identical names but one is local (private) while the other is global, attempts to view or extend the global dataset use results from the local dataset instead. |
| 2016-06-17 | SPL-122984 | Searching renamed sourcetype is case-sensitive |
| 2015-08-10 | SPL-105061, SOLNESS-7274 | Broken module prevents splunkweb from starting |
| 2015-06-17 | SPL-103247 | Filtering on _time uses different semantics for the "=" operator on microseconds depending on whether the value is quoted. |
| 2015-04-23 | SPL-100170 | Automatic Lookups limitation: No results returned in Smart Mode when there are nested lookups and the intermediate field is not mentioned in the search. |
| 2014-12-22 | SPL-94910 | The replace function does not apply to fields names with an underscore in them. Workaround: Rename the fields before the replace. ... | rename *_* AS *-* | replace "something" by "somethingelse" |
| 2014-11-13 | SPL-93039 | The relevancy search command does not work, always returning 0 or -inf. |
| 2014-10-02 | SPL-91638, SPL-107375 | For scheduled searches in a search head cluster, empty search jobs may appear in the job inspector for a cluster member. |
| 2014-09-15 | SPL-90861, SPL-90396, SPL-90886 | If search encounters invalid offsets or invalid rawdata at TSIDX offsets, it skips reading any number of events from that bucket. No message is displayed, though the information is added to search.log. |
| 2014-04-16 | SPL-83129 | Eval function strptime does not return results when 1970 date is used. |
| 2014-04-04 | SPL-82650 | A report created and scheduled by admin cannot be embedded by a power user. |
| 2014-03-27 | SPL-82357 | The splunk clean all -f CLI command doesn't remove data from the main index on Windows systems. |
| 2014-03-15 | SPL-81934 | For clusters, may be unable to open search results output file for search results in a cluster. Workaround: Write to a temp file and rename to the target file. |
| 2014-02-21 | SPL-80942 | Flashtimeline: 500 Internal Server Error when pasting long URL into panel name. |
| 2013-12-18 | SPL-78179 | REST /saved/searches App names with special characters have invalid links. |
| 2013-08-19 | SPL-73386 | Users are not allowed to run historical scheduled search Workaround: 1. Create a special power/admin user who can run scheduled searches. 2. Assign this user ownership of the scheduled searches. 3. Share the searches at the app level and grant read/write permission to the correct set of users. |
Saved search, alerting, scheduling, and job management issues
| Date filed | Issue number | Description |
|---|---|---|
| 2019-07-18 | SPL-173414, SPL-179491, SPL-179612, SPL-179627 | Splunk unable to load defined Saved Searches in a conf file if a bad/malformed cron_schedule value is present/set Workaround: If this behavior is noticed (savedsearches - alerts/searches/reports) not showing up in the GUI, verify cron jobs are valid, if they're not, then correct them. A quick check would be to create a duplicate savedsearches.conf and removing all cron_schedule definitions and rebooting splunk with that config to identify if it is this issue. |
| 2019-03-22 | SPL-168110, SPL-164733 | tstats searches do not run on datamodels that contain only a streamable BaseSearch object |
| 2019-03-22 | SPL-168124, SPL-161055 | Internal Error: datamodel - invalid or unaccelerable root object |
| 2018-11-20 | SPL-163095, SPL-163054 | Scheduler crashed due to divide by zero exception (FPE) when allow_skew is configured |
| 2018-10-25 | SPL-162249, SPL-170857, SPL-177527, SPL-177529, SPL-183262 | The filter function of <splunk-search-dropdown> UI component is not working on on Splunk Enterprise 7.1 and later. |
| 2018-09-04 | SPL-159602, SPL-159053 | Trigger Time format in alert emails without AM/PM designators and no Timezone information. |
| 2018-09-04 | SPL-159604, SPL-159053 | Trigger Time format in alert emails without AM/PM designators and no Timezone information |
| 2018-07-11 | SPL-157118, SPL-161721, SPL-164488, SPL-164489 | invisible datamodels /data/models/._*.json files are causing the manager to fail finding the datamodel definition Workaround: in the apps folders, data/models, find and delete all the ._*.json files |
| 2018-06-06 | SPL-155219, SPL-155560 | DMA accelerating too much data when acceleration.backfill_time unset, resulting in heavy indexer load Workaround: acceleration.backfill_time needs to be set for all DMA searches in datamodels.conf. Otherwise the DMA will attempt to accelerate for "all time". |
| 2018-05-04 | SPL-154136, SPL-154836 | Duplicate alerts are triggered for real time alert type on Splunk Enterprise 7.1.0 |
| 2018-04-21 | SPL-153649, SPL-156991, SPL-157792, SPL-157793 | Search scheduler shifts earliest_time and latest_time based on the skew, when using allow_skew Workaround: Don't use allow_skew for searches where this behaviour is a problem. |
| 2017-12-13 | SPL-147319, SPL-154403, SPL-154405 | SHC AuthenticationManagerLDAP complains "Could not find user="system"" flooding splunkd.log Workaround: + creating a local user called "system" would clear the INFO logging + or customer can turn off INFO logging by setting logging level to NOTICE or above:
splunk set log-level AuthenticationManagerLDAP -level NOTICE |
| 2017-11-29 | SPL-146802 | Distributed environment requires index defined on search head for log event alerts |
| 2015-11-15 | SPL-109471 | For Real Time Scheduled Search in search head cluster, alerts are triggered twice when members cannot HB to captain |
| 2015-04-09 | SPL-99421 | Long name of app causes accelerated search to not complete normally and shows invalid results on Windows 2008 R2 Workaround: Reduce length of name of the app and report acceleration searches will run properly within the context of the app. |
| 2014-08-15 | SPL-89332 | Report acceleration summaries do not show in Settings when you have hundreds of reports accelerated. |
| 2014-08-05 | SPL-88396 | After configuring a client name for a deployment client, the name is not shown in the Forwarder Management UI Workaround: Create a server class, where you can see the client name, and use that group when you add data. |
| 2014-05-01 | SPL-83686 | Data Model Pivot: Extra NULL column displays in Pivot with big data and Numbered Attribute in Split Columns. Workaround: The workaround is to add filter status=*, or make a more refined Data Model that has an object for events with status. |
| 2014-03-24 | SPL-82262, SPL-82241 | Pivot search command fails for an admin trying to pivot on a Private Data Model created by a User. |
| 2014-03-20 | SPL-82164 | Migrating invalid data models from 6.0 to 6.x fails. |
| 2014-03-19 | SPL-82133 | Data model allows users to upload a JSON file which has Field names with spaces but will not validate it. |
| 2014-03-10 | SPL-81645 | Creating data model with root transaction name starting with root event name fails |
| 2014-03-10 | SPL-81637 | Splunkd preview runs indefinitely on any file preview with "DATETIME_CONFIG=none". |
| 2013-11-26 | SPL-77054, SPL-77055 | Data model objects that have names starting with an underscore character ("_") do not work correctly and cannot be used in Pivot. |
Charting, reporting, and visualization issues
| Date filed | Issue number | Description |
|---|---|---|
| 2020-01-23 | SPL-182114, SPL-179348 | autoLB not switching IDX when reaching frequency limit Workaround: Reduce maxKBps on the UF - this was tested in the customer environment and showed some improvement in IDX switching. DEV also suggested increasing the number of pipelines on the UF, though this has not been verified in the customer deployment as far as I'm aware. |
| 2020-01-23 | SPL-182113, SPL-179348 | autoLB not switching IDX when reaching frequency limit Workaround: Reduce maxKBps on the UF - this was tested in the customer environment and showed some improvement in IDX switching. DEV also suggested increasing the number of pipelines on the UF, though this has not been verified in the customer deployment as far as I'm aware. |
| 2019-12-19 | SPL-181194, SPL-179348 | autoLB not switching IDX when reaching frequency limit Workaround: Reduce maxKBps on the UF - this was tested in the customer environment and showed some improvement in IDX switching. DEV also suggested increasing the number of pipelines on the UF, though this has not been verified in the customer deployment as far as I'm aware. |
| 2019-01-15 | SPL-164920, SPL-166952, SPL-167850, SPL-169010, SPL-169011 | Dashboard issue: Multiselect URL retains single value after Hide Filters selected |
| 2018-05-02 | SPL-154054, SPL-163446, SPL-164721 | Dashboard Editor in de-DE locale CSS error in Format visualization modal for Stats Table/Line/Bar Charts |
| 2018-04-30 | SPL-153976, SPL-157687 | Splunkd Crashes When Opening A Simple Dashboard |
| 2017-12-06 | SPL-147115 | Drilldown search fails when a timeformat is specified Workaround: Remove the timeformat specification from the drilldown search or manually remove the search from the URL and run it in a new window. |
| 2016-09-15 | SPL-128819, SPL-130243, SPL-130245 | Editing panel in dashboard removes charting.legend.masterlegend option Workaround: Use <option name="charting.legend.masterLegend">null</option> |
| 2016-04-27 | SPL-118911 | In SHC, referenced saved real-time searches in a dashboard do not stream results. Workaround: See Troubleshoot referenced real-time searches for workaround details.
|
| 2015-02-23 | SPL-97193 | The initial value for Multiselect input does not display properly in Visualizations Editor if input has empty string. |
Distributed search and search head clustering issues
| Date filed | Issue number | Description |
|---|---|---|
| 2019-11-11 | SPL-179351 | loadjob fails when loading a job using savedsearch name - for specific regexes used in search string |
| 2019-10-25 | SPL-178412, SPL-155281 | Indexer Clustering Search Performance - search manifest updates should be locked per site+genid |
| 2018-12-14 | SPL-164011, SPL-164677, SPL-164731, SPL-164732 | SHC: when captain node is in AutomaticDetention status, all alerts (scheduled searches) appear to have stopped as well. |
| 2018-10-26 | SPL-162318, SPL-162906, SPL-163487, SPL-163488, SPL-163751 | DispatchReaper fails to reap artifacts from fill_summary_index.py in SH Cluster Workaround: ** Manual deletion ** To get sidlist.txt . splunk search '|REST /services/search/jobs label=searchname | table sid' --maxout 0 --preview To delete dispatch dirs.
for i in `cat sidlist.txt`
do
rm -rf $i
done |
| 2018-06-15 | SPL-155639, SPL-154654 | SHC captain stops delegating DMA searches after a delegated DMA search job fails (status=delegated_remote_completion, success=0). |
| 2018-05-23 | SPL-154830, SPL-141363 | Indexers report "Unknown search command" for external search commands even though the indexers contain the search bundle with the external command Workaround: Use any of the following 3 workarounds: 1. Transform the "| command" part of the search into "| script command"
2. Transform the "| command" part of the search into "| localop | command"
3. Distribute the app to the indexers via the CM. |
| 2018-05-03 | SPL-154089, SPL-154739 | Search heads may fail with "Skip search X during searchable rolling process" in invalid configurations where they communicate with cluster masters in an older version. |
| 2018-05-01 | SPL-154032, SPL-154067, SPL-154926, SPL-156192 | SHC bundle rejected at push-time because of built-in apps warning is still created and picked up by SHC members Workaround: * Remove the bundle on SHC deployer, e.g. $SPLUNK_HOME/var/run/splunk/deploy/apps/search-0f00e250ca395564de84b53b3ae644617d2d3860.bundle
|
| 2018-04-03 | SPL-152935, SPL-154616, SPL-154617, SPL-154618 | KVStore Replication Error: replSetReconfig got BadValue _id field value of 256 is out of range |
| 2018-03-14 | SPL-152148 | KV store replication fails on the upgrade search head during SHC member-by-member upgrade. Workaround: To ensure there is no kvstore activity during upgrade, perform an offline upgrade as follows:
|
| 2017-11-29 | SPL-146802 | Distributed environment requires index defined on search head for log event alerts |
| 2017-03-13 | SPL-138654 | Splunk searches fail when filepath gets too long on Windows |
| 2016-07-12 | SPL-124085 | On Search Head Cluster It is not possible to remove an App from the SHs once it has been disabled. |
| 2015-11-15 | SPL-109471 | For Real Time Scheduled Search in search head cluster, alerts are triggered twice when members cannot HB to captain |
| 2015-09-23 | SPL-106978 | Failed SHC captain election causes unnecessary change in server.conf |
| 2015-02-26 | SPL-97385 | $SPLUNK_HOME/var/run/splunk/snapshot contains large tarballs in the presence of large ES lookup table files. Workaround: The allowable size of the download can be increased by setting the following in server.conf. [httpServer] max_content_length = 1500MB The other option is to disable the search which controls the generation of the large lookup file. In this case, the search is: [Endpoint - Local Processes Tracker - Lookup Gen] |
| 2014-08-25 | SPL-90028 | Using "inputcsv dispatch=true" to read a CSV from a dispatch directory may not work on search head cluster members that have a replica of the desired artifact. |
| 2014-08-14 | SPL-89131 | In a search head cluster, the search Job management page on cluster member doesn't immediately reflect 'isSaved' state after you click Save. |
| 2014-08-02 | SPL-88228 | When user clicks on the RSS feed for an alert, search pool information is not displayed. Individual pool member information is displayed, however. |
Data model and pivot issues
| Date filed | Issue number | Description |
|---|---|---|
| 2019-03-22 | SPL-168124, SPL-161055 | Internal Error: datamodel - invalid or unaccelerable root object |
| 2019-03-22 | SPL-168110, SPL-164733 | tstats searches do not run on datamodels that contain only a streamable BaseSearch object |
| 2018-07-11 | SPL-157118, SPL-161721, SPL-164488, SPL-164489 | invisible datamodels /data/models/._*.json files are causing the manager to fail finding the datamodel definition Workaround: in the apps folders, data/models, find and delete all the ._*.json files |
| 2018-06-06 | SPL-155219, SPL-155560 | DMA accelerating too much data when acceleration.backfill_time unset, resulting in heavy indexer load Workaround: acceleration.backfill_time needs to be set for all DMA searches in datamodels.conf. Otherwise the DMA will attempt to accelerate for "all time". |
| 2017-12-13 | SPL-147319, SPL-154403, SPL-154405 | SHC AuthenticationManagerLDAP complains "Could not find user="system"" flooding splunkd.log Workaround: + creating a local user called "system" would clear the INFO logging + or customer can turn off INFO logging by setting logging level to NOTICE or above:
splunk set log-level AuthenticationManagerLDAP -level NOTICE |
| 2014-12-08 | SPL-94047, SPL-98628 | While creating a Pivot and using the _time column as a Split column, the table columns aren't formatted in a human readable way, but displayed with the epoc timestamp.It works when using _time as a 'Split Row' column. |
| 2014-05-01 | SPL-83686 | Data Model Pivot: Extra NULL column displays in Pivot with big data and Numbered Attribute in Split Columns. Workaround: The workaround is to add filter status=*, or make a more refined Data Model that has an object for events with status. |
| 2014-03-24 | SPL-82262, SPL-82241 | Pivot search command fails for an admin trying to pivot on a Private Data Model created by a User. |
| 2014-03-20 | SPL-82164 | Migrating invalid data models from 6.0 to 6.x fails. |
| 2014-03-19 | SPL-82133 | Data model allows users to upload a JSON file which has Field names with spaces but will not validate it. |
| 2014-03-11 | SPL-81701 | Data Model Pivot, "Legend Position" and "Stack Mode" change to default settings if you change the X/Y-Axis more than once. |
| 2014-03-10 | SPL-81645 | Creating data model with root transaction name starting with root event name fails |
| 2014-03-07 | SPL-81538 | When using Pivot, stack mode is lost when "Scatter Chart" is selected. |
| 2013-11-26 | SPL-77054, SPL-77055 | Data model objects that have names starting with an underscore character ("_") do not work correctly and cannot be used in Pivot. |
Indexer and indexer clustering issues
| Date filed | Issue number | Description |
|---|---|---|
| 2019-10-25 | SPL-178413, SPL-155281 | Indexer Clustering Search Performance - search manifest updates should be locked per site+genid |
| 2019-10-25 | SPL-178414, SPL-155281, SPL-179523 | Indexer Clustering Search Performance - search manifest updates should be locked per site+genid |
| 2019-10-25 | SPL-178412, SPL-155281 | Indexer Clustering Search Performance - search manifest updates should be locked per site+genid |
| 2019-05-29 | SPL-171257, SPL-171303 | Index Cluster Bundle Status stuck in "Bundle validation is in progress" Workaround: If the CM cluster-bundle-status gets stuck indefinitely in "Bundle validation is in progress" 1.) cancel the bundle push operation curl -k -u admin:pass https://host:mPort/services/cluster/master/control/default/cancel_bundle_push -X POST 2.) rollback to previous bundle or push a new bundle rollback: curl -k -u admin:pass https://host:mPort/services/cluster/master/control/default/rollback -X POST push bundle: curl -k -u admin:pass https://host:mPort/services/cluster/master/control/default/apply -X POST 3.) Restart the CM or peer if the above does not result in all peers on the same bundle
ie:
on CM server.conf
[clustering]
max_peers_to_download_bundle = 3
|
| 2019-03-13 | SPL-167708, SPL-170943, SPL-170937, SPL-170938 | Apply cluster bundle does not apply bundle to any indexers which are in progress of adding to cluster Workaround: restart affected indexer(s) |
| 2018-11-12 | SPL-162801, SPL-161301 | For a multisite cluster, splunk is not reaping prior search-buckets manifests after new generation |
| 2018-11-12 | SPL-162802, SPL-161301 | For a multisite cluster, splunk is not reaping prior search-buckets manifests after new generation Workaround: Do manual cleanup of $SPLUNK_HOME/var/run/splunk/cluster/search-buckets leaving the gen0 and 10 of the latest files per site as minimum To automate this you can do something like this in cron once you're happy with the manual run, you just need to add the delete flag for find: find $SPLUNK_HOME/var/run/splunk/cluster/search-buckets -regextype posix-extended -regex '.+_gen([0-9]{2,}|[1-9])\.csv\.gz' -mtime +2
|
| 2018-10-23 | SPL-161815 | Thawed buckets in a indexer cluster are sporadically unsearchable upon restart |
| 2018-06-22 | SPL-156164, SPL-162309, SPL-162310, SPL-162311 | Shutdown sequence does not begin after master has instructed peer to restart during rolling restart phase of a bundle push Workaround: Setting restart_timeout to the minimum value possible (server.conf on the CM). |
| 2018-06-06 | SPL-155226, SPL-153036 | CMBucketId has lock contention from std::map log(n) lookup time |
| 2018-04-05 | SPL-153051, SPL-152821 | Contention on DatabaseManager::_mux and CMIndexId mutex impacting search performance and indexer cluster stability. |
| 2018-03-22 | SPL-152465, SPL-153596, SPL-153597, SPL-154595, SPL-154647, SPL-154648 | Clustering - when a peer is in detention, we will make excess copies Workaround: If any indexers are in detention run `splunk remove excess-buckets` periodically. |
| 2018-03-15 | SPL-152168 | Batch-mode retry can return more or less events than it should due to reordering from thread pool processing. Workaround: Before you initiate searchable rolling restart or rolling upgrade, make sure the search_retry attribute in the [search] stanza of limits.conf is set to false (the default).
|
| 2017-03-16 | SPL-138846 | In multisite clustering, deletion of events in hot buckets is not pushed to other sites |
| 2016-08-25 | SPL-127353 | Data rebalance finishes early when one peer is the source for all buckets Workaround: when only one indexer in a cluster indexed data (has all the searchable copies), rebalance once before adding the new indexer, and then rebalance a second time |
| 2015-05-08 | SPL-101184 | Rolling restart in an Indexer Cluster may not be successful on a peer if a oneshot command is also running on that peer. Perform a manual restart to revive the peer. |
| 2014-10-13 | SPL-91861 | On Windows indexer on an ec2 instance, splunk-optimize main thread can crash on buckets on the temporary drive z:\>. |
| 2014-09-29 | SPL-91432 | On Windows when the master is down, the CLI command splunk offlinehangs when run from one of the streaming target peers. |
| 2014-09-08 | SPL-90630 | On a multisite cluster, no warning is given when search head names are the same. |
| 2014-07-29 | SPL-87816 | When implementing an indexer cluster or search head cluster, you cannot set pass4SymmKey in the general stanza. The system default values in the clustering and shclustering stanzas override any user-provided values in the general stanza. Workaround: Set the value in the [clustering] or [shclustering] stanza, depending on the type of cluster you're implementing. |
| 2014-07-14 | SPL-86799 | After adding a new license to the clustering search head, splunkd on restart cannot be reached by splunkweb. |
| 2014-04-29 | SPL-83636 | When configuring a multi-site cluster using cluster-config, the error messages are incorrect if the SF/RF was previously set. |
| 2014-03-18 | SPL-82038 | Cluster-config does not work if a parameter value includes a space character. |
| 2014-03-17 | SPL-81955 | Multisite: Peer takes approximately 6 minutes to restart when its site configuration is changed. |
| 2014-01-06 | SPL-78688 | Peer is able to change to an invalid (empty) replication port |
| 2013-08-06 | SPL-72484 | You cannot use the CLI to delete an index with a capital letter in its name. |
Universal forwarder issues
| Date filed | Issue number | Description |
|---|---|---|
| 2019-01-28 | SPL-165635, SPL-191773, SPL-189789 | splunk not reading file after log rotation |
| 2018-11-28 | SPL-163271, SPL-159337 | Splunk UF crashing due to invalid EVENT_BREAKER |
| 2018-09-25 | SPL-160530, SPL-156698 | splunk-netmon consumes additional 2GB memory every day on Universal Forwarder |
| 2018-04-10 | SPL-153251 | Universal Forwarder txz package cannot be installed on FreeBSD 11.1 Workaround: 1. Use pkg install instead of pkg add OR
2. Install package by untarring tgz file to /opt/splunkforwarder |
| 2017-05-23 | SPL-141961 | Older 6.0, 6.1, 6.2, 6.3 maintenance release versions unable to connect to 6.6.x and later via management port. Workaround: This applies to License Master/Slave, Deployment Server/Client, Cluster Master/Peers, Search Head/Peers and affects Splunk 6.6.x and the following versions:
Upgrade your older instances to the latest maintenance releases or on your 6.6.x Splunk instances. Add the following stanza to server.conf: [sslConfig]
|
| 2017-03-20 | SPL-139019 | Possible compatibility issues between Python / SDK clients and new 6.6 and later default sslVersions, cipherSuites Workaround: Users can do either of the following: 1. Overwrite the new Splunk 6.6 server.conf [sslConfig] sslVersions, cipherSuites with your own settings that are compatible with your version of OpenSSL, e.g. the previous defaults from 6.5.x are compatible with OpenSSL 0.9.8 on Mac OSX: [sslConfig] 2. For some more up-to-date clients, it is possible to enforce TLS1.2 (e.g. --tlsv1.2 for curl) in order to connect successfully. 3. Upgrade OpenSSL on your platform and link it with your client (e.g. Python, curl, etc..). For example, OpenSSL 1.0.2 is currently available on Mac OSX via Homebrew (see https://brew.sh) and is compatible with the new Splunk 6.6 default sslVersions, cipherSuites. |
| 2017-03-14 | SPL-138731 | New 6.6 and later default SHA256/2048-bit key certificates are not compatible with previous versions SHA1/1024-bit key certificates if cert verification is enabled Workaround: Users can do any of the following: 1. Disable certificate verification - the same root certificate is available with every Splunk download so enabling certificate verification while using the default certificates provides very little additional security. 2. Generate new SHA256/2048-bit key certificates using the new 6.6 root certificate and distribute to older versions of Splunk 3. Generate SHA1/1024-bit key certificates using the old root certificate to use with your new 6.6 instance. For convenience, the old root certificate is included in 6.6 in $SPLUNK_HOME/etc/auth/prev_release/ |
| 2015-06-10 | SPL-103010 | Indexing throughput on a forwarder with four pipelinesets drops 30% compared to a forwarder with two pipelinesets. |
| 2015-04-14 | SPL-99687, SPL-129637 | Splunk universal forwarder is 7-10 days behind recent Windows Security and system log events. Workaround: To mitigate this, edit the following stanza in inputs.conf: [WinEventLog://Security] evt_resolve_ad_obj = 0. |
| 2015-04-07 | SPL-99316 | Universal Forwarders stop sending data repeatedly throughout the day Workaround: In limits.conf, try changing file_tracking_db_threshold_mb in the [inputproc] stanza to a lower value. |
| 2015-03-25 | SPL-98594 | Routing events to two different groups not working as expected. Workaround: 1 On the original UF, instead of configuring 1 s2s and 1 syslog group, configure 2 s2s groups. 2 Setup a proxy UF which takes input from the original UF and send input out syslog server.
This solution only requires config change and no patch release is required. |
| 2014-08-05 | SPL-88396 | After configuring a client name for a deployment client, the name is not shown in the Forwarder Management UI Workaround: Create a server class, where you can see the client name, and use that group when you add data. |
| 2013-09-18 | SPL-74427, SPL-74448 | The Splunk universal forwarder installer for Solaris 10 does not add the splunk user when you attempt to install it using the pkgadd command. This results in the script generating lots of errors. Workaround: To work around this issue, create a splunk user on your system before attempting to run the installer. |
Distributed deployment, forwarder, deployment server issues
| Date filed | Issue number | Description |
|---|---|---|
| 2018-10-08 | SPL-161044, SPL-141772 | App deployment fails sporadically on Windows |
| 2018-06-29 | SPL-156539, SPL-155035 | Splunk Fowarders splunkd process stopping - Crashing thread: HttpClientPollingThread |
| 2014-10-02 | SPL-91648, SPL-91358 | Forwarder unable to push scripted inputs to a Linux deployment client from a Windows deployment server. |
| 2014-08-15 | SPL-89333 | Using client filtering in forwarder management interface when the deployment server is servicing a large numbers of deployment clients (over approximately 5000) can cause a temporary spike in memory usage. |
| 2014-06-20 | SPL-85739 | When running a high number of deployment clients for a server, memory growth may be excessive. Workaround: To mitigate this, set forceHttp10=always. |
Monitoring Console issues
| Date filed | Issue number | Description |
|---|---|---|
| 2019-01-24 | SPL-165397, SPL-160335 | No custom checklist item examples in checklist.conf.spec |
| 2019-01-23 | SPL-165338, SPL-160335 | No custom checklist item examples in checklist.conf.spec |
| 2018-10-19 | SPL-161714, SPL-163188, SPL-163189 | Saving server roles in DMC "configure" view results in 409 Conflict error response code when there is more than 30 groups defined |
| 2018-09-30 | SPL-160867, SPL-158166 | Monitoring Console does not allow user to select 'All Queues' in Queues to Measure dropdown |
| 2018-09-20 | SPL-160349, SPL-158166 | Monitoring Console does not allow user to select 'All Queues' in Queues to Measure dropdown |
| 2018-09-20 | SPL-160348, SPL-158166 | Monitoring Console does not allow user to select 'All Queues' in Queues to Measure dropdown |
| 2017-08-18 | SPL-144193 | Bundle validation errors prevent future app deployment to indexer cluster |
| 2017-08-14 | SPL-143981 | Uninstall app dialog does not show the app name correctly when the app doesn't have the label |
| 2017-08-04 | SPL-143664 | Uploaded apps page makes two calls to packages endpoint |
| 2017-05-24 | SPL-141982 | Upload modal should use size=large File element |
| 2017-04-19 | SPL-141274 | Clicking Install multiple times in Install dialog causes error |
| 2017-04-19 | SPL-141273 | Task endpoint fetch once even when there's no last deploy task id |
| 2017-03-30 | SPL-140654, SPL-178056 | wrong integrity check alert for file etc/users/users.ini |
| 2017-03-07 | SPL-138351, SPL-172626 | The role change of DMC via UI does not reflect to distsearch.conf Workaround: As a workaround can the customer manually modify the distsearch.conf. |
| 2016-11-14 | SPL-132151 | XML error when trying to download uninstalled app |
Splunk Web and interface issues
| Date filed | Issue number | Description |
|---|---|---|
| 2019-07-11 | SPL-173061 | UI exposes a nonfunctional option for modifying permissions on custom search commands |
| 2019-01-22 | SPL-165253, SPL-166047, SPL-166776, SPL-166777 | Using "%" in dashboard XML can cause infinite 'Loading...' loop for dashboards with no error reported. Workaround: Do manual URI encoding. For example, this would load just fine:
This is the <a href="http://%25%25problem%25%25/index.html">problem.</a> </html>
|
| 2018-10-25 | SPL-162249, SPL-170857, SPL-177527, SPL-177529, SPL-183262 | The filter function of <splunk-search-dropdown> UI component is not working on on Splunk Enterprise 7.1 and later. |
| 2018-07-16 | SPL-157354, SPL-152481 | Setting "tools.sessions.forceSecure = True" in web.conf doesn't set the secure flag on session_id_* cookies |
| 2018-06-26 | SPL-156316, SPL-145546 | When assigning indexes to roles, indexes defined on the indexer tier are not displayed Workaround: Replace the "$SPLUNK_HOME/etc/apps/search/default/data/ui/manager/authentication_roles.xml" file on the search head with a version from any Splunk Enterprise 6.6.x release. Refresh the configuration on the search head by calling a debug refresh (http[s]://[splunkweb hostname]:[splunkweb port]/debug/refresh) using a supported web browser. |
| 2018-05-30 | SPL-154973, SPL-155773, SPL-158832 | timeline preview shows random events, but not the ones based on the selected timeline segment Workaround: Appending following to the SPL seems to do the trick: | sort - _time |
| 2018-05-27 | SPL-154920, SPL-156245, SPL-159249 | Search Removal With Case Insensitive Capability Workaround: To use the same letter case as the search function. |
| 2017-08-23 | SPL-144350 | Archived Index is created without error when the splunk index is invalid |
| 2017-07-13 | SPL-143111 | "Splunkd daemon is not responding" when edit local windows event log collection |
| 2016-11-14 | SPL-132133 | App Browser filtering of the apps does not work |
| 2015-11-09 | SPL-109165 | Interactive Field Extractor hangs when using "^" as delimiter. Workaround: Use props and transforms to specify the delimiter of your choice. |
| 2015-08-10 | SPL-105061, SOLNESS-7274 | Broken module prevents splunkweb from starting |
| 2015-06-30 | SPL-103701 | Actions links should be removed for "Apps Browser" |
| 2014-07-16 | SPL-87015 | chart count by source and *| cluster showcount=t | table cluster_count _raw) no metadata/ result is available when user drills down on Count and Percent columns. |
| 2014-04-04 | SPL-82650 | A report created and scheduled by admin cannot be embedded by a power user. |
| 2014-02-26 | SPL-81103 | Username surrounded by dollar signs cannot create saved searches. |
| 2013-08-19 | SPL-73386 | Users are not allowed to run historical scheduled search Workaround: 1. Create a special power/admin user who can run scheduled searches. 2. Assign this user ownership of the scheduled searches. 3. Share the searches at the app level and grant read/write permission to the correct set of users. |
Windows-specific issues
| Date filed | Issue number | Description |
|---|---|---|
| 2019-06-07 | SPL-171658, SPL-166645 | Splunk is filling the "C:/Windows/Temp" folder with .tmp files |
| 2019-04-18 | SPL-169287, SPL-155149 | Registry changes under SYSTEM\CurrentControlSet are not being read by WinRegMon Workaround: Monitor SYSTEM\\ControlSet\d+ instead. |
| 2018-11-07 | SPL-162659, SPL-80589 | On Windows Server 2012 and Server 2012 R2, an external bug causes the %_Processor_Time counter to display 100 for multiple processes, even when the number of available CPU cores precludes that possibility. |
| 2018-10-29 | SPL-162353, SPL-158197 | splunk-regmon - failed to start the driver due to permission issue |
| 2018-09-25 | SPL-160530, SPL-156698 | splunk-netmon consumes additional 2GB memory every day on Universal Forwarder |
| 2018-08-31 | SPL-159549, SPL-153030 | PowerShell inputs fail after several runs |
| 2018-06-15 | SPL-155603, SPL-155149 | Registry changes under SYSTEM\CurrentControlSet are not being read by WinRegMon Workaround: Monitor SYSTEM\\ControlSet\d+ instead. |
| 2015-11-13 | SPL-109430 | In Windows only, inheritance is broken for folders created by splunkd. Files created are accessible only to the user as whom splunkd is running. |
| 2015-04-14 | SPL-99687, SPL-129637 | Splunk universal forwarder is 7-10 days behind recent Windows Security and system log events. Workaround: To mitigate this, edit the following stanza in inputs.conf: [WinEventLog://Security] evt_resolve_ad_obj = 0. |
| 2015-04-01 | SPL-98978 | On differing versions of Splunk Enterprise indexer (5.0.1) and universal forwarder (6.2.2), collection of the Security Event log can take increasingly longer over time. Workaround: To fix the problem, restart Windows on the forwarder.
|
| 2014-10-31 | SPL-92596 | After upgrade from Splunk Enterprise 6.1 or earlier to 6.4.x on Windows, splunkweb service does not start automatically. Attempts to start it manually show "Error 1053: The service did not respond to the start or control request in a timely fashion." Workaround: This is expected behavior. See the Splunk Answers post: http://answers.splunk.com/answers/177187/why-is-the-splunk-web-service-not-running-after-an.html |
| 2014-09-25 | SPL-91279 | Splunk Universal Forwarder on Windows (specifically, the splunk-perfmon.exe process) does not release key handles. Workaround: See "Handle leak when an application collects performance data in Windows Vista, in Windows 7, in Windows Server 2008 or in Windows Server 2008 R2" on the Microsoft Support website for a hotfix download. |
| 2013-10-11 | SPL-75116 | The UI does not show configured items of some newly converted windows modular inputs that contain the name "default" in the stanza Workaround: Edit inputs.conf: in stanzas that contain WinRegMon://default, replace "default" with something else, then restart splunk. |
REST, Simple XML, and Advanced XML issues
| Date filed | Issue number | Description |
|---|---|---|
| 2017-07-13 | SPL-143111 | "Splunkd daemon is not responding" when edit local windows event log collection |
| 2016-10-31 | SPL-131072 | Datamodel backend allows invalid time values |
| 2013-05-15 | SPL-67453 | When sending the following XML data as a GET or POST param to a custom splunkd endpoint: <dashboard><foo></dashboard>, the endpoint actually receives:<dashboard><foo></dashboard>. |
PDF issues
| Date filed | Issue number | Description |
|---|---|---|
| 2016-11-23 | SPL-132925 | Table data rows generated with the addcoltotals command do not show up in PDF Workaround: If you are using addcoltotals to generate a totals data row, renaming the _time field can cause PDF generation issues.
Remove the label and |
| 2015-03-31 | SPL-98890 | Maps printed from Report page do not honor custom zoom and center. |
| 2014-06-16 | SPL-85497 | Unable to save generated PDFs using Chrome internal PDF viewer. Workaround: Workaround: Enable Adobe Acrobat or Acrobat Reader as the default PDF viewer in Chrome. For more information, seehttps://support.google.com/chrome/answer/142056.
|
Admin and CLI issues
| Date filed | Issue number | Description |
|---|---|---|
| 2020-08-22 | SPL-194053, SPL-193257 | create_context=usr: notify mothership for newly created file Workaround: Change permissions after each lookup table creation Upload a pre-created/pre-existing csv lookup, but it is often not possible. |
| 2019-07-18 | SPL-173414, SPL-179491, SPL-179612, SPL-179627 | Splunk unable to load defined Saved Searches in a conf file if a bad/malformed cron_schedule value is present/set Workaround: If this behavior is noticed (savedsearches - alerts/searches/reports) not showing up in the GUI, verify cron jobs are valid, if they're not, then correct them. A quick check would be to create a duplicate savedsearches.conf and removing all cron_schedule definitions and rebooting splunk with that config to identify if it is this issue. |
| 2019-01-30 | SPL-165767, SPL-145827 | Capability rtsearch is enabling for power user after being remove when running CLI cmd and restarting splunk |
| 2018-11-01 | SPL-162465, SPL-142345 | SHOULD_LINEMERGE always shows true on UI when there is a LINE_BREAKER setting in sourcetype |
| 2018-10-11 | SPL-161286, SPL-154594 | system/default/props.conf for python.log just plain WRONG |
| 2018-10-09 | SPL-161134, SPL-142345 | SHOULD_LINEMERGE always shows true on UI when there is a LINE_BREAKER setting in sourcetype |
| 2018-07-23 | SPL-157731, SPL-154594 | system/default/props.conf for python.log just plain WRONG |
| 2018-05-17 | SPL-154589, SPL-154772, SPL-155190, SPL-155191, SPL-155194 | Enabling splunk boot-start won't work with ubuntu-like distro Workaround: 1. make a copy of /etc/os-release 2. remove /etc/os-release
3. run enable boot-start on splunk
4. restore /etc/os-release |
| 2018-02-06 | SPL-148877, SPL-145579 | chkconfig directive missing for AWS with enable boot-start |
| 2017-11-29 | SPL-146820 | Unable to access some settings/manager pages (data model editor) if starting from the setup page of a non-visible app Workaround: Navigate to a visible app, such as the search and reporting app, and access the Splunk settings pages from that app context. |
| 2017-04-11 | SPL-141051 | When LINE_BREAKER is defined for a sourcetype, UI forces SHOULD_LINEMERGE to true Workaround: None in Splunk Cloud. For on-prem, manually edit the props.conf file to set SHOULD_LINEMERGE to 'false'. |
| 2017-04-03 | SPL-140747 | SSL connection in Python when using new ciphers may be slow. |
| 2016-11-09 | SPL-131880 | Reports/Alerts owned by the deleted user cannot be found in the Orphaned filter for the Reassign Knowledge Objects page |
| 2015-09-23 | SPL-106978 | Failed SHC captain election causes unnecessary change in server.conf |
| 2015-03-11 | SPL-97942 | Capability defined in an app does not take effect when assigned to a role Workaround: The workaround is to change the ui-prefs in ./etc/users/username/local/ui-prefs.conf to look like this: [search]
display.events.fields = ["description","except_extract_1","except_extract_2","except_extract_3","sap_order_status","sourcetype","source","status","request_mode","request_id","request_status_id","object_id","BillToCity_","Airline_","BillToName_","BillToCountry_","City_"]
display.events.type = table |
| 2014-04-07 | SPL-82699 | SSO: Acceleration icon fails to display in Searches, Reports, and Alerts page. |
| 2013-05-25 | SPL-68010 | The error thrown when your Splunk instance cannot connect to splunkbase/.../checkforupdate is not an ERROR, should be lowered to INFO. Workaround: Set server.conf [applicationsManager] allowInternetAccess = false |
| 2013-05-02 | SPL-66511 | If $SPLUNK_HOME/etc is located on a case-insensitive filesystem, creating a new view with the same name as an existing view but with different case (capital letters vs lowercase, etc) silently overwrites the existing view. |
Uncategorized issues
| Date filed | Issue number | Description |
|---|---|---|
| 2019-11-08 | SPL-179256, SPL-179703, SPL-180148, SPL-180149 | kvstore inputlookup with large 'where' filter fails silently when hitting 300 second timeout Workaround: Change logic of your search, do filtering later in | search |
| 2019-07-16 | SPL-173213, SPL-169562 | EXTRACT with REGEX capture groups are not extracting fields without specifying FORMAT. |
| 2019-07-11 | SPL-173038 | Deprecated Feature SH Pooling has several functional problems in versions 7.1.x and above Workaround: Customers are strongly advised to use Search Head Clustering instead. |
| 2019-06-06 | SPL-171600, SPL-167453 | Replicated bucket in indexer cluster is timestamped with earliest time 0 (January 1970) if its last slice is empty. |
| 2019-05-22 | SPL-170880, SPL-169429 | Do not evict bucket contents from target indexers after S3 upload |
| 2019-04-03 | SPL-168740, SPL-151627 | Force user reset password is not replicated in SHC |
| 2019-03-20 | SPL-168023, SPL-167635 | Failed to localize because of CacheManager inconsistent bucket state after a truncate Workaround: https://confluence.splunk.com/display/PROD/Fixing+Failed+to+localize+errors+due+to+a+truncate |
| 2019-03-20 | SPL-168025, SPL-167635 | Failed to localize because of CacheManager inconsistent bucket state after a truncate Workaround: https://confluence.splunk.com/display/PROD/Fixing+Failed+to+localize+errors+due+to+a+truncate |
| 2019-03-20 | SPL-168026, SPL-167635 | Failed to localize because of CacheManager inconsistent bucket state after a truncate Workaround: https://confluence.splunk.com/display/PROD/Fixing+Failed+to+localize+errors+due+to+a+truncate |
| 2019-02-27 | SPL-167014, SPL-143275 | Bucket rebuild fails with reason: Failed to process delete journals |
| 2019-02-08 | SPL-166228, SPL-166798, SPL-167655 | Splunk crashes in _mongoc_openssl_ctx_new on shutdown |
| 2019-01-29 | SPL-165718, SPL-162399 | Relative, realtime, and date range tabs on time picker on a dashboard is not translated |
| 2019-01-28 | SPL-165614, SPL-162969 | Splunk upgrade failures due to kv store migration issues |
| 2019-01-25 | SPL-165574 | Splunk Search head cluster initial kvstore sync continues to fail with "OplogOperationUnsupported: error applying batch: Applying renameCollection not supported in initial sync" in mongod.log Workaround: For 7.1.4+: add the following to $SPLUNK_HOME/etc/system/local/server.conf: [kvstore] allowUnsafeRenamesDuringInitialSync = true For 7.1.3 and below:
Try and stop all kvstore (outputlookup) activity, the most common reason for the rename of a collection in mongo is when running | outputlookup without append=true |
| 2019-01-15 | SPL-164976, SPL-164862 | After migration, Splunk Cloud customer seeing unexpected large increase in outbound network bandwidth from forwarders |
| 2019-01-15 | SPL-164979, SPL-166184, SPL-166510, SPL-166511 | Search deadlock in StateStoreWorkerScheduler when executing kvstore lookup |
| 2019-01-04 | SPL-164557, SPL-175202 | Add capability of skewing time validation for SAML assertions |
| 2018-12-04 | SPL-163475, SPL-161299 | ES Search Head Captain Crash: 'it != _summary_inprogress.end()' |
| 2018-11-19 | SPL-163032, SPL-157014 | Search results on a dashboard are given with system timezone instead of user timezone |
| 2018-11-19 | SPL-163030, SPL-157014 | Search results on a dashboard are given with system timezone instead of user timezone |
| 2018-11-12 | SPL-162794 | Internal logs not being sent after enabling SplunkForwarder app |
| 2018-10-10 | SPL-161224, SPL-153371 | S2 - Search of a frozen bucket returns with a "failed to localize" error. |
| 2018-09-28 | SPL-160841, SPL-164523, SPL-164531 | Received fatal signal 11 (SEGV) on TailWatcher on Heavy Forwarders RegexExtractionProcessor race condition Workaround: disable regex profiling in limits.conf: regex_cpu_profiling = false or upgrade to 7.1.7, 7.2.4, 7.3.0 or higher |
| 2018-09-10 | SPL-159813, SPL-163056, SPL-164724, SPL-164725 | Post 6.6 / 7.0 upgrade, power user role cannot edit alert.expires from UI |
| 2018-08-31 | SPL-159552, SPL-163568, SPL-161659, SPL-161660, SPL-161688, SPL-162483, SPL-162484 | SAML - "role" not parsing comma separated list |
| 2018-08-22 | SPL-159203, SPL-159254, SPL-163561 | splunk_archiver app is included in the Splunk Cloud package in error, needs to be excluded at packaging time |
| 2018-08-15 | SPL-158875, SPL-159174, SPL-159613, SPL-159614, SPL-159644 | splunk shipped python in *nix doesn't work with iso2022_jp |
| 2018-07-13 | SPL-157230, SPL-163803, SPL-163974, SPL-163975 | conf-mutator.pid cleanup error during GUI initiated restart Workaround: The error message can be safely ignored. |
| 2018-06-28 | SPL-156504 | CIM Setup page is showing single line because of Indexes.js collection not executing callbacks Workaround: Edit configuration files as needed and restart the search head. |
| 2018-06-13 | SPL-155513, SPL-159646, SPL-159848 | Mstats not honoring time picker range for windowed real time search |
| 2018-05-28 | SPL-154925, SPL-161164, SPL-162971 | KVstore restore failure sometimes when having >1000 rows in the collection Workaround: Change the limit in server.conf from 1000 to a big enough value before backup, and allow restoring the backup e.g.: [kvstore]
max_documents_per_batch_save = 50000 |
| 2018-05-21 | SPL-155427, SPL-155716, SPL-155719 | CIM Setup page is showing single line because of Indexes.js collection not executing callbacks |
| 2018-05-18 | SPL-154616, SPL-152935 | KVStore Replication Error: replSetReconfig got BadValue _id field value of 256 is out of range |
| 2018-05-17 | SPL-154593 | Chunks of summary index data are routed to the wrong index when queues are blocked |
| 2018-05-10 | SPL-154382, SPL-166025, SPL-167034, SPL-167035, SPL-167036 | Role Capability To See Indexes for Summary Indexing Gives Role Index Edit Ability Workaround: Enable indexes_edit and dispatch_rest_to_indexers capabilities for the Power role for all indexes to be listed |
| 2018-05-08 | SPL-154263 | Splunk diag fails on files with modification time before 1970, "error: integer out of range for 'l' format code". Workaround: Change the timestamps of any files under SPLUNK_HOME dated prior to 1970. |
| 2018-05-04 | SPL-154139, SPL-154567 | embedded report uses oldest search artifact from the history endpoint Workaround: Same as Jira SPL-122982: cron job to delete the old artifacts |
| 2018-04-27 | SPL-153958, SPL-153724, SPL-154459 | mcollect should check index permissions for the index that it is trying to write to. |
| 2018-04-25 | SPL-153758, SPL-153687 | Dashboard time range picker selected state does not correctly display certain ranges |
| 2018-04-18 | SPL-153555, SPL-152283 | mongod errors out on distros with older glibc (2.7 and below) with " Invalid access at address: 0x10" |
| 2018-04-12 | SPL-153352 | Users are not notified about password expiration in Splunk Web Workaround: To mitigate this issue, the admin can create a saved search for password expiration alerts that appear in splunkd.log in order to find user passwords that are expiring. The expiration warning messages are in the format: "Password for user '{user}' is set to expire in {num} days" |
| 2018-04-02 | SPL-152888, SPL-154243, SPL-155000, SPL-155019, SPL-155451 | Chunks of summary index data are routed to the wrong index when queues are blocked |
| 2018-03-29 | SPL-152761, SPL-151501 | Enabling/Disabling acceleration for a data model creates an unnecessary copy of the data model JSON in <appname>/local/data/models/<model>.json |
| 2018-03-22 | SPL-152457, SPL-177446, SPL-178720, SPL-178721, SPL-178722, SPL-178723 | splunkd.log WARN TelemetryHandler - 1521648000.000000 |
| 2018-03-20 | SPL-152330, SPL-151992 | After installing Splunk on Windows using msiexec and the "GENRANDOMPASSWORD=1" option (and if generated password ends with backslash) admin is unable to login with msg "No users exist. Please set up a new user." Workaround: Create a $SPLUNK_HOME/etc/system/local/user-seed.conf and restart Splunk [user_info] |
| 2018-03-14 | SPL-152095 | Edit Summary Indexing - Index List empty/incomplete for User with Power role after upgrading to 6.6.0+ Workaround: add indexes_edit and dispatch_rest_to_indexers capability to the Power role for all indexes to be listed |
| 2018-02-14 | SPL-149190, SPL-141808 | (Windows Only) Support sslRootCAPath on Windows |
| 2018-01-25 | SPL-148514 | Splunk not starting on Linux kernel version 4.13.0-31 Workaround: Do not upgrade kernel to version 4.13.0-31. Use either an older release or 4.13.0-32.35+ |
| 2017-11-06 | SPL-146229 | Clustering S2 - set indexing-ready does not recreate buckets |
| 2017-10-17 | SPL-145749 | S2 bootstrapping doesnt respect multisite origin policies |
| 2017-05-09 | SPL-141693 | DataModel Editor - when child object has same name as inherited field, inherited field does not show in the inherited fields list. |
| 2017-03-27 | SPL-140442, SOLNESS-11786 | In Splunk Enterprise 6.6.0 and later, with Enterprise Security 4.5.2 and 4.6.0, roles without "edit_roles" capability cannot perform operations on notable event review statuses. Workaround: If users cannot perform operations on notable event review statuses or have issues viewing "Edit all selected" links on Incident Review, user roles must be provided with the "edit_roles" capability. |
| 2017-02-13 | SPL-136709 | Chart retains legend and title after enabling trellis layout in splunk.js |
| 2017-01-18 | SPL-135260 | Documentation for Search formatting keyboard shortcut for non-English languages |
| 2017-01-06 | SPL-134707 | Splunk restart does not create missing server.pem certificate on Windows Workaround: Use bin/splunk createssl server-cert -d etc/auth/ -n server to generate a new certificate. |
| 2016-11-21 | SPL-132670 | Mac OS 10.11: disable boot-start doesn't remove the file /Library/LaunchAgents//com.splunk.plist by enabling boot-start in prior Splunk/UF |
| 2016-08-31 | SPL-127800 | Opting in to data sharing on a monitoring console produces duplicate data |
| 2016-07-26 | SPL-125052 | Sole Admin can demote themself to Power without path of recovery in GUI. Workaround: Through the command line, you can open notepad and modify the password file to regain 'Admin' status. |
| 2016-06-22 | SPL-123301, SPL-95164, SPL-167968 | Aggressive calls to LDAP for non-existent/inactive users causes slow logins, performance issues/ skipped searches/ indexing pause |
| 2016-06-21 | SPL-123174 | JSON indexed_extractions doesn't work for TCP inputs |
| 2015-10-07 | SPL-107606 | Inconsistency between summary and datamodel_summary files. |
| 2015-06-18 | SPL-103302 | Files ownership are failed to be changed when using debian package to install splunk and $SPLUNK_HOME is a symlink Workaround: Run a recursive chown from the command line on $SPLUNK_HOME manually, post install. |
| 2015-05-24 | SPL-102008 | On Internet Explorer, a warning message does not display when you cannot log in due to a time zone difference. |
| 2015-05-11 | SPL-101289 | When the number of indexing pipeline sets is greater than four, indexing throughput decreases. |
| 2015-05-06 | SPL-100980 | Single indexer does not scale when receiving parsed data from multiple PipelineSets. |
| 2015-05-04 | SPL-100792 | There are multiple group=thruput metrics lines in metrics.log. Searches that do not differentiate among them may get falsely high totals. Workaround: Searches that key off these lines need to select their desired name=x category in order to see a single thruput value. |
| 2015-04-24 | SPL-100322 | A view gets stuck with "loading" due to problematic navigation (default.xml) Workaround: Workaround is to use label attribute for collection element. <collection label="Others"> <view source="unclassified" match="Dashboard"/>
</collection>
|
| 2015-03-26 | SPL-98700 | splunkd Indexer crashes in IndexerTPoolWorker due to duplicated bucket id. Workaround: The workaround is to remove the duplicated bucket. |
| 2015-02-26 | SPL-97389 | When using timechart command, the embedded report shows different time format than the original report. |
| 2015-01-08 | SPL-95144, SPL-101986, SPL-101987, SPL-106884, SPL-107317, SPL-142789 | Indexed message for Windows security event logs shows "FormatMessage error" Workaround: Splunk believes this was introduced in a Microsoft Windows patch. The workaround is to configure a delayed start of the Splunk service(s) so that it starts after the Windows Event Log service. |
| 2014-11-10 | SPL-92831 | A mismatch of versions between the license-master and the license-slave is generating Warning messages like "WARN LMDirective - directive cmd=D_set_feature_state args='Acceleration,ENABLED' failed: reason='feature='Acceleration' is invalid' ." Workaround: The warnings can be ignored, the workaround is use same major versions (all on 6.2 or all on 6.1).
|
| 2014-10-24 | SPL-92432, SPL-99583 | Chart in dashboard panel does not honor interval settings. Workaround: In the panel XML, specify a larger height to use the correct interval settings. |
| 2014-10-17 | SPL-92162 | Writing large amounts of data (> 20 GB) to KV store collections using outputlookup can result in high memory usage on the machine. |
| 2014-09-11 | SPL-90738 | Monitoring a directory with an unknown sourcetype produces indexing errors. |
| 2014-08-26 | SPL-90139 | <timestamp> does not display in the Patterns tab when searches are run in fast mode. |
| 2014-04-22 | SPL-83365 | Splunk Enterprise on Windows does not show an error message when a user without the edit_license capability tries to add a license through the CLI. |
| 2014-04-14 | SPL-83068 | Default index can be set to random index. |
| 2014-04-01 | SPL-82517, SPL-208875 | Paper Size and Layout in PDF Schedule dialog do not respect Paper Size and Layout in Email Settings. |
| 2014-03-23 | SPL-82238 | Datamodel fails to drill down further when the same attribute for Split Rows and Split Columns are selected. |
| 2014-03-13 | SPL-81856 | Show all lines does not work in data model editor preview. |
| 2014-03-12 | SPL-81810 | Licensing - license pool warning at license master keeps coming back after deleting it. Workaround: Delete the warnings on the peers first, then the License Manager. |
| 2014-03-12 | SPL-81781 | In the Data Model Manager, "Acceleration Status" and "Access Count" fail to update when you click "Update". |
| 2014-02-13 | SPL-80568 | Highcharts determines Y-axis values based on first point outside visible range. |
| 2014-02-07 | SPL-80285 | In the Data Model Editor, the Edit Lookup page is blank if Lookup is shared only in Lookup Definitions. Workaround: For more information, see Add lookup files to Splunk. |
| 2014-02-06 | SPL-80187 | In the Data Model Editor, lookup pages open with options displayed for other Lookup when the data model definition is private but the file is app or globally shared. Workaround: Share the definition. For more information, see Add lookup files to Splunk. |
| 2014-01-31 | SPL-79842 | On Windows, Indexer doesnt accept new connections on splunktcpin port after queue blockage is resolved |
| 2013-11-27 | SPL-77139 | Licenser pool usage gets reflected only after restarting splunkd. |
| 2013-10-29 | SPL-75764 | Forwarder forwards duplicate data after props.conf is in place for cross platform scenario/when the forwarder is on Solaris and the indexer is on Linux. |
| 2013-09-13 | SPL-74337, BETA-496 | You cannot specify a destination folder when installing on OSX. |
| 2013-09-10 | SPL-74209, SPL-74167 | Persistent queues are not created on Windows for stanzas that contain unusual characters (such as < and >). Workaround: Specify the persistentQueue explicitly in the input definition. |
| 2013-08-28 | SPL-73826 | Windows: hostname override not working properly |
| 2013-06-13 | SPL-69304 | If license slaves are running <6.0 version, they do not have the idx field and in theLicense Usage view, the split by index field will show a field named UNKNOWN. |
| 2013-04-30 | SPL-66213 | PDF server app is not working with latest Xvfb |
| 2012-02-22 | SPL-48342 | LDAP strategy host field cannot work with ipv6 format address but computer name is okay |
| 2010-10-08 | SPL-34347 | wmi input default fields - with value including newlines doesn't search properly becasue of \r\n issue |
Splunk Analytics for Hadoop
| Date filed | Issue number | Description |
|---|---|---|
| 2017-04-04 | ERP-2040 | Splunk archiving fails for large block sizes (buckets) due to HDFS write crashes for Hadoop version 2.8, 2.7.x Workaround: Upgrade Hadoop to 2.8.2 or higher. |
| 2015-09-09 | ERP-1650 | timestamp data type not properly deserialized. |
| 2015-08-05 | ERP-1619 | Searching on a newly created archive index before the bucket copy saved search is run causes a filenotfound exception. Workaround: Reenable the bucket copy saved search and let it run, or force the archiving to happen via | archivebuckets force=1 and then rerun the search. |
| 2015-07-07 | ERP-1598 | minsplit rampup - splits generation takes too long. Workaround: Set minsplits=maxsplits |
| 2015-05-12 | ERP-1502 | Non-accelerated pivot search on Pivot UI page waits for a long time to return result. |
| 2015-01-08 | ERP-1343, SPL-95174 | Splunk Analytics for Hadoop searches fail on corrupted journal.gz files, although Splunk searches run without error. Workaround: Add the journal.gz to the input path's blacklist (vix.input.1.ignore = ....) |
| 2014-10-27 | ERP-1216 | Data Explorer preview does not honor existing sourcetypes for big5/sjis files. |
| 2014-10-03 | ERP-1164 | Report acceleration summary gets deleted when two Splunk Analytics for Hadoop instances point to the same Splunk working directory. Workaround: To mitigate this issue, make sure that vix.splunk.home.hdfs (or Working directory in the UI) is unique on both search heads that are not in a pool. To keep your instances in the same working directory, configure vix.splunk.search.cache.path to be unique on both search heads. |
Last modified on 10 December, 2024
| Welcome to Splunk Enterprise 7.1 | Splunk Enterprise and anti-virus products |
This documentation applies to the following versions of Splunk® Enterprise: 7.1.0
Download manual
Feedback submitted, thanks!