Known Issues for version 3.4.10

Known Issues for version 3.4.10

This page contains known issues and workarounds for this release of Splunk.

If you are upgrading from Splunk version 3.4.7

Splunk 3.4.7 contained an issue related to password encryption affecting all passwords over 8 characters in length, and was removed from distribution. If you installed or upgraded to version 3.4.7, you must ensure that any user password over 8 characters in length is temporarily reset to be 8 characters or fewer before upgrading to 3.4.8 or later. Once you have upgraded to 3.4.8 or later, user passwords can be any length desired.

If you are upgrading from 3.4.6 or earlier, this issue will not affect you.

Events dated 2010 not returned by searches

Splunk is not auto-recognizing some timestamps from the year 2010. The problem is specific to two-digit year representations; the timestamp for these events are not correctly indexed by Splunk and so the events are not returned correctly by search. This is a particular issue with Windows Event Log events, but affects all events with timestamps that use two digits to represent the year.

If events from 2010 are not returned by searches, replace the datetime.xml file in your Splunk installation with this one:

http://download.splunk.com/support/config/2010fixed.datetime.xml.gz

The datetime.xml file is located in $SPLUNK_HOME/etc. You must apply this file to all indexers, and to regular Splunk forwarders (but you do not have to apply it to light forwarders, since indexing is not occurring on them.)

To apply this file to your instance:

1. Download the file.
2. Decompress it: gzip -d 2010fixed.datetime.xml.gz
3. Copy it to your install: cp 2010fixed.datetime.xml path/to/splunk/etc/datetime.xml

We are currently working on a step-by-step procedure for recovering events between 01/Jan/2010 00:00:00 and the time you replace datetime.xml, but the general recommendation is to examine your buckets, locate those that include events for the timeframe in question, export them, and re-import them. For more information on buckets and how to identify their timeranges, refer to this topic on the Splunk Wiki.

Vulnerability in sample PAM script

This release contains a security issue described in detail on this page in the Splunk security portal.

General issues and considerations

This section contains general considerations, issues and workarounds for this release of Splunk.

• LDAP bind account password won't work if it contains XML-unsafe characters such as '&'. To workaround this, change the bind password so that it does not contain any XML-unsafe characters. (SPL-18170)
• If you have configured timestamp offsets using pre-Splunk 3.2 POSIX instructions, you must reconfigure them using this information. If you do not do this, your timestamp information will be incorrect. If you have not configured timezone offsets, you can ignore this note.
• Live tail is a powerful feature, and as such can tax system resources. For this reason, Splunk defaults to only allowing you to run one Live Tail at a time. However, you can edit web.conf to allow for multiple Live Tails. You must enable HTTP pipelining for this to function correctly. Refer to web.conf for more details. (SPL-11839)
• Live tail does not work through a proxy at this time. (SPL-13095)
• Live tail does not work in a distributed search environment.
• If you are using Splunk Deployment server, version 3.2 and earlier will only work with other deployed servers of exactly the same version, but 3.3.x will work with 3.2.x and 3.3.x.
• If you are running two different instances of Splunk on one machine, you cannot log into both instances at once, even with different shell sessions. However, you can use the -auth option in your search string to provide credentials for a different user on the fly. (SPL-11924)
• Splunk's authentication module does not work with Domino LDAP.
• 2.0.x licenses will NEVER work with 3.x+. If you have a current Plus Support contract you are entitled to upgrade your license to 3.x. If you do not have a current support agreement in place, contact sales@splunk.com.
• The File System Change Monitor does not monitor the directory or directories directly referenced in inputs.conf, rather the contents of those directories. If a directly referenced directory is deleted, renamed, or otherwise changed, you will not receive an alert. However, if any file in the directory is changed, you will receive an alert. (SPL-12418)
• If you switch from LDAP authentication to Splunk's built-in authentication, you must restart from the command line before you can log in again. (SPL-11737)
• You cannot specify a relative path when setting $SPLUNK_DB. (SPL-11867)
• Export and import of user data may not work properly.
• Log file rotation does not currently work while tailing SMB mounts. Work around this by mounting as CIFS.
• Upgrading using rpm does not create a etc.bak file.
• Some SUSE 10.x users might experience incorrectly displayed dialog boxes and searches may return the message "Unable to get a properly formatted response from the server; canceling the current search." This is a problem with the mime.types configuration. Instructions on how to correct this problem can be found here.
• Live tail does not currently respect the use of srchfilter within a role. To prevent users from accessing restricted information, explicitly disable Live tail in their user role. (SPL-13534)
• When enabling LDAP authentication, saved searches running as the admin user no longer function. To work around this, change the user the search runs as to a different user. (SPL-13870)
• Intermediary CAs are not yet supported in SSL certificates. (SPL-14463)
• LDAP authentication does not work when LDAP has no groups. (SPL-14439)
• Server-class CLI commands fail authentication. (SPL-14059)
• Wildcards in file system change monitor stanzas are ignored. (SPL-14487)
• Export to txt is broken from report view (SPL-16581).
• When adding a forwarder via the command line, splunk add forward-server, a vestigial setting isLoadBalanced = False is added to outputs.conf. This setting is ignored and has no effect. (SPL-17878)
• When using Splunk Applications with inputs (such as Splunk for Unix) in combination with a forwarder which does not parse the data, such as a SplunkLightForwarder, the application must be installed on both the node which acquires the data, and the node which parses the data (usually the direct receiver). (SPL-17915).
• Failure to load the Splunk Web interface (infinite checking for versions / login does not fail but returns to login screen) can be caused by multiple Internet Explorer problems. Windows XP prior to service pack 2 will not allow cookies from localhost, 127.0.0.1 may work. IE 6 and 7 typically will not accept cookies from host names containing underscores. Other sources of cookie blocking will cause the same failure.
• In a deployment server and client setup, the following configuration is supported.
  • Deployment Server 3.2.x with Deployment Client 3.2.x, 3.1.x and 3.0.x.
  • Deployment Server 3.3.x and 3.4.x with Deployment Client 3.4.x,and 3.3.x.
  • NOTE: Mixing 3.4.x and 3.3.x with 3.2.x or older versions is not supported.
• In versions 3.4.8 and later, server.conf is created in the /etc/system/local directory. If server.conf is rsynced, upon login it will cause the browser to hang. To work around this, delete server.conf from /etc/system/local and restart Splunk to regenerate server.conf.
• For timestamps containing an hour but no minutes, splunk will discard the hour data, reverting the hour to 0. This can be worked around by inserting a zero-value minute field. (SPL-23777)
• Splunk does not automatically break events on timestamps that do not contain a date. Use other breaking features to choose the desired behavior.
• Enabling forwarding in the SplunkWeb ui will silently enable the SplunkLightForwarder app, causing many changes. Some inputs are disabled, the SplunkWeb interface will go away, etc. (SPL-24722)

Search issues, including deprecated commands

• The readlevel and readlimit modifiers are deprecated as of version 3.2. Splunk now handles the verbosity of events intelligently with no need for specification.
• The maxresults and maxtime modifiers have been deprecated. If you have saved searches that use maxresults, they will no longer function starting with version 3.2.
  • Use the Preferences menu in Splunk Web to configure these values.
  • From within the CLI, use of maxresults has changed from being inside your query (for example, splunk search "search foo maxresults::100") to being outside your query (for example, splunk search "foo" -maxresults 100).
• The remote command is deprecated.
  • In Splunk Web, perform remote functionality in the Distributed tab of the Admin interface.
  • Click Admin in the upper-right corner of Splunk Web.
  • Click Distributed from the Distributed tab to turn on Distributed searching and then restart the server.
  • Add the servers you want search requests to be distributed to.
  • Restart Splunk. Once you restart Splunk, all search requests are sent to the servers you specify in the list.
  • In the CLI, use the dispatch command to execute remote functionality. You must have distributed search configured prior to running dispatch.
• The header argument for the diff command has no effect; the header data is always displayed.
• Performing multiple searches at once from the Web UI can occasionally return a "search was canceled" error.
• Field filtering does not work correctly on 'eventtype'-specific fields. To workaround, use '| search field=value' in your search string. (SPL-15700)
• Searches that operate on large events, such as configuration files and tabular data (top/ps ouput, logs containing multi-line events), can stress the memory available on 32-bit systems. Splunk recommends that you reduce the maximum number of results from the Preferences menu in Splunk Web or consider searching asynchronously using the command line interface when you are performing these types of searches. This issue can be compounded in distributed search scenarios, where the pool for results is greater. Additionally, the optimizations Splunk applies when displaying non-distributed search results are not available when performing distributed searches; this will also affect memory consumption.
• The pattern feature of transactiontypes and the transaction command only implements literal types (A, B, C) in a usable way. The regex-like functionality is not currently useful. (SPL-18442)
• Searching for tags that expand to larger than 8K characters in distributed search does not work. (SPL-22480)

Splunk Web issues and considerations

• Due to a change in Firefox 3, enabling SSL for a Splunk deployment may result in an "invalid security exception" being displayed in the browser. Refer to this workaround documentation for more information.
• Splunk 3.2 and later requires Flash 9. (download). Flash is available for Firefox 1.5 and 2.0, and Internet Explorer 6 and 7. See the Adobe Flash system requirements. You can check which version of Flash you are running here.
• Firefox 3.0b1 will not currently display any data with Splunk Web. Use Firefox 2.0.0.10 or earlier.
• If you create an event type that contains a space in the name and also specify tags for the event type at the same time, you cannot search on the tags.
• If you pipe into a saved search, time range specifications are ignored in Splunk Web. (SPL-12017)
• Section headers may sometimes display incorrectly in Splunk Web. (SPL-10138)
• If you are using IE7, you may experience inconsistent results in the timeline display. (SPL-11052)
• Time ranges are not retained in snapshots.
• To specify a label for a report column that includes spaces (with quotes surrounding the label name), do not use eval. Use rename and specify it as the last search processor in your string. (SPL-12200)
• Values for starttimeu or endtimeu are not recognized in Splunk Web, but do function correctly in the CLI. (SPL-13141)
• CSV export of searches that make use of field + will include all fields not those limited to the search results displayed in Splunk Web. (SPL-16562)
• In Splunk Web, you cannot filter searches on fields extracted by the REX command (SPL-15699), or based on eventtype::foobar in $SPLUNK_HOME/etc/system/local/props.conf. (SPL-15700)
• Decreasing the number of events shown in Splunk Web (by editing the number of cards and decks) to a low number causes Splunk Web to keep reloading. (SPL-14267)
• If you rename fields in a search pipeline, and generate a graph, the graph clickthrough searches will not account for the field renames, so the clickthrough search will often not work properly. (SPL-17688)
• For event types containing certain character sequences (eg %--), the web interface to typelearner produces an error when clicking the "Add Type" link. (SPL-18055)
• In the 'Authentication' screen, if you click on 'Save' without actually changing from Splunk authentication to LDAP, you will lose your session state and be routed to the login page. You will then be unable to log-in again without restarting the Splunk service (SPL-18543)
• You cannot give a dashboard a name with a hyphen in it. (SPL-19581)
• Form search with dynamically loaded list do not work in IE. (SPL-22880)
• Permalinks do not work in Firefox 3.5 (SPL-24913)

Windows-specific considerations and known issues

As a result of porting Splunk to the Windows platform, some functionality is not available or works differently due to platform differences or limitations:

• FIFO data inputs are not supported.
• 'Watch and symlink' operation is not supported with file-based data inputs, however, DFS is supported.
• The exporttool function does not support exporting to the original source, but does support export to csv. (SPL-12313)
• You must use two backslashes \\ to escape wildcards in stanza names in inputs.conf. (SPL-7270)
• The Windows installation package does not include the sample data (referred to in the tutorial portion of the User Guide) that is included on other platforms.
• The Windows release has been tested on English versions of the operating system only. Foreign language versions are unsupported.
• Changing the service login credentials of splunkd after installation is not supported via Splunk Web. If you must change the user Splunk runs as after you have installed, you must ensure that the user you create has the permissions described in this Windows installation topic, and also ensure that that user has Full Control permissions to the $SPLUNK_HOME/var directory. (SPL-14631)
• Regular expressions do not currently work in the Registry baselining feature. (SPL-14743)
• If you have made manual changes to the etc/system/local/inputs.conf file they may not be correctly preserved on upgrade. Make a backup copy of this file before upgrading. If Windows data input items do not exist, they will be added at the beginning of the file rather than the bottom, incorrectly including some conf items in the wrong stanza. This primarily applies to global "host = foohost" settings at the top of the file.
• There is an issue with stopping and restarting Splunk currently affecting users of remote WMI polling. If one or more of your WMI sources is unavailable at the time that you stop Splunk, Splunk will not come back up unless you wait for the splunk-wmi.exe process to exit, or kill it manually. To avoid this issue, do not unnecessarily list non-existent/non-functioning machines in wmi.conf. (SPL-16612)
• Issues with date/timestamping of data collected before 2007 may be the result of an OS-level issue on all pre-Vista systems. All Windows systems prior to Vista did not recognize that recognition of Daylight Savings Time (DST) can vary by location. A patch was issued by Microsoft in 2007, which, when applied, can cause the parsing of timestamps in Splunk to fail for data collected before 2007. (SPL-12503)
• During upgrade, Splunk does not inherit the original installation services setting. Therefore, during the upgrade, you must make sure to input the user account used for the original installation, or it will default to the local system user. (SPL-18195).
• Windows may generate a 3013 error in the System event log whenever Splunk locks a file depending on your file system audit level. This should not affect Windows or Splunk. (SPL-18263)
• If you are attempting to enable the SplunkLightForwarder app with Deployment Server via the EnableLightForwarder class, the class does not enable the app on Windows systems. An updated setup.py will be shipped in a future release that will enable the app across all platforms. (SPL-23008)
• A Windows event log input set up to pull events from an event log category which is not available on the system will acquire data from the Application event log (SPL-22613).
• The block signing feature enabled by blockSignSize causes splunkd to crash on Windows. (SPL-23359)
• Indexed Windows Vista or 2008 files may contain "The specified resource language ID cannot be found in the image file". The same message is in the Windows Event Viewer export but not in the Windows Event Viewer UI. (SPL-24031).
• High CPU utilization is observed with splunk-regmon. (SPL-24689)

Distributed search issues and considerations

• If you are adding or changing a license on any server in your distributed cluster, restart all of them to ensure that they display correctly on each others' dashboards. (SPL-12122)
• Autodiscovery of hosts for distributed search is unreliable.
• If you are using Splunk in a distributed search cluster you can mix 3.3.x with 3.2.x, but mixing 3.1.x and 3.2.x nodes in a distributed search cluster is not supported.

Configuration considerations and issues

• Entries in indexes.conf are case sensitive, including the stanza name itself. (SPL-12063)
• Reusing a field name in fields.conf results in the field being undefined. (SPL-12008)
• Use props.conf to alter Splunk's settings. The properties.xml file is still included with the product, but its settings have no effect.
• Configuring the file system change monitor to monitor the same thing in two different application with differing settings can causes conflicts which result in those differences being ignored (SPL-15680)
• Ensure that all instances of Splunk that are indexing data in a round-robin configuration have plenty of disk space. A current limitation of Splunk exists such that if a Splunk indexer runs out of disk space, all forwarders involved in the round-robin configuration will stop forwarding data to all Splunk indexers. (SPL-16602)
• Some splunk configuration files in etc/system/local, such as eventtypes.conf and savedsearches.conf will be rewritten with their entries in alpha-order when changed via the Splunk Web interface. Comment lines are preserved, but positional meaning may be lost in some cases. This will not affect the functionality of your searches or eventtypes, just that any in-line comments on a particular search/eventtype will not be moved. (SPL-18292)
• When using the 3.4.x SplunkLightForwarder, there must not be any queue=indexQueue statements in inputs.conf on the forwarder. This is likely when migrating old 3.3.x forwarders which were configured to avoid parsing.

Splunk Toolbar considerations and issues

• The Internet Explorer version of the toolbar does not work on instances of Splunk running over HTTPS (SPL-12821)
• The Splunk Toolbar sometimes incorrectly displays two drop-down arrows in the search box. This is has no effect on functionality.
• When running a free Splunk license, or an unlicensed copy of Splunk, the toolbar may not get past the "Welcome to Splunk" start page.
• Occasionally a search done in the toolbar will not return results. This may cause the browser to hang. The searches will work correctly if run directly in Splunk Web or the command line (CLI).
• In some cases, the toolbar will prevent "Find in this page" functionality from running multiple times on the same page. These reports have been limited to users running multiple browser add-ons (e.g. colorful tabs, dom inspector, user agent switcher).
• Autologin does not work if the Autologin is set to off prior to configuring a Splunk server in the toolbar.
  • To login automatically set Autologin to on prior to configuring the server.
• The toolbar does not have a mechanism for alerting if its credentials are invalid.
  • When a Splunk server is configured to talk to an LDAP server that locks accounts after N failed login attempts, users should verify that their credentials are correct.
• There are some cases where the toolbar may take over the current user session if the toolbar is configured to talk to a Splunk instance that is different than the one a user is currently logged into.
• There may be conflicts if a user is logged into one Splunk instance and runs a toolbar search on a different Splunk instance.

• Was this topic useful?
• Post a comment