Known issues

Known issues

The following are issues and workarounds for version 4.0.2 of Splunk.

Epoch timestamps not parsed correctly after March 12, 2011

This problem affects ALL Splunk versions: 3.x, 4.x, and 4.2.

In Splunk's datetime.xml, the regular expression for parsing epoch time assumes values from 2002 through to March 12th 2011. Those values started with 10,11,12. On March 12th, 2011, the seconds since 1970 became 1300000000, starting with 13.

First, make a backup copy of $SPLUNK_HOME/etc/datetime.xml, and then modify it. Change the _utcepoch regex (at around line 200) to the following:

<define name="_utcepoch" extract="utcepoch, subsecond">
    <!-- update regex before 2017! :) -->
    <text><![CDATA[((?<=^|[\s#,"=([\|{])(?:1[012345]|9)\d{8}|^@[\da-fA-F]{16,24})(?:.?(\d{1,6}))?(?![\d(])]]></text>
</define>

Alternatively, for your sources that use epoch time, explicitly specify a strptime format in props.conf, by using TIME_FORMAT and TIME_PREFIX fields.

Example:

[asterisk]
TIME_FORMAT = %s

Security issues

This version of Splunk contains several security flaws described on this page in the Splunk Security Portal. Splunk strongly recommends that you upgrade to 4.0.11 (or 4.1.2 or later) as soon as possible.

Events dated 2010 not returned by searches

Splunk is not auto-recognizing some timestamps from the year 2010. The problem is specific to two-digit year representations; the timestamp for these events are not correctly indexed by Splunk and so the events are not returned correctly by search. This is a particular issue with Windows Event Log events, but affects all events with timestamps that use two digits to represent the year.

If events from 2010 are not returned by searches, replace the datetime.xml file in your Splunk installation with this one:

http://download.splunk.com/support/config/2010fixed.datetime.xml.gz

The datetime.xml file is located in $SPLUNK_HOME/etc. You must apply this file to all indexers, and to regular Splunk forwarders (but you do not have to apply it to light forwarders, since indexing is not occurring on them.)

To apply this file to your instance:

1. Download the file.
2. Decompress it: gzip -d 2010fixed.datetime.xml.gz
3. copy it to your install: cp 2010fixed.datetime.xml path/to/splunk/etc/datetime.xml

This issue will be resolved in 4.0.8. We are currently working on a step-by-step procedure for recovering events between 01/Jan/2010 00:00:00 and the time you replace datetime.xml, but the general recommendation is to examine your buckets, locate those that include events for the timeframe in question, export them, and re-import them. For more information on buckets and how to identify their timeranges, refer to this topic on the Splunk Wiki.

Security

• A cross-site scripting vulnerability has been identified in Splunkweb. To remediate the issue download this file and untar into your $SPLUNK_HOME directory. (SPL-27560)

General issues

• The .spec file for inputs.conf states an inaccurate default value for rcvbuf. (SPL-24860)
• On shutdown, many WARN lines are displayed in splunk.log that should actually be INFO. These lines can be safely ignored. (SPL-24862)
• The splunkmon.log file reports many more restarts than may be accurate. (SPL-24928)
• Editing an existing role to inherit capabilities from another role causes those capabilities to be removed from the first role if it already had them. (SPL-24194)
• Uploading a local file over 105MB is currently not supported. (SPL-24292)
• The idsprobe command is deprecated and should no longer be used. (SPL-24403)
• You must manually distribute certificates to a host before you can successfully add it as a distributed search peer using the CLI. (SPL-24786)
• The admin role does not currently see all non-internal indexes by default. This will be changed in a future release. (SPL-24962)
• The 'always' alert condition does not trigger unless you get back at least one event. (SPL-25504)
• LDAP auth to AD may not list all users if realNameAttribute = cn. Set realNameAttribute to 'name' or 'displayName' as a workaround. (SPL-25462)
• If your username has a space in it, you cannot edit views. (SPL-25537)
• The path for results sent to scripts via alerts is incorrect. To work around this, parse the URL fields from sid= and tack that onto $SPLUNK_HOME/var/run/splunk/dispatch to get to the raw data of the alert. (SPL-25512)
• Deploying scripts via deployment server changes permissions on the client side to non-executable. To resolve this issue, change the permissions back to 755. (SPL-24168)
• If you disable Splunk Web from the CLI, you cannot re-enable it without editing web.conf. (SPL-25669)
• splunk-search may crash (in factorCommonTerms) when processing moderately complex boolean compound expressions, especially those involving tags. (SPL-27495)

Migrating your license

Splunk 4.x does not work with licenses from older releases. When you install 4.0.2, your existing 3.x license will be moved aside and replaced with a 4.x Enterprise trial license, which you can use while you procure an updated license.

• If you are an current Enterprise customer, check your splunk.com orders page for an updated license.
• If you are running with a 3.x Free or Enterprise trial license, delete the $SPLUNK_HOME/etc/splunk.license file before you start Splunk 4.x. The instance will then pick up the 60-day Enterprise trial license.

Localization, internationalization, and character set issues

• There are issues with character set auto detection when certain combinations of Unicode characters appear in an active file. Explicitly setting the correct character set will resolve this issue. (SPL-20780)
• Certain Japanese language OSes, including most versions of Windows, use the ¥ (Yen) symbol to denote backslashes in path names. This can cause issues when monitoring or spooling files, and may require custom regex configurations where a file path is part of the dataset. (SPL-23307)
• In Splunk 4.0-4.0.2, the browser's selected locale will not always be respected, and Splunk Web may fall back to en_US. To work around this, change the URL (for example, to en_GB) for the session and bookmark it for use in subsequent sessions. (SPL-25432)
• Characters that are not UTF-8-mapped are parsed as hexadecimal. (SPL-22935)

App and App development issues

• An issue exists in the first time run experience around input collisions: if you enable the *Nix App, the inputs it adds put their data in the "os" index, which by default is only searchable from the *Nix App interface. If you then try to add /var/log as an input (through the Getting Started App or any other App), an error is displayed stating that this input already exists. (SPL-25138)
• It's possible to get to the setup page for an App without enabling it first. (SPL-24852)
• No dashboards are added to the navigation menus for the Windows and *Nix Apps. (SPL-24933)
• It's not possible to delete views from Manager. (SPL-24908)
• Old modules, templates, and other App components are not deleted on upgrade. (SPL-22494)
• Selecting the "manager" link from the Launcher or Search App will sometimes hang. This happens when Splunk cannot connect to splunk.com. Set remote_tab = false in the $SPLUNK_HOME/etc/apps/launcher/default/app.conf file (SPL-25520, SPL-24670)
• If a dashboard.xml file - the default first view for most Apps - from any App is invalid, Splunk Web will not allow users to logon until it is fixed. (SPL-24799)
• If you delete a saved search that is referred to in a dashboard, the dashboard will not load. (SPL-25145)
• The *Nix App is not supported on AIX.
• Simple XML searchPostProcess doesn't work with <chart> and <fields>. (SPL-27248)
• in custom form search views, Lister modules like SearchSelectLister can not be configured to run their internal searches over the time range selected in a TimeRangePicker. (SPL-31706)

Search and search app issues

• Clicking on a field value that contains a quote character does not escape it in the search string. (SPL-25001)
• Subsearch clauses are not resurrected when running a saved search with a subsearch. To work around this, open the saved search in Manager > Searches and reports, copy the string, return to the search interface, and paste in the full search as needed. (SPL-24957)
• The schedule for a scheduled saved search is lost when that saved search is disabled. (SPL-25073)
• Creating an invalid event type does not generate an error. (SPL-25091)
• The error displayed ("Your entry was not saved. The following error was reported: undefined.") when trying to save a search with a name that already exists is too generic. (SPL-24267)
• In some cases, a savedsearch may not correctly reflect the entered string. Known situations are when using top for multiple fields "|top field1, field2" or "top x by x". The error may appear as Unknown search operator: Undefined (SPL-25447), or simply be a different search (SPL-25446).
• When clicking on the duration field in transaction results, the search string is getting inserted too early into the pipeline (SPL-25697).

Splunk Web and Manager display issues

• Linewrapping does not work correctly in Firefox 3.5; long events are shown on one long line. (SPL-24856)
• Scrolling in panel layout view does not work correctly in IE7. (SPL-24861)
• The number of users to display per page in Manager > Users does not retain its state if you change it. (SPL-24896)
• The page forManager > Authentication method displays the Switch to Splunk Authentication button, even if you already are using Splunk authentication. (SPL-24993)
• Pausing a search job in the job manager does not update the job's displayed status (SPL-24999)
• Splunk Web produces an error saying AttributeError: 'int' object has no attribute 'strip', if updateCheckerBaseUrl = 0 in web.conf. Change this setting to "none" or an empty value to disable the updateChecker, or remove the line to enable it (SPL-25319)
• If you have cookies disabled or if the server and/or client CPU time are not correct, you will be returned to the login page. Both machines must be on same GMT time when cookie timestamp is verified. (SPL-22393)
• If you schedule a search that's a report and have it emailed, the link that is included in the email will link to something that does not have the chart formatting you specified. (SPL-25671)
• Setting tools.sessions.timeout in web.conf is ignored. (SPL-26243)

Windows-specific issues

• When migrating from an earlier version of Splunk to version 4.0, there is no warning provided that some files cannot be automatically migrated. This is expected; migration of complex installs on Windows should proceed manually. (SPL-25161)
• The crawl feature is not applicable on Windows. (SPL-24843)
• The header for admon.conf erroneously states that changes to the file should be made to $SPLUNK_HOME/etc/system/local when changes should be made to $SPLUNK_HOME/etc/apps/windows/local. (SPL-24859)
• Enumeration of schema classes from Active Directory fails when there are more than 200 elements in a single object's definition. Only the first 200 are captured for use in metadata lookups. (SPL-25011)
• Auto host-extraction will not work on Windows Event logs when there is no default host value in inputs.conf. (SPL-25098)
• Forcing roll from hot to warm from the command line requires a backslash on the pipe character before the debug command, eg splunk search "\|debug cmd=roll index=your_index". (SPL-27534)

Considerations for users of Splunk 3.4.x

Splunk 4 is a huge stride forward in performance and flexibility, but there are a few interaction changes vs. 3.4.x which upgraders should be aware of, and even some reasons why you might want to wait for a future release before upgrading. Below are some capabilities that have changed with the introduction of Splunk 4:

Live tail

• With Splunk 4's dramatically improved search and indexing speed, along with the ability to provide intermediate search results, you don't really need a separate live event console to see data in near real-time. However, if your use case relies on version 3.4.x's "Live tail" feature, you may want to wait on upgrading to Splunk 4. Future roadmap plans involve re-architecting the live tail functionality to scale across much larger data flows, and across distributed environments. Additionally, look out for improve real-time alerting and dashboard updates down the road as a result of these upcoming architectural changes.

Custom field actions

• Based on customer feedback, we decided to re-architect this feature to improve flexibility and allow for event actions based on multiple fields. Expect this functionality to be reintroduced in a near term 4.x release. If you rely on this functionality, but still want to upgrade, you may want to consider Splunk 4's new "Dynamic field lookups" as an alternative which allows you to map data from external databases and lists into Splunk.

Snapshots

• In Splunk 4, we've improved upon 3.x's ability to take a timeline snapshots of individual searches. Try out Splunk 4's new job manager which allows you to retrieve the entire cached search result, including reports, from existing searches.

Event scrolling

• In Splunk 4, the new page selector allows you to hop between results with greater flexibility, even as a search runs. However, for those who still prefer a scroll bar, expect this capability to be re-introduced as an option in a future 4.x release.

Timeline and timestamp interaction

• In Splunk 4, we improved the timeline to allow users to quickly view any time range within search results, without having to rerun a search. Also try clicking "zoom-in" on the timeline, which now allows you to lock-in a time range, and specify follow on search.
• We're also planning to improve the usability of some related 3.4.x functionality including clicking on timestamps, and double clicking on timeline bars in future versions of 4.x.

Crawl

• Crawl is no longer configurable via the UI, but is still available as a search command. Based on customer feedback, we have decided to re-architect this feature to make it easier and more effective. Expect improved functionality, along with a new user interface to be introduced in a future release.

FIFO inputs

• This input type has been depreciated with Splunk 4, and we do not recommend using it as a best practice due to data loss considerations. Please contact support@splunk.com if you currently rely on this input type for alternative input methods.

• Was this topic useful?
• Post a comment