Known issues

Known issues

The following are issues and workarounds for version 4.0.5 of Splunk.

Epoch timestamps not parsed correctly after March 12, 2011

This problem affects ALL Splunk versions: 3.x, 4.x, and 4.2.

In Splunk's datetime.xml, the regular expression for parsing epoch time assumes values from 2002 through to March 12th 2011. Those values started with 10,11,12. On March 12th, 2011, the seconds since 1970 became 1300000000, starting with 13.

First, make a backup copy of $SPLUNK_HOME/etc/datetime.xml, and then modify it. Change the _utcepoch regex (at around line 200) to the following:

<define name="_utcepoch" extract="utcepoch, subsecond">
    <!-- update regex before 2017! :) -->
    <text><![CDATA[((?<=^|[\s#,"=([\|{])(?:1[012345]|9)\d{8}|^@[\da-fA-F]{16,24})(?:.?(\d{1,6}))?(?![\d(])]]></text>
</define>

Alternatively, for your sources that use epoch time, explicitly specify a strptime format in props.conf, by using TIME_FORMAT and TIME_PREFIX fields.

Example:

[asterisk]
TIME_FORMAT = %s

Security issues

This version of Splunk contains several security flaws described on this page in the Splunk Security Portal. Splunk strongly recommends that you upgrade to 4.0.11 (or 4.1.2 or later) as soon as possible.

Events dated 2010 not returned by searches

Splunk is not auto-recognizing some timestamps from the year 2010. The problem is specific to two-digit year representations; the timestamp for these events are not correctly indexed by Splunk and so the events are not returned correctly by search. This is a particular issue with Windows Event Log events, but affects all events with timestamps that use two digits to represent the year.

If events from 2010 are not returned by searches, replace the datetime.xml file in your Splunk installation with this one:

http://download.splunk.com/support/config/2010fixed.datetime.xml.gz

The datetime.xml file is located in $SPLUNK_HOME/etc. You must apply this file to all indexers, and to regular Splunk forwarders (but you do not have to apply it to light forwarders, since indexing is not occurring on them.)

To apply this file to your instance:

1. Download the file.
2. Decompress it: gzip -d 2010fixed.datetime.xml.gz
3. copy it to your install: cp 2010fixed.datetime.xml path/to/splunk/etc/datetime.xml

This issue will be resolved in 4.0.8. We are currently working on a step-by-step procedure for recovering events between 01/Jan/2010 00:00:00 and the time you replace datetime.xml, but the general recommendation is to examine your buckets, locate those that include events for the timeframe in question, export them, and re-import them. For more information on buckets and how to identify their timeranges, refer to this topic on the Splunk Wiki.

Security

• A cross-site scripting vulnerability has been identified in Splunkweb. To remediate the issue download this file and untar into your $SPLUNK_HOME directory. (SPL-27560)

General issues

• On shutdown, many WARN lines are displayed in splunk.log that should actually be INFO. These lines can be safely ignored. (SPL-24862)
• You must manually distribute certificates to a host before you can successfully add it as a distributed search peer using the CLI. (SPL-24786)
• If you expand the view of a large event to the full event and back again to the summary view, subsequent attempts to expand to view the entire event will be restricted to 500 lines. (SPL-27109)
• Show source will show no text if there are more than 10,000 events per second from that host, source and sourcetype. (SPL-26792)
• Admin Activity dashboard reports IE8 as Mozilla. (SPL-27296)
• Saving an event type via Splunk Web adds an extra literal "search" to the full search. (SPL-27049)
• Starting Splunk with the --debug option will result in a non-responsive UI, CLI will still work. If you need debug messages, temporarily copy log-debug.cfg to log.cfg (save a copy to replace) and start normally. Or enable individual logging options in log.cfg instead. (SPL-27398)
• splunk-search may crash (in factorCommonTerms) when processing moderately complex boolean compound expressions, especially those involving tags. (SPL-27495)
• Indexing performance can be impacted if you are running anti-virus scans on Splunk's index directories. This is caused primarily by the additional CPU and disk overhead of the anti-virus accessing these files at the same time as Splunk.

Data input issues

• monitor inputs using the followTail setting sometimes will index some older events or all events from log files which are updated when not intended. (SPL-23555)
• The no_appending_timestamp = true setting doesn't work in UDP inputs (SPL-26783)

Localization, internationalization, and character set issues

• Certain Japanese language OSes, including most versions of Windows, use the ¥ (Yen) symbol to denote backslashes in path names. This can cause issues when monitoring or spooling files, and may require custom regex configurations where a file path is part of the dataset. (SPL-23307)

(This issue is also present in the Japanese PDFs of the documentation.)

App and App development issues

• An issue exists in the first time run experience around input collisions: if you enable the *Nix App, the inputs it adds put their data in the "os" index, which by default is only searchable from the *Nix App interface. If you then try to add /var/log as an input (through the Getting Started App or any other App), an error is displayed stating that this input already exists. (SPL-25138)
• It's possible to get to the setup page for an App without enabling it first. (SPL-24852)
• No dashboards are added to the navigation menus for the Windows and *Nix Apps. (SPL-24933)
• It's not possible to delete views from Manager. (SPL-24908)
• Old modules, templates, and other App components are not deleted on upgrade. (SPL-22494)
• The *Nix App is not supported on AIX. (ENH-3001)
• Simple XML searchPostProcess doesn't work with <chart> and <fields>. (SPL-27248)
• in custom form search views, Lister modules like SearchSelectLister can not be configured to run their internal searches over the time range selected in a TimeRangePicker. (SPL-31706)

Search and search App issues

• Saved searches that contain NOT expressions that include quoted terms will not run properly when loaded from the Searches and Reports menu because the quotation marks do not escape correctly. In some cases, removing the quotation marks is sufficient to work around the issue. (SPL-26944)
• Creating an invalid event type does not generate an error. (SPL-25091)
• The All indexed data dashboard count for number of sources stops incrementing at 10,000 sources. (SPL-27300)
• The xpath operator does not work. (SPL-26985)

Splunk Web and Manager display issues

• The number of users to display per page in Manager > Users does not retain its state if you change it. (SPL-24896)
• Pausing a search job in the job manager does not update the job's displayed status (SPL-24999)
• If you have cookies disabled or if the server and/or client CPU time are not in sync, you will be returned to the login page. Both machines must have the correct time set when cookie timestamp is verified. (SPL-22393)
• If you schedule a search that's a report and have it emailed, the link that is included in the email will link to something that does not have the chart formatting you specified. (SPL-25671)

Windows-specific issues

• The crawl feature is not applicable on Windows. (SPL-24843)
• The Windows App was enabled by default in its app.conf file in versions 4.0-4.0.2. Starting in version 4.0.3, it is disabled in this file by default. (SPL-25487) Read on for important details:
  • If you're upgrading from 4.0-4.0.2 to 4.0.3 or later, the Windows App will be disabled, even if it was enabled in the version you're upgrading from.
  • If you're doing a fresh installation of 4.0.3 or later, the Windows App is enabled by default via the MSI and if you want to install it in a disabled state, you must specify this using the SPLUNK_APP msiexec command as described in "Install on Windows via the commandline" .
• In props.conf the source stanza is ignored. Use sourcetype as a workaround. (SPL-25898)
• Specifying a non-existent Windows Event Log category in inputs.conf will acquire general Application Log data and assign it the non-existent category (SPL-22613)
• Forcing roll from hot to warm from the command line requires a backslash on the pipe character before the debug command, eg splunk search "\|debug cmd=roll index=your_index". (SPL-27534)
• A perpetual license may report expiration in approximately 1.5 years. (SPL-27005)

Migrating your license

Splunk 4.x does not work with licenses from older releases. When you install Splunk 4.0.2 or later, your existing 3.x license will be moved aside and replaced with a 4.x Enterprise trial license, which you can use while you procure an updated license.

• If you are an current Enterprise customer, check your splunk.com orders page for an updated license.
• If you are running with a 3.x Free or Enterprise trial license, delete the $SPLUNK_HOME/etc/splunk.license file before you start Splunk 4.x. The instance will then pick up the 60-day Enterprise trial license.

Considerations for users of Splunk 3.4.x

Splunk 4 is a huge stride forward in performance and flexibility, but there are a few interaction changes vs. 3.4.x which upgraders should be aware of, and even some reasons why you might want to wait for a future release before upgrading. Below are some capabilities that have changed with the introduction of Splunk 4:

Live tail

• With Splunk 4's dramatically improved search and indexing speed, along with the ability to provide intermediate search results, you don't really need a separate live event console to see data in near real-time. However, if your use case relies on version 3.4.x's "Live tail" feature, you may want to wait on upgrading to Splunk 4. Future roadmap plans involve re-architecting the live tail functionality to scale across much larger data flows, and across distributed environments. Additionally, look out for improve real-time alerting and dashboard updates down the road as a result of these upcoming architectural changes.

Custom field actions

• Based on customer feedback, we decided to re-architect this feature to improve flexibility and allow for event actions based on multiple fields. Expect this functionality to be reintroduced in a near term 4.x release. If you rely on this functionality, but still want to upgrade, you may want to consider Splunk 4's new "Dynamic field lookups" as an alternative which allows you to map data from external databases and lists into Splunk.

Snapshots

• In Splunk 4, we've improved upon 3.x's ability to take a timeline snapshots of individual searches. Try out Splunk 4's new job manager which allows you to retrieve the entire cached search result, including reports, from existing searches.

Event scrolling

• In Splunk 4, the new page selector allows you to hop between results with greater flexibility, even as a search runs. However, for those who still prefer a scroll bar, expect this capability to be re-introduced as an option in a future 4.x release.

Timeline and timestamp interaction

• In Splunk 4, we improved the timeline to allow users to quickly view any time range within search results, without having to rerun a search. Also try clicking "zoom-in" on the timeline, which now allows you to lock-in a time range, and specify follow on search.
• We're also planning to improve the usability of some related 3.4.x functionality including clicking on timestamps, and double clicking on timeline bars in future versions of 4.x.

Crawl

• Crawl is no longer configurable via the UI, but is still available as a search command. Based on customer feedback, we have decided to re-architect this feature to make it easier and more effective. Expect improved functionality, along with a new user interface to be introduced in a future release.

FIFO inputs

• This input type has been depreciated with Splunk 4, and we do not recommend using it as a best practice due to data loss considerations. Please contact support@splunk.com if you currently rely on this input type for alternative input methods.

RSS Feed alerts

• Splunk 4 now has improved capabilities for creating email alerts based off searches on your data. However, the RSS feed alerting option from 3.x is currently being re-architected based on customer feedback, and will be reintroduced as an option in a future 4.x release.

Deployment

• Splunk 4.0.x Deployment server is not compatible with Splunk Deployment client 3.x.

• Was this topic useful?
• Post a comment