Documentation > Splunk > Release Notes > Reindexing data affected by the 2010 timestamp issue

Release Notes

 


What's new
Known issues for this release
Changelogs by version
Credits

Reindexing data affected by the 2010 timestamp issue

This documentation does not apply to the most recent version of Splunk. Click here for the latest version.

Contents

Reindexing data affected by the 2010 timestamp issue

This topic provides a procedure for recovering data that was mis-indexed as a result of a timestamp recognition issue.

What was the issue?

In versions previous to 4.0.8, Splunk did not auto-recognize some timestamps from the year 2010. The problem was specific to two-digit year representations; the timestamp for these events were not correctly indexed by Splunk and so the events are not returned correctly by search. This was a particular issue with Windows Event Log events, but affected all events with timestamps that use two digits to represent the year.

The timestamp recognition issue was resolved in 4.0.8 by replacing datetime.xml. Any data indexed since upgrading to 4.0.8 or replacing datetime.xml is timestamped correctly.

The following is a step-by-step procedure for recovering events between 01/Jan/2010 00:00:00 and the time you replaced datetime.xml.

Caveats

Procedure

  1. Be sure you have updated your Splunk to 4.0.8+ or installed a fixed datetime.xml.
  2. Create an index in which to store the reindexed events. The provided app defaults to 'test'.
  3. Download the winevt_timefix app and unzip it into your Splunk instance, under the $SPLUNK_HOME/etc/apps directory.
  4. If your temporary index name is not 'test', edit the etc/apps/winevt_timefix/bin/evtrindex.py script included with the app to set the target_index to the index you created for this purpose.
  5. Run a search that captures a finite amount of data. Recommended: sourcetype=wineventlog* _indextime>=1262332800 endtime=01/25/2010:00:00:00'. The search should be bounded, so you can delete the data later, and you should remember the conditions, for example by running this from the command line.
  6. After verifying the above search gets what you want , run your_search | evtreindex from the command line, or background the search from Splunk Web.
  7. Wait for the search to complete. You should be able to validate that new data is arriving in your target index with the correct time, by searching, for example, in index=test
  8. Verify that you got all the data you expected, and that it looks correct.
  9. Delete the old version of your data (not reversible) with <your prior search> | delete. For example, sourcetype=wineventlog* _indextime>=1262332800 endtime=01/25/2010:00:00:00 | delete
  10. Stop Splunk.
  11. Merge the buckets from the test index into the main index. A script is provided for this purpose, substitute in the name of the index you used: splunk cmd python etc/apps/winevt_timefix/bin/index_merge.py test main. This topic discuss how to do this by hand: http://www.splunk.com/wiki/Community:MoveIndexes.
  12. Start Splunk.
Retrieved from "http://docs.splunk.com/Documentation/Splunk/4.0.5/ReleaseNotes/Reindexingdataaffectedbythe2010timestampissue"

This documentation applies to the following versions of Splunk: 4.0 , 4.0.1 , 4.0.2 , 4.0.3 , 4.0.4 , 4.0.5 , 4.0.6 , 4.0.7 , 4.0.8 , 4.0.9 , 4.0.10 , 4.0.11 View the Article History for its revisions.


You must be logged into splunk.com in order to post comments. Log in now.

Was this documentation topic helpful?

If you'd like to hear back from us, please provide your email address:

We'd love to hear what you think about this topic or the documentation as a whole. Feedback you enter here will be delivered to the documentation team.

Feedback submitted, thanks!