Splunk® Enterprise

Splunk Analytics for Hadoop

Acrobat logo Download manual as PDF

Splunk Enterprise version 8.0 is no longer supported as of October 22, 2021. See the Splunk Software Support Policy for details. For information about upgrading to a supported version, see How to upgrade Splunk Enterprise.
Acrobat logo Download topic as PDF

Configure and run unified search

Splunk Analytics for Hadoop archiving lets you search archived data in virtual archive indexes as well as the live data in the Splunk Enterprise indexes that feed those archives. Depending on how you configure archiving, your archived data may overlap with the unarchived data in your indexes.

For example, we recommend that you set your Splunk Enterprise indexes to archive data before the data is set to be expunged from the Splunk Enterprise index to avoid the risk of data being temporarily unavailable for search. This would create some data overlap.

You configure unified search for any virtual index that is also configured for archiving. Then, any time you run a search against that Splunk Enterprise index, unified search automatically checks data in both Splunk Enterprise indexes and archives, while skipping the duplicated data.

How to search with unified search

Unified search works only for indexes which are explicitly specified in the search. Unified search will not search archives for indexes that are implicitly specified, for example, via default index(es) or indexes specified via wildcards. Unified search does not sort by the event's create date, this includes results that only come from real Splunk indexers where data has not been archived yet. Splunk Analytics for Hadoop does not support real-time searches with unified search.

For more about how Splunk Analytics for Hadoop handles searches and time/dates, see Search a virtual index.

Here are some examples of explicit searches where unified search can help improve your searches:

  • index=myindex someterm
  • index=myindexname OR index=foo | top limit=20 "result.category_id"

Here are some examples of non-explicit searches that will not cause unified search to search archives:

  • wildcards
  • someterm
  • index=m* someterm
  • index!=my_splunk_index_with_an_archive
  • NOT index=my_splunk_index_with_an_archive

Configuring unified search

Important: to use unified search, the indexes must be defined on the search head as well as indexers. If indexes are not defined in the search head, Splunk creates empty indexes

Turn on unified search in limits.conf by setting unified_search to true:

# turn on/off feature
unified_search = true

In indexes.conf add the following attribute to your index archive stanza:

vix.unified.search.cutoff_sec = = <window length, before present time, in seconds>

A query against myindex will automatically look for events older than this cutoff in the archive index (i.e. myindex_archive), and will look for younger events in myindex itself. We recommend putting the unified search cutoff to occur right before the Splunk index is configured to move buckets from the cold state to the frozen state.

See Archiving Splunk indexes for more about archive configuration.

Here's an example of a virtual index configured to use unified search:

[root@sandbox bin]# more $SPLUNK_HOME/etc/apps/search/local/limits.conf 
unified_search = true

[root@sandbox bin]# more $SPLUNK_HOME/etc/apps/search/local/indexes.conf 
vix.output.buckets.from.indexes = myindex
vix.output.buckets.older.than = 3600
vix.output.buckets.path = /user/root/archive/myindex_archive
vix.provider = hdp2provider
vix.unified.search.cutoff_sec = 14400
# 14400 is 4 hours
Last modified on 05 April, 2019
Configure data model acceleration
Troubleshoot Splunk Analytics for Hadoop

This documentation applies to the following versions of Splunk® Enterprise: 7.0.0, 7.0.1, 7.0.2, 7.0.3, 7.0.4, 7.0.5, 7.0.6, 7.0.7, 7.0.8, 7.0.9, 7.0.10, 7.0.11, 7.0.13, 7.1.0, 7.1.1, 7.1.2, 7.1.3, 7.1.4, 7.1.5, 7.1.6, 7.1.7, 7.1.8, 7.1.9, 7.1.10, 7.2.0, 7.2.1, 7.2.2, 7.2.3, 7.2.4, 7.2.5, 7.2.6, 7.2.7, 7.2.8, 7.2.9, 7.2.10, 7.3.0, 7.3.1, 7.3.2, 7.3.3, 7.3.4, 7.3.5, 7.3.6, 7.3.7, 7.3.8, 7.3.9, 8.0.0, 8.0.1, 8.0.2, 8.0.3, 8.0.4, 8.0.5, 8.0.6, 8.0.7, 8.0.8, 8.0.9, 8.0.10, 8.1.0, 8.1.1, 8.1.2, 8.1.3, 8.1.4, 8.1.5, 8.1.6, 8.1.7, 8.1.8, 8.1.9, 8.1.10, 8.1.11, 8.1.12, 8.1.13, 8.1.14, 8.2.0, 8.2.1, 8.2.2, 8.2.3, 8.2.4, 8.2.5, 8.2.6, 8.2.7, 8.2.8, 8.2.9, 8.2.10, 8.2.11, 8.2.12, 9.0.0, 9.0.1, 9.0.2, 9.0.3, 9.0.4, 9.0.5, 9.0.6, 9.1.0, 9.1.1

Was this documentation topic helpful?

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters