Use search commands in Hadoop Connect
Distributable search commands are more effective in Hadoop Connect, because they can be distributed to search heads and virtual indexes.
This topic discusses the types of commands that work best with Hadoop Connect and commands that should be reserved for use with the Splunk Enterprise local directories.
Distributable commands are commands that can be run on a local indexer but can also be distributed to search heads and virtual indexes.
- Distributable streaming commands: Any streaming command that operates on each event returned by a search. Distributable streaming commands include:
- Distributable generating commands: Event-generating commands that are distributable return an events list or a table of results. Generating commands are invoked at the beginning of the search and with a leading pipe. A search cannot be piped into a generating command. The exception is the search command, because it is implicit at the start of a search and does not need to be invoked. Distributable event-generating commands include:
Non-distributable commands, which are also referred to as non-streaming commands, require all data to come back to the local indexer.
Reserve non-streaming commands for when part of your searching involves local indexes in some capacity. Types of non-distributable or non-streaming commands are:
- Centralized streaming commands: These commands are sometimes referred to as "stateful streaming" commands and include:
- Transforming streaming commands: A transforming command orders events into values that the Splunk platform can use for statistical purposes and includes:
- Non-distributable Generating commands: Generating commands that are either centralized event-generating or report-generating do not work with Hadoop Connect.
Hadoop Connect cannot export searches that contain any report-generating commands.
There are a handful of commands that do not fit into these categories. These commands are non-reporting, not distributable, and not streaming: sort, eventstats, some modes of dedup, and some modes of cluster.
Import from HDFS
Use the Troubleshoot menu
This documentation applies to the following versions of Splunk® Hadoop Connect: 1.0, 1.1, 1.2, 1.2.1, 1.2.2, 1.2.3, 1.2.4, 1.2.5