Which instance should host the console?
This topic is a step in the process of setting up the Distributed Management Console (DMC) for a Splunk Enterprise deployment. See About the Distributed Management Console above.
Once you have configured the DMC in distributed mode, you will be able to navigate to it on only one instance in your deployment and view the console information for your entire deployment. But which instance should this be?
You have several options for where to host the DMC. The instance you choose must be provisioned as a search head. See Reference hardware in the Capacity Planning Manual. For security and some performance reasons, only Splunk Enterprise administrators should have access to this instance.
Important: Except for the case of a single, standalone Splunk Enterprise instance, the instance hosting the DMC should not be used as a production search head and should not run any searches unrelated to its function as the DMC.
This table describes the recommended locations for the DMC, based on deployment type.
|Distributed mode?||Indexer clustering?||Search head clustering?||DMC options|
|No||N/A||N/A||The standalone instance.|
|Yes||No||No||The license master or a deployment server servicing a small number (<50) of clients. Use of the instance should be limited to DMC and these specific functions. If neither a license master nor a deployment server is available, run the DMC on a dedicated search head not used for other purposes.|
|Yes||Single cluster||Not relevant||The master node. If preferred, you can instead run the DMC on a dedicated search head not used for other purposes.|
|Yes||Multiple clusters||Not relevant||A search head that is configured as a search head node across all the clusters. This search head must be limited only to DMC use.|
|Yes||No||Yes||The search head cluster deployer. If preferred, you can instead run the DMC on a dedicated search head not used for other purposes. Do not run the DMC on a search head cluster member.|
In a deployment with a single indexer cluster: On the master node
In an indexer cluster (either single-site or multisite), host the DMC on the cluster master. See System requirements in Managing Indexes and Clusters.
As an alternative, you can host the DMC on a search head node in the cluster. If you do so, however, you cannot use the search head to run any non-DMC searches. See Enable the search head in Managing Indexers and Clusters of Indexers.
In a deployment with multiple indexer clusters: On a search head node
If your deployment has multiple indexer clusters, host the DMC on a search head configured as a search head node on each of the clusters. Do not use this search head to run any non-DMC searches.
The main steps to accomplish this are:
1. Configure a single search head as a node on each of the indexer clusters. See Search across multiple indexer clusters in the Managing Indexes and Clusters Manual. This is your DMC instance.
2. Configure each master node, as well as all search head nodes in the clusters, as search peers of the DMC instance. See Add instances as search peers.
Caution: Do not configure the cluster peer nodes (indexers) as search peers to the DMC node. As nodes in the indexer clusters, they are already known to all search head nodes in their cluster, including the DMC node.
In a non-indexer-cluster environment, option 1: On the license master
You can configure the monitoring console on your dedicated license master if the following are true:
- Your license master can handle the search workload, that is, meets or exceeds the search head reference hardware requirements. See Reference hardware in the Capacity Planning Manual.
- Only Splunk Enterprise admins can access your license master.
In a non-indexer-cluster environment, option 2: On a new instance
Another option is to provision a new instance, configure it as a search head of search heads and a search head of indexers, and configure the DMC in distributed mode there.
In a search head cluster environment
Use a deployer or dedicated license master for hosting the DMC. The DMC cannot be on a search head cluster member. The app is disabled on startup on a search head cluster member. See System requirements and other deployment considerations for search head clusters in the Distributed Search Manual.
DMC is not supported in a search head pooled environment.
Why not to host the console on a production search head
You should configure DMC on an instance that is not used as a production search head (a search head that runs your own searches or apps) for a few reasons.
The DMC's distributed search groups modify default search behavior. This modification ensures that the searches for the DMC dashboards are as narrowly scoped as possible to the list of search peers they target. When you set up the DMC in distributed mode, it creates one search group per server role, identified cluster, or custom group. Then it makes sure that unless you use a "splunk_server_group" or the "splunk_server" option, only search peers that are members of the indexer group are searched by default. But since all searches follow this behavior, non-DMC searches might have incomplete results.
Additionally, you should not host the DMC on a production search head because all production search heads should be monitored for performance, and the DMC itself affects performance. It can be difficult to disentangle DMC resource usage from production resource usage on the same instance.
The DMC and deployment server
In most cases, you cannot host the distributed DMC on a deployment server. The exception is if the deployment server handles only a small number of deployment clients, no more than 50. The DMC and deployment server functionalities can interfere with each other at larger client counts. See Deployment server provisioning in the Updating Splunk Enterprise Instances manual.
To continue setting up the DMC in distributed mode, your next step is to make sure your deployment meets the prerequisites. See DMC setup prerequisites.
Single-instance DMC setup steps
This documentation applies to the following versions of Splunk® Enterprise: 6.3.0, 6.3.1, 6.3.2, 6.3.3, 6.3.4, 6.3.5, 6.3.6, 6.3.7, 6.3.8, 6.3.9, 6.3.10, 6.3.11, 6.3.12, 6.3.13, 6.3.14, 6.4.0, 6.4.1, 6.4.2, 6.4.3, 6.4.4, 6.4.5, 6.4.6, 6.4.7, 6.4.8, 6.4.9, 6.4.10, 6.4.11