Splunk® Enterprise

Admin Manual

Download manual as PDF

This documentation does not apply to the most recent version of Splunk. Click here for the latest version.
Download topic as PDF

Configure the distributed management console

What is the distributed management console?

The distributed management console lets you view detailed performance information about your Splunk Enterprise deployment. The topics in this chapter describe the available dashboards and alerts.

The available dashboards provide insight into your deployment's indexing performance, search performance, operating system resource usage, Splunk Enterprise app key value store performance, and license usage.

Find the distributed management console

From anywhere in Splunk Web, click Settings, and then click the Distributed Management Console icon on the left.

FindDMC.png

The distributed management console (DMC) is visible only to admin users.

You can leave DMC in standalone mode on your Splunk Enterprise instance, which means that you can navigate to the DMC on your individual instance in your deployment and see that particular instance's performance. Or you can go through the configuration steps, still in standalone mode, which lets you access the default platform alerts. Finally, if you go through the configuration steps for distributed mode, you can log into one instance and view performance information for every instance in the deployment.

Which instance should host the console?

After you have configured the DMC in distributed mode, you can navigate to it on only one instance in your deployment and view the console information for your entire deployment.

You have several options for where to host the distributed management console. The instance you choose must be provisioned as a search head. See "Reference hardware" in the Capacity Planning Manual. For security and some performance reasons, only Splunk Enterprise administrators should have access to this instance.

Important: Except for the case of a standalone, non-distributed Splunk Enterprise deployment, the instance hosting the DMC should not be used as a production search head and should not run any searches unrelated to its function as the DMC. This table describes the recommended locations for the DMC, based on deployment type:

Distributed Indexer clustering Search head clustering DMC options
No N/A N/A The standalone instance.
Yes No No The license master or a deployment server servicing a small number (<50) of clients. Use of the instance should be limited to DMC and these specific functions. If neither a license master nor a deployment server is available, run the DMC on a dedicated search head not used for other purposes.
Yes Single cluster Not relevant The master node. If preferred, you can instead run the DMC on a dedicated search head not used for other purposes.
Yes Multiple clusters Not relevant A search head that is configured as a search head node across all the clusters. This search head must be limited only to DMC use.
Yes No Yes The search head cluster deployer. If preferred, you can instead run the DMC on a dedicated search head not used for other purposes.

In a deployment with a single indexer cluster: On the master node

In an indexer cluster, host the DMC on the master node. See "System requirements" in the Managing Indexes and Clusters Manual.

As an alternative, you can host the DMC on a search head node in the cluster. If you do so, however, you cannot use the search head to run any non-DMC searches.

In a deployment with multiple indexer clusters: On a dedicated search head node

If your deployment has multiple indexer clusters, host the DMC on a search head configured as a search head node on each of the clusters. Do not use this search head to run any non-DMC searches.

The main steps to accomplish this are:

1. Configure a single search head as a node on each of the indexer clusters. See Search across multiple indexer clusters" in the Managing Indexes and Clusters Manual. This is your DMC instance.

2. Configure each master node, as well as all search head nodes in the clusters, as search peers of the DMC instance. See Add instances as search peers."

Caution: Do not configure the cluster peer nodes (indexers) as search peers to the DMC node. As nodes in the indexer clusters, they are already known to all search head nodes in their cluster, including the DMC node.

In a non-indexer-cluster environment, option 1: On license master

You can configure the monitoring console on your license master if the following are true:

  • Your license master can handle the search workload, that is, meets or exceeds the search head reference hardware requirements. See "Reference hardware" in the Capacity Planning Manual.
  • Only Splunk Enterprise admins can access your dedicated license master.

In a non-indexer-cluster environment, option 2: On a new instance

Another option is to provision a new instance, configure it as a search head of search heads and a search head of indexers, and configure the DMC in distributed mode there.

DMCarch.png

In a search head cluster environment

Use a deployer or dedicated license master for hosting the DMC. The DMC cannot be on a search head cluster member. See "System requirements and other deployment considerations for search head clusters" in the Distributed Search Manual.

The distributed management console is not supported in a search head pooled environment.

The DMC and deployment server

In most cases, you cannot host the distributed DMC on a deployment server. The exception is if the deployment server handles only a small number of deployment clients, no more than 50. The DMC and deployment server functionalities can interfere with each other at larger client counts. See "Deployment server provisioning" in the Updating Splunk Enterprise Instances manual.

Configure your DMC to monitor a deployment

Prerequisites

  • Have a functional Splunk Enterprise deployment. See "Distributed Splunk Enterprise overview" in the Distributed Deployment Manual. Any instance that you want to monitor must be running Splunk Enterprise 6.1 or higher.
  • Check whether your deployment is healthy, that is, that all peers are up.
  • Make sure that each instance in the deployment (each search head, license master, and so on) has a unique server.conf serverName value and inputs.conf host value.
  • Forward internal logs (both $SPLUNK_HOME/var/log/splunk and $SPLUNK_HOME/var/log/introspection) to indexers from all other instance types. See "Forward search head data" in the Distributed Search Manual. Without this step, many dashboards will lack data. These other instance types include:
    • Search heads.
    • License masters.
    • Cluster masters.
    • Deployment servers.
  • The user setting up the Distributed Management Console needs the "admin_all_objects" capability.

Add instances as search peers

1. Log into the instance on which you want to configure the distributed management console.

2. In Splunk Web, select Settings > Distributed search > Search peers.

3. Add each search head, deployment server, license master, and standalone indexer as a distributed search peer to the instance hosting the distributed management console. You do not need to add clustered indexers, but you must add clustered search heads.

Set up DMC in distributed mode

1. Log into the instance on which you want to configure the distributed management console. The instance by default is in standalone mode, unconfigured.

2. In Splunk Web, select Distributed management console > Setup.

3. Turn on distributed mode at the top left.

4. Check that:

  • The columns labeled instance and machine are populated correctly and populated with values that are unique within a column. Note: If your deployment has nodes running Splunk Enterprise 6.1.x (instead of 6.2.0+), their instance (host) and machine values will not be populated.
    • To find the value of machine, typically you can log into the 6.1.x instance and run hostname on *nix or Windows. Here machine represents the FQDN of the machine.
    • To find the value of instance (host), use btool: splunk cmd btool inputs list default.
    • When you know these values, in the Setup page, click Edit > Edit instance. A popup presents you with two fields to fill in: Instance (host) name and Machine name.
  • The server roles are correct, with the primary or major roles. For example, a search head that is also a license master should have both roles marked. If not, click Edit to correct.
  • A cluster master is identified if you are using indexer clustering. If not, click Edit to correct.

Caution: Make sure anything marked an indexer is really an indexer.

5. (Optional) Set custom groups. Custom groups are tags that map directly to distributed search groups. You don't need to add groups the first time you go through DMC setup (or ever). You might find groups useful, for example, if you have multisite indexer clustering (each group can consist of the indexers in one location) or an indexer cluster plus standalone peers. Custom groups are allowed to overlap. That is, one indexer can belong to multiple groups. See distributed search groups in the Distributed Search Manual.

6. Click Save.

7. (Optional) Set up platform alerts.

If you add another node to your deployment later, return to Setup and check that the items in step 4 are accurate.

Configure on a single instance

On a single Splunk Enterprise instance operating by itself, you must configure standalone mode before you can use platform alerts.

To configure:

1. Navigate to the Setup page in DMC.

2. Check that search head, license master, and indexer are listed under Server Roles, and nothing else. If not, click Edit.

3. Click Apply Changes to complete setup.

PREVIOUS
Use the license usage report view
  NEXT
Return the DMC to default settings

This documentation applies to the following versions of Splunk® Enterprise: 6.2.0, 6.2.1, 6.2.2, 6.2.3, 6.2.4, 6.2.5, 6.2.6, 6.2.7, 6.2.8, 6.2.9, 6.2.10, 6.2.11, 6.2.12, 6.2.13, 6.2.14, 6.2.15


Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters