Enable workload management

After you create your workload pools and workload rules, you must globally enable the workload management feature. When you enable workload management, a series of health checks run in the background to validate your configuration. If these health checks fail, you cannot enable workload management and a failure message appears.

For more information on Linux configuration requirements, see Set up Linux for workload management.

You can enable workload management using Splunk Web, the CLI, or REST API.

Enable workload management in Splunk Web

1. In Splunk Web, click Settings > Workload Management.
2. Toggle the switch to Enabled.
   This applies any pending configuration changes and enables workload management.
   To disable workload management, toggle the switch to Disabled.

Enable workload management using the CLI

To enable or disable workload management, run the following CLI command:

./splunk <enable|disable> workload-management

Enable workload management using REST

You can enable or disable workload management using the REST API. For endpoint details, see workloads/config/enable or workloads/config/disable in the REST API Reference Manual.

Enable individual workload rules

You can enable or disable individual workload rules. This lets you create and save multiple different workload rules and apply them as needed. Individual workload rules are enabled by default when you create them. Disabled workload rules are not evaluated and have no effect on searches.

When you enable or disable a workload rule, the setting is stored under the [workload_rule:<rule_name>] stanza for the individual rule in workload_rules.conf. For more information, see workload_rules.conf.

You can enable or disable individual workload rules using Splunk Web, the CLI or REST API.

Enable individual workload rules using Splunk Web

To enable or disable an individual workload rule using Splunk Web:

1. In Splunk Web, click Settings > Workload Management > Workload Rules.
2. In the Status column, toggle the switch to enable or disable the individual workload rule.

Enable individual workload rules using the CLI

To enable or disable an individual workload rule, run the following CLI command,

./splunk <enable|disable> workload-rule <rule_name>

Enable individual workload rules using REST

Send a POST request to the workloads/rules endpoint, as shown:

curl -k -u <user:pass> -X POST https://localhost:8089/services/workloads/rules/<rule_name>/<enable|disable> 

For endpoint details, see workloads/rules in the REST API Reference Manual.

Check workload management status

You can check the current status of workload management using the CLI or REST API. The output shows configuration details of all workload pools and rules, and whether workload management is supported and enabled on the instance. You can also view runtime information for in-progress searches in any pool.

Check workload management status using the CLI

Run the following CLI command:

./splunk show workload-management-status --verbose

The command output will differ slightly depending on whether your system is using cgroups v1 or cgroups v2.

The following is an example of the output for cgroups v1. Note the lines that show V1 preceding the cgroup path.

        Admission Control Status:
                Enabled: 0
        Workload Management Status:
                Enabled: 1
                Supported: 1
                Error:
        Workload Category: ingest
                        CPU Group: V1:/sys/fs/cgroup/cpu,cpuacct:/system.slice/Splunkd.service:/sys/fs/cgroup/memory:/system.slice/Splunkd.service:/ingest:cpu
                        Memory Group: V1:/sys/fs/cgroup/cpu,cpuacct:/system.slice/Splunkd.service:/sys/fs/cgroup/memory:/system.slice/Splunkd.service:/ingest:memory
                        CPU Weight: 20
                        Memory Weight: 100
                        Default Category Pool: p2
                p2:
                        CPU Group: V1:/sys/fs/cgroup/cpu,cpuacct:/system.slice/Splunkd.service:/sys/fs/cgroup/memory:/system.slice/Splunkd.service:/ingest/p2:cpu
                        Memory Group: V1:/sys/fs/cgroup/cpu,cpuacct:/system.slice/Splunkd.service:/sys/fs/cgroup/memory:/system.slice/Splunkd.service:/ingest/p2:memory
                        CPU Weight: 100
                        Memory Weight: 100
        Workload Category: search
                        CPU Group: V1:/sys/fs/cgroup/cpu,cpuacct:/system.slice/Splunkd.service:/sys/fs/cgroup/memory:/system.slice/Splunkd.service:/search:cpu
                        Memory Group: V1:/sys/fs/cgroup/cpu,cpuacct:/system.slice/Splunkd.service:/sys/fs/cgroup/memory:/system.slice/Splunkd.service:/search:memory
                        CPU Weight: 70
                        Memory Weight: 70
                        Default Category Pool: p1
                p1:
                        CPU Group: V1:/sys/fs/cgroup/cpu,cpuacct:/system.slice/Splunkd.service:/sys/fs/cgroup/memory:/system.slice/Splunkd.service:/search/p1:cpu
                        Memory Group: V1:/sys/fs/cgroup/cpu,cpuacct:/system.slice/Splunkd.service:/sys/fs/cgroup/memory:/system.slice/Splunkd.service:/search/p1:memory
                        CPU Weight: 100
                        Memory Weight: 100
        Workload Category: misc
                        CPU Group: V1:/sys/fs/cgroup/cpu,cpuacct:/system.slice/Splunkd.service:/sys/fs/cgroup/memory:/system.slice/Splunkd.service:/misc:cpu
                        Memory Group: V1:/sys/fs/cgroup/cpu,cpuacct:/system.slice/Splunkd.service:/sys/fs/cgroup/memory:/system.slice/Splunkd.service:/misc:memory
                        CPU Weight: 10
                        Memory Weight: 10
                        Default Category Pool:
Workload Rules:
		rule_1:
			Order: 1
			Predicate: app="search"
			Workload Pool: pool_1

		rule_2:
			Order: 2
			Predicate: app="search" AND (NOT index="_internal")
			Workload Pool: pool_3

The following is an example of the output for cgroups v2. Note the lines that show V2 preceding the cgroup path.

        Admission Control Status:
                Enabled: 0
        Workload Management Status:
                Enabled: 1
                Supported: 1
                Error:
        Workload Category: ingest
                        CPU Group: V2:/sys/fs/cgroup:/system.slice/Splunkd.service:/sys/fs/cgroup:/system.slice/Splunkd.service:/ingest:cpu
                        Memory Group: V2:/sys/fs/cgroup:/system.slice/Splunkd.service:/sys/fs/cgroup:/system.slice/Splunkd.service:/ingest:memory
                        CPU Weight: 20
                        Memory Weight: 100
                        Default Category Pool: p2
                p2:
                        CPU Group: V2:/sys/fs/cgroup:/system.slice/Splunkd.service:/sys/fs/cgroup:/system.slice/Splunkd.service:/ingest/p2:cpu
                        Memory Group: V2:/sys/fs/cgroup:/system.slice/Splunkd.service:/sys/fs/cgroup:/system.slice/Splunkd.service:/ingest/p2:memory
                        CPU Weight: 100
                        Memory Weight: 100
        Workload Category: search
                        CPU Group: V2:/sys/fs/cgroup:/system.slice/Splunkd.service:/sys/fs/cgroup:/system.slice/Splunkd.service:/search:cpu
                        Memory Group: V2:/sys/fs/cgroup:/system.slice/Splunkd.service:/sys/fs/cgroup:/system.slice/Splunkd.service:/search:memory
                        CPU Weight: 70
                        Memory Weight: 70
                        Default Category Pool: p1
                p1:
                        CPU Group: V2:/sys/fs/cgroup:/system.slice/Splunkd.service:/sys/fs/cgroup:/system.slice/Splunkd.service:/search/p1:cpu
                        Memory Group: V2:/sys/fs/cgroup:/system.slice/Splunkd.service:/sys/fs/cgroup:/system.slice/Splunkd.service:/search/p1:memory
                        CPU Weight: 100
                        Memory Weight: 100
        Workload Category: misc
                        CPU Group: V2:/sys/fs/cgroup:/system.slice/Splunkd.service:/sys/fs/cgroup:/system.slice/Splunkd.service:/misc:cpu
                        Memory Group: V2:/sys/fs/cgroup:/system.slice/Splunkd.service:/sys/fs/cgroup:/system.slice/Splunkd.service:/misc:memory
                        CPU Weight: 10
                        Memory Weight: 10
                        Default Category Pool:
Workload Rules:
		rule_1:
			Order: 1
			Predicate: app="search"
			Workload Pool: pool_1

		rule_2:
			Order: 2
			Predicate: app="search" AND (NOT index="_internal")
			Workload Pool: pool_3

The command output shows only those workload rules that are currently enabled.

Check workload management status using REST

To view workload management status information, send an HTTP GET request to the workload/status endpoint. For endpoint details, see workloads/status in the REST API Reference Manual.