Manually assign searches to workload pools

Workload rules let you automatically assign searches to workload pools. While using workload rules to assign searches is considered the best practice, you can also assign searches to workload pools manually.

This page shows you how to assign searches to workload pools manually. For detailed instructions on how to assign searches to workload pools automatically using workload rules, see Create workload rules.

To assign searches manually, a role must have list_workload_pools and select_workload_pools capabilities. How you manually assign a search to a workload pool depends on the type of search.

Assign a scheduled search to a workload pool

You can assign a scheduled search to a workload pool using Splunk Web, CLI, or REST.

When you assign a scheduled search to a workload pool, the pool information is written to savedsearches.conf. For more information, see savedsearches.conf.spec.

Assign a scheduled search using Splunk Web

To assign a scheduled search to a workload pool using Splunk Web, follow these steps:

1. Click on Settings > Searches, Reports, and Alerts.
2. Find the specific saved search, and click Edit > Advanced Edit.
3. In the Workload Pool field, enter the name of the pool.
4. Click Save.
   The workload pool information is written to local/savedsearches.conf and the scheduled search runs in the specified pool.

Assign a scheduled search using the CLI

To assign a scheduled search to a workload pool, run the following CLI command:

./splunk add saved-search -name <search_name> -workload_pool <pool_name>

Assign a scheduled search using REST

Send a POST request to the saved/searches/{name} endpoint. For example:

curl -k -u admin:pass https://localhost:8089/services/searches/<search_name> -d workload_pool=<pool_name>

Assign an ad-hoc search to a workload pool

You can assign an ad-hoc search to a workload pool using Splunk Web, CLI, or REST.

Assign an ad-hoc search using Splunk Web

1. In the Search bar, enter your ad-hoc search string.
2. Select a workload pool from the menu.
3. Run the search.
   The ad-hoc search job runs in the specified workload pool.
   
   

   If you select Policy-Based Pool, workload management automatically assigns the search to a pool based on any defined workload rules. If an explicit match for the search is not found, workload management assigns the search to the default pool.

   The workload pool menu is only visible to roles that have list_workload_pools and select_workload_pools capabilities.

4. Click Job > Inspect Job > Search job properties.
5. Confirm that the ad-hoc search ran in the specified pool. For example:
   

Assign an ad-hoc search using CLI

To assign an ad-hoc search, run the following CLI command:

./splunk search "index=_internal" -workload_pool=<pool_name>

Assign an ad-hoc search using REST

Send a POST request to the search/jobs endpoint. For example:

curl -k -u admin:pass https://localhost:8089/services/search/jobs -d search="search index=_internal" -d workload_pool=pool_1

Change the workload pool for a running search

You can re-assign an actively running search to a different workload pool using Splunk Web or REST. This applies to both scheduled searches and ad-hoc searches.

To change the workload pool for a running search, a role must have the list_workload_pools and select_workload_pools capabilities. See Set access controls for workload management.

Change workload pool using Splunk Web

1. Click Activity > Jobs.
2. For the specific running search, click Job > Edit Job Settings.
3. Select a new pool from the Workload Pool menu.
   

Re-assigning an ad-hoc search on the Search bar triggers a new search process in the new pool. To continue running the same search process in a new pool, re-assign the search via the Job Activity page or REST endpoint.

Change workload pool using REST

Send a POST request to the search/jobs/{search_id}/control endpoint. For example:

curl -k -u admin:pass https://localhost:8089/services/search/jobs/{search_id}/control -d action=setworkloadpool -d workload_pool=<pool_name>

Assign accelerated reports to workload pools

You can assign any report that qualifies for acceleration to a workload pool.

Assigning an accelerated report to a workload pool with ample cpu and memory resources can help you minimize performance issues that can occur during report acceleration, which can be resource intensive.

You can assign an accelerated report to a workload pool using Splunk Web.

1. Click Settings > Searches, Reports, and Alerts.
2. Find the report you want to accelerate and click Edit > Edit Acceleration.
3. Select the Accelerate Report checkbox.
4. Select the Summary Range for the report acceleration.
5. Select a workload pool from the menu.
6. Click Save.

The workload pool that you specify in the Edit Acceleration modal is written to the auto_summarize.workload_pool setting in savedsearches.conf

For more information on report acceleration, see Accelerate reports in the Reporting Manual.

Assign accelerated data models to workload pools

You can assign an accelerated data model to a workload pool using Splunk Web:

1. Click Settings > Data models.
2. Find the data model you want to accelerate and click Edit > Edit Acceleration.
3. Select the Accelerate checkbox.
4. Select the Summary Range for the data model acceleration.
5. Select a workload pool from the menu.
6. Click Save.

For more information on accelerated data models, see Accelerate data models in the Knowledge Manager Manual.

CPU overflow and memory sharing

If a search exceeds the maximum CPU resources allocated to its workload pool, it is considered a soft limit, and the pool can borrow available CPU resources from other pools.

Similarly, if a search exceeds the maximum memory limit allocated to its pool, the pool can share memory resources from other pools up to the total amount of memory available for all pools in the category.

For more information, see Resource allocation in workload management.

Search concurrency considerations in workload management

It's important to consider search concurrency constraints when you assign searches to workload pools.

Search concurrency limits in workload management

Splunk Enterprise enforces concurrent search limits globally. As a result, in the context of resource reservation in workload management, searches are not entirely isolated, and increasing search load in one workload pool can limit the number of searches you can run in other pools.

The following concurrent search quotas can impact search performance in workload management:

Scheduler concurrency limits

This limit determines the maximum number of searches that the scheduler can run concurrently. For detailed information, see How the scheduler determines concurrent search limits.

User/role search quotas

This quota determines the maximum number of historical searches allowable for a specific user/role. These quotas are configured with srchJobsQuota and related settings in authorize.conf. See Authorize.conf.

To minimize search performance issues due to concurrent search limits, make sure adequate search quota is available.

For detailed information on how concurrent search quotas work in a search head cluster environment, see How the cluster handles concurrent search quotas.

Search priority in workload management

Search priority in workload management is determined by two main factors:

Search scheduler priority

When the total number of searches reaches the maximum concurrent search limit, the search scheduler runs additional searches in priority order as search quota becomes available. To ensure that important searches are not skipped, you can set a scheduled search to high-priority in the search scheduler. For more information, see Configure the priority of scheduled reports.

Workload rules order

Workload rules control access to resources in workload pools based on app or role. The order of a rule determines which apps or roles, and therefore which searches, have priority access to a workload pool. For more information, see Create workload rules.

To avoid skipped searches and other search concurrency issues due to search priority, make sure to assign high-priority searches to workload pools that provide sufficient resources.