Manage a rolling restart in Splunk Cloud
Some configuration updates can cause the indexers in your Splunk Cloud deployment to begin a process called a rolling restart. To minimize the impact of a rolling restart, deploy these updates during off-peak hours.
What users experience during a rolling restart
A rolling restart is a sequential restart of Splunk indexers that allows indexing to continue during the restart process.
While indexing remains available at all times during a rolling restart, non-Splunk clients that do not follow best practices for retrying connections and managing backpressure might be impacted by an individual node restarting. Using forwarders or other types of load balancers, rather than network inputs alone, increases the robustness of your indexing during a rolling restart.
Searches still run during a rolling restart, but they might return incomplete results. Users running searches in Splunk Web receive a message warning of incomplete search results.
What triggers a rolling restart
Deploying certain configuration changes triggers a rolling restart. Examples of changes that trigger a rolling restart include, but are not limited to, the following tasks:
- Source type management
- Deleting an index
- Deleting the last HEC token (which deletes the app, causing a rolling restart)
- Installing some apps and add-ons ( See Restart vs. reload behavior of common apps and .conf files).
Adding an index, for example, does not trigger a restart by itself. But if you or another admin has made other configuration changes and not deployed them, then when you deploy your change that adds an index, you also deploy the previous changes. In this way, deploying a seemingly safe change can indirectly trigger a rolling restart.
Restart vs. reload behavior of common apps and .conf files
Many configuration files do not trigger a rolling restart when configuration changes occur, but instead trigger a less time consuming file reload. To minimize service disruptions, before installing apps or deploying configuration changes, consider the restart behavior of frequently used configuration files. The following tables list common apps and configuration files and show whether they trigger a restart or a reload.
Rolling restart behavior of common .conf files
The table shows rolling restart behavior of common configuration files in Splunk Cloud.
|.conf file name||Used for||Reload or restart|
|authorize.conf||This file is used to configure roles and granular access controls.||reload|
|collections.conf||This file is used to configure KV store settings for a given app.||reload|
|distsearch.conf||This file is used to configure attributes and values you can use to configure distributed search.||reload|
|indexes.conf||This file is used to configure indexes and their properties.
|inputs.conf||This file is used for HEC CRUD operations, configuring tcp ports for forwarders, configuring scripted inputs for apps, and configuring file system monitoring.
|multikv.conf||This file is used to configure multikv rules for extracting events from table-like events, such as the output of top, ps, ls, netstat, etc.||reload|
|restmap.conf||This file is used to create custom REST endpoints.||reload|
|server.conf||This file is used to configure which settings should be replicated within a search head cluster.
|ui-tour||This file is used to configure in-product tours of Splunk software features.||reload|
|web.conf||This file is used to configure tcp port to listen to incoming connections, appserverports, connectiontimeout.||reload|
|wmi.conf||This file is used to configure access to Windows Management Instrumentation (WMI).||reload|
Rolling restart behavior of common apps
The table shows rolling restart behavior of common apps and add-ons in Splunk Cloud.
|App name||Used for||Reload or restart|
|Force Directed App For Splunk||The Force Directed App For Splunk helps you graph out attack paths and review links in your data. Built on D3 this app will allow you to search any form of data that has a source and target.||reload|
|Lookup File Editor||This app provides an Excel-like interface for editing, importing, and exporting lookup files (both KV store and CSV based lookups)||reload|
|Python for Scientific Computing
(for Linux 64-bit)
|This add-on contains a Python interpreter bundled with the following scientific and machine learning libraries: numpy, scipy, pandas, scikit-learn, and statsmodels. With this add-on, you can import these powerful libraries in your own custom search commands, custom rest endpoints, modular inputs, and so forth.||reload|
|Punchcard Custom Visualization||This Punchcard Custom Visualization app provides interactive ways to visualize and investigate cyclical trends in your data.||reload|
|Splunk Add-on for Unix and Linux||The Splunk Add-on for Unix and Linux allows a Splunk software administrator to collect *nix data from *nix hosts.||reload|
|Splunk Dashboard Examples||The Splunk Dashboard app delivers examples that give you a hands-on way to learn the basic concepts and tools needed to rapidly create rich dashboards using Simple XML.||reload|
Guidance for managing a rolling restart
To minimize impact to users, deploy configuration changes during times that are off peak for both indexing and searching. You can identify off-peak times from the Snapshots in your Splunk Cloud Monitoring Console. See Monitor your Splunk Cloud Deployment.
During a rolling restart, monitor indexing and search performance with the Splunk Cloud Monitoring Console.
For more information about how a rolling restart works, see Perform a rolling restart of an indexer cluster in the Splunk Enterprise documentation. Note that some of the advanced options are not available by default in Splunk Cloud.
Manage private apps in your Splunk Cloud deployment
This documentation applies to the following versions of Splunk Cloud™: 8.0.2006