Manage a rolling restart in Splunk Cloud Platform
Some configuration updates can cause the indexers in your Splunk Cloud Platform deployment to begin a process called a rolling restart. To minimize the impact of a rolling restart, deploy these updates during off-peak hours.
What users experience during a rolling restart
A rolling restart is a sequential restart of Splunk indexers that allows indexing to continue during the restart process.
While indexing remains available at all times during a rolling restart, non-Splunk clients that do not follow best practices for retrying connections and managing backpressure might be impacted by an individual node restarting. Using forwarders or other types of load balancers, rather than network inputs alone, increases the robustness of your indexing during a rolling restart.
Searches still run during a rolling restart, but they might return incomplete results. Users running searches in Splunk Web receive a message warning of incomplete search results.
What triggers a rolling restart
Deploying certain changes triggers a rolling restart. Examples of changes that trigger a rolling restart include, but are not limited to, the following tasks:
- Deleting the last HEC token (which deletes the app, causing a rolling restart).
- Installing some apps and add-ons. See Restart versus reload behavior of common apps and .conf files.
Deploying a seemingly safe change can indirectly trigger a rolling restart. For example, adding an index doesn't trigger a restart by itself. But if you or another admin has made other changes that trigger a rolling restart and not deployed them, then when you deploy your change that adds an index, you also deploy the previous changes that trigger the rolling restart.
Restart versus reload behavior of common apps and .conf files
Most configuration files do not trigger a restart when configuration changes occur, but instead trigger a less time-consuming file reload. To minimize service disruptions, before you install apps or deploy configuration updates in Splunk Cloud Platform, consider the restart versus reload behavior of relevant apps and configuration files.
For more information on configuration file reload behavior, see Configuration file reload triggers in app.conf.
The following tables list some common apps and configuration files and show whether they trigger a restart or a reload.
Reload or restart behavior of common .conf files
Most Splunk configuration files are now reloadable. The following table shows the reload or restart behavior of some frequently used configuration files in Splunk Cloud Platform:
| .conf file name | Used for | Reload or restart |
|---|---|---|
| authorize.conf | This file is used to configure roles and granular access controls. | reload |
| collections.conf | This file is used to configure KV store settings for a given app. | reload |
| distsearch.conf | This file is used to configure attributes and values you can use to configure distributed search. | reload |
| indexes.conf | This file is used to configure indexes and their properties.
|
reload/restart |
| inputs.conf | This file is used for HEC CRUD operations, configuring tcp ports for forwarders, configuring scripted inputs for apps, and configuring file system monitoring.
|
reload/restart |
| multikv.conf | This file is used to configure multikv rules for extracting events from table-like events, such as the output of top, ps, ls, netstat, etc. | reload |
| props.conf | This file is used to set indexing property configuration, including timezone offset, custom source type rules, and pattern collision properties. Also, map transforms to even properties. | reload |
| restmap.conf | This file is used to create custom REST endpoints. | reload |
| server.conf | This file is used to configure which settings should be replicated within a search head cluster.
|
reload/restart |
| transforms.conf | This file is used to configure regex transformations to perform on data inputs. Use in tandem with props.conf. | reload |
| ui-tour | This file is used to configure in-product tours of Splunk software features. | reload |
| web.conf | This file is used to configure tcp port to listen to incoming connections, appserverports, connectiontimeout. | reload |
| wmi.conf | This file is used to configure access to Windows Management Instrumentation (WMI). | reload |
Reload behavior of common apps
The following table shows the reload behavior of frequently used apps and add-ons in Splunk Cloud Platform:
This list pertains to the specified version of each app. Changes made to an app's configuration settings in subsequent app versions might trigger a restart instead of a reload.
| App name | Version | Used for | Reload |
|---|---|---|---|
| Cisco Networks Add-on for Splunk Enterprise | 2.5.8 | This add-on sets the correct sourcetype and fields for identifying data from Cisco IOS, IOS XE, IOS XR, NX-OS devices in Splunk® Enterprise. | reload |
| Force Directed App For Splunk | 3.0.1 | The Force Directed App For Splunk helps you graph out attack paths and review links in your data. Built on D3 this app will allow you to search any form of data that has a source and target. | reload |
| Lookup File Editor | 3.3.2 | This app provides an Excel-like interface for editing, importing, and exporting lookup files (both KV store and CSV based lookups) | reload |
| Palo Alto Networks Add-on for Splunk | 6.1.1 | This add-on collects and correlates data from Firewalls, Panorama, Traps Endpoints, Aperture SaaS Security, AutoFocus, MineMeld, and WildFire. | reload |
| Palo Alto Networks App for Splunk | 6.1.1 | This app combines Palo Alto Networks security platform features with Splunk's investigation and visualization capabilities to provide advanced security reporting and analysis. | reload |
| Python for Scientific Computing (for Linux 64-bit) | 1.4 | This add-on contains a Python interpreter bundled with the following scientific and machine learning libraries: numpy, scipy, pandas, scikit-learn, and statsmodels. With this add-on, you can import these powerful libraries in your own custom search commands, custom rest endpoints, modular inputs, and so forth. | reload |
| Punchcard Custom Visualization | 1.3.0 | This Punchcard Custom Visualization app provides interactive ways to visualize and investigate cyclical trends in your data. | reload |
| Qualys Technology Add-on (TA) for Splunk | 1.4.3 | This add-on provides pre-built inputs for Qualys Cloud Platform data. | reload |
| Splunk Add-on for Amazon Web Services | 4.6.0 | This add-on lets Splunk admins collect data from AWS accounts, including configuration details, EC2 instance and EBS metadata, compliance information, CloudWatch log data, performance and billing metrics, S3 bucket stats, and more. | reload |
| Splunk Add-on for Cisco ASA | 3.4.0 | The Splunk Add-on for Cisco ASA allows a Splunk software administrator to map Cisco ASA devices, Cisco PIX, and Cisco FWSM events to the Splunk CIM. | reload |
| Splunk Add-on for Microsoft Cloud Services | 3.1.0 | This add-on lets Splunk admins pull activity logs, service status, operational messages, Azure audit, Azure resource data and Azure Storage Table and Blob data from a variety of Microsoft cloud services using the Office 365 Management APIs, Azure Service Management APIs and Azure Storage API. | reload |
| Splunk Add-on for Microsoft Office 365 | 1.1.0 | This add-on lets Splunk admins pull service status, service messages, and management activity logs from the Office 365 Management API. | reload |
| Splunk Add-on for Microsoft Windows | 6.0.0 | This add-on provides predefined inputs to collect data from Windows systems and maps data to the Common Information Model. | reload |
| Splunk Add-on for Unix and Linux | 6.0.2 | The Splunk Add-on for Unix and Linux allows a Splunk software administrator to collect *nix data from *nix hosts. | reload |
| Splunk App for AWS | 5.1.3 | This app provides insight into your Amazon Web Services account. The app includes pre-built dashboards, reports, and alerts that provide real-time visibility into your AWS environment, including your AWS Config, CloudWatch, CloudTrail, Billing, S3, VPC Flow Log, Amazon Inspector, and Metadata inputs. | reload |
| Splunk App for Windows Infrastructure | 1.5.2 | This app provides pre-built data inputs, searches, reports, and dashboards that let you monitor, manage, and troubleshoot Windows operating systems, including Active Directory elements, from a single location. | reload |
| Splunk Common Information Model (CIM) | 4.13.0 | This add-on contains a collection of pre-configured data models that support the consistent, normalized treatment of data for maximum efficiency at search time. | reload |
| Splunk Dashboard Examples | 7.3.0 | The Splunk Dashboard app delivers examples that give you a hands-on way to learn the basic concepts and tools needed to rapidly create rich dashboards using Simple XML. | reload |
| Splunk Datasets Add-on | 1.0 | This app delivers new SPL commands, custom visualizations, assistants, and examples to explore a variety of machine learning concepts. | reload |
| Splunk Machine Learning Toolkit | 5.2.0 | This add-on provides an intuitive interface to build, edit and analyze table datasets (tables) without SPL. | reload |
| Splunk Sankey Diagram - Custom Visualizations | 1.5.0 | Sankey diagrams show metric flows and category relationships. You can use a Sankey diagram to visualize relationship density and trends. | reload |
| Splunk Supporting Add-on for Active Directory | 2.2.1 | This app provides support functions to the Windows Infrastructure, Active Directory, and Exchange apps that enable you to extract information from an Active Directory database. | reload |
| Splunk Timeline - Custom Visualization | 1.4.0 | A timeline visualization shows activity time intervals and discrete events for a resource set. | reload |
Guidance for managing a rolling restart
To minimize impact to users, deploy configuration changes during times that are off peak for both indexing and searching. You can identify off-peak times from the Snapshots in your Splunk Cloud Monitoring Console. See Monitor your Splunk Cloud Platform Deployment.
During a rolling restart, monitor indexing and search performance with the Splunk Cloud Monitoring Console.
More information
For more information about how a rolling restart works, see Perform a rolling restart of an indexer cluster in the Splunk Enterprise documentation. Note that some of the advanced options are not available by default in Splunk Cloud Platform.
| Manage the Splunk Product Guidance app on your Splunk Cloud Platform deployment | Configure hybrid search |
This documentation applies to the following versions of Splunk Cloud Platform™: 8.2.2112, 8.2.2201, 8.2.2202, 8.2.2203, 9.0.2205, 9.0.2208, 9.0.2209, 9.0.2303, 9.0.2305, 9.1.2308, 9.1.2312, 9.2.2403, 9.2.2406 (latest FedRAMP release)
Download manual
Feedback submitted, thanks!