Splunk® Enterprise

Python 3 Migration

Download manual as PDF

This documentation does not apply to the most recent version of Splunk. Click here for the latest version.
Download topic as PDF

Changes to Splunk Enterprise

Check this manual often for updated information about the Splunk platform Python 3 migration. The content is subject to change.

The following changes to Splunk Enterprise cause breaking changes to existing Python scripts:

  • Python scripts in the deprecated module system. Any scripts with Python 2 syntax in [app]>appserver>modules>[module name] that aren't Python 3 compatible will cause UI errors, and should be made dual compatible with both Python 2 and 3.
  • Custom web controllers (such as CherryPy endpoints). Requires script-level compatibility with Python 3.7. Failure to make scripts compatible with Python 3.7 may cause issues starting Splunk Web.
  • Custom Mako templates. Requires script-level compatibility with Python 3.7. Failure to make scripts compatible with Python 3.7 may cause issues starting Splunk Web.
  • Advanced XML (deprecated in Splunk version 6.3): removed. If possible, replace Advanced XML with Simple XML. For more information about alternatives to Advanced XML available in Splunk Enterprise, see Building customizations for the Splunk platform.
  • Splunk Web Legacy Mode (deprecated in Splunk version 6.4): removed. Do not set appServerPorts = 0 in web.conf.

To prevent issues starting Splunk Web, revise apps for Python 3 compatibility. If an app cannot be upgraded, it must be removed for Splunk Web to start.

In addition to breaking changes and upgrade steps, a number of features and components are impacted with Splunk Enterprise version 8.x, but deployments can be migrated to Python 3.7 over time. Splunk Enterprise version 8.x includes both Python 2.7 and Python 3.7 interpreters, with Python 2.7 enabled for all custom Python scripts outside of those shipped with Splunk Enterprise.

The Python 2.7 interpreter is still used by default in Splunk Enterprise version 8.x for features that include:

  • Custom search commands
  • Custom REST endpoints
  • Scripted authentication
  • Scripted inputs
  • Modular inputs
  • Scripted lookups
  • Custom alert actions
  • Modular alerts
  • Cold-to-frozen scripts

Note that scripted alerts were deprecated in version 6.3 and are not supported in Python 3.7. Convert any scripted alerts to custom alert actions in order to adopt Python 3.7.

Using Python 3-only syntax for these features might be incompatible with the Python 2.7 interpreter which will be enabled by default in the Splunk Enterprise version 8.x. Python scripts that are compatible with both Python 2 and Python 3 should work with either Python interpreter used by Splunk Enterprise, and is recommended for developers. For more information about making Python scripts dual-compatible, see Python Development in Splunk Enterprise.

Splunk Web

Unlike Splunk Enterprise version 8.x, Splunk Web supports only Python version 3.7. Any scripts that depend on Splunk Web must be upgraded to use syntax compatible with both Python 2.7 and 3.7. This will allow upgrades to 8.x from 7.x versions of Splunk Enterprise.

To prevent issues starting Splunk Web, revise apps for Python 3 compatibility. If an app cannot be upgraded, it must be removed for Splunk Web to start.

Python interpreter settings

Splunk Enterprise version 8.x includes a global setting, python.version, to specify which Python interpreter to use across an instance. The global setting resides in the server.conf file, located in $SPLUNK_HOME/etc/system/local/. The stanza that controls Python version is [general]. For more information how Splunk Enterprise uses configuration files, see About configuration files.

For Splunk Enterprise version 8.1, python.version defaults to python3 within server.conf. For prior versions of Splunk Enterprise, python.version defaults to python2.

To force Splunk Enterprise to use only the Python 3 interpreter regardless of script-level setting, set python.version = force_python3. Use this setting if you cannot run Python 2.7 past its EOL date of January 1, 2020, or if your Splunk Enterprise deployment and all Splunk apps and add-ons are fully migrated and ready to run Python 3 only.

Splunk Enterprise also includes python.version settings to control which version of the Python interpreter is used by Splunk Enterprise at the script-level. For the following scripts, the python.version setting resides in the corresponding conf file:

Script type File
Custom search commands commands.conf
Modular inputs inputs.conf
Scripted inputs inputs.conf
Custom alert actions alert_actions.conf
Scripted lookups transforms.conf
Custom REST endpoints restmap.conf
Scripted authentication authentication.conf
coldToFrozenScript indexes.conf

By default, the script-level setting of python.version is not set, and the script will use the Python interpreter specified by the global setting in server.conf. Setting python.version to default or python also uses the Python interpreter specified by the global setting in server.conf. If set to python2 or python3, the corresponding Python interpreter will be used. This overrides the global setting, except if the global setting is force3, in which case Python 3 is always used.

Set python.version to python3 or default to remove Python 3 migration-related start up warnings for your impacted apps.

Setting python.version for coldToFrozenScript applies if the canonical path to the Python interpreter is used. However, note that for coldToFrozen:

    * scripts set executable on UNIX with a #! shebang line pointing to a valid interpreter.

If your script is specified with #! /usr/bin/env python, then python.version will be ignored for coldToFrozen. Also note that for warmToCold, this is always how the Python script is specified, so there is no applicable python.version for warmToCold.

Check this manual often for updated information about the Splunk platform Python 3 migration. The content is subject to change.

Search and Reporting

If you have modified Splunk Search and Reporting with scripts or other customizations using Python 2, you must update these scripts to use Python 3 syntax or to be dual-compatible with both Python 2 and Python 3.

If you must maintain Python 2 compatibility, use Python compatibility libraries provided with Splunk Enterprise to help make apps and add-ons compatible with both Python 2 and Python 3. (Six, Python-future, 2to3). Splunk also updated Splunk Enterprise 7.x maintenance releases to include these Python compatibility libraries. These Splunk-provided libraries should not be used for any other apps; all custom or Splunkbase apps should package their own libraries to respect app structures.

Analytics for Hadoop and Hadoop Data Roll

Analytics for Hadoop and Hadoop Data roll do not support Python 3 in Splunk Enterprise version 8.x. When using Hadoop with Splunk Enterprise:

  • Do not set set python.version = python3 for the global python.version setting, which resides in the server.conf file.
  • Do not remove the Python 2.7 runtime. If your deployment requires the removal of Python 2.7 for for security compliance reasons, contact Splunk Support.

Splunk Platform Upgrade Readiness app

Splunk provides the Splunk Platform Upgrade Readiness app for admins to scan deployed apps for any components impacted by migration to Python 3. The app is recommended to prepare for an upgrade to Splunk Enterprise version 8.x. For more information, see the Splunk Platform Upgrade Readiness App.

Splunkbase apps and add-ons

Impacted Splunkbase apps and add ons must be resubmitted to Splunkbase after validation of compatibility with the Splunk Enterprise version 8.x, including Python 3 testing with AppInspect. Apps that are marked compatible with Splunk Enterprise 7.x and below are Python 2.7-compatible only, while apps that are marked compatible with Splunk Enterprise 8.x are Python 3.7-compatible only. Apps that are marked compatible with Splunk Enterprise 7.x and 8.x are compatible with both Python 2.7 and 3.7.

Last modified on 19 October, 2020
PREVIOUS
Python 3 migration with the Splunk platform
  NEXT
Python development with Splunk Enterprise

This documentation applies to the following versions of Splunk® Enterprise: 7.0.0, 7.0.1, 7.0.2, 7.0.3, 7.0.4, 7.0.5, 7.0.6, 7.0.7, 7.0.8, 7.0.9, 7.0.10, 7.0.11, 7.0.13, 7.1.0, 7.1.1, 7.1.2, 7.1.3, 7.1.4, 7.1.5, 7.1.6, 7.1.7, 7.1.8, 7.1.9, 7.1.10, 7.2.0, 7.2.1, 7.2.2, 7.2.3, 7.2.4, 7.2.5, 7.2.6, 7.2.7, 7.2.8, 7.2.9, 7.2.10, 7.3.0, 7.3.1, 7.3.2, 7.3.3, 7.3.4, 7.3.5, 7.3.6, 7.3.7, 8.0.0, 8.0.1, 8.0.2, 8.0.3, 8.0.4, 8.0.5, 8.0.6


Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters