Python development with Splunk Enterprise

Check this manual often for updated information about the Splunk platform Python 3 migration. The content is subject to change.

The migration to Python 3 impacts Python scripts developed by Splunk app and add on developers and admins. In addition to changes to Python scripts, there are additional settings for Splunk administrators and prerequisites for upgrading to Splunk Enterprise version 8.x. For more information, see Changes to Splunk Enterprise. For Splunk Enterprise version 8.x upgrade instructions, see Choose your Splunk Enterprise upgrade path for the Python 3 migration.

Developers must update Python scripts used in apps and add-ons for compatibility with Splunk Enterprise version 8.x. For guidelines for updating Python scripts in general, see Python Code Compatibility.

The following Splunk Enterprise features will require script-level compatibility with Python 3.7:

• Custom web controllers (such as CherryPy endpoints)
• Custom Mako templates

These must be made dual-compatible with both Python 2 and 3 to prevent breakage for customers upon upgrade.

Splunk Enterprise provides settings to specify which Python interpreter to use at global and script levels, covered in Changes to Splunk Enterprise.

Removal of deprecated Splunk platform features

Some deprecated features have been removed from Splunk Enterprise version 8.x, which might require file revisions:

• Advanced XML (deprecated in Splunk version 6.3). If possible, replace Advanced XML with Simple XML. For more information about alternatives to Advanced XML available in Splunk Enterprise, see Building customizations for the Splunk platform.
• Splunk Web Legacy Mode (deprecated in Splunk version 6.4): do not set appServerPorts = 0 in web.conf.

Writing scripts compatible with Python 2 and Python 3

Developers must make all Python files and scripts compatible with Python 3 to be compatible with Splunk Enterprise version 8.x. When making apps and scripts compatible with Python 3, Splunk recommends writing dual-compatible Python code that works with both Python 2 and Python 3 interpreters. For more information about Python compatibility libraries, see Python Code Compatibility.

Apps that must be written in Python 3-only syntax should set python.version to python3 in the appropriate .conf files for individual scripts. Developers should not set python.version in server.conf. For more information about python.version settings, see Changes to Splunk Enterprise.

You should properly store and import cross-compatible Python libraries and update the Python path according to guidelines provided in The directory structure of a Splunk App in Splunk developer docs.

Running against earlier indexer tiers

For apps that might run against a Splunk Enterprise version 7.3.x or earlier indexer tier, admins should ensure those apps use dual-compatible Python syntax. This is because custom search commands and scripted lookups will be passed to the indexer tier as part of the knowledge bundle, and any Python 3-specific syntax will fail on the indexer.

Module naming conflicts

You should also rename any files that conflict with Python standard modules or Splunk libraries, such as files named test.py or html.py. Use different, non-reserved names to avoid namespace conflicts in Python 3.

Splunk SDK for Python

The Splunk SDK for Python API and service wrappers are dual-compatible with Python 2 and Python 3, starting with version 1.6.5. Upgrade to the latest version of the Splunk SDK for Python to help make scripts that use the Splunk SDK for Python compatible with the Splunk Enterprise version 8.x.

Identifying Python scripts

Splunk provides the Splunk Platform Upgrade Readiness app for admins to scan deployed apps for any components impacted by migration to Python 3. For more information, see the Splunk Platform Upgrade Readiness App.

You can also manually identify possibly impacted Python scripts in your app or deployment by taking the following steps:

• Identify files ending in *.py.
• Identify files in $SPLUNK_HOME/etc/apps/$<app_name>/bin/. These are typically custom scripts or inputs, which might not necessarily end in *.py. but can still be implicitly executed by the Python interpreter used by Splunk Enterprise.
• Identify any other files explicitly executed by the Python interpreter. These files are often executed by the command splunk cmd python $<script_name>.py. These files could contain shell scripts or could exist outside an app's or deployment's standard directories.

Splunk Web

Unlike Splunk Enterprise version 8.x, Splunk Web supports only Python version 3.7. Any scripts that depend on Splunk Web must be upgraded to use syntax compatible with both Python 2.7 and 3.7. This will allow upgrades to 8.x from 7.x versions of Splunk Enterprise.

Testing your app

For apps with scripts cross-compatible with both versions of Python, you will need to test your application in at least two Splunk Enterprise test deployments:

1. Use a Splunk Enterprise 7 deployment (version 7.2 or later) to test that your app runs as expected with a Python 2 runtime. Splunk Enterprise 7.2 or later forces your entire application to run in Python 2. 2. Use a Splunk Enterprise 8.x deployment with specific configuration settings to force your application to run in Python 3. You have two options:

• Set python.version=python3 in server.conf's [general] stanza to force all scripts in all applications to run in Python 3 only.
• Set python.version=python3 in the appropriate stanza of every .conf file that specifies Python scripts in your application.

For more information, see Python interpreter settings.

If you are creating an app that only runs in Python 3, you will only need a test environment for Splunk Enterprise 8.x.

If your app uses Python that runs in the appserver, such as in the module system, you will not be able to select which Python runtime will be used for these scripts. Splunk Enterprise versions previous to Splunk Enterprise 8.x will always attempt to run these scripts with Python 2, and Splunk Enterprise 8.x will always attempt to run these scripts with Python 3. Because of this, any app that you upload to Splunkbase and flag as 8.x compatible must be Python 3 compatible.