Python development with Splunk Enterprise

Check this manual often for updated information about the Splunk platform Python 3 migration. The content is subject to change.

The migration to Python 3 impacts Python scripts developed by Splunk app and add on developers and admins. In addition to changes to Python scripts, there are additional settings for Splunk administrators and prerequisites for upgrading to Splunk Enterprise version 8.x and higher. For more information, see Changes to Splunk Enterprise. For Splunk Enterprise version 8.x upgrade instructions, see Choose your Splunk Enterprise upgrade path for the Python 3 migration.

Developers must update Python scripts used in apps and add-ons for compatibility with Splunk Enterprise version 8.x. For guidelines for updating Python scripts in general, see Python Code Compatibility.

The following Splunk Enterprise features will require script-level compatibility with Python 3.7:

• Custom web controllers (such as CherryPy endpoints)
• Custom Mako templates

Splunk Enterprise provides settings to specify which Python interpreter to use at global and script levels, covered in Changes to Splunk Enterprise.

Removal of deprecated Splunk platform features

Some deprecated features have been removed from Splunk Enterprise versions 8.x and higher, which might require file revisions:

• Advanced XML (deprecated in Splunk version 6.3). If possible, replace Advanced XML with Simple XML. For more information about alternatives to Advanced XML available in Splunk Enterprise, see Building customizations for the Splunk platform.
• Splunk Web Legacy Mode (deprecated in Splunk version 6.4): do not set appServerPorts = 0 in web.conf.

Running against earlier indexer tiers

For apps that might run against a Splunk Enterprise version 7.3.x or earlier indexer tier, admins should ensure those apps use dual-compatible Python syntax. This is because custom search commands and scripted lookups will be passed to the indexer tier as part of the knowledge bundle, and any Python 3-specific syntax will fail on the indexer.

Module naming conflicts

You should also rename any files that conflict with Python standard modules or Splunk libraries, such as files named test.py or html.py. Use different, non-reserved names to avoid namespace conflicts in Python 3.

Splunk SDK for Python

The Splunk SDK for Python API and service wrappers are dual-compatible with Python 2 and Python 3, starting with version 1.6.5. Upgrade to the latest version of the Splunk SDK for Python to help make scripts that use the Splunk SDK for Python compatible with Splunk Enterprise versions 8.x and higher.

Identifying Python scripts

Splunk provides the Splunk Platform Upgrade Readiness app for admins to scan deployed apps for any components impacted by migration to Python 3. For more information, see the Splunk Platform Upgrade Readiness App.

You can also manually identify possibly impacted Python scripts in your app or deployment by taking the following steps:

• Identify files ending in *.py.
• Identify files in $SPLUNK_HOME/etc/apps/$<app_name>/bin/. These are typically custom scripts or inputs, which might not necessarily end in *.py. but can still be implicitly executed by the Python interpreter used by Splunk Enterprise.
• Identify any other files explicitly executed by the Python interpreter. These files are often executed by the command splunk cmd python $<script_name>.py. These files could contain shell scripts or could exist outside an app's or deployment's standard directories.

Splunk Web

To prevent issues starting Splunk Web, revise apps for Python 3 compatibility. If an app cannot be upgraded, it must be removed for Splunk Web to start.

Testing cross-compatible apps

For apps with scripts cross-compatible with Python 2 and 3, you will need to test your application in at least two Splunk Enterprise test deployments:

1. Use a Splunk Enterprise 7 deployment (version 7.2 or later) to test that your app runs as expected with a Python 2 runtime. Splunk Enterprise 7.2 or later forces your entire application to run in Python 2. 2. Use a Splunk Enterprise 8.x deployment with specific configuration settings to force your application to run in Python 3. You have two options:

• Set python.version=python3 in server.conf's [general] stanza to force all scripts in all applications to run in Python 3 only.
• Set python.version=python3 in the appropriate stanza of every .conf file that specifies Python scripts in your application.

For more information, see Python interpreter settings.

If you are creating an app that only runs in Python 3, you will only need a test environment for Splunk Enterprise 8.x or higher.

If your app uses Python that runs in the appserver, such as in the module system, you will not be able to select which Python runtime will be used for these scripts. Splunk Enterprise versions previous to Splunk Enterprise 8.x will always attempt to run these scripts with Python 2, and Splunk Enterprise 8.x and higher will always attempt to run these scripts with Python 3. Because of this, any app that you upload to Splunkbase and flag as 8.x compatible must be Python 3 compatible.