Convert apps from Python 2 to Python 3

Convert your existing apps from Python 2 to Python 3.

Prerequisites

Your code is syntactically correct and you're able to successfully run the script for <app>_connector.py on the existing Python 2 app version. In the following example, phantom is the user, phipinfoio is the app directory, and ipinfoio is the app name:

[phantom@phantom phipinfoio]$ phenv python2.7 ipinfoio_connector.py

Run the 2to3 tool on the app

The 2to3 tool is located in ${PHANTOM_HOME}/bin

1. Run the following command:
   [phantom@phantom <app-dir>]$ phenv 2to3 <app-name>_connector.py
   This will output the recommended changes to make the code Python 3 compatible.
2. Review the recommended changes and make sure they make sense before putting them in.
   Some suggestions are wrong or unnecessary. The following is an example of a necessary change:

   -        print (json.dumps(json.loads(ret_val), indent=4))
   +        print((json.dumps(json.loads(ret_val), indent=4)))
   

Make further changes that 2to3 missed

2to3 is not perfect. It misses things like certain default module name changes.

The best way to figure that out is to test your app. You can test from the UI or from the command line. Run the app, and make changes where it crashes until it works. See Compile and install.

Revise any d.iteritems() in the app .py file

Check if your code contains the following statement:

for k,v in d.iteritems():

If so, replace d.iteritems() with six.iteritems(d).

Six is included, but you need to import it at the top of your file.

import six
...
six.iteritems(d)
...

Update the app .json file

Update the python_version and app_version keys in the app JSON file for the changes to take effect.

Update python_version key

Update the python_version key in the app JSON to the string '3'.

{
  "app_config_render": null,
  "product_version_regex": ".*",
  "python_version": "3",
  "uber_view": null,
  "disabled": false,
...
}

To compile using Python 3, the compile_app.pyc script expects the python_version key in the app JSON to be set to 3. Other acceptable values are strings of 3.6 and 2.7. Use 3 in preparation for any potential platform migrations to python3.7 or python3.8. For example, if you don't use 3, then you have to upgrade your app if Python is upgraded on the platform.

If you only have .py files, the compile script will use the version that you specify in your app .json file. That generates the .pyc files that the interpreter uses.

See App development script installation path for script usage.

Update app_version key

Update the app_version key to a higher version number.

{
...
 "app_version": "1.0.9",
  "type": "reputation",
  "product_name": "AbuseIPDB",
...
}

The app_version key accepts any update, such as from 1.0.0 to 1.0.1 or from 1.0.0 to 2.0.0. As long as the number is higher, the UI understands that it's an upgrade.

It is possible to switch back and forth in the UI between the app version that uses Python 2 & the version that uses Python 3. If you downgrade and go back to using the Python 2 version in the UI, it automatically runs the Python 2 code.

Dependencies for older versions

In the case of converting an app that is compatible with current and older versions of Splunk Phantom, there are multiple dependencies to consider.

If your app has custom views or its own REST handler, it needs to be compatible with Python 2 and ship with the source until a future release of Splunk Phantom when the entire platform is fully python3 compatible.

If your converted app has a dependency with functionally-equivalent versions, one of which is only available for Python 2 and the other for Python 3, then you might need to define pip3_dependencies in the app JSON alongside pip_dependencies so that the platform will install the different versions as needed. You only need to define pip3_dependencies if your app has a different set of dependencies when running in Python 3 than when running in Python 2. Otherwise, the platform will accept a Python 3 app that only defines pip_dependencies if the dependencies are the same.

Compile and reinstall

After completing all the steps on this page, verify if your code is syntactically correct. Verify it by running the compatible Python 3 equivalent script for <app>_connector.py, such as in the following example:

[phantom@phantom phipinfoio]$ phenv python3 ipinfoio_connector.pyc

Use the compatible Python 3 script to reinstall the app in the app directory. This compiles and installs it in the UI and lets you test the actions, such as in the following example:

[phantom@phantom phipinfoio]$ phenv python /opt/phantom/bin/compile_app.pyc -i 

See Compatible Python 3 scripts for more information about Python 3 scripts.

See your new app version in the UI

Both app versions are available in the Splunk Phantom UI.

1. Navigate to the Main Menu.
2. Select Apps.
3. Scroll to find the app or search for it by name.
4. Use the drop-down menu to to see the version numbers.