Write better playbooks by following these guidelines

This topic is a collection of guidance for users who create playbooks, or custom functions for playbooks. This is not an exhaustive list, but is instead designed to cover certain key ideas that impact how you should handle variables in your playbooks and custom code.

How playbooks and playbook blocks are run

When you run a playbook, the blocks are run sequentially by a Python runner. A playbook block or custom function must be completed before the runner executes the next playbook or custom function in sequence.

With the release of 6.3.0 your playbook's blocks can be run by multiple Python runners, each python runner in its own isolated container.

• When a playbook run is started, the DECIDED daemon assigns the playbook's blocks to an available Python runner in a round-robin fashion.
• Blocks from the same playbook run are run sequentially. A single playbook run will only run one block at a time.
• Previous releases of guaranteed that all the blocks of a playbook run would be run by the same Python runner. Now, playbook blocks can be run by any available Python runner.
• The number of Python runners is automatically scaled up or down as needed.

How multiple python runners impact playbook behavior

When multiple python runners are active, playbooks behave differently than in earlier releases of .

• The playbook API save_data may return incorrect results when playbook blocks are run by different runners if the key:value pairs are not unique across playbook runs. Use the save_object() API instead of the save_data() API.
• The playbook API save_object may return incorrect results if the same playbook is run against the same container multiple times. Use the playbook_name and container_id parameters with save_object to make sure that saved objects are unique across multiple runs of the same playbook.
• Playbooks cannot share information between playbook runs by using the host's file system.
• The directories /tmp and /opt/phantom/tmp cannot be used to share information between playbook runs. These directories can still be used to share information in the context of a single playbook run.
• If you need to save information specifically about a playbook run, use the save_run_data() and get_run_data() APIs.
• Playbooks cannot read or modify the directory /opt/phantom/vault by using the file system. Playbooks that interact with the vault must use the Vault automation API.
• Playbooks should not create subprocesses, either by using the built-in os.system python function or the built-in subprocess python module.

Use local variables instead of global variables

Values stored in a global variable could be modified by another instance of the playbook, a child playbook, or another process that uses the same variable resulting in unexpected or incorrect results. This means new practices for handling variables are required.

When creating or editing playbook source code, use local variables rather than global ones. New Pylint rules have been implemented to properly enforce the handling of global variables.

• You may use global variables if they are read only.
• You can use global variables at the module level, but the variable must be contained within a static condition check.
• You cannot update a global variable on non-module level, such as at the function, class, or enclosing scope level. Read is allowed.

Examples:

import datetime

mutable_list = ["test0"]
mutable_set = {0}
mutable_dict = {"test0":"val0"}
immutable_var = 1
immutable_str = "test"

# Updating global variables at the module level is allowed
mutable_list = ["test"]
mutable_set.add("test")
mutable_dict["test"] = "val"
immutable_var += 2

# Updating global variables within static conditions at module level is allowed
if immutable_str:
    immutable_var += 3

while False:
    mutable_dict["test"] = "updated"

# Updating global variables within non-static conditions at module level is not allowed
if datetime.datetime.now().day == 3:
    immutable_var = 3

def main():
    # At non-module level
    # Assignment of local immutable variables is allowed
    immutable_str = "test_local"
    immutable_str[0] = "f"
    immutable_str = immutable_str.lower()
    # Updating global variables with assignment is not allowed
    mutable_list = ["test1"]
    mutable_set = set("test1")
    mutable_dict = {"test1":"val1"}
    global immutable_var
    immutable_var = 3
    # Updating global variables with function calls is not allowed
    mutable_list.append("test2")
    mutable_set.add("test2")
    mutable_dict.update({"test1":"val2"})
    # Updating global variables with subscripting is not allowed
    mutable_list[0] = "test3"
    mutable_dict["test1"] = "val3"
    # Updating global variables with augmented assignments is not allowed
    mutable_list+=["test4"]
    mutable_dict+={"test4":"val4"}
    global immutable_var
    immutable_var+=1
    # Reading global variables is allowed
    print(mutable_list)
    print(mutable_dict)
    print(mutable_set)
    print(mutable_list[0])
    print(mutable_dict["test"])
    print(mutable_set[0])
    for e in enumerate(mutable_list):
        print(e)

    for k,v in mutable_dict.items():
        print((k,v))

    global immutable_var
    print(immutable_var)
    garbage = undefined + 1
    garbage = undefined + 1

if __name__ == "__main__":
    mutable_list = ["test"]
    immutable_str += "3"

How to exchange data between playbooks or blocks

You must exercise caution about data consistency in your playbooks. When there was only a single Python runner, playbooks ran in series so no other playbook would attempt to access or modify information during the playbook run. Now that your playbook blocks can be run by different Python runners, multiple playbooks could be using the same variables or objects, so data consistency is more difficult to guarantee.

Using local variables and the correct APIs for passing data between playbook blocks or runs helps to ensure your data is consistent and correct.

• If you want to share data within the same playbook run use the APIs save_run_data and get_run_data to exchange information about specific keys. See get_run_data and save_run_data in the Python Playbook API Reference for .
• If you want to share data between playbook runs with the same playbook name, between playbook runs that operate on the same container, or across all playbook runs, then you need to use the APIs clear_object, get_object, and save_object. See get_object, save_object, and clear_object in the Python Playbook API Reference for .
• When exchanging data between playbook runs, remember that the order in which playbooks are run is not guaranteed. If you need to ensure results from a specific playbook are available, call that playbook as a dependent playbook.