Data management automation API

The Splunk Phantom Automation API allows security operations teams to develop detailed and precise automation strategies. Playbooks can serve many purposes, ranging from automating minimal investigative tasks that can speed up analysis to large-scale responses to a security breach. The following APIs are supported to leverage the capabilities of data management automation using playbooks.

add_list

Use the add_list API to append a new row of data to the custom list named by list name. If the list does not exist, it is created. Values are converted to a list of strings. Upon completion, the API returns a tuple of a success flag and response messages.

The add_list API is supported from within a custom function.

phantom.add_list(list_name=None, values=None)

Parameter	Required?	Type	Description
list_name	Required	String	The name of the custom list to add an item to.
values	Required	String or list of strings	The values to be added.

This sample uses the phantom.add_list API.

import phantom.rules as phantom
import json

def on_start(container):
    #in the product in 'Playbooks / Custom Lists', define a
    # list called 'executives' and then access it here
    success, message = phantom.add_list(
        list_name='executives', values=[ 'bob.jones@splunk.com' ] )

    phantom.debug(
        'phantom.add_list results: success: {}, message: {}' \
        .format(success, message)
    )

    success, message = phantom.add_list(
        list_name='executives', values=[ 'susan.smith@splunk.com' ] )

    phantom.debug(
        'phantom.add_list results: success: {}, message: {}' \
        .format(success, message)
    )

    return

def on_finish(container, summary):
    return

check_list

Use the check_list API to check whether a value is in a custom list or not. The default behavior, case_sensitive=False, is case insensitive and searches complete strings using substring=False. The API returns a tuple of a success flag, any response messages, and the number of matching rows in the custom list.

The check_list API is not supported from within a custom function.

phantom.check_list(list_name=None, value=None, case_sensitive=False, substring=False)

Parameter	Required?	Type	Description
list_name	Required	String	The name of the custom list to be searched.
value	Required	String or list of strings	The value to be searched.
case_sensitive	Optional	Boolean	Change the case sensitivity with this parameter. The default behavior is case insensitive.
substring	Optional	Boolean	Change the substring with this parameter. The default behavior is complete string match.

This sample uses the phantom.check_list() API.

import phantom.rules as phantom

def on_start(container):
    phantom.debug('phantom.check_list start')

    success, message, matched_row_count = \
        phantom.check_list(list_name='Example List', value='Example Value')

    phantom.debug(
      'phantom.check_list results: success: {}, message: {}, matched_row_count: {}'\
      .format(success, message, matched_row_count)
    )

    return

def on_finish(container, summary):
    return

clear_object

Use the clear_object API to delete data saved through the save_object() API.

The clear_object API is not supported from within a custom function.

phantom.clear_object(key=None, container_id=None, playbook_name=None, repo_name=None)

Parameter	Required?	Description
key	Required	The key parameter that was used in the save_object() API.
container_id	Optional, unless playbook_name is not provided. Can also be used together with a playbook_name.	The same container_id parameter that was used in save_object() API.
playbook_name	Optional, unless container_id is not provided. Can also be used together with container_id.	The same playbook_name parameter that was used in save_object() API.
repo_name	Optional	The same repo_name parameter that was used in save_object() API.

The container_id and playbook_name parameters support Postgres LIKE patterns. If the pattern does not contain percent signs or underscores, then the pattern only represents the string itself. In that case, LIKE acts like the equals operator. An underscore in the pattern stands for any single character and a percent sign matches any sequence of zero or more characters.

delete_from_list

Use the delete_from_list API to remove rows from a list that contain a specific value. The delete_from_list API returns a tuple of a success flag and any response messages.

The delete_from_list API is supported from within a custom function.

phantom.delete_from_list(list_name=None, value=None, column=None, remove_all=False, remove_row=False)

Parameter	Required?	Type	Description
list_name	Required	String	The name of the custom list to be modified.
value	Required	String	Replaces cells containing value with None.
column	Optional	Positive integer	Zero-based index, checks for values in this column.
remove_all	Optional	boolean	If True, replace all occurrences of value with None. Otherwise, the API fails if multiple are found.
remove_row	Optional	boolean	If True, remove the full row where value was found.

The following sample uses the phantom.delete_from_list() API.

import phantom.rules as phantom
import json

def on_start(container):
    #in the product in 'Playbooks / Custom Lists', define a
    # list called 'example' with two rows. Creates the list if it does not exist.
    success, message = phantom.delete_from_list(
        list_name='example',
        value='deleteme')
    phantom.debug(
      'phantom.delete_from_list results: success: {}, message: {}'\
      .format(success, message)
    )
    return

def on_finish(container, summary):
    return

You can't delete all of the rows from the list, and phantom.delete_from_list commands that attempt to do so result in errors. At least one row must be present in the list.

get_list

Use the get_list API to access custom lists. Custom lists are lists of dictionaries that allow you to manage data that can be referenced in Splunk Phantom playbooks. You can view or maintain these lists from navigation to the Splunk Phantom main menu, and then click Playbooks > Custom Lists.

The get_list API is supported from within a custom function.

phantom.get_list(list_name=None, values=None, column_index=-1, trace=False)

Parameter	Required?	Type	Description
list_name	Required	String	The name of the custom list to retrieve.
values	Optional	String or list of strings	A value or a list of values to search. If a value isn't included, the full list is retrieved.
column_index	Optional	Integer	Used to specify a specific column to retrieve.
trace	Optional	Boolean	Trace is a flag related to the level of logging. If trace is on (True), more logging is enabled. When set to True, more detailed output is displayed in debug output.

This sample uses the phantom.get_list() API.

import phantom.rules as phantom
import json

def on_start(container):
    #in the product in 'Playbooks / Custom Lists', define a
    # list called 'executives' and then access it here
    success, message, execs = phantom.get_list(list_name='executives')

    phantom.debug(
        'phantom.get_list results: success: {}, message: {}, execs: {}'\
        .format(success, message, execs)
    )

    return

def on_finish(container, summary):
    return

get_object

Use the get_object API to retrieve data that was saved through the save_object API. See save_object for a sample playbook and usage of this API.

The get_object API is not supported from within a custom function.

phantom.get_object(key=None, clear_data=False, container_id=None, playbook_name=None, repo_name=None)

Parameter	Required?	Description
key	Required	The key specified in the save_object() API that is used when saving data.
clear_data	Optional	If set True, this parameter clears the data after fetching. Defaults to False.
container_id	Optional, unless playbook_name is not specified. Can also be specified with a playbook_name.	The container ID specified when the data was saved.
playbook_name	Optional, unless container_id is not specified. Can also be specified with container_id	The playbook name as specified when the data was saved.
repo_name	Optional	The repository name as specified when saving the data in the save_object API.

The key parameter supports Postgres LIKE patterns. If the pattern does not contain percent signs or underscores, then the pattern only represents the string itself. In that case, LIKE acts like the equals operator. An underscore in the pattern stands for any single character and a percent sign matches any sequence of zero or more characters.

This sample shows the return value as a list of dictionaries where each dictionary has search criteria and the value for the specified combination of parameters.

[
    {
        "composite_key": {
            "container_id": <>,
            "playbook_name": <>,
            "key": <>
        },
        "value": {<>
        }
    }
]

get_run_data

Use the get_run_data method to return the data value saved for the specified key through the phantom.save_run_data() method. If the key is not specified, the get_run_data API returns data for all the keys as a string object.

The get_run_data API is supported from within a custom function.

phantom.get_run_data(key=None)

This sample shows result data if a key is not specified when calling get_run_data.

{
   "specified_key_on_save_run_data_call" : {
      "auto_" : true,
      "data_" : "specified value on save run data call with key provided"
   },
   "ef106ce1-301d-490d-9b96-d16b7e3a1a85" : {
      "auto_" : true,
      "data_" : "specified value on save run data call without key provided"
   }
}

remove_list

Use the remove_list API to delete a list. If you use the empty_list option, the list still exists, but is cleared of all values.

The remove_list API is supported from within a custom function.

phantom.remove_list(list_name=None, empty_list=False, trace=False)

Parameter	Required?	Type	Description
list_name	Required	String	The name of the custom list to delete.
empty_list	Optional	Boolean	Setting this parameter to True clears the list contents instead of removing the list.
trace	Optional	Boolean	Trace is a flag related to the level of logging. If trace is on (True), more logging is enabled. When set to True, more detailed output is displayed in debug output.

For remove_list API to work, give the automation user permissions to delete lists.

This sample uses the phantom.remove_list() API.

import phantom.rules as phantom
import json

def on_start(container):
    phantom.debug('phantom.remove_list start')

    success, message = phantom.remove_list(list_name='example')

    phantom.debug(
      'phantom.remove_list results: success: {}, message: {}'\
      .format(success, message)
    )

    return

def on_finish(container, summary):
    return

save_object

Use the save_object API to save data by key, container ID or the playbook name to be retrieved when executing playbooks on containers. You can save a key and value pair along with the context of container ID or the playbook name. Only JSON compliant objects, dictionaries, lists, strings, and numbers are supported as objects that can be saved and retrieved.

The save_object API is not supported from within a custom function.

 phantom.save_object(key=None, value=None, container_id=None,
                    auto_delete=False, playbook_name=None,
                    repo_name=None)

Parameter	Required?	Description
key	Required	Specify key to save and retrieve data by this unique key.
value	Required	The data to be saved. Use a expected to be a Python dictionary object.
container_id	Optional, unless the playbook_name parameter is not specified.	This parameter is the container ID as a context to the data being saved. You must provide this parameter if auto_delete is True.
auto_delete	Optional, unless the container_ID parameter is not provided	Defaults to False. If set to True, the data is deleted when the container is closed. You can use the clear_object parameter to delete the data. If the parameter is set True, you must provide the container ID.
playbook_name	Optional	The playbook name, which is also saved as context to the data.
repo_name	Optional	Specify this parameter when a playbook name is provided, because a particular playbook might exist in more than one repository.

This sample playbook uses save_object.

import phantom.rules as phantom
import json
from datetime import datetime, timedelta

def on_start(container):
    phantom.debug('on_start() called {}'.format(container))

    pb_info = phantom.get_playbook_info()
    phantom.debug(pb_info)
    if not pb_info:
        return

    playbook_name = pb_info[0].get('name', None)
    container_id = container['id']


    # SAVE data with key, container id and Playbook name
    phantom.save_object(key="key1", value={'value':'key 1 data for
        container and playbook'}, auto_delete=True, container_id = container_id,
        playbook_name=playbook_name)

    phantom.save_object(key="key2", value={'value':'key 2 data for
        container and playbook'}, auto_delete=True, container_id = container_id,
        playbook_name=playbook_name)

    # SAVE data with key, container id but NO Playbook name
    phantom.save_object(key="key1", value={'value':'key 1 data for
        only container and NO playbook'}, auto_delete=True,
        container_id = container_id)

    # SAVE data with key, Playbook name and NO container id
    phantom.save_object(key="key1", value={'value':'key 1 data for
        only playbook and not container'}, auto_delete=False,
        playbook_name=playbook_name)


    my_key = "key1"
    data = phantom.get_object(key=my_key,
            container_id = container_id, playbook_name=playbook_name)

    phantom.debug("get by key, container_id and playbook name:
        {} records found".format(len(data)))
    phantom.debug(data)

    data = phantom.get_object(key=my_key, container_id = container_id)

    phantom.debug("get by key, and container_id and
        NO playbook name: {} records found".format(len(data)))
    phantom.debug(data)

    data = phantom.get_object(key=my_key,
        playbook_name=playbook_name)

    phantom.debug("get by key, and playbook name and no container id:
        {} records found".format(len(data)))
    phantom.debug(data)

    data = phantom.get_object(key=my_key, playbook_name="%%")

    phantom.debug("get by key, and ALL playbook name and
        no container id: {} records found".format(len(data)))
    phantom.debug(data)

    data = phantom.get_object(key="%%",
        container_id = container_id)

    phantom.debug("get for ALL key, and container_id and
        no playbook name: {} records found".format(len(data)))
    phantom.debug(data)

    data = phantom.get_object(key="%%", container_id = container_id,
        playbook_name=playbook_name)

    phantom.debug("get for ALL key, and container_id and playbook name:
        {} records found".format(len(data)))
    phantom.debug(data)

    # TESTING CLEAR API

    data = phantom.get_object(key="key1", container_id = container_id,
        playbook_name=playbook_name)
    phantom.debug("BEFORE clear ... get for key1, and container_id and playbook
        name: {} records found".format(len(data)))
    phantom.debug(data)

    phantom.clear_object(key="key1", container_id = container_id,
        playbook_name=playbook_name)

    data = phantom.get_object(key="key1", container_id = container_id,
        playbook_name=playbook_name)

    phantom.debug("AFTER clear ... get for key1, and container_id and playbook
        name: {} records found".format(len(data)))
    phantom.debug(data)


    return

def on_finish(container, summary):
    phantom.debug('on_finish() called')
    return

The output of the playbook in debugger shows the following results.

Fri Mar 24 2017 07:13:08 GMT-0700 (PDT): Starting playbook 'local/automation_data' (id: 112, version: 35) on 'incident'(id: 10) with playbook run id: 72
Fri Mar 24 2017 07:13:08 GMT-0700 (PDT): calling on_start() on incident 'CryptoLocker Ransomware Infection (new, SLA breached) (192.168.1.41)'(id: 10).
Fri Mar 24 2017 07:13:08 GMT-0700 (PDT): on_start() called {'sensitivity': 'amber', 'create_time': '2017-03-23 03:54:12.981519+00', 'owner': 'bob.tailor@splunk.com', 'id': 10, 'close_time': '', 'severity': 'high', 'label': 'incident', 'due_time': '2017-03-18 21:34:47.217516+00', 'version': '1', 'current_rule_run_id': 72, 'status': 'new', 'owner_name': '', 'hash': 'f407a85b849baecdb34d27da1e1431dc', 'description': 'CryptoLocker has been detected on finance system 192.168.1.41 running on ESXi server 192.168.1.40', 'tags': [], 'start_time': '2014-09-04 14:40:33+00', 'asset_name': 'qradar_entr', 'artifact_update_time': '2017-03-23 03:54:15.990076+00', 'container_update_time': '', 'kill_chain': '', 'name': 'CryptoLocker Ransomware Infection (new, SLA breached) (192.168.1.41)', 'ingest_app_id': '', 'source_data_identifier': '45', 'end_time': '', 'artifact_count': 2}
Fri Mar 24 2017 07:13:08 GMT-0700 (PDT):
[
    {
        "parent_playbook_run_id": "0",
        "name": "automation_data",
        "run_id": "72",
        "scope_artifacts": [],
        "scope": "new",
        "id": "112",
        "repo_name": "local"
    }
]
Fri Mar 24 2017 07:13:08 GMT-0700 (PDT): save_object() called:key=key1,auto_delete=True,container_id=10,playbook_name=automation_data,repo_name=None
Fri Mar 24 2017 07:13:08 GMT-0700 (PDT): save_object() called:key=key2,auto_delete=True,container_id=10,playbook_name=automation_data,repo_name=None
Fri Mar 24 2017 07:13:08 GMT-0700 (PDT): save_object() called:key=key1,auto_delete=True,container_id=10,playbook_name=None,repo_name=None
Fri Mar 24 2017 07:13:08 GMT-0700 (PDT): save_object() called:key=key1,auto_delete=False,container_id=None,playbook_name=automation_data,repo_name=None
Fri Mar 24 2017 07:13:08 GMT-0700 (PDT): get_object() called:key=key1, clear_data=False,container_id=10,playbook_name=automation_data,repo_name=None
Fri Mar 24 2017 07:13:08 GMT-0700 (PDT): get by key, container_id and playbook name: 1 records found
Fri Mar 24 2017 07:13:08 GMT-0700 (PDT):
[
    {
        "composite_key": {
            "container_id": 10,
            "playbook_name": "automation_data",
            "key": "key1"
        },
        "value": {
            "value": "key 1 data for container and playbook"
        }
    }
]
Fri Mar 24 2017 07:13:08 GMT-0700 (PDT): get_object() called:key=key1, clear_data=False,container_id=10,playbook_name=None,repo_name=None
Fri Mar 24 2017 07:13:08 GMT-0700 (PDT): get by key, and container_id and NO playbook name: 1 records found
Fri Mar 24 2017 07:13:08 GMT-0700 (PDT):
[
    {
        "composite_key": {
            "container_id": 10,
            "playbook_name": "",
            "key": "key1"
        },
        "value": {
            "value": "key 1 data for only container and NO playbook"
        }
    }
]
Fri Mar 24 2017 07:13:08 GMT-0700 (PDT): get_object() called:key=key1, clear_data=False,container_id=None,playbook_name=automation_data,repo_name=None
Fri Mar 24 2017 07:13:08 GMT-0700 (PDT): get by key, and playbook name and no container id: 1 records found
Fri Mar 24 2017 07:13:08 GMT-0700 (PDT):
[
    {
        "composite_key": {
            "container_id": 0,
            "playbook_name": "automation_data",
            "key": "key1"
        },
        "value": {
            "value": "key 1 data for only playbook and not container"
        }
    }
]
Fri Mar 24 2017 07:13:08 GMT-0700 (PDT): get_object() called:key=key1, clear_data=False,container_id=None,playbook_name=%%,repo_name=None
Fri Mar 24 2017 07:13:08 GMT-0700 (PDT): get by key, and ALL playbook name and no container id: 1 records found
Fri Mar 24 2017 07:13:08 GMT-0700 (PDT):
[
    {
        "composite_key": {
            "container_id": 0,
            "playbook_name": "automation_data",
            "key": "key1"
        },
        "value": {
            "value": "key 1 data for only playbook and not container"
        }
    }
]
Fri Mar 24 2017 07:13:08 GMT-0700 (PDT): get_object() called:key=%%, clear_data=False,container_id=10,playbook_name=None,repo_name=None
Fri Mar 24 2017 07:13:08 GMT-0700 (PDT): get for ALL key, and container_id and no playbook name: 1 records found
Fri Mar 24 2017 07:13:08 GMT-0700 (PDT):
[
    {
        "composite_key": {
            "container_id": 10,
            "playbook_name": "",
            "key": "key1"
        },
        "value": {
            "value": "key 1 data for only container and NO playbook"
        }
    }
]
Fri Mar 24 2017 07:13:08 GMT-0700 (PDT): get_object() called:key=%%, clear_data=False,container_id=10,playbook_name=automation_data,repo_name=None
Fri Mar 24 2017 07:13:08 GMT-0700 (PDT): get for ALL key, and container_id and playbook name: 2 records found
Fri Mar 24 2017 07:13:08 GMT-0700 (PDT):
[
    {
        "composite_key": {
            "container_id": 10,
            "playbook_name": "automation_data",
            "key": "key2"
        },
        "value": {
            "value": "key 2 data for container and playbook"
        }
    },
    {
        "composite_key": {
            "container_id": 10,
            "playbook_name": "automation_data",
            "key": "key1"
        },
        "value": {
            "value": "key 1 data for container and playbook"
        }
    }
]
Fri Mar 24 2017 07:13:08 GMT-0700 (PDT): get_object() called:key=key1, clear_data=False,container_id=10,playbook_name=automation_data,repo_name=None
Fri Mar 24 2017 07:13:08 GMT-0700 (PDT): BEFORE clear ... get for key1, and container_id and playbook name: 1 records found
Fri Mar 24 2017 07:13:08 GMT-0700 (PDT):
[
    {
        "composite_key": {
            "container_id": 10,
            "playbook_name": "automation_data",
            "key": "key1"
        },
        "value": {
            "value": "key 1 data for container and playbook"
        }
    }
]
Fri Mar 24 2017 07:13:08 GMT-0700 (PDT): clear_object() called:key=key1,container_id=10,playbook_name=automation_data,repo_name=None
Fri Mar 24 2017 07:13:08 GMT-0700 (PDT): get_object() called:key=key1, clear_data=False,container_id=10,playbook_name=automation_data,repo_name=None
Fri Mar 24 2017 07:13:08 GMT-0700 (PDT): AFTER clear ... get for key1, and container_id and playbook name: 0 records found
Fri Mar 24 2017 07:13:08 GMT-0700 (PDT):
[]

Fri Mar 24 2017 07:13:08 GMT-0700 (PDT): No actions were executed
Fri Mar 24 2017 07:13:08 GMT-0700 (PDT):

Playbook 'automation_data' (playbook id: 112) executed (playbook run id: 72) on incident 'CryptoLocker Ransomware Infection (new, SLA breached) (192.168.1.41)'(container id: 10).
   Playbook execution status is 'success'
	Total actions executed: 0

Fri Mar 24 2017 07:13:08 GMT-0700 (PDT): on_finish() called
Fri Mar 24 2017 07:13:08 GMT-0700 (PDT): {"message":"No actions were executed","playbook_run_id":72,"result":[],"status":"success"}

If auto_delete is False, no container ID is provided, or the container isn't closed, the data isn't deleted and can waste space.

save_run_data

Use the save_run_data API to save the value in a key only in the context of the playbook run or execution. This data is automatically deleted when the playbook execution completes unless the auto parameter is set to False.

The save_run_data API is not supported from within a custom function.

phantom.save_run_data(value=None, key=None, auto=True)

set_action_limit

Use the set_action_limit API in your playbook's on_start() block to set the maximum number of action calls that can be executed. The default is 50 action calls per container per playbook. Each phantom.act() call can still result in multiple actions performed, resulting in more actions than this setting.

The set_action_limit API is not supported from within a custom function.

phantom.set_action_limit(limit)

set_list

Use the set_list API to replace the contents of a list. This API returns a tuple of a success flag and any response messages.

The set_list API is supported from within a custom function.

phantom.set_list(list_name=None, values=None)

Parameter	Required?	Type	Description
list_name	Required	String	The name of the custom list to modify.
values	Required	List of Lists	The values to set.

This sample uses the phantom.set_list() API.

import phantom.rules as phantom
import json

def on_start(container):
    #in the product in 'Playbooks / Custom Lists', define a
    # list called 'example' with two rows. Creates the list if it does not exist.
    success, message = phantom.set_list(
        list_name='example',
        values=[ ['a', 'list', 'of', 'values'], ['second', 'row'] ])
    phantom.debug(
      'phantom.set_list results: success: {}, message: {}'\
      .format(success, message)
    )
    return

def on_finish(container, summary):
    return