Phantom app metadata

Phantom apps use a JSON schema to define the configuration of an app. This JSON can vary in complexity, depending on the sophistication of the app and the author's desire to customize how data produced by their app is rendered.

Top level definition

The top level keys define the primary facets of the app:

{
  "appid" : "fd4dc1ce-b63c-4d0e-94e6-df0ae680d99c",
  "name" : "SampleApp",
  "description" : "Query the Sample reputation service",
  "type" : "reputation",
  "main_module" : "sampleapp_connector.py",
  "app_version" : "1.0.0",
  "product_vendor" : "Contoso",
  "product_name" : "Contoso Reputation",
  "product_version_regex": ".*",
  "logo": "contoso.svg",
  "logo_dark": "contoso_dark.svg",
  "min_phantom_version": "1.0.240",
  "publisher" : "Contoso Corp",
  "package_name": "phantom_whois",
  "license": "Copyright (c) 2019 Splunk, Inc.",
  "configuration": {},
  "actions": []
}

Key	Required?	Description
appid	Required	This is the ID of the app for unique identification, in GUID/UUID version 4 format. It can be generated by applications for the required platform. uuidgen for linux, guidgen.exe on windows, or through www.guidgenerator.com. As of this writing, the uuid command on linux will not work, you will need to use the -v4 command line switch to generate a version 4 UUID. Another quick handy command is: python -c "import uuid;print str(uuid.uuid4())"
name	Required	This is the name of the app. App names should be as short as possible (1-2 words).
description	Required	This is a more verbose description of the app. Use this when displaying a description of the app in the user interface, and also for automatically generated documentation on an app.
type	Required	This is the app type. The type is used to group apps at a level above the product_vendor name. Some of the types are currently defined by Splunk Phantom. Apps include SIEM, email, endpoint, firewall, forensic, information, investigative, network access control, reputation, sandbox, threat intel, ticketing, virtualization and generic. Please let us know if you intend to create a new app type, so we can better maintain this list.
main_module	Required	This is the file name containing the module with the entry point code. It usually (also) contains the action implementations for this app. This file is loaded when an action that this app supports is executed. The Python code should be located in the current app's directory.
app_version	Required	This is the app version. There may be multiple versions of an app on a given system in order to support different versions of a vendor/product combination. Versions should start at 1.0.
pip_dependencies	Optional	The platform allows the app author to specify pip dependencies that need to be installed on the OVA for the app to properly work. Please see the Specifying pip dependencies section for more information.
pip3_dependencies	Only required for apps that need to be compatible with both python2 and python3.	Use this key for specifying dependencies that you want installed when the app is running in python3. If this key is present, pip_dependencies will be ignored if the app is running in python3. If this key is not present, but pip_dependencies is, that information will be used for both python2 and python3.
min_phantom_version	Optional	Use this key to specify the minimum Splunk Phantom version that the app supports. The platform will validate the Phantom version during app installation and fail if the minimum criteria is not met. To get a list of released Phantom versions head to my.phantom.us
product_vendor	Required	This is the vendor associated with the product that this app supports. For example, ReversingLabs, Cisco, IBM, FireEye.
product_name	Required	This is the name of the product that this app supports.
product_version_regex	Required	This is the version of the product that this app supports integration with. It is a regex and is rendered by the platform in the app documentation. The app will require to code extracting the version from the 3rd party device or service that it integrates with and perform any required validation. A '.*' represents all versions.
publisher	Required	This is your name, as the publisher and creator of this app. This is your company name if you are a vendor, or your own name if you are an individual.
package_name	Required	This is the package name that will be used to install the app via the yum/rpm commands.
license	Required	This is the license that the app is released under. This should be the license string that is to be displayed in the yum/rpm info commands output.
consolidate_widgets	Optional	If false, the platform will render every action in its own widget, or else a single widget will contain all actions consolidated.
configuration	Required	A set of asset configuration variables, defined in more detail below. (Can be empty if none are needed).
url	Optional	This is the URL associated with the app. It should be a landing page that gives brief information about the app or extra documentation.
logo	Optional	This is the name of the icon file that is rendered at multiple places in the product in Light Theme. It has to be in the app folder with the rest of the files. SVG, PNG and JPEG files are supported, but SVGs are preferred. It needs to have a transparent background and should represent the product being integrated. A product icon is preferred over the vendor icon.
logo_dark	Optional	This is the icon that is rendered in Dark Theme.
actions	Required	Detailed specification of the actions that this app supports. This defines the input parameters that the app needs for each action and the output that they generate.
python_version	Optional when using python2.7, but required when using python3.	Defines the Python version used to run the app's actions. This value is either "2.7" or "3". If not specified, "2.7" is assumed.

Specifying pip dependencies

The app author can specify the Python modules that the platform should install during app installation. The format of this dictionary is:

"pip_dependencies": {
   "pypi": [
      {"module": "requests_ntlm"}
   ],
   "wheel": [
      {"module": "python-rtkit", "input_file": "wheels/python_rtkit-0.7.0-py27-none-any.whl"}
   ]
},
 

The pip_dependencies dictionary supports two keys but only one key is required:

• pypi
  A list of modules that need to be installed for the app to work properly. Every item in the list is a dictionary with a single key called module, the value of which is the module name to use as a parameter to the pip2.7 install [module_name] command
• wheel
  Use this key to install the list of modules, for which the app supplies the input wheel files to install from. Every item in the list is a dictionary with two keys:
  • module
    The module name
  • input_file
    Path to the wheel file relative to the app directory in the app tarball.

Wheel installations are the preferred method to install dependencies because they allow app dependencies to be installed when the Splunk Phantom instance does not have direct connectivity to the internet. The platform will execute pip with the --no-deps switch during wheel file installations, so all required dependencies must be included with the app and properly configured in the app JSON.

The platform will install all the modules in the app's install directory under the dependencies subfolder (using the --target command line switch of pip2.7). This means that the app will get its own version and copy of the installed Python module. The platform while executing every action, adds the /opt/phantom/apps/[app_directory]/dependencies sub folder as part of the PYTHONPATH. The pip2.7 install command will continue to install all the dependencies of the module being installed in the same dependencies folder.

The best way to figure out the complete dependency list is to pick a module that is required by the app and download the files in a directory.

In your app folder, you may need to create a wheels subfolder. Once it is created, download the required dependencies.

[phantom@phantom sampleapp]$ phenv pip2.7 download --dest ./wheels python-rtkit
Collecting python-rtkit
  Downloading python_rtkit-0.7.2-py2-none-any.whl
  Saved ./wheels/python_rtkit-0.7.2-py2-none-any.whl
Successfully downloaded python-rtkit
[phantom@phantom wheels]$ ls
python_rtkit-0.7.2-py2-none-any.whl

The download command downloads all the wheel files (including the dependencies) required for the Python module(s) to function properly. Next edit the app JSON to include these wheel files.

...
...
    "pip_dependencies": {
      "wheel": [
        {"module": "python-rtkit", "input_file": "wheels/python_rtkit-0.7.2-py2-none-any.whl"}
        ...
      ]
    },
...
...

Now compile and install the app

[phantom@phantom sampleapp]$ phenv python2.7 ../compile_app.pyc -i
cd'ing into ./
Validating app Json
  Working on: ./sampleapp.json
    Looks like an app json
  Processing app Json
  Processing actions
  ...
Installing app...
  Creating tarball...
  ../sampleapp.tgz
  Calling installer...
  Success
Done
[phantom@phantom sampleapp]$

If you now check the contents of the folder, you will notice that the dependencies folder has been created.

[phantom@phantom sampleapp]$
[phantom@phantom sampleapp]$ ls /opt/phantom/apps/sampleapp_UUID
dependencies  __init__.py  __init__.pyc  readme.html  sampleapp_connector.py  sampleapp_connector.pyc  sampleapp_consts.py  sampleapp_consts.pyc  sampleapp.json  sample_logo.svg sample_logo_dark.svg wheels

Modules that require compilation

Some modules when downloaded, require compilation before they can be used by a Splunk Phantom app. In these cases when the module is downloaded, it is a zip file that contains source files that need to be compiled before a self contained wheel file can be used. One way to do this is to create a wheel file using pip2.7. Do note that since the module will require to be compiled in order to create a wheel file, things like gcc, g++ etc. will require to be installed on the OVA that is creating the wheel file. Once the wheel file is created it can be packaged with the app and installed without any other compilers.

Create a requirements.txt file that contains the list of modules. One module per line. Then run the following command:

...]$ phenv pip2.7 wheel --wheel-dir=./wheels/ -r ./requirements.txt

This will download and compile the module if required and create a wheel file that can be packaged with the app. Please make sure that the module that you are packaging with the app has a license that allows you to do so.

Configuration Section

In order to execute an action, an app must operate on an asset that has been configured by the end user within the Phantom platform. First, the platform must have one or more instances of an asset configured; at least one that is directly supported by the app, and that match the vendor and product that the app supports (as specified in the JSON header above).

The configuration section contains variables that are used when configuring an asset, essentially serving as a template for that. As a result, the configuration of an asset is data-driven based on the variables defined here. Each asset vendor/product combination will typically have a single app that operates on it. For example, there would be a single app that supports and operates on Cisco ASA devices, and the configuration variables required by the app to connect to that device are defined here. It should be noted that these are static configuration variables for an asset that serve as a template when configuring the asset under the Administration menu, such as a username, password, or API key as shown in the example here. Dynamic parameters that change each time an app executes an action are defined in the "parameters" section of the corresponding action and are not "configuration" variables.

{
  ...
  "configuration": {
    "apikey" : {
      "data_type": "string",
      "description": "Contoso API key",
      "required": true
    }
  },

The configuration section contains a list of variables (keys) that each contain a dictionary to describe that key. This defines how that variable should be rendered and used within the system:

Key	Required?	Description
data_type	Required	The type of variable. Supported types are: string, password, numeric, boolean.
description	Required	A description of this variable. This value is used as a label that is displayed on the UI next to the input control. It should be very brief, prefrably a single line, since it is displayed on the UI. If you need to explain things in more detail regarding asset configuration or the app in general use a readme.html file.
required	Optional	Whether the variable is mandatory. Either true or false If mandatory, the variable MUST be configured by the user when defining an asset that this variable operates on. Default is false
value_list	Optional	To allow the user to choose from a pre-defined list of values displayed in a drop-down, specify them as a list for e.g. ["one", "two", "three"]
default	Optional	To set the default value of a variable in the UI, use this key. The user will be able to modify this value, so the app will need to validate it nonetheless. This key also works in conjunction with a value_list.
order	Optional	It's preferable to display a config parameter like Username before something like Password, the order key (starting at 0) allows the app author to control the diplay order of the controls in the UI.

Not all apps require a configuration section. For example, Splunk Phantom's MaxMind app has no need for one, since there is nothing to configure. It can be left empty:

"configuration": {},

During app updates, configuration requirements might change. If the new app version adds a new parameter that is required, the currently configured asset will become invalid and user intervention is required for the app to function. For a smoother upgrade process, the app author should try to set the new asset configuration values as optional and with a default value instead of adding a required. The default value should be specified in the app json and also handled in the app code.

README file

An app author can bundle a readme.html file in the app directory, which the platform renders as part of the app documentation. It is rendered between the app description and the asset configuration parameters.

Place Holder Data Type

As mentioned above the order key allows an app author to specify the order in which the controls are displayed to the user. In order to insert a blank (no control) at a specific location, one can use the ph data_type with the order key. For e.g. To display a blank space between the 1st and 2nd control define the configuration as follows

{
  ...
    "configuration": {
      "server": {
        "data_type": "string",
        "description": "Server IP/Hostname",
        "order": 0,
        "required": true
      },
      "ph": {
        "data_type": "ph",
        "order": 1
      },
      "username": {
        "data_type": "string",
        "description": "Username",
        "order": 2,
        "required": false
      },
      "password": {
        "data_type": "password",
        "description": "Password",
        "order": 3,
        "required": false
      },

This data type is supported for action parameters also.

Actions Section

The actions key defines an array of actions that this app supports. This exposes the core functionality that the app makes available to the Phantom platform. It is the most complex (but powerful) part of the JSON. The actions section is required, as there is no sense in having an app that has no actions.

{
  ...
  actions": [
    {
      "action": "file reputation",
      "description": "Queries Contoso for file info",
      "type": "investigate",
      "read_only": true,

Parameter	Required?	Description
action	Required	The name of the action. Action names are high level primitives that are used throughout the Phantom platform. They are used in playbooks when an action is requested and in the UI when an action is executed. It is very important to name an action appropriately. Please read the section Naming actions for more info.
identifier	Required	A unique identifier for this action. The action key above is mapped to this identifier and that is what is passed down to the app. The app code then uses this unique identifier to determine which action to execute. This additional level of abstraction allows us to easily change the action key without changing code. identifier must be longer than 5 characters.
description	Required	A description of this variable. This value is used as a label that is displayed on the UI next to the input control. It should be very brief, prefrably a single line, since it is displayed on the UI. If you need to explain things in more detail regarding a parameter or the action in general, use the verbose key.
verbose	Optional	Use this key to fill in any information that you wish to document about the action. This text supports html formatting. It's a good place to explain certain caveats about the action or any specific values of any parameter. Since it supports html formatting it's considered good practice to mention action or parameter names in bold formatting.
type	Required	The type of action. This is used to organize actions into logical groups based on their purpose. Type must be one of the following five: contain, correct, generic, investigate and test. Note that only test connectivity action uses the test type. The test type of actions are not displayed in the UI in the Run Action dialog.
read_only	Required	Indicates whether the action is "read-only", or in other words, non-destructive. Executing this action will have no adverse affect on the asset. This typically applies to information gathering actions that do not make any configuration changes to an asset.
undo	Optional	The action that reverts or undoes this action. This is typically an action that will take the exact same parameters as this action.

Naming Actions

Phantom users are used to a particular naming convention for actions names. In order to get them to understand your app quickly it helps to re-use action names. There may be multiple apps that each expose the same action. For example, there may be a "block ip" action exposed by a Cisco ASA app, a Palo Alto Networks app, and a Microsoft Windows app. It is therefore important to have a good understanding of other action names within the Splunk Phantom platform when writing your app in order to match them to an existing name that has the same purpose. This is to your advantage since your app will benefit from Playbooks that already execute this action, if one exists. Action names should be as short as possible while still conveying their intent. They may contain spaces. They should follow the verb object naming convention. For e.g. list processes or detonate file. In case the action is gathering infomation, then this rule can be flipped. Case in point: file reputation is preferred to get file reputation. Another e.g. while implmenting a search action, it's best to name the action as run query, since phantom users are already aware of this action and therefore will quickly know what the action does when listed in your app. Every app's documentation lists the actions names, so it's very easy to study the action names on the prodcut. Also the Run Action dialog box lists the names segregated by their type.

Action Section: Versions

The versions key specifies which versions of the product that this action supports (product_name). It contains a regular expression that is matched against a configured asset in order to find the app and action within that app that best supports a specific asset.

"versions": "EQ(*)"

The versions key supports a short number of matching functions which can be one of the following:

Function	Description
EQ()	Equal - the version matches a specific version: EQ(*), or EQ(12.*)
NEQ()	Not Equal - the version does not match a specific version: NEQ(12.*)
IN()	In - The version matches a list of comma separated versions: IN(12.*, 13.*)
NIN()	Not In - The version is not in a list of comma separated versions: NIN(12.*, 13.*)

Version matching is used to find the BEST app to run on a given asset, while allowing app authors to maintain support for various versions of a vendor's product. It's quite possible that a product's command syntax changes, a protocol changes, or features change or added as products evolve, requiring the need to be as granular as possible when creating apps. When making configuration changes it is especially important to ensure that we don't mistakenly execute the wrong command on a given asset.

There are two ways to handle multiple versions of a product. First, you can simply create a new action within your JSON which has a different version expression and calls a different function within your app. In this case, your app code has two different functions, each one supporting a different version or versions of a product.

Second option, you can create a separate JSON which contains the required meta-data for the version that you would like to support. In this case, you can have a separate app altogether, with its own appid; one that supports only the versions specified in this separate JSON.

Action Section: Parameters

Within an action entry, the parameters key defines a list of run time parameters that are passed to this action. These parameters are typically specified in Phantom automation playbooks and pass through to the app when an action is executed. The name of the key represents the parameter name. For e.g. in the below example hash is the name of the parameter. Parameter naming is as important as action name. Each action and it's parameters are documented on the platform, so please take a look at the names used by action parameters of the same action in other apps. For e.g. an action like list processes should take as input an ip address or hostname named ip_hostname, another E.g. domain reputation should take a parameter named domain

"parameters": {
   "hash": {
      "description": "the hash of the file to be queried",
      "data_type": "string",
      "contains": ["hash", "sha256", "sha1", "md5"],
      "required": true,
      "allow_list": true
   }
},

Parameter	Required?	Description
description	Required	A short description of this parameter. This description is displayed to the user in the user interface when running an action manually.
data_type	Required	The type of variable. Supported types are: string, password, numeric, boolean.
contains	Optional	Specifies what kind of content this field contains. The purpose of contains is to allow matching of output from one action to another action's parameters. This allows the platform to more easily chain together a series of actions where one action's results are directly consumed by another action. One place that this is used is in the visual playbook editor to allow blocks to provide a list of input parameters based on the output from the previous blocks that they are attached to. For example, app A which produces a "hash" matches nicely with app C which consumes "hash" as an input parameter. contains is a very powerful feature of Phantom that app authors should not ignore. Please see the contains section of this guide for more information.
required	Optional	Whether this parameter is mandatory for this action to function. If this parameter is not provided, the action will not succeed.
primary	Optional	Specifies if the action acts primarily on this parameter. It is used in conjunction with the contains field to display a list of contextual actions when the user clicks on a piece of data in the UI. Please see the contains section of this guide for more information.
value_list	Optional	To allow the user to choose from a pre-defined list of values displayed in a drop-down for this parameter, specify them as a list for e.g. ["one", "two", "three"]. Do note that an action can be executed from the playbook, in which case the user can pass an arbitary value for the parameter, so the app needs to validate this parameter on its own.
default	Optional	To set the default value of a variable in the UI, use this key. The user will be able to modify this value, so the app will need to validate it nonetheless. This key also works in conjunction with a value_list.
order	Optional	It's preferable to display an action parameter like Name before something like Type, the order key (starting at 0) allows the app author to control the diplay order of the controls in the UI.
allow_list	Optional	Use this key to specify if the parameter supports specifying multiple values as a comma separated string.

Even before the action is passed onto the app, it reaches the BaseConnector first. The BaseConnector parses the app json and matches it against the config and action parameters that was passed from the Phantom Core. In case it finds that any of the parameters marked as required are missing, it reports an error back to the Phantom Core. This allows the app writer to write code without checking for the presence of a required param.

Action Section: Synchronization

Within an action entry, the optional lock key defines a set of parameters that app authors can set to synchronize actions. A lock is represented by it's name. Multiple actions locking on the same name will be serialized even if the actions are from different apps. In the absence of a lock dictionary, the platform will synchronize the action using the asset as the lock name. So to disable synchronization for an action, the lock dictionary will require to be present and the 'enabled' key set to false.

"lock": {
   "enabled": true,
   "data_path": "parameters.hash",
   "timeout": 600
}

Parameter	Required?	Description
enabled	Required	Boolean value that specifies if the lock is enabled or not for this action.
data_path	Optional	The name of the lock. Only valid if lock is enabled. This value can be one of three things. Either a data path that points to a parameter of the action for e.g. "parameters.hash" where 'hash' is one of the parameters of the action, or a data path that points to a config parameter for e.g. "configuration.server". At runtime the platform will read the value stored in these data paths and use it as a name of the lock. The third option that the app author can use is a constant string. I.e. any string that does not start with "configuration." or "parameters.". The platform will use this value as is. In case the data_path is not specified, the asset will be used as the lock name.
timeout	Optional	Specifies the number of seconds to wait to acquire the lock, before an error condition is reported.

Action Section: Render key

Within an action entry, the render key specifies how output from this app will be displayed to the user within the Phantom product. This section is optional. Output is typically displayed in two places. First, in the results from an action when viewing a specific action, and second in the Mission Control panel of the product. In Mission Control, the output is encapsulated within a widget container that can be displayed, expanded, and moved by the user when managing security operations.

The value associated with this key is a dictionary which must contain a key called type.

"render": {
   "type": "custom",
   "width": 4,
   "height": 5,
   "view": "sampleapp_view.file_reputation",
   "title": "FILE REPUTATION",
   "menu_name": "SampleApp File Lookup"
}

Parameter	Required?	Description
type	Required	This specifies the type of render function to use when displaying output from this app. Phantom supports a number of built-in widgets that display data and also provides you the ability to define your own view to render data from your app in any form that you wish. The built-in render options include: table and map. Please see the documentation of the table and map views for more info. If you would like to create your own view, you can do so by specifying the view in the view variable below and choosing a type of custom.
width	Required	Specifies the width (in columns) of the container that houses this content. The Mission Control screen is 12 columns wide and widgets can be anywhere from 1 to 6 columns in width.
height	Required	Specifies the height (in rows) of the container that houses this content. The height must be between 0 and 10.
view	Optional	Specifies the custom view to load when rendering data produced by this app. The format of this value is file.function, referring to the Django Python view file that you have authored, and the function for Phantom to call when rendering your view.
title	Required	Specifies the title that is displayed within the widget header on the Mission Control screen.
menu_name	Optional	Specifies the name to be displayed in the widget menu on the Mission Control screen when listing this widget.

Action Section: Output

Within an action entry, the output key defines an array of results that are generated by this app. Each member of the array is one variable that matches the corresponding variable in the CommandResult class that the Python app code produces.

"output":[
   {
      "data_path": "action_result.data.*.permalink",
      "data_type": "string"
   },
   {
   "data_path": "action_result.data.*.sha1",
   "data_type": "string",
   "contains": ["sha1"],
   "example_values": [
      "f0eb87677a689a88b58a0613dfd7252fb850393c",
      "d969e7e0b0571370cd6763192bc24ac56c255472"
      ]
   },

Parameter	Required?	Description
data_path	Optional	Specifies the data path of this field. Data path's are a method of indexing into the JSON in an abstract fashion that allows others who want to access that data to do so by specifying the appropriate path. The data path specified here must be populated by the app code that creates and returns this field.
data_type	Required	The type of variable. Supported types are: string, password, numeric, boolean.
contains	Optional	Specifies what kind of content this field contains. The purpose of contains is to allow matching of output from one action to another action's parameters. This allows the platform to more easily chain together a series of actions where one action's results are directly consumed by another action. One place that this is used is in the visual block-based rule creation process to allow blocks to provide a list of input parameters based on the output from the parent block that they are attached to. For example, app A which produces a "hash" matches nicely with app C which consumes "hash" as an input parameter. contains is a very powerful feature of Phantom that app authors should not ignore. Please see the contains section of this guide for more information.
column_name	Optional	If the output from this app is to be rendered in the default table widget (when render "type" is table), then column_name specifies the name of this column within the rendered widget when it is displayed. This provides a convenient way to show data in Mission Control and when viewer action results in tabular form.
column_order	Optional	If the output from this app is to be rendered in the default table widget (when render "type" is table), then column_order specifies the order of this column. Column ordering starts at 0.
example_values	Optional	A list of example values for the data path. These will be displayed in the app's documentation page.
map_info	Optional	If the output from this app is to be rendered in the default map widget (when render "type" is map), then map_info specifies the name of this field within the details for this item.