Package an algorithm for Splunkbase

To package an algorithm for Splunkbase, create an App, then add the custom algorithm and test it in the application. For more information on Splunkbase, see Working with Splunkbase.

Create an App in Splunkbase

To build an App in Splunkbase, see Create a Splunk App in the Splunk Developer portal. Before you choose a name for your app, see Naming Conventions for apps and add-ons.

There is a set of required fields that must be included in your app. The following table shows an example of an app with the barebones template and corresponding user input for the required fields. You do not need to load upload assets in the app.

Required Field	Example User Input
Name	application name
Folder name	application name
Template	barebones

Add the custom algorithm

The process of adding a custom algorithm to an app is similar to adding an algorithm to the Splunk Machine Learning Toolkit, see Examples. You must have access to the application's file system to add a custom algorithm to the app.

To add an algorithm to your app:

1. Adhere to the namespace constraints.
2. Add the implementation file.
3. Modify the algorithm configuration file.

Namespace Constraints

There are restrictions on algorithm names in the Splunk Machine Learning Toolkit. These restrictions apply to individual packaging in the application, but only affect the user of the application.

• The algorithm name must be unique across all of the Splunk Machine Learning Toolkit and its add-ons.

For example, after installing the SVR_app application, there should be no other instances of SVR.py within the Splunk Machine Learning Toolkit's environment. If there is more than one instance, the most recently added copy takes precedence.

• You cannot use algos as a package_name, because algos is the default folder for the Splunk Machine Learning Toolkit.
• Any references to algorithm source files in the register_codecs method must also reference the same package name.

Add the implementation file

The following example uses the algorithm Support Vector Regression, which is referred to as SVR.

1. Open the directory SPLUNK_HOME/etc/apps/SVR_app/bin/
2. Create a folder inside your app's bin folder named app_algos.
   Here, the name app_algos is arbitrary, however it must conform to the namespace constraints.
3. Create an empty file within app_algos named __init__.py.
   This converts the directory into a python package, and lets you import modules such as SVR.
4. Create an empty file within that same folder named SVR.py.
5. Add the following lines of code to SVR.py:

   from sklearn.svm import SVR as _SVR

   from base import BaseAlgo, RegressorMixin

   from util.param_util import convert_params

   class SVR(RegressorMixin, BaseAlgo):

      def __init__(self, options):
          self.handle_options(options)
   

          params = options.get('params', {})
          out_params = convert_params(
              params,
              floats=['C', 'gamma'],
              strs=['kernel'],
              ints=['degree'],
          )
   

          self.estimator = _SVR(**out_params)
   

      @staticmethod
      def register_codecs():
          from codec.codecs import SimpleObjectCodec
          from codec import codecs_manager
          codecs_manager.add_codec('app_algos.SVR', 'SVR', SimpleObjectCodec)
          codecs_manager.add_codec('sklearn.svm.classes', 'SVR', SimpleObjectCodec)
   

For a detailed look at how the above code works, see Support Vector Regressor Example.

Modify the algorithm configuration file

The code example below registers the algorithm SVR and identifies the location of algorithm.py in the directory of the Splunk Machine Learning Toolkit. To modify the algorithm configuration file:

1. Add a configuration file name algos.conf to the directory SPLUNK_HOME/etc/apps/SVR_app/local/.
2. Add the following code to the algos.conf file:

   [SVR]
   package=app_algos
   disabled=false
   

   The stanza algorithm class name, must always match the name of the algorithm.py. So, in this example [SVR] matches with the SVR.py file contained in the package SPLUNK_HOME/etc/apps/<app_name>/bin/<app_algos>/.

   In order for Splunk Machine Learning Toolkit to find the algos.conf file, you must export its content system-wide.
3. Open the SPLUNK_HOME/etc/apps/SVR_app/metadata/local.meta file and add the following code:

   [algos]
   export = system
   

   This code exports the algorithm to the system and makes the algorithms within the add-on viewable across other apps such as the Splunk Machine Learning Toolkit. The stanza name [algos] is not configurable. Any other name will not be recognized by the Splunk Machine Learning Toolkit.
4. Restart Splunk Enterprise.

Test the packaged algorithm

When you export algos.conf system-wide, any Splunk Machine Learning Toolkit add-on, and Splunk Machine Learning Toolkit itself, can reference the algorithms contained in your application. Then you can use ML-SPL commands to reference the algorithm within any Splunk Machine Learning Toolkit add-on, and in the Splunk Machine Learning Toolkit.

Test in the MLTK default search application

When you create and export an algorithm, you can call it the same way you call an algorithm shipped with Splunk Machine Learning Toolkit.

To test the algorithm in the default search application:

1. Navigate to the search bar in the Splunk Machine Learning Toolkit.
2. Enter the following SPL:

   |inputlookup iris.csv | fit SVR petal_width from sepal_length
   

If your code executes without errors, then your algorithm application is correct.

Test in the add-on

The process for calling an algorithm is the same when working within the add-on as in the MLTK default search application.

To test the example algorithm in the add on:

1. Navigate to your application app_name from Splunk Enterprise home page.
2. Enter the following SPL:

   index=_internal | head 1000 | fit SVR data_hour from cpu_seconds
   

If your code executes without errors, then your algorithm application is correct.