Splunk® Machine Learning Toolkit

ML-SPL API Guide

Acrobat logo Download manual as PDF


This documentation does not apply to the most recent version of Splunk® Machine Learning Toolkit. For documentation on the most recent version, go to the latest release.
Acrobat logo Download topic as PDF

Package an algorithm for Splunkbase

To package an algorithm for Splunkbase, create an App, then add the custom algorithm and test it in the application. For more information on Splunkbase, see Working with Splunkbase.

Create an App in Splunkbase

To build an App in Splunkbase, see Create a Splunk App in the Splunk Developer portal. Before you choose a name for your app, see Naming Conventions for apps and add-ons.

There is a set of required fields that must be included in your app. The following table shows an example of an app with the barebones template and corresponding user input for the required fields. You do not need to load upload assets in the app.


Required Field Example User Input
Name application name
Folder name application name
Template barebones

Add the custom algorithm

The process of adding a custom algorithm to an app is similar to adding an algorithm to the Splunk Machine Learning Toolkit, see Examples. You must have access to the application's file system to add a custom algorithm to the app.

To add an algorithm to your app:

  1. Adhere to the namespace constraints.
  2. Add the implementation file.
  3. Modify the algorithm configuration file.

Namespace Constraints

There are restrictions on algorithm names in the Splunk Machine Learning Toolkit. These restrictions apply to individual packaging in the application, but only affect the user of the application.

  • The algorithm name must be unique across all of the Splunk Machine Learning Toolkit and its add-ons.
For example, after installing the SVR_app application, there should be no other instances of SVR.py within the Splunk Machine Learning Toolkit's environment. If there is more than one instance, the most recently added copy takes precedence.
  • You cannot use algos as a package_name, because algos is the default folder for the Splunk Machine Learning Toolkit.
  • Any references to algorithm source files in the register_codecs method must also reference the same package name.

Add the implementation file

The following example uses the algorithm Support Vector Regression, which is referred to as SVR.

  1. Open the directory SPLUNK_HOME/etc/apps/SVR_app/bin/
  2. Create a folder inside your app's bin folder named app_algos.
    Here, the name app_algos is arbitrary, however it must conform to the namespace constraints.
  3. Create an empty file within app_algos named __init__.py.
    This converts the directory into a python package, and lets you import modules such as SVR.
  4. Create an empty file within that same folder named SVR.py.
  5. Add the following lines of code to SVR.py:

    from sklearn.svm import SVR as _SVR

    from base import BaseAlgo, RegressorMixin

    from util.param_util import convert_params

    class SVR(RegressorMixin, BaseAlgo):

       def __init__(self, options):
           self.handle_options(options)
    
           params = options.get('params', {})
           out_params = convert_params(
               params,
               floats=['C', 'gamma'],
               strs=['kernel'],
               ints=['degree'],
           )
    
           self.estimator = _SVR(**out_params)
    
       @staticmethod
       def register_codecs():
           from codec.codecs import SimpleObjectCodec
           from codec import codecs_manager
           codecs_manager.add_codec('app_algos.SVR', 'SVR', SimpleObjectCodec)
           codecs_manager.add_codec('sklearn.svm.classes', 'SVR', SimpleObjectCodec)
    

For a detailed look at how the above code works, see Support Vector Regressor Example.

Modify the algorithm configuration file

The code example below registers the algorithm SVR and identifies the location of algorithm.py in the directory of the Splunk Machine Learning Toolkit. To modify the algorithm configuration file:

  1. Add a configuration file name algos.conf to the directory SPLUNK_HOME/etc/apps/SVR_app/local/.
  2. Add the following code to the algos.conf file:
    [SVR]
    package=app_algos
    disabled=false
    

    The stanza algorithm class name, must always match the name of the algorithm.py. So, in this example [SVR] matches with the SVR.py file contained in the package SPLUNK_HOME/etc/apps/<app_name>/bin/<app_algos>/.

    In order for Splunk Machine Learning Toolkit to find the algos.conf file, you must export its content system-wide.
  3. Open the SPLUNK_HOME/etc/apps/SVR_app/metadata/local.meta file and add the following code:
    [algos]
    export = system
    
    This code exports the algorithm to the system and makes the algorithms within the add-on viewable across other apps such as the Splunk Machine Learning Toolkit. The stanza name [algos] is not configurable. Any other name will not be recognized by the Splunk Machine Learning Toolkit.
  4. Restart Splunk Enterprise.

Test the packaged algorithm

When you export algos.conf system-wide, any Splunk Machine Learning Toolkit add-on, and Splunk Machine Learning Toolkit itself, can reference the algorithms contained in your application. Then you can use ML-SPL commands to reference the algorithm within any Splunk Machine Learning Toolkit add-on, and in the Splunk Machine Learning Toolkit.

Test in the MLTK default search application

When you create and export an algorithm, you can call it the same way you call an algorithm shipped with Splunk Machine Learning Toolkit.

To test the algorithm in the default search application:

  1. Navigate to the search bar in the Splunk Machine Learning Toolkit.
  2. Enter the following SPL:
    |inputlookup iris.csv | fit SVR petal_width from sepal_length
    

If your code executes without errors, then your algorithm application is correct.

Test in the add-on

The process for calling an algorithm is the same when working within the add-on as in the MLTK default search application.

To test the example algorithm in the add on:

  1. Navigate to your application app_name from Splunk Enterprise home page.
  2. Enter the following SPL:
    index=_internal | head 1000 | fit SVR data_hour from cpu_seconds
    

If your code executes without errors, then your algorithm application is correct.

Last modified on 03 July, 2019
PREVIOUS
Saving models
  NEXT
Correlation Matrix

This documentation applies to the following versions of Splunk® Machine Learning Toolkit: 2.4.0, 3.0.0, 3.1.0, 3.2.0, 3.3.0, 3.4.0, 4.0.0, 4.1.0, 4.2.0, 4.3.0


Was this documentation topic helpful?


You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters