Splunk® Machine Learning Toolkit

ML-SPL API Guide

Package an algorithm for Splunkbase

Complete the following steps to package an algorithm for Splunkbase:

  1. Create the app in Splunkbase.
  2. Add the custom algorithm.
  3. Test the custom algorithm.

For more information on Splunkbase, see Publish apps for Splunk Cloud Platform or Splunk Enterprise to Splunkbase on the Splunk Developer Portal.

Create the app in Splunkbase

To learn how to build an app in Splunkbase, see Create a Splunk app in the Splunk Developer Portal. And before choosing a name for your app, see Naming Conventions for apps and add-ons.

You do not need to load upload assets in the app.

The following fields are required for any app being added to Splunkbase:

Required field Example input
Name application name
Folder name folder name
Template barebones

Add the custom algorithm

The process of adding a custom algorithm to an app is similar to adding an algorithm to the Splunk Machine Learning Toolkit (MLTK). Complete the following steps to add an algorithm to your Splunkbase app:

You need access to the application's file system to add a custom algorithm to the app.

  1. Name the custom algorithm.
  2. Add the implementation file.
  3. Modify the algorithm configuration file.

Name the custom algorithm

There are restrictions on algorithm names in MLTK. These namespace constraints apply to individual packaging in the application, but only affect the user of the application.

  • The algorithm name must be unique across all of MLTK and its add-ons.
  • You cannot use algos as a package_name, because algos is the default folder for MLTK.
  • Any references to algorithm source files in the register_codecs method must also reference the same package name.

Example

Following installation of the SVR_app application, there must be no other instances of SVR.py within MLTK environment. If there is more than one instance, the most recently added copy takes precedence.

Add the implementation file

The following example uses the algorithm Support Vector Regression, which is referred to as SVR:

  1. Open the directory SPLUNK_HOME/etc/apps/SVR_app/bin/
  2. Create a folder inside the bin folder of the app. For example, app_algos. The folder name must conform to the namespace constraints.
  3. Create an empty file within the folder. For example, __init__.py. This step converts the directory into a python package, and lets you import modules such as SVR.
  4. Create another empty file within that same folder. For example, SVR.py.
  5. Add the following lines of code to SVR.py:

    from sklearn.svm import SVR as _SVR

    from base import BaseAlgo, RegressorMixin

    from util.param_util import convert_params

    class SVR(RegressorMixin, BaseAlgo):

       def __init__(self, options):
           self.handle_options(options)
    
           params = options.get('params', {})
           out_params = convert_params(
               params,
               floats=['C', 'gamma'],
               strs=['kernel'],
               ints=['degree'],
           )
    
           self.estimator = _SVR(**out_params)
    
       @staticmethod
       def register_codecs():
           from codec.codecs import SimpleObjectCodec
           from codec import codecs_manager
           codecs_manager.add_codec('app_algos.SVR', 'SVR', SimpleObjectCodec)
           codecs_manager.add_codec('sklearn.svm.classes', 'SVR', SimpleObjectCodec)
    

Modify the algorithm configuration file

The following example registers the algorithm SVR and identifies the location of algorithm.py in the directory of the Splunk Machine Learning Toolkit. Complete the following steps to modify the algorithm configuration file:

  1. Add a configuration file name algos.conf to the directory SPLUNK_HOME/etc/apps/SVR_app/local/.
  2. Add the following code to the algos.conf file:
    [SVR]
    package=app_algos
    disabled=false
    

    The stanza algorithm class name, must always match the name of the algorithm.py. So, in this example [SVR] matches with the SVR.py file contained in the package SPLUNK_HOME/etc/apps/<app_name>/bin/<app_algos>/.

    For the Splunk Machine Learning Toolkit to find the algos.conf file, you must export its content system-wide.

  3. Open the SPLUNK_HOME/etc/apps/SVR_app/metadata/local.meta file and add the following code:
    [algos]
    export = system
    
    This code exports the algorithm to the system and makes the algorithms within the add-on viewable across other apps. The stanza name [algos] is not configurable. Any other name will not be recognized by the Splunk Machine Learning Toolkit.
  4. Restart Splunk Enterprise.

Test the custom algorithm

When you export algos.conf system-wide, you can then use ML-SPL commands to reference the algorithm in MLTK and any MLTK add-on.

Test in the MLTK default search application

When you create and export an algorithm, you can call it the same way you call any algorithm shipped with MLTK.

Complete the following steps to test the algorithm in the search application:

  1. Navigate to the search tab in MLTK.
  2. Input the following SPL:
    |inputlookup iris.csv | fit SVR petal_width from sepal_length
    

If your code executes without errors, then your custom algorithm is working correctly.

Test in an add-on

The process for calling an algorithm is the same when working within an add-on as in the MLTK search tab.

To test the example algorithm in the add on:

  1. Navigate to an application app_name from Splunk Enterprise home page.
  2. Enter the following SPL:
    index=_internal | head 1000 | fit SVR data_hour from cpu_seconds
    

If your code executes without errors, then your custom algorithm is working correctly.

Last modified on 05 February, 2024
Using codecs   Correlation Matrix example

This documentation applies to the following versions of Splunk® Machine Learning Toolkit: 5.1.0, 5.2.0, 5.2.1, 5.2.2, 5.3.0, 5.3.1, 5.3.3, 5.4.0, 5.4.1, 5.4.2


Was this topic useful?







You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters