Splunk® Machine Learning Toolkit

ML-SPL API Guide

Acrobat logo Download manual as PDF


This documentation does not apply to the most recent version of Splunk® Machine Learning Toolkit. For documentation on the most recent version, go to the latest release.
Acrobat logo Download topic as PDF

Savitzky-Golay Filter example

This example adds SciPy's implementation of a Savitzky-Golay signal processing filter to the Splunk Machine Learning Toolkit. This Savitzky-Golay Filter example covers the following tasks:

  • Using the BaseAlgo class
  • Converting parameters
  • Using the prepare_features utility
  • Using an arbitrary function to transform data

Since SciPy's savgol_filter is only a function, work is performed using the fit method which returns the transformed values. See the SciPy documentation for details on the filter.

Steps

Follow these steps to add the Savitzky-Golay Filter algorithm.

  1. Register the algorithm in algos.conf using one of the following methods.
    1. Register the algorithm using the REST API:
      $ curl -k -u admin:<admin pass> https://localhost:8089/servicesNS/nobody/Splunk_ML_Toolkit/configs/conf-algos -d name="SavgolFilter"
      
    2. Register the algorithm manually:
      Modify or create the algos.conf file located in $SPLUNK_HOME/etc/apps/Splunk_ML_Toolkit/local/ and add the following stanza to register your algorithm
       [SavgolFilter]
      

      When you register the algorithm with this method, you will need to restart Splunk.

  2. Create the python file in the algos folder. For this example, we create $SPLUNK_HOME/etc/apps/Splunk_ML_Toolkit/bin/algos/SavgolFilter.py.
    import numpy as np
    from scipy.signal import savgol_filter
     
    from base import BaseAlgo
    from util.param_util import convert_params
    from util import df_util
    
  3. Define the class.
    class SavgolFilter(BaseAlgo):
        """Use SciPy's savgol_filter to run a filter over fields."""
    
  4. Define the __init__ method.
    Since there isn't an estimator class like other examples, attach the params to the object (self) to use later.
        def __init__(self, options):
            # set parameters
            params = options.get('params', {})
            out_params = convert_params(
                params,
                ints=['window_length', 'polyorder', 'deriv']
            )
    
            # set defaults for parameters
            if 'window_length' in out_params:
                self.window_length = out_params['window_length']
            else:
                self.window_length = 5
    
            if 'polyorder' in out_params:
                self.polyorder = out_params['polyorder']
            else:
                self.polyorder = 2
    
            if 'deriv' in out_params:
                self.deriv = out_params['deriv']
            else:
                self.deriv = 0
    
  5. Define the fit method.
       def fit(self, df, options):
            X = df.copy()
            X, nans, columns = df_util.prepare_features(X, self.feature_variables)
     
    	# Define a wrapper function
            def f(x):
                return savgol_filter(x, self.window_length, self.polyorder, self.deriv)
     
    	# Apply that function along each column of X
            y_hat = np.apply_along_axis(f, 0, X)
     
            names = ['SG_%s' % col for col in columns]
            output_df = df_util.create_output_dataframe(y_hat, nans, names)
            df = df_util.merge_predictions(df, output_df)
     
            return df
    

    Finished example

    import numpy as np
    from scipy.signal import savgol_filter
    
    from base import BaseAlgo
    from util.param_util import convert_params
    from util import df_util
    
    
    class SavgolFilter(BaseAlgo):
    
        def __init__(self, options):
            # set parameters
            params = options.get('params', {})
            out_params = convert_params(
                params,
                ints=['window_length', 'polyorder', 'deriv']
            )
    
            # set defaults for parameters
            if 'window_length' in out_params:
                self.window_length = out_params['window_length']
            else:
                self.window_length = 5
    
            if 'polyorder' in out_params:
                self.polyorder = out_params['polyorder']
            else:
                self.polyorder = 2
    
            if 'deriv' in out_params:
                self.deriv = out_params['deriv']
            else:
                self.deriv = 0
    
        def fit(self, df, options):
            X = df.copy()
            X, nans, columns = df_util.prepare_features(X, self.feature_variables)
    
            def f(x):
                return savgol_filter(x, self.window_length, self.polyorder, self.deriv)
    
            y_hat = np.apply_along_axis(f, 0, X)
    
            names = ['SG_%s' % col for col in columns]
            output_df = df_util.create_output_dataframe(y_hat, nans, names)
            df = df_util.merge_predictions(df, output_df)
    
            return df
    
Last modified on 13 February, 2024
PREVIOUS
Support Vector Regressor example
  NEXT
Custom algorithms and PSC libraries version dependencies

This documentation applies to the following versions of Splunk® Machine Learning Toolkit: 4.4.0, 4.4.1, 4.4.2, 4.5.0, 5.0.0


Was this documentation topic helpful?


You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters