Savitzky-Golay Filter example
You can add custom algorithms to the MLTK app. This example adds a Savitzky-Golay signal processing from SciPy.
Because the SciPy savgol_filter
is only a function, work is performed using the fit method which returns the transformed values.
For more information see the SciPy documentation: https://docs.scipy.org/doc/scipy/
Add the Savitzky-Golay Filter algorithm
Follow these steps to add the Savitzky-Golay Filter algorithm.
- Register the Savitzky-Golay Filter algorithm.
- Create the Python file.
- Define the class.
- Define the init method.
- Define the fit method.
Register the Savitzky-Golay Filter algorithm
Register the algorithm in algos.conf
using one of the following methods.
Register the algorithm using the REST API
Use the following curl command to register using the REST API:
$ curl -k -u admin:<admin pass> https://localhost:8089/servicesNS/nobody/Splunk_ML_Toolkit/configs/conf-algos -d name="SavgolFilter"
Register the algorithm manually
Modify or create the algos.conf
file located in $SPLUNK_HOME/etc/apps/Splunk_ML_Toolkit/local/
and add the following stanza to register your algorithm:
[SavgolFilter]
When you register the algorithm with this method, you must restart Splunk Enterprise.
Create the Python file
Create the Python file in the algos
folder. For this example, you create $SPLUNK_HOME/etc/apps/Splunk_ML_Toolkit/bin/algos/SavgolFilter.py
.
import numpy as np from scipy.signal import savgol_filter from base import BaseAlgo from util.param_util import convert_params from util import df_util
Define the class
Inherit from the BaseAlgo. The class name is the name of the algorithm.
class SavgolFilter(BaseAlgo): """Use SciPy's savgol_filter to run a filter over fields."""
Define the init method
The __init__
method passes the options from the search to the algorithm. Because there isn't an estimator class like other examples, attach the params to the object (self
) to use later.
def __init__(self, options): # set parameters params = options.get('params', {}) out_params = convert_params( params, ints=['window_length', 'polyorder', 'deriv'] ) # set defaults for parameters if 'window_length' in out_params: self.window_length = out_params['window_length'] else: self.window_length = 5 if 'polyorder' in out_params: self.polyorder = out_params['polyorder'] else: self.polyorder = 2 if 'deriv' in out_params: self.deriv = out_params['deriv'] else: self.deriv = 0
Define the fit method
The fit
method is where you compute the correlations. And then returns the DataFrame.
def fit(self, df, options): X = df.copy() X, nans, columns = df_util.prepare_features(X, self.feature_variables) # Define a wrapper function def f(x): return savgol_filter(x, self.window_length, self.polyorder, self.deriv) # Apply that function along each column of X y_hat = np.apply_along_axis(f, 0, X) names = ['SG_%s' % col for col in columns] output_df = df_util.create_output_dataframe(y_hat, nans, names) df = df_util.merge_predictions(df, output_df) return df
End-to-end example
This Savitzky-Golay Filter example covers the following tasks:
- Using the BaseAlgo class
- Converting parameters
- Using the prepare_features utility
- Using an arbitrary function to transform data
import numpy as np from scipy.signal import savgol_filter from base import BaseAlgo from util.param_util import convert_params from util import df_util class SavgolFilter(BaseAlgo): def __init__(self, options): # set parameters params = options.get('params', {}) out_params = convert_params( params, ints=['window_length', 'polyorder', 'deriv'] ) # set defaults for parameters if 'window_length' in out_params: self.window_length = out_params['window_length'] else: self.window_length = 5 if 'polyorder' in out_params: self.polyorder = out_params['polyorder'] else: self.polyorder = 2 if 'deriv' in out_params: self.deriv = out_params['deriv'] else: self.deriv = 0 def fit(self, df, options): X = df.copy() X, nans, columns = df_util.prepare_features(X, self.feature_variables) def f(x): return savgol_filter(x, self.window_length, self.polyorder, self.deriv) y_hat = np.apply_along_axis(f, 0, X) names = ['SG_%s' % col for col in columns] output_df = df_util.create_output_dataframe(y_hat, nans, names) df = df_util.merge_predictions(df, output_df) return df
Support Vector Regressor example | Custom algorithms and PSC libraries version dependencies |
This documentation applies to the following versions of Splunk® Machine Learning Toolkit: 5.1.0, 5.2.0, 5.2.1, 5.2.2, 5.3.0, 5.3.1, 5.3.3, 5.4.0, 5.4.1, 5.4.2, 5.5.0
Feedback submitted, thanks!