Splunk® Machine Learning Toolkit

User Guide

This documentation does not apply to the most recent version of Splunk® Machine Learning Toolkit. For documentation on the most recent version, go to the latest release.

What's new

Here's what's new in each version of the Splunk Machine Learning Toolkit:

Version 4.0.0

Features and improvements

  • A number of new demos have been added to the Showcase tab based on customer requests.
  • The Experiment History tab now captures the model history from scheduled retraining.
  • Introduction of the LocalOutlierFactor algorithm for unsupervised anomaly detection. Accessing this algorithm requires an upgrade to PSC version 1.3.
  • The MLPClassifier algorithm now supports partial_fit for incremental learning.
  • Introduction of the score command to validate models and statistical tests for any use case.
  • Introduction of k-fold cross validation as a method to help test for model overfitting.
  • A new tab called Settings is now part of the MLTK nav bar. Users with admin access can work within this interface to configure the toolkit settings of the fit and apply commands. Ensure you know the impact of making changes to the algorithm settings by adding the ML-SPL Performance App for the Machine Learning Toolkit to your setup via Splunkbase.
  • Customers can now share and reuse custom algorithms in the Splunk Community for MLTK on GitHub.
  • The Splunk MLTK Connector for Apache Spark™ allows users to leverage their own Spark clusters for large data sets. This is a limited availability release. Reach out to Splunk's ML Spark team for LAR application access.
  • The Splunk MLTK Container for TensorFlow™ is an add-on docker container, allowing multiple local GPU/ CPU acceleration. This is available via Splunk's Professional Services department.
  • For bug fixes, see Fixed issues.

Version 3.4.0

Features and improvements

  • Version 1.3 of the Python for Scientific Computing add-on is now available in Splunkbase. Upgrading to version 3.4 of the MLTK requires upgrading to PSC version 1.3 .
  • Introduction of the MLPClassifier algorithm. Accessing this algorithm requires an upgrade to PSC version 1.3.
  • Introduction of Boxplot Chart to the search visualization options.
  • Models created within the Experiments framework can now be published and more easily used outside of the MLTK environment.
  • For bug fixes, see Fixed issues.

Upgrading to version 3.4.0 of the MLTK requires upgrading to version 1.3 of the Python for Scientific Computing add-on. Two previous versions of the MLTK (3.2.0 and 3.3.0) will successfully operate on version 1.2 or 1.3 of the PSC add-on. However, users cannot access new features in the 3.4.0 MLTK without updating to that version.

Version 3.3.0

Features and improvements

Version 3.4.0 of the MLTK will require an update to Version 1.3 of Python for Scientific Computing. The release for the MLTK 3.4.0 will coincide with the availability of PSC 1.3 in Splunkbase.

Version 3.2.0

Features and improvements

  • Introduction of the Experiment Management Framework. This framework ties the experiment, along with any alerts or scheduled trainings, together. Users can now see which alerts or scheduled trainings are assigned to any experiment, and which experiment has or has not undergone pre-processing steps. See Using the Splunk Machine Learning Toolkit and Experiments.
  • Relocation within the Machine Learning Toolkit of the previously free-standing Assistant module. This version of Assistants now lives under the Legacy tab of the MLTK bar. It is recommended that you do not create Models via this version of Assistants, and instead create Models via the Experiments Management Framework. Doing so will ensure that you can both:
    • Create Alerts and Scheduled Trainings on the saved Experiment
    • See Alerts and Scheduled Trainings organized by Experiment
  • The Splunk Machine Learning Toolkit version 3.2 does not support Splunk Enterprise version 6.4 or earlier.
  • For bug fixes, see Fixed issues.

Version 3.1.0

Features and improvements

  • The FieldSelector algorithm can now be used in the preprocessing panel. See FieldSelector in the Machine Learning Toolkit User Guide.
  • The maximum number of distinct values supported for categorical fields, formerly 100, can now be configured for both features fields and target fields.
  • The Splunk Machine Learning Toolkit has a new clustering algorithm:
  • For bug fixes, see Fixed issues.

Version 3.0.0

Features and improvements

Introduced a new interface for managing models. You can now easily see what types of models you have, inspect the settings of each model (such as which variables were used to train it), and view or update each model's sharing settings. For more information, see Manage models.

Version 2.4.0

Features and improvements

Version 2.3.0

Features and improvements

  • Entries in the "Load Existing Settings" tab are now unique per-user instead of being shared with all users. Entries created prior to version 2.3 will continue to be accessible by all users.
  • Two new algorithms have been added:
    • ACF (autocorrelation function)
    • PACF (partial autocorrelation function)
  • The Forecast Time Series assistant now allows for the selection of the ARIMA forecasting algorithm. Additional panels have been added for inspecting properties unique to ARIMA models.

Version 2.2.1

This version contains bug fixes. See Fixed issues for details.

Version 2.2.0

Features and improvements

  • The preprocessing feature has been redesigned and is offered in the Predict Numeric Fields, Predict Categorical Fields, and Clustering Numeric Events assistants. See Preprocessing for information.
  • The ML-SPL API has been updated to make it easier for developers and partners to import custom algorithms in order to extend the capabilities of the Splunk Machine Learning Toolkit. See ML-SPL API Guide for information.

Version 2.1.0

Features and improvements

Enhancements to the Detect Numeric Outliers assistant:

  • You can now specify one or more fields to split by (up to 5). Specifying one or more split by fields enables you to see the values of the field you are analyzing grouped by the values of the split by fields in visualizations.
  • Enhanced visualizations including a new Data Distribution histogram that shows the number of data points within the threshold and the number of data points outside the threshold.

For more information, see Detect Numeric Outliers.

Version 2.0.1

The Downsampled Line Chart custom visualization now supports the same drilldown actions as the built-in Line Chart visualization.

Version 2.0.0

Features and improvements

  • The app has been renamed to "Machine Learning Toolkit."
  • New Cluster Numeric Events assistant that steps you through how to perform clustering on your own data. This assistant includes the ability to preprocess data by applying StandardScaler, PCA, or KernelPCA methods. See Cluster Numeric Events.
  • Updated examples for the Cluster Numeric Events showcase.
  • A streaming_apply setting has been added to the mlspl.conf file, which allows you to run the apply command on your indexers. For details, see Use your indexers to apply models.
  • The Predict Numeric Fields and Predict Categorical Fields assistants now support multiple algorithms.
  • A new visualization type has been added: Scatterplot matrix. This visualization is available in the Cluster Numeric Events assistant.
  • The Machine Learning Toolkit app has a walk-through tour and each assistant has its own walk-through tour.
  • A link to machine learning video tutorials has been added to the top menu bar and the Showcase page.
  • Tooltips have been added for the fields in each of the assistants.

Algorithms

  • The SGDClassifier algorithm is now supported. For details, see Algorithms.
  • The SGDRegressor algorithm is now supported. For details, see Algorithms.
  • The ARIMA algorithm is now supported. For details, see Algorithms.
  • The LogisticRegression algorithm supports a new parameter probabilities=<true|false>. For details, see Algorithms.
  • Summary support has been added to the RandomForestClassifier and RandomForestRegressor algorithms. For details, see Algorithms.
  • The BernoulliNB, GaussianNB, Birch, and StandardScaler algorithms support a new parameter partial_fit=<true|false>. For details, see Algorithms.

Version 1.3.0

Features and improvements

  • You can now create alerts within the Machine Learning Toolkit from some of the panels in the assistants. Alerts can be viewed under Scheduled Jobs > Alerts.
  • You can now schedule model training in the Predict Numeric Fields and Predict Categorical Fields assistants by clicking the icon on the right side of the Fit Model button.
Mlapp fitmodelscheduleicon.jpg
Schedules can be viewed under Scheduled Jobs > Scheduled Training.
  • The Training/Test split can now be set to a 100/0 split (no split).

Version 1.2.0

Features and improvements

  • The DecisionTreeClassifier and DecisionTreeRegressor algorithms are now supported. For details, see Algorithms.
  • The Detect Numeric Outliers assistant now includes an Include current point checkbox to support the "current" parameter of the streamstats command.
  • The Predict Numeric Fields assistant has an improved Actual vs. Predicted Line Chart, which replaces the Actual vs. Predicted Overlay.
  • Two macros in the Forecast Time Series assistant have been merged into one macro.
  • The max_features parameter of the RandomForestClassifier and RandomForestRegressor algorithms now accepts values with the float data type.
  • The Remove from history confirmation dialog box has been improved.
  • A basic framework has been implemented for displaying Bootstrap's modal dialog boxes in the Machine Learning Toolkit and Showcase UI.

Version 1.1.0

Features and improvements

  • The visualizations in the Cluster Events showcase have been updated.
  • The Predict Numeric Fields and Predict Categorical Fields assistants now allow you to enter wildcards in Fields to use for predicting. For example, to specify both the Packets Received and Packets Sent fields, enter "Packets*". Wildcards are case sensitive.
  • The Select All and Select None buttons on the Predict Numeric Fields and Predict Categorical Fields assistants have been moved inside the dropdown list.

Algorithms

  • The KernelRidge regression algorithm is now supported. For details, see Algorithms.

Bug fixes

The following bugs were fixed:

  • Changing the time range or search mode on assistant search bars will now re-run the search in the search bar, the same as the default Search page in Splunk Enterprise.
  • Custom visualizations will now display time stamps correctly when the event time differs from browser time.
  • Caching issues have been fixed, and the app no longer loads old versions of resources after an update.
  • Exit points in assistants now correctly have the same time range as that assistant's search bar

Version 1.0.0

This is the first release of the Machine Learning Toolkit and Showcase app.

Last modified on 10 December, 2018
Preprocessing machine data   Known issues

This documentation applies to the following versions of Splunk® Machine Learning Toolkit: 4.0.0

Acrobat logo Download manual
Acrobat logo Download this page

Was this topic useful?







You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters