Upgrade the Splunk Machine Learning Toolkit

The Splunk Machine Learning Toolkit (MLTK) regularly releases new features and enhancements. You can learn about new features and enhancements by release version, on the What's new page.

The Splunk Enterprise Security App relies on MLTK and the PSC add-on. If you are a Splunk Enterprise Security App user, and you are upgrading that app, restart your Splunk instance first. Doing so closes any background PSC processes that can cause the Splunk Enterprise Security App upgrade to error out.

Requirements

Running MLTK version 5.4.2 requires Splunk Enterprise 8.2.x or higher or Splunk Cloud Platform. If you choose to upgrade to MLTK version 5.4.2 you must use Python for Scientific Computing (PSC) add-on version 3.2.1 or 4.2.1. Both MLTK and PSC must be updated.

If you choose to upgrade to MLTK version 5.4.2 and you have trained models that are built using certain algorithms, then you will need to retrain those models. This need for retraining is the result of the PSC add-on upgrading scikit-learn from version 0.24.2 to version 1.5.1. See the table for details.

• To retrain a model re-run the search that used the fit command.
• If you use a validation script as part of your machine learning process, also consider revalidating these algorithms.
• If you use the ML-SPL API additional steps might also be required to validate any custom algorithms. See the Adding a custom algorithm to the Splunk Machine Learning Toolkit in the ML-SPL API Guide for more information.

Models trained using the following algorithms will require retraining if you upgrade to MLTK version 5.4.2:

Old models cannot be loaded and need re-training. Run the fit command again.

Algorithm	How impacted
AutoPrediction	This algorithm uses RandomForestClassifier and RandomForestRegressor. Error: node array from the pickle has an incompatible dtype
DecisionTreeClassifier	Error: node array from the pickle has an incompatible dtype
DecisionTreeRegressor	Error: node array from the pickle has an incompatible dtype
GaussianNB	sigma_ variable is replaced with the var_ variable.
GradientBoosting	New codecs are added to save the model file. loss params values changed from {ls, lad, huber, quantile} to {squared_error, absolute_error, huber, quantile}. loss=ls and loss=lad are deprecated. Use squared_error and absolute_error respectively. Error: loss must be one of: squared_error, absolute_error, huber, quantile Error while loading an old model: loss must be one of: squared_error, absolute_error, huber, quantile
Imputer	New codecs are added to save the model file. Model saving functionality might break with older PSC versions.
KernelPCA	gamma_ variable is introduced in sklearn version 1.3. Older models don't have this variable. Error: KernelPCA object has no attribute gamma_.
RandomForestClassifier	Error: node array from the pickle has an incompatible dtype
RandomForestRegressor	criterion=mse and criterion=mae are deprecated. Use squared_error and absolute_error respectively. auto has been deprecated . base_estimator_ is replaced with estimator_. See https://scikit-learn.org/stable/whats_new/v1.3.html
SystemIdentification	Error: DataFrame object has no attribute _mgr.
TFIDF	Error:This TfidfTransformer instance is not fitted yet.

Python for Scientific Computing add-on

The Splunk Machine Learning Toolkit relies on the Python for Scientific Computing (PSC) add-on. Install or upgrade to a compatible version of the PSC add-on based on your operating system:

On some Windows operating systems, installing PSC through the Splunk Manage Apps user interface results in an error. This error is usually benign and you can ignore it. In some cases, you might need to manually unpack the package in the apps directory to get past the error.

• Python for Scientific Computing (for Mac Intel)
• Python for Scientific Computing (for Mac Apple Silicon)
• Python for Scientific Computing (for Linux 64-bit)
• Python for Scientific Computing (for Windows 64-bit)

If you have any custom algorithms that rely on the PSC libraries, upgrading the PSC add-on impacts those algorithms. You must re-train any models (re-run the search that used the fit command) using those algorithms after you upgrade PSC.

Version dependencies

For version information that includes MLTK, the PSC add-on, Python, and the Splunk platform, see Splunk Machine Learning Toolkit version dependencies.

If a newer version of PSC is required for the version of MLTK you upgrade to, an in-app instruction to upgrade PSC appears when you run the upgraded version of MLTK.

MLTK version	PSC add-on version	Splunk platform version
5.4.2	4.2.1	Splunk Enterprise 8.2.x, 9.0.x, 9.1.x, or 9.2.x or Splunk Cloud Platform
	3.2.1	Splunk Enterprise 8.2.x, 9.0.x, 9.1.x, or 9.2.x or Splunk Cloud Platform
5.4.1	4.1.2 or 4.2.0	Splunk Enterprise 9.2.x or Splunk Cloud Platform
	3.1.0, 3.2.0, 4.1.0, 4.1.2, or 4.2.0	Splunk Enterprise 8.1.x, 8.2.x, 9.0.0, 9.0.1, 9.0.5, or 9.1.0 or Splunk Cloud Platform
5.4.0	4.1.2	Splunk Enterprise 9.2.x or Splunk Cloud Platform
	3.1.0, 4.1.0, or 4.1.2	Splunk Enterprise 8.2.x, 9.0.0, 9.0.1, 9.0.5, or 9.1.0 or Splunk Cloud Platform
5.3.3	3.0.2, 3.1.0, 4.0.0, 4.1.0, or 4.1.2	Splunk Enterprise 8.1.x, 8.2.x, or 9.0.0 or Splunk Cloud Platform
5.3.1	3.0.0, 3.0.1, or 3.0.2	Splunk Enterprise 8.0.x, 8.1.x, 8.2.x, or 9.0.0 or Splunk Cloud Platform
5.3.0	3.0.0, 3.0.1, or 3.0.2	Splunk Enterprise 8.0.x, 8.1.x, 8.2.x, or 9.0.0 or Splunk Cloud Platform
5.2.2	2.0.0, 2.0.1, or 2.0.2	Splunk Enterprise 8.0.x, 8.1.x, or 8.2.0 or Splunk Cloud Platform
5.2.1	2.0.0, 2.0.1, or 2.0.2	Splunk Enterprise 8.0.x, 8.1.x, or 8.2.0 or Splunk Cloud Platform
5.2.0	2.0.0, 2.0.1, or 2.0.2	Splunk Enterprise 8.0.x, 8.1.x, or 8.2.0 or Splunk Cloud Platform
5.1.0	2.0.0, 2.0.1, or 2.0.2	Splunk Enterprise 8.0.x or 8.1.x or Splunk Cloud Platform
5.0.0	2.0.0, 2.0.1, or 2.0.2	Splunk Enterprise 8.0.x or 8.1.x or Splunk Cloud Platform

Upgrade in Splunk Web

In Splunk Web, click the Update option on the app icon in the left-hand Apps bar. The Update option appears when a new version of an app is available on Splunkbase.

Upgrade from Manage Apps

Follow these steps to upgrade the Splunk Machine Learning Toolkit using Manage Apps:

1. Download the latest version of the Splunk Machine Learning Toolkit app from Splunkbase
2. In Splunk Web, select the Manage icon next to Apps in the left navigation bar as shown in the following image:
3. On the Apps page, select Install app from file.
4. Select Choose File to navigate to and select the package file for MLTK. Then click Open.
5. Check the Upgrade app box.
6. Click Upload.

In a distributed deployment and depending on your environment, you might need to upgrade MLTK and PSC in multiple places.

Upgrade from the command line

On the command line (CLI), run the command that corresponds to your operating system:

Operating system	Command line
Unix/Linux	./splunk install app <app_package_filename> -update 1 -auth <username>:<password>
Windows	splunk install app <app_package_filename> -update 1 -auth <username>:<password>

Alternatively, unpack/unzip the file then copy the app directory to $SPLUNK_HOME/etc/apps on Unix based systems or %SPLUNK_HOME%\etc\apps on Windows systems.