Install the Machine Learning Toolkit
The Splunk Machine Learning Toolkit (MLTK) enables users to create, validate, manage, and operationalize machine learning models through a guided user interface. Use the following directions to install the MLTK on to your system(s).
Requirements
The current version of the Splunk Machine Learning Toolkit is 4.4.1. You cannot access new features in the MLTK without upgrading to the latest version of the Machine Learning Toolkit. In order to successfully run the latest version of MLTK, the following is required:
- Splunk Enterprise 7.0 or later or Splunk Cloud
- Installation of the Python for Scientific Computing (PSC) add-on version 1.3 or 1.4 from Splunkbase
- Installation of the Splunk Machine Learning Toolkit app from Splunkbase
It is recommended that you install the Python for Scientific Computing (PSC) add-on first, followed by the Splunk Machine Learning Toolkit app.
You can choose the appropriate version of the Python for Scientific Computing (PSC) add-on for your environment:
Linux 32-bit support is not available should you upgrade to version 1.4 of the Python for Scientific Computing add-on.
Versions 3.4.0 and above of the MLTK require upgrading to versions 1.3 or 1.4 of the PSC add-on. See the version dependencies table for the specific requirements between toolkit and PSC add-on versions.
Specific version dependencies
MLTK Version PSC Version 4.4.1 1.3 or 1.4 4.4.0 1.3 or 1.4 4.3.0 1.3 or 1.4 4.2.0 1.3 or 1.4 4.1.0 1.3 4.0.0 1.3 3.4.0 1.3 3.3.0 1.2 or 1.3 3.2.0 1.2 or 1.3 3.1.0 1.2
If you have written any custom algorithms that rely on the PSC libraries, upgrading to version 1.3 or 1.4 the PSC library add-on will impact those algorithms. You will need to re-train any models (re-run the search that used the fit
command) using those algorithms after you upgrade the PSC add-on.
Splunk Cloud deployments
Follow the appropriate directions for your instance of self-service or managed Splunk Cloud.
Splunk Cloud trial and self-service Splunk Cloud
Install the Python for Scientific Computing add-on and the Splunk Machine Learning Toolkit app to your self-service instance of Splunk Cloud using the app browser in Splunk Cloud.
- Log in to your Splunk Cloud instance.
- From the Splunk Web home screen, click on the gear icon next to Apps in the left navigation bar.
- Click Browse more apps.
- Search for the Python for Scientific Computing add-on and install it.
- Search for the Splunk Machine Learning Toolkit app and install it.
Managed Splunk Cloud
Open a ticket with support and request the Python for Scientific Computing add-on and Splunk Machine Learning Tooklit app to be installed for you.
Splunk Enterprise single instance deployments
Follow these directions for single instance deployments.
Install the Python for Scientific Computing add-on and Splunk Machine Learning Toolkit app onto your single instance Splunk Enterprise
- Install the Python for Scientific Computing add-on first (required).
- Install the Splunk Machine Learning Toolkit app.
Install an app or add-on in Splunk Web
- In Splunk Web, click on the gear icon next to Apps in the left navigation bar.
- On the Apps page, click Install app from file.
- Click Choose File, navigate to and select the package file for the app or add-on, then click Open.
- Click Upload.
Install an app or add-on from the command line
At the command line, enter the following content, depending on your operating system.
- Unix/Linux:
./splunk install app <path/packagename>
- Windows:
splunk install app <path\packagename>
Alternatively, unpack/unzip the file then copy the app directory to $SPLUNK_HOME/etc/apps
on Unix based systems or %SPLUNK_HOME%\etc\apps
on Windows systems.
Splunk Enterprise distributed deployments
Follow these directions for distributed deployments.
Use the following tables to determine where and how to install the Splunk Machine Learning Toolkit and Python for Scientific Computing add-on in a distributed deployment of Splunk Enterprise. Depending on your environment, you may need to install the Splunk Machine Learning Toolkit and Python for Scientific Computing add-on in multiple places.
Where to install Splunk Machine Learning Toolkit and Python for Scientific Computing
This table provides a reference for installing the Splunk Machine Learning Toolkit and Python for Scientific Computing to a distributed deployment of Splunk Enterprise.
Splunk instance type Supported Required Actions required / Comments Search Heads Yes Yes Install Python for Scientific Computing and the Splunk Machine Learning Toolkit to all search heads where the Splunk Machine Learning Toolkit is used. Search heads must be running Splunk Enterprise 6.6 or greater. Indexers Yes Conditional If you want to use the distributed apply feature of the Splunk Machine Learning Toolkit, install Python for Scientific Computing on all of your indexers. This feature is disabled by default. See Use your indexers to apply models for information. Indexers must be running Splunk Enterprise 6.6 or greater. The Splunk Machine Learning Toolkit does not need to be installed on the indexers to enable this functionality. Heavy Forwarders Yes No These apps do not contain a data collection component. Universal Forwarders Yes No These apps do not contain a data collection component. Light Forwarders Yes No These apps do not contain a data collection component.
Distributed deployment feature compatibility
This table describes the compatibility of the Splunk Machine Learning Toolkit and Python for Scientific Computing add-on with Splunk distributed deployment features.
Distributed deployment feature Supported Actions required Search Head Clusters Yes Search heads must be running Splunk Enterprise 6.6 or greater. Indexer Clusters Yes If you want to use the distributed apply feature of the Splunk Machine Learning Toolkit, install Python for Scientific Computing on the indexers in your cluster. This feature is disabled by default. See Use your indexers to apply models for information. Indexers must be running Splunk Enterprise 6.6 or greater. The Splunk Machine Learning Toolkit does not need to be installed on the indexers in your cluster to enable this functionality.
Use your indexers to apply models
If you have more than one Splunk indexer and want to take advantage of the parallel computing power available on your standalone Splunk indexers or Splunk indexing cluster, you can configure your indexers to run the apply
command, a CPU-intensive task that applies machine-learning models.
Follow these steps to use your indexers to apply model:
- Install the Python for Scientific Computing add-on on all of your indexers.
- On each search head in your deployment, open the local
mlspl.conf
configuration file in a text editor.
For Unix based systems:$SPLUNK_HOME/etc/apps/Splunk_ML_Toolkit/local/mlspl.conf
For Windows systems:%SPLUNK_HOME%\etc\apps\Splunk_ML_Toolkit\local\mlspl.conf
Create themlspl.conf
in the local directory if one does not exist. - Copy the
[default]
stanza from the defaultmlspl.conf
configuration file to the local version of the configuration file if this stanza is not present.
Location of defaultmlspl.conf
file on Unix based systems:$SPLUNK_HOME/etc/apps/Splunk_ML_Toolkit/default/mlspl.conf
Location of defaultmlspl.conf
file on Windows systems:%SPLUNK_HOME%\etc\apps\Splunk_ML_Toolkit\default\mlspl.conf
- Change the
streaming_apply
command to be true as follows:streaming_apply = true
Use the deployment methodology of your choice to make these configuration changes.
- To learn about updating search head cluster members, see Use the deployer to distribute apps and configuration updates in the Distributed Search manual.
- To learn about updating peers in an indexer cluster, see Manage app deployment across all peers in the Managing Indexers and Clusters of Indexers manual.
Machine Learning Toolkit files
You can view the source code for the Machine Learning Toolkit app in Unix and Windows environments:
- For Unix-based systems, see
$SPLUNK_HOME/etc/apps/Splunk_ML_Toolkit
. - For Windows systems, see
%SPLUNK_HOME%\etc\apps\Splunk_ML_Toolkit
.
The MLTK is not open source. MLTK source code is provided as an example and for educational purposes only.
Refer to the following table for subdirectory names and descriptions:
Subdirectory | Description |
---|---|
appserver/static and /bin
|
Contains the underlying code files for Python, JavaScript, CSS, and images. |
/default
|
Contains configuration and dashboard files. |
/lookups
|
Contains the sample datasets used in the Showcase examples, along with more information about the datasets and their licenses. |
Bundle replication
Permanent model files, sometimes referred to as learned models or encoded lookups, are saved on disk. These files follow Splunk knowledge object rules, including permissions and bundle replication. Bundle replication is the process by which knowledge objects on the search head are distributed to the indexers.
The Machine Learning Toolkit includes a number of example model files that support the Showcase page. These examples are powered by .csv lookup files. To prevent performance issues, these .csv lookup files are not included in the MLTK bundle replication process.
Scoring metrics in the Machine Learning Toolkit | Install the ML-SPL Performance App |
This documentation applies to the following versions of Splunk® Machine Learning Toolkit: 4.4.1
Feedback submitted, thanks!