Experiments

Introduced in version 3.2 of the Splunk Machine Learning Toolkit (MLTK), an Experiment is an exclusive knowledge object in Splunk that keeps track of its settings and history, as well as its affiliated alerts and scheduled trainings.

Experiments manage the data source, algorithm used and any additional parameters to configure that algorithm, within one framework. Use any of the Machine Learning Toolkit Assistants to edit your experiment.

Create an experiment

1. Create an experiment under the Experiments tab of the MLTK navigation bar:
   • If this is the first experiment in your toolkit, you will land on a display screen of all 6 assistants. Select one and continue.
   • If you already have at least one experiment in your toolkit, click the Create New Experiment button in the top right of the screen.
2. Give the experiment a name, and (optional) add a description. Both the name and description can be edited later as needed.
3. Click Create.

Configure experiment settings

Workflow will vary depending on the assistant selected. For details, see the assistant documents:

• Predict Numeric Fields
• Predict Categorical Fields
• Detect Numeric Outliers
• Detect Categorical Outliers
• Forecast Time Series
• Cluster Numeric Events

Run and save the experiment

After you configure your experiment settings, run the experiment. This will vary depending upon the selected assistant:

• Predict Numeric and Predict Categorical Fields use the fit model
• Detect Numeric Outliers and Detect Categorical Outliers use detect outliers
• Forecast Time Series uses forecast
• Cluster Numeric Events uses cluster

Important note: After you successfully run an experiment it is saved in a Draft state. This experiment is not stored to Splunk until it is saved. When ready, click the Save button in the top right of the page.

Once you are happy with the results of your experiment, save it. The action of saving will:

1. Save the assistant settings to the experiment knowledge object.
2. (As applicable) Update the draft model to an experiment model.
3. (As applicable) Update all the affiliated scheduled trainings and alerts to synchronize with the search SPL and trigger conditions.

The table below shows the comparison of a running an experiment versus saving an experiment:

Results	Run Experiment	Saved Experiment
Create new experiment history record	Yes	No
Run experiment search jobs	Yes	No
(As applicable) Save and update experiment model	No	Yes
(As applicable) Update all experiment alerts	No	Yes
(As applicable) Update experiment scheduled trainings	No	Yes

Load an experiment

You can load a saved experiment by clicking the experiment name. Doing so will retrieve your last saved experiment settings.

Manage experiments

You can manage your experiments through the Experiments tab on the MLTK navigation bar. Once on that view, click the Manage button available under the Actions column.

The Splunk Machine Learning Toolkit (MLTK) supports the following experiment management options:

1. Create experiment level alerts.
2. Edit the title (name) and description of the experiment.
3. (As applicable) Manage alerts for a single experiment.

   If you make changes to the saved experiment you may impact affiliated alerts. Re-validate your alerts once experiment changes are complete.

4. (As applicable) Schedule a training job for an experiment.
5. Delete an experiment.

Experiments are always stored under the user's namespace, meaning that changing sharing settings and permissions on experiments is not supported at this time.