Splunk® Machine Learning Toolkit

User Guide

This documentation does not apply to the most recent version of Splunk® Machine Learning Toolkit. For documentation on the most recent version, go to the latest release.

Smart Forecasting Assistant

The Smart Forecasting Assistant enables machine learning outcomes for users with little to no SPL knowledge. Introduced in version 4.3.0 of the Machine Learning Toolkit, this new Assistant is built on the backbone of the Experiment Management Framework (EMF), offering enhanced time-series forecasting abilities. The Smart Forecasting Assistant offers a segmented, guided workflow with an updated user interface. Move through the stages of Define, Learn, Review, and Operationalize to load data, build your model, and put that model into production. Each stage offers a data preview and visualization panel.

This Assistant leverages the StateSpaceForecast algorithm which persists a model using the fit command that can be used with the apply command. StateSpaceForecast is based on Kalman filters, supports incremental fit, and automatically imputes any missing values in your data. To help improve the accuracy of your forecast, this algorithm includes the ability to account for the effects of specific days that need to be treated differently.

To learn more about the Smart Forecasting Assistant algorithm, see StateSpaceForecast algorithm.

Version 4.4.0 of the Smart Forecasting Assistant supports both univariate and multivariate forecasting . Version 4.3.0 of the Smart Forecasting Assistant is limited to univariate forecasting.

Smart Forecasting Assistant Showcase

You can gain familiarity of this new Assistant through the MLTK Showcase, accessed under its own tab. The Smart Forecasting Assistant Showcase examples include:

  • Forecast the Number of Calls to a Call Center
  • Forecast App Logons with Special Days
  • Forecast App Expenses
  • Forecast App Expenses from Multiple Variables

This image shows the landing page for the Machine Learning Toolkit Showcase page. A new Showcase option is highlighted with four examples listed for the Smart Forecasting Assistant.

Click the name of any Smart Forecasting Showcase to see this new Assistant and its updated interface using pre-loaded test data and pre-selected forecast parameters.

Smart Forecasting Assistant showcases require you to click to continue the demonstration. Please refer to the following screenshots to learn where and when to take action to move the Showcases forward.

Forecast the Number of Calls to a Call Center Showcase example

The Showcase begins in the Define stage. The call center data set that ships with the MLTK is pre-loaded into the search panel for you. Click Next to continue the walk-through.

This image shows the Define stage of the Showcase. The data set for call center data is already present in the Search bar. The magnifying glass icon and button labeled Next are highlighted.

Clicking Next moves you into the Learn stage. Open the Smart forecasting menu to see the pre-selected forecasting options. You can hover over any helper icon in the Smart forecasting menu to learn more about a particular field and its function. Click Run to view these forecasting parameters on the test data and then click Next continue the walk-through. Alternately, click Edit to view these fields in edit mode, followed by Forecast to view the these forecasting parameters on the test data, then Next.

This image shows the Learn stage of the Showcase. The Smart Forecasting drop down menu is open showing fields including Field to forecast and Future timespan. The button labeled Run and the button labeled Next are highlighted.

Clicking Next moves you to the Review stage. The top of the page displays the forecasting parameters set in the Learn stage in plain English. This stage enables you to review your forecast settings prior to putting this model into production. Navigate back to the Define or Learn stages as needed to adjust the results at the Review stage. You do not see an option to save work here, as this is only a Showcase. When working in the Assistant itself, options to save as well as to operationalize the model are available.

Click Cancel to return to the main Showcase page.

This image shows the Review stage of the Showcase. The top of the page shows a written overview of the fields selected at the Learn stage. The Cancel button is highlighted.

Smart Forecasting Assistant univariate workflow

Move through the stages of Define, Learn, Review, and Operationalize to draw in data, build your model, and put that model into production.

This example workflow uses the call_center.csv dataset that ships with the MLTK. You can use this dataset or another of your choice to explore the Smart Forecasting Assistant and its features before building a model with your own data.

To begin, select Smart Forecasting from the Experiments landing page and the Create New Experiment button in the top right.

This image shows the Machine Learning Toolkit and the view under the Experiments tab. The Experiment types are displayed from which a user can create a new Experiment of that type. The new Experiment type of Smart Forecasting Assistant is highlighted.

Enter an Experiment Title, and optionally add a Description. Click Create to move into the Assistant interface.

This image shows the resulting modal window that generates following clicking the Create New Experiment button. Fields are filled in for Experiment Title and Description and a button labeled Create is highlighted in the bottom right corner of the modal window.

Define

Use the Define stage to select and preview the data you want to use for the forecast. You can pull in data from anywhere in the Splunk platform. Use the Search bar as you would with any other Experiment Assistant. You can the Search bar to modify your dataset data in advance of using it within the Learn step.

This image shows the Define stage of the Assistant. The Search bar is highlighted and contains an inputlookup for the call center dataset.

As an alternative to accessing data via Search, you can choose the Datasets option. Under Datasets, you can find any data you have ingested into Splunk, as well as any datasets that ship with Splunk Enterprise and the Machine Learning Toolkit. You can filter by type to find your preferred data faster.

This image shows the Define stage of the Assistant. This view is if the alternate option to getting data into the Assistant called Datasets. The View Datasets menu is open with the call center dataset selected from the list

As with other Experiment Assistants, the Smart Forecasting Assistant includes a time-range picker to narrow down the data time-frame to a particular date or date range. The default setting of 'All time' can be changed to suit your needs. Once data is selected, the Data Preview and Visualization tabs populate.

This image shows the Define stage of the Assistant. The menu option to change the default time range for the data from All time to another preset time frame or a custom time frame is open.

Following the selection of data, the Learn stage becomes the available next step. Click Learn in the left menu, or Next in the top right of the page to continue.

This image shows the Define stage of the Assistant. The left hand side menu option of Learn is highlighted. The green button labeled Next in the top right corner of the page is also highlighted.

Learn

Use the Learn stage to perform any preprocessing on your data, and to create your forecasting model. Select from the interactive learning steps to customize and complete the forecasting outcome.

The Learn stage begins with the Initial data menu open, showing the successful addition of data from the Define stage with a green check mark icon. A preview and visual evaluation of the data is available in the main page body.

This image shows the Learn stage of the Assistant. In the body of the page, the Initial data menu is open showing the search string used to bring in data during the Define stage. A green check mark icon is highlighted and indicates the data was brought in successfully.

Choose your forecast parameters using the Smart forecasting menu. Click Edit to add or update the available fields.

This image shows the Learn stage of the Assistant. The menu for Smart Forecasting is open. Available field options for input include Field to forecast, Holdback period, Future timespan, and Confidence interval.

You can refer to the following table for details of each available field. Certain fields are required. Hovering over the question mark helper icons beside each field also provides field descriptions.

Field name Description
Field to forecast Required field. Select one (univariate) or more (multivariate) numeric field to forecast.
Holdback period Required field. Holdback is the number of data points held back from training to compare the forecast against known values. This comparison is done using R2 and RMSE statistics.
Future timespan Required field. Pick number of days you want to forecast into the future. The further into the future the forecast, the less accurate it is likely to be.
Confidence interval Required field. Specify and integer between 0 and 99, where a larger value means a greater tolerance for forecast uncertainty.
Special days field Optional field. Accessed using the Join special time entries preprocessing step. Special days data can improve your forecast by accounting for days which should be treated differently such as Black Friday sales or IP traffic on July 4th.
Period Optional field. Specify if the data has a known periodicity. The units of the period are equal to the span of the _time field. For example, hourly data may have a period of 24 (one day), whereas daily data may have a period of 7 (one week).
Notes Optional field. Use this free form block of text to track the selections made in the parameter fields. Refer back to notes to review which parameter combinations yield the best results.

The fields for Holdback period and Future timespan allow you to choose the measurement value from a pre-populated list. Click Forecast to see a preview of your parameter settings.

This image shows the available Smart Forecasting fields in edit mode. The Holdback Period field is highlighted to show the option to set this period to be measured by different values including Points, Seconds, Hours, and Years.

Clicking Forecast produces a preview and visualization of the forecast. A plain English description of the chosen parameters also displays at the top of the screen. Use the Edit button to further adjust your forecast parameters.

This image shows the Learn stage with field selections set under the Smart Forecasting menu. A visualization of the data is displayed on the Evaluate screen. The Edit button is highlighted. You can use Edit to continue to adjust the forecast settings.

Leveraging the StateSpaceForecast algorithm provides the option to take company or business calendar specific days into account when building your forecasting model. Include these special days in your forecast using the Join special time entries preprocessing step option.

This image highlights the button within the Learn stage to add a preprocessing step. When hovered over, this button displays the text of Join special time entries.

Add any special days data using joins from CSV lookups. For details on how to work with data from CSV lookups, see Define a CSV lookup in Splunk Web.

This image shows the resulting view following selecting the Add a preprocessing step button. Available fields include Lookup, Time field in data, and Lookup time format. The button labeled Join is highlighted.

As you work through the Smart Forecasting Assistant, SPL is created for you and can be viewed via the SPL button.

This image shows the modal window generated from choosing the SPL button in the Assistant. As you work through the Assistant, SPL is written for you, and can be viewed anytime using the SPL button.

Changes to the Experiment are also tracked and can be viewed by clicking View History.

This image shows the modal window generated from choosing the View History button in the Assistant. As you adjust settings within the Assistant, these changes are tracked and can be referred to using the View History option.

When you are happy with the results from the parameter settings in the Learn stage, click Next in the top right of the page to continue.

This image shows the Learn stage of the Assistant. Green check mark icons appear beside the Initial data and Smart forecasting options indicating you can proceed to the Review stage. The button labeled Next in the top right corner is highlighted.

Review

From the Review stage, you can see the forecast accuracy based on the parameters selected at the Learn stage. The Review panels give you the opportunity to assess your forecasting results prior to putting the model into production.

Use the provided model statistics from R2 and RMSE to assess model accuracy and error rate. You can also choose to set a threshold for the amount of time before a metric reaches a certain value. Navigate back to the Learn stage to make forecast adjustments, or click Save and Next to continue.

This image shows the Review stage of the Assistant. Four panels allow you to review your results prior to putting the model into production. Panels include R squared statistic, Root Mean Squared Error, Forecasted Value, and Earliest Threshold Violation. Both Forecasted Value, and Earliest Threshold Violation allow you to define dates and thresholds on screen. A Save and Next button in the top right is highlighted.

After you click Save and Next a modal window offers the opportunity to update the Experiment name or description. When ready, click Save.

This image shows the modal window generated from clicking the Save and Next button. This Save Experiment window allows you to give the Experiment an updated name or description. A green button labeled Save in the bottom right of the window is highlighted.

Operationalize

The Operationalize stage provides publishing, alerting, and scheduled training in one place. Click Done to move to the Experiments listings page.

This image shows the Operationalize stage of the Assistant. Options on this page include Publish Forecasting Models, Create Alert, Manage Alerts, Schedule Model Training, and View Scheduled Training Jobs. A green button labeled Done in the top right of the page is highlighted.

The Experiments listing page provides a place to publish, set up alerts, and schedule training for any of your saved Experiments across all Assistant types including Smart Forecasting.

This image shows the page you land on after you mark your Experiment as Done. This is the Experiment list view page. The Experiment created in this document is listed, and options to Manage and Publish the Experiment are highlighted.

Smart Forecasting Assistant multivariate workflow highlights

Introduced in version 4.4.0 of the MLTK, you can perform both univariate and multivariate forecasting with the Smart Forecasting Assistant. Version 4.3.0 of the Assistant is limited to univariate forecasting.

The multivariate workflow is the same as univariate in most ways. Particular screens offer some key differences associated to a multivariate workflow. Highlights of the multivariate workflow include:

Workflow stage Highlight
Learn Field to forecast menu is a multi-pick of up to five fields.
The Evaluate view offers combined view of fields to forecast.
Review See the number of fields to forecast as well as fields by name.
Use the View fields menu to filter results shown.
Choose to see fields in a combined or split view.
Set the Earliest Threshold Violation values in one place or set individual thresholds in split view.
Toggle the confidence interval on or off for the combined chart or per chart in split view.
Operationalize When setting Alerts you can choose which target field to alert on, based on the fields selected when building the Experiment.

The maximum number of fields to forecast you can select for the Smart Forecasting Assistant is five.

Learn

Use the Learn stage to perform any preprocessing on your data, and to create your forecasting model. In the multivariate workflow, the Field to forecast menu is multi-pick up to five fields and the list of fields is populated based on your data.

This image shows the Learn stage of the Smart Forecasting Assistant. The Field to forecast menu is open showing the ability to select multiple fields.

Review

Use the Review stage to assess the forecast based on your selections at the Learn stage. The Review panels give you the opportunity to assess your forecasting results prior to putting the model into production.

See the total number of chosen fields to forecast as well as those fields by name in their own drop-down. Choose to review the forecast charts in a combined view of one chart or a split view. Toggle the confidence interval on or off for the entire chart using the combined chart view, or by individual chart in the split chart view.

This image shows the Review stage of the Smart Forecasting Assistant. The top of the page shows the plain English summary of the forecast including the number of fields used to forecast and the a names of those fields. The option to view fields as combined chart is highlighted. Also highlighted is the option on the chart to toggle the confidence interval on or off.

In the combined view you can set the Earliest Threshold Violations for the all fields to forecast on one screen. Settings selected are immediately reflected in the chart results.

This image shows the Review stage of the Smart Forecasting Assistant. The Earliest Threshold Violation area is highlighted to show that these values can be set on screen for one or more of your fields to forecast.

In the split view you can set the Earliest Threshold Violations by individual fields to forecast. Settings selected are reflected in the chart results when you click Apply.

This image shows the Review stage of the Smart Forecasting Assistant. The split chart view option is selected which has broken out the two fields to forecast into their own charts. One is for CRM and one is for ERP. The ERP chart is selected. Each chart has its own section to set the Earliest Threshold Violations. Clicking the Apply button updates the chart with any chosen settings.

In addition to the combined and split chart view options you can also customize fields by which to review results from the View fields drop-down menu.

This image shows the Review stage of the Smart Forecasting Assistant. The View fields area is highlighted to show a drop-down option by which you can filter the chart view of forecast results.

Operationalize

The Operationalize stage provides publishing, alerting, and scheduled training in one place. In a multivariate workflow you can choose which field to alert on, based on the fields selected when building the Experiment.

This image shows the Save As Alert modal that is accessed from the Operationalize stage. The field selector menu is highlighted, showing the ability to select from the list of fields chosen when building the Experiment. In univariate forecasting the one field is pre-populated for this alert setting.

Learn more

To learn about implementing analytics and data science projects using Splunk's statistics, machine learning, built-in and custom visualization capabilities, see the Splunk for Analytics and Data Science course.

Last modified on 17 October, 2019
Preprocessing machine data using MLTK Assistants   Predict Numeric Fields Experiment Assistant workflow

This documentation applies to the following versions of Splunk® Machine Learning Toolkit: 4.4.0, 4.4.1, 4.4.2, 4.5.0

Acrobat logo Download manual
Acrobat logo Download this page

Was this topic useful?







You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters