Smart Forecasting Assistant
The Smart Forecasting Assistant enables machine learning outcomes for users with little to no SPL knowledge. Introduced in version 4.3.0 of the Machine Learning Toolkit, this new Assistant is built on the backbone of the Experiment Management Framework (EMF), offering enhanced time-series forecasting abilities. The Smart Forecasting Assistant offers a segmented, guided workflow with an updated user interface. Move through the stages of Define, Learn, Review, and Operationalize to load data, build your model, and put that model into production. Each stage offers a data preview and visualization panel.
This Assistant leverages the StateSpaceForecast algorithm which persists a model using the
fit command that can be used with the
apply command. StateSpaceForecast is based on Kalman filters, supports incremental fit, and automatically imputes any missing values in your data. To help improve the accuracy of your forecast, this algorithm includes the ability to account for the effects of specific days that need to be treated differently.
To learn more about the Smart Forecasting Assistant algorithm, see StateSpaceForecast algorithm.
The Smart Forecasting Assistant only supports univariate forecasting.
Smart Forecasting Assistant Showcase
You can gain familiarity of this new Assistant through the MLTK Showcase, accessed under its own tab. The three new Showcase examples include:
- Forecast the Number of Calls to a Call Center
- Forecast App Logons with Special Days
- Forecast App Expenses
Click the name of any Smart Forecasting Showcase to see this new Assistant and its updated interface using pre-loaded test data and pre-selected forecast parameters.
Unlike previous Showcase examples, Smart Forecasting Assistant showcases require you to click to continue the demonstration. Please refer to the following screenshots to learn where and when to take action to move the Showcase forward.
Forecast the Number of Calls to a Call Center Showcase example
The Showcase begins in the Define stage. The call center data set that ships with the MLTK is pre-loaded into the search panel for you. Click Next to continue the walk-through.
Clicking Next moves you into the Learn stage. Open the Smart forecasting menu to see the pre-selected forecasting options. You can hover over any helper icon in the Smart forecasting menu to learn more about a particular field and its function. Click Run to view these forecasting parameters on the test data and then click Next continue the walk-through. Alternately, click Edit to view these fields in edit mode, followed by Forecast to view the these forecasting parameters on the test data, then Next.
Clicking Next moves you to the Review stage. The top of the page displays the forecasting parameters set in the Learn stage in plain English. This stage enables you to review your forecast settings prior to putting this model into production. Navigate back to the Define or Learn stages as needed to adjust the results at the Review stage. You do not see an option to save work here, as this is only a Showcase. When working in the Assistant itself, options to save as well as to operationalize the model are available.
Click Cancel to return to the main Showcase page.
Smart Forecasting Assistant workflow
Move through the stages of Define, Learn, Review, and Operationalize to draw in data, build your model, and put that model into production.
This document uses the example dataset of
call_center.csv that ships with the MLTK. You can use this dataset or another of your choice to explore the Smart Forecasting Assistant and its features before building a model with your own data.
To begin, select Smart Forecasting from the Experiments landing page and the Create New Experiment button in the top right.
Enter an Experiment Title, and optionally add a Description. Click Create to move into the Assistant interface.
Use the Define stage to select and preview the data you wish to use for the forecast. You can pull in data from anywhere in the Splunk platform. Use the Search bar as you would with any other Experiment Assistant. You can the Search bar to modify your dataset data in advance of using it within the Learn step.
As an alternative to accessing data via Search, you can choose the Datasets option. Under Datasets, you can find any data you have ingested into Splunk, as well as any datasets that ship with Splunk Enterprise and the Machine Learning Toolkit. You can filter by type to find your preferred data faster.
As with other Experiment Assistants, the Smart Forecasting Assistant includes a time-range picker to narrow down the data time-frame to a particular date or date range. The default setting of 'All time' can be changed to suit your needs. Once data is selected, the Data Preview and Visualization tabs populate.
Following the selection of data, the Learn stage becomes the available next step. Click Learn in the left menu, or Next in the top right of the page to continue.
Use the Learn stage to perform any preprocessing on your data, and to create your forecasting model. Select from the interactive learning steps to customize and complete the forecasting outcome.
The Learn stage begins with the Initial data menu open, showing the successful addition of data from the Define stage with a green check mark icon. A preview and visual evaluation of the data is available in the main page body.
Choose your forecast parameters using the Smart forecasting menu. Click Edit to add or update the available fields.
You can refer to the following table for details of each available field. Certain fields are required. Hovering over the question mark helper icons beside each field also provides field descriptions.
|Field to forecast||Required field. Select a numeric field to forecast.|
|Holdback period||Required field. Holdback is the number of data points held back from training to compare the forecast against known values. This comparison is done using R2 and RMSE statistics.|
|Future timespan||Required field. Pick number of days you want to forecast into the future. The further into the future the forecast, the less accurate it is likely to be.|
|Confidence interval||Required field. Specify and integer between 0 and 99, where a larger value means a greater tolerance for forecast uncertainty.|
|Special days field||Optional field. Accessed using the Join special time entries preprocessing step. Special days data can improve your forecast by accounting for days which should be treated differently such as Black Friday sales or IP traffic on July 4th.|
|Period||Optional field. Specify if the data has a known periodicity. The units of the period are equal to the span of the |
|Notes||Optional field. Use this free form block of text to track the selections made in the parameter fields. Refer back to notes to review which parameter combinations yield the best results.|
The fields for Holdback period and Future timespan allow you to choose the measurement value from a pre-populated list. Click Forecast to see a preview of your parameter settings.
Clicking Forecast produces a preview and visualization of the forecast. A plain English description of the chosen parameters also displays at the top of the screen. Use the Edit button to further adjust your forecast parameters.
Leveraging the StateSpaceForecast algorithm provides the option to take company or business calendar specific days into account when building your forecasting model. Include these special days in your forecast using the Join special time entries preprocessing step option.
Add any special days data using joins from CSV lookups. For details on how to work with data from CSV lookups, see Define a CSV lookup in Splunk Web.
As you work through the Smart Forecasting Assistant, SPL is created for you and can be viewed via the SPL button.
Changes to the Experiment are also tracked and can be viewed by clicking View History.
When you are happy with the results from the parameter settings in the Learn stage, click Next in the top right of the page to continue.
From the Review stage, you can see the forecast accuracy based on the parameters selected at the Learn stage. The panels on the Review page let you confirm you are seeing good results prior to putting this model into production.
Use the provided model statistics from R2 and RMSE to assess model accuracy and error rate. You can also choose to set a threshold for the amount of time before a metric reaches a certain value. Navigate back to the Learn stage to make forecast adjustments, or click Save and Next to continue.
After you click Save and Next a modal window offers the opportunity to update the Experiment name or description. When ready, click Save.
The Operationalize stage provides publishing, alerting, and scheduled training in one place. Click Done to move to the Experiments listings page.
The Experiments listing page provides a place to publish, set up alerts, and schedule training for any of your saved Experiments across all Assistant types including Smart Forecasting.
To learn about implementing analytics and data science projects using Splunk's statistics, machine learning, built-in and custom visualization capabilities, see the Splunk for Analytics and Data Science course.
This documentation applies to the following versions of Splunk® Machine Learning Toolkit: 4.3.0