Predict Categorical Fields Classic Assistant workflow
Classic Assistants enable machine learning through a guided user interface. The Predict Categorical Fields Classic Assistant displays a type of learning known as classification. A classification algorithm learns the tendency for data to belong to one category or another based on related data.
The following classification table shows the actual state of the field versus predicted state of the field. The yellow bar highlights an incorrect prediction.
The Predict Categorical Fields assistant uses the following classification algorithms to predict fields:
Create a model to predict a categorical field
Before you begin
- The Predict Numeric Fields Assistant offers the option to preprocess your data. For more information on Assistant-based preprocessing algorithms, see Preprocessing machine data using Assistants.
- The toolkit default selects the Logistic Regression algorithm. Use this default if you aren't sure which algorithm is best for you. For further details on any algorithm, see Algorithms in the Machine Learning Toolkit.
Follow these steps for the Predict Categorical Fields Classic Assistant.
- From the MLTK navigation bar select Classic > Assistants > Predict Categorical Fields.
- Run a search, and be sure to select a date range.
- (Optional) Click + Add a step to add preprocessing steps.
- Select an algorithm from the
- Select a target field from the drop-down menu
Field to predict.
When you select the
Field to predict, the
Fields to use for predictingdrop-down populates with available fields to include in your model.
- Select a combination of fields from the drop-down menu
Fields to use for predicting.
- Split your data into training and testing data. The default split is 50/50, and the data is divided randomly into two groups.
The algorithm selected determines the fields available to build your model. Hover over any field name to get more information about that field
- Type the name the model in
Save the model asfield.
You must specify a name for the model in order to fit a model on a schedule or schedule an alert.
- Click Fit Model.
Interpret and validate
After you fit the model, review the prediction results and visualizations to see how well the model predicted the categorical field. In this analysis, metrics are related to mis-classifying the field, and are based on false positives and negatives, and true positives and negatives.
|Precision||This statistic is the percentage of the time a predicted class is the correct class.|
|Recall||This statistic is the percentage of time that the correct class is predicted.|
|Accuracy||This statistic is the overall percentage of correct predictions.|
|F1||This statistic is the the weighted average of precision and recall, based on a scale from zero to one. The closer the statistic is to one, the better the fit of the model.|
|Classification Results (Confusion Matrix)||This table charts the number of actual results against predicted results, also known as a Confusion Matrix. The shaded diagonal numbers should be high (closer to 100%), while the other numbers should be closer to 0.|
Refine the model
After you validate the model, refine the model and run the
fit command again.
Consider trying the following:
- Reduce the number of fields selected in the
Fields to use for predictingdrop-down menu. Having too many fields can generate a distraction.
- Bring in new data sources to enrich your modeling space.
- Build features on raw data, model on behaviors of the data instead of raw data points, using SPL. Streamstats, eventstats, etc.
- Check your fields - are you using categorical values correctly? For example are you using DayOfWeek as a number (0 to 6) instead of "Monday", "Tuesday" , etc ? Make sure you have the right type of value.
- Bring in context via lookups - holidays, external anomalies, etc.
- Increase the number of fields ( from additional data, feature building as above,etc) selected in the
Fields to use for predictingdrop-down menu.
Deploy the model
After you validate and refine the model, you can deploy the model.
Within the classic assistant framework
- Click the Schedule Training button to the right of Fit Model to schedule model training.
This open a new modal/ window overlay with fields to fill out including Report title, time range and trigger actions. You can set up a regular interval to fit the model.
Outside the Classic Assistant framework
- Click Open in Search to to generate a New Search tab for this same dataset. This new search will open in a new browser tab, away from the Classic Assistant.
This search query that uses all data, not just the training set. You can adjust the SPL directly and see results immediately. You can also save the query as a Report, Dashboard Panel or Alert.
- Click Show SPL to generate a new window showing the search query that was used to fit the model. Copy the SPL here for use in other aspects of your Splunk instance.
- Click Schedule Alert to set up an alert that is triggered when the predicted value meets a threshold you specify.
Once you navigate away from the Classic Assistant page, you cannot return to it through the Classic or Models tabs. Classic Assistants are great for generating SPL, but may not be ideal for longer-term projects.
For more information about alerts, see Getting started with alerts in the Splunk Enterprise Alerting Manual.
Predict Numeric Fields Classic Assistant workflow
Detect Numeric Outliers Classic Assistant workflow
This documentation applies to the following versions of Splunk® Machine Learning Toolkit: 4.4.0, 4.4.1, 4.4.2, 5.0.0