Splunk® Machine Learning Toolkit

User Guide

Search macros in the Splunk Machine Learning Toolkit

Splunk Machine Learning Toolkit (MLTK) includes 3 search macros. Search macros are reusable blocks of Splunk Search Processing Language (SPL) that you can insert into other searches. Search macros can be any part of a search, such as an eval statement or search term, and do not need to be a complete command. You can also specify whether the macro takes any arguments.

Use these macros to save time when writing SPL searches and to validate models. The following macros are included with MLTK:

View the MLTK search macros

You can view the available macros from the Settings drop down menu on the main navigation bar, and by selecting Advanced Settings.

This screenshot of the Splunk interface shows the menu options available by clicking the Settings menu in the Splunk bar. The option for Advanced Settings is highlighted.

On the resulting page, choose Search macros.

This screenshot of the Advanced Settings page shows options for Search macros and Search commands. Search macros is highlighted.

From the Apps menu, choose the Splunk Machine Learning Toolkit for MLTK search macros. Listed information includes the name, definition, and status.

Search macros that take arguments are identified by a bracketed number following the name. For example, confusionmatrix(2) and regressionstatistics(2). Confusion matrix and regression statistics are the search macro names, each of which can take two (2) arguments.

This screenshot of the Search macros page shows a table of information for the macros for the Machine Learning Toolkit. Content includes the macro name, description, sharing permissions, and status. At the top of the table, users can choose from other Splunk products in the App menu to see their macros.

Insert search macros into search strings

To include a search macro in your saved or ad hoc searches, place a backtick character ( ` ) before and after the macro name. You can also reference a search macro within other search macros using this same syntax.

For search macros that take arguments, define those arguments when you insert the macro into the search string. The following example shows a search macro with the arguments defined: 

... | `classificationstatistics("DiskFailure", "predicted(DiskFailure)`

Classification statistics macro

Use the classification statistics macro to save time when measuring the statistics of your classification model.

Syntax 

... | `classificationstatistics(response, prediction)`

Example

The following example shows the classification statistics macro on sample data. The first code block shows the passing of the fit command with the LogisticRegression algorithm: 

| inputlookup disk_failures.csv | eventstats max(SMART_1_Raw) as max1 min(SMART_1_Raw) as min1 | eventstats max(SMART_2_Raw) as max2 min(SMART_2_Raw) as min2 | eventstats max(SMART_3_Raw) as max3 min(SMART_3_Raw) as min3 | eventstats max(SMART_4_Raw) as max4 min(SMART_4_Raw) as min4 | eventstats max(SMART_5_Raw) as max5 min(SMART_5_Raw) as min5 | eval SMART_1_Transformed = (SMART_1_Raw - min1)/(max1-min1) | eval SMART_2_Transformed = (SMART_2_Raw - min2)/(max2-min2) | eval SMART_3_Transformed = (SMART_3_Raw - min3)/(max3-min3) | eval SMART_4_Transformed = (SMART_4_Raw - min4)/(max4-min4) | eval SMART_5_Transformed = (SMART_5_Raw - min5)/(max5-min5) | table Date Model CapacityBytes SerialNumber DiskFailure SMART_1_Raw SMART_1_Transformed SMART_2_Raw SMART_2_Transformed SMART_3_Raw SMART_3_Transformed SMART_4_Raw SMART_4_Transformed SMART_5_Raw SMART_5_Transformed | fit LogisticRegression fit_intercept=true "DiskFailure" from "Model" "SMART_1_Transformed" "SMART_2_Transformed" "SMART_3_Transformed" "SMART_4_Transformed" "SMART_5_Transformed" into "example_disk_failures"

The second code block shows the passing of the apply command, followed by the macro: 

| inputlookup disk_failures.csv | eventstats max(SMART_1_Raw) as max1 min(SMART_1_Raw) as min1 | eventstats max(SMART_2_Raw) as max2 min(SMART_2_Raw) as min2 | eventstats max(SMART_3_Raw) as max3 min(SMART_3_Raw) as min3 | eventstats max(SMART_4_Raw) as max4 min(SMART_4_Raw) as min4 | eventstats max(SMART_5_Raw) as max5 min(SMART_5_Raw) as min5 | eval SMART_1_Transformed = (SMART_1_Raw - min1)/(max1-min1) | eval SMART_2_Transformed = (SMART_2_Raw - min2)/(max2-min2) | eval SMART_3_Transformed = (SMART_3_Raw - min3)/(max3-min3) | eval SMART_4_Transformed = (SMART_4_Raw - min4)/(max4-min4) | eval SMART_5_Transformed = (SMART_5_Raw - min5)/(max5-min5) | table Date Model CapacityBytes SerialNumber DiskFailure SMART_1_Raw SMART_1_Transformed SMART_2_Raw SMART_2_Transformed SMART_3_Raw SMART_3_Transformed SMART_4_Raw SMART_4_Transformed SMART_5_Raw SMART_5_Transformed | apply "example_disk_failures" | `classificationstatistics("DiskFailure", "predicted(DiskFailure)")`

Example output

This screenshot shows the Statistics tab of the Search page in the toolkit. There is one results row under columns for class, accuracy, precision, recall, f1, and count.

Classification report macro

You can view classification statistics results by class using the classification report macro. The classification report macro provides weighted average for each of the classification statistics classes.

Example output

This screenshot shows the Statistics tab of the Search page in the toolkit. The macro at the end of the search string for classification statistics is replaced with the macro for classification report. The results show a new row with weighted averages divided by class for the columns of accuracy, precision, recall, f1, and count.

Confusion matrix macro

Use the confusion matrix macro to save time when assessing the performance of your classification model.

Syntax 

... | `confusionmatrix(response, prediction)`

Example

The following example shows the confusion matrix macro on sample data. The first code block shows the passing of the fit command with the LogisticRegression algorithm: 

| inputlookup diabetes.csv
| sample partitions=3 seed=42
| search partition_number < 2
| fit LogisticRegression response from BMI age into LogisticRegressionClassifier

The second code block shows the passing of the apply command, followed by the macro: 

| inputlookup diabetes.csv
| sample partitions=3 seed=42
| search partition_number = 2
| apply LogisticRegressionClassifier as prediction
| `confusionmatrix(response, prediction)`

Example output

This screenshot shows the Statistics tab of the Search page in the toolkit. There are two rows of results under columns for Predicted actual, Predicted 0, and Predicted 1.

Confusion matrix report macro

You can view the confusion matrix results by class using the confusion matrix report macro. The confusion report macro provides the weighted average for each of the confusion matrix classes.

Example output

This screenshot shows the Statistics tab of the Search page in the toolkit. The macro at the end of the search string for confusion matrix is replaced with the macro for classification report. The results show a new row with weighted averages divided by class for the columns of accuracy, precision, recall, f1, and count.

Regression statistics macro

Use the regression statistics macro to save time when measuring the statistics of your regression model.

Syntax 

... | `regressionstatistics(response, prediction)`

Example

The following example shows the regression statistics macro on sample data. The first code block shows the passing of the fit command with the LinearRegression algorithm: 

| inputlookup server_power.csv | fit LinearRegression fit_intercept=true "ac_power" from "total-unhalted_core_cycles" "total-instructions_retired" "total-last_level_cache_references" "total-memory_bus_transactions" "total-cpu-utilization" "total-disk-accesses" "total-disk-blocks" "total-disk-utilization" into "example_server_power"

The second code block shows the passing of the apply command, followed by the macro: 

| inputlookup server_power.csv | apply "example_server_power" | `regressionstatistics("ac_power", "predicted(ac_power)")`

Example output

This screenshot shows the Statistics tab of the Search page in the toolkit. There is one row of results under columns for rSquared, and RMSE.

Learn more

See the following resources to learn more about search macros in the Splunk platform:

Last modified on 27 June, 2024
Search commands for machine learning safeguards   Custom visualizations in the Splunk Machine Learning Toolkit

This documentation applies to the following versions of Splunk® Machine Learning Toolkit: 5.3.3, 5.4.0, 5.4.1, 5.4.2, 5.5.0

Acrobat logo Download manual
Acrobat logo Download this page

Was this topic useful?







You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters