Custom eval functions

You can create your own custom eval functions to extend SPL2. Custom functions are user-defined functions that you declare in an SPL2 module. Custom eval functions have zero or more parameters and return a single value.

You can use custom eval functions in the WHERE clause of the from command and in the eval and where commands.

Custom functions provide a structured way to share and reuse blocks of SPL2. Custom functions are similar to macros, but they have more capabilities than macros, such as data type checking and advanced optimizations.

For information about the built-in eval functions, see Quick Reference for SPL2 eval functions.

Custom eval function syntax

The required syntax is in bold.

function <function-name>

( [ $<parameter-name> [: <parameter-type>] [=<default-value>] ], ...) [: <function-type> ]

{

[ <SPL-statement>... ]

return <expression>

}

The parenthesis are required, even if a <function-parameter> isn't specified.

The curly braces { } are required, even if a <SPL-statement> isn't specified.

Required arguments

function-name: Syntax: function <function-name>; Description: The word function followed by the name that you want to give to the function. Function names must start with a letter or underscore ( _ ) character and cannot contain spaces.

expression: Syntax: return <expression>; Description: The word return followed by the expression that you want returned from the function. Expressions can be composed of literals, functions, fields, parameters, comparisons, and other expressions. See Types of expressions in the SPL2 Search Manual.

Optional arguments

parameter-name: Syntax: $<parameter-name>; Description: The name that you want to give to a parameter that you want to use with a custom function. You can specify zero or more parameters. Separate multiple parameters with commas. Parameter names must start with a dollar sign ( $ ) followed by a letter or an underscore ( _ ) character and cannot contain spaces. When you specify a parameter name, you have the option to include a parameter type. The default parameter type is any.; Example: ($firstname, $surname: string); Default: None

parameter-type: Syntax: <parameter-type>; Description: The data type that the parameter accepts. This is the input data type. The default data type is any. See Built-in data types. If you specify a parameter type, you must separate the <parameter-name> and the <parameter-type> with a colon ( : ).; Example: ($ipaddress: string, $kbps: int); Default: any

default-value: Syntax: <default-value>; Description: A default value for the parameter. The value must be a constant, either a literal or a string. The value can't be a field name. Users can override the default value by specifying the parameter with a value.; Example: ($count: int=5); Default: None

function-type: Syntax: <function-type>; Description: The data type that the function returns. This is the output data type. The default data type is any. See Built-in data types.; Default: any

SPL-statement: Syntax: <SPL-statement>...; Description: One or more SPL2 statements to include in your custom function. See Examples.; Default: None

Using custom eval functions

Consider this custom eval function example:

function hello ($fullname: string): string { 
    return  "Howdy" + " " + $fullname 
}

The following table describes each part of this function example:

Syntax	Example	Description
function <function_name>	function hello	The name of the function is `hello`.
$<parameter-name>: <parameter-type>	$fullname: string	One parameter is defined, with the name `fullname`. The value that you specify for the parameter must be an expression that resolves to a string, such as a field that contains string values or a string literal.
<function-type>	string	The function returns a string.
{ <SPL-statements> return <expression> }	{ return "Howdy" + " " + $fullname }	There is no SPL-statement. The expression returned is the string `Howdy` , a space, and the value specified for the parameter `$fullname`.

Using functions in modules

You create custom eval functions in an SPL2 module. You can use a custom function multiple times within that module. Custom functions can also be imported into other modules.

Using functions in the API

You can also create custom functions using the Search Service API.

Supported function types

When you create a custom eval function, you have the option to specify the data type for the function or the function parameters. The built-in data types include strings, numbers, Booleans, arrays, objects, and times.

If you don't specify a data type, the default data type any is used.

The dataset data type isn't applicable to custom eval functions. You use the dataset data type with a custom command function. See Custom command functions.

To learn more about the supported data types, see Built-in data types in the SPL2 Search Reference.

Custom function limitations

An SPL2 custom function cannot reference a search statement that is defined outside of the SPL2 function.

In the following example, the search $aws_security_cloudtrail_sourcetype is referenced in the function aws_security_cloudtrail.

$aws_security_cloudtrail_sourcetype = 
from $aws_security_cloudtrail_index 
where sourcetype="aws:cloudtrail"

function aws_security_cloudtrail($accountId, $region) {
    return from $aws_security_cloudtrail_sourcetype 
   | search $accountId $region
}

The following is an example of the error message that is returned when a function references a search that is outside of the function.

Error: Invalid function declaration: Unknown variable 
'$aws_security_cloudtrail_sourcetype' referenced 
in the function 'aws_security_cloudtrail'.

Testing custom functions

Testing custom functions makes it easier to verify that the function works as you intend.

You can test custom functions in isolation, without impacting other SPL2 statements, by using dataset literals. There are multiple sample dataset literals that you can use to test your functions. See Sample dataset literals in the SPL2 Search Manual.

Consider the following dataset:

name	age	city
Wei Zhang	39	Seattle
Rutherford Sullivan	57	Dublin
Vanya Patel	47	Bangalore

This dataset will be used to test the following function. This function, called hello, returns the string "Howdy", followed by a space and the values from the expression specified for the fullname parameter.

function hello ($fullname: string): string { 
    return  "Howdy" + " " + $fullname 
}

The number of events in the dataset literal that you use for testing depends on how complex your function is. In simple examples, you could use one event. This example uses three.

This example invokes the function by using the eval command. This example creates a new field called greeting to store the results of the function:

$search = | from [ {name: "Wei Zhang", age: 39, city: "Seattle"}, {name: "Rutherford Sullivan", age:57, city: "Dublin"}, {name: "Vanya Patel", age: 47, city: "Bangalore"}] | eval greeting = hello(name)

These are results of invoking this function:

name	age	city	greeting
Wei Zhang	39	Seattle	Howdy Wei Zhang
Rutherford Sullivan	57	Dublin	Howdy Rutherford Sullivan
Vanya Patel	47	Bangalore	Howdy Vanya Patel

Examples

1. A function with a simple return expression

This example shows how to use a custom function with a simple expression.

This example returns error codes that are greater than or equal to 400. The custom function is called isError and returns a Boolean value. A parameter is defined, called $code, which has an input data type of number. There is no SPL statement in this example. The expression for this function returns results only when the value of the $code parameter is greater than or equal to 400.

function isError($code : number) : boolean {
  return $code >= 400 
}

This example invokes the function in the WHERE clause of the from command:

| FROM jsondata WHERE method="GET" AND isError(status_code)

In this example, the isError function is used as a filter in the WHERE clause of a from command. The status_code field is the value used for the $code parameter.

As the events from the jsondata dataset are read, the events are filtered by the WHERE clause. Only the events that have a method field with a value of GET and have an error status code of 400 or greater are returned.

Suppose the following events are in the jsondata dataset:

_time	method	status_code
22 Mar 2022 12:00:00 PM	PUT	200
22 Mar 2022 11:45:00 AM	GET	400
22 Mar 2022 11:13:00 AM	GET	200
22 Mar 2022 10:59:00 AM	POST	408
22 Mar 2022 10:38:00 AM	GET	404

When the search is run with the custom eval function, the following results are returned:

_time	method	status_code
22 Mar 2022 11:45:00 AM	GET	400
22 Mar 2022 10:38:00 AM	GET	404

2. Using and overriding a default value

This example shows how to specify a default value in a custom function and override that default value.

Suppose you want to create a custom function that combines the built-in if and round functions. You want to take the values in a field and round the values to a specific number of decimal points, but only if the number is greater than 1.

Start by creating some sample data. You can use a dataset literal like this:

$sample = FROM [{score: 2.345678},{score: 0.1234},{score: 1.0076543}]

Next, combine the functions that you want to use explicitly. For example, if you want to apply the function to the score field, this would be the function expression:

if(score>1, round(score, 2), score)

Then test your explicit function expression against the sample data:

$function_test1 = FROM $sample SELECT score, if(score>1, round(score, 2), score)  AS newscore

You can convert the explicit function expression into a custom function, which includes a default value for $precision:

function roundIf($field:int, $precision:number=2): int {
  return if($field>1, round($field, $precision), $field) 
}

Now you're ready to test the function. Notice this search doesn't specify a precision. It uses the default $precision in the function.

$function_test2 = FROM $sample SELECT score, roundIf(score) AS newscore

You can override the default $precision by specifying a different precision in your search. For example:

$function_test2 = FROM $sample SELECT score, roundIf(score, 4) AS newscore

3. Reusing a custom function

You can use your custom eval functions in multiple searches within the same SPL module. To use a custom function in another SPL module, you must export the custom function.

This custom function, isError, is identical to the custom function described in the previous example.

This custom function is used in three different searches:

function isError($code : number) : boolean {
  return $code >= 400 
}

| FROM jsondata WHERE method="GET" AND isError(status_code)

| FROM jsondata WHERE isError(status_code)
| stats count(method) BY status_code, method

| FROM main WHERE isError(status_code)
| stats (earliest_time(_time)) AS 'First Instance'
| eval 'First Instance'=strftime('First Instance', "%Y-%m-%d %H:%M:%S")

First search

This search returns events where the method is GET and the values in the status_code field are equal to or greater than 400:

| FROM jsondata WHERE method="GET" AND isError(status_code)

Second search

This search returns events where the values in the status_code field are equal to or greater than 400. The search then counts the values in the method field and organizes the results by the values in the status_code field and the method field.

| FROM jsondata WHERE isError(status_code) | stats count(method) BY status_code, method

The results look something like this:

status_code	method	count(method)
400	GET	1
404	GET	1
408	POST	1

Third search

This search returns events from a different dataset where the values in the status_code field are equal to or greater than 400. The search returns the time from the earliest event in the search results, which is the first occurrence of the 4xx status codes for the time range specified. The field returned is renamed First Instance. Because the earliest_time function returns a UNIX time, the strftime eval function is used to change the time into a human-readable time format.

| FROM main WHERE isError(status_code) |stats (earliest_time(_time)) AS 'First Instance' |eval 'First Instance'=strftime('First Instance', "%Y-%m-%d %H:%M:%S")

The results look something like this:

First instance
2022/03/22 10:38