All DSP releases prior to DSP 1.4.0 use Gravity, a Kubernetes orchestrator, which has been announced end-of-life. We have replaced Gravity with an alternative component in DSP 1.4.0. Therefore, we will no longer provide support for versions of DSP prior to DSP 1.4.0 after July 1, 2023. We advise all of our customers to upgrade to DSP 1.4.0 in order to continue to receive full product support from Splunk.
Map
The following topic contains the functions that you can use to modify or return maps, as well as information about how to use dot notation to access map elements.
Accessing map elements using dot notation
You can choose to use dot notation instead of calling the map_get scalar function to access elements from a map. The syntax for dot notation is <map-name>.<element-name>
, where <map-name>
is the name of the map and <element-name>
is the name of the element.
For example, assume you're streaming a record that contains the following map field named zoo
:
"zoo":{"mammals":"tiger"}
To access the tiger
element in this map, you can use the dot notation zoo.mammals
. The following Eval function extracts the tiger
value from the map and returns it in a top-level field named species
:
...| eval species = zoo.mammals;
You can also access elements in maps that are nested in other maps by prepending the names of as many parent-level maps as needed, using syntax like <parent-map-name>.<nested-map-name>.<element-name>
.
As another example, assume that the zoo
map field is nested in a z
map field:
"z":"zoo":{"mammals":"tiger"}
To access the tiger
element in this nested map, you can use the dot notation z.zoo.mammals
, as shown in the following Eval function:
...| eval species = z.zoo.mammals;
Currently, dot notation can only be used to access elements in maps. You cannot use dot notation to assign values to elements. For example, ... | eval zoo.mammals = cheetah;
results in an error. If you want to assign values to elements, use the map_set
function instead.
Accessing nested elements using bracket notation and dot notation
You can use bracket notation in combination with the dot notation for accessing map elements. See Accessing list elements using bracket notation for more information about dot notation. Using dot and bracket notation, you can simplify the SPL2 expression for accessing elements in nested lists and maps.
For example, assume you're streaming a record that contains the following list named mammals
, which is nested in a zoo
map, which is further nested in the z
map:
"z":{"zoo":{"mammals":["elephant","tiger","lion"]}}
To extract the tiger
value from the list and return it as a top-level field named species
, you can write the following Eval function using dot and bracket notation:
...| eval species = z.zoo.mammals[1];
To achieve the same results using scalar functions instead of dot and bracket notation, you would need to write an Eval function with multiple expressions:
...| eval zoo=map_get(z, "zoo"), mammals=map_get(zoo, "mammals"), species=mvindex(mammals, 1);
contains_key(input, key)
Checks a map for a specified key.
- Function Input
- input: map<string, T>
- key: string
- Function Output
- boolean
SPL2 examples
Checks the inputted map to see if the key "foo" exists.
When working in the SPL View, you can write the function by using the following syntax.
...| eval n=contains_key(from_json_object("{\"foo\": [{\"bar\": \"baz\"}]}"), "foo");
Alternatively, you can use named arguments to list the arguments in any order.
...| eval n=contains_key(key: "foo", input: from_json_object("{\"foo\": [{\"bar\": \"baz\"}]}"));
create_map(keys_and_values)
Creates a new map object at runtime. Returns a map of key-value pairs. Use create_map
if you want to create a map whose keys are determined by the output of another scalar function. See the SPL2 example.
- Function Input
- keys_and_values: collection<expression<any>>
- Function Output
- map<string, T>
SPL2 examples
Creates a map with the value of host
in uppercase and value, for example: {"MYHOST":"value"}
.
When working in the SPL View, you can write the function by using the following syntax.
...| eval my_map=create_map([upper(host), "value"]) |...;
Alternatively, you can use named arguments.
...| eval my_map=create_map(keys_and_values: [upper(host), "value"]) |...;
flatten(input, separator)
Flattens a list or a map. If used to flatten a map, takes an optional second argument (Y) as a delimiter. Flattening fields with nested values can simplify SPL2 syntax and field extraction.
This function flattens either a list or a map. If you pass a list to flatten that is a list of nested maps, only the list will be flattened, not the maps. Similarly, if you pass in a map with nested lists, the map is flattened but the lists are not. See the fourth row in the table for an example.
- Function Input
- input: collection<any> or map<string,any>
- This function accepts one of two different data types. It either accepts a collection of lists, where the list can be of any type, including a list of lists. Or, it accepts maps, where the data type of the map is <string, any>.
- separator: string
- If you are flattening a map, you can optionally pass in a delimiter to separate the keys in a map.
- Function Output
- collection<any> if a list was passed in, where all nested lists are flattened to a single list.
- map<string,any> if a map was passed in, where all nested maps are flattened to a single top-level map.
SPL2 examples
The following table shows the original nested list or map, the flattened version of that list or map, and an accompanying SPL2 example. When working in the SPL View, you can write the function by using the syntax shown in the "SPL2 example" column.
Incoming, nested data | Outgoing, flattened data | SPL2 example | Notes |
---|---|---|---|
[1, null, "foo", ["1-deep", ["2-deep"]], [], 100]
|
[1, null, "foo", "1-deep", "2-deep", 100]
|
eval flattened_list= flatten(list_field); | Returns the flattened list in a new top-level field called flattened_list .
|
{"baz": {"foo": 1, "bar": "thing"}, "quux": 3}
|
{"quux":3,"baz.foo":1,"baz.bar":"thing"}
|
eval flattened_map=flatten(map_field); | Returns the flattened map in a new top-level field called flattened_map .
|
{"baz": {"foo": 1, "bar": "thing"}, "quux": 3}
|
{"quux":3,"baz::bar":"thing","baz::foo":1}
|
eval flattened_map=flatten({"baz": {"foo": 1, "bar": "thing"}, "quux": 3}, "::"); | Returns the flattened map in a new top-level field called flattened_map . Also, delimits the keys in the map with :: .
|
[[1, 2, 3], [{"key1": {"innerkey1": "innerval1"}}]]
|
[1,2,3,{"key1":{"innerkey1":"innerval1"}}]
|
eval flattened_map=flatten([[1, 2, 3], [{"key1": {"innerkey1": "innerval1"}}]]); | Returns the flattened lists in a new top-level field called flattened_list_with_nested_map . Does not flatten the nested maps that are included in the original list.
|
Alternatively, you can also use named arguments to list the arguments in any order.
...| eval flattened_map=flatten(separator: "::", input: {"baz": {"foo": 1, "bar": "thing"}, "quux": 3});
length(input)
Returns the character length of the provided input. The input can be a map, collection, bytes, or a string.
- Function Input
- input: any
- Function Output
- integer
SPL2 example
Returns 3.
When working in the SPL View, you can write the function by using the following syntax.
...| eval n=length({"1": "bar", "2": "baz", "3": "foo"});
Alternatively, you can use named arguments.
...| eval n=length(input: {"1": "bar", "2": "baz", "3": "foo"});
map_delete(input, key)
Deletes a given key from a map.
- Function Input
- input: map<string,T>
- keys: collection<string>
- Function Output
- map<string,T>
SPL2 example
If your data has a field called defaults
containing the following map, {"name":"read_throughput","unit":"MBps","type":"metrics","value":null}
, then remove the key-value pairs unit
and value
from the data and output the new map {"name":"read_throughput","type":"metrics"}
in a newmap
field.
When working in the SPL View, you can write the function by using the following syntax.
... | eval newmap=map_delete(defaults, ["unit", "value"]);
Alternatively, you can also use named arguments to list the arguments in any order.
... | eval newmap=map_delete(keys: ["unit", "value"], input: defaults);
map_get(input, key)
Returns the value corresponding to a key in the map input.
As an alternative, you can use dot notation to return the value of a given key in a map. See the Accessing map elements using dot notation section for more information.
- Function Input
- input: map<string, T>
- This function inputs a map between a key and a value of any specific data type, T.
- key: string
- Function Output
- type: T
This function outputs the value, which can be of any specific data type T, of the inputted key.
SPL2 examples
Returns the value of the key "key" from the attributes field map. Converts that value to a string and outputs the result as a single field.
When working in the SPL View, you can write the function by using the following syntax.
... | select to_string(get(map_get(attributes, "key"), "bytes")) AS stringified;
Alternatively, you can also use named arguments to list the arguments in any order.
... | select to_string(value: get(map_get(attributes, "key"), "bytes")) AS stringified;
map_keys(input)
Returns a list of keys in a map.
- Function Input
- input: map<string, any>
- Function Output
- collection<string>
SPL2 examples
Outputs a list of keys from the attributes
field in a new top-level field called keys
.
When working in the SPL View, you can write the function by using the following syntax.
...| eval keys=map_keys(attributes);
Alternatively, you can also use named arguments.
...| eval keys=map_keys(input: attributes);
map_merge(maps, mode)
This function takes two or more maps and merges them together to return a single map using a specified merge mode. The available merge modes are left, top, or recursive.
- Function Input
- maps: collection<map<string, any>>
- mode: string (one of "left", "top", or "recursive")
- Function Output
- map<string, any>
Argument | Input | Description |
---|---|---|
maps | collection<map<string, any>> | This function accepts a collection of maps where the data type of the map is <string, any>. |
mode | left, top, or recursive | Specifies the merging behavior for the incoming maps. Defaults to left. See the SPL2 examples section to see the different ways to use the map_merge function to merge two or more maps.
|
1. SPL2 example
Merges two maps using left-hand precedence. Returns {"a" : [17, 33]}
.
When working in the SPL View, you can write the function by using the following syntax.
... | eval n = map_merge([{"a" : [17, 33]}, {"a" : [100]}], "left") |...;
2. SPL2 example
Merges only the top-level fields of two maps. Returns {"a":[{"abc":123},{"abc":456,"xyz":890}]}
.
When working in the SPL View, you can write the function by using the following syntax.
... | eval n = map_merge([{"a" : {"abc" : 123 } }, {"a" : { "abc" : 456 , "xyz" : 890 } }], "top") |...;
3. SPL2 example
Merges two maps, including all nested fields within the maps. Returns {"a":{"abc":[123,456],"xyz":890}}
.
When working in the SPL View, you can write the function by using the following syntax.
... | eval n = map_merge([{"a" : {"abc" : 123 } }, {"a" : {"abc" : 456 , "xyz" : 890 } }], "recursive") |...;
4. SPL2 example
In the following example, the fields m1
and m2
both contain a map. This example uses the map_merge
function to merge the two fields together to return a single map in a new field called n1
. Assume that m1
contains {"a":{"abc":[123,456],"xyz":890}}
and m2
contains {"a":[17,33]}
, then n1
returns {"a":[17,33,{"abc":[123,456],"xyz":890}]}.
When working in the SPL View, you can write the function by using the following syntax.
... | eval n1 = map_merge([m1, m2], "top") |...;
5. SPL2 example
Alternatively, you can use named arguments to list the arguments in any order.
... | eval n1 = map_merge(mode: "top", maps: [m1, m2]) |...;
map_set(map, keys_and_values)
Insert new key-value pairs or overwrite existing key-value pairs in a map. Returns an updated map.
- Function Input
- map: map<K,V>
- keys_and_values: collection<expression<any>>
- Function Output
- map<K,V>
SPL2 examples
This example adds the key-value pair "some_key": "some_value"
to the attributes
map.
When working in the SPL View, you can write the function by using the following syntax.
...| eval attributes=map_set(attributes, "some_key", "some_value") |...;
Alternatively, you can also use named arguments to list the arguments in any order.
...| eval attributes=map_set(keys_and_values: ["some_key", "some_value"], map: attributes) |...;
map_values(input)
Returns a list of values in a map.
- Function Input
- map: map<string,T>
- Function Output
- collection<T>
SPL2 example
In this example, if your attributes
field contains a map with key-value pairs {"_splunk_connection_id":"rest_api:all","foo":"bar"}
, then the following example would return the values in the map in a new field called n
: ["rest_api:all","bar"]
.
When working in the SPL View, you can write the function by using the following syntax.
...| eval n=map_values(attributes) |...;
Alternatively, you can also use named arguments.
...| eval n=map_values(input: attributes) |...;
spath(input, path)
This function extracts a value from a map or collection. Use this function to extract a value from a nested map or collection. To extract a value from a JSON string, convert your JSON string to a map. See the from_json_object scalar function and the SPL2 example below.
The Data Stream Processor spath function does not have an auto-extract mode and does not accept JSON or XML strings.
- Function Input
- input: union<collection<any>, map<string,any>>
- path: string
- The location path to the value that you want to extract. If you do not specify the
path=
, the first unlabeled argument is used as the location path. A location path is composed of one or more location steps, separated by periods. An example of this is 'foo.bar.baz'. - Function Output
- union<string, bytes, boolean, integer, long, float, double, collection<any>, map<string,any>>
1. SPL2 example
Returns baz.
When working in the SPL View, you can write the function by using the following syntax.
spath(from_json_object("{\"foo\": [{\"bar\": \"baz\"}]}"), "foo{0}.bar");
2. SPL2 example
Extracts forwarder_channel_id from attributes
.
When working in the SPL View, you can write the function by using the following syntax.
... | eval forwarder_channel_id_spath=spath(attributes, "spl_forwarder_channel_info.forwarder_channel_id");
3. SPL2 example
Alternatively, you can use named arguments to list the arguments in any order.
... | eval forwarder_channel_id_spath=spath(path: "spl_forwarder_channel_info.forwarder_channel_id", input: attributes);
List | Mathematical |
This documentation applies to the following versions of Splunk® Data Stream Processor: 1.2.0, 1.2.1-patch02, 1.2.1, 1.2.2-patch02, 1.2.4, 1.2.5, 1.3.0, 1.3.1, 1.4.0, 1.4.1, 1.4.2, 1.4.3, 1.4.4, 1.4.5
Feedback submitted, thanks!