Related Answers
Download topic as PDF
streamstats
Description
Adds summary statistics to all search results in a streaming manner.
The streamstats command is similar to the eventstats command except that it uses events before a given event to compute the aggregate statistics applied to each event. If you want to include the given event in the stats calculations, use current=true, which is the default.
The streamstats command is also similar to the stats command in that streamstats calculates summary statistics on search results. Unlike stats, which works on the results as a whole, streamstats calculates statistics for each event at the time the event is seen.
Syntax
streamstats [current=<bool>] [window=<int>] [global=<bool>] [allnum=<bool>] <stats-agg-term>... [<by clause>]
Required arguments
- stats-agg-term
- Syntax: <stats-func>( <evaled-field> | <wc-field> ) [AS <wc-field>]
- Description: A statistical aggregation function. See Stats function options. The function can be applied to an eval expression, or to a field or set of fields. Use the AS clause to place the result into a new field with a name that you specify. You can use wild card characters in field names.
Optional arguments
- current
- Syntax: current=<bool>
- Description: If true, tells the search to include the given, or current, event in the summary calculations. If false, tells the search to use the field value from the previous event.
- Default: true
- window
- Syntax: window=<int>
- Description: The window option specifies the number of events to use when computing the statistics.
- Default: 0, which means that all previous (plus current) events are used.
- global
- Syntax: global=<bool>
- Description: Defines whether the window is global or for each field in the by clause. If
global=falseandwindowis set to a non-zero value, a separate window is used for each group of values of the group by fields. - Default: true
- allnum
- Syntax: allnum=<bool>
- Description: If true, computes numerical statistics on each field if and only if all of the values of that field are numerical.
- Default: false
- by clause
- Syntax: BY <field-list>
- Description: The name of one or more fields to group by.
Stats function options
- stats-func
- Syntax: The syntax depends on the function you use. Refer to the table below.
- Description: Statistical functions that you can use with the
streamstatscommand. Each time you invoke thestreamstatscommand, you can use one or more functions. However, you can only use oneBYclause. See Usage.
- The following table lists the supported functions by type of function. For descriptions and examples, see Statistical and charting functions.
Type of function Supported functions and syntax Aggregate functions avg()
count()
distinct_count()
estdc()
estdc_error()
max()
median()
min()
mode()
perc<int>
range()
stdev()
stdevp()
sum()
sumsq()
var()
varp()
Event order functions earliest()
first()
last()
latest()
Multivalue stats and chart functions list(X)
values(X)
Usage
Functions and memory usage
Some functions are inherently more expensive, from a memory standpoint, than other functions. For example, the distinct_count function requires far more memory than the count function. The values and list functions also can consume a lot of memory.
If you are using the distinct_count function without a split-by field or with a low-cardinality split-by by field, consider replacing the distinct_count function with the the estdc function (estimated distinct count). The estdc function might result in significantly lower memory usage and run times.
Examples
Example 1
Each day you track unique users, and you would like to track the cumulative count of distinct users. This example calculates the running total of distinct users over time.
eventtype="download" | bin _time span=1d as day | stats values(clientip) as ips dc(clientip) by day | streamstats dc(ips) as "Cumulative total"
The bin command breaks the time into days. The stats command calculates the distinct users (clientip) and user count per day. The streamstats command finds the running distinct count of users.
This search returns a table that includes: day, ips, dc(clientip), and Cumulative total.
Example 2
This example uses streamstats to produce hourly cumulative totals for category values.
... | timechart span=1h sum(value) as total by category | streamstats global=f sum(total) as accu_total
The timechart command buckets the events into spans of 1 hour and counts the total values for each category. The timechart command also fills NULL values, so that there are no missing values. Then, the streamstats command is used to calculate the accumulated total.
Example 3
This example uses streamstats to figure out when a DHCP IP lease address changed for a MAC address, 54:00:00:00:00:00.
source=dhcp MAC=54:00:00:00:00:00 | head 10 | streamstats current=f last(DHCP_IP) as new_dhcp_ip last(_time) as time_of_change by MAC
You can also clean up the presentation to display a table of the DHCP IP address changes and the times the occurred.
source=dhcp MAC=54:00:00:00:00:00 | head 10 | streamstats current=f last(DHCP_IP) as new_dhcp_ip last(_time) as time_of_change by MAC | where DHCP_IP!=new_dhcp_ip | convert ctime(time_of_change) as time_of_change | rename DHCP_IP as old_dhcp_ip | table time_of_change, MAC, old_dhcp_ip, new_dhcp_ip
For more details, refer to the Splunk Blogs post for this example.
More examples
Example 1:
Compute the average value of foo for each value of bar including only 5 events, specified by the window size, with that value of bar.
... | streamstats avg(foo) by bar window=5 global=f
Example 2:
For each event, compute the average of field foo over the last 5 events, including the current event. Similar to doing trendline sma5(foo)
... | streamstats avg(foo) window=5
Example 3:
This example adds to each event a count field that represents the number of events seen so far, including that event. For example, it adds 1 for the first event, 2 for the second event, and so on.
... | streamstats count
If you did not want to include the current event, you would specify:
... | streamstats count current=f
See also
accum, autoregress, delta, fillnull, eventstats, trendline
Answers
Have questions? Visit Splunk Answers and see what questions and answers the Splunk community has using the streamstats command.
|
PREVIOUS strcat |
NEXT table |
This documentation applies to the following versions of Splunk® Enterprise: 4.3, 4.3.1, 4.3.2, 4.3.3, 4.3.4, 4.3.5, 4.3.6, 4.3.7, 5.0, 5.0.1, 5.0.2, 5.0.3, 5.0.4, 5.0.5, 5.0.6, 5.0.7, 5.0.8, 5.0.9, 5.0.10, 5.0.11, 5.0.12, 5.0.13, 5.0.14, 5.0.15, 5.0.16, 5.0.17, 5.0.18, 6.0, 6.0.1, 6.0.2, 6.0.3, 6.0.4, 6.0.5, 6.0.6, 6.0.7, 6.0.8, 6.0.9, 6.0.10, 6.0.11, 6.0.12, 6.0.13, 6.0.14, 6.0.15, 6.1, 6.1.1, 6.1.2, 6.1.3, 6.1.4, 6.1.5, 6.1.6, 6.1.7, 6.1.8, 6.1.9, 6.1.10, 6.1.11, 6.1.12, 6.1.13, 6.1.14, 6.2.0, 6.2.1, 6.2.2, 6.2.3, 6.2.4, 6.2.5, 6.2.6, 6.2.7, 6.2.8, 6.2.9, 6.2.10, 6.2.11, 6.2.12, 6.2.13, 6.2.14, 6.2.15, 6.3.0, 6.3.1, 6.3.2, 6.3.3, 6.3.4, 6.3.5, 6.3.6, 6.3.7, 6.3.8, 6.3.9, 6.3.10, 6.3.11, 6.3.12, 6.3.13, 6.3.14
Comments
I second Lalleman's question. The answer would clarify the mechanic of the global option.
When would you want to group "by" a field and yet use "global=t" (which is the default). I'm not sure what happens in this case. (Or should you always use "global=f" whenever you have a "by")
The example 2 may be incorrect here. There is not field "total" after the timechart, so, the streamstats would fail to do anything. This will only work if one of the values of the category is "total".