Splunk® Enterprise

Metrics

Download manual as PDF

Download topic as PDF

Metrics indexing performance

This topic summarizes the results of metrics indexing performance.

Size on disk

When ingesting typical metrics payloads with supported metrics source types (collectd_http, statsd, metrics_csv), a metrics index requires about 50% less disk storage space compared to storing the same payload in an events index.

Throughput

Consider the following when deciding whether to scale horizontally by adding additional indexers.
Using the collectd_http source type with an HTTP Event Collector (HEC) input, testing reached a constant of around 55,000 events per second maximum ingestion throughput, and around 58,000 events per second without additional search load.
  • The default batch size was 5,000 events per batch. A significant difference in ingestion performance was not observed between batch sizes of 100 to 5,000 events.
  • The keep-alive setting was enabled for these tests.
  • A typical event size was about 214 bytes.
Using the statsd source type with a UDP input, throughput was highly variable depending on other network activity. For UDP inputs we recommend using a universal forwarder as close as possible to where metrics are collected.

Speed

Consider the results from the following test for running metrics queries. This test used metrics from 1,000 hosts, with a total event count of 6 billion events in the metrics index, where queries were representative and did not use wildcards in metric_name.
Time range Events Query speed
1 hour 35 million < 0.1s
1 day 850 million ~3-5s
1 week 6 billion ~20-22s


See the Capacity Planning Manual.

PREVIOUS
Investigate counter metrics
  NEXT
Best practices for metrics

This documentation applies to the following versions of Splunk® Enterprise: 7.0.0, 7.0.1, 7.0.2, 7.0.3, 7.0.4, 7.0.5, 7.0.6, 7.0.7, 7.0.8, 7.0.9, 7.0.10, 7.0.11, 7.1.0, 7.1.1, 7.1.2, 7.1.3, 7.1.4, 7.1.5, 7.1.6, 7.1.7, 7.1.8, 7.1.9, 7.2.0, 7.2.1, 7.2.2, 7.2.3, 7.2.4, 7.2.5, 7.2.6, 7.2.7, 7.2.8, 7.3.0, 7.3.1, 7.3.2


Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters