Splunk® Supported Add-ons

Splunk Add-on for Microsoft Cloud Services

Acrobat logo Download manual as PDF


Acrobat logo Download topic as PDF

Performance reference for the Azure Event Hub input in the Splunk Add-on for Microsoft Cloud Services

The following is reference information about Splunk's performance testing of the Azure Event Hub input in the Splunk Add-on for Microsoft Cloud Services. The testing occurred with version 4.3.2. Use this information to enhance the performance of your own Azure Event Hub data collection tasks.

Many factors impact performance results, including file size, file compression, event size, deployment architecture, batch size for event hub file size, and hardware. Results represent reference information and do not represent performance in all environments.

Event hub input CO2 performance characteristics

Common Setup Event Type Event Size Scenario Ingest Rate Data lag IDM CPU Index CPU Bottleneck
  • 2 add-on inputs
  • 2 Event hub partitions
  • Default co2 stack
    • 1 cluster master (c5.xlarge)
    • 1 search head (c5.2xlarge)
    • 3 indexer cluster (i3.large)
    • 1 Inputs Data Manager (IDM) (r5.xlarge)
  • default TA configs
  • 1 IDM (4 cores, 32 GB memory)
JSON 1 KB Sweet Spot 187 GB/Day (2200 eps) 4 seconds 50-89% 86-92% IDX CPU utilized
Non-JSON (mcas-cef) 994 bytes Sweet Spot 133 GB/day (2000 eps) 8 seconds 65-94% 99% IDX CPU utilized

Event hub CO2 Scale Up performance characteristics

Environment setup

Cluster setup Event Hub namespace Event Hub Add-on inputs Splunk software configurations
Scaled up co2 stack
  • 1 cluster master (c5.xlarge)
  • 1 search head (c5.9xlarge)
  • 3 indexer cluster (i3.8xlarge)
    • 32 CPU cores
    • 244 GB memory
  • 2 Input Data Manager (c5.9xlarge)
    • 36 CPU cores
    • 72 GB memory
  • 2 event hub namespaces
  • 40 throughput units (TU) on each namespace
  • 80 throughput units (TU) in total (2*40)
  • 2 event hubs per namespace
  • 4 event hubs in total (2*2)
  • 20 partitions per event hub
  • 80 partitions in total (4*20)
  • 80 inputs in total (2*40)
  • 40 inputs per add-on
  • 2 add-on installed on Inputs Data Manager
  • default add-on configurations
  • max wait 10s
  • batch size 300
  • 1 add-on pulls from 1 namespace only
  • 1 add-on pulls from 2 event hubs on same namespace
  • 40 inputs on each add-on
  • 20 inputs on each event hub (20 partitions)
Inputs Data Manager and indexer configuration


  • parallelIngestionPipelines = 8

Scale up result summary

Event Type Number of inputs Event Size Scenario Ingest Rate Ingest Events per second Max Index Rate Data lag in seconds Inputs Data Manager (IDM) CPU % IDM CPU Cores % (Percentage of total IDM cores) Indexer CPU %
JSON 80 1 Kb Sweet Spot 6.106 TB/day 78K 7.55 TB/day 4s 51% 36% 30.5%
JSON 10 1 Kb Sweet Spot 1.764 TB/day 20.8K 2.19 TB/day 9s 24% 8.6% 10%
Non JSON 80 0.998 Kb Sweet Spot 5.555 TB/day 83.2K 7.22 TB/day 4s 67% 48% 47%
Non JSON 10 0.998 Kb Sweet Spot 1.595 TB/day 24K 2.07 TB/day 3s 29% 10.4% 11%
Last modified on 01 December, 2021
PREVIOUS
Lookups for the Splunk Add-on for Microsoft Cloud Services
  NEXT
Performance reference for the Azure Storage Table input in the Splunk Add-on for Microsoft Cloud Services

This documentation applies to the following versions of Splunk® Supported Add-ons: released


Was this documentation topic helpful?


You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters