
Performance reference for the Azure Event Hub input in the Splunk Add-on for Microsoft Cloud Services
The following is reference information about Splunk's performance testing of the Azure Event Hub input in the Splunk Add-on for Microsoft Cloud Services. The testing occurred with version 4.1.5. Use this information to enhance the performance of your own Azure Event Hub data collection tasks.
Many factors impact performance results, including file size, file compression, event size, deployment architecture, batch size for event hub file size, and hardware. Results represent reference information and do not represent performance in all environments.
Event hub input CO2 performance characteristics
Common Setup | Event Type | Event Size | Scenario | Ingest Rate | Data lag | IDM CPU | Index CPU | Bottleneck |
---|---|---|---|---|---|---|---|---|
| ||||||||
JSON | 1 KB | Sweet Spot | 187 GB/Day (2200 eps) | 4 seconds | 50-89% | 86-92% | IDX CPU utilized | |
Non-JSON (mcas-cef) | 994 bytes | Sweet Spot | 133 GB/day (2000 eps) | 8 seconds | 65-94% | 99% | IDX CPU utilized |
Event hub CO2 Scale Up performance characteristics
Environment setup
Cluster setup | Event Hub namespace | Event Hub | Add-on inputs | Splunk software configurations |
---|---|---|---|---|
Scaled up co2 stack
|
|
|
|
Inputs Data Manager and indexer configuration
|
Scale up result summary
Event Type | Number of inputs | Event Size | Scenario | Ingest Rate | Ingest Events per second | Max Index Rate | Data lag in seconds | Inputs Data Manager (IDM) CPU % | IDM CPU Cores % (Percentage of total IDM cores) | Indexer CPU % |
---|---|---|---|---|---|---|---|---|---|---|
JSON | 80 | 1 Kb | Sweet Spot | 6.106 TB/day | 78K | 7.55 TB/day | 4s | 51% | 36% | 30.5% |
JSON | 10 | 1 Kb | Sweet Spot | 1.764 TB/day | 20.8K | 2.19 TB/day | 9s | 24% | 8.6% | 10% |
Non JSON | 80 | 0.998 Kb | Sweet Spot | 5.555 TB/day | 83.2K | 7.22 TB/day | 4s | 67% | 48% | 47% |
Non JSON | 10 | 0.998 Kb | Sweet Spot | 1.595 TB/day | 24K | 2.07 TB/day | 3s | 29% | 10.4% | 11% |
PREVIOUS Lookups for the Splunk Add-on for Microsoft Cloud Services |
NEXT Performance reference for the Azure Storage Table input in the Splunk Add-on for Microsoft Cloud Services |
This documentation applies to the following versions of Splunk® Supported Add-ons: released
Feedback submitted, thanks!