
Performance reference for the Azure Storage Blob input in the Splunk Add-on for Microsoft Cloud Services
The following tables show reference information about performance testing of the Azure Storage Blob input in the Splunk Add-on for Microsoft Cloud Services. Use this information to enhance the performance of your own Azure Storage Blob data collection tasks.
Many factors impact performance results, including file size, file compression, event size, deployment architecture, and hardware. These results represent reference information and do not represent performance in all environments.
Version 5.0.0 Blob input performance characteristics
Common architecture setup | Scenario | Blob type | Event size | Ingest rate | IDM CPU |
---|---|---|---|---|---|
CO2 Stack - Victoria Search Head Cluster
| |||||
1 input | AppendBlob | 1MB | 8155.26 kb/s | N/A | |
5 inputs | AppendBlob | 1MB | 21390.76 kb/s | N/A | |
1 input | BlockBlob | 1MB | 13100.21 kb/s | N/A | |
5 inputs | BlockBlob | 1MB | 24328.95 kb/s | N/A | |
CO2 Stack - Classic Cluster (1 IDM)
| |||||
1 input | AppendBlob | 1MB | 6850.66 kb/s | 55.85% | |
5 inputs | AppendBlob | 1MB | 8049.20 kb/s | 33.35% | |
1 input | BlockBlob | 1MB | 3808.87 kb/s | 44.55% | |
5 inputs | BlockBlob | 1MB | 3838.31 kb/s | 32.00% |
Continuous append scenario | |||
---|---|---|---|
Append 6 files sequentially to 1GB. |
|
|
Version 4.1.4 Blob input performance characteristics
Block blob
Common architecture setup | Blob type | Event size | Ingest rate | Events per minute | Data lag | IDM CPU Cores % (Percentage of total IDM cores) |
---|---|---|---|---|---|---|
Scaled up stack
|
block blob | 10kb | 48GB/day | 3522 | ~120 seconds | 1.8% |
1MB | 50GB/day | 36 | ~121 seconds | 1.84% |
Append blob
Common architecture setup | Blob type | Event size | Ingest rate | Events per minute | Data lag | IDM CPU Cores % (Percentage of total IDM cores) |
---|---|---|---|---|---|---|
Scaled up stack
|
append blob | 1 kb | 33GB/day | 21522 | ~120 seconds | 3.7% |
10 kb | 281GB/day | 20496 | ~121 seconds | 2.67% | ||
100 kb | 2.69TB/day | 18402 | ~129 seconds | 6.5% |
Additional Blob input scenarios
Scenario | Workflow | Result |
---|---|---|
Append file up to 4GB |
|
|
Append 3 files sequentially to 100 MB |
|
|
PREVIOUS Performance reference for the Azure Storage Table input in the Splunk Add-on for Microsoft Cloud Services |
NEXT APIs used in the Splunk Add-on for Microsoft Cloud Services |
This documentation applies to the following versions of Splunk® Supported Add-ons: released
Feedback submitted, thanks!