Splunk® Supported Add-ons

Splunk Add-on for Microsoft Cloud Services

Acrobat logo Download manual as PDF


Acrobat logo Download topic as PDF

Performance reference for the Azure Storage Blob input in the Splunk Add-on for Microsoft Cloud Services

The following is reference information about Splunk's performance testing of the Azure Storage Blob input in the Splunk Add-on for Microsoft Cloud Services. The testing occurred with version 4.3.2 of the Splunk Add-on for Microsoft Cloud Services. Use this information to enhance the performance of your own Azure Storage Blob data collection tasks.

Many factors impact performance results, including file size, file compression, event size, deployment architecture, and hardware. These results represent reference information and do not represent performance in all environments.

Blob input CO2 performance characteristics

Block blob

Common Architecture Setup Blob Type Event Size Ingest Rate Events per minute Data lag IDM CPU Cores % (Percentage of total IDM cores)

Scaled up co2 stack

  • 20 containers
  • 20 add-on inputs
  • 1 cluster master (c5.xlarge)
  • 1 search head (c5.2xlarge)
  • 3 indexer cluster (i3.large)
    • 32 CPU cores
    • 244 GB memory
  • 1 input data manager (c5.9xlarge)
    • 32 CPU cores
    • 72 GB memory
  • 20 Splunk Add-on for Microsoft Cloud Services inputs
  • 120 seconds collection interval
block blob 10kb 48GB/day 3522 ~120 seconds 1.8%
1MB 50GB/day 36 ~121 seconds 1.84%



Append blob

Common Architecture Setup Blob Type Event Size Ingest Rate Events per minute Data lag IDM CPU Cores % (Percentage of total IDM cores)
Scaled up co2 stack
  • 20 containers
  • 20 add-on inputs
  • 1 cluster master (c5.xlarge)
  • 1 search head (c5.2xlarge)
  • 3 indexer cluster (i3.large)
    • 32 CPU cores
    • 244 GB memory
  • 1 input data manager (c5.9xlarge)
    • 32 CPU cores
    • 72 GB memory
  • 20 Splunk Add-on for Microsoft Cloud Services inputs
  • 120 seconds collection interval
append blob 1 kb 33GB/day 21522 ~120 seconds 3.7%
10 kb 281GB/day 20496 ~121 seconds 2.67%
100 kb 2.69TB/day 18402 ~129 seconds 6.5%


Additional Blob input scenarios

Scenario Workflow Result
Append file up to 4GB
  • A single user creates an append blob.
  • Append 100 kb each time.
  • Append 40000 times.
  • Pause up to 100 ms in each append.
  • All 40000 append blocks pulled back.
  • Total event size matches original file size in Azure.
  • Minimal to no data lag.
  • Collection interval is 12 seconds.
Append 3 files sequentially to 100 MB
  • A single user creates an append blob.
  • 100000 append each time.
  • Pause up to 100 ms after each append.
  • Append 1000 times to get to 100 MB for the file.
  • Move on to next file and repeat.
  • In total, append 3 files to 100 MB sequentially.
  • All 3000 append blocks pulled back.
  • Total event size matches original file size in Azure.
Last modified on 01 December, 2021
PREVIOUS
Performance reference for the Azure Storage Table input in the Splunk Add-on for Microsoft Cloud Services
  NEXT
APIs used in the Splunk Add-on for Microsoft Cloud Services

This documentation applies to the following versions of Splunk® Supported Add-ons: released


Was this documentation topic helpful?


You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters