Splunk® Supported Add-ons

Splunk Add-on for Microsoft Cloud Services

Download manual as PDF

Download topic as PDF

Performance reference for the Azure storage input in the Splunk Add-on for Microsoft Cloud Services

The following is reference information about Splunk's performance testing of the Azure storage input in the Splunk Add-on for Microsoft Cloud Services. The testing occurred with version 2.0.0, when the Azure storage input was first introduced. Use this information to enhance the performance of your own Azure storage data collection tasks.

Many factors impact performance results, including file size, file compression, event size, deployment architecture, and hardware. These results represent reference information and do not represent performance in all environments.

Testing Architecture

Splunk tested the performance of the Storage input using a single-instance Splunk Enterprise 6.4.3 on an C4 High-CPU Double Extra Large instance to ensure CPU, memory, storage, and network do not introduce any bottlenecks. Instance specs:

The EC2 in the testing environment is in the same area of Azure storage input, the network latency is low.

Instance type C4 High-CPU Double Extra Large
Memory 15 GB
Compute Units (ECU) 31 Unit
Cores 8
Storage Type GP2(SSD)
Architecture 64-bit
Network performance High
EBS Optimized: Max Bandwidth 1000 Mbps

Test Environment

Deployment Type Role EC2 Type Count
Standalone Deployment Standalone C4 High-CPU Double Extra Large 1

Testing Result

The detailed test result follows.

  • The input number stands for the number of the inputs, one input collects one table.
  • Each table contains 2,131,200 events.
  • Each event is 500 Bytes.
Input Number Throughput (MB/s) Throughput (GB/day) Throughput (Event/s)
1 3.44 290 7045
2 5.7 480 11670
4 6.84 577 14000
8 6.12 516 12533

The maximum throughput is 6.8MB/s with four data inputs. 100% higher than one data input. It can scale 100% with max throughput 577GB/day for single instance.

Testing Result for Azure Storage Blob inputs

The detailed test result follows.

  • Each Blob input contains 259,202 events.
  • Each event is 190 Bytes.
Blob Number Blob Size Input Number Throughput (MB/s) Throughput (GB/day) Throughput (Event/s)
20 50 M 1 3.8 MB/s 320 21136
40 50 M 2 7.7 MB/s 650 42092
80 50 M 4 12.5 MB/s 1054 72960
160 50 M 8 12.1 MB/s 1020 66500

Using a large file size (50M in our test environment), the max throughput is 12.5MB/s with four data inputs, about 300% higher than the single input.

PREVIOUS
Lookups for the Splunk Add-on for Microsoft Cloud Services
  NEXT
APIs used in the Splunk Add-on for Microsoft Cloud Services

This documentation applies to the following versions of Splunk® Supported Add-ons: released


Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters