Splunk® Supported Add-ons

Splunk Add-on for Microsoft Cloud Services

Download manual as PDF

Download topic as PDF

Configure Azure Storage Blob Modular Input for Splunk Add-on for Microsoft Cloud Services

Before you enable inputs, complete the previous steps in the configuration process:

Configure your inputs on the Splunk platform instance responsible for collecting data for this add-on, usually a heavy forwarder. You can configure inputs using Splunk Web (recommended) or using the configuration files.

Since the format of the data in Azure Storage Blob channel varies (including text and binary data), Splunk suggests you to leverage the options for sourcetypes to make the event data more effective. See Overview of Event Processing for details.

Configure inputs using Splunk Web

Configure your inputs using Splunk Web on the Splunk platform instance responsible for collecting data for this add-on, usually a heavy forwarder.

  1. In the Splunk Add-on for Microsoft Cloud Services, click Inputs.
  2. Click Create New Input and select Azure Storage Blob.
  3. Enter the Name, Storage Account, Container Name, Blob list, Interval, Index and Sourcetype using the inputs parameters table below.

Configure inputs using Configuration File

  1. Create a file called inputs.conf under $SPLUNK_HOME/etc/apps/Splunk_TA_microsoft-cloudservices/local.
  2. Configure Azure Storage Blob input with the following stanza:
    [mscs_storage_blob://<input_name>]                                  
    account =  <value>                                    
    container_name = <value>                           
    blob_list = <value>  
    exclude_blob_list = <value>        
    blob_mode = <value>                                   
    decoding = <value>                                    
    index = <value>                                     
    interval = <value>                                           
    sourcetype = <value>
    

Inputs Parameter

Each attribute in the following table corresponds to a field in Splunk Web.

Attribute Corresponding field in Splunk Web Description
mscs:storage:blob://<input_name> Name Enter a friendly name of your inputs.
account Azure Storage Account Select the storage account name you configured.
container_name Container Name Enter the container name under the storage account. You can only add one container name for each input.
blob_list Blob List Enter the Blob name which you want to collect the data from. You can add multiple blob names separated by commas. If you leave this field empty, this add-on will collect all the blob lists under the Container Name you just configured.

You can enter the specific blob list name, use wildcard or use regex expression in this field.

  • If you want to collect data from a specific blob list, just enter the name of the blob list here, such as blob_name.
  • You can use wildcard in this field, e.g. blob*, this add-on will collect data from the blob lists of the names starting from blob. And you can also use comma to separate multiple blob names, e.g. blob, name*.
  • If you want to use regex, the syntax is JSON format: {"regex syntax":3}, 3 stands for regex file.
  • If you want to enter the blob list which has both wildcard and regex, you can enter both separated by commas, for example, {"regex syntax" :3, blob* :2}, 2 stands for wildcard list.
  • If you want to enter the blob list using all of the three expressions, you can use the syntax like {"regex syntax" :3, blob* :2, blob :1}, 1 stands for using a specific blob list name.

Constraints:

  • The blob name must be at least one character long but cannot be more than 1,024 characters.
  • Blob names are case-sensitive.
  • Reserved URL characters must be properly escaped.
  • The number of path segments comprising the blob name cannot exceed 254.
exclude_blob_list Excluded Blob List Optional. Enter the Blob name that you do not want to collect the data from. You can add multiple blob names separated by commas. The syntax of the Excluded Blob List is the same as Blob List.
blob_mode NULL The default is append. Do not change the value of this field.
decoding Decoding Specify the character set of the file, such as UTF-8 or UTF-32. If you leave this field blank, this add-on will use the default character set of the file.
collection_interval Interval The number of seconds to wait before the Splunk platform runs the command again. The default is 3600 seconds.
index Index The index in which to store Azure Storage Blob data.
sourcetype Sourcetype The default is mscs:storage:blob. To simplify field extraction, enter one of the following predefined sourcetypes: mscs:storage:blob:json, or mscs:storage:blob:xml.

If there is a file match the syntax both in Blob List and Exclude Blob List, Exclude Blob List is in higher priority. For example, if there is a blob list name blob1, and it matches the syntax you set in Blob List and Exclude Blob List, this add-on will exclude this list because Exclude Blob List is in higher priority.

PREVIOUS
Configure Azure Storage Table Modular Input for Splunk Add-on for Microsoft Cloud Services
  NEXT
Configure Azure Virtual Machine Metrics Modular Input for Splunk Add-on for Microsoft Cloud Services

This documentation applies to the following versions of Splunk® Supported Add-ons: released


Comments

for csv files in blob, which sourcetype we have to use to get default parsing for csv?

Rajasekhar14
November 14, 2019

Hello Michaelrosello,
This could be specific to your configuration. For issues this specific, please post the question to Splunk Answers (http://answers.splunk.com) so the broader community of Splunk customers and employees can help you. Alternatively, file a Support case via the Support portal (https://login.splunk.com/page/sso_redirect?type=portal) if you have an active Support entitlement.

Mglauser splunk, Splunker
September 4, 2019

I am trying to onboard multilple file in one account under different sourcetypes. some files are being onboarded just fine, some are not and saw this the internal logs. "The number of qualified_storage is 0" is this issue regarding the storage account, and how to resolve?

Michaelrosello
September 4, 2019

the URL to access storage account in China differs and is not handled.
Please refer to:
https://docs.microsoft.com/es-es/azure/china/china-get-started-developer-guide#check-endpoints-in-azure
It's quite similar as what has been achieved to support gov site following this: https://docs.microsoft.com/en-us/azure/azure-government/documentation-government-developer-guide

Ronteix
April 24, 2018

Hi Chad, thanks for posting your comment. At present, there is not another blob_mode to use with this app and, as per the table above, you should not change the default value of blob_mode. I'm afraid that is probably not what you wanted to hear but I hope that helps.
Janet
Director, Splunk> Docs

Jrevell splunk, Splunker
November 22, 2017

I'm in a scenario where I need Splunk to index a blob file once a day, regardless of whether there are data changes or not. Is there another blob_mode that would support this?

Chadmedeiros
November 16, 2017

Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters