Get Microsoft Azure data into Splunk Cloud
This topic guides you through the steps to get Microsoft Azure data into Splunk Cloud.
Before you begin
To get Microsoft Azure data into Splunk Cloud, you need a high-level understanding of the following concepts:
- Indexes. The index is the repository for your data. When Splunk Cloud indexes raw data, it transforms the data into searchable events.
- Inputs Data Manager. The Inputs Data Manager (IDM) is a component of your Splunk Cloud environment optimized for data ingestion. It is intended for use with cloud data sources or when using add-ons that require inputs on the Search tier.
- Source types. A source type is one of the critical default fields that Splunk Cloud assigns to all incoming data. It tells Splunk Cloud what kind of data you have so that it can format the data intelligently during indexing.
- Splunk Add-ons. In this configuration, you use an add-on to help get data in. Add-ons support and extend the functionality of Splunk Cloud and the apps that run on it, usually by providing inputs for a specific technology or vendor.
Prerequisites in your Splunk Cloud environment
You must meet the following prerequisites before you can get Azure data into Splunk Cloud:
- This document assumes that you have an
sc_adminrole on your Splunk Cloud instance. If you do not have this role assigned to you, you'll need to do this first.
- IMPORTANT: You'll need to get the Splunk Add-on for Microsoft Cloud Services installed on your Inputs Data Manager and your Splunk Cloud instance via Splunk Support. Ensure you allow adequate time to complete this task before you attempt to get data in.
- You should create a test index in your Splunk Cloud instance so that you can test your installation before going into production.
Prerequisites in your Microsoft Azure environment
This document describes how to get data from your Microsoft Azure activity logs into Splunk Cloud. The activity logs contain information on events and users' actions and when those actions occurred. Ensure you meet the following requirements:
- This document assumes you have activity logs and subscriptions in your Microsoft Azure environment.
- This document assumes you have the permissions necessary to make changes in your Microsoft Azure environment. If you do not have these permissions, work closely with your Microsoft Azure Administrator to complete these steps.
This document walks you through the steps to get your Microsoft Azure activity data into your Splunk Cloud instance. In this procedure, you'll create an application registration, which is similar to a service account that you can use to authenticate to Microsoft Azure. The application registration has an application ID (similar to a user name) and an application key or secret (which is similar to a password). This allows your Splunk Cloud instance to authenticate to Microsoft Azure and get the activity log data in.
Then, you'll configure the Splunk Add-on for Microsoft Cloud Services on the Inputs Data Manager and on Splunk Cloud to make it easy to get the data into Splunk Cloud.
There are many other types of data you may want to get into your Splunk Cloud instance, and this document is not intended to be a comprehensive guide for getting all your Microsoft Azure data into Splunk Cloud. Instead, use this process as a template that you can repeat until you've included all of the relevant source types.
To get Microsoft Azure data into Splunk Cloud, you'll need to complete the following high-level steps:
- Configure an index on your Splunk Cloud instance. You create an index for the Microsoft Azure data you want to bring into your Splunk Cloud deployment.
- Configure Microsoft Azure so that you can authenticate and ingest data from Splunk Cloud. While you are configuring Microsoft Azure, you'll also need to record some information that you'll use to connect Splunk Cloud with Microsoft Azure.
- Configure the Splunk Add-on for Microsoft Cloud Services on your Inputs Data Manager (IDM). You will also need to configure inputs on the IDM. The IDM is responsible for data ingestion.
- Verify that data is flowing to your Splunk Cloud instance. After configuring Azure settings and add-on settings, check to see if data is flowing to your Splunk Cloud instance.
Step 1: Configure a new index on your Splunk Cloud instance
First, you need to create indexes to store the events you will send from your Microsoft Azure instance. It's a best practice to create separate indexes for different types of data. For this initial index, you'll create an index to store Microsoft Azure activity data:
- From your Splunk Cloud instance, go to Settings > Indexes.
- Click New Index.
- In the Index name field, as an example, enter azure-activity. Alternatively, you can select a name that is consistent with your company's index naming convention.
- For Index Data Type, select Events.
- For Searchable time (days), enter the number of days you want data to be searchable. As an example, enter 30.
- Click No Additional Storage, and click Save.
Step 2: Configure Microsoft Azure so that you can authenticate and ingest activity data
You need to configure an Application Registration on Microsoft Azure and give it read access to resources in your subscription:
- From your Microsoft Azure portal, go to Azure Active Directory > App Registrations > New Registration:
- In the New Registration field, enter a name for the Application Registration.
- Leave the Application type at the default value (Web app/API).
- Leave the Supported account type as the default value.
- Click the Register button.
- In a separate location, note the application ID value. You can think of the application ID as a user ID. The application ID value maps to the Client ID field when you configure Microsoft Azure in Splunk Cloud via the Microsoft Cloud Services add-on.
- Now, you need to create an application secret. This is comparable to a password or key. To do this, go to Certificates & secrets, and click New client secret:
- Enter a name for the secret in the Description field, and select the Never radio button under the Expires field. Click Add.
- In a separate location, copy the value for the secret key from the clipboard as this is the only time it will display. You will need this value later when you configure the add-on to connect to Microsoft Azure. The secret key maps to the Key (Client Secret) value when you configure Azure in Splunk via the Microsoft Cloud Services add-on. Note that you can create a new secret if you lose this value.
- You need to grant the application registration read access to resources in your subscription. To do this, click Subscriptions, and choose the subscription from which you want to ingest data. In this example, the Pay-As-You-Go subscription is selected:
- Click Access control (IAM) > Add > Add role assignment.
- From the Role dropdown menu, select Reader.
- In the Assign Access to field, leave the default value.
- In the select field, type the name of the Application Registration you just created:
- Click Save
- If you have multiple subscriptions, you can continue to add access to each of the subscriptions you want to include. When you have added all of the subscriptions that you want to include, your Microsoft Azure configuration should be complete.
Step 3: Configure the Splunk Add-on for Microsoft Cloud Services on your Inputs Data Manager (IDM)
- Log into your IDM instance at
https://idm-<cloudname>.splunkcloud.comwhere <cloudname> represents your Splunk Cloud name.
- Go to Apps > Splunk Add-on for Microsoft Cloud Services.
- From the Add-on, select Configuration > Add Azure App Account.
- In the Name field, enter a name for the Microsoft Azure App account.
- In the Client ID field, enter the application ID value that you saved earlier.
- In the Key (Client Secret) field, enter the value for the client secret you saved earlier
- In the Tenant ID field, enter the Microsoft Azure Directory ID. You can find the Directory ID in the Azure Portal, by going to Azure Active Directory > Properties.
- Click Add.
- Now you'll need to configure the inputs. To do this, click the Inputs tab.
- Click Create New Input > Azure Audit.
- Enter a name for the input.
- From the dropdown menu, select the account you created earlier.
- In the Subscription ID field, enter the Azure Subscription ID. You can find this in the Azure Portal by going to All services > Subscriptions. Copy the value of the Subscription ID field.
- Leave the start time value as the default value and the interval at 3600.
- For the index, select the azure-activity index you created earlier.
- Click Add.
Step 4: Confirm data is flowing to your Splunk Cloud instance
- From your Splunk Cloud instance, go to Apps > Search and Reporting.
- In the search field, enter index= azure-activity
- For the time range, select Presets > Last 30 days.
- Click the search icon.
- Events from your Microsoft Azure environment should display.
Get Amazon Web Services (AWS) data into Splunk Cloud
Get *nix data into Splunk Cloud
This documentation applies to the following versions of Splunk Cloud™: 7.2.9, 7.2.10, 8.0.2006, 8.0.2007, 8.1.2008, 8.1.2009, 8.1.2011, 8.1.2012 (latest FedRAMP release), 8.1.2101, 8.1.2103, 8.2.2104