Configure AWS Data Collection for Splunk App for Infrastructure

Admin privileges are required to configure data collection.

To collect data and monitor your AWS accounts, add your AWS account information to Splunk App for Infrastructure (SII) and collect data from your entities such as EC2, EBS, ELB, and CloudWatch logs.

Video demonstration

For a video demonstration of configuring AWS data collection, see Video: Configuring AWS collection.

Prerequisites

• If configuring on an on-premises instance, you need your AWS account Name, Key ID, and Secret Key information for Step 1.
• If configuring on an AWS EC2 instance, you need to configure an IAM role for AWS data collection in Step 1.

Steps

Step 1: Connect to your AWS account

For on-premises instances:

1. In the Splunk App for Infrastructure user interface, click the Add Data tab.
2. In the left panel click AWS.
3. Enter your AWS account Name, Key ID, Secret Key, and select a Region Category. Note all fields are required.
4. Click Add AWS account.

For AWS EC2 instances:

1. In the Splunk App for Infrastructure user interface, click the Add Data tab.
2. In the left panel click AWS.
3. Attach IAM role. Click the instructions link for directions for how to attach an IAM role needed for AWS data collection, or see Configure Identity and Access Management (IAM) policies for AWS data collection. There can be only one IAM role attached to an instance, and the user interface updates when the IAM role is detected.
4. Click Verify IAM role attachment. A green checkmark and an identified IAM detected role display.

Step 2: Collect data from

1. Select the AWS Entity Types you want to collect data from.
2. Select the AWS Regions that apply.
3. Enter Custom Dimensions for troubleshooting, analysis, and filtering hosts.
   • Dimensions are key/value pairs that provide meta data about the metric (describes the measurement) used for searching and filtering relevant datasets (distinct time series) during an investigation.
   • Use the format of dimension:value, such as location:seattle or role:webserver.
4. If you want to collect data from CloudWatch Logs, select Yes and click Add AWS data source.
   • When setting up CloudWatch Logs agent configuration in AWS, edit the log stream name (log_stream_name) with a unique name (instance_id) for each log group within the configuration file. This defines the log stream’s identity for correlation of logs to individual instances and metric data. For example:

   [/var/log/messages]
   file = /var/log/messages
   log_group_name = /var/log/messages
   log_stream_name = {instance_id}
   

   • Select the region and enter the log file name. Click the Add to add more log files.
5. Click Update AWS data source.

Step 3: Once your AWS account is added, verify your data connection

1. When a connection is made to your AWS account(s), connected entities display.
   • If no new entities are connected after a few minutes, click Refresh.
   • When new entities are connected, click New host found to view your entity.

Summary

After you have added your AWS entities, and validate new entities are connected, you can start monitoring your infrastructure. Go to the Investigate page to monitor your entities in the Infrastructure Overview or List View. You can group your entities to monitor them more easily, and drilldown to the Analysis Workspace to further analyze your infrastructure.