Splunk® Supported Add-ons

Splunk Add-on for AWS

Download manual as PDF

Download topic as PDF

Configure AWS services for the Splunk Add-on for AWS

Before installing the Splunk Add-on for AWS, configure AWS services.

Before configuring you Splunk platform deployment to work with your AWS data, make sure that your AWS deployment is properly configured to send data.

Prerequisites

  • You must have administrator access to your AWS account. If you do not have necessary permissions, work with an AWS administrator to complete the tasks described in this manual.
  • If your account is in the AWS China region or the AWS GovCloud region, not all AWS services are available.


Configure AWS Config

The Splunk Add-on for AWS collects events from a Simple Queue Service (SQS) that subscribes to the Simple Notification Service (SNS) notification events from AWS Config. Configure AWS Config to produce SNS notifications, and then create the SQS that the add-on can access. For more information about AWS Config, see the AWS Config documentation.

  1. Enable AWS Config by following the AWS Config setup guide.
  2. Specify a new S3 bucket to save the data and an SNS Topic to which Splunk software will stream Config notifications. Do not use an existing bucket or SNS.
  3. Verify that you have successfully completed the setup process. If you used the AWS console, the Resource Lookup page displays.
  4. Create a new SQS.
  5. Subscribe the SQS exclusively to the SNS Topic that you created in Step 2.
  6. Grant IAM permissions to access the S3 bucket and SQS to the AWS account that the add-on uses to connect to your AWS environment.


Configure AWS Config Rules

AWS Config Rules requires no additional configuration beyond that described in the AWS documentation.

  1. Enable AWS Config for all regions for which you want to collect data in the add-on. Follow the AWS Config setup guide: http://docs.aws.amazon.com/config/latest/developerguide/setting-up.html.
  2. Set up AWS Config Rules by following the instructions in the AWS Config documentation: http://docs.aws.amazon.com/config/latest/developerguide/evaluate-config_set-up.html
  3. Grant the necessary permissions to the AWS account used for this input. See Configure AWS permissions for details.

Configure CloudTrail

The Splunk Add-on for AWS collects events from a Simple Queue Service (SQS) that subscribes to the Simple Notification Service (SNS) notification events from CloudTrail. Configure CloudTrail to produce these notifications, then create an SQS in each region for the add-on to access them.

Although AWS offers global trails, or one CloudTrail configuration in one region to collect trail data from all regions, SQS messages do not arrive as expected in this case. Either configure separate CloudTrail S3 > SNS > SQS paths for each region to ensure that you capture all your data or, if you want to configure a global CloudTrail, skip steps 3 through 6 below and instead configure the add-on to collect data from that S3 bucket directly.

  1. Enable CloudTrail. Follow the instructions in the AWS documentation: http://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-create-and-update-a-trail.html.
  2. Create an S3 Bucket in which to store the CloudTrail events. Follow the AWS documentation to ensure the permissions for this bucket are correct: http://docs.aws.amazon.com/awscloudtrail/latest/userguide/create-s3-bucket-policy-for-cloudtrail.html
  3. Enable SNS Notifications. See the AWS documentation for instructions: http://docs.aws.amazon.com/awscloudtrail/latest/userguide/getting_notifications_top_level.html
  4. Create a new SQS.
  5. If you are in the China region, explicitly grant DeleteMessage and SendMessage permissions to the SQS that you just created. This step is not necessary in commercial regions.
  6. Subscribe the SQS to the SNS Notifications that you enabled in step 3.
  7. Grant IAM permissions to access the S3 bucket and SQS to the AWS account that the add-on uses to connect to your AWS environment. See Configure AWS permissions for details.

Configure CloudWatch

To enable AWS to produce billing metrics in CloudWatch, turn on Receive Billing Alerts in the Preferences section of the Billing and Cost Management console.

The CloudWatch service is automatically enabled to collect free metrics for your AWS services and requires no additional configuration for the Splunk Add-on for AWS. However, you do need to grant permissions to the AWS account(s) that the add-on uses to connect to the CloudWatch API. See Configure AWS permissions for details.


Configure CloudWatch Logs, including VPC Flow Logs

Ingesting data from CloudWatch Logs requires no additional configuration beyond that described in the AWS documentation. Similarly, VPC Flow Logs require no additional configuration for the Splunk Add-on for AWS, other than enabling them for your VPCs. However, you do need to grant permissions to the AWS account(s) that the add-on uses to connect to the VPC Flow Log groups and streams. See Configure AWS permissions for details.

See the AWS documentation for how to enable Flow Logs for your VPCs and configure an IAM role for them: http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/flow-logs.html.

Configure Inspector

Inspector requires no additional configuration for the Splunk Add-on for AWS. However, you do need to grant permissions to the AWS accounts or EC2 IAM roles that the add-on uses to connect to the Amazon Inspector API. See Configure AWS permissions for details.


Configure Kinesis

Kinesis requires no additional configuration for the Splunk Add-on for AWS. However, you do need to grant permissions to the AWS accounts or EC2 IAM roles that the add-on uses to connect to the Kinesis API. See Configure AWS permissions for details.

Configure S3

To collect access logs, configure logging in the AWS console to collect the logs in a dedicated S3 bucket. See the AWS documentation for more information on how to configure access logs:

Refer to the AWS S3 documentation for more information about how to configure S3 buckets and objects. http://docs.aws.amazon.com/gettingstarted/latest/swh/getting-started-create-bucket.html

Configure Billing

The Splunk App for AWS collects Billing metrics through CloudWatch and Billing Reports by collecting them from an S3 bucket.

To enable AWS to produce Billing Metrics in CloudWatch, turn on Receive Billing Alerts in the Preferences section of the Billing and Cost Management console.

To enable Billing Reports, turn on Receive Billing Reports in the Preferences section of the Billing and Cost Management console. Be sure to verify your S3 bucket in the Billing and Cost Management console and select the report types that you want to collect.

For more details on managing your AWS Billing Reports, see the Amazon Web Services documentation


Configure SNS

You need to grant permissions to the AWS accounts or EC2 IAM roles that the add-on uses to connect to the Amazon SNS API. See Configure AWS permissions for details.

If you plan to use the SQS-based S3 input, you must enable Amazon S3 bucket events to send notification messages to an SQS queue whenever the events occur. For instructions on setting up S3 bucket event notifications, see the AWS documentation:
https://docs.aws.amazon.com/AmazonS3/latest/UG/SettingBucketNotifications.html
http://docs.aws.amazon.com/AmazonS3/latest/dev/NotificationHowTo.html

Configure SQS

You need to grant permissions to the AWS accounts or EC2 IAM roles that the add-on uses to connect to the Amazon SQS API. See Configure AWS permissions for details.

If you plan to use the SQS-based S3 input, you must perform the following:

PREVIOUS
Sizing, performance, and cost considerations for the Splunk Add-on for AWS
  NEXT
Configure AWS permissions for the Splunk Add-on for AWS

This documentation applies to the following versions of Splunk® Supported Add-ons: released


Comments

The Splunk Add-on for AWS is very feature-rich, seamlessly supporting data collection from AWS services. You can also configure API Gateway on the AWS side to integrate Splunk into your AWS “serverless” infrastructure. For detailed information, see http://blogs.splunk.com/2017/01/06/splunk-and-aws-monitoring-metrics-in-a-serverless-world .

Hunters splunk, Splunker
January 9, 2017

Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters