
Configure CloudWatch Log inputs for the Splunk Add-on for AWS
Complete the steps to configure CloudWatch Log inputs for the Splunk Add-on for Amazon Web Services (AWS):
- You must manage accounts for the add-on as a prerequisite. See Manage accounts for the Splunk Add-on for AWS.
- Configure AWS services for the CloudWatch Log input.
- Configure AWS permissions for the CloudWatch Log input. You can skip this step and configure AWS permissions at once, if you prefer. See Configure AWS permissions for all Splunk Add-on for AWS inputs at once.
- Configure CloudWatch Log inputs either through Splunk Web or configuration files.
Due to rate limitations, don't use the Splunk Add-on for AWS to collect CloudWatch Log data which has the source type aws:cloudwatchlogs:*
. Instead, use the Splunk Add-on for Amazon Kinesis Firehose to collect CloudWatch Log and VPC Flow Logs. The Spunk Add-on for Amazon Kinesis Firehose includes index-time logic to perform the correct knowledge extraction for these events through the Kinesis input as well.
Configure AWS permissions for the CloudWatch Log input
Required permissions for Logs:
DescribeLogGroups
DescribeLogStreams
GetLogEvents
See the following sample inline policy to configure CloudWatch Log input permissions:
{ "Version": "2012-10-17", "Statement": [ { "Action": [ "logs:DescribeLogGroups", "logs:DescribeLogStreams", "logs:GetLogEvents" ], "Effect": "Allow", "Resource": "*" } ] }
You must also ensure that your role has a trust relationship that allows the flow logs service to assume the role. While viewing the IAM role, choose Edit Trust Relationship and replace that policy with this one:
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Service": "vpc-flow-logs.amazonaws.com" }, "Action": "sts:AssumeRole" } ] }
Configure a CloudWatch Logs input using Splunk Web
To configure inputs using Splunk Web, click Splunk Add-on for AWS in the navigation bar on Splunk Web home, then choose one of the following menu paths depending on the data type you want to collect:
- Create New Input > VPC Flow Logs > CloudWatch Logs
- Create New Input > Others > CloudWatch Logs
Fill out the fields as described in the table:
Argument in configuration file | Field in Splunk Web | Description |
---|---|---|
account
|
AWS Account | The AWS account or EC2 IAM role the Splunk platform uses to access your CloudWatch Logs data. In Splunk Web, select an account from the drop-down list. In aws_cloudwatch_logs_tasks.conf , enter the friendly name of one of the AWS accounts that you configured on the Configuration page or the name of the automatically discovered EC2 IAM role.
|
region
|
AWS Region | The AWS region that contains the data. In aws_cloudwatch_logs_tasks.conf , enter the region ID.
|
groups
|
Log group | A comma-separated list of log group names. Do not use wildcards. |
only_after
|
Only After | GMT time string in '%Y-%m-%dT%H:%M:%S' format. If set, only events after this time are queried and indexed. Defaults to 1970-01-01T00:00:00. |
stream_matcher
|
Stream Matching Regex | REGEX to strictly match stream names. Defaults to .*
|
interval
|
Interval | The number of seconds to wait before the Splunk platform runs the command again. The default is 600 seconds. |
sourcetype
|
Source type | A source type for the events. Enter aws:cloudwatchlogs:vpcflow if you are indexing VPC Flow Log data. Enter aws:cloudwatchlogs if you are collecting any other CloudWatch Logs data.
|
index
|
Index | The index name where the Splunk platform puts the CloudWatch Logs data. The default is main. |
Configure a CloudWatch Logs input using configuration files
To configure the input using configuration files, create $SPLUNK_HOME/etc/apps/Splunk_TA_aws/local/aws_cloudwatch_logs_tasks.conf
using the following template:
[<name>] account = <value> groups = <value> index = <value> interval = <value> only_after = <value> region = <value> sourcetype = <value> stream_matcher = <value>
Here is an example stanza that collects VPC Flow Log data from two log groups:
[splunkapp2:us-west-2] account = splunkapp2 groups = SomeName/DefaultLogGroup, SomeOtherName/SomeOtherLogGroup index = default interval = 600 only_after = 1970-01-01T00:00:00 region = us-west-2 sourcetype = aws:cloudwatchlogs:vpcflow stream_matcher = eni.*
PREVIOUS Configure CloudWatch inputs for the Splunk Add-on for AWS |
NEXT Configure Description inputs for the Splunk Add-on for AWS |
This documentation applies to the following versions of Splunk® Supported Add-ons: released, released
Feedback submitted, thanks!