Splunk Add-on for AWS

Configure CloudWatch Log inputs for the Splunk Add-on for AWS

Due to rate limitations, Splunk strongly recommends against using the Splunk Add-on for AWS to collect CloudWatch Log data (source type: aws:cloudwatchlogs:*). Use to Splunk Add-on for Amazon Kinesis Firehose to collect CloudWatch Log and VPC Flow Logs instead. The Spunk Add-on for Amazon Kinesis Firehose includes index-time logic to perform the correct knowledge extraction for these events through the Kinesis input as well.

Configure a CloudWatch Logs input for the Splunk Add-on for Amazon Web Services on your data collection node through Splunk Web (recommended), or in local/aws_cloudwatch_logs_tasks.conf.

• Configure a CloudWatch Logs input using Splunk Web (recommended)
• Configure a CloudWatch Logs input using configuration file

Configure a CloudWatch Logs input using Splunk Web

To configure inputs using Splunk Web, click Splunk Add-on for AWS in the left navigation bar on Splunk Web home, then choose one of the following menu paths depending on the data type you want to collect:

• Create New Input > VPC Flow Logs > CloudWatch Logs.
• Create New Input > Others > CloudWatch Logs.

Fill out the fields as described in the table:

Argument in configuration file	Field in Splunk Web	Description
account	AWS Account	The AWS account or EC2 IAM role the Splunk platform uses to access your CloudWatch Logs data. In Splunk Web, select an account from the drop-down list. In aws_cloudwatch_logs_tasks.conf, enter the friendly name of one of the AWS accounts that you configured on the Configuration page or the name of the autodiscovered EC2 IAM role.
region	AWS Region	The AWS region that contains the data. In aws_cloudwatch_logs_tasks.conf, enter the region ID.
groups	Log group	A comma-separated list of log group names. Note: Do not use wildcards.
only_after	Only After	GMT time string in '%Y-%m-%dT%H:%M:%S' format. If set, only events after this time are queried and indexed. Defaults to 1970-01-01T00:00:00.
stream_matcher	Stream Matching Regex	REGEX to strictly match stream names. Defaults to .*
interval	Interval	The number of seconds to wait before the Splunk platform runs the command again. Default is 600 seconds.
sourcetype	Source type	A source type for the events. Enter aws:cloudwatchlogs:vpcflow if you are indexing VPC Flow Log data. Enter aws:cloudwatchlogs if you are collecting any other CloudWatch Logs data.
index	Index	The index name where the Splunk platform puts the CloudWatch Logs data. The default is main.

Configure a CloudWatch Logs input using configuration file

To configure the input using configuration file, create $SPLUNK_HOME/etc/apps/Splunk_TA_aws/local/aws_cloudwatch_logs_tasks.conf using the following template.

[<name>]
account = <value>
groups = <value>
index = <value>
interval = <value>
only_after = <value>
region = <value>
sourcetype = <value>
stream_matcher = <value>

Here is an example stanza that collects VPC Flow Log data from two log groups.

[splunkapp2:us-west-2]
account = splunkapp2
groups = SomeName/DefaultLogGroup, SomeOtherName/SomeOtherLogGroup
index = default
interval = 600
only_after = 1970-01-01T00:00:00
region = us-west-2
sourcetype = aws:cloudwatchlogs:vpcflow
stream_matcher = eni.*