Configure CloudTrail inputs for the Splunk Add-on for AWS
The CloudTrail input type supports the collection of CloudTrail data (source type:
aws:cloudtrail). However, it is highly recommended that you configure SQS-based S3 inputs to collect this type of data.
Before you begin configuring your CloudTrail inputs, be aware of the following behaviors:
1. You should only have a single enabled CloudTrail modular input for each unique SQS > SNS > S3 bucket path. Multiple enabled modular inputs can cause conflicts when trying to delete SQS messages or S3 records that another modular input is attempting to access and parse. Be sure to disable or delete testing configs before going to production.
2. If you have multiple AWS regions from which you want to gather CloudTrail data, Amazon Web Services recommends that you configure a trail that applies to all regions in the AWS partition in which you are working. You then set up one CloudTrail input to collect data from the centralized S3 bucket where log files from all the regions are stored.
Configure a CloudTrail input on the data collection node using one of the following ways:
- Configure a CloudTrail input using Splunk Web (recommended)
- Configure a CloudTrail input using configuration file
Configure a CloudTrail input using Splunk Web
To configure inputs in Splunk Web, click on Splunk Add-on for AWS in the left navigation bar on Splunk Web home, then click Create New Input > CloudTrail.
|Argument in configuration file||Field in Splunk Web||Description|
||AWS Account|| The AWS account or EC2 IAM role the Splunk platform uses to access your CloudTrail data. In Splunk Web, select an account from the drop-down list. In |
||AWS Region|| The AWS region that contains the log notification SQS queue. In |
||SQS queue name|| The name of the queue to which AWS sends new CloudTrail log notifications. In Splunk Web, you can select a queue from the drop-down list, if your account permissions allow you to list queues, or enter the queue name manually. The queue name is the final segment of the full queue URL. For example, if your SQS queue URL is |
||Remove logs when done||A boolean value indicating whether the Splunk platform should delete log files from the S3 bucket after indexing is complete. Default is false.|
||Exclude events||A boolean value indicating whether or not to exclude certain events, such as read-only events that can produce a high volume of data. Default is true.|
||Blacklist for exclusion|| A PCRE regular expression that specifies event names to if exclude_describe_events is set to |
||Excluded events index||The name of the index in which the Splunk platform should put excluded events. Default is empty, which discards the events.|
||Interval||The number of seconds to wait before the Splunk platform runs the command again. Default is 30 seconds.|
||Source type|| A source type for the events. Enter a value only if you want to override the default of |
||Index||The index name where the Splunk platform puts the CloudTrail data. The default is main.|
Configure a CloudTrail input using configuration file
To configure inputs manually in
inputs.conf, create a stanza using the following template and add it to
$SPLUNK_HOME/etc/apps/Splunk_TA_aws/local/inputs.conf. If the file or path does not exist, create it.
[aws_cloudtrail://<name>] aws_account = <value> aws_region = <value> sqs_queue = <value> exclude_describe_events = <value> remove_files_when_done = <value> blacklist = <value> excluded_events_index = <value> interval = <value> sourcetype = <value> index = <value>
Some of these settings have default values that can be found in
[aws_cloudtrail] aws_account = sourcetype = aws:cloudtrail exclude_describe_events = true remove_files_when_done = false queueSize = 128KB persistentQueueSize = 24MB interval = 30
The values above correspond to the default values in Splunk Web as well as some internal values that are not exposed in Splunk Web for configuration. If you choose to copy this stanza to
/local and use it as a starting point to configure your
inputs.conf manually, change the stanza title from
Switch from a CloudTrail input to an SQS-based S3 input
The SQS-based S3 input is a more fault-tolerant and higher-performing alternative to the CloudTrail input for collecting CloudTrail data. If you are already collecting CloudTrail data using a CloudTrail input, you can configure an SQS-based S3 input and seamlessly switch to the new input for CloudTrail data collection with little disruption.
- Disable the CloudTrail input you are using to collect CloudTrail data.
- Set up a dead-letter queue and the SQS visibility timeout setting for the SQS queue from which you are collecting CloudTrail data. See Configure SQS.
- Create an SQS-based S3 input, pointing to the SQS Queue you configured in the last step. Refer to Configure SQS-based S3 inputs for the Splunk Add-on for AWS for the detailed configuration steps.
Once configured, the new SQS-based S3 input will replace the old CloudTrail input to collect CloudTrail data from the same SQS queue.
Configure Inspector inputs for the Splunk Add-on for AWS
Configure CloudWatch Log inputs for the Splunk Add-on for AWS
This documentation applies to the following versions of Splunk® Supported Add-ons: released