Configure CloudTrail inputs for the Splunk Add-on for AWS

Complete the steps to configure CloudTrail inputs for the Splunk Add-on for Amazon Web Services (AWS):

1. You must manage accounts for the add-on as a prerequisite. See Manage accounts for the Splunk Add-on for AWS.
2. Configure AWS services for the CloudTrail input.
3. Configure AWS permissions for the CloudTrail input.
4. (Optional) Configure VPC Interface Endpoints for SQS, STS and S3 services from your AWS Console if you want to use private endpoints for data collection and authentication. For more information, see the Interface VPC endpoints (AWS PrivateLink) topic in the Amazon Virtual Private Cloud documentation.
5. Configure CloudTrail inputs either through Splunk Web or configuration files.

The CloudTrail input type supports the collection of CloudTrail data (source type: aws:cloudtrail). However, you might want to configure SQS-based S3 inputs to collect this type of data. See Configure SQS-based S3 inputs for the Splunk Add-on for AWS

Before you begin configuring your CloudTrail inputs, be aware of the following behaviors:

• Create a single enabled CloudTrail modular input for each unique Simple Queue Service (SQS) > Simple Notification Service (SNS) > S3 bucket path. Multiple enabled modular inputs can cause conflicts when trying to delete SQS messages or S3 records that another modular input is attempting to access and parse. Be sure to disable or delete testing configurations before going to production.

• If you have multiple AWS regions from which you want to gather CloudTrail data, the Amazon Web Services best practice is that you configure a trail that applies to all regions in the AWS partition in which you are working. You can then set up one CloudTrail input to collect data from the centralized S3 bucket where log files from all the regions are stored.

Configure AWS services for the CloudTrail input

The Splunk Add-on for AWS collects events from an SQS that subscribes to the SNS notification events from CloudTrail. Configure CloudTrail to produce these notifications, then create an SQS in each region for the add-on to access them. The best practice for creating one CloudTrail configuration in one region in order to collect SQS messages of CloudTrail data from all regions, is to perform one of the following tasks:

• Configure one CloudTrail S3 bucket, separate SNS and SQS paths for each region, and configure S3 Event Notification to send to SNS.
• Configure a global CloudTrail, skip steps 3 through 6 below, and configure a Generic S3 input on the add-on to collect data directly from your AWS deployment's S3 bucket.

Configure AWS services

1. Enable CloudTrail. Follow the instructions in the AWS documentation. See http://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-create-and-update-a-trail.html.
2. Create an S3 bucket in which to store the CloudTrail events. Follow the AWS documentation to ensure the permissions for this bucket are correct. See http://docs.aws.amazon.com/awscloudtrail/latest/userguide/create-s3-bucket-policy-for-cloudtrail.html.
3. Enable SNS notifications. See: http://docs.aws.amazon.com/awscloudtrail/latest/userguide/getting_notifications_top_level.html.
4. Create a new SQS.
5. If you are in the China region, explicitly grant DeleteMessage and SendMessage permissions to the SQS that you just created. This step is not necessary in commercial regions.
6. Subscribe the SQS to the SNS notifications that you enabled in step 3.
7. Grant IAM permissions to access the AWS account that the add-on uses to connect to your AWS environment. See Manage accounts for the Splunk Add-on for AWS for details.

Configure AWS permissions for the CloudTrail input

Required permissions for the S3 bucket that collects your CloudTrail logs:

• Get*
• List*
• Delete*

Granting the delete permission is required to support the option to remove log files when done collecting them with the add-on. If you set this parameter to false, you do not need to grant delete permissions.

Required permissions for the SQS subscribed to the S3 bucket that collects CloudTrail logs:

• GetQueueAttributes
• ListQueues
• ReceiveMessage
• GetQueueUrl
• DeleteMessage

In the Resource section of the policy, specify the ARNs of the S3 buckets and SQS queues from which you want to collect data.

See the following sample inline policy to configure CloudTrail input permissions:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "sqs:GetQueueAttributes",
                "sqs:ListQueues",
                "sqs:ReceiveMessage",
                "sqs:GetQueueUrl",
                "sqs:DeleteMessage",
                "s3:Get*",
                "s3:List*",
                "s3:Delete*"
            ],
            "Resource": [
               "*"
            ]
        }
    ]
}

For more information and sample policies, see these resources in the AWS documentation:

• For SQS, see http://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/UsingIAM.html.
• For S3, see http://docs.aws.amazon.com/AmazonS3/latest/dev/s3-access-control.html.

Configure a CloudTrail input using Splunk Web

To configure inputs in Splunk Web:

1. Click on Splunk Add-on for AWS in the navigation bar on Splunk Web home.
2. Click Create New Input > CloudTrail.
3. Use the following table to complete the fields for the new input in the .conf file or in Splunk Web:

Argument in configuration file	Field in Splunk Web	Description
aws_account	AWS Account	The AWS account or EC2 IAM role the Splunk platform uses to access your CloudTrail data. In Splunk Web, select an account from the drop-down list. In inputs.conf, enter the friendly name of one of the AWS accounts that you configured on the Configuration page or the name of the automatically discovered EC2 IAM role.
aws_region	AWS Region	The AWS region that contains the log notification SQS queue. In inputs.conf, enter the region ID. See http://docs.aws.amazon.com/general/latest/gr/rande.html#d0e371.
private_endpoint_enabled	Use Private Endpoints	Check the checkbox to use private endpoints of AWS Security Token Service (STS) and AWS Simple Cloud Storage (S3) services for authentication and data collection. In inputs.conf, enter 0 or 1 to respectively disable or enable use of private endpoints.
s3_private_endpoint_url	Private Endpoint (S3)	Private Endpoint (Interface VPC Endpoint) of your S3 service, which can be configured from your AWS console. Supported Formats : <http/https>://bucket.vpce-<endpoint_id>-<unique_id>.s3.<region_id>.vpce.amazonaws.com <http/https>://bucket.vpce-<endpoint_id>-<unique_id>-<availability_zone>.s3.<region_id>.vpce.amazonaws.com
sts_private_endpoint_url	Private Endpoint (STS)	Private Endpoint (Interface VPC Endpoint) of your STS service, which can be configured from your AWS console. Supported Formats : <http/https>://vpce-<endpoint_id>-<unique_id>.sts.<region_id>.vpce.amazonaws.com <http/https>://vpce-<endpoint_id>-<unique_id>-<availability_zone>.sts.<region_id>.vpce.amazonaws.com
sqs_queue	SQS queue name	The name of the queue to which AWS sends new CloudTrail log notifications. In Splunk Web, you can select a queue from the drop-down list, if your account permissions allow you to list queues, or enter the queue name manually. The queue name is the final segment of the full queue URL. For example, if your SQS queue URL is http://sqs.us-east-1.amazonaws.com/123456789012/testQueue, then your SQS queue name is testQueue.
sqs_private_endpoint_url	Private Endpoint (SQS)	Private Endpoint (Interface VPC Endpoint) of your SQS service, which can be configured from your AWS console. Supported Formats : <http/https>://vpce-<endpoint_id>-<unique_id>.sqs.<region_id>.vpce.amazonaws.com <http/https>://vpce-<endpoint_id>-<unique_id>-<availability_zone>.sqs.<region_id>.vpce.amazonaws.com
remove_files_when_done	Remove logs when done	A Boolean value indicating whether the Splunk platform should delete log files from the S3 bucket after indexing is complete. The default is false.
exclude_describe_events	Exclude events	A Boolean value indicating whether or not to exclude certain events, such as read-only events that can produce a high volume of data. The default is true.
blacklist	Deny list for exclusion	A PCRE regular expression that specifies event names to exclude if exclude_describe_events is set to true. Leave blank to use the default regex ^(?:Describe|List|Get).
excluded_events_index	Excluded events index	The name of the index in which the Splunk platform puts excluded events. The default is empty, which discards the events.
interval	Interval	The number of seconds to wait before the Splunk platform runs the command again. The default is 30 seconds.
log_partitions	n/a	Configure partitions of a log file to be ingested. This add-on searches the log files for <Region ID> and <Account ID>. For example, log_partitions = AWSLogs/<Account ID>/CloudTrail/<Region>.
sourcetype	Source type	A source type for the events. Enter a value only if you want to override the default of aws:cloudtrail. Event extraction relies on the default value of source type. If you change the default value, you must update props.conf as well.
index	Index	The index name where the Splunk platform puts the CloudTrail data. The default is main.

Configure a CloudTrail input using configuration files

To configure inputs manually in inputs.conf, create a stanza using the following template and add it to $SPLUNK_HOME/etc/apps/Splunk_TA_aws/local/inputs.conf. If the file or path does not exist, create it.

[aws_cloudtrail://<name>]
aws_account = <value>
aws_region = <value>
private_endpoint_enabled = <value>
sqs_queue = <value>
sqs_private_endpoint_url = <value>
s3_private_endpoint_url = <value>
sts_private_endpoint_url = <value>
exclude_describe_events = <value>
remove_files_when_done = <value>
blacklist = <value>
excluded_events_index = <value>
interval = <value>
sourcetype = <value>
index = <value>

Some of these settings have default values that can be found in $SPLUNK_HOME/etc/apps/Splunk_TA_aws/default/inputs.conf:

[aws_cloudtrail]
aws_account =
sourcetype = aws:cloudtrail
exclude_describe_events = true
remove_files_when_done = false
queueSize = 128KB
persistentQueueSize = 24MB
interval = 30

The values in default/inputs.conf correspond to the default values in Splunk Web as well as some internal values that are not exposed in Splunk Web for configuration. If you choose to copy this stanza to /local and use it as a starting point to configure your inputs.conf manually, change the stanza title from aws_cloudtrail to aws_cloudtrail://<name>.

Switch from a CloudTrail input to an SQS-based S3 input

The SQS-based S3 input is a more fault-tolerant and higher-performing alternative to the CloudTrail input for collecting CloudTrail data. If you are already collecting CloudTrail data using a CloudTrail input, you can configure an SQS-based S3 input and seamlessly switch to the new input for CloudTrail data collection with little disruption.

1. Disable the CloudTrail input you are using to collect CloudTrail data.
2. Set up a Dead-Letter Queue (DLQ) and the SQS visibility timeout setting for the SQS queue from which you are collecting CloudTrail data. See Configure SQS-based S3 inputs for the Splunk Add-on for AWS.
3. Create an SQS-based S3 input, pointing to the SQS queue you configured in the last step. Configure SQS-based S3 inputs for the Splunk Add-on for AWS for the detailed configuration steps.

Once configured, the new SQS-based S3 input replaces the old CloudTrail input to collect CloudTrail data from the same SQS queue.