Add a CloudWatch input for the Splunk App for AWS

Create a CloudWatch input to gather performance and billing metrics from the CloudWatch service.

Prerequisites

Before you can successfully configure a CloudWatch input, you need to make sure that the account friendly name you use to configure this input corresponds to an AWS Account Access Key ID or EC2 IAM role that has the necessary permissions to gather this data. If you have not already done this, see Configure your AWS permissions for the Splunk App for AWS in this manual.

If you have a very large amount of CloudWatch data to collect, consider the following factors before you create your data inputs.

• AWS allows a limited number of API calls to CloudWatch per month by default. You can pay for a higher rate limit or attempt to configure your inputs efficiently so that you do not encounter the limit. For more information, see Sizing, performance, and cost considerations for the Splunk App for AWS.
• AWS has different minimum granularities for different metrics, so you will waste API calls if you try to collect metrics more frequently that AWS produces new data for them. As a best practice, configure different CloudWatch inputs for each metric or collection of metrics for which you have different granularity and polling interval requirements.
• When you configure CloudWatch inputs through the app and select "All" for a given namespace, the app collects all metrics that AWS defines for that namespace, including any new ones defined after you create the input. If you instead select "Custom" for a given namespace, or if you configure this input using the Splunk Add-on for AWS instead of through the app, you are limiting the metric collection to just the ones specified in the metrics field. Thus, if AWS adds additional metrics to this namespace later, your input will not collect them.
• If you want to customize which dimensions you collect in a certain namespace, but the app displays a message saying there are too many to list, you can use the Splunk Add-on for AWS to write a custom JSON object to collect only the dimensions that you want, using strings as keys and regular expressions as values. See Add a CloudWatch input for the Splunk Add-on for AWS for instructions.

Add a new CloudWatch input

1. In the app, click Configure in the app navigation bar.

2. Under Data Sources, in the CloudWatch box, click New Input.

3. Select the friendly name of the AWS Account that you want to use to collect CloudWatch data. If you have not yet configured the account you need, click Add New Account to configure one now.

4. Under Regions, select Custom if you want to limit data collection with this input to certain regions. Otherwise, leave All selected.

Note: Selecting all regions may greatly increase the amount of data you collect, incurring charges in AWS and indexing volume. Select only the regions that you need.

5. Under Services, select All, Custom, or None for each service name listed. If you were expecting to see other services which are not listed, check that those services are enabled in the regions you have selected, and that the account you are using has permissions to list all the services. For example, you can only collect Billing metrics if the Virginia region is enabled and you have enabled billing alerts in the Billing and Cost Management console.

Note: Selecting All for all the services listed may greatly increase the amount of data you collect, incurring charges in AWS and indexing volume. Select only the services that you need, and customize the metrics and dimensions that you collect to avoid charges for collecting data that you do not need.

6. If you selected Custom for any of the services, you can remove individual Metrics to reduce the data that you collect. Collect only the metrics that you require to reduce unnecessary API calls.

7. By default, Dimensions is set to All, but you can change the selector to By <Dimension> to open a customizable field to further specify what data the app should collect. Click in the field to open a drop-down menu of dimensions to select from. If there are so many dimensions in the metric that the app cannot list them all, you can revert to selecting all dimensions or you can configure this input using the add-on instead. The dimensions field also supports regular expressions. For example:

• For EBS, you can specify that only the metrics for Volume IDs vol-b8f600b6 and vol-692f6d61 should be collected by selecting those from the drop down menu.
• For SQS, you can collect only the metrics for Queue Names that start with "splunk" and end with "_current" by entering splunk.*_current\s.

8. (Recommended) Configure a custom Index.

9. (Optional) Open the Advanced Settings and configure a Granularity for your input between 60 and 21,600 seconds (between one minute and six hours). The granularity is the sampling period for the data. The smaller your granularity, the more precise your metrics data becomes. Configuring a small granularity is useful when you want to do precise analysis of metrics and you are not concerned about limiting your data volume. Configure a larger granularity when a broader view is acceptable or you want to limit the amount of data you collect from AWS.

Note: If you configure a granularity that is smaller than the minimum sampling period allowed by AWS for a particular metric, your granularity configuration does not override the AWS limit. The app attempts to collect metric data at the granularity you specify even if AWS does not support that granularity, resulting in your indexed data being labeled with an incorrect granularity. This has no affect on app dashboards, which are configured to interpret your raw data correctly.

10. (Optional) Configure an Interval for your input between 60 and 21,600 seconds (between one minute and six hours). The interval is how often the app should poll CloudWatch for new data. By default, the interval is set to 12 times the granularity. You can type a lower multiplier to override the default. As a best practice, do not set an interval greater than 12 times the granularity.

11. Click Add to save and enable this data input.

When you create the data input, the Splunk App for AWS immediately begins collecting your CloudWatch data. If you did not adjust the advanced settings, the app collects data using a granularity of five minutes and polls for new data every hour.

Edit or delete a CloudWatch input

You can view, edit, or delete your existing CloudWatch inputs from the CloudWatch Inputs screen.

1. In the app, click Configure in the app navigation bar.

2. Under Data Sources, in the CloudWatch box, click the link that tells you how many inputs you currently have configured for CloudWatch.

3. The CloudWatch Inputs screen displays a list of CloudWatch inputs, organized by the name auto-assigned to the input.

4. From here, you can click the names to open the individual inputs to edit them, or you can delete an input by clicking the trash can icon.