Create a DSP connection to Amazon CloudWatch

The Amazon CloudWatch connector is planned for deprecation. See the Release Notes for more information.

To get metrics data from Amazon CloudWatch into a data pipeline in Splunk Data Stream Processor, you must first create a connection using the Amazon CloudWatch connector. In the connection settings, provide your Identity and Access Management (IAM) user credentials so that DSP can access your data, and schedule a data collection job to specify how frequently DSP retrieves the data. You can then use the connection in the Amazon CloudWatch source function to get data from Amazon CloudWatch into a DSP pipeline.

Prerequisites

Before you can create the Amazon CloudWatch connection, you must have the following:

• An IAM user that has the necessary permissions assigned to it.
• The access key ID and secret access key for that IAM user. Search for "Access Keys (Access Key ID and Secret Access Key)" in the Amazon Web Services (AWS) documentation for more information about access key credentials.

Your IAM user must have the following permissions for Amazon CloudWatch Metrics:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "cloudwatch:GetMetricData",
                "cloudwatch:GetMetricStatistics",
                "cloudwatch:ListMetrics"
            ],
            "Resource": "*"
        }
    ]
}

You have the option of filtering empty metrics from the AWS/EC2 namespace out of the data that Amazon CloudWatch returns. Filtering out these empty metrics allows the connector to collect other metrics more quickly while making fewer AWS API calls. If you want to filter out empty metrics, make sure that your IAM user also has the following optional permissions:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "1",
            "Effect": "Allow",
            "Action": [
                "ec2:DescribeInstances",
                "autoscaling:DescribeAutoScalingGroups"
            ],
            "Resource": "*"
        }
    ]
}

If you don't have an IAM user with the necessary permissions, ask your AWS administrator for assistance.

Steps

1. In DSP, select the Connections page.
2. On the Connections page, click Create Connection.
3. On the Source tab, select Amazon CloudWatch and then click Next.
4. Complete the following fields:

   Field	Description
   Connection Name	A unique name for your Amazon CloudWatch connection.
   Access Key ID	The access key ID for your IAM user.
   Secret Access Key	The secret access key for your IAM user.
   Metrics	A list of metric queries, which determine the namespaces and regions that the connector collects data from. For each metric query that you want to include, click Add Group and select the appropriate values from the following drop-down lists: Namespaces: A list of namespaces that you want to collect data from. Regions: A list of regions that you want to collect data from. Statistics: A list of statistics that you want to collect. Each statistic represents an aggregation of metrics data collected over a specified period of time.
   Period	The length of time (in seconds) associated with each Amazon CloudWatch statistic.
   Delay	(Optional) The number of seconds to delay the metrics data collection, to compensate for latency in the availability of Amazon CloudWatch Metrics data points. The default is 300 seconds.
   Scheduled	This parameter is on by default, indicating that jobs run automatically. Toggle this parameter off to stop the scheduled job from automatically running. Jobs that are currently running are not affected.
   Schedule	The time-based job schedule that determines when the connector executes jobs for collecting data. Select a predefined value or write a custom CRON schedule. All CRON schedules are based on UTC.
   Workers	The number of workers you want to use to collect data.

   Any credentials that you upload are transmitted securely by HTTPS, encrypted, and securely stored in a secrets manager.

5. Click Save.

   If you're editing a connection that's being used by an active pipeline, you must reactivate that pipeline after making your changes. When you reactivate a pipeline, you must select where you want to resume data ingestion. See Using activation checkpoints to activate your pipeline in the Use the Data Stream Processor manual for more information.

You can now use your connection in an Amazon CloudWatch source function at the start of your data pipeline to get metrics data from Amazon CloudWatch. For instructions on how to build a data pipeline, see the Building a pipeline chapter in the Use the Data Stream Processor manual. For information about the source function, see Get data from Amazon CloudWatch in the Function Reference manual.