Connect to AWS using the Splunk Observability Cloud API ๐
To connect Splunk Observability Cloud to your AWS account, complete the following steps:
Create an AWS connection. See the available AWS regions.
Review your IAM policy. Specify the permissions youโll require to connect to AWS.
Configure how to connect to AWS: Select polling or Metric Streams.
See next steps.
1. Create an AWS connection ๐
To connect Splunk Observability Cloud to AWS through the Splunk Observability Cloud API, perform the following steps:
Create an external AWS ID ๐
To create an external AWS ID, open your command-line interface and use the following command to create an AWS connection that generates an external ID:
curl -X POST 'https://app.<realm>.signalfx.com/v2/integration' \
-H 'accept: application/json, text/plain, */*' \
-H 'x-sf-token: <USER_API_ACCESS_TOKEN>' \
-H 'content-type: application/json' \
--data-raw '{"name":"AWS-connection-name","type":"AWSCloudWatch","authMethod":"ExternalId","pollRate":300000,"services":[],"regions":["us-east-1", "us-east-2", "us-west-1", "us-west-2"]}'
Your system response looks something like this:
{
"authMethod" : "ExternalId",
"enabled" : false,
"externalId" : "<externalId>",
"id" : "<integrationId>",
"importCloudWatch" : false,
"name" : "AWS",
"pollRate" : 300000,
"regions" : [ "us-east-1", "us-east-2", "us-west-1", "us-west-2" ],
"roleArn" : null,
"services" : [ ],
"sfxAwsAccountArn" : "arn:aws:iam::<accountId>:root"
"type" : "AWSCloudWatch"
}
In the system response, note the following fields:
The
id
field is the id of the integration you just created.externalId
andaccountId
will be used when you create an IAM (Identity and Access Management) role in the AWS console later on.The
importCloudWatch
value is set tofalse
because CloudWatch Metrics collection has not been configured.
Create an AWS policy and IAM role ๐
To create an AWS policy and an AWS IAM role with a unique Amazon Resource Name (ARN), go to the AWS console and follow the instructions in Authenticate in AWS using an External ID (recommended). Use the externalId
and accountId
values generated in the previous step.
2. Review the IAM policy and required permissions ๐
These are the required permissions to collect AWS data:
3. Configure your setup ๐
Provide the ARN role to Splunk Observability Cloud.
You can also configure your connection to support any of the following use cases:
Collect metrics for selected regions and services using the CloudWatch API.
Collect metrics for all services using the CloudWatch API.
Collect metrics using CloudWatch Metric Streams by itself or together with log collection.
The following example shows how to collect metrics from selected regions and all services by leaving the services value unspecified.
curl -X PUT 'https://app.<realm>.signalfx.com/v2/integration/<integrationId>' \
-H 'accept: application/json, text/plain, */*' \
-H 'x-sf-token: <USER_API_ACCESS_TOKEN>' \
-H 'content-type: application/json' \
--data-raw '{
"authMethod" : "ExternalId",
"created" : 1690856052734,
"createdByName" : null,
"creator" : "FVaMfXTAIAA",
"customCloudWatchNamespaces" : null,
"enableAwsUsage" : false,
"enableCheckLargeVolume" : false,
"enabled" : true,
"externalId" : "<externalId>",
"id" : "<integrationId>",
"importCloudWatch" : true,
"largeVolume" : false,
"lastUpdated" : 1690856052734,
"lastUpdatedBy" : "FVaMfXTAIAA",
"lastUpdatedByName" : null,
"name" : "AWS-connection-name",
"pollRate" : 300000,
"regions" : [ "us-east-1", "us-east-2", "us-west-1", "us-west-2" ],
"roleArn" : "<your-aws-iam-role-arn>",
"services" : [ ],
"sfxAwsAccountArn" : "arn:aws:iam::<accountId>:root",
"syncCustomNamespacesOnly" : false,
"syncLoadBalancerTargetGroupTags" : false,
"type" : "AWSCloudWatch"}'
4. Configure how to connect to AWS ๐
Configure how to connect to AWS: via API polling, or using Metric Streams.
Configure API polling (default) ๐
If youโre retrieving AWS metrics polling CloudWatch APIs, keep in mind the following intervals:
First, the list of metrics is retrieved with the
ListMetrics
API every 15 minutes.Next, data points are retrieved with the
GetMetricData
API. Note that theGetMetricStatistics
API is deprecated, see more in GetMetricStatistics API deprecation notice.
Use
pollRate
to configure the polling interval for metrics.Use
metadataPollRate
to configure the polling interval for metadata.See how to configure the APIs developer portal for more information.
Configure Metric Streams ๐
To activate CloudWatch Metric Streams, follow these steps:
Submit a GET request to
https://api.<realm>.signalfx.com/v2/integration/<integrationId>
to retrieve your current settings. Make sure to substitute your own realm and integration ID in the URL.Set the
metricStreamsSyncState
field toENABLED
.Set the
importCloudWatch
field totrue
.Set the
enabled
field totrue
.Submit a PUT request to the
https://api.<realm>.signalfx.com/v2/integration/<integrationId>
endpoint to save your updated settings.
Caution
CloudWatch Metric Streams supports filtering by namespace and metric name but doesnโt support filtering based on resource tags.
Next, to complete the activation of Metric Streams:
If you havenโt already, add the relevant permissions to your AWS IAM policy.
In every region from which you want to stream metrics from, deploy CloudFormation.
This creates:
Kinesis Firehose.
The S3 bucket, to back up the events that Kinesis Data Firehose fails to send to the specified HTTP endpoint.
The IAM role that Metric Streams will use.
The IAM role that allows Kinesis Firehose to write the S3 bucket.
See Create an AWS integration using an external ID and ARN in the Splunk developer documentation for syntax examples.
Configure Metric Streams from the AWS console ๐
If you want to integrate from the AWS console, make sure the following fields are configured:
"importCloudWatch": true // import metrics
"metricStreamsSyncState": "ENABLED" // Metric Streams is activated
"metricStreamsManagedExternally": true // Metric Streams managed by AWS
Deactivate Metric Streams ๐
To deactivate Metric Streams, follow these steps:
Submit a GET request to
https://api.<realm>.signalfx.com/v2/integration/<integrationId>
to retrieve your current settings. Make sure to substitute your own realm and integration ID in the URL.Set the
metricStreamsSyncState
field toCANCELLING
.Wait for Splunk Observability Cloud to clean up. This can take up to 15 minutes.
If Splunk Observability Cloud sets
metricStreamsSyncState
toDISABLED
, Metric Streams has been deactivated sucessfully.If Splunk Observability Cloud sets
metricStreamsSyncState
toCANCELLATION_FAILED
, try again, or refer to Troubleshoot Metric Streams in AWS.
5. (Optional) Deploy CloudFormation ๐
To collect CloudWatch Metric Streams or logs from all supported AWS services across all regions, select and deploy a CloudFormation template that supports metric streams or logs. Deploying the template creates the additional resources on your AWS account required both by Metric Streams (Kinesis Firehose, S3 bucket, IAM roles) and logs (Splunk AWS log collector lambda function, IAM role).
See the CloudFormation templates table for more information.
You can find your access token in your accountโs profile settings.
6. Collect logs ๐
To collect logs, see Send AWS logs to Splunk Platform.
Next steps ๐
After you connect Splunk Observability Cloud with AWS, youโll be able to track a series of metrics and analyze your AWS data in real time. See how to leverage data from integration with AWS for more information.
Note
When you edit an AWS integration through the user interface for Splunk Observability Cloud, the integration ID shows in your browser address bar as an alphanumeric string in quotation marks (โ) after a colon (:) at the end of the URL.