Connect to AWS and send data to Splunk Observability Cloud 🔗

To leverage the benefits of data monitoring across your infrastructure, connect Splunk Observability Cloud to Amazon Web Services (AWS). Follow these steps:

Verify the prerequisites.
Evaluate your needs and learn about Observability Cloud’s options to ingest AWS data.
Choose how to connect with AWS.
Next steps.

Note

Check the list of AWS integrations available in Splunk Observability Cloud.

AWS integration prerequisites¶

To connect AWS to Observability Cloud you need:

Administrator privileges in Observability Cloud and your AWS accounts.
An authentication method.

Supported AWS regions¶

Observability Cloud supports the following regions:

Regular
- ap-northeast-1: Asia Pacific (Tokyo)
- ap-northeast-2: Asia Pacific (Seoul)
- ap-northeast-3: Asia Pacific (Osaka)
- ap-south-1: Asia Pacific (Mumbai)
- ap-southeast-1: Asia Pacific (Singapore)
- ap-southeast-2: Asia Pacific (Sydney)
- ca-central-1: Canada (Central)
- eu-central-1: Europe (Frankfurt)
- eu-north-1: Europe (Stockholm)
- eu-west-1: Europe (Ireland)
- eu-west-2: Europe (London)
- eu-west-3: Europe (Paris)
- sa-east-1: South America (Sao Paulo)
- us-east-1: US East (N. Virginia)
- us-east-2: US East (Ohio)
- us-west-1: US West (N. California)
- us-west-2: US West (Oregon)
Optional
- af-south-1: Africa (Cape Town)
- ap-east-1: Asia Pacific (Hong Kong)
- ap-south-2: Asia Pacific (Hyderabad)
- ap-southeast-3: Asia Pacific (Jakarta)
- ap-southeast-4: Asia Pacific (Melbourne)
- eu-central-2: Europe (Zurich)
- eu-south-1: Europe (Milan)
- eu-south-2: Europe (Spain)
- me-central-1: Middle East (UAE)
- me-south-1: Middle East (Bahrain)
GovCloud
- us-gov-east-1: AWS GovCloud (US-East)
- us-gov-west-1: AWS GovCloud (US-West)
China
- cn-north-1: China (Beijing)
- cn-northwest-1: China (Ningxia)

Notes:

If you want to activate a specific optional region, you need to do it before adding it to the integration. Make sure you’ve activated the optional regions you’ll need in your AWS console first. Regular regions are activated in AWS by default.
If you’re using the UI guided setup to create the integration, you’ll be prompted to select which AWS regions you work with.
If you’re using the API and supply an empty list in an API call, Observability Cloud activates all regular regions. If you add the ec2:DescribeRegions permission to your AWS policy, optional regions you’ve activated on your AWS account are activated in Observability Cloud as well.

AWS authentication methods¶

In most AWS regions, use an Identity and Access Management (IAM) policy, an AWS IAM role, and an external ID from Observability Cloud.

An external ID is a random string used to establish a trust relationship between Observability Cloud and your AWS account. It’s automatically generated for you when you create a new AWS integration in Observability Cloud. See How to use an external ID when granting access to your AWS resources to a third party in AWS documentation.

For the GovCloud or China regions, select the option to authenticate using a secure token, which combines an access key ID and a secret access key.

Create an AWS IAM policy¶

The AWS IAM policy is a JSON object to which Observability Cloud refers for permission to collect data from every supported AWS service. To create a new AWS IAM policy, follow these steps.

Log into your Amazon Web Services account and look for the Identity and Access Management service.
Create a new policy. In the JSON tab, replace the placeholder JSON with the pertinent AWS IAM policy JSON. Guided setup provides this policy in the Prepare AWS Account step. See also some policy examples.
Follow the instructions to complete the process and create the policy.

If you have any doubts, check AWS documentation.

Create an AWS IAM role¶

After creating an AWS IAM policy, you need to assign that policy to a particular role by performing the following steps in the Amazon Web Services console:

Go to Roles > Create Role and select Another AWS account as the type of trusted entity.
Copy and paste the Account ID displayed in guided setup into the Account ID field.
Select Require external ID. Copy and paste the External ID displayed in the guided setup into the External ID field.
Continue with Next: Permissions. Under Policy name, select the policy you made in the previous step.
Follow the instructions, and name and create your new AWS IAM role.

Creating the AWS IAM role generates the Role ARN used to establish connection with AWS. Copy the created ARN role, and paste it into the Role ARN field in the guided setup.

If you have any doubts, check AWS documentation.

Evaluate your data ingest needs and costs¶

Regardless of the connection option you choose, you can configure your system more efficiently if you decide beforehand what data types and sources you want.

To determine the best connection method and configuration settings, answer the following questions before you connect AWS to Splunk Observability Cloud:

Which AWS regions do you need to work with.
Do you want to collect metrics through API polling at specified intervals, or through CloudWatch Metric Streams? You can activate Metric Streams using the Splunk Observability Cloud API.
If you want to collect logs in addition to metrics, include logs while configuring the API or when given that option while performing a guided setup.

Poll data from AWS using CloudWatch APIs¶

You can poll data from AWS at specified intervals using CloudWatch APIs. Due to the CloudWatch metrics instability, for certain namespaces some metrics might be delayed a few minutes. See more in Configure API polling.

High data volume warning¶

After you create an AWS integration, if more than 100,000 metrics are retrieved from CloudWatch, Observability Cloud automatically deactivates the integration and sends you a warning email.

This check runs just once per integration. If you activate the integration afterwards, it will work correctly.

You can deactivate this check by setting the enableCheckLargeVolume field in the AWS integration to false using the API.

Tag filtering¶

If you filter data based on tags, your costs for Amazon CloudWatch and Splunk Infrastructure Monitoring might decrease.

Be careful when choosing tag names: Splunk Observability Cloud only allows alphanumeric characters, and the underscore and minus symbols. Unsupported characters include ., :, /, =, +, @, and spaces, which are replaced by the underscore character.

Caution

CloudWatch Metric Streams doesn’t support filtering based on resource tags.

Use Metric Streams to forward data to Observability Cloud¶

Rather than polling for metrics data, CloudWatch Metric Streams sends metrics to a Kinesis Data Firehose stream, reducing latency. See Low Latency Observability Into AWS Services With Splunk in the DevOps blog for more information.

CloudWatch Metric Streams continually stream Amazon CloudWatch metrics. Although they’re more efficient than API polling, consider the constraints below.

Data collection interval and costs¶

In most cases, metrics are reported every minute. However, some services use a different cadence: For example, selected S3 metrics are reported on a daily basis. Check AWS documentation to verify how often your services’ metrics are reported.

Collecting Amazon CloudWatch metrics via the polling APIs at the default polling rate of 300 seconds (5 minutes) is generally cheaper than using Metric Streams. On the other hand, if you set polling intervals to one minute, generally you’ll see an increase in Amazon CloudWatch usage costs compared to Metric Streams.

Learn more at Amazon CloudWatch usage costs.

Connect with AWS¶

You can connect Observability Cloud to AWS in several ways. By default, Observability Cloud brings in data from all supported AWS services associated with your account. To limit the amount of data to import, see Specify and limit the data and metadata to import.

Choose the connection method that best matches your needs:

Connection option	Why use this?
Connect to AWS using the guided setup in Splunk Observability Cloud	Guides you step-by-step to set up an AWS connection and default configuration in Observability Cloud. Guided setup includes links to Amazon CloudFormation templates that you can select to create needed AWS IAM roles.
Connect to AWS using the Splunk Observability Cloud API	Requires knowledge of POST and PUT call syntax, but includes options and automation that are not part of the guided setup. Choose this method if you want to configure many integrations at once.
Connect to AWS using Splunk Terraform	Use this connection method if you already manage your infrastructure as code by deploying through Terraform.

If you can’t connect AWS to Observability Cloud, see Troubleshoot your AWS connection.