Splunk Observability Cloud metrics in Splunk Cloud Platform

There are two ways to view observability metrics from Splunk Observability Cloud in Splunk Cloud Platform:

• Create an observability metrics-based chart in Splunk Dashboard Studio
• Import a chart from Splunk Observability Cloud into Splunk Dashboard Studio

Prerequisites

To view Splunk Observability Cloud metrics in Splunk Cloud Platform, you must pair your Splunk Cloud Platform organization and your Splunk Observability Cloud organization. To pair, you must add an observability API access token. We recommend that you also set up Unified Identity, which has the benefits of real-time streaming and the ability to log in to Splunk Observability Cloud with your Splunk Cloud Platform credentials. To see if your organization meets the requirements to use Unified Identity, see Who can access Single Sign On (SSO) and the benefits of Unified Identity? If your organization can set up Unified Identity, follow the instructions in How to set up Unified Identity to pair.

If your organization cannot pair through Unified Identity, you can still use this feature by adding an observability API access token. Follow the instructions in Access tokens to pair.

If you have Unified Identity configured between your Splunk Cloud Platform stack and an Observability Cloud organization, the observability metrics in Dashboard Studio default to On. When observability metrics default to On, any observability metric-based chart with a Global Time Range set to "Last <value> <minutes/hours/days>" receives data and streams values into the chart.

Check for observability capabilities

To see observability data in dashboards, you need the read_o11y_content capability. To add charts with observability content, you need the write_o11y_content capability. You can check if your role has these observability capabilities by following these steps:

1. Navigate to the Settings drop-down list.
2. Under Users and Authentication, select Roles.
3. Under the Name column, find the role you want to check.
4. Navigate to the Actions column associated with that role, and select the Edit drop-down list.
5. Select View Capabilities. The View Capabilities modal will pop up.
6. In the search bar, filter for read_o11y_content or write_o11y_content.

Deactivate the observability functionality

The feature flag for the observability functionality is called activate_o11y_dashboards and located in your web-features.conf file. The flag defaults to True. You can deactivate the observability functionality by setting the feature flag to False. If the feature flag is set to False after you've added observability content to a dashboard, an error message appears on any charts using observability content. To change the feature flag setting, contact your support team. For more details, see Contact Support in the Troubleshooting Manual.

Create an observability metrics-based chart in Dashboard Studio

To create a chart based on observability metrics in Splunk Cloud Platform, follow these steps:

1. Navigate to the dashboard where you want to add a metrics-based chart or create a new dashboard. See Splunk Dashboard Studio Part 2: Create a dashboard to learn how.
2. Select the add chart icon () in the editing toolbar, then select the chart type you want to use.
3. In the Select data source panel, select O11y Metric Search, then select + Create o11y metric search.
4. Give your new data source a name in the Data source name field.
5. In the Metric field, enter the first few letters of the observability metric you want to analyze, then select the metric when it appears in the drop-down list.
6. Select Filter, select a dimension, then add the values you want to filter by.
7. Select Analytics, select + Add Analytics, then select the analytics type you want from the options Count, Sum, Mean, Max, or Min. Select the value or values you want to use.
   1. To create a chart with multiple metrics, you can construct and import a chart from Splunk Observability Cloud. For more details, see the following section Import a Splunk Observability Cloud chart into Dashboard Studio.
8. Select Apply Analytics.
9. Select the time range you want your chart to track.
10. Select Apply and close.

Suggested chart types for observability metrics

You can use all Dashboard Studio chart types with observability metrics, but some charts are better suited for the structure of observability data. The following charts are particularly effective in handling observability data:

• Area
• Column
• Line
• Single value
• Table

Import a Splunk Observability Cloud chart into Dashboard Studio

To import a Splunk Observability Cloud chart into Dashboard Studio, follow these steps:

1. Navigate to the Splunk Observability Cloud chart that you want to import into Dashboard Studio. Select the More menu, then select Open. Do not select Copy from the More menu drop-down list because it is a temporary URL.
2. Copy the URL of the chart from the address bar.
3. Navigate to the Splunk Cloud Platform dashboard where you want to add a metrics-based chart or create a new dashboard. See Splunk Dashboard Studio Part 2: Create a dashboard to learn how.
4. Select Edit in the upper right corner, then select the Import content icon (). The Import content panel opens on the right.
5. In the Import content panel, paste the URL of the Splunk Observability Cloud chart you want to import in the Content URL field.

Filter observability metrics by dimension

You can filter observability-based metrics charts by dimension to look more granularly at something. For example, you can look at the dimensions associated with a certain metric that might be related to an outage. Dimensions can be metadata such as region, cluster name, team names, etc. Filter by the metrics associated with a particular cluster to see all of the metrics related to that cluster. Another usage is to create a custom tag for services you own, then filter by that tag to see the performance of only your services.

To filter or drill down to a dimension, you must filter by token. You can create or use a token that is associated with a dimension of your choice. Filtering by the token assigned to that dimension allows you to instantly recreate the entire environment you drilled down to to see that dimension. Tokens also let you share context between multiple charts. For example, when you set a token for a particular cluster, and then filter a dashboard on that token, multiple charts update to the cluster that you assigned to that token.

To filter observability metrics by dimension or token, follow these steps:

1. Navigate to the dashboard you want to filter and examine in Splunk Cloud Platform. You can open an existing dashboard or create a new dashboard in Dashboard Studio.
2. Select Edit.
3. Select the filter icon and then select Dimension value filter.
4. Configure the token by giving it a title and a token name. Then select a dimension.

The title will be the name of the new field that displays on your dashboard. For example, you might title a token "Region", give it a token name "region", and select the dimension "cloud.region". Your dashboard then shows a field called Region with a drop-down list of the values you have for cloud.region, such as "us-east-1" and "us-west-1". You can then select a region from this drop-down list, and all charts on your dashboard update to the region you select.

To use the token for logs or SPL-based data sources, associate the token with the data source of your choice by editing the data source and adding the token name in the Dimension filter tokens field. Then select Apply and close.

Limitations

• Observability metrics in Dashboard Studio do not support creating chain searches from observability metrics.
• Observability metrics in Dashboard Studio do not support the Open in Search feature.
• Observability metrics in Dashboard Studio do not support observability data sources as token values. However, you can use tokens to filter observability metric data sources.