Docs » Connect to your cloud service provider » Connect to GCP

Connect to GCP 🔗

You can monitor Google Cloud Platform (GCP) in Splunk Infrastructure Monitoring using Google StackDriver metrics. If you haven’t already done so, follow the instructions to connect Infrastructure Monitoring to GCP.

Infrastructure Monitoring provides a robust integration with StackDriver, has a StackDriver-powered mode for the Infrastructure Navigator, and includes many built-in-dashboards to help you get started monitoring Google Cloud Platform services.

Create a GCP connection 🔗

This section shows you how to configure a GCP integration in Splunk Infrastructure Monitoring by creating a new service account key and specifying services to monitor in your GCP account.

Prerequisites 🔗

Before you can create a GCP connection, you must be an administrator of your Splunk account.

Part 1: Select a role for the GCP service account 🔗

The primary way to connect to GCP is to use a Project Viewer role for the service account you create. Choosing this role ensures that any functionality updates implemented in Infrastructure Monitoring will not require changes to your GCP setup. If you plan to use this role, you can skip to Part 2: Configure GCP.

If you want to create a role with more restrictive permissions than those available to the Project Viewer role, you can create a new role to use for the service account you create.

About permissions

If you don’t specify sufficient permissions for the role associated with the service account, trying to connect the project in Infrastructure Monitoring generates an error message. If you see this error, review the permissions assigned to the role and add any permissions that have not been enabled, or change the role for the service account to Project Viewer.

The following table specifies the permissions required for this role.

Permission

Required?

monitoring.metricDescriptors.get

yes

monitoring.metricDescriptors.list

yes

monitoring.timeSeries.list

yes

resourcemanager.projects.get

if you want to sync project metadata, such as labels

compute.instances.list

if the Compute Engine service is enabled

compute.machineTypes.list

if the Compute Engine service is enabled

spanner.instances.list

if the Spanner service is enabled

storage.buckets.list

if the Spanner service is enabled

Part 2: Configure GCP 🔗

Note: You have to repeat the following steps to for each project you want to monitor with the GCP integration.

  1. In a new window or tab, go to the Google Cloud Platform website and log into your GCP account.

  2. Open the GCP web console and select a project you want to monitor.

  3. From the sidebar, select IAM & admin ‣ Service Accounts.

  4. Click Create Service Account at the top of the screen.

  5. Complete the following fields:

    Field

    Description

    Service account name

    Enter Splunk.

    Service account ID

    This field is autofilled after you enter Splunk for Service account name.

    Service account description

    Enter the description for your service account.

  6. Click CREATE.

  7. (Optional) Select a role to grant this Service account access to the selected project, then click CONTINUE.

  8. Enable Key type JSON, and then click CREATE. A new service account key JSON file is then downloaded to your computer.

  9. In a new window or tab, go to Cloud Resource Manager API and enable the Cloud Resource Manager API. You need to enable this API so Splunk Infrastructure Monitoring can use it to validate permissions on the service account keys.

Part 3: Start the integration 🔗

  1. On the Observability Cloud home page, click Data Setup. The Connect Your Data page is displayed.

  2. Select All.

  3. Select GCP in the list of integration tiles.

  4. Click New Integration.

  5. Enter a name for this GCP integration.

  6. Click Add Project, then click Import Service Account Key.

  7. Select one or more of the JSON key files that you downloaded from GCP in step 8 of Part 2: Configure GCP. When you finish selecting all the keys you want to add, click Open. You can then see the project IDs corresponding to the service account keys you selected.

  8. By default, all available services are monitored and any new services added later are also be monitored. To import metrics from only some of the available services, click All Services to display a list of the services you can monitor. Select the services you want to monitor, then click Apply.

  9. Set the poll rate to 5 minutes or 1 minute.

  10. (Optional) If you select Compute Engine as one of the services to monitor, you can enter a comma-separated list of Compute Engine Instance metadata keys to send as properties. These metadata keys are sent as properties named gcp_metadata_<metadata-key> in the Google Compute Engine metrics table.

  11. Click Save.

Your GCP integration is now complete.

StackDriver metrics 🔗

After you connect to GCP, metrics from StackDriver under Google Cloud metrics in the StackDriver metric list sync with Infrastructure Monitoring. Agent metrics from AWS or StackDriver do not sync. You can use AWS integration to monitor those metrics.

The metrics from StackDriver contain dimensions that correspond to the Labels described in the Google Cloud metrics reference and the StackDriver Monitored Resource Types reference. Use the monitored_resource dimension to determine which metric corresponds to a particular resource.

Ingest GCP log data 🔗

Scenario documentation in the GCP Cloud Architecture Center describes both pull- and push-based ways to ingest Google Cloud data, but Splunk Observability Cloud supports only the push-based method.

To export Cloud Logging data from GCP to Splunk Observability Cloud, you prepare GCP log export by creating a Pub/Sub subscription and using the Pub/Sub to Splunk Dataflow template to create a Dataflow job that pulls messages from the Pub/Sub subscription, converts payloads into Splunk HEC event format, and forwards those payloads to Splunk Observability Cloud, where the whole event (JSON payload and its information) is ingested.

You can use the example shown in Option A: Stream logs using Pub/Sub to Splunk Dataflow, with the following changes:

  • Change the Access token in the sample syntax (token=your-splunk-hec-token) to the Splunk Observability Cloud access token instead of the Splunk HEC token. Splunk Observability Cloud exposes the Real-time Data Ingest endpoint to be used instead of Splunk HTTP Event Collector endpoint, and you do not have to create any endpoint manually.

  • Revise the URL in the sample syntax to point to the Real-time Data Ingest endpoint for Splunk Observability Cloud as shown in your profile.

Note

Any response code that is not 2xx, including throttling, indicates a message delivery failure. If message delivery fails, you can replay unprocessed messages manually by following the instructions in the “Handling delivery failures” section of GCP documentation for deploying production-ready log exports to Splunk using Dataflow.

Metrics 🔗

These are the metrics available for the Google Cloud Platform integration with Splunk Observability Cloud, grouped according to GCP resource. All metrics are included by default.

Google App Engine metrics 🔗

Google BigQuery metrics 🔗

Google Cloud Bigtable metrics 🔗

Google Cloud Datastore metrics 🔗

Google Cloud Functions metrics 🔗

Google Cloud Pub/Sub metrics 🔗

Google Cloud Router metrics 🔗

Google Cloud Spanner metrics 🔗

Google Cloud Storage metrics 🔗

Google Compute Engine metrics 🔗

Google Container Engine metrics 🔗

Google Kubernetes Engine metrics 🔗