Splunk® Data Stream Processor

Connect to Data Sources and Destinations with DSP

Acrobat logo Download manual as PDF

Acrobat logo Download topic as PDF

Create a DSP connection to Google Cloud Pub/Sub

To get messages from a Google Cloud Pub/Sub topic into a data pipeline in the , you must first create a connection using the Connector for Google Cloud Pub/Sub. You can then use the connection in the Google Cloud Pub/Sub source function to get data from a Pub/Sub topic into a pipeline.


Before you can create the Google Cloud Pub/Sub connection, you must have the following:

  • A Google Cloud service account with the following permissions:
    • pubsub.snapshots.seek
    • pubsub.snapshots.create
    • pubsub.snapshots.update
    • pubsub.subscriptions.create
    • pubsub.subscriptions.delete
    • pubsub.subscriptions.consume
    • pubsub.topics.attachSubscription

    If the "Pub/Sub Editor" role is assigned to your account, then your account already has all these permissions.

  • A JSON file containing the credentials of your Google Cloud service account.

If you don't have the required permissions or the credentials file, ask your Google Cloud Platform administrator for assistance. For information about managing permissions and roles, search for "Access control" in the Google Cloud Pub/Sub documentation. For information about creating the credentials file, search for "Creating service account keys" in the Google Cloud Identity and Access Management (IAM) documentation.


  1. In the , select the Connections tab and then click Create Connection.
  2. Select Connector for Google Cloud Pub/Sub and then click Next.
  3. Complete the following fields:
    Field Description
    Connection Name A unique name for your Google Cloud Pub/Sub connection.
    Description (Optional) A description of your connection.
    Service Account Credentials JSON The JSON file containing the credentials of your Google Cloud service account.

    Any credentials that you upload are transmitted securely by HTTPS, encrypted, and securely stored in a secrets manager.

  4. Click Save.

    If you're editing a connection that's being used by an active pipeline, you must reactivate that pipeline after making your changes.

You can now use your connection in a Google Cloud Pub/Sub source function at the start of your data pipeline to get messages from a Pub/Sub topic. For instructions on how to build a data pipeline, see the Building a pipeline chapter in the Use the manual. For information about the source function, see Get data from Google Cloud Pub/Sub in the Function Reference manual.

Last modified on 21 June, 2021
Connecting Google Cloud Pub/Sub to your DSP pipeline
Deserialize and preview data from Google Cloud Pub/Sub in DSP

This documentation applies to the following versions of Splunk® Data Stream Processor: 1.2.0, 1.2.1

Was this documentation topic helpful?

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters