Splunk® Data Stream Processor

Connect to Data Sources and Destinations with DSP

Acrobat logo Download manual as PDF


Acrobat logo Download topic as PDF

Connecting Google Cloud Pub/Sub to your DSP pipeline

When creating a data pipeline in the , you can connect to Google Cloud Pub/Sub and use it as a data source. You can get message data from Pub/Sub topics into a pipeline, transform the data as needed, and then send the transformed data out from the pipeline to a destination of your choosing.

To connect to Google Cloud Pub/Sub as a data source, you must complete the following tasks:

  1. Create a connection that allows the the to access your Pub/Sub data. See Create a DSP connection to Google Cloud Pub/Sub.
  2. Create a pipeline that starts with the Google Cloud Pub/Sub source function. See the Building a pipeline chapter in the Use the manual for instructions on how to build a data pipeline.
  3. Configure the Google Cloud Pub/Sub source function to use your Pub/Sub connection and receive data from a Pub/Sub topic. See Get data from Google Cloud Pub/Sub in the Function Reference manual.
  4. (Optional) Convert the byte-encoded data from Pub/Sub into strings that are human-readable during data preview and usable in streaming functions that require string input. See Deserialize and preview data from Google Cloud Pub/Sub in the .

When you activate the pipeline, the source function subscribes to the specified Pub/Sub topic and starts receiving any messages that are delivered to the topic after the subscription was created. Each message is received into the pipeline as a record containing byte-encoded data.

If your data fails to get into the pipeline, check the connection settings to make sure you have the correct credentials file for your Google Cloud service account.

Last modified on 18 January, 2021
PREVIOUS
Create a DSP connection to Google Cloud Monitoring
  NEXT
Create a DSP connection to Google Cloud Pub/Sub

This documentation applies to the following versions of Splunk® Data Stream Processor: 1.2.0, 1.2.1


Was this documentation topic helpful?

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters