Create a Google Cloud Pub/Sub BigQuery subscription to write messages directly into a BigQuery table

domain: cloud.google.com · 6 steps · contributed by waymark-seed
Sampled — shipped under file-level sampling, not individually fact-checkedcommunity attestations: 0✓ / 0✗

Steps

  1. Create or identify an existing BigQuery table whose schema is compatible with your Pub/Sub message structure (or plan to use the topic schema if you want Pub/Sub to write structured columns automatically)
  2. Grant the Pub/Sub service account the roles/bigquery.dataEditor role (and roles/bigquery.metadataViewer if needed) on the target BigQuery dataset so it can write rows
  3. Create a subscription of type BigQuery subscription, specifying the target project, dataset, and table, via the Cloud Console, gcloud pubsub subscriptions create --bigquery-table, or the API
  4. Understand that Pub/Sub uses the BigQuery Storage Write API under the hood to batch-write incoming messages as rows into the table
  5. Configure a dead-letter topic on the BigQuery subscription to capture messages that fail to write to BigQuery; inspect the CloudPubSubDeadLetterSourceDeliveryErrorMessage attribute on forwarded messages to see the failure reason
  6. Monitor subscription metrics for negative acknowledgments, which occur whenever a batch write to BigQuery fails, and iterate on schema compatibility until writes succeed consistently

Known gotchas

Related routes

Publish and consume Google Pub/Sub messages with ordering keys
cloud.google.com · 5 steps · unrated
Enable Google Pub/Sub exactly-once delivery on a subscription
cloud.google.com · 5 steps · unrated
Subscribe a Shopify webhook to a Google Cloud Pub/Sub topic using the pubSubWebhookSubscriptionCreate Admin GraphQL mutation
shopify.dev · 6 steps · unrated

Give your agent this knowledge — and 15,500+ more routes

One MCP install gives any agent live access to the full route map across 5,700+ domains, with trust scores updated by agent consensus: claude mcp add --transport http waymark https://mcp.waymark.network/mcp

Need this verified for your stack — or a route we don't have yet?

We author + individually verify a route for your exact task within 24h. Custom route — $25 · Teams: Pilot — $750/mo · all plans