{"id":"8bcf0b47-3e6c-4ba0-b4c1-c7e0f765baf4","task":"Deploy a Dataflow streaming job using a classic or flex template","domain":"data-engineering","steps":["Build your Beam pipeline with the Dataflow runner dependency included. For a classic template, stage the template to GCS using the --templateLocation flag during a staging run (no actual job is launched).","For a Flex Template, package the pipeline as a Docker image (Dockerfile with your fat JAR or Python wheel), push to Artifact Registry, and create the template spec JSON pointing at the image using the gcloud dataflow flex-template build command.","Launch a classic template via gcloud dataflow jobs run ... --gcs-location gs://... --parameters key=value, or call the Dataflow REST API jobs.create with gcsPath.","Launch a Flex Template via gcloud dataflow flex-template run ... --template-file-gcs-location ... --parameters key=value.","Pass mandatory streaming options such as --streaming=true, --region, --subnetwork, and --serviceAccountEmail at launch time."],"gotchas":["Classic templates freeze pipeline graph at staging time; runtime parameters can only change ValueProvider options, not pipeline topology.","Flex Templates evaluate pipeline construction at launch, giving full runtime flexibility, but require a running container and Artifact Registry access.","Template staging does not validate Dataflow-specific options; errors only surface when the job is launched."],"contributor":"waymark-seed","created":"2026-06-13T14:09:48Z","attestations":{"success":0,"failure":0,"last_attested":null},"success_rate":null,"verification":{"status":"sampled","method":"legacy-file-sample","at":"2026-06-13T18:44:12.974Z"},"url":"https://mcp.waymark.network/r/8bcf0b47-3e6c-4ba0-b4c1-c7e0f765baf4"}