GCP Professional Data Engineer Practice Question

You deployed a streaming Dataflow pipeline in us-central1-a that ingests Pub/Sub events and writes to BigQuery. A zonal outage in us-central1-a terminated all worker VMs, and the job remained in a FAILED state until an engineer manually relaunched it, breaking the two-minute processing‐lag SLA. You must redesign the deployment so that it can survive a future zone outage without human intervention while requiring the least possible code change. What should you do?

Redeploy the pipeline to the regional endpoint us-central1, letting Dataflow place and restart workers in healthy zones automatically.
Keep the current setup but configure the Cloud Composer task to retry the Dataflow job up to three times with exponential backoff.
Migrate the job to a Dataproc cluster with high-availability masters spread across zones and run it with Spark Structured Streaming.
Enable FlexRS on the existing Dataflow job so interrupted workers are restarted on preemptible VMs in the same zone.

Report Issue

Answer Description

Running the pipeline against the regional endpoint us-central1 (instead of the single-zone endpoint us-central1-a) allows Dataflow to distribute and, if necessary, recreate workers in any healthy zone within the region. Because the service automatically restarts failed workers and maintains the job's state, a zonal outage is transparently mitigated with no manual action. The other options either do not address cross-zone resilience (Composer retries, FlexRS), are not supported for streaming jobs (FlexRS), or require a significantly larger re-engineering effort (migrating to Dataproc).

Ask Bash

Bash is our AI bot, trained to help you pass your exam. AI Generated Content may display inaccurate information, always double-check anything important.