Your company ingests IoT telemetry at 30 000 messages per second via Cloud Pub/Sub. A streaming Dataflow job in us-central1 transforms the data and writes to a BigQuery dataset also in us-central1. The business requires that if the entire us-central1 region becomes unavailable, no more than 60 seconds of data may be lost (RPO ≤ 1 minute) and processing must resume in another region within 15 minutes (RTO ≤ 15 minutes) without manual code changes. Which design meets these objectives with the least operational overhead?
Modify the existing Dataflow job to enable drain-and-restore, set a 60-second checkpoint interval, and rely on BigQuery regional redundancy for protection.
Enable Pub/Sub topic replication to us-east1 and use a Cloud Composer DAG that launches the Dataflow template in us-east1 when a regional health check fails; keep the dataset in a us-east1 regional BigQuery location.
Create a second pull subscription to the Pub/Sub topic and deploy an identical streaming Dataflow Flex Template in us-east1 writing to a multi-region BigQuery dataset; run both pipelines continuously with idempotent writes.
Configure the job to autoscale across all zones in us-central1 and snapshot state to a dual-region Cloud Storage bucket every minute; redeploy the template manually in another region during an outage.
Running two identical streaming jobs in separate regions provides active-active redundancy: if one region fails, the other continues to process new messages with no intervention, keeping RTO effectively zero and RPO limited only by Pub/Sub delivery guarantees. Using separate subscriptions prevents message acknowledgment coupling, and a multi-region BigQuery dataset remains reachable from either job. Idempotent or exactly-once semantics in the pipeline mitigate duplicate writes that can occur when both jobs are running. Relying on a single-region job, manual restarts from snapshots, or workflow-driven failover all introduce higher operational burden or longer recovery times that threaten the 15-minute RTO and 1-minute RPO targets.
Ask Bash
Bash is our AI bot, trained to help you pass your exam. AI Generated Content may display inaccurate information, always double-check anything important.
What is RPO and RTO in cloud architecture?
Open an interactive chat with Bash
What are idempotent writes in data pipelines?
Open an interactive chat with Bash
How does Pub/Sub support high availability with multiple subscriptions?
Open an interactive chat with Bash
What is active-active redundancy in streaming data pipelines?
Open an interactive chat with Bash
What are idempotent writes in data pipelines?
Open an interactive chat with Bash
How does Pub/Sub ensure message delivery guarantees for RPO requirements?
Open an interactive chat with Bash
GCP Professional Data Engineer
Designing data processing systems
Your Score:
Report Issue
Bash, the Crucial Exams Chat Bot
AI Bot
Loading...
Loading...
Loading...
Pass with Confidence.
IT & Cybersecurity Package
You have hit the limits of our free tier, become a Premium Member today for unlimited access.
Military, Healthcare worker, Gov. employee or Teacher? See if you qualify for a Community Discount.
Monthly
$19.99 $11.99
$11.99/mo
Billed monthly, Cancel any time.
$19.99 after promotion ends
3 Month Pass
$44.99 $26.99
$8.99/mo
One time purchase of $26.99, Does not auto-renew.
$44.99 after promotion ends
Save $18!
MOST POPULAR
Annual Pass
$119.99 $71.99
$5.99/mo
One time purchase of $71.99, Does not auto-renew.
$119.99 after promotion ends
Save $48!
BEST DEAL
Lifetime Pass
$189.99 $113.99
One time purchase, Good for life.
Save $76!
What You Get
All IT & Cybersecurity Package plans include the following perks and exams .