GCP Professional Data Engineer Practice Question

Your team must plan a pipeline for a global e-commerce platform. Mobile apps produce 50 000 events/sec that must power millisecond-latency personalization and be available for ad-hoc analytics in BigQuery. Compliance demands customer-managed encryption for stored data, no public IP addresses on processing workers, and a perimeter that blocks data exfiltration. Which architecture meets every requirement while minimising operational overhead?

Publish events to Pub/Sub; process them with streaming Dataflow jobs that use CMEK, disable public IPs, and run inside a VPC Service Controls perimeter; have the pipeline write in parallel to Bigtable for low-latency serving and BigQuery for analytics.
Use Cloud Data Fusion real-time pipelines to read from Pub/Sub and write to Spanner for both analytics and serving; rely on Cloud NAT for egress and Cloud DLP for data encryption.
Stream events to a self-managed Apache Kafka cluster on Compute Engine; use Spark Streaming to land data in Cloud Storage; query it through BigQuery external tables.
Send events to Cloud Storage with default encryption; schedule Dataproc Spark batch jobs on preemptible VMs via Cloud NAT; load the results into BigQuery and Cloud SQL for serving.

Report Issue

Answer Description

The correct design streams events through Pub/Sub, a fully-managed messaging service that scales to high throughput and requires little operational work. A streaming Dataflow job can read from Pub/Sub and simultaneously write to Bigtable and BigQuery. Dataflow supports customer-managed encryption keys, can run without public IP addresses, and can be placed inside a VPC Service Controls perimeter, satisfying all security and compliance constraints. Bigtable delivers single-digit-millisecond reads for personalization, while BigQuery provides interactive analytics. The other options fail at least one requirement: the Dataproc batch design does not meet the low-latency or CMEK needs, the self-managed Kafka approach raises operational overhead and lacks built-in CMEK and perimeter controls, and the Cloud Data Fusion option stores data in Spanner (not ideal for analytics), relies on Cloud NAT (still public egress), and treats Cloud DLP as encryption rather than masking.

Ask Bash

Bash is our AI bot, trained to help you pass your exam. AI Generated Content may display inaccurate information, always double-check anything important.