GCP Professional Data Engineer Practice Question

Your company streams 5 million web-click events per minute from Cloud Pub/Sub. Data analysts need to run ad-hoc SQL queries and build near-real-time Looker Studio dashboards with sub-second response times on the latest 90 days of data, which may grow to several petabytes. You must choose a managed storage solution that minimizes operational overhead while meeting these analytical requirements. Which design best satisfies the business goals?

Persist events in Cloud Bigtable and expose them to analysts via custom aggregation services that query Bigtable through client libraries.
Write events as daily Avro files to Cloud Storage, then load them hourly into a scaled-up Cloud SQL instance for reporting queries.
Use Dataflow's streaming connector to write events into an ingestion-time-partitioned BigQuery table and power dashboards through BigQuery BI Engine.
Ingest events into Cloud Spanner and query them from BigQuery using federated connections for dashboarding.

Report Issue

Answer Description

Writing the streaming events from Dataflow directly into a BigQuery table that is partitioned (for example, by ingestion time) lets new rows become queryable within seconds. BigQuery's serverless architecture automatically scales storage and compute to petabyte levels, eliminating cluster management. Enabling BigQuery BI Engine on the 90-day dataset keeps frequently accessed data in memory, giving Looker Studio dashboards sub-second performance without additional infrastructure.

In contrast, Cloud Bigtable provides low-latency key-value access but no native SQL and would require custom aggregation services. Cloud SQL cannot sustain millions of events per minute or scale to multi-petabyte volumes. Cloud Spanner is optimized for globally consistent OLTP workloads; using BigQuery federated queries on Spanner would add cost and complexity while still limiting scan performance for analytical dashboards.

Ask Bash

Bash is our AI bot, trained to help you pass your exam. AI Generated Content may display inaccurate information, always double-check anything important.