GCP Professional Data Engineer Practice Question

Your trading platform ingests about 5 million JSON trade events per second from globally distributed exchanges. Events are append-only and must be retrieved within a few milliseconds by a risk-evaluation service that scans all trades from the most recent hour using the event timestamp. Data should be kept for 30 days and then bulk-exported to BigQuery for historical analytics. Which Google Cloud storage service best fits the raw event store, given these ingestion, low-latency, and range-scan requirements?

Store the events in Cloud Bigtable using a row key that starts with an inverted timestamp followed by the trade ID.
Store each event as a document in Firestore (Native mode) keyed by trade ID, and query recent trades with composite indexes.
Stream each event into a partitioned BigQuery table and let the risk-evaluation service query the latest partition.
Batch events into hourly JSON files in a Cloud Storage Nearline bucket and have the service read the most recent file.

Report Issue

Answer Description

Cloud Bigtable is engineered for very high write throughput and single-digit millisecond read latency at petabyte scale. Designing the row key with a reversed timestamp (optionally combined with a unique suffix such as trade ID) supports time-ordered range scans that the risk-evaluation service needs. Bigtable can easily handle millions of mutations per second and integrates with Dataflow or Data Dataproc for periodic exports to BigQuery. Firestore can provide millisecond reads but its write throughput per collection and query patterns do not scale to millions of events per second, and range scans over large time windows quickly become inefficient. BigQuery excels at analytics but its query latency (seconds) and streaming cost model are unsuitable for sub-millisecond operational access. Cloud Storage provides durable object storage, yet retrieving individual events with millisecond latency or performing range scans over numerous small objects is not practical.

Ask Bash

Bash is our AI bot, trained to help you pass your exam. AI Generated Content may display inaccurate information, always double-check anything important.