GCP Professional Data Engineer Practice Question

A smart-city analytics team ingests billions of JSON sensor readings each day through Pub/Sub and immediately writes them to a raw staging location. Compliance rules require that the unmodified records be retained for five years, with older data automatically moved to colder, less expensive storage classes. Engineers will later run Dataflow jobs that cleanse the data and load curated subsets into BigQuery on demand. Which sink best satisfies the retention, cost, and future-processing requirements for the raw data layer?

Persist the records in a wide-column Cloud Bigtable instance
Insert the records into a partitioned BigQuery table using streaming inserts
Store the records as objects in a Cloud Storage bucket with lifecycle rules
Load the records into a Cloud SQL PostgreSQL database and enable point-in-time recovery

Report Issue

Answer Description

Cloud Storage is designed for data-lake use cases where raw, unstructured, or semi-structured files must be kept for long periods at low cost. Bucket lifecycle policies can automatically transition objects from Standard to Nearline, Coldline, or Archive classes to meet five-year retention at minimal expense. Dataflow can natively read from and write to Cloud Storage, allowing downstream cleansing and loading into BigQuery without relocating the data. BigQuery is an analytics warehouse optimized for SQL query performance; loading every raw record would incur storage and streaming-insert costs and prevents tiered lifecycle storage. Cloud Bigtable provides low-latency key/value serving but lacks object lifecycle tiering and is cost-inefficient for write-once archival data. Cloud SQL is a managed relational OLTP service with size limits, higher per-GB cost, and no automated tiering, making it unsuitable for petabyte-scale, append-only raw data.

Ask Bash

Bash is our AI bot, trained to help you pass your exam. AI Generated Content may display inaccurate information, always double-check anything important.