GCP Professional Data Engineer Practice Question

Your organization operates a Dataplex-managed data lake that spans three Google Cloud projects. Each project contains Cloud Storage buckets, always-running streaming Dataflow jobs, on-demand Dataproc clusters, and several BigQuery reservations. The governance team needs a single pane of glass that can (1) chart stored bytes and egress traffic per bucket, (2) show streaming backlog and YARN pending containers, (3) track month-to-date BigQuery slot-hours, and (4) fire alerts when storage costs or pipeline lag exceed predefined thresholds. You must rely only on native Google-provided metrics and avoid granting the monitoring analysts any editor roles on the workload projects. Which architecture best satisfies these requirements?

Schedule weekly Cloud Asset Inventory exports for each project, process them with Cloud Functions to calculate usage statistics, and send email notifications via Cloud Scheduler when thresholds are breached.
Enable Cloud Storage access logs, Dataflow job logs, and Dataproc job history export to Pub/Sub; write a Dataflow pipeline that aggregates the logs into BigQuery and build dashboards and alerts in Looker Studio.
Create a Cloud Monitoring workspace in a dedicated observability project, add the three lake projects to that workspace, and build dashboards and alerting policies using the built-in Cloud Storage, Dataflow, Dataproc, and BigQuery metric families.
Install OpenTelemetry collectors on every Dataproc cluster and Dataflow worker to push metrics to a self-managed Prometheus cluster hosted on Compute Engine, then visualize and alert with Grafana.

Report Issue

Answer Description

A Cloud Monitoring workspace created in a separate "observability" project can aggregate built-in metrics from multiple projects by simply adding those projects to the workspace. The monitoring team only needs the Monitoring Viewer/IAM roles on the workspace project, not editor rights on the data-lake projects. The workspace exposes the native metrics families for Cloud Storage (storage/total_bytes, network/egress_bytes_count), Dataflow (job/element_count, job/system_lag), Dataproc (hadoop/yarn/pending_memory), and BigQuery (slots/allocated). Dashboards and alerting policies can therefore be built entirely inside Cloud Monitoring without custom pipelines. The other options either collect metrics with custom pipelines, require third-party tooling, or deliver only periodic snapshots rather than real-time metric streams, so they do not fully meet the stated goals.

Ask Bash

Bash is our AI bot, trained to help you pass your exam. AI Generated Content may display inaccurate information, always double-check anything important.