GCP Professional Data Engineer Practice Test

Session windows naturally group events that are separated by less than a specified gap, matching the 30-minute inactivity rule for user sessions. Adding an early processing-time trigger such as AfterProcessingTime.pastFirstElementInPane().plusDelayOf(5 minutes) produces interim aggregates every five minutes. Leaving the default on-time firing (watermark past end of window) ensures a pane is emitted when the 30-minute gap elapses, and setting allowedLateness to 15 minutes lets late events merge into the existing session. Using accumulating mode updates prior results without opening new windows. Fixed, sliding, or global windows fail to capture true user sessions or to meet late-data requirements without extra complexity.

The BigQuery Data Editor role (roles/bigquery.dataEditor) is scoped to datasets and grants permissions such as bigquery.tables.create and bigquery.tables.updateData, which allow a principal to create tables and write rows. Although it also includes bigquery.tables.getData (read access), it does not include permissions like bigquery.tables.update (alter table schemas) or bigquery.datasets.update (change access controls). Thus it provides the minimum required capabilities without granting schema-modification rights. BigQuery Data Owner and BigQuery Admin include schema and access-control permissions, violating least privilege. BigQuery Data Viewer is read-only, and BigQuery Job User controls job execution but provides no direct dataset write permissions. Therefore granting roles/bigquery.dataEditor on the dataset is the correct choice.

Cloud Dataflow executes Apache Beam pipelines that run in either true streaming or batch mode from the same code base. It supports user-defined transforms for complex cleansing and deduplication, side inputs for periodically refreshed lookup tables, exactly-once BigQuery sinks, and automatic horizontal scaling without any cluster to manage. Dataproc would require cluster provisioning and separate batch/stream jobs; Cloud Data Fusion and Dataform focus on GUI or SQL-based ETL and are not ideal for high-volume, low-latency streaming with code reuse across batch and streaming workloads.

Attaching a dedicated service account to the Compute Engine instances and granting it only the permissions required for the two target resources meets the principle of least privilege. Because the service account is bound to the VM, the workload can obtain short-lived OAuth 2.0 tokens from the instance metadata server at run time; no user credentials or downloadable keys need to be stored on disk, so key rotation happens automatically. Granting broad project-level roles or distributing user-managed keys would violate least-privilege goals and introduce operational overhead. Using HMAC or signed URLs still requires managing long-lived secrets and does not cover BigQuery access.

Cloud Dataflow automatically exports a rich set of built-in metrics (for example, elements-per-second, data freshness, CPU and memory usage) to Cloud Monitoring. Apache Beam user-defined counters can also be published as custom metrics that appear in the same Monitoring workspace. Using these metrics, engineers can build dashboards that combine Dataflow, Pub/Sub, and BigQuery data and can create alerting policies for latency and sustained CPU utilization. This provides a single pane of glass and proactive notifications without deploying or managing extra tooling.

The other options fall short:

• Enabling Cloud Trace surfaces trace spans but not CPU, memory, or custom Beam counters.
• Log-based metrics would require per-service extraction rules and still miss real-time Dataflow internals.
• Running Prometheus and Grafana introduces additional infrastructure and contradicts the low-overhead requirement.

Cloud External Key Manager (EKM) lets Google Cloud services such as BigQuery and Cloud Storage use an encryption key that is stored and managed in an external HSM. When you enable CMEK protection with an EKM key, the key material never resides in Google Cloud, yet BigQuery and Dataflow can transparently encrypt and decrypt data without code changes. Every call to the external key is captured in Cloud Audit Logs, satisfying the auditing requirement. Customer-supplied encryption keys cannot be used with BigQuery and require applications to pass the key on each request, while CMEK keys kept entirely inside Cloud KMS do not meet the mandate that the key stay on-premises. Default Google-managed keys provide no customer control.

Cloud Data Loss Prevention (DLP) can de-identify sensitive data by applying deterministic cryptographic transformations. When you configure a CryptoDeterministicConfig with a customer-managed (or external) key, DLP replaces each instance of the identified PII with a stable surrogate token: the same input value always maps to the same output, enabling joins across data sets. Because only the keyed cryptographic function can reverse the process, partners cannot recover the original PII, meeting GDPR data-minimization requirements.

Merely encrypting the BigQuery dataset with CMEK protects data at rest but still exposes clear-text PII to anyone who can query the table. Column-level access controls would block partners from seeing the identifiers entirely, preventing the required joins. Off-loading PII to Cloud SQL and sharing foreign keys would still leave linkage information exposed and complicate operations without providing managed tokenization. Therefore, deterministic tokenization with Cloud DLP and a customer-managed key is the correct solution.

Dataplex natively unifies governance for Cloud Storage and BigQuery by grouping assets into lakes and zones. When you attach a bucket or dataset to a zone, Dataplex automatically runs discovery jobs that register the assets in the Dataplex (Data Catalog) catalog, creates data profiles with statistics such as null counts, and can run built-in data quality scans. Access to the cataloged assets is controlled through IAM roles on Dataplex resources, giving fine-grained governance without custom pipelines. The other options either rely on separate services that do not provide automated profiling (BigQuery INFORMATION_SCHEMA), require custom functions and pipelines to keep metadata up to date, or use services (Cloud Asset Inventory, BigQuery Data Transfer Service) that are not intended for end-user data discovery and quality management.

Running two identical streaming jobs in separate regions provides active-active redundancy: if one region fails, the other continues to process new messages with no intervention, keeping RTO effectively zero and RPO limited only by Pub/Sub delivery guarantees. Using separate subscriptions prevents message acknowledgment coupling, and a multi-region BigQuery dataset remains reachable from either job. Idempotent or exactly-once semantics in the pipeline mitigate duplicate writes that can occur when both jobs are running. Relying on a single-region job, manual restarts from snapshots, or workflow-driven failover all introduce higher operational burden or longer recovery times that threaten the 15-minute RTO and 1-minute RPO targets.

BigQuery enforces column-level security through policy tags that live in Data Catalog taxonomies. By tagging each sensitive column with a PII policy tag and granting access to that tag only to the Risk group (via the Data Catalog Fine-Grained Reader or BigQuery Data Policy User role), you ensure they can query the protected columns while Marketing-who has dataset-level read rights but no access to the tag-cannot. When new PII columns are added, attaching the same policy tag immediately protects them, so no views or table restructuring is required. Row-level security filters rows, not columns; splitting tables or using authorized views would work but requires ongoing schema maintenance; per-column encryption with customer-managed keys is not natively enforced by BigQuery for column masking and would block all users without the key, not only Marketing.

The correct approach enforces the constraints as follows:

• Apply the organization-policy constraint constraints/gcp.resourceLocations to allow only EU regions, ensuring Cloud Storage buckets, Dataflow jobs, BigQuery datasets, and Cloud KMS key rings are created inside the EU.
• Within the Dataflow job, call Cloud DLP to perform deterministic encryption on the PAN field, using a customer-managed key (CMEK) stored in a Cloud KMS key ring located in an EU region. Deterministic encryption provides reversible pseudonymization while preserving referential integrity.
• Persist the transformed data to a BigQuery dataset in an EU region such as europe-west1, satisfying data-residency rules.
• Grant data analysts only the roles/bigquery.dataViewer role on the dataset so they can query pseudonymized data but lack any Cloud KMS permissions required to decrypt it.
• Grant the Dataflow worker service account the minimum required privileges: roles/bigquery.dataEditor on the target dataset and roles/cloudkms.cryptoKeyEncrypterDecrypter on the specific CMEK key, preventing broader access.

Alternative solutions fail to satisfy one or more requirements: allowing non-EU regions violates residency rules; using irreversible redaction or hashing breaks the reversibility requirement; storing keys outside the EU or granting analysts decrypter privileges violates compliance and least-privilege principles.

Because the regulation stipulates that encryption keys must stay in an on-premises FIPS 140-2 Level 3 HSM under the customer's sole control, the only Google Cloud option that satisfies this is Cloud External Key Manager (EKM). With EKM, BigQuery can be configured to use a customer-managed encryption key that never leaves the customer-owned HSM; Google Cloud retrieves key material on-demand over a secure channel, so the data remains encrypted at rest with an externally hosted key. Key rotation is handled in the external HSM and is transparent to BigQuery clients, so no SQL jobs or application code need to change.

Using CMEK backed by Cloud HSM would store the key inside Google Cloud, violating the requirement that keys remain on-premises. Default Google-managed keys do not satisfy customer-control or residency requirements. Customer-supplied encryption keys (CSEK) are not supported for BigQuery and require applications to supply a key with every request, which would break existing workloads and fail to automate rotation. Therefore, configuring BigQuery with an external key through Cloud EKM is the correct solution.

Deterministic cryptographic transformation with a centrally-managed Cloud KMS key converts identical names to the same surrogate value across all branches, so joins are still possible and the process can be reversed only by a team that controls the key.
Character masking that replaces the first 12 digits of the PAN with a fixed symbol irreversibly removes sensitive data yet keeps the last four digits available to analysts.
The other proposals either (1) use format-preserving encryption that is still reversible by anyone with a key, (2) encrypt data in bulk without selectively exposing the last four digits, or (3) rely only on access controls without actually de-identifying the data, and therefore do not satisfy the stated compliance goals.

Datastream is Google Cloud's fully managed, serverless change-data-capture (CDC) and replication service. It supports Oracle sources and can continuously stream inserts, updates, and deletes to BigQuery with sub-second to minute-level latency, requiring little operational overhead.

The BigQuery Data Transfer Service only performs scheduled batch loads and cannot read directly from Oracle. Transfer Appliance is designed for one-time, offline bulk transfers of files to Cloud Storage, not continuous CDC, so it would not keep the dataset current. Database Migration Service focuses on moving databases into Cloud SQL (and currently offers limited support for Oracle) and would still require additional tooling to feed changes into BigQuery. Therefore, Datastream best meets the requirement for managed, serverless, near-real-time replication from an on-prem Oracle database to BigQuery.

Running Google's open-source Data Validation Tool (DVT) as a Dataflow flex template fulfills all constraints. The template compares row counts and column-level checksums between Oracle and BigQuery for every table based on a single YAML configuration, so no table-specific code is required. It writes detailed match and mismatch results to BigQuery tables and Cloud Logging, from which log-based metrics can feed Cloud Monitoring alert policies. Datastream lacks built-in validation, custom Dataflow jobs would require per-table logic, BigQuery snapshots with manual SQL diffing do not scale, and BigQuery Data Transfer Service offers no validation capability. Therefore, deploying the DVT Dataflow template is the most efficient, scalable, and monitorable option.

Cloud Data Loss Prevention (DLP) offers fully managed inspection and de-identification capabilities that scale automatically. By calling DLP from the "Cloud Storage Text to BigQuery with Cloud DLP" Dataflow template and configuring a CryptoDeterministicConfig that uses a customer-managed Cloud KMS key, all detected PII (for example, PERSON_NAME and PHONE_NUMBER infoTypes) is replaced with stable, non-reversible tokens before the data is written to BigQuery. Because the same source value always maps to the same token when the same key is used, analysts can reliably join or aggregate records that refer to the same individual without exposing the original sensitive values. BigQuery's column-level security or views alone would leave the raw PII in storage, and client-side encryption or ad-hoc regex masking would break deterministic linking and add significant custom development and maintenance. Therefore, orchestrating a Dataflow DLP de-identification template with deterministic cryptographic tokenization best satisfies the privacy, scalability, and maintainability requirements.

Storing the raw tables in a single-region EU dataset (for example europe-west1) keeps data physically in the EU. Enforcing the organization policy constraint gcp.resourceLocations ensures no one can accidentally create resources outside approved EU regions. An authorized view can expose only aggregated results, so analysts can run their queries without gaining direct access to the sensitive columns. Granting analysts bigquery.dataViewer on the dataset that contains the views plus bigquery.jobUser on the project lets them execute queries but not modify data. The Dataflow pipeline needs to load and overwrite table partitions, so bigquery.dataEditor on the raw dataset is sufficient-there is no need for broader Owner privileges. The other options either do not restrict residency to the EU, expose raw columns through insufficient controls, or assign overly broad IAM roles.

The Editor primitive role grants thousands of permissions across nearly every Google Cloud service, including the ability to create, modify, and delete resources such as Compute Engine instances and Cloud Storage buckets. To comply with least-privilege guidelines, you should remove this broad role and replace it with a predefined BigQuery-specific role that contains only the permissions required for the scientists' tasks. Granting roles/bigquery.dataEditor at the dataset level lets them create and update tables without exposing the project to unnecessary risk. The other options either continue to over-provision access, add unnecessary impersonation complexity, or rely solely on monitoring rather than removing excessive permissions.

Dataplex can attach Cloud Storage buckets and BigQuery datasets from any project into logical data lakes and zones. When an asset is attached, Dataplex's built-in metadata service automatically crawls the underlying storage, extracts technical metadata (schemas, locations, partitions), and surfaces it-together with user-defined business tags-in the Dataplex Universal Catalog. The catalog entries reside in Data Catalog, so analysts can use Data Catalog search APIs to look for business terms such as "video-stream metrics" or "ad-impressions" without knowing the physical location of the data. IAM policies applied at the project, dataset, or bucket level are respected because Dataplex manages only references and does not copy data. Running a third-party catalog or exporting metadata to another system would add operational overhead and would not benefit from Dataplex's tight integration with Google Cloud services and IAM, reducing portability and increasing maintenance effort. Therefore, enabling Dataplex, organizing assets into lakes and zones, and relying on the Universal Catalog best meets the stated goals.

BigQuery regional locations already replicate table data synchronously to at least two separate zones inside the same region and automatically reroute queries to healthy replicas if a zone is unavailable. This provides a Recovery Point Objective of zero and near-instant failover (very low RTO) for single-zone outages without any additional storage, pipelines, or operational overhead. Moving to a multi-region location or creating cross-region copies would exceed the stated requirements and add cost or complexity, while periodic exports or scheduled transfers would introduce an RPO greater than zero and require extra maintenance. Therefore, no architectural change is required.