GCP Professional Data Engineer Practice Question

Your genomics research institute must move 3 PB of raw sequencing files from an on-premises HPC cluster to a new data lake in a regional Cloud Storage bucket. The data center's WAN link averages 150 Mbps and cannot be upgraded in time. Compliance prohibits transferring this data across the public internet, yet the migration must finish within one month so downstream analytics in BigQuery can begin on schedule. Which Google solution best meets these constraints while requiring the least change to existing infrastructure?

Configure BigQuery Data Transfer Service to pull the sequencing files directly into BigQuery as external data sources.
Provision Dedicated Interconnect and use gsutil -m rsync to stream the data from the cluster to Cloud Storage.
Install Storage Transfer Service agents on the HPC cluster and schedule continuous transfers to Cloud Storage over the existing link.
Order a Google Transfer Appliance, copy the datasets locally, then return it for bulk ingestion into Cloud Storage.

Report Issue

Answer Description

With only a 150 Mbps link, sending 3 PB online would take many months, exceeding the one-month window. Google Cloud Transfer Appliance ships an encrypted, high-capacity (up to 1 PB per device) storage server to the customer site, allowing data to be copied locally at LAN speed. The appliance is then returned to Google, where its contents are uploaded directly into the target Cloud Storage bucket inside Google's network-satisfying both the tight timeline and the requirement to avoid public-internet transfer.

Other options move data over the network, so they are constrained by the 150 Mbps link:

Cloud Storage Transfer Service with an on-prem agent performs online transfers and would not meet the schedule.
Cloud Interconnect still requires data to traverse the link to reach the edge and cannot be provisioned quickly enough.
BigQuery Data Transfer Service handles SaaS or cloud-resident sources, not bulk on-prem file systems. Therefore, Transfer Appliance is the only viable choice.

Ask Bash

Bash is our AI bot, trained to help you pass your exam. AI Generated Content may display inaccurate information, always double-check anything important.