Your analytics team stores raw log files in several Cloud Storage buckets and curated tables in BigQuery datasets that span four GCP projects. They complain that discovering datasets is difficult and security administrators want a single place to apply and audit access policies and tag sensitive columns. You are asked to implement a solution that automatically keeps metadata current on a nightly schedule without moving data or deploying custom crawlers. What should you do?
Create a Dataplex lake spanning the four projects, add the buckets and datasets as assets in appropriate zones, enable automatic discovery with a daily schedule, and manage policy tags and IAM centrally in the Dataplex Catalog.
Enable Cloud Asset Inventory across the projects and export the inventory to BigQuery; schedule a Cloud Function to update column-level tags each night.
Build a nightly Dataflow job that reads object metadata and BigQuery INFORMATION_SCHEMA views, writes the results to a central BigQuery table, and controls access through BigQuery-level IAM.
Deploy an Apache Atlas cluster on GKE to crawl the buckets and datasets nightly and expose the collected metadata through its REST API.
Dataplex provides a lake-centric abstraction that can reference Cloud Storage buckets and BigQuery datasets as assets without copying data. When discovery is enabled on a zone, Dataplex automatically scans the underlying storage on the configured schedule (hourly, daily, or weekly) and syncs technical metadata, partitions, and inferred schema into the unified Dataplex Catalog (built on Data Catalog). Because the assets remain in their original projects, IAM policies and column-level policy tags can be managed centrally from Dataplex and automatically apply to both BigQuery and Cloud Storage. The alternative options rely on custom code or third-party services and do not offer the built-in, policy-driven metadata discovery requested.
Ask Bash
Bash is our AI bot, trained to help you pass your exam. AI Generated Content may display inaccurate information, always double-check anything important.
What is Dataplex in GCP?
Open an interactive chat with Bash
What is automatic discovery in Dataplex?
Open an interactive chat with Bash
What are IAM policies and policy tags in Dataplex?
Open an interactive chat with Bash
What is Dataplex?
Open an interactive chat with Bash
How does automatic discovery work in Dataplex?
Open an interactive chat with Bash
What are IAM and policy tags in Dataplex?
Open an interactive chat with Bash
GCP Professional Data Engineer
Storing the data
Your Score:
Report Issue
Bash, the Crucial Exams Chat Bot
AI Bot
Loading...
Loading...
Loading...
Pass with Confidence.
IT & Cybersecurity Package
You have hit the limits of our free tier, become a Premium Member today for unlimited access.
Military, Healthcare worker, Gov. employee or Teacher? See if you qualify for a Community Discount.
Monthly
$19.99
$19.99/mo
Billed monthly, Cancel any time.
3 Month Pass
$44.99
$14.99/mo
One time purchase of $44.99, Does not auto-renew.
MOST POPULAR
Annual Pass
$119.99
$9.99/mo
One time purchase of $119.99, Does not auto-renew.
BEST DEAL
Lifetime Pass
$189.99
One time purchase, Good for life.
What You Get
All IT & Cybersecurity Package plans include the following perks and exams .