Your manufacturing company ingests several terabytes of IoT sensor data each day as partitioned Parquet files in a Cloud Storage data lake. Data engineering teams must give hundreds of analysts interactive SQL access while enforcing BigQuery row-level security policies and a unified audit trail. Management wants to avoid duplicating the data and minimise storage cost, but they still need BigQuery-style metadata caching for performance. Which architecture best satisfies these requirements?
Load the Parquet data into Cloud Bigtable using Dataflow and query it from BigQuery through a Bigtable external connection.
Run a daily Dataflow pipeline that loads the Parquet files into partitioned BigQuery native tables and apply column- and row-level security there.
Expose the bucket as a BigQuery external table and control access only through Cloud Storage IAM roles.
Create BigLake tables in BigQuery that reference the Parquet objects in Cloud Storage and apply row-level security policies on those tables.
BigLake tables extend BigQuery's external table capability by adding fine-grained authorization (row- and column-level security), uniform audit logging, and an intelligent caching layer-while leaving the underlying Parquet data in Cloud Storage. This meets the need for interactive SQL access with BigQuery semantics, unified governance, and no data duplication, keeping storage costs low. Loading the files into native BigQuery tables would satisfy governance but duplicates data and adds storage cost. Standard external tables over Cloud Storage avoid duplication, yet they cannot use BigQuery row-level security and rely only on bucket-level IAM, so they fail the governance requirement. Moving the data into Bigtable introduces a different engine that lacks BigQuery's row-level policies and incurs extra data movement and storage.
Ask Bash
Bash is our AI bot, trained to help you pass your exam. AI Generated Content may display inaccurate information, always double-check anything important.
What is a BigLake table in BigQuery?
Open an interactive chat with Bash
How does row-level security work in BigQuery?
Open an interactive chat with Bash
What are the advantages of using Parquet files for data storage?
Open an interactive chat with Bash
What is a BigLake table?
Open an interactive chat with Bash
What are the key differences between BigLake tables and BigQuery native tables?
Open an interactive chat with Bash
How does metadata caching improve performance in BigLake and BigQuery?
Open an interactive chat with Bash
GCP Professional Data Engineer
Storing the data
Your Score:
Report Issue
Bash, the Crucial Exams Chat Bot
AI Bot
Loading...
Loading...
Loading...
Pass with Confidence.
IT & Cybersecurity Package
You have hit the limits of our free tier, become a Premium Member today for unlimited access.
Military, Healthcare worker, Gov. employee or Teacher? See if you qualify for a Community Discount.
Monthly
$19.99 $11.99
$11.99/mo
Billed monthly, Cancel any time.
$19.99 after promotion ends
3 Month Pass
$44.99 $26.99
$8.99/mo
One time purchase of $26.99, Does not auto-renew.
$44.99 after promotion ends
Save $18!
MOST POPULAR
Annual Pass
$119.99 $71.99
$5.99/mo
One time purchase of $71.99, Does not auto-renew.
$119.99 after promotion ends
Save $48!
BEST DEAL
Lifetime Pass
$189.99 $113.99
One time purchase, Good for life.
Save $76!
What You Get
All IT & Cybersecurity Package plans include the following perks and exams .