GCP Professional Data Engineer Practice Question

You are designing the vector store layer of a retrieval-augmented generation (RAG) pipeline for 10 million technical-manual paragraphs. Each paragraph will be embedded with ML.GENERATE_EMBEDDING using the Vertex AI model textembedding-gecko. At query time, a user question must be embedded once and matched against the stored vectors with millisecond-level latency directly inside BigQuery. Future product manuals will arrive daily, so the solution must keep the index current without manual rebuilds and avoid unnecessary I/O when the RAG query runs. Which approach best meets these requirements?

Store each paragraph and its ARRAY embedding in a partitioned, clustered BigQuery table, then create a VECTOR index on the embedding column for approximate nearest-neighbor search.
Build a materialized view that calls ML.GENERATE_EMBEDDING at query time so vectors are not stored; scan the view with a WHERE clause that compares embeddings using the COSINE_DISTANCE function.
Export embeddings as JSON to Cloud Storage, ingest them into Vertex AI Matching Engine, and perform the similarity search there, returning IDs to BigQuery for a join.
Write embeddings to an external BigLake table over Cloud Storage files and invoke ML.ANN_SEARCH with a full brute-force scan on every query.

Report Issue

Answer Description

Storing the paragraph text and its embedding together in a native BigQuery table keeps all data in the same analytic system, so no cross-service joins are needed. Persisting the embedding as an ARRAY column allows creation of a VECTOR index, which BigQuery keeps up to date automatically as new rows are inserted. Partitioning by ingestion date and clustering by product identifier limits the data scanned to only the relevant partitions and clusters, further reducing latency and cost. External BigLake files (choice B) or an external Matching Engine (choice C) introduce extra network hops and join logic that add latency. Generating embeddings on-the-fly through a materialized view (choice D) prevents index creation and forces full compute for every query, violating the latency goal.

Ask Bash

Bash is our AI bot, trained to help you pass your exam. AI Generated Content may display inaccurate information, always double-check anything important.