During business hours, data analysts regularly hit BigQuery's 100-query interactive concurrency limit, and their queries fail with HTTP 429 errors indicating too many concurrent requests. A separate hourly ETL workflow launches several long-running SQL transformation jobs that worsen the congestion. You need to keep the analysts productive without purchasing additional slot capacity or reservations. What should you do?
Migrate the ETL workload to a persistent Dataproc cluster and write its results back to BigQuery.
Purchase a BigQuery reservation of 500 slots for the analysts and leave the ETL workload on on-demand capacity.
Apply a maximum bytes billed limit to each ETL query to reduce its resource usage and avoid concurrency errors.
Submit the ETL queries with batch priority so they queue until idle capacity is available and do not count toward the interactive concurrency limit.
Submitting the ETL statements as batch-priority queries queues them for execution only when idle resources are available. Because batch jobs do not consume interactive-query concurrency slots, they cannot block the analysts' interactive queries. Purchasing reservations would add cost, moving the workload to Dataproc would not eliminate the concurrency limit for the analysts' interactive queries, and setting a maximum-bytes-billed cap controls cost rather than concurrency.
Ask Bash
Bash is our AI bot, trained to help you pass your exam. AI Generated Content may display inaccurate information, always double-check anything important.
What is the difference between batch priority and interactive priority in BigQuery?
Open an interactive chat with Bash
What is BigQuery's 100-query interactive concurrency limit?
Open an interactive chat with Bash
How does HTTP 429 error help indicate concurrency issues in BigQuery?
Open an interactive chat with Bash
What is BigQuery's interactive concurrency limit?
Open an interactive chat with Bash
What is the difference between batch priority and interactive priority in BigQuery?
Open an interactive chat with Bash
What does submitting ETL workflows as batch-priority queries achieve?
Open an interactive chat with Bash
GCP Professional Data Engineer
Maintaining and automating data workloads
Your Score:
Report Issue
Bash, the Crucial Exams Chat Bot
AI Bot
Loading...
Loading...
Loading...
Pass with Confidence.
IT & Cybersecurity Package
You have hit the limits of our free tier, become a Premium Member today for unlimited access.
Military, Healthcare worker, Gov. employee or Teacher? See if you qualify for a Community Discount.
Monthly
$19.99
$19.99/mo
Billed monthly, Cancel any time.
3 Month Pass
$44.99
$14.99/mo
One time purchase of $44.99, Does not auto-renew.
MOST POPULAR
Annual Pass
$119.99
$9.99/mo
One time purchase of $119.99, Does not auto-renew.
BEST DEAL
Lifetime Pass
$189.99
One time purchase, Good for life.
What You Get
All IT & Cybersecurity Package plans include the following perks and exams .