Your team schedules an hourly Cloud Composer DAG that triggers a Dataflow batch job to aggregate the previous hour's click-stream data and then archives the raw files. After an upcoming four-hour maintenance pause you must ensure that Composer does not automatically create four delayed DAG runs, yet any individual task in the next scheduled run must retry up to three times at 10-minute intervals if it fails. With minimal custom code, how should you configure the DAG?
Disable catchup but set max_active_runs=24, and require any failed run to be relaunched manually.
Leave catchup enabled, set execution_timeout=600 seconds, and disable retries to avoid duplicate processing.
Set schedule_interval='@hourly', configure catchup=False, and add retries: 3 and retry_delay: timedelta(minutes=10) in default_args.
Use schedule_interval='@hourly', keep catchup=True, and add depends_on_past=True with the same retry settings.
Setting catchup=False on the DAG tells Airflow (and therefore Cloud Composer) not to create runs for any schedule intervals that were missed while the environment was paused, eliminating unwanted backfill. Defining retries=3 and retry_delay=timedelta(minutes=10) in the DAG's default_args enables each task to retry up to three times with a 10-minute wait between attempts. Using the canonical hourly schedule expression (@hourly or an equivalent cron string) keeps the schedule simple. The alternative choices either leave catchup enabled (causing backfill) or omit automatic retries, or require additional manual intervention, so they do not satisfy both requirements as cleanly.
Ask Bash
Bash is our AI bot, trained to help you pass your exam. AI Generated Content may display inaccurate information, always double-check anything important.
What is DAG and how does it work in Cloud Composer?
Open an interactive chat with Bash
What is the purpose of `catchup` in Cloud Composer?
Open an interactive chat with Bash
What happens if retries and retry delays are not configured in the DAG?
Open an interactive chat with Bash
What does `catchup=False` do in Cloud Composer?
Open an interactive chat with Bash
How does `retries` and `retry_delay` in `default_args` impact task execution?
Open an interactive chat with Bash
What is the purpose of `schedule_interval='@hourly'` in a DAG?
Open an interactive chat with Bash
GCP Professional Data Engineer
Maintaining and automating data workloads
Your Score:
Report Issue
Bash, the Crucial Exams Chat Bot
AI Bot
Loading...
Loading...
Loading...
Pass with Confidence.
IT & Cybersecurity Package
You have hit the limits of our free tier, become a Premium Member today for unlimited access.
Military, Healthcare worker, Gov. employee or Teacher? See if you qualify for a Community Discount.
Monthly
$19.99
$19.99/mo
Billed monthly, Cancel any time.
3 Month Pass
$44.99
$14.99/mo
One time purchase of $44.99, Does not auto-renew.
MOST POPULAR
Annual Pass
$119.99
$9.99/mo
One time purchase of $119.99, Does not auto-renew.
BEST DEAL
Lifetime Pass
$189.99
One time purchase, Good for life.
What You Get
All IT & Cybersecurity Package plans include the following perks and exams .