A retail analytics team deploys the following daily Cloud Composer workflow:
LocalFilesystemToGCSOperator uploads the previous day's CSV files
DataflowTemplateOperator transforms the files
GCSToBigQueryOperator loads the results into a partitioned table
After the DAG is first published, Airflow immediately tries to schedule every daily interval back to the start_date, quickly exceeding the project's Dataflow quota. The team wants the DAG to run only for intervals that begin after the deployment moment, while keeping the existing schedule_interval and start_date. What modification to the DAG definition will accomplish this goal?
Change schedule_interval to None to prevent historical and future automatic scheduling.
Set depends_on_past=True so each run waits for the previous one to finish before starting.
Configure max_active_runs=1 to limit the DAG to a single concurrent run.
Add catchup=False to the DAG constructor to disable back-filling of missed intervals.
Airflow's scheduler generates one DAG run for each unexecuted schedule interval between the start_date and the current time. Setting the catchup parameter to False when constructing the DAG disables this back-fill behavior, so the scheduler will create a run only for the most recent interval (and subsequent ones). Changing depends_on_past or max_active_runs would still create all of the historical runs-those parameters merely affect ordering and concurrency. Setting schedule_interval to None would stop all future automated runs, which is not the requirement.
Ask Bash
Bash is our AI bot, trained to help you pass your exam. AI Generated Content may display inaccurate information, always double-check anything important.
What is the purpose of Airflow's `catchup=False` parameter?
Open an interactive chat with Bash
How does `max_active_runs=1` differ from `catchup=False` in Airflow?
Open an interactive chat with Bash
What happens when a DAG uses `schedule_interval=None`?
Open an interactive chat with Bash
What does `catchup=False` do in an Airflow DAG?
Open an interactive chat with Bash
What is the role of `schedule_interval` in Airflow?
Open an interactive chat with Bash
Why is `depends_on_past` not suitable for preventing back-filling?
Open an interactive chat with Bash
GCP Professional Data Engineer
Maintaining and automating data workloads
Your Score:
Report Issue
Bash, the Crucial Exams Chat Bot
AI Bot
Loading...
Loading...
Loading...
Pass with Confidence.
IT & Cybersecurity Package
You have hit the limits of our free tier, become a Premium Member today for unlimited access.
Military, Healthcare worker, Gov. employee or Teacher? See if you qualify for a Community Discount.
Monthly
$19.99
$19.99/mo
Billed monthly, Cancel any time.
3 Month Pass
$44.99
$14.99/mo
One time purchase of $44.99, Does not auto-renew.
MOST POPULAR
Annual Pass
$119.99
$9.99/mo
One time purchase of $119.99, Does not auto-renew.
BEST DEAL
Lifetime Pass
$189.99
One time purchase, Good for life.
What You Get
All IT & Cybersecurity Package plans include the following perks and exams .