Crucial Exams

GCP Professional Data Engineer Practice Question

Your team is writing an Apache Airflow DAG in Cloud Composer that launches an ephemeral Dataproc cluster, submits a Spark job, and then deletes the cluster. The cluster must be deleted even if the Spark job fails so you do not leave costly resources running. Which Airflow configuration will best ensure that the task deleting the cluster always runs, regardless of the outcome of the upstream Spark job task?

  • Assign the delete task to an isolated Airflow queue to ensure it is not blocked by failed tasks in the default queue.

  • Set the trigger_rule argument of the DataprocDeleteClusterOperator to "all_done" so it runs after upstream tasks complete in any state.

  • Increase the retries parameter on the Spark job task and rely on retry exhaustion to terminate the cluster if the job continues to fail.

  • Enable depends_on_past=True on the delete task so it ignores the status of the current run and always executes.

GCP Professional Data Engineer
Maintaining and automating data workloads
Your Score:
Settings & Objectives
Random Mixed
Questions are selected randomly from all chosen topics, with a preference for those you haven’t seen before. You may see several questions from the same objective or domain in a row.
Rotate by Objective
Questions cycle through each objective or domain in turn, helping you avoid long streaks of questions from the same area. You may see some repeat questions, but the distribution will be more balanced across topics.

Check or uncheck an objective to set which questions you will receive.

Wipe Score
Bash, the Crucial Exams Chat Bot
AI Bot