🔥 40% Off Crucial Exams Memberships — Deal ends today!

45 minutes, 0 second remaining!
Crucial Exams

GCP Professional Data Engineer Practice Question

Your team has packaged a nightly Apache Beam batch pipeline as a Dataflow Flex Template located at gs://analytics-pipelines/daily-etl.json. You need to expand an existing Cloud Composer DAG so that:

  • input_path and load_date values are passed to the template at run time
  • the Composer worker releases its slot immediately after the job is launched, yet downstream tasks must wait until the Dataflow job finishes successfully
  • any job still running after 6 hours is cancelled automatically to control costs

Which solution meets these requirements while requiring the least custom code?

  • Use DataflowStartFlexTemplateOperator with wait_until_finished=True and set its timeout to 6 hours, allowing the operator itself to block until completion or error.

  • Start the job with DataflowStartFlexTemplateOperator (wait_until_finished=False), passing input_path and load_date through the operator's body; chain a DataflowJobStatusSensor with a 6-hour timeout, and on sensor failure run a DataflowStopJobOperator to cancel the job before downstream tasks.

  • Invoke the template from a BashOperator that calls gcloud dataflow flex-template run, then use a PythonSensor to poll Cloud Logging for completion and a BashOperator to cancel the job if polling exceeds 6 hours.

  • Launch the template with DataflowTemplateOperator and set max_num_workers=0 so autoscaling stops the job after 6 hours, allowing downstream tasks to start immediately.

GCP Professional Data Engineer
Maintaining and automating data workloads
Your Score:
Settings & Objectives
Random Mixed
Questions are selected randomly from all chosen topics, with a preference for those you haven’t seen before. You may see several questions from the same objective or domain in a row.
Rotate by Objective
Questions cycle through each objective or domain in turn, helping you avoid long streaks of questions from the same area. You may see some repeat questions, but the distribution will be more balanced across topics.

Check or uncheck an objective to set which questions you will receive.

Wipe Score
Bash, the Crucial Exams Chat Bot
AI Bot