GCP Professional Data Engineer Practice Question

To keep a mission-critical Spark Streaming job within a 1-minute latency objective, you run it on a persistent Dataproc cluster and attach a YARN-based autoscaling policy. Spikes must be absorbed quickly, but scale-downs must avoid repeatedly killing executors. Which policy setting combination best satisfies these constraints while balancing cost?

scaleUpFactor = 0.5, scaleDownFactor = 1.0, gracefulDecommissionTimeout = 30 minutes
scaleUpFactor = 0.2, scaleDownFactor = 0.1, gracefulDecommissionTimeout = 0 seconds
scaleUpFactor = 0.5, scaleDownFactor = 0.5, gracefulDecommissionTimeout = 0 seconds
scaleUpFactor = 1.0, scaleDownFactor = 0.1, gracefulDecommissionTimeout = 30 minutes

Report Issue

Answer Description

A high scale-up factor tells Dataproc to add enough workers to cover all pending YARN memory, so the cluster expands rapidly when traffic surges and protects the latency SLO. A low scale-down factor removes only a small fraction of the excess memory each evaluation cycle, preventing aggressive shrinkage that could thrash the job. Finally, a 30-minute graceful decommission timeout lets running tasks finish before workers are deleted, further protecting the streaming workload. The other options either scale up too slowly, scale down too aggressively, or terminate workers immediately, any of which risks missing the latency target or wasting resources.

Ask Bash

Bash is our AI bot, trained to help you pass your exam. AI Generated Content may display inaccurate information, always double-check anything important.