GCP Professional Data Engineer Practice Question

Your team operates a long-running Dataproc cluster that executes a mission-critical Spark Streaming application. Traffic spikes are unpredictable, and recent surges have caused processing lag. You are asked to attach an autoscaling policy that (1) guarantees at least five primary worker nodes are always available and (2) can both grow quickly during spikes and shrink when load drops to control costs. Which policy configuration meets these requirements?

minNumWorkers = 0, scaleUpFactor = 0.5, scaleDownFactor = 0.5
minNumWorkers = 5, scaleUpFactor = 0.5, scaleDownFactor = 0.0, scaleDownMinWorkerFraction = 1.0
minNumWorkers = 5, scaleUpFactor = 0.5, scaleDownFactor = 0.5
minNumWorkers = 5, scaleUpFactor = 1.0, scaleDownFactor = 0.0

Report Issue

Answer Description

The minimum number of primary workers that Dataproc keeps alive is controlled by the minNumWorkers setting. Setting this value to 5 reserves five workers for the streaming job even when the cluster is otherwise idle. A scaleUpFactor of 0.5 means that up to 50 % of the unsatisfied YARN memory will be provisioned on each evaluation cycle, allowing the cluster to react quickly to demand. A scaleDownFactor of 0.5 permits Dataproc to remove up to half of the excess idle CPU cores each cycle, so the cluster can contract once the surge passes. Together, these parameters ensure resources never drop below the required floor while still enabling both rapid scale-out and scale-in. Options that leave minNumWorkers at 0 risk the cluster shrinking to zero workers, while configurations that set scaleDownFactor to 0 prevent any reduction in size and therefore do not meet the cost-control requirement.

Ask Bash

Bash is our AI bot, trained to help you pass your exam. AI Generated Content may display inaccurate information, always double-check anything important.