Microsoft Fabric Data Engineer Associate DP-700 Practice Question

In Spark Structured Streaming, state created by windowed aggregations is kept for as long as late data might still arrive for a given window. You therefore declare a watermark on the event-time column that specifies the maximum allowed lateness. A call to withWatermark("eventTime", "8 minutes") tells Spark to retain state for 8 minutes beyond each window's end time; after that, the engine can safely drop the state and any later arriving records are ignored. This satisfies the requirement to include events that arrive within eight minutes while preventing unbounded state growth.

Using a processing-time trigger controls how often results are emitted but does not bound event-time state. Setting outputMode("update") only changes what is written, not how long state is kept. dropDuplicates removes repeated rows, not state kept by aggregations.