Microsoft Fabric Data Engineer Associate DP-700 Practice Question

Broadcasting a very small table (such as the 5 000-row dimension table) lets Spark send a copy of that table to every executor, eliminating shuffles during the join. With a broadcast hash join, each executor can join its partition of the large FactSales table to the in-memory broadcasted lookup table locally, greatly reducing network I/O and execution time. Increasing the number of shuffle partitions adds task overhead without removing the shuffle. Repartitioning the large fact table triggers an additional full shuffle and still does not avoid the expensive join shuffle. Converting Delta files to CSV sacrifices ACID benefits while doing nothing to address the shuffle bottleneck. Therefore, applying the broadcast join hint to the dimension DataFrame is the most effective optimization here.