Microsoft Fabric Data Engineer Associate DP-700 Practice Question
A Microsoft Fabric pipeline triggers a PySpark notebook that transforms about 300 GB of Parquet files in a lakehouse. Several pipeline runs fail with the error message "org.apache.spark.SparkException: Job aborted due to stage failure - ExecutorLostFailure" although the same notebook sometimes finishes when executed manually. You need to determine the root cause of the failures and make the smallest possible change to prevent them from recurring. What should you do first?
Inspect the failed run in the Monitor hub, download the executor logs, and optimize the notebook's transformations (for example by repartitioning) based on the observed shuffle or spill issues.
Configure the Notebook activity in the pipeline to retry the run three times with a five-minute interval between attempts.
Add a setup cell that sets the configuration spark.executor.memoryOverhead to 4 GB before any transformations run.
Scale the workspace capacity to the next SKU tier to provide additional vCores for Spark compute.
Opening the failed Spark run in the Monitor hub lets you drill into the Spark UI and download the executor logs. Those logs typically show whether executors were killed because of excessive memory usage, shuffle spill, or data skew. With that information you can repartition or otherwise optimize the DataFrame so that the workload fits into the available resources. Merely increasing capacity, changing a single Spark configuration value, or adding automatic retries adds cost or masks the symptom without identifying the underlying problem.
Ask Bash
Bash is our AI bot, trained to help you pass your exam. AI Generated Content may display inaccurate information, always double-check anything important.
What is the Monitor hub in Microsoft Fabric?
Open an interactive chat with Bash
What are shuffle spill issues in Spark?
Open an interactive chat with Bash
How does data skew affect Spark performance?
Open an interactive chat with Bash
Microsoft Fabric Data Engineer Associate DP-700
Monitor and optimize an analytics solution
Your Score:
Report Issue
Bash, the Crucial Exams Chat Bot
AI Bot
Loading...
Loading...
Loading...
Pass with Confidence.
IT & Cybersecurity Package
You have hit the limits of our free tier, become a Premium Member today for unlimited access.
Military, Healthcare worker, Gov. employee or Teacher? See if you qualify for a Community Discount.
Monthly
$19.99
$19.99/mo
Billed monthly, Cancel any time.
3 Month Pass
$44.99
$14.99/mo
One time purchase of $44.99, Does not auto-renew.
MOST POPULAR
Annual Pass
$119.99
$9.99/mo
One time purchase of $119.99, Does not auto-renew.
BEST DEAL
Lifetime Pass
$189.99
One time purchase, Good for life.
What You Get
All IT & Cybersecurity Package plans include the following perks and exams .