Microsoft Fabric Data Engineer Associate DP-700 Practice Question
You are authoring a PySpark notebook in Microsoft Fabric. A Delta table named Transactions contains the columns productId (string), saleTs (timestamp), qty (int), and unitPrice (decimal). For the nightly batch, you must create a DataFrame that keeps only the row with the most recent saleTs for each productId on each calendar day before writing the results back to the Lakehouse. Which transformation should you apply?
Create a Window partitioned by productId and to_date(col("saleTs")), ordered by col("saleTs").desc(); add row_number() over that window, then filter rows where row_number equals 1.
Select distinct rows, cache the DataFrame, and order by productId and saleTs descending.
Group by productId only, aggregate max("saleTs"), then join the result back to the original DataFrame.
Call dropDuplicates(["productId", "saleTs"]) to remove earlier sales for the same product on the same day.
Using a window specification that partitions by both productId and the calendar date guarantees that rows are compared only within the same product and day. Ordering the window by saleTs in descending order and then assigning a row_number allows you to flag the most recent record as 1. Filtering on row_number == 1 eliminates earlier rows while preserving all other columns. The other options either do not partition by day, lose non-key columns, or rely on dropDuplicates/distinct, which cannot identify the latest rows based on timestamps.
Ask Bash
Bash is our AI bot, trained to help you pass your exam. AI Generated Content may display inaccurate information, always double-check anything important.
What is a Window specification in PySpark?
Open an interactive chat with Bash
What does row_number() do in PySpark?
Open an interactive chat with Bash
What is the role of to_date(col("saleTs")) in the transformation?
Open an interactive chat with Bash
Microsoft Fabric Data Engineer Associate DP-700
Ingest and transform data
Your Score:
Report Issue
Bash, the Crucial Exams Chat Bot
AI Bot
Loading...
Loading...
Loading...
Pass with Confidence.
IT & Cybersecurity Package
You have hit the limits of our free tier, become a Premium Member today for unlimited access.
Military, Healthcare worker, Gov. employee or Teacher? See if you qualify for a Community Discount.
Monthly
$19.99
$19.99/mo
Billed monthly, Cancel any time.
3 Month Pass
$44.99
$14.99/mo
One time purchase of $44.99, Does not auto-renew.
MOST POPULAR
Annual Pass
$119.99
$9.99/mo
One time purchase of $119.99, Does not auto-renew.
BEST DEAL
Lifetime Pass
$189.99
One time purchase, Good for life.
What You Get
All IT & Cybersecurity Package plans include the following perks and exams .