AWS Certified Data Engineer Associate DEA-C01 Practice Question
Your data lake stores hourly IoT records in Amazon S3 using the prefix structure raw/iot/year=YYYY/month=MM/day=DD/hour=HH/. Analysts query the data with Amazon Athena. You must ensure that new hourly partitions become available for querying within five minutes of arrival, while keeping operational overhead and crawl costs low. Which solution meets these requirements?
Create an AWS Step Functions workflow that runs the statement MSCK REPAIR TABLE in Athena every five minutes.
Configure an Amazon S3 event notification that triggers an AWS Glue crawler set to "Crawl new folders only" on the dataset prefix.
Enable Amazon Athena partition projection and disable the AWS Glue crawler for the table.
Use AWS Database Migration Service (AWS DMS) to replicate new S3 objects into the AWS Glue Data Catalog as partitions.
An Amazon S3 event notification can invoke an AWS Glue crawler whenever a new object is written to a partition folder. By configuring the crawler's recrawl policy to "Crawl new folders only", the crawler adds just the newly detected partitions to the existing Glue Data Catalog table rather than re-crawling the entire dataset. This keeps catalog metadata current within minutes and minimizes both API calls and costs.
Automatic partition projection eliminates the need for a crawler but requires manually defining projection parameters; the scenario specifically asks to synchronize partitions with the catalog. Scheduling MSCK REPAIR TABLE every five minutes or running ALTER TABLE statements introduces more operational overhead and still scans the entire table each time, increasing cost and complexity. Using AWS DMS is intended for database replication and does not update Glue catalog partitions for S3 data.
Ask Bash
Bash is our AI bot, trained to help you pass your exam. AI Generated Content may display inaccurate information, always double-check anything important.
What is an AWS Glue crawler?
Open an interactive chat with Bash
What is Amazon Athena partition projection?
Open an interactive chat with Bash
Why is running MSCK REPAIR TABLE in Athena not an ideal solution here?
Open an interactive chat with Bash
AWS Certified Data Engineer Associate DEA-C01
Data Store Management
Your Score:
Report Issue
Bash, the Crucial Exams Chat Bot
AI Bot
Loading...
Loading...
Loading...
Pass with Confidence.
IT & Cybersecurity Package
You have hit the limits of our free tier, become a Premium Member today for unlimited access.
Military, Healthcare worker, Gov. employee or Teacher? See if you qualify for a Community Discount.
Monthly
$19.99
$19.99/mo
Billed monthly, Cancel any time.
3 Month Pass
$44.99
$14.99/mo
One time purchase of $44.99, Does not auto-renew.
MOST POPULAR
Annual Pass
$119.99
$9.99/mo
One time purchase of $119.99, Does not auto-renew.
BEST DEAL
Lifetime Pass
$189.99
One time purchase, Good for life.
What You Get
All IT & Cybersecurity Package plans include the following perks and exams .