AWS Certified Data Engineer Associate DEA-C01 Practice Question
An analytics team stores click-stream data as Parquet files in Amazon S3, partitioned by year/month/day (for example, s3://datalake/events/year=2025/month=10/day=07/). A daily AWS Glue crawler adds partitions to the AWS Glue Data Catalog so analysts can query the table in Amazon Athena. After two years the crawler's runtime and cost have increased significantly. The team wants to keep automatic partition discovery while minimizing ongoing cost and administration. What should they do?
Enable partition projection for the Athena table, configure the year, month, and day keys, and stop scheduling the AWS Glue crawler.
Change the existing crawler's recrawl policy to crawl new folders only and enable partition indexes on the Data Catalog table.
Create an AWS Lambda function that runs MSCK REPAIR TABLE after each crawler run to update the Data Catalog incrementally.
Switch to Amazon S3 event notifications that invoke an AWS Glue job calling the batchCreatePartition API to add each new partition to the Data Catalog.
Athena partition projection lets you define the partition keys (year, month, day) as template variables. Athena then resolves the partitions at query time instead of reading them from the AWS Glue Data Catalog. Once projection is configured, the table no longer needs explicit partition objects, so the daily crawler can be disabled, eliminating both the crawl time and the related cost.
Using an S3 event-driven AWS Glue job or Lambda function would still require authoring and maintaining custom code. Setting the crawler to recrawl only new folders reduces, but does not eliminate, the growing scan cost. Glue partition indexes accelerate certain lookups but do not shorten crawler runtime or remove the need to maintain partitions.
Ask Bash
Bash is our AI bot, trained to help you pass your exam. AI Generated Content may display inaccurate information, always double-check anything important.
What is Athena partition projection?
Open an interactive chat with Bash
How do S3 event notifications work with AWS Glue?
Open an interactive chat with Bash
What is MSCK REPAIR TABLE and how does it differ from partition projection?
Open an interactive chat with Bash
What is Athena Partition Projection?
Open an interactive chat with Bash
Why is enabling Partition Projection more efficient than using AWS Glue crawlers?
Open an interactive chat with Bash
How does using S3 event-driven AWS Glue jobs differ from Partition Projection?
Open an interactive chat with Bash
AWS Certified Data Engineer Associate DEA-C01
Data Store Management
Your Score:
Report Issue
Bash, the Crucial Exams Chat Bot
AI Bot
Loading...
Loading...
Loading...
Pass with Confidence.
IT & Cybersecurity Package
You have hit the limits of our free tier, become a Premium Member today for unlimited access.
Military, Healthcare worker, Gov. employee or Teacher? See if you qualify for a Community Discount.
Monthly
$19.99
$19.99/mo
Billed monthly, Cancel any time.
3 Month Pass
$44.99
$14.99/mo
One time purchase of $44.99, Does not auto-renew.
MOST POPULAR
Annual Pass
$119.99
$9.99/mo
One time purchase of $119.99, Does not auto-renew.
BEST DEAL
Lifetime Pass
$189.99
One time purchase, Good for life.
What You Get
All IT & Cybersecurity Package plans include the following perks and exams .