AWS Certified Data Engineer Associate DEA-C01 Practice Question
A data engineer stores application logs in Amazon S3 using the prefix pattern s3://company-logs/year=YYYY/month=MM/day=DD/. An AWS Glue Data Catalog table is used by analysts to query the data with Amazon Athena. New log files arrive every hour, and the analysts must be able to query them within five minutes with minimal operational cost and without manual partition updates. Which solution meets these requirements most cost-effectively?
Configure the Athena table to use partition projection with date-based ranges so that new partitions are inferred at query time without crawling.
Use Amazon S3 event notifications to invoke an AWS Lambda function that calls the AWS Glue BatchCreatePartition API whenever a new day folder is created.
Enable AWS Glue partition indexes and run the existing crawler once per day; Athena will detect new partitions automatically through the index.
Schedule an AWS Glue crawler with an S3 data store to run every five minutes so that new partitions are discovered quickly.
Partition projection in Amazon Athena eliminates the need to crawl or explicitly add partitions. The table definition contains template variables that Athena resolves at query time, so newly arriving data is immediately queryable. This approach has no recurring crawler or Lambda execution cost. Running an AWS Glue crawler every five minutes would meet the latency goal but would be more expensive. Partition indexes speed up metadata lookups but still require partition creation. An S3-triggered Lambda that calls BatchCreatePartition meets the latency goal but introduces additional operational overhead and Lambda costs.
Ask Bash
Bash is our AI bot, trained to help you pass your exam. AI Generated Content may display inaccurate information, always double-check anything important.
What is partition projection in Amazon Athena?
Open an interactive chat with Bash
How do S3 event notifications work for processing new data?
Open an interactive chat with Bash
Why is using an AWS Glue crawler costly for frequent updates?
Open an interactive chat with Bash
What is partition projection in Amazon Athena?
Open an interactive chat with Bash
How are S3 event notifications used with AWS Lambda for partition management?
Open an interactive chat with Bash
What are the advantages of using Athena partition projection over Glue crawlers?
Open an interactive chat with Bash
AWS Certified Data Engineer Associate DEA-C01
Data Store Management
Your Score:
Report Issue
Bash, the Crucial Exams Chat Bot
AI Bot
Loading...
Loading...
Loading...
Pass with Confidence.
IT & Cybersecurity Package
You have hit the limits of our free tier, become a Premium Member today for unlimited access.
Military, Healthcare worker, Gov. employee or Teacher? See if you qualify for a Community Discount.
Monthly
$19.99
$19.99/mo
Billed monthly, Cancel any time.
3 Month Pass
$44.99
$14.99/mo
One time purchase of $44.99, Does not auto-renew.
MOST POPULAR
Annual Pass
$119.99
$9.99/mo
One time purchase of $119.99, Does not auto-renew.
BEST DEAL
Lifetime Pass
$189.99
One time purchase, Good for life.
What You Get
All IT & Cybersecurity Package plans include the following perks and exams .