AWS Certified Data Engineer Associate DEA-C01 Practice Question
An AWS Glue crawler registers daily Parquet files stored under the Amazon S3 prefix s3://datalake/iot/year=YYYY/month=MM/day=DD/. Business analysts query the table from Amazon Athena, but the current day's data is not visible until the crawler's nightly run. As a data engineer, how can you expose new partitions to Athena within minutes of arrival while keeping operational effort low?
Trigger an AWS Step Functions workflow from CloudWatch Events that calls ALTER TABLE ADD PARTITION for each new file detected.
Change the crawler to run every five minutes on a fixed schedule.
Replace the crawler with Athena partition projection and define formulas that generate the year, month, and day partitions.
Enable Amazon S3 event notifications to invoke the crawler in incremental mode whenever new objects are created.
An Amazon S3 event-driven, incremental crawler updates only the partitions that correspond to the newly arrived objects. When an object is written to a monitored prefix, S3 publishes an event that triggers the crawler through EventBridge. The crawler quickly adds the new year/month/day partition to the AWS Glue Data Catalog, making the data immediately queryable from Athena. A frequent schedule works but still polls on a fixed interval. Partition projection removes the dependency on the Data Catalog but requires additional table-property configuration and is not necessary when a catalog is already in use. A custom Lambda or Step Functions workflow achieves the same goal but adds more code and maintenance overhead compared with the built-in event-driven crawler capability.
Ask Bash
Bash is our AI bot, trained to help you pass your exam. AI Generated Content may display inaccurate information, always double-check anything important.
What is Amazon S3 event notification?
Open an interactive chat with Bash
What is an AWS Glue incremental crawler?
Open an interactive chat with Bash
How does partition projection work in Athena?
Open an interactive chat with Bash
What is an Amazon S3 event notification?
Open an interactive chat with Bash
What is an incremental crawler in AWS Glue?
Open an interactive chat with Bash
What is partition projection in Athena, and why wasn’t it chosen here?
Open an interactive chat with Bash
AWS Certified Data Engineer Associate DEA-C01
Data Store Management
Your Score:
Report Issue
Bash, the Crucial Exams Chat Bot
AI Bot
Loading...
Loading...
Loading...
Pass with Confidence.
IT & Cybersecurity Package
You have hit the limits of our free tier, become a Premium Member today for unlimited access.
Military, Healthcare worker, Gov. employee or Teacher? See if you qualify for a Community Discount.
Monthly
$19.99
$19.99/mo
Billed monthly, Cancel any time.
3 Month Pass
$44.99
$14.99/mo
One time purchase of $44.99, Does not auto-renew.
MOST POPULAR
Annual Pass
$119.99
$9.99/mo
One time purchase of $119.99, Does not auto-renew.
BEST DEAL
Lifetime Pass
$189.99
One time purchase, Good for life.
What You Get
All IT & Cybersecurity Package plans include the following perks and exams .