AWS Certified Data Engineer Associate DEA-C01 Practice Question
An EMR cluster writes detailed Spark application logs to an S3 bucket. Engineers must perform ad-hoc, interactive troubleshooting by filtering on executor IDs and calculating average task duration. They want a solution that requires no clusters to manage and keeps costs low, paying only for queries run. Which approach meets these requirements?
Spin up a temporary EMR cluster with Hive, copy the logs to HDFS, and issue Hive queries.
Stream the logs to CloudWatch Logs and analyze them with CloudWatch Logs Insights.
Ingest the logs with Logstash into an Amazon OpenSearch Service domain and query them in Kibana.
Create an Amazon Athena external table that points to the S3 log prefix and run SQL queries against it.
Amazon Athena lets you define an external table on the log files that EMR archived to S3 and submit SQL queries immediately. Because Athena is a serverless service, the team does not provision or operate any compute resources. Charges are incurred only for the amount of data scanned by each query, satisfying the cost and operations constraints. Launching a transient EMR cluster or an OpenSearch domain would add infrastructure to manage. Moving the logs to CloudWatch Logs first would introduce extra ingestion steps and ongoing storage costs while still not eliminating managed resources.
Ask Bash
Bash is our AI bot, trained to help you pass your exam. AI Generated Content may display inaccurate information, always double-check anything important.
Why is Amazon Athena a good choice for analyzing EMR Spark logs on S3?
Open an interactive chat with Bash
What is an external table in Amazon Athena, and how does it work?
Open an interactive chat with Bash
How does Amazon Athena compare to CloudWatch Logs Insights for log analysis?
Open an interactive chat with Bash
AWS Certified Data Engineer Associate DEA-C01
Data Operations and Support
Your Score:
Report Issue
Bash, the Crucial Exams Chat Bot
AI Bot
Loading...
Loading...
Loading...
Pass with Confidence.
IT & Cybersecurity Package
You have hit the limits of our free tier, become a Premium Member today for unlimited access.
Military, Healthcare worker, Gov. employee or Teacher? See if you qualify for a Community Discount.
Monthly
$19.99
$19.99/mo
Billed monthly, Cancel any time.
3 Month Pass
$44.99
$14.99/mo
One time purchase of $44.99, Does not auto-renew.
MOST POPULAR
Annual Pass
$119.99
$9.99/mo
One time purchase of $119.99, Does not auto-renew.
BEST DEAL
Lifetime Pass
$189.99
One time purchase, Good for life.
What You Get
All IT & Cybersecurity Package plans include the following perks and exams .