AWS Certified Data Engineer Associate DEA-C01 Practice Question

A data engineering team uses AWS Step Functions to launch a transient Amazon EMR 6.x cluster nightly to run a PySpark ETL step, after which the cluster terminates automatically. When a step fails, the cluster shuts down before engineers can view Spark driver and executor logs. The team must retain detailed logs and the Spark history UI for post-mortem analysis while adding minimal EC2 cost. Which action meets these requirements?

Enable CloudTrail data events on the input data bucket to capture Spark driver logs for later review.
Specify an Amazon S3 log URI and enable persistent application user interfaces for Spark when creating the EMR cluster.
Configure EMRFS Consistent View so logs are automatically synchronized to Amazon S3 after each task.
Enable termination protection and disable auto-termination so the cluster remains available for manual log retrieval via SSH.

Report Issue

Answer Description

Specifying an Amazon S3 log URI causes Amazon EMR to copy YARN, Spark, and system logs to the bucket as each step completes. Enabling the persistent application user interfaces feature stores Spark History Server data off-cluster in the same S3 location, so the UI can be opened from the Amazon EMR console even after the cluster terminates. This preserves all troubleshooting information without incurring additional EC2 costs. Leaving the cluster up with termination protection meets the troubleshooting goal but accrues ongoing instance charges. EMRFS Consistent View only improves S3 list consistency and does not archive logs. Enabling CloudTrail data events on the input bucket records object-level API activity, not Spark application logs, so it does not provide the required diagnostics.

Ask Bash

Bash is our AI bot, trained to help you pass your exam. AI Generated Content may display inaccurate information, always double-check anything important.