AWS Certified Data Engineer Associate DEA-C01 Practice Question

A data engineering team must ingest thousands of click-stream events per second from a public website, enrich each record, and store the results in Amazon S3. The solution must provide sub-second end-to-end latency, allow any record to be replayed when a downstream transformation fails, and minimize custom state management. Which approach best meets these requirements according to AWS best practices?

Configure an Amazon Kinesis Data Firehose delivery stream with data transformation enabled and an Amazon S3 destination.
Create an Amazon Kinesis Data Stream and use an AWS Glue streaming ETL job to read from the stream, enrich the records, and write the output to Amazon S3.
Send events to Amazon EventBridge, invoke an AWS Lambda function for enrichment, and write the results to Amazon S3.
Stream events directly to AWS Lambda using an Amazon Kinesis Data Stream event source mapping; the function enriches data and stores it in Amazon S3.

Report Issue

Answer Description

Using an AWS Glue streaming ETL job that reads from Amazon Kinesis Data Streams makes the consumer stateful without requiring engineers to manage that state themselves. Glue's Spark Structured Streaming engine automatically writes checkpoint data-including the end offset for each shard-to an Amazon S3 location that you specify. If a failure occurs, simply deleting or rolling back the checkpoint directory causes the job to start from an earlier position and replay any data that is still within the stream's retention period.

EventBridge with Lambda can replay events only after you first enable an archive and wait for events to arrive in that archive (AWS recommends a delay of about 10 minutes), which conflicts with the near-instant replay requirement. Kinesis Data Firehose offers no built-in way to resend records that have already been delivered, and a Lambda consumer that uses a Kinesis event-source mapping would require creating a new mapping or manipulating iterator positions to force reprocessing. Therefore, the Glue Streaming solution best satisfies throughput, latency, replayability, and operational-overhead constraints.

Ask Bash

Bash is our AI bot, trained to help you pass your exam. AI Generated Content may display inaccurate information, always double-check anything important.