AWS Certified Data Engineer Associate DEA-C01 Practice Question

A company ingests 50 000 JSON events per second from IoT sensors into an Amazon Kinesis Data Stream. The analytics team needs each record converted to Apache Parquet with sub-second latency and written to Amazon S3. The solution must scale automatically with the unpredictable event rate and require minimal infrastructure management. Which approach meets these requirements most effectively?

Configure an Amazon EMR cluster with Spark Structured Streaming to poll the stream and convert data to Parquet in Amazon S3.
Use AWS Lambda with Kinesis Data Streams as the event source; each invocation converts the JSON record to Parquet and writes it to Amazon S3.
Deliver the stream to Amazon S3 through Kinesis Data Firehose with a Lambda transformation that converts incoming records to Parquet format.
Create an AWS Glue streaming ETL job that reads from the Kinesis Data Stream and writes Parquet files to Amazon S3.

Report Issue

Answer Description

An AWS Glue streaming ETL job is serverless, so there are no clusters to provision or manage. It can read directly from Kinesis Data Streams, perform Spark-based transformations, and write Parquet files to Amazon S3 with micro-batch windows as small as 1 s. Glue streaming jobs support Auto Scaling, so they handle spikes in event volume without manual intervention.

An Amazon EMR cluster running Spark Structured Streaming could work, but the company would still have to size, monitor, and scale the cluster, increasing operational overhead.

An AWS Lambda consumer risks hitting concurrency limits and would struggle to serialize each record to Parquet efficiently at 50 000 TPS.

Kinesis Data Firehose can deliver to S3 and convert formats, but its buffering interval (minimum 60 s) prevents sub-second latency. Therefore, the Glue streaming ETL job best satisfies latency, scalability, and management requirements.

Ask Bash

Bash is our AI bot, trained to help you pass your exam. AI Generated Content may display inaccurate information, always double-check anything important.