AWS Certified Data Engineer Associate DEA-C01 Practice Question

An IoT application ingests 50,000 1 KB JSON events per second from 100,000 sensors. Each event has a device_id, timestamp, and payload. Engineers must query the past 24 hours of data for a given device_id in single-digit milliseconds. Records older than 24 hours are exported nightly to Amazon S3 for analytics. Which storage solution best supports this hot dataset with minimal operational effort?

Use an Amazon RDS for MySQL database with Auto Scaling read replicas to handle the throughput.
Load the incoming data into an Amazon Redshift cluster and run SQL queries for each device lookup.
Write the events to Amazon S3 in hourly Parquet files and query them with Amazon Athena when a device lookup is needed.
Store the hot dataset in an Amazon DynamoDB table with device_id as the partition key and a sort key on timestamp.

Report Issue

Answer Description

The workload needs high-throughput writes and millisecond key-value reads on device_id. Amazon DynamoDB provides predictable single-digit-millisecond latency at any scale and can auto-scale to tens of thousands of writes per second. A partition key on device_id and sort key on timestamp enables efficient time-range queries, and DynamoDB TTL can mark items older than 24 hours for expiry, simplifying nightly offloading to S3. RDS MySQL can add read replicas but still has a single writer, complicating scaling. Amazon Redshift is optimized for analytic, not transactional, workloads, with query times typically in seconds. Storing hourly Parquet files in S3 and querying with Athena would require full-object scans, resulting in higher latency and cost. Therefore, DynamoDB aligns best with the required pattern while minimizing operations.

Ask Bash

Bash is our AI bot, trained to help you pass your exam. AI Generated Content may display inaccurate information, always double-check anything important.