AWS Certified Data Engineer Associate DEA-C01 Practice Question

A data engineering team receives hourly CSV files in an Amazon S3 bucket. Each time a file arrives they must 1) launch an AWS Glue ETL job, 2) run an Amazon Athena CTAS query to aggregate the transformed data, and 3) send an Amazon SNS notification. The solution must provide built-in retries, visual workflow monitoring, JSON-based infrastructure-as-code definitions, and minimal operational overhead. Which service should orchestrate this pipeline?

Deploy an Amazon Managed Workflows for Apache Airflow environment and implement a DAG that calls Glue and Athena operators, then publishes an SNS message.
Define an AWS Step Functions state machine triggered by an EventBridge rule that invokes the Glue job, runs the Athena query with the SDK integration, and publishes to SNS.
Create an Amazon EventBridge Scheduler cron expression that invokes three Lambda functions in sequence to run Glue, Athena, and SNS.
Use an AWS Glue Workflow to run the Glue job, followed by a crawler and a trigger that starts an Athena query via Lambda, then send an SNS notification.

Report Issue

Answer Description

AWS Step Functions is a serverless, fully managed workflow engine defined in JSON (the Amazon States Language). It offers visual execution graphs in the console, integrated CloudWatch metrics, and configurable retry and error-handling policies. Step Functions provides direct service integrations to invoke AWS Glue StartJobRun, run Amazon Athena StartQueryExecution, and publish Amazon SNS messages, so no custom code is necessary. EventBridge Scheduler supports only a single target per schedule, so it cannot chain the three required steps. Amazon MWAA could orchestrate the tasks but introduces additional effort to provision and maintain an Airflow environment. AWS Glue Workflows can coordinate Glue jobs and crawlers but cannot directly invoke Athena queries or publish SNS notifications without additional Lambda code. Therefore, Step Functions best meets all stated requirements.

Ask Bash

Bash is our AI bot, trained to help you pass your exam. AI Generated Content may display inaccurate information, always double-check anything important.