AWS Certified Data Engineer Associate DEA-C01 Practice Question

A data engineering team must run a weekly batch pipeline to extract data from Amazon RDS, transform it with an AWS Glue job, and load the results into Amazon Redshift. The team wants a fully managed orchestration solution that provides visual workflow monitoring, native retry logic, and minimal operational overhead. Which approach meets these requirements?

Implement the pipeline as a DAG in Amazon Managed Workflows for Apache Airflow, using Python and Redshift operators, and rely on Airflow's scheduler to run it weekly.
Build an AWS Glue Workflow that triggers the Glue job followed by an AWS Lambda function that executes the Redshift COPY command, and invoke the workflow weekly with EventBridge.
Configure two EventBridge rules: one to start the AWS Glue job and a second rule scheduled an hour later to run the Redshift COPY operation. Monitor both steps with CloudWatch alarms.
Create an Amazon EventBridge rule that triggers an AWS Step Functions Standard workflow. The workflow first uses the native StartJobRun integration to run the AWS Glue job, then calls the Amazon Redshift Data API to load the data.

Report Issue

Answer Description

AWS Step Functions Standard workflows offer direct service integrations for both AWS Glue StartJobRun and Amazon Redshift Data API, supply built-in retry and error-handling policies, and provide a visual execution history in the Step Functions console. A single Amazon EventBridge rule can invoke the state machine on the required weekly schedule. Using separate EventBridge rules cannot guarantee that the Redshift load waits for the Glue job to finish. Amazon MWAA can orchestrate the steps, but it requires provisioning and managing an Airflow environment, increasing operational effort. AWS Glue Workflows can chain Glue crawlers and jobs only; they cannot include a Lambda function or other non-Glue steps, so they cannot natively run the Redshift load.

Ask Bash

Bash is our AI bot, trained to help you pass your exam. AI Generated Content may display inaccurate information, always double-check anything important.