AWS Certified Data Engineer Associate DEA-C01 Practice Question
A data engineer is building an AWS Step Functions Standard workflow that will invoke an AWS Glue job for each of 200 daily S3 partitions. No more than 10 Glue jobs should run at the same time, each invocation must automatically retry twice with exponential backoff for transient errors, and the workflow must fail immediately on a custom "DATA_VALIDATION_FAILED" error returned by the job. Which Step Functions design will meet these requirements with the least custom code?
Run an Express Step Functions workflow triggered by Amazon EventBridge rules that submit Glue jobs in batches of 10 until all partitions are processed.
Use a Parallel state with 10 static branches; each branch invokes the Glue job for a subset of partitions.
Create a Map state that passes the array of partition prefixes, set MaxConcurrency to 10, and configure Retry with backoffRate and a Catch clause for the DATA_VALIDATION_FAILED error.
Invoke the Glue job from a Lambda function in a Task state and write custom code in the function to iterate through partitions, manage retries, and enforce a 10-job concurrency limit.
A Map state natively iterates over a JSON array and can control parallelism with the MaxConcurrency field, ensuring that no more than the specified number of iterations (10) run simultaneously. Inside the Map state's Item processor, you can add a Task state that starts the Glue job, apply a Retry clause that specifies a maximum of two attempts with an exponential backoffRate, and add a Catch clause that matches the custom "DATA_VALIDATION_FAILED" error to fail the workflow immediately. This solution uses only Step Functions features and requires no additional Lambda code or complex branching logic.
Parallel states launch a fixed number of branches and therefore cannot dynamically scale to 200 partitions while still limiting concurrency to 10 without extra logic. Iterating in a Lambda function shifts the retry and concurrency control to application code, adding operational overhead. Express workflows cannot directly throttle concurrent Glue invocations and would still require external coordination.
Ask Bash
Bash is our AI bot, trained to help you pass your exam. AI Generated Content may display inaccurate information, always double-check anything important.
What is a Map state in AWS Step Functions?
Open an interactive chat with Bash
How does MaxConcurrency work in Step Functions?
Open an interactive chat with Bash
What is exponential backoff in Step Functions retry policy?
Open an interactive chat with Bash
AWS Certified Data Engineer Associate DEA-C01
Data Operations and Support
Your Score:
Report Issue
Bash, the Crucial Exams Chat Bot
AI Bot
Loading...
Loading...
Loading...
Pass with Confidence.
IT & Cybersecurity Package
You have hit the limits of our free tier, become a Premium Member today for unlimited access.
Military, Healthcare worker, Gov. employee or Teacher? See if you qualify for a Community Discount.
Monthly
$19.99
$19.99/mo
Billed monthly, Cancel any time.
3 Month Pass
$44.99
$14.99/mo
One time purchase of $44.99, Does not auto-renew.
MOST POPULAR
Annual Pass
$119.99
$9.99/mo
One time purchase of $119.99, Does not auto-renew.
BEST DEAL
Lifetime Pass
$189.99
One time purchase, Good for life.
What You Get
All IT & Cybersecurity Package plans include the following perks and exams .