AWS Certified Data Engineer Associate DEA-C01 Practice Question

Adding an infrastructure-as-code step to the deployment pipeline ensures that every Glue job is accompanied by the required CloudWatch Logs configuration. A CloudFormation stack can create a log group whose name is passed in as a parameter, set RetentionInDays to 14, and define a metric filter that searches for the "ERROR" pattern. Because the stack is executed automatically by CodePipeline, the process is repeatable and works for any number of jobs without manual intervention.

Creating the log group in the job's Python code causes the resource creation logic to run every execution, introduces IAM overhead, and does not guarantee that the metric filter is in place before logs arrive. Manually creating log groups in the console does not scale to frequent job launches and is prone to human error. Enabling CloudTrail data events only records API activity and does not capture the driver or executor log lines needed by Operations.