Bash, the Crucial Exams Chat Bot
AI Bot
Data Pipelines and ETL Processes Flashcards
AWS Certified Data Engineer Associate DEA-C01 Flashcards
Study our Data Pipelines and ETL Processes flashcards for the AWS Certified Data Engineer Associate DEA-C01 exam with 15+ flashcards. View as flashcards, a searchable table, or as a fun matching game.
| Front | Back |
| Advantages of AWS Step Functions for ETL | Enables orchestration and monitoring of workflows using state machines |
| Difference between Glue Jobs and Glue Workflows | Jobs handle specific ETL tasks while workflows orchestrate multiple jobs and crawlers |
| Difference between on-demand and scheduled triggers in Glue | On-demand triggers run manually, while scheduled triggers operate at set intervals |
| How Data Pipeline handles retry logic | Automatically retries failed activities based on defined conditions |
| How Glue integrates with S3 | Allows reading datasets stored in S3 for transformation and loading |
| How Step Functions differ from Glue Workflow | Step Functions offer more flexibility for orchestrating complex workflows across AWS services |
| Key benefits of using AWS Glue | Dynamically scales, reduces coding effort, integrated tools for seamless data preparation |
| Optimal use case for AWS Glue vs Data Pipeline | Glue for complex ETL; Data Pipeline for simpler scheduled data copy/movement |
| Primary benefit of using Glue with Redshift | Simplifies loading and querying large-scale datasets into Redshift |
| Purpose of AWS Glue Data Catalog | A centralized metadata repository for data assets that integrates with other AWS services |
| What is an AWS Glue Crawler | A tool to automatically infer schema and metadata of data stored in various sources |
| What is AWS Data Pipeline | A service to schedule and automate data movement and transformation |
| What is AWS Glue | A managed ETL service used to prepare and transform data for analytics |
| What is ETL | Extract Transform Load - a process to extract data, transform it into a usable format, and load it into a target system |
| What is partitioning in AWS Glue | Dividing data into subsets based on a key to optimize querying and storage |
About the Flashcards
Flashcards for the AWS Certified Data Engineer Associate exam focus on core ETL terminology and the AWS services and patterns used to build scalable data pipelines. Cards define ETL, explain AWS Glue roles like jobs, crawlers, and the Data Catalog, and highlight Glue advantages for data preparation. They also cover partitioning, metadata management, and common integration patterns.
Students can review orchestration and scheduling concepts (workflows, on-demand vs scheduled triggers, Step Functions), integration points with S3 and Redshift, partitioning strategies, and when to choose Glue versus AWS Data Pipeline for different use cases, with quick recall practice for common exam scenarios.
Topics covered in this flashcard deck:
- ETL fundamentals
- AWS Glue components
- Glue orchestration and Step Functions
- Partitioning in Glue
- AWS Data Pipeline
- S3 and Redshift integration