Your team maintains several Apache Beam pipelines packaged as Dataflow Flex Templates together with Cloud Composer DAGs that invoke those templates. You must introduce CI/CD so that any change to pipeline code or DAG definitions is version-controlled, traced, tested, and then automatically deployed to development, staging, and production projects. Operations effort must stay minimal and only managed Google Cloud services may be used. Which approach should you recommend?
Keep the Flex Template specification files inside each environment's Composer bucket with object versioning enabled and rely on Airflow's GitSync sidecar to propagate changes.
Commit pipeline code to a private GitHub repository and manually run Cloud Composer's Import DAG feature after every merge, triggering Dataflow jobs through a Cloud Function webhook.
Publish compiled pipeline JARs and DAG Python files directly to Artifact Registry and schedule a Cloud Scheduler job that invokes gcloud builds submit weekly to redeploy them.
Store Beam source and DAG files in Cloud Source Repositories and create Cloud Build triggers that run tests, build the Flex Template container, push it to Artifact Registry, and copy updated DAGs to each Cloud Composer environment's bucket.
Storing the Beam source code and Composer DAG files in Cloud Source Repositories gives you a fully managed Git repository inside Google Cloud. Cloud Build can watch that repository through build triggers, run unit tests, build the Flex Template container image, push it to Artifact Registry, and copy updated DAG files to each environment's Composer bucket (for example, with a gsutil or REST API step). This provides auditable version control and automated, multi-environment deployment while relying solely on managed GCP services. The alternative proposals lack either proper version control (object versioning or Artifact Registry alone), do not deliver automated multi-environment promotion, or require manual actions that violate the low-overhead requirement.
Ask Bash
Bash is our AI bot, trained to help you pass your exam. AI Generated Content may display inaccurate information, always double-check anything important.
What are Dataflow Flex Templates and why are they used?
Open an interactive chat with Bash
How does Cloud Build interact with Cloud Source Repositories in CI/CD pipelines?
Open an interactive chat with Bash
What is the role of Artifact Registry and how does it support pipeline deployment?
Open an interactive chat with Bash
What is a Flex Template in Dataflow?
Open an interactive chat with Bash
What is Cloud Composer and its role in this approach?
Open an interactive chat with Bash
How does Cloud Build integrate with CI/CD pipelines?
Open an interactive chat with Bash
GCP Professional Data Engineer
Ingesting and processing the data
Your Score:
Report Issue
Bash, the Crucial Exams Chat Bot
AI Bot
Loading...
Loading...
Loading...
Pass with Confidence.
IT & Cybersecurity Package
You have hit the limits of our free tier, become a Premium Member today for unlimited access.
Military, Healthcare worker, Gov. employee or Teacher? See if you qualify for a Community Discount.
Monthly
$19.99
$19.99/mo
Billed monthly, Cancel any time.
3 Month Pass
$44.99
$14.99/mo
One time purchase of $44.99, Does not auto-renew.
MOST POPULAR
Annual Pass
$119.99
$9.99/mo
One time purchase of $119.99, Does not auto-renew.
BEST DEAL
Lifetime Pass
$189.99
One time purchase, Good for life.
What You Get
All IT & Cybersecurity Package plans include the following perks and exams .