- Managed Airflow
Explore how to implement managed Airflow with Google Cloud Composer to automate and manage scalable model workflows. Understand the steps for deployment, authentication, and adding DAGs to ensure high-availability in production environments.
We'll cover the following...
We'll cover the following...
We now have a workflow tool set up for managing model workflows, but the default configuration does not provide
high-availability, where new machines are provisioned when failures occur. While it’s possible to set up a distributed version of Airflow using Celery to set up different Scheduler and Worker nodes, one recent trend is using Kubernetes to create more robust Airflow deployments.
