- Managed Airflow

We now have a workflow tool set up for managing model workflows, but the default configuration does not provide high-availability, where new machines are provisioned when failures occur. While it’s possible to set up a distributed version of Airflow using Celery to set up different Scheduler and Worker nodes, one recent trend is using Kubernetes to create more robust Airflow deployments.