Welcome to Introduction to Airflow, Module 2 of Orchestrating Data Pipelines with Airflow. Module 1 gave you the DAG/scheduler/executor/webserver mental model entirely on paper — a real topological_order function, but no Docker, no install, nothing running. That was deliberate: you needed the ideas fixed in your head before looking at the real thing. This module is the real thing.

You’ll start by grounding that mental model in an actual running stack — three real Docker containers, each doing one job, talking to each other the way Lesson 4 of Module 1 described in the abstract. Then you’ll install that stack yourself, from the official Docker Compose file down to confirming a real health check and logging into a real browser UI. From there you write your first real DAG — a small “hello CityFlow” pipeline using the TaskFlow API — and watch Airflow’s scheduler pick it up, run it, and record its success, entirely on its own. A guided tour of the UI shows you where every piece of that lives: the DAGs list, a run’s Grid view, a task’s actual log output. The module closes with a guided project that takes Module 1’s paper design — CityFlow’s six-task, branching zone-and-fare report — and runs it for real, verified two independent ways: a screenshot of the finished run and a CLI query of every task’s final state.

Every command in this module is real, executed against a live Airflow 2.10.4 install, and verified — the output you read is the output that command actually produced.

Start with Lesson 1, and see exactly which real container plays each part of the mental model you already have.