Monitoring and running pipelines
See whether your pipelines ran, read their logs, and trigger a run yourself.
Opening Pipelines opens the Airflow web interface. A pipeline (Airflow calls it a DAG) is a job that loads data into tables — see authoring pipelines. Visibility is fleet-wide: if you can open Pipelines, you see every pipeline in the deployment, not a subset. An empty list means none are deployed.
Prerequisite
can_view_pipelines opens Pipelines and reads runs, logs and code. can_run_pipelines adds
trigger, clear, mark and pause, and implies view. Missing from your sidebar means you hold neither.
Everything below is standard Airflow; the Airflow UI docs cover each screen.
What to look at
- Pipeline list — the landing screen: every pipeline with its recent runs, schedule, and paused state.
- Runs — select a pipeline for its run history; each run has a state (success, running, failed) and start time.
- Task states — within a run, each task’s state, in a grid and a graph showing dependencies and which task failed.
- Logs — open a task instance for the log it wrote. A failed run tells you what went wrong here.
Operating a run
With can_run_pipelines you operate every pipeline in the deployment:
- Trigger a run on demand (unpause the pipeline first if it is paused).
- Clear a task or run so it runs again.
- Mark a task or run as success or failed.
- Pause and unpause a pipeline’s schedule.
Deleting a pipeline (removes its history irrecoverably) and editing Airflow’s Connections,
Variables, Pools, Config and Backfills need can_admin. If a control is refused,
check what it needs against the permissions reference.