Airflow

https://airflow.apache.org

Usage

Creating DAGs

In your airflow directory (~/airflow) create the directory "dags" and place your DAGs in there and the airflow scheduler will pick them up.

Creating Custom Operators

Create the directory "plugins" inside your airflow directory and add your operators to that.

• https://github.com/apache/airflow/tree/main/airflow/providers

API

The airflow webserver has an api for interacting with it from other clients.

• https://airflow.readthedocs.io/en/1.10.7/index.html