Awesome Open Source
Awesome Open Source
Combined Topics
airflow x
Advertising 📦 10
All Projects
Application Programming Interfaces 📦 124
Applications 📦 192
Artificial Intelligence 📦 78
Blockchain 📦 73
Build Tools 📦 113
Cloud Computing 📦 80
Code Quality 📦 28
Collaboration 📦 32
Command Line Interface 📦 49
Community 📦 83
Companies 📦 60
Compilers 📦 63
Computer Science 📦 80
Configuration Management 📦 42
Content Management 📦 175
Control Flow 📦 213
Data Formats 📦 78
Data Processing 📦 276
Data Storage 📦 135
Economics 📦 64
Frameworks 📦 215
Games 📦 129
Graphics 📦 110
Hardware 📦 152
Integrated Development Environments 📦 49
Learning Resources 📦 166
Legal 📦 29
Libraries 📦 129
Lists Of Projects 📦 22
Machine Learning 📦 347
Mapping 📦 64
Marketing 📦 15
Mathematics 📦 55
Media 📦 239
Messaging 📦 98
Networking 📦 315
Operating Systems 📦 89
Operations 📦 121
Package Managers 📦 55
Programming Languages 📦 245
Runtime Environments 📦 100
Science 📦 42
Security 📦 396
Social Media 📦 27
Software Architecture 📦 72
Software Development 📦 72
Software Performance 📦 58
Software Quality 📦 133
Text Editors 📦 49
Text Processing 📦 136
User Interface 📦 330
User Interface Components 📦 514
Version Control 📦 30
Virtualization 📦 71
Web Browsers 📦 42
Web Servers 📦 26
Web User Interface 📦 210

The Top 49 Airflow Open Source Projects

Categories > Control Flow > Airflow
Airflow ⭐ 20,665
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
Argo Workflows ⭐ 7,811
Workflow engine for Kubernetes
Incubator Dolphinscheduler ⭐ 5,172
Apache DolphinScheduler is a distributed and extensible workflow scheduler platform with powerful DAG visual interfaces, dedicated to solving complex job dependencies in the data pipeline and providing various types of jobs available `out of the box`.
Pipeline ⭐ 4,108
PipelineAI Kubeflow Distribution
Docker Airflow ⭐ 3,088
Docker Apache Airflow
Awesome Apache Airflow ⭐ 2,275
Curated list of resources about Apache Airflow
Dataspherestudio ⭐ 1,166
DataSphereStudio is a one stop data application development& management portal, covering scenarios including data exchange, desensitization/cleansing, analysis/mining, quality measurement, visualization, and task scheduling.
Airflow Maintenance Dags ⭐ 903
A series of DAGs/Workflows to help maintain the operation of Airflow
Elyra ⭐ 819
Elyra extends JupyterLab Notebooks with an AI centric approach.
Goodreads_etl_pipeline ⭐ 785
An end-to-end GoodReads Data Pipeline for Building Data Lake, Data Warehouse and Analytics Platform.
Udacity Data Engineering Projects ⭐ 445
Few projects related to Data Engineering including Data Modeling, Infrastructure setup on cloud, Data Warehousing and Data Lake development.
Agile_data_code_2 ⭐ 413
Code for Agile Data Science 2.0, O'Reilly 2017, Second Edition
Dag Factory ⭐ 376
Dynamically generate Apache Airflow DAGs from YAML configuration files
Aws Airflow Stack ⭐ 352
Turbine: the bare metals that gets you Airflow
Airflow Tutorial ⭐ 304
Airflow basics tutorial
Airflow Rest Api Plugin ⭐ 281
A plugin for Apache Airflow that exposes rest end points for the Command Line Interfaces
Airflow Operator ⭐ 275
Kubernetes custom controller and CRDs to managing Airflow
Around Dataengineering ⭐ 252
A Data Engineering & Machine Learning Knowledge Hub
Example Airflow Dags ⭐ 241
Example DAGs using hooks and operators from Airflow Plugins
Paperboy ⭐ 218
A web frontend for scheduling Jupyter notebook reports
Airflow Scheduler Failover Controller ⭐ 204
A process that runs in unison with Apache Airflow to control the Scheduler process to ensure High Availability
Airflow Testing ⭐ 170
Airflow Unit Tests and Integration Tests
Airflow Doc Zh ⭐ 169
📖 [译] Airflow 中文文档
Airflow Exporter ⭐ 160
Airflow plugin to export dag and task based metrics to Prometheus.
Data Science Stack Cookiecutter ⭐ 153
🐳📊🤓Cookiecutter template to launch an awesome dockerized Data Science toolstack (incl. Jupyster, Superset, Postgres, Minio, AirFlow & API Star)
Beyond Jupyter ⭐ 135
🐍💻📊 All material from the PyCon.DE 2018 Talk "Beyond Jupyter Notebooks - Building your own data science platform with Python & Docker" (incl. Slides, Video, Udemy MOOC & other References)
Airflow Chart ⭐ 131
A Helm chart to install Apache Airflow on Kubernetes
Airflow Autoscaling Ecs ⭐ 130
Airflow Deployment on AWS ECS Fargate Using Cloudformation
Airflow Pipeline ⭐ 127
An Airflow docker image preconfigured to work well with Spark and Hadoop/EMR
Telemetry Airflow ⭐ 125
Airflow configuration for Telemetry
Afctl ⭐ 115
afctl helps to manage and deploy Apache Airflow projects faster and smoother.
Aws Ecs Airflow ⭐ 107
Run Airflow in AWS ECS(Elastic Container Service) using Fargate tasks
Whirl ⭐ 107
Fast iterative local development and testing of Apache Airflow workflows
Airflow_in_docker_compose ⭐ 107
Apache Airflow in Docker Compose (for both versions 1.10.* and 2.*)
Bitnami Docker Airflow ⭐ 88
Bitnami Docker Image for Apache Airflow
Udacity Data Engineering ⭐ 88
Udacity Data Engineering Nano Degree (DEND)
Airflow Training ⭐ 82
Airflow training for the crunch conf
Dataengineeringproject ⭐ 78
Example end to end data engineering project.
Terraform Aws Airflow ⭐ 68
Terraform module to deploy an Apache Airflow cluster on AWS, backed by RDS PostgreSQL for metadata, S3 for logs and SQS as message broker with CeleryExecutor
Discreetly ⭐ 59
ETLy is an add-on dashboard service on top of Apache Airflow.
Airflow Cookbook ⭐ 58
Airflow workflow management platform chef cookbook.
Xene ⭐ 56
A distributed workflow runner focusing on performance and simplicity.
Airflow Toolkit ⭐ 50
Any Airflow project day 1, you can spin up a local desktop Kubernetes Airflow environment AND one in Google Cloud Composer with tested data pipelines(DAGs) 🖥 >> [ 🚀, 🚢 ]
Airflow On Kubernetes ⭐ 38
Bare minimal Airflow on Kubernetes (Local, EKS, AKS)
Docker Airflow ⭐ 29
Repo for building docker based airflow image. Containers support multiple features like writing logs to local or S3 folder and Initializing GCP while container booting. https://abhioncbr.github.io/docker-airflow/
Objinsync ⭐ 29
Continuously synchronize directories from remote object store to local filesystem
Databook ⭐ 26
A facebook for data
Automating Your Data Pipeline With Apache Airflow ⭐ 19
Automating Your Data Pipeline with Apache Airflow
Phila Airflow ⭐ 16
1-49 of 49 projects
Advertising 📦 10
All Projects
Application Programming Interfaces 📦 124
Applications 📦 192
Artificial Intelligence 📦 78
Blockchain 📦 73
Build Tools 📦 113
Cloud Computing 📦 80
Code Quality 📦 28
Collaboration 📦 32
Command Line Interface 📦 49
Community 📦 83
Companies 📦 60
Compilers 📦 63
Computer Science 📦 80
Configuration Management 📦 42
Content Management 📦 175
Control Flow 📦 213
Data Formats 📦 78
Data Processing 📦 276
Data Storage 📦 135
Economics 📦 64
Frameworks 📦 215
Games 📦 129
Graphics 📦 110
Hardware 📦 152
Integrated Development Environments 📦 49
Learning Resources 📦 166
Legal 📦 29
Libraries 📦 129
Lists Of Projects 📦 22
Machine Learning 📦 347
Mapping 📦 64
Marketing 📦 15
Mathematics 📦 55
Media 📦 239
Messaging 📦 98
Networking 📦 315
Operating Systems 📦 89
Operations 📦 121
Package Managers 📦 55
Programming Languages 📦 245
Runtime Environments 📦 100
Science 📦 42
Security 📦 396
Social Media 📦 27
Software Architecture 📦 72
Software Development 📦 72
Software Performance 📦 58
Software Quality 📦 133
Text Editors 📦 49
Text Processing 📦 136
User Interface 📦 330
User Interface Components 📦 514
Version Control 📦 30
Virtualization 📦 71
Web Browsers 📦 42
Web Servers 📦 26
Web User Interface 📦 210