Awesome Open Source
Awesome Open Source
Combined Topics
airflow
x
Advertising
📦 10
All Projects
Application Programming Interfaces
📦 124
Applications
📦 192
Artificial Intelligence
📦 78
Blockchain
📦 73
Build Tools
📦 113
Cloud Computing
📦 80
Code Quality
📦 28
Collaboration
📦 32
Command Line Interface
📦 49
Community
📦 83
Companies
📦 60
Compilers
📦 63
Computer Science
📦 80
Configuration Management
📦 42
Content Management
📦 175
Control Flow
📦 213
Data Formats
📦 78
Data Processing
📦 276
Data Storage
📦 135
Economics
📦 64
Frameworks
📦 215
Games
📦 129
Graphics
📦 110
Hardware
📦 152
Integrated Development Environments
📦 49
Learning Resources
📦 166
Legal
📦 29
Libraries
📦 129
Lists Of Projects
📦 22
Machine Learning
📦 347
Mapping
📦 64
Marketing
📦 15
Mathematics
📦 55
Media
📦 239
Messaging
📦 98
Networking
📦 315
Operating Systems
📦 89
Operations
📦 121
Package Managers
📦 55
Programming Languages
📦 245
Runtime Environments
📦 100
Science
📦 42
Security
📦 396
Social Media
📦 27
Software Architecture
📦 72
Software Development
📦 72
Software Performance
📦 58
Software Quality
📦 133
Text Editors
📦 49
Text Processing
📦 136
User Interface
📦 330
User Interface Components
📦 514
Version Control
📦 30
Virtualization
📦 71
Web Browsers
📦 42
Web Servers
📦 26
Web User Interface
📦 210
The Top 49 Airflow Open Source Projects
Categories
>
Control Flow
>
Airflow
Airflow
⭐
20,665
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
Argo Workflows
⭐
7,811
Workflow engine for Kubernetes
Incubator Dolphinscheduler
⭐
5,172
Apache DolphinScheduler is a distributed and extensible workflow scheduler platform with powerful DAG visual interfaces, dedicated to solving complex job dependencies in the data pipeline and providing various types of jobs available `out of the box`.
Pipeline
⭐
4,108
PipelineAI Kubeflow Distribution
Docker Airflow
⭐
3,088
Docker Apache Airflow
Awesome Apache Airflow
⭐
2,275
Curated list of resources about Apache Airflow
Dataspherestudio
⭐
1,166
DataSphereStudio is a one stop data application development& management portal, covering scenarios including data exchange, desensitization/cleansing, analysis/mining, quality measurement, visualization, and task scheduling.
Airflow Maintenance Dags
⭐
903
A series of DAGs/Workflows to help maintain the operation of Airflow
Elyra
⭐
819
Elyra extends JupyterLab Notebooks with an AI centric approach.
Goodreads_etl_pipeline
⭐
785
An end-to-end GoodReads Data Pipeline for Building Data Lake, Data Warehouse and Analytics Platform.
Udacity Data Engineering Projects
⭐
445
Few projects related to Data Engineering including Data Modeling, Infrastructure setup on cloud, Data Warehousing and Data Lake development.
Agile_data_code_2
⭐
413
Code for Agile Data Science 2.0, O'Reilly 2017, Second Edition
Dag Factory
⭐
376
Dynamically generate Apache Airflow DAGs from YAML configuration files
Aws Airflow Stack
⭐
352
Turbine: the bare metals that gets you Airflow
Airflow Tutorial
⭐
304
Airflow basics tutorial
Airflow Rest Api Plugin
⭐
281
A plugin for Apache Airflow that exposes rest end points for the Command Line Interfaces
Airflow Operator
⭐
275
Kubernetes custom controller and CRDs to managing Airflow
Around Dataengineering
⭐
252
A Data Engineering & Machine Learning Knowledge Hub
Example Airflow Dags
⭐
241
Example DAGs using hooks and operators from Airflow Plugins
Paperboy
⭐
218
A web frontend for scheduling Jupyter notebook reports
Airflow Scheduler Failover Controller
⭐
204
A process that runs in unison with Apache Airflow to control the Scheduler process to ensure High Availability
Airflow Testing
⭐
170
Airflow Unit Tests and Integration Tests
Airflow Doc Zh
⭐
169
📖 [译] Airflow 中文文档
Airflow Exporter
⭐
160
Airflow plugin to export dag and task based metrics to Prometheus.
Data Science Stack Cookiecutter
⭐
153
🐳📊🤓Cookiecutter template to launch an awesome dockerized Data Science toolstack (incl. Jupyster, Superset, Postgres, Minio, AirFlow & API Star)
Beyond Jupyter
⭐
135
🐍💻📊 All material from the PyCon.DE 2018 Talk "Beyond Jupyter Notebooks - Building your own data science platform with Python & Docker" (incl. Slides, Video, Udemy MOOC & other References)
Airflow Chart
⭐
131
A Helm chart to install Apache Airflow on Kubernetes
Airflow Autoscaling Ecs
⭐
130
Airflow Deployment on AWS ECS Fargate Using Cloudformation
Airflow Pipeline
⭐
127
An Airflow docker image preconfigured to work well with Spark and Hadoop/EMR
Telemetry Airflow
⭐
125
Airflow configuration for Telemetry
Afctl
⭐
115
afctl helps to manage and deploy Apache Airflow projects faster and smoother.
Aws Ecs Airflow
⭐
107
Run Airflow in AWS ECS(Elastic Container Service) using Fargate tasks
Whirl
⭐
107
Fast iterative local development and testing of Apache Airflow workflows
Airflow_in_docker_compose
⭐
107
Apache Airflow in Docker Compose (for both versions 1.10.* and 2.*)
Bitnami Docker Airflow
⭐
88
Bitnami Docker Image for Apache Airflow
Udacity Data Engineering
⭐
88
Udacity Data Engineering Nano Degree (DEND)
Airflow Training
⭐
82
Airflow training for the crunch conf
Dataengineeringproject
⭐
78
Example end to end data engineering project.
Terraform Aws Airflow
⭐
68
Terraform module to deploy an Apache Airflow cluster on AWS, backed by RDS PostgreSQL for metadata, S3 for logs and SQS as message broker with CeleryExecutor
Discreetly
⭐
59
ETLy is an add-on dashboard service on top of Apache Airflow.
Airflow Cookbook
⭐
58
Airflow workflow management platform chef cookbook.
Xene
⭐
56
A distributed workflow runner focusing on performance and simplicity.
Airflow Toolkit
⭐
50
Any Airflow project day 1, you can spin up a local desktop Kubernetes Airflow environment AND one in Google Cloud Composer with tested data pipelines(DAGs) 🖥 >> [ 🚀, 🚢 ]
Airflow On Kubernetes
⭐
38
Bare minimal Airflow on Kubernetes (Local, EKS, AKS)
Docker Airflow
⭐
29
Repo for building docker based airflow image. Containers support multiple features like writing logs to local or S3 folder and Initializing GCP while container booting. https://abhioncbr.github.io/docker-airflow/
Objinsync
⭐
29
Continuously synchronize directories from remote object store to local filesystem
Databook
⭐
26
A facebook for data
Automating Your Data Pipeline With Apache Airflow
⭐
19
Automating Your Data Pipeline with Apache Airflow
Phila Airflow
⭐
16
1-49 of 49 projects
Advertising
📦 10
All Projects
Application Programming Interfaces
📦 124
Applications
📦 192
Artificial Intelligence
📦 78
Blockchain
📦 73
Build Tools
📦 113
Cloud Computing
📦 80
Code Quality
📦 28
Collaboration
📦 32
Command Line Interface
📦 49
Community
📦 83
Companies
📦 60
Compilers
📦 63
Computer Science
📦 80
Configuration Management
📦 42
Content Management
📦 175
Control Flow
📦 213
Data Formats
📦 78
Data Processing
📦 276
Data Storage
📦 135
Economics
📦 64
Frameworks
📦 215
Games
📦 129
Graphics
📦 110
Hardware
📦 152
Integrated Development Environments
📦 49
Learning Resources
📦 166
Legal
📦 29
Libraries
📦 129
Lists Of Projects
📦 22
Machine Learning
📦 347
Mapping
📦 64
Marketing
📦 15
Mathematics
📦 55
Media
📦 239
Messaging
📦 98
Networking
📦 315
Operating Systems
📦 89
Operations
📦 121
Package Managers
📦 55
Programming Languages
📦 245
Runtime Environments
📦 100
Science
📦 42
Security
📦 396
Social Media
📦 27
Software Architecture
📦 72
Software Development
📦 72
Software Performance
📦 58
Software Quality
📦 133
Text Editors
📦 49
Text Processing
📦 136
User Interface
📦 330
User Interface Components
📦 514
Version Control
📦 30
Virtualization
📦 71
Web Browsers
📦 42
Web Servers
📦 26
Web User Interface
📦 210