Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About
Search results for docker airflow
airflow
x
docker
x
102 search results found
Pipeline
โญย
4,158
PipelineAI
Orchest
โญย
3,876
Build data pipelines, the easy way ๐ ๏ธ
Docker Airflow
โญย
3,589
Docker Apache Airflow
Awesome Apache Airflow
โญย
3,468
Curated list of resources about Apache Airflow
Elyra
โญย
1,721
Elyra extends JupyterLab with an AI centric approach.
Airflow Tutorial
โญย
669
Apache Airflow tutorial
Energy Forecasting
โญย
663
๐ ๐ง๐ต๐ฒ ๐๐๐น๐น ๐ฆ๐๐ฎ๐ฐ๐ธ ๐ณ-๐ฆ๐๐ฒ๐ฝ๐ ๐ ๐๐ข๐ฝ๐ ๐๐ฟ๐ฎ๐บ๐ฒ๐๐ผ๐ฟ๐ธ | ๐๐ฒ๐ฎ๐ฟ๐ป ๐ ๐๐ & ๐ ๐๐ข๐ฝ๐ for free by designing, building and deploying an end-to-end ML batch system ~ ๐ด๐ฐ๐ถ๐ณ๐ค๐ฆ ๐ค๐ฐ๐ฅ๐ฆ + 2.5 ๐ฉ๐ฐ๐ถ๐ณ๐ด ๐ฐ๐ง ๐ณ๐ฆ๐ข๐ฅ๐ช๐ฏ๐จ & ๐ท๐ช๐ฅ๐ฆ๐ฐ ๐ฎ๐ข๐ต๐ฆ๐ณ๐ช๐ข๐ญ๐ด
Data Pipelines With Apache Airflow
โญย
587
Code for Data Pipelines with Apache Airflow
Kube Airflow
โญย
556
A docker image and kubernetes config files to run Airflow on Kubernetes
Astronomer
โญย
454
Helm Charts for the Astronomer Platform, Apache Airflow as a Service on Kubernetes
Gather Deployment
โญย
351
Gathers Python deployment, infrastructure and practices.
Microservices Based Algorithmic Trading System
โญย
302
MBATS is a docker based platform for developing, testing and deploying Algorthmic Trading strategies with a focus on Machine Learning based algorithms.
Beginner_de_project
โญย
276
Beginner data engineering project - batch edition
Airflow Chart
โญย
261
A Helm chart to install Apache Airflow on Kubernetes
Paperboy
โญย
231
A web frontend for scheduling Jupyter notebook reports
Kyle School
โญย
204
์์นด ๋ฐ์ดํฐ ๊ทธ๋ฃน ์ฌ๋ด ์ ์ /์ธํด์ ๋์์ผ๋ก ํ ์นด์ผ์ค์ฟจ
Whirl
โญย
183
Fast iterative local development and testing of Apache Airflow workflows
Airflow_in_docker_compose
โญย
182
Apache Airflow in Docker Compose (for both versions 1.10.* and 2.*)
Airflow Pipeline
โญย
168
An Airflow docker image preconfigured to work well with Spark and Hadoop/EMR
Incubator Liminal
โญย
131
Apache Liminals goal is to operationalise the machine learning process, allowing data scientists to quickly transition from a successful experiment to an automated pipeline of model training, validation, deployment and inference in production. Liminal provides a Domain Specific Language to build ML workflows on top of Apache Airflow.
Beyond Jupyter
โญย
125
๐๐ป๐ All material from the PyCon.DE 2018 Talk "Beyond Jupyter Notebooks - Building your own data science platform with Python & Docker" (incl. Slides, Video, Udemy MOOC & other References)
Airflow Docker
โญย
120
Source code of the Apache Airflow Tutorial for Beginners on YouTube Channel Coder2j (https://www.youtube.com/c/coder2j)
Data Science Stack Cookiecutter
โญย
120
๐ณ๐๐คCookiecutter template to launch an awesome dockerized Data Science toolstack (incl. Jupyster, Superset, Postgres, Minio, AirFlow & API Star)
Airflow On Kubernetes
โญย
119
A guide to running Airflow on Kubernetes
Movalytics Data Warehouse
โญย
117
Data pipeline performing ETL to AWS Redshift using Spark, orchestrated with Apache Airflow
Aws Ecs Airflow
โญย
110
Run Airflow in AWS ECS(Elastic Container Service) using Fargate tasks
Afctl
โญย
103
afctl helps to manage and deploy Apache Airflow projects faster and smoother.
Airflow Toolkit
โญย
101
Any Airflow project day 1, you can spin up a local desktop Kubernetes Airflow environment AND one in Google Cloud Composer with tested data pipelines(DAGs) ๐ฅ๏ธ >> [ ๐, ๐ข ]
Ap Airflow
โญย
100
Astronomer Core Docker Images
Incremental_training
โญย
96
Repo that relates to the Medium blog 'Keeping your ML model in shape with Kafka, Airflow' andย MLFlow'
Deployml_course
โญย
93
ะ ะตะฟะพะทะธัะพัะธะน ะดะปั ะพัะบัััะพะณะพ ะบัััะฐ ยซะัะพะผััะปะตะฝะฝะฐั ัะบัะฟะปัะฐัะฐัะธั ะผะพะดะตะปะตะน ะผะฐัะธะฝะฝะพะณะพ ะพะฑััะตะฝะธัยป
Udacity Data Eng Proj 1
โญย
81
Developed a data pipeline to automate data warehouse ETL by building custom airflow operators that handle the extraction, transformation, validation and loading of data from S3 -> Redshift -> S3
Airflow_multi_dagrun
โญย
76
triggering a DAG run multiple times
Discreetly
โญย
70
ETLy is an add-on dashboard service on top of Apache Airflow.
Airflow Spark
โญย
64
Docker with Airflow and Spark standalone cluster
Dbt Airflow Docker Compose
โญย
64
Execution of DBT models using Apache Airflow through Docker Compose
Xene
โญย
52
A distributed workflow runner focusing on performance and simplicity.
Algotrading_stack
โญย
48
Microservices Stack for AlgoTrading
Pyconde2019 Airflow Ml Workshop
โญย
42
PyConDE & PyData Berlin 2019 Airflow Workshop: Airflow for machine learning pipelines.
Alphasql
โญย
39
AlphaSQL provides Integrated Type and Schema Check and Parallelization for SQL file set mainly for BigQuery
Airflow Kube Setup
โญย
35
How to deploy airflow on Kubernetes
Ecs Airflow
โญย
32
Cloudformation templates for deploying Airflow in ECS
Sagerx
โญย
30
Open drug data pipelines curated by pharmacists.
Docker Airflow
โญย
28
Repo for building docker based airflow image. Containers support multiple features like writing logs to local or S3 folder and Initializing GCP while container booting. https://abhioncbr.github.io/docker-airflow/
Airflow Prometheus Exporter
โญย
25
Export Airflow metrics (from mysql) in prometheus format
Reach
โญย
23
Wellcome tool to parse references scraped from policy documents using machine learning
Udacity Data Eng Proj2
โญย
23
A production-grade data pipeline has been designed to automate the parsing of user search patterns to analyze user engagement. Extract data from S3, apply a series of transformations and load into S3 and Redshift.
Heroku Airflow
โญย
21
Demo of Author data workflows with Airflow on Heroku (not maintained)
Terraform Infra
โญย
20
Production Grade Terraform for Provisioning Infrastructure
Airflow Tutorial
โญย
19
Tutorial like code for how to deploy airflow using docker and how to use the DockerOperator.
Spotify Api
โญย
19
Pipeline that extracts data from the Spotify API to build a more detailed version of Spotify Wrapped
Airflow Docker
โญย
19
This is my Apache Airflow Local development setup on Windows 10 WSL2/Mac using docker-compose. It will also include some sample DAGs and workflows.
Final Project Level3 Recsys 06
โญย
18
๐ฅ ๋น์ ์ ์ถ์ฒํ๋ ๊ฐ์ฅ ์ข์ ๋ฐฉ๋ฒ RecommendU @boostcampaitech4
Airflow Git Sync
โญย
17
Sync DAG changes from Git to Airflow
Airflow Docker
โญย
16
Apache Airflow Docker Image.
Qunomon
โญย
15
Testbed of AI Systems Quality Management
Airflow Docker Compose Example
โญย
15
Example implementation running Airflow as separate services with docker-compose.
Airflow Ci
โญย
15
Apache Airflow CI pipeline
Health Equity Tracker
โญย
15
Health Equity Tracker is a free-to-use data visualization platform that is enabling new insights into the impact of COVID-19 and other social and political determinants of health on historically underrepresented groups in the United States.
Opentrials Airflow
โญย
15
Configuration and definitions of Airflow for OpenTrials
Ghcn D
โญย
14
Data Pipeline from the Global Historical Climatology Network DataSet
Ml Airflow
โญย
13
Generalized project for running Airflow DAGs, with possibility of skipping tasks already done for some set of input parameters.
Level3_nlp_finalproject Nlp 08
โญย
12
์ฌ์ฉ์๊ฐ ์ฑํ ์น์ ํตํด ์์ ์ด ์ฒํ ๋ฒ๋ฅ ์ ์ํฉ์ ์ ์ํ๋ฉด, ์ ๋ ฅ์ ๋ํ ๋ฌธ๋งฅ์ ๋ชจ๋ธ์ด ์ดํดํ์ฌ ๊ฐ์ด๋๋ผ์ธ์ ์ ์ํ๊ณ , ์ ์ฌํ ์ํฉ์ ํ๋ก๋ฅผ ์ ๊ณตํ๋ ์น ์๋น์ค์ ๋๋ค. (2023.08.18 ์๋น์ค ์ข ๋ฃ)
Marxan Cloud
โญย
12
Monorepo for the Marxan Cloud platform
Airflowjob
โญย
11
Airflow POC demo : 1) env set up 2) airflow DAG 3) Spark/ML pipeline | #DE
Greatex
โญย
10
A project for exploring how Great Expectations can be used to ensure data quality and validate batches within a data pipeline defined in Airflow.
Shopme
โญย
10
ShopME: An E2E fashion recommendation System
Miniflow
โญย
9
This repository aims to help you have an Airflow installation running in Kubernetes minikube.
Airflow Ubuntu Ansible Terraform
โญย
9
Demo deployment of Apache Airflow on EC2/Ubuntu with Docker-compose, Terraform, and Ansible
Docker Airflow
โญย
9
A basic dockerized implementation of (clustered) Airflow
Docker Airflow
โญย
9
Guide on how to setup Apache Airflow containers using Docker and IBM Bluemix
Airflow Kubernetes
โญย
9
Simple Airflow on Kubernetes (GKE)
From Keras To Production Workshop
โญย
8
Udacity Airflow
โญย
8
Udacity Data Pipeline Exercises
Coronalytics
โญย
7
A full ETL pipeline to visualize the corona virus outbreak.
Openeo Eodc Driver
โญย
7
openEO EODC driver
Docker Bigdata Cluster
โญย
7
Infrastructure automation to deploy Hadoop,Hive,Spark,airflow nodes on a docker host
30daysofairflow
โญย
7
30 Days of Airflow
Dsflow
โญย
7
a sandbox for data pipelines
Airflow Docker
โญย
6
Run Apache Airflow using Docker containers
Iac Terraform Emr
โญย
6
AWS Summit 2022 ASEAN --- COM203 Using IaC with Terraform to provision Big Data Platform on Amazon EMR
Airflow_selenium
โญย
6
This repo demonstrates how to set up an airflow environment and custom Selenium plugin.
Dockerized_data_science_playground
โญย
6
Multi-docker container data science / engineering playground (w/ Kafka, Airflow, MLFlow, Tensorflow-Keras / SKLearn) for simulating a microservices-oriented architecture
My Notes
โญย
6
My personal notes on different topics I'm studying
Spark Mesos Airflow Tutorial
โญย
6
Tfl Bikes Data Pipeline
โญย
6
Processing TFL data for bike usage with Google Cloud Platform.
Real Estate Sale Analytics
โญย
5
Create data pipeline using Lambda architecture with Spark, Kafka, Airflow and Snowflake
Docker Airflow
โญย
5
Docker for airflow with mysql as backend
Docker Airflow
โญย
5
Docker Apache Airflow
Aiscalator
โญย
5
Tools to streamline Jupyter Notebook Prototypes into robust Data Products
Mlops
โญย
5
Full Machine Learning Lifecycle using Airflow, MLflow, and AWS S3
Data_infra_repo
โญย
5
Collections of POC/dev data infrastructure. | #SE
Stock_streaming_pipeline_project
โญย
5
Built a real-time streaming pipeline to extract stock data, using Apache Nifi, Debezium, Kafka, and Spark Streaming. Loaded the transformed data into Glue database and created real-time dashboards using Power BI and Tableau with Athena. The pipeline is orchestrated using Airflow.
Global Data Pipeline
โญย
5
Code Repository for my 3rd Data Project.
Analytics_data_where_house
โญย
5
An analytics engineering sandbox focusing on real estates prices in Cook County, IL
Docker Airflow Datax
โญย
5
Docker Compose Airflow
โญย
5
Docker Compose Apache Airflow (Official Docker Images) with CeleryExecutor, InitDB and InitUser
Airflow Tutorial
โญย
5
Steam Data Engineering
โญย
5
A data engineering project with Airflow, dbt, Terrafrom, GCP and much more!
Lakehouse
โญย
5
Playground for Lakehouse (Iceberg, Hudi, Spark, Flink, Trino, DBT, Airflow, Kafka, Debezium CDC)
Related Searches
Shell Docker (20,660)
Docker Dockerfile (16,395)
Python Docker (16,341)
Javascript Docker (10,426)
Golang Docker (7,702)
Php Docker (6,192)
Java Docker (6,071)
Docker Nginx (5,238)
Typescript Docker (4,630)
Docker Postgresql (4,363)
1-100 of 102 search results
Next >
Privacy
ย |ย
About
ย |ย
Terms
ย |ย
Follow Us On Twitter
Copyright 2018-2024 Awesome Open Source.ย All rights reserved.