Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About
Search results for pipeline airflow
airflow
x
pipeline
x
37 search results found
Argo Workflows
⭐
14,264
Workflow Engine for Kubernetes
Orchest
⭐
3,876
Build data pipelines, the easy way 🛠️
Ploomber
⭐
3,318
The fastest ⚡️ way to build data pipelines. Develop iteratively, deploy anywhere. ☁️
Awesome Etl
⭐
3,041
A curated list of awesome ETL frameworks, libraries, and software.
Elyra
⭐
1,721
Elyra extends JupyterLab with an AI centric approach.
Couler
⭐
847
Unified Interface for Constructing and Managing Workflows on different workflow engines, such as Argo Workflows, Tekton Pipelines, and Apache Airflow.
Goodreads_etl_pipeline
⭐
593
An end-to-end GoodReads Data Pipeline for Building Data Lake, Data Warehouse and Analytics Platform.
Data Pipelines With Apache Airflow
⭐
587
Code for Data Pipelines with Apache Airflow
Torchx
⭐
275
TorchX is a universal job launcher for PyTorch applications. TorchX is designed to have fast iteration time for training/research and support for E2E production ML pipelines when you're ready.
Data Testing With Airflow
⭐
190
Whirl
⭐
183
Fast iterative local development and testing of Apache Airflow workflows
Dataplane
⭐
171
Dataplane is an Airflow inspired unified data platform with additional data mesh and RPA capability to automate, schedule and design data pipelines and workflows. Dataplane is written in Golang with a React front end.
Airflow_for_beginners
⭐
166
Incubator Liminal
⭐
131
Apache Liminals goal is to operationalise the machine learning process, allowing data scientists to quickly transition from a successful experiment to an automated pipeline of model training, validation, deployment and inference in production. Liminal provides a Domain Specific Language to build ML workflows on top of Apache Airflow.
Kedro Airflow
⭐
120
Kedro-Airflow makes it easy to deploy Kedro projects to Airflow.
Hark Lang
⭐
104
A serverless virtual machine
Deployml_course
⭐
93
Репозиторий для открытого курса «Промышленная эксплуатация моделей машинного обучения»
Data Engineering Nanodegree
⭐
76
Projects done in the Data Engineering Nanodegree by Udacity.com
Aws Concurrent Data Orchestration Pipeline Emr Livy
⭐
66
This code demonstrates the architecture featured on the AWS Big Data blog (https://aws.amazon.com/blogs/big-data/ ) which creates a concurrent data pipeline by using Amazon EMR and Apache Livy. This pipeline is orchestrated by Apache Airflow.
Airflow Tutorial
⭐
51
🐍💨 Airflow tutorial for PyCon 2019
Soopervisor
⭐
42
☁️ Export Ploomber pipelines to Kubernetes (Argo), Airflow, AWS Batch, SLURM, and Kubeflow.
Pyconde2019 Airflow Ml Workshop
⭐
42
PyConDE & PyData Berlin 2019 Airflow Workshop: Airflow for machine learning pipelines.
Ml In Production
⭐
39
The practical use-cases of how to make your Machine Learning Pipelines robust and reliable using Apache Airflow.
Cloud Data Lake
⭐
34
Data lake, data warehouse on GCP
Data Engineer Nanodegree Projects Udacity
⭐
27
Projects done in the Data Engineer Nanodegree Program by Udacity.com
Awesome Kedro
⭐
22
Driblet
⭐
20
Pipeline Editor
⭐
19
Common pipeline-editor components used in different clients (e.g. Elyra application, Web browser extensions, etc)
Airflowetl
⭐
16
Blog post on ETL pipelines with Airflow
Phila Airflow
⭐
16
Airflowdatapipeline
⭐
15
Example of an ETL Pipeline using Airflow
Airflow Ci
⭐
15
Apache Airflow CI pipeline
Airflow Imaging Plugins
⭐
14
Set of plugins helping to work with imaging data in Airflow.
Thepipelinetool
⭐
13
A pipeline orchestration tool
Airflow Rbac Roles Cli
⭐
12
A tool to create Airflow RBAC roles with dag-level permissions from cli.
Security Token Analytics
⭐
12
Blockchain data pipeline using Airflow, Kubernetes, Redshift, and Grafana
Snql
⭐
11
Code for SNQL, an app and data pipeline to track my sneaker habits
Greatex
⭐
10
A project for exploring how Great Expectations can be used to ensure data quality and validate batches within a data pipeline defined in Airflow.
Python_packages_for_pipeline_workflow
⭐
9
Opendata Airflow Tutorial
⭐
9
Tutorials on airflow pipelines with open data sets
Airflow
⭐
8
This set of code and instructions has the porpouse to instanciate a compiled environment with set of docker images like airflow webserver, airflow scheduler, postgresql, pyspark, Data Pipeline consuming data from weather api , processing with pyspark and storing in postgresql
Airflow
⭐
8
Workflow manager from Airbnb
Udacity Airflow
⭐
8
Udacity Data Pipeline Exercises
Airflow_to_manage_talend_etl_jobs
⭐
8
Airflow, an open source platform, is used to orchestrate workflows as Directed Acyclic Graphs (DAGs) of tasks in a programmatic manner. An airflow scheduler is used to schedule workflows and data processing pipelines. Airflow user interface allows easy visualization of pipelines running in production environment, monitoring of the progress of the workflows, and troubleshooting issues when needed.
Airflow Data Pipeline
⭐
7
Udacity Data Engineer Nanodegree - Airflow data pipeline
Aws Etl
⭐
7
This is an ETL application on AWS with general open sales and customer data that you can find here: https://github.com/camposvinicius/data/blob/main/A it's a zipped file with some .csvs inside that we will apply transformations.
Fn_graph
⭐
7
Lightweight function pipelines for Python
Meetup Spark Airflow Demo
⭐
7
Spark & Airflow demo for meetup
30daysofairflow
⭐
7
30 Days of Airflow
Coronalytics
⭐
7
A full ETL pipeline to visualize the corona virus outbreak.
Dsflow
⭐
7
a sandbox for data pipelines
Demotrends
⭐
6
Code required to setup the demo trends website (http://demotrends.qubole.com)
Sheenflow
⭐
6
A next-generation open source orchestration platform for the development, production, and observation of data assets.
Airflow4ds
⭐
6
Using Apache Airflow to author, run and monitor complex data pipelines.
Airflow Ml Pipeline
⭐
5
The repository to learn Machine Learning with Airflow
Tfx Gpt2
⭐
5
Code for creating end-end TFX production pipeline for GPT-2.
Airflow Dags
⭐
5
Related Searches
Python Pipeline (4,255)
Javascript Pipeline (1,369)
Pipeline Jenkins (1,150)
Shell Pipeline (1,143)
Docker Pipeline (1,018)
Jupyter Notebook Pipeline (976)
Java Pipeline (868)
Pipeline Continuous Integration (808)
Golang Pipeline (682)
Python Airflow (646)
1-37 of 37 search results
Privacy
|
About
|
Terms
|
Follow Us On Twitter
Copyright 2018-2024 Awesome Open Source. All rights reserved.