Awesome Open Source

Programming Languages

Search results for etl airflow

58 search results found

Airflow ⭐ 34,468

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

Orchest ⭐ 3,876

Build data pipelines, the easy way 🛠️

Awesome Etl ⭐ 3,041

A curated list of awesome ETL frameworks, libraries, and software.

Etl With Airflow ⭐ 1,053

ETL best practices with airflow, with examples

Optimus ⭐ 707

Optimus is an easy-to-use, reliable, and performant workflow orchestrator for data transformation, data modeling, pipelines, and data quality management.

Goodreads_etl_pipeline ⭐ 593

An end-to-end GoodReads Data Pipeline for Building Data Lake, Data Warehouse and Analytics Platform.

Ethereum Etl Airflow ⭐ 378

Airflow DAGs for exporting, loading, and parsing the Ethereum blockchain data. How to get any Ethereum smart contract into BigQuery https://towardsdatascience.com/how-to-get-any-ethe

Data Engineering Projects ⭐ 322

Personal Data Engineering Projects

Astro Sdk ⭐ 303

Astro SDK allows rapid and clean development of {Extract, Load, Transform} workflows using Python and SQL, powered by Apache Airflow.

Beginner_de_project ⭐ 276

Beginner data engineering project - batch edition

Example Airflow Dags ⭐ 204

Example DAGs using hooks and operators from Airflow Plugins

Dataplane ⭐ 171

Dataplane is an Airflow inspired unified data platform with additional data mesh and RPA capability to automate, schedule and design data pipelines and workflows. Dataplane is written in Golang with a React front end.

Airflow_for_beginners ⭐ 166

Aws Ecs Airflow ⭐ 110

Run Airflow in AWS ECS(Elastic Container Service) using Fargate tasks

Deployml_course ⭐ 93

Репозиторий для открытого курса «Промышленная эксплуатация моделей машинного обучения»

Polygon Etl ⭐ 93

ETL (extract, transform and load) tools for ingesting Polygon blockchain data to Google BigQuery and Pub/Sub

Data Engineering Nanodegree ⭐ 76

Projects done in the Data Engineering Nanodegree by Udacity.com

Discreetly ⭐ 70

ETLy is an add-on dashboard service on top of Apache Airflow.

Data Engineering made simple - An opinionated Data Engineering framework

Udacity Data Engineering ⭐ 42

Udacity Data Engineering Nano Degree (DEND)

Blockchain Etl Architecture ⭐ 36

Blockchain ETL Architecture

Stellar Etl Airflow ⭐ 31

Airflow DAGs for the Stellar ETL project

Data Engineer Nanodegree Projects Udacity ⭐ 27

Projects done in the Data Engineer Nanodegree Program by Udacity.com

Python_mozetl ⭐ 26

ETL jobs for Firefox Telemetry

Airflow Jupyter Docker Compose ⭐ 24

Orchestration of data science and earth observation models in Apache Airflow, scale-up with Celery Executor, experiment with jupyter notebook using a docker containers composition

Forklift ⭐ 22

🚚 ETL for Spark and Airflow

Airflow Snowflake ⭐ 22

Code to be contributed to the Apache Airflow (incubating) project for ETL workflow management for integrating with the Snowflake Data Warehouse.

Airflow Tutorial ⭐ 19

Tutorial like code for how to deploy airflow using docker and how to use the DockerOperator.

Dolthub Etl Jobs ⭐ 17

ETL jobs that DoltHub maintained that load public data into DoltHub.

Phila Airflow ⭐ 16

Airflowetl ⭐ 16

Blog post on ETL pipelines with Airflow

Airflow Provider Fivetran Async ⭐ 15

A new Airflow Provider for Fivetran, maintained by Astronomer and Fivetran

Greenish ⭐ 15

Data monitoring tool, monitors the result, not the run

Airflowdatapipeline ⭐ 15

Example of an ETL Pipeline using Airflow

Telemetry Streaming ⭐ 15

Spark Streaming ETL jobs for Mozilla Telemetry

Etl Airflow S3 ⭐ 14

ETL of newspaper article keywords using Apache Airflow, Newspaper3k, Quilt T4 and AWS S3

Data Pipeline from the Global Historical Climatology Network DataSet

Airflow Ml Prediction ⭐ 13

Running ECS task for ML prediction orchestrated by Airflow

Thepipelinetool ⭐ 13

A pipeline orchestration tool

Airflowjob ⭐ 11

Airflow POC demo : 1) env set up 2) airflow DAG 3) Spark/ML pipeline | #DE

Rb_status_plugin ⭐ 11

rb_status_plugin : Data confidence tool for Airflow

Apache Airflow Providers Transfers ⭐ 10

Data Pipeline With Dbt Using Airflow On Gcp ⭐ 10

This project demonstrates how to build and automate an ETL pipeline using DAGs in Airflow and load the transformed data to Bigquery. There are different tools that have been used in this project such as Astro, DBT, GCP, Airflow, Metabase.

A project for exploring how Great Expectations can be used to ensure data quality and validate batches within a data pipeline defined in Airflow.

Awesome Dataops ⭐ 10

Awesome list of dataops products, open source and resources

Python_packages_for_pipeline_workflow ⭐ 9

Ca_visit_tracking ⭐ 9

Tac Airflow Plugin ⭐ 8

TAC is an airflow plugin which helps you to Extract transform and Load your data, bit more easily

Coronalytics ⭐ 7

A full ETL pipeline to visualize the corona virus outbreak.

Meetup Spark Airflow Demo ⭐ 7

Spark & Airflow demo for meetup

30daysofairflow ⭐ 7

30 Days of Airflow

This is an ETL application on AWS with general open sales and customer data that you can find here: https://github.com/camposvinicius/data/blob/main/A it's a zipped file with some .csvs inside that we will apply transformations.

Data Engineer Portfolio ⭐ 6

This is a repository to demonstrate my details, skills, projects and to keep track of my progression in Data Analytics and Data Science topics.

Awesome Airflow ⭐ 6

A curated list of awesome Airflow platform, links, tips and resources

ETL repository for DoIT work

Airflow Etl Mssql Sample ⭐ 5

Airflow ETL MS SQL Sample Project

Udacity Data Engineering Nanodegree ⭐ 5

This is a repository to hold the files and notebooks produced throughout my Udacity's Nanodegree Data Engineering program.

Stock_streaming_pipeline_project ⭐ 5

Built a real-time streaming pipeline to extract stock data, using Apache Nifi, Debezium, Kafka, and Spark Streaming. Loaded the transformed data into Glue database and created real-time dashboards using Power BI and Tableau with Athena. The pipeline is orchestrated using Airflow.

Related Searches

Python Etl (814)

Python Airflow (646)

1-58 of 58 search results

Privacy | About | Terms | Follow Us On Twitter

Copyright 2018-2024 Awesome Open Source. All rights reserved.