Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

The Top 23 Spark Airflow Open Source Projects

Open source projects categorized as Spark Airflow
Categories > Control Flow > Airflow
Pipeline ⭐ 4,158
PipelineAI
total releases 85most recent commit a year ago
Dataspherestudio ⭐ 2,860
DataSphereStudio is a one stop data application development& management portal, covering scenarios including data exchange, desensitization/cleansing, analysis/mining, quality measurement, visualization, and task scheduling.
dependent packages 39total releases 7most recent commit 3 months ago
Around Dataengineering ⭐ 926
A Data Engineering & Machine Learning Knowledge Hub
most recent commit 2 years ago
Goodreads_etl_pipeline ⭐ 593
An end-to-end GoodReads Data Pipeline for Building Data Lake, Data Warehouse and Analytics Platform.
most recent commit 4 years ago
Data Engineering Interview Questions ⭐ 554
More than 2000+ Data engineer interview questions.
most recent commit 7 months ago
Agile_data_code_2 ⭐ 435
Code for Agile Data Science 2.0, O'Reilly 2017, Second Edition
most recent commit a year ago
Data Engineering Projects ⭐ 322
Personal Data Engineering Projects
most recent commit a year ago
Compass ⭐ 284
Compass is a task diagnosis platform for bigdata
most recent commit 3 months ago
Beginner_de_project ⭐ 276
Beginner data engineering project - batch edition
most recent commit a year ago
Airflow Pipeline ⭐ 168
An Airflow docker image preconfigured to work well with Spark and Hadoop/EMR
most recent commit 5 months ago
Movalytics Data Warehouse ⭐ 114
Data pipeline performing ETL to AWS Redshift using Spark, orchestrated with Apache Airflow
most recent commit 4 years ago
Streamify ⭐ 97
A data engineering project with Kafka, Spark Streaming, dbt, Docker, Airflow, Terraform, GCP and much more!
most recent commit 2 years ago
Data Engineering Nanodegree ⭐ 76
Projects done in the Data Engineering Nanodegree by Udacity.com
most recent commit 5 years ago
Airflow Spark Operator Plugin ⭐ 66
A plugin to Apache Airflow to allow you to run Spark Submit Commands as an Operator
most recent commit 5 years ago
Airflow Spark ⭐ 64
Docker with Airflow and Spark standalone cluster
most recent commit 2 years ago
Cloud Bigdata Book ⭐ 53
write book
most recent commit 3 years ago
Works With Determined ⭐ 43
This repository contains example integrations between Determined and other ML products
most recent commit 7 months ago
Udacity Data Engineering ⭐ 42
Udacity Data Engineering Nano Degree (DEND)
most recent commit 4 years ago
Pyjaws ⭐ 36
PyJaws: A Pythonic Way to Define Databricks Jobs and Workflows
most recent commit 5 months ago
Debussy_concert ⭐ 29
Debussy is an opinionated Data Architecture and Engineering framework, enabling data analysts and engineers to build better platforms and pipelines.
total releases 2most recent commit a year ago
Data Engineer Nanodegree Projects Udacity ⭐ 27
Projects done in the Data Engineer Nanodegree Program by Udacity.com
most recent commit 2 years ago
Data Engineering Nanodegree ⭐ 27
Solution to all projects of Udacity's Data Engineering Nanodegree: Data Modeling with Postgres & Cassandra, Data Warehouse with Redshift, Data Lake with Spark and Data Pipeline with Airflow.
most recent commit a year ago
Forklift ⭐ 22
🚚 ETL for Spark and Airflow
most recent commit 6 years ago
The 4 Latest Releases In Spark Airflow Open Source Projects
Airflow Livy Operators ⭐ 17
Lets Airflow DAGs run Spark jobs via Livy: sessions and/or batches.
total releases 8latest release August 27, 2021most recent commit a year ago
Pipeline ⭐ 4,158
PipelineAI
total releases 85latest release July 18, 2017most recent commit a year ago
Popular Control Flow Categories
Stream
Thread
Promise
Series
Webhook
Daemon
Parallel
Reactive
Rxjava
Cron
Categories
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2024 Awesome Open Source.  All rights reserved.