Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

The Top 23 Spark Etl Open Source Projects

Open source projects categorized as Spark Etl
Doris ⭐ 7,619
Apache Doris is an easy-to-use, high performance and unified analytics database.
most recent commit 6 hours ago
Dagster ⭐ 6,968
An orchestration platform for the development, production, and observation of data assets.
dependent packages 89total releases 495most recent commit 2 hours agopypi dagster} Downloads
Mage Ai ⭐ 3,611
🧙 The modern replacement for Airflow. Build, run, and manage data pipelines for integrating and transforming data.
total releases 9most recent commit 2 hours ago
Aws Glue Samples ⭐ 1,250
AWS Glue code samples
most recent commit 18 hours ago
Pyspark Example Project ⭐ 1,034
Example project implementing best practices for PySpark ETL jobs and applications.
most recent commit 5 months ago
Zingg ⭐ 714
Scalable identity resolution, entity resolution, data mastering and deduplication using ML
most recent commit 8 hours ago
Goodreads_etl_pipeline ⭐ 593
An end-to-end GoodReads Data Pipeline for Building Data Lake, Data Warehouse and Analytics Platform.
most recent commit 3 years ago
Metorikku ⭐ 536
A simplified, lightweight ETL Framework based on Apache Spark
total releases 122most recent commit a month ago
Aws Glue Libs ⭐ 514
AWS Glue Libraries are additions and enhancements to Spark for ETL operations.
most recent commit 3 months ago
Spark Excel ⭐ 353
A Spark plugin for reading and writing Excel files
dependent packages 6total releases 43most recent commit 3 days ago
Big_data_architect_skills ⭐ 353
一个大数据架构师应该掌握的技能
most recent commit 4 years ago
Data Engineering Projects ⭐ 322
Personal Data Engineering Projects
most recent commit 2 months ago
Zdh_web ⭐ 282
大数据采集,抽取平台
most recent commit 6 months ago
Beginner_de_project ⭐ 276
Beginner data engineering project - batch edition
most recent commit a month ago
Butterfree ⭐ 249
A tool for building feature stores.
dependent packages 1total releases 30most recent commit 3 months agopypi butterfree} Downloads
Setl ⭐ 170
A simple Spark-powered ETL framework that just works 🍺
total releases 4most recent commit a month ago
Cobrix ⭐ 120
A COBOL parser and Mainframe/EBCDIC data source for Apache Spark
dependent packages 2total releases 31most recent commit 6 days ago
Easy_sql ⭐ 106
A library developed to ease the data ETL development process.
total releases 7most recent commit 11 days ago
Chombo ⭐ 102
Big Data ETL and Utilities for Hadoop Map Reduce, Spark and Storm
most recent commit 2 years ago
Data Engineering Nanodegree ⭐ 76
Projects done in the Data Engineering Nanodegree by Udacity.com
most recent commit 4 years ago
Luigi Warehouse ⭐ 73
A luigi powered analytics / warehouse stack
most recent commit 6 years ago
Gallia Core ⭐ 73
A schema-aware Scala library for data transformation
total releases 4most recent commit 25 days ago
Spark Etl ⭐ 62
Apache Spark based ETL Engine
most recent commit 6 years ago
The 5 Most Depended On Spark Etl Open Source Projects
Dagster ⭐ 6,968
An orchestration platform for the development, production, and observation of data assets.
dependent packages 89total releases 495latest release July 06, 2022most recent commit 2 hours agopypi dagster} Downloads
Spark Excel ⭐ 353
A Spark plugin for reading and writing Excel files
dependent packages 6total releases 43latest release February 22, 2021most recent commit 3 days ago
Cobrix ⭐ 120
A COBOL parser and Mainframe/EBCDIC data source for Apache Spark
dependent packages 2total releases 31latest release August 24, 2022most recent commit 6 days ago
Butterfree ⭐ 249
A tool for building feature stores.
dependent packages 1total releases 30latest release June 15, 2021most recent commit 3 months agopypi butterfree} Downloads
The 10 Latest Releases In Spark Etl Open Source Projects
Cobrix ⭐ 120
A COBOL parser and Mainframe/EBCDIC data source for Apache Spark
dependent packages 2total releases 31latest release August 24, 2022most recent commit 6 days ago
Metorikku ⭐ 536
A simplified, lightweight ETL Framework based on Apache Spark
total releases 122latest release July 12, 2022most recent commit a month ago
Dagster ⭐ 6,968
An orchestration platform for the development, production, and observation of data assets.
dependent packages 89total releases 495latest release July 06, 2022most recent commit 2 hours agopypi dagster} Downloads
Yaetos ⭐ 23
Write data & AI pipelines in (SQL, Spark, Pandas) and deploy to the cloud, simplified
total releases 19latest release June 03, 2022most recent commit 6 months ago
Easy_sql ⭐ 106
A library developed to ease the data ETL development process.
total releases 7latest release May 15, 2022most recent commit 11 days ago
Starlake ⭐ 22
Starlake is a Spark Based On Premise and Cloud ELT/ETL Framework for Batch & Stream Processing
total releases 20latest release April 14, 2022most recent commit a day ago
Gallia Core ⭐ 73
A schema-aware Scala library for data transformation
total releases 4latest release January 05, 2022most recent commit 25 days ago
Getl ⭐ 7
An elegant way to ETL'ing
total releases 42latest release December 17, 2021most recent commit 5 months ago
Popular Data Processing Categories
Jupyter Notebook
Dataset
Sql
Validation
Pipeline
Translation
Classification
Scraper
Transaction
Data Science
Categories
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2023 Awesome Open Source.  All rights reserved.