Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

The Top 23 Spark Etl Open Source Projects

Open source projects categorized as Spark Etl
Doris ⭐ 11,243
Apache Doris is an easy-to-use, high performance and unified analytics database.
total releases 8most recent commit 11 days ago
Dagster ⭐ 9,467
An orchestration platform for the development, production, and observation of data assets.
dependent packages 133total releases 585most recent commit 3 months agopypi dagster} Downloads
Mage Ai ⭐ 6,324
🧙 The modern replacement for Airflow. Build, run, and manage data pipelines for integrating and transforming data.
total releases 314most recent commit 3 months ago
Aws Glue Samples ⭐ 1,334
AWS Glue code samples
most recent commit 6 months ago
Pyspark Example Project ⭐ 1,034
Example project implementing best practices for PySpark ETL jobs and applications.
most recent commit a year ago
Zingg ⭐ 828
Scalable identity resolution, entity resolution, data mastering and deduplication using ML
total releases 1most recent commit 3 months ago
Goodreads_etl_pipeline ⭐ 593
An end-to-end GoodReads Data Pipeline for Building Data Lake, Data Warehouse and Analytics Platform.
most recent commit 4 years ago
Aws Glue Libs ⭐ 568
AWS Glue Libraries are additions and enhancements to Spark for ETL operations.
most recent commit 9 months ago
Metorikku ⭐ 536
A simplified, lightweight ETL Framework based on Apache Spark
total releases 126most recent commit a year ago
Spark Excel ⭐ 421
A Spark plugin for reading and writing Excel files
dependent packages 6total releases 43most recent commit 3 months ago
Zdh_web ⭐ 379
大数据采集,抽取平台,zdh_web是zdh系列服务的可视化管理平台，包含数据采集,调度,权限,审批
most recent commit 3 months ago
Big_data_architect_skills ⭐ 353
一个大数据架构师应该掌握的技能
most recent commit 5 years ago
Data Engineering Projects ⭐ 322
Personal Data Engineering Projects
most recent commit a year ago
Beginner_de_project ⭐ 276
Beginner data engineering project - batch edition
most recent commit a year ago
Butterfree ⭐ 269
A tool for building feature stores.
dependent packages 1total releases 35most recent commit 5 months agopypi butterfree} Downloads
Setl ⭐ 173
A simple Spark-powered ETL framework that just works 🍺
total releases 4most recent commit 4 months ago
Cobrix ⭐ 131
A COBOL parser and Mainframe/EBCDIC data source for Apache Spark
dependent packages 2total releases 43most recent commit 3 months ago
Easy_sql ⭐ 126
A library developed to ease the data ETL development process.
total releases 10most recent commit 3 months ago
Chombo ⭐ 102
Big Data ETL and Utilities for Hadoop Map Reduce, Spark and Storm
most recent commit 3 years ago
Flowman ⭐ 85
Flowman is an ETL framework powered by Apache Spark. With its declarative approach, Flowman simplifies the development of complex data pipelines.
dependent packages 24total releases 65most recent commit 4 months ago
Gallia Core ⭐ 79
A schema-aware Scala library for data transformation
dependent packages 1total releases 6most recent commit 4 months ago
Data Engineering Nanodegree ⭐ 76
Projects done in the Data Engineering Nanodegree by Udacity.com
most recent commit 5 years ago
Luigi Warehouse ⭐ 73
A luigi powered analytics / warehouse stack
most recent commit 7 years ago
The 10 Most Depended On Spark Etl Open Source Projects
Dagster ⭐ 9,467
An orchestration platform for the development, production, and observation of data assets.
dependent packages 133total releases 585latest release December 07, 2023most recent commit 3 months agopypi dagster} Downloads
Spark Excel ⭐ 421
A Spark plugin for reading and writing Excel files
dependent packages 6total releases 43latest release February 22, 2021most recent commit 3 months ago
Diem ⭐ 10
DIEM Data Integration Engine Multipurpose
dependent packages 3total releases 2latest release February 24, 2021most recent commit 4 months agonpm @mydiem/diem-util} Downloads
Pramen ⭐ 20
Resilient data pipeline framework running on Apache Spark
dependent packages 2total releases 19latest release November 10, 2023most recent commit 3 months ago
Cobrix ⭐ 131
A COBOL parser and Mainframe/EBCDIC data source for Apache Spark
dependent packages 2total releases 43latest release October 16, 2023most recent commit 3 months ago
Gallia Core ⭐ 79
A schema-aware Scala library for data transformation
dependent packages 1total releases 6latest release September 19, 2023most recent commit 4 months ago
Butterfree ⭐ 269
A tool for building feature stores.
dependent packages 1total releases 35latest release November 14, 2023most recent commit 5 months agopypi butterfree} Downloads
The 10 Latest Releases In Spark Etl Open Source Projects
Dagster ⭐ 9,467
An orchestration platform for the development, production, and observation of data assets.
dependent packages 133total releases 585latest release December 07, 2023most recent commit 3 months agopypi dagster} Downloads
Yaetos ⭐ 32
Write data & AI pipelines in (SQL, Spark, Pandas) and deploy to the cloud, simplified
total releases 32latest release November 27, 2023most recent commit 3 months ago
Butterfree ⭐ 269
A tool for building feature stores.
dependent packages 1total releases 35latest release November 14, 2023most recent commit 5 months agopypi butterfree} Downloads
Pramen ⭐ 20
Resilient data pipeline framework running on Apache Spark
dependent packages 2total releases 19latest release November 10, 2023most recent commit 3 months ago
Cobrix ⭐ 131
A COBOL parser and Mainframe/EBCDIC data source for Apache Spark
dependent packages 2total releases 43latest release October 16, 2023most recent commit 3 months ago
Onetl ⭐ 55
One ETL tool to rule them all
total releases 11latest release October 10, 2023most recent commit 3 months ago
Doris ⭐ 11,243
Apache Doris is an easy-to-use, high performance and unified analytics database.
total releases 8latest release September 27, 2023most recent commit 11 days ago
Gallia Core ⭐ 79
A schema-aware Scala library for data transformation
dependent packages 1total releases 6latest release September 19, 2023most recent commit 4 months ago
Popular Data Processing Categories
Jupyter Notebook
Dataset
Sql
Validation
Pipeline
Translation
Data Science
Classification
Transaction
Scraper
Categories
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2024 Awesome Open Source.  All rights reserved.