Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

The Top 19 Spark Big Data Etl Open Source Projects

Open source projects categorized as Spark Big Data Etl
Metorikku ⭐ 536
A simplified, lightweight ETL Framework based on Apache Spark
total releases 126most recent commit a year ago
Zdh_web ⭐ 379
大数据采集,抽取平台,zdh_web是zdh系列服务的可视化管理平台，包含数据采集,调度,权限,审批
most recent commit 4 months ago
Big_data_architect_skills ⭐ 353
一个大数据架构师应该掌握的技能
most recent commit 5 years ago
Setl ⭐ 173
A simple Spark-powered ETL framework that just works 🍺
total releases 4most recent commit 5 months ago
Flowman ⭐ 85
Flowman is an ETL framework powered by Apache Spark. With its declarative approach, Flowman simplifies the development of complex data pipelines.
dependent packages 24total releases 65most recent commit 5 months ago
Udacity Data Engineer Nanodegree ⭐ 64
Classwork projects and home works done through Udacity data engineering nano degree
most recent commit 4 months ago
Datapipelines Essentials Python ⭐ 45
Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validation, Column extensions, SQL functions, and DataFrame transformations
most recent commit a year ago
Sharpetl ⭐ 36
Write ETL using your favorite SQL dialects
most recent commit 4 months ago
Ides ⭐ 32
智能数据探索服务(Intelligent Data Exploration Service)，一站式Data + AI数据解决方案！
most recent commit 10 months ago
Yaetos ⭐ 32
Write data & AI pipelines in (SQL, Spark, Pandas) and deploy to the cloud, simplified
total releases 32most recent commit 3 months ago
Zephyr ⭐ 21
Zephyr is a big data, platform agnostic ETL API, with Hadoop MapReduce, Storm, and other big data bindings.
most recent commit 8 years ago
Pramen ⭐ 20
Resilient data pipeline framework running on Apache Spark
dependent packages 2total releases 19most recent commit 3 months ago
Jun_bigdata ⭐ 18
jun_bigdata大数据平台服务框架。实现了Kafka实时数据过滤、清洗、转换、消费，实现了Sp SQL对Redis、MongoDB等非关系型数据库的数据的读写；集成了规则引擎，可基于规则引擎实现客
most recent commit a year ago
Bigdata Etl Pipeline ⭐ 10
The Data Pipeline and Analytics Stack is a comprehensive solution designed for processing, storing, and visualizing data. Explore a complete data pipeline with all components seamlessly set up and ready to use
most recent commit 4 months ago
Yasp ⭐ 9
Yet Another SPark Framework
most recent commit a year ago
Dlt With Debug ⭐ 8
A lightweight helper utility which allows developers to do interactive pipeline development by having a unified source code for both DLT run and Non-DLT interactive notebook run.
total releases 4most recent commit 2 years ago
Spooq ⭐ 8
total releases 10most recent commit a year ago
Data Engineer Portfolio ⭐ 6
This is a repository to demonstrate my details, skills, projects and to keep track of my progression in Data Analytics and Data Science topics.
most recent commit 5 months ago
Spark Databricks ⭐ 6
🔥 Master Apache Spark & Databricks! Dive into a world of big data with exclusive insights from Udemy courses, personal notes, and practical guides. Whether you're starting out or scaling new heights in data engineering, this is your ultimate resource hub! 🌟🚀
most recent commit 4 months ago
The 2 Most Depended On Spark Big Data Etl Open Source Projects
Pramen ⭐ 20
Resilient data pipeline framework running on Apache Spark
dependent packages 2total releases 19latest release November 10, 2023most recent commit 3 months ago
The 7 Latest Releases In Spark Big Data Etl Open Source Projects
Yaetos ⭐ 32
Write data & AI pipelines in (SQL, Spark, Pandas) and deploy to the cloud, simplified
total releases 32latest release November 27, 2023most recent commit 3 months ago
Pramen ⭐ 20
Resilient data pipeline framework running on Apache Spark
dependent packages 2total releases 19latest release November 10, 2023most recent commit 3 months ago
Metorikku ⭐ 536
A simplified, lightweight ETL Framework based on Apache Spark
total releases 126latest release February 27, 2023most recent commit a year ago
Spooq ⭐ 8
total releases 10latest release August 16, 2022most recent commit a year ago
Setl ⭐ 173
A simple Spark-powered ETL framework that just works 🍺
total releases 4latest release August 21, 2020most recent commit 5 months ago
Popular Data Processing Categories
Jupyter Notebook
Dataset
Sql
Validation
Pipeline
Translation
Data Science
Classification
Transaction
Scraper
Categories
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2024 Awesome Open Source.  All rights reserved.