Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

The Top 23 Scala Spark Etl Open Source Projects

Open source projects categorized as Scala Spark Etl
Metorikku ⭐ 536
A simplified, lightweight ETL Framework based on Apache Spark
total releases 126most recent commit a year ago
Spark Excel ⭐ 421
A Spark plugin for reading and writing Excel files
dependent packages 6total releases 43most recent commit 3 months ago
Setl ⭐ 173
A simple Spark-powered ETL framework that just works 🍺
total releases 4most recent commit 5 months ago
Cobrix ⭐ 131
A COBOL parser and Mainframe/EBCDIC data source for Apache Spark
dependent packages 2total releases 43most recent commit 3 months ago
Flowman ⭐ 85
Flowman is an ETL framework powered by Apache Spark. With its declarative approach, Flowman simplifies the development of complex data pipelines.
dependent packages 24total releases 65most recent commit 5 months ago
Gallia Core ⭐ 79
A schema-aware Scala library for data transformation
dependent packages 1total releases 6most recent commit 4 months ago
Spark Etl ⭐ 62
Apache Spark based ETL Engine
most recent commit 8 years ago
Zdh_server ⭐ 56
数据采集平台zdh,etl 处理服务
most recent commit 10 months ago
Etlflow ⭐ 43
EtlFlow is an ecosystem of functional libraries in Scala based on ZIO for running complex Auditable workflows which can interact with Google Cloud Platform, AWS, Kubernetes, Databases, SFTP servers, On-Prem Systems and more.
dependent packages 11total releases 37most recent commit 9 months ago
Spark Ref Architecture ⭐ 38
Reference Architectures for Apache Spark
most recent commit 7 years ago
Etl Light ⭐ 38
A light Kafka to HDFS/S3 ETL library based on Apache Spark
most recent commit 7 years ago
Sope ⭐ 37
Apache Spark ETL Utilities
most recent commit 10 months ago
Sharpetl ⭐ 36
Write ETL using your favorite SQL dialects
most recent commit 4 months ago
Amazon Eks Apache Spark Etl Sample ⭐ 35
Spark ETL example processing New York taxi rides public dataset on EKS
most recent commit a year ago
Ides ⭐ 32
智能数据探索服务(Intelligent Data Exploration Service)，一站式Data + AI数据解决方案！
most recent commit 10 months ago
Starlake ⭐ 29
Starlake is an On Premise and Cloud ELT/ETL Framework for Batch & Stream Processing
total releases 55most recent commit 3 months ago
Nebula Exchange ⭐ 26
NebulaGraph Exchange is an Apache Spark application to parse data from different sources to NebulaGraph in a distributed environment. It supports both batch and streaming data in various formats and sources including other Graph Databases, RDBMS, Data warehouses, NoSQL, Message Bus, File systems, etc.
most recent commit 3 months ago
Wasp ⭐ 25
WASP is a framework to build complex real time big data applications. It relies on a kind of Kappa/Lambda architecture mainly leveraging Kafka and Spark. If you need to ingest huge amount of heterogeneous data and analyze them through complex pipelines, this is the framework for you.
dependent packages 15total releases 25most recent commit 7 months ago
Daflow ⭐ 24
Apache-Spark based Data Flow(ETL) Framework which supports multiple read, write destinations of different types and also support multiple categories of transformation rules.
most recent commit 4 years ago
Pramen ⭐ 20
Resilient data pipeline framework running on Apache Spark
dependent packages 2total releases 19most recent commit 3 months ago
Cda Client ⭐ 19
Cloud Data Access client
most recent commit a year ago
Spark Etl ⭐ 15
Set of ETL utils for Spark
most recent commit 4 years ago
Telemetry Streaming ⭐ 15
Spark Streaming ETL jobs for Mozilla Telemetry
most recent commit 4 years ago
The 7 Most Depended On Scala Spark Etl Open Source Projects
Spark Excel ⭐ 421
A Spark plugin for reading and writing Excel files
dependent packages 6total releases 43latest release February 22, 2021most recent commit 3 months ago
Cobrix ⭐ 131
A COBOL parser and Mainframe/EBCDIC data source for Apache Spark
dependent packages 2total releases 43latest release October 16, 2023most recent commit 3 months ago
Pramen ⭐ 20
Resilient data pipeline framework running on Apache Spark
dependent packages 2total releases 19latest release November 10, 2023most recent commit 3 months ago
Gallia Core ⭐ 79
A schema-aware Scala library for data transformation
dependent packages 1total releases 6latest release September 19, 2023most recent commit 4 months ago
The 10 Latest Releases In Scala Spark Etl Open Source Projects
Pramen ⭐ 20
Resilient data pipeline framework running on Apache Spark
dependent packages 2total releases 19latest release November 10, 2023most recent commit 3 months ago
Cobrix ⭐ 131
A COBOL parser and Mainframe/EBCDIC data source for Apache Spark
dependent packages 2total releases 43latest release October 16, 2023most recent commit 3 months ago
Gallia Core ⭐ 79
A schema-aware Scala library for data transformation
dependent packages 1total releases 6latest release September 19, 2023most recent commit 4 months ago
Starlake ⭐ 29
Starlake is an On Premise and Cloud ELT/ETL Framework for Batch & Stream Processing
total releases 55latest release September 14, 2023most recent commit 3 months ago
Metorikku ⭐ 536
A simplified, lightweight ETL Framework based on Apache Spark
total releases 126latest release February 27, 2023most recent commit a year ago
Spark Excel ⭐ 421
A Spark plugin for reading and writing Excel files
dependent packages 6total releases 43latest release February 22, 2021most recent commit 3 months ago
Setl ⭐ 173
A simple Spark-powered ETL framework that just works 🍺
total releases 4latest release August 21, 2020most recent commit 5 months ago
Popular Data Processing Categories
Jupyter Notebook
Dataset
Sql
Validation
Pipeline
Translation
Data Science
Classification
Transaction
Scraper
Categories
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2024 Awesome Open Source.  All rights reserved.