Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About
Search results for scala etl
etl
x
scala
x
57 search results found
Metorikku
⭐
536
A simplified, lightweight ETL Framework based on Apache Spark
Spark Excel
⭐
421
A Spark plugin for reading and writing Excel files
Setl
⭐
173
A simple Spark-powered ETL framework that just works 🍺
Eel Sdk
⭐
140
Big Data Toolkit for the JVM
Hale
⭐
136
(Spatial) data harmonisation with hale studio (formerly HUMBOLDT Alignment Editor)
Cobrix
⭐
131
A COBOL parser and Mainframe/EBCDIC data source for Apache Spark
Dataxserver
⭐
106
为DataX(https://github.com/alibaba/DataX) 提供远程多语言调用(ThriftServer,HttpServer) 分布式运行(DataX on YARN) 功能
Flowman
⭐
85
Flowman is an ETL framework powered by Apache Spark. With its declarative approach, Flowman simplifies the development of complex data pipelines.
Gallia Core
⭐
79
A schema-aware Scala library for data transformation
Rocket Bi
⭐
79
A free, open-source, web-based self-service BI tailor-made for clickhouse, google bigquery, mysql, postgresql, vertica
Dataexpress
⭐
69
[NOT MAINTAINED] DataExpress is a simple, Scala-based cross database ETL toolkit supporting Postgres, MySql, Oracle, SQLServer, and Sqlite
Spark Etl
⭐
62
Apache Spark based ETL Engine
Zdh_server
⭐
56
数据采集平台zdh,etl 处理服务
Etlflow
⭐
43
EtlFlow is an ecosystem of functional libraries in Scala based on ZIO for running complex Auditable workflows which can interact with Google Cloud Platform, AWS, Kubernetes, Databases, SFTP servers, On-Prem Systems and more.
Etl Light
⭐
38
A light Kafka to HDFS/S3 ETL library based on Apache Spark
Spark Ref Architecture
⭐
38
Reference Architectures for Apache Spark
Sope
⭐
37
Apache Spark ETL Utilities
Sharpetl
⭐
36
Write ETL using your favorite SQL dialects
Amazon Eks Apache Spark Etl Sample
⭐
35
Spark ETL example processing New York taxi rides public dataset on EKS
Ides
⭐
32
智能数据探索服务(Intelligent Data Exploration Service),一站式Data + AI数据解决方案!
Starlake
⭐
29
Starlake is an On Premise and Cloud ELT/ETL Framework for Batch & Stream Processing
Nebula Exchange
⭐
26
NebulaGraph Exchange is an Apache Spark application to parse data from different sources to NebulaGraph in a distributed environment. It supports both batch and streaming data in various formats and sources including other Graph Databases, RDBMS, Data warehouses, NoSQL, Message Bus, File systems, etc.
Wasp
⭐
25
WASP is a framework to build complex real time big data applications. It relies on a kind of Kappa/Lambda architecture mainly leveraging Kafka and Spark. If you need to ingest huge amount of heterogeneous data and analyze them through complex pipelines, this is the framework for you.
Estuary
⭐
24
基于Akka实现的数据实时流式同步的应用,支持高可用
Daflow
⭐
24
Apache-Spark based Data Flow(ETL) Framework which supports multiple read, write destinations of different types and also support multiple categories of transformation rules.
Enrich
⭐
20
Snowplow Enrichment jobs and library
Bandar Log
⭐
20
Monitoring tool to measure flow throughput of data sources and processing components that are part of Data Ingestion and ETL pipelines.
Bigdata Project
⭐
20
大数据相关笔记
Pramen
⭐
20
Resilient data pipeline framework running on Apache Spark
Cda Client
⭐
19
Cloud Data Access client
Etl Starter Kit
⭐
18
📁 Extract, Transform, Load (ETL) 👷 refers to a process in database usage and especially in data warehousing. This repository contains a starter kit featuring ETL related work.
Telemetry Streaming
⭐
15
Spark Streaming ETL jobs for Mozilla Telemetry
Greenish
⭐
15
Data monitoring tool, monitors the result, not the run
Spark Etl
⭐
15
Set of ETL utils for Spark
Datalink
⭐
13
简单易用的ETL工具
Data Brewery
⭐
12
Data Brewery is an ETL (Extract-Transform-Load) program that connect to many data sources (cloud services, databases, ...) and manage data warehouse workflow.
Devestates
⭐
10
Database for visualizing and understanding the impacts of declared disasters on real estate prices
Eigenflow
⭐
9
ETL orchestration platform with recoverability and process monitoring features
Yasp
⭐
9
Yet Another SPark Framework
Hbaseetl
⭐
8
Spark HbaseETL Tools. Support bulk
Spark Etl Demo
⭐
7
Demo of an ETL Spark Job
Meetup Spark Airflow Demo
⭐
7
Spark & Airflow demo for meetup
Parcsv
⭐
7
📂📰 Scala library for manipulating CSV dataframes.
Platform Etl Backend
⭐
7
Etl Processes Using Sqoop Hadoop Hive Spark And Scala
⭐
7
I implemented various ETL processes like loading the data using sqoop from mysql to hdfs, transform the data using Spark and Scala, perform analytics using Spark and Scala and loading the data back to HDFS.
Openmrs Etl
⭐
7
openmrs - mysql - debezium - kafka - spark - scala
Freetle
⭐
7
streaming XML transformations
Mongodb Elasticsearch Spark Etl
⭐
7
Generic template to read MongoDB and migrate to ElasticSearch
Spark Etl Framework
⭐
7
A generic ETL framework with Spark_SQL for transforming data by constructing pipelines with Yaml/Json/Xml
Data Engineer Portfolio
⭐
6
This is a repository to demonstrate my details, skills, projects and to keep track of my progression in Data Analytics and Data Science topics.
Yl Spark Sql
⭐
6
一个Spark SQL方言,增强了批处理、机器学习、模型服务等语义;基于统一的SQL语法,提供了一个ETL、机器学习
Setl Examples
⭐
6
Learn SETL with examples, lessons and exercises
Example
⭐
5
HbaseETL
Omnidatahouse
⭐
5
Utilities for OMNILab data warehouse.
Amadou
⭐
5
Ignite your Spark ETL jobs
Kf Portal Etl
⭐
5
🏭 Extract-Transform-Load Pipeline for producing data for the Kids First Data Resource Portal
Planet7
⭐
5
Scala library for fast ETL and reconciliation.
Related Searches
Scala Sbt (4,178)
Scala Spark (3,279)
Scala Akka (2,120)
Java Scala (1,794)
Scala Play Framework (1,309)
Plugin Scala (1,079)
Scala Kafka (969)
Scala Functional Programming (942)
Scala Scalajs (887)
Python Etl (814)
1-57 of 57 search results
Privacy
|
About
|
Terms
|
Follow Us On Twitter
Copyright 2018-2024 Awesome Open Source. All rights reserved.