Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

The Top 23 Big Data Etl Open Source Projects

Open source projects categorized as Big Data Etl
Eland ⭐ 588
Python Client and Toolkit for DataFrames, Big Data, Machine Learning and ETL in Elasticsearch
dependent packages 3total releases 30most recent commit 3 months agopypi eland} Downloads
Metorikku ⭐ 536
A simplified, lightweight ETL Framework based on Apache Spark
total releases 126most recent commit a year ago
Bigslice ⭐ 525
A serverless cluster computing system for the Go programming language
total releases 13most recent commit a year ago
Zdh_web ⭐ 379
大数据采集,抽取平台,zdh_web是zdh系列服务的可视化管理平台，包含数据采集,调度,权限,审批
most recent commit 4 months ago
Smooks ⭐ 377
Extensible data integration Java framework for building XML and non-XML fragment-based applications
dependent packages 14total releases 5most recent commit 3 months ago
Big_data_architect_skills ⭐ 353
一个大数据架构师应该掌握的技能
most recent commit 5 years ago
Aws Etl Orchestrator ⭐ 185
A serverless architecture for orchestrating ETL jobs in arbitrarily-complex workflows using AWS Step Functions and AWS Lambda.
most recent commit 4 years ago
Setl ⭐ 173
A simple Spark-powered ETL framework that just works 🍺
total releases 4most recent commit 5 months ago
Graphar ⭐ 145
An open source, standard data file format for graph data storage and retrieval
most recent commit 3 months ago
Eel Sdk ⭐ 140
Big Data Toolkit for the JVM
dependent packages 17total releases 103most recent commit 4 years ago
Hydrograph ⭐ 138
A visual ETL development and debugging tool for big data
most recent commit 4 years ago
Flowman ⭐ 85
Flowman is an ETL framework powered by Apache Spark. With its declarative approach, Flowman simplifies the development of complex data pipelines.
dependent packages 24total releases 65most recent commit 5 months ago
Rocket Bi ⭐ 79
A free, open-source, web-based self-service BI tailor-made for clickhouse, google bigquery, mysql, postgresql, vertica
most recent commit 5 months ago
Spark ⭐ 65
Open Source D-APM (Data-Application Performance Monitoring) for Apache Spark
most recent commit 4 months ago
Udacity Data Engineer Nanodegree ⭐ 64
Classwork projects and home works done through Udacity data engineering nano degree
most recent commit 5 months ago
Datapipelines Essentials Python ⭐ 45
Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validation, Column extensions, SQL functions, and DataFrame transformations
most recent commit a year ago
Sharpetl ⭐ 36
Write ETL using your favorite SQL dialects
most recent commit 4 months ago
Yaetos ⭐ 32
Write data & AI pipelines in (SQL, Spark, Pandas) and deploy to the cloud, simplified
total releases 32most recent commit 4 months ago
Ides ⭐ 32
智能数据探索服务(Intelligent Data Exploration Service)，一站式Data + AI数据解决方案！
most recent commit 10 months ago
Aws Auto Terminate Idle Emr ⭐ 26
AWS Auto Terminate Idle AWS EMR Clusters Framework is an AWS based solution using AWS CloudWatch and AWS Lambda using a Python script that is using Boto3 to terminate AWS EMR clusters that have been idle for a specified period of time.
most recent commit 3 years ago
Cubed ⭐ 24
Data Mart As A Service
most recent commit a year ago
Zephyr ⭐ 21
Zephyr is a big data, platform agnostic ETL API, with Hadoop MapReduce, Storm, and other big data bindings.
most recent commit 8 years ago
Bigdata Project ⭐ 20
大数据相关笔记
most recent commit 3 years ago
The 6 Most Depended On Big Data Etl Open Source Projects
Eel Sdk ⭐ 140
Big Data Toolkit for the JVM
dependent packages 17total releases 103latest release February 11, 2019most recent commit 4 years ago
Smooks ⭐ 377
Extensible data integration Java framework for building XML and non-XML fragment-based applications
dependent packages 14total releases 5latest release June 19, 2023most recent commit 3 months ago
Eland ⭐ 588
Python Client and Toolkit for DataFrames, Big Data, Machine Learning and ETL in Elasticsearch
dependent packages 3total releases 30latest release November 22, 2023most recent commit 3 months agopypi eland} Downloads
Bigquery Kafka Connect ⭐ 17
:cloud: nodejs kafka connect connector for Google BigQuery
dependent packages 3total releases 23latest release February 13, 2019most recent commit 3 years agonpm bigquery-kafka-connect} Downloads
Pramen ⭐ 20
Resilient data pipeline framework running on Apache Spark
dependent packages 2total releases 19latest release November 10, 2023most recent commit 4 months ago
The 10 Latest Releases In Big Data Etl Open Source Projects
Yaetos ⭐ 32
Write data & AI pipelines in (SQL, Spark, Pandas) and deploy to the cloud, simplified
total releases 32latest release November 27, 2023most recent commit 4 months ago
Eland ⭐ 588
Python Client and Toolkit for DataFrames, Big Data, Machine Learning and ETL in Elasticsearch
dependent packages 3total releases 30latest release November 22, 2023most recent commit 3 months agopypi eland} Downloads
Pramen ⭐ 20
Resilient data pipeline framework running on Apache Spark
dependent packages 2total releases 19latest release November 10, 2023most recent commit 4 months ago
Smooks ⭐ 377
Extensible data integration Java framework for building XML and non-XML fragment-based applications
dependent packages 14total releases 5latest release June 19, 2023most recent commit 3 months ago
Metorikku ⭐ 536
A simplified, lightweight ETL Framework based on Apache Spark
total releases 126latest release February 27, 2023most recent commit a year ago
Clickhouse Highlevel Sinker ⭐ 18
clickhouse-highlevel-sinker
total releases 5latest release September 09, 2022most recent commit 2 years ago
Spooq ⭐ 8
total releases 10latest release August 16, 2022most recent commit a year ago
Bigslice ⭐ 525
A serverless cluster computing system for the Go programming language
total releases 13latest release April 05, 2021most recent commit a year ago
Popular Data Processing Categories
Jupyter Notebook
Dataset
Sql
Validation
Pipeline
Translation
Data Science
Classification
Transaction
Scraper
Categories
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2024 Awesome Open Source.  All rights reserved.