Awesome Open Source

Programming Languages

Search results for big data etl

23 search results found

Python Client and Toolkit for DataFrames, Big Data, Machine Learning and ETL in Elasticsearch

Metorikku ⭐ 536

A simplified, lightweight ETL Framework based on Apache Spark

Bigslice ⭐ 525

A serverless cluster computing system for the Go programming language

Zdh_web ⭐ 379

大数据采集,抽取平台,zdh_web是zdh系列服务的可视化管理平台，包含数据采集,调度,权限,审批

Extensible data integration Java framework for building XML and non-XML fragment-based applications

Big_data_architect_skills ⭐ 353

一个大数据架构师应该掌握的技能

Aws Etl Orchestrator ⭐ 185

A serverless architecture for orchestrating ETL jobs in arbitrarily-complex workflows using AWS Step Functions and AWS Lambda.

A simple Spark-powered ETL framework that just works 🍺

Graphar ⭐ 145

An open source, standard data file format for graph data storage and retrieval

Eel Sdk ⭐ 140

Big Data Toolkit for the JVM

Hydrograph ⭐ 138

A visual ETL development and debugging tool for big data

Flowman is an ETL framework powered by Apache Spark. With its declarative approach, Flowman simplifies the development of complex data pipelines.

Rocket Bi ⭐ 79

A free, open-source, web-based self-service BI tailor-made for clickhouse, google bigquery, mysql, postgresql, vertica

Open Source D-APM (Data-Application Performance Monitoring) for Apache Spark

Udacity Data Engineer Nanodegree ⭐ 64

Classwork projects and home works done through Udacity data engineering nano degree

Datapipelines Essentials Python ⭐ 45

Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validation, Column extensions, SQL functions, and DataFrame transformations

Sharpetl ⭐ 36

Write ETL using your favorite SQL dialects

Write data & AI pipelines in (SQL, Spark, Pandas) and deploy to the cloud, simplified

智能数据探索服务(Intelligent Data Exploration Service)，一站式Data + AI数据解决方案！

Aws Auto Terminate Idle Emr ⭐ 26

AWS Auto Terminate Idle AWS EMR Clusters Framework is an AWS based solution using AWS CloudWatch and AWS Lambda using a Python script that is using Boto3 to terminate AWS EMR clusters that have been idle for a specified period of time.

Data Mart As A Service

Zephyr is a big data, platform agnostic ETL API, with Hadoop MapReduce, Storm, and other big data bindings.

Bigdata Project ⭐ 20

大数据相关笔记

Bandar Log ⭐ 20

Monitoring tool to measure flow throughput of data sources and processing components that are part of Data Ingestion and ETL pipelines.

Resilient data pipeline framework running on Apache Spark

Jun_bigdata ⭐ 18

jun_bigdata大数据平台服务框架。实现了Kafka实时数据过滤、清洗、转换、消费，实现了Sp SQL对Redis、MongoDB等非关系型数据库的数据的读写；集成了规则引擎，可基于规则引擎实现客

Clickhouse Highlevel Sinker ⭐ 18

clickhouse-highlevel-sinker

Etl Starter Kit ⭐ 18

📁 Extract, Transform, Load (ETL) 👷 refers to a process in database usage and especially in data warehousing. This repository contains a starter kit featuring ETL related work.

Bigquery Kafka Connect ⭐ 17

☁️ nodejs kafka connect connector for Google BigQuery

Bigdata Tech Index ⭐ 16

Big Data Technology Index

Hadoop Data Ingestion Tool ⭐ 15

OLAP and ETL of Big Data

Bigdata Etl Pipeline ⭐ 10

The Data Pipeline and Analytics Stack is a comprehensive solution designed for processing, storing, and visualizing data. Explore a complete data pipeline with all components seamlessly set up and ready to use

Yet Another SPark Framework

Flinksupport ⭐ 8

Flink应用程序开发支持框架

Dlt With Debug ⭐ 8

A lightweight helper utility which allows developers to do interactive pipeline development by having a unified source code for both DLT run and Non-DLT interactive notebook run.

Data Engineer Portfolio ⭐ 6

This is a repository to demonstrate my details, skills, projects and to keep track of my progression in Data Analytics and Data Science topics.

Spark Databricks ⭐ 6

🔥 Master Apache Spark & Databricks! Dive into a world of big data with exclusive insights from Udemy courses, personal notes, and practical guides. Whether you're starting out or scaling new heights in data engineering, this is your ultimate resource hub! 🌟🚀

Northstar ⭐ 6

北极星数据管理中台

Dshackle Archive ⭐ 5

ETL for Bitcoin and Ethereum data

Related Searches

Python Etl (814)

1-23 of 23 search results

Privacy | About | Terms | Follow Us On Twitter

Copyright 2018-2024 Awesome Open Source. All rights reserved.