Awesome Open Source

Programming Languages

Search results for spark delta lake

23 search results found

Doris ⭐ 11,243

Apache Doris is an easy-to-use, high performance and unified analytics database.

Delta ⭐ 6,656

An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs

Delta Sharing ⭐ 654

An open protocol for secure data sharing

Learningsparkv2 ⭐ 570

This is the github repo for Learning Spark: Lightning-Fast Data Analytics [2nd Edition]

Connectors ⭐ 383

This library allows Scala and Java-based projects (including Apache Flink, Apache Hive, Apache Beam, and PrestoDB) to read from and write to Delta Lake.

Dbldatagen ⭐ 234

Generate relevant synthetic data quickly for your projects. The Databricks Labs synthetic data generator (aka `dbldatagen`) may be used to generate large simulated / synthetic data sets for test, POCs, and other uses in Databricks environments including in Delta Live Tables pipelines

Lakehouse Engine ⭐ 154

The Lakehouse Engine is a configuration driven Spark framework, written in Python, serving as a scalable and distributed engine for several lakehouse algorithms, data flows and utilities for Data Products.

Smart Data Lake ⭐ 87

Smart Automation Tool for building modern Data Lakes and Data Pipelines

Delta Architecture ⭐ 66

Streaming data changes to a Data Lake with Debezium and Delta Lake pipeline

Apachespark ⭐ 59

This repository will help you to learn about databricks concept with the help of examples. It will include all the important topics which we need in our real life experience as a data engineer. We will be using pyspark & sparksql for the development. At the end of the course we also cover few case studies.

Edc Mod1 Exercise Igti ⭐ 42

Exercícios do módulo 1 - Bootcamp EDC - IGTI 2021

Databricks ⭐ 41

Databricks Platform - Architecture, Security, Automation and much more!!

Building Data Lakehouse ⭐ 32

Creation of a data lakehouse and an ELT pipeline to enable the efficient analysis and use of data

Real Time Data Warehouse ⭐ 29

Real-time Data Warehouse with Apache Flink & Apache Kafka & Apache Hudi

Delta Go ⭐ 26

Native Delta Lake Implementation in Go

Spark Structured Streaming Examples ⭐ 25

Spark structured streaming examples with using of version 3.4.0

Spark Movies Etl ⭐ 21

Spark data pipeline that ingests and transforms movie ratings data.

Open source stack lakehouse

Amazon Emr With Delta Lake ⭐ 17

Amazon EMR Notebook to show how to read from and write to Delta tables with Amazon EMR

Workshop Data Lakehouse ⭐ 11

Repositório dedicado a Workshop de Data Lakehouse com Delta Lake

Lighthouse ⭐ 8

Shed light on your data layout in order to monitor the health of your Lakehouse tables and identify when data maintenance operations should be performed.

Cdk Emrserverless With Delta Lake ⭐ 8

This construct builds some elements for you to quickly launch an EMR Serverless application. After submitting the Emr Serverless job, you could also launch an EMR notebook via cluster template to check the outcome from the EMR Serverless application.

Hive helper functions for apache spark users

Net.jgp.books.spark.ch17 ⭐ 7

Spark in Action, 2nd edition - chapter 16 - exporting data, using delta lake

Spark Databricks ⭐ 6

🔥 Master Apache Spark & Databricks! Dive into a world of big data with exclusive insights from Udemy courses, personal notes, and practical guides. Whether you're starting out or scaling new heights in data engineering, this is your ultimate resource hub! 🌟🚀

Waterbear ⭐ 5

Automated provisioning of an industry Lakehouse with enterprise data model

Spark Structured Streaming Kafka ⭐ 5

Spark Structured Streaming + Kafka + Delta pipeline.

Doris Sdk ⭐ 5

SDK for Apache Doris

Genomic Bigdata Spark ⭐ 5

Genomic BigData Warehousing with Apache Spark and LakeHouse Architecture

Related Searches

Scala Spark (3,279)

Python Spark (2,053)

Java Spark (1,591)

Apache Spark (1,207)

Spark Hadoop (1,188)

Jupyter Notebook Spark (1,151)

Spark Kafka (985)

Spark Streaming (817)

Spark Pyspark (812)

Shell Spark (707)

1-23 of 23 search results

Privacy | About | Terms | Follow Us On Twitter

Copyright 2018-2024 Awesome Open Source. All rights reserved.