Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

The Top 23 Database Data Engineering Open Source Projects

Open source projects categorized as Database Data Engineering
Categories > Data Processing > Data Engineering
Categories > Data Storage > Database
Airbyte ⭐ 12,918
The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
dependent packages 11total releases 311most recent commit 3 months agopypi airbyte-cdk} Downloads
Risingwave ⭐ 5,799
The distributed streaming database. Engineered to offer the simplest and most cost-efficient way for stream processing and management.
total releases 14most recent commit 3 months ago
Data Diff ⭐ 2,707
Compare tables within or across databases
dependent packages 1total releases 69most recent commit 3 months agopypi data-diff} Downloads
Data Engineering Wiki ⭐ 934
The best place to learn data engineering. Built and maintained by the data engineering community.
most recent commit 3 months ago
Data Engineering Book ⭐ 744
Accumulated knowledge and experience in the field of Data Engineering
most recent commit a year ago
Neumai ⭐ 693
Neum AI is a best-in-class framework to manage the creation and synchronization of vector embeddings at large scale.
most recent commit 3 months ago
Versatile Data Kit ⭐ 389
One framework to develop, deploy and operate data workflows with Python and SQL.
dependent packages 25total releases 181most recent commit 3 months agopypi vdk-core} Downloads
Data Engineering With Python ⭐ 302
Data Engineering with Python, published by Packt
most recent commit a year ago
Yuniql ⭐ 292
Free and open source schema versioning and database migration made natively with .NET/6. NEW THIS MAY 2022! v1.3.15 released!
dependent packages 7total releases 25most recent commit 2 years agonuget Yuniql.Extensibility} Downloads
Morph Kgc ⭐ 151
Powerful RDF Knowledge Graph Generation with RML Mappings
dependent packages 2total releases 35most recent commit 3 days agopypi morph-kgc} Downloads
Datachecks ⭐ 117
Open Source Data Quality Monitoring.
total releases 14most recent commit 3 months ago
Dataengineeringpilipinas ⭐ 80
Data Engineering Pilipinas is a community for data engineers, data analysts, data scientists, developers, AI / ML engineers, and users of closed and open source data tools and methods / techniques in the Philippines. Data Engineering Pilipinas is a PyData group.
most recent commit 3 months ago
Apachespark ⭐ 59
This repository will help you to learn about databricks concept with the help of examples. It will include all the important topics which we need in our real life experience as a data engineer. We will be using pyspark & sparksql for the development. At the end of the course we also cover few case studies.
most recent commit 2 years ago
Doltpy ⭐ 53
A Python API for Dolt
dependent packages 2total releases 60most recent commit a year agopypi doltpy} Downloads
De 100 Days ⭐ 22
data engineering 100 days 🤖 🧲 🦾 | #DE
most recent commit 7 months ago
Etl_manager ⭐ 21
A python package to create a database on the platform using our moj data warehousing framework
most recent commit 5 months ago
Big Data Engineering Indonesia ⭐ 18
A curated list of big data engineering tools, resources and communities.
most recent commit 4 years ago
Airflowetl ⭐ 16
Blog post on ETL pipelines with Airflow
most recent commit 4 years ago
Data Engineering Salaries ⭐ 16
A Streamlit app to explore data engineering salary data.
most recent commit 4 months ago
Azure Certification Dp 200 ⭐ 16
Road to Azure Data Engineer Part-I: DP-200 - Implementing an Azure Data Solution
most recent commit 4 years ago
Sheetwork ⭐ 14
A handy package to load Google Sheets to your database right from the CLI and with easy configuration via YAML files.
total releases 27most recent commit a year ago
Akka Lift Ml ⭐ 12
akka http service for serving spark machine learning models
most recent commit 7 years ago
Data Brewery ⭐ 12
Data Brewery is an ETL (Extract-Transform-Load) program that connect to many data sources (cloud services, databases, ...) and manage data warehouse workflow.
most recent commit 3 years ago
The 6 Most Depended On Database Data Engineering Open Source Projects
Versatile Data Kit ⭐ 389
One framework to develop, deploy and operate data workflows with Python and SQL.
dependent packages 25total releases 181latest release November 28, 2023most recent commit 3 months agopypi vdk-core} Downloads
Yuniql ⭐ 292
Free and open source schema versioning and database migration made natively with .NET/6. NEW THIS MAY 2022! v1.3.15 released!
dependent packages 7total releases 25latest release May 25, 2022most recent commit 2 years agonuget Yuniql.Extensibility} Downloads
Doltpy ⭐ 53
A Python API for Dolt
dependent packages 2total releases 60latest release January 30, 2023most recent commit a year agopypi doltpy} Downloads
Morph Kgc ⭐ 151
Powerful RDF Knowledge Graph Generation with RML Mappings
dependent packages 2total releases 35latest release November 21, 2023most recent commit 3 days agopypi morph-kgc} Downloads
Data Diff ⭐ 2,707
Compare tables within or across databases
dependent packages 1total releases 69latest release November 30, 2023most recent commit 3 months agopypi data-diff} Downloads
The 9 Latest Releases In Database Data Engineering Open Source Projects
Datachecks ⭐ 117
Open Source Data Quality Monitoring.
total releases 14latest release December 07, 2023most recent commit 3 months ago
Data Diff ⭐ 2,707
Compare tables within or across databases
dependent packages 1total releases 69latest release November 30, 2023most recent commit 3 months agopypi data-diff} Downloads
Versatile Data Kit ⭐ 389
One framework to develop, deploy and operate data workflows with Python and SQL.
dependent packages 25total releases 181latest release November 28, 2023most recent commit 3 months agopypi vdk-core} Downloads
Morph Kgc ⭐ 151
Powerful RDF Knowledge Graph Generation with RML Mappings
dependent packages 2total releases 35latest release November 21, 2023most recent commit 3 days agopypi morph-kgc} Downloads
Doltpy ⭐ 53
A Python API for Dolt
dependent packages 2total releases 60latest release January 30, 2023most recent commit a year agopypi doltpy} Downloads
Yuniql ⭐ 292
Free and open source schema versioning and database migration made natively with .NET/6. NEW THIS MAY 2022! v1.3.15 released!
dependent packages 7total releases 25latest release May 25, 2022most recent commit 2 years agonuget Yuniql.Extensibility} Downloads
Popular Data Processing Categories
Jupyter Notebook
Dataset
Sql
Validation
Pipeline
Translation
Data Science
Classification
Transaction
Scraper
Categories
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2024 Awesome Open Source.  All rights reserved.