Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

The Top 23 Python Pipeline Etl Open Source Projects

Open source projects categorized as Python Pipeline Etl
Airbyte ⭐ 12,918
The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
dependent packages 11total releases 311most recent commit 3 months agopypi airbyte-cdk} Downloads
Dagster ⭐ 9,467
An orchestration platform for the development, production, and observation of data assets.
dependent packages 133total releases 585most recent commit 3 months agopypi dagster} Downloads
Mage Ai ⭐ 6,324
🧙 The modern replacement for Airflow. Build, run, and manage data pipelines for integrating and transforming data.
total releases 314most recent commit 3 months ago
Orchest ⭐ 3,876
Build data pipelines, the easy way 🛠️
total releases 19most recent commit a year ago
Mara Pipelines ⭐ 2,053
A lightweight opinionated ETL framework, halfway between plain scripts and Apache Airflow
dependent packages 1total releases 8most recent commit 4 months agopypi mara-pipelines} Downloads
Vdp ⭐ 1,556
💧 Instill VDP (Versatile Data Pipeline) is an open-source tool to seamlessly integrate AI to process unstructured data in the modern data stack
dependent packages 1total releases 42most recent commit 3 months ago
Neumai ⭐ 693
Neum AI is a best-in-class framework to manage the creation and synchronization of vector embeddings at large scale.
most recent commit 3 months ago
Goodreads_etl_pipeline ⭐ 593
An end-to-end GoodReads Data Pipeline for Building Data Lake, Data Warehouse and Analytics Platform.
most recent commit 4 years ago
Versatile Data Kit ⭐ 389
One framework to develop, deploy and operate data workflows with Python and SQL.
dependent packages 25total releases 181most recent commit 3 months agopypi vdk-core} Downloads
Butterfree ⭐ 269
A tool for building feature stores.
dependent packages 1total releases 35most recent commit 5 months agopypi butterfree} Downloads
Naas ⭐ 266
Low-code Python library to safely use notebooks in production: schedule workflows, generate assets, trigger webhooks, send notifications, build pipelines, manage secrets (Cloud-only)
most recent commit 3 months ago
Airflow_for_beginners ⭐ 166
most recent commit 2 years ago
Metl ⭐ 154
mito ETL tool
most recent commit 3 years ago
Patterns Devkit ⭐ 101
Data pipelines from re-usable components
most recent commit a year ago
Airbyte_serverless ⭐ 83
Airbyte made simple (no UI, no database, no cluster)
most recent commit 4 months ago
Stetl ⭐ 81
Stetl, Streaming ETL, is a lightweight geospatial processing and ETL framework written in Python.
total releases 14most recent commit 4 months ago
Prism ⭐ 70
Prism is the easiest way to develop, orchestrate, and execute data pipelines in Python.
most recent commit 3 months ago
Sqlbucket ⭐ 67
Lightweight library to write, orchestrate and test your SQL ETL. Writing ETL with data integrity in mind.
total releases 72most recent commit a year ago
Drivers ⭐ 53
Low-code Python library enabling access to APIs, tools, data sources in seconds.
total releases 403most recent commit 3 months ago
Blast ⭐ 31
Blast is a data orchestration tool that can run SQL and Python against Google BigQuery and Snowflake. It supports templating with Jinja, data quality tests, query validation, environment management and more.
total releases 26most recent commit a year ago
Datayoga ⭐ 27
streaming data pipeline platform
dependent packages 3total releases 34most recent commit 5 months agonpm @datayoga-io/shared} Downloads
Covalent Slurm Plugin ⭐ 26
Executor plugin interfacing Covalent with Slurm
total releases 4most recent commit 3 months ago
Sample_etl_structure ⭐ 26
most recent commit 8 months ago
The 8 Most Depended On Python Pipeline Etl Open Source Projects
Dagster ⭐ 9,467
An orchestration platform for the development, production, and observation of data assets.
dependent packages 133total releases 585latest release December 07, 2023most recent commit 3 months agopypi dagster} Downloads
Versatile Data Kit ⭐ 389
One framework to develop, deploy and operate data workflows with Python and SQL.
dependent packages 25total releases 181latest release November 28, 2023most recent commit 3 months agopypi vdk-core} Downloads
Datayoga ⭐ 27
streaming data pipeline platform
dependent packages 3total releases 34latest release November 26, 2022most recent commit 5 months agonpm @datayoga-io/shared} Downloads
Diem ⭐ 10
DIEM Data Integration Engine Multipurpose
dependent packages 3total releases 2latest release February 24, 2021most recent commit 4 months agonpm @mydiem/diem-util} Downloads
Mara Pipelines ⭐ 2,053
A lightweight opinionated ETL framework, halfway between plain scripts and Apache Airflow
dependent packages 1total releases 8latest release December 06, 2023most recent commit 4 months agopypi mara-pipelines} Downloads
Butterfree ⭐ 269
A tool for building feature stores.
dependent packages 1total releases 35latest release November 14, 2023most recent commit 5 months agopypi butterfree} Downloads
The 10 Latest Releases In Python Pipeline Etl Open Source Projects
Dagster ⭐ 9,467
An orchestration platform for the development, production, and observation of data assets.
dependent packages 133total releases 585latest release December 07, 2023most recent commit 3 months agopypi dagster} Downloads
Mara Pipelines ⭐ 2,053
A lightweight opinionated ETL framework, halfway between plain scripts and Apache Airflow
dependent packages 1total releases 8latest release December 06, 2023most recent commit 4 months agopypi mara-pipelines} Downloads
Drivers ⭐ 53
Low-code Python library enabling access to APIs, tools, data sources in seconds.
total releases 403latest release November 29, 2023most recent commit 3 months ago
Versatile Data Kit ⭐ 389
One framework to develop, deploy and operate data workflows with Python and SQL.
dependent packages 25total releases 181latest release November 28, 2023most recent commit 3 months agopypi vdk-core} Downloads
Covalent Ecs Plugin ⭐ 7
Executor plugin interfacing Covalent with Amazon ECS Fargate
total releases 35latest release November 21, 2023most recent commit 5 months ago
Butterfree ⭐ 269
A tool for building feature stores.
dependent packages 1total releases 35latest release November 14, 2023most recent commit 5 months agopypi butterfree} Downloads
Covalent Ssh Plugin ⭐ 8
Executor plugin interfacing Covalent with remote backends using SSH
total releases 31latest release October 31, 2023most recent commit 3 months ago
Popular Data Processing Categories
Jupyter Notebook
Dataset
Sql
Validation
Pipeline
Translation
Data Science
Classification
Transaction
Scraper
Categories
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2024 Awesome Open Source.  All rights reserved.