Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

The Top 23 Python Data Integration Open Source Projects

Open source projects categorized as Python Data Integration
Categories > Data Integration
Airflow ⭐ 34,299
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
dependent packages 320total releases 169most recent commit 11 days agopypi apache-airflow} Downloads
Airbyte ⭐ 12,918
The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
dependent packages 11total releases 311most recent commit 3 months agopypi airbyte-cdk} Downloads
Dagster ⭐ 9,467
An orchestration platform for the development, production, and observation of data assets.
dependent packages 133total releases 585most recent commit 3 months agopypi dagster} Downloads
Mage Ai ⭐ 6,324
🧙 The modern replacement for Airflow. Build, run, and manage data pipelines for integrating and transforming data.
total releases 314most recent commit 3 months ago
Awesome Single Cell ⭐ 2,784
Community-curated list of software packages and data resources for single-cell, including RNA-seq, ATAC-seq, etc.
most recent commit 3 months ago
Mara Pipelines ⭐ 2,053
A lightweight opinionated ETL framework, halfway between plain scripts and Apache Airflow
dependent packages 1total releases 8most recent commit 4 months agopypi mara-pipelines} Downloads
Kuwala ⭐ 610
Kuwala is the no-code data platform for BI analysts and engineers enabling you to build powerful analytics workflows. We are set out to bring state-of-the-art data engineering tools you love, such as Airbyte, dbt, or Great Expectations together in one intuitive interface built with React Flow. In addition we provide third-party data into data science models and products with a focus on geospatial data. Currently, the following data connectors are available worldwide: a) High-resolution demograp
most recent commit 2 years ago
Recap ⭐ 292
Work with your web service, database, and streaming schemas in a single format.
total releases 29most recent commit 3 months ago
Nomenklatura ⭐ 171
Framework and command-line tools for integrating FollowTheMoney data streams from multiple sources
dependent packages 5total releases 122most recent commit 3 months agopypi nomenklatura} Downloads
Metl ⭐ 154
mito ETL tool
most recent commit 3 years ago
Morph Kgc ⭐ 143
Powerful RDF Knowledge Graph Generation with RML Mappings
dependent packages 2total releases 35most recent commit a month agopypi morph-kgc} Downloads
Harmonypy ⭐ 116
🎼 Integrate multiple high-dimensional datasets with fuzzy k-means and locally linear adjustments.
dependent packages 14total releases 8most recent commit a year agopypi harmonypy} Downloads
Megalista ⭐ 109
First Party data integration solution built for marketing teams to enable audience and conversion onboarding into Google Marketing products (Google Ads, Campaign Manager, Google Analytics).
most recent commit 3 months ago
Sdm Rdfizer ⭐ 99
An Efficient RML-Compliant Engine for Knowledge Graph Construction
dependent packages 1total releases 550most recent commit 5 months agopypi rdfizer} Downloads
Prism ⭐ 70
Prism is the easiest way to develop, orchestrate, and execute data pipelines in Python.
most recent commit 3 months ago
Gecko ⭐ 54
Toolbox for including enzyme constraints on a genome-scale model.
dependent packages 1total releases 11most recent commit 3 months agopypi geckopy} Downloads
Drivers ⭐ 53
Low-code Python library enabling access to APIs, tools, data sources in seconds.
total releases 403most recent commit 3 months ago
Cellhint ⭐ 44
A tool for semi-automatic cell type harmonization and integration
most recent commit 3 months ago
Nfcompose ⭐ 30
Build REST APIs/Integrations in minutes instead of hours - NF Compose is a (data) integration platform that allows developers to define REST APIs in seconds instead of hours. Generated REST APIs are backed by postgres and support automatic consumer webhook notifications on data changes out of the box.
most recent commit 3 months ago
Cogstack Nifi ⭐ 29
Building data processing pipelines for documents processing with NLP using Apache NiFi and related services
most recent commit 3 months ago
Mapeathor ⭐ 26
Translator of spreadsheet mappings into R2RML, RML or YARRRML
total releases 84most recent commit a year ago
Pyradigm ⭐ 24
Research data management in biomedical and machine learning applications
dependent packages 4total releases 21most recent commit a year agopypi pyradigm} Downloads
Portal ⭐ 24
Adversarial domain translation networks for integrating large-scale atlas-level single-cell datasets
most recent commit 9 months ago
The 10 Most Depended On Python Data Integration Open Source Projects
Airflow ⭐ 34,299
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
dependent packages 320total releases 169latest release November 27, 2023most recent commit 11 days agopypi apache-airflow} Downloads
Dagster ⭐ 9,467
An orchestration platform for the development, production, and observation of data assets.
dependent packages 133total releases 585latest release December 07, 2023most recent commit 3 months agopypi dagster} Downloads
Bio2bel ⭐ 14
A Python framework for integrating biological databases and structured data sources in Biological Expression Language (BEL)
dependent packages 21total releases 40latest release November 03, 2021most recent commit 2 years agopypi bio2bel} Downloads
Harmonypy ⭐ 116
🎼 Integrate multiple high-dimensional datasets with fuzzy k-means and locally linear adjustments.
dependent packages 14total releases 8latest release November 23, 2022most recent commit a year agopypi harmonypy} Downloads
Nomenklatura ⭐ 171
Framework and command-line tools for integrating FollowTheMoney data streams from multiple sources
dependent packages 5total releases 122latest release December 01, 2023most recent commit 3 months agopypi nomenklatura} Downloads
Python Extractor Utils ⭐ 12
Framework for developing extractors in Python
dependent packages 4total releases 99latest release November 21, 2023most recent commit 3 months agopypi cognite-extractor-utils} Downloads
Pyradigm ⭐ 24
Research data management in biomedical and machine learning applications
dependent packages 4total releases 21latest release July 22, 2020most recent commit a year agopypi pyradigm} Downloads
Morph Kgc ⭐ 143
Powerful RDF Knowledge Graph Generation with RML Mappings
dependent packages 2total releases 35latest release November 21, 2023most recent commit a month agopypi morph-kgc} Downloads
Mbpls ⭐ 10
(Multiblock) Partial Least Squares Regression for Python
dependent packages 2total releases 15latest release October 30, 2018most recent commit 4 years agopypi mbpls} Downloads
The 10 Latest Releases In Python Data Integration Open Source Projects
Dagster ⭐ 9,467
An orchestration platform for the development, production, and observation of data assets.
dependent packages 133total releases 585latest release December 07, 2023most recent commit 3 months agopypi dagster} Downloads
Mara Pipelines ⭐ 2,053
A lightweight opinionated ETL framework, halfway between plain scripts and Apache Airflow
dependent packages 1total releases 8latest release December 06, 2023most recent commit 4 months agopypi mara-pipelines} Downloads
Nomenklatura ⭐ 171
Framework and command-line tools for integrating FollowTheMoney data streams from multiple sources
dependent packages 5total releases 122latest release December 01, 2023most recent commit 3 months agopypi nomenklatura} Downloads
Drivers ⭐ 53
Low-code Python library enabling access to APIs, tools, data sources in seconds.
total releases 403latest release November 29, 2023most recent commit 3 months ago
Sdm Rdfizer ⭐ 99
An Efficient RML-Compliant Engine for Knowledge Graph Construction
dependent packages 1total releases 550latest release November 28, 2023most recent commit 5 months agopypi rdfizer} Downloads
Airflow ⭐ 34,299
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
dependent packages 320total releases 169latest release November 27, 2023most recent commit 11 days agopypi apache-airflow} Downloads
Python Extractor Utils ⭐ 12
Framework for developing extractors in Python
dependent packages 4total releases 99latest release November 21, 2023most recent commit 3 months agopypi cognite-extractor-utils} Downloads
Morph Kgc ⭐ 143
Powerful RDF Knowledge Graph Generation with RML Mappings
dependent packages 2total releases 35latest release November 21, 2023most recent commit a month agopypi morph-kgc} Downloads
Categories
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2024 Awesome Open Source.  All rights reserved.