Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

The Top 23 Python Data Science Etl Open Source Projects

Open source projects categorized as Python Data Science Etl
Airflow ⭐ 34,468
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
dependent packages 320total releases 169most recent commit a day agopypi apache-airflow} Downloads
Dagster ⭐ 9,467
An orchestration platform for the development, production, and observation of data assets.
dependent packages 133total releases 585most recent commit 3 months agopypi dagster} Downloads
Mage Ai ⭐ 6,324
🧙 The modern replacement for Airflow. Build, run, and manage data pipelines for integrating and transforming data.
total releases 314most recent commit 3 months ago
Orchest ⭐ 3,876
Build data pipelines, the easy way 🛠️
total releases 19most recent commit a year ago
Aws Sdk Pandas ⭐ 3,779
pandas on AWS - Easy integration with Athena, Glue, Redshift, Timestream, Neptune, OpenSearch, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON and EXCEL).
dependent packages 65total releases 143most recent commit a month agopypi awswrangler} Downloads
Quadratic ⭐ 2,485
Quadratic | Data Science Spreadsheet with Python & SQL
most recent commit 3 months ago
Hamilton ⭐ 1,272
Hamilton helps data scientists and engineers define testable, modular, self-documenting dataflows, that encode lineage and metadata. Runs and scales everywhere python does.
dependent packages 2total releases 116most recent commit a month agopypi sf-hamilton} Downloads
Pyspark Example Project ⭐ 1,034
Example project implementing best practices for PySpark ETL jobs and applications.
most recent commit a year ago
Hamilton ⭐ 877
A scalable general purpose micro-framework for defining dataflows. THIS REPOSITORY HAS BEEN MOVED TO www.github.com/dagworks-inc/hamilton
total releases 10most recent commit 10 months ago
Redun ⭐ 464
Yet another redundant workflow engine
dependent packages 1total releases 18most recent commit 5 months agopypi redun} Downloads
Versatile Data Kit ⭐ 389
One framework to develop, deploy and operate data workflows with Python and SQL.
dependent packages 25total releases 181most recent commit 3 months agopypi vdk-core} Downloads
Astro Sdk ⭐ 303
Astro SDK allows rapid and clean development of {Extract, Load, Transform} workflows using Python and SQL, powered by Apache Airflow.
dependent packages 2total releases 49most recent commit 3 months agopypi astro-sdk-python} Downloads
Butterfree ⭐ 269
A tool for building feature stores.
dependent packages 1total releases 35most recent commit 5 months agopypi butterfree} Downloads
Naas ⭐ 266
Low-code Python library to safely use notebooks in production: schedule workflows, generate assets, trigger webhooks, send notifications, build pipelines, manage secrets (Cloud-only)
most recent commit 3 months ago
Sayn ⭐ 117
Data processing and modelling framework for automating tasks (incl. Python & SQL transformations).
total releases 32most recent commit 3 months ago
Patterns Devkit ⭐ 101
Data pipelines from re-usable components
most recent commit a year ago
Dataengineeringpilipinas ⭐ 80
Data Engineering Pilipinas is a community for data engineers, data analysts, data scientists, developers, AI / ML engineers, and users of closed and open source data tools and methods / techniques in the Philippines. Data Engineering Pilipinas is a PyData group.
most recent commit 3 months ago
Prism ⭐ 70
Prism is the easiest way to develop, orchestrate, and execute data pipelines in Python.
most recent commit 3 months ago
Data Wrangling With Python ⭐ 66
Simplify your ETL processes with these hands-on data sanitation tips, tricks, and best practices
most recent commit 2 years ago
Beneath ⭐ 64
Beneath is a serverless real-time data platform ⚡️
total releases 38most recent commit 2 years ago
Etl_with_python ⭐ 57
ETL with Python - Taught at DWH course 2017 (TAU)
most recent commit 7 years ago
Drivers ⭐ 53
Low-code Python library enabling access to APIs, tools, data sources in seconds.
total releases 403most recent commit 3 months ago
Uptasticsearch ⭐ 48
An Elasticsearch client tailored to data science workflows.
total releases 1most recent commit 3 years ago
The 8 Most Depended On Python Data Science Etl Open Source Projects
Airflow ⭐ 34,468
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
dependent packages 320total releases 169latest release November 27, 2023most recent commit a day agopypi apache-airflow} Downloads
Dagster ⭐ 9,467
An orchestration platform for the development, production, and observation of data assets.
dependent packages 133total releases 585latest release December 07, 2023most recent commit 3 months agopypi dagster} Downloads
Versatile Data Kit ⭐ 389
One framework to develop, deploy and operate data workflows with Python and SQL.
dependent packages 25total releases 181latest release November 28, 2023most recent commit 3 months agopypi vdk-core} Downloads
Butterfree ⭐ 269
A tool for building feature stores.
dependent packages 1total releases 35latest release November 14, 2023most recent commit 5 months agopypi butterfree} Downloads
Redun ⭐ 464
Yet another redundant workflow engine
dependent packages 1total releases 18latest release November 12, 2023most recent commit 5 months agopypi redun} Downloads
The 10 Latest Releases In Python Data Science Etl Open Source Projects
Dagster ⭐ 9,467
An orchestration platform for the development, production, and observation of data assets.
dependent packages 133total releases 585latest release December 07, 2023most recent commit 3 months agopypi dagster} Downloads
Drivers ⭐ 53
Low-code Python library enabling access to APIs, tools, data sources in seconds.
total releases 403latest release November 29, 2023most recent commit 3 months ago
Versatile Data Kit ⭐ 389
One framework to develop, deploy and operate data workflows with Python and SQL.
dependent packages 25total releases 181latest release November 28, 2023most recent commit 3 months agopypi vdk-core} Downloads
Airflow ⭐ 34,468
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
dependent packages 320total releases 169latest release November 27, 2023most recent commit a day agopypi apache-airflow} Downloads
Tablite ⭐ 36
multiprocessing enabled out-of-memory data analysis library for tabular data.
total releases 144latest release November 24, 2023most recent commit 3 months ago
Butterfree ⭐ 269
A tool for building feature stores.
dependent packages 1total releases 35latest release November 14, 2023most recent commit 5 months agopypi butterfree} Downloads
Redun ⭐ 464
Yet another redundant workflow engine
dependent packages 1total releases 18latest release November 12, 2023most recent commit 5 months agopypi redun} Downloads
Popular Data Processing Categories
Jupyter Notebook
Dataset
Sql
Validation
Pipeline
Translation
Data Science
Classification
Transaction
Scraper
Categories
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2024 Awesome Open Source.  All rights reserved.