Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

The Top 23 Data Science Etl Open Source Projects

Open source projects categorized as Data Science Etl
Categories > Data Processing > Data Science
Airflow ⭐ 34,468
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
dependent packages 320total releases 169most recent commit an hour agopypi apache-airflow} Downloads
Dagster ⭐ 9,467
An orchestration platform for the development, production, and observation of data assets.
dependent packages 133total releases 585most recent commit 3 months agopypi dagster} Downloads
Mage Ai ⭐ 6,324
🧙 The modern replacement for Airflow. Build, run, and manage data pipelines for integrating and transforming data.
total releases 314most recent commit 3 months ago
Orchest ⭐ 3,876
Build data pipelines, the easy way 🛠️
total releases 19most recent commit a year ago
Aws Sdk Pandas ⭐ 3,779
pandas on AWS - Easy integration with Athena, Glue, Redshift, Timestream, Neptune, OpenSearch, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON and EXCEL).
dependent packages 65total releases 143most recent commit a month agopypi awswrangler} Downloads
Quadratic ⭐ 2,485
Quadratic | Data Science Spreadsheet with Python & SQL
most recent commit 3 months ago
Awesome Business Intelligence ⭐ 1,862
Actively curated list of awesome BI tools. PRs welcome!
most recent commit 6 months ago
Hamilton ⭐ 1,272
Hamilton helps data scientists and engineers define testable, modular, self-documenting dataflows, that encode lineage and metadata. Runs and scales everywhere python does.
dependent packages 2total releases 116most recent commit a month agopypi sf-hamilton} Downloads
Pyspark Example Project ⭐ 1,034
Example project implementing best practices for PySpark ETL jobs and applications.
most recent commit a year ago
Hamilton ⭐ 877
A scalable general purpose micro-framework for defining dataflows. THIS REPOSITORY HAS BEEN MOVED TO www.github.com/dagworks-inc/hamilton
total releases 10most recent commit 10 months ago
Zingg ⭐ 828
Scalable identity resolution, entity resolution, data mastering and deduplication using ML
total releases 1most recent commit 3 months ago
Datacleaner ⭐ 557
The premier open source Data Quality solution
dependent packages 10total releases 53most recent commit 5 months ago
Redun ⭐ 464
Yet another redundant workflow engine
dependent packages 1total releases 18most recent commit 5 months agopypi redun} Downloads
Versatile Data Kit ⭐ 389
One framework to develop, deploy and operate data workflows with Python and SQL.
dependent packages 25total releases 181most recent commit 3 months agopypi vdk-core} Downloads
Awesome Bigquery Views ⭐ 322
Useful SQL queries for Blockchain ETL datasets in BigQuery.
most recent commit a year ago
Astro Sdk ⭐ 303
Astro SDK allows rapid and clean development of {Extract, Load, Transform} workflows using Python and SQL, powered by Apache Airflow.
dependent packages 2total releases 49most recent commit 3 months agopypi astro-sdk-python} Downloads
Cql ⭐ 285
Categorical Query Language IDE
most recent commit 24 days ago
Butterfree ⭐ 269
A tool for building feature stores.
dependent packages 1total releases 35most recent commit 5 months agopypi butterfree} Downloads
Naas ⭐ 266
Low-code Python library to safely use notebooks in production: schedule workflows, generate assets, trigger webhooks, send notifications, build pipelines, manage secrets (Cloud-only)
most recent commit 3 months ago
Elastic ⭐ 242
R client for the Elasticsearch HTTP API
dependent packages 3total releases 13most recent commit a year ago
Setl ⭐ 173
A simple Spark-powered ETL framework that just works 🍺
total releases 4most recent commit 5 months ago
Dataplane ⭐ 171
Dataplane is an Airflow inspired unified data platform with additional data mesh and RPA capability to automate, schedule and design data pipelines and workflows. Dataplane is written in Golang with a React front end.
total releases 30most recent commit 4 months ago
Sayn ⭐ 117
Data processing and modelling framework for automating tasks (incl. Python & SQL transformations).
total releases 32most recent commit 3 months ago
The 10 Most Depended On Data Science Etl Open Source Projects
Airflow ⭐ 34,468
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
dependent packages 320total releases 169latest release November 27, 2023most recent commit an hour agopypi apache-airflow} Downloads
Dagster ⭐ 9,467
An orchestration platform for the development, production, and observation of data assets.
dependent packages 133total releases 585latest release December 07, 2023most recent commit 3 months agopypi dagster} Downloads
Versatile Data Kit ⭐ 389
One framework to develop, deploy and operate data workflows with Python and SQL.
dependent packages 25total releases 181latest release November 28, 2023most recent commit 3 months agopypi vdk-core} Downloads
Datacleaner ⭐ 557
The premier open source Data Quality solution
dependent packages 10total releases 53latest release February 09, 2022most recent commit 5 months ago
Elastic ⭐ 242
R client for the Elasticsearch HTTP API
dependent packages 3total releases 13latest release January 11, 2020most recent commit a year ago
Redun ⭐ 464
Yet another redundant workflow engine
dependent packages 1total releases 18latest release November 12, 2023most recent commit 5 months agopypi redun} Downloads
Gallia Core ⭐ 79
A schema-aware Scala library for data transformation
dependent packages 1total releases 6latest release September 19, 2023most recent commit 4 months ago
The 10 Latest Releases In Data Science Etl Open Source Projects
Dagster ⭐ 9,467
An orchestration platform for the development, production, and observation of data assets.
dependent packages 133total releases 585latest release December 07, 2023most recent commit 3 months agopypi dagster} Downloads
Drivers ⭐ 53
Low-code Python library enabling access to APIs, tools, data sources in seconds.
total releases 403latest release November 29, 2023most recent commit 3 months ago
Versatile Data Kit ⭐ 389
One framework to develop, deploy and operate data workflows with Python and SQL.
dependent packages 25total releases 181latest release November 28, 2023most recent commit 3 months agopypi vdk-core} Downloads
Airflow ⭐ 34,468
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
dependent packages 320total releases 169latest release November 27, 2023most recent commit an hour agopypi apache-airflow} Downloads
Tablite ⭐ 36
multiprocessing enabled out-of-memory data analysis library for tabular data.
total releases 144latest release November 24, 2023most recent commit 3 months ago
Butterfree ⭐ 269
A tool for building feature stores.
dependent packages 1total releases 35latest release November 14, 2023most recent commit 5 months agopypi butterfree} Downloads
Redun ⭐ 464
Yet another redundant workflow engine
dependent packages 1total releases 18latest release November 12, 2023most recent commit 5 months agopypi redun} Downloads
Popular Data Processing Categories
Jupyter Notebook
Dataset
Sql
Validation
Pipeline
Translation
Data Science
Classification
Transaction
Scraper
Categories
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2024 Awesome Open Source.  All rights reserved.