Project Name | Stars | Downloads | Repos Using This | Packages Using This | Most Recent Commit | Total Releases | Latest Release | Open Issues | License | Language |
---|---|---|---|---|---|---|---|---|---|---|
Prefect | 12,915 | 1 | 138 | 6 hours ago | 225 | August 01, 2023 | 565 | apache-2.0 | Python | |
Prefect is a workflow orchestration tool empowering developers to build, observe, and react to data pipelines | ||||||||||
Tpot | 9,213 | 40 | 20 | 23 days ago | 61 | January 06, 2021 | 281 | lgpl-3.0 | Python | |
A Python Automated Machine Learning tool that optimizes machine learning pipelines using genetic programming. | ||||||||||
Great_expectations | 8,855 | 35 | 6 hours ago | 236 | August 04, 2023 | 143 | apache-2.0 | Python | ||
Always know what to expect from your data. | ||||||||||
Dagster | 8,548 | 41 | 6 hours ago | 105 | September 30, 2022 | 2,024 | apache-2.0 | Python | ||
An orchestration platform for the development, production, and observation of data assets. | ||||||||||
Pachyderm | 5,979 | 1 | 12 hours ago | 504 | August 04, 2023 | 882 | apache-2.0 | Go | ||
Data-Centric Pipelines and Data Versioning | ||||||||||
Mage Ai | 5,572 | 6 hours ago | 278 | August 08, 2023 | 140 | apache-2.0 | Python | |||
🧙 The modern replacement for Airflow. Build, run, and manage data pipelines for integrating and transforming data. | ||||||||||
Orchest | 3,876 | 4 months ago | 19 | December 13, 2022 | 125 | apache-2.0 | TypeScript | |||
Build data pipelines, the easy way 🛠️ | ||||||||||
Datascienceresources | 3,826 | a month ago | 20 | |||||||
Open Source Data Science Resources. | ||||||||||
Polyaxon | 3,387 | 4 | 12 | a day ago | 377 | August 14, 2023 | 122 | apache-2.0 | ||
MLOps Tools For Managing & Orchestrating The Machine Learning LifeCycle | ||||||||||
Pipelines | 3,293 | 2 | 71 | 20 hours ago | 125 | July 28, 2023 | 1,043 | apache-2.0 | Python | |
Machine Learning Pipelines for Kubeflow |
Notice: we’re no longer actively developing Orchest. We could not find a way to make building a workflow orchestrator commercially viable. Check out Apache Airflow for a robust workflow solution.
No frameworks. No YAML. Just write your data processing code directly in Python, R or Julia.
💡 Watch the full narrated video to learn more about building data pipelines in Orchest.
Note: Orchest is in beta.
When to use Orchest? Read it in the docs.
👉 Get started with our quickstart tutorial or have a look at our video tutorials explaining some of Orchest's core concepts.
Missing a feature? Have a look at our public roadmap to see what the team is working on in the short and medium term. Still missing it? Please let us know by opening an issue!
Get started with an example project:
👉 Check out the full list of example projects.
Want to skip the installation and jump right in? Then try out our managed service: Orchest Cloud.
Join our Slack to chat about Orchest, ask questions, and share tips.
The software in this repository is licensed as follows:
orchest-sdk/
and orchest-cli/
directories of this repository
are licensed under the Apache-2.0
license as defined in orchest-sdk/LICENSE
and
orchest-cli/LICENSE
respectively.AGPL-3.0
license.Contributions are more than welcome! Please see our contributor guides for more details.
Alternatively, you can submit your pipeline to the curated list of Orchest examples that are automatically loaded in every Orchest deployment! 🔥