Awesome Open Source
Awesome Open Source

Build data pipelines, the easy way 🙌

No frameworks. No YAML. Just write your data processing code directly in Python, R or Julia.

orchest-0.3.0-demo

💡 Watch the full narrated video to learn more about building data pipelines in Orchest.

Note: Orchest is in beta.

Features

  • Visually construct pipelines through our user-friendly UI
  • Code in Notebooks and scripts (quickstart)
  • Run any subset of a pipelines directly or periodically (jobs)
  • Easily define your dependencies to run on any machine (environments)
  • Spin up services whose lifetime spans across the entire pipeline run (services)
  • Version your projects using git (projects)

When to use Orchest? Read it in the docs.

👉 Get started with our quickstart tutorial or have a look at our video tutorials explaining some of Orchest's core concepts.

Examples

Get started with an example project:

👉 Check out the full list of example projects.

Installation

Want to skip the installation and jump right in? Then try out our managed service by clicking:

Open in Orchest

For macOS and Linux we provide an automated convience script to install Orchest on minikube. Run it with:

curl -fsSL https://get.orchest.io > convenience_install.sh
bash convenience_install.sh

👉 For detailed instructions on how to deploy a self-hosted version, check out our installation docs.

License

The software in this repository is licensed as follows:

  • All content residing under the orchest-sdk/ and orchest-cli/ directories of this repository are licensed under the Apache-2.0 license as defined in orchest-sdk/LICENSE and orchest-cli/LICENSE respectively.
  • Content outside of the above mentioned directories is available under the AGPL-3.0 license.

Slack Community

Join our Slack to chat about Orchest, ask questions, and share tips.

Join us on Slack

Contributing

Contributions are more than welcome! Please see our contributor guides for more details.

Alternatively, you can submit your pipeline to the curated list of Orchest examples that are automatically loaded in every Orchest deployment! 🔥

Contributors

Alternatives To Orchest
Select To Compare


Alternative Project Comparisons
Related Awesome Lists
Top Programming Languages
Top Projects

Get A Weekly Email With Trending Projects For These Topics
No Spam. Unsubscribe easily at any time.
Python (806,114
Jupyter Notebook (153,976
Docker (97,273
Deployment (57,658
Machine Learning (37,040
Cloud Computing (29,215
Kubernetes (24,680
Pipeline (15,544
Data Science (10,142
Ide (8,891
Self Hosted (1,193
Data Pipeline (247
Orchest (4