Prefect is a workflow orchestration tool empowering developers to build, observe, and react to data pipelines
Alternatives To Prefect
Project NameStarsDownloadsRepos Using ThisPackages Using ThisMost Recent CommitTotal ReleasesLatest ReleaseOpen IssuesLicenseLanguage
Prefect12,91511386 hours ago225August 01, 2023565apache-2.0Python
Prefect is a workflow orchestration tool empowering developers to build, observe, and react to data pipelines
Tpot9,213402023 days ago61January 06, 2021281lgpl-3.0Python
A Python Automated Machine Learning tool that optimizes machine learning pipelines using genetic programming.
Great_expectations8,855356 hours ago236August 04, 2023143apache-2.0Python
Always know what to expect from your data.
Dagster8,548416 hours ago105September 30, 20222,024apache-2.0Python
An orchestration platform for the development, production, and observation of data assets.
Pachyderm5,979113 hours ago504August 04, 2023882apache-2.0Go
Data-Centric Pipelines and Data Versioning
Mage Ai5,572
6 hours ago278August 08, 2023140apache-2.0Python
🧙 The modern replacement for Airflow. Build, run, and manage data pipelines for integrating and transforming data.
4 months ago19December 13, 2022125apache-2.0TypeScript
Build data pipelines, the easy way 🛠️
a month ago20
Open Source Data Science Resources.
Polyaxon3,387412a day ago377August 14, 2023122apache-2.0
MLOps Tools For Managing & Orchestrating The Machine Learning LifeCycle
Pipelines3,29327121 hours ago125July 28, 20231,043apache-2.0Python
Machine Learning Pipelines for Kubeflow
Alternatives To Prefect
Select To Compare

Alternative Project Comparisons



Prefect is an orchestrator for data-intensive workflows. It's the simplest way to transform any Python function into a unit of work that can be observed and orchestrated. With Prefect, you can build resilient, dynamic workflows that react to the world around them and recover from unexpected changes. With just a few decorators, Prefect supercharges your code with features like automatic retries, distributed execution, scheduling, caching, and much more. Every activity is tracked and can be monitored with the Prefect server or Prefect Cloud dashboard.

from prefect import flow, task
from typing import List
import httpx

def get_stars(repo: str):
    url = f"{repo}"
    count = httpx.get(url).json()["stargazers_count"]
    print(f"{repo} has {count} stars!")

@flow(name="GitHub Stars")
def github_stars(repos: List[str]):
    for repo in repos:

# run the flow!

After running some flows, fire up the Prefect UI to see what happened:

prefect server start

Prefect UI dashboard

From here, you can continue to use Prefect interactively or deploy your flows to remote environments, running on a scheduled or event-driven basis.

Getting Started

Prefect requires Python 3.8 or later. To install Prefect, run the following command in a shell or terminal session:

pip install prefect

Start by then exploring the core concepts of Prefect workflows, then follow one of our friendly tutorials to learn by example.

Join the community

Prefect is made possible by the fastest growing community of thousands of friendly data engineers. Join us in building a new kind of workflow system. The Prefect Slack community is a fantastic place to learn more about Prefect, ask questions, or get help with workflow design. The Prefect Discourse is a community-driven knowledge base to find answers to your Prefect-related questions. All community forums, including code contributions, issue discussions, and slack messages are subject to our Code of Conduct.


See our documentation on contributing to Prefect.

Thanks for being part of the mission to build a new kind of workflow system and, of course, happy engineering!

Popular Pipeline Projects
Popular Data Science Projects
Popular Data Processing Categories
Related Searches

Get A Weekly Email With Trending Projects For These Categories
No Spam. Unsubscribe easily at any time.
Data Science
Data Engineering
Workflow Engine