Project Name	Stars	Repos Using This	Packages Using This	Most Recent Commit	Total Releases	Latest Release	Open Issues	License	Language
Couler	847			6 months ago			33	apache-2.0	Python
Unified Interface for Constructing and Managing Workflows on different workflow engines, such as Argo Workflows, Tekton Pipelines, and Apache Airflow.
Rain	713	1	2	a year ago	8	September 05, 2018	36	mit	Rust
Framework for large distributed pipelines
Sparktorch	297			a year ago	11	December 07, 2019	12	mit	Python
Train and run Pytorch models on Apache Spark.
Prime_server	53			5 months ago			26	other	C++
non-blocking (web)server API for distributed computing and SOA based on zeromq
Sciflow	50	2		2 years ago	11	May 14, 2018		mit	Haskell
Scientific workflow management
Pythonframeworks	49			4 months ago			1	mit	Python
Another curated list of Python frameworks
Stairs	35			4 years ago	18	April 02, 2020		apache-2.0	Python
Framework which helps you to make parallel/distributed calculations using data pipelines
Shallowspeed	21			6 months ago					Python
Small scale distributed training of sequential deep learning models, built on Numpy and MPI.
Serpytor	18			a year ago			5	mit	Python
A distributed, low-code, end-to-end data collection and analysis tool for data folks. Take the pain out of data collection from your pipeline!
Pipeedge	14			6 months ago			3	bsd-3-clause	Python
PipeEdge: Pipeline Parallelism for Large-Scale Model Inference on Heterogeneous Edge Devices

Alternatives To Rain

Select To Compare

Couler ⭐ 847

Unified Interface for Constructing and Managing Workflows on different workflow engines, such as Argo Workflows, Tekton Pipelines, and Apache Airflow.

most recent commit 6 months ago

Rain ⭐ 713

Framework for large distributed pipelines

dependent packages 2total releases 8most recent commit a year ago

Sparktorch ⭐ 297

Train and run Pytorch models on Apache Spark.

total releases 11most recent commit a year ago

Prime_server ⭐ 53

non-blocking (web)server API for distributed computing and SOA based on zeromq

most recent commit 5 months ago

Sciflow ⭐ 50

Scientific workflow management

total releases 11most recent commit 2 years ago

Pythonframeworks ⭐ 49

Another curated list of Python frameworks

most recent commit 4 months ago

Stairs ⭐ 35

Framework which helps you to make parallel/distributed calculations using data pipelines

total releases 18most recent commit 4 years ago

Shallowspeed ⭐ 21

Small scale distributed training of sequential deep learning models, built on Numpy and MPI.

most recent commit 6 months ago

Serpytor ⭐ 18

A distributed, low-code, end-to-end data collection and analysis tool for data folks. Take the pain out of data collection from your pipeline!

most recent commit a year ago

Pipeedge ⭐ 14

PipeEdge: Pipeline Parallelism for Large-Scale Model Inference on Heterogeneous Edge Devices

most recent commit 6 months ago

Suggest An Alternative To rain

Alternative Project Comparisons

Rain vs Pythonframeworks

Popular Pipeline Projects

Nushell ⭐ 28,304

A new type of shell

dependent packages 82total releases 77latest release November 20, 2023most recent commit 3 months ago

Jina ⭐ 19,573

☁️ Build multimodal AI applications with cloud-native stack

dependent packages 16total releases 2,453latest release December 01, 2023most recent commit 3 months ago

Luigi ⭐ 17,046

Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.

dependent packages 76total releases 80latest release October 05, 2023most recent commit 3 months ago

Vector ⭐ 16,326

A high-performance observability data pipeline.

total releases 3latest release March 13, 2021most recent commit 17 days ago

Argo Cd ⭐ 15,229

Declarative Continuous Deployment for Kubernetes

dependent packages 65total releases 359latest release December 01, 2023most recent commit 3 months ago

Popular Distributed Computing Projects

Colossalai ⭐ 37,814

Making large AI models cheaper, faster and more accessible

most recent commit 8 days ago

Qix ⭐ 13,926

Machine Learning、Deep Learning、PostgreSQL、Distributed System、Node.Js、Golang

most recent commit 2 years ago

Hazelcast ⭐ 5,738

Hazelcast is a unified real-time data platform combining stream processing with a fast data store, allowing customers to act instantly on data-in-motion for real-time insights.

dependent packages 450total releases 264latest release November 09, 2023most recent commit 3 months ago

Protoactor Go ⭐ 4,859

Proto Actor - Ultra fast distributed actors for Go, C# and Java/Kotlin

dependent packages 23total releases 13latest release April 14, 2023most recent commit 18 days ago

Gleam ⭐ 3,266

Fast, efficient, and scalable distributed map/reduce system, DAG execution, in memory or on disk, written in pure Go, runs standalone or distributedly.

dependent packages 1total releases 1latest release May 13, 2021most recent commit 5 months ago

Popular Data Processing Categories