Awesome Open Source

Programming Languages

Search results for distributed computing

distributed-computing x

434 search results found

Colossalai ⭐ 37,814

Making large AI models cheaper, faster and more accessible

Machine Learning、Deep Learning、PostgreSQL、Distributed System、Node.Js、Golang

Hazelcast ⭐ 5,738

Hazelcast is a unified real-time data platform combining stream processing with a fast data store, allowing customers to act instantly on data-in-motion for real-time insights.

Protoactor Go ⭐ 4,859

Proto Actor - Ultra fast distributed actors for Go, C# and Java/Kotlin

Gleam ⭐ 3,266

Fast, efficient, and scalable distributed map/reduce system, DAG execution, in memory or on disk, written in pure Go, runs standalone or distributedly.

Catalyst ⭐ 3,151

Accelerated deep learning R&D

Training and serving large-scale neural networks with auto parallelization.

Nakedtensor ⭐ 2,471

Bare bone examples of machine learning in TensorFlow

Distributed Consensus Reading List ⭐ 2,276

A list of papers about distributed consensus.

Graphengine ⭐ 2,176

Microsoft Graph Engine

Fugue ⭐ 1,821

A unified interface for distributed computing. Fugue executes SQL, Python, Pandas, and Polars code on Spark, Dask and Ray without any rewrites.

Boinc ⭐ 1,820

Open-source software for volunteer computing and grid computing.

A modular, primitive-first, python-first PyTorch library for Reinforcement Learning.

Protoactor Dotnet ⭐ 1,655

Proto Actor - Ultra fast distributed actors for Go, C# and Java/Kotlin

Disco ⭐ 1,611

a Map/Reduce framework for distributed computing

Elephas ⭐ 1,548

Distributed Deep learning with Keras & Spark

Distributed ⭐ 1,514

A distributed task scheduler for Dask

Moosefs ⭐ 1,509

MooseFS – Open Source, Petabyte, Fault-Tolerant, Highly Performing, Scalable Network Distributed File System (Software-Defined Storage)

Distributedsystem Notes ⭐ 1,478

📚 深入浅出分布式基础架构，Linux 与操作系统篇 | 分布式系统篇 | 分布式计算篇 | 数据库篇 | 网络篇 | 虚拟化与编排篇 | 大数据与云计算篇

Awesome Federated Machine Learning ⭐ 1,425

Everything about federated learning, including research papers, books, codes, tutorials, videos and beyond

Server ⭐ 1,327

Hashtopolis - A Hashcat wrapper for distributed password recovery

Pfllib ⭐ 1,147

We expose this user-friendly algorithm library (with an integrated evaluation platform) for beginners who intend to start federated learning (FL) study

Vizier ⭐ 1,142

Python-based research interface for blackbox and hyperparameter optimization, based on the internal Google Vizier Service.

Zenoh ⭐ 1,116

zenoh unifies data in motion, data in-use, data at rest and computations. It carefully blends traditional pub/sub with geo-distributed storages, queries and computations, while retaining a level of time and space efficiency that is well beyond any of the mainstream stacks.

Sparkit Learn ⭐ 1,054

PySpark + Scikit-learn = Sparkit-learn

Smartsql ⭐ 1,035

SmartSql = MyBatis in C# + .NET Core+ Cache(Memory | Redis) + R/W Splitting + PropertyChangedTrack +Dynamic Repository + InvokeSync + Diagnostics

Distributed DataFrame for Python designed for the cloud, powered by Rust

Data Algorithms Book ⭐ 973

MapReduce, Spark, Java, and Scala for Data Algorithms Book

A Golang based high performance, scalable and distributed workflow framework

Lizardfs ⭐ 924

LizardFS is an Open Source Distributed File System licensed under GPLv3.

🚀 R package: future: Unified Parallel and Distributed Processing in R for Everyone

Federated Learning Pytorch ⭐ 902

Implementation of Communication-Efficient Learning of Deep Networks from Decentralized Data

Ai Horde ⭐ 892

A crowdsourced distributed cluster for AI art and text generation

Holochain ⭐ 873

The current, performant & industrial strength version of Holochain on Rust.

Distributed Computing for AI Made Simple

Unified Interface for Constructing and Managing Workflows on different workflow engines, such as Argo Workflows, Tekton Pipelines, and Apache Airflow.

Bagua Speeds up PyTorch

Framework for large distributed pipelines

Fishnet ⭐ 692

Distributed Stockfish analysis for lichess.org

An open framework for Federated Learning.

A light-weight library for building distributed applications such as microservices

Transient ⭐ 629

A full stack, reactive architecture for general purpose programming. Algebraic and monadically composable primitives for concurrency, parallelism, event handling, transactions, multithreading, Web, and distributed computing with complete de-inversion of control (No callbacks, no blocking, pure state)

Awesome Parallel Computing ⭐ 585

A curated list of awesome parallel computing resources

Awesome Distributed Systems ⭐ 575

Awesome list of distributed systems resources

Metorikku ⭐ 536

A simplified, lightweight ETL Framework based on Apache Spark

Fast Raft framework using the Redis protocol for Go

Thrill - An EXPERIMENTAL Algorithmic Distributed Big Data Batch Processing Framework in C++

Auto Tune Models - A multi-tenant, multi-data system for automated machine learning (model selection and tuning).

Shadowclone ⭐ 459

Unleash the power of cloud

Easylambda ⭐ 456

distributed dataflows with functional list operations for data processing with C++14

Backend.ai ⭐ 449

Backend.AI is a streamlined, container-based computing cluster platform that hosts popular computing/ML frameworks and diverse programming languages, with pluggable heterogeneous accelerator support including CUDA GPU, ROCm GPU, TPU, IPU and other NPUs.

Awesome Federated Computing ⭐ 448

📚 👓 A collection of research papers, codes, tutorials and blogs on Federated Computing/Learning.

Platon Go ⭐ 444

Golang implementation of the PlatON protocol

Transducers.jl ⭐ 414

Efficient transducers for Julia

Yggdrasil Decision Forests ⭐ 403

A library to train, evaluate, interpret, and productionize decision forest models such as Random Forest and Gradient Boosted Decision Trees.

An experimental platform to quickly realize and compare with popular centralized federated learning algorithms. A realization of federated learning algorithm on fairness (FedFV, Federated Learning with Fair Averaging, https://fanxlxmu.github.io/publication/ijcai2021/) was accepted by IJCAI-21 (https://www.ijcai.org/proceedings/2021/223).

Mpi Operator ⭐ 373

Kubernetes Operator for MPI-based applications (distributed training, HPC, etc.)

Diplomat ⭐ 369

A HTTP Ruby API for Consul

High performance distributed framework for training deep learning recommendation models based on PyTorch.

A Go library for master-less peer-to-peer autodiscovery and RPC between HTTP services

Machinaris ⭐ 341

An easy-to-use WebUI for crypto plotting and farming. Offers Bladebit, Gigahorse, MadMax, Chiadog and Plotman in a Docker container. Supports Chia, MMX, Chives, Flax, and HDDCoin among others.

Amadeus ⭐ 334

Harmonious distributed data analysis in Rust.

Tdigest ⭐ 332

t-Digest data structure in Python. Useful for percentiles and quantiles, including distributed enviroments like PySpark

Paracel ⭐ 326

Distributed training framework with parameter server

Python Seminar ⭐ 310

Python for Data Science (Seminar Course at UC Berkeley; AY 250)

Awesome Distributed Deep Learning ⭐ 309

A curated list of awesome Distributed Deep Learning resources.

Floops.jl ⭐ 300

Fast sequential, threaded, and distributed for-loops for Julia—fold for humans™

Sparktorch ⭐ 297

Train and run Pytorch models on Apache Spark.

Distributed Llama ⭐ 292

Run LLMs on weak devices or make powerful devices even more powerful by distributing the workload and dividing the RAM usage.

Bluefog ⭐ 282

Distributed and decentralized training framework for PyTorch over graph

Charm4py ⭐ 280

Parallel Programming with Python and Charm++

Selinon ⭐ 277

An advanced distributed task flow management on top of Celery

Swarm Learning ⭐ 275

A simplified library for decentralized, privacy preserving machine learning

A Clojure dataframe library that runs on Spark

Good Karma Kit ⭐ 264

😇 A Docker Compose bundle to run on servers with spare CPU, RAM, disk, and bandwidth to help the world. Includes Tor, ArchiveWarrior, BOINC, and more...

Torch Quiver ⭐ 256

PyTorch Library for Low-Latency, High-Throughput Graph Learning on GPUs.

Distributed and Parallel Computing Framework with / for Python

Hyperqueue ⭐ 252

Scheduler for sub-node tasks for HPC systems with batch scheduling

Programming Models and Languages for Distributed Computation

Scalecube Cluster ⭐ 231

ScaleCube Cluster is a lightweight Java VM implementation of SWIM: Scalable Weakly-consistent Infection-style Process Group Membership Protocol. features cluster membership, failure detection, and gossip protocol library.

Fleex makes it easy to create multiple VPS on cloud providers and use them to distribute workloads.

Cuneiform ⭐ 215

Cuneiform distributed programming language

Arkouda ⭐ 215

Arkouda (αρκούδα): Interactive Data Analytics at Supercomputing Scale 🐻

Linearsolve.jl ⭐ 211

LinearSolve.jl: High-Performance Unified Interface for Linear Solvers in Julia. Easily switch between factorization and Krylov methods, add preconditioners, and all in one interface.

Future.apply ⭐ 203

🚀 R package: future.apply - Apply Function to Elements in Parallel using Futures

Gosiris ⭐ 191

An actor framework for Go

Local Cluster ⭐ 187

Easy local cluster creation for Elixir to aid in unit testing

Distributed DAG (Directed acyclic graph) framework for machine learning with UI

Js Spark ⭐ 186

Realtime calculation distributed system. AKA distributed lodash

Hydra Express ⭐ 183

A module which wraps Hydra and ExpressJS into a library for building distributed applications - such as microservices

Orleans.clustering.kubernetes ⭐ 177

Orleans Membership provider for Kubernetes

Ml Systems ⭐ 175

papers on scalable and efficient machine learning systems

A message-passing distributed computing framework for node.js

An intuitive remotely-accessible system performance monitoring and task management tool for servers and headless Raspberry Pi setups.

Artbot For Stable Diffusion ⭐ 153

A front-end GUI for interacting with the AI Horde / Stable Diffusion distributed cluster

State-first Streaming Applications in Rust

Wukong Agent ⭐ 152

Web scan foundation framework

Geminigraph ⭐ 149

A computation-centric distributed graph processing system.

A distributed block-based data storage and compute engine

Openmole ⭐ 141

Workflow engine for exploration of simulation models using high throughput computing

1-100 of 434 search results

Privacy | About | Terms | Follow Us On Twitter

Copyright 2018-2024 Awesome Open Source. All rights reserved.