Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About
Search results for python distributed computing
distributed-computing
x
python
x
146 search results found
Catalyst
⭐
3,151
Accelerated deep learning R&D
Alpa
⭐
2,878
Training and serving large-scale neural networks with auto parallelization.
Nakedtensor
⭐
2,471
Bare bone examples of machine learning in TensorFlow
Fugue
⭐
1,821
A unified interface for distributed computing. Fugue executes SQL, Python, Pandas, and Polars code on Spark, Dask and Ray without any rewrites.
Rl
⭐
1,658
A modular, primitive-first, python-first PyTorch library for Reinforcement Learning.
Elephas
⭐
1,548
Distributed Deep learning with Keras & Spark
Distributed
⭐
1,514
A distributed task scheduler for Dask
Pfllib
⭐
1,147
We expose this user-friendly algorithm library (with an integrated evaluation platform) for beginners who intend to start federated learning (FL) study
Vizier
⭐
1,142
Python-based research interface for blackbox and hyperparameter optimization, based on the internal Google Vizier Service.
Sparkit Learn
⭐
1,054
PySpark + Scikit-learn = Sparkit-learn
Daft
⭐
1,012
Distributed DataFrame for Python designed for the cloud, powered by Rust
Federated Learning Pytorch
⭐
902
Implementation of Communication-Efficient Learning of Deep Networks from Decentralized Data
Ai Horde
⭐
892
A crowdsourced distributed cluster for AI art and text generation
Fiber
⭐
864
Distributed Computing for AI Made Simple
Couler
⭐
847
Unified Interface for Constructing and Managing Workflows on different workflow engines, such as Argo Workflows, Tekton Pipelines, and Apache Airflow.
Bagua
⭐
842
Bagua Speeds up PyTorch
Rain
⭐
713
Framework for large distributed pipelines
Openfl
⭐
657
An open framework for Federated Learning.
Atm
⭐
481
Auto Tune Models - A multi-tenant, multi-data system for automated machine learning (model selection and tuning).
Shadowclone
⭐
459
Unleash the power of cloud
Backend.ai
⭐
449
Backend.AI is a streamlined, container-based computing cluster platform that hosts popular computing/ML frameworks and diverse programming languages, with pluggable heterogeneous accelerator support including CUDA GPU, ROCm GPU, TPU, IPU and other NPUs.
Yggdrasil Decision Forests
⭐
403
A library to train, evaluate, interpret, and productionize decision forest models such as Random Forest and Gradient Boosted Decision Trees.
Machinaris
⭐
341
An easy-to-use WebUI for crypto plotting and farming. Offers Bladebit, Gigahorse, MadMax, Chiadog and Plotman in a Docker container. Supports Chia, MMX, Chives, Flax, and HDDCoin among others.
Tdigest
⭐
332
t-Digest data structure in Python. Useful for percentiles and quantiles, including distributed enviroments like PySpark
Python Seminar
⭐
310
Python for Data Science (Seminar Course at UC Berkeley; AY 250)
Sparktorch
⭐
297
Train and run Pytorch models on Apache Spark.
Bluefog
⭐
282
Distributed and decentralized training framework for PyTorch over graph
Charm4py
⭐
280
Parallel Programming with Python and Charm++
Selinon
⭐
277
An advanced distributed task flow management on top of Celery
Swarm Learning
⭐
275
A simplified library for decentralized, privacy preserving machine learning
Torch Quiver
⭐
256
PyTorch Library for Low-Latency, High-Throughput Graph Learning on GPUs.
Dispy
⭐
254
Distributed and Parallel Computing Framework with / for Python
Arkouda
⭐
215
Arkouda (αρκούδα): Interactive Data Analytics at Supercomputing Scale 🐻
Mlcomp
⭐
187
Distributed DAG (Directed acyclic graph) framework for machine learning with UI
Wukong Agent
⭐
152
Web scan foundation framework
Stable Diffusion Webui Distributed
⭐
135
Chains stable-diffusion-webui instances together to facilitate faster image generation.
Qcfractal
⭐
135
A distributed compute and database platform for quantum chemistry.
Dirac
⭐
109
DIRAC Grid
Replay
⭐
109
A Comprehensive Framework for Building End-to-End Recommendation Systems with State-of-the-Art Models
Distex
⭐
102
Distributed process pool for Python
Spark With Python
⭐
98
Fundamentals of Spark with Python (using PySpark), code examples
Autodist
⭐
92
Simple Distributed Deep Learning on TensorFlow
Learning_ray
⭐
84
Code for the upcoming book "Learning Ray" with O'Reilly
Ganga
⭐
84
Ganga is an easy-to-use frontend for job definition and management
Cubed
⭐
80
Bounded-memory serverless distributed N-dimensional array processing
Ensae_teaching_cs
⭐
71
Teaching materials in python at the @ENSAE
Tasq
⭐
70
A simple task queue implementation to enqeue jobs on local or remote processes.
Awesome Asynchronous Federated Learning
⭐
68
📦 Collect some Asynchronous Federated Learning papers.
Playdoh
⭐
68
UNMAINTAINED - USE AT YOUR OWN RISKS
Mltoolkits
⭐
65
learningOrchestra is a distributed Machine Learning integration tool that facilitates and streamlines iterative processes in a Data Science project.
Wrapyfi
⭐
64
Python Wrapper for Message-Oriented and Robotics Middleware
Frovedis
⭐
62
Framework of vectorized and distributed data analytics
Paleo
⭐
58
An analytical performance modeling tool for deep neural networks.
Pyscreener
⭐
56
pythonic interface to virtual screening software
Theano Mpi
⭐
55
MPI Parallel framework for training deep learning models built in Theano
Easydist
⭐
55
Automated Parallelization System and Infrastructure for Multiple Ecosystems
Python Json Socket
⭐
53
JSON messaging based socket interface with multi-threaded server and client
Spacy Ray
⭐
51
☄️ Parallel and distributed training with spaCy and Ray
Integrated Design Diffusion Model
⭐
50
IDDM (Industrial, landscape, animate...), support DDPM, DDIM, webui and multi-GPU distributed training. Pytorch实现,生成模型,扩散模型,分布式训练
Xoscar
⭐
50
Python actor framework for heterogeneous computing.
Pythonframeworks
⭐
49
Another curated list of Python frameworks
Stable Diffusion Webui Stable Horde
⭐
48
Stable Horde client for AUTOMATIC1111's Stable Diffusion Web UI
Ray Core Tutorial
⭐
45
Introduction to Ray Core Design Patterns and APIs.
Lazycluster
⭐
43
🎛 Distributed machine learning made simple.
Pycos
⭐
42
Concurrent, Asynchronous, Distributed, Communicating Tasks with Python
Dislib
⭐
41
The Distributed Computing library for python implemented using PyCOMPSs programming model for HPC.
Compss
⭐
39
COMP Superscalar (COMPSs) is a framework which aims to ease the development and execution of applications for distributed infrastructures, such as Clusters, Grids and Clouds.
Ravenverse
⭐
39
Decentralized Computing Backend for Artificial Intelligence, Web3, Metaverse, and Gaming Application
Aane_python
⭐
35
Accelerated Attributed Network Embedding, SDM 2017
Lstm Tensorspark
⭐
35
Implementation of a LSTM with TensorFlow and distributed on Apache Spark
Proxima
⭐
35
Transcode source media directly from DaVinci Resolve using multiple machines for encoding. Great for creating proxies quickly.
Stairs
⭐
35
Framework which helps you to make parallel/distributed calculations using data pipelines
Pyspark Algorithms
⭐
33
PySpark Algorithms Book: https://www.amazon.com/dp/B07X4B2218/ref=sr_1_2
Dlsa
⭐
33
Distributed least squares approximation (dlsa) implemented with Apache Spark
Disbatch
⭐
31
Tool to distribute a list of computational tasks over a pool of compute resources. The pool can grow or shrink.
Pandata
⭐
31
The Pandata scalable open-source analysis stack
Lipizzaner Gan
⭐
30
[AAAI 2018] "Towards distributed coevolutionary GANs" by Abdullah Al-Dujaili, Tom Schmiedlechner, Erik Hemberg, Una-May O'Reilly
Tune
⭐
30
An abstraction layer for parameter tuning
Spark Ray Data Science
⭐
29
Supporting content (slides and exercises) for the Pearson video series covering best practices for developing scalable applications with Spark and Ray in the context of a data scientist's standard workflow.
Htmap
⭐
29
High-Throughput Computing in Python, powered by HTCondor
Redis Dict
⭐
29
Python Redis Dictionary. Pythonic wrapper that provides name spacing for everyone favorite caching database Redis.
Gordo
⭐
26
An API-first distributed deployment system of deep learning models using timeseries data to predict the behaviour of systems
Xlang
⭐
26
A next-generation dynamic and high-performance language for AI and IOT with natural born distributed computing ability.
Iotpy
⭐
25
Python for streams
Adaptive Scheduler
⭐
24
Run many functions (adaptively) on many cores (>10k) using mpi4py.futures, ipyparallel, loky, or dask-mpi. 🎉
Flip
⭐
24
FLIP: A Provable Defense Framework for Backdoor Mitigation in Federated Learning [ICLR‘23, Best Paper Award at ECCV’22 AROW Workshop]
Grid Control
⭐
23
The grid-control job submission tool
Darima
⭐
23
Distributed ARIMA Models
Easydata
⭐
23
A simple and lightweight database library in python.
Pycondor
⭐
23
Build and submit workflows to HTCondor in Python
Shallowspeed
⭐
21
Small scale distributed training of sequential deep learning models, built on Numpy and MPI.
Dasksge
⭐
21
deploy Dask.distributed to GridEngine (sge/pbs) cluster systems
Kaleidoscope Sdk
⭐
20
A user toolkit for analyzing and interfacing with Large Language Models (LLMs)
Findthemag2
⭐
19
A tool to determine optimal projects for Gridcoin & BOINC crunchers. Maximize your magnitude!
Hydra
⭐
19
Execution framework for multi-task model parallelism. Enables the training of arbitrarily large models with a single GPU, with linear speedups for multi-gpu multi-task execution.
Serpytor
⭐
18
A distributed, low-code, end-to-end data collection and analysis tool for data folks. Take the pain out of data collection from your pipeline!
Kaleidoscope
⭐
17
A user toolkit for analyzing and interfacing with Large Language Models (LLMs)
Federated Learning And Split Learning With Raspberry Pi
⭐
17
SRDS 2020: End-to-End Evaluation of Federated Learning and Split Learning for Internet of Things
Ml_notebooks
⭐
17
Collection of machine learning related notebooks to share.
Optuna Distributed
⭐
17
Distributed hyperparameter optimization made easy
Related Searches
Python Machine Learning (20,195)
Python Docker (14,113)
Python Tensorflow (13,736)
Python Deep Learning (13,092)
Python Jupyter Notebook (12,976)
Python Natural Language Processing (9,064)
Python Artificial Intelligence (8,580)
Python Pytorch (7,877)
Python Convolutional Neural Networks (6,998)
Python Keras (6,821)
1-100 of 146 search results
Next >
Privacy
|
About
|
Terms
|
Follow Us On Twitter
Copyright 2018-2024 Awesome Open Source. All rights reserved.