Awesome Open Source

Programming Languages

Search results for mixture of experts

mixture-of-experts x

32 search results found

Deepspeed ⭐ 32,358

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

Llama Factory ⭐ 10,715

Easy-to-use LLM fine-tuning framework (LLaMA, BLOOM, Mistral, Baichuan, Qwen, ChatGLM)

Mixtral Offloading ⭐ 1,943

Run Mixtral-8x7B models in Colab or consumer desktops

Hivemind ⭐ 1,716

Decentralized deep learning in PyTorch. Built to train models on thousands of volunteers across the world.

Mixture Of Experts ⭐ 656

PyTorch Re-Implementation of "The Sparsely-Gated Mixture-of-Experts Layer" by Noam Shazeer et al. https://arxiv.org/abs/1701.06538

Surrogate Modeling Toolbox

Tutel MoE: An Optimized Mixture-of-Experts Implementation

Keras Mmoe ⭐ 529

A TensorFlow Keras implementation of "Modeling Task Relationships in Multi-task Learning with Multi-gate Mixture-of-Experts" (KDD 2018)

Llama Moe ⭐ 497

⛷️ LLaMA-MoE: Building Mixture-of-Experts from LLaMA with Continual Pre-training

Mixture Of Experts ⭐ 347

A Pytorch implementation of Sparsely-Gated Mixture of Experts, for massively increasing the parameter count of language models

Generalizable Mixture Of Experts ⭐ 280

GMoE could be the next backbone model for many kinds of generalization task.

St Moe Pytorch ⭐ 160

Implementation of ST-Moe, the latest incarnation of MoE after years of research at Brain, in Pytorch

Soft Moe Pytorch ⭐ 153

Implementation of Soft MoE, proposed by Brain's Vision team, in Pytorch

Awesome Adaptive Computation ⭐ 79

A curated reading list of research in Adaptive Computation (AC) & Mixture of Experts (MoE).

Mixture Of Attention ⭐ 74

Some personal experiments around routing tokens to different autoregressive attention, akin to mixture-of-experts

Pipegoose ⭐ 58

Large scale 4D parallelism pre-training for 🤗 transformers in Mixture of Experts *(still work in progress)*

Efficient global optimization toolbox in Rust: bayesian optimization, mixture of gaussian processes, sampling methods

Mixture Models ⭐ 38

Hierarchical Mixture of Experts,Mixture Density Neural Network

Soft Mixture Of Experts ⭐ 30

PyTorch implementation of Soft MoE by Google Brain in "From Sparse to Soft Mixtures of Experts" (https://arxiv.org/pdf/2308.00951.pdf)

Neural Implicit Dict ⭐ 28

[ICML 2022] "Neural Implicit Dictionary via Mixture-of-Expert Training" by Peihao Wang, Zhiwen Fan, Tianlong Chen, Zhangyang Wang

MoEL: Mixture of Empathetic Listeners

Multi-Task Learning package built with tensorflow 2 (Multi-Gate Mixture of Experts, Cross-Stitch, Ucertainty Weighting)

Mixtools ⭐ 14

Tools for Analyzing Finite Mixture Models

A toolbox for inference of mixture models

Mose Auseg ⭐ 13

The official code repo for the paper "Mixture of Stochastic Experts for Modeling Aleatoric Uncertainty in Segmentation". (ICLR 2023)

Learning At Home ⭐ 13

«Towards Crowdsourced Training of Large Neural Networks using Decentralized Mixture-of-Experts» (NeurIPS 2020), original PyTorch implementation

Gaussian Parsimonious Clustering Models with Gating and Expert Network Covariates

ecomix is a package to implement model based species level (Species Archetype Models) or site level (Regions of Common Profile) grouping of community data.

Classifier ⭐ 6

Machine learning code, derivatives calculation and optimization algorithms developed during the Machine Learning course at Universidade de Sao Paulo. All codes in Python, NumPy and Matplotlib with example in the end of file.

Modular_deep_learning ⭐ 5

This repository contains scripts for implementing various learning from expert architectures, such as mixture of experts and product of experts, and performing various experiments with these architectures.

Mixtures of Exponential-Distance Models for Clustering Longitudinal Life-Course Sequences with Gating Covariates and Sampling Weights

Pre-training Multi-task Contrastive Learning Models for Scientific Literature Understanding (Findings of EMNLP'23)

Related Searches

Python Mixture Of Experts (23)

Deep Learning Mixture Of Experts (13)

Pytorch Mixture Of Experts (10)

Machine Learning Mixture Of Experts (10)

Artificial Intelligence Mixture Of Experts (7)

Jupyter Notebook Mixture Of Experts (6)

1-32 of 32 search results

Privacy | About | Terms | Follow Us On Twitter

Copyright 2018-2024 Awesome Open Source. All rights reserved.