Awesome Open Source

Programming Languages

Search results for python mixture of experts

mixture-of-experts x

21 search results found

Llama Factory ⭐ 10,715

Easy-to-use LLM fine-tuning framework (LLaMA, BLOOM, Mistral, Baichuan, Qwen, ChatGLM)

Mixtral Offloading ⭐ 1,943

Run Mixtral-8x7B models in Colab or consumer desktops

Hivemind ⭐ 1,716

Decentralized deep learning in PyTorch. Built to train models on thousands of volunteers across the world.

Mixture Of Experts ⭐ 656

PyTorch Re-Implementation of "The Sparsely-Gated Mixture-of-Experts Layer" by Noam Shazeer et al. https://arxiv.org/abs/1701.06538

Tutel MoE: An Optimized Mixture-of-Experts Implementation

Keras Mmoe ⭐ 529

A TensorFlow Keras implementation of "Modeling Task Relationships in Multi-task Learning with Multi-gate Mixture-of-Experts" (KDD 2018)

Llama Moe ⭐ 497

⛷️ LLaMA-MoE: Building Mixture-of-Experts from LLaMA with Continual Pre-training

Mixture Of Experts ⭐ 347

A Pytorch implementation of Sparsely-Gated Mixture of Experts, for massively increasing the parameter count of language models

St Moe Pytorch ⭐ 160

Implementation of ST-Moe, the latest incarnation of MoE after years of research at Brain, in Pytorch

Soft Moe Pytorch ⭐ 153

Implementation of Soft MoE, proposed by Brain's Vision team, in Pytorch

Generalizable Mixture Of Experts ⭐ 75

GMoE could be the next backbone model for many kinds of generalization task.

Mixture Of Attention ⭐ 74

Some personal experiments around routing tokens to different autoregressive attention, akin to mixture-of-experts

Pipegoose ⭐ 58

Large scale 4D parallelism pre-training for 🤗 transformers in Mixture of Experts *(still work in progress)*

Soft Mixture Of Experts ⭐ 30

PyTorch implementation of Soft MoE by Google Brain in "From Sparse to Soft Mixtures of Experts" (https://arxiv.org/pdf/2308.00951.pdf)

Neural Implicit Dict ⭐ 28

[ICML 2022] "Neural Implicit Dictionary via Mixture-of-Expert Training" by Peihao Wang, Zhiwen Fan, Tianlong Chen, Zhangyang Wang

MoEL: Mixture of Empathetic Listeners

Multi-Task Learning package built with tensorflow 2 (Multi-Gate Mixture of Experts, Cross-Stitch, Ucertainty Weighting)

A toolbox for inference of mixture models

Mose Auseg ⭐ 13

The official code repo for the paper "Mixture of Stochastic Experts for Modeling Aleatoric Uncertainty in Segmentation". (ICLR 2023)

Classifier ⭐ 6

Machine learning code, derivatives calculation and optimization algorithms developed during the Machine Learning course at Universidade de Sao Paulo. All codes in Python, NumPy and Matplotlib with example in the end of file.

Pre-training Multi-task Contrastive Learning Models for Scientific Literature Understanding (Findings of EMNLP'23)

Related Searches

Python Machine Learning (20,195)

Python Dataset (14,792)

Python Tensorflow (13,736)

Python Deep Learning (13,092)

Python Testing (9,479)

Python Artificial Intelligence (8,580)

Python Pytorch (7,877)

Python Convolutional Neural Networks (6,862)

Python Keras (6,821)

Python Paper (6,580)

1-21 of 21 search results

Privacy | About | Terms | Follow Us On Twitter

Copyright 2018-2024 Awesome Open Source. All rights reserved.