Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

The Top 23 Transformer Attention Mechanism Open Source Projects

Open source projects categorized as Transformer Attention Mechanism
Categories > Machine Learning > Attention Mechanism
Vit Pytorch ⭐ 16,298
Implementation of Vision Transformer, a simple way to achieve SOTA in vision classification with only a single transformer encoder, in Pytorch
dependent packages 6total releases 184most recent commit 4 months agopypi vit-pytorch} Downloads
Rwkv Lm ⭐ 10,705
RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding.
most recent commit 3 months ago
Palm Rlhf Pytorch ⭐ 7,496
Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the PaLM architecture. Basically ChatGPT but with PaLM
total releases 69most recent commit 3 months ago
Dalle Pytorch ⭐ 5,477
Implementation / replication of DALL-E, OpenAI's Text to Image Transformer, in Pytorch
total releases 174most recent commit 2 months ago
Awesome Transformer Attention ⭐ 3,895
An ultimately comprehensive paper list of Vision Transformer/Attention, including papers, codes, and related websites
most recent commit 3 months ago
Transformer ⭐ 3,882
A TensorFlow Implementation of the Transformer: Attention Is All You Need
most recent commit a year ago
X Transformers ⭐ 3,840
A simple but complete full-attention transformer with a set of promising experimental features from various papers
dependent packages 10total releases 317most recent commit 3 months agopypi x-transformers} Downloads
Musiclm Pytorch ⭐ 2,686
Implementation of MusicLM, Google's new SOTA model for music generation using attention networks, in Pytorch
total releases 38most recent commit 8 months ago
Audiolm Pytorch ⭐ 2,112
Implementation of AudioLM, a SOTA Language Modeling Approach to Audio Generation out of Google Research, in Pytorch
dependent packages 5total releases 311most recent commit 3 months agopypi audiolm-pytorch} Downloads
Reformer Pytorch ⭐ 1,917
Reformer, the efficient Transformer, in Pytorch
total releases 139most recent commit 10 months ago
Toolformer Pytorch ⭐ 1,788
Implementation of Toolformer, Language Models That Can Use Tools, by MetaAI
total releases 23most recent commit 4 months ago
Hopfield Layers ⭐ 1,258
Hopfield Networks is All You Need
most recent commit 2 years ago
Whisper Timestamped ⭐ 1,217
Multilingual Automatic Speech Recognition with word-level timestamps and confidence
dependent packages 3total releases 3most recent commit 3 months agopypi whisper-timestamped} Downloads
Sockeye ⭐ 1,190
Sequence-to-sequence framework with a focus on Neural Machine Translation based on PyTorch
total releases 85most recent commit 6 months ago
Soundstorm Pytorch ⭐ 1,054
Implementation of SoundStorm, Efficient Parallel Audio Generation from Google Deepmind, in Pytorch
total releases 33most recent commit 5 months ago
Awesome Transformer Nlp ⭐ 1,022
A curated list of NLP resources focused on Transformer networks, attention mechanism, GPT, BERT, ChatGPT, LLMs, and transfer learning.
most recent commit 18 days ago
Coca Pytorch ⭐ 900
Implementation of CoCa, Contrastive Captioners are Image-Text Foundation Models, in Pytorch
dependent packages 1total releases 12most recent commit 4 months agopypi CoCa-pytorch} Downloads
Retro Pytorch ⭐ 784
Implementation of RETRO, Deepmind's Retrieval based Attention net, in Pytorch
total releases 67most recent commit 6 months ago
Performer Pytorch ⭐ 777
An implementation of Performer, a linear attention-based transformer, in Pytorch
dependent packages 5total releases 77most recent commit 2 years agopypi performer-pytorch} Downloads
Muse Maskgit Pytorch ⭐ 739
Implementation of Muse: Text-to-Image Generation via Masked Generative Transformers, in Pytorch
total releases 39most recent commit 5 months ago
Phenaki Pytorch ⭐ 674
Implementation of Phenaki Video, which uses Mask GIT to produce text guided videos of up to 2 minutes in length, in Pytorch
total releases 73most recent commit 5 months ago
Longnet ⭐ 613
Implementation of plug in and play Attention from "LongNet: Scaling Transformers to 1,000,000,000 Tokens"
total releases 37most recent commit 4 months ago
Tab Transformer Pytorch ⭐ 609
Implementation of TabTransformer, attention network for tabular data, in Pytorch
total releases 19most recent commit 5 months ago
The 10 Most Depended On Transformer Attention Mechanism Open Source Projects
X Transformers ⭐ 3,840
A simple but complete full-attention transformer with a set of promising experimental features from various papers
dependent packages 10total releases 317latest release December 02, 2023most recent commit 3 months agopypi x-transformers} Downloads
Performer Pytorch ⭐ 777
An implementation of Performer, a linear attention-based transformer, in Pytorch
dependent packages 5total releases 77latest release February 02, 2022most recent commit 2 years agopypi performer-pytorch} Downloads
Audiolm Pytorch ⭐ 2,112
Implementation of AudioLM, a SOTA Language Modeling Approach to Audio Generation out of Google Research, in Pytorch
dependent packages 5total releases 311latest release December 02, 2023most recent commit 3 months agopypi audiolm-pytorch} Downloads
Eqtransformer ⭐ 260
EQTransformer, a python package for earthquake signal detection and phase picking using AI.
dependent packages 3total releases 46latest release January 30, 2021most recent commit 5 months agopypi EQTransformer} Downloads
Whisper Timestamped ⭐ 1,217
Multilingual Automatic Speech Recognition with word-level timestamps and confidence
dependent packages 3total releases 3latest release December 08, 2023most recent commit 3 months agopypi whisper-timestamped} Downloads
Linformer ⭐ 194
Implementation of Linformer for Pytorch
dependent packages 2total releases 8latest release December 13, 2020most recent commit 4 months agopypi linformer} Downloads
Linear Attention Transformer ⭐ 278
Transformer based on a variant of attention that is linear complexity in respect to sequence length
dependent packages 2total releases 50latest release September 07, 2021most recent commit 3 years agopypi linear-attention-transformer} Downloads
Memory Transformer Xl ⭐ 6
A variant of Transformer-XL where the memory is updated not with a queue, but with attention
dependent packages 2total releases 6latest release July 31, 2020most recent commit 4 years agopypi memory-transformer-xl} Downloads
The 10 Latest Releases In Transformer Attention Mechanism Open Source Projects
Rt 2 ⭐ 215
Democratization of RT-2 "RT-2: New model translates vision and language into action"
total releases 8latest release December 23, 2023most recent commit 3 months ago
Whisper Timestamped ⭐ 1,217
Multilingual Automatic Speech Recognition with word-level timestamps and confidence
dependent packages 3total releases 3latest release December 08, 2023most recent commit 3 months agopypi whisper-timestamped} Downloads
Robotic Transformer Pytorch ⭐ 306
Implementation of RT1 (Robotic Transformer) in Pytorch
total releases 22latest release December 08, 2023most recent commit 5 months ago
Parti Pytorch ⭐ 487
Implementation of Parti, Google's pure attention-based text-to-image neural network, in Pytorch
total releases 21latest release December 08, 2023most recent commit 5 months ago
Audiolm Pytorch ⭐ 2,112
Implementation of AudioLM, a SOTA Language Modeling Approach to Audio Generation out of Google Research, in Pytorch
dependent packages 5total releases 311latest release December 02, 2023most recent commit 3 months agopypi audiolm-pytorch} Downloads
X Transformers ⭐ 3,840
A simple but complete full-attention transformer with a set of promising experimental features from various papers
dependent packages 10total releases 317latest release December 02, 2023most recent commit 3 months agopypi x-transformers} Downloads
Popular Machine Learning Categories
Deep Learning
Machine Learning
Pytorch
Tensorflow
Natural Language Processing
Neural Network
Neural
Computer Vision
Convolutional Neural Networks
Opencv
Categories
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2024 Awesome Open Source.  All rights reserved.