Awesome Open Source

Programming Languages

Search results for deep learning attention mechanism

attention-mechanism x

deep-learning x

249 search results found

Ml Nlp ⭐ 10,874

此项目是机器学习(Machine Learning)、深度学习(Deep Learning)、NLP面试中常考到的知识点和代码实现，也是作为一个算法工程师必会的理论基础知识。

Rwkv Lm ⭐ 10,705

RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding.

Palm Rlhf Pytorch ⭐ 7,496

Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the PaLM architecture. Basically ChatGPT but with PaLM

Dalle Pytorch ⭐ 5,477

Implementation / replication of DALL-E, OpenAI's Text to Image Transformer, in Pytorch

Awesome Transformer Attention ⭐ 3,895

An ultimately comprehensive paper list of Vision Transformer/Attention, including papers, codes, and related websites

X Transformers ⭐ 3,840

A simple but complete full-attention transformer with a set of promising experimental features from various papers

Keras Attention ⭐ 2,794

Keras Attention Layer (Luong and Bahdanau scores).

Musiclm Pytorch ⭐ 2,686

Implementation of MusicLM, Google's new SOTA model for music generation using attention networks, in Pytorch

Audiolm Pytorch ⭐ 2,112

Implementation of AudioLM, a SOTA Language Modeling Approach to Audio Generation out of Google Research, in Pytorch

Pytorch Gat ⭐ 1,815

My implementation of the original GAT paper (Veličković et al.). I've additionally included the playground.py file for visualizing the Cora dataset, GAT embeddings, an attention mechanism, and entropy histograms. I've supported both Cora (transductive) and PPI (inductive) examples!

Toolformer Pytorch ⭐ 1,788

Implementation of Toolformer, Language Models That Can Use Tools, by MetaAI

Make A Video Pytorch ⭐ 1,449

Implementation of Make-A-Video, new SOTA text to video generator from Meta AI, in Pytorch

Whisper Timestamped ⭐ 1,217

Multilingual Automatic Speech Recognition with word-level timestamps and confidence

Sockeye ⭐ 1,190

Sequence-to-sequence framework with a focus on Neural Machine Translation based on PyTorch

Lambda Networks ⭐ 1,110

Implementation of LambdaNetworks, a new approach to image recognition that reaches SOTA with less compute

Alphafold2 ⭐ 1,086

To eventually become an unofficial Pytorch implementation / replication of Alphafold2, as details of the architecture get released

Soundstorm Pytorch ⭐ 1,054

Implementation of SoundStorm, Efficient Parallel Audio Generation from Google Deepmind, in Pytorch

Perceiver Pytorch ⭐ 980

Implementation of Perceiver, General Perception with Iterative Attention, in Pytorch

Coca Pytorch ⭐ 900

Implementation of CoCa, Contrastive Captioners are Image-Text Foundation Models, in Pytorch

Pointer_summarizer ⭐ 859

pytorch implementation of "Get To The Point: Summarization with Pointer-Generator Networks"

Chatbot_cn ⭐ 850

基于金融-司法领域(兼有闲聊性质)的聊天机器人，其中的主要模块有信息抽取、NLU、NLG、知识图谱等

Retro Pytorch ⭐ 784

Implementation of RETRO, Deepmind's Retrieval based Attention net, in Pytorch

Performer Pytorch ⭐ 777

An implementation of Performer, a linear attention-based transformer, in Pytorch

Muse Maskgit Pytorch ⭐ 739

Implementation of Muse: Text-to-Image Generation via Masked Generative Transformers, in Pytorch

Phenaki Pytorch ⭐ 674

Implementation of Phenaki Video, which uses Mask GIT to produce text guided videos of up to 2 minutes in length, in Pytorch

Keras Attention ⭐ 656

Visualizing RNNs using the attention mechanism

Seq2seq Pytorch ⭐ 653

Sequence to Sequence Models with PyTorch

Tab Transformer Pytorch ⭐ 609

Implementation of TabTransformer, attention network for tabular data, in Pytorch

Transformer Tts ⭐ 599

A Pytorch Implementation of "Neural Speech Synthesis with Transformer Network"

A Deep Learning library for EEG Tasks (Signals) Classification, based on TensorFlow.

Memorizing Transformers Pytorch ⭐ 556

Implementation of Memorizing Transformers (ICLR 2022), attention net augmented with indexing and retrieval of memories using approximate nearest neighbors, in Pytorch

Self Attention Cv ⭐ 550

Implementation of various self-attention mechanisms focused on computer vision. Ongoing repository.

Flamingo Pytorch ⭐ 549

Implementation of 🦩 Flamingo, state-of-the-art few-shot visual question answering attention net out of Deepmind, in Pytorch

A PyTorch implementation of "SimGNN: A Neural Network Approach to Fast Graph Similarity Computation" (WSDM 2019).

Bottleneck Transformer Pytorch ⭐ 523

Implementation of Bottleneck Transformer in Pytorch

Deeplearning.ai Natural Language Processing Specialization ⭐ 523

This repository contains my full work and notes on Coursera's NLP Specialization (Natural Language Processing) taught by the instructor Younes Bensouda Mourri and Łukasz Kaiser offered by deeplearning.ai

Nmt Keras ⭐ 514

Neural Machine Translation with Keras

Parti Pytorch ⭐ 487

Implementation of Parti, Google's pure attention-based text-to-image neural network, in Pytorch

Timesformer Pytorch ⭐ 483

Implementation of TimeSformer from Facebook AI, a pure attention-based solution for video classification

Nuwa Pytorch ⭐ 466

Implementation of NÜWA, state of the art attention network for text to video synthesis, in Pytorch

Megabyte Pytorch ⭐ 458

Implementation of MEGABYTE, Predicting Million-byte Sequences with Multiscale Transformers, in Pytorch

Openstl ⭐ 450

OpenSTL: A Comprehensive Benchmark of Spatio-Temporal Predictive Learning

Lamda Rlhf Pytorch ⭐ 444

Open-source pre-training implementation of Google's LaMDA in PyTorch. Adding RLHF similar to ChatGPT.

Palm Pytorch ⭐ 439

Implementation of the specific Transformer architecture from PaLM - Scaling Language Modeling with Pathways

Geotransformer ⭐ 422

[CVPR2022] Geometric Transformer for Fast and Robust Point Cloud Registration

Structured Self Attention ⭐ 412

A Structured Self-attentive Sentence Embedding

Point Transformer Pytorch ⭐ 402

Implementation of the Point Transformer layer, in Pytorch

Meshgpt Pytorch ⭐ 394

Implementation of MeshGPT, SOTA Mesh generation using Attention, in Pytorch

Pytorch Original Transformer ⭐ 376

My implementation of the original transformer model (Vaswani et al.). I've additionally included the playground.py file for visualizing otherwise seemingly hard concepts. Currently included IWSLT pretrained models.

Enformer Pytorch ⭐ 359

Implementation of Enformer, Deepmind's attention network for predicting gene expression, in Pytorch

Triplet Attention ⭐ 347

Official PyTorch Implementation for "Rotate to Attend: Convolutional Triplet Attention Module." [WACV 2021]

Action Recognition Visual Attention ⭐ 338

Action recognition using soft attention based deep recurrent neural networks

Recurrent Memory Transformer Pytorch ⭐ 335

Implementation of Recurrent Memory Transformer, Neurips 2022 paper, in Pytorch

Changeformer ⭐ 335

[IGARSS'22]: A Transformer-Based Siamese Network for Change Detection

Linformer Pytorch ⭐ 323

My take on a practical implementation of Linformer for Pytorch.

Robotic Transformer Pytorch ⭐ 306

Implementation of RT1 (Robotic Transformer) in Pytorch

Keras Gat ⭐ 301

Keras implementation of the graph attention networks (GAT) by Veličković et al. (2017; https://arxiv.org/abs/1710.10903)

Medical Chatgpt ⭐ 300

Implementation of ChatGPT, but tailored towards primary care medicine, with the reward being able to collect patient histories in a thorough and efficient manner and come up with a reasonable differential diagnosis

Flash Pytorch ⭐ 298

Implementation of the Transformer variant proposed in "Transformer Quality in Linear Time"

Attention_is_all_you_need ⭐ 293

Transformer of "Attention Is All You Need" (Vaswani et al. 2017) by Chainer.

Exploring attention weights in transformer-based models with linguistic knowledge.

Slot Attention ⭐ 286

Implementation of Slot Attention from GoogleAI

Linear Attention Transformer ⭐ 278

Transformer based on a variant of attention that is linear complexity in respect to sequence length

Itransformer ⭐ 275

Unofficial implementation of iTransformer - SOTA Time Series Forecasting using Attention networks, out of Tsinghua / Ant group

Local Attention ⭐ 270

An implementation of local windowed attention for language modeling

Memory Efficient Attention Pytorch ⭐ 267

Implementation of a memory efficient multi-head attention as proposed in the paper, "Self-attention Does Not Need O(n²) Memory"

Eqtransformer ⭐ 260

EQTransformer, a python package for earthquake signal detection and phase picking using AI.

Q Transformer ⭐ 253

Implementation of Q-Transformer, Scalable Offline Reinforcement Learning via Autoregressive Q-Functions, out of Google Deepmind

Magvit2 Pytorch ⭐ 244

Implementation of MagViT2 Tokenizer in Pytorch

📃 **Unofficial** PyTorch Implementation of DA-RNN (arXiv:1704.02971)

Attentive Gan Derainnet ⭐ 222

Unofficial tensorflow implemention of "Attentive Generative Adversarial Network for Raindrop Removal from A Single Image (CVPR 2018) " model https://maybeshewill-cv.github.io/attentive-gan-de

Colt5 Attention ⭐ 207

Implementation of the conditionally routed attention in the CoLT5 architecture, in Pytorch

Equiformer Pytorch ⭐ 207

Implementation of the Equiformer, SE3/E3 equivariant attention network that reaches new SOTA, and adopted for use by EquiFold for protein folding

Se3 Transformer Pytorch ⭐ 205

Implementation of SE3-Transformers for Equivariant Self-Attention, in Pytorch. This specific repository is geared towards integration with eventual Alphafold2 replication.

Routing Transformer ⭐ 202

Fully featured implementation of Routing Transformer

Ttslearn ⭐ 197

ttslearn: Library for Pythonで学ぶ音声合成 (Text-to-speech with Python)

Mega Pytorch ⭐ 195

Implementation of Mega, the Single-head Attention with Multi-headed EMA architecture that currently holds SOTA on Long Range Arena

Linformer ⭐ 194

Implementation of Linformer for Pytorch

En Transformer ⭐ 192

Implementation of E(n)-Transformer, which incorporates attention mechanisms into Welling's E(n)-Equivariant Graph Neural Network

Deformable Attention ⭐ 190

Implementation of Deformable Attention in Pytorch from the paper "Vision Transformer with Deformable Attention"

Sparse Structured Attention ⭐ 190

Sparse and structured neural attention mechanisms

Guided Attention Inference Network ⭐ 187

Contains implementation of Guided Attention Inference Network (GAIN) presented in Tell Me Where to Look(CVPR 2018). This repository aims to apply GAIN on fcn8 architecture used for segmentation.

Attention ⭐ 187

This repository will house a visualization that will attempt to convey instant enlightenment of how Attention works to someone not working in artificial intelligence, with 3Blue1Brown as inspiration

Train and visualize Hierarchical Attention Networks

Palm Jax ⭐ 181

Implementation of the specific Transformer architecture from PaLM - Scaling Language Modeling with Pathways - in Jax (Equinox framework)

Sinkhorn Transformer ⭐ 178

Sinkhorn Transformer - Practical implementation of Sparse Sinkhorn Attention

Neat Vision ⭐ 175

Neat (Neural Attention) Vision, is a visualization tool for the attention mechanisms of deep-learning models for Natural Language Processing (NLP) tasks. (framework-agnostic)

Flash Cosine Sim Attention ⭐ 173

Implementation of fused cosine similarity attention in the same style as Flash Attention

Multihead Siamese Nets ⭐ 173

Implementation of Siamese Neural Networks built upon multihead attention mechanism for text semantic similarity task.

Simple Hierarchical Transformer ⭐ 172

Experiments around a simple idea for inducing multiple hierarchical predictive model within a GPT

Datastories Semeval2017 Task4 ⭐ 171

Deep-learning model presented in "DataStories at SemEval-2017 Task 4: Deep LSTM with Attention for Message-level and Topic-based Sentiment Analysis".

Recurrent Interface Network Pytorch ⭐ 170

Implementation of Recurrent Interface Network (RIN), for highly efficient generation of images and video without cascading networks, in Pytorch

Multi Scale Attention ⭐ 166

Code for our paper "Multi-scale Guided Attention for Medical Image Segmentation"

Block Recurrent Transformer Pytorch ⭐ 164

Implementation of Block Recurrent Transformer - Pytorch

Galerkin Transformer ⭐ 152

[NeurIPS 2021] Galerkin Transformer: a linear attention without softmax

Speech2text ⭐ 148

A Deep-Learning-Based Persian Speech Recognition System

Picanet Implementation ⭐ 147

Pytorch Implementation of PiCANet: Learning Pixel-wise Contextual Attention for Saliency Detection

Bs Roformer ⭐ 144

Implementation of Band Split Roformer, SOTA Attention network for music source separation out of ByteDance AI Labs

Implementation of "PaLM-E: An Embodied Multimodal Language Model"

Visualization ⭐ 142

a collection of visualization function

Related Searches

Python Deep Learning (13,092)

Jupyter Notebook Deep Learning (10,328)

Deep Learning Pytorch (6,767)

Deep Learning Tensorflow (5,868)

Deep Learning Neural Network (5,801)

Deep Learning Convolutional Neural Networks (4,142)

Deep Learning Neural (3,734)

Network Deep Learning (3,532)

Deep Learning Computer Vision (3,365)

Deep Learning Keras (3,258)

1-100 of 249 search results

Privacy | About | Terms | Follow Us On Twitter

Copyright 2018-2024 Awesome Open Source. All rights reserved.