Awesome Open Source

Programming Languages

Search results for python multimodal deep learning

multimodal-deep-learning x

82 search results found

Pytorch Widedeep ⭐ 1,194

A flexible package for multimodal-deep-learning to combine tabular data with text and images using Wide and Deep models in Pytorch

Deepviewagg ⭐ 195

[CVPR'22 Best Paper Finalist] Official PyTorch implementation of the method presented in "Learning Multi-View Aggregation In the Wild for Large-Scale 3D Semantic Segmentation"

Prophet ⭐ 179

Implementation of CVPR 2023 paper "Prompting Large Language Models with Answer Heuristics for Knowledge-based Visual Question Answering".

This repo contains evaluation code for the paper "MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI"

CapDec: SOTA Zero Shot Image Captioning Using CLIP and GPT2, EMNLP 2022 (findings)

Fusilli ⭐ 120

A Python package housing a collection of deep-learning multi-modal data fusion method pipelines! From data loading, to training, to evaluation - fusilli's got you covered 🌸

The Compiler ⭐ 119

Seed, Code, Harvest: Grow Your Own App with Tree of Thoughts!

Pseudo Q ⭐ 116

[CVPR 2022] Pseudo-Q: Generating Pseudo Language Queries for Visual Grounding

Implementation of "BitNet: Scaling 1-bit Transformers for Large Language Models" in pytorch

Video Captioning ⭐ 102

This repository contains the code for a video captioning system inspired by Sequence to Sequence -- Video to Text. This system takes as input a video and generates a caption in English describing the video.

Implementation of PALI3 from the paper PALI-3 VISION LANGUAGE MODELS: SMALLER, FASTER, STRONGER"

As a part of HAKE project (HAKE-3D). Code for our CVPR2020 paper "Detailed 2D-3D Joint Representation for Human-Object Interaction".

Multimodal Infomax ⭐ 82

This repository contains the official implementation code of the paper Improving Multimodal Fusion with Hierarchical Mutual Information Maximization for Multimodal Sentiment Analysis, accepted at EMNLP 2021.

Swarms Pytorch ⭐ 67

Swarming algorithms like PSO, Ant Colony, Sakana, and more in PyTorch 😊

Awesome Rvos ⭐ 66

Referring Video Object Segmentation / Multi-Object Tracking Repo

Mmsa Fet ⭐ 58

A Tool for extracting multimodal features from videos.

3dcompat V2 ⭐ 57

3DCoMPaT++: An improved large-scale 3D vision dataset for compositional recognition

MIntRec: A New Dataset for Multimodal Intent Recognition (ACM MM 2022)

Embracenet ⭐ 50

Robust multimodal integration method implemented in PyTorch and TensorFlow

Cvt2distilgpt2 ⭐ 46

Improving Chest X-Ray Report Generation by Leveraging Warm-Starting

Democratization of "PaLI: A Jointly-Scaled Multilingual Language-Image Model"

This repository contains the implementation of the paper -- Bi-Bimodal Modality Fusion for Correlation-Controlled Multimodal Sentiment Analysis

CLIPMH:CLIP Multi-modal Hashing

Visual Spatial Reasoning ⭐ 38

[TACL'23] VSR: A probing benchmark for spatial undersranding of vision-language models.

Kosmos2.5 ⭐ 34

My implementation of Kosmos2.5 from the paper: "KOSMOS-2.5: A Multimodal Literate Model"

Self Supervised Embedding Fusion Transformer ⭐ 29

The code for our IEEE ACCESS (2020) paper Multimodal Emotion Recognition with Transformer-Based Self Supervised Feature Fusion.

Learning Cross-Modal Retrieval with Noisy Labels (CVPR 2021, PyTorch Code)

Official code release for ARTEMIS: Attention-based Retrieval with Text-Explicit Matching and Implicit Similarity (published at ICLR 2022)

Attention Feature Fusion base on spatial-temporal Graph Convolutional Network（AFFGCN）

CFCNet for depth completion, NeurIPS 2019.

Code for "Cross-modal Learning for Image-Guided Point Cloud Shape Completion" (NeurIPS 2022)

Utils and modules for Speech Language and Multimodal processing using pytorch and pytorch lightning

Social Iq ⭐ 20

[CVPR 2019 Oral] Social-IQ: A Question Answering Benchmark for Artificial Social Intelligence

In Progress Implementation of GATO style Generalist Multimodal model capable of image, text, RL and Robotics tasks

Code for the InterSpeech 2023 paper: MMER: Multimodal Multi-task learning for Speech Emotion Recognition

Multimodal Future Prediction ⭐ 18

The official repository for the CVPR 2019 paper "Overcoming Limitations of Mixture Density Networks: A Sampling and Fitting Framework for Multimodal Future Prediction"

Msa Robustness ⭐ 17

NAACL 2022 paper on Analyzing Modality Robustness in Multimodal Sentiment Analysis

Multigraphgan ⭐ 17

MultiGraphGAN for predicting multiple target graphs from a source graph using geometric deep learning.

Meme_challenge ⭐ 16

Repository containing code from team Kingsterdam for the Hateful Memes Challenge

Official Code for REVIVE: Regional Visual Representation Matters in Knowledge-Based Visual Question Answering (NeurIPS 2022)

3d Bounding Boxes From Monocular Images ⭐ 15

A two stage multi-modal loss model along with rigid body transformations to regress 3D bounding boxes

Circdeep ⭐ 15

End-to-End learning framework for circular RNA classification from other long non-coding RNA using multimodal deep learning

Entity-Driven Image Search over Multimodal Web Content (EMNLP 2023)

Uniteandconquer ⭐ 14

[CVPR '23] Unite and Conquer: Plug & Play Multi-Modal Synthesis using Diffusion Models

Documentclip ⭐ 14

Attentive Modality Hopping For Ser ⭐ 14

TensorFlow implementation of "Attentive Modality Hopping for Speech Emotion Recognition," ICASSP-20

C3vqg Official ⭐ 14

Code for the paper "C3VQG: Category Consistent Cyclic Visual Question Generation".

Offical implementation of paper "MSAF: Multimodal Split Attention Fusion"

Whos Waldo ⭐ 13

Who's Waldo? Linking People Across Text and Images. ICCV 2021.

[NeurIPS 2023] Official Pytorch code for LOVM: Language-Only Vision Model Selection

Ligand_generation ⭐ 12

Target-aware Variational Auto-encoders for Ligand Generation with Multimodal Protein Representation Learning

Mica Deep Mcca ⭐ 12

Deep Multiset Canonical Correlation Analysis - An extension of CCA to multiple datasets

Move2hear Active Av Separation ⭐ 11

Code and datasets for 'Move2Hear: Active Audio-Visual Source Separation' (ICCV 2021)

[MM 2022 Oral] MM-ALT: A Multimodal Automatic Lyric Transcription System

Pytorch Implementation of FOCAL: Contrastive Learning for Multimodal Time-Series Sensing Signals in Factorized Orthogonal Latent Space

Piano Skills Assessment ⭐ 10

Piano Skills Assessment [IEEE MMSP 2021]

Deep Representations Of Visual Descriptions ⭐ 9

Pytorch implementation of CVPR'16 paper "Learning Deep Representations of Fine-Grained Visual Descriptions", by Reed et al.

Job Recommend Competition ⭐ 9

🥇KNOW기반 직업 추천 알고리즘 경진대회 1등 솔루션입니다🥇

Multimodal Robustness ⭐ 9

Code and resources for EMNLP 2022 paper on 'Robustness of Fusion-based Multimodal Classifiers to Cross-Modal Content Dilutions'

PegasusX: The Future of Multimodal Embeddings 🦄 🦄

This repository contains the code for the publication "Harnessing the Power of Multi-Task Pretraining for Ground-Truth Level Natural Language Explanations"

Package for Multimodal Autoencoders in TensorFlow / Keras

Deepcu Ijcai19 ⭐ 8

DeepCU: Integrating Both Common and Unique Latent Information for Multimodal Sentiment Analysis, IJCAI-19

VTC: Improving Video-Text Retrieval with User Comments

This repo has the PyTorch implementation and datasets of our WSDM 2023 paper: “Multi-queue Momentum Contrast for Microvideo-Product Retrieval”.

[NeurIPS 2023] Bootstrapping Vision-Language Learning with Decoupled Language Pre-training

Segresearchtoolkit ⭐ 8

A High-Efficient Research Development Toolkit for Image Segmentation Based on Pytorch.

[Bioinformatics 2022] Cross-Modality and Self-Supervised Protein Embedding for Compound-Protein Affinity and Contact Prediction

Stacked Attention Networks For Visual Question Answering ⭐ 7

Implementation of the paper "Stacked Attention Networks for Image Question Answering" in Tensorflow

M2h2 Dataset ⭐ 7

This repository contains the dataset and baselines explained in the paper: M2H2: A Multimodal Multiparty Hindi Dataset For HumorRecognition in Conversations

Accepted at ACM Multimedia 2023 in the Open Source track.

Lemanchot Analysis ⭐ 7

LeManchot-Analysis is a system for abnormal detection in coupled visible-thermal images

Image Text Verification ⭐ 7

Official repository for the "VERITE: A Robust Benchmark for Multimodal Misinformation Detection Accounting for Unimodal Bias" paper.

[IROS 2023] GVCCI: Lifelong Learning of Visual Grounding for Language-Guided Robotic Manipulation

Multimodal Dl Framework ⭐ 7

An extensible PyTorch framework to experiment with neural-networks-based deep learning algorithms on multiple data modalities for binary classification.

Transformer-based online speech recognition system with TensorFlow 2

Greedy_multimodal_learning ⭐ 6

Characterizing and overcoming the greedy nature of learning in multi-modal deep neural networks

Formal Multimod Rec ⭐ 6

Formalizing Multimedia Recommendation through Multimodal Deep Learning, under review at TORS.

Protein_pretrain ⭐ 5

Multimodal Pretraining for Unsupervised Protein Representation Learning

MOGONET (Multi-Omics Graph cOnvolutional NETworks) is multi-omics data integrative analysis framework for classification tasks in biomedical applications.

This repository contains the official implementation of the paper: MM-Align: Learning Optimal Transport-based Alignment Dynamics for Fast and Accurate Inference on Missing Modality Sequences (EMNLP 2022)

A Multi-modal Framework for Sentimental Analysis of Meme

Related Searches

Python Machine Learning (20,195)

Python Dataset (14,792)

Python Tensorflow (13,736)

Python Deep Learning (13,092)

Python Network (11,495)

Python Natural Language Processing (9,064)

Python Artificial Intelligence (8,580)

Python Pytorch (7,877)

Python Convolutional Neural Networks (7,365)

Python Paper (6,577)

1-82 of 82 search results

Privacy | About | Terms | Follow Us On Twitter

Copyright 2018-2024 Awesome Open Source. All rights reserved.