LinkNet

Name: jiayan97/linknet-pytorch
Brand: jiayan97/linknet-pytorch
SKU: project/jiayan97/linknet-pytorch
Rating: 4.43 (21 reviews)

Now in experimental release, suggestions welcome.

This is a Pytorch reimplementation of LinkNet for Scene Graph Generation.

Core code is rel_model_linknet.py, built on top of neural-motifs.

Setup

Install Python3.6 & PyTorch3. conda install pytorch=0.3.0 torchvision=0.2.0 cuda90 -c pytorch
Download Visual Genome dataset, see data/stanford_filtered/README.md for details.
Compile everything, run make in the main directory.
Fix PYTHONPATH. export PYTHONPATH=/data/yjy/Workspace/linknet
Click here for more detailed instructions.

Train

Train Object Detection (You can also download the pretrained detector checkpoint here.)

CUDA_VISIBLE_DEVICES=0,1,2 python models/train_detector.py -b 6 -lr 1e-3 -save_dir checkpoints/vgdet -nepoch 50 -ngpu 3 -nwork 3 -p 100 -clip 5

Train Scene Graph Classification

CUDA_VISIBLE_DEVICES=0 python models/train_rels.py -m sgcls -model linknet -b 6 -clip 5 -p 100 -hidden_dim 256 -pooling_dim 4096 -lr 1e-3 -ngpu 1 -ckpt checkpoints/vgdet/vg-24.tar -save_dir checkpoints/linknet-sgcls -nepoch 50 -use_bias

Refine Scene Graph Detection

CUDA_VISIBLE_DEVICES=0 python models/train_rels.py -m sgdet -model linknet -b 6 -clip 5 -p 100 -hidden_dim 256 -pooling_dim 4096 -lr 1e-4 -ngpu 1 -ckpt checkpoints/linknet-sgcls/vgrel-10.tar -save_dir checkpoints/linknet-sgdet -nepoch 10 -use_bias

Test

Evaluate Predicate Classification

CUDA_VISIBLE_DEVICES=0 python models/eval_rels.py -m predcls -model linknet -b 6 -clip 5 -p 100 -hidden_dim 256 -pooling_dim 4096 -lr 1e-3 -ngpu 1 -test -ckpt checkpoints/linknet-sgcls/vgrel-10.tar -nepoch 50 -use_bias -cache linknet_predcls

Evaluate Scene Graph Classification

CUDA_VISIBLE_DEVICES=0 python models/eval_rels.py -m sgcls -model linknet -b 6 -clip 5 -p 100 -hidden_dim 256 -pooling_dim 4096 -lr 1e-3 -ngpu 1 -test -ckpt checkpoints/linknet-sgcls/vgrel-10.tar -nepoch 50 -use_bias -cache linknet_sgcls

Evaluate Scene Graph Detection

CUDA_VISIBLE_DEVICES=0 python models/eval_rels.py -m sgdet -model linknet -b 6 -clip 5 -p 100 -hidden_dim 256 -pooling_dim 4096 -lr 1e-3 -ngpu 1 -test -ckpt checkpoints/linknet-sgdet/vgrel-18.tar -nepoch 50 -use_bias -cache linknet_sgdet

Result

Mode	[email protected]	[email protected]	[email protected]
Predicate Classification	58.8	65.5	67.4
Scene Graph Classification	32.6	35.5	36.1
Scene Graph Detection	13.6	20.5	25.0

TODO

Use Faster RCNN with a ResNet backbone

Contact

For any question, please contact:

Jiayan Yang: [email protected]
Zhiwei Dong: [email protected]

Project Name	Stars	Packages Using This	Most Recent Commit	Total Releases	Latest Release	Open Issues	License	Language
Scabbard	766		6 months ago			21	apache-2.0	Kotlin
🗡 A tool to visualize Dagger 2 dependency graphs
Scene Graph Benchmark.pytorch	760		8 months ago			100	mit	Jupyter Notebook
A new codebase for popular Scene Graph Generation methods (2020). Visualization & Scene Graph Extraction on custom images/datasets are provided. It's also a PyTorch implementation of paper “Unbiased Scene Graph Generation from Biased Training CVPR 2020”
Graph Rcnn.pytorch	482		3 years ago			32		Python
Pytorch code for our ECCV 2018 paper "Graph R-CNN for Scene Graph Generation" and other papers
Gran	363		a year ago			7	mit	C++
Efficient Graph Generation with Graph Recurrent Attention Networks, Deep Generative Model of Graphs, Graph Neural Networks, NeurIPS 2019
Graphwriter	222		3 years ago			13		Python
Code for "Text Generation from Knowledge Graphs with Graph Transformers"
Rl_graph_generation	216		2 years ago			7	bsd-3-clause	Python

Beebug	210		4 years ago				gpl-3.0	Python
A tool for checking exploitability
Jngen	189		4 months ago			4	mit	C++
Library for generating tests for olympiad problems
Ccm	187		4 years ago			6	apache-2.0	Python
This project is a tensorflow implement of our work, CCM.
Amrlib	161	2	2 months ago	13	March 08, 2022	4	mit	Python
A python library that makes AMR parsing, generation and visualization simple.

Linknet Pytorch