Factorizable Net (Multi-GPU version): An Efficient Subgraph-based Framework for Scene Graph Generation
Alternatives To Factorizablenet
Project NameStarsDownloadsRepos Using ThisPackages Using ThisMost Recent CommitTotal ReleasesLatest ReleaseOpen IssuesLicenseLanguage
Scene Graph Benchmark.pytorch914
2 months ago108mitJupyter Notebook
A new codebase for popular Scene Graph Generation methods (2020). Visualization & Scene Graph Extraction on custom images/datasets are provided. It's also a PyTorch implementation of paper “Unbiased Scene Graph Generation from Biased Training CVPR 2020”
6 months ago22apache-2.0Kotlin
🗡 A tool to visualize Dagger 2 dependency graphs
Graph Rcnn.pytorch482
3 years ago32Python
Pytorch code for our ECCV 2018 paper "Graph R-CNN for Scene Graph Generation" and other papers
a year ago7mitC++
Efficient Graph Generation with Graph Recurrent Attention Networks, Deep Generative Model of Graphs, Graph Neural Networks, NeurIPS 2019
3 years ago13Python
Code for "Text Generation from Knowledge Graphs with Graph Transformers"
3 years ago7bsd-3-clausePython
4 years agogpl-3.0Python
A tool for checking exploitability
10 months ago4mitC++
Library for generating tests for olympiad problems
5 years ago6apache-2.0Python
This project is a tensorflow implement of our work, CCM.
Amrlib16128 months ago13March 08, 20224mitPython
A python library that makes AMR parsing, generation and visualization simple.
Alternatives To Factorizablenet
Select To Compare

Alternative Project Comparisons

Factorizable Net (F-Net)

This is pytorch implementation of our ECCV-2018 paper: Factorizable Net: An Efficient Subgraph-based Framework for Scene Graph Generation. This project is based on our previous work: Multi-level Scene Description Network.


  • [x] Guide for Project Setup
  • [x] Guide for Model Evaluation with pretrained model
  • [x] Guide for Model Training
  • [x] Uploading pretrained model and format-compatible datasets.
  • [x] Update the Model link for VG-DR-Net (We will upload a new model by Aug. 27).
  • [x] Update the Dataset link for VG-DR-Net.
  • [ ] A demonstration of our Factorizable Net
  • [x] Migrate to PyTorch 1.0.1
  • [x] Multi-GPU support (beta version): one image per GPU


  • Feb 26, 2019: Now we release our beta [Multi-GPU] version of Factorizable Net. Find the stable version at branch 0.3.1
  • Aug 28, 2018: Bug fix for running the evaluation with "--use_gt_boxes". VG-DR-Net has some self-relations, e.g. A-relation-A. Previously, we assumed there is no such relation. This commit may influence the model performance on Scene Graph Generation.

Project Settings

  1. Install the requirements (you can use pip or Anaconda):

    conda install pip pyyaml sympy h5py cython numpy scipy click
    conda install -c menpo opencv3
    conda install pytorch torchvision cudatoolkit=8.0 -c pytorch 
    pip install easydict
  2. Clone the Factorizable Net repository

    git clone [email protected]:yikang-li/FactorizableNet.git
  3. Build the Cython modules for nms, roi pooling,roi align modules

    cd lib
    make all
    cd ..
  4. Download the three datasets VG-MSDN, VG-DR-Net, VRD to F-Net/data. And extract the folders with tar xzvf ${Dataset}.tgz. We have converted the original annotations to json version.

  5. Download Visual Genome images and VRD images.

  6. Link the image data folder to target folder: ln -s /path/to/images F-Net/data/${Dataset}/images

    • p.s. You can change the default data directory by modifying dir in options/data_xxx.json.
  7. [optional] Download the pretrained RPN for Visual Genome and VRD. Place them into output/.

  8. [optional] Download the pretrained Factorizable Net on VG-MSDN, VG-DR-Net and VG-VRD, and place them to output/trained_models/

Project Organization

There are several subfolders contained:

  • lib: dataset Loader, NMS, ROI-Pooling, evaluation metrics, etc. are listed in the folder.
  • options: configurations for Data, RPN, F-Net and hyperparameters.
  • models: model definitions for RPN, Factorizable and related modules.
  • data: containing VG-DR-Net (svg/), VG-MSDN (visual_genome/) and VRD (VRD/).
  • output: storing the trained model, checkpoints and loggers.

Evaluation with our Pretrained Models

Pretrained models on VG-MSDN, VG-DR-Net and VG-VRD are provided. --evaluate is provided to enable evaluation mode. Additionally, we also provide --use_gt_boxes to fed the ground-truth object bounding boxes instead of RPN proposals.

  • Evaluation on VG-MSDN with pretrained. Scene Graph Generation results: Recall@50: 12.984%, Recall@100: 16.506%.
CUDA_VISIBLE_DEVICES=0 python train_FN.py --evaluate --dataset_option=normal \
	--path_opt options/models/VG-MSDN.yaml \
	--pretrained_model output/trained_models/Model-VG-MSDN.h5
  • Evaluation on VG-VRD with pretrained. : Scene Graph Generation results: Recall@50: 19.453%, Recall@100: 24.640%.
CUDA_VISIBLE_DEVICES=0 python train_FN.py --evaluate \
	--path_opt options/models/VRD.yaml \
	--pretrained_model output/trained_models/Model-VRD.h5
  • Evaluation on VG-DR-Net with pretrained. Scene Graph Generation results: Recall@50: 19.807%, Recall@100: 25.488%.
CUDA_VISIBLE_DEVICES=0 python train_FN.py --evaluate --dataset_option=normal \
	--path_opt options/models/VG-DR-Net.yaml \
	--pretrained_model output/trained_models/Model-VG-DR-Net.h5


  • Training Region Proposal Network (RPN). The shared conv layers are fixed. We also provide pretrained RPN on Visual Genome and VRD.

     # Train RPN for VG-MSDN and VG-DR-Net
     CUDA_VISIBLE_DEVICES=0 python train_rpn.py --dataset_option=normal 
     # Train RPN for VRD
     CUDA_VISIBLE_DEVICES=0 python train_rpn_VRD.py 
  • Training Factorizable Net: detailed training options are included in options/models/.

     # Train F-Net on VG-MSDN:
     CUDA_VISIBLE_DEVICES=0 python train_FN.py --dataset_option=normal \
     	--path_opt options/models/VG-MSDN.yaml --rpn output/RPN.h5
     # Train F-Net on VRD:
     CUDA_VISIBLE_DEVICES=0 python train_FN.py  \
     	--path_opt options/models/VRD.yaml --rpn output/RPN_VRD.h5
     # Train F-Net on VG-DR-Net:
     CUDA_VISIBLE_DEVICES=0 python train_FN.py --dataset_option=normal \
     	--path_opt options/models/VG-DR-Net.yaml --rpn output/RPN.h5

    --rpn xxx.h5 can be ignored in end-to-end training from pretrained VGG16. Sometime, unexpected and confusing errors appear. Ignore it and restart to training.

  • For better results, we usually re-train the model with additional epochs by resuming the training from the checkpoint with --resume ckpt:

     # Resume F-Net training on VG-MSDN:
     CUDA_VISIBLE_DEVICES=0 python train_FN.py --dataset_option=normal \
     	--path_opt options/models/VG-MSDN.yaml --resume ckpt --epochs 30


We thank longcw for his generous release of the PyTorch Implementation of Faster R-CNN.


If you find our project helpful, your citations are highly appreciated:

author={Li, Yikang and Ouyang, Wanli and Bolei, Zhou and Jianping, Shi and Chao, Zhang and Wang, Xiaogang},
title={Factorizable Net: An Efficient Subgraph-based Framework for Scene Graph Generation},
booktitle = {ECCV},
year = {2018}

We also have two papers regarding to scene graph generation / visual relationship detection:

author={Li, Yikang and Ouyang, Wanli and Zhou, Bolei and Wang, Kun and Wang, Xiaogang},
title={Scene graph generation from objects, phrases and region captions},
booktitle = {ICCV},
year = {2017}

author={Li, Yikang and Ouyang, Wanli and Zhou, Bolei and Wang, Kun and Wang, Xiaogang},
title={ViP-CNN: Visual Phrase Guided Convolutional Neural Network},
booktitle = {CVPR},
year = {2017}


The pre-trained models and the Factorizable Network technique are released for uncommercial use.

Contact Yikang LI if you have questions.

Popular Generation Projects
Popular Graph Projects
Popular Software Development Categories

Get A Weekly Email With Trending Projects For These Categories
No Spam. Unsubscribe easily at any time.