The code of SCFusion: Real-time Incremental Scene Reconstruction with Semantic Completion
Alternatives To Scfusion
Project NameStarsDownloadsRepos Using ThisPackages Using ThisMost Recent CommitTotal ReleasesLatest ReleaseOpen IssuesLicenseLanguage
Co Fusion414
2 years ago5otherC++
Co-Fusion: Real-time Segmentation, Tracking and Fusion of Multiple Objects
7 years ago7otherC++
3D reconstruction system to creating detailed scene geometry from range video.
a year ago2bsd-3-clauseC++
Shading-aware Multi-view Stereo
a year ago10mitPython
Neural Point-Based Graphics
8 months ago13otherPython
MoreFusion: Multi-object Reasoning for 6D Pose Estimation from Volumetric Fusion, CVPR 2020
5 years ago6mitPython
Code for ECCV 2018 paper - Holistic 3D Scene Parsing and Reconstruction from a Single RGB Image
4 years ago1otherC++
Robust Point Cloud Based Reconstruction of Large-Scale Outdoor Scenes
5 years ago6bsd-3-clauseC++
real-time 3D reconstruction of non-rigidly deforming scenes using depth data
3 years agomitJulia
A Julia project demonstrating the fast f-k migration algorithm.
4 years agootherPython
InvSFM: Revealing Scenes by Inverting Structure from Motion Reconstructions [CVPR 2019]
Alternatives To Scfusion
Select To Compare

Alternative Project Comparisons



Authors: Shun-Cheng Wu, Keisuke Tateno, Nassir Navab and Federico Tombari

This is an implementation of our SCFusion on InfiniTAM for open source purpose.


apt-get update
apt install python3-pip
pip3 install cmake
git clone
cd eigen-git-mirror; mkdir build;cd build;cmake ..; make install

sudo apt install libpcl-dev

git clone
cd opencv; mkdir build; cd build;
cmake ..; make; make install
glfw3 (if with GUI)
apt-get install libglfw3 libglfw3-dev
Assimp (if with data generator)
apt-get install libassimp-dev assimp-utils
TooN (if with data generator)
git clone; cd TooN;
./configure && make && make install

Please follow the instruction on NVIDIA offical site. assimp (for data generation).


libTorch : You can download the pre-built version from Pytorch website or build it from source


Using docker

If you have nvidia-docker2:

docker pull gn02077706/ubuntu18.04_cuda_dev:torch


# only if you haven't installed docker yet
curl | sh
sudo systemctl start docker
sudo systemctl enable docker

# set up container toolkits
distribution=$(. /etc/os-release;echo $ID$VERSION_ID) \
 && curl -s -L | sudo apt-key add - \
 && curl -s -L$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list

# install nvidia-docker2
sudo apt-get update
sudo apt-get install -y nvidia-docker2

# restart docker
sudo systemctl restart docker

The docker image has all the dependencies our method needs.


On your system or withint docker:

git clone --recurse-submodules
cd SCFusion; mkdir build; cd build; 

#`-DBUILD_DATA_GENERATOR=True` to build data generator. 
#`-DWITH_PYTORCH=True -DPYTORCH_PATH=/path/to/libtorch/` to enable pytorch support
#`-DWITH_GUI=ON` to build with GUI
cmake -DCMAKE_BUILD_TYPE=Release ..;

Scene reconstruction

Our system takes a configuration file. You can find an example file under Configurations folder. It also takes some command line arguments, you can find them by pass --h or in /SLAMWrapper/include/SLAMTools/Parser.hpp.

The trained model can be downloaded here. Put the downloaded model under Models folder.
Change libSettings->scParams.pth_to_pb in a config file to where this file is stored.

To reconstruct

# useGTPose 0: no, 1: assist, 2: yes
./build/App/SCFusion/exe_scfusion_OFusionRGB1Label ./Configurations/Config_ScanNet_CRF.txt --useGTPose 2 --useSC 1 --pthOut /pth/to/output/folder/
# or with GUI (left button: translation; right button: rotation; scrolling: zoom. )
./build/App/SCFusion/exe_scfusion_gui_OFusionRGB1Label ./Configurations/Config_ScanNet_CRF.txt --useGTPose 2 --useSC 1 --pthOut /pth/to/output/folder/

Get mesh

./exe_Map2Mesh_OFusionRGB1Label --pth_in /pth/to/output/folder/ --pth_out /pth/to/output_mesh/folder/

CompleteScanNet Data Generation

Our dataset is built based on ScanNet, Scan2CAD and ShapeNetCore.v2. You will need to download the dataset from them first in order to build CompleteScanNet.

You will need to download the *_vh_clean_2.labels.ply and *.sens files.

Generate GT

Generated ground truth is available at this link. Please make sure you have agreed all the terms of use from ScanNet, Scan2CAD and ShapeNetCore.v2.

**Flip Clouds**

Our system uses a different coordinate system than ScanNet. The input ply files must be flipped in order to correctly align it to our system.

It's also possible to use the original ScanNet coordinate system, but you will need to change some codes accordingly. We are working on that.

**Update main 7886e87 **
This is no longer needed. At least in the data generation. Haven't fully tested yet.

# Generate aligned poses
cd ../build/App/extractPoseFromScanNet;
./exe_ExtractPoseFromScanNet --pth_in /pth/to/scannet/scans/ --pth_out /pth/to/output/pose_folder/

replace --fill 1 to --fill 0 if you want to generate not filled ground truth.

# Prepare ground truth
./exe_GroundtruthGenerator_ScanNetScan2CAD \
../../Configurations/Config_ScanNet.txt \
 --pth_scan /pth/to/scannet/scans/ \
 --pth_shapenet  /pth/to/shanetcore.v2/ \
 --pth_annotations /pth/to/Scan2CAD/Routines/Script/full_annotations.json \
 --gui 0 \
 --verbose 0 \
 --pthOut /path/to/output/folder/ \
 --fill 1 \
 --labelNum 12

Generate Training data

cd scripts
# Need to change paths


Please check the network repository: SCFusion_Network

  author={Wu, Shun-Cheng and Tateno, Kesuke and Navab, Nassir and Tombari, Federico},
  booktitle={2020 International Conference on 3D Vision (3DV)}, 
  title={SCFusion: Real-time Incremental Scene Reconstruction with Semantic Completion}, 


This work is supported by the German Research Foundation (DFG, project number 407378162) and the Bavarian State Ministry of Education, Science and the Arts in the framework of the Centre Digitisation Bavaria (ZD.B).

Popular Reconstruction Projects
Popular Scene Projects
Popular Machine Learning Categories
Related Searches

Get A Weekly Email With Trending Projects For These Categories
No Spam. Unsubscribe easily at any time.
C Plus Plus