Awesome Open Source
Awesome Open Source

360SD-Net

project page | paper | dataset

This is the implementation of our ICRA 2020 paper "360° Stereo Depth Estimation with Learnable Cost Volume" by Ning-Hsu Wang

Overview

How to Use

  • Setup a directory for all experiments. All you have to do in advance may look like this,
# SETUP REPO
>> git clone https://github.com/albert100121/360SD-Net.git
>> cd 360SD-Net
>> mkdir output
>> cd conda_env
>> conda create --name 360SD-Net python=2.7
>> conda activate 360SD-Net
>> conda install --file requirement.txt

# DOWNLOAD MP3D Dataset
>> cd ./data
# reqest download MP3D Dataset
>> unzip MP3D Dataset
# request download SF3D Dataset
>> unzip SF3D Dataset
  • Setup data and directories (opt to you as long as the data is linked correctly). Set the directory structure for data as follows:
# MP3D Dataset
./data/
     |--MP3D/
                 |--train/
                       |--image_up/
                       |--image_down/
                       |--disp_up/
                 |--val/
                       |--image_up/
                       |--image_down/
                       |--disp_up/
                 |--test/
                       |--image_up/
                       |--image_down/
                       |--disp_up/
# SF3D Dataset
./data/
     |--SF3D/
                 |--train/
                       |--image_up/
                       |--image_down/
                       |--disp_up/
                 |--val/
                       |--image_up/
                       |--image_down/
                       |--disp_up/
                 |--test/
                       |--image_up/
                       |--image_down/
                       |--disp_up/
  • Training procedure:
# For MP3D Dataset
>> python main.py --datapath data/MP3D/train/ --datapath_val data/MP3D/val/ --batch 8

# For SF3D Dataset
>> python main.py --datapath data/SF3D/train/ --datapath_val data/SF3D/val/ --batch 8 --SF3D
  • Testing prodedure:
# For MP3D Dataset
>> python testing.py --datapath data/MP3D/test/ --checkpoint checkpoints/MP3D_checkpoint/checkpoint.tar --outfile output/MP3D

# For SF3D Dataset
>> python testing.py --datapath data/SF3D/test/ --checkpoint checkpoints/SF3D_checkpoint/checkpoint.tar --outfile output/SF3D

# For Real World Data
>> python testing.py --datapath data/realworld/ --checkpoint checkpoints/Realworld_checkpoint/checkpoint.tar --real --outfile output/realworld

# For small inference
>> python testing.py --datapath data/inference/MP3D/ --checkpoint checkpoints/MP3D_checkpoint/checkpoint.tar --outfile output/small_inference
  • Disparity to Depth:
>> python utils/disp2de.py --path PATH_TO_DISPARITY

Notes

  • The training process will cost a lot of GPU memory. Please make sure you have a GPU with 32G or larger memory.
  • For testing, 1080Ti (12G) is enough for a 512 x 1024 image.

Synthetic Results

  • Depth / Error Map
* Projected PCL

Real-World Results

  • Camera Setting
* Real World Results

Citation

@article{wang2019360sdnet,
	title={360SD-Net: 360° Stereo Depth Estimation with Learnable Cost Volume},
	author={Ning-Hsu Wang and Bolivar Solarte and Yi-Hsuan Tsai and Wei-Chen Chiu and Min Sun},
	journal={arXiv preprint arXiv:1911.04460},
	year={2019}
}


Alternative Project Comparisons
Related Awesome Lists
Top Programming Languages

Get A Weekly Email With Trending Projects For These Topics
No Spam. Unsubscribe easily at any time.
Python (861,508
Deep Learning (38,525
Pytorch (22,149
Paper (18,773
Computer Vision (9,482
Volume (8,260
Cost (4,206
Deep Neural Networks (3,995
Depth Estimation (375
Stereo Vision (276
Deep Learning Algorithms (128
Stereo Matching (91
360 (90
Stereo (79
360 Photo (57
Icra (23
360degree (13
Stereo Depth Estimation (11
Icra2020 (8
Disparity Predictions (5