An Invitation to 3D Vision: A Tutorial for Everyone

An Invitation to 3D Vision is an introductory tutorial on 3D vision (a.k.a. geometric vision or visual geometry or multi-view geometry). It aims to make beginners understand basic theory of 3D vision and implement their own applications using OpenCV. In addition to tutorial slides, example codes are provided in the purpose of education. They include simple but interesting and practical applications. The example codes are written as short as possible (mostly less than 100 lines) to be clear and easy to understand.

What does its name come from?

The main title, An Invitation to 3D Vision, came from a legendary book by Yi Ma, Stefano Soatto, Jana Kosecka, and Shankar S. Sastry. We wish that our tutorial will be the first gentle invitation card for beginners to 3D vision and its applications.
The subtitle, for everyone, was inspired from Prof. Kim's online lecture (in Korean). Our tutorial is also intended not only for students and researchers in academia, but also for hobbyists and developers in industries. We tried to describe important and typical problems and their solutions in OpenCV. We hope readers understand it easily without serious mathematical background.

Examples

Single-view Geometry
- Camera Projection Model
  - Object Localization and Measurement: object_localization.cpp (result: image)
  - Image Formation: image_formation.cpp (result: image0, image1, image2, image3, image4)
  - Geometric Distortion Correction: distortion_correction.cpp (result: video)
- General 2D-3D Geometry
  - Camera Calibration: camera_calibration.cpp (result: text)
  - Camera Pose Estimation (Chessboard): pose_estimation_chessboard.cpp (result: video)
  - Camera Pose Estimation (Book): pose_estimation_book1.cpp
  - Camera Pose Estimation and Calibration: pose_estimation_book2.cpp
  - Camera Pose Estimation and Calibration w/o Initially Given Camera Parameters: pose_estimation_book3.cpp (result: video)
Two-view Geometry
- Planar 2D-2D Geometry (Projective Geometry)
  - Perspective Distortion Correction: perspective_correction.cpp (result: original, rectified)
  - Planar Image Stitching: image_stitching.cpp (result: image)
  - 2D Video Stabilization: video_stabilization.cpp (result: video)
- General 2D-2D Geometry (Epipolar Geometry)
  - Visual Odometry (Monocular, Epipolar Version): vo_epipolar.cpp
  - Triangulation (Two-view Reconstruction): triangulation.cpp
Multi-view Geometry
- Bundle Adjustment
  - Global Version: bundle_adjustment_global.cpp
  - Incremental Version: bundle_adjustment_inc.cpp
- Structure-from-Motion
  - Global SfM: sfm_global.cpp
  - Incremental SfM: sfm_inc.cpp
- Feature-based Visual Odometry and SLAM
  - Visual Odometry (Monocular, Epipolar Version): vo_epipolar.cpp
  - Visual Odometry (Stereo Version)
  - Visual Odometry (Monocular, PnP and BA Version)
  - Visual SLAM (Monocular Version)
- Direct Visual Odometry and SLAM
  - Visual Odometry (Monocular, Direct Version)
- c.f. The above examples need Ceres Solver for bundle adjustment.
Correspondence Problem
- Line Fitting with RANSAC: line_fitting_ransac.cpp
- Line Fitting with M-estimators: line_fitting_m_est.cpp
Appendix
- Line Fitting
- Planar Homograph Estimation
- Fundamental Matrix Estimation

Dependencies

OpenCV (> 3.0.0, 3-clause BSD License)
- OpenCV is a base of all example codes for basic computer vision algorithms, linear algebra, image/video manipulation, and GUI.
Ceres Solver (3-clause BSD License): A numerical optimization library
- Ceres Solver is additionally used by m-estimator, bundle adjustment, structure-from-motion, and visual odometry/SLAM.

License

Beerware

Authors

Sunglok Choi (sunglok AT hanmail DOT net)

Acknowledgement

The authors thank the following contributors and projects.

Jae-Yeong Lee: He motivated many examples.
Giseop Kim: He contributed the initial version of SfM codes with Toy-SfM and cvsba.
The KITTI Vision Benchmark Suite: The KITTI odometry dataset #07 was used to demonstrate visual odometry and SLAM.
Russell Hewett: His two hill images were used to demonstrate image stitching.
Kang Li: His shaking CCTV video was used to demonstrate video stabilization.
Richard Blais: His book cover and video in the OpenCV tutorial were used to demonstrate camera pose estimation and augmented reality.