Winter Project in MSR, Northwestern University (Winter 2018)
A simple demo of 3D reconstruction of a sneaker 'Kyrie4 CNY' with this pipeline. More examples see 'Demo' part.
Demo Video in Youtube: Mini 3D-Scanner
The goal of this project is to develop a pipeline that can do online 3D reconstruction of small scale obejects, only using visual data from RGB-D camera.
While some commercial 3D scanner can provide with more acurate reconstruction results, they have some disadvantages. Some of them are very expensive, some of them have no colors (laser scan), and some of them are limited only to be used in some particular environments with special setup.
In this project, I explored to only use visual data (color images and range images), to robustly and conveniently reconstruct objects online, which seems a more natural and smarter way for a robot to explore the world.
I used ROS package iai_kinect2 to drive the RGB-D camera, do registration of range image and color image, and generate raw point cloud as input data of this 3D reconstruction pipeline.
Open3D is a brand new library for 3D data processing, which was just released in Jan 2018. The reason I used this library for point cloud registration is that they implemented the state-of-the-art Colored ICP algorithm. Plus, the library has Python interface with complete documentation and examples, which is easy to use.
[Park2017] J. Park, Q.-Y. Zhou, and V. Koltun, Colored Point Cloud Registration Revisited, ICCV, 2017.
The image above shows how effective this algorithm is. The left only used ICP with normal info of each points, while the right image first used Normal ICP for rough registration, plus Colored ICP for refinement. For some objects with rich textures on surface, with color information synthesized, this algorithm would be super powerful.
This pipeline works in ROS framework.
/kinect2/qhd/pointsgenerated by Kinect V2)
The image above shows the setup of the 3D reconstruction pipeline. With all the ROS nodes ready, when people rotates the turntable slowly, the 3D reconstruction process is shown on the computer screen.
Feel free to download this pipeline and play with it. Hope you enjoy it. There are some tips you may want to know for an easy start.
catkin buildto build this ROS package)
Intsructions for use:
$ roslaunch kinect_bridge kinect_bridge.launchto connect Kinect V2 thtough iai_kinect2;
$ roslaunch scanner pipeline.launchto run the pipeline.