Awesome Open Source
Awesome Open Source

EAO-SLAM

Related Paper:

  • Wu Y, Zhang Y, Zhu D, et al. EAO-SLAM: Monocular Semi-Dense Object SLAM Based on Ensemble Data Association[C]//2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2020: 4966-4973. [PDF] [YouTube] [bilibili] [Project page].
  • Extended Work
    • Wu Y, Zhang Y, Zhu D, et al. Object-Driven Active Mapping for More Accurate Object Pose Estimation and Robotic Grasping[J]. arXiv preprint arXiv:2012.01788, 2020. [PDF] [Project page].
    • Robotic Grasping demo: YouTube | bilibili
    • Augmented Reality demo: YouTube | bilibili
  • If you use the code in your academic work, please cite the above paper.

1. Prerequisites

  • Prerequisites are the same as semidense-lines. If compiling problems met, please refer to semidense-lines and ORB_SLAM2.
  • The code is tested in Ubuntu 16.04, opencv 3.2.0/3.3.1, Eigen 3.2.1.

2. Building

chmod +x build.sh       
./build.sh

3. Examples

    1. We provide a demo that uses the TUM rgbd_dataset_freiburg3_long_office_household sequence; please download the dataset beforehand. The offline object bounding boxes are in data/yolo_txts folder.
    1. Object size and orientation estimation.
    • use iForest and line alignment:
      ./Examples/Monocular/mono_tum LineAndiForest [path of tum fr3_long_office]
      
    • only use iForest:
      ./Examples/Monocular/mono_tum iForest [path of tum fr3_long_office]
      
    • without iForest and line alignment:
      ./Examples/Monocular/mono_tum None [path of tum fr3_long_office]
      

    Figure 1

    1. Data association
    • without data association:
      ./Examples/Monocular/mono_tum NA [path of tum fr3_long_office]
      
    • data association by IoU only:
      ./Examples/Monocular/mono_tum IoU [path of tum fr3_long_office]
      
    • data association by Non-Parametric-test only:
      ./Examples/Monocular/mono_tum NP [path of tum fr3_long_office]
      
    • data association by our ensemble method:
      ./Examples/Monocular/mono_tum EAO [path of tum fr3_long_office]
      

    Figure 1

    1. The full demo on TUM fr3_long_office sequence:
    ./Examples/Monocular/mono_tum Full [path of tum fr3_long_office]
    
    • If you want to see the semi-dense map, you may have to wait a while after the sequence ends.
    • Since YOLO (which was not trained in this scenario) made a lot of false detections at the start of the sequence, so we adopted a stricter elimination mechanism, which resulted in the deletion of many objects at the start.

4. Videos

5. Note

  • This is an incomplete version of our paper. If you want to use it in your work or with other datasets, you should prepare the offline semantic detection/segmentation results or switch to online mode. Besides, you may need to adjust the data association strategy and abnormal object elimination mechanism (We found the misdetection from YOLO has a great impact on the results).

6. Acknowledgement

Thanks for the great work: ORB-SLAM2, Cube SLAM, and Semidense-Lines.

  • Mur-Artal R, Tardós J D. Orb-slam2: An open-source slam system for monocular, stereo, and rgb-d cameras[J]. IEEE Transactions on Robotics, 2017, 33(5): 1255-1262. PDF, Code
  • Yang S, Scherer S. Cubeslam: Monocular 3-d object slam[J]. IEEE Transactions on Robotics, 2019, 35(4): 925-938. PDF, Code
  • He S, Qin X, Zhang Z, et al. Incremental 3d line segment extraction from semi-dense slam[C]//2018 24th International Conference on Pattern Recognition (ICPR). IEEE, 2018: 1658-1663. PDF, Code

7. Contact


Get A Weekly Email With Trending Projects For These Topics
No Spam. Unsubscribe easily at any time.
c-plus-plus (18,401
slam (138
mapping (135
pose-estimation (109
semantic (60