DH-PTAM: A Deep Hybrid Stereo Events-Frames Parallel Tracking And Mapping System
Abstract
This paper presents a robust approach for a visual parallel tracking and mapping (PTAM) system that excels in challenging environments. Our proposed method combines the strengths of heterogeneous multi-modal visual sensors, including stereo event-based and frame-based sensors, in a unified reference frame through a novel spatio-temporal synchronization of stereo visual frames and stereo event streams. We employ deep learning-based feature extraction and description for estimation to enhance robustness further. We also introduce an end-to-end parallel tracking and mapping optimization layer complemented by a simple loop-closure algorithm for efficient SLAM behavior. Through comprehensive experiments on both small-scale and large-scale real-world sequences of VECtor and TUM-VIE benchmarks, our proposed method (DH-PTAM) demonstrates superior performance in terms of robustness and accuracy in adverse conditions, especially in large-scale HDR scenarios. Our implementation's research-based Python API is publicly available on GitHub for further research and development: https://github.com/AbanobSoliman/DH-PTAM.
- Publication:
-
arXiv e-prints
- Pub Date:
- June 2023
- DOI:
- 10.48550/arXiv.2306.01891
- arXiv:
- arXiv:2306.01891
- Bibcode:
- 2023arXiv230601891S
- Keywords:
-
- Computer Science - Computer Vision and Pattern Recognition;
- Computer Science - Robotics;
- Electrical Engineering and Systems Science - Image and Video Processing;
- Electrical Engineering and Systems Science - Signal Processing
- E-Print:
- Accepted for publication in the IEEE Transactions on Intelligent Vehicles