Mesh-based Object Tracking for Dynamic Semantic 3D Scene Graphs via Ray Tracing
Abstract
In this paper, we present a novel method for 3D geometric scene graph generation using range sensors and RGB cameras. We initially detect instance-wise keypoints with a YOLOv8s model to compute 6D pose estimates of known objects by solving PnP. We use a ray tracing approach to track a geometric scene graph consisting of mesh models of object instances. In contrast to classical point-to-point matching, this leads to more robust results, especially under occlusions between objects instances. We show that using this hybrid strategy leads to robust self-localization, pre-segmentation of the range sensor data and accurate pose tracking of objects using the same environmental representation. All detected objects are integrated into a semantic scene graph. This scene graph then serves as a front end to a semantic mapping framework to allow spatial reasoning.
- Publication:
-
arXiv e-prints
- Pub Date:
- August 2024
- DOI:
- 10.48550/arXiv.2408.04979
- arXiv:
- arXiv:2408.04979
- Bibcode:
- 2024arXiv240804979N
- Keywords:
-
- Computer Science - Robotics
- E-Print:
- RSS Workshop on Semantics for Robotics: From Environment Understanding and Reasoning to Safe Interaction 2024