Endoscopic Depth Measurement and Super-Spectral-Resolution Imaging
Abstract
Intra-operative measurements of tissue shape and multi/ hyperspectral information have the potential to provide surgical guidance and decision making support. We report an optical probe based system to combine sparse hyperspectral measurements and spectrally-encoded structured lighting (SL) for surface measurements. The system provides informative signals for navigation with a surgical interface. By rapidly switching between SL and white light (WL) modes, SL information is combined with structure-from-motion (SfM) from white light images, based on SURF feature detection and Lucas-Kanade (LK) optical flow to provide quasi-dense surface shape reconstruction with known scale in real-time. Furthermore, "super-spectral-resolution" was realized, whereby the RGB images and sparse hyperspectral data were integrated to recover dense pixel-level hyperspectral stacks, by using convolutional neural networks to upscale the wavelength dimension. Validation and demonstration of this system is reported on ex vivo/in vivo animal/ human experiments.
- Publication:
-
arXiv e-prints
- Pub Date:
- June 2017
- DOI:
- 10.48550/arXiv.1706.06081
- arXiv:
- arXiv:1706.06081
- Bibcode:
- 2017arXiv170606081L
- Keywords:
-
- Computer Science - Computer Vision and Pattern Recognition
- E-Print:
- accepted by MICCAI2017