Stereoscopic augmented reality for single camera endoscopy: a virtual study

Yen Yu Wang, Atul Kumar, Kai Che Liu, Shih Wei Huang, Ching Chun Huang, Wei Chia Su, Fu Li Hsiao, Wen Nung Lie

研究成果: Article同行評審


Endoscopic surgery causes less tissue injury compared to open surgical techniques, thus promoting more rapid recuperation and reduced post-operative pain. Endoscopy, however, allows the surgeon to visualise only the anatomical surface of the surgical site, with a relatively narrow field of view. Moreover, the 2D video captured by the conventional endoscope does not provide depth perception of the surgical scene. In this study, these limitations have been addressed with the development of an augmented reality (AR) system with stereoscopic visualisation. A phantom and its 3D CT model were used, respectively, to form the real and virtual parts of the AR. The virtual environment camera pose was tracked using algorithms for image feature detection, feature matching and Perspective-n-Point applied on the endoscopic image and the 3D virtual model-rendered image. The endoscope video frame- and the virtual model-rendered images were superimposed to form the AR composite view. The depth buffer (z-buffer) of the rendering window was further used to make a stereo pair of the AR image. The AR system produced a stereo composite view having well-aligned real and virtual components. The RMS error of the real and virtual image contours registration was 9.6 ± 6.7 mm. Correlation coefficients between the depth map from z-buffer and a depth camera was between 0.60 and 0.96 (p < 0.05). The AR system requires further improvement to be applicable at a higher frame rate of endoscope image acquisition. It also needs to include motion and deformation models when applied to animals or patients.

頁(從 - 到)182-191
期刊Computer Methods in Biomechanics and Biomedical Engineering: Imaging and Visualization
出版狀態Published - 2018 三月 4

All Science Journal Classification (ASJC) codes

  • Computational Mechanics
  • Biomedical Engineering
  • Radiology Nuclear Medicine and imaging
  • Computer Science Applications

指紋 深入研究「Stereoscopic augmented reality for single camera endoscopy: a virtual study」主題。共同形成了獨特的指紋。