TY - JOUR
T1 - Stereoscopic augmented reality for single camera endoscopy
T2 - a virtual study
AU - Wang, Yen Yu
AU - Kumar, Atul
AU - Liu, Kai Che
AU - Huang, Shih Wei
AU - Huang, Ching Chun
AU - Su, Wei Chia
AU - Hsiao, Fu Li
AU - Lie, Wen Nung
N1 - Funding Information:
The study was supported by the Ministry of Science and Technology (grant number 103WHK5400012) and Chang Bing Show Chwan Memorial Hospital (grant number RA14003), Taiwan, R.o.C.
PY - 2018/3/4
Y1 - 2018/3/4
N2 - Endoscopic surgery causes less tissue injury compared to open surgical techniques, thus promoting more rapid recuperation and reduced post-operative pain. Endoscopy, however, allows the surgeon to visualise only the anatomical surface of the surgical site, with a relatively narrow field of view. Moreover, the 2D video captured by the conventional endoscope does not provide depth perception of the surgical scene. In this study, these limitations have been addressed with the development of an augmented reality (AR) system with stereoscopic visualisation. A phantom and its 3D CT model were used, respectively, to form the real and virtual parts of the AR. The virtual environment camera pose was tracked using algorithms for image feature detection, feature matching and Perspective-n-Point applied on the endoscopic image and the 3D virtual model-rendered image. The endoscope video frame- and the virtual model-rendered images were superimposed to form the AR composite view. The depth buffer (z-buffer) of the rendering window was further used to make a stereo pair of the AR image. The AR system produced a stereo composite view having well-aligned real and virtual components. The RMS error of the real and virtual image contours registration was 9.6 ± 6.7 mm. Correlation coefficients between the depth map from z-buffer and a depth camera was between 0.60 and 0.96 (p < 0.05). The AR system requires further improvement to be applicable at a higher frame rate of endoscope image acquisition. It also needs to include motion and deformation models when applied to animals or patients.
AB - Endoscopic surgery causes less tissue injury compared to open surgical techniques, thus promoting more rapid recuperation and reduced post-operative pain. Endoscopy, however, allows the surgeon to visualise only the anatomical surface of the surgical site, with a relatively narrow field of view. Moreover, the 2D video captured by the conventional endoscope does not provide depth perception of the surgical scene. In this study, these limitations have been addressed with the development of an augmented reality (AR) system with stereoscopic visualisation. A phantom and its 3D CT model were used, respectively, to form the real and virtual parts of the AR. The virtual environment camera pose was tracked using algorithms for image feature detection, feature matching and Perspective-n-Point applied on the endoscopic image and the 3D virtual model-rendered image. The endoscope video frame- and the virtual model-rendered images were superimposed to form the AR composite view. The depth buffer (z-buffer) of the rendering window was further used to make a stereo pair of the AR image. The AR system produced a stereo composite view having well-aligned real and virtual components. The RMS error of the real and virtual image contours registration was 9.6 ± 6.7 mm. Correlation coefficients between the depth map from z-buffer and a depth camera was between 0.60 and 0.96 (p < 0.05). The AR system requires further improvement to be applicable at a higher frame rate of endoscope image acquisition. It also needs to include motion and deformation models when applied to animals or patients.
UR - http://www.scopus.com/inward/record.url?scp=85006222299&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85006222299&partnerID=8YFLogxK
U2 - 10.1080/21681163.2016.1197798
DO - 10.1080/21681163.2016.1197798
M3 - Article
AN - SCOPUS:85006222299
VL - 6
SP - 182
EP - 191
JO - Computer Methods in Biomechanics and Biomedical Engineering: Imaging and Visualization
JF - Computer Methods in Biomechanics and Biomedical Engineering: Imaging and Visualization
SN - 2168-1163
IS - 2
ER -