北京生物医学工程

光学定位技术辅助的手术场景全景拼接算法

Panoramic stitching of surgical scenes assisted by optical positioning algorithm

作者：杨巧玲叶灿梁楠武博张楠

单位：首都医科大学生物医学工程学院（北京 100069） 首都医科大学临床生物力学应用基础研究北京市重点实验室（北京 100069） 通信作者：武博。E-mail：wubogo@ccmu.edu.cn

关键词：全景拼接；手术场景；高关注度区域；光学定位；深度信息

分类号：R318.04

出版年·卷·期（页码）：2022·41·3（242-248）

摘要：

目的提出一种光学定位技术辅助的手术场景全景拼接算法(panoramic stitching assisted by optical positioning technique, PSAOP)，以提高全景拼接质量。方法通过光学定位系统获得手术场景高关注度区域中严格匹配的光学定位点对。结合SIFT特征匹配点对，建立更准确的局部单应性模型。为了进一步减少透视失真，提出基于深度信息优化的全局相似变换(depth optimized global similarity transformation algorithm, DOGST)算法来补偿摄像机的运动。结果对模拟手术场景进行拼接，该算法可以极大提高场景拼接的自然性，获得了高主观质量。对齐后，视图之间重叠区域的结构相似性(structural similarity, SSIM)得分平均高于其他算法。DOGST获得的全局相似矩阵最小旋转角度小于RANSAC获得的角度，提高了拼接结果的自然度。结论 PSAOP能够提供更自然的拼接效果，在重叠区域中没有明显的重影，且进一步减少了非重叠区域中的透视失真。未来可与VR(Virtual Reality)结合使用，为远程医疗提供更身临其境、更立体直观的医疗场景。

Objective To propose a panoramic stitching assisted by optical positioning (PSAOP) technique for surgical scenes for surgical scenes，and to improve the quality of panoramic stitching, Methods The strictly matching optical positioning point pairs in high-attention area of the surgical scenes are obtained by the optical positioning system. Combining SIFT feature matching point pairs, a more accurate local homography model is established. In order to further reduce the perspective distortion, a depth optimized global similarity transformation (DOGST) algorithm is proposed to compensate for the camera motion. Results Experiments on simulated surgery scenes demonstrate that PSAOP can greatly improve the naturalness of the stitching result and obtain high subjective quality. Through subjective observation, the PSAOP can greatly improve the naturalness of stitching. The structural similarity (SSIM) scores of the overlapping region between views after alignment are higher than the other algorithms in average. The minimum rotation angle of global similarity transformation obtained by DOGST is smaller than that obtained by RANSAC, which improves the naturalness of stitching results. Conclusions The PSAOP algorithm provides a more natural stitching with no visible ghosting in the overlapping regions and further reduces the perspective distortion in the non-overlapping regions. Combined with Virtual reality (VR) in the future, it can provide more immersive and intuitive medical situations for telemedicine.

参考文献：

[1].? Ahrendt A, Schneider H, Kaiser M,et al. Prototypical development of a remote patient monitoring system for efficient treatment of COPD patients[J]. Biomedical Engineering / Biomedizinische Technik, 2013,58(1):4317-4318.

[2].? Ji H, Wang J, Gao J, Liu X. Research on telemedicine technology and implement based on virtual reality[C]//Advanced Information Management, Communicates, Electronic and Automation Control Conference (IMCEC). Xi'an：IEEE,2016.

[3].? Szeliski R. Computer vision: algorithms and applications[M]. New York：Springer-Verlag, Inc, 2010.

[4].? Zhang Y, Lai YK, Zhang F. Content-preserving image stitching with piecewise rectangular boundary constraints[J]. IEEE Transactions on Visualization and Computer Graphics, 2021, ?27（7）：3198-3212.

[5].? 张威. 不同光照条件下的图像拼接技术研究[D].沈阳：沈阳工业大学,2017.

Zhang W. Research on image stitching under different illumination[D]. Shenyang: Shenyang University of Technology,2017.

[6].? Lowe DG. Distinctive image features from scale-invariant keypoints[J]. International Journal of Computer Vision,2004,60(2):91-110.

[7].? Gao J, Kim SJ, Brown MS. Constructing image panoramas using dual-homography warping[C]// IEEE Computer Vision and Pattern Recognition (CVPR). Colorado Springs, USA：IEEE，2011.

[8].? Lin W, Liu S, Matsushita Y,et al. Smoothly varying affine stitching[C]//Computer Vision & Pattern Recognition, Colorado Springs, CO, USA：IEEE，2011.

[9].? Zaragoza J, Chin T, Tran Q,et al. As-projective-as-possible image stitching with moving DLT[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014,36(7):1285-1298.

[10]. Chang C, Sato Y, Chuang Y. Shape-preserving half-projective warps for image stitching[C]//IEEE Computer Vision and Pattern Recognition (CVPR). Columbus, OH, USA：IEEE, 2014.

[11]. Zheng J, Wang Y, Wang H, et al. A novel projective-consistent plane based image stitching method[J]. IEEE Transaction on Multimedia，2019,21(10):2561-2575.

[12]. Lin CC, Pankanti SU, Ramamurthy KN,et al. Adaptive as-natural-as-possible image stitching[C]//Computer Vision & Pattern Recognition. Boston, USA：IEEE, 2015.

[13]. Li J, Wang Z, Lai S,et al. Parallax-tolerant image stitching based on robust elastic warping[J]. IEEE Transactions On Multimedia, 2018,20(7):1672-1687.

[14]. Chen YS, Chuang YY. Natural image stitching with the global similarity prior[C]// European Conference on Computer Vision. Amsterdam, Netherlands：Springer International Publishing, 2016.

[15]. Zhang F, Liu F. Parallax-tolerant image stitching[C]// IEEE Computer Vision and Pattern Recognition (CVPR).Columbus, USA：IEEE, 2014.

[16]. Wang Z, Yang Z. Seam elimination based on curvelet for image stitching[J]. Soft Computing, 2019,23(13):5065-5080.

[17]. Lin K, Jiang N, Cheong L,et al. SEAGULL: seam-guided local alignment for parallax-tolerant image stitching[C]// Amsterdam, Netherlands：European Conference on Computer Vision. Springer (ECCV), 2016.

[18]. Hejazifar H, Khotanlou H. Fast and robust seam estimation to seamless image stitching[J]. Signal Image and Video Processing, 2018,12(1):1-9.

[19]. Jiang G, Luo M, Bai K. Optical positioning technology of an assisted puncture robot based on binocular vision[J]. International Journal of Imaging Systems & Technology, 2019, 29(2):180-190.

[20]. Zhou Z, Wu B, Duan J,et al. Optical surgical instrument tracking system based on the principle of stereo vision[J]. Journal of Biomedical Optics, 2017,22(6):1-14.

[21]. Zhang M, Wu B, Ye C,et al. Multiple instruments motion trajectory tracking in optical surgical navigation[J]. Optics Express, 2019,27(11):1-19.

[22]. 梁楠, 贾博奇, 张梦诗,等. 基于改进尺度不变特征变换的手术室多视点图像拼接算法[J]. 北京生物医学工程, 2018,37(1):9-14.

Zhang MS, Jia BQ, Liang N,et al. Optical tracking algorithm based on motion vectors for multiple surgical instruments[J]. Beijing Biomedical Engineering, 2018,37(4):345-350.

[23]. Fischler MA, Bolles RC. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography[J]. Communications of the ACM,1981,24(6):381-395.

[24]. Wang Z, Bovik AC, Sheikh HR, et al. Image quality assessment: from error visibility to structural similarity[J]. IEEE Trans Image Process, 2004,13(4):1-13.

服务与反馈：

【文章下载】【加入收藏】

提示：您还未登录，请登录！点此登录