面向低分辨率单目内窥镜图像的三维多孔结构重建

张建勋; 韩明慧; 代煜

doi:10.37188/OPE.20202809.2085

您当前的位置：

首页 >

文章列表页 >

面向低分辨率单目内窥镜图像的三维多孔结构重建

信息科学 | 更新时间：2020-09-28

- 面向低分辨率单目内窥镜图像的三维多孔结构重建
- Three-dimensional porous structure reconstruction for low-resolution monocular endoscopic images
- 光学精密工程 2020年28卷第9期页码：2085-2095
- 作者机构：
  
  1.南开大学人工智能学院, 天津 300350
  2.南开大学机器人与信息自动化研究所, 天津 300350
- 作者简介：
  
  [ "张建勋(1961-), 男, 天津人, 博士, 教授, 分别于1984年和1987年于天津大学获得学士学位和硕士学位, 1994年于中国科学院自动化研究所获得博士学位, 主要从事医疗机器人技术、智能机器人控制等方面的研究。E-mail:zhangjx@nankai.edu.cn" ]
  代煜(1981-), 男, 天津人, 副教授, 分别于2002年、2004年、2009年于哈尔滨工业大学获得学士学位, 硕士学位和博士学位, 主要从事面向手术机器人的智能感知技术等方面的研究。E-mail:daiyu@nankai.edu.cn DAI Yu, E-mail:daiyu@nankai.edu.cn
- 基金信息：
  
  国家重点研发计划资助项目(2017YFB1302803);天津市自然科学基金资助项目(18JCYBJC18800)
- DOI：10.37188/OPE.20202809.2085
  中图分类号： TP391
- 收稿日期：2020-04-07，
  
  修回日期：2020-04-12，
  
  录用日期：2020-4-12，
  
  纸质出版日期：2020-09-25
- 稿件说明：
移动端阅览
张建勋, 韩明慧, 代煜. 面向低分辨率单目内窥镜图像的三维多孔结构重建[J]. 光学精密工程, 2020,28(9):2085-2095.

Jian-xun ZHANG, Ming-hui HAN, Yu DAI. Three-dimensional porous structure reconstruction for low-resolution monocular endoscopic images[J]. Optics and precision engineering, 2020, 28(9): 2085-2095.
张建勋, 韩明慧, 代煜. 面向低分辨率单目内窥镜图像的三维多孔结构重建[J]. 光学精密工程, 2020,28(9):2085-2095. DOI： 10.37188/OPE.20202809.2085.

Jian-xun ZHANG, Ming-hui HAN, Yu DAI. Three-dimensional porous structure reconstruction for low-resolution monocular endoscopic images[J]. Optics and precision engineering, 2020, 28(9): 2085-2095. DOI： 10.37188/OPE.20202809.2085.

摘要

三维多孔结构是内窥镜在医疗外科手术和工业检查领域常见的一种场景，然而内窥镜成像分辨率低，其应用场景纹理特征弱、噪声大，这导致应用传统的从运动中恢复结构（SFM）算法进行多孔结构的三维重建和定位难度大，可行性低。为此，本文提出了一种面向低分辨率单目内窥镜图像序列的三维多孔结构重建方法。首先将基于加权引导滤波的图像增强算法应用于内窥镜图像纹理细节的提升；接着利用快速行进（FMM）算法去除漂浮杂质；然后提出了一种检测离群点对的方法用于优化特征点匹配结果，通过SFM计算获得稀疏点云；最后针对本文研究的多孔结构，提出了一种基于模板匹配和最小二乘拟合的孔洞三维结构提取方法。实验结果表明：多孔球面上的重建结果中孔间距和孔直径的相对误差均小于10%，真实肾结石手术场景上的重建与实际相符，验证了本文整体算法流程的有效性、准确性和可行性。

Abstract

A three-dimensional porous structure is a common application of the endoscope in the field of medical surgery and industrial inspection. However

the limitations of low endoscopic imaging resolution

weak texture features

and high noise make it difficult to utilize a traditional structure from motion (SFM) algorithm for reconstructing and locating the three-dimensional porous structure. In this study

a three-dimensional porous structure reconstruction method based on low-resolution monocular endoscopic images was proposed. The image enhancement algorithm based on weighted guided filtering is first applied to improve the endoscopic image details. The fast marching method is then used to remove floating impurities

and an outlier detection method is proposed to optimize the results of feature matching. Additionally

the sparse point cloud is obtained by an SFM algorithm. Finally

a method based on template matching and least square fitting is presented to extract the three-dimensional structures of holes. Experimental results indicated that the relative errors in the spacing between holes and their diameters of reconstruction on a porous hemisphere were < 10%

and the reconstruction results on the real renal surgery video were consistent with reality. Therefore

the validity

accuracy

and feasibility of the whole reconstruction algorithm flow were verified.

关键词

Keywords

references

王欣 , 袁坤 , 于晓 , 等 . 基于运动恢复的双目视觉三维重建系统设计 . 光学精密工程 , 2014 . 22 ( 5 ): 1379 - 1387 . http://ope.lightpublishing.cn/thesisDetails?columnId=2218545&Fpath=&index=-1&l=zh http://ope.lightpublishing.cn/thesisDetails?columnId=2218545&Fpath=&index=-1&l=zh .

X WANG , K YUAN , X YU , 等 . Design of binocular vision 3D reconstruction system based on motion recovery . Opt. Precision Eng. , 2014 . 22 ( 5 ): 1379 - 1387 . http://ope.lightpublishing.cn/thesisDetails?columnId=2218545&Fpath=&index=-1&l=zh http://ope.lightpublishing.cn/thesisDetails?columnId=2218545&Fpath=&index=-1&l=zh .

李巍 , 董明利 , 吕乃光 , 等 . 基于二阶锥规划无标定参照物的手眼标定 . 光学精密工程 , 2018 . 26 ( 10 ): 2536 - 2545 . http://ope.lightpublishing.cn/thesisDetails?columnId=1420150&Fpath=&index=-1&l=zh http://ope.lightpublishing.cn/thesisDetails?columnId=1420150&Fpath=&index=-1&l=zh .

W LI , M L DONG , N G LV , 等 . Hand-eye calibration method without a calibration reference based on second-order cone programming . Opt. Precision Eng. , 2018 . 26 ( 10 ): 2536 - 2545 . http://ope.lightpublishing.cn/thesisDetails?columnId=1420150&Fpath=&index=-1&l=zh http://ope.lightpublishing.cn/thesisDetails?columnId=1420150&Fpath=&index=-1&l=zh .

谢理想 , 万刚 , 曹雪峰 , 等 . 基于凸优化改进的相机全局位置估计方法 . 自动化学报 , 2018 . 44 ( 3 ): 506 - 516 . http://www.cnki.com.cn/Article/CJFDTotal-MOTO201803011.htm http://www.cnki.com.cn/Article/CJFDTotal-MOTO201803011.htm .

L X XIE , G WAN , X F CAO , 等 . An improved method for camera location estimation through convex optimization . Acta Automatica Sinica , 2018 . 44 ( 3 ): 506 - 516 . http://www.cnki.com.cn/Article/CJFDTotal-MOTO201803011.htm http://www.cnki.com.cn/Article/CJFDTotal-MOTO201803011.htm .

曾露露 , 盖绍彦 , 达飞鹏 , 等 . 基于从运动中恢复结构的三维点云孔洞修补算法研究 . 光学学报 , 2018 . 38 ( 6 ): 131 - 137 . http://www.cnki.com.cn/Article/CJFDTotal-GXXB201806018.htm http://www.cnki.com.cn/Article/CJFDTotal-GXXB201806018.htm .

L L ZENG , SH Y GAI , F P DA , 等 . Hole filling algorithm of three-dimensional point cloud based on structure from motion . Acta Optica Sinica , 2018 . 38 ( 6 ): 131 - 137 . http://www.cnki.com.cn/Article/CJFDTotal-GXXB201806018.htm http://www.cnki.com.cn/Article/CJFDTotal-GXXB201806018.htm .

T BERGEN , T WITTENBERG . Stitching and surface reconstruction from endoscopic imagesequences:a review of applications and methods . IEEE journal of biomedical and health informatics , 2014 . 20 ( 1 ): 304 - 321 . https://ieeexplore.ieee.org/document/6991521/ https://ieeexplore.ieee.org/document/6991521/ .

郭俊锋 , 刘鹏 , 焦国华 , 等 . 三维测量工业内窥镜的双目光学系统 . 光学精密工程 , 2014 . 22 ( 9 ): 2337 - 2344 . http://ope.lightpublishing.cn/thesisDetails?columnId=2225424&Fpath=&index=-1&l=zh http://ope.lightpublishing.cn/thesisDetails?columnId=2225424&Fpath=&index=-1&l=zh .

J F GUO , P LIU , G H JIAO , 等 . Binocular optical system of industrial endoscope for three-dimensional measurement . Opt. Precision Eng. , 2014 . 22 ( 9 ): 2337 - 2344 . http://ope.lightpublishing.cn/thesisDetails?columnId=2225424&Fpath=&index=-1&l=zh http://ope.lightpublishing.cn/thesisDetails?columnId=2225424&Fpath=&index=-1&l=zh .

杨东林 , 卢栋 , 冯大伟 . 基于图像处理的三维工业内窥镜研究 . 仪器仪表学报 , 2009 . 30 ( 6 ): 1285 - 1289 . http://www.cqvip.com/Main/Detail.aspx?id=30862093 http://www.cqvip.com/Main/Detail.aspx?id=30862093 .

D L YANG , D LU , D W FENG . Research on 3D industrial endoscope based on imageprocessing . Chinese Journal of Scientific Instrument. , 2009 . 30 ( 6 ): 1285 - 1289 . http://www.cqvip.com/Main/Detail.aspx?id=30862093 http://www.cqvip.com/Main/Detail.aspx?id=30862093 .

BEN-HAMADOU A, DAUL C, SOUSSEN C, et al . A novel 3D surface construction approach: Application to three-dimensional endoscopic data[C]. 2010 IEEE International Conference on Image Processing , 2010: 4425-4428.

D SUN , C A LINTE , H DUAN , 等 . Surface reconstruction from tracked endoscopic video using the structure from motion approach . Augmented Reality Environments for Medical Imaging and Computer-Assisted Interventions:Springer , 2013 . 127 - 135 . http://cn.bing.com/academic/profile?id=d16325a3c629bd28214dea68afaf7dfe&encoded=0&v=paper_preview&mkt=zh-cn http://cn.bing.com/academic/profile?id=d16325a3c629bd28214dea68afaf7dfe&encoded=0&v=paper_preview&mkt=zh-cn .

M VISENTINI-SCARZANELLA , T SUGIURA , T KANEKO , 等 . Deep monocular 3D reconstruction for assisted navigation in bronchoscopy . International Journal of Computer Assisted Radiology & Surgery , 2017 . http://cn.bing.com/academic/profile?id=a41c337ada9f8d37064c08063dd0b669&encoded=0&v=paper_preview&mkt=zh-cn http://cn.bing.com/academic/profile?id=a41c337ada9f8d37064c08063dd0b669&encoded=0&v=paper_preview&mkt=zh-cn .

N MAHMOUD , T COLLINS , A HOSTETTLER , 等 . Live tracking and dense reconstruction for handheld monocular endoscopy . IEEE transactions on medical imaging , 2018 . 38 ( 1 ): 79 - 89 . http://cn.bing.com/academic/profile?id=8f0c6d41d2cb2c2249c968def6831177&encoded=0&v=paper_preview&mkt=zh-cn http://cn.bing.com/academic/profile?id=8f0c6d41d2cb2c2249c968def6831177&encoded=0&v=paper_preview&mkt=zh-cn .

M TURAN , Y Y PILAVCI , I GANIYUSUFOGLU , 等 . Sparse-then-dense alignment-based 3D map reconstruction method for endoscopic capsule robots . Machine Vision and Applications , 2018 . 29 ( 2 ): 345 - 359 . http://cn.bing.com/academic/profile?id=d79fcaeabfe02c2a8786749c4c481215&encoded=0&v=paper_preview&mkt=zh-cn http://cn.bing.com/academic/profile?id=d79fcaeabfe02c2a8786749c4c481215&encoded=0&v=paper_preview&mkt=zh-cn .

陈晓冬 , 吉佳瑞 , 盛婧 , 等 . 分数阶微分加权引导滤波对超声图像的纹理保持 . 光学精密工程 , 2020 . 28 ( 1 ): 174 - 181 . http://ope.lightpublishing.cn/thesisDetails?columnId=1496362&Fpath=&index=-1&l=zh http://ope.lightpublishing.cn/thesisDetails?columnId=1496362&Fpath=&index=-1&l=zh .

X D CHEN , J R JI , J SHENG , 等 . Fractional differential weighted guided filtering for image texture preservation for medical ultrasound . Opt. Precision Eng. , 2020 . 28 ( 1 ): 174 - 181 . http://ope.lightpublishing.cn/thesisDetails?columnId=1496362&Fpath=&index=-1&l=zh http://ope.lightpublishing.cn/thesisDetails?columnId=1496362&Fpath=&index=-1&l=zh .

陈清江 , 张雪 . 混合残差学习与导向滤波算法在图像去雾中的应用 . 光学精密工程 , 2019 . 27 ( 12 ): 2702 - 2712 . http://ope.lightpublishing.cn/thesisDetails?columnId=2141016&Fpath=&index=-1&l=zh http://ope.lightpublishing.cn/thesisDetails?columnId=2141016&Fpath=&index=-1&l=zh .

Q J CHEN , X ZHANG . Application of hybrid residual learning and guided filtering algorithm in image defogging . Opt. Precision Eng. , 2019 . 27 ( 12 ): 2702 - 2712 . http://ope.lightpublishing.cn/thesisDetails?columnId=2141016&Fpath=&index=-1&l=zh http://ope.lightpublishing.cn/thesisDetails?columnId=2141016&Fpath=&index=-1&l=zh .

宫剑 , 吕俊伟 , 刘亮 , 等 . 红外偏振舰船目标自适应尺度局部对比度检测 . 光学精密工程 , 2020 . 28 ( 1 ): 223 - 233 . http://ope.lightpublishing.cn/thesisDetails?columnId=1496837&Fpath=&index=-1&l=zh http://ope.lightpublishing.cn/thesisDetails?columnId=1496837&Fpath=&index=-1&l=zh .

J GONG , J W LV , L LIU , 等 . Adaptive sale local contrast detection for infrared polarization ship targets . Opt. Precision Eng. , 2020 . 28 ( 1 ): 223 - 233 . http://ope.lightpublishing.cn/thesisDetails?columnId=1496837&Fpath=&index=-1&l=zh http://ope.lightpublishing.cn/thesisDetails?columnId=1496837&Fpath=&index=-1&l=zh .

Z LI , J ZHENG , W YAO , 等 . Weighted guided image filtering . IEEE Transactions on Image Processing , 2015 . 24 ( 1 ): 120 - 129 .

A TELEA . An image inpainting technique based on the fast marching method . Journal of graphics tools , 2004 . 9 ( 1 ): 23 - 34 . DOI: 10.1080/10867651.2004.10487596 http://doi.org/10.1080/10867651.2004.10487596 .

解则晓 , 王晓东 , 宫韩磊 . 基于双目视觉的薄壁零件圆孔轮廓测量 . 中国激光 , 2019 . 46 ( 12 ): 179 - 186 .

Z X XIE , X D WANG , H L GONG , 等 . Round-hole profile measurement of thin-walled parts based on binocular vision . Chinese Journal of Lasers , 2019 . 46 ( 12 ): 179 - 186 .

方文辉 , 陈熙源 , 柳笛 . 基于改进Harris的低动态载体速度的快速计算方法 . 光学学报 , 2018 . 38 ( 4 ): 270 - 276 .

W H FANG , X Y CHEN , D LIU , 等 . Fast calculation method for low dynamic carrier velocity based on improved Harris . Acta Optica Sinica , 2018 . 38 ( 4 ): 270 - 276 .

M M BREUNIG , H P KRIEGEL , R T NG , 等 . LOF:identifying density-based local outliers . ACM sigmod record , 2000 . 93 - 104 . http://cn.bing.com/academic/profile?id=95956f2ccd5a6941f3e71ccfb2988419&encoded=0&v=paper_preview&mkt=zh-cn http://cn.bing.com/academic/profile?id=95956f2ccd5a6941f3e71ccfb2988419&encoded=0&v=paper_preview&mkt=zh-cn .

R MUR-ARTAL , J M M MONTIEL , J D TARDOS . ORB-SLAM:a versatile and accurate monocular SLAM system . IEEE transactions on robotics , 2015 . 31 ( 5 ): 1147 - 1163 . DOI: 10.1109/TRO.2015.2463671 http://doi.org/10.1109/TRO.2015.2463671 .

浏览量

100

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于双投射结构光系统的多尺度点云融合

基于频域-空域混合注意力的降质遥感图像质量增强

基于单目视觉的火箭回收高度测量技术研究

纹理感知联合颜色直方图特征的水下图像增强

双路混合注意力的跨层次特征聚合图像增强