Three-dimensional porous structure reconstruction for low-resolution monocular endoscopic images

Jian-xun ZHANG; Ming-hui HAN; Yu DAI

doi:10.37188/OPE.20202809.2085

您当前的位置：

首页 >

文章列表页 >

Three-dimensional porous structure reconstruction for low-resolution monocular endoscopic images

Information Sciences | 更新时间：2020-09-28

- Three-dimensional porous structure reconstruction for low-resolution monocular endoscopic images
- Optics and Precision Engineering Vol. 28, Issue 9, Pages: 2085-2095(2020)
- 作者机构：
  
  1.南开大学人工智能学院, 天津 300350
  2.南开大学机器人与信息自动化研究所, 天津 300350
- 作者简介：
- 基金信息：
- DOI：10.37188/OPE.20202809.2085
  CLC： TP391
- Received：07 April 2020，
  
  Revised：12 April 2020，
  
  Accepted：12 April 2020，
  
  Published：25 September 2020
- 稿件说明：
移动端阅览
Jian-xun ZHANG, Ming-hui HAN, Yu DAI. Three-dimensional porous structure reconstruction for low-resolution monocular endoscopic images[J]. Optics and precision engineering, 2020, 28(9): 2085-2095.
DOI：

Jian-xun ZHANG, Ming-hui HAN, Yu DAI. Three-dimensional porous structure reconstruction for low-resolution monocular endoscopic images[J]. Optics and precision engineering, 2020, 28(9): 2085-2095. DOI： 10.37188/OPE.20202809.2085.

摘要

三维多孔结构是内窥镜在医疗外科手术和工业检查领域常见的一种场景，然而内窥镜成像分辨率低，其应用场景纹理特征弱、噪声大，这导致应用传统的从运动中恢复结构（SFM）算法进行多孔结构的三维重建和定位难度大，可行性低。为此，本文提出了一种面向低分辨率单目内窥镜图像序列的三维多孔结构重建方法。首先将基于加权引导滤波的图像增强算法应用于内窥镜图像纹理细节的提升；接着利用快速行进（FMM）算法去除漂浮杂质；然后提出了一种检测离群点对的方法用于优化特征点匹配结果，通过SFM计算获得稀疏点云；最后针对本文研究的多孔结构，提出了一种基于模板匹配和最小二乘拟合的孔洞三维结构提取方法。实验结果表明：多孔球面上的重建结果中孔间距和孔直径的相对误差均小于10%，真实肾结石手术场景上的重建与实际相符，验证了本文整体算法流程的有效性、准确性和可行性。

Abstract

A three-dimensional porous structure is a common application of the endoscope in the field of medical surgery and industrial inspection. However

the limitations of low endoscopic imaging resolution

weak texture features

and high noise make it difficult to utilize a traditional structure from motion (SFM) algorithm for reconstructing and locating the three-dimensional porous structure. In this study

a three-dimensional porous structure reconstruction method based on low-resolution monocular endoscopic images was proposed. The image enhancement algorithm based on weighted guided filtering is first applied to improve the endoscopic image details. The fast marching method is then used to remove floating impurities

and an outlier detection method is proposed to optimize the results of feature matching. Additionally

the sparse point cloud is obtained by an SFM algorithm. Finally

a method based on template matching and least square fitting is presented to extract the three-dimensional structures of holes. Experimental results indicated that the relative errors in the spacing between holes and their diameters of reconstruction on a porous hemisphere were < 10%

and the reconstruction results on the real renal surgery video were consistent with reality. Therefore

the validity

accuracy

and feasibility of the whole reconstruction algorithm flow were verified.

关键词

Keywords

references

王欣 , 袁坤 , 于晓 , 等 . 基于运动恢复的双目视觉三维重建系统设计 . 光学精密工程 , 2014 . 22 ( 5 ): 1379 - 1387 . http://ope.lightpublishing.cn/thesisDetails?columnId=2218545&Fpath=&index=-1&l=zh http://ope.lightpublishing.cn/thesisDetails?columnId=2218545&Fpath=&index=-1&l=zh .

X WANG , K YUAN , X YU , 等 . Design of binocular vision 3D reconstruction system based on motion recovery . Opt. Precision Eng. , 2014 . 22 ( 5 ): 1379 - 1387 . http://ope.lightpublishing.cn/thesisDetails?columnId=2218545&Fpath=&index=-1&l=zh http://ope.lightpublishing.cn/thesisDetails?columnId=2218545&Fpath=&index=-1&l=zh .

李巍 , 董明利 , 吕乃光 , 等 . 基于二阶锥规划无标定参照物的手眼标定 . 光学精密工程 , 2018 . 26 ( 10 ): 2536 - 2545 . http://ope.lightpublishing.cn/thesisDetails?columnId=1420150&Fpath=&index=-1&l=zh http://ope.lightpublishing.cn/thesisDetails?columnId=1420150&Fpath=&index=-1&l=zh .

W LI , M L DONG , N G LV , 等 . Hand-eye calibration method without a calibration reference based on second-order cone programming . Opt. Precision Eng. , 2018 . 26 ( 10 ): 2536 - 2545 . http://ope.lightpublishing.cn/thesisDetails?columnId=1420150&Fpath=&index=-1&l=zh http://ope.lightpublishing.cn/thesisDetails?columnId=1420150&Fpath=&index=-1&l=zh .

谢理想 , 万刚 , 曹雪峰 , 等 . 基于凸优化改进的相机全局位置估计方法 . 自动化学报 , 2018 . 44 ( 3 ): 506 - 516 . http://www.cnki.com.cn/Article/CJFDTotal-MOTO201803011.htm http://www.cnki.com.cn/Article/CJFDTotal-MOTO201803011.htm .

L X XIE , G WAN , X F CAO , 等 . An improved method for camera location estimation through convex optimization . Acta Automatica Sinica , 2018 . 44 ( 3 ): 506 - 516 . http://www.cnki.com.cn/Article/CJFDTotal-MOTO201803011.htm http://www.cnki.com.cn/Article/CJFDTotal-MOTO201803011.htm .

曾露露 , 盖绍彦 , 达飞鹏 , 等 . 基于从运动中恢复结构的三维点云孔洞修补算法研究 . 光学学报 , 2018 . 38 ( 6 ): 131 - 137 . http://www.cnki.com.cn/Article/CJFDTotal-GXXB201806018.htm http://www.cnki.com.cn/Article/CJFDTotal-GXXB201806018.htm .

L L ZENG , SH Y GAI , F P DA , 等 . Hole filling algorithm of three-dimensional point cloud based on structure from motion . Acta Optica Sinica , 2018 . 38 ( 6 ): 131 - 137 . http://www.cnki.com.cn/Article/CJFDTotal-GXXB201806018.htm http://www.cnki.com.cn/Article/CJFDTotal-GXXB201806018.htm .

T BERGEN , T WITTENBERG . Stitching and surface reconstruction from endoscopic imagesequences:a review of applications and methods . IEEE journal of biomedical and health informatics , 2014 . 20 ( 1 ): 304 - 321 . https://ieeexplore.ieee.org/document/6991521/ https://ieeexplore.ieee.org/document/6991521/ .

郭俊锋 , 刘鹏 , 焦国华 , 等 . 三维测量工业内窥镜的双目光学系统 . 光学精密工程 , 2014 . 22 ( 9 ): 2337 - 2344 . http://ope.lightpublishing.cn/thesisDetails?columnId=2225424&Fpath=&index=-1&l=zh http://ope.lightpublishing.cn/thesisDetails?columnId=2225424&Fpath=&index=-1&l=zh .

J F GUO , P LIU , G H JIAO , 等 . Binocular optical system of industrial endoscope for three-dimensional measurement . Opt. Precision Eng. , 2014 . 22 ( 9 ): 2337 - 2344 . http://ope.lightpublishing.cn/thesisDetails?columnId=2225424&Fpath=&index=-1&l=zh http://ope.lightpublishing.cn/thesisDetails?columnId=2225424&Fpath=&index=-1&l=zh .

杨东林 , 卢栋 , 冯大伟 . 基于图像处理的三维工业内窥镜研究 . 仪器仪表学报 , 2009 . 30 ( 6 ): 1285 - 1289 . http://www.cqvip.com/Main/Detail.aspx?id=30862093 http://www.cqvip.com/Main/Detail.aspx?id=30862093 .

D L YANG , D LU , D W FENG . Research on 3D industrial endoscope based on imageprocessing . Chinese Journal of Scientific Instrument. , 2009 . 30 ( 6 ): 1285 - 1289 . http://www.cqvip.com/Main/Detail.aspx?id=30862093 http://www.cqvip.com/Main/Detail.aspx?id=30862093 .

BEN-HAMADOU A, DAUL C, SOUSSEN C, et al . A novel 3D surface construction approach: Application to three-dimensional endoscopic data[C]. 2010 IEEE International Conference on Image Processing , 2010: 4425-4428.

D SUN , C A LINTE , H DUAN , 等 . Surface reconstruction from tracked endoscopic video using the structure from motion approach . Augmented Reality Environments for Medical Imaging and Computer-Assisted Interventions:Springer , 2013 . 127 - 135 . http://cn.bing.com/academic/profile?id=d16325a3c629bd28214dea68afaf7dfe&encoded=0&v=paper_preview&mkt=zh-cn http://cn.bing.com/academic/profile?id=d16325a3c629bd28214dea68afaf7dfe&encoded=0&v=paper_preview&mkt=zh-cn .

M VISENTINI-SCARZANELLA , T SUGIURA , T KANEKO , 等 . Deep monocular 3D reconstruction for assisted navigation in bronchoscopy . International Journal of Computer Assisted Radiology & Surgery , 2017 . http://cn.bing.com/academic/profile?id=a41c337ada9f8d37064c08063dd0b669&encoded=0&v=paper_preview&mkt=zh-cn http://cn.bing.com/academic/profile?id=a41c337ada9f8d37064c08063dd0b669&encoded=0&v=paper_preview&mkt=zh-cn .

N MAHMOUD , T COLLINS , A HOSTETTLER , 等 . Live tracking and dense reconstruction for handheld monocular endoscopy . IEEE transactions on medical imaging , 2018 . 38 ( 1 ): 79 - 89 . http://cn.bing.com/academic/profile?id=8f0c6d41d2cb2c2249c968def6831177&encoded=0&v=paper_preview&mkt=zh-cn http://cn.bing.com/academic/profile?id=8f0c6d41d2cb2c2249c968def6831177&encoded=0&v=paper_preview&mkt=zh-cn .

M TURAN , Y Y PILAVCI , I GANIYUSUFOGLU , 等 . Sparse-then-dense alignment-based 3D map reconstruction method for endoscopic capsule robots . Machine Vision and Applications , 2018 . 29 ( 2 ): 345 - 359 . http://cn.bing.com/academic/profile?id=d79fcaeabfe02c2a8786749c4c481215&encoded=0&v=paper_preview&mkt=zh-cn http://cn.bing.com/academic/profile?id=d79fcaeabfe02c2a8786749c4c481215&encoded=0&v=paper_preview&mkt=zh-cn .

陈晓冬 , 吉佳瑞 , 盛婧 , 等 . 分数阶微分加权引导滤波对超声图像的纹理保持 . 光学精密工程 , 2020 . 28 ( 1 ): 174 - 181 . http://ope.lightpublishing.cn/thesisDetails?columnId=1496362&Fpath=&index=-1&l=zh http://ope.lightpublishing.cn/thesisDetails?columnId=1496362&Fpath=&index=-1&l=zh .

X D CHEN , J R JI , J SHENG , 等 . Fractional differential weighted guided filtering for image texture preservation for medical ultrasound . Opt. Precision Eng. , 2020 . 28 ( 1 ): 174 - 181 . http://ope.lightpublishing.cn/thesisDetails?columnId=1496362&Fpath=&index=-1&l=zh http://ope.lightpublishing.cn/thesisDetails?columnId=1496362&Fpath=&index=-1&l=zh .

陈清江 , 张雪 . 混合残差学习与导向滤波算法在图像去雾中的应用 . 光学精密工程 , 2019 . 27 ( 12 ): 2702 - 2712 . http://ope.lightpublishing.cn/thesisDetails?columnId=2141016&Fpath=&index=-1&l=zh http://ope.lightpublishing.cn/thesisDetails?columnId=2141016&Fpath=&index=-1&l=zh .

Q J CHEN , X ZHANG . Application of hybrid residual learning and guided filtering algorithm in image defogging . Opt. Precision Eng. , 2019 . 27 ( 12 ): 2702 - 2712 . http://ope.lightpublishing.cn/thesisDetails?columnId=2141016&Fpath=&index=-1&l=zh http://ope.lightpublishing.cn/thesisDetails?columnId=2141016&Fpath=&index=-1&l=zh .

宫剑 , 吕俊伟 , 刘亮 , 等 . 红外偏振舰船目标自适应尺度局部对比度检测 . 光学精密工程 , 2020 . 28 ( 1 ): 223 - 233 . http://ope.lightpublishing.cn/thesisDetails?columnId=1496837&Fpath=&index=-1&l=zh http://ope.lightpublishing.cn/thesisDetails?columnId=1496837&Fpath=&index=-1&l=zh .

J GONG , J W LV , L LIU , 等 . Adaptive sale local contrast detection for infrared polarization ship targets . Opt. Precision Eng. , 2020 . 28 ( 1 ): 223 - 233 . http://ope.lightpublishing.cn/thesisDetails?columnId=1496837&Fpath=&index=-1&l=zh http://ope.lightpublishing.cn/thesisDetails?columnId=1496837&Fpath=&index=-1&l=zh .

Z LI , J ZHENG , W YAO , 等 . Weighted guided image filtering . IEEE Transactions on Image Processing , 2015 . 24 ( 1 ): 120 - 129 .

A TELEA . An image inpainting technique based on the fast marching method . Journal of graphics tools , 2004 . 9 ( 1 ): 23 - 34 . DOI: 10.1080/10867651.2004.10487596 http://doi.org/10.1080/10867651.2004.10487596 .

解则晓 , 王晓东 , 宫韩磊 . 基于双目视觉的薄壁零件圆孔轮廓测量 . 中国激光 , 2019 . 46 ( 12 ): 179 - 186 .

Z X XIE , X D WANG , H L GONG , 等 . Round-hole profile measurement of thin-walled parts based on binocular vision . Chinese Journal of Lasers , 2019 . 46 ( 12 ): 179 - 186 .

方文辉 , 陈熙源 , 柳笛 . 基于改进Harris的低动态载体速度的快速计算方法 . 光学学报 , 2018 . 38 ( 4 ): 270 - 276 .

W H FANG , X Y CHEN , D LIU , 等 . Fast calculation method for low dynamic carrier velocity based on improved Harris . Acta Optica Sinica , 2018 . 38 ( 4 ): 270 - 276 .

M M BREUNIG , H P KRIEGEL , R T NG , 等 . LOF:identifying density-based local outliers . ACM sigmod record , 2000 . 93 - 104 . http://cn.bing.com/academic/profile?id=95956f2ccd5a6941f3e71ccfb2988419&encoded=0&v=paper_preview&mkt=zh-cn http://cn.bing.com/academic/profile?id=95956f2ccd5a6941f3e71ccfb2988419&encoded=0&v=paper_preview&mkt=zh-cn .

R MUR-ARTAL , J M M MONTIEL , J D TARDOS . ORB-SLAM:a versatile and accurate monocular SLAM system . IEEE transactions on robotics , 2015 . 31 ( 5 ): 1147 - 1163 . DOI: 10.1109/TRO.2015.2463671 http://doi.org/10.1109/TRO.2015.2463671 .

Views

153

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Review of MEMS vibration-mirror-based 3D reconstruction of structured light

Adaptive color compensation and multi-scale fusion for underwater image enhancement

Registration of heterogeneous point cloud and precise reconstruction of 3D terrain via complex slope surface fitting

Binocular vision-based trackside pantograph anomaly detection under strong environmental noise

Multi-scale point cloud fusion based on dual-projection structured light system

Related Author

HAN Min

ZHANG Chenping

ZHANG Zonghua

LI Xinghui

LIU Cuiyin

JING Tengyun

WANG Feng

CHEN Yuanshuai

Related Institution

Shenzhen International Graduate School， Tsinghua University

School of Mechanical Engineering， Hebei University of Technology

School of Information Engineering and Automation， Kunming University of Science and Technology

School of Physics and Material Sciences， Guangzhou University

College of Geological Engineering and Geomatics， Chang'an University

AI问答

Address：No.3888 Dong Nanhu Road, Changchun, Jilin, China Postal code：130033
Tel：0431-86176855 Email：gxjmgc@ciomp.ac.cn
Technical support is provided by Beijing Founder electronics co., LTD 吉ICP备11002662号-17 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰