采用轮廓特征匹配的红外-可见光视频自动配准

孙兴龙; 韩广良; 刘培勋; 许廷发

doi:10.3788/OPE.20202805.1140

您当前的位置：

首页 >

文章列表页 >

采用轮廓特征匹配的红外-可见光视频自动配准

信息科学 | 更新时间：2020-07-09

- 采用轮廓特征匹配的红外-可见光视频自动配准
- Infrared-visible video automatic registration with contour feature matching
- 光学精密工程 2020年28卷第5期页码：1140-1151
- 作者机构：
  
  1.中国科学院长春光学精密机械与物理研究所, 吉林长春 130033
  2.中国科学院大学, 北京 100049
  3.北京理工大学光电学院, 北京 100081
- 作者简介：
  
  [ "孙兴龙(1993-), 男, 吉林长春人, 博士研究生, 研究实习员, 2018年于北京理工大学获得硕士学位, 主要从事异源图像配准融合、目标检测跟踪等方面的研究。E-mail:sunxinglong1123@163.com" ]
  韩广良(1968-), 男, 山东嘉祥人, 博士, 研究员, 博士生导师, 1990年于内蒙古工业大学获得学士学位, 2000和2003年于中科院长春光机所分别获得硕士学位和博士学位, 研究方向为图像和视频信息处理、计算机视觉、机器视觉与人工智能等。E-mail:hangl@ciomp.ac.cn HAN Guang-liang, E-mail:hangl@ciomp.ac.cn
- 基金信息：
  
  国家自然科学基金资助项目(61602432)
- DOI：10.3788/OPE.20202805.1140
  中图分类号： TP394.1;TH691.9
- 收稿日期：2019-11-11，
  
  修回日期：2020-01-29，
  
  录用日期：2020-1-29，
  
  纸质出版日期：2020-05-15
- 稿件说明：
移动端阅览
孙兴龙, 韩广良, 刘培勋, 等. 采用轮廓特征匹配的红外-可见光视频自动配准[J]. 光学精密工程, 2020,28(5):1140-1151.

Xing-long SUN, Guang-liang HAN, Pei-xun LIU, et al. Infrared-visible video automatic registration with contour feature matching[J]. Optics and precision engineering, 2020, 28(5): 1140-1151.
孙兴龙, 韩广良, 刘培勋, 等. 采用轮廓特征匹配的红外-可见光视频自动配准[J]. 光学精密工程, 2020,28(5):1140-1151. DOI： 10.3788/OPE.20202805.1140.

Xing-long SUN, Guang-liang HAN, Pei-xun LIU, et al. Infrared-visible video automatic registration with contour feature matching[J]. Optics and precision engineering, 2020, 28(5): 1140-1151. DOI： 10.3788/OPE.20202805.1140.

摘要

为了精确地配准近平面场景下的红外-可见光视频序列

本文提出了一种基于轮廓特征匹配的自动配准方法

通过迭代匹配目标轮廓特征来解决异源图像中配准特征的提取和匹配难题。首先

采用运动目标检测技术获取目标轮廓

并由曲率尺度空间(CSS)角点检测算法提取轮廓特征点。此后

建立全局形状上下文描述子和局部边缘方向直方图描述子描述特征

从而实现可靠的特征匹配。来自不同时刻的匹配点对被保存在一个基于高斯距离准则的特征匹配库中。最后

为了克服近平面场景中目标深度变化的影响

本文结合前景样本随机抽样策略计算配准矩阵的损失函数

完成对全局配准矩阵的更新。在LITIV数据库上对方法进行实验验证

结果表明：本文方法的配准精度优于当前先进的对比方法

在9个测试视频上的平均重叠率误差仅为0.194

与对比方法相比下降了18.5%。基本满足了近平面场景下红外-可见光视频序列配准的精度要求

且具有较高的鲁棒性。

Abstract

To register infrared-visible video sequences precisely inalmost-planar scenes

an automatic registration method based on matching the contour features was proposed in this paper. This method could solve the challenging problem regarding extracting and matching features in multimodal images by iteratively matching the contour features of targets. First

this method adopted the technology of moving target detection to identify the contours of targets and extracted the contour feature points with the corner detection algorithm of Curvature Scale Space(CSS). Then

the global shape context descriptors and the local histogram of edge orientation descriptors were established to describe the features; theseareuseful forreliable feature matching. The matched feature pairs from different times were reserved in a reservoir based on the Gaussian distance criterion. Finally

to overcome the influence of target depth variationin almost-planar scenes

the loss function of the registration matrix was calculated by incorporating the strategy of randomly sampling foreground samples

after which the global registration matrix was updated. The method was validated using the LITIV dataset

and the results demonstrate that the proposed method outperforms state-of-the-art methods. The average overlap error of our method on nine test sequences is only 0.194;this value for the compared methods demonstrate a decrease of 18.5%. This essentially satisfies the precise requirement of infrared-visible video registration in almost-planar scenes

and this method is fairly robust.

关键词

Keywords

references

Y YANG , Z NIE , S HUANG , 等 . Multi-level features convolutional neural network for multi-focus image fusion . IEEE Transactions on Computational Imaging , 2019 . 5 ( 2 ): 262 - 273 . DOI: 10.1109/TCI.2018.2889959 http://doi.org/10.1109/TCI.2018.2889959 .

NB CHANG , K BAI , S IMEN , 等 . Multisensor satellite image fusion and networking for all-weather environmental monitoring . IEEE Systems Journal , 2018 . 12 ( 2 ): 1341 - 1357 . DOI: 10.1109/JSYST.2016.2565900 http://doi.org/10.1109/JSYST.2016.2565900 .

EL-HOSENY HM, RABAIE ESME, ELRAHMAN WA, et al .. Medical image fusion techniques based on combined discrete transform domains [C]. IEEE 34th National Radio Science Conference, Alexandria, Egypt: NRSC , 2017: 471-480.

J MA , Y MA , C LI . Infrared and visible image fusion methods and applications: A survey . Information Fusion , 2018 . 45 153 - 178 . http://cn.bing.com/academic/profile?id=f98ae899b89b135a522486601ff17c11&encoded=0&v=paper_preview&mkt=zh-cn http://cn.bing.com/academic/profile?id=f98ae899b89b135a522486601ff17c11&encoded=0&v=paper_preview&mkt=zh-cn .

SJ KROTOSKY , MM TRIVEDI . Person surveillance using visual and infrared imagery . IEEE Transactions on Circuits & Systems for Video Technology , 2008 . 18 ( 8 ): 1096 - 1105 . http://d.old.wanfangdata.com.cn/NSTLQK/NSTL_QKJJ024146246/ http://d.old.wanfangdata.com.cn/NSTLQK/NSTL_QKJJ024146246/ .

J MA , J ZHAO , Y MA , 等 . Non-rigid visible and infrared face registration via regularized Gaussian fields criterion . Pattern Recognition , 2015 . 48 ( 3 ): 772 - 784 . http://d.old.wanfangdata.com.cn/NSTLQK/NSTL_QKJJ0234287840/ http://d.old.wanfangdata.com.cn/NSTLQK/NSTL_QKJJ0234287840/ .

B ZITOVA , J FLUSSER . Image registration methods: a survey . Image & Vision Computing , 2003 . 21 ( 11 ): 977 - 1000 . http://d.old.wanfangdata.com.cn/Periodical/jsjgcyyy200511059 http://d.old.wanfangdata.com.cn/Periodical/jsjgcyyy200511059 .

P VIOLA , W M W. VIOLA PA Ⅲ , WM WELLS . Alignment by maximization of mutual information . International Journal of Computer Vision , 1997 . 24 ( 2 ): 137 - 154 . http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=ebc82ea41912813953d4150e177b31be http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=ebc82ea41912813953d4150e177b31be .

ROCHE A, MALANDAIN G, PENNEC X, et al .. The correlation ratio as a new similarity measure for multimodal image registration [C]. Proceedings of the Springer International Conference on Medical Image computing and Computer-Assisted Intervention, Cambridge, MA, USA, 1998: 1115-1124.

P A LEGG , P L RDSIN , D MARSHALL , 等 . Feature neighbourhood mutual information for multi-modal image registration: an application to eye fundus imaging . Pattern Recognition , 2015 . 48 ( 6 ): 1937 - 1946 . DOI: 10.1016/j.patcog.2014.12.014 http://doi.org/10.1016/j.patcog.2014.12.014 .

车芳 , 韩俊刚 , 郭志全 . SMT-PAAG下的Harris角点检测与匹配算法 . 电子技术应用 , 2017 . 43 ( 4 ): 138 - 140 . http://d.old.wanfangdata.com.cn/Periodical/dzjsyy201704036 http://d.old.wanfangdata.com.cn/Periodical/dzjsyy201704036 .

F CHE , J G HAN , ZH Q GUO . Harris corner detection and matching algorithm at the SMT-PAAG . Application of Electronic Technique , 2017 . 43 ( 4 ): 138 - 140 . http://d.old.wanfangdata.com.cn/Periodical/dzjsyy201704036 http://d.old.wanfangdata.com.cn/Periodical/dzjsyy201704036 .

耿庆田 , 赵浩宇 , 王宇婷 , 等 . 基于改进SIFT特征提取的车标识别 . 光学精密工程 , 2018 . 26 ( 5 ): 269 - 276 . http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201805028 http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201805028 .

Q T GENG , H Y ZHAO , Y T WANG , 等 . A vehicle logo recognition algorithm based on the improved SIFT feature . Opt. Precision Eng. , 2018 . 26 ( 5 ): 269 - 276 . http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201805028 http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201805028 .

Z L SONG , J ZHANG . Remote sensing image registration based on retrofitted SURF algorithm and trajectories generated from lissajous figures . IEEE Geoscience & Remote Sensing Letters , 2010 . 7 ( 3 ): 491 - 495 . http://cn.bing.com/academic/profile?id=278d251a2f5f0f68034d65eae220a90d&encoded=0&v=paper_preview&mkt=zh-cn http://cn.bing.com/academic/profile?id=278d251a2f5f0f68034d65eae220a90d&encoded=0&v=paper_preview&mkt=zh-cn .

KLIMASZEWSKI J, KONDEJ M, KAWECKI M, et al .. Registration of Infrared and Visible Images Based on Edge Extraction and Phase Correlation Approaches [M]. Berlin: Springer Berlin Heidelberg: Image Processing and Communications Challenges 4, 2013.

C MING , P WONKA , A RAZDAN , 等 . A new image registration scheme based on curvature scale space curve matching . Visual Computer , 2007 . 23 ( 8 ): 607 - 618 . DOI: 10.1007/s00371-007-0164-1 http://doi.org/10.1007/s00371-007-0164-1 .

于晓明 , 李思颖 , 史胜楠 . 混合高斯融合三帧差的运动目标检测改进算法 . 红外技术 , 2019 . 41 ( 3 ): 256 - 261 . http://d.old.wanfangdata.com.cn/Periodical/hwjs201903010 http://d.old.wanfangdata.com.cn/Periodical/hwjs201903010 .

X M YU , S Y LI , SH N SHI . An improved algorithm for moving target detection using a gaussian mixture with three-frame difference . Infrared technology , 2019 . 41 ( 3 ): 256 - 261 . http://d.old.wanfangdata.com.cn/Periodical/hwjs201903010 http://d.old.wanfangdata.com.cn/Periodical/hwjs201903010 .

ST-CHARLES P L, BILODEAU G A, BERGEVIN R. A self-adjusting approach to change detection based on background word consensus [C]. IEEE Winter Conference on Applications of Computer Vision, Waikoloa, HI, USA: IWCACV , 2015: 990-997.

王洪庆 , 许廷发 , 孙兴龙 , 等 . 目标运动轨迹匹配式的红外-可见光视频自动配准 . 光学精密工程 , 2018 . 26 ( 6 ): 1533 - 1541 . http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201806028 http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201806028 .

H Q WANG , T F XU , X L SUN , 等 . Infrared-visible video registration with matching motion trajectories of targets . Opt. Precision Eng. , 2018 . 26 ( 6 ): 1533 - 1541 . http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201806028 http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201806028 .

A TORABI , G MASSE , G A BILODEAU . Feedback scheme for thermal-visible video registration, sensor fusion, and people tracking . Journal of Biomedical Materials Research Part A , 2010 . 2 ( 1 ): 15 - 22 . http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=CC0212365299 http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=CC0212365299 .

SONN S, BILODEAU GA, GALINIER P. Fast and accurate registration of visible and infrared videos [C]. IEEE Conference on Computer Vision and Pattern Recognition Workshops, Portland, OR, USA: CVPRW , 2013: 308-313.

ST-CHARLES P L, BILODEAU G A, BERGEVIN R. Online multimodal video registration based on shape matching [C]. IEEE Conference on Computer Vision and Pattern Recognition Workshops, Boston, MA, USA: CVPRW, 2015: 26-34.

X SUN , T XU , J ZHANG , 等 . A hierarchical framework combining motion and feature information for infrared-visible Video registration . Sensors , 2017 . 17 ( 2 ): 384 http://cn.bing.com/academic/profile?id=9a78c4cac7fffc83e4b10a0a20abc563&encoded=0&v=paper_preview&mkt=zh-cn http://cn.bing.com/academic/profile?id=9a78c4cac7fffc83e4b10a0a20abc563&encoded=0&v=paper_preview&mkt=zh-cn .

樊彦国 , 柴江龙 , 许明明 , 等 . 基于ORB与RANSAC融合改进的图像配准 . 光学精密工程 , 2019 . 27 ( 3 ): 197 - 212 . http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201903022 http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201903022 .

Y G FAN , J L CHAI , M M XU , 等 . Improved fast Image registration algorithm based on ORB and RANSAC fusion . Opt. Precision Eng. , 2019 . 27 ( 3 ): 197 - 212 . http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201903022 http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201903022 .

浏览量

552

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

目标运动轨迹匹配式的红外-可见光视频自动配准

变分近似解算KL散度红外可见光遥感图像配准

多光谱图像融合的IC器件表面缺陷检测

弱纹理环境下融合点线特征的双目视觉同步定位与建图

针对目标遮挡的自适应特征匹配网络