Infrared-visible video automatic registration with contour feature matching

Xing-long SUN; Guang-liang HAN; Pei-xun LIU; Ting-fa XU

doi:10.3788/OPE.20202805.1140

您当前的位置：

首页 >

文章列表页 >

Infrared-visible video automatic registration with contour feature matching

Information Sciences | 更新时间：2020-07-09

- Infrared-visible video automatic registration with contour feature matching
- Optics and Precision Engineering Vol. 28, Issue 5, Pages: 1140-1151(2020)
- 作者机构：
  
  1.中国科学院长春光学精密机械与物理研究所, 吉林长春 130033
  2.中国科学院大学, 北京 100049
  3.北京理工大学光电学院, 北京 100081
- 作者简介：
- 基金信息：
- DOI：10.3788/OPE.20202805.1140
  CLC： TP394.1;TH691.9
- Received：11 November 2019，
  
  Revised：29 January 2020，
  
  Accepted：29 January 2020，
  
  Published：15 May 2020
- 稿件说明：
移动端阅览
Xing-long SUN, Guang-liang HAN, Pei-xun LIU, et al. Infrared-visible video automatic registration with contour feature matching[J]. Optics and precision engineering, 2020, 28(5): 1140-1151.
DOI：

Xing-long SUN, Guang-liang HAN, Pei-xun LIU, et al. Infrared-visible video automatic registration with contour feature matching[J]. Optics and precision engineering, 2020, 28(5): 1140-1151. DOI： 10.3788/OPE.20202805.1140.

摘要

为了精确地配准近平面场景下的红外-可见光视频序列

本文提出了一种基于轮廓特征匹配的自动配准方法

通过迭代匹配目标轮廓特征来解决异源图像中配准特征的提取和匹配难题。首先

采用运动目标检测技术获取目标轮廓

并由曲率尺度空间(CSS)角点检测算法提取轮廓特征点。此后

建立全局形状上下文描述子和局部边缘方向直方图描述子描述特征

从而实现可靠的特征匹配。来自不同时刻的匹配点对被保存在一个基于高斯距离准则的特征匹配库中。最后

为了克服近平面场景中目标深度变化的影响

本文结合前景样本随机抽样策略计算配准矩阵的损失函数

完成对全局配准矩阵的更新。在LITIV数据库上对方法进行实验验证

结果表明：本文方法的配准精度优于当前先进的对比方法

在9个测试视频上的平均重叠率误差仅为0.194

与对比方法相比下降了18.5%。基本满足了近平面场景下红外-可见光视频序列配准的精度要求

且具有较高的鲁棒性。

Abstract

To register infrared-visible video sequences precisely inalmost-planar scenes

an automatic registration method based on matching the contour features was proposed in this paper. This method could solve the challenging problem regarding extracting and matching features in multimodal images by iteratively matching the contour features of targets. First

this method adopted the technology of moving target detection to identify the contours of targets and extracted the contour feature points with the corner detection algorithm of Curvature Scale Space(CSS). Then

the global shape context descriptors and the local histogram of edge orientation descriptors were established to describe the features; theseareuseful forreliable feature matching. The matched feature pairs from different times were reserved in a reservoir based on the Gaussian distance criterion. Finally

to overcome the influence of target depth variationin almost-planar scenes

the loss function of the registration matrix was calculated by incorporating the strategy of randomly sampling foreground samples

after which the global registration matrix was updated. The method was validated using the LITIV dataset

and the results demonstrate that the proposed method outperforms state-of-the-art methods. The average overlap error of our method on nine test sequences is only 0.194;this value for the compared methods demonstrate a decrease of 18.5%. This essentially satisfies the precise requirement of infrared-visible video registration in almost-planar scenes

and this method is fairly robust.

关键词

Keywords

references

Y YANG , Z NIE , S HUANG , 等 . Multi-level features convolutional neural network for multi-focus image fusion . IEEE Transactions on Computational Imaging , 2019 . 5 ( 2 ): 262 - 273 . DOI: 10.1109/TCI.2018.2889959 http://doi.org/10.1109/TCI.2018.2889959 .

NB CHANG , K BAI , S IMEN , 等 . Multisensor satellite image fusion and networking for all-weather environmental monitoring . IEEE Systems Journal , 2018 . 12 ( 2 ): 1341 - 1357 . DOI: 10.1109/JSYST.2016.2565900 http://doi.org/10.1109/JSYST.2016.2565900 .

EL-HOSENY HM, RABAIE ESME, ELRAHMAN WA, et al .. Medical image fusion techniques based on combined discrete transform domains [C]. IEEE 34th National Radio Science Conference, Alexandria, Egypt: NRSC , 2017: 471-480.

J MA , Y MA , C LI . Infrared and visible image fusion methods and applications: A survey . Information Fusion , 2018 . 45 153 - 178 . http://cn.bing.com/academic/profile?id=f98ae899b89b135a522486601ff17c11&encoded=0&v=paper_preview&mkt=zh-cn http://cn.bing.com/academic/profile?id=f98ae899b89b135a522486601ff17c11&encoded=0&v=paper_preview&mkt=zh-cn .

SJ KROTOSKY , MM TRIVEDI . Person surveillance using visual and infrared imagery . IEEE Transactions on Circuits & Systems for Video Technology , 2008 . 18 ( 8 ): 1096 - 1105 . http://d.old.wanfangdata.com.cn/NSTLQK/NSTL_QKJJ024146246/ http://d.old.wanfangdata.com.cn/NSTLQK/NSTL_QKJJ024146246/ .

J MA , J ZHAO , Y MA , 等 . Non-rigid visible and infrared face registration via regularized Gaussian fields criterion . Pattern Recognition , 2015 . 48 ( 3 ): 772 - 784 . http://d.old.wanfangdata.com.cn/NSTLQK/NSTL_QKJJ0234287840/ http://d.old.wanfangdata.com.cn/NSTLQK/NSTL_QKJJ0234287840/ .

B ZITOVA , J FLUSSER . Image registration methods: a survey . Image & Vision Computing , 2003 . 21 ( 11 ): 977 - 1000 . http://d.old.wanfangdata.com.cn/Periodical/jsjgcyyy200511059 http://d.old.wanfangdata.com.cn/Periodical/jsjgcyyy200511059 .

P VIOLA , W M W. VIOLA PA Ⅲ , WM WELLS . Alignment by maximization of mutual information . International Journal of Computer Vision , 1997 . 24 ( 2 ): 137 - 154 . http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=ebc82ea41912813953d4150e177b31be http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=ebc82ea41912813953d4150e177b31be .

ROCHE A, MALANDAIN G, PENNEC X, et al .. The correlation ratio as a new similarity measure for multimodal image registration [C]. Proceedings of the Springer International Conference on Medical Image computing and Computer-Assisted Intervention, Cambridge, MA, USA, 1998: 1115-1124.

P A LEGG , P L RDSIN , D MARSHALL , 等 . Feature neighbourhood mutual information for multi-modal image registration: an application to eye fundus imaging . Pattern Recognition , 2015 . 48 ( 6 ): 1937 - 1946 . DOI: 10.1016/j.patcog.2014.12.014 http://doi.org/10.1016/j.patcog.2014.12.014 .

车芳 , 韩俊刚 , 郭志全 . SMT-PAAG下的Harris角点检测与匹配算法 . 电子技术应用 , 2017 . 43 ( 4 ): 138 - 140 . http://d.old.wanfangdata.com.cn/Periodical/dzjsyy201704036 http://d.old.wanfangdata.com.cn/Periodical/dzjsyy201704036 .

F CHE , J G HAN , ZH Q GUO . Harris corner detection and matching algorithm at the SMT-PAAG . Application of Electronic Technique , 2017 . 43 ( 4 ): 138 - 140 . http://d.old.wanfangdata.com.cn/Periodical/dzjsyy201704036 http://d.old.wanfangdata.com.cn/Periodical/dzjsyy201704036 .

耿庆田 , 赵浩宇 , 王宇婷 , 等 . 基于改进SIFT特征提取的车标识别 . 光学精密工程 , 2018 . 26 ( 5 ): 269 - 276 . http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201805028 http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201805028 .

Q T GENG , H Y ZHAO , Y T WANG , 等 . A vehicle logo recognition algorithm based on the improved SIFT feature . Opt. Precision Eng. , 2018 . 26 ( 5 ): 269 - 276 . http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201805028 http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201805028 .

Z L SONG , J ZHANG . Remote sensing image registration based on retrofitted SURF algorithm and trajectories generated from lissajous figures . IEEE Geoscience & Remote Sensing Letters , 2010 . 7 ( 3 ): 491 - 495 . http://cn.bing.com/academic/profile?id=278d251a2f5f0f68034d65eae220a90d&encoded=0&v=paper_preview&mkt=zh-cn http://cn.bing.com/academic/profile?id=278d251a2f5f0f68034d65eae220a90d&encoded=0&v=paper_preview&mkt=zh-cn .

KLIMASZEWSKI J, KONDEJ M, KAWECKI M, et al .. Registration of Infrared and Visible Images Based on Edge Extraction and Phase Correlation Approaches [M]. Berlin: Springer Berlin Heidelberg: Image Processing and Communications Challenges 4, 2013.

C MING , P WONKA , A RAZDAN , 等 . A new image registration scheme based on curvature scale space curve matching . Visual Computer , 2007 . 23 ( 8 ): 607 - 618 . DOI: 10.1007/s00371-007-0164-1 http://doi.org/10.1007/s00371-007-0164-1 .

于晓明 , 李思颖 , 史胜楠 . 混合高斯融合三帧差的运动目标检测改进算法 . 红外技术 , 2019 . 41 ( 3 ): 256 - 261 . http://d.old.wanfangdata.com.cn/Periodical/hwjs201903010 http://d.old.wanfangdata.com.cn/Periodical/hwjs201903010 .

X M YU , S Y LI , SH N SHI . An improved algorithm for moving target detection using a gaussian mixture with three-frame difference . Infrared technology , 2019 . 41 ( 3 ): 256 - 261 . http://d.old.wanfangdata.com.cn/Periodical/hwjs201903010 http://d.old.wanfangdata.com.cn/Periodical/hwjs201903010 .

ST-CHARLES P L, BILODEAU G A, BERGEVIN R. A self-adjusting approach to change detection based on background word consensus [C]. IEEE Winter Conference on Applications of Computer Vision, Waikoloa, HI, USA: IWCACV , 2015: 990-997.

王洪庆 , 许廷发 , 孙兴龙 , 等 . 目标运动轨迹匹配式的红外-可见光视频自动配准 . 光学精密工程 , 2018 . 26 ( 6 ): 1533 - 1541 . http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201806028 http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201806028 .

H Q WANG , T F XU , X L SUN , 等 . Infrared-visible video registration with matching motion trajectories of targets . Opt. Precision Eng. , 2018 . 26 ( 6 ): 1533 - 1541 . http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201806028 http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201806028 .

A TORABI , G MASSE , G A BILODEAU . Feedback scheme for thermal-visible video registration, sensor fusion, and people tracking . Journal of Biomedical Materials Research Part A , 2010 . 2 ( 1 ): 15 - 22 . http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=CC0212365299 http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=CC0212365299 .

SONN S, BILODEAU GA, GALINIER P. Fast and accurate registration of visible and infrared videos [C]. IEEE Conference on Computer Vision and Pattern Recognition Workshops, Portland, OR, USA: CVPRW , 2013: 308-313.

ST-CHARLES P L, BILODEAU G A, BERGEVIN R. Online multimodal video registration based on shape matching [C]. IEEE Conference on Computer Vision and Pattern Recognition Workshops, Boston, MA, USA: CVPRW, 2015: 26-34.

X SUN , T XU , J ZHANG , 等 . A hierarchical framework combining motion and feature information for infrared-visible Video registration . Sensors , 2017 . 17 ( 2 ): 384 http://cn.bing.com/academic/profile?id=9a78c4cac7fffc83e4b10a0a20abc563&encoded=0&v=paper_preview&mkt=zh-cn http://cn.bing.com/academic/profile?id=9a78c4cac7fffc83e4b10a0a20abc563&encoded=0&v=paper_preview&mkt=zh-cn .

樊彦国 , 柴江龙 , 许明明 , 等 . 基于ORB与RANSAC融合改进的图像配准 . 光学精密工程 , 2019 . 27 ( 3 ): 197 - 212 . http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201903022 http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201903022 .

Y G FAN , J L CHAI , M M XU , 等 . Improved fast Image registration algorithm based on ORB and RANSAC fusion . Opt. Precision Eng. , 2019 . 27 ( 3 ): 197 - 212 . http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201903022 http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201903022 .

Views

572

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Infrared-visible video registration with matching motion trajectories of targets

Infrared-visible remote sensing image registration method based on Kullback-Leibler divergence using variational approximation

Multispectral image fusion method for surface defect detection of IC devices

Binocular vision SLAM with fused point and line features in weak texture environment

Adaptive feature matching network for object occlusion

Related Author

Hong-qing WANG

Xing-long SUN

Xiang-min LI

Tai-hui LIU

WANG Jia

WU Hao

FU Ruigang

KONG Lingshuang

Related Institution

College of Computer Science and Technology, Beihua University

College of Electronic Science and Technology， National University of Defense Technology

Institute of Dataspace， Hefei Comprehensive National Science Center

School of Electronic Information and Electrical Engineering， Changsha University

College of Mechanical and Electrical Engineering， Guangdong University of Technology

AI问答

Address：No.3888 Dong Nanhu Road, Changchun, Jilin, China Postal code：130033
Tel：0431-86176855 Email：gxjmgc@ciomp.ac.cn
Technical support is provided by Beijing Founder electronics co., LTD 吉ICP备11002662号-17 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰