Infrared-visible video registration with matching motion trajectories of targets

Hong-qing WANG; Xing-long SUN; Xiang-min LI; Tai-hui LIU

doi:10.3788/OPE.20182606.1533

您当前的位置：

首页 >

文章列表页 >

Infrared-visible video registration with matching motion trajectories of targets

Information Sciences | 更新时间：2020-07-05

- Infrared-visible video registration with matching motion trajectories of targets
- Optics and Precision Engineering Vol. 26, Issue 6, Pages: 1533-1541(2018)
- 作者机构：
  
  1.北京理工大学光电学院, 北京 100081
  2.北华大学计算机科学技术学院, 吉林吉林 132013
- 作者简介：
- 基金信息：
- DOI：10.3788/OPE.20182606.1533
  CLC： TP394.1;TH691.9
- Received：02 November 2017，
  
  Accepted：11 December 2017，
  
  Published：25 June 2018
- 稿件说明：
移动端阅览
Hong-qing WANG, Xing-long SUN, Xiang-min LI, et al. Infrared-visible video registration with matching motion trajectories of targets[J]. Optics and precision engineering, 2018, 26(6): 1533-1541.
DOI：

Hong-qing WANG, Xing-long SUN, Xiang-min LI, et al. Infrared-visible video registration with matching motion trajectories of targets[J]. Optics and precision engineering, 2018, 26(6): 1533-1541. DOI： 10.3788/OPE.20182606.1533.

摘要

为实现精确的红外-可见光视频序列的自动配准，提出了一种新的基于目标轨迹线匹配的配准方法。首先，利用运动目标检测技术提取目标前景，并由基于相关滤波器（KCF）的多目标跟踪算法对每个前景顶点进行跟踪，进而获取每个目标的运动轨迹。此后，为每条轨迹线建立归一化运动方向描述子与归一化运动幅度描述子，通过时序分析、方向描述子匹配及幅值描述子匹配建立分步约束的匹配机制，完成轨迹线匹配工作。最后，采用迭代更新的方式获取最佳全局配准矩阵，实现对异源视频的配准。在LITIV数据库上的9组视频上进行测试验证，实验的结果表明：本文配准算法的重叠率误差一般小于0.2，接近或已超过手动的Ground-Truth矩阵。通过充分利用目标的运动信息，该算法实现了精确的红外-可见光图像序列配准。

Abstract

In order to realize accurate and automatic infrared-visible video registration

a novel registration method was proposed based on matching the motion trajectories of targets. First

the top pixel of each foreground was tracked by using the multi-target tracking algorithm based on KCF. In this way

the trajectory of each target was obtained. Then

the normalized motion orientation descriptor and the normalized motion magnitude descriptor were established for each trajectory. The stepwise constraint matching framework was structured by using time analysis

orientation descriptor matching and magnitude descriptor matching. Finally

the best registration matrix was obtained with iterative updating. The method proposed was validated with the nine pairs of videos in the LITIV database. The results indicate that the overlap error of the proposed method is smaller than 0.2

which is close to or better than the manual Ground-Truth matrix. By adequately using the motion information of target

the algorithm can realize precise infrared-visible image sequence registration.

关键词

Keywords

references

HERMOSILLA G, GALLARDO F, FARIAS G, et al.. Fusion of visible and thermal descriptors using genetic algorithms for face recognition systems[J]. Sensors, 2015, 15:17944-17962.

KROTOSKY S J, TRIVEDI M M. Person surveillance using visual and infrared imagery[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2008, 18(8):1096-1105.

MA J, ZHAO J, MA Y, et al.. Non-rigid visible and infrared face registration via regularized Gaussian fields criterion[J]. Pattern Recognition, 2015, 48(3):772-784.

陈继红, 刘素真.可见光与红外CPCT彩色图像融合研究与实现[J].现代电子技术, 2017, 40(9):5-9.

CHEN J H, LIU S Z.Research and implementation of visible-light and infrared CPCT color image fusion[J]. Modern Electronics Technique, 2017, 40(9):5-9.(in Chinese)

ROCHE A, MALANDAIN G, PENNEC X, et al . . The correlation ratio as a new similarity measure for multimodal image registration[C]. Proceedings of the Springer International Conference on Medical Image computing and Computer-Assisted Intervention , Cambridge , MA , USA : MICCAI '98, 1998: 1115-1124. http://www-sop.inria.fr/asclepios/Publications/Roche/miccai98.pdf

VIOLA P, Ⅲ W M W. VIOLA PA, WELLS WM. Alignment by maximization of mutual information[J]. International Journal of Computer Vision, 1997, 24(2):137-154.

LEGG P A, RDSIN P L, MARSHALL D, et al.. Feature neighbourhood mutual information for multi-modal image registration:an application to eye fundus imaging[J]. Pattern Recognition, 2015, 48(6):1937-1946.

车芳, 韩俊刚, 郭志全. SMT-PAAG下的Harris角点检测与匹配算法[J].电子技术应用, 2017, 43(4):138-140.

CHE F, HAN J G, GUO ZH Q. Harris corner detection and matching algorithm at the SMT-PAAG[J]. Application of ElectronicTechnique, 2017, 43(4):138-140.(in Chinese)

张宏伟, 樊祥, 朱斌, 等.引入外点剔除机制的双波段红外图像的配准[J].红外与激光工程, 2015, 44(s1):23-28.

ZHANG HW, FAN X, ZHU B, et al.. Dual-band infrared image registration with the introduction ofoutliers rejection mechanism[J].Infrared and Laser Engineering, 2015, 44(s1):23-28.(in Chinese)

KONG S G, HEO J, BOUGHORBEL F, et al.. Multiscale fusion of visible and thermal IR Images for illumination-invariant face recognition[J]. International Journal of Computer Vision, 2007, 71(2):215-233.

COIRAS E, SANTAMARIA J, MIRAVET C. Segment-based registration technique for visual-infrared images[J]. Optical Engineering, 2000, 39(1):202-7.

BILODEAU G A, TORABI A, MORIN F. Visible and infrared image registration using trajectories and composite foreground images[J]. Image & Vision Computing, 2011, 29(1):41-50.

TORABI A, MASSE G, BILODEAU G A. Feedback scheme for thermal-visible video registration, sensor fusion, and people tracking[J]. Journal of Biomedical Materials Research Part A, 2010, 2(1):15-22.

ST-CHARLES P L, BILODEAU G A, BERGEVINR. A self-adjusting approach to change detection based on background word consensus[C]. IEEE Winter Conference on Applications of Computer Vision , Waikoloa , HI , USA : IWCACV , 2015: 990-997. http://www.polymtl.ca/litiv/doc/StCharlesetalWACV2015.pdf

FUENTES L M, VELATIN S A. People tracking in surveillance applications[J].Image & Vision Computing, 2006, 24(11):1165-1171.

YANG Y, BILODEAU G A. Multiple object tracking with kernelized correlation filters in urban mixed traffic[EB/OL]. (2017-4-24). https://arxiv.org/abs/1611.02364 https://arxiv.org/abs/1611.02364 . (accessed on 24 March 2017).

ST-CHARLES P L, BILODEAU G A, BERGEVIN R. Online multimodal video registration based on shape matching[C]. IEEE Conference on Computer Vision and Pattern Recognition Workshops , Boston , MA , USA : CVPRW , 2015: 26-34.

Views

370

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

IR/visible image registration based on EM iteration of log-likelihood function

Infrared and visible image fusion based on target enhancement and butterfly optimization

Image fusion of dual-discriminator generative adversarial network and latent low-rank representation

Infrared-visible video automatic registration with contour feature matching

Improved rattle snake dual mode cell model for image fusion

Related Author

NIE Hong-bin

HOU Qing-yu

ZHAO Ming

ZHANG Wei

HAO Shuai

LI Tong

MA Xu

HE Tian

Related Institution

Research Center for Space Optical Engineering, Harbin Institute of Technology

College of Electrical and Control Engineering， Xi'an University of Science and Technology

Key Laboratory of Nondestructive Testing （Ministry of Education）， Nanchang Hangkong University

AECC Shenyang Liming Aero-Engine Co.， LTD.

Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences

AI问答

Address：No.3888 Dong Nanhu Road, Changchun, Jilin, China Postal code：130033
Tel：0431-86176855 Email：gxjmgc@ciomp.ac.cn
Technical support is provided by Beijing Founder electronics co., LTD 吉ICP备11002662号-17 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰