Trajectory detection of small targets based on convolutional long short-term memory with attention mechanisms

Qi-li YANG; Bing-hong ZHOU; Wei ZHENG; Ming-tao LI

doi:10.37188/OPE.20202811.2535

您当前的位置：

首页 >

文章列表页 >

Trajectory detection of small targets based on convolutional long short-term memory with attention mechanisms

Monitoring and Early Warning of NEOs | 更新时间：2020-12-09

- Trajectory detection of small targets based on convolutional long short-term memory with attention mechanisms
- Optics and Precision Engineering Vol. 28, Issue 11, Pages: 2535-2548(2020)
- 作者机构：
  
  1.中国科学院国家空间科学中心, 北京 100190
  2.中国科学院大学工程科学学院, 北京 100049
- 作者简介：
- 基金信息：
- DOI：10.37188/OPE.20202811.2535
  CLC： TP391.4
- Received：10 March 2020，
  
  Revised：31 March 2020，
  
  Accepted：31 March 2020，
  
  Published：25 November 2020
- 稿件说明：
移动端阅览
Qi-li YANG, Bing-hong ZHOU, Wei ZHENG, et al. Trajectory detection of small targets based on convolutional long short-term memory with attention mechanisms[J]. Optics and precision engineering, 2020, 28(11): 2535-2548.
DOI：

Qi-li YANG, Bing-hong ZHOU, Wei ZHENG, et al. Trajectory detection of small targets based on convolutional long short-term memory with attention mechanisms[J]. Optics and precision engineering, 2020, 28(11): 2535-2548. DOI： 10.37188/OPE.20202811.2535.

摘要

红外弱小目标的轨迹检测是红外导引的一项关键技术，在小天体探测、导弹制导和战场侦察等航空航天领域具有重要地位。针对传统基于检测前跟踪技术的轨迹搜索算法存在需要事先获取目标的灰度分布函数或运动速度等先验知识限定条件的问题，提出了基于注意力机制卷积长短时记忆神经网络的弱小目标轨迹检测算法。这种算法通过3D卷积核提取连续15帧红外图像序列的短期时间维信息和空间维信息，结合卷积长短时记忆网络提取红外序列的长期时空信息，利用注意力机制关注与弱小目标运动轨迹有关的关键信息并舍弃无关信息，实现了网络端对端的预测输出。在5个红外图像序列上进行了均方根误差、平均绝对误差、峰值信噪比和结构相似度等4个客观度量指标的实验评估。结果表明，基于输出门注意力机制的卷积长短时记忆网络在均方根误差和平均绝对误差上相对于3DCNN，3D-ConvLSTM，3D-AIConvLSTM方法平均降低了32.8%和46.3%，在峰值信噪比和结构相似度指标上平均提高了18.3%和4.3%，能够优秀地检测低于6 pixel红外目标的运动轨迹，预测轨迹与真实轨迹非常吻合，且背景杂波残留最少，检测效果最优。

Abstract

Trajectory detection of small infrared targets is important for infrared guidance and is widely used in aerospace for functions such as detection of small celestial bodies

missile guidance

and battlefield reconnaissance. Aiming at the traditional trajectory-detection algorithm based on track-before-detect technology

which requires prior knowledge of the target

such as the gray distribution or target speed

this paper proposes a convolutional long short-term memory method based on the attention mechanism in neural networks to detect weak target trajectories. This method involves the use of 3D convolutional kernels to extract short-term temporal and spatial information and combines the convolutional long short-term memory network to extract long-term spatial-temporal information of 15 consecutive infrared image sequences. It also uses the attention mechanism to focus on the key information related to the trajectory and discard irrelevant background information. This method enables the network to realize end-to-end prediction. Four objective indicators such as root mean square error

mean absolute error

peak signal-to-noise ratio

and structural similarity index were applied to three infrared sequences for an experimental evaluation. The experimental results indicated that

compared with 3DCNN

3D-ConvLSTM

and 3D-AIConvLSTM

3D-AOConvLSTM achieved an average reduction of 32.8% and 46.3% in the root mean square error and average absolute error

respectively

and an average increase of 18.3% and 4.3% in peak signal-to-noise and structural similarity

respectively. It can detect the trajectory of an infrared target with fewer than six pixels. The predicted trajectory is highly consistent with the actual trajectory. The proposed method provides the least background clutter and the best detection result.

关键词

Keywords

references

李翠芸 , 姬红兵 . Rao-Balckwellized粒子滤波的红外多个弱目标检测前跟踪 . 光学精密工程 , 2009 . 17 ( 9 ): 2342 - 2349 . http://ope.lightpublishing.cn/thesisDetails?columnId=1749990&Fpath=&index=-1&l=zh http://ope.lightpublishing.cn/thesisDetails?columnId=1749990&Fpath=&index=-1&l=zh .

C Y LI , H B JI . Track-before-detection for multi weak targets based on Rao-Blackwellized particle filter . Opt. Precision Eng. , 2009 . 17 ( 9 ): 2342 - 2349 . http://ope.lightpublishing.cn/thesisDetails?columnId=1749990&Fpath=&index=-1&l=zh http://ope.lightpublishing.cn/thesisDetails?columnId=1749990&Fpath=&index=-1&l=zh .

S D BLOSTEIN , T S HUANG . Detecting small moving objects in image sequences using sequential hypothesis testing . IEEE Transactions on Signal Processing , 1991 . 39 ( 7 ): 1611 - 1629 . DOI: 10.1109/78.134399 http://doi.org/10.1109/78.134399 .

R J LIOU , M R AZIMI-SADJADI . Dim target detection using high order correlation method . IEEE Transactions on Aerospace and Electronic Systems , 1993 . 29 ( 3 ): 841 - 856 . DOI: 10.1109/7.220935 http://doi.org/10.1109/7.220935 .

Y BARNIV , O KELLA . Dynamic programming solution for detecting dim moving targets . IEEE Transactions on Aerospace and Electronic Systems , 1987 . 23 ( 6 ): 776 - 788 . http://ieeexplore.ieee.org/document/4104027 http://ieeexplore.ieee.org/document/4104027 .

I S REED , R M GAGLIARDI , H M SHAO . Application of three-dimensional filtering to moving target detecton . IEEE Transactions on Aerospace and Electronic Systems , 1983 . 19 ( 6 ): 898 - 905 . http://ieeexplore.ieee.org/document/4102882 http://ieeexplore.ieee.org/document/4102882 .

KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[C]. Proceedings of the 25 th International Conference on Neural Information Processing Systems , San Francisco : Curran Associates Inc ., 2012: 1097-1105.

SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[C]. Proceedings of 2015 International Conference on Learning Representations , New York : Computing Research Repository , 2015: 1-14.

SZEGEDY C, LIU W, JIA Y, et al . Going deeper with convolutions[C]. Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition , New York : IEEE , 2015: 1-9.

IRSOY O, CARDIE C. Opinion mining with deep recurrent neural networks[C]. Proceedings of Conference on Empirical Methods in Natural Language Processing , Stroudsburg : Association for Computational Linguistics , 2014: 720-728.

S HOCHREITER , J SCHMIDHUBER . Long short-term memory . Neural Computation , 1997 . 9 ( 8 ): 1735 - 1780 . DOI: 10.1162/neco.1997.9.8.1735 http://doi.org/10.1162/neco.1997.9.8.1735 .

SHI X J, CHEN Z R, WANG H, et al .. Convolutional LSTM network: a machine learning approach for precipitation nowcasting[C]. Proceedings of the 28 th International Conference on Neural Information Processing Systems , Cambridge : MIT Press , 2015: 802-810.

ZHANG L, ZHU G M, MEI L, et al .. Attention in convolutional LSTM for gesture recognition[C]. Proceedings of the 32 st Conference on Neural Information Processing Systems , Cambridge : MIT Press , 2018: 1957-1966.

谢学立 , 李传祥 , 杨小冈 , 等 . 双注意力循环卷积显著性目标检测算法 . 光学学报 , 2019 . 39 ( 9 ): 0915005 http://www.cnki.com.cn/Article/CJFDTotal-GXXB201909032.htm http://www.cnki.com.cn/Article/CJFDTotal-GXXB201909032.htm .

X L XIE , CH X LI , X G YANG , 等 . Salient object detection algorithm based on dual-attention recurrent convolution . Acta Optica Sinica , 2019 . 39 ( 9 ): 0915005 http://www.cnki.com.cn/Article/CJFDTotal-GXXB201909032.htm http://www.cnki.com.cn/Article/CJFDTotal-GXXB201909032.htm .

Z Y LI , K GAVRILYUK , E GAVVES , 等 . Videolstm convolves, attends and flows for action recognition . Computer Vision and Image Understanding , 2018 . 166 41 - 50 . DOI: 10.1016/j.cviu.2017.10.011 http://doi.org/10.1016/j.cviu.2017.10.011 .

WANG Y B, LONG M S, WANG J M, et al .. PredRNN: recurrent neural networks for predictive learning using spatiotemporal lstms[C]. Proceedings of the 31 st Conference on Neural Information Processing Systems , Cambridge : MIT Press , 2017: 879-888.

KARPATHY A, TODERICI G, SHETTY S, et al .. Large-scale video classification with convolutional neural networks[C]. Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition , New York : IEEE , 2003: 1725-1732.

S W JI , W XU , M YANG , 等 . 3D convolutional neural networks for human action recognition . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2013 . 35 ( 1 ): 221 - 231 . DOI: 10.1109/TPAMI.2012.59 http://doi.org/10.1109/TPAMI.2012.59 .

吕晓琪 , 吴凉 , 谷宇 , 等 . 基于三维卷积神经网络的低剂量CT肺结节检测 . 光学精密工程 , 2018 . 26 ( 5 ): 1211 - 1218 . http://ope.lightpublishing.cn/thesisDetails?columnId=1415633&Fpath=&index=-1&l=zh http://ope.lightpublishing.cn/thesisDetails?columnId=1415633&Fpath=&index=-1&l=zh .

X Q LÜ , L WU , Y GU , 等 . Detection of low dose CT pulmonary modules based on 3D convolution neural network . Opt. Precision Eng. , 2018 . 26 ( 5 ): 1211 - 1218 . http://ope.lightpublishing.cn/thesisDetails?columnId=1415633&Fpath=&index=-1&l=zh http://ope.lightpublishing.cn/thesisDetails?columnId=1415633&Fpath=&index=-1&l=zh .

潘仙张 , 张石清 , 郭文平 . 多模深度卷积神经网络应用于视频表情识别 . 光学精密工程 , 2019 . 27 ( 4 ): 963 - 970 . http://ope.lightpublishing.cn/thesisDetails?columnId=1425472&Fpath=&index=-1&l=zh http://ope.lightpublishing.cn/thesisDetails?columnId=1425472&Fpath=&index=-1&l=zh .

X ZH PAN , SH Q ZHANG , W P GUO . Video-based facial expression recognition using multimodal deep convolutional neural networks . Opt. Precision Eng. , 2019 . 27 ( 4 ): 963 - 970 . http://ope.lightpublishing.cn/thesisDetails?columnId=1425472&Fpath=&index=-1&l=zh http://ope.lightpublishing.cn/thesisDetails?columnId=1425472&Fpath=&index=-1&l=zh .

王中宇 , 倪显扬 , 尚振东 . 利用卷积神经网络的自动驾驶场景语义分割 . 光学精密工程 , 2019 . 27 ( 11 ): 2429 - 2438 . http://ope.lightpublishing.cn/thesisDetails?columnId=1453147&Fpath=&index=-1&l=zh http://ope.lightpublishing.cn/thesisDetails?columnId=1453147&Fpath=&index=-1&l=zh .

ZH Y WANG , X Y NI , ZH D SHANG . Autonomous driving semantic segmentation with convolution neural networks . Opt. Precision Eng. , 2019 . 27 ( 11 ): 2429 - 2438 . http://ope.lightpublishing.cn/thesisDetails?columnId=1453147&Fpath=&index=-1&l=zh http://ope.lightpublishing.cn/thesisDetails?columnId=1453147&Fpath=&index=-1&l=zh .

Views

521

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Underwater image enhancement based on multi-branch residual attention network

Cross-modality image matching algorithm based on policy gradient and pseudo-twin network

Shape adaptive feature aggregation network for point cloud classification and segmentation

Real-time super-resolution for infrared dynamic object video based on airborne platform

Multimode composite guidance data fusion method and its experimental validation

Related Author

CHENG Zhuming

LI Jiaxuan

HUANG San'ao

HAN Lichao

WANG Peizhen

ZHANG Jian

LIANG Ao

HUA Haiyang

Related Institution

School of Electrical and Information Engineering， Anhui University of Technology， Maanshan

Key Laboratory of Opto-Electronic Information Processing，Chinese Academy of Sciences

Shenyang Institute of Automation， Chinese Academy of Sciences

University of Chinese Academy of Sciences

Key Laboratory of Optoelectronic Technology and System， Ministry of Education， Chongqing University

AI问答

Address：No.3888 Dong Nanhu Road, Changchun, Jilin, China Postal code：130033
Tel：0431-86176855 Email：gxjmgc@ciomp.ac.cn
Technical support is provided by Beijing Founder electronics co., LTD 吉ICP备11002662号-17 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰