Underwater polarization image fusion based on unsupervised learning and attention mechanisms

GONG Wenzhe; CHU Jinkui; CHENG Haoyuan; ZHANG Ran

doi:10.37188/OPE.20233121.3212

您当前的位置：

首页 >

文章列表页 >

Underwater polarization image fusion based on unsupervised learning and attention mechanisms

Information Sciences | 更新时间：2023-11-16

- Underwater polarization image fusion based on unsupervised learning and attention mechanisms
- Optics and Precision Engineering Vol. 31, Issue 21, Pages: 3212-3220(2023)
- 作者机构：
  
  大连理工大学辽宁省微纳米系统重点实验室，辽宁大连 116024
- 作者简介：
- 基金信息：
- DOI：10.37188/OPE.20233121.3212
  CLC： TN29;TP391.4
- Received：23 February 2023，
  
  Revised：13 April 2023，
  
  Published：10 November 2023
- 稿件说明：
移动端阅览
巩文哲,褚金奎,成昊远等.基于无监督学习和注意力机制的水下偏振图像融合[J].光学精密工程,2023,31(21):3212-3220.

GONG Wenzhe,CHU Jinkui,CHENG Haoyuan,et al.Underwater polarization image fusion based on unsupervised learning and attention mechanisms[J].Optics and Precision Engineering,2023,31(21):3212-3220.
巩文哲,褚金奎,成昊远等.基于无监督学习和注意力机制的水下偏振图像融合[J].光学精密工程,2023,31(21):3212-3220. DOI： 10.37188/OPE.20233121.3212.

GONG Wenzhe,CHU Jinkui,CHENG Haoyuan,et al.Underwater polarization image fusion based on unsupervised learning and attention mechanisms[J].Optics and Precision Engineering,2023,31(21):3212-3220. DOI： 10.37188/OPE.20233121.3212.

摘要

针对光线在水体中传播会受到吸收和散射影响，仅使用传统的强度相机获取水下图像存在成像结果亮度偏低、图像模糊、细节丢失等问题，将深度融合网络应用于水下偏振图像，即用深度学习的方法将水下偏振度图像与光强图像融合。分析水下主动偏振成像模型，搭建实验装置获取水下偏振图像构建训练数据集，并进行适当变换以扩充数据集。构建基于无监督学习和注意力机制引导的用于融合偏振度和光强图像的端到端学习框架，并对损失函数及权重参数进行阐述。使用制作的数据集在不同的损失权重参数下进行网络训练，基于不同的图像评价指标对融合后的图像进行评估。实验结果表明，融合后的水下图像细节更为丰富，相比于光强图像信息熵提升24.48%，标准差提升139%。相比于其他传统融合算法，该方法不需要人工调节权重参数，运算速度较快，具有较强的稳定性和自适应性，对于海洋探测、水下目标识别等领域的应用研究具有重要意义。

Abstract

As light propagation in water is subject to absorption and scattering effects， acquiring underwater images using conventional intensity cameras can result in low brightness of imaging results， blurred images， and loss of details. In this study， a deep fusion network was applied to underwater polarimetric images； the underwater polarimetric images were fused with light-intensity images using deep learning. First， the underwater active polarization imaging model was analyzed， an experimental setup was built to obtain underwater polarization images to construct a training dataset， and appropriate transformations were performed to expand the dataset. Second， an end-to-end learning framework was constructed based on unsupervised learning and guided by attention mechanism for fusing polarimetric and light intensity images and the loss function and weight parameters were elaborated. Finally， the produced dataset was used to train the network under different loss weight parameters and the fused images were evaluated based on different image evaluation metrics. The experimental results show that the fused underwater images are more detailed， with 24.48% higher information entropy and 139% higher standard deviation than light-intensity images. Unlike other traditional fusion algorithms， the method does not require manual weight parameter adjustment， has faster operation speed， strong robustness， and self-adaptability， which is important for ocean detection and underwater target recognition.

关键词

Keywords

references

LI C Y ， GUO C L ， REN W Q ， et al . An underwater image enhancement benchmark dataset and beyond ［J］. IEEE Transactions on Image Processing ， 2020 ， 29 ： 4376 - 4389 . doi: 10.1109/tip.2019.2955241 http://dx.doi.org/10.1109/tip.2019.2955241

GONZÁLEZ-GARCÍA J ， GÓMEZ-ESPINOSA A ， CUAN-URQUIZO E ， et al . Autonomous underwater vehicles： localization， navigation， and communication for collaborative missions ［J］. Applied Sciences ， 2020 ， 10 （ 4 ）： 1256 . doi: 10.3390/app10041256 http://dx.doi.org/10.3390/app10041256

褚金奎，林木音，王寅龙，等 . 偏振光传感器的无人船导航与编队应用［J］. 光学精密工程， 2020 ， 28 （ 8 ）： 1661 - 1669 .

CHU J K ， LIN M Y ， WANG Y L ， et al . Application of polarization sensor to unmanned surface vehicle navigation and formation ［J］. Opt. Precision Eng. ， 2020 ， 28 （ 8 ）： 1661 - 1669 . （in Chinese）

张然，桂心远，成昊远，等 . 基于偏振成像的低光照强背景噪声下的目标位姿估计［J］. 光学精密工程， 2021 ， 29 （ 4 ）： 647 - 655 . doi: 10.37188/ope.20212904.0647 http://dx.doi.org/10.37188/ope.20212904.0647

ZHANG R ， GUI X Y ， CHENG H Y ， et al . Target pose estimation based on polarization imaging in low light and strong background noise ［J］. Opt. Precision Eng. ， 2021 ， 29 （ 4 ）： 647 - 655 . （in Chinese） . doi: 10.37188/ope.20212904.0647 http://dx.doi.org/10.37188/ope.20212904.0647

CHENG H Y ， CHU J K ， ZHANG R ， et al . Real-time position and attitude estimation for homing and docking of an autonomous underwater vehicle based on bionic polarized optical guidance ［J］. Journal of Ocean University of China ， 2020 ， 19 （ 5 ）： 1042 - 1050 . doi: 10.1007/s11802-020-4399-z http://dx.doi.org/10.1007/s11802-020-4399-z

韩平丽 . 水下目标偏振成像探测技术研究［D］. 西安：西安电子科技大学， 2018 . doi: 10.7498/aps.67.20172009 http://dx.doi.org/10.7498/aps.67.20172009

HAN P L . Research on Polarization Imaging Detection Technology of Underwater Target ［D］. Xi'an ： Xidian University ， 2018 . （in Chinese） . doi: 10.7498/aps.67.20172009 http://dx.doi.org/10.7498/aps.67.20172009

褚金奎，张培奇，成昊远，等 . 基于特定偏振态成像的水下图像去散射方法［J］. 光学精密工程， 2021 ， 29 （ 5 ）： 1207 - 1215 . doi: 10.37188/OPE.20212905.1207 http://dx.doi.org/10.37188/OPE.20212905.1207

CHU J K ， ZHANG P Q ， CHENG H Y ， et al . De-scattering method of underwater image based on imaging of specific polarization state ［J］. Opt. Precision Eng. ， 2021 ， 29 （ 5 ）： 1207 - 1215 . （in Chinese） . doi: 10.37188/OPE.20212905.1207 http://dx.doi.org/10.37188/OPE.20212905.1207

HUANG B J ， LIU T G ， HU H F ， et al . Underwater image recovery considering polarization effects of objects ［J］. Optics Express ， 2016 ， 24 （ 9 ）： 9826 - 9838 . doi: 10.1364/oe.24.009826 http://dx.doi.org/10.1364/oe.24.009826

TREIBITZ T ， SCHECHNER Y Y . Active polarization descattering ［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence ， 2009 ， 31 （ 3 ）： 385 - 399 . doi: 10.1109/tpami.2008.85 http://dx.doi.org/10.1109/tpami.2008.85

TOET A . Image fusion by a ratio of low-pass pyramid ［J］. Pattern Recognition Letters ， 1989 ， 9 （ 4 ）： 245 - 253 . doi: 10.1016/0167-8655(89)90003-2 http://dx.doi.org/10.1016/0167-8655(89)90003-2

PAJARES G ， MANUEL DE LA CRUZ J . A wavelet-based image fusion tutorial ［J］. Pattern Recognition ， 2004 ， 37 （ 9 ）： 1855 - 1872 . doi: 10.1016/j.patcog.2004.03.010 http://dx.doi.org/10.1016/j.patcog.2004.03.010

CANDÈS E J ， LEIGH DONOHO D . Ridgelets ： Theory and Applications ［M］. Stanford University ， 1998 .

NENCINI F ， GARZELLI A ， BARONTI S ， et al . Remote sensing image fusion using the curvelet transform ［J］. Information Fusion ， 2007 ， 8 （ 2 ）： 143 - 156 . doi: 10.1016/j.inffus.2006.02.001 http://dx.doi.org/10.1016/j.inffus.2006.02.001

RAHMANI S ， STRAIT M ， MERKURJEV D ， et al . An adaptive IHS pan-sharpening method ［J］. IEEE Geoscience and Remote Sensing Letters ， 2010 ， 7 （ 4 ）： 746 - 750 . doi: 10.1109/lgrs.2010.2046715 http://dx.doi.org/10.1109/lgrs.2010.2046715

XIE H Q ， LIU Y F ， THUERING T ， et al . Photon-counting spectral CT with de-noised principal component analysis （PCA）［C］. 15th International Meeting on Fully Three-Dimensional Image Reconstruction in Radiology and Nuclear Medicine. June 2 - 6 ， 2019 . Philadelphia， USA. SPIE ， 2019 ， 11072 ： 177 - 181 .

LIU Y ， CHEN X ， PENG H ， et al . Multi-focus image fusion with a deep convolutional neural network ［J］. Information Fusion ， 2017 ， 36 ： 191 - 207 . doi: 10.1016/j.inffus.2016.12.001 http://dx.doi.org/10.1016/j.inffus.2016.12.001

KALANTARI N K ， RAMAMOORTHI R . Deep high dynamic range imaging of dynamic scenes ［J］. ACM Transactions on Graphics ， 2017 ， 36 （ 4 ）： 1 - 12 . doi: 10.1145/3072959.3073609 http://dx.doi.org/10.1145/3072959.3073609

MA J ， MA Y ， LI C . Infrared and visible image fusion methods and applications： a survey ［J］. Information Fusion ， 2019 ， 45 ： 153 - 178 . doi: 10.1016/j.inffus.2018.02.004 http://dx.doi.org/10.1016/j.inffus.2018.02.004

刘志豪，金伟其，李力，等 . 四波段共光轴成像实验平台及其图像融合［J］. 光学精密工程， 2022 ， 30 （ 1 ）： 1 - 11 . doi: 10.37188/OPE.20223001.0001 http://dx.doi.org/10.37188/OPE.20223001.0001

LIU ZH H ， JIN W Q ， LI L ， et al . Four-band coaxial imaging experimental platform and its image fusion methods ［J］. Optics and Precision Engineering ， 2022 ， 30 （ 1 ）： 1 - 11 . （in Chinese） . doi: 10.37188/OPE.20223001.0001 http://dx.doi.org/10.37188/OPE.20223001.0001

丁贵鹏，陶钢，李英超，等 . 基于非下采样轮廓波变换与引导滤波器的红外及可见光图像融合［J］. 兵工学报， 2021 ， 42 （ 9 ）： 1911 - 1922 . doi: 10.3969/j.issn.1000-1093.2021.09.012 http://dx.doi.org/10.3969/j.issn.1000-1093.2021.09.012

DING G P ， TAO G ， LI Y CH ， et al . Infrared and visible images fusion based on non-subsampled contourlet transform and guided filter ［J］. Acta Armamentarii ， 2021 ， 42 （ 9 ）： 1911 - 1922 . （in Chinese） . doi: 10.3969/j.issn.1000-1093.2021.09.012 http://dx.doi.org/10.3969/j.issn.1000-1093.2021.09.012

郭全民，王言，李翰山 . 改进IHS-Curvelet变换融合可见光与红外图像抗晕光方法［J］. 红外与激光工程， 2018 ， 47 （ 11 ）： 432 - 440 . doi: 10.3788/IRLA201847.1126002 http://dx.doi.org/10.3788/IRLA201847.1126002

GUO Q M ， WANG Y ， LI H SH . Anti-halation method of visible and infrared image fusion based on improved IHS-Curvelet transform ［J］. Infrared and Laser Engineering ， 2018 ， 47 （ 11 ）： 432 - 440 . （in Chinese） . doi: 10.3788/IRLA201847.1126002 http://dx.doi.org/10.3788/IRLA201847.1126002

ELZEKI O M ， ABD ELFATTAH M ， SALEM H ， et al . A novel perceptual two layer image fusion using deep learning for imbalanced COVID-19 dataset ［J］. PeerJ Computer Science ， 2021 ， 7 ： e364 . doi: 10.7717/peerj-cs.364 http://dx.doi.org/10.7717/peerj-cs.364

ZHANG J ， SHAO J ， CHEN J ， et al . Polarization image fusion with self-learned fusion strategy ［J］. Pattern Recognition ， 2021 ， 118 ： 108045 . doi: 10.1016/j.patcog.2021.108045 http://dx.doi.org/10.1016/j.patcog.2021.108045

LIU X ， LIU Q ， WANG Y . Remote sensing image fusion based on two-stream fusion network ［J］. Information Fusion ， 2020 ， 55 ： 1 - 15 . doi: 10.1016/j.inffus.2019.07.010 http://dx.doi.org/10.1016/j.inffus.2019.07.010

MCGLAMERY B L . A computer model for underwater camera systems ［C］. Proc SPIE 0208， Ocean Optics VI ， 1980 ， 0208 ： 221 - 231 . doi: 10.1117/12.958279 http://dx.doi.org/10.1117/12.958279

JAFFE J S . Computer modeling and the design of optimal underwater imaging systems ［J］. IEEE Journal of Oceanic Engineering ， 1990 ， 15 （ 2 ）： 101 - 111 . doi: 10.1109/48.50695 http://dx.doi.org/10.1109/48.50695

MA J ， CHEN C ， LI C ， et al . Infrared and visible image fusion via gradient transfer and total variation minimization ［J］. Information Fusion ， 2016 ， 31 ： 100 - 109 . doi: 10.1016/j.inffus.2016.02.001 http://dx.doi.org/10.1016/j.inffus.2016.02.001

NAIDU V P S . Image fusion technique using multi-resolution singular value decomposition ［J］. Defence Science Journal ， 2011 ， 61 （ 5 ）： 479 . doi: 10.14429/dsj.61.705 http://dx.doi.org/10.14429/dsj.61.705

Views

676

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Conditional diffusion and multi-channel high-low frequency parallel fusion of infrared and visible light images

Computed tomography image segmentation of cell pole piece via strip attention mechanism

YOLOv8 model-based additive manufacturing micro porosity defect detection and its dimension measurement

3DRes-ViT knee osteoarthritis classification model based on multimodal fusion

Degradation remote sensing image quality enhancement based on frequency-domain-spatial-domain hybrid attention

Related Author

DI Jing

WANG Heran

LIANG Chan

LIU Jizhao

LIAN Jing

LIU Zefang

LONG Chao

LIU Xueshuan

Related Institution

School of Electronic and Information Engineering， Lanzhou Jiaotong University

School of Information Science and Engineering， Lanzhou University

ICT Research Center， Key Laboratory of Optoelectronic Technology and Systems of Ministry of Education， Chongqing University

College of Optoelectronic Engineering， Chongqing University

Project Management Center， Equipment Department of Rocket Force

Address：No.3888 Dong Nanhu Road, Changchun, Jilin, China Postal code：130033
Tel：0431-86176855 Email：gxjmgc@ciomp.ac.cn
Technical support is provided by Beijing Founder electronics co., LTD 吉ICP备11002662号-17 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰