Underwater image enhancement synthesizing multi-scale information and attention mechanisms

XIA Xiaohua; ZHONG Yuquan; HU Peng; YAO Yunshi; GENG Jiguang; ZHANG Liangqi

doi:10.37188/OPE.20243210.1582

您当前的位置：

首页 >

文章列表页 >

Underwater image enhancement synthesizing multi-scale information and attention mechanisms

Information Sciences | 更新时间：2024-06-11

- Underwater image enhancement synthesizing multi-scale information and attention mechanisms
- Optics and Precision Engineering Vol. 32, Issue 10, Pages: 1582-1594(2024)
- 作者机构：
  
  1.长安大学道路施工技术与装备教育部重点实验室，陕西西安 710064
  2.河南万里交通科技集团股份有限公司，河南许昌 461000
- 作者简介：
- 基金信息：
- DOI：10.37188/OPE.20243210.1582
  CLC： TP391
- Published：25 May 2024，
  
  Received：14 December 2023，
  
  Revised：20 February 2024，
- 稿件说明：
移动端阅览
夏晓华,钟预全,胡鹏等.综合多尺度信息和注意力机制的水下图像增强[J].光学精密工程,2024,32(10):1582-1594.

XIA Xiaohua,ZHONG Yuquan,HU Peng,et al.Underwater image enhancement synthesizing multi-scale information and attention mechanisms[J].Optics and Precision Engineering,2024,32(10):1582-1594.
夏晓华,钟预全,胡鹏等.综合多尺度信息和注意力机制的水下图像增强[J].光学精密工程,2024,32(10):1582-1594. DOI： 10.37188/OPE.20243210.1582.

XIA Xiaohua,ZHONG Yuquan,HU Peng,et al.Underwater image enhancement synthesizing multi-scale information and attention mechanisms[J].Optics and Precision Engineering,2024,32(10):1582-1594. DOI： 10.37188/OPE.20243210.1582.

摘要

针对水下图像由于水的散射和吸收而存在颜色失真和细节丢失等问题，提出了一种综合多尺度信息和注意力机制的生成对抗网络模型来增强水下图像。首先，为了充分利用和增强图像的局部信息和全局信息，使用局部编码器和全局编码器分别提取图像的局部特征和全局特征，并互相融合以实现互补性。接着，设计多尺度混合卷积来捕捉多尺度信息，增加网络对不同尺度特征的适应性。然后，利用注意力机制增加特征提取的准确性，加强网络对高价值特征的关注度。最后，重复使用多尺度混合卷积和注意力机制进一步细化特征后，逐步上采样得到增强图像。与六种经典和最新的方法相比，提出的模型不仅在主观评价中取得了最好的视觉感受，而且在整个测试集上，峰值信噪比（PSNR）、结构相似指数（SSIM）、水下图像质量指标（UIQM）和自然图像质量（NIQE）四种客观评价指标分别取得了22.499，0.789，2.911和4.175的平均分数，均优于六种对比方法，较对比方法中的最优值分别提升0.353，0.002，0.025和0.307，证明提出的模型不仅能够矫正图像颜色失真，而且在恢复图像细节、增加图像对比度和清晰度等方面均有较好的表现，具有良好的应用前景。

Abstract

Aiming at the problems of color distortion and detail loss in underwater images due to water scattering and absorption， a generative adversarial network model integrating multi-scale information and attention mechanism was proposed to enhance underwater images. Firstly， to fully exploit and enhance both local and global information of the image， local encoders and global encoders were employed to extract local and global features respectively， which were then fused to achieve complementarity. Next， a multi-scale hybrid convolution was designed to capture multi-scale information， increasing the network's adaptability to features at different scales. Subsequently， attention mechanisms were utilized to enhance the accuracy of feature extraction， emphasizing the focus on high-value features. Finally， by iteratively applying multi-scale hybrid convolution and attention mechanisms to refine features， the enhanced image was gradually up-sampled. Compared with the six classical and state-of-the-art methods， the proposed model not only achieved the best visual perception in subjective evaluations but also outperformed the six comparative methods on the entire test set in terms of four objective evaluation metrics peak signal-to-noise ratio （PSNR）， structural similarity （SSIM）， underwater image quality measurement （UIQM）， and natural image quality evaluation （NIQE） with average scores of 22.499， 0.789， 2.911， and 4.175， respectively. The improvements over the best scores among the comparative methods are 0.353， 0.002， 0.025， and 0.307， respectively. These results indicate that the proposed model not only corrects image color distortion but also performs well in restoring image details， increasing image contrast， and enhancing clarity. Therefore， it shows promising prospects for practical applications in underwater image enhancement.

关键词

水下图像增强生成对抗网络编码器多尺度混合卷积注意力机制

Keywords

underwater image enhancementgenerative adversarial networkencodermulti-scale hybrid convolutionattention mechanism

references

郭继昌，李重仪，郭春乐，等. 水下图像增强和复原方法研究进展［J］. 中国图象图形学报， 2017， 22（3）： 273-287. doi: 10.11834/jig.20170301http://dx.doi.org/10.11834/jig.20170301

GUO J C， LI C Y， GUO C L， et al. Research progress of underwater image enhancement and restoration methods［J］. Journal of Image and Graphics， 2017， 22（3）： 273-287.（in Chinese）. doi: 10.11834/jig.20170301http://dx.doi.org/10.11834/jig.20170301

袁国铭，杨光，王金峰，等. 由粗到细的多级小波变换水下图像增强［J］. 光学精密工程， 2022， 30（22）： 2939-2951. doi: 10.37188/ope.20223022.2939http://dx.doi.org/10.37188/ope.20223022.2939

YUAN G M， YANG G， WANG J F， et al. Coarse-to-fine underwater image enhancement based on multi-level wavelet transform［J］. Opt. Precision Eng.， 2022， 30（22）： 2939-2951.（in Chinese）. doi: 10.37188/ope.20223022.2939http://dx.doi.org/10.37188/ope.20223022.2939

林森，刘旭. 门控融合对抗网络的水下图像增强［J］. 图学学报， 2021， 42（6）： 948-956. doi: 10.11996/JG.j.2095-302X.2021060948http://dx.doi.org/10.11996/JG.j.2095-302X.2021060948

LIN S， LIU X. Underwater image enhancement algorithm using gated fusion generative adversarial network［J］. Journal of Graphics， 2021， 42（6）： 948-956.（in Chinese）. doi: 10.11996/JG.j.2095-302X.2021060948http://dx.doi.org/10.11996/JG.j.2095-302X.2021060948

王明军，彭月，刘燕荣，等. 模拟水体湍流环境下目标激光点云数据的三维重建与分析［J］. 光电工程， 2023， 50（6）： 230004.

WANG M J， PENG Y， LIU Y R， et al. Three-dimensional reconstruction and analysis of target laser point cloud data in simulated turbulent water environment［J］. Opto-Electronic Engineering， 2023， 50（6）： 230004.（in Chinese）

GHANI A S A， ISA N A M. Enhancement of low quality underwater image through integrated global and local contrast correction［J］. Applied Soft Computing， 2015， 37（C）： 332-344. doi: 10.1016/j.asoc.2015.08.033http://dx.doi.org/10.1016/j.asoc.2015.08.033

胡振宇，陈琦，朱大奇. 基于颜色平衡和多尺度融合的水下图像增强［J］. 光学精密工程， 2022， 30（17）： 2133-2146. doi: 10.37188/OPE.20223017.2133http://dx.doi.org/10.37188/OPE.20223017.2133

HU Z Y， CHEN Q， ZHU D Q. Underwater image enhancement based on color balance and multi-scale fusion［J］. Opt. Precision Eng.， 2022， 30（17）： 2133-2146.（in Chinese）. doi: 10.37188/OPE.20223017.2133http://dx.doi.org/10.37188/OPE.20223017.2133

张楠楠，李志伟，郭新军，等. 使用改进型大气散射模型的双阶段图像修复［J］. 光学精密工程， 2022， 30（18）： 2267-2279. doi: 10.37188/OPE.20223018.2267http://dx.doi.org/10.37188/OPE.20223018.2267

ZHANG N N， LI Z W， GUO X J， et al. Two-stage image restoration using improved atmospheric scattering model［J］. Opt. Precision Eng.， 2022， 30（18）： 2267-2279.（in Chinese）. doi: 10.37188/OPE.20223018.2267http://dx.doi.org/10.37188/OPE.20223018.2267

丛润民，张禹墨，张晨，等. 深度学习驱动的水下图像增强与复原研究进展［J］. 信号处理， 2020， 36（9）： 1377-1389. doi: 10.16798/j.issn.1003-0530.2020.09.001http://dx.doi.org/10.16798/j.issn.1003-0530.2020.09.001

CONG R M， ZHANG Y M， ZHANG C， et al. Research progress of deep learning driven underwater image enhancement and restoration［J］. Journal of Signal Processing， 2020， 36（9）： 1377-1389.（in Chinese）. doi: 10.16798/j.issn.1003-0530.2020.09.001http://dx.doi.org/10.16798/j.issn.1003-0530.2020.09.001

FABBRI C， ISLAM M J， SATTAR J. Enhancing underwater imagery using generative adversarial networks［C］. 2018 IEEE International Conference on Robotics and Automation （ICRA）. Brisbane， QLD， Australia. IEEE， 2018： 7159-7165. doi: 10.1109/icra.2018.8460552http://dx.doi.org/10.1109/icra.2018.8460552

ZHU J Y， PARK T， ISOLA P， et al. Unpaired image-to-image translation using cycle-consistent adversarial networks［C］. 2017 IEEE International Conference on Computer Vision （ICCV）. Venice， Italy. IEEE， 2017： 2242-2251. doi: 10.1109/iccv.2017.244http://dx.doi.org/10.1109/iccv.2017.244

ISLAM M J， XIA Y Y， SATTAR J. Fast underwater image enhancement for improved visual perception［J］. IEEE Robotics and Automation Letters， 2020， 5（2）： 3227-3234. doi: 10.1109/lra.2020.2974710http://dx.doi.org/10.1109/lra.2020.2974710

LI C Y， ANWAR S， HOU J H， et al. Underwater image enhancement via medium transmission-guided multi-color space embedding［J］. IEEE Transactions on Image Processing， 2021， 30： 4985-5000. doi: 10.1109/tip.2021.3076367http://dx.doi.org/10.1109/tip.2021.3076367

MA Z Y， OH C. A wavelet-based dual-stream network for underwater image enhancement［C］. ICASSP 2022 - 2022 IEEE International Conference on Acoustics， Speech and Signal Processing （ICASSP）. Singapore， Singapore. IEEE， 2022： 2769-2773. doi: 10.1109/icassp43922.2022.9747781http://dx.doi.org/10.1109/icassp43922.2022.9747781

FU Z Q， WANG W， HUANG Y， et al. Uncertainty inspired underwater image enhancement［C］. AVIDAN S， BROSTOW G， CISSÉ M， et al. European Conference on Computer Vision. Cham： Springer， 2022： 465-482. doi: 10.1007/978-3-031-19797-0_27http://dx.doi.org/10.1007/978-3-031-19797-0_27

米泽田，晋洁，李圆圆，等. 基于多尺度级联网络的水下图像增强方法［J］. 电子与信息学报， 2022， 44（10）： 3353-3362. doi: 10.11999/JEIT220375http://dx.doi.org/10.11999/JEIT220375

MI Z T， JIN J， LI Y Y， et al. Underwater image enhancement method based on multi-scale cascade network［J］. Journal of Electronics & Information Technology， 2022， 44（10）： 3353-3362.（in Chinese）. doi: 10.11999/JEIT220375http://dx.doi.org/10.11999/JEIT220375

李云，孙山林，黄晴，等. 基于多路混合注意力机制的水下图像增强网络［J］. 电子与信息学报， 2024， 46（1）： 118-128. doi: 10.11999/JEIT230495http://dx.doi.org/10.11999/JEIT230495

LI Y， SUN S L， HUANG Q， et al. Underwater image enhancement network based on multi-channel hybrid attention mechanism［J］. Journal of Electronics & Information Technology， 2024， 46（1）： 118-128.（in Chinese）. doi: 10.11999/JEIT230495http://dx.doi.org/10.11999/JEIT230495

ZHANG D H， WU C Y， ZHOU J C， et al. Hierarchical attention aggregation with multi-resolution feature learning for GAN-based underwater image enhancement［J］. Engineering Applications of Artificial Intelligence， 2023， 125： 106743. doi: 10.1016/j.engappai.2023.106743http://dx.doi.org/10.1016/j.engappai.2023.106743

RONNEBERGER O， FISCHER P， BROX T. U-Net： convolutional networks for biomedical image segmentation［C］. Medical Image Computing and Computer-Assisted Intervention-MICCAI 2015： 18th International Conference， Munich， Germany， October 5-9， 2015， Proceedings， Part III 18. Springer International Publishing， 2015： 234-241. doi: 10.1007/978-3-319-24574-4_28http://dx.doi.org/10.1007/978-3-319-24574-4_28

KINOSHITA Y， KIYA H. Convolutional neural networks considering local and global features for image enhancement［C］. 2019 IEEE International Conference on Image Processing （ICIP）. Taipei， China. IEEE， 2019： 2110-2114. doi: 10.1109/icip.2019.8803194http://dx.doi.org/10.1109/icip.2019.8803194

WOO S， PARK J， LEE J Y， et al. CBAM： convolutional block attention module［C］. European Conference on Computer Vision. Cham： Springer， 2018： 3-19. doi: 10.1007/978-3-030-01234-2_1http://dx.doi.org/10.1007/978-3-030-01234-2_1

ISOLA P， ZHU J Y， ZHOU T H， et al. Image-to-image translation with conditional adversarial networks［C］. 2017 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. Honolulu， HI， USA. IEEE， 2017： 5967-5976. doi: 10.1109/cvpr.2017.632http://dx.doi.org/10.1109/cvpr.2017.632

YI Z L， ZHANG H， TAN P， et al. DualGAN： unsupervised dual learning for image-to-image translation［C］. 2017 IEEE International Conference on Computer Vision （ICCV）. Venice， Italy. IEEE， 2017： 2868-2876. doi: 10.1109/iccv.2017.310http://dx.doi.org/10.1109/iccv.2017.310

范新南，杨鑫，史朋飞，等. 特征融合生成对抗网络的水下图像增强［J］. 计算机辅助设计与图形学学报， 2022， 34（2）： 264-272. doi: 10.3724/sp.j.1089.2022.18843http://dx.doi.org/10.3724/sp.j.1089.2022.18843

FAN X N， YANG X， SHI P F， et al. Underwater image enhancement based on feature fusion generative adversaral networks［J］. Journal of Computer-Aided Design & Computer Graphics， 2022， 34（2）： 264-272.（in Chinese）. doi: 10.3724/sp.j.1089.2022.18843http://dx.doi.org/10.3724/sp.j.1089.2022.18843

WANG Y， SONG W， FORTINO G， et al. An experimental-based review of image enhancement and image restoration methods for underwater imaging［J］. IEEE Access， 2019， 7： 140233-140251. doi: 10.1109/access.2019.2932130http://dx.doi.org/10.1109/access.2019.2932130

PENG Y T， COSMAN P C. Underwater image restoration based on image blurriness and light absorption［J］. IEEE Transactions on Image Processing， 2017， 26（4）： 1579-1594. doi: 10.1109/tip.2017.2663846http://dx.doi.org/10.1109/tip.2017.2663846

PANETTA K， GAO C， AGAIAN S. Human-visual-system-inspired underwater image quality measures［J］. IEEE Journal of Oceanic Engineering， 2016， 41（3）： 541-551. doi: 10.1109/joe.2015.2469915http://dx.doi.org/10.1109/joe.2015.2469915

MITTAL A， SOUNDARARAJAN R， BOVIK A C. Making a “completely blind” image quality analyzer［J］. IEEE Signal Processing Letters， 2013， 20（3）： 209-212. doi: 10.1109/lsp.2012.2227726http://dx.doi.org/10.1109/lsp.2012.2227726

HU K， WENG C H， ZHANG Y W， et al. An overview of underwater vision enhancement： from traditional methods to recent deep learning［J］. Journal of Marine Science and Engineering， 2022， 10（2）： 241. doi: 10.3390/jmse10020241http://dx.doi.org/10.3390/jmse10020241

王欣，石慧. DRHA-UIE：基于双重残差混合注意力模块的水下图像增强方法［J］. 电子学报， 2023， 51（9）： 2398-2407. doi: 10.12263/DZXB.20211619http://dx.doi.org/10.12263/DZXB.20211619

WANG X， SHI H. DRHA-UIE： an underwater image enhancement method based on dual residual hybrid attention block［J］. Acta Electronica Sinica， 2023， 51（9）： 2398-2407.（in Chinese）. doi: 10.12263/DZXB.20211619http://dx.doi.org/10.12263/DZXB.20211619

ZHOU W H， ZHU D M， SHI M， et al. Deep images enhancement for turbid underwater images based on unsupervised learning［J］. Computers and Electronics in Agriculture， 2022， 202： 107372. doi: 10.1016/j.compag.2022.107372http://dx.doi.org/10.1016/j.compag.2022.107372

WANG T H， WANG L L， ZHANG E， et al. Underwater image enhancement based on optimal contrast and attenuation difference［J］. IEEE Access， 2023， 11： 68538-68549. doi: 10.1109/access.2023.3292275http://dx.doi.org/10.1109/access.2023.3292275

巩文哲，褚金奎，成昊远，等. 基于无监督学习和注意力机制的水下偏振图像融合［J］. 光学精密工程， 2023， 31（21）： 3212-3220. doi: 10.37188/OPE.20233121.3212http://dx.doi.org/10.37188/OPE.20233121.3212

GONG W Z， CHU J K， CHENG H Y， et al. Underwater polarization image fusion based on unsupervised learning and attention mechanisms［J］. Opt. Precision Eng.， 2023， 31（21）： 3212-3220.（in Chinese）. doi: 10.37188/OPE.20233121.3212http://dx.doi.org/10.37188/OPE.20233121.3212

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Aircraft target detection in SAR images based on MA-DETR

3D point cloud classification and segmentation based on dual attention and weighted dynamic graph convolution

Cascade residual-optimized image super-resolution reconstruction in Transformer network

Few-shot warhead fragment group object detection based on feature reassembly and attention

Segmentation network for metastatic lymph nodes of head and neck tumors

Related Author

ZHOU Wenjun

HUANG Shuo

ZHANG Ning

SONG Chuanlong

ZHAO Yuxuan

DUAN Yifan

XU Guoqing

XIAO Jian

Related Institution

Shanghai Key Laboratory of Space-based Heterogeneous Network Collaborative Computing， Shanghai Aerospace Electronic Communication Equipment Research Institute

School of Communication and Information Engineering， Shanghai University

School of Mining， Guizhou University

College of Physics and Telecommunication Engineering， Fuzhou University

Fujian Science and Technology Innovation Laboratory for Optoelectronic Information of China

Address：No.3888 Dong Nanhu Road, Changchun, Jilin, China Postal code：130033
Tel：0431-86176855 Email：gxjmgc@ciomp.ac.cn
Technical support is provided by Beijing Founder electronics co., LTD 吉ICP备11002662号-17 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰