Improved CycleGAN network for underwater microscopic image color correction

Haotian WANG; Qingsheng LIU; Liang CHEN; Wangquan YE; Yuan LU; Jinjia GUO; Ronger ZHENG

doi:10.37188/OPE.20223012.1499

您当前的位置：

首页 >

文章列表页 >

Improved CycleGAN network for underwater microscopic image color correction

Information Sciences | 更新时间：2022-06-25

- Improved CycleGAN network for underwater microscopic image color correction
- Optics and Precision Engineering Vol. 30, Issue 12, Pages: 1499-1508(2022)
- 作者机构：
  
  中国海洋大学信息科学与工程学部物理与光电工程学院，山东青岛 266100
- 作者简介：
- 基金信息：
- DOI：10.37188/OPE.20223012.1499
  CLC： TP391.4;TH691.9
- Received：01 March 2022，
  
  Revised：20 March 2022，
  
  Published：25 June 2022
- 稿件说明：
移动端阅览
王昊天,刘庆省,陈亮等.改进的CycleGAN网络用于水下显微图像颜色校正[J].光学精密工程,2022,30(12):1499-1508.

WANG Haotian,LIU Qingsheng,CHEN Liang,et al.Improved CycleGAN network for underwater microscopic image color correction[J].Optics and Precision Engineering,2022,30(12):1499-1508.
王昊天,刘庆省,陈亮等.改进的CycleGAN网络用于水下显微图像颜色校正[J].光学精密工程,2022,30(12):1499-1508. DOI： 10.37188/OPE.20223012.1499.

WANG Haotian,LIU Qingsheng,CHEN Liang,et al.Improved CycleGAN network for underwater microscopic image color correction[J].Optics and Precision Engineering,2022,30(12):1499-1508. DOI： 10.37188/OPE.20223012.1499.

摘要

针对海洋水体及悬浮颗粒物吸收和散射所导致的水下显微图像的颜色信息失真问题，本文提出了一种改进的循环一致性对抗网络（Cycle-consistent Adversarial Network， CycleGAN）算法，实现对水下目标物图像的颜色校正。通过在原始水下降质图像和重构水下图像之间加入R、G、B三个通道的结构相似性（Structure Similarity Index Measure， SSIM）损失函数，度量二者图像之间的信息损失，进而实现R、G、B三个通道颜色的精准调控，不仅增强了CycleGAN网络的整体性能，也提高了生成器生成图像的质量。然后，利用水下多色自制标靶及天然矿石的显微图像组成的训练数据集对本文所提的改进网络进行训练，所得的模型可用于实际矿石样品表面的显微图像颜色校正。结果表明，本文所提的改进的CycleGAN算法较其它方法在颜色校正方面有着明显的优势。与传统的Retinex算法相比，处理后的图像峰值信噪比（Peak Signal-to-Noise Ratio， PSNR）和结构相似性指标分别提高41.85%、35.62%，而且，从主观视觉角度可发现经过校正的水下显微图像与空气中图像颜色相似度最高。综上，本文方法可以有效地对水下目标物图像进行颜色校正，并提升水下显微图像的质量，有望在海洋地质和海洋生物学方面得到应用。

Abstract

The absorption and scattering of light by marine water and suspended particles lead to the distortion of color in underwater microscopic images. This paper presents an improved cycle generative adversarial network （CycleGAN） algorithm for effectively correcting the color of microscopic images of underwater targets. The structural similarity index （SSIM） loss function， which measures the loss of color information among images， of the R， G， and B channels was added between the original underwater images and the reconstructed images. Therefore， the color of the R， G and B channels was accurately regulated. This enhanced not only the overall performance of the CycleGAN， but also the quality of images produced by the generator. Subsequently， the improved network was trained by using a training data set， which consisted of underwater multicolor self-made target images and microscopic images of natural stones. The trained network model was used to correct the color of the microscopic images of underwater stones. The results showed that the improved CycleGAN algorithm had distinct advantages in color correction over other methods. The peak signal-to-noise ratio and SSIM of the images processed by using this algorithm were 41.85% and 35.62% higher than those processed by using the traditional Retinex algorithm， respectively. Moreover， in terms of subjective vision， the corrected underwater microscopic images had the highest color similarity with the images taken in air. In conclusion， this method can effectively correct the color of underwater target images and improve the quality of underwater microscopic images. It can be applied in marine geology and marine biology.

关键词

Keywords

references

NOZAKI T ， NAGASE T ， USHIKUBO T ， et al . Microbial sulfate reduction plays an important role at the initial stage of subseafloor sulfide mineralization ［J］. Geology ， 2021 ， 49 （ 2 ）： 222 - 227 . doi: 10.1130/g47943.1 http://dx.doi.org/10.1130/g47943.1

SINHA R K ， KRISHNAN K P ， THOMAS F A ， et al . Polyphasic approach revealed complex bacterial community structure and function in deep sea sediment of ultra-slow spreading Southwest Indian Ridge ［J］. Ecological Indicators ， 2019 ， 96 ： 40 - 51 . doi: 10.1016/j.ecolind.2018.08.063 http://dx.doi.org/10.1016/j.ecolind.2018.08.063

NIKOLAUS R ， AMMERMAN J W ， MACDONALD I R . Distinct pigmentation and trophic modes in beggiatoa from hydrocarbon seeps in the gulf of Mexico ［J］. Aquatic Microbial Ecology ， 2003 ， 32 ： 85 - 93 . doi: 10.3354/ame032085 http://dx.doi.org/10.3354/ame032085

ZHANG R ， GUI X Y ， CHENG H Y ， et al . Underwater image recovery utilizing polarimetric imaging based on neural networks ［J］. Applied Optics ， 2021 ， 60 （ 27 ）： 8419 . doi: 10.1364/ao.431299 http://dx.doi.org/10.1364/ao.431299

WU S C ， LUO T ， JIANG G Y ， et al . A two-stage underwater enhancement network based on structure decomposition and characteristics of underwater imaging ［J］. IEEE Journal of Oceanic Engineering ， 2021 ， 46 （ 4 ）： 1213 - 1227 . doi: 10.1109/joe.2021.3064093 http://dx.doi.org/10.1109/joe.2021.3064093

HAN R Y ， GUAN Y ， YU Z B ， et al . Underwater image enhancement based on a spiral generative adversarial framework ［J］. IEEE Access ， 2020 ， 8 ： 218838 - 218852 . doi: 10.1109/access.2020.3041280 http://dx.doi.org/10.1109/access.2020.3041280

ZONG X H ， CHEN Z H ， WANG D D . Local-CycleGAN： a general end-to-end network for visual enhancement in complex deep-water environment ［J］. Applied Intelligence ， 2021 ， 51 （ 4 ）： 1947 - 1958 . doi: 10.1007/s10489-020-01931-w http://dx.doi.org/10.1007/s10489-020-01931-w

FU X Y ， CAO X Y . Underwater image enhancement with global-local networks and compressed-histogram equalization ［J］. Signal Processing： Image Communication ， 2020 ， 86 ： 115892 . doi: 10.1016/j.image.2020.115892 http://dx.doi.org/10.1016/j.image.2020.115892

SONG W ， WANG Y ， HUANG D M ， et al . Enhancement of underwater images with statistical model of background light and optimization of transmission map ［J］. IEEE Transactions on Broadcasting ， 2020 ， 66 （ 1 ）： 153 - 169 . doi: 10.1109/tbc.2019.2960942 http://dx.doi.org/10.1109/tbc.2019.2960942

褚金奎，张培奇，成昊远，等 . 基于特定偏振态成像的水下图像去散射方法［J］. 光学精密工程， 2021 ， 29 （ 5 ）： 1207 - 1215 . doi: 10.37188/OPE.20212905.1207 http://dx.doi.org/10.37188/OPE.20212905.1207

CHU J K ， ZHANG P Q ， CHENG H Y ， et al . De-scattering method of underwater image based on imaging of specific polarization state ［J］. Opt. Precision Eng. ， 2021 ， 29 （ 5 ）： 1207 - 1215 . （in Chinese） . doi: 10.37188/OPE.20212905.1207 http://dx.doi.org/10.37188/OPE.20212905.1207

ZHUANG P X ， DING X H . Underwater image enhancement using an edge-preserving filtering Retinex algorithm ［J］. Multimedia Tools and Applications ， 2020 ， 79 （ 25/26 ）： 17257 - 17277 . doi: 10.1007/s11042-019-08404-4 http://dx.doi.org/10.1007/s11042-019-08404-4

ZHU J Y ， PARK T ， ISOLA P ， et al . Unpaired image-to-image translation using cycle-consistent adversarial networks ［C］. 2017 IEEE International Conference on Computer Vision . 2229，2017 ， Venice， Italy . IEEE ， 2017 ： 2242 - 2251 . doi: 10.1109/iccv.2017.244 http://dx.doi.org/10.1109/iccv.2017.244

杜振龙，沈海洋，宋国美，等 . 基于改进CycleGAN的图像风格迁移［J］. 光学精密工程， 2019 ， 27 （ 8 ）： 1836 - 1844 . doi: 10.3788/ope.20192708.1836 http://dx.doi.org/10.3788/ope.20192708.1836

DU Z L ， SHEN H Y ， SONG G M ， et al . Image style transfer based on improved CycleGAN ［J］. Opt. Precision Eng. ， 2019 ， 27 （ 8 ）： 1836 - 1844 . （in Chinese） . doi: 10.3788/ope.20192708.1836 http://dx.doi.org/10.3788/ope.20192708.1836

LU J Y ， LI N ， ZHANG S Y ， et al . Multi-scale adversarial network for underwater image restoration ［J］. Optics & Laser Technology ， 2019 ， 110 ： 105 - 113 . doi: 10.1016/j.optlastec.2018.05.048 http://dx.doi.org/10.1016/j.optlastec.2018.05.048

LIU F ， HAN P L ， WEI Y ， et al . Deeply seeing through highly turbid water by active polarization imaging ［J］. Optics Letters ， 2018 ， 43 （ 20 ）： 4903 - 4906 . doi: 10.1364/ol.43.004903 http://dx.doi.org/10.1364/ol.43.004903

AMER K O ， ELBOUZ M ， ALFALOU A . Phase Contrast Enhancement Combined with Dark Channel Prior for Underwater Target Recognition ［J］. The Eurasia Proceedings of Science Technology Engineering and Mathematics ， 2018 （ 4 ）： 74 - 81 . doi: 10.1364/oe.27.000621 http://dx.doi.org/10.1364/oe.27.000621

聂瑛，何志毅 . 不同波长光源照明的水下成像及光学图像实时处理［J］. 光学学报， 2014 ， 34 （ 7 ）： 0710002 . doi: 10.3788/aos201434.0710002 http://dx.doi.org/10.3788/aos201434.0710002

NIE Y ， HE Z Y . Underwater imaging and real-time optical image processing under illumination by light sources with different wavelengths ［J］. Acta Optica Sinica ， 2014 ， 34 （ 7 ）： 0710002 . （in Chinese） . doi: 10.3788/aos201434.0710002 http://dx.doi.org/10.3788/aos201434.0710002

王卫星，赵恒 . 结合改进Retinex及自适应分数阶微分的雾霾公路交通图像增强［J］. 光学精密工程， 2020 ， 28 （ 8 ）： 1820 - 1834 . doi: 10.3788/OPE.20202808.1820 http://dx.doi.org/10.3788/OPE.20202808.1820

WANG W X ， ZHAO H . Haze traffic image enhancement based on improved retinex and adaptive fractional differential ［J］. Opt. Precision Eng. ， 2020 ， 28 （ 8 ）： 1820 - 1834 . （in Chinese） . doi: 10.3788/OPE.20202808.1820 http://dx.doi.org/10.3788/OPE.20202808.1820

王冬云，唐楚，鄂世举，等 . 基于导向滤波Retinex和自适应Canny的图像边缘检测［J］. 光学精密工程， 2021 ， 29 （ 2 ）： 443 - 451 . doi: 10.37188/OPE.20212902.0443 http://dx.doi.org/10.37188/OPE.20212902.0443

WANG D Y ， TANG C ， E S J ， et al . Image edge detection based on guided filter Retinex and adaptive Canny ［J］. Opt. Precision Eng. ， 2021 ， 29 （ 2 ）： 443 - 451 . （in Chinese） . doi: 10.37188/OPE.20212902.0443 http://dx.doi.org/10.37188/OPE.20212902.0443

HORÉ A ， ZIOU D . Image quality metrics： PSNR vs. SSIM ［C］. 2010 20th International Conference on Pattern Recognition . 2326，2010 ， Istanbul， Turkey . IEEE ， 2010 ： 2366 - 2369 . doi: 10.1109/icpr.2010.579 http://dx.doi.org/10.1109/icpr.2010.579

YANG M ， SOWMYA A . An underwater color image quality evaluation metric ［J］. IEEE Transactions on Image Processing ， 2015 ， 24 （ 12 ）： 6062 - 6071 . doi: 10.1109/tip.2015.2491020 http://dx.doi.org/10.1109/tip.2015.2491020

Views

2261

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Equipment fault dataset amplification method combine 3D model with improved CycleGAN

Sand-dust image enhancement using RGB color balance method

Coarse-to-fine underwater image enhancement based on multi-level wavelet transform

Multi-projection color correction and brightness fusion technology based on Lab

Related Author

LI Baoping

QI Hengyi

WANG Manli

WEI Po

DING Yuan

WU Kaijun

YUAN Guoming

YANG Guang

Related Institution

College of Physics and Electronic Information， Henan Polytechnic University

College of Electronics and Information Engineering， Lanzhou Jiaotong University

Department of emergency management， Institute of Disaster Prevention

Department of Information Engineering， Institute of Disaster Prevention

School of electrical engineering， Yanshan university

AI问答

Address：No.3888 Dong Nanhu Road, Changchun, Jilin, China Postal code：130033
Tel：0431-86176855 Email：gxjmgc@ciomp.ac.cn
Technical support is provided by Beijing Founder electronics co., LTD 吉ICP备11002662号-17 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰