Deep learning image denoising based on multi-stage supervised with Res2-Unet

LIU Yan; CHEN Gang; YU Chunyu; WANG Shiyun; SUN Bin

doi:10.37188/OPE.20233106.0920

您当前的位置：

首页 >

文章列表页 >

Deep learning image denoising based on multi-stage supervised with Res2-Unet

Information Sciences | 更新时间：2023-03-25

- Deep learning image denoising based on multi-stage supervised with Res2-Unet
- Optics and Precision Engineering Vol. 31, Issue 6, Pages: 920-935(2023)
- 作者机构：
  
  1.南京邮电大学电子与光学工程学院，江苏南京 210023
  2.江苏北方湖光光电有限公司，江苏无锡 214035
  3.南京邮电大学自动化学院，江苏南京 210023
- 作者简介：
- 基金信息：
- DOI：10.37188/OPE.20233106.0920
  CLC： TP391.41
- Received：30 June 2022，
  
  Revised：15 August 2022，
  
  Published：25 March 2023
- 稿件说明：
移动端阅览
刘言,陈刚,喻春雨等.基于Res2-Unet多阶段监督的图像降噪[J].光学精密工程,2023,31(06):920-935.

LIU Yan,CHEN Gang,YU Chunyu,et al.Deep learning image denoising based on multi-stage supervised with Res2-Unet[J].Optics and Precision Engineering,2023,31(06):920-935.
刘言,陈刚,喻春雨等.基于Res2-Unet多阶段监督的图像降噪[J].光学精密工程,2023,31(06):920-935. DOI： 10.37188/OPE.20233106.0920.

LIU Yan,CHEN Gang,YU Chunyu,et al.Deep learning image denoising based on multi-stage supervised with Res2-Unet[J].Optics and Precision Engineering,2023,31(06):920-935. DOI： 10.37188/OPE.20233106.0920.

摘要

为了提高基于深度学习的图像降噪效率，提出了一种基于Res2-Unet-SE的多阶段监督深度残差（Multi-stage Supervised Deep Residual，MSDR）降噪神经网络。首先基于该神经网络，将图像降噪分为多阶段处理过程；然后在各处理阶段，将不同分辨率图像块输入到Res2-Unet子网络中获取不同尺度特征信息，并通过通道注意力机制将自适应学习的特征融合信息传递到下阶段；最后将不同尺度特征信息叠加，完成高质量的图像降噪。实验选择BSD400数据集用于训练，通过Set12数据集进行高斯噪声的降噪测试；通过SIDD数据集完成真实噪声的降噪测试。通过与常见的降噪神经网络对比表明，对图像添加

=15，25，50的高斯噪声时，经本文算法降噪后的图像PSNR比对高斯噪声消除性能较好的DNCNN分别提高0.03 dB，0.05 dB，0.14 dB；在

=25，50时，相较于MPRNET分别提高了0.02 dB， 0.06 dB。对含真实噪声的图像，经本文算法降噪后的图像PSNR比CBDNET算法提高0.48 dB。实验分析表明，本文算法在图像降噪上具有较高的鲁棒性，不仅能从噪声中有效恢复图像细节，还能充分保持图像的全局依赖关系。

Abstract

To restore high quality images from different types of noise images， this study developed a multi-stage supervised deep residual （MSDR） neural network based on Res2-Unet-SE. First， using the neural network， the image denoising task was devised as a multi-stage process. Then， in each processing stage， image blocks with different resolutions were input into a Res2-Unet sub-network to obtain feature information at different scales， and an adaptive learning of the feature fusion information was transferred to the next stage through a channel attention mechanism. Finally， the feature information of different scales was superimposed to achieve high-quality image noise reduction. The BSD400 dataset was selected for training in the experiments， and a Gaussian noise reduction test was performed using the Set12 data set. Real noise reduction test was conducted using the SIDD data set. Compared with the common denoising neural network， the peak signal-to-noise ratios （PSNRs） of the proposed denoising convolutional neural network （DnCNN） improved by 0.03 dB， 0.05 dB， and 0.14 dB when Gaussian noises of σ = 15， 25 and 50， respectively， were added to the image data set. Compared with the latest dual residual block network （DuRN） algorithm， the PSNR of the image denoised using the proposed algorithm was higher by 0.06 dB， 0.57 dB， and 0.39 dB， respectively. For images containing real noise， the PSNR of the image denoised by the proposed algorithm was 0.6 dB higher than that by the convolutional blind denoising network （CBDNET） algorithm. The results indicate that the proposed algorithm is highly robust in the task of image denoising， and it can effectively remove noise and restore the details of an image， as well as fully maintain the global dependence of the image.

关键词

Keywords

references

DABOV K ， FOI A ， KATKOVNIK V ， et al . Image denoising by sparse 3-D transform-domain collaborative filtering ［J］. IEEE Transactions on Image Processing ， 2007 ， 16 （ 8 ）： 2080 - 2095 . doi: 10.1109/tip.2007.901238 http://dx.doi.org/10.1109/tip.2007.901238

GU S H ， ZHANG L ， ZUO W M ， et al . Weighted Nuclear Norm Minimization with Application to Image Denoising ［C］. 2014 IEEE Conference on Computer Vision and Pattern Recognition . 2328，2014 ， Columbus ， OH ， USA . IEEE ， 2014 ： 2862 - 2869 . doi: 10.1109/cvpr.2014.366 http://dx.doi.org/10.1109/cvpr.2014.366

JAIN V ， MURRAY J F ， ROTH F ， et al . Supervised Learning of Image Restoration with Convolutional Networks ［C］. 2007 IEEE 11th International Conference on Computer Vision . 1421，2007 ， Rio de Janeiro ， Brazil . IEEE ， 2007 ： 1 - 8 . doi: 10.1109/iccv.2007.4408909 http://dx.doi.org/10.1109/iccv.2007.4408909

ZHANG K ， ZUO W M ， CHEN Y J ， et al . Beyond a Gaussian denoiser： residual learning of deep CNN for image denoising ［J］. IEEE Transactions on Image Processing ， 2017 ， 26 （ 7 ）： 3142 - 3155 . doi: 10.1109/tip.2017.2662206 http://dx.doi.org/10.1109/tip.2017.2662206

ZHANG K ， ZUO W M ， ZHANG L . FFDNet： toward a fast and flexible solution for CNN-based image denoising ［J］. IEEE Transactions on Image Processing ， 2018 ， 27 （ 9 ）： 4608 - 4622 . doi: 10.1109/TIP.2018.2839891 http://dx.doi.org/10.1109/TIP.2018.2839891

GUO S ， YAN Z F ， ZHANG K ， et al . Toward Convolutional Blind Denoising of Real Photographs ［C］. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. 1520，2019 ， Long Beach， CA， USA. IEEE ， 2020 ： 1712 - 1722 . doi: 10.1109/cvpr.2019.00181 http://dx.doi.org/10.1109/cvpr.2019.00181

LIU X ， SUGANUMA M ， SUN Z ， et al . Dual Residual Networks Leveraging the Potential of Paired Operations for Image Restoration ［C］. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. 1520，2019 ， Long Beach， CA， USA. IEEE ， 2020 ： 7000 - 7009 . doi: 10.1109/cvpr.2019.00717 http://dx.doi.org/10.1109/cvpr.2019.00717

ZAMIR S W ， ARORA A ， KHAN S ， et al . Multi-stage Progressive Image Restoration ［C］. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. 2025，2021 ， Nashville， TN， USA. IEEE ， 2021 ： 14816 - 14826 . doi: 10.1109/cvpr46437.2021.01458 http://dx.doi.org/10.1109/cvpr46437.2021.01458

HU J ， SHEN L ， ALBANIE S ， et al . Squeeze-and-excitation networks ［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence ， 2020 ， 42 （ 8 ）： 2011 - 2023 . doi: 10.1109/tpami.2019.2913372 http://dx.doi.org/10.1109/tpami.2019.2913372

秦传波，宋子玉，曾军英，等 . 联合多尺度和注意力-残差的深度监督乳腺癌分割［J］. 光学精密工程， 2021 ， 29 （ 4 ）： 877 - 895 . doi: 10.37188/OPE.20212904.0877 http://dx.doi.org/10.37188/OPE.20212904.0877

QIN CH B ， SONG Z Y ， ZENG J Y ， et al . Deeply supervised breast cancer segmentation combined with multi-scale and attention-residuals ［J］. Opt. Precision Eng. ， 2021 ， 29 （ 4 ）： 877 - 895 . （in Chinese） . doi: 10.37188/OPE.20212904.0877 http://dx.doi.org/10.37188/OPE.20212904.0877

GAO S H ， CHENG M M ， ZHAO K ， et al . Res2Net： a new multi-scale backbone architecture ［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence ， 2021 ， 43 （ 2 ）： 652 - 662 . doi: 10.1109/tpami.2019.2938758 http://dx.doi.org/10.1109/tpami.2019.2938758

蔡体健，彭潇雨，石亚鹏，等 . 通道注意力与残差级联的图像超分辨率重建［J］. 光学精密工程， 2021 ， 29 （ 1 ）： 142 - 151 . doi: 10.37188/OPE.20212901.0142 http://dx.doi.org/10.37188/OPE.20212901.0142

CAI T J ， PENG X Y ， SHI Y P ， et al . Channel attention and residual concatenation network for image super-resolution ［J］. Opt. Precision Eng. ， 2021 ， 29 （ 1 ）： 142 - 151 . （in Chinese） . doi: 10.37188/OPE.20212901.0142 http://dx.doi.org/10.37188/OPE.20212901.0142

RONNEBERGER O ， FISCHER P ， BROX T . U-net： convolutional networks for biomedical image segmentation ［EB/OL］. 2015 ： arXiv ： 1505 . 04597 . https：//arxiv.org/abs/1505.04597 https://arxiv.org/abs/1505.04597 . doi: 10.1007/978-3-319-24574-4_28 http://dx.doi.org/10.1007/978-3-319-24574-4_28

ALOM M Z ， YAKOPCIC C ， TAHA T M ， et al . Nuclei Segmentation with Recurrent Residual Convolutional Neural Networks Based U-net （R2U-net）［C］. NAECON 2018 - IEEE National Aerospace and Electronics Conference . 2326，2018 ， Dayton ， OH ， USA . IEEE ， 2018 ： 228 - 233 . doi: 10.1109/naecon.2018.8556686 http://dx.doi.org/10.1109/naecon.2018.8556686

GU X F ， LIU J ， ZOU X X ， et al . Using Checkerboard Rendering and Deconvolution to Eliminate Checkerboard Artifacts in Images Generated by Neural Networks ［C］. 2017 14th International Computer Conference on Wavelet Active Media Technology and Information Processing （ICCWAMTIP）. 1517，2017 ， Chengdu， China. IEEE ， 2018 ： 197 - 200 . doi: 10.1109/iccwamtip.2017.8301478 http://dx.doi.org/10.1109/iccwamtip.2017.8301478

JOHNSON J ， ALAHI A ， LI" F F . Perceptual losses for real-time style transfer and super-resolution ［EB/OL］. 2016 ： arXiv ： 1603 . 08155 . https：//arxiv.org/abs/1603.08155 https://arxiv.org/abs/1603.08155 . doi: 10.1007/978-3-319-46475-6_43 http://dx.doi.org/10.1007/978-3-319-46475-6_43

佟雨兵，张其善，祁云平 . 基于PSNR与SSIM联合的图像质量评价模型［J］. 中国图象图形学报， 2006 ， 11 （ 12 ）： 1758 - 1763 . doi: 10.11834/jig.2006012307 http://dx.doi.org/10.11834/jig.2006012307

TONG Y B ， ZHANG Q SH ， QI Y P . Image quality assessing by combining PSNR with SSIM ［J］. Journal of Image and Graphics ， 2006 ， 11 （ 12 ）： 1758 - 1763 . （in Chinese） . doi: 10.11834/jig.2006012307 http://dx.doi.org/10.11834/jig.2006012307

ARBELÁEZ P ， MAIRE M ， FOWLKES C ， et al . Contour detection and hierarchical image segmentation ［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence ， 2011 ， 33 （ 5 ）： 898 - 916 . doi: 10.1109/tpami.2010.161 http://dx.doi.org/10.1109/tpami.2010.161

ABDELHAMED A ， LIN S ， BROWN M S . A High-Quality Denoising Dataset for Smartphone Cameras ［C］. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . 1823，2018 ， Salt Lake City， UT， USA . IEEE ， 2018 ： 1692 - 1700 . doi: 10.1109/cvpr.2018.00182 http://dx.doi.org/10.1109/cvpr.2018.00182

李泽田，雷志春 . 基于局部期望最大化注意力的图像降噪［J］. 液晶与显示， 2020 ， 35 （ 4 ）： 350 - 359 . doi: 10.3788/yjyxs20203504.0350 http://dx.doi.org/10.3788/yjyxs20203504.0350

LI Z T ， LEI ZH CH . Local expectation-maximization attention network for image denoising ［J］. Chinese Journal of Liquid Crystals and Displays ， 2020 ， 35 （ 4 ）： 350 - 359 . （in Chinese） . doi: 10.3788/yjyxs20203504.0350 http://dx.doi.org/10.3788/yjyxs20203504.0350

陈欣，万敏杰，马超，等 . 采用多尺度特征融合SSD的遥感图像小目标检测［J］. 光学精密工程， 2021 ， 29 （ 11 ）： 2672 - 2682 . doi: 10.37188/OPE.20212911.2672 http://dx.doi.org/10.37188/OPE.20212911.2672

CHEN X ， WAN M J ， MA CH ， et al . Recognition of small targets in remote sensing image using multi-scale feature fusion-based shot multi-box detector ［J］. Opt. Precision Eng. ， 2021 ， 29 （ 11 ）： 2672 - 2682 . （in Chinese） . doi: 10.37188/OPE.20212911.2672 http://dx.doi.org/10.37188/OPE.20212911.2672

Views

502

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Cascade residual-optimized image super-resolution reconstruction in Transformer network

Design of lightweight re-parameterized remote sensing image super-resolution network

Multidimensional attention mechanism and selective feature fusion for image super-resolution reconstruction

Image super-resolution reconstruction based on attention and wide-activated dense residual network

Design of intensity image and polarization image fusion network

Related Author

LIN Shanling

GUO Tailiang

LIN Zhixian

WANG Kunfu

WU Zhencheng

LIN Jianpu

XIE Weijia

LI Jun

Related Institution

College of Physics and Telecommunication Engineering， Fuzhou University

Fujian Science and Technology Innovation Laboratory for Optoelectronic Information of China

School of Advanced Manufacturing， Fuzhou University

College of Information Engineering， Jiangxi University of Science and Technology

Yunnan Police Unmanned System Innovation Research Institute， Yunnan Police Officer Academy

Address：No.3888 Dong Nanhu Road, Changchun, Jilin, China Postal code：130033
Tel：0431-86176855 Email：gxjmgc@ciomp.ac.cn
Technical support is provided by Beijing Founder electronics co., LTD 吉ICP备11002662号-17 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰