Multi-level features progressive refinement and edge enhancement network for image dehazing

Yangfang FU; Shibai YIN; Zhen DENG; Yibin WANG; Shuhao HU

doi:10.37188/OPE.20223009.1091

您当前的位置：

首页 >

文章列表页 >

Multi-level features progressive refinement and edge enhancement network for image dehazing

Information Sciences | 更新时间：2022-05-26

- Multi-level features progressive refinement and edge enhancement network for image dehazing
- Optics and Precision Engineering Vol. 30, Issue 9, Pages: 1091-1100(2022)
- 作者机构：
  
  1.西安工业大学计算机科学与工程学院，陕西西安 710021
  2.西南财经大学经济信息工程学院，四川成都 611130
  3.宁夏大学信息工程学院，宁夏银川 750021
  4.四川师范大学工学院，四川成都 610101
- 作者简介：
- 基金信息：
- DOI：10.37188/OPE.20223009.1091
  CLC： TP391.41
- Received：22 September 2021，
  
  Revised：15 October 2021，
  
  Published：10 May 2022
- 稿件说明：
移动端阅览
傅妍芳,尹诗白,邓箴等.多级特征逐步细化及边缘增强的图像去雾[J].光学精密工程,2022,30(09):1091-1100.

FU Yangfang,YIN Shibai,DENG Zhen,et al.Multi-level features progressive refinement and edge enhancement network for image dehazing[J].Optics and Precision Engineering,2022,30(09):1091-1100.
傅妍芳,尹诗白,邓箴等.多级特征逐步细化及边缘增强的图像去雾[J].光学精密工程,2022,30(09):1091-1100. DOI： 10.37188/OPE.20223009.1091.

FU Yangfang,YIN Shibai,DENG Zhen,et al.Multi-level features progressive refinement and edge enhancement network for image dehazing[J].Optics and Precision Engineering,2022,30(09):1091-1100. DOI： 10.37188/OPE.20223009.1091.

摘要

为设计合理有效的神经网络框架，提高去雾算法的精度，保留完整的边缘细节，提出了常微分方程（Ordinary Differential Equation， ODE）启发的多级特征逐步细化及边缘增强的去雾算法。利用多级特征提取子网络，从输入雾图中提取出包含细节信息的低级特征和包含语义信息的高级特征，用于后续去雾结果的逐步细化。受残差网络框架与ODE求解策略关联性启发，依据两步两阶的蛙跳方法Leapfrog设计出Leapfrog模块，并串联多个Leapfrog模块，模拟ODE离散的逼近求解过程，构造逐步细化的去雾子网络。此子网络中，每个Leapfrog模块在交替输入的低级/高级特征的互补信息引导下，不断细化前一个Leapfrog模块估计的去雾结果。受二阶微分算子实施边缘增强的启发，边缘增强子网络利用预训练的UNet估计最后一个Leapfrog模块的去雾图像边缘，并叠加到此去雾图像上得到增强边缘，保留细节的最终去雾结果。实验表明，在真实图像及合成图像上，本算法均能取得较好的去雾效果，且在视觉评价和客观评价方面优于已有的去雾算法，与EAAN相比去雾精度提高了5%，运行时间仅有0.032 s，能有效用于图像去雾的工程实践中。

Abstract

An ordinary differential equation （ODE）-based multi-level feature progressive refinement and edge enhancement network is proposed for image dehazing to provide an effective convolutional neural network framework with an algorithm designed to preserve edges while improving accuracy. The study mainly comprises subnetworks of multi-level feature extraction， ODE-based progressive refinement， and edge enhancement. First， the multi-level features extraction subnetwork is leveraged to extract low-level features with detailed information and high-level features with semantic information from hazy images， to enable the subsequent dehazing operations. Second， a novel Leapfrog module is designed based on the relationship between the residual framework and ODE solver by cascading Leapfrog modules to model an approximation solution for ODEs. Finally， the progressive refinement subnetwork is developed. Notably， each Leapfrog module refines the output of the previous Leapfrog with alternative low/high-level features. Finally， motivated by the effectiveness of edge enhancement via second-order differential operators in the edge enhancement network， the edge of the dehazing result predicted by the last Leapfrog module is calculated using the pretrained UNet and added back into dehazing to enhance edges and preserve details. The experimental results demonstrate that the proposed method outperforms the existing methods on both synthetic images and real images quantitatively as well as qualitatively. The dehazing accuracy is improved by 5% and the runtime is only 0.032 s. Hence， the proposed method can be incorporated into practical dehazing applications in engineering.

关键词

Keywords

references

杨燕，梁小珍，张金龙 . 分离特征和协同网络下的端到端图像去雾［J］. 光学精密工程， 2021 ， 29 （ 8 ）： 1931 - 1941 . doi: 10.37188/OPE.2021.0003 http://dx.doi.org/10.37188/OPE.2021.0003

YANG Y ， LIANG X ZH ， ZHANG J L . End-to-end image dehazing under separated features and collaborative network ［J］. Opt. Precision Eng. ， 2021 ， 29 （ 8 ）： 1931 - 1941 . （in Chinese） . doi: 10.37188/OPE.2021.0003 http://dx.doi.org/10.37188/OPE.2021.0003

HE K M ， SUN J ， TANG X O . Single image haze removal using dark channel prior ［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence ， 2011 ， 33 （ 12 ）： 2341 - 2353 . doi: 10.1109/tpami.2010.168 http://dx.doi.org/10.1109/tpami.2010.168

FENG T ， WANG C S ， CHEN X W ， et al . URNet： a U-Net based residual network for image dehazing ［J］. Applied Soft Computing ， 2021 ， 102 ： 106884 . doi: 10.1016/j.asoc.2020.106884 http://dx.doi.org/10.1016/j.asoc.2020.106884

BERMAN D ， TREIBITZ T ， AVIDAN S . Non-local image dehazing ［C］. 2016 IEEE Conference on Computer Vision and Pattern Recognition ， 2730，2016 ， Las Vegas， NV， USA . IEEE ， 2016 ： 1674 - 1682 . doi: 10.1109/cvpr.2016.185 http://dx.doi.org/10.1109/cvpr.2016.185

PANAGOPOULOS A ， WANG C H ， SAMARAS D ， et al . Estimating shadows with the bright channel cue ［C］. Trends and Topics in Computer Vision ， 2012 ： 1 - 12 . doi: 10.1007/978-3-642-35740-4_1 http://dx.doi.org/10.1007/978-3-642-35740-4_1

YU T ， SONG K ， MIAO P ， et al . Nighttime single image dehazing via pixel-wise alpha blending ［J］. IEEE Access ， 2019 ， 7 ： 114619 - 114630 . doi: 10.1109/access.2019.2936049 http://dx.doi.org/10.1109/access.2019.2936049

YIN S B ， WANG Y B ， YANG Y H . A novel image-dehazing network with a parallel attention block ［J］. Pattern Recognition ， 2020 ， 102 ： 107255 . doi: 10.1016/j.patcog.2020.107255 http://dx.doi.org/10.1016/j.patcog.2020.107255

YIN ， WANG ， YANG . A novel residual dense pyramid network for\r image dehazing ［J］. Entropy ， 2019 ， 21 （ 11 ）： 1123 . doi: 10.3390/e21111123 http://dx.doi.org/10.3390/e21111123

SHAO Y J ， LI L ， REN W Q ， et al . Domain adaptation for image dehazing ［C］. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. 1319，2020 ， Seattle， WA， USA. IEEE ， 2020 ： 2805 - 2814 . doi: 10.1109/cvpr42600.2020.00288 http://dx.doi.org/10.1109/cvpr42600.2020.00288

QU Y Y ， CHEN Y Z ， HUANG J Y ， et al . Enhanced Pix2pix dehazing network ［C］. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. 1520，2019 ， Long Beach， CA， USA. IEEE ， 2019 ： 8152 - 8160 . doi: 10.1109/cvpr.2019.00835 http://dx.doi.org/10.1109/cvpr.2019.00835

WANG C ， SHEN H Z ， FAN F ， et al . EAA-Net： a novel edge assisted attention network for single image dehazing ［J］. Knowledge-Based Systems ， 2021 ， 228 ： 107279 . doi: 10.1016/j.knosys.2021.107279 http://dx.doi.org/10.1016/j.knosys.2021.107279

SHEN J W ， LI Z Y ， YU L ， et al . Implicit Euler ODE networks for single-image dehazing ［C］. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops （CVPRW）. 1419，2020 ， Seattle， WA， USA. IEEE ， 2020 ： 877 - 886 . doi: 10.1109/cvprw50498.2020.00117 http://dx.doi.org/10.1109/cvprw50498.2020.00117

NAGY Á ， OMLE I ， KAREEM H ， et al . Stable， explicit， leapfrog-hopscotch algorithms for the diffusion equation ［J］. Computation ， 2021 ， 9 （ 8 ）： 92 . doi: 10.3390/computation9080092 http://dx.doi.org/10.3390/computation9080092

王卫星，赵恒 . 结合改进Retinex及自适应分数阶微分的雾霾公路交通图像增强［J］. 光学精密工程， 2020 ， 28 （ 8 ）： 1820 - 1834 . doi: 10.3788/OPE.20202808.1820 http://dx.doi.org/10.3788/OPE.20202808.1820

WANG W X ， ZHAO H . Haze traffic image enhancement based on improved retinex and adaptive fractional differential ［J］. Opt. Precision Eng. ， 2020 ， 28 （ 8 ）： 1820 - 1834 . （in Chinese） . doi: 10.3788/OPE.20202808.1820 http://dx.doi.org/10.3788/OPE.20202808.1820

LI B ， REN W ， FU D ， et al . Benchmarking single image dehazing and beyond ［J］. IEEE Transactions on Image Processing ， 2018 ， 28 （ 1 ）： 492 - 505 . doi: 10.1109/tip.2018.2867951 http://dx.doi.org/10.1109/tip.2018.2867951

ZHANG H ， PATEL V M . Densely connected pyramid dehazing network ［C］. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition ， 1823，2018 ， Salt Lake City， UT， USA . IEEE ， 2018 ： 3194 - 3203 . doi: 10.1109/cvpr.2018.00337 http://dx.doi.org/10.1109/cvpr.2018.00337

XU Z X ， WU K ， HUANG L ， et al . Cloudy image arithmetic： a cloudy scene synthesis paradigm with an application to deep-learning-based thin cloud removal ［J］. IEEE Transactions on Geoscience and Remote Sensing ， 2022 ， 60 ： 1 - 16 . doi: 10.1109/tgrs.2021.3122253 http://dx.doi.org/10.1109/tgrs.2021.3122253

Views

427

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Dual scale fusion image dehazing algorithm based on frequency-domain feature distillation

Global and local feature fusion image dehazing

Generate adversarial network for super-resolution reconstruction of remote sensing images by fusing edge enhancement and non-local modules

Image dehazing based on polarization optimization and atmosphere light correction

Two-stage image restoration using improved atmospheric scattering model

Related Author

YANG Shuang

CHEN Qingjiang

ZHU Ming

NIE Haitao

JIANG Xin

HAN Ke

QI Ruo

LIU Jie

Related Institution

Xi'an University of Architecture and Technology， School of Science

Changchun Institute of Optics，Fine Mechanics and Physics，Chinese Academy of Sciences

School of Computer and Information Engineering， Harbin University of Commerce

College of Measurement and Control Technology and Communication Engineering， Harbin University of Science and Technology

Institute of Advanced Technology Innovation， Fuzhou University

AI问答

Address：No.3888 Dong Nanhu Road, Changchun, Jilin, China Postal code：130033
Tel：0431-86176855 Email：gxjmgc@ciomp.ac.cn
Technical support is provided by Beijing Founder electronics co., LTD 吉ICP备11002662号-17 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰