Dual attention refinement single image desnowing

SHI Mingzhu; ZAO Bin; SU Yuhao; LIN Xinhui; KONG Siqi; TAN Muxian

doi:10.37188/OPE.20243212.1954

您当前的位置：

首页 >

文章列表页 >

Dual attention refinement single image desnowing

Information Sciences | 更新时间：2024-07-26

- Dual attention refinement single image desnowing
- Optics and Precision Engineering Vol. 32, Issue 12, Pages: 1954-1964(2024)
- 作者机构：
  
  1.天津师范大学电子与通信工程学院，天津 300387
  2.天津无线移动通信与无线电能传输重点实验室，天津 300387
- 作者简介：
- 基金信息：
- DOI：10.37188/OPE.20243212.1954
  CLC： TP394.1;TH691.9
- Published：25 June 2024，
  
  Received：13 November 2023，
  
  Revised：12 December 2023，
- 稿件说明：
移动端阅览
石明珠,糟斌,苏宇皓等.联合图像通道与像素双注意力机制精细化单幅图像去雪[J].光学精密工程,2024,32(12):1954-1964.

SHI Mingzhu,ZAO Bin,SU Yuhao,et al.Dual attention refinement single image desnowing[J].Optics and Precision Engineering,2024,32(12):1954-1964.
石明珠,糟斌,苏宇皓等.联合图像通道与像素双注意力机制精细化单幅图像去雪[J].光学精密工程,2024,32(12):1954-1964. DOI： 10.37188/OPE.20243212.1954.

SHI Mingzhu,ZAO Bin,SU Yuhao,et al.Dual attention refinement single image desnowing[J].Optics and Precision Engineering,2024,32(12):1954-1964. DOI： 10.37188/OPE.20243212.1954.

摘要

针对雪天退化图像中不规则和多变的雪花形态，提出一种双注意力机制的精细化图像去雪网络（Dual Attention Refinement Desnowing Network， DARDNet）。网络引入维度拆分处理策略，并行处理通道和像素双维度特征，旨在有效配置两种注意力机制，兼顾提取复杂特征和保护纹理细节。其中，通道注意力机制针对雪花形态构建基础模块，形成U型金字塔结构分层提取深层次特征；像素注意力机制结合卷积形成自校准模块，串联高效Transformer关注图像纹理细节；两种注意力机制并行化处理后进行特征融合，提升信息融合度。在CSD，SRRS和Snow100K三个数据集上进行验证测试，其中在CSD数据集上PSNR达到32.16 dB，SSIM达到0.96。本文方法在处理多种雪花形态方面具有一定优势，能很好地重建纹理细节，获得高质量的去雪图像。

Abstract

Snow degradation is complex and variable， including various snowflakes， snow spots and snow streaks. To this end， we proposed a dual attention refinement desnowing network （DARDNet）. The network introduced a dimensional splitting strategy to handle two-dimensional features of channel and pixel in parallel， aiming to achieve a good trade-off between complex features and texture details. The channel attention mechanism built a module for the multiple degradation and forms a U-shaped pyramid structure to extract the depth features； the pixel attention mechanism combined the convolution to form the self-calibration module， and connected the efficient Transformer to preserve texture details； The parallel processed information streams were fused to improve the reconstruction quality of the image. Experiments were carried out on CSD， SRRS and Snow100K datasets， where PSNR reached 32.56 dB and SSIM reached 0.96 on CSD dataset. The experimental results show that our proposed method has obvious advantages in dealing with various snow degradations， which can better reconstruct the detail information and achieve satisfactory snow removal results.

关键词

单幅图像去雪通道注意力机制像素注意力机制深度图像先验

Keywords

single image desnowingchannel attentionpixel attentiondeep image prior

references

BOSSU J， HAUTIÈRE N， TAREL J P. Rain or snow detection in image sequences through use of a Histogram of orientation of streaks［J］. International Journal of Computer Vision， 2011， 93（3）： 348-367. doi: 10.1007/s11263-011-0421-7http://dx.doi.org/10.1007/s11263-011-0421-7

ZHENG X H， LIAO Y H， GUO W， et al. Single-image-based rain and snow removal using multi-guided filter［C］. International Conference on Neural Information Processing. Berlin， Heidelberg： Springer， 2013： 258-265. doi: 10.1007/978-3-642-42051-1_33http://dx.doi.org/10.1007/978-3-642-42051-1_33

PEI S C， TSAI Y T， LEE C Y. Removing rain and snow in a single image using saturation and visibility features［C］. 2014 IEEE International Conference on Multimedia and Expo Workshops （ICMEW）. Chengdu. IEEE， 2014： 1-6. doi: 10.1109/icmew.2014.6890551http://dx.doi.org/10.1109/icmew.2014.6890551

RAJDERKAR D， MOHOD P S. Removing snow from an image via image decomposition［C］. 2013 IEEE International Conference on Emerging Trends in Computing， Communication and Nanotechnology （ICECCN）. Tirunelveli， India. IEEE， 2013： 576-579. doi: 10.1109/ice-ccn.2013.6528565http://dx.doi.org/10.1109/ice-ccn.2013.6528565

LIU Y F， JAW D W， HUANG S C， et al. DesnowNet： context-aware deep network for snow removal［J］. IEEE Transactions on Image Processing， 2018， 27（6）： 3064-3073. doi: 10.1109/tip.2018.2806202http://dx.doi.org/10.1109/tip.2018.2806202

CHEN W T， FANG H Y， DING J J， et al. JSTASR： joint size and transparency-aware snow removal algorithm based on modified partial convolution and veiling effect removal［C］. VEDALDI A， BISCHOF H， BROX T， et al. European Conference on Computer Vision. Cham： Springer， 2020： 754-770. doi: 10.1007/978-3-030-58589-1_45http://dx.doi.org/10.1007/978-3-030-58589-1_45

CHEN W T， FANG H Y， HSIEH C L， et al. ALL snow removed： single image desnowing algorithm using hierarchical dual-tree complex wavelet representation and contradict channel loss［C］. 2021 IEEE/CVF International Conference on Computer Vision （ICCV）. Montreal， QC， Canada. IEEE， 2021： 4176-4185. doi: 10.1109/iccv48922.2021.00416http://dx.doi.org/10.1109/iccv48922.2021.00416

WANG Y L， MA C， LIU J Z. SmartAssign： learning a smart knowledge assignment strategy for deraining and desnowing［C］. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Vancouver， BC， Canada. IEEE， 2023： 3677-3686. doi: 10.1109/cvpr52729.2023.00358http://dx.doi.org/10.1109/cvpr52729.2023.00358

WOO S， PARK J， LEE J Y， et al. CBAM： convolutional block attention module［C］. European Conference on Computer Vision. Cham： Springer， 2018： 3-19. doi: 10.1007/978-3-030-01234-2_1http://dx.doi.org/10.1007/978-3-030-01234-2_1

FU J， LIU J， TIAN H J， et al. Dual attention network for scene segmentation［C］. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Long Beach， CA， USA. IEEE， 2019： 3141-3149. doi: 10.1109/cvpr.2019.00326http://dx.doi.org/10.1109/cvpr.2019.00326

DING M Y， XIAO B， CODELLA N， et al. DaViT： dual attention vision transformers［C］. European Conference on Computer Vision. Cham： Springer， 2022： 74-92. doi: 10.1007/978-3-031-20053-3_5http://dx.doi.org/10.1007/978-3-031-20053-3_5

HU J， SHEN L， SUN G. Squeeze-and-excitation networks［C］. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City， UT， USA. IEEE， 2018： 7132-7141. doi: 10.1109/cvpr.2018.00745http://dx.doi.org/10.1109/cvpr.2018.00745

CHEN L Y， CHU X J， ZHANG X Y， et al. Simple baselines for image restoration［C］. European Conference on Computer Vision. Cham： Springer， 2022： 17-33. doi: 10.1007/978-3-031-20071-7_2http://dx.doi.org/10.1007/978-3-031-20071-7_2

LI Y W， FAN Y C， XIANG X Y， et al. Efficient and explicit modelling of image hierarchies for image restoration［C］. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Vancouver， BC， Canada. IEEE， 2023： 18278-18289. doi: 10.1109/cvpr52729.2023.01753http://dx.doi.org/10.1109/cvpr52729.2023.01753

ZHAO H Y， KONG X T， HE J W， et al. Efficient image super-resolution using pixel attention［C］. European Conference on Computer Vision. Cham： Springer， 2020： 56-72. doi: 10.1007/978-3-030-67070-2_3http://dx.doi.org/10.1007/978-3-030-67070-2_3

ZOU W B， YE T， ZHENG W X， et al. Self-calibrated efficient transformer for lightweight super-resolution［C］. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops （CVPRW）. New Orleans， LA， USA. IEEE， 2022： 929-938. doi: 10.1109/cvprw56347.2022.00107http://dx.doi.org/10.1109/cvprw56347.2022.00107

RONNEBERGER O， FISCHER P， BROX T. U-net： convolutional networks for biomedical image segmentation［C］. International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham： Springer， 2015： 234-241. doi: 10.1007/978-3-319-24574-4_28http://dx.doi.org/10.1007/978-3-319-24574-4_28

ZAMIR S W， ARORA A， KHAN S， et al. Restormer： efficient transformer for high-resolution image restoration［C］. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. New Orleans， LA， USA. IEEE， 2022： 5718-5729. doi: 10.1109/cvpr52688.2022.00564http://dx.doi.org/10.1109/cvpr52688.2022.00564

CHEN L Y， LU X， ZHANG J， et al. HINet： half instance normalization network for image restoration［C］. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops （CVPRW）. Nashville， TN， USA. IEEE， 2021： 182-192. doi: 10.1109/cvprw53098.2021.00027http://dx.doi.org/10.1109/cvprw53098.2021.00027

WANG Z， BOVIK A C， SHEIKH H R， et al. Image quality assessment： from error visibility to structural similarity［J］. IEEE Transactions on Image Processing， 2004， 13（4）： 600-612. doi: 10.1109/tip.2003.819861http://dx.doi.org/10.1109/tip.2003.819861

ENGIN D， GENC A， EKENEL H K. Cycle-dehaze： enhanced cyclegan for single image dehazing［C］. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops （CVPRW）. Salt Lake City， UT， USA. IEEE， 2018： 938-9388. doi: 10.1109/cvprw.2018.00127http://dx.doi.org/10.1109/cvprw.2018.00127

LI X， WU J L， LIN Z C， et al. Recurrent squeeze-and-excitation context aggregation net for single image deraining［C］. European Conference on Computer Vision. Cham： Springer， 2018： 262-277. doi: 10.1007/978-3-030-01234-2_16http://dx.doi.org/10.1007/978-3-030-01234-2_16

LI R T， TAN R T， CHEONG L F. All in one bad weather removal using architectural search［C］. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Seattle， WA， USA. IEEE， 2020： 3172-3182. doi: 10.1109/cvpr42600.2020.00324http://dx.doi.org/10.1109/cvpr42600.2020.00324

JOSE VALANARASU J M， YASARLA R， PATEL V M. TransWeather： transformer-based restoration of images degraded by adverse weather conditions［C］. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. New Orleans， LA， USA. IEEE， 2022： 2343-2353. doi: 10.1109/cvpr52688.2022.00239http://dx.doi.org/10.1109/cvpr52688.2022.00239

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

No data

Related Author

No data

Related Institution

No data

Address：No.3888 Dong Nanhu Road, Changchun, Jilin, China Postal code：130033
Tel：0431-86176855 Email：gxjmgc@ciomp.ac.cn
Technical support is provided by Beijing Founder electronics co., LTD 吉ICP备11002662号-17 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰