Image super-resolution reconstruction based on attention and wide-activated dense residual network

KOU Qiqi; LI Chao; CHENG Deqiang; CHEN Liangliang; MA Haohui; ZHANG Jianying

doi:10.37188/OPE.20233115.2273

您当前的位置：

首页 >

文章列表页 >

Image super-resolution reconstruction based on attention and wide-activated dense residual network

Information Sciences | 更新时间：2023-08-28

- Image super-resolution reconstruction based on attention and wide-activated dense residual network
- Optics and Precision Engineering Vol. 31, Issue 15, Pages: 2273-2286(2023)
- 作者机构：
  
  1.中国矿业大学计算机科学与技术学院，江苏徐州 221116
  2.中国矿业大学信息与控制工程学院，江苏徐州 221116
- 作者简介：
- 基金信息：
- DOI：10.37188/OPE.20233115.2273
  CLC：
扫描看全文
KOU Qiqi, LI Chao, CHENG Deqiang, et al. Image super-resolution reconstruction based on attention and wide-activated dense residual network. [J]. Optics and Precision Engineering 31(15):2273-2286(2023)
DOI：

KOU Qiqi, LI Chao, CHENG Deqiang, et al. Image super-resolution reconstruction based on attention and wide-activated dense residual network. [J]. Optics and Precision Engineering 31(15):2273-2286(2023) DOI： 10.37188/OPE.20233115.2273.

摘要

针对全局和局部高低频空间信息利用不足而导致重建图像纹理细节模糊的问题，提出一种基于注意力和宽激活密集残差网络的图像超分辨率重建模型。首先，四个不同尺度且平行的卷积核被用来充分提取图像低频特征作为空间特征转换的先验信息。在深层特征映射模块中构建融合注意力的宽激活残差块，并利用低频先验信息来引导高频特征的提取。该宽激活残差块通过扩大激活函数前的特征通道数来提取更深层次的特征图，且所构造的全局和局部残差连接在加强残差块和网络特征前向传播的同时，在不增加参数情况下使得所提取特征的多样性更加丰富。最后，对得到的特征图进行上采样和重建以得到清晰的高分辨率图像。实验表明，所提算法在BSD100数据集上4倍超分辨率时，相比LatticeNet模型的PSNR指标提升了0.14 dB，SSIM提升了0.001，在主观视觉方面，重建出的图像局部纹理细节也更加清晰。

Abstract

To address the problem of the blurring of the texture details of reconstructed images due to the insufficient utilization of global and local high- and low-frequency spatial information， this paper proposes an image super-resolution reconstruction model based on attention and a wide-activated dense residual network. First， four parallel convolution kernels with different scales are used to fully extract the low-frequency features of the image as the prior information for spatial feature transformation. Second， a wide-activated residual block fused with attention is constructed in the deep feature mapping module， and the low-frequency prior information is used to guide the extraction of the high-frequency features. In addition， the wide-activated residual block extracts deeper feature maps by expanding the number of feature channels before the activation function. As a result， the constructed global and local residual connections not only strengthen the forward propagation of the residual blocks and network features， but also enrich the diversity of the extracted features without increasing the number of parameters. Finally， the feature map is upsampled and reconstructed to obtain a clear high-resolution image. the experimental results show that compared with the LatticeNet model， the peak signal-to-noise ratio of the proposed algorithm is improved by 0.14 dB， and the structural similarity is improved by 0.001 at 4× super resolution on the BSD100 dataset. In addition， the local texture details of the reconstructed image are also clearer in subjective visualization.

关键词

残差网络超分辨率宽激活注意力机制密集连接

Keywords

residual networksuper-resolutionwide activationattention mechanismdense connectivity

references

耿铭昆，吴凡路，王栋. 轻量化火星遥感影像超分辨率重建网络［J］. 光学精密工程， 2022， 30（12）：1487-1498. doi: 10.37188/OPE.20223012.1487http://dx.doi.org/10.37188/OPE.20223012.1487

GENG M K， WU F L， WANG D. Lightweight Mars remote sensing image super-resolution reconstruction network［J］. Opt. Precision Eng.， 2022， 30（12）：1487-1498.（in Chinese）. doi: 10.37188/OPE.20223012.1487http://dx.doi.org/10.37188/OPE.20223012.1487

李勇，王珂，张立保，等. 多断层融合的肺CT肿瘤靶区超分辨率重建［J］. 光学精密工程， 2010， 18（5）：1212-1218. doi: 10.3788/OPE.20101805.1212http://dx.doi.org/10.3788/OPE.20101805.1212

LI Y， WANG K， ZHANG L B， et al. Super-resolution reconstruction of pulmonary nodules based on CT multi-section fusion［J］. Opt. Precision Eng.， 2010， 18（5）：1212-1218.（in Chinese）. doi: 10.3788/OPE.20101805.1212http://dx.doi.org/10.3788/OPE.20101805.1212

王杰，徐国明，马健，等. 轻量级注意力级联网络的偏振计算成像超分辨率重建［J］. 光学精密工程， 2022， 30（19）：2404-2419. doi: 10.37188/OPE.20223019.2404http://dx.doi.org/10.37188/OPE.20223019.2404

WANG J， XU G M， MA J， et al. Polarization computational imaging super-resolution reconstruction with lightweight attention cascading network［J］. Opt. Precision Eng.， 2022， 30（19）：2404-2419.（in Chinese）. doi: 10.37188/OPE.20223019.2404http://dx.doi.org/10.37188/OPE.20223019.2404

LI X， ORCHARD M T. New edge-directed interpolation［J］. IEEE Transactions on Image Processing， 2001， 10（10）： 1521-1527. doi: 10.1109/83.951537http://dx.doi.org/10.1109/83.951537

LIU X M， ZHAO D B， ZHOU J T， et al. Image interpolation via graph-based Bayesian label propagation［J］. IEEE Transactions on Image Processing， 2014， 23（3）： 1084-1096. doi: 10.1109/tip.2013.2294543http://dx.doi.org/10.1109/tip.2013.2294543

DAI S Y， HAN M， XU W， et al. SoftCuts： a soft edge smoothness prior for color image super-resolution［J］. IEEE Transactions on Image Processing， 2009， 18（5）： 969-981. doi: 10.1109/tip.2009.2012908http://dx.doi.org/10.1109/tip.2009.2012908

DONG C， LOY C C， HE K M， et al. Image super-resolution using deep convolutional networks［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2016， 38（2）： 295-307. doi: 10.1109/tpami.2015.2439281http://dx.doi.org/10.1109/tpami.2015.2439281

SHI W Z， CABALLERO J， HUSZÁR F， et al. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network［C］. 2016 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）.2730，2016， Las Vegas， NV， USA. IEEE， 2016： 1874-1883. doi: 10.1109/cvpr.2016.207http://dx.doi.org/10.1109/cvpr.2016.207

KIM J， LEE J K， LEE K M. Accurate image super-resolution using very deep convolutional networks［C］. 2016 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）.2730，2016， Las Vegas， NV， USA. IEEE， 2016： 1646-1654. doi: 10.1109/cvpr.2016.182http://dx.doi.org/10.1109/cvpr.2016.182

HE K M， ZHANG X Y， REN S Q， et al. Deep residual learning for image recognition［C］. 2016 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）.2730，2016， Las Vegas， NV， USA. IEEE， 2016： 770-778. doi: 10.1109/cvpr.2016.90http://dx.doi.org/10.1109/cvpr.2016.90

KIM J， LEE J K， LEE K M. Deeply-recursive convolutional network for image super-resolution［C］. 2016 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）.2730，2016， Las Vegas， NV， USA. IEEE， 2016： 1637-1645. doi: 10.1109/cvpr.2016.181http://dx.doi.org/10.1109/cvpr.2016.181

TAI Y， YANG J， LIU X M. Image super-resolution via deep recursive residual network［C］. 2017 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）.2126，2017， Honolulu， HI， USA. IEEE， 2017： 2790-2798. doi: 10.1109/cvpr.2017.298http://dx.doi.org/10.1109/cvpr.2017.298

LIM B， SON S， KIM H， et al. Enhanced deep residual networks for single image super-resolution［C］. 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops （CVPRW）.2126，2017， Honolulu， HI， USA. IEEE， 2017： 1132-1140. doi: 10.1109/cvprw.2017.151http://dx.doi.org/10.1109/cvprw.2017.151

HUI Z， GAO X B， YANG Y C， et al. Lightweight image super-resolution with information multi-distillation network［C］. Proceedings of the 27th ACM International Conference on Multimedia. October 21 - 25， 2019， Nice， France. New York： ACM， 2019： 2024-2032. doi: 10.1145/3343031.3351084http://dx.doi.org/10.1145/3343031.3351084

LUO X T， XIE Y， ZHANG Y L， et al. LatticeNet： Towards Lightweight Image Super-resolution with Lattice Block［M］. Computer Vision - ECCV 2020. Cham： Springer International Publishing， 2020： 272-289. doi: 10.1007/978-3-030-58542-6_17http://dx.doi.org/10.1007/978-3-030-58542-6_17

TONG T， LI G， LIU X J， et al. Image super-resolution using dense skip connections［C］. 2017 IEEE International Conference on Computer Vision （ICCV）.2229，2017， Venice， Italy. IEEE， 2017： 4809-4817. doi: 10.1109/iccv.2017.514http://dx.doi.org/10.1109/iccv.2017.514

ZHANG Y L， TIAN Y P， KONG Y， et al. Residual dense network for image super-resolution［C］. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 1823，2018， Salt Lake City， UT， USA. IEEE， 2018： 2472-2481. doi: 10.1109/cvpr.2018.00262http://dx.doi.org/10.1109/cvpr.2018.00262

DAI T， CAI J R， ZHANG Y B， et al. Second-order attention network for single image super-resolution［C］. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）.1520，2019， Long Beach， CA， USA. IEEE， 2020： 11057-11066. doi: 10.1109/cvpr.2019.01132http://dx.doi.org/10.1109/cvpr.2019.01132

WANG X L， GIRSHICK R， GUPTA A， et al. Non-local neural networks［C］. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 1823，2018， Salt Lake City， UT， USA. IEEE， 2018： 7794-7803. doi: 10.1109/cvpr.2018.00813http://dx.doi.org/10.1109/cvpr.2018.00813

CHEN H， GU J， ZHANG Z. Attention in attention network for image super-resolution［EB/OL］. 2021： arXiv： 2104.09497. https：//arxiv.org/abs/2104.09497https://arxiv.org/abs/2104.09497. doi: 10.48550/arXiv.2104.09497http://dx.doi.org/10.48550/arXiv.2104.09497

LEDIG C， THEIS L， HUSZÁR F， et al. Photo-realistic single image super-resolution using a generative adversarial network［C］. 2017 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）.2126，2017， Honolulu， HI， USA. IEEE， 2017： 105-114. doi: 10.1109/cvpr.2017.19http://dx.doi.org/10.1109/cvpr.2017.19

YU J， FAN Y， YANG J， et al. Wide activation for efficient and accurate image super-resolution［J］. arXiv preprint arXiv： 1808. 08718， 2018.

SZEGEDY C， LIU W， JIA Y Q， et al. Going deeper with convolutions［C］. 2015 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）.712，2015， Boston， MA， USA. IEEE， 2015： 1-9. doi: 10.1109/cvpr.2015.7298594http://dx.doi.org/10.1109/cvpr.2015.7298594

LI X， WANG W H， HU X L， et al. Selective kernel networks［C］. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）.1520，2019， Long Beach， CA， USA. IEEE， 2020： 510-519. doi: 10.1109/cvpr.2019.00060http://dx.doi.org/10.1109/cvpr.2019.00060

WANG X T， YU K， DONG C， et al. Recovering realistic texture in image super-resolution by deep spatial feature transform［C］. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 1823，2018， Salt Lake City， UT， USA. IEEE， 2018： 606-615. doi: 10.1109/cvpr.2018.00070http://dx.doi.org/10.1109/cvpr.2018.00070

ZEYDE R， ELAD M， PROTTER M. On Single Image Scale-up Using Sparse-representations［M］. Curves and Surfaces. Berlin， Heidelberg： Springer Berlin Heidelberg， 2012： 711-730. doi: 10.1007/978-3-642-27413-8_47http://dx.doi.org/10.1007/978-3-642-27413-8_47

MARTIN D， FOWLKES C， TAL D， et al. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics［C］. Proceedings Eighth IEEE International Conference on Computer Vision. ICCV.714，2001， Vancouver， BC， Canada. IEEE， 2002： 416-423.

HUANG J B， SINGH A， AHUJA N. Single image super-resolution from transformed self-exemplars［C］. 2015 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）.712，2015， Boston， MA， USA. IEEE， 2015： 5197-5206. doi: 10.1109/cvpr.2015.7299156http://dx.doi.org/10.1109/cvpr.2015.7299156

LAI W S， HUANG J B， AHUJA N， et al. Deep Laplacian pyramid networks for fast and accurate super-resolution［C］. 2017 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）.2126，2017， Honolulu， HI， USA. IEEE， 2017： 5835-5843. doi: 10.1109/cvpr.2017.618http://dx.doi.org/10.1109/cvpr.2017.618

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Multidimensional attention mechanism and selective feature fusion for image super-resolution reconstruction

Multi-stage frame alignment video super- resolution network

Generate adversarial network for super-resolution reconstruction of remote sensing images by fusing edge enhancement and non-local modules

Weather recognition combining improved ConvNeXt models with knowledge distillation

Low-light image enhancement algorithm based on multi-channel fusion attention network

Related Author

No data

Related Institution

Yunnan Police Unmanned System Innovation Research Institute， Yunnan Police Officer Academy

Faculty of Information Engineering and Automation， Kunming University of Science and Technology

Faculty of Mechanical and Electrical Engineering， Kunming University of Science and Technology

School of Computer and Information Engineering， Harbin University of Commerce

College of Measurement and Control Technology and Communication Engineering， Harbin University of Science and Technology

Address：No.3888 Dong Nanhu Road, Changchun, Jilin, China Postal code：130033
Tel：0431-86176855 Email：gxjmgc@ciomp.ac.cn
Technical support is provided by Beijing Founder electronics co., LTD 吉ICP备11002662号-17 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰