Image reconstruction based on deep compressive sensing combined with global and local features

ZHONG Yuanhong; XU Qianfeng; ZHOU Yujie; WANG Shanshan

doi:10.37188/OPE.20233114.2135

您当前的位置：

首页 >

文章列表页 >

Image reconstruction based on deep compressive sensing combined with global and local features

Information Sciences | 更新时间：2023-08-26

- Image reconstruction based on deep compressive sensing combined with global and local features
- Optics and Precision Engineering Vol. 31, Issue 14, Pages: 2135-2146(2023)
- 作者机构：
  
  1.重庆大学微电子与通信工程学院，重庆 400044
  2.安徽大学物质科学与信息技术研究院，安徽合肥 230039
- 作者简介：
- 基金信息：
- DOI：10.37188/OPE.20233114.2135
  CLC： TP394.1;TH691.9
- Received：06 December 2022，
  
  Revised：05 January 2023，
  
  Published：25 July 2023
- 稿件说明：
移动端阅览
仲元红,徐乾锋,周宇杰等.联合全局与局部特征的深度压缩感知图像重构[J].光学精密工程,2023,31(14):2135-2146.

ZHONG Yuanhong,XU Qianfeng,ZHOU Yujie,et al.Image reconstruction based on deep compressive sensing combined with global and local features[J].Optics and Precision Engineering,2023,31(14):2135-2146.
仲元红,徐乾锋,周宇杰等.联合全局与局部特征的深度压缩感知图像重构[J].光学精密工程,2023,31(14):2135-2146. DOI： 10.37188/OPE.20233114.2135.

ZHONG Yuanhong,XU Qianfeng,ZHOU Yujie,et al.Image reconstruction based on deep compressive sensing combined with global and local features[J].Optics and Precision Engineering,2023,31(14):2135-2146. DOI： 10.37188/OPE.20233114.2135.

摘要

从极少量的测量值中有效且高概率高质量恢复出原始信号是压缩感知图像重建研究的核心问题，学者们相继提出了传统和基于深度学习的压缩感知图像重建算法，传统算法通常基于优化模型迭代求解，重建质量和重建速度都无法保证；基于深度学习的算法重建质量相对较高，但缺乏物理可解释性。受滤波流的启发，本文提出了联合全局与局部的深度压缩感知图像重建模型（G2LNet），其以卷积层执行压缩采样以及初始重建过程，利用快速傅里叶卷积与滤波流，同时考虑了图像全局上下文信息和图像像素局部邻域信息，联合学习优化测量矩阵与滤波流，建立了完整的端到端可训练的深度图像压缩感知重建网络。经实验验证，在压缩感知图像重建领域常用的Set5，Set11，BSD68测试集上取得了良好的重建效果，在采样率为20%的情况下，G2LNet的图像重建质量相比于经典的传统算法MH与基于深度学习的算法CSNet的平均PSNR分别提高了2.29 dB，0.51 dB，有效提升了重建图像质量。

Abstract

Effectively restoring the original signal with high probability and high quality from a very small number of measured values is the core issue of compressive sensing for image reconstruction. Researchers have successively proposed traditional and deep learning-based compressive sensing image reconstruction algorithms. The traditional algorithms are based on mathematical derivation. Although they are comprehensible， their reconstruction quality is relatively poor. On the contrary， deep learning-based algorithms have relatively high reconstruction quality， but they cannot guarantee intelligibility. Inspired by filter flow， this study proposes a global-to-local compressive sensing image reconstruction model called G2LNet， which performs compressed sampling and initial reconstruction processes with convolutional layers using fast Fourier convolution and convolutional filter flow， taking into account the global contextual information of the image and local neighborhood information of the image pixel simultaneously. It learns to jointly optimize the measurement matrix and convolution filter flow and establishes a complete end-to-end trainable deep image reconstruction network. Verification experiments were performed on the Set5， Set11， and BSD68 test datasets commonly used in the field of compressive sensing image reconstruction at a 20% sampling rate. The image reconstruction quality of G2LNet was compared with that of the traditional algorithm MH and algorithm based on deep learning； the average peak signal-to-noise ratio of CSNet increased by 2.29 dB and 0.51 dB， respectively， effectively improving the quality of the reconstructed image.

关键词

Keywords

references

SHANNON C E . Communication in the presence of noise ［J］. Proceedings of the IRE ， 1949 ， 37 （ 1 ）： 10 - 21 . doi: 10.1109/jrproc.1949.232969 http://dx.doi.org/10.1109/jrproc.1949.232969

CANDÈS E J ， WAKIN M B ， BOYD S P . Enhancing sparsity by reweighted ℓ ［J］. Journal of Fourier Analysis and Applications ， 2008 ， 14 （ 5 ）： 877 - 905 . doi: 10.1007/s00041-008-9045-x http://dx.doi.org/10.1007/s00041-008-9045-x

DONOHO D L . Compressed sensing ［J］. IEEE Transactions on Information Theory ， 2006 ， 52 （ 4 ）： 1289 - 1306 . doi: 10.1109/tit.2006.871582 http://dx.doi.org/10.1109/tit.2006.871582

BARANIUK R ， CANDES E ， NOWAK R ， et al . Compressive sampling ［J］. IEEE Signal Processing Magazine ， 2008 ， 25 （ 2 ）： 12 - 13 . doi: 10.1109/msp.2008.915557 http://dx.doi.org/10.1109/msp.2008.915557

SHI W Z ， LIU S H ， JIANG F ， et al . Video compressed sensing using a convolutional neural network ［J］. IEEE Transactions on Circuits and Systems for Video Technology ， 2021 ， 31 （ 2 ）： 425 - 438 . doi: 10.1109/tcsvt.2020.2978703 http://dx.doi.org/10.1109/tcsvt.2020.2978703

ROGERS C A ， POPESCU D C . Compressed sensing MIMO radar system for extended target detection ［J］. IEEE Systems Journal ， 2021 ， 15 （ 1 ）： 1381 - 1389 . doi: 10.1109/jsyst.2020.3000500 http://dx.doi.org/10.1109/jsyst.2020.3000500

黄博，周劼，江舸 . 稳健型双层叠组LASSO逆合成孔径雷达高分辨成像算法［J］. 电子与信息学报， 2021 ， 43 （ 3 ）： 674 - 682 . doi: 10.11999/JEIT200338 http://dx.doi.org/10.11999/JEIT200338

HUANG B ， ZHOU J ， JIANG G . High resolution ISAR imaging algorithm based on robust two-tier group LASSO alternating direction method of multipliers ［J］. Journal of Electronics & Information Technology ， 2021 ， 43 （ 3 ）： 674 - 682 . （in Chinese） . doi: 10.11999/JEIT200338 http://dx.doi.org/10.11999/JEIT200338

KUMAR P A ， GUNASUNDARI R ， AARTHI R . Systematic analysis and review of magnetic resonance imaging （MRI） reconstruction techniques ［J］. Current Medical Imaging Formerly Current Medical Imaging Reviews ， 2021 ， 17 （ 8 ）： 943 - 955 . doi: 10.2174/1573405616666210105125542 http://dx.doi.org/10.2174/1573405616666210105125542

SHI W Z ， JIANG F ， LIU S H ， et al . Image compressed sensing using convolutional neural network ［J］. IEEE Transactions on Image Processing ， 2020 ， 29 ： 375 - 388 . doi: 10.1109/tip.2019.2928136 http://dx.doi.org/10.1109/tip.2019.2928136

MALLAT S G ， ZHANG Z F . Matching pursuits with time-frequency dictionaries ［J］. IEEE Transactions on Signal Processing ， 1993 ， 41 （ 12 ）： 3397 - 3415 . doi: 10.1109/78.258082 http://dx.doi.org/10.1109/78.258082

TROPP J A ， GILBERT A C . Signal recovery from random measurements via orthogonal matching pursuit ［J］. IEEE Transactions on Information Theory ， 2007 ， 53 （ 12 ）： 4655 - 4666 . doi: 10.1109/tit.2007.909108 http://dx.doi.org/10.1109/tit.2007.909108

FORNASIER M ， RAUHUT H . Iterative thresholding algorithms ［J］. Applied and Computational Harmonic Analysis ， 2008 ， 25 （ 2 ）： 187 - 208 . doi: 10.1016/j.acha.2007.10.005 http://dx.doi.org/10.1016/j.acha.2007.10.005

BECK A ， TEBOULLE M . A fast iterative shrinkage-thresholding algorithm for linear inverse problems ［J］. SIAM Journal on Imaging Sciences ， 2009 ， 2 （ 1 ）： 183 - 202 . doi: 10.1137/080716542 http://dx.doi.org/10.1137/080716542

WRIGHT S J ， NOWAK R D ， FIGUEIREDO M A T . Sparse reconstruction by separable approximation ［J］. IEEE Transactions on Signal Processing ， 2009 ， 57 （ 7 ）： 2479 - 2493 . doi: 10.1109/tsp.2009.2016892 http://dx.doi.org/10.1109/tsp.2009.2016892

FIGUEIREDO M A T ， NOWAK R D ， WRIGHT S J . Gradient projection for sparse reconstruction： application to compressed sensing and other inverse problems ［J］. IEEE Journal of Selected Topics in Signal Processing ， 2007 ， 1 （ 4 ）： 586 - 597 . doi: 10.1109/jstsp.2007.910281 http://dx.doi.org/10.1109/jstsp.2007.910281

MUN S ， FOWLER J E . Block Compressed Sensing of Images Using Directional Transforms ［C］. 2010 Data Compression Conference . 24 - 26 ， 2010， Snowbird， UT， USA. IEEE ， 2010： 547 . doi: 10.1109/dcc.2010.90 http://dx.doi.org/10.1109/dcc.2010.90

景宁，姚鼎一，王志斌，等 . 等效时间采样压缩感知高频信号重建［J］. 光学精密工程， 2022 ， 30 （ 10 ）： 1240 - 1245 . doi: 10.37188/OPE.20223010.1240 http://dx.doi.org/10.37188/OPE.20223010.1240

JING N ， YAO D Y ， WANG Z B ， et al . High frequency signal reconstruction based on compressive sensing and equivalent-time sampling ［J］. Opt. Precision Eng. ， 2022 ， 30（ 10 ） 1240 - 1245 . （in Chinese） . doi: 10.37188/OPE.20223010.1240 http://dx.doi.org/10.37188/OPE.20223010.1240

KULKARNI K ， LOHIT S ， TURAGA P ， et al . ReconNet： Non-Iterative Reconstruction of Images From Compressively Sensed Measurements ［C］. 2016 IEEE Conference on Computer Vision and Pattern Recognition （CVPR） . 27 - 30 ， 2016， Las Vegas， NV， USA. IEEE ， 2016： 449 - 458 . doi: 10.1109/cvpr.2016.55 http://dx.doi.org/10.1109/cvpr.2016.55

SHI W Z ， JIANG F ， ZHANG S P ， et al . Deep Networks for Compressed Image Sensing ［C］. 2017 IEEE International Conference on Multimedia and Expo （ICME） . 10 - 14 ， 2017， Hong Kong， China. IEEE ， 2017： 877 - 882 . doi: 10.1109/icme.2017.8019428 http://dx.doi.org/10.1109/icme.2017.8019428

SHI W Z ， JIANG F ， LIU S H ， et al . Scalable Convolutional Neural Network for Image Compressed Sensing ［C］. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR） . 15 - 20 ， 2019， Long Beach， CA， USA. IEEE ， 2020： 12282 - 12291 . doi: 10.1109/cvpr.2019.01257 http://dx.doi.org/10.1109/cvpr.2019.01257

YANG Y ， SUN J ， LI H B ， et al . ADMM-CSNet： a deep learning approach for image compressive sensing ［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence ， 2020 ， 42 （ 3 ）： 521 - 538 . doi: 10.1109/tpami.2018.2883941 http://dx.doi.org/10.1109/tpami.2018.2883941

ZHANG J ， ZHAO C ， GAO W . Optimization-inspired compact deep compressive sensing ［J］. IEEE Journal of Selected Topics in Signal Processing ， 2020 ， 14 （ 4 ）： 765 - 774 . doi: 10.1109/jstsp.2020.2977507 http://dx.doi.org/10.1109/jstsp.2020.2977507

CHEN C ， TRAMEL E W ， FOWLER J E . Compressed-Sensing Recovery of Images and Video Using Multihypothesis Predictions ［C］. 2011 Conference Record of the Forty Fifth Asilomar Conference on Signals， Systems and Computers （ASILOMAR） . 6 - 9 ， 2011， Pacific Grove， CA， USA. IEEE ， 2012： 1193 - 1198 . doi: 10.1109/acssc.2011.6190204 http://dx.doi.org/10.1109/acssc.2011.6190204

ZHANG J ， GHANEM B . ISTA-net： Interpretable Optimization-Inspired Deep Network for Image Compressive Sensing ［C］. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . 18 - 23 ， 2018， Salt Lake City， UT， USA. IEEE ， 2018： 1828 - 1837 . doi: 10.1109/cvpr.2018.00196 http://dx.doi.org/10.1109/cvpr.2018.00196

KRIZHEVSKY A ， SUTSKEVER I ， HINTON G E . ImageNet classification with deep convolutional neural networks ［J］. Communications of the ACM ， 2017 ， 60 （ 6 ）： 84 - 90 . doi: 10.1145/3065386 http://dx.doi.org/10.1145/3065386

HE K M ， ZHANG X Y ， REN S Q ， et al . Deep Residual Learning for Image Recognition ［C］. 2016 IEEE Conference on Computer Vision and Pattern Recognition （CVPR） . 27 - 30 ， 2016， Las Vegas， NV， USA. IEEE ， 2016： 770 - 778 . doi: 10.1109/cvpr.2016.90 http://dx.doi.org/10.1109/cvpr.2016.90

CHI L ， JIANG B ， MU Y . Fast fourier convolution ［J］. Advances in Neural Information Processing Systems ， 2020 ， 33 ： 4479 - 4488 .

SEITZ S M ， BAKER S . Filter Flow ［C］. 2009 IEEE 12th International Conference on Computer Vision. September 29 - October 2 ， 2009 ， Kyoto， Japan. IEEE ， 2010 ： 143 - 150 . doi: 10.1109/iccv.2009.5459155 http://dx.doi.org/10.1109/iccv.2009.5459155

SHI W Z ， CABALLERO J ， HUSZÁR F ， et al . Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network ［C］. 2016 IEEE Conference on Computer Vision and Pattern Recognition （CVPR） . 27 - 30 ， 2016， Las Vegas， NV， USA. IEEE ， 2016： 1874 - 1883 . doi: 10.1109/cvpr.2016.207 http://dx.doi.org/10.1109/cvpr.2016.207

YANG J C ， WRIGHT J ， HUANG T S ， et al . Image super-resolution via sparse representation ［J］. IEEE Transactions on Image Processing： a Publication of the IEEE Signal Processing Society ， 2010 ， 19 （ 11 ）： 2861 - 2873 . doi: 10.1109/tip.2010.2050625 http://dx.doi.org/10.1109/tip.2010.2050625

BEVILACQUA M ， ROUMY A ， GUILLEMOT C ， et al . Low-Complexity Single-Image Super-Resolution Based on Nonnegative Neighbor Embedding ［C］. Proceedings of the British Machine Vision Conference 2012. Surrey. British Machine Vision Association ， 2012 . doi: 10.5244/c.26.135 http://dx.doi.org/10.5244/c.26.135

BO L ， LU H C ， LU Y J ， et al . FompNet： Compressive Sensing Reconstruction with Deep Learning over Wireless Fading Channels ［C］. 2017 9th International Conference on Wireless Communications and Signal Processing （WCSP） . 11 - 13 ， 2017， Nanjing， China. IEEE ， 2017： 1 - 6 . doi: 10.1109/wcsp.2017.8171076 http://dx.doi.org/10.1109/wcsp.2017.8171076

MARTIN D ， FOWLKES C ， TAL D ， et al . A Database of Human Segmented Natural Images and its Application to Evaluating Segmentation Algorithms and Measuring Ecological Statistics ［C］. Proceedings Eighth IEEE International Conference on Computer Vision. ICCV.7 - 14 ， 2001， Vancouver， BC， Canada. IEEE ， 2002： 416 - 423 . doi: 10.1109/iccv.2001.937655 http://dx.doi.org/10.1109/iccv.2001.937655

WANG Z ， BOVIK A C ， SHEIKH H R ， et al . Image quality assessment： from error visibility to structural similarity ［J］. IEEE Transactions on Image Processing ， 2004 ， 13 （ 4 ）： 600 - 612 . doi: 10.1109/tip.2003.819861 http://dx.doi.org/10.1109/tip.2003.819861

Views

437

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Iterative reconstruction of compressive sensing combining image hierarchical-feature

Super-resolution strain measurement in phase-contrast optical coherence elastography

Optical remote sensing road extraction network based on GCN guided model viewpoint

Ship detection oriented to compressive sensing measurements of space optical remote sensing scenes

Lens-less imaging via score-based generative model

Related Author

LIU Yuhong

YANG Heng

ZHANG Zhanhua

CAO Xin

ZHAN Weihao

DONG Bo

XIE Shengli

BAI Yulei

Related Institution

School of Electronic and Information Engineering， Lanzhou Jiaotong University

School of Automation， Guangdong University of Technology

Xi'an Key Laboratory of Intelligent Technology for Building and Manufacturing

College of Information and Control Engineering，Xi'an University of Architecture and Technology

University of Chinese Academy of Sciences

Address：No.3888 Dong Nanhu Road, Changchun, Jilin, China Postal code：130033
Tel：0431-86176855 Email：gxjmgc@ciomp.ac.cn
Technical support is provided by Beijing Founder electronics co., LTD 吉ICP备11002662号-17 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰