1.哈尔滨理工大学 测控技术与通信工程学院,黑龙江 哈尔滨 150080
2.哈尔滨商业大学 计算机与信息工程学院,黑龙江 哈尔滨 150028
扫 描 看 全 文
LIU Jie, QI Ruo, HAN Ke. Generate adversarial network for super-resolution reconstruction of remote sensing images by fusing edge enhancement and non-local modules. [J]. Optics and Precision Engineering 31(14):2080-2092(2023)
LIU Jie, QI Ruo, HAN Ke. Generate adversarial network for super-resolution reconstruction of remote sensing images by fusing edge enhancement and non-local modules. [J]. Optics and Precision Engineering 31(14):2080-2092(2023) DOI: 10.37188/OPE.20233114.2080.
针对遥感图像成像过程中噪声污染严重,超分辨率重建图像存在目标边缘模糊和伪影等问题,本文提出一种融合边缘增强与非局部模块的遥感图像超分辨率算法(Edge-Enhanced and Non-local Modules Generative Adversarial Network,ENGAN)。为了使图像细节边缘更清晰,本文融合一种图像边缘增强模块;同时,为进一步扩大模型感受野和增强去除边缘噪声性能,改进边缘增强模块中的Mask分支;此外,引入非局部模块,通过更好地利用图像的内在特征相关性,进一步提升了网络的重建性能。本文在UCAS-AOD和NWPU VHR-10两种遥感图像数据集上进行多个算法的对比实验,结果表明本文提出的方法在多个评价指标上均有所改善。以退化类型Ⅳ为例,本文方法相比深度盲超分辨率退化模型,4倍超分辨率的SSIM提升了0.068,PSNR提升了1.400 dB,RMSE减少了12.5%,且重建后的遥感图像相较于原始图像可以得到更好的地面目标检测结果。
To address the serious noise pollution in the process of image remote sensing and the existence of object edge blur and artifacts in the super-resolution reconstructed image, this study proposes a remote sensing image super-resolution algorithm called edge-enhanced and non-local modules generative adversarial network (ENGAN). To make the image edge details clearer, the proposed algorithm integrated an image edge enhancement module. To further expand the receptive field of the model and enhance the edge noise removal, the Mask branch in the edge enhancement module was simultaneously improved. The use of the intrinsic feature correlation of images further improved the reconstruction performance of the network. In this study, comparison experiments of multiple algorithms were performed on two remote sensing image datasets, UCAS-AOD and NWPU VHR-10. The proposed method showed improvement in multiple evaluation indicators. Taking degradation type IV as an example, the 4x super-resolution SSIM was increased by 0.068, PSNR increased by 1.400 dB, and RMSE reduced by 12.5% compared with the deep-blind super-resolution degradation model. Moreover, the reconstructed remote sensing image can obtain better ground target detection results than the original image.
遥感图像超分辨率边缘增强非局部特征生成对抗网络
remote sensing imagesuper-resolutionedge enhancementnon-local featuresgenerative adversarial network
孙家抦, 舒宁, 关泽群. 遥感原理、方法和应用[M]. 北京: 测绘出版社, 1997.
SUN J B, SHU N, GUAN Z Q. Principle, Method and Application of Remote Sensing[M]. Beijing: Sino Maps Press, 1997.(in Chinese)
柏宇阳. 基于卷积神经网络的遥感图像超分辨重建[D]. 哈尔滨: 黑龙江大学, 2021.
BAI Y Y. Super-Resolution Reconstruction of Remote Sensing Image Based on Convolutional Neural Network[D]. Harbin: Helongjiang University, 2021. (in Chinese)
唐艳秋, 潘泓, 朱亚平, 等. 图像超分辨率重建研究综述[J]. 电子学报, 2020, 48(7):1407-1420. doi: 10.3969/j.issn.0372-2112.2020.07.022http://dx.doi.org/10.3969/j.issn.0372-2112.2020.07.022
TANG Y Q, PAN H, ZHU Y P, et al. A survey of image super-resolution reconstruction[J]. Acta Electronica Sinica, 2020, 48(7):1407-1420.(in Chinese). doi: 10.3969/j.issn.0372-2112.2020.07.022http://dx.doi.org/10.3969/j.issn.0372-2112.2020.07.022
熊鹰飞. 基于生成对抗网络的多源跨区域遥感图像超分辨[D]. 深圳: 中国科学院大学(中国科学院深圳先进技术研究院), 2021.
XIONG Y F. Super-Resolution of Multi-Source and Cross-Regional Remote Sensing Images Based on Generative Countermeasure Network[D]. Shenzhen: Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, 2021. (in Chinese)
邵旻昊. 基于深度学习的遥感图像超分辨率重建算法研究[D]. 西安: 西安电子科技大学, 2021.
SHAO M H. Research on Super-Resolution Reconstruction Algorithm of Remote Sensing Image Based on Deep Learning[D]. Xi'an: Xidian University, 2021. (in Chinese)
DONG C, LOY C C, HE K M, et al. Learning a Deep Convolutional Network for Image Super-Resolution[M]. Computer Vision - ECCV 2014. Cham: Springer International Publishing, 2014: 184-199. doi: 10.1007/978-3-319-10593-2_13http://dx.doi.org/10.1007/978-3-319-10593-2_13
蔡体健, 彭潇雨, 石亚鹏, 等. 通道注意力与残差级联的图像超分辨率重建[J]. 光学 精密工程, 2021, 29(1):142-151. doi: 10.37188/OPE.20212901.0142http://dx.doi.org/10.37188/OPE.20212901.0142
CAI T J, PENG X Y, SHI Y P, et al. Channel attention and residual concatenation network for image super-resolution[J]. Opt. Precision Eng., 2021, 29(1):142-151.(in Chinese). doi: 10.37188/OPE.20212901.0142http://dx.doi.org/10.37188/OPE.20212901.0142
刘雪岩, 许聿达, 雷建昕, 等. 基于视差放大与超分辨率的三维光场腹腔镜标定[J]. 光学 精密工程, 2022, 30(5):510-517. doi: 10.37188/OPE.2021.0332http://dx.doi.org/10.37188/OPE.2021.0332
LIU X Y, XU Y D, LEI J X, et al. Three-dimensional light field endoscope calibration based on light field disparity amplifier and super-resolution network[J]. Opt. Precision Eng., 2022, 30(5):510-517.(in Chinese). doi: 10.37188/OPE.2021.0332http://dx.doi.org/10.37188/OPE.2021.0332
耿铭昆, 吴凡路, 王栋. 轻量化火星遥感影像超分辨率重建网络[J]. 光学 精密工程, 2022, 30(12):1487-1498. doi: 10.37188/OPE.20223012.1487http://dx.doi.org/10.37188/OPE.20223012.1487
GENG M K, WU F L, WANG D. Lightweight Mars remote sensing image super-resolution reconstruction network[J]. Opt. Precision Eng., 2022, 30(12):1487-1498.(in Chinese). doi: 10.37188/OPE.20223012.1487http://dx.doi.org/10.37188/OPE.20223012.1487
WANG X T, YU K, WU S X, et al. ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks[M]. Lecture Notes in Computer Science. Cham: Springer International Publishing, 2019: 63-79. doi: 10.1007/978-3-030-11021-5_5http://dx.doi.org/10.1007/978-3-030-11021-5_5
JI X Z, CAO Y, TAI Y, et al. Real-World Super-Resolution via Kernel Estimation and Noise Injection[C]. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).14-19, 2020, Seattle, WA, USA. IEEE, 2020: 1914-1923. doi: 10.1109/cvprw50498.2020.00241http://dx.doi.org/10.1109/cvprw50498.2020.00241
XU Y Y, LUO W, HU A N, et al. TE-SAGAN: an improved generative adversarial network for remote sensing super-resolution images[J]. Remote Sensing, 2022, 14(10): 2425. doi: 10.3390/rs14102425http://dx.doi.org/10.3390/rs14102425
WANG X T, XIE L B, DONG C, et al. Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data[C]. 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW).11-17, 2021, Montreal, BC, Canada. IEEE, 2021: 1905-1914. doi: 10.1109/iccvw54120.2021.00217http://dx.doi.org/10.1109/iccvw54120.2021.00217
ZHANG K, LIANG J Y, VAN GOOL L, et al. Designing a Practical Degradation Model for Deep Blind Image Super-Resolution[C]. 2021 IEEE/CVF International Conference on Computer Vision (ICCV).10-17, 2021, Montreal, QC, Canada. IEEE, 2022: 4771-4780. doi: 10.1109/iccv48922.2021.00475http://dx.doi.org/10.1109/iccv48922.2021.00475
LEDIG C, THEIS L, HUSZÁR F, et al. Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network[C]. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).21-26, 2017, Honolulu, HI, USA. IEEE, 2017: 105-114. doi: 10.1109/cvpr.2017.19http://dx.doi.org/10.1109/cvpr.2017.19
SCHÖNFELD E, SCHIELE B, KHOREVA A. A U-Net Based Discriminator for Generative Adversarial Networks[C]. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).13-19, 2020, Seattle, WA, USA. IEEE, 2020: 8204-8213. doi: 10.1109/cvpr42600.2020.00823http://dx.doi.org/10.1109/cvpr42600.2020.00823
XU L, REN J S J, YAN Q, et al. Deep edge-aware filters[J]. 32nd International Conference on Machine Learning, ICML 2015, 2015, 2: 1669-1678.
LIU S F, PAN J S, YANG M H. Learning Recursive Filters for Low-Level Vision via a Hybrid Neural Network[M]. Computer Vision - ECCV 2016. Cham: Springer International Publishing, 2016: 560-576. doi: 10.1007/978-3-319-46493-0_34http://dx.doi.org/10.1007/978-3-319-46493-0_34
KAMGAR-PARSI B, KAMGAR-PARSI B, ROSENFELD A. Optimally isotropic Laplacian operator[J]. IEEE Transactions on Image Processing, 1999, 8(10): 1467-1472. doi: 10.1109/83.791975http://dx.doi.org/10.1109/83.791975
LAI W S, HUANG J B, AHUJA N, et al. Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution[C]. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).21-26, 2017, Honolulu, HI, USA. IEEE, 2017: 5835-5843. doi: 10.1109/cvpr.2017.618http://dx.doi.org/10.1109/cvpr.2017.618
LIM B, SON S, KIM H, et al. Enhanced Deep Residual Networks for Single Image Super-Resolution[C]. 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).21-26, 2017, Honolulu, HI, USA. IEEE, 2017: 1132-1140. doi: 10.1109/cvprw.2017.151http://dx.doi.org/10.1109/cvprw.2017.151
BUADES A, COLL B, MOREL J M. A Non-Local Algorithm for Image Denoising[C]. 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).20-25, 2005, San Diego, CA, USA. IEEE, 2005: 60-65. doi: 10.1109/cvpr.2005.38http://dx.doi.org/10.1109/cvpr.2005.38
DABOV K, FOI A, KATKOVNIK V, et al. Image denoising by sparse 3-D transform-domain collaborative filtering[J]. IEEE Transactions on Image Processing: a Publication of the IEEE Signal Processing Society, 2007, 16(8): 2080-2095. doi: 10.1109/tip.2007.901238http://dx.doi.org/10.1109/tip.2007.901238
WANG X L, GIRSHICK R, GUPTA A, et al. Non-Local Neural Networks[C]. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.18-23, 2018, Salt Lake City, UT, USA. IEEE, 2018: 7794-7803. doi: 10.1109/cvpr.2018.00813http://dx.doi.org/10.1109/cvpr.2018.00813
DAI T, CAI J R, ZHANG Y B, et al. Second-Order Attention Network for Single Image Super-Resolution[C]. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).15-20, 2019, Long Beach, CA, USA. IEEE, 2020: 11057-11066. doi: 10.1109/cvpr.2019.01132http://dx.doi.org/10.1109/cvpr.2019.01132
WANG Z, BOVIK A C, SHEIKH H R, et al. Image quality assessment: from error visibility to structural similarity[J]. IEEE Transactions on Image Processing: a Publication of the IEEE Signal Processing Society, 2004, 13(4): 600-612. doi: 10.1109/tip.2003.819861http://dx.doi.org/10.1109/tip.2003.819861
YUAN F, HUANG L F, YAO Y. An Improved PSNR Algorithm for Objective Video Quality Evaluation[C]. 2007 Chinese Control Conference.26-31, 2007, Zhangjiajie, China. IEEE, 2007: 376-380. doi: 10.1109/chicc.2006.4347144http://dx.doi.org/10.1109/chicc.2006.4347144
0
Views
32
下载量
0
CSCD
Publicity Resources
Related Articles
Related Author
Related Institution