1.中国科学院 长春光学精密机械与物理研究所,吉林 长春 130033
扫 描 看 全 文
JIANG Xin, NIE Haitao, ZHU Ming. Global and local feature fusion image dehazing. [J]. Optics and Precision Engineering 31(18):2687-2699(2023)
JIANG Xin, NIE Haitao, ZHU Ming. Global and local feature fusion image dehazing. [J]. Optics and Precision Engineering 31(18):2687-2699(2023) DOI: 10.37188/OPE.20233118.2687.
具有参数共享特性的卷积操作主要关注于图像局部特征的提取,而无法对超出感受野范围的特征进行建模,同时整幅图像共享同一个卷积核参数也忽略了不同区域的特性不同。为了克服现有方法的表达不足,提出了全局和局部特征融合去雾网络,分别利用Transformer和卷积操作提取图像全局和局部特征信息,并将两者融合后输出,充分发挥了Transformer建模长距离依赖关系和卷积操作局部感知特性的优势,实现了特征的高效表达。在最终输出复原图像前,设计了包含多尺度图像块的增强模块,利用Transformer进一步聚合全局特征信息,丰富复原图像细节。同时,提出了一个全局位置编码生成器,可自适应地根据全局图像内容信息生成位置编码,进而实现对像素点间依赖关系的二维空间位置建模。实验结果表明,所提出的去雾网络在合成和真实图像数据集上均展现出了较好的去雾性能,复原图像更加真实,细节还原度高。
Convolution operations with parameter sharing features primarily focus on the extraction of local features of images but fail to model the features beyond the range of the receptive field. Moreover, when the parameters of an entire image share the same convolution kernel, the characteristics of different regions are ignored. To address this limitation in existing methods, a global and local feature fusion dehazing network is proposed. We utilize transformer and convolution operations to extract global and local feature information from images, respectively. Subsequently, we merge and output these features, effectively employing the advantages of transformers in modeling long-distance dependencies and the local perception of convolution operations, thus achieving efficient feature expression. Before the final output of restored images, we incorporate an enhancement module that includes multi-scale patches to further aggregate global feature information and enhance the details of the restored images using a transformer. Simultaneously, we introduce a global positional encoding generator, which can adaptively generate positional encodings based on the global content information of images, thereby enabling 2D spatial location modeling of the dependency relationship between pixels. Experimental results demonstrate the superior performance of the proposed dehazing network on both synthetic and real image datasets, producing more realistic restored images and significantly reducing detail loss.
图像去雾生成式对抗网络Transformer位置编码生成器
image dehazinggenerative adversarial networktransformerpositional encoding generator
王卫星, 赵恒. 结合改进Retinex及自适应分数阶微分的雾霾公路交通图像增强[J]. 光学 精密工程, 2020, 28(8): 1820-1834.
WANG W X, ZHAO H. Haze traffic image enhancement based on improved retinex and adaptive fractional differential[J]. Opt. Precision Eng., 2020, 28(8):1820-1834.(in Chinese)
JIANG X, ZHAO C L, ZHU M, et al. Residual spatial and channel attention networks for single image dehazing[J]. Sensors (Basel, Switzerland), 2021, 21(23): 7922. doi: 10.3390/s21237922http://dx.doi.org/10.3390/s21237922
TAN R T. Visibility in Bad Weather from a Single Image[C]. 2008 IEEE Conference on Computer Vision and Pattern Recognition.23-28, 2008, Anchorage, AK, USA. IEEE, 2008: 1-8. doi: 10.1109/cvpr.2008.4587643http://dx.doi.org/10.1109/cvpr.2008.4587643
HE K M, SUN J, TANG X O. Single image haze removal using dark channel prior[C]. 2009 IEEE Conference on Computer Vision and Pattern Recognition.20-25, 2009, Miami, FL. IEEE, 2009: 1956-1963. doi: 10.1109/cvpr.2009.5206515http://dx.doi.org/10.1109/cvpr.2009.5206515
ANCUTI C O, ANCUTI C, HERMANS C, et al. A Fast Semi-Inverse Approach to Detect and Remove the Haze from a Single Image[M]. Computer Vision-ACCV 2010. Berlin, Heidelberg: Springer Berlin Heidelberg, 2011: 501-514. doi: 10.1007/978-3-642-19309-5_39http://dx.doi.org/10.1007/978-3-642-19309-5_39
ZHU Q S, MAI J M, SHAO L. A fast single image haze removal algorithm using color attenuation prior[J]. IEEE Transactions on Image Processing: a Publication of the IEEE Signal Processing Society, 2015, 24(11): 3522-3533. doi: 10.1109/tip.2015.2446191http://dx.doi.org/10.1109/tip.2015.2446191
CAI B L, XU X M, JIA K, et al. DehazeNet: an end-to-end system for single image haze removal[J]. IEEE Transactions on Image Processing: a Publication of the IEEE Signal Processing Society, 2016, 25(11): 5187-5198. doi: 10.1109/tip.2016.2598681http://dx.doi.org/10.1109/tip.2016.2598681
LI B Y, PENG X L, WANG Z Y, et al. AOD-Net: all-in-one dehazing network[C]. 2017 IEEE International Conference on Computer Vision (ICCV).22-29, 2017, Venice, Italy. IEEE, 2017: 4780-4788. doi: 10.1109/iccv.2017.511http://dx.doi.org/10.1109/iccv.2017.511
陈清江, 张雪. 混合残差学习与导向滤波算法在图像去雾中的应用[J]. 光学 精密工程, 2019, 27(12):2702-2712. doi: 10.3788/ope.20192712.2702http://dx.doi.org/10.3788/ope.20192712.2702
CHEN Q J, ZHANG X. Application of hybrid residual learning and guided filtering algorithm in image defogging[J]. Opt. Precision Eng., 2019, 27(12):2702-2712.(in Chinese). doi: 10.3788/ope.20192712.2702http://dx.doi.org/10.3788/ope.20192712.2702
冯燕茹. 双视觉注意网络的联合图像去雾和透射率估计[J]. 光学 精密工程, 2021, 29(4): 854-863. doi: 10.37188/OPE.20212904.0854http://dx.doi.org/10.37188/OPE.20212904.0854
FENG Y R. Joint transmission map estimation and image dehazing using dual vision attention network[J]. Opt. Precision Eng., 2021, 29(4): 854-863.(in Chinese). doi: 10.37188/OPE.20212904.0854http://dx.doi.org/10.37188/OPE.20212904.0854
杨燕, 梁小珍, 张金龙. 分离特征和协同网络下的端到端图像去雾[J]. 光学 精密工程, 2021, 29(8):1931-1941. doi: 10.37188/OPE.2021.0003http://dx.doi.org/10.37188/OPE.2021.0003
YANG Y, LIANG X Z, ZHANG J L. End-to-end image dehazing under separated features and collaborative network[J]. Opt. Precision Eng., 2021, 29(8):1931-1941.(in Chinese). doi: 10.37188/OPE.2021.0003http://dx.doi.org/10.37188/OPE.2021.0003
VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]. Proceedings of the 31st International Conference on Neural Information Processing Systems. December 4 - 9, 2017, Long Beach, California, USA. New York: ACM, 2017: 6000-6010.
LIANG J Y, CAO J Z, SUN G L, et al. SwinIR: image restoration using swin transformer[C]. 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW).11-17, 2021, Montreal, BC, Canada. IEEE, 2021: 1833-1844. doi: 10.1109/iccvw54120.2021.00210http://dx.doi.org/10.1109/iccvw54120.2021.00210
CHEN H T, WANG Y H, GUO T Y, et al. Pre-trained image processing transformer[C]. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).20-25, 2021, Nashville, TN, USA. IEEE, 2021: 12294-12305. doi: 10.1109/cvpr46437.2021.01212http://dx.doi.org/10.1109/cvpr46437.2021.01212
QIN Q, YAN J K, WANG Q, et al. ETDNet: an efficient transformer deraining model[J]. IEEE Access, 2021, 9: 119881-119893. doi: 10.1109/access.2021.3108516http://dx.doi.org/10.1109/access.2021.3108516
MIRZA M, OSINDERO S. Conditional Generative Adversarial Nets[EB/OL]. 2014: arXiv: 1411.1784. https://arxiv.org/abs/1411.1784https://arxiv.org/abs/1411.1784
ZHANG H B, SAKURAI K. Conditional generative adversarial network-based image denoising for defending against adversarial attack[J]. IEEE Access, 2021, 9: 169031-169043. doi: 10.1109/access.2021.3137637http://dx.doi.org/10.1109/access.2021.3137637
WANG P. Image denoising using deep cgan with bi-skip connections[C]. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).16-17, 2019, Long Beach, CA, USA. IEEE, 2020: 724-729. doi: 10.1109/cvprw.2019.00099http://dx.doi.org/10.1109/cvprw.2019.00099
ZHANG H, SINDAGI V, PATEL V M. Image de-raining using a conditional generative adversarial network[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2020, 30(11): 3943-3956. doi: 10.1109/tcsvt.2019.2920407http://dx.doi.org/10.1109/tcsvt.2019.2920407
LI R T, CHEONG L F, TAN R T. Heavy rain image restoration: integrating physics model and conditional adversarial learning[C]. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).15-20, 2019, Long Beach, CA, USA. IEEE, 2020: 1633-1642. doi: 10.1109/cvpr.2019.00173http://dx.doi.org/10.1109/cvpr.2019.00173
PARK J, HAN D K, KO H. Fusion of heterogeneous adversarial networks for single image dehazing[J]. IEEE Transactions on Image Processing, 2020, 29: 4721-4732. doi: 10.1109/tip.2020.2975986http://dx.doi.org/10.1109/tip.2020.2975986
LI R D, PAN J S, LI Z C, et al. Single image dehazing via conditional generative adversarial network[C]. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.18-23, 2018, Salt Lake City, UT, USA. IEEE, 2018: 8202-8211. doi: 10.1109/cvpr.2018.00856http://dx.doi.org/10.1109/cvpr.2018.00856
ISOLA P, ZHU J Y, ZHOU T H, et al. Image-to-image translation with conditional adversarial networks[C]. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).21-26, 2017, Honolulu, HI, USA. IEEE, 2017: 5967-5976. doi: 10.1109/cvpr.2017.632http://dx.doi.org/10.1109/cvpr.2017.632
RONNEBERGER O, FISCHER P, BROX T. U-Net: Convolutional Networks for Biomedical Image Segmentation[M]. Lecture Notes in Computer Science. Cham: Springer International Publishing, 2015: 234-241. doi: 10.1007/978-3-319-24574-4_28http://dx.doi.org/10.1007/978-3-319-24574-4_28
SHI W Z, CABALLERO J, HUSZÁR F, et al. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[C]. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).27-30, 2016, Las Vegas, NV, USA. IEEE, 2016: 1874-1883. doi: 10.1109/cvpr.2016.207http://dx.doi.org/10.1109/cvpr.2016.207
CHU X, TIAN Z, ZHANG B, et al. Conditional Positional Encodings for Vision Transformers[EB/OL]. 2021: arXiv: 2102.10882. https://arxiv.org/abs/2102.10882.pdfhttps://arxiv.org/abs/2102.10882.pdf
JIANG X, LU L, ZHU M, et al. Haze relevant feature attention network for single image dehazing[J]. IEEE Access, 2021, 9: 106476-106488. doi: 10.1109/access.2021.3100604http://dx.doi.org/10.1109/access.2021.3100604
ENGIN D, GENC A, EKENEL H K. Cycle-dehaze: enhanced cyclegan for single image dehazing[C]. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).18-22, 2018, Salt Lake City, UT, USA. IEEE, 2018: 938-9388. doi: 10.1109/cvprw.2018.00127http://dx.doi.org/10.1109/cvprw.2018.00127
QU Y Y, CHEN Y Z, HUANG J Y, et al. Enhanced pix2pix dehazing network[C]. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).15-20, 2019, Long Beach, CA, USA. IEEE, 2020: 8152-8160. doi: 10.1109/cvpr.2019.00835http://dx.doi.org/10.1109/cvpr.2019.00835
QIN X, WANG Z L, BAI Y C, et al. FFA-net: feature fusion attention network for single image dehazing[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2020, 34(7): 11908-11915. doi: 10.1609/aaai.v34i07.6865http://dx.doi.org/10.1609/aaai.v34i07.6865
ULLAH H, MUHAMMAD K, IRFAN M, et al. Light-DehazeNet: a novel lightweight CNN architecture for single image dehazing[J]. IEEE Transactions on Image Processing: a Publication of the IEEE Signal Processing Society, 2021, 30: 8968-8982. doi: 10.1109/tip.2021.3116790http://dx.doi.org/10.1109/tip.2021.3116790
LI B Y, REN W Q, FU D P, et al. Benchmarking single-image dehazing and beyond[J]. IEEE Transactions on Image Processing, 2019, 28(1): 492-505. doi: 10.1109/tip.2018.2867951http://dx.doi.org/10.1109/tip.2018.2867951
ANCUTI C, ANCUTI C O, TIMOFTE R, et al. NTIRE 2018 challenge on image dehazing: methods and results[C]. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).18-22, 2018, Salt Lake City, UT, USA. IEEE, 2018: 1004-100410. doi: 10.1109/cvprw.2018.00119http://dx.doi.org/10.1109/cvprw.2018.00119
ANCUTI C, ANCUTI C, TIMOFTE R, et al. NTIRE 2019 image dehazing challenge report[C]. Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Long Beach, CA, USA: CVPRW, 2019:2241-2253. doi: 10.1109/cvprw50498.2020.00230http://dx.doi.org/10.1109/cvprw50498.2020.00230
ANCUTI C O, ANCUTI C, VASLUIANU F A, et al. NTIRE 2020 challenge on nonhomogeneous dehazing[C]. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).14-19, 2020, Seattle, WA, USA. IEEE, 2020: 2029-2044.
ANCUTI C O, ANCUTI C, TIMOFTE R, et al. O-HAZE: a dehazing benchmark with real hazy and haze-free outdoor images[C]. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).18-22, 2018, Salt Lake City, UT, USA. IEEE, 2018: 867-8678. doi: 10.1109/cvprw.2018.00119http://dx.doi.org/10.1109/cvprw.2018.00119
ANCUTI C O, ANCUTI C, SBERT M, et al. Dense-Haze: a benchmark for image dehazing with dense-haze and haze-free images[C]. 2019 IEEE International Conference on Image Processing (ICIP).22-25, 2019, Taipei, China. IEEE, 2019: 1014-1018. doi: 10.1109/icip.2019.8803046http://dx.doi.org/10.1109/icip.2019.8803046
ANCUTI C O, ANCUTI C, TIMOFTE R. NH-HAZE: an image dehazing benchmark with non-homogeneous hazy and haze-free images[C]. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).14-19, 2020, Seattle, WA, USA. IEEE, 2020: 1798-1805. doi: 10.1109/cvprw50498.2020.00230http://dx.doi.org/10.1109/cvprw50498.2020.00230
0
Views
21
下载量
0
CSCD
Publicity Resources
Related Articles
Related Author
Related Institution