Optical remote sensing road extraction network based on GCN guided model viewpoint

LIU Guanghui; SHAN Zhe; YANG Yuanhai; WANG Heng; MENG Yuebo; XU Shengjun

doi:10.37188/OPE.20243210.1552

您当前的位置：

首页 >

文章列表页 >

Optical remote sensing road extraction network based on GCN guided model viewpoint

Information Sciences | 更新时间：2024-06-11

- Optical remote sensing road extraction network based on GCN guided model viewpoint
- Optics and Precision Engineering Vol. 32, Issue 10, Pages: 1552-1566(2024)
- 作者机构：
  
  1.西安建筑科技大学信息与控制工程学院，陕西西安 710055
  2.西安市建筑制造智动化技术重点实验室，陕西西安 710055
- 作者简介：
- 基金信息：
- DOI：10.37188/OPE.20243210.1552
  CLC： TP394.1;TH691.9
- Published：25 May 2024，
  
  Received：14 November 2023，
  
  Revised：04 January 2024，
- 稿件说明：
移动端阅览
刘光辉,单哲,杨塬海等.GCN引导模型视点的光学遥感道路提取网络[J].光学精密工程,2024,32(10):1552-1566.

LIU Guanghui,SHAN Zhe,YANG Yuanhai,et al.Optical remote sensing road extraction network based on GCN guided model viewpoint[J].Optics and Precision Engineering,2024,32(10):1552-1566.
刘光辉,单哲,杨塬海等.GCN引导模型视点的光学遥感道路提取网络[J].光学精密工程,2024,32(10):1552-1566. DOI： 10.37188/OPE.20243210.1552.

LIU Guanghui,SHAN Zhe,YANG Yuanhai,et al.Optical remote sensing road extraction network based on GCN guided model viewpoint[J].Optics and Precision Engineering,2024,32(10):1552-1566. DOI： 10.37188/OPE.20243210.1552.

摘要

在光学遥感图像中，道路易受遮挡物、铺装材料以及周围环境等多重因素的影响，导致其特征模糊不清。然而，现有道路提取方法即使增强其特征感知能力，仍在特征模糊区域存在大量误判。为解决上述问题，本文提出基于GCN引导模型视点的道路提取网络（RGGVNet）。RGGVNet采用编解码结构，并设计基于GCN的视点引导模块（GVPG）在编解码器的连接处反复引导模型视点，从而增强对特征模糊区域的关注。GVPG利用GCN信息传播过程具有平均特征权重的特性，将特征图中不同区域道路显著性水平作为拉普拉斯矩阵，参与到GCN信息传播从而实现引导模型视点。同时，提出密集引导视点策略（DGVS），采用密集连接的方式将编码器、GVPG和解码器相互连接，确保有效引导模型视点的同时缓解优化困难。在解码阶段设计多分辨率特征融合（MRFF）模块，最小化不同尺度道路特征在特征融合和上采样过程中的信息偏移和损失。在两个公开遥感道路数据集中，本文方法

IoU

分别达到65.84%和69.36%，

F1-score

分别达到79.40%和81.90%。从定量和定性两方面实验结果可以看出，本文所提方法性能优于其他主流方法。

Abstract

In optical remote sensing images， roads are easily affected by multiple factors such as obstructions， pavement materials， and surrounding environments， resulting in blurred featur

es. However， even if existing road extraction methods enhance their feature perception capabilities， they still suffer from a large number of misjudgments in feature-blurred areas. To address the above issues， this paper proposed the road extraction network based on GCN guided model viewpoint （RGGVNet）. RGGVNet adopted the encoder-decoder structure and designed a GCN based viewpoint guidance module （GVPG） to repeatedly guide the model viewpoint at the connection of the encoder and decoder， thereby enhancing attention to feature blurred areas. GVPG took advantage of the fact that the GCN information propagation process had the characteristic of average feature weight， used the road salience levels in different areas as a Laplacian matrix， and participated in GCN information propagation to realize the guidance model perspective. At the same time， a dense guidance viewpoint strategy （DGVS） was proposed， which uses dense connections to connect the encoder， GVPG module， and decoder to each other to ensure effective guidance of model viewpoints while alleviating optimization difficulties. In the decoding stage， a multi-resolution feature fusion module （MRFF） was designed to minimize the information offset and loss of road features of different scales in the feature fusion and upsampling process. In two public remote sensing road datasets， the

IoU

of our method reached 65.84% and 69.36%， respectively， and the

F1-score

reached 79.40% and 81.90%， respectively. It can be seen from the quantitative and qualitative experimental results that the performance of our method is superior to other mainstream methods.

关键词

光学遥感图像道路提取深度神经网络图卷积网络

Keywords

optical remote sensing imagesroad extractiondeep neural networkgraph convolution network

references

贾建鑫，孙海彬，蒋长辉，等. 多源遥感数据的道路提取技术研究现状及展望［J］. 光学精密工程， 2021， 29（2）： 430-442. doi: 10.37188/OPE.20212902.0430http://dx.doi.org/10.37188/OPE.20212902.0430

JIA J X， SUN H B， JIANG C H， et al. Road extraction technology based on multi-source remote sensing：［J］. Opt. Precision Eng.， 2021， 29（2）： 430-442.（in Chinese）. doi: 10.37188/OPE.20212902.0430http://dx.doi.org/10.37188/OPE.20212902.0430

姚凯旋，曹飞龙. 基于多输入密集连接神经网络的遥感图像时空融合算法［J］. 模式识别与人工智能， 2019， 32（5）： 429-435. doi: 10.16451/j.cnki.issn1003-6059.201905005http://dx.doi.org/10.16451/j.cnki.issn1003-6059.201905005

YAO K X， CAO F L. Spatial-temporal fusion algorithm for remote sensing images based on multi-input dense connected neural network［J］. Pattern Recognition and Artificial Intelligence， 2019， 32（5）： 429-435.（in Chinese）. doi: 10.16451/j.cnki.issn1003-6059.201905005http://dx.doi.org/10.16451/j.cnki.issn1003-6059.201905005

VALERO S， CHANUSSOT J， BENEDIKTSSON J A， et al. Advanced directional mathematical morphology for the detection of the road network in very high resolution remote sensing images［J］. Pattern Recognition Letters， 2010， 31（10）： 1120-1127. doi: 10.1016/j.patrec.2009.12.018http://dx.doi.org/10.1016/j.patrec.2009.12.018

CHAUDHURI D， KUSHWAHA N K， SAMAL A. Semi-automated road detection from high resolution satellite images by directional morphological enhancement and segmentation techniques［J］. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing， 2012， 5（5）： 1538-1544. doi: 10.1109/jstars.2012.2199085http://dx.doi.org/10.1109/jstars.2012.2199085

GRINIAS I， PANAGIOTAKIS C， TZIRITAS G. MRF-based segmentation and unsupervised classification for building and road detection in peri-urban areas of high-resolution satellite images［J］. ISPRS Journal of Photogrammetry and Remote Sensing， 2016， 122： 145-166. doi: 10.1016/j.isprsjprs.2016.10.010http://dx.doi.org/10.1016/j.isprsjprs.2016.10.010

SHAO Y Z， GUO B X， HU X Y， et al. Application of a fast linear feature detector to road extraction from remotely sensed imagery［J］. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing， 2011， 4（3）： 626-631. doi: 10.1109/jstars.2010.2094181http://dx.doi.org/10.1109/jstars.2010.2094181

POULLIS C. Tensor-Cuts： a simultaneous multi-type feature extractor and classifier and its application to road extraction from satellite images［J］. ISPRS Journal of Photogrammetry and Remote Sensing， 2014， 95： 93-108. doi: 10.1016/j.isprsjprs.2014.06.006http://dx.doi.org/10.1016/j.isprsjprs.2014.06.006

MABOUDI M， AMINI J， HAHN M， et al. Road network extraction from VHR satellite images using context aware object feature integration and tensor voting［J］. Remote Sensing， 2016， 8（8）： 637. doi: 10.3390/rs8080637http://dx.doi.org/10.3390/rs8080637

SHELHAMER E， LONG J， DARRELL T. Fully Convolutional Networks for Semantic Segmentation［C］. IEEE Transactions on Pattern Analysis and Machine Intelligence. IEEE， 2017： 640-651. doi: 10.1109/tpami.2016.2572683http://dx.doi.org/10.1109/tpami.2016.2572683

RONNEBERGER O， FISCHER P， BROX T. U-Net： convolutional networks for biomedical image segmentation［C］. International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham： Springer， 2015： 234-241. doi: 10.1007/978-3-319-24574-4_28http://dx.doi.org/10.1007/978-3-319-24574-4_28

CHAURASIA A， CULURCIELLO E. LinkNet： exploiting encoder representations for efficient semantic segmentation［C］. 2017 IEEE Visual Communications and Image Processing （VCIP）. St. Petersburg， FL， USA. IEEE， 2017： 1-4. doi: 10.1109/vcip.2017.8305148http://dx.doi.org/10.1109/vcip.2017.8305148

CHEN L C， ZHU Y K， PAPANDREOU G， et al. Encoder-decoder with atrous separable convolution for semantic image segmentation［C］. Computer Vision-ECCV 2018： 15th European Conference， Munich， Germany，8-14， 2018， Proceedings， Part VII. ACM， 2018： 833–851. doi: 10.1007/978-3-030-01234-2_49http://dx.doi.org/10.1007/978-3-030-01234-2_49

BUSLAEV A， SEFERBEKOV S， IGLOVIKOV V， et al. Fully convolutional network for automatic road extraction from satellite imagery［C］. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops （CVPRW）. Salt Lake City， UT， USA. IEEE， 2018： 197-1973. doi: 10.1109/cvprw.2018.00035http://dx.doi.org/10.1109/cvprw.2018.00035

HE K M， ZHANG X Y， REN S Q， et al. Deep residual learning for image recognition［C］. 2016 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. Las Vegas， NV， USA. IEEE， 2016： 770-778. doi: 10.1109/cvpr.2016.90http://dx.doi.org/10.1109/cvpr.2016.90

XIN J， ZHANG X C， ZHANG Z Q， et al. Road extraction of high-resolution remote sensing images derived from DenseUNet［J］. Remote Sensing， 2019， 11（21）： 2499. doi: 10.3390/rs11212499http://dx.doi.org/10.3390/rs11212499

HUANG G， LIU Z， VAN DER MAATEN L， et al. Densely connected convolutional networks［C］. 2017 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. Honolulu， HI， USA. IEEE， 2017： 2261-2269. doi: 10.1109/cvpr.2017.243http://dx.doi.org/10.1109/cvpr.2017.243

WU Q Q， LUO F， WU P H， et al. Automatic road extraction from high-resolution remote sensing images using a method based on densely connected spatial feature-enhanced pyramid［J］. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing， 2021， 14： 3-17. doi: 10.1007/978-3-319-24574-4_28http://dx.doi.org/10.1007/978-3-319-24574-4_28

WANG Y， SEO J， JEON T. NL-LinkNet： toward lighter but more accurate road extraction with nonlocal operations［J］. IEEE Geoscience and Remote Sensing Letters， 2022， 19： 3000105. doi: 10.1109/lgrs.2021.3050477http://dx.doi.org/10.1109/lgrs.2021.3050477

ZHANG Z X， SUN X， LIU Y X. GMR-net： road-extraction network based on fusion of local and global information［J］. Remote Sensing， 2022， 14（21）： 5476. doi: 10.3390/rs14215476http://dx.doi.org/10.3390/rs14215476

JIE Y S， HE H Y， XING K， et al. MECA-net： a MultiScale feature encoding and long-range context-aware network for road extraction from remote sensing images［J］. Remote Sensing， 2022， 14（21）： 5342. doi: 10.3390/rs14215342http://dx.doi.org/10.3390/rs14215342

VASWANI A， SHAZEER N， PARMAR N， et al. Attention is all you need［C］. Advances in Neural Information Processing Systems， California， USA， 2017， 30： 6000–6010.

DOSOVITSKIY A， BEYER L， KOLESNIKOV A， et al. An Image is Worth 16x16 Words： Transformers for Image Recognition at Scale［EB/OL］. 2020： arXiv： 2010.11929. http：//arxiv.org/abs/2010.11929http://arxiv.org/abs/2010.11929

WAN J， XIE Z， XU Y Y， et al. DA-RoadNet： a dual-attention network for road extraction from high resolution satellite imagery［J］. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing， 2021， 14： 6302-6315. doi: 10.1109/jstars.2021.3083055http://dx.doi.org/10.1109/jstars.2021.3083055

DAI L， ZHANG G Y， ZHANG R T. RADANet： road augmented deformable attention network for road extraction from complex high-resolution remote-sensing images［J］. IEEE Transactions on Geoscience and Remote Sensing， 2023， 61： 5602213. doi: 10.1109/tgrs.2023.3237561http://dx.doi.org/10.1109/tgrs.2023.3237561

XU Q X， LONG C， YU L， et al. Road extraction with satellite images and partial road maps［J］. IEEE Transactions on Geoscience and Remote Sensing， 2023， 61： 4501214. doi: 10.1109/tgrs.2023.3261332http://dx.doi.org/10.1109/tgrs.2023.3261332

LUO L， WANG J X， CHEN S B， et al. BDTNet： road extraction by Bi-direction transformer from remote sensing images［J］. IEEE Geoscience and Remote Sensing Letters， 1998， 19： 2505605.

ZHANG Z， MIAO C L， LIU C A， et al. DCS-TransUperNet： road segmentation network based on CSwin transformer with dual resolution［J］. Applied Sciences， 2022， 12（7）： 3511. doi: 10.3390/app12073511http://dx.doi.org/10.3390/app12073511

LIU X Z， WANG Z Y， WAN J T， et al. RoadFormer： road extraction using a swin transformer combined with a spatial and channel separable convolution［J］. Remote Sensing， 2023， 15（4）： 1049. doi: 10.3390/rs15041049http://dx.doi.org/10.3390/rs15041049

SIMONYAN K， VEDALDI A， ZISSERMAN A. Deep inside convolutional networks： Visualising image classification models and saliency maps［J］. 2nd International Conference on Learning Representations， ICLR 2014-Workshop Track Proceedings， 2014： 1-8.

ZEILER M D， FERGUS R. Visualizing and Understanding Convolutional Networks［M］. Computer Vision-ECCV 2014. Cham： Springer International Publishing， 2014： 818-833. doi: 10.1007/978-3-319-10590-1_53http://dx.doi.org/10.1007/978-3-319-10590-1_53

SELVARAJU R R， COGSWELL M， DAS A， et al. Grad-CAM： visual explanations from deep networks via gradient-based localization［J］. International Journal of Computer Vision， 2020， 128（2）： 336-359. doi: 10.1007/s11263-019-01228-7http://dx.doi.org/10.1007/s11263-019-01228-7

LIU Z， MAO H Z， WU C Y， et al. A convnet for the 2020s［C］. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. New Orleans， LA， USA. IEEE， 2022： 11966-11976. doi: 10.1109/cvpr52688.2022.01167http://dx.doi.org/10.1109/cvpr52688.2022.01167

LI Z H， GU T C， LI B， et al. ConvNeXt-based fine-grained image classification and bilinear attention mechanism model［J］. Applied Sciences， 2022， 12（18）： 9016. doi: 10.3390/app12189016http://dx.doi.org/10.3390/app12189016

ZHOU J J， ZHANG B H， YUAN X L， et al. YOLO-CIR： the network based on YOLO and ConvNeXt for infrared object detection［J］. Infrared Physics and Technology， 2023， 131： 104703. doi: 10.1016/j.infrared.2023.104703http://dx.doi.org/10.1016/j.infrared.2023.104703

HAN Z M， JIAN M W， WANG G G. ConvUNeXt： an efficient convolution neural network for medical image segmentation［J］. Knowledge-Based Systems， 2022， 253： 109512. doi: 10.1016/j.knosys.2022.109512http://dx.doi.org/10.1016/j.knosys.2022.109512

KIPF T N， WELLING M. Semi-Supervised Classification with Graph Convolutional Networks［EB/OL］. 2016： arXiv： 1609.02907. http：//arxiv.org/abs/1609.02907http://arxiv.org/abs/1609.02907. doi: 10.48550/arXiv.1609.02907http://dx.doi.org/10.48550/arXiv.1609.02907

LI Y， GUPTA A. Beyond grids： learning graph representations for visual recognition［C］. Proceedings of the 32nd International Conference on Neural Information Processing Systems. 3-8，2018， Montréal， Canada. ACM， 2018： 9245-9255.

LIU Q H， KAMPFFMEYER M， JENSSEN R， et al. Self-constructing graph convolutional networks for semantic labeling［C］. IGARSS 2020 - 2020 IEEE International Geoscience and Remote Sensing Symposium. Waikoloa， HI， USA. IEEE， 2020： 1801-1804. doi: 10.1109/igarss39084.2020.9324719http://dx.doi.org/10.1109/igarss39084.2020.9324719

LI Q M， HAN Z C， WU X M. Deeper insights into graph convolutional networks for semi-supervised learning［J］. Proceedings of the AAAI Conference on Artificial Intelligence， 2018， 32（1）： 1. doi: 10.1609/aaai.v32i1.11604http://dx.doi.org/10.1609/aaai.v32i1.11604

XU K， LI C T， TIAN Y L， et al. Representation Learning on Graphs with Jumping Knowledge Networks［EB/OL］. 2018： arXiv： 1806.03536. http：//arxiv.org/abs/1806.03536http://arxiv.org/abs/1806.03536

MNIH V. Machine Learning for Aerial Image Labeling［M］. Canada： University of Toronto， 2013.

DEMIR I， KOPERSKI K， LINDENBAUM D， et al. DeepGlobe 2018： a challenge to parse the earth through satellite images［C］. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops （CVPRW）. Salt Lake City， UT， USA. IEEE， 2018： 172-17209. doi: 10.1109/cvprw.2018.00031http://dx.doi.org/10.1109/cvprw.2018.00031

MILLETARI F， NAVAB N， AHMADI S A. V-Net： fully convolutional neural networks for volumetric medical image segmentation［C］. 2016 Fourth International Conference on 3D Vision （3DV）. Stanford， CA， USA. IEEE， 2016： 565-571. doi: 10.1109/3dv.2016.79http://dx.doi.org/10.1109/3dv.2016.79

BADRINARAYANAN V， KENDALL A， CIPOLLA R. SegNet： a deep convolutional encoder-decoder architecture for image segmentation［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2017， 39（12）： 2481-2495. doi: 10.1109/tpami.2016.2644615http://dx.doi.org/10.1109/tpami.2016.2644615

ZHAO H S， SHI J P， QI X J， et al. Pyramid scene parsing network［C］. 2017 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. Honolulu， HI， USA. IEEE， 2017： 6230-6239. doi: 10.1109/cvpr.2017.660http://dx.doi.org/10.1109/cvpr.2017.660

WANG S， YANG H， WU Q Q， et al. An improved method for road extraction from high-resolution remote-sensing images that enhances boundary information［J］. Sensors， 2020， 20（7）： 2064. doi: 10.3390/s20072064http://dx.doi.org/10.3390/s20072064

LI J， LIU Y， ZHANG Y D， et al. Cascaded attention DenseUNet （CADUNet） for road extraction from very-high-resolution images［J］. ISPRS International Journal of Geo-Information， 2021， 10（5）： 329. doi: 10.3390/ijgi10050329http://dx.doi.org/10.3390/ijgi10050329

WANG Y， PENG Y X， LI W， et al. DDU-net： dual-decoder-U-net for road extraction using high-resolution remote sensing images［J］. IEEE Transactions on Geoscience and Remote Sensing， 2022， 60： 4412612. doi: 10.1109/tgrs.2022.3197546http://dx.doi.org/10.1109/tgrs.2022.3197546

ZHOU L C， ZHANG C， WU M. D-LinkNet： LinkNet with pretrained encoder and dilated convolution for high resolution satellite imagery road extraction［C］. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops （CVPRW）. Salt Lake City， UT， USA. IEEE， 2018： 192-1924. doi: 10.1109/cvprw.2018.00034http://dx.doi.org/10.1109/cvprw.2018.00034

YANG M X， YUAN Y， LIU G C. SDUNet： road extraction via spatial enhanced and densely connected UNet［J］. Pattern Recognition， 2022， 126： 108549. doi: 10.1016/j.patcog.2022.108549http://dx.doi.org/10.1016/j.patcog.2022.108549

MEI J， LI R J， GAO W， et al. CoANet： connectivity attention network for road extraction from satellite imagery［J］. IEEE Transactions on Image Processing， 2021， 30： 8540-8552. doi: 10.1109/tip.2021.3117076http://dx.doi.org/10.1109/tip.2021.3117076

SHAO S W， XIAO L X， LIN L P， et al. Road extraction convolutional neural network with embedded attention mechanism for remote sensing imagery［J］. Remote Sensing， 2022， 14（9）： 2061. doi: 10.3390/rs14092061http://dx.doi.org/10.3390/rs14092061

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Degradation remote sensing image quality enhancement based on frequency-domain-spatial-domain hybrid attention

Super-resolution strain measurement in phase-contrast optical coherence elastography

Image reconstruction based on deep compressive sensing combined with global and local features

Single-view 3D object reconstruction based on NFFD and graph convolution

Polarization orientation method for cloudy sky

Related Author

XU Fanjiang

XIA Yuanyuan

YANG Hanxiang

WANG Jing

NIE Haitao

TANG Xiongxin

WEI Hua

XIE Shengli

Related Institution

Laboratory of Science and Technology on Integrated Information System， Institute of Software， Chinese Academy of Sciences

Changchun Institute of Optics， Fine Mechanics and Physics， Chinese Academy of Sciences

School of Automation， Guangdong University of Technology

School of Microelectronics and Communication Engineering， Chongqing University

Institute of Physical Science and Information Technology， Anhui University

Address：No.3888 Dong Nanhu Road, Changchun, Jilin, China Postal code：130033
Tel：0431-86176855 Email：gxjmgc@ciomp.ac.cn
Technical support is provided by Beijing Founder electronics co., LTD 吉ICP备11002662号-17 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰