Remote sensing image feature extraction and classification based on contrastive learning method

Xiao-dong MU; Kun BAI; Xuan-ang YOU; Yong-qing ZHU; Xue-bing CHEN

doi:10.37188/OPE.20212909.2222

您当前的位置：

首页 >

文章列表页 >

Remote sensing image feature extraction and classification based on contrastive learning method

Information Sciences | 更新时间：2021-10-08

- Remote sensing image feature extraction and classification based on contrastive learning method
- Optics and Precision Engineering Vol. 29, Issue 9, Pages: 2222-2234(2021)
- 作者机构：
  
  1.火箭军工程大学作战保障学院，陕西西安 710025
  2.解放军61068部队，陕西西安 710100
- 作者简介：
- 基金信息：
- DOI：10.37188/OPE.20212909.2222
  CLC： TP751
- Received：29 January 2021，
  
  Revised：19 March 2021，
  
  Published：15 September 2021
- 稿件说明：
移动端阅览
慕晓冬,白坤,尤轩昂等.基于对比学习方法的遥感影像特征提取与分类[J].光学精密工程,2021,29(09):2222-2234.

MU Xiao-dong,BAI Kun,YOU Xuan-ang,et al.Remote sensing image feature extraction and classification based on contrastive learning method[J].Optics and Precision Engineering,2021,29(09):2222-2234.
慕晓冬,白坤,尤轩昂等.基于对比学习方法的遥感影像特征提取与分类[J].光学精密工程,2021,29(09):2222-2234. DOI： 10.37188/OPE.20212909.2222.

MU Xiao-dong,BAI Kun,YOU Xuan-ang,et al.Remote sensing image feature extraction and classification based on contrastive learning method[J].Optics and Precision Engineering,2021,29(09):2222-2234. DOI： 10.37188/OPE.20212909.2222.

摘要

为了解决使用深度学习进行遥感影像特征提取与分类时标注数据不足的问题，提出一种基于非对称预测算子的简易对比学习方法。首先，使用水平翻转、颜色抖动和灰度化方法对输入图像进行数据增强，得到同一幅图像的两个相关视图。接着，将其分别输入到孪生网络的两个分支进行特征提取。然后，使用非对称预测算子对特征进行变换，通过最大化两种特征间的相似度优化网络。最后，固定特征提取网络的参数，训练一个线性分类器完成特征分类。在四个公开数据集NWPU-RESISC45，EuroSAT，UC Merced，SIRI-WHU上使用20%的标注样本进行微调，分类精度分别达到77.57%，87.70%，60.52%和65.83%。本文提出的方法能够在不使用数据标签的情况下充分挖掘遥感影像中的高层语义特征，在只使用少量标注样本的情况下性能优于有监督方法得到的ImageNet预训练模型和目前最新的对比学习方法SimSiam。

Abstract

To solve the problem of lack of labeled data in the feature extraction and classification from remote sensing images using deep learning， a simple contrastive learning method involving the use of an asymmetric predictor was proposed. First， the input image is enhanced using horizontal flipping， color jitter， and grayscale methods to obtain two related views of the same image. Subsequently， they are fed into the two branches of a Siamese network for feature extraction. Next， asymmetric predictors are used to transform the features， and the network is optimized by maximizing the similarity between them. Finally， a linear classifier is trained by fixing its parameters to complete the feature classification. When 20% of the labeled samples are used for fine-tuning in the four public remote sensing image datasets， NWPU-Resisc45， EuroSAT， UC Merced， and Siri-WHU， the classification accuracies of the experiments are 77.57%， 87.70%， 60.52%， and 65.83%， respectively. Our proposed method can effectively extract the high-level semantic features of remote sensing images without using data labels and has better performance than the ImageNet pre-trained model and the latest contrastive learning method SimSiam under the conditions of insufficient number of labeled samples.

关键词

Keywords

references

陈科峻，张叶 . 循环神经网络多标签航空图像分类［J］. 光学精密工程， 2020 ， 28 （ 6 ）： 1404 - 1413 . doi: 10.3788/ope.20202806.1404 http://dx.doi.org/10.3788/ope.20202806.1404

CHEN K J ， ZHANG Y . Recurrent neural network multi-label aerial images classification ［J］. Opt. Precision Eng. ， 2020 ， 28 （ 6 ）： 1404 - 1413 . （in Chinese） . doi: 10.3788/ope.20202806.1404 http://dx.doi.org/10.3788/ope.20202806.1404

杨州，慕晓冬，王舒洋，等 . 基于多尺度特征融合的遥感图像场景分类［J］. 光学精密工程， 2018 ， 26 （ 12 ）： 3099 - 3107 . doi: 10.3788/ope.20182612.3099 http://dx.doi.org/10.3788/ope.20182612.3099

YANG ZH ， MU X D ， WANG SH Y ， et al . Scene classification of remote sensing images based on multiscale features fusion ［J］. Opt. Precision Eng. ， 2018 ， 26 （ 12 ）： 3099 - 3107 . （in Chinese） . doi: 10.3788/ope.20182612.3099 http://dx.doi.org/10.3788/ope.20182612.3099

PENATTI O A B ， NOGUEIRA K ， SANTOS J ADOS . Do deep features generalize from everyday objects to remote sensing and aerial scenes domains？［C］. 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops （CVPRW） . 712，2015 ， Boston， MA， USA . IEEE ， 2015 ： 44 - 51 . doi: 10.1109/cvprw.2015.7301382 http://dx.doi.org/10.1109/cvprw.2015.7301382

MARMANIS D ， DATCU M ， ESCH T ， et al . Deep learning earth observation classification using ImageNet pretrained networks ［J］. IEEE Geoscience and Remote Sensing Letters ， 2016 ， 13 （ 1 ）： 105 - 109 . doi: 10.1109/lgrs.2015.2499239 http://dx.doi.org/10.1109/lgrs.2015.2499239

ZHANG R ， ISOLA P ， EFROS A A . Colorful image colorization ［M］. Computer Vision-ECCV 2016. Cham ： Springer International Publishing ， 2016 ： 649 - 666 . doi: 10.1007/978-3-319-46487-9_40 http://dx.doi.org/10.1007/978-3-319-46487-9_40

CHEN T ， ZHAI X H ， RITTER M ， et al . Self-supervised GANs via auxiliary rotation loss ［C］. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR） . 1520，2019 ， Long Beach， CA， USA . IEEE ， 2019 ： 12146 - 12155 . doi: 10.1109/cvpr.2019.01243 http://dx.doi.org/10.1109/cvpr.2019.01243

CHEN T ， KORNBLITH S ， NOROUZI M ， et al . A simple framework for contrastive learning of visual representations ［EB/OL］. Arxiv Preprint Arxiv ： 2002 . 05709 ， 2020 . https：//arxiv.org/abs/2002.05709v1 https://arxiv.org/abs/2002.05709v1

HE K M ， FAN H Q ， WU Y X ， et al . Momentum contrast for unsupervised visual representation learning ［C］. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR） . 1319，2020 ， Seattle ， WA ， USA . IEEE ， 2020 ： 9726 - 9735 . doi: 10.1109/cvpr42600.2020.00975 http://dx.doi.org/10.1109/cvpr42600.2020.00975

Chen X ， Fan H ， Girshick R ， et al . Improved baselines with momentum contrastive learning ［EB/OL］. Arxiv Preprint Arxiv ： 2003 ， 04297 ， 2020 . https：//www.arxiv-vanity.com/papers/2003. 04297/ https://www.arxiv-vanity.com/papers/2003.04297/

Kalantidis Y ， Sariyildiz M B ， Pion N ， et al . Hard negative mixing for contrastive learning ［EB/OL］. Arxiv Preprint Arxiv ： 2010 ， 01028 ， 2020 . https：//arxiv.org/abs/2010.01028 https://arxiv.org/abs/2010.01028

LI T H ， FAN L J ， YUAN Y ， et al . Information-preserving contrastive learning for self-supervised representations ［EB/OL］. Arxiv Preprint Arxiv ： 2012 . 09962 ， 2020 . https：//arxiv.org/abs/2012.09962v1 https://arxiv.org/abs/2012.09962v1

LIU H ， ABBEEL P . Hybrid discriminative-generative training via contrastive learning ［EB/OL］. Arxiv Preprint Arxiv ： 2007 ， 09070 ， 2020 . https：//arxiv.org/abs/2007.09070 https://arxiv.org/abs/2007.09070

Caron M ， Misra I ， Mairal J ， et al . Unsupervised Learning of Visual Features By Contrasting Cluster Assignments ［EB/OL］. Arxiv Preprint Arxiv ： 2006 ， 09882 ， 2020 .

GRILL J ， STRUB F ， ALTCHE F ， et al . Bootstrap your own latent-a new approach to self-supervised learning ［J］. Advances in Neural Information Processing Systems ， 2020 ， 33 .

Chen X ， He K . Exploring Simple Siamese Representation Learning ［EB/OL］. Arxiv Preprint Arxiv ： 2011 ， 10566 2020. https：//arxiv.org/abs/2011.10566 https://arxiv.org/abs/2011.10566

HE K M ， ZHANG X Y ， REN S Q ， et al . Deep residual learning for image recognition ［C］. 2016 IEEE Conference on Computer Vision and Pattern Recognition （CVPR） . 2730，2016 ， Las Vegas， NV， USA . IEEE ， 2016 ： 770 - 778 . doi: 10.1109/cvpr.2016.90 http://dx.doi.org/10.1109/cvpr.2016.90

CHEN W J ， PU S L ， XIE D ， et al . Unsupervised image classification for deep representation learning ［EB/OL］. Arxiv Preprint Arxiv ： 2006 ， 11480 ， 2020 . https：//arxiv.org/abs/2006.11480v1 https://arxiv.org/abs/2006.11480v1

ULYANOV D ， VEDALDI A ， LEMPITSKY V . Deep image prior ［J］. International Journal of Computer Vision ， 2020 ， 128 （ 7 ）： 1867 - 1888 . doi: 10.1007/s11263-020-01303-4 http://dx.doi.org/10.1007/s11263-020-01303-4

Zhu J ， Li Y ， Zhou SK . Aggregative Self-supervised Feature Learning ［EB/OL］. Arxiv Preprint Arxiv ： 2012 ， 07477 2020. https：//arxiv.org/abs/2012.07477 https://arxiv.org/abs/2012.07477

CHENG G ， HAN J W ， LU X Q . Remote sensing image scene classification： benchmark and state of the art ［J］. Proceedings of the IEEE ， 2017 ， 105 （ 10 ）： 1865 - 1883 . doi: 10.1109/jproc.2017.2675998 http://dx.doi.org/10.1109/jproc.2017.2675998

HELBER P ， BISCHKE B ， DENGEL A ， et al . EuroSAT： a novel dataset and deep learning benchmark for land use and land cover classification ［J］. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing ， 2019 ， 12 （ 7 ）： 2217 - 2226 . doi: 10.1109/jstars.2019.2918242 http://dx.doi.org/10.1109/jstars.2019.2918242

ZOU Q ， NI L H ， ZHANG T ， et al . Deep learning based feature selection for remote sensing scene classification ［J］. IEEE Geoscience and Remote Sensing Letters ， 2015 ， 12 （ 11 ）： 2321 - 2325 . doi: 10.1109/lgrs.2015.2475299 http://dx.doi.org/10.1109/lgrs.2015.2475299

ZHU Q Q ， ZHONG Y F ， ZHAO B ， et al . Bag-of-visual-words scene classifier with local and global features for high spatial resolution remote sensing imagery ［J］. IEEE Geoscience and Remote Sensing Letters ， 2016 ， 13 （ 6 ）： 747 - 751 . doi: 10.1109/lgrs.2015.2513443 http://dx.doi.org/10.1109/lgrs.2015.2513443

Kornblith S ， Norouzi M ， Lee H ， et al . Similarity of Neural Network Representations Revisited ［C］. Proceedings of the 36th International Conference on Machine Learning ： Pmlr ， 2019 ： -3529 - 3519 .

Views

1924

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Collaborative classification of hyperspectral and LiDAR data based on CNN-transformer

Hyperspectral images feature extraction and classification based on fractional differentiation

Lightweight deep global-local knowledge distillation network for hyperspectral image scene classification

Dense pedestrian detection algorithm in multi-branch non-anchor frame network

Application of SENet generative adversarial network in image semantics description

Related Author

WU Haibin

DAI Shiyu

WANG Aili

YUJI Iwahori

YU Xiaoyu

LIU Jing

LI Yang

LIU Yi

Related Institution

College of Electron and Information， University of Electronic Science and Technology of China，Zhongshan Institute

Heilongjiang Province Key Laboratory of Laser Spectroscopy Technology and Application， College of Measurement and Control Technology and Communication Engineering， Harbin University of Science and Technology

Department of Computer Science， Chubu University

School of Electronic Engineering， Xi’an University of Posts and Telecommunications

School of Electronic Engineering， Xidian University

AI问答

Address：No.3888 Dong Nanhu Road, Changchun, Jilin, China Postal code：130033
Tel：0431-86176855 Email：gxjmgc@ciomp.ac.cn
Technical support is provided by Beijing Founder electronics co., LTD 吉ICP备11002662号-17 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰