Rotation-invariant 2D views-3D point clouds auto-encoder

LIU Xianying; WU Qiuxia; KANG Wenxiong; LI Yuqiong

doi:10.37188/OPE.20233105.0656

您当前的位置：

首页 >

文章列表页 >

Rotation-invariant 2D views-3D point clouds auto-encoder

Three-dimensional topographic mapping | 更新时间：2023-03-22

- Rotation-invariant 2D views-3D point clouds auto-encoder
- Optics and Precision Engineering Vol. 31, Issue 5, Pages: 656-666(2023)
- 作者机构：
  
  1.华南理工大学软件学院，广东广州 510006
  2.华南理工大学自动化科学与工程学院，广东广州 510641
  3.中国科学院力学研究所，北京 100190
- 作者简介：
- 基金信息：
- DOI：10.37188/OPE.20233105.0656
  CLC： TP391
- Received：27 July 2022，
  
  Revised：30 August 2022，
  
  Published：10 March 2023
- 稿件说明：
移动端阅览
刘贤颖,吴秋遐,康文雄等.旋转不变的2D视图-3D点云自编码器[J].光学精密工程,2023,31(05):656-666.

LIU Xianying,WU Qiuxia,KANG Wenxiong,et al.Rotation-invariant 2D views-3D point clouds auto-encoder[J].Optics and Precision Engineering,2023,31(05):656-666.
刘贤颖,吴秋遐,康文雄等.旋转不变的2D视图-3D点云自编码器[J].光学精密工程,2023,31(05):656-666. DOI： 10.37188/OPE.20233105.0656.

LIU Xianying,WU Qiuxia,KANG Wenxiong,et al.Rotation-invariant 2D views-3D point clouds auto-encoder[J].Optics and Precision Engineering,2023,31(05):656-666. DOI： 10.37188/OPE.20233105.0656.

摘要

点云的无监督表征学习对于理解和分析点云至关重要，基于三维重建的自动编码器是无监督学习中的重要架构。针对现有的自编码器存在旋转干扰和特征学习能力不足的问题，本文提出一个旋转不变的2D视图-3D点云自编码器。首先，设计局部融合全局的旋转不变特征转换策略。对于局部表示，利用手工设计特征对输入点云进行转换，生成旋转不变的点云表征；对于全局表示，提出一个基于主成分分析（Principal Component Analysis， PCA）的对齐模块，将旋转点云对齐同一姿态下，在补充全局信息的同时排除旋转干扰。然后，在编码器设计局部和非局部特征提取模块，充分提取点云的局部空间特征和非局部上下文相关性，并建模不同层次特征之间的语义一致性。最后，提出一个基于PCA对齐的2D-3D 重构的解码方法，重建对齐后的三维点云和二维视图，使编码器输出的点云表征集成来自3D点云和2D视图的丰富学习信号。实验结果表明：本算法在随机旋转的合成数据集ModelNet40和真实数据集ScanObjectNN上的识别精度分别为90.84%和89.02%，学习的点云表征在没有任何标签监督的情况下实现了良好的可辨别性，并且具有较好的旋转鲁棒性。

Abstract

The unsupervised representation learning of point clouds is crucial for understanding and analyzing point clouds， and a 3D reconstruction-based autoencoder is an important architecture in unsupervised learning. To address the rotation interference and insufficient feature learning capability of existing autoencoders， this study proposes a rotation-invariant 2D views-3D point clouds autoencoder. First， a local fusion global rotation-invariant feature conversion strategy is designed. For the local representation， the input point clouds are transformed into handcrafted rotation-invariant features； for the global representation， an alignment module based on PCA is proposed to align the rotating point clouds under the same pose to exclude the rotation interference while complementing the global information. Then， for the encoder， the local and non-local module are designed to fully extract the local spatial features and non-local contextual correlations of the point cloud and model the semantic consistency between different levels of features. Finally， a PCA alignment-based decoding method for 2D-3D reconstruction is proposed for reconstructing the aligned 3D point clouds and 2D views such that the point-cloud representation output from the encoder integrates rich learning signals from the 3D point clouds and 2D views. Experiments demonstrate that the recognition accuracies of this algorithm are 90.84% and 89.02% on the randomly rotated synthetic dataset ModelNet40 and real dataset ScanObjectNN， respectively. Moreover， the learned point-cloud representations achieve good discriminability without label supervision and have a good rotational robustness.

关键词

Keywords

references

GUO Y L ， WANG H Y ， HU Q Y ， et al . Deep learning for 3D point clouds： a survey ［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence ， 2021 ， 43 （ 12 ）： 4338 - 4364 . doi: 10.1109/tpami.2020.3005434 http://dx.doi.org/10.1109/tpami.2020.3005434

CHARLES R Q ， HAO S ， MO K C ， et al . PointNet： deep learning on point sets for 3D classification and segmentation ［C］. 2017 IEEE Conference on Computer Vision and Pattern Recognition . Honolulu， HI， USA . IEEE ， 2017 ： 77 - 85 . doi: 10.1109/cvpr.2017.16 http://dx.doi.org/10.1109/cvpr.2017.16

QI C R ， YI L ， SU H ， et al . Pointnet++： Deep hierarchical feature learning on point sets in a metric space ［J］. Advances in Neural Information Processing Systems ， 2017 ， 30 .

WANG Y ， SUN Y B ， LIU Z W ， et al . Dynamic graph CNN for learning on point clouds ［J］. ACM Transactions on Graphics ， 2019 ， 38 （ 5 ）： 1 - 12 . doi: 10.1145/3326362 http://dx.doi.org/10.1145/3326362

杨军，党吉圣 . 采用深度级联卷积神经网络的三维点云识别与分割［J］. 光学精密工程， 2020 ， 28 （ 5 ）： 1187 - 1199 . doi: 10.3788/OPE.20202805.1187 http://dx.doi.org/10.3788/OPE.20202805.1187

YANG J ， DANG J S . Recognition and segmentation of three-dimensional point cloud based on deep cascade convolutional neural network ［J］. Opt. Precision Eng. ， 2020 ， 28 （ 5 ）： 1187 - 1199 . （in Chinese） . doi: 10.3788/OPE.20202805.1187 http://dx.doi.org/10.3788/OPE.20202805.1187

杨军，李博赞 . 基于自注意力特征融合组卷积神经网络的三维点云语义分割［J］. 光学精密工程， 2022 ， 30 （ 7 ）： 840 - 853 . doi: 10.37188/OPE.20223007.0840 http://dx.doi.org/10.37188/OPE.20223007.0840

YANG J ， LI B Z . Semantic segmentation of 3D point cloud based on self-attention feature fusion group convolutional neural network ［J］. Opt. Precision Eng. ， 2022 ， 30 （ 7 ）： 840 - 853 . （in Chinese） . doi: 10.37188/OPE.20223007.0840 http://dx.doi.org/10.37188/OPE.20223007.0840

陈俊英，白童垚，赵亮 . 互注意力融合图像和点云数据的3D目标检测［J］. 光学精密工程， 2021 ， 29 （ 9 ）： 2247 - 2254 . doi: 10.37188/OPE.20212909.2247 http://dx.doi.org/10.37188/OPE.20212909.2247

CHEN J Y ， BAI T Y ， ZHAO L . 3D object detection based on fusion of point cloud and image by mutual attention ［J］. Opt. Precision Eng. ， 2021 ， 29 （ 9 ）： 2247 - 2254 . （in Chinese） . doi: 10.37188/OPE.20212909.2247 http://dx.doi.org/10.37188/OPE.20212909.2247

DENG H W ， BIRDAL T ， ILIC S . PPF-FoldNet： unsupervised learning of rotation invariant 3D local descriptors ［C］. Computer Vision-ECCV 2018 ， 2018 ： 602 - 618 . doi: 10.1007/978-3-030-01228-1_37 http://dx.doi.org/10.1007/978-3-030-01228-1_37

GU R B ， WU Q X ， LI Y Q ， et al . Enhanced local and global learning for rotation-invariant point cloud representation ［J］. IEEE MultiMedia ， 1906 ， PP（99）： 1.

LIU X H ， HAN Z Z ， WEN X ， et al . L2G auto-encoder： understanding point clouds by local-to-global reconstruction with hierarchical self-attention ［C］. MM '19 ： Proceedings of the 27th ACM International Conference on Multimedia . 2019 ： 989 - 997 . doi: 10.1145/3343031.3350960 http://dx.doi.org/10.1145/3343031.3350960

HAN Z Z ， WANG X Y ， LIU Y S ， et al . Multi-angle point cloud-VAE： unsupervised feature learning for 3D point clouds from multiple angles by joint self-reconstruction and half-to-half prediction ［C］. 2019 IEEE/CVF International Conference on Computer Vision （ICCV）. Seoul ， Korea （South） . IEEE ， 2019 ： 10441 - 10450 . doi: 10.1109/iccv.2019.01054 http://dx.doi.org/10.1109/iccv.2019.01054

XIE S N ， GU J T ， GUO D M ， et al . PointContrast： unsupervised pre-training for 3D point cloud understanding ［C］. Computer Vision-ECCV 2020 ， 2020 ： 574 - 591 . doi: 10.1007/978-3-030-58580-8_34 http://dx.doi.org/10.1007/978-3-030-58580-8_34

RAO Y M ， LU J W ， ZHOU J . Global-local bidirectional reasoning for unsupervised representation learning of 3D point clouds ［C］. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Seattle ， WA ， USA . IEEE ， 2020 ： 5375 - 5384 . doi: 10.1109/cvpr42600.2020.00542 http://dx.doi.org/10.1109/cvpr42600.2020.00542

AFHAM M ， DISSANAYAKE I ， DISSANAYAKE D ， et al . CrossPoint： self-supervised cross-modal contrastive learning for 3D point cloud understanding ［C］. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. New Orleans ， LA， USA . IEEE ， 2022 ： 9892 - 9902 . doi: 10.1109/cvpr52688.2022.00967 http://dx.doi.org/10.1109/cvpr52688.2022.00967

YANG Y Q ， FENG C ， SHEN Y R ， et al . FoldingNet： point cloud auto-encoder via deep grid deformation ［C］. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City， UT， USA . IEEE ， 2018 ： 206 - 215 . doi: 10.1109/cvpr.2018.00029 http://dx.doi.org/10.1109/cvpr.2018.00029

LI Q M ， HAN Z C ， WU X M . Deeper insights into graph convolutional networks for semi-supervised learning ［J］. Proceedings of the AAAI Conference on Artificial Intelligence ， 2018 ， 32 （ 1 ）. doi: 10.1609/aaai.v32i1.11604 http://dx.doi.org/10.1609/aaai.v32i1.11604

WU Z R ， SONG S R ， KHOSLA A ， et al . 3D ShapeNets： a deep representation for volumetric shapes ［C］. 2015 IEEE Conference on Computer Vision and Pattern Recognition . Boston， MA， USA . IEEE ， 2015 ： 1912 - 1920 . doi: 10.1109/cvpr.2015.7298801 http://dx.doi.org/10.1109/cvpr.2015.7298801

UY M A ， PHAM Q H ， HUA B S ， et al . Revisiting point cloud classification： a new benchmark dataset and classification model on real-world data ［C］. 2019 IEEE/CVF International Conference on Computer Vision （ICCV）. Seoul ， Korea （South） . IEEE ， 2019 ： 1588 - 1597 . doi: 10.1109/iccv.2019.00167 http://dx.doi.org/10.1109/iccv.2019.00167

Views

497

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Semantic segmentation of 3D point cloud based on self-attention feature fusion group convolutional neural network

Color restoration of object surface point cloud in three-dimensional laser scanning system

Fast scan method of digital speckle and three-dimensional human body

Related Author

Jun YANG

Bozan LI

Ou SHA

Yuan-zhi LÜ

Jian-yong LING

Han-yi FU

WANG Yong-xin

LU Bing-heng

Related Institution

Faculty of Geomatics， Lanzhou Jiaotong University

School of Automation and Electrical Engineering， Lanzhou Jiaotong University

Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences

University of Chinese Academy of Sciences

State Key Laboratory for Manufacturing Systems Engineering, School of Mechanical Engineering, Xi'an Jiaotong University

Address：No.3888 Dong Nanhu Road, Changchun, Jilin, China Postal code：130033
Tel：0431-86176855 Email：gxjmgc@ciomp.ac.cn
Technical support is provided by Beijing Founder electronics co., LTD 吉ICP备11002662号-17 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰