采用深度级联卷积神经网络的三维点云识别与分割

杨军; 党吉圣

doi:10.3788/OPE.20202805.1187

您当前的位置：

首页 >

文章列表页 >

采用深度级联卷积神经网络的三维点云识别与分割

信息科学 | 更新时间：2020-07-09

- 采用深度级联卷积神经网络的三维点云识别与分割
- Recognition and segmentation of three-dimensional point cloud based on deep cascade convolutional neural network
- 光学精密工程 2020年28卷第5期页码：1187-1199
- 作者机构：
  
  兰州交通大学电子与信息工程学院，甘肃兰州 730070
- 作者简介：
  
  [ "杨军(1973-)，男，宁夏吴忠人，博士后，教授，博士生导师，1995年于西北师范大学获得学士学位，2002年于兰州交通大学获硕士学位，2007年于西南交通大学计算机科学专业获博士学位，主要从事三维模型的空间分析、模式识别等方面的研究。E-mail: yangj@mail.lzjtu.cn" ]
  [ "党吉圣(1991-)，男，甘肃武威人，硕士研究生，2016年于兰州交通大学获得学士学位, 主要从事模式识别、计算机视觉方面的研究。E-mail: 1442342449@qq.com" ]
- 基金信息：
  
  国家自然科学基金资助项目(61862039)
- DOI：10.3788/OPE.20202805.1187
  中图分类号： TP391
- 收稿日期：2019-12-02，
  
  修回日期：2020-03-13，
  
  录用日期：2020-3-13，
  
  纸质出版日期：2020-05-25
- 稿件说明：
移动端阅览
杨军, 党吉圣. 采用深度级联卷积神经网络的三维点云识别与分割[J]. 光学精密工程, 2020,28(5):1187-1199.

Jun YANG, Ji-sheng DANG. Recognition and segmentation of three-dimensional point cloud based on deep cascade convolutional neural network[J]. Optics and precision engineering, 2020, 28(5): 1187-1199.
杨军, 党吉圣. 采用深度级联卷积神经网络的三维点云识别与分割[J]. 光学精密工程, 2020,28(5):1187-1199. DOI： 10.3788/OPE.20202805.1187.

Jun YANG, Ji-sheng DANG. Recognition and segmentation of three-dimensional point cloud based on deep cascade convolutional neural network[J]. Optics and precision engineering, 2020, 28(5): 1187-1199. DOI： 10.3788/OPE.20202805.1187.

摘要

三维目标识别和模型语义分割在自动驾驶、机器人导航、3D打印和智能交通等领域均有着广泛应用。针对PointNet++未能结合三维模型的上下文几何结构信息的问题，提出一种采用深度级联卷积神经网络的三维点云识别与分割方法。首先，通过构建深度动态图卷积神经网络捕捉点云的深层语义几何特征；其次，通过将深度动态图卷积神经网络作为深度级联卷积神经网络的子网络递归地应用于输入点集的嵌套分区，以充分挖掘三维模型的深层细粒度几何特征；最后，针对点集特征学习中的点云采样不均匀问题，构建一种密度自适应层，利用循环神经网络编码每个采样点的多尺度邻域特征以捕捉上下文细粒度几何特征。实验结果表明

本算法在三维目标识别数据集ModelNet40和MoelNet10上的识别准确率分别为91.9%和94.3%，在语义分割数据集ShapeNet Part

S3DIS和vKITTI上的平均交并比分别为85.6%

58.3%和38.6%。该算法能够提高三维点云目标识别和模型语义分割的准确率，且具有较高的鲁棒性。

Abstract

Three-dimensional (3D) object recognition and model semantic segmentation are widely appliedin fields such as automatic driving

robot navigation

3D printing

and intelligent transportation. With a focuson the inability of PointNet++ to integrate contextual geometric structure information

a method for recognition and segmentation of 3D point cloud modes based on a deep cascade Convolutional Neural Network (CNN) was proposed herein. The deep semantic geometric features of the point cloud could be captured via construction of a deep dynamic graph CNN. Subsequently

the deep dynamic graph CNN was applied recursively as a subnetwork of a deep cascade CNN for nested partition of the input point set for full exploration of the fine-grained geometric features of the 3D model. Finally

to address the point cloud sampling nonuniformity problem in point set feature learning

a density adaptive layer was constructed.A recurrent neural network was used to encode the multiscale neighborhood features of each sample point to capture the contextual fine-grained geometric features. The experimental results showed that the recognition accuracy of this algorithm on ModelNet40 and ModelNet10 were 91.9% and 94.3%

respectively.The mean intersection-over-union on the ShapeNet Part

S3DIS

and vKITTI datasets was 85.6%

58.3%

and 38.6%

respectively. This algorithm can improve the accuracy of 3D point cloud recognition and model semantic segmentation

and it shows high robustness.

关键词

Keywords

references

R OSADA , T FUNKHOUSER , B CHAZELLE , 等 . Shape distributions . ACM Transactions on Graphics , 2002 . 21 ( 4 ): 807 - 832 . DOI: 10.1145/571647.571648 http://doi.org/10.1145/571647.571648 .

J SUN , M OVSJANIKOV , L GUIBAS . A concise and provably informative multi-scale signature based on heat diffusion . Computer Graphics Forum , 2009 . 28 ( 5 ): 1383 - 1392 . DOI: 10.1111/j.1467-8659.2009.01515.x http://doi.org/10.1111/j.1467-8659.2009.01515.x .

KRIZHEVSKY A, SUTSKEVER I, HINTON G E. Imagenet classification with deep convolutional neural networks[C]. Advances in Neural Information Processing Systems, Long Beach, USA: NIPS , 2012: 1097-1105.

吕晓琪 , 吴凉 , 谷宇 , 等 . 基于三维卷积神经网络的低剂量CT肺结节检测 . 光学精密工程 , 2018 . 26 ( 5 ): 1211 - 1218 . http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201805023 http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201805023 .

X Q LV , L WU , Y GU , 等 . Detection of low dose CT pulmonary nodules based on 3D convolution neural network . Opt. Precision Eng. , 2018 . 26 ( 5 ): 1211 - 1218 . http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201805023 http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201805023 .

潘仙张 , 张石清 , 郭文平 . 多模深度卷积神经网络应用于视频表情识别 . 光学精密工程 , 2019 . 27 ( 4 ): 963 - 970 . http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201904023 http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201904023 .

X ZH PAN , SH Q ZHANG , W P GUO . Video-based facial expression recognition using multimodal deep convolutional neural networks . Opt. Precision Eng. , 2019 . 27 ( 4 ): 963 - 970 . http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201904023 http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201904023 .

郑斌琪 , 李宝清 , 刘华巍 , 等 . 采用自适应一致性UKF的分布式目标跟踪 . 光学精密工程 , 2019 . 27 ( 1 ): 260 - 270 . http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201901029 http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201901029 .

B Q ZHENG , B Q LI , H W LIU , 等 . Distributed target tracking based on adaptive consensus UKF . Opt. Precision Eng. , 2019 . 27 ( 1 ): 260 - 270 . http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201901029 http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201901029 .

SU H, MAJI S, KALOGERAKIS E, et al . Multi-view convolutional neural networks for 3D shape recognition[C]. IEEE International Conference on Computer Vision, New York, USA: IEEE , 2015: 945-953.

SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[J]. ArXiv Preprint ArXiv : 1409.1556, 2014.

MATURANA D, SCHERER S, FRANKA A. VoxNet: a 3D convolutional neural network for real-time object recognition[C]. IEEE International Conference on Intelligent Robots and Systems, New York, USA: IEEE , 2015: 922-928.

杨军 , 王亦民 . 基于深度级联卷积神经网络的三维模型识别 . 重庆邮电大学学报 , 2019 . 31 ( 2 ): 253 - 260 .

J YANG , Y M WANG . 3D model recognition and classification based on deep convolution neural network . Journal of Chongqing University , 2019 . 31 ( 2 ): 253 - 260 .

杨军 , 王顺 , 周鹏 . 基于深度体素卷积神经网络的三维模型识别分类 . 光学学报 , 2019 . 39 ( 4 ): 0415007 http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=gxxb201904037 http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=gxxb201904037 .

J YANG , SH WANG , P ZHOU . 3D model recognition and classification based on deep voxel convolution neural network . Acta Optica Sinica , 2019 . 39 ( 4 ): 0415007 http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=gxxb201904037 http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=gxxb201904037 .

KLOKOV R, LEMPITSKY V. Escape from cells: deep kd-networks for the recognition of 3D point cloud models[C]. Proceedings of the IEEE International Conference on Computer Vision, New York, USA: IEEE , 2017: 863-872.

ZHU Y, MOTTAGHI R, KOLVE E, et al . Target-driven visual navigation in indoor scenes using deep reinforcement learning[C]. IEEE International Conference on Robotics and Automation, New York, USA: IEEE , 2017: 3357-3364.

QI C, LIU W, WU C, et al . Frustum pointnets for 3D object detection from RGB-D data[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, New York, USA: IEEE , 2018: 918-927.

R RUSU , Z MARTON , BLODOW , 等 . Towards 3D point cloud based object maps for household environments . Robotics and Autonomous Systems , 2008 . 56 ( 11 ): 927 - 941 . DOI: 10.1016/j.robot.2008.08.005 http://doi.org/10.1016/j.robot.2008.08.005 .

赵传 , 张保明 , 余东行 , 等 . 利用迁移学习的机载激光雷达点云分类 . 光学精密工程 , 2019 . 27 ( 7 ): 1601 - 1612 . http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201907022 http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201907022 .

CH ZHAO , B M ZHANG , D X ZHANG , 等 . Airborne LiDAR point cloud classification using transfer learning . Opt. Precision Eng. , 2019 . 27 ( 7 ): 1601 - 1612 . http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201907022 http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201907022 .

QI C, SU H, MO K, et al . Pointnet: deep learning on point sets for 3D classification and segmentation [C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, New York, USA: IEEE , 2017: 652-660.

QI C R, YI L, SU H, et al . Pointnet++: deep hierarchical feature learning on point sets in a metric space[C]. Advances in neural information processing systems, Long Beach, USA: NIPS , 2017: 5099-5108.

Y WANG , Y SUN , Z LIU , 等 . Dynamic graph cnn for learning on point clouds . ACM Transactions on Graphics , 2019 . 38 ( 5 ): 146 http://cn.bing.com/academic/profile?id=b05eb60575571c953271e13e882a2899&encoded=0&v=paper_preview&mkt=zh-cn http://cn.bing.com/academic/profile?id=b05eb60575571c953271e13e882a2899&encoded=0&v=paper_preview&mkt=zh-cn .

ZHANG K, HAO M, WANG J, et al . Linked dynamic graph CNN: learning on point cloud via linking hierarchical features[J]. ArXiv Preprint ArXiv: 1904.10014, 2019.

HE K, ZHANG X, REN S, et al . Deep residual learning for image recognition[C]. Proceedings of the IEEE conference on computer vision and pattern recognition. New York, USA: IEEE , 2016: 770-778.

WU Z, SONG S, KHOSLA A, et al . 3D shapenets: a deep representation for volumetric shapes[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, New York, USA: IEEE , 2015: 1912-1920.

L YI , V G KIM , D CEYLAN , 等 . A scalable active framework for region annotation in 3D shape collections . ACM Transactions on Graphics , 2016 . 35 ( 6 ): 210 http://cn.bing.com/academic/profile?id=5c6a0813f7dd1714a58a37d5977d7f0f&encoded=0&v=paper_preview&mkt=zh-cn http://cn.bing.com/academic/profile?id=5c6a0813f7dd1714a58a37d5977d7f0f&encoded=0&v=paper_preview&mkt=zh-cn .

ARMENI I, SENER O, ZAMIR A R, et al . 3D semantic parsing of large-scale indoor spaces[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. New York, USA: IEEE , 2016: 1534-1543.

ENGELMANN F, KONTOGIANNI T, HERMANS A, et al . Exploring spatial context for 3D semantic segmentation of point clouds[C]. Proceedings of the IEEE International Conference on Computer Vision. New York, USA: IEEE , 2017: 716-724.

KLAMBAUER G, UNTERHINER T, MAYR A, et al . Self-normalizing neural networks[C]. Advances in Neural Information Processing Systems, Long Beach, USA: NIPS , 2017: 971-980.

GAIDON A, WANG Q, CABON Y, et al . Virtual worlds as proxy for multi-object tracking analysis[C]. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. New York, USA: IEEE , 2016.

浏览量

819

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于跨层次聚合网络的实时城市街景语义分割

基于机载的红外动态目标视频实时超分辨率重建

面向遥感图像道路提取的多尺度上下文感知网络

融合生成对抗网络的大气无线光信道密钥提取

融合注意力机制的改进型DeepLabv3+语义分割