Recognition and segmentation of three-dimensional point cloud based on deep cascade convolutional neural network

Jun YANG; Ji-sheng DANG

doi:10.3788/OPE.20202805.1187

您当前的位置：

首页 >

文章列表页 >

Recognition and segmentation of three-dimensional point cloud based on deep cascade convolutional neural network

Information Sciences | 更新时间：2020-07-09

- Recognition and segmentation of three-dimensional point cloud based on deep cascade convolutional neural network
- Optics and Precision Engineering Vol. 28, Issue 5, Pages: 1187-1199(2020)
- 作者机构：
  
  兰州交通大学电子与信息工程学院，甘肃兰州 730070
- 作者简介：
- 基金信息：
- DOI：10.3788/OPE.20202805.1187
  CLC： TP391
- Received：02 December 2019，
  
  Revised：13 March 2020，
  
  Accepted：13 March 2020，
  
  Published：25 May 2020
- 稿件说明：
移动端阅览
Jun YANG, Ji-sheng DANG. Recognition and segmentation of three-dimensional point cloud based on deep cascade convolutional neural network[J]. Optics and precision engineering, 2020, 28(5): 1187-1199.
DOI：

Jun YANG, Ji-sheng DANG. Recognition and segmentation of three-dimensional point cloud based on deep cascade convolutional neural network[J]. Optics and precision engineering, 2020, 28(5): 1187-1199. DOI： 10.3788/OPE.20202805.1187.

摘要

三维目标识别和模型语义分割在自动驾驶、机器人导航、3D打印和智能交通等领域均有着广泛应用。针对PointNet++未能结合三维模型的上下文几何结构信息的问题，提出一种采用深度级联卷积神经网络的三维点云识别与分割方法。首先，通过构建深度动态图卷积神经网络捕捉点云的深层语义几何特征；其次，通过将深度动态图卷积神经网络作为深度级联卷积神经网络的子网络递归地应用于输入点集的嵌套分区，以充分挖掘三维模型的深层细粒度几何特征；最后，针对点集特征学习中的点云采样不均匀问题，构建一种密度自适应层，利用循环神经网络编码每个采样点的多尺度邻域特征以捕捉上下文细粒度几何特征。实验结果表明

本算法在三维目标识别数据集ModelNet40和MoelNet10上的识别准确率分别为91.9%和94.3%，在语义分割数据集ShapeNet Part

S3DIS和vKITTI上的平均交并比分别为85.6%

58.3%和38.6%。该算法能够提高三维点云目标识别和模型语义分割的准确率，且具有较高的鲁棒性。

Abstract

Three-dimensional (3D) object recognition and model semantic segmentation are widely appliedin fields such as automatic driving

robot navigation

3D printing

and intelligent transportation. With a focuson the inability of PointNet++ to integrate contextual geometric structure information

a method for recognition and segmentation of 3D point cloud modes based on a deep cascade Convolutional Neural Network (CNN) was proposed herein. The deep semantic geometric features of the point cloud could be captured via construction of a deep dynamic graph CNN. Subsequently

the deep dynamic graph CNN was applied recursively as a subnetwork of a deep cascade CNN for nested partition of the input point set for full exploration of the fine-grained geometric features of the 3D model. Finally

to address the point cloud sampling nonuniformity problem in point set feature learning

a density adaptive layer was constructed.A recurrent neural network was used to encode the multiscale neighborhood features of each sample point to capture the contextual fine-grained geometric features. The experimental results showed that the recognition accuracy of this algorithm on ModelNet40 and ModelNet10 were 91.9% and 94.3%

respectively.The mean intersection-over-union on the ShapeNet Part

S3DIS

and vKITTI datasets was 85.6%

58.3%

and 38.6%

respectively. This algorithm can improve the accuracy of 3D point cloud recognition and model semantic segmentation

and it shows high robustness.

关键词

Keywords

references

R OSADA , T FUNKHOUSER , B CHAZELLE , 等 . Shape distributions . ACM Transactions on Graphics , 2002 . 21 ( 4 ): 807 - 832 . DOI: 10.1145/571647.571648 http://doi.org/10.1145/571647.571648 .

J SUN , M OVSJANIKOV , L GUIBAS . A concise and provably informative multi-scale signature based on heat diffusion . Computer Graphics Forum , 2009 . 28 ( 5 ): 1383 - 1392 . DOI: 10.1111/j.1467-8659.2009.01515.x http://doi.org/10.1111/j.1467-8659.2009.01515.x .

KRIZHEVSKY A, SUTSKEVER I, HINTON G E. Imagenet classification with deep convolutional neural networks[C]. Advances in Neural Information Processing Systems, Long Beach, USA: NIPS , 2012: 1097-1105.

吕晓琪 , 吴凉 , 谷宇 , 等 . 基于三维卷积神经网络的低剂量CT肺结节检测 . 光学精密工程 , 2018 . 26 ( 5 ): 1211 - 1218 . http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201805023 http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201805023 .

X Q LV , L WU , Y GU , 等 . Detection of low dose CT pulmonary nodules based on 3D convolution neural network . Opt. Precision Eng. , 2018 . 26 ( 5 ): 1211 - 1218 . http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201805023 http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201805023 .

潘仙张 , 张石清 , 郭文平 . 多模深度卷积神经网络应用于视频表情识别 . 光学精密工程 , 2019 . 27 ( 4 ): 963 - 970 . http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201904023 http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201904023 .

X ZH PAN , SH Q ZHANG , W P GUO . Video-based facial expression recognition using multimodal deep convolutional neural networks . Opt. Precision Eng. , 2019 . 27 ( 4 ): 963 - 970 . http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201904023 http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201904023 .

郑斌琪 , 李宝清 , 刘华巍 , 等 . 采用自适应一致性UKF的分布式目标跟踪 . 光学精密工程 , 2019 . 27 ( 1 ): 260 - 270 . http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201901029 http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201901029 .

B Q ZHENG , B Q LI , H W LIU , 等 . Distributed target tracking based on adaptive consensus UKF . Opt. Precision Eng. , 2019 . 27 ( 1 ): 260 - 270 . http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201901029 http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201901029 .

SU H, MAJI S, KALOGERAKIS E, et al . Multi-view convolutional neural networks for 3D shape recognition[C]. IEEE International Conference on Computer Vision, New York, USA: IEEE , 2015: 945-953.

SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[J]. ArXiv Preprint ArXiv : 1409.1556, 2014.

MATURANA D, SCHERER S, FRANKA A. VoxNet: a 3D convolutional neural network for real-time object recognition[C]. IEEE International Conference on Intelligent Robots and Systems, New York, USA: IEEE , 2015: 922-928.

杨军 , 王亦民 . 基于深度级联卷积神经网络的三维模型识别 . 重庆邮电大学学报 , 2019 . 31 ( 2 ): 253 - 260 .

J YANG , Y M WANG . 3D model recognition and classification based on deep convolution neural network . Journal of Chongqing University , 2019 . 31 ( 2 ): 253 - 260 .

杨军 , 王顺 , 周鹏 . 基于深度体素卷积神经网络的三维模型识别分类 . 光学学报 , 2019 . 39 ( 4 ): 0415007 http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=gxxb201904037 http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=gxxb201904037 .

J YANG , SH WANG , P ZHOU . 3D model recognition and classification based on deep voxel convolution neural network . Acta Optica Sinica , 2019 . 39 ( 4 ): 0415007 http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=gxxb201904037 http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=gxxb201904037 .

KLOKOV R, LEMPITSKY V. Escape from cells: deep kd-networks for the recognition of 3D point cloud models[C]. Proceedings of the IEEE International Conference on Computer Vision, New York, USA: IEEE , 2017: 863-872.

ZHU Y, MOTTAGHI R, KOLVE E, et al . Target-driven visual navigation in indoor scenes using deep reinforcement learning[C]. IEEE International Conference on Robotics and Automation, New York, USA: IEEE , 2017: 3357-3364.

QI C, LIU W, WU C, et al . Frustum pointnets for 3D object detection from RGB-D data[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, New York, USA: IEEE , 2018: 918-927.

R RUSU , Z MARTON , BLODOW , 等 . Towards 3D point cloud based object maps for household environments . Robotics and Autonomous Systems , 2008 . 56 ( 11 ): 927 - 941 . DOI: 10.1016/j.robot.2008.08.005 http://doi.org/10.1016/j.robot.2008.08.005 .

赵传 , 张保明 , 余东行 , 等 . 利用迁移学习的机载激光雷达点云分类 . 光学精密工程 , 2019 . 27 ( 7 ): 1601 - 1612 . http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201907022 http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201907022 .

CH ZHAO , B M ZHANG , D X ZHANG , 等 . Airborne LiDAR point cloud classification using transfer learning . Opt. Precision Eng. , 2019 . 27 ( 7 ): 1601 - 1612 . http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201907022 http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201907022 .

QI C, SU H, MO K, et al . Pointnet: deep learning on point sets for 3D classification and segmentation [C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, New York, USA: IEEE , 2017: 652-660.

QI C R, YI L, SU H, et al . Pointnet++: deep hierarchical feature learning on point sets in a metric space[C]. Advances in neural information processing systems, Long Beach, USA: NIPS , 2017: 5099-5108.

Y WANG , Y SUN , Z LIU , 等 . Dynamic graph cnn for learning on point clouds . ACM Transactions on Graphics , 2019 . 38 ( 5 ): 146 http://cn.bing.com/academic/profile?id=b05eb60575571c953271e13e882a2899&encoded=0&v=paper_preview&mkt=zh-cn http://cn.bing.com/academic/profile?id=b05eb60575571c953271e13e882a2899&encoded=0&v=paper_preview&mkt=zh-cn .

ZHANG K, HAO M, WANG J, et al . Linked dynamic graph CNN: learning on point cloud via linking hierarchical features[J]. ArXiv Preprint ArXiv: 1904.10014, 2019.

HE K, ZHANG X, REN S, et al . Deep residual learning for image recognition[C]. Proceedings of the IEEE conference on computer vision and pattern recognition. New York, USA: IEEE , 2016: 770-778.

WU Z, SONG S, KHOSLA A, et al . 3D shapenets: a deep representation for volumetric shapes[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, New York, USA: IEEE , 2015: 1912-1920.

L YI , V G KIM , D CEYLAN , 等 . A scalable active framework for region annotation in 3D shape collections . ACM Transactions on Graphics , 2016 . 35 ( 6 ): 210 http://cn.bing.com/academic/profile?id=5c6a0813f7dd1714a58a37d5977d7f0f&encoded=0&v=paper_preview&mkt=zh-cn http://cn.bing.com/academic/profile?id=5c6a0813f7dd1714a58a37d5977d7f0f&encoded=0&v=paper_preview&mkt=zh-cn .

ARMENI I, SENER O, ZAMIR A R, et al . 3D semantic parsing of large-scale indoor spaces[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. New York, USA: IEEE , 2016: 1534-1543.

ENGELMANN F, KONTOGIANNI T, HERMANS A, et al . Exploring spatial context for 3D semantic segmentation of point clouds[C]. Proceedings of the IEEE International Conference on Computer Vision. New York, USA: IEEE , 2017: 716-724.

KLAMBAUER G, UNTERHINER T, MAYR A, et al . Self-normalizing neural networks[C]. Advances in Neural Information Processing Systems, Long Beach, USA: NIPS , 2017: 971-980.

GAIDON A, WANG Q, CABON Y, et al . Virtual worlds as proxy for multi-object tracking analysis[C]. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. New York, USA: IEEE , 2016.

Views

831

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Real-time urban street view semantic segmentation based on cross-layer aggregation network

Real-time super-resolution for infrared dynamic object video based on airborne platform

Multi-scale context-aware network for road extraction in remote sensing images

Secret key extraction from atmospheric wireless optical channels by combing with generative adversarial network

Improved DeepLabv3+ semantic segmentation incorporating attention mechanisms

Related Author

HOU Zhiqiang

CHENG Minjie

MA Sugang

QU Minjie

YANG Xiaobao

ZHU Deyan

XU Jiayi

AO Yongqi

Related Institution

Xi'an University of Posts and Telecommunications， Shaanxi Key Laboratory of Network Data Analysis and Intelligent Processing

Xi'an University of Posts and Telecommunications， Institute of Computer

Key Laboratory of Space Photoelectric Detection and Sensing of Industry and Information Technology， Nanjing University of Aeronautics and Astronautics

College of Astronautics， Nanjing University of Aeronautics and Astronautics

School of Information and Control Engineering， Xi’an University of Architectural Science and Technology

AI问答

Address：No.3888 Dong Nanhu Road, Changchun, Jilin, China Postal code：130033
Tel：0431-86176855 Email：gxjmgc@ciomp.ac.cn
Technical support is provided by Beijing Founder electronics co., LTD 吉ICP备11002662号-17 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰