浏览全部资源
扫码关注微信
1.兰州交通大学 电子与信息工程学院,甘肃 兰州 730070
2.兰州交通大学 测绘与地理信息学院,甘肃 兰州 730070
Received:15 February 2022,
Revised:11 March 2022,
Published:10 September 2022
移动端阅览
杨军,张景发.基于投票机制的神经架构搜索[J].光学精密工程,2022,30(17):2119-2132.
YANG Jun,ZHANG Jingfa.Neural architecture search algorithm based on voting scheme[J].Optics and Precision Engineering,2022,30(17):2119-2132.
杨军,张景发.基于投票机制的神经架构搜索[J].光学精密工程,2022,30(17):2119-2132. DOI: 10.37188/OPE.20223017.2119.
YANG Jun,ZHANG Jingfa.Neural architecture search algorithm based on voting scheme[J].Optics and Precision Engineering,2022,30(17):2119-2132. DOI: 10.37188/OPE.20223017.2119.
针对现有神经架构搜索算法自动搜索到的网络架构与评估的网络架构之间存在较大差异的问题,提出了基于投票机制的神经架构搜索算法。首先,利用小批量训练数据上测试的训练损失作为性能估计器对候选网络进行采样,将计算资源集中于潜在的性能表现良好的候选网络架构,以解决均匀采样忽略了各网络架构之间重要性程度的问题;其次,对于各节点中候选操作难以选择的问题,利用组稀疏正则化策略对所有候选操作进行排名,以筛选出合适的候选操作,进一步提高Cell结构中路径选择的准确性;最后,将可微架构搜索策略、噪声策略和组稀疏正则化策略加以融合,以加权投票的方法选择出最优的Cell结构,构建出性能优秀的三维模型识别与分类网络架构。在数据集ModelNet40上的实验结果表明,所构建的网络对三维模型的分类准确率达到了93.9%,优于目前的主流算法。本算法有效缩小了搜索和评估阶段网络架构之间的差异,解决了以往神经架构搜索方法中均匀采样所导致的网络训练效率低的问题。
A neural-architecture search algorithm based on a voting scheme was proposed to address the difference between network architectures that are automatically searched by existing algorithms and those that were evaluated by the algorithm. First, to solve the problem that uniform sampling ignores the importance of each network architecture, the training losses tested on small batch training data were used as performance estimators to sample candidate networks, thus concentrating computing resources on high-performance candidate network architectures. Second, a group sparsity regularization strategy was adopted to rank all candidate operations to solve the problem of selecting candidate operations in each node. This strategy could screen suitable candidate operations and further enhance the precision of path selection in the cell structure. Finally, by integrating the differentiable architecture search, noise and sparse regularization strategies, the optimal cell structure was selected using a weighted voting scheme, and the network architecture for 3D model recognition and classification was constructed. Experimental results indicate that the classification accuracy of the constructed network for 3D models reaches 93.9% on the ModelNet40 dataset, which is higher than that of current mainstream algorithms. The proposed algorithm effectively narrows the gap between the network architecture during the search and evaluation phases, thereby resolving the problem of inefficient network training caused by uniform sampling in previous neural-architecture search methods.
GUO Y , CHEN J , DU Q , et al . Multi-way backpropagation for training compact deep neural networks [J]. Neural Networks , 2020 , 126 : 250 - 261 . doi: 10.1016/j.neunet.2020.03.001 http://dx.doi.org/10.1016/j.neunet.2020.03.001
SIMONYAN K , ZISSERMAN A . Very deep convolutional networks for large-scale image recognition [EB/OL]. 2014: arXiv : 1409 . 1556 . https://arxiv.org/abs/1409.1556 https://arxiv.org/abs/1409.1556
HE K M , ZHANG X Y , REN S Q , et al . Deep residual learning for image recognition [C]. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). June 27 - 30 , 2016 . Las Vegas, NV, USA. IEEE , 2016 : 770 - 778 . doi: 10.1109/cvpr.2016.90 http://dx.doi.org/10.1109/cvpr.2016.90
CHARLES R Q , HAO S , MO K C , et al . PointNet: deep learning on point sets for 3D classification and segmentation [C]. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). July 21 - 26 , 2017 . Honolulu, HI. IEEE , 2017 : 652 - 660 . doi: 10.1109/cvpr.2017.16 http://dx.doi.org/10.1109/cvpr.2017.16
ZOPH B , VASUDEVAN V , SHLENS J , et al . Learning transferable architectures for scalable image recognition [C]. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City, UT, USA . IEEE , 2018 : 8697 - 8710 . doi: 10.1109/cvpr.2018.00907 http://dx.doi.org/10.1109/cvpr.2018.00907
LIU H X , SIMONYAN K , YANG Y M . DARTS: differentiable architecture search [EB/OL]. 2018: arXiv : 1806 . 09055 . https://arxiv.org/abs/1806.09055 https://arxiv.org/abs/1806.09055
CHEN X , XIE L X , WU J , et al . Progressive differentiable architecture search: bridging the depth gap between search and evaluation [C]. 2019 IEEE/CVF International Conference on Computer Vision (ICCV). Seoul , Korea (South) . IEEE , 2019 : 1294 - 1303 . doi: 10.48550/arXiv.1904.12760 http://dx.doi.org/10.48550/arXiv.1904.12760
DONG X Y , YANG Y . Searching for a robust neural architecture in four GPU hours [C]. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach , CA, USA . IEEE , 2019 : 1761 - 1770 . doi: 10.1109/cvpr.2019.00186 http://dx.doi.org/10.1109/cvpr.2019.00186
ZOPH B , LE Q V . Neural architecture search with reinforcement learning [EB/OL]. 2016: arXiv : 1611 . 01578 . https://arxiv.org/abs/1611.01578 https://arxiv.org/abs/1611.01578
ZHONG Z , YAN J J , WU W , et al . Practical block-wise neural network architecture generation [C]. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City, UT, USA . IEEE , 2018 : 2423 - 2432 . doi: 10.1109/cvpr.2018.00257 http://dx.doi.org/10.1109/cvpr.2018.00257
TAN M X , LE Q V . EfficientNet: rethinking model scaling for convolutional neural networks [EB/OL]. 2019: arXiv : 1905 . 11946 . https://arxiv.org/abs/1905.11946 https://arxiv.org/abs/1905.11946
PHAM H , GUAN M Y , ZOPH B , et al . Efficient neural architecture search via parameter sharing [EB/OL]. 2018: arXiv : 1802 . 03268 . https://arxiv.org/abs/1802.03268 https://arxiv.org/abs/1802.03268
REAL E , AGGARWAL A , HUANG Y P , et al . Regularized evolution for image classifier architecture search [C]. Proceedings of the AAAI Conference on Artificial Intelligence , 2019 , 33 : 4780 - 4789 . doi: 10.1609/aaai.v33i01.33014780 http://dx.doi.org/10.1609/aaai.v33i01.33014780
LIANG T T , WANG Y T , TANG Z , et al . OPANAS: one-shot path aggregation network architecture search for object detection [C]. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Nashville, TN, USA. IEEE , : 10190- 10198 .
CASALE F P , GORDON J , FUSI N . Probabilistic neural architecture search [EB/OL]. 2019: arXiv : 1902 . 05116 . https://arxiv.org/abs/1902.05116 https://arxiv.org/abs/1902.05116
CAI H , ZHU L G , HAN S . ProxylessNAS: direct neural architecture search on target task and hardware [EB/OL]. 2018: arXiv : 1812 . 00332 . https://arxiv.org/abs/1812.00332 https://arxiv.org/abs/1812.00332
YOU S , HUANG T , YANG M M , et al . GreedyNAS: towards fast one-shot NAS with greedy supernet [C]. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle , WA , USA . IEEE , 2020 : 1996 - 2005 . doi: 10.1109/cvpr42600.2020.00207 http://dx.doi.org/10.1109/cvpr42600.2020.00207
LI G H , QIAN G C , DELGADILLO I C , et al . SGAS: sequential greedy architecture search [C]. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle , WA , USA . IEEE , 2020 : 1617 - 1627 . doi: 10.1109/cvpr42600.2020.00169 http://dx.doi.org/10.1109/cvpr42600.2020.00169
YU K C , SCIUTO C , JAGGI M , et al . Evaluating the search phase of neural architecture search [EB/OL]. 2019: arXiv : 1902 . 08142 . https://arxiv.org/abs/1902.08142 https://arxiv.org/abs/1902.08142
CHU X X , ZHOU T B , ZHANG B , et al . Fair DARTS : Eliminating Unfair Advantages in Differentiable Architecture Search [M]. Computer Vision - ECCV 2020 . Cham : Springer International Publishing , 2020 : 465 - 480 . doi: 10.1007/978-3-030-58555-6_28 http://dx.doi.org/10.1007/978-3-030-58555-6_28
LIU H X , SIMONYAN K , YANG Y M . DARTS: differentiable architecture search [EB/OL]. 2018: arXiv : 1806 . 09055 . https://arxiv.org/abs/1806.09055 https://arxiv.org/abs/1806.09055
THOMAS H , QI C R , DESCHAUD J E , et al . KPConv: flexible and deformable convolution for point clouds [C]. 2019 IEEE/CVF International Conference on Computer Vision (ICCV). Seoul , Korea (South) . IEEE , 2019 : 6410 - 6419 . doi: 10.1109/iccv.2019.00651 http://dx.doi.org/10.1109/iccv.2019.00651
KOMARICHEV A , ZHONG Z C , HUA J . A-CNN: annularly convolutional neural networks on point clouds [C]. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach , CA, USA . IEEE , 2019 : 7413 - 7422 . doi: 10.1109/cvpr.2019.00760 http://dx.doi.org/10.1109/cvpr.2019.00760
LIU X H , HAN Z Z , LIU Y S , et al . Point2Sequence: learning the shape representation of 3D point clouds with an attention-based sequence to sequence network [C]. Proceedings of the AAAI Conference on Artificial Intelligence , 2019 , 33 : 8778 - 8785 . doi: 10.1609/aaai.v33i01.33018778 http://dx.doi.org/10.1609/aaai.v33i01.33018778
WANG Y , SUN Y B , LIU Z W , et al . Dynamic graph CNN for learning on point clouds [J]. ACM Transactions on Graphics , 2019 , 38 ( 5 ): 1 - 12 . doi: 10.1145/3326362 http://dx.doi.org/10.1145/3326362
XU Q G , SUN X D , WU C Y , et al . Grid-GCN for fast and scalable point cloud learning [C]. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle , WA , USA . IEEE , 2020 : 5660 - 5669 . doi: 10.1109/cvpr42600.2020.00570 http://dx.doi.org/10.1109/cvpr42600.2020.00570
BEN-SHABAT Y , LINDENBAUM M , FISCHER A . 3DmFV: three-dimensional point cloud classification in real-time using convolutional neural networks [J]. IEEE Robotics and Automation Letters , 2018 , 3 ( 4 ): 3145 - 3152 . doi: 10.1109/lra.2018.2850061 http://dx.doi.org/10.1109/lra.2018.2850061
LAN S Y , YU R C , YU G , et al . Modeling local geometric structure of 3D point clouds using geo-CNN [C]. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach , CA, USA . IEEE , 2019 : 998 - 1008 . doi: 10.1109/cvpr.2019.00109 http://dx.doi.org/10.1109/cvpr.2019.00109
XU M T , DING R Y , ZHAO H S , et al . PAConv: position adaptive convolution with dynamic kernel assembling on point clouds [C]. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Nashville , TN , USA . IEEE , 2021 : 3172 - 3181 . doi: 10.1109/cvpr46437.2021.00319 http://dx.doi.org/10.1109/cvpr46437.2021.00319
XU M Y , ZHOU Z P , QIAO Y . Geometry sharing network for 3D point cloud classification and segmentation [C]. Proceedings of the AAAI Conference on Artificial Intelligence , 2020 , 34 ( 7 ): 12500 - 12507 . doi: 10.1609/aaai.v34i07.6938 http://dx.doi.org/10.1609/aaai.v34i07.6938
0
Views
675
下载量
0
CSCD
Publicity Resources
Related Articles
Related Author
Related Institution