Automatic attitude measurement of laser tracker based on deep learning and PnP model

Daode ZHOU; Doudou GAO; Dengfeng DONG; Weihu ZHOU; Chengjun CUI

doi:10.37188/OPE.20223009.1047

您当前的位置：

首页 >

文章列表页 >

Automatic attitude measurement of laser tracker based on deep learning and PnP model

Modern Applied Optics | 更新时间：2022-05-26

- Automatic attitude measurement of laser tracker based on deep learning and PnP model
- Optics and Precision Engineering Vol. 30, Issue 9, Pages: 1047-1057(2022)
- 作者机构：
  
  1.中国科学院微电子研究所，北京100029
  2.中国科学院大学，北京 100049
- 作者简介：
- 基金信息：
- DOI：10.37188/OPE.20223009.1047
  CLC： TP391.4;TH744
- Received：04 March 2022，
  
  Revised：16 March 2022，
  
  Published：10 May 2022
- 稿件说明：
移动端阅览
周道德,高豆豆,董登峰等.基于深度学习和PnP模型的激光跟踪仪自动姿态测量[J].光学精密工程,2022,30(09):1047-1057.

ZHOU Daode,GAO Doudou,DONG Dengfeng,et al.Automatic attitude measurement of laser tracker based on deep learning and PnP model[J].Optics and Precision Engineering,2022,30(09):1047-1057.
周道德,高豆豆,董登峰等.基于深度学习和PnP模型的激光跟踪仪自动姿态测量[J].光学精密工程,2022,30(09):1047-1057. DOI： 10.37188/OPE.20223009.1047.

ZHOU Daode,GAO Doudou,DONG Dengfeng,et al.Automatic attitude measurement of laser tracker based on deep learning and PnP model[J].Optics and Precision Engineering,2022,30(09):1047-1057. DOI： 10.37188/OPE.20223009.1047.

摘要

针对航空航天、汽车装配等高端制造领域对姿态测量的迫切需求，提出一种面向激光跟踪仪的快速高精度姿态测量方法，利用深度学习结合视觉PnP模型实现了激光跟踪过程中被测件姿态的自动测量。针对PnP姿态求解模型所需的3D特征点和2D特征点之间的对应关系难以直接确定的问题，设计了一个特征提取网络用于提取特征点对应的高维特征，采用最优传输理论确定特征向量之间的联合概率分布，从而完成3D-2D特征点的自动匹配；使用Ransac-P3P结合EPnP算法对匹配好的3D特征点和2D像素点进行姿态求解，获得高精度的姿态信息；在此基础上，利用隐式微分理论计算PnP求解过程的雅克比矩阵，从而将PnP姿态求解模型集成到网络中并指导网络训练，实现了深度网络匹配能力与PnP模型姿态求解能力的优势互补，提高了解算精度。最后，制作了一个含有丰富标注信息的数据集，用于训练面向激光跟踪仪的姿态测量网络。基于高精度二维转台进行了姿态测量实验，结果表明，该方法在3 m处对俯仰角的测量精度优于0.31°，横滚角精度优于0.03°，单次测量耗时约40 ms，能够实现激光跟踪仪的高精度姿态测量。

Abstract

In view of the urgent demand for attitude measurement in high-end manufacturing applications， such as aerospace and automobile assembly， a fast and high-precision attitude measurement method for a laser tracker was proposed. The method employed deep learning in conjunction with the visual PnP model to realize automatic attitude measurement of the laser tracker. The correspondence between 3D feature points and 2D feature points required by the traditional PnP model were directly determined through a feature extraction network designed to extract high-dimensional features. The joint probability distribution between feature vectors was determined using optimal transmission theory to complete the matching of 3D-2D feature points. Subsequently， Ransac-P3P combined with EPnP algorithm was used to obtain high-precision attitude information； Based on this， the Jacobian matrix of PnP solution process was calculated using implicit differential theory， and the PnP attitude solution model was integrated into the network to guide the training of the network. The complementary advantages of strong depth network matching ability and high attitude solution accuracy of the PnP model improved the solution accuracy of the network. In addition， a dataset with rich annotation information was used to train the attitude measurement network for the laser tracker. Finally， an attitude measurement test was conducted using a high-precision two-dimensional turntable. The experimental results show that the calculation error of pitch angle is less than 0.31°， the rolling angle error is less than 0.03°， and the single measurement takes approximately 40 ms. The proposed method can potentially be applied to attitude measurement scene of the laser tracker.

关键词

Keywords

references

朱永国，张文博，邓正平，等 . 基于激光跟踪仪和机器视觉的飞机翼身对接装配偏差动态综合修正［J］. 机械工程学报， 2019 ， 55 （ 24 ）： 187 - 196 . doi: 10.3901/jme.2019.24.187 http://dx.doi.org/10.3901/jme.2019.24.187

ZHU Y G ， ZHANG W B ， DENG ZH P ， et al . Dynamic synthesis correction of deviation for aircraft wing-fuselage docking assembly based on laser tracker and machine vision ［J］. Journal of Mechanical Engineering ， 2019 ， 55 （ 24 ）： 187 - 196 . （in Chinese） . doi: 10.3901/jme.2019.24.187 http://dx.doi.org/10.3901/jme.2019.24.187

杨文辉，林嘉睿，高扬，等 . 激光标靶位姿测量系统的建模与误差分析［J］. 纳米技术与精密工程， 2015 ， 13 （ 4 ）： 293 - 298 . doi: 10.13494/j.npe.20140069 http://dx.doi.org/10.13494/j.npe.20140069

YANG W H ， LIN J R ， GAO Y ， et al . Modeling and error analysis of laser target pose measurement system ［J］. Nanotechnology and Precision Engineering ， 2015 ， 13 （ 4 ）： 293 - 298 . （in Chinese） . doi: 10.13494/j.npe.20140069 http://dx.doi.org/10.13494/j.npe.20140069

迟书凯，叶旋，高翔，等 . 基于编码标记点的高精度运动估计［J］. 光学精密工程， 2021 ， 29 （ 7 ）： 1720 - 1730 . doi: 10.37188/OPE.20212907.1720 http://dx.doi.org/10.37188/OPE.20212907.1720

CHI SH K ， YE X ， GAO X ， et al . Coded marker-based high-accuracy motion estimation ［J］. Opt. Precision Eng. ， 2021 ， 29 （ 7 ）： 1720 - 1730 . （in Chinese） . doi: 10.37188/OPE.20212907.1720 http://dx.doi.org/10.37188/OPE.20212907.1720

WU D ， ZHUANG Z Y ， XIANG C Q ， et al . 6D-VNet： end-to-end 6DoF vehicle pose estimation from monocular RGB images ［C］. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops （CVPRW）. 1617，2019 ， Long Beach， CA， USA. IEEE ， 2019 ： 1238 - 1247 . doi: 10.1109/cvprw.2019.00163 http://dx.doi.org/10.1109/cvprw.2019.00163

KEHL W ， MANHARDT F ， TOMBARI F ， et al . SSD-6D： making RGB-based 3D detection and 6D pose estimation great again ［C］. 2017 IEEE International Conference on Computer Vision . 2229，2017 ， Venice， Italy . IEEE ， 2017 ： 1530 - 1538 . doi: 10.1109/iccv.2017.169 http://dx.doi.org/10.1109/iccv.2017.169

许凌志，符钦伟，陶卫，等 . 基于三维模型的单目车辆位姿估计［J］. 光学精密工程， 2021 ， 29 （ 6 ）： 1346 - 1355 . doi: 10.37188/OPE.20212906.1346 http://dx.doi.org/10.37188/OPE.20212906.1346

XU L ZH ， FU Q W ， TAO W ， et al . Monocular vehicle pose estimation based on 3D model ［J］. Opt. Precision Eng. ， 2021 ， 29 （ 6 ）： 1346 - 1355 . （in Chinese） . doi: 10.37188/OPE.20212906.1346 http://dx.doi.org/10.37188/OPE.20212906.1346

SATTLER T ， ZHOU Q J ， POLLEFEYS M ， et al . Understanding the limitations of CNN-based absolute camera pose regression ［C］. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. 1520，2019 ， Long Beach， CA， USA. IEEE ， 2019 ： 3297 - 3307 . doi: 10.1109/cvpr.2019.00342 http://dx.doi.org/10.1109/cvpr.2019.00342

屈也频，侯旺 . 基于误差传播理论的PnP问题姿态精度分析［J］. 光学精密工程， 2019 ， 27 （ 2 ）： 479 - 487 . doi: 10.3788/ope.20192702.0479 http://dx.doi.org/10.3788/ope.20192702.0479

QU Y P ， HOU W . Attitude accuracy analysis of PnP based on error propagation theory ［J］. Opt. Precision Eng. ， 2019 ， 27 （ 2 ）： 479 - 487 . （in Chinese） . doi: 10.3788/ope.20192702.0479 http://dx.doi.org/10.3788/ope.20192702.0479

PENG S D ， LIU Y ， HUANG Q X ， et al . PVNet： pixel-wise voting network for 6DoF pose estimation ［C］. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. 1520，2019 ， Long Beach， CA， USA. IEEE ， 2019 ： 4556 - 4565 . doi: 10.1109/cvpr.2019.00469 http://dx.doi.org/10.1109/cvpr.2019.00469

CHEN B ， CAO J W ， PARRA A ， et al . Satellite pose estimation with deep landmark regression and nonlinear pose refinement ［C］. 2019 IEEE/CVF International Conference on Computer Vision Workshop （ICCVW）. 2728，2019 ， Seoul， Korea （South）. IEEE ， 2019 ： 2816 - 2824 . doi: 10.1109/iccvw.2019.00343 http://dx.doi.org/10.1109/iccvw.2019.00343

TEKIN B ， SINHA S N ， FUA P . Real-time seamless single shot 6D object pose prediction ［C］. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. June 18 - 23 ， 2018 . Salt Lake City， UT， USA. IEEE ， 2018 ： 292 - 301 . doi: 10.1109/cvpr.2018.00038 http://dx.doi.org/10.1109/cvpr.2018.00038

BRACHMANN E ， KRULL A ， NOWOZIN S ， et al . DSAC-differentiable RANSAC for camera localization ［C］. 2017 IEEE Conference on Computer Vision and Pattern Recognition . 2126，2017 ， Honolulu， HI， USA . IEEE ， 2017 ： 2492 - 2500 . doi: 10.1109/cvpr.2017.267 http://dx.doi.org/10.1109/cvpr.2017.267

CAMPBELL D ， LIU L ， GOULD S . Solving the blind perspective-n-point problem end-to-end with robust differentiable geometric optimization ［C］. Computer Vision-ECCV 2020 ， 2020 ： 244 - 261 . doi: 10.1007/978-3-030-58536-5_15 http://dx.doi.org/10.1007/978-3-030-58536-5_15

YI K M ， TRULLS E ， ONO Y ， et al . Learning to find good correspondences ［C］. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . 1823，2018 ， Salt Lake City， UT， USA . IEEE ， 2018 ： 2666 - 2674 . doi: 10.1109/cvpr.2018.00282 http://dx.doi.org/10.1109/cvpr.2018.00282

SINKHORN R . Diagonal equivalence to matrices with prescribed row and column sums. II ［J］. Proceedings of the American Mathematical Society ， 1974 ， 45 （ 2 ）： 195 - 198 . doi: 10.1090/s0002-9939-1974-0357434-8 http://dx.doi.org/10.1090/s0002-9939-1974-0357434-8

LEPETIT V ， MORENO-NOGUER F ， FUA P . EPnP： an accurate O（n） solution to the PnP problem ［J］. International Journal of Computer Vision ， 2008 ， 81 （ 2 ）： 155 - 166 . doi: 10.1007/s11263-008-0152-6 http://dx.doi.org/10.1007/s11263-008-0152-6

CHEN B ， PARRA Á ， CAO J W ， et al . End-to-end learnable geometric vision by backpropagating PnP optimization ［C］. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. 1319，2020 ， Seattle， WA， USA. IEEE ， 2020 ： 8097 - 8106 . doi: 10.1109/cvpr42600.2020.00812 http://dx.doi.org/10.1109/cvpr42600.2020.00812

张刘港，熊芝，冯维，等 . 基于视觉与激光准直的激光跟踪姿态角测量方法［J］. 仪器仪表学报， 2020 ， 41 （ 8 ）： 30 - 36 . doi: 10.19650/j.cnki.cjsi.J2006544 http://dx.doi.org/10.19650/j.cnki.cjsi.J2006544

ZHANG L G ， XIONG ZH ， FENG W ， et al . Laser tracking attitude angle measurement method based on vision and laser collimation ［J］. Chinese Journal of Scientific Instrument ， 2020 ， 41 （ 8 ）： 30 - 36 . （in Chinese） . doi: 10.19650/j.cnki.cjsi.J2006544 http://dx.doi.org/10.19650/j.cnki.cjsi.J2006544

王天宇，董文博，王震宇 . 基于单目视觉和固定靶标的位姿测量系统［J］. 红外与激光工程， 2017 ， 46 （ 4 ）： 153 - 160 . doi: 10.3788/IRLA201746.0427003 http://dx.doi.org/10.3788/IRLA201746.0427003

WANG T Y ， DONG W B ， WANG ZH Y . Position and orientation measurement system based on monocular vision and fixed target ［J］. Infrared and Laser Engineering ， 2017 ， 46 （ 4 ）： 153 - 160 . （in Chinese） . doi: 10.3788/IRLA201746.0427003 http://dx.doi.org/10.3788/IRLA201746.0427003

许航，熊芝，张刘港等 . 基于加权最小二乘的激光跟踪姿态角测量方法［J/OL］. 红外与激光工程： 1 - 6 ［ 2022-01-26 ］. http：//kns.cnki.net/kcms/detail/12.1261.TN.20211206.2201.016.html http://kns.cnki.net/kcms/detail/12.1261.TN.20211206.2201.016.html .

XU H ， XIONG ZH ， ZHANG L G ， et al . Laser tracking attitude angle measurement method based on weighted least square ［J/OL］. Infrared and Laser Engineering ： 1 - 6 ［ 2022-01-26 ］. http：//kns.cnki.net/kcms/detail/12.1261.TN.20211206.2201. 016.html. http://kns.cnki.net/kcms/detail/12.1261.TN.20211206.2201.016.html. （in Chinese）

Views

582

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

High-precision attitude measurement with active aiming composite sensing for laser tracker

Large range automatic attitude measurement method for laser tracking measurement

Driving obstacles prediction network merged with spatial attention

Target plate calibration for vision-based airborne equipment mounted attitude boresight

Underwater image enhancement based on multi-branch residual attention network

Related Author

CHENG Zhi

DONG Dengfeng

LI Yang

QIU Qifan

WANG Jianguo

HUO Hanlin

HUANG Yunpeng

ZHOU Weihu

Related Institution

Institute of Microelectronics， Chinese Academy of Sciences

College of Automation Engineering， Nanjing University of Aeronautics and Astronautics

Electronic Information School, Wuhan University

College of Mechanical and Electrical Engineering, Nanjing University of Aeronautics and Astronautics

School of Electrical and Information Engineering， Anhui University of Technology， Maanshan

AI问答

Address：No.3888 Dong Nanhu Road, Changchun, Jilin, China Postal code：130033
Tel：0431-86176855 Email：gxjmgc@ciomp.ac.cn
Technical support is provided by Beijing Founder electronics co., LTD 吉ICP备11002662号-17 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰