Robot vision system for keyframe global map establishment and robot localization based on graphic content matching

Tian-yang CAO; Hao-yuan CAI; Dong-ming FANG; Chang LIU

doi:10.3788/OPE.20172508.2221

您当前的位置：

首页 >

文章列表页 >

Robot vision system for keyframe global map establishment and robot localization based on graphic content matching

Information Sceiences | 更新时间：2020-07-07

- Robot vision system for keyframe global map establishment and robot localization based on graphic content matching
- Optics and Precision Engineering Vol. 25, Issue 8, Pages: 2221-2232(2017)
- 作者机构：
  
  1.中国科学院电子学研究所传感技术联合国家重点实验室, 北京 100190
  2.中国科学院大学, 北京 100190
- 作者简介：
- 基金信息：
- DOI：10.3788/OPE.20172508.2221
  CLC： TP242.6
- Received：03 January 2017，
  
  Accepted：24 March 2017，
  
  Published：25 August 2017
- 稿件说明：
移动端阅览
Tian-yang CAO, Hao-yuan CAI, Dong-ming FANG, et al. Robot vision system for keyframe global map establishment and robot localization based on graphic content matching[J]. Optics and precision engineering, 2017, 25(8): 2221-2232.
DOI：

Tian-yang CAO, Hao-yuan CAI, Dong-ming FANG, et al. Robot vision system for keyframe global map establishment and robot localization based on graphic content matching[J]. Optics and precision engineering, 2017, 25(8): 2221-2232. DOI： 10.3788/OPE.20172508.2221.

摘要

为了解决机器人室内定位时的绑架问题和相似物体的干扰，设计了一种具有图像内容匹配功能的视觉系统，从而使机器人能有效提取关键帧序列构建室内全局地图并实现自主定位。考虑影响图像内容匹配的主要干扰是机器人视角和位移造成的图像畸变，本文通过对室内物体的图像畸变建模与特征分析，设计了一种图像内容匹配方法。该方法以图像重叠区提取、基于子块分解匹配的重叠区重建两部分为核心，可将待匹配的两帧图像畸变调整为一致后再进行内容匹配并准确解算它们的相似度。其能有效利用各个房间内不同的景物和布局信息来消除相似物体的影响，从机器人学习环境时采集的视频中提取空间间距大且重叠相连的关键帧序列建立整栋建筑内部的全局导航地图。机器人工作时，实时视觉的图像内容与地图关键帧序列匹配，提取出与每个时刻视觉图像最相似的关键帧对机器人实施定位。在由3个房间和2条走廊组成的实验区进行了实验测试，结果表明：机器人可有效消除相似物体的干扰，绑架发生时仍可通过与全局地图匹配实施准确自主定位，匹配准确率≥93%，定位精度误差（RMSE） < 0.5 m。

Abstract

To solve kidnapping problem and similar object interference at the time of indoor localization of robots

a visual system with graphic contents matching function was designed to make robots extract constructed indoor global map of key frame sequence effectively and to realize self-localization.Since the main interferences influencing graphic content matching was image distortion caused by visual angels of robots and displacement

a graphic content matching method was designed by image distortion modeling and feature analysis of indoor objects. With the method

both parts of image overlapping area extraction and overlapping area reconstruction based on sub-block decomposition and matching were taken as a core.The content matching could be implemented after two image distortions waiting for matching were adjusted to be conformity and their similarity could be calculated accurately. The method could make use of different sceneries and arrangement information in various rooms effectively to eliminate the influence of similar objects and to extract the overlapped and connected key frame sequence with great space distance from collected video when the robots learn environment to construct global navigation map in the whole building. When robots work

graphic contents of real-time vision and key frame sequence of map was matching

and key frame that is the most similar with visual image in all moments was extracted to implement localization on robots. Experimental test was performed in an experimental area composed of 3 rooms and 2 corridors. Experimental result indicates that robots can eliminate interference of similar objects effectively and can implement accurate self-localization by matching with global map when kidnapping happens. The matching accuracy is ≥ 93% and localization accuracy error (RMSE) is < 0.5m.

关键词

Keywords

references

BRAND C, SCHUSTER M J, HIRSCHM? LLER H, et al.. Submap matching for stereo-vision based indoor/outdoor SLAM[C]. Proceedings of 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), IEEE, 2015:5670-5677. http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=7354182

温丰, 柴晓杰, 朱智平, 等.基于单目视觉的SLAM算法研究[J].系统科学与数学, 2010, 30(6):827-839.

WEN F, CHAI X J, ZHU ZH P, et al.. A visual slam algorithm based on monocular vision[J]. Journal of Systems Science and Mathematical Sciences, 2010, 30(6):827-839. (in Chinese)

杜航原, 郝燕玲, 赵玉新, 等.用概率假设密度滤波实现同步定位与地图创建[J].光学精密工程, 2011, 19(12):3064-3073.

DU H Y, HAO Y L, ZHAO Y X, et al.. Implementation of SLAM by probability hypothesis density filter[J]. Opt. Precision Eng., 2011, 19(12):3064-3073. (in Chinese)

付梦印, 吕宪伟, 刘彤, 等.基于RGB-D数据的实时SLAM算法[J].机器人, 2015, 37(6):683-692.

FU M Y, LV X W, LIU T, et al.. Real-time SLAM algorithm based on RGB-D data[J]. Robot, 2015, 37(6):683-692. (in Chinese)

GARCIA-FIDALGO E, ORTIZ A. Vision-based topological mapping and localization by means of local invariant features and map refinement[J]. Robotica, 2015, 33(7):1446-1470.

MUR-ARTAL R, MONTIEL J M M, TARDOS J D. ORB-SLAM:a versatile and accurate monocular SLAM system[J]. IEEE Transactions on Robotics, 2015, 31(5):1147-1163.

TATENO K, TOMBARI F, NAVAB N. Real-time and scalable incremental segmentation on dense SLAM[C]. Proceedings of 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), IEEE, 2015:4465-4472. http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=7354011

KIM J H, CADENA C, REID I. Direct semi-dense SLAM for rolling shutter cameras[C]. Proceedings of 2016 IEEE International Conference on Robotics and Automation (ICRA), IEEE, 2016:1308-1315. http://ieeexplore.ieee.org/document/7487263/

BOIKOS K, BOUGANIS C S. Semi-dense SLAM on an FPGA SoC[C]. Proceedings of the 26th International Conference on Field Programmable Logic and Applications (FPL), IEEE, 2015:4465-4472. http://ieeexplore.ieee.org/document/7577365/

CHEN P J, GU Z P, ZHANG G D, et al.. Ceiling vision localization with feature pairs for home service robots[C]. Proceedings of 2014 IEEE International Conference on Robotics and Biomimetics, IEEE, 2015:2274-2279. http://ieeexplore.ieee.org/document/7090676/

JO S, CHOI H, KIM E. Ceiling vision based SLAM approach using sensor fusion of sonar sensor and monocular camera[C]. Proceedings of the 12th International Conference on Control, Automation and Systems (ICCAS), IEEE, 2012:1461-1464. http://www.dbpia.co.kr/Journal/ArticleDetail/NODE02171568

GAO X, ZHANG T. Robust RGB-D simultaneous localization and mapping using planar point features[J]. Robotics and Autonomous Systems, 2015, 72:1-14.

刘志斌, 吴显亮, 徐文立, 等.视觉SLAM中基于误匹配风险预测的特征选择[J].机器人, 2010, 32(5):635-641.

LIU ZH B, WU X L, XU W L, et al.. Erroneous matching risk prediction based feature selection for Visual SLAM[J]. Robot, 2015, 32(5):635-641. (in Chinese)

ZHOU H Z, ZOU D P, PEI L, et al.. StructSLAM:visual SLAM with building structure lines[J]. IEEE Transactions on Vehicular Technology, 2015, 64(4):1364-1375.

GAO X, ZHANG T. Unsupervised learning to detect loops using deep neural networks for visual SLAM system[J]. Autonomous Robots, 2017, 41(1):1-18.

LEE S, LEE S, BAEK S. Vision-based kidnap recovery with SLAM for home cleaning robots[J]. Journal of Intelligent & Robotic Systems, 2012, 67(1):7-24.

YI C, CHOI B U. Detection and recovery for kidnapped-robot problem using measurement entropy[C]. Communications in Computer and Information Science, Springer, 2011, 261:293-299. http://www.springerlink.com/index/h057332564r6l11t.pdf

STEWART R L, ZHANG H. Image similarity from feature-flow for keyframe detection in appearance-based SLAM[C]. Proceedings of 2011 IEEE International Conference on Robotics and Biomimetics (ROBIO), IEEE, 2011:305-312. http://ieeexplore.ieee.org/xpls/icp.jsp?arnumber=6181303

KIM J, KWEON I S. Robust feature matching for loop closing and localization[C]. Proceedings of 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems, IEEE, 2007:3905-3910. http://ieeexplore.ieee.org/xpls/icp.jsp?arnumber=4399436

KIM D Y, CHOI H, LEE H, et al.. A new cvSLAM exploiting a partially known landmark association[J]. Advanced Robotics, 2013, 27(14):1073-1086.

张宝龙, 李丹, 郭艳艳, 等.基于OV7962的车载全景摄像头设计[J].液晶与显示, 2015, 30(4):634-640.

ZHANG B L, LI D, GUO Y Y, et al.. Panoramic camera design for automotive applications based on OV7962[J]. Chinese Journal of Liquid Crystals and Displays, 2015, 30(4):634-640. (in Chinese)

ZHUO J J, SUN L Q, SHI J J, et al.. Research on a type of camera calibration method based on high precision detection of X corners[C]. Proceedings of the 8th International Symposium on Computational Intelligence and Design (ISCID), IEEE, 2015:193-196. http://ieeexplore.ieee.org/document/7469112/

STROBL K H, LINGENAUBER M. Stepwise calibration of focused plenoptic cameras[J]. Computer Vision and Image Understanding, 2016, 145:140-147.

陈杰春, 丁振良, 袁峰.三维重构不确定度的两阶估计方法[J].光学精密工程, 2008, 16(6):1110-1116.

CHEN J C, DING ZH L, YUAN F. Two-stage uncertainty evaluation of 3D reconstruction[J]. Opt. Precision Eng., 2008, 16(6):1110-1116. (in Chinese)

翟优, 曾峦, 熊伟.不同局部邻域划分SURF描述符的性能分析[J].光学精密工程, 2013, 21(9):2395-2404.

ZHAI Y, ZENG L, XIONG W. Performance analysis of SURF descriptor with different local region partition[J]. Opt. Precision Eng., 2013, 21(9):2395-2404. (in Chinese)

刘洋, 韩广良, 史春蕾.基于SIFT算法的多表情人脸识别[J].液晶与显示, 2016, 31(12):1156-1160.

LIU Y, HAN G L, SHI CH L. Recognition of expression-variant faces based on SIFT method[J]. Chinese Journal of Liquid Crystals and Displays, 2016, 31(12):1156-1160. (in Chinese)

LI Z H, SHAO C Y, LIU Y M. Motion estimation based on axis affine model[C]. Proceedings of 2010 International Conference on Mechatronics and Automation (ICMA), IEEE, 2010:572-576. https://www.infona.pl/resource/bwmeta1.element.ieee-art-000005588407

CABALLERO D, ANTEQUERA T, CARO A, et al.. Data mining on MRI-computational texture features to predict sensory characteristics in ham[J]. Food and Bioprocess Technology, 2016, 9(4):699-708.

CHO J S, LEE H J, PARK J H, et al.. Image analysis to evaluate the browning degree of banana (Musa spp.) peel[J]. Food Chemistry, 2016, 194:1028-1033.

MOTEKI A, YAMAGUCHI N, KARASUDANI A, et al.. Fast and accurate relocalization for keyframe-based SLAM using geometric model selection[C]. Proceedings of 2016 IEEE Virtual Reality, IEEE, 2016. http://ieeexplore.ieee.org/document/7504740/

Views

797

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Distortion analysis and reconstruction of OCT image in sector scan mode

Correction of distortion in microscopic imaging with resonant scanning

Robot vision system for 3D reconstruction in low texture environment

Design of AFM system with high speed and large scanning range

Distortion correction for images in planar metrology

Related Author

Li-cheng FAN

Zuo-pan ZHAI

Lian-sheng LIU

Fan YANG

XIONG Da-xi

LIU Yun

LIANG Yong

LOU Yan-yang

Related Institution

School of Mechanical and Electrical Engineering， Soochow University

Jiangsu Key Laboratory of Medical Optics, Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences

Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences

Institute of Optical Communication & Optoelectronics, Beijing University of Posts & Telecommunications

Beijing Key Laboratory of Optoelectronic Measurement Technology, Beijing Information Science & Technology University

AI问答

Address：No.3888 Dong Nanhu Road, Changchun, Jilin, China Postal code：130033
Tel：0431-86176855 Email：gxjmgc@ciomp.ac.cn
Technical support is provided by Beijing Founder electronics co., LTD 吉ICP备11002662号-17 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰