A method for dense occlusion target recognition of service robots based on improved YOLOv7

CHEN Renxiang; QIU Tianran; YANG Lixia; YU Tenwei; JIA Fei; CHEN Cai

doi:10.37188/OPE.20243210.1595

您当前的位置：

首页 >

文章列表页 >

A method for dense occlusion target recognition of service robots based on improved YOLOv7

Information Sciences | 更新时间：2024-06-11

- A method for dense occlusion target recognition of service robots based on improved YOLOv7
- Optics and Precision Engineering Vol. 32, Issue 10, Pages: 1595-1605(2024)
- 作者机构：
  
  1.重庆交通大学交通工程应用机器人重庆市工程实验室，重庆 400074
  2.重庆科技学院工商管理学院，重庆 401331
  3.重庆智能机器人研究院，重庆 400714
- 作者简介：
- 基金信息：
- DOI：10.37188/OPE.20243210.1595
  CLC： TP391
- Published：25 May 2024，
  
  Received：14 November 2023，
  
  Revised：04 January 2024，
- 稿件说明：
移动端阅览
陈仁祥,邱天然,杨黎霞等.改进YOLOv7的服务机器人密集遮挡目标识别[J].光学精密工程,2024,32(10):1595-1605.

CHEN Renxiang,QIU Tianran,YANG Lixia,et al.A method for dense occlusion target recognition of service robots based on improved YOLOv7[J].Optics and Precision Engineering,2024,32(10):1595-1605.
陈仁祥,邱天然,杨黎霞等.改进YOLOv7的服务机器人密集遮挡目标识别[J].光学精密工程,2024,32(10):1595-1605. DOI： 10.37188/OPE.20243210.1595.

CHEN Renxiang,QIU Tianran,YANG Lixia,et al.A method for dense occlusion target recognition of service robots based on improved YOLOv7[J].Optics and Precision Engineering,2024,32(10):1595-1605. DOI： 10.37188/OPE.20243210.1595.

摘要

针对服务机器人视觉抓取时待识别目标存在密集遮挡导致识别效果差的问题，提出改进YOLOv7的服务机器人密集遮挡目标识别方法。首先，为改善密集遮挡目标特征信息丢失导致识别困难的问题，使用深度过参数化卷积构建深度过参数化高效聚合网络，利用不同卷积核对每个通道进行运算，增强网络感知能力，使网络关注目标未遮挡区域特征；其次，为抑制密集遮挡目标边界不易区分对识别造成的影响，将坐标注意力机制嵌入主干网络中，使网络获取目标位置信息并更好地关注特征图中的重要区域，增强网络特征提取能力；最后，使用Ghost网络进行轻量化改进，减少计算量并降低模型内存占用。在自建数据集与公共数据集分别对模型进行对比实验，实验结果表明，改进后模型mAP分别达到92.9%，87.8%。本文模型在降低内存占用的同时，识别精度和识别效率提升，整体性能更优。

Abstract

Aiming at the problem of poor recognition effect due to dense occlusion of the target to be recognized during visual grasping of service robots， we propose to improve the dense occlusion target recognition method for service robots with YOLOv7. First， in order to improve the problem of recognition difficulties caused by the loss of feature information of densely occluded targets， a deep over-parameterized convolution was used to construct a deep over-parameterized high-efficiency aggregation network， and different convolution kernels were used to operate on each channel to enhance the network sensing ability， so that the network focused on the features of the target's uncovered area； second， in order to suppress the influence caused by dense occlusions and indistinguishable target boundaries on recognition， the coordinate attention mechanism was embedded into the backbone network. This enabled the network to obtain target position information and paid more attention to the important areas in the feature map， thereby enhancing the capability of the network to extract features； finally， the Ghost network was used to improve the lightweighting， reduce the number of parameters of the network model and the number of floating-point operations to realize the lightweighting， reduce the memory occupation of the model， and increase the model operation efficiency. Comparison experiments were conducted on the model in the self-constructed dataset and the public dataset respectively， and the experimental results show that the improved model achieves a mAP of 92.9% on the self-constructed dataset and 87.8% on the public dataset， which is better than the original method and the other commonly used methods. In this paper， the model reduces the memory footprint while the recognition accuracy and recognition efficiency are improved， and the overall performance is better.

关键词

密集遮挡改进YOLOv7服务机器人目标识别

Keywords

dense occlusionimproved YOLOv7service robotstarget recognition

references

崔永成，田国会，周昭旭，等. 智能空间下面向动作序列生成的服务机器人指令解析方法［J］. 机器人， 2024， 46（1）： 1-15.

CUI Y C， TIAN G H， ZHOU Z X， et al. A service robot instruction parsing method for action sequence generation in intelligent space［J］. Robot， 2024， 46（1）： 1-15.（in Chinese）

陶永，刘海涛，王田苗，等. 我国服务机器人技术研究进展与产业化发展趋势［J］. 机械工程学报， 2022， 58（18）： 56-74. doi: 10.3901/jme.2022.18.056http://dx.doi.org/10.3901/jme.2022.18.056

TAO Y， LIU H T， WANG T M， et al. Research progress and industrialization development trend of Chinese service robot［J］. Journal of Mechanical Engineering， 2022， 58（18）： 56-74.（in Chinese）. doi: 10.3901/jme.2022.18.056http://dx.doi.org/10.3901/jme.2022.18.056

REN Y， ZHU C R， XIAO S P. Object detection based on fast/faster RCNN employing fully convolutional architectures［J］. Mathematical Problems in Engineering， 2018， 2018： 3598316. doi: 10.1155/2018/3598316http://dx.doi.org/10.1155/2018/3598316

HE K M， GKIOXARI G， DOLLÁR P， et al. Mask R-CNN［C］. 2017 IEEE International Conference on Computer Vision （ICCV）. Venice， Italy. IEEE， 2017： 2980-2988. doi: 10.1109/iccv.2017.322http://dx.doi.org/10.1109/iccv.2017.322

CARION N， MASSA F， SYNNAEVE G， et al. End-to-End Object Detection with Transformers［M］. Computer Vision-ECCV 2020. Cham： Springer International Publishing， 2020： 213-229. doi: 10.1007/978-3-030-58452-8_13http://dx.doi.org/10.1007/978-3-030-58452-8_13

YANG J， HE W Y， ZHANG T L， et al. Research on subway pedestrian detection algorithms based on SSD model［J］. IET Intelligent Transport Systems， 2020， 14（11）： 1491-1496. doi: 10.1049/iet-its.2019.0806http://dx.doi.org/10.1049/iet-its.2019.0806

REDMON J， DIVVALA S， GIRSHICK R， et al. You only look once： unified， real-time object detection［C］. 2016 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. Las Vegas， NV， USA. IEEE， 2016： 779-788. doi: 10.1109/cvpr.2016.91http://dx.doi.org/10.1109/cvpr.2016.91

杨坚，钱振，张燕军，等. 采用改进YOLOv4-tiny的复杂环境下番茄实时识别［J］. 农业工程学报， 2022， 38（9）： 215-221. doi: 10.11975/j.issn.1002-6819.2022.09.023http://dx.doi.org/10.11975/j.issn.1002-6819.2022.09.023

YANG J， QIAN Z， ZHANG Y J， et al. Real-time recognition of tomatoes in complex environments based on improved YOLOv4-tiny［J］. Transactions of the Chinese Society of Agricultural Engineering， 2022， 38（9）： 215-221.（in Chinese）. doi: 10.11975/j.issn.1002-6819.2022.09.023http://dx.doi.org/10.11975/j.issn.1002-6819.2022.09.023

滕臻，崔国华，高鹏，等. 基于机器视觉及深度学习的静脉药物调配机器人药瓶识别［J］. 机床与液压， 2022， 50（5）： 33-37. doi: 10.3969/j.issn.1001-3881.2022.05.007http://dx.doi.org/10.3969/j.issn.1001-3881.2022.05.007

TENG Z， CUI G H， GAO P， et al. Drug bottle recognition of intravenous drug dispensing robot based on machine vision and deep learning［J］. Machine Tool & Hydraulics， 2022， 50（5）： 33-37.（in Chinese）. doi: 10.3969/j.issn.1001-3881.2022.05.007http://dx.doi.org/10.3969/j.issn.1001-3881.2022.05.007

张伏，陈自均，鲍若飞，等. 基于改进型YOLOv4-LITE轻量级神经网络的密集圣女果识别［J］. 农业工程学报， 2021， 37（16）： 270-278. doi: 10.11975/j.issn.1002-6819.2021.16.033http://dx.doi.org/10.11975/j.issn.1002-6819.2021.16.033

ZHANG F， CHEN Z J， BAO R F， et al. Recognition of dense cherry tomatoes based on improved YOLOv4-LITE lightweight neural network［J］. Transactions of the Chinese Society of Agricultural Engineering， 2021， 37（16）： 270-278.（in Chinese）. doi: 10.11975/j.issn.1002-6819.2021.16.033http://dx.doi.org/10.11975/j.issn.1002-6819.2021.16.033

高毅，何淼. 密集遮挡条件下的步态识别［J］. 光学精密工程， 2023， 31（2）： 263-276. doi: 10.37188/ope.20233102.0263http://dx.doi.org/10.37188/ope.20233102.0263

GAO Y， HE M. Gait recognition algorithm in dense occlusion scene［J］. Opt. Precision Eng.， 2023， 31（2）： 263-276.（in Chinese）. doi: 10.37188/ope.20233102.0263http://dx.doi.org/10.37188/ope.20233102.0263

LI Y D， GUO K， LU Y G， et al. Cropping and attention based approach for masked face recognition［J］. Applied Intelligence， 2021， 51（5）： 3012-3025. doi: 10.1007/s10489-020-02100-9http://dx.doi.org/10.1007/s10489-020-02100-9

CAO J M， LI Y Y， SUN M C， et al. DO-conv： depthwise over-parameterized convolutional layer［J］. IEEE Transactions on Image Processing： a Publication of the IEEE Signal Processing Society， 2022， 31： 3726-3736. doi: 10.1109/tip.2022.3175432http://dx.doi.org/10.1109/tip.2022.3175432

HOU Q B， ZHOU D Q， FENG J S. Coordinate attention for efficient mobile network design［C］. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Nashville， TN， USA. IEEE， 2021： 13708-13717. doi: 10.1109/cvpr46437.2021.01350http://dx.doi.org/10.1109/cvpr46437.2021.01350

HAN K， WANG Y H， TIAN Q， et al. GhostNet： more features from cheap operations［C］. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Seattle， WA， USA. IEEE， 2020： 1577-1586. doi: 10.1109/cvpr42600.2020.00165http://dx.doi.org/10.1109/cvpr42600.2020.00165

李大湘，苏仲恒，刘颖. 基于改进YOLOv4的道路交通标志识别［J］. 光学精密工程， 2023， 31（9）： 1366-1378. doi: 10.37188/OPE.20233109.1366http://dx.doi.org/10.37188/OPE.20233109.1366

LI D X， SU Z H， LIU Y. Road traffic sign recognition algorithm based on improved YOLOv4［J］. Opt. Precision Eng.， 2023， 31（9）： 1366-1378.（in Chinese）. doi: 10.37188/OPE.20233109.1366http://dx.doi.org/10.37188/OPE.20233109.1366

司永胜，孔德浩，王克俭，等. 基于CRV-YOLO的苹果中心花和边花识别方法［J］. 农业机械学报， 2024， 55（2）： 278-286. doi: 10.6041/j.issn.1000-1298.2024.02.027http://dx.doi.org/10.6041/j.issn.1000-1298.2024.02.027

SI Y S， KONG D H， WANG K J， et al. Recognition of apple king flower and side flower based on CRV-YOLO［J］. Transactions of the Chinese Society for Agricultural Machinery， 2024， 55（2）： 278-286.（in Chinese）. doi: 10.6041/j.issn.1000-1298.2024.02.027http://dx.doi.org/10.6041/j.issn.1000-1298.2024.02.027

石杰，周亚丽，张奇志. 基于改进Mask RCNN和Kinect的服务机器人物品识别系统［J］. 仪器仪表学报， 2019， 40（4）： 216-228.

SHI J， ZHOU Y L， ZHANG Q Z. Service robot item recognition system based on improved Mask RCNN and Kinect［J］. Chinese Journal of Scientific Instrument， 2019， 40（4）： 216-228.（in Chinese）

刘颖，姜威，李冠典，等. 基于标签嵌入方法的纺织品瑕疵识别网络［J］. 光学精密工程， 2023， 31（10）： 1563-1579. doi: 10.37188/OPE.20233110.1563http://dx.doi.org/10.37188/OPE.20233110.1563

LIU Y， JIANG W， LI G D， et al. Textile defect recognition network based on label embedding［J］. Opt. Precision Eng.， 2023， 31（10）： 1563-1579.（in Chinese）. doi: 10.37188/OPE.20233110.1563http://dx.doi.org/10.37188/OPE.20233110.1563

SELVARAJU R R， COGSWELL M， DAS A， et al. Grad-CAM： visual explanations from deep networks via gradient-based localization［J］. International Journal of Computer Vision， 2020， 128（2）： 336-359. doi: 10.1007/s11263-019-01228-7http://dx.doi.org/10.1007/s11263-019-01228-7

YU Z， HUANG H， CHEN W， et al. YOLO-FaceV2： a scale and occlusion aware face detector［J］. arXiv preprint arXiv：2208. 02019， 2022： 1-18.

CAI Z A， ZHANG J Z， REN D X， et al. MessyTable： Instance Association in Multiple Camera Views［M］. Computer Vision-ECCV 2020. Cham： Springer International Publishing， 2020： 1-16. doi: 10.1007/978-3-030-58621-8_1http://dx.doi.org/10.1007/978-3-030-58621-8_1

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Saliency maps for point cloud recognition models

Gait recognition algorithm in dense occlusion scene

Super-resolution reconstruction of micro-scanning images

Path planning strategy of amphibious spherical robot

SAR ATR based on disentangled representation learning generative adversarial networks and support vector machine

Related Author

LIANG Ao

ZHANG Hao

HUA Haiyang

GAO Yi

HE Miao

Hao-guang ZHAO

Han-shi QU

Xin WANG

Related Institution

University of Chinese Academy of Sciences

Key Laboratory of Opto-Electronic Information Processing，Chinese Academy of Sciences

Shenyang Institute of Automation， Chinese Academy of Sciences

Institutes for Robotics and Intelligent Manufacturing， Chinese Academy of Sciences

Key Laboratory of Impression Evidence Examination and Identification Technology （Criminal Investigation Police University of China）， Ministry of Public Security， People's Republic of China

Address：No.3888 Dong Nanhu Road, Changchun, Jilin, China Postal code：130033
Tel：0431-86176855 Email：gxjmgc@ciomp.ac.cn
Technical support is provided by Beijing Founder electronics co., LTD 吉ICP备11002662号-17 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰