Joint self-attention and branch sampling for object detection on drone imagery

ZHANG Yunzuo; WU Cunyu; LIU Yameng; ZHANG Tian; ZHENG Yuxin

doi:10.37188/OPE.20233118.2723

您当前的位置：

首页 >

文章列表页 >

Joint self-attention and branch sampling for object detection on drone imagery

Information Sciences | 更新时间：2023-09-25

- Joint self-attention and branch sampling for object detection on drone imagery
- Optics and Precision Engineering Vol. 31, Issue 18, Pages: 2723-2735(2023)
- 作者机构：
  
  1.石家庄铁道大学信息科学与技术学院，河北石家庄 050043
  2.河北省电磁环境效应与信息处理重点实验室，河北石家庄 050043
- 作者简介：
- 基金信息：
- DOI：10.37188/OPE.20233118.2723
  CLC：
扫描看全文
ZHANG Yunzuo, WU Cunyu, LIU Yameng, et al. Joint self-attention and branch sampling for object detection on drone imagery. [J]. Optics and Precision Engineering 31(18):2723-2735(2023)
DOI：

ZHANG Yunzuo, WU Cunyu, LIU Yameng, et al. Joint self-attention and branch sampling for object detection on drone imagery. [J]. Optics and Precision Engineering 31(18):2723-2735(2023) DOI： 10.37188/OPE.20233118.2723.

摘要

无人机图像目标检测在诸多领域被广泛应用，但受制于图像背景复杂、目标密集，目标尺度变化剧烈，现有的无人机图像目标检测算法检测效果不够精准。为解决此类问题，提出了一种联合自注意力和分支采样的无人机图像目标检测方法。首先，设计了自注意力和卷积相融合的嵌套残差结构以实现全局信息和局部信息的有效结合，让模型聚焦于待测目标，从而淡化复杂背景的影响。其次，设计了一种基于分支采样的特征融合模块以弥补目标信息丢失。最后，引入浅层细粒度特征图，新增了针对微小目标的改进检测头以缓解尺度剧烈变化，并基于此提出一种特征增强模块，用于捕获更多具有鉴别性的小目标特征。经实验验证，本文所提算法在多种场景中性能良好。其中s模型在VisDrone2019数据集上的mAP,50,和mAP分别达到59.3%和37.1%，相较于基线模型增长了5.6%和5.4%，在UAVDT数据集上的mAP,50,和mAP分别达到44.1%和24.9%，相较于基线模型提高了5.8%和3.2%。

Abstract

Object detection on drone imagery is widely used in many fields. However， due to the complexity of the image background， the dense small objects and the dramatic scale changes， the existing object detection on drone imagery methods are not accurate enough. In order to solve this problem， we propose an accurate object detection method for drone imagery joint self attention and branch sampling. Firstly， a nested residual structure integrating self attention and convolution is designed to achieve the effective combination of global and local information， which makes the model to focus on the object area and ignore invalid features. Secondly， we design a feature fusion module based on branch sampling to mitigate the loss of object information. Finally， an improved detector for small objects is added to alleviate the problem of sharp scale changes. Furthermore， we propose a feature enhancement module to obtain more discriminative small object features. The experimental results show that the proposed algorithm performs well in various scenarios. Specifically， the mAP,50, and mAP of the s model on the VisDrone2019 reached 59.3% and 37.1% respectively， an increase of 5.6% and 5.4% compared with the baseline. The mAP,50 ,and mAP on the UAVDT reached 44.1% and 24.9% respectively， an increase of 5.8% and 3.2% compared with the baseline.

关键词

无人机图像自注意力分支采样多尺度特征融合

Keywords

UAV imageself attentionbranch samplingmulti-scalefeature fusion

references

曲优，李文辉.基于锚框变换的单阶段旋转目标检测方法［J］.吉林大学学报（工学版），2022，52（01）：162-173.

QU Y， LI W H. Single-stage rotated object detection network based on anchor transformation［J］. Journal of Jilin University （Engineering and Technology Edition）， 2022， 52（1）： 162-173.（in Chinese）

李超，黄新宇，王凯. 基于特征融合和自学习锚框的高分辨率图像小目标检测算法［J］. 电子学报， 2022， 50（7）：1684-1695. doi: 10.12263/DZXB.20200917http://dx.doi.org/10.12263/DZXB.20200917

LI C， HUANG X Y， WANG K. Small object detection of high-resolution images based on feature fusion and learnable anchor［J］. Acta Electronica Sinica， 2022， 50（7）： 1684-1695.（in Chinese）. doi: 10.12263/DZXB.20200917http://dx.doi.org/10.12263/DZXB.20200917

刘芳，韩笑. 基于多尺度深度学习的自适应航拍目标检测［J］. 航空学报， 2022， 43（5）：463-474. doi: 10.7527/S1000-6893.2021.25270http://dx.doi.org/10.7527/S1000-6893.2021.25270

LIU F， HAN X. Adaptive aerial object detection based on multi-scale deep learning［J］. Acta Aeronautica et Astronautica Sinica， 2022， 43（5）：463-474.（in Chinese）. doi: 10.7527/S1000-6893.2021.25270http://dx.doi.org/10.7527/S1000-6893.2021.25270

DALAL N， TRIGGS B. Histograms of Oriented Gradients for Human Detection［C］. 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition （CVPR'05）. 20-25， 2005， San Diego， CA， USA. IEEE， 2005： 886-893. doi: 10.1109/cvpr.2005.4http://dx.doi.org/10.1109/cvpr.2005.4

LIENHART R， MAYDT J. An Extended Set of Haar-Like Features for Rapid Object Detection［C］. Proceedings of International Conference on Image Processing. 22-25， 2002， Rochester， NY， USA. IEEE， 2002： I.

VIOLA P， JONES M. Rapid object detection using a boosted cascade of simple features［C］. Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR. 8-14， 2001， Kauai， HI， USA. IEEE， 2003： I.

林浩，李雷孝，王慧. 支持向量机在智能交通系统中的研究应用综述［J］. 计算机科学与探索， 2020， 14（6）901-917.

XU D G， WANG L， LI F. Review of typical object detection algorithms for deep learning［J］. Computer Engineering and Applications， 2021， 57（8）： 10-25.（in Chinese）

张财广，熊博莅，匡纲要. 光学卫星遥感图像舰船目标检测综述［J］. 电波科学学报， 2020， 35（5）：637-647. doi: 10.13443/j.cjors.2020040603http://dx.doi.org/10.13443/j.cjors.2020040603

ZHANG C G， XIONG B L， KUANG G Y. A survey of ship detection in optical satellite remote sensing images［J］. Chinese Journal of Radio Science， 2020， 35（5）：637-647.（in Chinese）. doi: 10.13443/j.cjors.2020040603http://dx.doi.org/10.13443/j.cjors.2020040603

YANG F， FAN H， CHU P， et al. Clustered object detection in aerial images［C］. 2019 IEEE/CVF International Conference on Computer Vision （ICCV）. 27-2， 2019， Seoul， Korea （South）. IEEE， 2020： 8310-8319. doi: 10.1109/iccv.2019.00840http://dx.doi.org/10.1109/iccv.2019.00840

YU W P， YANG T， CHEN C. Towards resolving the challenge of long-tail distribution in uav images for object detection［C］. 2021 IEEE Winter Conference on Applications of Computer Vision （WACV）. 3-8， 2021， Waikoloa， HI， USA. IEEE， 2021： 3257-3266. doi: 10.1109/wacv48630.2021.00330http://dx.doi.org/10.1109/wacv48630.2021.00330

LIU Z M， GAO G Y， SUN L， et al. HRDNet： high-resolution detection network for small objects［C］. 2021 IEEE International Conference on Multimedia and Expo （ICME）. 5-9， 2021， Shenzhen， China. IEEE， 2021： 1-6. doi: 10.1109/icme51207.2021.9428241http://dx.doi.org/10.1109/icme51207.2021.9428241

WU Z Y， SURESH K， NARAYANAN P， et al. Delving into robust object detection from unmanned aerial vehicles： a deep nuisance disentanglement approach［C］. 2019 IEEE/CVF International Conference on Computer Vision （ICCV）. 27- 2， 2019， Seoul， Korea （South）. IEEE， 2020： 1201-1210. doi: 10.1109/iccv.2019.00129http://dx.doi.org/10.1109/iccv.2019.00129

YOUSSEF Y， ELSHENAWY M. Automatic vehicle counting and tracking in aerial video feeds using cascade region-based convolutional neural networks and feature pyramid networks［J］. Transportation Research Record： Journal of the Transportation Research Board， 2021， 2675（8）： 304-317. doi: 10.1177/0361198121997833http://dx.doi.org/10.1177/0361198121997833

LI J N， LIANG X D， WEI Y C， et al. Perceptual Generative Adversarial Networks for Small Object Detection［C］. 2017 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. 21-26， 2017， Honolulu， HI， USA. IEEE， 2017： 1951-1959. doi: 10.1109/cvpr.2017.211http://dx.doi.org/10.1109/cvpr.2017.211

TANG Z Y， LIU X， YANG B J. PENet： object detection using points estimation in high definition aerial images［C］. 2020 19th IEEE International Conference on Machine Learning and Applications （ICMLA）. 14-17， 2020， Miami， FL， USA. IEEE， 2021： 392-398. doi: 10.1109/icmla51294.2020.00069http://dx.doi.org/10.1109/icmla51294.2020.00069

LAMINE MEKHALFI M， BEJIGA M B， SORESINA D， et al. Capsule networks for object detection in UAV imagery［J］. Remote Sensing， 2019， 11（14）： 1694. doi: 10.3390/rs11141694http://dx.doi.org/10.3390/rs11141694

CHEN C Y， GONG W G， CHEN Y L， et al. Object detection in remote sensing images based on a scene-contextual feature pyramid network［J］. Remote Sensing， 2019， 11（3）： 339. doi: 10.3390/rs11030339http://dx.doi.org/10.3390/rs11030339

GIRSHICK R. Fast R-CNN［C］. 2015 IEEE International Conference on Computer Vision （ICCV）. 7-13， 2015， Santiago， Chile. IEEE， 2016： 1440-1448. doi: 10.1109/iccv.2015.169http://dx.doi.org/10.1109/iccv.2015.169

REN S Q， HE K M， GIRSHICK R， et al. Faster R-CNN： towards real-time object detection with region proposal networks［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2017， 39（6）： 1137-1149. doi: 10.1109/tpami.2016.2577031http://dx.doi.org/10.1109/tpami.2016.2577031

BOCHKOVSKIY A， WANG C Y， LIAO H Y M. YOLOv4： Optimal Speed and Accuracy of Object Detection［EB/OL］. 2020： arXiv： 2004.10934.https：//arxiv.org/abs/2004.10934https://arxiv.org/abs/2004.10934

WANG C Y， BOCHKOVSKIY A， LIAO H Y M. YOLOv7： Trainable Bag-of-Freebies Sets New State-Of-The-Art for Real-Time Object Detectors［EB/OL］. 2022： arXiv： 2207.02696.https：//arxiv.org/abs/2207.02696https://arxiv.org/abs/2207.02696. doi: 10.1109/cvpr52729.2023.00721http://dx.doi.org/10.1109/cvpr52729.2023.00721

DOSOVITSKIY A， BEYER L， KOLESNIKOV A， et al. An Image is Worth 16x16 Words： Transformers for Image Recognition at Scale［EB/OL］. 2020： arXiv： 2010.11929.https：//arxiv.org/abs/2010.11929https://arxiv.org/abs/2010.11929

SUN Z Q， CAO S C， YANG Y M， et al. Rethinking transformer-based set prediction for object detection［C］. 2021 IEEE/CVF International Conference on Computer Vision （ICCV）. 10-17， 2021， Montreal， QC， Canada. IEEE， 2022： 3591-3600. doi: 10.1109/iccv48922.2021.00359http://dx.doi.org/10.1109/iccv48922.2021.00359

DU D W， ZHU P F， WEN L Y， et al. VisDrone-DET2019： the vision meets drone object detection in image challenge results［C］. 2019 IEEE/CVF International Conference on Computer Vision Workshop （ICCVW）. 27-28， 2019， Seoul， Korea （South）. IEEE， 2020： 213-226. doi: 10.1109/iccvw48693.2019http://dx.doi.org/10.1109/iccvw48693.2019

DU D W， QI Y K， YU H Y， et al. The Unmanned Aerial Vehicle Benchmark： Object Detection and Tracking［M］. Computer Vision - ECCV 2018. Cham： Springer International Publishing， 2018： 375-391. doi: 10.1007/978-3-030-01249-6_23http://dx.doi.org/10.1007/978-3-030-01249-6_23

CHALAVADI V， JERIPOTHULA P， DATLA R， et al. mSODANet： a network for multi-scale object detection in aerial images using hierarchical dilated convolutions［J］. Pattern Recognition， 2022， 126： 108548. doi: 10.1016/j.patcog.2022.108548http://dx.doi.org/10.1016/j.patcog.2022.108548

LI C L， YANG T， ZHU S J， et al. Density Map Guided Object Detection in Aerial Images［C］. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops （CVPRW）. 14-19， 2020， Seattle， WA， USA. IEEE， 2020： 737-746. doi: 10.1109/cvprw50498.2020.00103http://dx.doi.org/10.1109/cvprw50498.2020.00103

DENG S T， LI S， XIE K， et al. A global-local self-adaptive network for drone-view object detection［J］. IEEE Transactions on Image Processing， 2021， 30： 1556-1569. doi: 10.1109/tip.2020.3045636http://dx.doi.org/10.1109/tip.2020.3045636

DUAN C Z， WEI Z W， ZHANG C， et al. Coarse-grained density map guided object detection in aerial images［C］. 2021 IEEE/CVF International Conference on Computer Vision Workshops （ICCVW）. 11-17， 2021， Montreal， BC， Canada. IEEE， 2021： 2789-2798. doi: 10.1109/iccvw54120.2021.00313http://dx.doi.org/10.1109/iccvw54120.2021.00313

LENG J X， MO M， ZHOU Y H， et al. Pareto refocusing for drone-view object detection［C］. IEEE Transactions on Circuits and Systems for Video Technology. 28， 2022， IEEE， 2022： 1320-1334. doi: 10.1109/tcsvt.2022.3210207http://dx.doi.org/10.1109/tcsvt.2022.3210207

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Multidimensional attention mechanism and selective feature fusion for image super-resolution reconstruction

Multi-scale dense feature fusion network for image super-resolution

Cross-scale infrared pedestrian detection based on dynamic feature optimization mechanism

Fusion of fractal geometric features Resnet remote sensing image building segmentation

Lightweight small object detection network with attention mechanism

Related Author

No data

Related Institution

Yunnan Police Unmanned System Innovation Research Institute， Yunnan Police Officer Academy

Faculty of Information Engineering and Automation， Kunming University of Science and Technology

School of Computer Science and Technology， China University of Mining and Technology

School of Information and Control Engineering， China University of Mining and Technology

College of Electrical and Control Engineering， Xi'an University of Science and Technology

Address：No.3888 Dong Nanhu Road, Changchun, Jilin, China Postal code：130033
Tel：0431-86176855 Email：gxjmgc@ciomp.ac.cn
Technical support is provided by Beijing Founder electronics co., LTD 吉ICP备11002662号-17 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰