1.宁夏大学 信息工程学院,宁夏 银川 750021
扫 描 看 全 文
LIU Libo, XI Siyu, DENG Zhen. Weather recognition combining improved ConvNeXt models with knowledge distillation. [J]. Optics and Precision Engineering 31(14):2123-2134(2023)
LIU Libo, XI Siyu, DENG Zhen. Weather recognition combining improved ConvNeXt models with knowledge distillation. [J]. Optics and Precision Engineering 31(14):2123-2134(2023) DOI: 10.37188/OPE.20233114.2123.
为提升复杂交通场景下天气识别准确率的同时实现网络轻量化,提出了一种结合改进ConvNeXt网络与知识蒸馏的天气识别方法。首先,在ConvNeXt网络的每组Block特征提取块后加入SimAm注意力机制,构建ConvNeXt_F网络,利用SimAm注意力机制对Block块提取的深层特征进行鉴权并校正权重,有效强化对天气判别性特征的捕获能力;其次,在网络训练过程中将Equalized Focal Loss(EFL)与Mutual-Channel Loss(MCL)采用平均占比的方式进行累加作为总损失函数,一方面利用EFL消除数据不均衡造成的影响,另一方面利用MCL减小同类天气下局部细节特征差异;最后,采用知识蒸馏技术将天气分类知识从ConvNeXt_F网络迁移到轻量级MobileNetV3网络,虽然精度略微损失但网络参数量大幅减少。实验结果表明,与其他算法相比,所提方法在本文构建的宁夏高速公路场景下的天气数据集weather-traffic和公开的自然天气数据集RSCM2017上准确率分别达到96.22%,84.8%,FPS分别达到157.6 Hz,137.6 Hz,FLOPs和Params仅为0.06 G和2.54 M,识别精度、速度和网络的轻量化较原网络均有提高,能够更好地应用于储存和计算能力受限的实际场景中。
A weather recognition method combining an improved ConvNeXt network and knowledge distillation is proposed to improve the accuracy of weather recognition in complex traffic scenes while achieving network lightweighting. Firstly, the ConvNeXt_F network was constructed, and the SimAm attention mechanism was added after each set of Block feature extraction of the ConvNeXt network to correct the weights of the extracted deep features and strengthen the ability to capture discriminative weather features. Secondly, during the network training, equalized focal loss (EFL) and mutual-channel loss (MCL) were aggregated as the total loss function by using the average occupancy ratio, eliminating the effect caused by data imbalance using EFL and reducing the difference of local detail features under similar weather using MCL. Finally, the knowledge distillation technique was used to migrate the weather classification knowledge from the ConvNeXt_F network to the lightweight MobileNetV3 network, which has a marginal loss of accuracy but significant reduction in the number of network parameters. The experimental results showed that compared with other algorithms, the proposed method achieved 96.22% and 84.8% accuracy on the weather-traffic dataset of Ningxia expressway and publicly-available natural weather dataset RSCM2017, respectively; the FPSs were 157.6 Hz and 137.6 Hz and FLOPs and Params were 0.06 G and 2.54 M. Compared with the original network, the recognition accuracy, speed, and lightness of the network were improved, making it better applicable to practical scenarios with limited storage and computational power.
天气识别ConvNeXt网络注意力机制知识蒸馏
weather recognitionConvNeXt networkattention mechanismknowledge distillation
路欣,田辉,严玉彬,等.低温雨雪冰冻对交通的影响与防范[C].第27届中国气象学会年会重大天气气候事件与应急气象服务分会场论文集, 2010:344-347.
LU X, TIAN H, YAN Y B, et al. Impact of Low Temperature, Rain, Snow and Freezing on Traffic and Prevention[C]. Proceedings of the 27th Annual Meeting of the Chinese Meteorological Society on Major Weather and Climate Events and Emergency Meteorological Services, 2010: 344-347.(in Chinese)
张红艳, 李茵茵, 万伟. 改进K近邻和支持向量机相融合的天气识别[J]. 计算机工程与应用, 2014, 50(14): 148-151, 167. doi: 10.3778/j.issn.1002-8331.1212-0372http://dx.doi.org/10.3778/j.issn.1002-8331.1212-0372
ZHANG H Y, LI Y Y, WAN W. Weather identification based on improved K nearest neighbor and sup-port vector machine[J]. Computer Engineering and Applications, 2014, 50(14): 148-151, 167.(in Chinese). doi: 10.3778/j.issn.1002-8331.1212-0372http://dx.doi.org/10.3778/j.issn.1002-8331.1212-0372
孟凡军, 李天伟, 徐冠雷, 等. 基于K均值聚类算法的雾天识别方法研究[J]. 现代电子技术, 2015, 38(22): 80-83.
MENG F J, LI T W, XU G L, et al. Research on method of foggy weather recognition based on K-means clustering algorithm[J]. Modern Electronics Technique, 2015, 38(22): 80-83.(in Chinese)
于浩, 王孝通, 徐冠雷. 基于贝叶斯分类的雾天和雨天两类天气图像自动识别[J]. 舰船电子工程, 2016, 36(9):73-75, 91. doi: 10.3969/j.issn.1672-9730.2016.09.019http://dx.doi.org/10.3969/j.issn.1672-9730.2016.09.019
YU H, WANG X T, XU G L. Automatic recognition of two different weathers of foggy and rainy based on Bayesian classification[J]. Ship Electronic Engineering, 2016, 36(9):73-75, 91.(in Chinese). doi: 10.3969/j.issn.1672-9730.2016.09.019http://dx.doi.org/10.3969/j.issn.1672-9730.2016.09.019
ZHANG Z, MA H, FU H, et al. Scene-free multi-class weather classification on single images[J]. Neurocomputing, 2016, 207: 365-373. doi: 10.1016/j.neucom.2016.05.015http://dx.doi.org/10.1016/j.neucom.2016.05.015
史静, 朱虹, 韩勇. 户外天气状况分类识别[J]. 计算机系统应用, 2018, 27(4):259-263.
SHI J, ZHU H, HAN Y. Outdoor weather classification[J]. Computer Systems & Applications, 2018, 27(4):259-263.(in Chinese)
李鹏程, 吕昌峰, 于向茹, 等. 基于分区域特征提取的单张图像天气识别[J]. 济南大学学报(自然科学版), 2020, 34(4): 321-327.
LI P C, LYU C F, YU X R, et al. Weather classification of single image based on sub-region feature extraction[J]. Journal of University of Jinan (Science and Technology), 2020, 34(4): 321-327.(in Chinese)
李冰村, 唐晓文, 何建新, 等. 基于机器学习的冰雹天气识别研究[J]. 气象科学, 2022, 42(5): 581-590.
LI B C, TANG X W, HE J X, et al. Hail identification based on machine learning methods[J]. Journal of the Meteorological Sciences, 2022, 42(5): 581-590.(in Chinese)
陈思玮, 贾克斌, 王聪聪, 等. 深度学习在多天气分类算法中的研究与应用[J]. 高技术通讯, 2020, 30(10):1010-1017. doi: 10.3772/j.issn.1002-0470.2020.10.003http://dx.doi.org/10.3772/j.issn.1002-0470.2020.10.003
CHEN S W, JIA K B, WANG C C, et al. Research and application of deep learning in multi-weather classification algorithms[J]. Chinese High Technology Letters, 2020, 30(10):1010-1017.(in Chinese). doi: 10.3772/j.issn.1002-0470.2020.10.003http://dx.doi.org/10.3772/j.issn.1002-0470.2020.10.003
刘奥强, 张旭. 基于ResNet50的恶劣天气识别研究[J]. 智能计算机与应用, 2022, 12(4): 168-171. doi: 10.3969/j.issn.2095-2163.2022.04.030http://dx.doi.org/10.3969/j.issn.2095-2163.2022.04.030
LIU A Q, ZHANG X. Research on severe weather recognition based on ResNet50[J]. Intelligent Computer and Applications, 2022, 12(4): 168-171.(in Chinese). doi: 10.3969/j.issn.2095-2163.2022.04.030http://dx.doi.org/10.3969/j.issn.2095-2163.2022.04.030
郭志强, 胡永武, 刘鹏, 等. 基于特征融合的室外天气图像分类[J]. 计算机应用, 2020, 40(4):1023-1029. doi: 10.11772/j.issn.1001-9081.2019081449http://dx.doi.org/10.11772/j.issn.1001-9081.2019081449
GUO ZH Q, HU Y W, LIU P, et al. Outdoor weather image classification based on feature fusion[J]. Journal of Computer Applications, 2020, 40(4):1023-1029.(in Chinese). doi: 10.11772/j.issn.1001-9081.2019081449http://dx.doi.org/10.11772/j.issn.1001-9081.2019081449
左杰格, 柳晓鸣, 蔡兵. 基于图像分块与特征融合的户外图像天气识别[J]. 计算机科学, 2022, 49(3):197-203. doi: 10.11896/jsjkx.201200263http://dx.doi.org/10.11896/jsjkx.201200263
ZUO J G, LIU X M, CAI B. Outdoor image weather recognition based on image blocks and feature fusion[J]. Computer Science, 2022, 49(3):197-203.(in Chinese). doi: 10.11896/jsjkx.201200263http://dx.doi.org/10.11896/jsjkx.201200263
崔洪涛, 曹科, 张虎, 等. 基于联合投票网络的交通场景天气分类方法[J]. 西南交通大学学报, 2021, 56(3):579-586.
CUI H T, CAO K, ZHANG H, et al. Weather classification in traffic scene based on joint voting network[J]. Journal of Southwest Jiaotong University, 2021, 56(3):579-586.(in Chinese)
HINTON G, VINYALS O, DEAN J. Distilling the Knowledge in a Neural Network[EB/OL]. 2015: arXiv: 1503.02531. https://arxiv.org/abs/1503.02531https://arxiv.org/abs/1503.02531. doi: 10.5860/choice.189890http://dx.doi.org/10.5860/choice.189890
LIU Z, MAO H Z, WU C Y, et al. A ConvNet for the 2020s[C]. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).18-24, 2022, New Orleans, LA, USA. IEEE, 2022: 11966-11976. doi: 10.1109/cvpr52688.2022.01167http://dx.doi.org/10.1109/cvpr52688.2022.01167
LI J, WANG C, HUANG B, et al. ConvNeXt-backbone HoVerNet for nuclei segmentation and classification[EB/OL]. 2022: arXiv: 2202. 13560. https://arxiv.org/abs/2202.13560https://arxiv.org/abs/2202.13560
YANG L, ZHANG R Y, Li L, et al. SimAM: A Simple, Parameter-Free Attention Module for Convolutional Neural Networks[C]. International conference on machine learning. PMLR, 2021: 11863-11874. doi: 10.1109/mlbdbi51377.2020.00079http://dx.doi.org/10.1109/mlbdbi51377.2020.00079
LIN T Y, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(2): 318-327. doi: 10.1109/tpami.2018.2858826http://dx.doi.org/10.1109/tpami.2018.2858826
LI B, YAO Y Q, TAN J R, et al. Equalized Focal Loss for Dense Long-Tailed Object Detection[C]. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).18-24, 2022, New Orleans, LA, USA. IEEE, 2022: 6980-6989. doi: 10.1109/cvpr52688.2022.00686http://dx.doi.org/10.1109/cvpr52688.2022.00686
CHANG D L, DING Y F, XIE J Y, et al. The devil is in the channels: mutual-channel loss for fine-grained image classification[J]. IEEE Transactions on Image Processing, 2020, 29: 4683-4695. doi: 10.1109/tip.2020.2973812http://dx.doi.org/10.1109/tip.2020.2973812
LIN D, LU C W, HUANG H, et al. RSCM: region selection and concurrency model for multi-class weather recognition[J]. IEEE Transactions on Image Processing, 2017, 26(9): 4154-4167. doi: 10.1109/tip.2017.2695883http://dx.doi.org/10.1109/tip.2017.2695883
LIU Z, LIN Y T, CAO Y, et al. Swin Transformer: Hierarchical Vision Transformer Using Shifted Windows[C]. 2021 IEEE/CVF International Conference on Computer Vision (ICCV).1017,2021, Montreal, QC, Canada. IEEE, 2022: 9992-10002. doi: 10.1109/iccv48922.2021.00986http://dx.doi.org/10.1109/iccv48922.2021.00986
HOWARD A, SANDLER M, CHEN B, et al. Searching for MobileNetV3[C]. 2019 IEEE/CVF International Conference on Computer Vision (ICCV). October 27 - November 2, 2019, Seoul, Korea (South). IEEE, 2020: 1314-1324. doi: 10.1109/iccv.2019.00140http://dx.doi.org/10.1109/iccv.2019.00140
0
Views
20
下载量
0
CSCD
Publicity Resources
Related Articles
Related Author
Related Institution