Spatial information adaptive regulation and feature alignment for infrared methane instance segmentation

HE Zifen; CAO Huizhu; ZHANG Yinhui; ZHUANG Hong

doi:10.37188/OPE.20233120.3034

您当前的位置：

首页 >

文章列表页 >

Spatial information adaptive regulation and feature alignment for infrared methane instance segmentation

Information Sciences | 更新时间：2023-11-27

- Spatial information adaptive regulation and feature alignment for infrared methane instance segmentation
- Optics and Precision Engineering Vol. 31, Issue 20, Pages: 3034-3049(2023)
- 作者机构：
  
  1.昆明理工大学机电工程学院，云南昆明 650000
- 作者简介：
- 基金信息：
- DOI：10.37188/OPE.20233120.3034
  CLC：
扫描看全文
何自芬,曹辉柱,张印辉等.空间信息自适应调控和特征对齐的红外甲烷实例分割[J].光学精密工程,2023,31(20):3034-3049.

HE Zifen,CAO Huizhu,ZHANG Yinhui,et al.Spatial information adaptive regulation and feature alignment for infrared methane instance segmentation[J].Optics and Precision Engineering,2023,31(20):3034-3049.
何自芬,曹辉柱,张印辉等.空间信息自适应调控和特征对齐的红外甲烷实例分割[J].光学精密工程,2023,31(20):3034-3049. DOI： 10.37188/OPE.20233120.3034.

HE Zifen,CAO Huizhu,ZHANG Yinhui,et al.Spatial information adaptive regulation and feature alignment for infrared methane instance segmentation[J].Optics and Precision Engineering,2023,31(20):3034-3049. DOI： 10.37188/OPE.20233120.3034.

摘要

传统接触式甲烷泄漏传感器检测范围小且效率低，而结合非接触式红外热成像的机器视觉算法可实现远距离、大范围红外甲烷实例分割，对于提高甲烷检测效率及保障人员安全具有显著优势。然而远距离甲烷气体图像轮廓模糊、泄漏的甲烷气体与背景对比度较低且形状易受大气流动因素影响等问题限制了红外甲烷实例分割性能。针对上述问题，本文提出一种空间信息自适应调控和特征对齐的网络模型（Adaptive spatial information regulation and Feature alignment Network， AFNet）实现甲烷泄漏红外实例分割。首先，为增强模型的特征提取能力，提出自适应空间信息调控模块赋予主干网络不同尺度残差块自适应权重丰富模型提取的特征空间；其次，构建加权双向金字塔弥补特征金字塔自顶而下的特征传播方式导致的低层特征空间位置和实例边缘信息弥散丢失问题，以适应甲烷气体复杂轮廓变化下前景目标定位检测和轮廓分割需求。最后，设计原型特征对齐模块捕获长距离气体特征之间的语义关系丰富原型语义信息量以改善生成目标掩码质量提高甲烷气体分割精度。实验结果表明，本文提出的AFNet模型AP50@95，AP50定量分割精度分别达到42.42%，92.18%，相比于原始Yolact模型分割精度，分别提高9.79%，6.18%，推理速度达到36.80 frame/s，满足甲烷泄漏分割需求。实验结果验证了本文算法对红外甲烷泄漏分割的有效性和工程实用性。

Abstract

Conventional contact methane leak sensors suffer from a small detection range and low efficiency， but machine vision algorithms combined with non-contact infrared thermal imaging can make infrared methane instance segmentation possible at long distances and large ranges. This is a significant advantage for improving methane detection efficiency and ensuring personnel safety. However， the segmentation performance of infrared methane instances is limited by such problems as blurred contour and low contrast between the leaking methane gas and the background， and it can be affected by atmospheric flow factors. In response to these problems， an adaptive spatial information regulation and feature alignment network （AFNet） is proposed to segment infrared instances of methane leakage. First， to enhance the model’s feature extraction， an adaptive spatial information regulation module is proposed to endow the backbone network with adaptive weights for different scale residual blocks， which enrich the feature space extracted by the model. Second， to meet the requirements of foreground target positioning detection and contour segmentation under complex methane gas contours， a weighted bidirectional pyramid is designed to reduce the diffusion， loss of spatial location， and instance edge information in low-level features， which are caused by the top-down propagation of the feature pyramid. Finally， a prototype feature alignment module is designed to capture the semantic relationships between long-distance gas features， enriching the semantic information of the prototype and improving the quality of generated target masks to improve the methane instance segmentation accuracy. Experimental results show that the proposed AFNet model achieves AP50@95 and AP50 quantitative segmentation accuracies of 42.42% and 92.18%， which are improved by 9.79% and 6.18% compared with the original Yolact， respectively. In addition， the inference speed achieves 36.80 frames/s and meets the requirements of methane leakage segmentation. The experimental results validate the effectiveness and engineering practicality of the algorithm proposed for infrared methane leakage segmentation.

关键词

红外甲烷自适应调控特征对齐特征金字塔实例分割

Keywords

infrared methaneadaptive regulationfeature alignmentfeature pyramidsinstance segmentation

references

熊仕富. 红外热成像甲烷气体探测与识别系统关键技术研究［D］. 长春：长春理工大学， 2018.

XIONG S F. Studies on Key Technologies of Infrared Thermal Imaging Detection and Identification System for Methane Gas［D］. Changchun： Changchun University of Science and Technology， 2018. （in Chinese）

KEYES T， RIDGE G， KLEIN M， et al. An enhanced procedure for urban mobile methane leak detection［J］. Heliyon， 2020， 6（10）： e04876. doi: 10.1016/j.heliyon.2020.e04876http://dx.doi.org/10.1016/j.heliyon.2020.e04876

GLOBAL METHANE INITIATIVE. Global Methane Emissions and Mitigation Opportunities［EB/OL］. Available： www.globalmethane.orgwww.globalmethane.org， 2011.

BOAZ L， KAIJAGE S， SINDE R. An Overview of Pipeline Leak Detection and Location Systems［C］. Proceedings of the 2nd Pan African International Conference on Science， Computing and Telecommunications （PACT 2014）.14-18， 2014， Arusha， Tanzania. IEEE， 2015： 133-137. doi: 10.1109/scat.2014.7055147http://dx.doi.org/10.1109/scat.2014.7055147

JIN H， ZHANG L， LIANG W， et al. Integrated leakage detection and localization model for gas pipelines based on the acoustic wave method［J］. Journal of Loss Prevention in the Process Industries， 2014， 27： 74-88. doi: 10.1016/j.jlp.2013.11.006http://dx.doi.org/10.1016/j.jlp.2013.11.006

YANG C B， ROMANAK K D， REEDY R C， et al. Soil gas dynamics monitoring at a CO2-EOR site for leakage detection［J］. Geomechanics and Geophysics for Geo-Energy and Geo-Resources， 2017， 3（3）： 351-364. doi: 10.1007/s40948-017-0053-7http://dx.doi.org/10.1007/s40948-017-0053-7

XU J H， NIE Z W， SHAN F Q， et al. Leak detection methods overview and summary［C］. ICPTT 2012. Wuhan， China. Reston， VA： American Society of Civil Engineers， 2012： 1034-1050. doi: 10.1061/9780784412619.105http://dx.doi.org/10.1061/9780784412619.105

SCAFUTTO RD， DE SOUZA FILHO CR. Detection of heavy hydrocarbon plumes （Ethane， propane and Butane） using airborne longwave （7.6-13.5 μm） infrared hyperspectral data［J］. Fuel， 2019， 242： 863-870. doi: 10.1016/j.fuel.2018.12.127http://dx.doi.org/10.1016/j.fuel.2018.12.127

HUANG K， MAO X. Detectability of infrared small targets［J］. Infrared Physics & Technology， 2010， 53（3）： 208-217. doi: 10.1016/j.infrared.2009.12.001http://dx.doi.org/10.1016/j.infrared.2009.12.001

丁德武，申屠灵女，邹兵，等. 红外热成像技术在石化装置泄漏隐患检测中的应用［J］. 安全、健康和环境， 2015， 15（12）： 17-20.

DING D W， SHENTU L N， ZOU B， et al. Application of infrared thermal imaging technology in detecting leakage of petrochemical plant［J］. Safety Health & Environment， 2015， 15（12）： 17-20. （in Chinese）

鞠默然，罗海波，刘广琦，等. 采用空间注意力机制的红外弱小目标检测网络［J］. 光学精密工程， 2021， 29（4）： 843-853. doi: 10.37188/OPE.20212904.0843http://dx.doi.org/10.37188/OPE.20212904.0843

JU M R， LUO H B， LIU G Q， et al. Infrared dim and small target detection network based on spatial attention mechanism［J］. Opt. Precision Eng.， 2021， 29（4）： 843-853. （in Chinese）. doi: 10.37188/OPE.20212904.0843http://dx.doi.org/10.37188/OPE.20212904.0843

WANG J， TCHAPMI L P， RAVIKUMARA A P， et al. Machine Vision for Natural Gas Methane Emissions Detection Using an Infrared Camera［EB/OL］. 2019： arXiv： 1904.08500. https：//arxiv.org/abs/1904.08500.pdfhttps://arxiv.org/abs/1904.08500.pdf. doi: 10.1016/j.apenergy.2019.113998http://dx.doi.org/10.1016/j.apenergy.2019.113998

XIONG K N， JIANG J B， PAN Y Y， et al. Deep learning approach for detection of underground natural gas micro-leakage using infrared thermal images［J］. Sensors， 2022， 22（14）： 5322. doi: 10.3390/s22145322http://dx.doi.org/10.3390/s22145322

MELO R O， COSTA M G F， FILHO C F F C. Applying convolutional neural networks to detect natural gas leaks in wellhead images［J］. IEEE Access， 2020， 8： 191775-191784. doi: 10.1109/access.2020.3031683http://dx.doi.org/10.1109/access.2020.3031683

SHI J， CHANG Y， XU C， et al. Real-time leak detection using an infrared camera and Faster R-CNN technique［J］. Computers & Chemical Engineering， 2020， 135： 106780. doi: 10.1016/j.compchemeng.2020.106780http://dx.doi.org/10.1016/j.compchemeng.2020.106780

HE K M， GKIOXARI G， DOLLÁR P， et al. Mask R-CNN［C］. 2017 IEEE International Conference on Computer Vision （ICCV）.22-29， 2017， Venice， Italy. IEEE， 2017： 2980-2988. doi: 10.1109/iccv.2017.322http://dx.doi.org/10.1109/iccv.2017.322

HUANG Z J， HUANG L C， GONG Y C， et al. Mask scoring R-CNN［C］. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）.15-20， 2019， Long Beach， CA， USA. IEEE， 2020： 6402-6411. doi: 10.1109/cvpr.2019.00657http://dx.doi.org/10.1109/cvpr.2019.00657

CHEN K， PANG J M， WANG J Q， et al. Hybrid task cascade for instance segmentation［C］. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）.15-20， 2019， Long Beach， CA， USA. IEEE， 2020： 4969-4978. doi: 10.1109/cvpr.2019.00511http://dx.doi.org/10.1109/cvpr.2019.00511

LIN T Y， GOYAL P， GIRSHICK R， et al. Focal loss for dense object detection［C］. 2017 IEEE International Conference on Computer Vision （ICCV）.22-29， 2017， Venice， Italy. IEEE， 2017： 2999-3007. doi: 10.1109/iccv.2017.324http://dx.doi.org/10.1109/iccv.2017.324

WANG X L， KONG T， SHEN C H， et al. SOLO： Segmenting Objects by Locations［M］. Computer Vision - ECCV 2020. Cham： Springer International Publishing， 2020： 649-665. doi: 10.1007/978-3-030-58523-5_38http://dx.doi.org/10.1007/978-3-030-58523-5_38

WANG X， ZHANG R， KONG T， et al. Solov2： Dynamic and fast instance segmentation［J］. Advances in Neural information processing systems， 2020， 33： 17721-17732.

TIAN Z， SHEN C H， CHEN H. Conditional Convolutions for Instance Segmentation［M］. Computer Vision - ECCV 2020. Cham： Springer International Publishing， 2020： 282-298. doi: 10.1007/978-3-030-58452-8_17http://dx.doi.org/10.1007/978-3-030-58452-8_17

BOLYA D， ZHOU C， XIAO F Y， et al. YOLACT： Real-Time Instance Segmentation［C］. 2019 IEEE/CVF International Conference on Computer Vision （ICCV）. October 27 - November 2， 2019， Seoul， Korea （South）. IEEE， 2020： 9156-9165. doi: 10.1109/iccv.2019.00925http://dx.doi.org/10.1109/iccv.2019.00925

HE K M， ZHANG X Y， REN S Q， et al. Deep residual learning for image recognition［C］. 2016 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）.27-30， 2016， Las Vegas， NV， USA. IEEE， 2016： 770-778. doi: 10.1109/cvpr.2016.90http://dx.doi.org/10.1109/cvpr.2016.90

TAN M X， PANG R M， LE Q V. EfficientDet： scalable and efficient object detection［C］. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）.13-19， 2020， Seattle， WA， USA. IEEE， 2020： 10778-10787. doi: 10.1109/cvpr42600.2020.01079http://dx.doi.org/10.1109/cvpr42600.2020.01079

SHELHAMER E， LONG J， DARRELL T. Fully convolutional networks for semantic segmentation［C］. IEEE Transactions on Pattern Analysis and Machine Intelligence.24， 2016， IEEE， 2016： 640-651. doi: 10.1109/tpami.2016.2572683http://dx.doi.org/10.1109/tpami.2016.2572683

郝帅，何田，马旭，等. 动态特征优化机制下的跨尺度红外行人检测［J］. 光学精密工程， 2022， 30（19）： 2390-2403. doi: 10.37188/OPE.20223019.2390http://dx.doi.org/10.37188/OPE.20223019.2390

HAO S， HE T， MA X， et al. Cross-scale infrared pedestrian detection based on dynamic feature optimization mechanism［J］. Opt. Precision Eng.， 2022， 30（19）： 2390-2403.（in Chinese）. doi: 10.37188/OPE.20223019.2390http://dx.doi.org/10.37188/OPE.20223019.2390

WANG K X， LIEW J H， ZOU Y T， et al. PANet： Few-shot image semantic segmentation with prototype alignment［C］. 2019 IEEE/CVF International Conference on Computer Vision （ICCV）. October 27 - November 2， 2019， Seoul， Korea （South）. IEEE， 2020： 9196-9205. doi: 10.1109/iccv.2019.00929http://dx.doi.org/10.1109/iccv.2019.00929

FAN M Y， LAI S Q， HUANG J S， et al. Rethinking BiSeNet for real-time semantic segmentation［C］. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）.20-25， 2021， Nashville， TN， USA. IEEE， 2021： 9711-9720. doi: 10.1109/cvpr46437.2021.00959http://dx.doi.org/10.1109/cvpr46437.2021.00959

CHENG T H， WANG X G， CHEN S Y， et al. Sparse instance activation for real-time instance segmentation［C］. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）.18-24， 2022， New Orleans， LA， USA. IEEE， 2022： 4423-4432. doi: 10.1109/cvpr52688.2022.00439http://dx.doi.org/10.1109/cvpr52688.2022.00439

ZHANG T， WEI S Q， JI S P. E2EC： an end-to-end contour-based method for high-quality high-speed instance segmentation［C］. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）.18-24， 2022， New Orleans， LA， USA. IEEE， 2022： 4433-4442. doi: 10.1109/cvpr52688.2022.00440http://dx.doi.org/10.1109/cvpr52688.2022.00440

LIU H T， RIVERA SOTO R A， XIAO F Y， et al. YolactEdge： real-time instance segmentation on the edge［C］. 2021 IEEE International Conference on Robotics and Automation （ICRA）.30-5，2021， Xi'an， China. IEEE， 2021： 9579-9585. doi: 10.1109/icra48506.2021.9561858http://dx.doi.org/10.1109/icra48506.2021.9561858

FANG Y X， YANG S S， WANG X G， et al. Instances as queries［C］. 2021 IEEE/CVF International Conference on Computer Vision （ICCV）.10-17， 2021， Montreal， QC， Canada. IEEE， 2022： 6890-6899. doi: 10.1109/iccv48922.2021.00683http://dx.doi.org/10.1109/iccv48922.2021.00683

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Vehicle pressure line detection based on improved Mask R-CNN + LaneNet

Related Author

No data

Related Institution

State Key Laboratory of Applied Optics， Changchun Institute of Optics， Fine Mechanics and Physics， Chinese Academy of Sciences

University of Chinese Academy of Sciences

Address：No.3888 Dong Nanhu Road, Changchun, Jilin, China Postal code：130033
Tel：0431-86176855 Email：gxjmgc@ciomp.ac.cn
Technical support is provided by Beijing Founder electronics co., LTD 吉ICP备11002662号-17 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰