Implementation and performance analysis of optical flow based motion estimation on FPGA

Xiang-jun WANG; Ji-long ZHANG; Lei YIN

doi:10.3788/OPE.20192701.0211

您当前的位置：

首页 >

文章列表页 >

Implementation and performance analysis of optical flow based motion estimation on FPGA

Information Sciences | 更新时间：2020-08-13

- Implementation and performance analysis of optical flow based motion estimation on FPGA
- Optics and Precision Engineering Vol. 27, Issue 1, Pages: 211-220(2019)
- 作者机构：
  
  1.天津大学精密测试技术及仪器国家重点实验室，天津 300072
  2.天津大学微光机电系统技术教育部重点实验室，天津 300072
- 作者简介：
- 基金信息：
- DOI：10.3788/OPE.20192701.0211
  CLC： TP391.4;TN79.1
- Received：28 June 2018，
  
  Accepted：29 August 2018，
  
  Published：25 January 2019
- 稿件说明：
移动端阅览
Xiang-jun WANG, Ji-long ZHANG, Lei YIN. Implementation and performance analysis of optical flow based motion estimation on FPGA[J]. Optics and precision engineering, 2019, 27(1): 211-220.
DOI：

Xiang-jun WANG, Ji-long ZHANG, Lei YIN. Implementation and performance analysis of optical flow based motion estimation on FPGA[J]. Optics and precision engineering, 2019, 27(1): 211-220. DOI： 10.3788/OPE.20192701.0211.

摘要

图像序列的光流估计理论在机器视觉领域已被提出多年，但算法的高计算复杂度限制了其在工业领域的应用。为了满足应用的实时性要求，阐述了一种光流实时估计的实现方法。为了提高算法精度及减少FPGA片内资源消耗，对L&K光流计算方法进行改进。首先，通过设计两层光流计算架构来提高精度。针对在此过程中出现的外部存储器读写速率不够的问题，提出一次读取同时分层缓存、分时计算的方法。考虑到两层光流在计算过程中的迭代关联性，设计了满足要求的外部存储器数据读出顺序表; 然后，针对卷积运算资源消耗大的问题，设计了新的卷积权重函数，能够将卷积计算量降低73%，从而节省了大量逻辑资源; 最后通过实验验证，所提出的FPGA光流计算方法的精度高于运行在PC平台的L&K方法，卷积计算资源消耗明显降低。设计的系统可以完成1 280×1 024 pixel、60 frame/s输入视频的计算，满足光流计算的实时性要求。

Abstract

Optical flow estimation theory has been proposed in the field of machine vision for many years

but the high computational complexity of the algorithm limits its application in the industrial field. To meet the real-time requirements

a method was realized by FPGA in this study. To improve the accuracy of the algorithm and reduce the consumption of hardware resources

the Lucas and Kanade optical flow calculation method was improved. First

a two-level optical flow computation framework was designed to improve the accuracy. To address the problem of insufficient read-write rate of the external memory

it was proposed that when the image was read

it was sampled and cached to two spaces at the same time for subsequent computation. Considering the iterative correlation between the two levels of optical flow

we designed the data readout order to be stored in external memory. Then

in this study

a new convolution weight function was designed to reduce the volume of convolution

which was reduced by 73%

thus saving a lot of hardware resources. Experimental results indicate that the accuracy of the hardware implementation is higher than that of the Lucas and Kanade method on a PC

and the convolution computation is significantly reduced. This system fulfills the specified real time constrains of 60 images per second with 1 280×1 024 image resolution.

关键词

Keywords

references

孙辉, 赵红颖, 熊经武, 等.基于光流模型的图像运动估计方法[J].光学精密工程, 2002, 10(5):443-447.

SUN H, ZHAO H Y, XIONG J W, et al ..Method of estimating image motion based on the optical flow model[J]. Opt.Precision Eng ., 2002, 10(5):443-447.(in Chinese)

BAGHAIE A, TAFTI A P, OWEN H A, et al ..Three-dimensional reconstruction of highly complex microscopic samples using scanning electron microscopy and optical flow estimation[J]. Plos One , 2017, 12(4):e0175078.

刘洪彬, 常发亮.权重系数自适应光流法运动目标检测[J].光学精密工程, 2016, 24(2):460-468.

LIU H B, CHANG F L.Moving object detection by optical flow method based on adaptive weight coefficient[J]. Opt.Precision Eng ., 2016, 24(2):460-468.(in Chinese)

CARVALHO B M, SANTOS T S, OLIVEIRA L M.Fuzzy segmentation of video shots using hybrid color spaces and motion information[J]. Pattern Analysis & Applications , 2014, 17(2):249-264.

王世刚, 鲁奉军, 赵文婷, 等.应用在线随机森林投票的动作识别[J].光学精密工程, 2016, 24(8):2010-2017.

WANG SH G, LU F J, ZHAO W T, et al ..Action recognition based on on-line random forest voting[J]. Opt.Precision Eng. , 2016, 24(8):2010-2017.(in Chinese)

BARRON J L, FLEET D J, BEAUCHEMIN S S.Performance of optical flow techniques[J]. International Journal of Computer Vision , 1994, 12(1):43-77.

LUCAS B D, KANADE T.An iterative image registration technique with an application to stereo vision[C]. International Joint Conference on Artificial Intelligence.Morgan Kaufmann Publishers Inc .1981: 674-679.

MACLEAN W J.An evaluation of the suitability of FPGAs for embedded vision systems[J].Research Gate, 2005, 408:131-131.

DIAZ J, ROS E, PELAYO F, et al ..FPGA-based real-time optical-flow system[J]. IEEE Transactions on Circuits & Systems for Video Technology , 2006, 16(2):274-279.

PYDA B, BRINDHA R.A novel high speed L-K based optical flow computation[C]. International Conference on Communication and Computational Intelligence.IEEE , 2011: 104-108. https://www.researchgate.net/publication/251993751_A_novel_high_speed_LK_based_optical_flow_computation

WEI Z, LEE D J, NELSON B E.FPGA-based real-time optical flow algorithm design and implementation[J]. Journal of Multimedia , 2007, 2(5):38-45.

BERTHOLD KP.Horn, Brian G.Schunck.Determining optical flow[J]. Artificial Intelligence , 1981, 17(1-3):185-203.

ARRIBAS P C, MACIA F M H.FPGA implementation of Santos-Victor optical flow algorithm for real-time image processing:an useful attempt[J]. Proceedings of SPIE-The International Society for Optical Engineering , 2003, 5117(5117):23-32.

ZULOAGA A, CUADRADO C, BIDARTE U.Hardware implementation of optical flow constraint equation using FPGAs[J]. Computer Vision & Image Understanding , 2005, 98(3):462-490.

ZHANG K, JIN X, WU A.Accelerating eulerian video magnification using FPGA[C]. International Conference on Advanced Communication Technology.IEEE , 2017: 554-559. https://www.researchgate.net/publication/315868083_Accelerating_Eulerian_Video_magnification_using_FPGA

Harn-Schunck Optical Flow with a Mutti-Scale Strategy.[OL](2005). http://demo.ipol.im/demo/20/ http://demo.ipol.im/demo/20/ .

DÍAZ J, ROS E, MOTA S, et al .Real time optical flow processing system[J]. Field Programmable Logic & Application , 2004, 3203:617-626.

Views

245

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

No data

Related Author

No data

Related Institution

No data

AI问答

Address：No.3888 Dong Nanhu Road, Changchun, Jilin, China Postal code：130033
Tel：0431-86176855 Email：gxjmgc@ciomp.ac.cn
Technical support is provided by Beijing Founder electronics co., LTD 吉ICP备11002662号-17 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰