融合通用形变模型信息的面部三维重建

高瞻宇; 顾营迎; 吕耀宇; 徐振邦; 吴清文

doi:10.3788/OPE.20182601.0184

您当前的位置：

首页 >

文章列表页 >

融合通用形变模型信息的面部三维重建

信息科学 | 更新时间：2020-08-13

- 融合通用形变模型信息的面部三维重建
- Face reconstruction fused with generic morphable model
- 光学精密工程 2018年26卷第1期页码：184-192
- 作者机构：
  
  1.中国科学院长春光学精密机械与物理研究所空间机器人中心创新研究室, 吉林长春 130033
  2.中国科学院大学, 北京 100049
- 作者简介：
  
  [ "高瞻宇(1991-), 男, 辽宁沈阳人, 博士研究生, 2013年于东北大学获得学士学位, 主要从事计算机视觉与三维重建方面的研究。E-mail:zhanyugao@icloud.com" ]
  [ "顾营迎(1984-), 男, 天津人, 博士, 副研究员, 2013年于中国科学院大学获得博士学位, 主要从事机器视觉感知与控制方面的研究。E-mail:18686681605@163.com" ]
- 基金信息：
  
  吉林省科技发展计划资助项目(20160520074JH)
- DOI：10.3788/OPE.20182601.0184
  中图分类号： TP181
- 收稿日期：2017-02-20，
  
  录用日期：2017-4-7，
  
  纸质出版日期：2018-01-25
- 稿件说明：
移动端阅览
高瞻宇, 顾营迎, 吕耀宇, 等. 融合通用形变模型信息的面部三维重建[J]. 光学精密工程, 2018,26(1):184-192.

Zhan-yu GAO, Ying-ying GU, Yao-yu LÜ, et al. Face reconstruction fused with generic morphable model[J]. Optics and precision engineering, 2018, 26(1): 184-192.
高瞻宇, 顾营迎, 吕耀宇, 等. 融合通用形变模型信息的面部三维重建[J]. 光学精密工程, 2018,26(1):184-192. DOI： 10.3788/OPE.20182601.0184.

Zhan-yu GAO, Ying-ying GU, Yao-yu LÜ, et al. Face reconstruction fused with generic morphable model[J]. Optics and precision engineering, 2018, 26(1): 184-192. DOI： 10.3788/OPE.20182601.0184.

摘要

基于图像对的立体重建是用于获取人脸三维信息的通用方法，但根据图像数据和重建算法所得到的三维重建结果存在各种误差，本文对通用形变模型进行改进并与三维立体重建融合以得到更精确的重建结果。首先使用Max-Margin对象检测算法来获取面部边界框，其中回归树集合法能直接从像素强度的稀疏子集识别面部特征点。然后通过PCA颜色模型生成形状和颜色的三维面部统计模型，利用ISOMAP算法将三维网格转换为二维表面并提取纹理信息，得到面部模型。最后在源网格上进行两步非刚性表面配准的变形过程：先通过对源网格进行二次采样来选择少量网格点来表示源的全局变化，并选取径向基函数（RBF）进行非刚性全局变形；再对源顶点进行Procrustes分析获得非刚性变换，再通过加权方案来进行

-近邻变换，得到平滑的局部变形。将单图像重建的面部模型，立体重建的面部模型和本文的面部变形模型与高质量扫描云图进行对齐比较，得到面部变形模型的3个RMS值分别为2.795 2，2.102 8和2.153 4，相比于其他模型，面部变形模型更接近高质量扫描云图，即与原图像一致性更高，误差更小。面部变形模型的定性和定量分析表明，立体重建与人脸一般形状信息的组合在几何信息的表达上优于基于通用模型的单个图像重建以及未考虑通用模型的立体重建。

Abstract

Stereo reconstruction from image pairs is a genetic method for 3D acquisition of human faces. Depending on available imagery and reconstruction algorithm

the resulting 3D reconstructions may have deficits. We improve generic morphable model and fused with stereo reconstruction to remedy such deficits. Firstly

we obtain the face bounding box by Max-Margin Object Detection

and recognition the face feature points directly from a sparse subset of pixel intensities by the Ensemble of Regression Treesmethod. Secondly

through the color PCA model to generate the shape and color of the 3D facial statistical model

the ISOMAP algorithm is used to convert the 3D mesh to 2D surface and extract the texture information to get the facial model.Finally

the surface registration with two step non-rigid change on source mesh:we mean the global deformation in the source by using few anchor points on the source mesh

and using the Radial Basis Function (RBF) to express non-rigid global deformation; and then Procrustes analysis is applied for non-rigid transformation of the source vertex

the

-nearest neighbor transformation through weighted scheme tosmooth the local deformation. The experiment result shows the high-quality scan which is used for cloud comparison with the face model

stereo reconstruction and our deformed face model

after aligning all models with the high quality scan

three RMS facial deformation model values are 2.795 2

2.102 8

2.153 4.Comparing to other models

deformed face model has smaller discrepancies dominate more strongly

and the deformed face model is visually more shape consistent to the image. Qualitative and quantitative analysis of the deformed face model shows that the combination of stereo reconstruction with general shape information about human faces is geometrically superior to both the reconstruction based on single image from generic model as well as the stereo reconstruction without consideration of the generic model.

关键词

Keywords

references

BLANZ V, VETTER T. A morphable model for the synthesis of 3D faces[C]. ACM Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques, 1999: 187-194. https://dl.acm.org/citation.cfm?id=311556

PAYSAN P, KNOTHE R, AMBERG B, et al. . A 3D face model for pose and illumination invariant face recognition[C]. IEEE Proceedings of the 6th IEEE International Conference on Advanced Video and Signal Based Surveillance, 2009: 296-301. http://ieeexplore.ieee.org/document/5279762/

ROMDHANI S, BLANZ V, VETTER T. Face identification by fitting a 3D morphable model using linear shape and texture error functions[M]//HEYDEN A, SPARR G, NIELSEN M, et al. . Computer Vision-ECCV 2002. Berlin, Heidelberg: Springer, 2002: 3-19. https://link.springer.com/chapter/10.1007%2F3-540-47979-1_1?LI=true

VAN ROOTSELER R T A, SPREEUWERS L J, VELDHUIS R N J. Using 3D morphable models for face recognition in video[C]. Proceedings of the 33rd WIC Symposium on Information Theory in the Benelux, Werkgemeenschap voor Informatie-en Co mmunicatietheorie(WIC), 2012: 235-242. https://research.utwente.nl/en/publications/using-3d-morphable-models-for-face-recognition-in-video

HUBER P, FENG ZH H, CHRISTMAS W, et al. . Fitting 3D morphable face models using local features[C]. IEEE Proceedings of 2015 IEEE International Conference on Image Processing (ICIP), 2015. http://ieeexplore.ieee.org/document/7350989/

高瞻宇, 顾营迎, 刘宇航, 等.采用简化Brown模型及改进BFGS法的相机自标定[J].光学精密工程, 2017, 25(9):2532-2540.

GAO ZH Y, GU Y Y, LIU Y H, et al.. Self-calibration based on simplified brown non-linear camera model and modified BFGS algorithm[J]. Opt. Precision Eng., 2017, 25(9):2532-2540. (in Chinese)

KAZEMI V, SULLIVAN J. One millisecond face alig nment with an ensemble of regression trees[C]. IEEE, Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014: 1867-1874. http://ieeexplore.ieee.org/document/6909637/

郑明秋, 杨帆.改进非负矩阵分解的神经网络人脸识别[J].液晶与显示, 2017, 32(3):213-218.

ZHENG M Q, YANG F.Face recognition based on improved NMF and neural network[J]. Chinese Journal of Liquid Crystals and Displays, 2017, 32(3):213-218. (in Chinese)

KING D E. Ma x -margin object det e ction[Z]. arXiv preprint arXiv: 1502.00046, 2015.

RICHARD H, ZISSERMAN A. Multiple View Geometry in Computer Vision[M]. Cambridge:Cambridge University Press, 2000.

LI H, SUMNER R W, PAULY M. Global correspondence optimization for non-rigid registration of depth scans[J]. Computer Graphics Forum, 2008, 27(5):1421-1430.

LEVI Z, LEVIN D. Shape deformation via interior RBF[J]. IEEE Transactions on Visualization and Computer Graphics, 2014, 20(7):1062-1075.

FASSHAUER G E. Meshfree Approximation Methods with MATLAB[M]. River Edge, NJ, USA:World Scientific, 2007.

SUMNER R W, SCHMID J, PAULY M. Embedded deformation for shape manipulation[J]. ACM Transactions on Graphics (TOG), 2007, 26(3):80.

CloudCompare (version2. 6. 2)[EB/OL]. http://www.cloudcompare.org/ http://www.cloudcompare.org/ .

KING D. Real-time face pose estimation[EB/OL]. (2015-12). http://blog.dlib.net/2014/08/real-time-face-pose-estimation.html http://blog.dlib.net/2014/08/real-time-face-pose-estimation.html .

浏览量

524

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

多层次信息融合在铁谱图像磨粒识别中的应用

神经网络辅助卡尔曼滤波在组合导航中的应用

基于改进支持向量机的目标威胁估计

基于信息融合实现的激光陀螺调腔检测

融合距离信息的红外自适应相关跟踪