Face reconstruction fused with generic morphable model

Zhan-yu GAO; Ying-ying GU; Yao-yu LÜ; Zhen-bang XU; Qing-wen WU

doi:10.3788/OPE.20182601.0184

您当前的位置：

首页 >

文章列表页 >

Face reconstruction fused with generic morphable model

Information Sciences | 更新时间：2020-08-13

- Face reconstruction fused with generic morphable model
- Optics and Precision Engineering Vol. 26, Issue 1, Pages: 184-192(2018)
- 作者机构：
  
  1.中国科学院长春光学精密机械与物理研究所空间机器人中心创新研究室, 吉林长春 130033
  2.中国科学院大学, 北京 100049
- 作者简介：
- 基金信息：
- DOI：10.3788/OPE.20182601.0184
  CLC： TP181
- Received：20 February 2017，
  
  Accepted：07 April 2017，
  
  Published：25 January 2018
- 稿件说明：
移动端阅览
Zhan-yu GAO, Ying-ying GU, Yao-yu LÜ, et al. Face reconstruction fused with generic morphable model[J]. Optics and precision engineering, 2018, 26(1): 184-192.
DOI：

Zhan-yu GAO, Ying-ying GU, Yao-yu LÜ, et al. Face reconstruction fused with generic morphable model[J]. Optics and precision engineering, 2018, 26(1): 184-192. DOI： 10.3788/OPE.20182601.0184.

摘要

基于图像对的立体重建是用于获取人脸三维信息的通用方法，但根据图像数据和重建算法所得到的三维重建结果存在各种误差，本文对通用形变模型进行改进并与三维立体重建融合以得到更精确的重建结果。首先使用Max-Margin对象检测算法来获取面部边界框，其中回归树集合法能直接从像素强度的稀疏子集识别面部特征点。然后通过PCA颜色模型生成形状和颜色的三维面部统计模型，利用ISOMAP算法将三维网格转换为二维表面并提取纹理信息，得到面部模型。最后在源网格上进行两步非刚性表面配准的变形过程：先通过对源网格进行二次采样来选择少量网格点来表示源的全局变化，并选取径向基函数（RBF）进行非刚性全局变形；再对源顶点进行Procrustes分析获得非刚性变换，再通过加权方案来进行

-近邻变换，得到平滑的局部变形。将单图像重建的面部模型，立体重建的面部模型和本文的面部变形模型与高质量扫描云图进行对齐比较，得到面部变形模型的3个RMS值分别为2.795 2，2.102 8和2.153 4，相比于其他模型，面部变形模型更接近高质量扫描云图，即与原图像一致性更高，误差更小。面部变形模型的定性和定量分析表明，立体重建与人脸一般形状信息的组合在几何信息的表达上优于基于通用模型的单个图像重建以及未考虑通用模型的立体重建。

Abstract

Stereo reconstruction from image pairs is a genetic method for 3D acquisition of human faces. Depending on available imagery and reconstruction algorithm

the resulting 3D reconstructions may have deficits. We improve generic morphable model and fused with stereo reconstruction to remedy such deficits. Firstly

we obtain the face bounding box by Max-Margin Object Detection

and recognition the face feature points directly from a sparse subset of pixel intensities by the Ensemble of Regression Treesmethod. Secondly

through the color PCA model to generate the shape and color of the 3D facial statistical model

the ISOMAP algorithm is used to convert the 3D mesh to 2D surface and extract the texture information to get the facial model.Finally

the surface registration with two step non-rigid change on source mesh:we mean the global deformation in the source by using few anchor points on the source mesh

and using the Radial Basis Function (RBF) to express non-rigid global deformation; and then Procrustes analysis is applied for non-rigid transformation of the source vertex

the

-nearest neighbor transformation through weighted scheme tosmooth the local deformation. The experiment result shows the high-quality scan which is used for cloud comparison with the face model

stereo reconstruction and our deformed face model

after aligning all models with the high quality scan

three RMS facial deformation model values are 2.795 2

2.102 8

2.153 4.Comparing to other models

deformed face model has smaller discrepancies dominate more strongly

and the deformed face model is visually more shape consistent to the image. Qualitative and quantitative analysis of the deformed face model shows that the combination of stereo reconstruction with general shape information about human faces is geometrically superior to both the reconstruction based on single image from generic model as well as the stereo reconstruction without consideration of the generic model.

关键词

Keywords

references

BLANZ V, VETTER T. A morphable model for the synthesis of 3D faces[C]. ACM Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques, 1999: 187-194. https://dl.acm.org/citation.cfm?id=311556

PAYSAN P, KNOTHE R, AMBERG B, et al. . A 3D face model for pose and illumination invariant face recognition[C]. IEEE Proceedings of the 6th IEEE International Conference on Advanced Video and Signal Based Surveillance, 2009: 296-301. http://ieeexplore.ieee.org/document/5279762/

ROMDHANI S, BLANZ V, VETTER T. Face identification by fitting a 3D morphable model using linear shape and texture error functions[M]//HEYDEN A, SPARR G, NIELSEN M, et al. . Computer Vision-ECCV 2002. Berlin, Heidelberg: Springer, 2002: 3-19. https://link.springer.com/chapter/10.1007%2F3-540-47979-1_1?LI=true

VAN ROOTSELER R T A, SPREEUWERS L J, VELDHUIS R N J. Using 3D morphable models for face recognition in video[C]. Proceedings of the 33rd WIC Symposium on Information Theory in the Benelux, Werkgemeenschap voor Informatie-en Co mmunicatietheorie(WIC), 2012: 235-242. https://research.utwente.nl/en/publications/using-3d-morphable-models-for-face-recognition-in-video

HUBER P, FENG ZH H, CHRISTMAS W, et al. . Fitting 3D morphable face models using local features[C]. IEEE Proceedings of 2015 IEEE International Conference on Image Processing (ICIP), 2015. http://ieeexplore.ieee.org/document/7350989/

高瞻宇, 顾营迎, 刘宇航, 等.采用简化Brown模型及改进BFGS法的相机自标定[J].光学精密工程, 2017, 25(9):2532-2540.

GAO ZH Y, GU Y Y, LIU Y H, et al.. Self-calibration based on simplified brown non-linear camera model and modified BFGS algorithm[J]. Opt. Precision Eng., 2017, 25(9):2532-2540. (in Chinese)

KAZEMI V, SULLIVAN J. One millisecond face alig nment with an ensemble of regression trees[C]. IEEE, Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014: 1867-1874. http://ieeexplore.ieee.org/document/6909637/

郑明秋, 杨帆.改进非负矩阵分解的神经网络人脸识别[J].液晶与显示, 2017, 32(3):213-218.

ZHENG M Q, YANG F.Face recognition based on improved NMF and neural network[J]. Chinese Journal of Liquid Crystals and Displays, 2017, 32(3):213-218. (in Chinese)

KING D E. Ma x -margin object det e ction[Z]. arXiv preprint arXiv: 1502.00046, 2015.

RICHARD H, ZISSERMAN A. Multiple View Geometry in Computer Vision[M]. Cambridge:Cambridge University Press, 2000.

LI H, SUMNER R W, PAULY M. Global correspondence optimization for non-rigid registration of depth scans[J]. Computer Graphics Forum, 2008, 27(5):1421-1430.

LEVI Z, LEVIN D. Shape deformation via interior RBF[J]. IEEE Transactions on Visualization and Computer Graphics, 2014, 20(7):1062-1075.

FASSHAUER G E. Meshfree Approximation Methods with MATLAB[M]. River Edge, NJ, USA:World Scientific, 2007.

SUMNER R W, SCHMID J, PAULY M. Embedded deformation for shape manipulation[J]. ACM Transactions on Graphics (TOG), 2007, 26(3):80.

CloudCompare (version2. 6. 2)[EB/OL]. http://www.cloudcompare.org/ http://www.cloudcompare.org/ .

KING D. Real-time face pose estimation[EB/OL]. (2015-12). http://blog.dlib.net/2014/08/real-time-face-pose-estimation.html http://blog.dlib.net/2014/08/real-time-face-pose-estimation.html .

Views

524

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Application of multi-level information fusion for wear particle recognition of ferrographic images

Application of neural network aided Kalman filtering to SINS/GPS

Target threat assessment using improved SVM

Detection of laser gyro cavity adjustment using information fusion

Infrared adaptive correlation tracking of fusing distance information

Related Author

Yu SU

Zhi-fen ZHANG

Feng CHEN

Yao-ning SUN

Bin XU

JIANG Rui-kai

CHU Hai-rong

JIA Hong-guang

Related Institution

School of Mechanical Engineering, Xi'an Jiaotong University

School of Mechanical Engineering, Xinjiang University, Wulumuqi

Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences

University of Chinese Academy of Sciences, Beijing 1300039

School of Mechatronic Engineering and Automation, Shanghai University

AI问答

Address：No.3888 Dong Nanhu Road, Changchun, Jilin, China Postal code：130033
Tel：0431-86176855 Email：gxjmgc@ciomp.ac.cn
Technical support is provided by Beijing Founder electronics co., LTD 吉ICP备11002662号-17 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰