浏览全部资源
扫码关注微信
1.西北大学 信息科学与技术学院,陕西 西安 710127
2.西北大学 文化遗产数字化国家地方联合工程研究中心,陕西 西安 710127
[ "周明全(1954-),男,陕西西安人,教授,博士生导师,1976年、1988年于西北大学分别获得学士、硕士学位,主要从事虚拟现实、可视化技术、智能信息处理等领域的基础理论及其应用工程创新的研究。E-mail: mqzhou@bnu.edu.cn" ]
[ "褚 彤(1996-),女,陕西宝鸡人,硕士研究生,助理研究员,主要从事三维形状补全和二维图像修复研究。E-mail: chutong_nwu@163.com" ]
收稿日期:2021-03-17,
修回日期:2021-05-16,
纸质出版日期:2022-04-25
移动端阅览
周明全,褚彤,耿国华等.结构与纹理融合的三维文物孔洞修复方法[J].光学精密工程,2022,30(08):894-907.
ZHOU Mingquan,CHU Tong,GENG Guohua,et al.Three-dimensional cultural relic hole repair method combining structure and texture[J].Optics and Precision Engineering,2022,30(08):894-907.
周明全,褚彤,耿国华等.结构与纹理融合的三维文物孔洞修复方法[J].光学精密工程,2022,30(08):894-907. DOI: 10.37188/OPE.20223008.0894.
ZHOU Mingquan,CHU Tong,GENG Guohua,et al.Three-dimensional cultural relic hole repair method combining structure and texture[J].Optics and Precision Engineering,2022,30(08):894-907. DOI: 10.37188/OPE.20223008.0894.
由于外在因素导致出土的文物模型呈现破碎状态,对其进行虚拟修复对考古学具有重要意义。现有孔洞修补方法大多仅针对三维模型结构进行补全,并且在三维结构修复后缺少表面的颜色纹理信息。本文基于文物三维空间结构和纹理信息,提出一种三维文物孔洞修复方法。首先,为了解决三维文物结构修复问题,利用基于径向基函数的算法填补三维文物网格模型的孔洞,通过拟合的曲面方程调整孔洞补丁顶点,使其与原有模型更好地融合。其次,为补全文物表面颜色纹理信息,使孔洞补丁与原有模型表面纹理自然过渡,将三维问题转化为二维图像修复问题,以EdgeConnect为框架,通过添加精细化网络生成更高分辨率的结果。最后,使用Mudbox软件映射二维图像到三维模型表面,融合结构和纹理修复的结果。本文改进后的二维修复网络在评价指标PSNR(Peak Signal to Noise Ratio)、SSIM(Structural Similarity)和MAE(Mean Absolute Deviation)的性能分别提高了0.54%、0.217%和6.52%,该方法能够有效地恢复兵马俑三维模型网格结构和表面纹理信息。
Due to external factors, the unearthed cultural relic model appears to be broken, and its virtual restoration is of great significance to archaeology. Most of the existing hole repair methods complete the hole based on the structure of 3D model, and the completed result is lack of color and texture information of the surface after the 3D(3-Dimension) model structure is repaired. Therefore, based on spatial structure and texture information of cultural relics, this paper proposes a 3D cultural relic hole repair method.First, in order to solve the problem of repairing the structure of 3D cultural relic, the algorithm based on the radial basis function is used to fill the holes of the mesh model , and the vertices of the hole patches are adjusted by fitting the surface equation to better integrate with the original model. Second, for in order to complete the surface color and texture information of the 3D model, make the hole patch and the original model surface texture natural transition, the 3D problem is converted into 2D(2-Dimension) image inpaint problem, and a refined network is added into EdgeConnect to generate higher resolution result. Final, using Mudbox software to map the 2D image to the surface of the 3D model to fuse the results of structure and texture repair.The results show that the improved 2D inpaint network in this paper improves the performance of the evaluation metrics PSNR, SSIM and MAE by 0.54%, 0.217% and 6.52%, respectively. In addition, the combination of 3D structure repair and 2D texture repair makes the restoration of the cultural relic model more complete.The proposed method can effectively restore the mesh structure and texture information of the 3D model of terracotta army.
DINESH C , BAJIĆ I V , CHEUNG G . Adaptive nonrigid inpainting of three-dimensional point cloud geometry [J]. IEEE Signal Processing Letters , 2018 , 25 ( 6 ): 878 - 882 . doi: 10.1109/lsp.2018.2831621 http://dx.doi.org/10.1109/lsp.2018.2831621
FORTES M A , GONZÁLEZ P , PALOMARES A , et al . Filling holes with geometric and volumetric constraints [J]. Computers & Mathematics With Applications , 2017 , 74 ( 4 ): 671 - 683 . doi: 10.1016/j.camwa.2017.05.009 http://dx.doi.org/10.1016/j.camwa.2017.05.009
GAI S Y , DA F P , ZENG L L , et al . Research on a hole filling algorithm of a point cloud based on structure from motion [J]. Journal of the Optical Society of America A, Optics, Image Science, and Vision , 2019 , 36 ( 2 ): A39 - A46 . doi: 10.1364/josaa.36.000a39 http://dx.doi.org/10.1364/josaa.36.000a39
LIN H B , CHEN J Y , ZHANG Y R , et al . Feature preserving filling of holes on point sampled surfaces based on tensor voting [J]. Mathematical Problems in Engineering , 2018 , 2018 : 8076910 . doi: 10.1155/2018/8076910 http://dx.doi.org/10.1155/2018/8076910
YANG B , ROSA S , MARKHAM A , et al . Dense 3D object reconstruction from a single depth view [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2019 , 41 ( 12 ): 2820 - 2834 . doi: 10.1109/tpami.2018.2868195 http://dx.doi.org/10.1109/tpami.2018.2868195
WANG W Y , HUANG Q G , YOU S Y , et al . Shape inpainting using 3D generative adversarial network and recurrent convolutional networks [C]. 2017 IEEE International Conference on Computer Vision . 2229,2017 , Venice, Italy . IEEE , 2017 : 2317 - 2325 . doi: 10.1109/iccv.2017.252 http://dx.doi.org/10.1109/iccv.2017.252
王茜艳 , 陈力 , 李萌 . 基于插值的图像修复算法 [J]. 汕头大学学报(自然科学版) , 2015 , 30 ( 2 ): 72 - 80 .
WANG X Y , CHEN L , LI M . Image inpainting algorithms based on interpolation [J]. Journal of Shantou University (Natural Science Edition) , 2015 , 30 ( 2 ): 72 - 80 . (in Chinese)
BERTALMIO M , BERTOZZI A L , SAPIRO G . Navier-stokes, fluid dynamics, and image and video inpainting [C]. Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001. 814,2001 , Kauai, HI, USA. IEEE , 2001 : I . doi: 10.1109/cvpr.2001.990497 http://dx.doi.org/10.1109/cvpr.2001.990497
KOMODAKIS N . Image completion using global optimization [C]. 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition . 1722,2006 , New York, NY, USA . IEEE , 2006 : 442 - 452 .
顾建平 , 韩华 , 彭思龙 . 基于水平线插值的图像修复算法 [J]. 计算机工程 , 2006 , 32 ( 9 ): 7 - 9, 12 . doi: 10.3969/j.issn.1000-3428.2006.09.003 http://dx.doi.org/10.3969/j.issn.1000-3428.2006.09.003
GU J P , HAN H , PENG S L . Digital image impainting using level line interpolation [J]. Computer Engineering , 2006 , 32 ( 9 ): 7 - 9, 12 . (in Chinese) . doi: 10.3969/j.issn.1000-3428.2006.09.003 http://dx.doi.org/10.3969/j.issn.1000-3428.2006.09.003
BARNES C , SHECHTMAN E , FINKELSTEIN A , et al . PatchMatch [J]. ACM Transactions on Graphics , 2009 , 28 ( 3 ): 1 - 11 . doi: 10.1145/1531326.1531330 http://dx.doi.org/10.1145/1531326.1531330
刘华明 , 毕学慧 , 叶中付 , 等 . 样本块搜索和优先权填充的弧形推进图像修复 [J]. 中国图象图形学报 , 2016 , 21 ( 8 ): 993 - 1003 . doi: 10.11834/jig.20160803 http://dx.doi.org/10.11834/jig.20160803
LIU H M , BI X H , YE Z F , et al . Arc promoting image inpainting using exemplar searching and priority filling [J]. Journal of Image and Graphics , 2016 , 21 ( 8 ): 993 - 1003 . (in Chinese) . doi: 10.11834/jig.20160803 http://dx.doi.org/10.11834/jig.20160803
曾接贤 , 王璨 . 基于优先权改进和块划分的图像修复 [J]. 中国图象图形学报 , 2017 , 22 ( 9 ): 1183 - 1193 . doi: 10.11834/jig.170054 http://dx.doi.org/10.11834/jig.170054
ZENG J X , WANG C . Image completion based on redefined priority and image division [J]. Journal of Image and Graphics , 2017 , 22 ( 9 ): 1183 - 1193 . (in Chinese) . doi: 10.11834/jig.170054 http://dx.doi.org/10.11834/jig.170054
QIANG Z P , HE L B , XU D . Exemplar-based Pixel by Pixel Inpainting Based on Patch Shift [M]. Communications in Computer and Information Science . Singapore : Springer Singapore , 2017 : 370 - 382 . doi: 10.1007/978-981-10-7302-1_31 http://dx.doi.org/10.1007/978-981-10-7302-1_31
CRIMINISI A , PEREZ P , TOYAMA K . Object removal by exemplar-based inpainting [C]. 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition , 2003 . Proceedings. 1820,2003 , Madison, WI, USA. IEEE , 2003: II . doi: 10.1109/cvpr.2003.1211538 http://dx.doi.org/10.1109/cvpr.2003.1211538
PATHAK D , KRÄHENBÜHL P , DONAHUE J , et al . Context encoders: feature learning by inpainting [C]. 2016 IEEE Conference on Computer Vision and Pattern Recognition . 2730,2016 , Las Vegas, NV, USA . IEEE , 2016 : 2536 - 2544 . doi: 10.1109/cvpr.2016.278 http://dx.doi.org/10.1109/cvpr.2016.278
IIZUKA S , SIMO-SERRA E , ISHIKAWA H . Globally and locally consistent image completion [J]. ACM Transactions on Graphics , 2017 , 36 ( 4 ): 1 - 14 . doi: 10.1145/3072959.3073659 http://dx.doi.org/10.1145/3072959.3073659
YU J H , LIN Z , YANG J M , et al . Generative image inpainting with contextual attention [C]. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . 1823,2018 , Salt Lake City, UT, USA . IEEE , 2018 : 5505 - 5514 . doi: 10.1109/cvpr.2018.00577 http://dx.doi.org/10.1109/cvpr.2018.00577
NAZERI K , NG E , JOSEPH T , et al . EdgeConnect: generative image inpainting with adversarial edge learning Arxiv, 2019, abs/1901 . 00212 .
杜佶 , 张丽艳 , 王宏涛 , 等 . 基于径向基函数的三角网格曲面孔洞修补算法 [J]. 计算机辅助设计与图形学学报 , 2005 , 17 ( 9 ): 1976 - 1982 . doi: 10.3321/j.issn:1003-9775.2005.09.014 http://dx.doi.org/10.3321/j.issn:1003-9775.2005.09.014
DU J , ZHANG L Y , WANG H T , et al . Hole repairing in triangular meshes based on radial basis function [J]. Journal of Computer Aided Design & Computer Graphics , 2005 , 17 ( 9 ): 1976 - 1982 . (in Chinese) . doi: 10.3321/j.issn:1003-9775.2005.09.014 http://dx.doi.org/10.3321/j.issn:1003-9775.2005.09.014
ISOLA P , ZHU J Y , ZHOU T H , et al . Image-to-image translation with conditional adversarial networks [C]. 2017 IEEE Conference on Computer Vision and Pattern Recognition . 2126,2017 , Honolulu, HI, USA . IEEE , 2017 : 5967 - 5976 . doi: 10.1109/cvpr.2017.632 http://dx.doi.org/10.1109/cvpr.2017.632
WANG T C , LIU M Y , ZHU J Y , et al . High-resolution image synthesis and semantic manipulation with conditional GANs [C]. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . 1823,2018 , Salt Lake City, UT, USA . IEEE , 2018 : 8798 - 8807 . doi: 10.1109/cvpr.2018.00917 http://dx.doi.org/10.1109/cvpr.2018.00917
JOHNSON J , ALAHI A , LI F F . Perceptual losses for real-time style transfer and super-resolution [C]. Computer Vision-ECCV 2016 . doi: 10.1007/978-3-319-46475-6_43 http://dx.doi.org/10.1007/978-3-319-46475-6_43
GATYS L A , ECKER A S , BETHGE M . Image style transfer using convolutional neural networks [C]. 2016 IEEE Conference on Computer Vision and Pattern Recognition . 2730,2016 , Las Vegas, NV, USA . IEEE , 2016 : 2414 - 2423 . doi: 10.1109/cvpr.2016.265 http://dx.doi.org/10.1109/cvpr.2016.265
LIU G L , REDA F A , SHIH K J , et al . Image inpainting for irregular holes using partial convolutions [EB/OL]. 2018: arXiv : 1804 .07723[cs.CV]. https://arxiv.org/abs/1804.07723 https://arxiv.org/abs/1804.07723
李克文 , 张文韬 , 邵明文 , 等 . 多尺度生成式对抗网络图像修复算法 [J]. 计算机科学与探索 , 2020 , 14 ( 1 ): 159 - 170 .
LI K W , ZHANG W T , SHAO M W , et al . Multi-scale generative adversarial networks image inpainting algorithm [J]. Journal of Frontiers of Computer Science and Technology , 2020 , 14 ( 1 ): 159 - 170 . (in Chinese)
0
浏览量
458
下载量
3
CSCD
关联资源
相关文章
相关作者
相关机构