基于改进CycleGAN的图像风格迁移

杜振龙; 沈海洋; 国美; 李晓丽

doi:10.3788/OPE.20192708.1836

您当前的位置：

首页 >

文章列表页 >

基于改进CycleGAN的图像风格迁移

信息科学 | 更新时间：2020-08-13

- 基于改进CycleGAN的图像风格迁移
- Image styletransfer based on improved Cycle
- 光学精密工程 2019年27卷第8期页码：1836-1844
- 作者机构：
  
  南京工业大学计算机科学与技术学院，江苏南京 211816
- 作者简介：
  
  [ "杜振龙(1971-)，男，陕西韩城人，博士，2007年于浙江大学获得博士学位，主要从事计算机视觉、多媒体取证、可视计算等。E-mail：duzhl@njtech.edu.cn" ]
  [ "沈海洋(1995-)，男，江苏启东人，硕士研究生，2017年于南京工业大学获得学士学位，主要从事计算机视觉、图像处理方面的研究。E-mail：seanyung@njtech.edu.cn" ]
- 基金信息：
  
  国家自然科学基金资助项目(61672279);江苏省“六大人才高峰”项目资助(2012-WLW-023);水文水资源与水利工程科学国家重点实验室开放基金资助项目(2016491411)
- DOI：10.3788/OPE.20192708.1836
  中图分类号： TP391.4
- 收稿日期：2018-12-11，
  
  录用日期：2019-2-21，
  
  纸质出版日期：2019-08-15
- 稿件说明：
移动端阅览
杜振龙, 沈海洋, 国美, 等. 基于改进CycleGAN的图像风格迁移[J]. 光学精密工程, 2019,27(8):1836-1844.

Zhen-long GANDU, Hai-yang SHEN, Guo-mei SONG, et al. Image styletransfer based on improved Cycle[J]. Optics and precision engineering, 2019, 27(8): 1836-1844.
杜振龙, 沈海洋, 国美, 等. 基于改进CycleGAN的图像风格迁移[J]. 光学精密工程, 2019,27(8):1836-1844. DOI： 10.3788/OPE.20192708.1836.

Zhen-long GANDU, Hai-yang SHEN, Guo-mei SONG, et al. Image styletransfer based on improved Cycle[J]. Optics and precision engineering, 2019, 27(8): 1836-1844. DOI： 10.3788/OPE.20192708.1836.

摘要

图像风格迁移是用风格图像对指定图像的内容进行重映射，利用GAN自动进行图像风格迁移，可减少工作量，且结果丰富。特定情况下GAN方法所用的配对数据集很难获得。为了避免利用传统GAN进行图像风格迁移受到成对数据集的限制，提高风格迁移效率，本文利用改进的循环一致性对抗网络CycleGAN实现图像风格迁移，用密集连接卷积网络DenseNet代替原来网络生成器的深度残差网络ResNet，用同一映射损失和感知损失组成的损失函数度量风格迁移损失。所做改进使网络性能得到了提升，取消了网络对成对样本的限制，提高了风格迁移生成图像的质量。同时进一步提高了稳定性，加快了网络收敛速度。论文所提方法对建筑图像进行了风格迁移，实验结果表明，生成图像的PSNR值平均提高了6.27%，SSIM值均提高了约10%。因此，本文提出的改进的CycleGAN图像风格迁移方法生成的风格图像效果更优。

Abstract

Image style transfer exploits a specified style to modify given image content. An automatic image style transfer based on a Generative Adversarial Network (GAN) can reduce the workload and yield rich results. In some cases

the pair datasets required by the classical GAN were difficult to obtain. To overcome the limitations of paired datasets by a traditional GAN and improve the efficiency of style transfer

this study proposed an image style transfer method based on an improved Cycle-consistent adversarial network (CycleGAN). In this study

the deep residual network adopted by the conventional network generator was replaced by the dense connection convolution network

and a novel loss function composed of the same mapping and perceptual losses was used to measure the style transfer loss. These improvements were shown to increase the network performance

overcome the network's limitations on paired samples

and improve the quality of images generated by style migration. In addition

the stability was further improved and the network convergence speed was accelerated. Experiments demonstrate that the peak signal-to-noise ratio of the image generated by the proposed method increase 6.27% on average

where as the structural similarity index measure increased by approximately 10%. The improved CycleGAN image style transfer method proposed in this study can thus generate better style images.

关键词

Keywords

references

LECUN Y, BENGIO Y, HINTON G. Deep learning[J]. Nature , 2015, 521(7553): 436-444.

吕晓琪, 吴凉, 谷宇, 等.基于三维卷积神经网络的低剂量CT肺结节检测[J].光学精密工程, 2018, 26(5): 1211-1218.

LV X Q, WU L, GU Y, et al .. Detection of low dose CT pulmonary nodules based on 3D convolution neural network[J]. Opt. Precision Eng ., 2018, 26(5): 1211-1218. (in Chinese)

廖欣, 郑欣, 邹娟, 等.基于神经网络集成模型的宫颈细胞病理计算机辅助诊断方法[J].液晶与显示, 2018, 33(4): 347-356.

LIAO X, ZHEN X, ZOU J, et al .. Intelligent auxiliary diagnosis method of cervical cytopathology based on deep convolutional neural networks[J]. Chinese Journal of Liquid Crystals and Displays , 2018, 33(4): 347-356. (in Chinese)

李宇, 刘雪莹, 张洪群, 等.基于卷积神经网络的光学遥感图像检索[J].光学精密工程, 2018, 26(1): 200-207.

LI Y, LIU X Y, ZHANG H Q, et al .. Optical remote sensing image retrieval based on convolutional neural networks[J]. Opt. Precision Eng ., 2018, 26(1): 200-207. (in Chinese)

林金花, 王延杰.三维语义场景复原网络[J].光学精密工程, 2018, 26(5): 1231-1241.

LIN J H, WANG Y J. Three-dimentional reconstruction of semantic scene based on RGB-D map[J]. Opt. Precision Eng ., 2018, 26(5): 1231-1241. (in Chinese)

GATYS L A, ECKER A S, BETHGE M. Image style transfer using convolutional neural networks[C]. Proceedings of the 29 th IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA : CVPR , 2016: 2414-2423.

JOHNSON J, ALAHI A, LI F. Perceptual losses for real-time style transfer and super-resolution[C]. Proceedings of the 14 th European Conference on Computer Vision, Amsterdam, Netherlands : ECCV , 2016: 694-711.

DANIEL H, IKHSANUL H, IKUO K, et al .. Fast neural style transfer for motion data[J]. IEEE Computer Graphics and Applications , 2017, 37(4):42-49.

MICHAE E, PEYMAN M. Style transfer via texture synthesis[J]. IEEE Transactions on Image Processing , 2017, 26(5): 2338-2351.

MANUEL R, ALEXEY D, THOMAS B. Artistic style transfer for videos and spherical images[J]. International Journal of Computer Vision , 2018, 126(11):1199-1219.

杜振龙, 赵中武, 李晓丽, 等.飞人:利用基灵矢量场的图像高保真变形[J].计算机辅助设计与图形学学报, 2016, 28(8): 1333-1340.

DU ZH L, ZHAO ZH W, LI X L, et al .. Flying people: high-fidelity image warping via killing vector fields[J]. Journal of Computer-Aided Design and Computer Graphics , 2016, 28(8): 1333-1340.

ISOLA P, ZHU J Y, ZHOU T, et al .. Image-to-image translation with conditional adversarial networks[C]. Proceedings of the 30 th IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, USA : CVPR , 2017: 1125-1134.

林懿伦, 戴星原, 李力, 等.人工智能研究的新前线:生成式对抗网络[J].自动化学报, 2018, 44(5): 775-792.

LIN Y L, DAI X Y, LI L, et al .. The new frontier of AI research: generative adversarial networks[J]. Acta Automatica Sinica , 2018, 44(5): 775-792. (in Chinese)

王坤峰, 苟超, 段艳杰, 等.生成式对抗网络GAN的研究进展与展望[J].自动化学报, 2017, 43(3): 321-332.

WANG K F, GOU CH, DUAN Y J, et al .. Generative adversarial networks: the state of the art and beyond[J]. Acta Automatica Sinica , 2017, 43(3): 321-332. (in Chinese)

王功明, 乔俊飞, 王磊.一种能量函数意义下的生成式对抗网络[J].自动化学报, 2018, 44(5): 793-803.

WANG G M, QIAO J F, WANG L. A generative adversarial network based on energy function[J]. Acta Automatica Sinica , 2018, 44(5): 793-803. (in Chinese)

RADFORD A, METZ L, CHINTALA S. Unsupervised representation learning with deep convolutional generative adversarial networks[C]. Proceedings of the 4 th International Conference on Learning Representations, Puerto Rico, USA : ICLR , 2016: 580-596.

ZHU J Y, PARK T, ISOLA P, et al .. Unpaired image-to-image translation using cycle-consistent adversarial networks[C]. Proceedings of the 16 th IEEE International Conference on Computer Vision, Venice, Italy : ICCV , 2017: 2223-2232.

HE K, ZHANG X, REN S, et al .. Deep residual learning for image recognition[C]. Proceedings of the 29 th IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA : CVPR , 2016: 770-778.

LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation[C]. Proceedings of the 28 th IEEE Conference on Computer Vision and Pattern Recognition, Boston, USA : CVPR , 2015: 3431-3440.

HUANG G, LIU Z, MAATEN L, et al .. Densely connected convolutional networks[C]. Proceedings of the 30 th IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, USA : CVPR , 2017: 4700-4708.

ZHANG ZH CH, LIANG X K, DONG X, et al .. A sparse-view CT reconstruction method based on combination of densenet and deconvolution[J]. IEEE Transactions on Medical Imaging , 2018, 37(6): 1407-1417.

IOFFES, SZEGEDY C. Batch normalization: accelerating deep network training by reducing internal covariate shift[C]. Proceedings of the 32 th International Conference on Machine Learning, Lille, France : ICML , 2015: 448-456.

GLOROT X, BORDES A, BENGIO Y. Deep sparse rectifier neural networks[C]. Proceedings of the 14 th International Conference on Artificial Intelligence and Statistics, Fort Lauderdale, USA : AISTATS , 2011: 315-323.

ZEILER M D, KRISHNAN D, TAYLOR G W, et al .. Deconvolutional networks[C]. Proceedings of the 23 th IEEE Conference on Computer Vision and Pattern Recognition, San Francisco, USA : CVPR , 2010: 2528-2535.

YU X, FATIH P. Imagining the unimaginable faces by deconvolutional networks[J]. IEEE Transactions on Image Processing , 2018: 2747-2761.

ARJOVSKY M, CHINTALA S, BOTTOU L. Wasserstein generative adversarial networks[C]. Proceedings of the 34 th International Conference on Machine Learning, Sydney, Australia : ICML , 2017: 214-233.

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于改进CycleGAN的图像风格迁移-第五届新型光电探测技术及其应用研讨会推荐

非成对训练样本条件下的红外图像生成

基于多尺度密集特征融合的单图像翻译