Image styletransfer based on improved Cycle

Zhen-long GANDU; Hai-yang SHEN; Guo-mei SONG; Xiao-li LI

doi:10.3788/OPE.20192708.1836

您当前的位置：

首页 >

文章列表页 >

Image styletransfer based on improved Cycle

Information Sciences | 更新时间：2020-08-13

- Image styletransfer based on improved Cycle
- Optics and Precision Engineering Vol. 27, Issue 8, Pages: 1836-1844(2019)
- 作者机构：
  
  南京工业大学计算机科学与技术学院，江苏南京 211816
- 作者简介：
  
  LI Xiao-li, E-mail: lixl@njtech.edu.cn
- 基金信息：
- DOI：10.3788/OPE.20192708.1836
  CLC： TP391.4
- Received：11 December 2018，
  
  Accepted：21 February 2019，
  
  Published：15 August 2019
- 稿件说明：
移动端阅览
Zhen-long GANDU, Hai-yang SHEN, Guo-mei SONG, et al. Image styletransfer based on improved Cycle[J]. Optics and precision engineering, 2019, 27(8): 1836-1844.
DOI：

Zhen-long GANDU, Hai-yang SHEN, Guo-mei SONG, et al. Image styletransfer based on improved Cycle[J]. Optics and precision engineering, 2019, 27(8): 1836-1844. DOI： 10.3788/OPE.20192708.1836.

摘要

图像风格迁移是用风格图像对指定图像的内容进行重映射，利用GAN自动进行图像风格迁移，可减少工作量，且结果丰富。特定情况下GAN方法所用的配对数据集很难获得。为了避免利用传统GAN进行图像风格迁移受到成对数据集的限制，提高风格迁移效率，本文利用改进的循环一致性对抗网络CycleGAN实现图像风格迁移，用密集连接卷积网络DenseNet代替原来网络生成器的深度残差网络ResNet，用同一映射损失和感知损失组成的损失函数度量风格迁移损失。所做改进使网络性能得到了提升，取消了网络对成对样本的限制，提高了风格迁移生成图像的质量。同时进一步提高了稳定性，加快了网络收敛速度。论文所提方法对建筑图像进行了风格迁移，实验结果表明，生成图像的PSNR值平均提高了6.27%，SSIM值均提高了约10%。因此，本文提出的改进的CycleGAN图像风格迁移方法生成的风格图像效果更优。

Abstract

Image style transfer exploits a specified style to modify given image content. An automatic image style transfer based on a Generative Adversarial Network (GAN) can reduce the workload and yield rich results. In some cases

the pair datasets required by the classical GAN were difficult to obtain. To overcome the limitations of paired datasets by a traditional GAN and improve the efficiency of style transfer

this study proposed an image style transfer method based on an improved Cycle-consistent adversarial network (CycleGAN). In this study

the deep residual network adopted by the conventional network generator was replaced by the dense connection convolution network

and a novel loss function composed of the same mapping and perceptual losses was used to measure the style transfer loss. These improvements were shown to increase the network performance

overcome the network's limitations on paired samples

and improve the quality of images generated by style migration. In addition

the stability was further improved and the network convergence speed was accelerated. Experiments demonstrate that the peak signal-to-noise ratio of the image generated by the proposed method increase 6.27% on average

where as the structural similarity index measure increased by approximately 10%. The improved CycleGAN image style transfer method proposed in this study can thus generate better style images.

关键词

Keywords

references

LECUN Y, BENGIO Y, HINTON G. Deep learning[J]. Nature , 2015, 521(7553): 436-444.

吕晓琪, 吴凉, 谷宇, 等.基于三维卷积神经网络的低剂量CT肺结节检测[J].光学精密工程, 2018, 26(5): 1211-1218.

LV X Q, WU L, GU Y, et al .. Detection of low dose CT pulmonary nodules based on 3D convolution neural network[J]. Opt. Precision Eng ., 2018, 26(5): 1211-1218. (in Chinese)

廖欣, 郑欣, 邹娟, 等.基于神经网络集成模型的宫颈细胞病理计算机辅助诊断方法[J].液晶与显示, 2018, 33(4): 347-356.

LIAO X, ZHEN X, ZOU J, et al .. Intelligent auxiliary diagnosis method of cervical cytopathology based on deep convolutional neural networks[J]. Chinese Journal of Liquid Crystals and Displays , 2018, 33(4): 347-356. (in Chinese)

李宇, 刘雪莹, 张洪群, 等.基于卷积神经网络的光学遥感图像检索[J].光学精密工程, 2018, 26(1): 200-207.

LI Y, LIU X Y, ZHANG H Q, et al .. Optical remote sensing image retrieval based on convolutional neural networks[J]. Opt. Precision Eng ., 2018, 26(1): 200-207. (in Chinese)

林金花, 王延杰.三维语义场景复原网络[J].光学精密工程, 2018, 26(5): 1231-1241.

LIN J H, WANG Y J. Three-dimentional reconstruction of semantic scene based on RGB-D map[J]. Opt. Precision Eng ., 2018, 26(5): 1231-1241. (in Chinese)

GATYS L A, ECKER A S, BETHGE M. Image style transfer using convolutional neural networks[C]. Proceedings of the 29 th IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA : CVPR , 2016: 2414-2423.

JOHNSON J, ALAHI A, LI F. Perceptual losses for real-time style transfer and super-resolution[C]. Proceedings of the 14 th European Conference on Computer Vision, Amsterdam, Netherlands : ECCV , 2016: 694-711.

DANIEL H, IKHSANUL H, IKUO K, et al .. Fast neural style transfer for motion data[J]. IEEE Computer Graphics and Applications , 2017, 37(4):42-49.

MICHAE E, PEYMAN M. Style transfer via texture synthesis[J]. IEEE Transactions on Image Processing , 2017, 26(5): 2338-2351.

MANUEL R, ALEXEY D, THOMAS B. Artistic style transfer for videos and spherical images[J]. International Journal of Computer Vision , 2018, 126(11):1199-1219.

杜振龙, 赵中武, 李晓丽, 等.飞人:利用基灵矢量场的图像高保真变形[J].计算机辅助设计与图形学学报, 2016, 28(8): 1333-1340.

DU ZH L, ZHAO ZH W, LI X L, et al .. Flying people: high-fidelity image warping via killing vector fields[J]. Journal of Computer-Aided Design and Computer Graphics , 2016, 28(8): 1333-1340.

ISOLA P, ZHU J Y, ZHOU T, et al .. Image-to-image translation with conditional adversarial networks[C]. Proceedings of the 30 th IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, USA : CVPR , 2017: 1125-1134.

林懿伦, 戴星原, 李力, 等.人工智能研究的新前线:生成式对抗网络[J].自动化学报, 2018, 44(5): 775-792.

LIN Y L, DAI X Y, LI L, et al .. The new frontier of AI research: generative adversarial networks[J]. Acta Automatica Sinica , 2018, 44(5): 775-792. (in Chinese)

王坤峰, 苟超, 段艳杰, 等.生成式对抗网络GAN的研究进展与展望[J].自动化学报, 2017, 43(3): 321-332.

WANG K F, GOU CH, DUAN Y J, et al .. Generative adversarial networks: the state of the art and beyond[J]. Acta Automatica Sinica , 2017, 43(3): 321-332. (in Chinese)

王功明, 乔俊飞, 王磊.一种能量函数意义下的生成式对抗网络[J].自动化学报, 2018, 44(5): 793-803.

WANG G M, QIAO J F, WANG L. A generative adversarial network based on energy function[J]. Acta Automatica Sinica , 2018, 44(5): 793-803. (in Chinese)

RADFORD A, METZ L, CHINTALA S. Unsupervised representation learning with deep convolutional generative adversarial networks[C]. Proceedings of the 4 th International Conference on Learning Representations, Puerto Rico, USA : ICLR , 2016: 580-596.

ZHU J Y, PARK T, ISOLA P, et al .. Unpaired image-to-image translation using cycle-consistent adversarial networks[C]. Proceedings of the 16 th IEEE International Conference on Computer Vision, Venice, Italy : ICCV , 2017: 2223-2232.

HE K, ZHANG X, REN S, et al .. Deep residual learning for image recognition[C]. Proceedings of the 29 th IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA : CVPR , 2016: 770-778.

LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation[C]. Proceedings of the 28 th IEEE Conference on Computer Vision and Pattern Recognition, Boston, USA : CVPR , 2015: 3431-3440.

HUANG G, LIU Z, MAATEN L, et al .. Densely connected convolutional networks[C]. Proceedings of the 30 th IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, USA : CVPR , 2017: 4700-4708.

ZHANG ZH CH, LIANG X K, DONG X, et al .. A sparse-view CT reconstruction method based on combination of densenet and deconvolution[J]. IEEE Transactions on Medical Imaging , 2018, 37(6): 1407-1417.

IOFFES, SZEGEDY C. Batch normalization: accelerating deep network training by reducing internal covariate shift[C]. Proceedings of the 32 th International Conference on Machine Learning, Lille, France : ICML , 2015: 448-456.

GLOROT X, BORDES A, BENGIO Y. Deep sparse rectifier neural networks[C]. Proceedings of the 14 th International Conference on Artificial Intelligence and Statistics, Fort Lauderdale, USA : AISTATS , 2011: 315-323.

ZEILER M D, KRISHNAN D, TAYLOR G W, et al .. Deconvolutional networks[C]. Proceedings of the 23 th IEEE Conference on Computer Vision and Pattern Recognition, San Francisco, USA : CVPR , 2010: 2528-2535.

YU X, FATIH P. Imagining the unimaginable faces by deconvolutional networks[J]. IEEE Transactions on Image Processing , 2018: 2747-2761.

ARJOVSKY M, CHINTALA S, BOTTOU L. Wasserstein generative adversarial networks[C]. Proceedings of the 34 th International Conference on Machine Learning, Sydney, Australia : ICML , 2017: 214-233.

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Image Style Transfer Based on Improved CycleGAN

Infrared image generation with unpaired training samples

Single-image translation based on multi-scale dense feature fusion

Related Author

DU Zhenlong SHEN Haiyang SONG Guome LI Xiaoli

CAI Wei

JIANG Bo

JIANG Xinhao

YANG Zhiyong

Qihang LI

Long FENG

Qing YANG

Related Institution

School of Computer Science and Technology， Nanjing TECH University

Armament Launch Theory and Technology Key Discipline Laboratory of PRC， Rocket Force University of Engineering， Xi′an

School of Information Science and Technology， Northwest University， Xi ’an

School of Mathematics， Northwest University， Xi ’an

AI问答

Address：No.3888 Dong Nanhu Road, Changchun, Jilin, China Postal code：130033
Tel：0431-86176855 Email：gxjmgc@ciomp.ac.cn
Technical support is provided by Beijing Founder electronics co., LTD 吉ICP备11002662号-17 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰