STYLE TRANSFER Three Parts 1 Introduction to the

风格迁移 STYLE TRANSFER 李克钹

Three Parts: 1. Introduction to the topic 2. Algorithm Principle 3. Reference 2

Introduction to the topic 3

Algorithm Principle • Convolutional neural network (CNN) • Gated-GAN 4

CNN • Use a VGG 19 model to be the network structure • The loss of the net has two parts: style loss and content loss • The loss of style use Gram matrix to describe • Content loss calculation generates the squared loss function between the output response of the image at the high level and the output response of the content image at the high level. 5

CNN 6

CNN 7

CNN Gram Matrix: The matrix formed by the inner product of two pairs of any k vectors in an n-dimensional Euclidean space is called the Gram matrix of these k vectors. What can Gram Matrix do in Style Transfer ? Quantitative the style differences. 8

CNN In style transfer, a Gram Matrix can be regarded as an eccentric covariance Matrix between the features of an image (that is, a covariance Matrix without subtracting the mean). Gram calculates the correlation between two features, which two features appear at the same time, which two are complementary and so on. 9

CNN Calculate the Gram Matrix in the lth layer, which can be on behalf of the related degree among every feature map Calculate the style loss in the lth layer Calculate the total style loss wl are weighting factors of the contribution of each layer to the total loss 10

CNN Calculate the content function, with the squared-error loss between the two feature representations But this way is too slow!! + || α and β are the weighting factors for content and style reconstruction, respectively. 11

CNN MSE loss -> perceptual loss Use perceptual loss to replace mse loss, which can generate the picture more quickly and natural 12

CNN Content Loss Function: Gram Matrix: Style Loss Function: With the model, we can improve Three orders of magnitude at time. 13

Gated-GAN 14

Gated-GAN 15

Gated-GAN The original generative adversarial value function The total loss of generator The total loss of discriminator 16

Gated-GAN The adversarial loss LGAN (G, D)(to generate realistic images) Classifier loss(to overcome the overlaid-gradient problem ) The total variation loss(to smooth the generated image) 17

Gated-GAN 18

Reference • Gatys L A , Ecker A S , Bethge M. A Neural Algorithm of Artistic Style[J]. Computer Science, 2015. • Gatys L A , Ecker A S , Bethge M. Image Style Transfer Using Convolutional Neural Networks[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2016. • Johnson J , Alahi A , Fei-Fei L. Perceptual Losses for Real-Time Style Transfer and Super-Resolution[J]. 2016. • Shen F , Yan S , Zeng G. Meta Networks for Neural Style Transfer[J]. 2017. • Chen X , Xu C , Yang X , et al. Gated-GAN: Adversarial Gated Networks for Multi. Collection Style Transfer[J]. IEEE Transactions on Image Processing, 2018: 1 -1. • https: //blog. csdn. net/u 014636245/article/details/88065269 19

DOWNLOADS at http: //vcc. szu. edu. cn Thank You!