Lecture 6 1 Generative Adversarial Network 2020 Photo

本科生课程：统计计算与机器学习 Lecture 6 -1: Generative Adversarial Network ▪ 许志钦 ▪ 2020年春

艺术是什么？ Photo via Art and Artificial Intelligence Laboratory, Rutgers University

▪ https: //github. com/Aixile/chainer-cyclegan

https: //becominghuman. ai/generative-adversarial-networks-gans-human-creativity 2 fc 61283 f 3 f 6

[Jun-Yan Zhu, et al. , ICCV, 2017]

Basic idea of GAN

Generation Image Generation NN Generator In a specific range Sentence Generation NN Generator Slides from 李宏毅 How are you? Good morning. Good afternoon.

Powered by: http: //mattya. github. io/chainer-DCGAN/ Basic Idea of GAN vector Generator Each dimension of input vector represents some characteristics. Generator Slides from 李宏毅 blue hair It is a neural network (NN), or a function. image Generator Longer hair Generator Open mouth high dimensional vector

Basic Idea of GAN Discriminator image Slides from 李宏毅 Discriminator It is a neural network (NN), or a function. scalar Larger value means real, smaller value means fake. 1. 0 Discriminator 1. 0 0. 1 Discriminator 0. 1

Basic Idea of GAN Generator Brown Butterflies are not brown Butterflies do not have veins ……. . Discriminator Slides from 李宏毅

This is where the term “adversarial” comes from. You can explain the process in different ways……. Basic Idea of GAN Slides from 李宏毅 NN Generator v 1 NN Generator v 2 NN Generator v 3 Discriminator v 1 Discriminator v 2 Discriminator v 3 Real images:

https: //stats. stackexchange. com/questions/277756/some-general-questions-on-generative-adversarial-networks

Algorithm • Learning D Learning G Slides from 李宏毅

• Given G, what is the optimal D* maximizing Assume that D(x) can be any function • Given x, the optimal D* maximizing Slides from 李宏毅

• Given x, the optimal D* maximizing a D b D • Find D* maximizing: Slides from 李宏毅 0< <1

2 Slides from 李宏毅 2

Jensen-Shannon divergence Slides from 李宏毅

[Goodfellow, et al. , NIPS, 2014]

[Goodfellow, et al. , NIPS, 2014] Rightmost column shows the nearest training example of the neighboring sample, in order to demonstrate that the model has not memorized the training set.

linearly interpolating between coordinates [Goodfellow, et al. , NIPS, 2014]

All Kinds of GAN … https: //github. com/hindupuravinash/the-gan-zoo GAN ACGAN BGAN CGAN DCGAN EBGAN f. GAN Go. GAN …… Mihaela Rosca, Balaji Lakshminarayanan, David Warde-Farley, Shakir Mohamed, “Variational Approaches for Auto. Encoding Generative Adversarial Networks”, ar. Xiv, 2017 Slides from 李宏毅

Conditional GAN

[Scott Reed, et al, ICML, 2016] Conditional GAN c: train G Image x = G(c, z) scalar x is realistic or not + c and x are matched or not Normal distribution D (better) True text-image pairs: (train , (cat , Slides from 李宏毅 ) 0 (train , ) 1 ) 0

Conditional GAN - Discriminator object x Network condition c Network (almost every paper) object x Slides from 李宏毅 x is realistic or not Network condition c [Augustus Odena et al. , ICML, 2017] [Takeru Miyato, et al. , ICLR, 2018] [Han Zhang, et al. , ar. Xiv, 2017] score x is realistic or not + c and x are matched or not Network c and x are matched or not

Conditional GAN paired data blue eyes red hair short hair red hair, green eyes blue hair, red eyes Slides from 李宏毅 The images are generated by Yen-Hao Chen, Po-Chun Chien, Jun-Chen Xie, Tsung -Han Wu. Collecting anime faces and the description of its characteristics

Cycle GAN

[Jun-Yan Zhu, et al. , ICCV, 2017]

Upair training [Jun-Yan Zhu, et al. , ICCV, 2017]

[Jun-Yan Zhu, et al. , ICCV, 2017] Direct Transformation as close as possible Cycle consistency Lack of information for reconstruction scalar Input image belongs to domain Y or not Slides from 李宏毅 Domain Y

Disco GAN Dual GAN [Taeksoo Kim, et al. , ICML, 2017] [Zili Yi, et al. , ICCV, 2017] Cycle GAN [Jun-Yan Zhu, et al. , ICCV, 2017]