ML style transfer outline CNN Visualization Neural Art

ML style transfer

outline • CNN Visualization • Neural Art

Padding, Stride, color

Hyperparameter, featureimage

Hyperparameter

Visual Perception of Computer

Visualization

Visualizing CNN http: //vision 03. csail. mit. edu/cnn_art/data/single_layer. png 10

Visualizing CNN flower random noise CNN filter response 11

Gradient Ascent 12

Gradient Ascent 13

Gradient Ascent 80

Different Layers of Visualization CNN 15

Multiscale Image Generation visualize resize visualize 16

Multiscale Image Generation 17

Deep Dream • https: //research. googleblog. com/2015/06/inceptionism- going-deeper-intoneural. html • Source code: https: //github. com/tensorflow/blob/master/tensorflow/ex amples/tutorials/deepdream. ipynb http: //download. tensorflow. org/example_images /flower_photos. tgz 18

Deep Dream 19

Deep Dream 20

Neural Art

Hybrid Images [siggraph 06]

Hybrid Images

VGGNet (2014) • Paper: https: //arxiv. org/abs/1409. 1556

VGG 19

Neural Art • Paper: https: //arxiv. org/abs/1508. 06576 • Source code : https: //github. com/ckmarkoh/neuralart_tensorflow content artwork ht tp: //www. taipei 101. com. tw/upload/news/201502/2015 021711505431705145. J PG style ht tps: //github. com/andersbll/neural_ar tistic_style/blob/master/images/starry_ night. jpg? raw=true 26

The Mechanism of Painting Artist Scene Brain Style Computer Art. Work Neural Networks 27

Content Generation Content Artist Brain Neural Stimulation Minimize the difference Canvas Draw 28

Content Generation Content VGG 19 Width*Height Filter Responses Depth Minimize the difference Canvas Result Update the color of the pixels 29

Content Generation Layer l’s Filter Responses: Input Canvas: Layer l’s Filter l Responses: Depth (i) Input Photo: Width*Height (j) 30

Content Generation • Backward Propagation Input Canvas: VGG 19 Layer l’s Filter l Responses: Update Canvas Learning Rate 31

Content Generation 32

Content Generation VGG 19 conv 1_2 conv 2_2 conv 3_4 conv 4_4 conv 5_1 conv 5_2 33

Style Generation Artwork VGG 19 Filter Responses Gram Matrix G Positiondependent Depth Width*Height G Depth Positionindependent 34

Style Generation Layer l’s Filter Responses Gram Matrix Width*Height 1. . 5 Depth . 5 1. k . 5 . 25 1. . 5 Depth G . 5 1. . 25. 5 1. Depth 35

Style Generation Input Artwork: Layer l’s Gram Matrix Input Canvas: Layer l’s Gram Matrix Layer l’s Filter Responses 36

Style Generation Style VGG 19 Gram Matrix Filter Responses G Minimize the difference Canvas G Result Update the color of the pixels 100

Style Generation 38

Style Generation VGG 19 Conv 1_1 Conv 2_1 Conv 3_1 Conv 4_1 Conv 1_1 Conv 2_1 Conv 3_1 Conv 4_1 Conv 5_1 39

Artwork Generation VGG 19 Filter Responses Gram Matrix 40

Artwork Generation VGG 19 Conv 4_2 Conv 1_1 Conv 2_1 Conv 3_1 Conv 4_1 Conv 5_1 41

Artwork Generation 42

Content v. s. Style 0. 15 0. 02 0. 007 43

Neural Doodle • Image analogy 44

Neural Doodle • Paper: https: //arxiv. org/abs/1603. 01768 • Source code: https: //github. com/alexjc/neural-doodle content semantic maps result style 45

Neural Doodle • Image analogy 46

Real-time Texture Synthesis • Paper: https: //arxiv. org/pdf/1604. 04382 v 1. pdf ◦ GAN: https: //arxiv. org/pdf/1406. 2661 v 1. pdf ◦ VAE: https: //arxiv. org/pdf/1312. 6114 v 10. pdf • Source Code : https: //github. com/chuanli 11/MGANs 110

Reference • 斎藤康毅 Deep Learning：用Python進行深度學習 的基礎理論實作 • https: //www. csie. ntu. edu. tw/~yvchen/f 105 adl/doc/161103_Convolutional. NN. pdf • https: //github. com/hunter-heidenreich/ML-Open. Source-Implementations/blob/master/Style. Transfer/Style%20 Transfer. ipynb