Style Transfer GAN 0229 Network Structure The network

Style Transfer GAN 02/29

Network Structure • The network structure is shown in the right figure. • The generation of high-quality photos are based on progressive GAN. • We expect that the progressive GAN is already able to handle high-quality photos’ generation and the style transfer GAN furthermore improve the performance.

Progressive GAN • Progressive GAN is a very powerful structure. • Potential problems of traditional GAN: • Unstable training process, especially with many layers • A considerable artifact when generating details • Solution for progressive GAN: • Try generating images from low to high resolutions • More semantic meaningful • More stable since each scale is supervised by GAN loss • Feature vector normalization • A more reasonable regularization term for generation

Styles & Mapping • Based on Progressive GAN, the style transfer GAN improves so that the input feature vectors are concatenated into intermediate layers. • Different layers are responsible for different sorts of information • Deeper layers -> lower resolution -> semantic information -> style/race/gender… • Shallower layers -> higher resolution -> texture information -> colour/edge/lighting… • The FC Network transfers the latent vector z to the styles and inserted into different layers for generation. • Remove the input latent vector for the first layer.

Noise input & mixing regularization (minor) • Adding noise will enhance the robustness of model in dealing with different inputs. • Adaptive instance normalization will make each output more stable (replacement of Batch. Norm)

Analysis & discussions about the model • Advantages: • The generation quality is very impressive • The progressive process will largely stabilize the training, and we need less worries about tuning the network structures. • Worries: • The model still has the possibility of overfitting. • The training process takes a long time.