Generative Adversarial Networks (GAN) is one a Neural Network architecture which simulates zero-sum game. There are 2 parts of this Neural Network. The first is called Generator and other is called Discriminator. Generator tries to mimic data and make the fake data likes the real data in distribution. Meanwhile, the Discriminator tries to maximize the difference between real data and fake data. It is reason we call zero-sum game. Two parts are coaction with each other. This structure makes the GAN to be a interesting Neural Network architecture and it has many application in both academic and industry.
In modeling, the GAN is an approach of equilibrium Networks such as Boltzmann Machine did. It is an optimization problem with objectives of: minimize Generator and maximum Discriminator simultaneously.
$max_{D}min_{G} V(D,G)$
$max_{D}min_{G} V(D,G) = E_{x\sim p_{data}(x)}[log(D(x))] + E_{z\sim p_{z}(z)}[log(1 - D(G(z)))]$
$V(D,G)$ is optimization problem subject to G and D.
$E$ means the expectation or training in long-term till get equilibrium.
To learn the generator's distribution $p_g$ over data x, we define a prior on input noise variable $p_z(z)$ and present by differentiable function $G(z, \theta_g)$. We also defind $D(z, \theta_x)$ that output the single scalar.
$x$ has distribution based on real data and $z$ is hidden parameters. In the beginning, D is easier to reject fake sample with high confident because they are clearly different from training data. In the case log(1-D(G(z))) saturates, Generator cannot be better, we train for log(D(z)) to maximum likelihood function.(x includes fake data here).
Algorithm pseudo code
Implementation
References:
https://papers.nips.cc/paper/5423-generative-adversarial-nets.pdf
https://arxiv.org/pdf/1609.03126.pdf
https://medium.com/ai-society/gans-from-scratch-1-a-deep-introduction-with-code-in-pytorch-and-tensorflow-cb03cdcdba0f
$max_{D}min_{G} V(D,G)$
$max_{D}min_{G} V(D,G) = E_{x\sim p_{data}(x)}[log(D(x))] + E_{z\sim p_{z}(z)}[log(1 - D(G(z)))]$
$V(D,G)$ is optimization problem subject to G and D.
$E$ means the expectation or training in long-term till get equilibrium.
To learn the generator's distribution $p_g$ over data x, we define a prior on input noise variable $p_z(z)$ and present by differentiable function $G(z, \theta_g)$. We also defind $D(z, \theta_x)$ that output the single scalar.
$x$ has distribution based on real data and $z$ is hidden parameters. In the beginning, D is easier to reject fake sample with high confident because they are clearly different from training data. In the case log(1-D(G(z))) saturates, Generator cannot be better, we train for log(D(z)) to maximum likelihood function.(x includes fake data here).
Algorithm pseudo code
Implementation
References:
https://papers.nips.cc/paper/5423-generative-adversarial-nets.pdf
https://arxiv.org/pdf/1609.03126.pdf
https://medium.com/ai-society/gans-from-scratch-1-a-deep-introduction-with-code-in-pytorch-and-tensorflow-cb03cdcdba0f
Comments