GANs — Generator vs Discriminator

A generative adversarial network (GAN) trains two networks against each other. A classifier D(x) learns to tell real images from fakes, and a generator G(z) learns to fool D. The loss signal for G is whatever D currently thinks makes something look real. That signal shifts as G improves, so G chases a moving target. If both networks converge, G has learned the data distribution without anyone writing down log p(x).

Here the starting point is the VAE's blur. A VAE's MSE decoder loss is Bayes-optimal for the mean image, and the mean of many plausible digits is a fuzzy digit. You want a loss that rewards plausibility instead of pixel-wise proximity to any one target. Plausibility has no closed form, so you learn it. Goodfellow's idea was to let the classifier be that learned loss.

The math is a minimax game:

min_G max_D  E_real[log D(x)] + E_fake[log(1 - D(G(z)))]

In 2026 GANs are no longer the SOTA generator, since diffusion and flow matching took that crown. StyleGAN 2/3 still rank among the sharpest face models, though, GAN discriminators serve as perceptual losses in diffusion training, and adversarial training powers the fast 1-step distillations (SDXL-Turbo, SD3-Turbo, LCM) that let you ship real-time diffusion.

Generator G(z) maps a noise vector z ~ N(0, I) to a sample x̂. It's a decoder-shaped network (dense layers in the toy script, transposed convs in DCGAN-style image models).

Discriminator D(x) maps a sample to a scalar probability (or score). Real should score 1, fake 0.

Loss. Two alternating updates:

• Train D: loss_D = -[ log D(x) + log(1 - D(G(z))) ]. Binary cross-entropy with real=1 and fake=0.
• Train G: loss_G = -log D(G(z)). This is the non-saturating form Goodfellow used. The original log(1 - D(G(z))) saturates and kills gradients when D is confident.

Training loop. One step of D, then one step of G, repeated.

Why it works. If G perfectly matches p_data, D can't do better than chance and outputs 0.5 everywhere, and G gets no more gradient. That's the equilibrium.

Why it breaks. Mode collapse: G finds one mode D can't classify and keeps producing it. Vanishing gradient: D learns too fast and log D saturates. Training instability: small changes to learning rates or batch sizes can tip the whole run.

code/main.py trains a tiny GAN on 1-D data drawn from a mixture of two Gaussians. Generator and discriminator are single-hidden-layer MLPs, and the forward pass, backward pass, and minimax loop are all written by hand. Every 100 steps the script prints the mean D(real) and D(fake) plus how many probe samples land in each mode, and it flags mode collapse when one mode is starved. It does not log losses or detect vanishing gradients, so you infer that failure from D(real) and D(fake) staying pinned.

Step 1: non-saturating loss

The vanilla Goodfellow loss log(1 - D(G(z))) goes to 0 when D classifies G's fake as fake with high confidence. G's gradient is basically zero at that point, so G can't improve. The non-saturating form -log D(G(z)) has the opposite asymptote: it blows up when D is confident, which gives G a strong signal.

main.py never computes this loss as a function. Excerpted from update_g, it starts from the analytic gradient directly:

# L = -log(p) with p = D(G(z)), so dL/dpre2 = -(1/p) * p*(1-p) = p - 1
dL_dpre2 = p - 1.0

When D is confident the fake is fake, p is near 0 and this gradient is near -1, a strong push on G.

Step 2: one discriminator step per generator step

for step in range(1, 801):
    # train D
    reals = sample_real(batch, rng)
    noise = sample_noise(batch, z_dim, rng)
    fakes = [forward_g(z, G)[0] for z in noise]
    update_d(reals, fakes, D, d_lr)
 
    # train G
    noise = sample_noise(batch, z_dim, rng)  # fresh noise
    update_g(noise, G, D, g_lr)

update_g takes fresh noise and reruns G's forward pass itself, so its gradients are never stale.

Step 3: watch for mode collapse

if step % 100 == 0:
    probe_fakes = [forward_g(z, G)[0][0] for z in sample_noise(400, z_dim, rng)]
    mode_a = sum(1 for v in probe_fakes if v < 0)
    mode_b = 400 - mode_a
    if min(mode_a, mode_b) < 50:
        print("  [!] mode collapse")

The canonical symptom is that one of the two real modes stops being generated. The discriminator stops correcting it because it doesn't see that mode among the fakes.

• Discriminator too strong. Cut D's learning rate by 2-5x, or add instance/layer noise. If D(real) sits near 1 and D(fake) near 0 and stays there, G is dead.
• Generator memorizes a mode. Add noise to D inputs, use a minibatch-discriminator layer, or switch to WGAN-GP.
• Batch norm leaking statistics. A real batch and a fake batch flowing through the same BN layer mix their statistics. Use instance norm or spectral norm instead (main.py has no norm layers, so you won't see this one in the toy run).
• Inception-score gaming. FID and IS are noisy at low sample counts. Use at least 10k samples at eval.
• One-shot sampling is a lie for conditional tasks. You still need CFG scales, truncation tricks, and re-sampling to get usable outputs.

The 2026 GAN stack:

GANs are sharp but narrow. Once your domain opens up (photos, arbitrary text prompts, video), switch to diffusion. The adversarial trick survives as a component (perceptual losses, distillation), not as a standalone generator.

Save outputs/skill-gan-debugger.md. The skill takes a failing GAN run (loss curves, sample grid, dataset size) and outputs a ranked list of likely causes, one-line fixes, and a rerun protocol.

1. Easy. Run code/main.py with the stock settings. Then in main() change d_lr from 0.01 to 5 * g_lr and rerun. Watch D(real), D(fake) and the mode counts: does D(fake) get pinned near 0, and how quickly?
2. Medium. Replace the Goodfellow BCE loss with the WGAN loss: loss_D = E[D(fake)] - E[D(real)], loss_G = -E[D(fake)], and clip D's weights to [-0.01, 0.01]. Is training more stable? Compare wall-clock convergence.
3. Hard. Extend the 1-D example to 2-D data (mixture of 8 Gaussians on a ring). Track how many of the 8 modes the generator captures at steps 1k, 5k, 10k. Implement minibatch discrimination and re-measure.

GANs no longer win on sample quality for open-domain generation, but they still win on inference cost. In production-inference terms, a GAN has:

• No prefill and decode stages. Generation is a single G(z) forward pass, so TTFT ≈ total latency.
• No KV-cache pressure. The only state is the weights. Batch size is bounded by activation memory, not cache.
• Trivial continuous batching. Each request takes the same fixed FLOPs, so a static batch at the server's target occupancy is usually optimal. No in-flight scheduler needed.

That's why GAN distillation (SDXL-Turbo, SD3-Turbo, ADD, LCM) is the dominant technique for fast text-to-image in 2026: it collapses a 20-50-step diffusion pipeline into 1-4 GAN-style forward passes while keeping the distribution of a diffusion base. The adversarial loss survives as a training-time knob for turning slow generators into fast ones.

• Goodfellow et al. (2014). Generative Adversarial Nets: the original GAN paper.
• Radford et al. (2015). Unsupervised Representation Learning with DCGAN: the first stable architecture.
• Arjovsky, Chintala, Bottou (2017). Wasserstein GAN: WGAN.
• Miyato et al. (2018). Spectral Normalization for GANs: SN.
• Karras et al. (2020). Analyzing and Improving the Image Quality of StyleGAN: StyleGAN2.
• Karras et al. (2021). Alias-Free Generative Adversarial Networks: StyleGAN3.
• Sauer et al. (2023). Adversarial Diffusion Distillation: SDXL-Turbo.