StyleGAN

StyleGAN is a GAN generator that maps z to an intermediate w first, then injects w at each resolution level through AdaIN, instead of feeding z straight into the convolutions.

The motivation is entanglement. A DCGAN maps z to an image through a stack of transposed convolutions, and z controls pose, lighting, identity, and background all at once. Move along one axis of z and all four change, so you can't ask the model for "same person, different pose." The representation doesn't factor that way.

Karras et al. (2019, NVIDIA) changed the plumbing. The network starts from a constant 4×4×512 tensor, and, in the published model, an 8-layer MLP maps z ∈ Z to w ∈ W. Each resolution gets w through adaptive instance normalization (AdaIN): normalize each conv feature map, then scale and shift it with affine projections of w. Per-layer noise adds stochastic detail like skin pores and hair strands.

The result is that W has roughly orthogonal axes for high-level style (pose, identity) versus fine style (lighting, color). To swap styles between two images, use image A's w for the low-resolution levels and image B's w for the high ones. That made editing, cross-domain stylization, and the StyleGAN-inversion line of research possible.

Mapping network. f: Z → W, an 8-layer MLP in the published model, with Z = N(0, I)^512. W isn't forced to be Gaussian. It learns a data-adapted shape.

Synthesis network. In the published model it starts from a learned constant 4×4×512. Each resolution block runs: upsample → conv → AdaIN(w_i) → noise → conv → AdaIN(w_i) → noise. The resolution doubles from block to block: 4, 8, 16, 32, 64, 128, 256, 512, 1024.

AdaIN.

AdaIN(x, y) = y_scale · (x - mean(x)) / std(x) + y_bias

where, in the published model, y_scale and y_bias come from per-channel affine projections of w. Normalize per feature map, then restyle. "Style" here means the first- and second-order statistics of the feature map.

Per-layer noise. Single-channel Gaussian noise is added to each feature map and scaled by a learned per-channel factor in the published model. It controls stochastic detail and leaves global structure alone.

Truncation trick. At inference, sample z, compute w = mapping(z), then w' = ŵ + ψ·(w - ŵ), where ŵ is the mean w over many samples. ψ < 1 trades diversity for quality. Most StyleGAN demos use ψ ≈ 0.7.

What the toy in main.py does instead. It keeps the data flow and drops almost everything else. The mapping network is 4 layers (depth=4), not 8, with z_dim = w_dim = 8. The constant is a 6-vector drawn once from gauss(0, 0.3), and nothing is ever trained. There are three synthesis blocks, each a matmul, a leaky ReLU, AdaIN, and noise, with no upsampling, convolutions, or resolution ladder. AdaIN's scale and bias are each one scalar per block, computed as a dot product of w with a fixed random vector, and the noise sigma is a single fixed scalar. Everything above in the published-model wording (8 layers, 512-d Z, 4×4×512, resolutions 4 to 1024, learned constant, per-channel parameters) describes the real StyleGAN, not the toy.

In 2026 StyleGAN3 remains the default for three jobs: (a) narrow-domain photorealism at high FPS, (b) few-shot domain adaptation (train on a new dataset with 100 images, freeze mapping), and (c) inversion-based editing (find the w that reconstructs a real photo, then edit that w). For open-domain text-to-image, reach for diffusion instead.

code/main.py implements a toy "style-GAN lite" in 1-D with three parts: a mapping MLP, a synthesis function that takes a constant vector (drawn once, never trained) and modulates it with w-derived scale/bias, and per-layer noise. The toy shows the mechanics of mapping, AdaIN modulation, truncation, and noise. It has no concatenation baseline and does not train anything, so it can't tell you whether modulating with w beats concatenating z.

The snippets below are illustrative. They show the idea in a compact form and don't match main.py's signatures exactly: there, mapping(z, layers) takes a list of (W, b) pairs, adain(features, scale, bias) uses a mean_std helper, and noise is added inline in stylegan_forward.

Step 1: mapping network

def mapping(z, M):
    h = z
    for i in range(num_layers):
        h = leaky_relu(add(matmul(M[f"W{i}"], h), M[f"b{i}"]))
    return h

Step 2: adaptive instance normalization

def adain(x, w_scale, w_bias):
    mu = mean(x)
    sd = std(x)
    x_norm = [(xi - mu) / (sd + 1e-8) for xi in x]
    return [w_scale * xi + w_bias for xi in x_norm]

In the published model w supplies a per-channel scale and bias through a learned linear projection. In the toy, each block's scale and bias are single scalars, each a dot product of w with a fixed random vector.

Step 3: per-layer noise

def add_noise(x, sigma, rng):
    return [xi + sigma * rng.gauss(0, 1) for xi in x]

In the published model sigma is a learned per-channel factor. In the toy it is one fixed scalar noise_sigma, applied once per block.

• Droplet artifacts. StyleGAN 1 produced a blobby droplet in the feature maps because AdaIN zeroed out the mean. StyleGAN 2's weight demodulation fixes it by scaling the convolution weights instead.
• Texture sticking. In StyleGAN 1 and 2, textures followed pixel coordinates instead of object coordinates, which shows when you interpolate. StyleGAN 3's alias-free convolutions fix this with windowed sinc filters.
• Mode coverage. Truncation ψ < 0.7 looks clean but samples from a narrow cone. Use ψ = 1.0 when you need diversity.
• Inversion is lossy. Inverting a real photo into W is usually done through optimization or an encoder (e4e, ReStyle, HyperStyle). Results drift over many iterations.

When the product is a photo of a person's face, StyleGAN beats diffusion on inference cost (single forward pass, roughly 10 ms or less on a high-end GPU; an approximate figure that varies with hardware and implementation) and on sharpness at the same quality bar.

Save outputs/skill-stylegan-inversion.md. The skill takes a real photo and outputs four things: the inversion method (e4e / ReStyle / HyperStyle), the expected latent loss, an editing budget (how far in W you can move before artifacts), and a list of known-good editing directions (age, expression, pose).

1. Easy. Run code/main.py. main() already loops over both modes (adain_on is a parameter of stylegan_forward), so you get the AdaIN on/off printout in one run. Then change it to feed the same five z samples to both modes, and compare the spread of outputs for a fixed latent versus a perturbed latent.
2. Medium. Implement mixing regularization. For a training batch, compute w_a and w_b, apply w_a for the first half of synthesis and w_b for the second half, and check whether the decoder learns disentangled styles.
3. Hard. Take a pretrained StyleGAN3 FFHQ model (ffhq-1024.pkl). Find the w direction that controls "smile" by training an SVM on labelled samples, and report how far you can push before identity drifts.

StyleGAN3 on a 4090 generates a 1024² FFHQ face in roughly 10 ms or less (an approximate, hardware-dependent figure) with num_steps = 1, no VAE decode, and no cross-attention pass. In production terms that's the floor latency for an image generator. A 50-step SDXL + VAE-decode pipeline at the same resolution takes on the order of a few seconds (again approximate). That is a gap of roughly two orders of magnitude, and for narrow-domain products (avatar services, ID document pipelines, stock face generation) it wins on TCO.

Two operational consequences:

• No scheduler, no batcher. A static batch at the target occupancy is the right fit. Continuous batching, which matters for LLMs and diffusion, has little to offer when each request takes the same FLOPs.
• Truncation ψ is the safety knob. ψ < 0.7 samples from a narrow cone of the mapping network's range, and that is the lever the serving layer has over sample variance. Lower ψ at peak load, raise it for premium users.

• Karras et al. (2019). A Style-Based Generator Architecture for GANs: StyleGAN.
• Karras et al. (2020). Analyzing and Improving the Image Quality of StyleGAN: StyleGAN2.
• Karras et al. (2021). Alias-Free Generative Adversarial Networks: StyleGAN3.
• Tov et al. (2021). Designing an Encoder for StyleGAN Image Manipulation: e4e inversion.
• Sauer et al. (2022). StyleGAN-XL: Scaling StyleGAN to Large Diverse Datasets: StyleGAN-XL.
• Huang et al. (2024). R3GAN: The GAN is dead; long live the GAN!: modern minimal GAN recipe.