← Back to curriculum
Phase 07 · 16 lessons
Transformers Deep Dive
The architecture that changed everything.
01Why Transformers — The Problems with RNNsLearn02Self-Attention from ScratchBuild03Multi-Head AttentionBuild04Positional Encoding — Sinusoidal, RoPE, ALiBiBuild05The Full Transformer — Encoder + DecoderBuild06BERT — Masked Language ModelingBuild07GPT — Causal Language ModelingBuild08T5, BART — Encoder-Decoder ModelsLearn09Vision Transformers (ViT)Build10Audio Transformers — Whisper ArchitectureLearn11Mixture of Experts (MoE)Build12KV Cache, Flash Attention & Inference OptimizationBuild13Scaling LawsLearn14Build a Transformer from Scratch — The CapstoneBuild15Attention Variants — Sliding Window, Sparse, DifferentialBuild16Speculative Decoding — Draft, Verify, RepeatBuild