← Back to curriculum

Phase 17 · 28 lessons

Infrastructure & Production

Ship AI to the real world.

01Managed LLM Platforms — Bedrock, Vertex AI, Azure OpenAILearn 02Inference Platform Economics — Fireworks, Together, Baseten, Modal, Replicate, AnyscaleLearn 03GPU Autoscaling on Kubernetes — Karpenter, KAI Scheduler, Gang SchedulingLearn 04vLLM Serving Internals: PagedAttention, Continuous Batching, Chunked PrefillLearn 05EAGLE-3 Speculative Decoding in ProductionLearn 06SGLang and RadixAttention for Prefix-Heavy WorkloadsLearn 07TensorRT-LLM on Blackwell with FP8 and NVFP4Learn 08Inference Metrics — TTFT, TPOT, ITL, Goodput, P99Learn 09Production Quantization — AWQ, GPTQ, GGUF K-quants, FP8, MXFP4/NVFP4Learn 10Cold Start Mitigation for Serverless LLMsLearn 11Multi-Region LLM Serving and KV Cache LocalityLearn 12Edge Inference — Apple Neural Engine, Qualcomm Hexagon, WebGPU/WebLLM, JetsonLearn 13LLM Observability Stack SelectionLearn 14Prompt Caching and Semantic Caching EconomicsLearn 15Batch APIs — the 50% Discount as Industry StandardLearn 16Model Routing as a Cost-Reduction PrimitiveLearn 17Disaggregated Prefill/Decode — NVIDIA Dynamo and llm-dLearn 18vLLM Production Stack with LMCache KV OffloadingLearn 19AI Gateways — LiteLLM, Portkey, Kong AI Gateway, BifrostLearn 20Shadow Traffic, Canary Rollout, and Progressive Deployment for LLMsLearn 21A/B Testing LLM Features — GrowthBook, Statsig, and the Vibes ProblemLearn 22Load Testing LLM APIs — Why k6 and Locust LieBuild 23SRE for AI — Multi-Agent Incident Response, Runbooks, Predictive DetectionLearn 24Chaos Engineering for LLM ProductionLearn 25Security — Secrets, API Key Rotation, Audit Logs, GuardrailsLearn 26Compliance — SOC 2, HIPAA, GDPR, PCI-DSS, EU AI Act, ISO 42001Learn 27FinOps for LLMs — Unit Economics and Multi-Tenant AttributionLearn 28Self-Hosted Serving Selection — llama.cpp, Ollama, TGI, vLLM, SGLangLearn