What is Mixed Precision?

Using float16 for forward pass and most operations (faster, less memory) but keeping float32 for gradient accumulation and weight updates (more precise). Gets 2x speedup with negligible accuracy loss.

What people say

Training trick for speed

Why it's called that