32 nodes (256 GPUs)
What each GPU does per optimizer step
Each colored block = 1 forward pass (8 samples)
Global batch (always fixed)4,096