Left: the spread of raw dot products q·k for random unit-variance vectors. Right: the softmax over 8 keys and how much gradient can get through it.