Splitting a matrix multiply

Tensor parallelism: column and row splits, four collectives a layer, sequence parallelism, and why the degree stops at a node