sde-cheatsheet
Parallelism - Concept
Parallelism Paradigms
DP
: Data Parallelism
PP
: Pipeline Parallelism
TP
: Tensor Parallelism
SP
: Sequence Parallelism