The Landscape of Distributed Parallelism Strategies
From DDP to hybrid parallelism — a systematic guide to every parallelism strategy in large model training.
Distributed-Training
Introduction to RLHF System Design
From the four-model RLHF architecture to verl’s system design — understanding why RLHF is fundamentally a systems problem.