Async GRPO with LoRA across HF Jobs: a bucket, a proxy, and no NCCL

## Async GRPO with LoRA across HF Jobs: a bucket, a proxy, and no NCCL

## Async GRPO with LoRA across HF Jobs: a bucket, a proxy, and no NCCL - +30 > The architecture: leveraging Hugging Face Jobs and Storage Buckets 🪣 The three Jobs The vLLM replicas The dataset choice: the Sanity set The trainer The proxy Routing rollouts by KV prefix Broadcasting the adapter Full run results Weight sync Routing Where the time goes Reward Chasing the bottleneck ping-pong Reading the dashboard Run 1, r1-dp2: a trainer that cannot keep up Run 2, r1-dp2-tb16k: pack the microbatch Run 3, r1-dp2-tb16k-nockpt: stop recomputing the forward Run 4, r1-dp3-tb16k-nockpt: three replicas,…

Читать полностью →

Источник: Hugging Face Blog

Подключаюсь к источникам…

30 главных источников
о мире ИИ

Автоматический перевод, курирование и красивая подача главных статей об искусственном интеллекте.

0
статей
0
источников
9
разделов