Using Multiple LPUs¶
Document status
Owner: LLMOps · Basis: vLLM HyperAccel 0.0.5 · Last checked: 2026-10-15
Overview¶
vLLM starts each engine worker in its own process, and every vLLM HyperAccel worker drives exactly one LPU. Scaling to several LPUs therefore uses vLLM's standard parallelism options.
| Pipeline parallelism (PP) | Data parallelism (DP) | |
|---|---|---|
| What it does | Splits one model's layers across LPUs | Runs one full copy of the model per LPU |
| Use it when | The model does not fit on one LPU | You need more throughput |
| Sizes | 2 or 4 | As many LPUs as you have |
| Validated with | Qwen3-8B on 2 and 4 LPUs | Qwen3-8B, 2 replicas |
| Option | --pipeline-parallel-size |
--data-parallel-size |
Pipeline Parallelism (PP)¶
The model's layers are divided into consecutive stages, one per LPU. Each step, a stage runs its layers and sends its activations to the next stage; the last stage computes the logits and samples. Activations travel between LPUs through the HyperAccel collective communication library, LCCL.
1 | |
1 2 3 4 5 6 7 8 9 10 11 12 | |
Requirements:
pipeline_parallel_sizeis 2 or 4, andtensor_parallel_sizeis 1.- vLLM's default multiprocessing executor is used.
- Stage
rruns onlpu:<r>, so the node needs at least as many LPUs as stages.
Validated with Qwen3-8B on 2 and on 4 LPUs, with output checked against a CPU reference.
Data Parallelism (DP)¶
Each replica is a complete, independent engine with its own scheduler and its own KV cache, bound to its own LPU:
replica r runs on lpu:<r>. vLLM's front end distributes incoming requests across the replicas.
1 | |
1 2 3 4 5 6 7 8 9 10 11 | |
Notes:
- Replicas do not share a KV cache, so a request stays on the replica it was assigned to.
- The node needs one LPU per replica.
Validated with Qwen3-8B as two replicas on two LPUs, with output checked against a CPU reference.
Device Binding¶
Each worker checks at startup that the LPU it selected is the one the runtime bound. If they differ, or if the requested LPU index does not exist, it stops with an error naming both.
Next¶
- Context and KV Cache — sizing each LPU's KV cache
- Kubernetes support — deploying multi-LPU pods