Troubleshooting¶
Document status
Owner: LLMOps · Basis: vLLM HyperAccel 0.0.5 · Last checked: 2026-10-15
Common problems and how to fix them. If this page does not solve your problem, contact taeseo.um@hyperaccel.ai or seunghun.shin@hyperaccel.ai.
1. The server takes a long time to become ready¶
Symptoms¶
After vllm serve starts, it takes a long time before Application startup complete. appears.
Causes¶
After the weights are loaded, the model is compiled for the LPU during startup. Compilation happens once, at startup.
Solution¶
This is expected. /health responds only after compilation finishes, so use a long initial delay for the readiness
probe on Kubernetes (Kubernetes support).
2. Parallelism configuration error¶
Symptoms¶
1 | |
Causes¶
tensor_parallel_size is not 1, or pipeline_parallel_size is not 1, 2, or 4.
Solution¶
Remove --tensor-parallel-size, and use --pipeline-parallel-size 2 or 4, or --data-parallel-size, for multiple
LPUs. Pipeline parallelism runs with vLLM's default multiprocessing executor. See
Using Multiple LPUs.
3. Not enough LPUs¶
Symptoms¶
1 | |
Causes¶
The run asked for more LPUs than the node has. Pipeline parallelism of size N and data parallelism of size N each need N LPUs.
Solution¶
Check how many cards are visible with hyperaccel-smi and match the parallel size to it. In a container, make sure all
needed cards were passed (--device hyperaccel.ai/lpu=all).
4. Context length error¶
Symptoms¶
1 | |
or max_model_len is reduced with a warning about a sliding window.
Causes¶
The maximum context is 16,384 tokens. For sliding-window models the LPU has no kernel for, the context is limited to the window so results stay correct. Gemma-4 is not affected.
Solution¶
Lower --max-model-len to 16,384 or less. See Supported Models for per-model limits.
5. Platform plugin conflict¶
Symptoms¶
1 | |
or
1 | |
Causes¶
vLLM allows a single platform plugin. Another platform plugin in the same environment is active as well, or a device
other than lpu was requested.
Solution¶
Leave VLLM_USE_HA_PLATFORM unset. Remove any option that overrides the device.
6. Sampling option error¶
Symptoms¶
1 | |
Causes¶
The request uses an option that LPU sampling does not support.
Solution¶
Remove the options named in the error from the request. The parameters you can use are listed in LPU Sampling.
7. Warnings at startup¶
| Warning | Meaning |
|---|---|
float16 is not supported on the LPU; forcing model dtype to bfloat16 |
The float16 checkpoint is run as bfloat16. Expected for Llama-2 |
enforce_eager=True is not supported. Setting enforce_eager=False |
Models always run compiled |
Block size N is not supported on EVT0; using 1 |
--block-size has no effect; the KV cache is paged one token per block |
Pin memory is not supported on vllm-hyperaccel |
Informational |
8. Fewer requests run at once than --max-num-seqs allows¶
Causes¶
Concurrency is also limited by KV cache memory. Each request reserves a region sized for --max-model-len.
Solution¶
Lower --max-model-len to what you actually need. See Context and KV Cache.
9. Requests are being preempted¶
Causes¶
The KV cache filled up while requests were generating, so one was paused and will be resumed. Results are unaffected; throughput is.
Solution¶
Lower --max-num-seqs or --max-model-len.
10. Output differs slightly from a GPU or CPU run¶
Causes¶
The LPU computes in bfloat16. When two tokens score almost the same, the LPU can choose the other one, and the text
diverges from there.
Solution¶
This is expected. The hardware tests are judged the same way: against the CPU's choice at each step, with a small tolerance for near-ties.
11. A profiling trace is empty¶
Causes¶
vllm servewas run underhaprofwithout--profile-children.- The process was killed. Traces are written when a process exits normally.
torch.profilerwas opened in a process that has no worker.
Solution¶
Follow Profiling: enable --profile-children, run the worker in the same process with
VLLM_ENABLE_V1_MULTIPROCESSING=0, or open the profiler in each worker through collective_rpc. Stop the server with
Ctrl+C.