Skip to content

Troubleshooting

Document status

Owner: LLMOps · Basis: vLLM HyperAccel 0.0.5 · Last checked: 2026-10-15

Common problems and how to fix them. If this page does not solve your problem, contact taeseo.um@hyperaccel.ai or seunghun.shin@hyperaccel.ai.

1. The server takes a long time to become ready

Symptoms

After vllm serve starts, it takes a long time before Application startup complete. appears.

Causes

After the weights are loaded, the model is compiled for the LPU during startup. Compilation happens once, at startup.

Solution

This is expected. /health responds only after compilation finishes, so use a long initial delay for the readiness probe on Kubernetes (Kubernetes support).

2. Parallelism configuration error

Symptoms

1
vllm-hyperaccel supports only PP=1, PP=2, or PP=4 with TP=1

Causes

tensor_parallel_size is not 1, or pipeline_parallel_size is not 1, 2, or 4.

Solution

Remove --tensor-parallel-size, and use --pipeline-parallel-size 2 or 4, or --data-parallel-size, for multiple LPUs. Pipeline parallelism runs with vLLM's default multiprocessing executor. See Using Multiple LPUs.

3. Not enough LPUs

Symptoms

1
LPU device N is unavailable (device_count=M)

Causes

The run asked for more LPUs than the node has. Pipeline parallelism of size N and data parallelism of size N each need N LPUs.

Solution

Check how many cards are visible with hyperaccel-smi and match the parallel size to it. In a container, make sure all needed cards were passed (--device hyperaccel.ai/lpu=all).

4. Context length error

Symptoms

1
sliding-window attention on EVT0 supports max_model_len <= 16384

or max_model_len is reduced with a warning about a sliding window.

Causes

The maximum context is 16,384 tokens. For sliding-window models the LPU has no kernel for, the context is limited to the window so results stay correct. Gemma-4 is not affected.

Solution

Lower --max-model-len to 16,384 or less. See Supported Models for per-model limits.

5. Platform plugin conflict

Symptoms

1
Only one platform plugin can be activated

or

1
HA mode only allows lpu device

Causes

vLLM allows a single platform plugin. Another platform plugin in the same environment is active as well, or a device other than lpu was requested.

Solution

Leave VLLM_USE_HA_PLATFORM unset. Remove any option that overrides the device.

6. Sampling option error

Symptoms

1
LPU sampling does not support: ...

Causes

The request uses an option that LPU sampling does not support.

Solution

Remove the options named in the error from the request. The parameters you can use are listed in LPU Sampling.

7. Warnings at startup

Warning Meaning
float16 is not supported on the LPU; forcing model dtype to bfloat16 The float16 checkpoint is run as bfloat16. Expected for Llama-2
enforce_eager=True is not supported. Setting enforce_eager=False Models always run compiled
Block size N is not supported on EVT0; using 1 --block-size has no effect; the KV cache is paged one token per block
Pin memory is not supported on vllm-hyperaccel Informational

8. Fewer requests run at once than --max-num-seqs allows

Causes

Concurrency is also limited by KV cache memory. Each request reserves a region sized for --max-model-len.

Solution

Lower --max-model-len to what you actually need. See Context and KV Cache.

9. Requests are being preempted

Causes

The KV cache filled up while requests were generating, so one was paused and will be resumed. Results are unaffected; throughput is.

Solution

Lower --max-num-seqs or --max-model-len.

10. Output differs slightly from a GPU or CPU run

Causes

The LPU computes in bfloat16. When two tokens score almost the same, the LPU can choose the other one, and the text diverges from there.

Solution

This is expected. The hardware tests are judged the same way: against the CPU's choice at each step, with a small tolerance for near-ties.

11. A profiling trace is empty

Causes

  • vllm serve was run under haprof without --profile-children.
  • The process was killed. Traces are written when a process exits normally.
  • torch.profiler was opened in a process that has no worker.

Solution

Follow Profiling: enable --profile-children, run the worker in the same process with VLLM_ENABLE_V1_MULTIPROCESSING=0, or open the profiler in each worker through collective_rpc. Stop the server with Ctrl+C.