Skip to content

Supported Features

Document status

Owner: LLMOps · Basis: vLLM HyperAccel 0.0.5 · Last checked: 2026-10-16

The feature support principle of vLLM HyperAccel is: aligned with vLLM. The plugin keeps vLLM's own engine and adds the LPU underneath it, so a vLLM feature becomes supported once the LPU path for it exists and is validated.

Below is vLLM HyperAccel 0.0.5 support for the features in vLLM's feature compatibility matrix and for its parallelism modes.

Features

Feature Status Description
Chunked Prefill (CP) ✅ Always on. A prompt longer than the per-step token budget is read over several steps
Tensor Parallelism (TP) ❌ tensor_parallel_size accepts only 1
Pipeline Parallelism (PP) ✅ 2 or 4 LPUs. Using Multiple LPUs
Data Parallelism (DP) ✅ One replica per LPU. Using Multiple LPUs
Quantization ✅ INT4 (EXAONE-4.5, Qwen3-8B, Qwen3-30B-A3B), FP8 (Qwen3-8B, Qwen3-30B-A3B). Quantization
Pooling ❌ Pooling requests, such as embeddings, are rejected with an error
Prompt Logprobs ❌ Requests with prompt_logprobs are rejected with an error

Sampling Parameters

Parameter Where it runs
temperature, top_k, top_p, seed LPU
stop, max_tokens, n vLLM

See LPU Sampling for details.

Using Features Together

  • Chunked prefill is always on, so it works with every feature.
  • INT4 is validated with EXAONE-4.5, Qwen3-8B, and Qwen3-30B-A3B, and FP8 with Qwen3-8B and Qwen3-30B-A3B.
  • PP and DP are each validated with a dense model (Qwen3-8B).
  • Sliding-window attention and KV cache preemption are validated together with Gemma-4.

Limits

Item Value
Context 16,384 tokens
KV cache read unit 64 tokens
Tokens per step LPU cores × 32 (1,024 with 32 cores)
PP sizes 2, 4
TP size 1
Compute dtype bfloat16

Configurations and request options outside these values are not silently routed elsewhere; they fail with an error that names them. Supported models are listed in the vLLM HyperAccel overview.