EXAONE-4.5 33B¶
Document status
Owner: LLMOps · Basis: vLLM HyperAccel 0.0.5 · Last checked: 2026-10-15
Overview¶
This tutorial runs LGAI-EXAONE/EXAONE-4.5-33B with vLLM HyperAccel.
Like Gemma-4, EXAONE-4.5 is published as a multimodal checkpoint and is served on the LPU as a text model from
that original checkpoint with --language-model-only.
Validated on the LPU with short prompts and with a full 4,096-token context, checked against a CPU reference.
Setup¶
Start from an environment where Installation is complete.
Online Serving¶
1. Start the server¶
1 2 3 4 5 6 | |
2. Send a request with cURL¶
1 2 3 4 5 6 7 8 | |
Offline Inference¶
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 | |
INT4¶
Serve the official INT4 checkpoint LGAI-EXAONE/EXAONE-4.5-33B-AWQ by its repository ID as is. The first run
converts it to the LPU format automatically and stores it in a cache; later runs read the cache.
1 | |
Local directory sources and the conversion cache are covered in Quantization.
Supported Scope¶
| dtype | bfloat16, INT4 weights |
| Concurrency | One request at a time (--max-num-seqs 1) |
| Context | Up to 4,096 tokens |
EXAONE-4.5 has local layers with a 4,096-token window and global layers that attend over everything. The LPU kernel
used for this model's attention layout does not apply a window mask, so the supported context is up to 4,096 tokens,
where results match the model's definition. The server reports an error at startup if max_model_len is above 4,096.
A prompt longer than the per-step token budget is read in chunks; the 4,096-token validation fills the context this way.
Next¶
- Quantization
- Gemma-4 31B — another model served with
--language-model-only - Supported Features