Skip to content

EXAONE-4.5 33B

Document status

Owner: LLMOps · Basis: vLLM HyperAccel 0.0.5 · Last checked: 2026-10-15

Overview

This tutorial runs LGAI-EXAONE/EXAONE-4.5-33B with vLLM HyperAccel.

Like Gemma-4, EXAONE-4.5 is published as a multimodal checkpoint and is served on the LPU as a text model from that original checkpoint with --language-model-only.

Validated on the LPU with short prompts and with a full 4,096-token context, checked against a CPU reference.

Setup

Start from an environment where Installation is complete.

Online Serving

1. Start the server

1
2
3
4
5
6
vllm serve LGAI-EXAONE/EXAONE-4.5-33B \
  --language-model-only \
  --max-model-len 4096 \
  --max-num-seqs 1 \
  --no-enable-prefix-caching \
  --no-async-scheduling

2. Send a request with cURL

1
2
3
4
5
6
7
8
curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "LGAI-EXAONE/EXAONE-4.5-33B",
    "messages": [{"role": "user", "content": "What is the capital of South Korea?"}],
    "max_tokens": 64,
    "temperature": 0
  }'

Offline Inference

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
from vllm import LLM, SamplingParams

llm = LLM(
    model="LGAI-EXAONE/EXAONE-4.5-33B",
    language_model_only=True,
    max_model_len=4096,
    max_num_seqs=1,
    dtype="bfloat16",
    enable_prefix_caching=False,
    async_scheduling=False,
)

messages = [{"role": "user", "content": "What is the capital of South Korea?"}]
outputs = llm.chat(messages, SamplingParams(temperature=0, max_tokens=64))
print(outputs[0].outputs[0].text)

INT4

Serve the official INT4 checkpoint LGAI-EXAONE/EXAONE-4.5-33B-AWQ by its repository ID as is. The first run converts it to the LPU format automatically and stores it in a cache; later runs read the cache.

1
vllm serve LGAI-EXAONE/EXAONE-4.5-33B-AWQ --language-model-only --max-model-len 4096 --max-num-seqs 1 --no-enable-prefix-caching --no-async-scheduling

Local directory sources and the conversion cache are covered in Quantization.

Supported Scope

dtype bfloat16, INT4 weights
Concurrency One request at a time (--max-num-seqs 1)
Context Up to 4,096 tokens

EXAONE-4.5 has local layers with a 4,096-token window and global layers that attend over everything. The LPU kernel used for this model's attention layout does not apply a window mask, so the supported context is up to 4,096 tokens, where results match the model's definition. The server reports an error at startup if max_model_len is above 4,096.

A prompt longer than the per-step token budget is read in chunks; the 4,096-token validation fills the context this way.

Next