HyperAccel SDK¶
HyperAccel SDK is the software you need to run LLMs on HyperAccel LPUs. The runtime, PyTorch backend, kernel compiler and vLLM plugin ship together; the current version is 0.0.6. The legato compiler and the driver have their own version numbers. See Releases for how they line up.
You do not rewrite model code for the LPU. In PyTorch you change the device to lpu; with vLLM you keep using vllm serve.
The LPU card¶
| Chip | Bertha |
| Memory | 128 GiB LPDDR5X |
| Host link | PCIe Gen5 x16 |
| Multiple cards | A model is split by layers across 2 or 4 cards (pipeline parallelism) |
To produce each token, an LLM reads all of its weights from memory once. Token generation is therefore bound more by how fast weights come out of memory than by compute, and the LPU is designed around that read.
What is in the SDK¶
From the layer closest to your code down to the hardware:
| Layer | Package | What it does |
|---|---|---|
| Model serving | vllm_hyperaccel |
vLLM hardware plugin. Adds the LPU without changing vLLM |
| PyTorch | torch_ha, torch_legato |
Adds the lpu device to PyTorch and a torch.compile backend |
| Kernels | legato, legato_kernels |
Legato, a Python language for LPU kernels, its compiler, and a library of LLM kernels |
| Runtime | hart |
Device memory, kernel launch, streams and events |
| Card-to-card communication | lccl |
Collective communication across cards |
| Tools | hapti, ha_intercept, ha_telemetry, halogger |
Profiling, call tracing, OpenTelemetry export, logging |
| Driver | ha kernel module |
Installed on the host. Creates a device file per card: /dev/ha0, /dev/ha1, ... |
Only the driver goes on the host. Everything else comes as container images. Both are downloaded after logging in with an account HyperAccel issues.
Supported in 0.0.6¶
| OS | Ubuntu 22.04, 24.04 (x86_64) |
| Python | 3.10 |
| PyTorch | 2.10 |
| vLLM | 0.28 |
| Models | Qwen3-8B, Qwen3-30B-A3B, Gemma-4-31B, EXAONE-4.5-33B, Llama-2-7B. Details in vLLM supported models |
| Precision | BF16; FP8 and INT4 weights for some models |
Where to start¶
| What you want to do | Start with |
|---|---|
| Install and run something once | Quick install, Quickstart |
| Serve a model behind an API | vLLM |
| Run PyTorch code on the LPU | Torch |
| Write your own operations | Compiler, Kernels |
| Run many servers on Kubernetes | Cloud native |
| Versions and where to download | Releases |
| Find examples for a component | Tutorials |