Skip to content

HyperAccel SDK

HyperAccel SDK is the software you need to run LLMs on HyperAccel LPUs. The runtime, PyTorch backend, kernel compiler and vLLM plugin ship together; the current version is 0.0.6. The legato compiler and the driver have their own version numbers. See Releases for how they line up.

You do not rewrite model code for the LPU. In PyTorch you change the device to lpu; with vLLM you keep using vllm serve.

The LPU card

Chip Bertha
Memory 128 GiB LPDDR5X
Host link PCIe Gen5 x16
Multiple cards A model is split by layers across 2 or 4 cards (pipeline parallelism)

To produce each token, an LLM reads all of its weights from memory once. Token generation is therefore bound more by how fast weights come out of memory than by compute, and the LPU is designed around that read.

What is in the SDK

From the layer closest to your code down to the hardware:

Layer Package What it does
Model serving vllm_hyperaccel vLLM hardware plugin. Adds the LPU without changing vLLM
PyTorch torch_ha, torch_legato Adds the lpu device to PyTorch and a torch.compile backend
Kernels legato, legato_kernels Legato, a Python language for LPU kernels, its compiler, and a library of LLM kernels
Runtime hart Device memory, kernel launch, streams and events
Card-to-card communication lccl Collective communication across cards
Tools hapti, ha_intercept, ha_telemetry, halogger Profiling, call tracing, OpenTelemetry export, logging
Driver ha kernel module Installed on the host. Creates a device file per card: /dev/ha0, /dev/ha1, ...

Only the driver goes on the host. Everything else comes as container images. Both are downloaded after logging in with an account HyperAccel issues.

Supported in 0.0.6

OS Ubuntu 22.04, 24.04 (x86_64)
Python 3.10
PyTorch 2.10
vLLM 0.28
Models Qwen3-8B, Qwen3-30B-A3B, Gemma-4-31B, EXAONE-4.5-33B, Llama-2-7B. Details in vLLM supported models
Precision BF16; FP8 and INT4 weights for some models

Where to start

What you want to do Start with
Install and run something once Quick install, Quickstart
Serve a model behind an API vLLM
Run PyTorch code on the LPU Torch
Write your own operations Compiler, Kernels
Run many servers on Kubernetes Cloud native
Versions and where to download Releases
Find examples for a component Tutorials