Introduction to Torch Legato¶
Document status
Owner: CL Team · Baseline: torch_legato 0.0.5, Legato 0.18.0+public, PyTorch 2.10.0+cpu, Bertha EVT0 · Last verified: 2026-10-22
The example was verified to reach the compiler entry point. Binary generation, LPU execution, and numerical correctness remain unverified because the Legato package restricted access to the HW backend (LE0144).
What it does¶
PyTorch is a framework for writing array computations and neural network models in Python.
torch_legato is a torch.compile backend that converts supported operations in PyTorch functions into Legato kernels. An LPU is a HyperAccel compute device that executes these kernels.
Write a computation such as addition in PyTorch and apply torch.compile(backend="legato") to the function. The package is named torch_legato; the backend is named "legato". This path handles supported operations without requiring the user to write kernels directly.
Components¶
| Component | Role |
|---|---|
| PyTorch | Expresses computation functions and tensors; extracts computations for compilation |
torch_ha |
Registers PyTorch's lpu device, allocates LPU memory, and copies data |
torch_legato |
Converts extracted computations into supported Legato kernels |
| Legato | Manages kernel compilation and execution |
A graph describes the relationships between computations. A kernel is a unit of computation executed on the LPU. The compilation path is:
1 | |
torch.compile returns a callable. Compilation occurs on its first invocation with inputs, so that call includes compilation cost. On this path the backend manages legato.session internally; a separate user session is unnecessary.
First example: adding two tensors¶
A tensor is a multidimensional array. This example adds two arrays with shape=[64, 256]. shape gives the number of elements in each dimension, and dtype=torch.bfloat16 specifies the BF16 numerical format.
Prerequisites:
- SDK installation and an available Bertha LPU.
torch_ha,torch_legato, and a compatible Legato installation.- A Legato package with the HW backend. If
LE0144occurs, check HW support with the package provider.
Download the complete executable and run it in the SDK environment:
1 | |
The key code is shown below. The full executable prepares a and b as LPU inputs.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 | |
"legato.device": "bertha": required compilation target.dynamic=False: compile for the input size; a size change may trigger recompilation.torch.no_grad(): inference example without gradient computation.
The full file creates inputs and a reference on the CPU, then copies the inputs to LPU memory. It allocates the inputs with torch.ops.lpu.empty_with_kind(..., "dram_word_interleaved", device="lpu:0"), the memory layout used by this example kernel. lpu:0 is the first assigned logical device.
After the call, result.cpu() copies the result to the CPU and torch.testing.assert_close compares every element with the reference. The inputs are small integers represented in BF16, so this addition is compared with zero tolerance. Expected output when all checks pass:
1 | |
This output remains unverified in the baseline environment because of the HW backend restriction.
Scope¶
This example covers [64, 256] BF16 addition. It does not guarantee support for other operations, input sizes, numerical formats, or complete models. Support requires both a torch_legato operation conversion and compliance with Legato compilation constraints.
General Python processing or unsupported operations may split the graph or cause compilation or execution errors. Do not assume every unsupported operation automatically runs on the CPU. This example does not validate backpropagation or performance.
Next steps¶
- Using the LPU device: tensor placement and data transfer.
- Writing kernels: write a kernel directly and run it with
legato.session. - Compilation integration reference: details of the integration and options; not a prerequisite for this example.