API Reference¶
torch_ha does not add a Python API of its own. It registers the LPU as a PyTorch device, so you write ordinary
PyTorch code. These pages describe what torch_ha registers and how each part behaves on the LPU.
| Page | Contents |
|---|---|
| Operator Coverage | ATen operators registered for the eager path, with their support status |
| Device API | Using torch.accelerator, device strings, and distributed backend registration |
| Custom Operators | Layout, quantization, debugging, and collective operators in torch.ops.lpu |
| VMM API | torch.ops.lpu.vmm_* for the paged KV cache |
| Environment Variables | Variables that torch_ha reads, with pointers to lower-layer variables |
| Support Matrix | Supported versions, dtypes, and features |
For device behavior, unregistered operators, and error reporting, see the Programming Guide.
Common Entry Points¶
| Task | Example | Reference |
|---|---|---|
| Moving tensors and taking views | x.to("lpu"), x[i] |
Operator Coverage. Views that device compute takes as is are contiguous slices of dim -3 and earlier that meet the conditions in Slicing |
| Compiling a model | torch.compile(model, backend="legato", ...) |
Compile PyTorch Model with Legato |
| Querying and selecting devices | torch.accelerator.is_available() |
Device API |
| Custom operators | torch.ops.lpu.get_storage_meta |
Custom Operators API |
| Paged KV cache | torch.ops.lpu.vmm_create_kv_pool(...) |
VMM API |
| Distributed communication | dist.init_process_group("lccl") |
Distributed |
You do not need to import torch_ha; import torch registers the backend.
Behavior at a Glance¶
| Property | Value |
|---|---|
| Device string | "lpu" |
| Internal dispatch key | PrivateUse1 — shown as lpu in error messages |
| Registration | Automatic on import torch. A failure is logged, not raised |
| Eager operators | Tensor moves, views, and slices, plus four experimental compute operators. Model computation runs through torch.compile(backend="legato") |
| Unregistered operators | Raise a dispatch error. There is no automatic CPU fallback |
| Primary dtype | bfloat16 |
| Tensor creation | Value factories accept device="lpu" or an explicit index. Create random tensors on the host, then call .to("lpu") |
| Synchronization | Reading a value on the host waits for it, or call torch.accelerator.synchronize() |
| Devices per process | Multiple. Select one per rank with torch.accelerator.set_device_index() |
| Distributed backend | "lccl" |