Skip to content

API Reference

torch_ha does not add a Python API of its own. It registers the LPU as a PyTorch device, so you write ordinary PyTorch code. These pages describe what torch_ha registers and how each part behaves on the LPU.

Page Contents
Operator Coverage ATen operators registered for the eager path, with their support status
Device API Using torch.accelerator, device strings, and distributed backend registration
Custom Operators Layout, quantization, debugging, and collective operators in torch.ops.lpu
VMM API torch.ops.lpu.vmm_* for the paged KV cache
Environment Variables Variables that torch_ha reads, with pointers to lower-layer variables
Support Matrix Supported versions, dtypes, and features

For device behavior, unregistered operators, and error reporting, see the Programming Guide.


Common Entry Points

Task Example Reference
Moving tensors and taking views x.to("lpu"), x[i] Operator Coverage. Views that device compute takes as is are contiguous slices of dim -3 and earlier that meet the conditions in Slicing
Compiling a model torch.compile(model, backend="legato", ...) Compile PyTorch Model with Legato
Querying and selecting devices torch.accelerator.is_available() Device API
Custom operators torch.ops.lpu.get_storage_meta Custom Operators API
Paged KV cache torch.ops.lpu.vmm_create_kv_pool(...) VMM API
Distributed communication dist.init_process_group("lccl") Distributed

You do not need to import torch_ha; import torch registers the backend.


Behavior at a Glance

Property Value
Device string "lpu"
Internal dispatch key PrivateUse1 — shown as lpu in error messages
Registration Automatic on import torch. A failure is logged, not raised
Eager operators Tensor moves, views, and slices, plus four experimental compute operators. Model computation runs through torch.compile(backend="legato")
Unregistered operators Raise a dispatch error. There is no automatic CPU fallback
Primary dtype bfloat16
Tensor creation Value factories accept device="lpu" or an explicit index. Create random tensors on the host, then call .to("lpu")
Synchronization Reading a value on the host waits for it, or call torch.accelerator.synchronize()
Devices per process Multiple. Select one per rank with torch.accelerator.set_device_index()
Distributed backend "lccl"

See Also