Device API¶
Query and select the lpu device with torch.accelerator, the standard PyTorch API. This page covers how CUDA APIs
map to it, the torch.ops.lpu operator namespace, device strings, and the lccl distributed backend.
torch.accelerator¶
Use torch.accelerator to query, select, and synchronize devices and to read memory statistics.
| CUDA | torch.accelerator |
|---|---|
torch.cuda.is_available() |
torch.accelerator.is_available() |
torch.cuda.device_count() |
torch.accelerator.device_count() |
torch.cuda.set_device(i) |
torch.accelerator.set_device_index(i) |
torch.cuda.current_device() |
torch.accelerator.current_device_index() |
torch.cuda.device(i) |
torch.accelerator.device_index(i) |
torch.cuda.synchronize() |
torch.accelerator.synchronize(), or synchronize by reading a value back |
torch.cuda.memory_allocated() and other statistics |
torch.accelerator.memory_allocated() and so on |
torch.cuda.empty_cache() |
torch.accelerator.empty_cache() |
torch.cuda.Stream, torch.cuda.Event |
Only the default stream (torch.accelerator.current_stream()) is used. torch.Stream("lpu") and torch.Event are not available yet |
set_device_index and device_index take an index, a string such as "lpu:1", or a torch.device.
A torch.device("lpu") without an index resolves to the device selected by the calling thread, as with CUDA.
See Distributed §4.2.
Do not use torch.autocast("lpu")
Entering the context succeeds, but operators inside it raise NotImplementedError naming the Autocastlpu
backend, in eager mode and under torch.compile. Cast explicitly.
torch.ops.lpu¶
The custom operator namespace, registered together with the ATen kernels. Its operators are documented on one page per area.
| Operators | Documentation |
|---|---|
Layout and memory, quantized weights, debugging, collective fast paths (empty_with_kind, lccl_*, and so on) |
Custom Operators |
Paged KV cache (vmm_*) |
VMM API |
Device Strings¶
| Form | Behavior |
|---|---|
torch.device("lpu") |
✅ Resolves to the device selected by the calling thread |
torch.device("lpu", 1) |
✅ Uses the explicit index |
tensor.to("lpu") |
✅ |
tensor.lpu() |
Use .to("lpu") instead |
module.to("lpu") |
✅ |
torch.zeros(n, device="lpu"), torch.zeros(n, device="lpu:0") |
✅ Value factories (zeros, ones, full, arange, empty) work with device="lpu" (the current device) or an explicit index |
torch.randn(n, device="lpu:0") |
Raises NotImplementedError (aten::normal_); use torch.randn(n).to("lpu") |
PrivateUse1 is the internal dispatch key. Error messages usually show lpu, but some low-level messages show the
internal key name instead. Read it as lpu.
See Error Handling.
Distributed¶
Registered during import torch.
1 2 | |
Use it with dist.init_process_group(backend="lccl"). It uses the same communicator as the torch.ops.lpu.lccl_* fast
path for compiled graphs. See Distributed §4.1. The supported
collectives and reductions are the ones LCCL implements, except that reduce, gather, scatter, and all_to_all
are not available. See Distributed §4.5.
See the LCCL support matrix.
Examples¶
torch.accelerator.current_device_index¶
Returns the index of the current device for the calling thread.
Example
1 2 3 4 | |
torch.accelerator.device_count¶
Returns the number of devices visible to the process.
Example
1 2 3 4 | |
torch.accelerator.device_index¶
Changes the current device within the scope and restores the previous device on exit.
Example
1 2 3 4 | |
torch.accelerator.is_available¶
Returns whether at least one device is available.
Example
1 2 3 4 | |
torch.accelerator.set_device_index¶
Sets the current device for the calling thread. After the call, "lpu" without an index resolves to this device.
Example
1 2 3 4 | |
torch.accelerator.synchronize¶
Waits for all work submitted to the device to complete.
Example
1 2 3 4 5 | |