Skip to content

Device API

Query and select the lpu device with torch.accelerator, the standard PyTorch API. This page covers how CUDA APIs map to it, the torch.ops.lpu operator namespace, device strings, and the lccl distributed backend.


torch.accelerator

Use torch.accelerator to query, select, and synchronize devices and to read memory statistics.

CUDA torch.accelerator
torch.cuda.is_available() torch.accelerator.is_available()
torch.cuda.device_count() torch.accelerator.device_count()
torch.cuda.set_device(i) torch.accelerator.set_device_index(i)
torch.cuda.current_device() torch.accelerator.current_device_index()
torch.cuda.device(i) torch.accelerator.device_index(i)
torch.cuda.synchronize() torch.accelerator.synchronize(), or synchronize by reading a value back
torch.cuda.memory_allocated() and other statistics torch.accelerator.memory_allocated() and so on
torch.cuda.empty_cache() torch.accelerator.empty_cache()
torch.cuda.Stream, torch.cuda.Event Only the default stream (torch.accelerator.current_stream()) is used. torch.Stream("lpu") and torch.Event are not available yet

set_device_index and device_index take an index, a string such as "lpu:1", or a torch.device. A torch.device("lpu") without an index resolves to the device selected by the calling thread, as with CUDA. See Distributed §4.2.

Do not use torch.autocast("lpu")

Entering the context succeeds, but operators inside it raise NotImplementedError naming the Autocastlpu backend, in eager mode and under torch.compile. Cast explicitly.

See Device Semantics §Autocast.


torch.ops.lpu

The custom operator namespace, registered together with the ATen kernels. Its operators are documented on one page per area.

Operators Documentation
Layout and memory, quantized weights, debugging, collective fast paths (empty_with_kind, lccl_*, and so on) Custom Operators
Paged KV cache (vmm_*) VMM API

Device Strings

Form Behavior
torch.device("lpu") ✅ Resolves to the device selected by the calling thread
torch.device("lpu", 1) ✅ Uses the explicit index
tensor.to("lpu") ✅
tensor.lpu() Use .to("lpu") instead
module.to("lpu") ✅
torch.zeros(n, device="lpu"), torch.zeros(n, device="lpu:0") ✅ Value factories (zeros, ones, full, arange, empty) work with device="lpu" (the current device) or an explicit index
torch.randn(n, device="lpu:0") Raises NotImplementedError (aten::normal_); use torch.randn(n).to("lpu")

PrivateUse1 is the internal dispatch key. Error messages usually show lpu, but some low-level messages show the internal key name instead. Read it as lpu. See Error Handling.


Distributed

Registered during import torch.

1
2
torch.distributed.Backend.register_backend(
    "lccl", ..., devices=["lpu"])

Use it with dist.init_process_group(backend="lccl"). It uses the same communicator as the torch.ops.lpu.lccl_* fast path for compiled graphs. See Distributed §4.1. The supported collectives and reductions are the ones LCCL implements, except that reduce, gather, scatter, and all_to_all are not available. See Distributed §4.5. See the LCCL support matrix.


Examples

torch.accelerator.current_device_index

Returns the index of the current device for the calling thread.

Example
1
2
3
4
import torch

torch.accelerator.set_device_index(1)
assert torch.accelerator.current_device_index() == 1

torch.accelerator.device_count

Returns the number of devices visible to the process.

Example
1
2
3
4
import torch

n = torch.accelerator.device_count()
print(f"{n} LPU device(s)")

torch.accelerator.device_index

Changes the current device within the scope and restores the previous device on exit.

Example
1
2
3
4
import torch

with torch.accelerator.device_index(1):
    x = torch.zeros(4, device="lpu")    # allocated on lpu:1

torch.accelerator.is_available

Returns whether at least one device is available.

Example
1
2
3
4
import torch

device = "lpu" if torch.accelerator.is_available() else "cpu"
x = torch.randn(4, 4).to(device)

torch.accelerator.set_device_index

Sets the current device for the calling thread. After the call, "lpu" without an index resolves to this device.

Example
1
2
3
4
import torch

torch.accelerator.set_device_index(0)
x = torch.randn(4).to("lpu")    # lpu:0

torch.accelerator.synchronize

Waits for all work submitted to the device to complete.

Example
1
2
3
4
5
import torch

x = torch.randn(64, 64, dtype=torch.bfloat16).to("lpu")
y = x @ x
torch.accelerator.synchronize()

See Also