Operator Coverage¶
This page lists the 99 ATen operators that torch_ha registers for the eager path, grouped by how they behave on the
device. Model inference is currently supported through the torch.compile(backend="legato") path.
Coverage on that path is determined by the legato resolvers, not by this table.
See Compile PyTorch Model with Legato.
An operator that has neither an LPU registration nor a PyTorch composite implementation raises a dispatch error
(NotImplementedError). Composite operators (for example transpose, permute, select, split, matmul)
decompose into the operators below.
See Operators §3.3.
The tables list only the variants that are registered directly. The remaining variants, such as functional and in-place variants, are provided by torchgen and the common PyTorch implementations.
1. Operators for the torch.compile Path¶
These operators move tensors to the device, create views and slices, and read values back. They run without device kernels, with restrictions that depend on the device layout of the tensor.
| Operator | Purpose |
|---|---|
aten::empty.memory_format |
Allocates a tensor in device memory with torch.empty(..., device="lpu:0") |
aten::empty_strided |
Allocates a tensor with the given strides. Used internally by .to("lpu") and for output tensor creation |
aten::_copy_from |
Performs host↔device and device-to-device copies for copy_, .to(), and .cpu(). A copy within one device uses the device copy kernel when it qualifies |
aten::_copy_from_and_resize |
Resizes the output tensor to match the input, then copies |
aten::_to_copy |
Implements .to(device) and .to(dtype). dtype conversion runs on the host |
aten::clone |
clone(). Creates a new tensor with the same tensor kind |
aten::view |
view(). Returns a view of the same storage with a different shape |
aten::view.dtype |
view(dtype). Reinterprets the same bytes as a different dtype |
aten::_reshape_alias |
Used when reshape() resolves to a view without a copy |
aten::_unsafe_view |
Used internally when another operator returns its result as a view |
aten::alias |
Creates a view of the same tensor, such as a full-range slice |
aten::as_strided |
as_strided(). Underlies strided views such as transpose |
aten::as_strided_ |
as_strided_(). Changes the strides of a tensor in place |
aten::slice.Tensor |
x[a:b] slicing. References the parent storage without a copy. Slices that device compute takes as is: Device Semantics §Slicing |
aten::view_as_complex |
torch.view_as_complex. Views pairs of real numbers as complex numbers (RoPE) |
aten::view_as_real |
torch.view_as_real. Views complex numbers as pairs of real numbers (RoPE) |
aten::set_.source_Storage |
set_(storage). Attaches a tensor to another storage |
aten::set_.source_Storage_storage_offset |
set_(storage, offset, size, stride). Used by torch.load and others |
aten::set_.source_Tensor |
set_(tensor). Shares the storage of another tensor |
aten::resize_ |
resize_(). Resizes an output buffer passed through out= |
aten::_has_compatible_shallow_copy_type |
Checks whether two tensor types are compatible, for example on .data assignment |
aten::fill_.Scalar |
fill_(), torch.full. Uses the device memset kernel for 1 MiB or more, and otherwise fills on the host and copies |
aten::zero_ |
zero_(), torch.zeros. Takes the same path as fill_(0) |
aten::arange.start_out |
torch.arange. Builds the sequence on the host and copies it to the device |
aten::cat.out |
torch.cat. Copies each input into its range of the output |
aten::_local_scalar_dense |
.item(). Reads the value of a one-element tensor back to the host |
2. Eager Mode Operators (Experimental)¶
These operators run as device kernels only for bfloat16 tensors with prebuilt shapes. Other dtypes and shapes fail
with NotImplementedError. Every operand, including the output, must be contiguous (the mm weight may also be w.t(),
the transpose of a contiguous weight); call .contiguous() on a non-contiguous view first, or
call .clone() on a slice that device compute does not take as is (Device Semantics §Slicing).
See Operators §3.1.
| Operator | Purpose |
|---|---|
aten::add.out |
+, torch.add. Supports same shape, row broadcast, and scalar operands |
aten::sub.out |
-, torch.sub. Supports same shape, row broadcast, and scalar operands |
aten::mul.out |
*, torch.mul. Supports same shape, row broadcast, and scalar operands |
aten::mm.out |
@, torch.mm. nn.Linear without bias also dispatches here |
3. Registered Operators Without an Eager Device Kernel¶
Apart from the two stubs below (multinomial, exponential_), calling these operators in eager mode raises
NotImplementedError. Use the torch.compile path for model computation.
multinomial and exponential_ return fixed values without an error
Device random number generation is not available yet, so these two operators are stubs for dry runs.
multinomial returns zeros, exponential_ returns ones, and neither raises an error. Do not use them for actual
sampling.
| Operator | Purpose |
|---|---|
aten::div.out |
/, torch.div |
aten::div.out_mode |
torch.div(..., rounding_mode="floor" | "trunc") |
aten::neg.out |
Unary -, torch.neg |
aten::pow.Scalar_out |
Scalar ** tensor (for example, base ** exp in RoPE) |
aten::pow.Tensor_Scalar_out |
Tensor ** scalar |
aten::pow.Tensor_Tensor_out |
Tensor ** tensor |
aten::floor_divide.out |
//, torch.floor_divide |
aten::remainder.Tensor_out |
%, torch.remainder |
aten::reciprocal.out |
torch.reciprocal, 1.0 / x |
aten::rsqrt.out |
torch.rsqrt. Used in RMSNorm, among others |
aten::abs.out |
abs(), torch.abs |
aten::clamp.out |
clamp(min, max) with scalar bounds |
aten::clamp.Tensor_out |
clamp(min, max) with tensor bounds |
aten::clamp_min.out |
clamp(min=...) with a scalar bound |
aten::clamp_min.Tensor_out |
clamp(min=...) with a tensor bound |
aten::silu.out |
nn.SiLU, F.silu |
aten::gelu.out |
nn.GELU, F.gelu |
aten::sigmoid.out |
torch.sigmoid, nn.Sigmoid |
aten::tanh.out |
torch.tanh, nn.Tanh |
aten::exp.out |
torch.exp |
aten::log.out |
torch.log |
aten::sin.out |
torch.sin. Used in RoPE |
aten::cos.out |
torch.cos. Used in RoPE |
aten::floor.out |
torch.floor |
aten::bmm.out |
torch.bmm and @ with a batch dimension |
aten::addmm.out |
torch.addmm. nn.Linear with bias dispatches here |
aten::sum.IntList_out |
sum(dim) |
aten::mean.out |
mean(dim) |
aten::amax.out |
amax(dim) |
aten::argmax.out |
argmax(dim). Used in greedy sampling |
aten::max.dim_max |
max(dim). Returns values and indices together |
aten::min.dim_min |
min(dim). Returns values and indices together |
aten::cumsum.out |
cumsum(dim) |
aten::_softmax.out |
softmax, F.softmax |
aten::_log_softmax.out |
log_softmax, F.log_softmax |
aten::eq.Scalar_out |
== between a tensor and a scalar |
aten::eq.Tensor_out |
== between two tensors |
aten::lt.Scalar_out |
< between a tensor and a scalar |
aten::lt.Tensor_out |
< between two tensors |
aten::le.Scalar_out |
<= between a tensor and a scalar |
aten::le.Tensor_out |
<= between two tensors |
aten::gt.Scalar_out |
> between a tensor and a scalar |
aten::gt.Tensor_out |
> between two tensors |
aten::ge.Scalar_out |
>= between a tensor and a scalar |
aten::ge.Tensor_out |
>= between two tensors |
aten::bitwise_and.Tensor_out |
&. Used to combine masks |
aten::bitwise_or.Tensor_out |
|. Used to combine masks |
aten::bitwise_not.out |
~. Used to invert masks |
aten::embedding |
nn.Embedding, F.embedding |
aten::index.Tensor |
Reads by tensor index, x[idx] |
aten::index_select.out |
index_select(dim, index) |
aten::_index_put_impl_ |
Writes by tensor index, x[idx] = v |
aten::index_copy.out |
index_copy, index_copy_ |
aten::gather.out |
gather(dim, index) |
aten::scatter.src_out |
scatter(dim, index, src), scatter_ |
aten::scatter.value_out |
scatter(dim, index, value), scatter_ |
aten::scatter.reduce_out |
scatter(dim, index, src, reduce=...) |
aten::scatter.value_reduce_out |
scatter(dim, index, value, reduce=...) |
aten::scatter_add.out |
scatter_add, scatter_add_ |
aten::topk.values |
topk(k). Used in top-k sampling |
aten::sort.values_stable |
sort() |
aten::nonzero.out |
nonzero(). Output size depends on the input data |
aten::where.self_out |
torch.where(cond, a, b) |
aten::masked_fill_.Scalar |
masked_fill_(mask, value). Used for attention masks, among others |
aten::flip |
flip(dims) |
aten::triu.out |
triu(). Used to build causal masks |
aten::tril.out |
tril(). Used to build causal masks |
aten::multinomial.out |
torch.multinomial. Returns zeros without an error |
aten::exponential_ |
exponential_(). Returns ones without an error |