Skip to content

Operator Coverage

This page lists the 99 ATen operators that torch_ha registers for the eager path, grouped by how they behave on the device. Model inference is currently supported through the torch.compile(backend="legato") path. Coverage on that path is determined by the legato resolvers, not by this table. See Compile PyTorch Model with Legato.

An operator that has neither an LPU registration nor a PyTorch composite implementation raises a dispatch error (NotImplementedError). Composite operators (for example transpose, permute, select, split, matmul) decompose into the operators below. See Operators §3.3.

The tables list only the variants that are registered directly. The remaining variants, such as functional and in-place variants, are provided by torchgen and the common PyTorch implementations.


1. Operators for the torch.compile Path

These operators move tensors to the device, create views and slices, and read values back. They run without device kernels, with restrictions that depend on the device layout of the tensor.

Operator Purpose
aten::empty.memory_format Allocates a tensor in device memory with torch.empty(..., device="lpu:0")
aten::empty_strided Allocates a tensor with the given strides. Used internally by .to("lpu") and for output tensor creation
aten::_copy_from Performs host↔device and device-to-device copies for copy_, .to(), and .cpu(). A copy within one device uses the device copy kernel when it qualifies
aten::_copy_from_and_resize Resizes the output tensor to match the input, then copies
aten::_to_copy Implements .to(device) and .to(dtype). dtype conversion runs on the host
aten::clone clone(). Creates a new tensor with the same tensor kind
aten::view view(). Returns a view of the same storage with a different shape
aten::view.dtype view(dtype). Reinterprets the same bytes as a different dtype
aten::_reshape_alias Used when reshape() resolves to a view without a copy
aten::_unsafe_view Used internally when another operator returns its result as a view
aten::alias Creates a view of the same tensor, such as a full-range slice
aten::as_strided as_strided(). Underlies strided views such as transpose
aten::as_strided_ as_strided_(). Changes the strides of a tensor in place
aten::slice.Tensor x[a:b] slicing. References the parent storage without a copy. Slices that device compute takes as is: Device Semantics §Slicing
aten::view_as_complex torch.view_as_complex. Views pairs of real numbers as complex numbers (RoPE)
aten::view_as_real torch.view_as_real. Views complex numbers as pairs of real numbers (RoPE)
aten::set_.source_Storage set_(storage). Attaches a tensor to another storage
aten::set_.source_Storage_storage_offset set_(storage, offset, size, stride). Used by torch.load and others
aten::set_.source_Tensor set_(tensor). Shares the storage of another tensor
aten::resize_ resize_(). Resizes an output buffer passed through out=
aten::_has_compatible_shallow_copy_type Checks whether two tensor types are compatible, for example on .data assignment
aten::fill_.Scalar fill_(), torch.full. Uses the device memset kernel for 1 MiB or more, and otherwise fills on the host and copies
aten::zero_ zero_(), torch.zeros. Takes the same path as fill_(0)
aten::arange.start_out torch.arange. Builds the sequence on the host and copies it to the device
aten::cat.out torch.cat. Copies each input into its range of the output
aten::_local_scalar_dense .item(). Reads the value of a one-element tensor back to the host

2. Eager Mode Operators (Experimental)

These operators run as device kernels only for bfloat16 tensors with prebuilt shapes. Other dtypes and shapes fail with NotImplementedError. Every operand, including the output, must be contiguous (the mm weight may also be w.t(), the transpose of a contiguous weight); call .contiguous() on a non-contiguous view first, or call .clone() on a slice that device compute does not take as is (Device Semantics §Slicing). See Operators §3.1.

Operator Purpose
aten::add.out +, torch.add. Supports same shape, row broadcast, and scalar operands
aten::sub.out -, torch.sub. Supports same shape, row broadcast, and scalar operands
aten::mul.out *, torch.mul. Supports same shape, row broadcast, and scalar operands
aten::mm.out @, torch.mm. nn.Linear without bias also dispatches here

3. Registered Operators Without an Eager Device Kernel

Apart from the two stubs below (multinomial, exponential_), calling these operators in eager mode raises NotImplementedError. Use the torch.compile path for model computation.

multinomial and exponential_ return fixed values without an error

Device random number generation is not available yet, so these two operators are stubs for dry runs. multinomial returns zeros, exponential_ returns ones, and neither raises an error. Do not use them for actual sampling.

Operator Purpose
aten::div.out /, torch.div
aten::div.out_mode torch.div(..., rounding_mode="floor" | "trunc")
aten::neg.out Unary -, torch.neg
aten::pow.Scalar_out Scalar ** tensor (for example, base ** exp in RoPE)
aten::pow.Tensor_Scalar_out Tensor ** scalar
aten::pow.Tensor_Tensor_out Tensor ** tensor
aten::floor_divide.out //, torch.floor_divide
aten::remainder.Tensor_out %, torch.remainder
aten::reciprocal.out torch.reciprocal, 1.0 / x
aten::rsqrt.out torch.rsqrt. Used in RMSNorm, among others
aten::abs.out abs(), torch.abs
aten::clamp.out clamp(min, max) with scalar bounds
aten::clamp.Tensor_out clamp(min, max) with tensor bounds
aten::clamp_min.out clamp(min=...) with a scalar bound
aten::clamp_min.Tensor_out clamp(min=...) with a tensor bound
aten::silu.out nn.SiLU, F.silu
aten::gelu.out nn.GELU, F.gelu
aten::sigmoid.out torch.sigmoid, nn.Sigmoid
aten::tanh.out torch.tanh, nn.Tanh
aten::exp.out torch.exp
aten::log.out torch.log
aten::sin.out torch.sin. Used in RoPE
aten::cos.out torch.cos. Used in RoPE
aten::floor.out torch.floor
aten::bmm.out torch.bmm and @ with a batch dimension
aten::addmm.out torch.addmm. nn.Linear with bias dispatches here
aten::sum.IntList_out sum(dim)
aten::mean.out mean(dim)
aten::amax.out amax(dim)
aten::argmax.out argmax(dim). Used in greedy sampling
aten::max.dim_max max(dim). Returns values and indices together
aten::min.dim_min min(dim). Returns values and indices together
aten::cumsum.out cumsum(dim)
aten::_softmax.out softmax, F.softmax
aten::_log_softmax.out log_softmax, F.log_softmax
aten::eq.Scalar_out == between a tensor and a scalar
aten::eq.Tensor_out == between two tensors
aten::lt.Scalar_out < between a tensor and a scalar
aten::lt.Tensor_out < between two tensors
aten::le.Scalar_out <= between a tensor and a scalar
aten::le.Tensor_out <= between two tensors
aten::gt.Scalar_out > between a tensor and a scalar
aten::gt.Tensor_out > between two tensors
aten::ge.Scalar_out >= between a tensor and a scalar
aten::ge.Tensor_out >= between two tensors
aten::bitwise_and.Tensor_out &. Used to combine masks
aten::bitwise_or.Tensor_out |. Used to combine masks
aten::bitwise_not.out ~. Used to invert masks
aten::embedding nn.Embedding, F.embedding
aten::index.Tensor Reads by tensor index, x[idx]
aten::index_select.out index_select(dim, index)
aten::_index_put_impl_ Writes by tensor index, x[idx] = v
aten::index_copy.out index_copy, index_copy_
aten::gather.out gather(dim, index)
aten::scatter.src_out scatter(dim, index, src), scatter_
aten::scatter.value_out scatter(dim, index, value), scatter_
aten::scatter.reduce_out scatter(dim, index, src, reduce=...)
aten::scatter.value_reduce_out scatter(dim, index, value, reduce=...)
aten::scatter_add.out scatter_add, scatter_add_
aten::topk.values topk(k). Used in top-k sampling
aten::sort.values_stable sort()
aten::nonzero.out nonzero(). Output size depends on the input data
aten::where.self_out torch.where(cond, a, b)
aten::masked_fill_.Scalar masked_fill_(mask, value). Used for attention masks, among others
aten::flip flip(dims)
aten::triu.out triu(). Used to build causal masks
aten::tril.out tril(). Used to build causal masks
aten::multinomial.out torch.multinomial. Returns zeros without an error
aten::exponential_ exponential_(). Returns ones without an error

See Also