Skip to content

Math Operations

Math operations live under legato.math. They emit Core compute.* ops and keep the logical tensor shape: a physically padded hole [logical, physical) is not an extra element.

This page is the language / Core contract. Shape broadcast and rank decomposition are in Broadcasting. Worked kernels are in Math And Masking and Causal Attention. Target-specific lowering belongs under Language/Bertha, not here.

Shared conventions

Compiled vs interpreted

Tensor operands (input, weight, bias, query, key, a supplied softmax maximum) must be compiled values: results of receive, allocate, @, or another legato.math call.

Geometry and control arguments are interpreted at compile time: axis, keepdim, stride, dilation, padding, padding_mode, kernel_size, epsilon, scale, causal, group_size. They are Python / constexpr values, not tensors.

mask_start on attention_scores is the exception: it is a compiled index so decode can pass a dynamic cache offset.

Axis

Negative axis counts from the end, then the frontend rewrites it to [0, rank):

1
2
3
if axis < -rank or axis >= rank:
    error  # LE: axis out of range
axis = axis + rank if axis < 0 else axis

Logical vs physical

Math uses the logical extent. On a last dim written (70, 256), RMSNorm / LayerNorm / softmax / reductions see length 70. The hole [70, 256) is filled with the op's identity (0 for sum and norms, -inf for max / softmax shift) so a physical sweep matches a logical one. See Broadcasting.

Channels-last spatial tensors

Conv, depthwise conv, and pooling are channels-last:

Rank class Activation Weight
1D [N*, X, C] or [N*, X, Cin] [KX, C] or [KX, Cin, Cout]
2D [N*, H, W, C] or [N*, H, W, Cin] [KH, KW, C] or [KH, KW, Cin, Cout]

N* is zero or more leading batch dims. They are copied to the result and become a prefix loop of unbatched leaves ([X, C] / [H, W, C]). Weights and bias have no N* prefix: the same SSA value is forwarded to every leaf. See Broadcasting: spatial N*.


Element-wise unary operations

Each call maps one function over every logical element and returns a tensor of the same rank, extents, and dtype. There is no axis and no cross-element reduction, so a packed rank-N tensor may flatten to one vector leaf; a padded last dim must stay a prefix of rank-1 leaves.

1
2
3
# tensor T, same shape and dtype as the result
for idx in logical_indices(T):
    result[idx] = f(T[idx])
Call f(x) Domain
exp e^x float
sqrt √x float; x < 0 is non-finite
abs \|x\| float or integer
log ln(x) (natural log) float; x ≤ 0 is non-finite
sin / cos / tan trig in radians float

The compiler does not insert domain checks. Out-of-domain inputs follow IEEE-754 for the operand dtype. log is not log2 / log10.

1
2
3
gate = x / (1.0 + legato.math.exp(-x))          # SiLU without a LUT
shifted = scores - row_max
weights = legato.math.exp(shifted)               # softmax numerator

exp

1
exp(tensor: Value)

Compute element-wise exponential of a tensor.

Parameters:

  • tensor (Value): The input tensor.

Returns:

  • result (Value): Tensor with exp applied element-wise.

sqrt

1
sqrt(tensor: Value)

Compute element-wise square root of a tensor.

Parameters:

  • tensor (Value): The input tensor.

Returns:

  • result (Value): Tensor with sqrt applied element-wise.

abs

1
abs(tensor: Value)

Compute element-wise absolute value of a tensor.

Parameters:

  • tensor (Value): The input tensor.

Returns:

  • result (Value): Tensor with abs applied element-wise.

log

1
log(tensor: Value)

Compute element-wise natural logarithm of a tensor.

Parameters:

  • tensor (Value): The input tensor.

Returns:

  • result (Value): Tensor with log applied element-wise.

sin

1
sin(tensor: Value)

Compute element-wise sine of a tensor.

Parameters:

  • tensor (Value): The input tensor.

Returns:

  • result (Value): Tensor with sin applied element-wise.

cos

1
cos(tensor: Value)

Compute element-wise cosine of a tensor.

Parameters:

  • tensor (Value): The input tensor.

Returns:

  • result (Value): Tensor with cos applied element-wise.

tan

1
tan(tensor: Value)

Compute element-wise tangent of a tensor.

Parameters:

  • tensor (Value): The input tensor.

Returns:

  • result (Value): Tensor with tan applied element-wise.

Backend support

This API reference describes frontend operations and mathematical semantics. Current EVT0 lowering supports recognized attention/normalization patterns, softmax with a supplied maximum, and standalone reduce_max under the constraints below. A known unsupported standalone reduction normally fails executor assignment with headline LE0121 and a recorded reason such as LE2119 (non-last axis) or LE2120 (missing keepdim=True). softmax(scores, -1) emits a maximum reduction and must satisfy its target constraints too. See the compile diagnostics example and the validated Math and masking and Causal attention examples.

Likewise, standalone full/full_like and general mask compositions do not currently lower to EVT0 binaries. Prefer the supported attention_scores pattern for causal attention, or supply masked scores and their post-mask maximum explicitly. Frontend MLIR alone does not test these backend paths.

On EVT1, last-axis BF16 softmax can start with reduce_max(scores, -1, True) and continue through sub → exp → reduce_sum(..., -1, True) → div. The simulator regression uses [32, 256] inputs, and the reduced width must be a multiple of 64 BF16 elements. This recognized pattern does not establish standalone support for every reduction or dtype on EVT1. See the EVT1 coverage table.

Reduction operations

A reduction collapses one axis. It is not a full-tensor fold: every other dimension is kept. keepdim=False (default) deletes axis; keepdim=True replaces it with size 1 so the result still broadcasts against the input (scores - reduce_max(scores, -1, True)).

On Bertha EVT0, standalone reduce_max uses the VPU for row-major FP16, BF16, and FP32 tensors when keepdim=True and the reduced axis is the last axis. Its logical width must be static and between 1 and 16384 elements. Prefix dimensions may be dynamic, and aligned row views may have a larger physical pitch. Physical capacity must hold the logical width rounded up to 64 elements. Aligned rows need no fill; otherwise lowering fills only the consumed source tail with -inf as required by the input contract.

1
2
3
4
5
6
7
8
9
# input shape S, axis a, keepdim k
if k:
    out_shape = S with S[a] replaced by 1
else:
    out_shape = S with axis a removed

for out_idx in indices(out_shape):
    slice = input along axis a at out_idx
    result[out_idx] = fold(slice)     # max / min / sum

fold is:

1
2
3
reduce_max:  acc = -inf;  acc = max(acc, x)
reduce_min:  acc = +inf;  acc = min(acc, x)
reduce_sum:  acc = 0;     acc = acc + x

Only logical elements participate. A physically padded tail is the identity of that fold (-inf / +inf / 0), not extra data.

axis must be an interpreted integer. The frontend accepts rank-dropping keepdim=False, but Bertha physical-shape inference can assert when that result has an incompatible physical extent. Use keepdim=True in attention/softmax patterns; this does not add standalone backend support to generic reductions. The _KEEPDIM_DEFAULT binding is listed with each API below.

1
2
row_max = legato.math.reduce_max(masked, -1, True)   # [rows, 1]
denom = legato.math.reduce_sum(exp_scores, -1, True)

reduce_max

1
reduce_max(tensor: Value, axis: Value, keepdim: Value = _KEEPDIM_DEFAULT)

Compute maximum along an axis.

Parameters:

  • tensor (Value): The input tensor.
  • axis (Value): Axis to reduce along. Negative values count from the end.
  • keepdim (Value): If True, the reduced axis is retained with size 1. Defaults to False. Default: _KEEPDIM_DEFAULT.

Returns:

  • result (Value): Reduced tensor. If keepdim=False (default), the axis is removed; if keepdim=True, the axis dimension becomes 1.
1
2
reduce_max(tensor, axis, keepdim=_KEEPDIM_DEFAULT)
_KEEPDIM_DEFAULT = False

reduce_min

1
reduce_min(tensor: Value, axis: Value, keepdim: Value = _KEEPDIM_DEFAULT)

Compute minimum along an axis.

Parameters:

  • tensor (Value): The input tensor.
  • axis (Value): Axis to reduce along. Negative values count from the end.
  • keepdim (Value): If True, the reduced axis is retained with size 1. Defaults to False. Default: _KEEPDIM_DEFAULT.

Returns:

  • result (Value): Reduced tensor. If keepdim=False (default), the axis is removed; if keepdim=True, the axis dimension becomes 1.
1
2
reduce_min(tensor, axis, keepdim=_KEEPDIM_DEFAULT)
_KEEPDIM_DEFAULT = False

reduce_sum

1
reduce_sum(tensor: Value, axis: Value, keepdim: Value = _KEEPDIM_DEFAULT)

Compute sum along an axis.

Parameters:

  • tensor (Value): The input tensor.
  • axis (Value): Axis to reduce along. Negative values count from the end.
  • keepdim (Value): If True, the reduced axis is retained with size 1. Defaults to False. Default: _KEEPDIM_DEFAULT.

Returns:

  • result (Value): Reduced tensor. If keepdim=False (default), the axis is removed; if keepdim=True, the axis dimension becomes 1.
1
2
reduce_sum(tensor, axis, keepdim=_KEEPDIM_DEFAULT)
_KEEPDIM_DEFAULT = False

Softmax

softmax is shape-preserving: result.shape == tensor.shape. It is the numerically stable map

1
2
3
4
5
# tensor X, axis a, shift M  (M is the per-fiber maximum)
shifted = X - M                         # broadcast M onto a
numer   = exp(shifted)
denom   = reduce_sum(numer, a, keepdim=True)
result  = numer / denom

When maximum is omitted, the frontend emits a keepdim=True reduce_max on axis and then compute.softmax. When you pass maximum, only compute.softmax is emitted — the compiler does not check that maximum equals max(X, axis).

maximum may keep axis as size 1 or squeeze it. Dtype and the non-reduced extents must match X. Its values are the caller's contract:

  • Prefer a finite max of the post-mask tensor.
  • Reusing a pre-mask max is only an upper bound. A large gap, or a row that is all -inf after masking, can underflow to 0/0 = NaN.

Softmax is group-agnostic. A GQA score tensor [group_size * q_len, seq_k] is ordinary rows; it does not unpack heads.

Last-axis softmax becomes rank-1 row leaves; see Broadcasting: last-axis rows.

1
2
probs = legato.math.softmax(scores, -1)              # emits reduce_max
probs = legato.math.softmax(scores, -1, row_max)     # reuse a computed max

softmax

1
softmax(tensor: Value, axis: Value, maximum: Value | None = None)

Numerically-stable softmax over axis.

Emits a keepdim compute.reduce_max before compute.softmax when maximum is omitted or None. A supplied maximum emits only compute.softmax. Group-agnostic: it treats the grouped [group_size * q_len, seq_k] score tensor as ordinary rows.

Parameters:

  • tensor (Value): The input scores (compiled).
  • axis (Value): Softmax axis. Negative values count from the end.
  • maximum (Value | None): Optional compiled precomputed shift with matching dtype and runtime non-reduced extents. It may retain axis with size one or squeeze it, and must have sufficient storage and bounds. The caller is responsible for its numerical values. For numerical stability, prefer a finite maximum of the actual post-mask tensor. Reusing a pre-mask maximum across nonpositive masks supplies only an upper bound; large gaps or all-masked rows can underflow or produce NaN. These numerical preconditions are not checked by the compiler. Default: None.

Returns:

  • result (Value): Softmax probabilities, the same shape as tensor.

Attention scores

attention_scores is scaled Q @ Kᵀ with optional causal masking. key is already matmul-ready (Kᵀ): there is no implicit transpose.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
# query : [B*, q_len, G * d_k]
# key   : [B*, d_k, seq_k]          # K^T
# result: [B*, G * q_len, seq_k]    # token-major per group
# scale > 0, G = group_size >= 1
# mask_start is 0 when omitted; legal only if causal is true

B* = broadcast(query.shape[:-2], key.shape[:-2])   # same rule as matmul

for g in 0 .. G-1:
    Qg = query[..., g*d_k : (g+1)*d_k]             # [B*, q_len, d_k]
    band = (Qg @ key) * scale                      # [B*, q_len, seq_k]
    result[..., g*q_len : (g+1)*q_len, :] = band
    if causal:
        for i in 0 .. q_len-1:
            for j in 0 .. seq_k-1:
                if j > i + mask_start:
                    result[..., g*q_len + i, j] = -inf

G = 1 is ordinary single-head / per-head attention: [q_len, d_k] @ [d_k, seq_k] → [q_len, seq_k].

G > 1 is grouped-query packing. Adjacent d_k bands of query share one key. The result stacks those G score matrices token-major ([G * q_len, seq_k]), not head-major. The causal diagonal is the token index i inside each band, not the packed row g*q_len + i.

mask_start shifts the diagonal for decode: a new token at cache offset past_len uses mask_start=past_len so it attends [0, past_len] and masks the rest of the K window. A constant mask_start must satisfy 0 <= mask_start < seq_k when seq_k is static.

scale is an interpreted positive float (typically 1/sqrt(d_k)). causal and group_size are interpreted. Query and key must share a dtype and have rank ≥ 2. Static inner dims must satisfy query.shape[-1] == group_size * key.shape[-2].

1
2
3
scores = legato.math.attention_scores(q, k_t, 1.0 / (d_k ** 0.5), True, 1)
scores = legato.math.attention_scores(q, k_t, scale, True, 1, past_len)
scores = legato.math.attention_scores(q_g, k_g_t, scale, True, group_size)

attention_scores

1
attention_scores(query: Value, key: Value, scale: Value, causal: Value, group_size: Value, mask_start: Value | None = None)

Compute scaled grouped-query attention scores (QK^T) with optional causal masking.

For group_size > 1, query packs adjacent d_k bands that share one K head. Scores are token-major [group_size * q_len, seq_k].

Parameters:

  • query (Value): Query tensor [q_len, group_size * d_k] (compiled).
  • key (Value): Key tensor in matmul-ready K^T form [d_k, seq_k] (compiled).
  • scale (Value): Score multiplier, e.g. 1/sqrt(d_k) (interpreted float).
  • causal (Value): Whether to apply the causal diagonal mask (interpreted bool).
  • group_size (Value): Grouped-query reuse factor, >= 1 (interpreted int).
  • mask_start (Value | None): Optional causal-mask offset index (compiled); valid only when causal is true. Defaults to no offset. Default: None.

Returns:

  • result (Value): The scaled (and optionally masked) scores [group_size * q_len, seq_k].

Grouped matrix multiplication

legato.math.matmul(lhs, rhs, group_size) is not Python @. @ is ordinary matmul (Arithmetic Operations). This helper is the GQA P @ V companion to attention_scores: token-major lhs rows are partitioned into G bands that share one rhs.

1
2
3
4
5
6
7
8
9
# lhs : [B*, G * M, K]
# rhs : [B*, K, N]
# out : [B*, M, G * N]
# G = group_size >= 1; static lhs.shape[-2] must be divisible by G

B* = broadcast(lhs.shape[:-2], rhs.shape[:-2])

for g in 0 .. G-1:
    out[..., :, g*N : (g+1)*N] = lhs[..., g*M : (g+1)*M, :] @ rhs

G = 1 is ordinary [M, K] @ [K, N] → [M, N]. G > 1 writes the G head results as adjacent column bands ([M, G*N]), which is the concat order used by grouped-query attention.

1
2
3
# scores_g is [G * q_len, seq_k], v_g is [seq_k, d_k]
# result is [q_len, G * d_k] — heads concatenated on the last dim
ctx = legato.math.matmul(probs_g, v_g, group_size)

matmul

1
matmul(lhs: Value, rhs: Value, group_size: Value)

Grouped P@V matrix multiplication.

For group_size > 1, lhs is token-major [group_size * M, K] and the result is banded [M, group_size * N].

Parameters:

  • lhs (Value): Left tensor [group_size * M, K] (compiled).
  • rhs (Value): Right (shared) tensor [K, N] (compiled).
  • group_size (Value): Grouped reuse factor, >= 1 (interpreted int).

Returns:

  • result (Value): The product [M, group_size * N].

Spatial geometry

Conv, depthwise conv, and pooling share one geometry: channels-last layout, per-side padding tuples, and one output-length formula per spatial axis. The subsections below are the contract for defaults and result shape. Window sampling is after that.

Tuple layout

Numeric geometry is always a Python tuple (or list) of ints. It is not PyTorch's scalar padding=1 or symmetric (H, W) pad pair.

Rank stride / dilation / kernel_size padding
1D (X,) (left, right)
2D (H, W) (top, bottom, left, right)

Length is checked. A 1D call with stride=(1, 1) or a 2D call with padding=(1, 1) is a frontend error. Each call's default constants are listed under that API.

Output shape

The result rank equals the input rank. Leading N* (everything before the spatial suffix) is copied unchanged. Each spatial axis is replaced by the formula below. The last axis is the output channel.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
# Split the input. spatial_rank is 1 or 2.
#   1D suffix = [I,  C_in]
#   2D suffix = [IH, IW, C_in]
N*     = input.shape[:-spatial_rank-1]
I[...] = input.shape[-spatial_rank-1 : -1]     # one or two spatial extents
C_in   = input.shape[-1]

# Kernel and output channel depend on the op.
if dwconv:
    K[...] = weight.shape[:spatial_rank]       # [KX] or [KH, KW]
    C_out  = weight.shape[-1]                  # must equal C_in
    dil[...] = 1
elif conv:
    K[...] = weight.shape[:spatial_rank]
    C_in_w = weight.shape[spatial_rank]        # must equal C_in
    C_out  = weight.shape[-1]
    dil[...] = dilation                        # default all ones
elif pool:
    K[...] = kernel_size
    C_out  = C_in
    dil[...] = 1

# Per-axis output length (integer division, same as the frontend
# `_spatial_output_dim` / Core `inferSpatialOutputDim`).
def spatial_out(I, K, stride, dilation, pad_before, pad_after):
    if I is dynamic or K is dynamic:
        return dynamic                          # written ? / -1
    effective = dilation * (K - 1) + 1
    O = (I + pad_before + pad_after - effective) // stride + 1
    if O <= 0:
        error                                   # non-positive output
    return O

# 1D padding is (left, right); 2D is (top, bottom, left, right).
# Axis d uses padding[2*d], padding[2*d+1] and stride[d], dil[d], K[d].
O[...] = [spatial_out(I[d], K[d], stride[d], dil[d],
                      padding[2*d], padding[2*d+1])
          for d in spatial axes]

result.shape = N* ++ O[...] ++ [C_out]

padding_mode does not change O. Reflect / replicate / circular only change which input element a tap reads; the padded length used in the formula is still I + pad_before + pad_after.

If a spatial I is dynamic (-1 / ?), that O stays dynamic. A static I and K always produce a static O, and that O must be > 0. stride, dilation, and kernel_size must be positive; padding entries must be ≥ 0.

N* may be empty ([X, C] is already a leaf) or several batch dims ([2, 4, 8, 64] 1D conv keeps [2, 4] and only rewrites 8 → Xout). The weight / kernel_size never contribute a batch axis. Worked numbers for each call sit with that API.

How a padded coordinate is sampled

A spatial index p lives in padded space [-pad_before, I + pad_after). The source index is s = p.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
def sample(x, s, I, mode, *, pad_value):
    if 0 <= s < I:
        return x[s]
    if mode == "zeros":
        return pad_value
    if mode == "replicate":
        return x[0] if s < 0 else x[I - 1]
    if mode == "circular":
        return x[s - I * floor(s / I)]  # wrap; not toward-zero remainder
    if mode == "reflect":
        # Bounce over the endpoints; the edge is not repeated.
        # s = -1 -> 1, s = I -> I-2. Requires I > 1.
        while s < 0 or s >= I:
            if s < 0:
                s = -s
            else:
                s = 2 * (I - 1) - s
        return x[s]

2D applies the 1D rule independently to H and W.

padding_mode is a conv-only string from _VALID_PADDING_MODES ("zeros", "reflect", "replicate", "circular"). Depthwise conv is implicit "zeros". Pooling has no mode: max pooling uses pad_value = -inf so a pad never wins the window; average pooling uses pad_value = 0 in the sum but still divides by the full kernel area (see Pooling). A target may reject some modes; that is a backend rule, not part of this contract.

Window walk

For one output coordinate o on one axis:

1
2
3
4
origin = o * stride - pad_before
for t in 0 .. K-1:
    s = origin + t * dilation          # dilation is 1 for pool / dwconv
    tap[t] = sample(..., s, ...)

Depthwise convolution

Each channel is an independent FIR. There is no Cin → Cout mix, no bias, and no dilation / padding_mode argument (dilation is always 1; pads are zeros).

1
2
3
4
5
6
7
# dwconv_1d: input [N*, X, C],  weight [KX, C]          → [N*, Xout, C]
# dwconv_2d: input [N*, H, W, C], weight [KH, KW, C]    → [N*, Hout, Wout, C]

for n in N*:
    for c in 0 .. C-1:
        result[n, ..., c] = conv1d_or_2d(input[n, ..., c], weight[..., c],
                                         stride, padding, dilation=1)

Unrolled 1D:

1
2
3
4
5
6
7
8
# Xout from spatial_out with K=KX, dilation=1
for n, xo, c in indices(N*, Xout, C):
    acc = 0
    origin = xo * stride_x - pad_left
    for t in 0 .. KX-1:
        xi = origin + t
        acc += sample_zeros(input[n, :, c], xi, X) * weight[t, c]
    result[n, xo, c] = acc

2D adds a KH loop over H with (pad_top, pad_bottom, stride_h). C on the weight must match C on the input. Kernel dims are taps, not batch.

dwconv_1d

1
dwconv_1d(input: Value, weight: Value, stride: Value = _DWCONV1D_STRIDE_DEFAULT, padding: Value = _DWCONV1D_PADDING_DEFAULT)

Channels-last 1D depthwise convolution.

input is [N*, X, C], weight is [KX, C], and the result is [N*, Xout, C]. stride has length 1; padding is (left, right).

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
dwconv_1d(input, weight, stride=_DWCONV1D_STRIDE_DEFAULT, padding=_DWCONV1D_PADDING_DEFAULT)

_DWCONV1D_STRIDE_DEFAULT  = (1,)       # (X,)
_DWCONV1D_PADDING_DEFAULT = (0, 0)     # (left, right)

# no bias, no dilation, no padding_mode — dilation is 1, pads are zeros
# result [N*, Xout, C]
#   Xout = spatial_out(X, KX, stride[0], 1, padding[0], padding[1])
# default call dwconv_1d(x, w) with x [19, 64], w [4, 64]:
#   Xout = (19 + 0 + 0 - 4) // 1 + 1 = 16  →  [16, 64]

dwconv_2d

1
dwconv_2d(input: Value, weight: Value, stride: Value = _CONV2D_STRIDE_DEFAULT, padding: Value = _CONV2D_PADDING_DEFAULT)

Channels-last 2D depthwise convolution.

input is [N*, H, W, C], weight is [KH, KW, C], and the result is [N*, Hout, Wout, C]. stride is (H, W); padding is (top, bottom, left, right).

1
2
3
4
5
6
7
8
9
dwconv_2d(input, weight, stride=_CONV2D_STRIDE_DEFAULT, padding=_CONV2D_PADDING_DEFAULT)

_CONV2D_STRIDE_DEFAULT  = (1, 1)          # (H, W)  — shared with conv_2d
_CONV2D_PADDING_DEFAULT = (0, 0, 0, 0)    # (top, bottom, left, right)

# no bias, no dilation, no padding_mode — dilation is 1, pads are zeros
# result [N*, Hout, Wout, C]
#   Hout = spatial_out(H, KH, stride[0], 1, padding[0], padding[1])
#   Wout = spatial_out(W, KW, stride[1], 1, padding[2], padding[3])

Convolution

Dense (non-depthwise) convolution mixes Cin into Cout and then optionally adds a rank-1 bias. Same channels-last layout and N* rule as depthwise; the weight gains an output-channel axis.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
# conv_1d: input [N*, X, Cin],    weight [KX, Cin, Cout]         → [N*, Xout, Cout]
# conv_2d: input [N*, H, W, Cin], weight [KH, KW, Cin, Cout]     → [N*, Hout, Wout, Cout]
# bias (optional): [Cout]

for n in N*:
    for cout in 0 .. Cout-1:
        acc = 0
        for cin in 0 .. Cin-1:
            acc += conv(input[n, ..., cin], weight[..., cin, cout],
                        stride, dilation, padding, padding_mode)
        if bias is not None:
            acc += bias[cout]
        result[n, ..., cout] = acc

Unrolled 1D with padding_mode:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
for n, xo, cout in indices(N*, Xout, Cout):
    acc = 0
    origin = xo * stride_x - pad_left
    for t in 0 .. KX-1:
        xi = origin + t * dilation_x
        for cin in 0 .. Cin-1:
            acc += sample(input[n, :, cin], xi, X, padding_mode, pad_value=0)
                    * weight[t, cin, cout]
    if bias is not None:
        acc += bias[cout]
    result[n, xo, cout] = acc

Cin on the input must match Cin on the weight. Bias, if present, must be rank-1 [Cout] with the same dtype. The frontend emits compute.conv.

conv_1d

1
conv_1d(input: Value, weight: Value, bias: Value | None = None, stride: Value = _CONV1D_STRIDE_DEFAULT, dilation: Value = _CONV1D_DILATION_DEFAULT, padding: Value = _CONV1D_PADDING_DEFAULT, padding_mode: Value = _PADDING_MODE_DEFAULT)

Channels-last 1D convolution.

input is [N*, X, Cin], weight is [KX, Cin, Cout], optional bias is [Cout], and the result is [N*, Xout, Cout]. stride and dilation have length 1; padding is (left, right).

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
conv_1d(input, weight, bias=None,
        stride=_CONV1D_STRIDE_DEFAULT,
        dilation=_CONV1D_DILATION_DEFAULT,
        padding=_CONV1D_PADDING_DEFAULT,
        padding_mode=_PADDING_MODE_DEFAULT)

_CONV1D_STRIDE_DEFAULT   = (1,)       # (X,)
_CONV1D_DILATION_DEFAULT = (1,)       # (X,) — one dilation, value 1
_CONV1D_PADDING_DEFAULT  = (0, 0)     # (left, right)
_PADDING_MODE_DEFAULT    = "zeros"

# result [N*, Xout, Cout]
#   effective = dilation[0] * (KX - 1) + 1
#   Xout = (X + padding[0] + padding[1] - effective) // stride[0] + 1
# conv_1d(x, w) with x [8, 64], w [3, 64, 64] uses every default:
#   Xout = (8 + 0 + 0 - (1*(3-1)+1)) // 1 + 1 = 6  →  [6, 64]
# conv_1d(x, w, None, (1,), (1,), (1, 1), "zeros"):
#   Xout = (8 + 1 + 1 - 3) // 1 + 1 = 8  →  [8, 64]

conv_2d

1
conv_2d(input: Value, weight: Value, bias: Value | None = None, stride: Value = _CONV2D_STRIDE_DEFAULT, dilation: Value = _CONV2D_DILATION_DEFAULT, padding: Value = _CONV2D_PADDING_DEFAULT, padding_mode: Value = _PADDING_MODE_DEFAULT)

Channels-last 2D convolution.

input is [N*, H, W, Cin], weight is [KH, KW, Cin, Cout], optional bias is [Cout], and the result is [N*, Hout, Wout, Cout]. stride and dilation are (H, W); padding is (top, bottom, left, right).

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
conv_2d(input, weight, bias=None,
        stride=_CONV2D_STRIDE_DEFAULT,
        dilation=_CONV2D_DILATION_DEFAULT,
        padding=_CONV2D_PADDING_DEFAULT,
        padding_mode=_PADDING_MODE_DEFAULT)

_CONV2D_STRIDE_DEFAULT   = (1, 1)          # (H, W)
_CONV2D_DILATION_DEFAULT = (1, 1)          # (H, W) — both axes 1
_CONV2D_PADDING_DEFAULT  = (0, 0, 0, 0)    # (top, bottom, left, right)
_PADDING_MODE_DEFAULT    = "zeros"

# result [N*, Hout, Wout, Cout]
#   Hout = spatial_out(H, KH, stride[0], dilation[0], padding[0], padding[1])
#   Wout = spatial_out(W, KW, stride[1], dilation[1], padding[2], padding[3])
# conv_2d(x, w) with default pads: Hout = H - KH + 1, Wout = W - KW + 1

Pooling

Pooling walks the same windows as a unit-dilation conv, then reduces the window per channel. There is no weight.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
# 1D: input [N*, X, C]       → [N*, Xout, C]
# 2D: input [N*, H, W, C]    → [N*, Hout, Wout, C]
# kernel_size is required; dilation is always 1

area = product(kernel_size)              # KX, or KH * KW

for n, o..., c in result indices:
    window = sampled taps at o (dilation=1)
    if max:
        result[n, o..., c] = max(window)          # pad taps are -inf
    if avg:
        result[n, o..., c] = sum(window) / area   # pad taps are 0

Average pooling always divides by the full kernel area, including pads (count_include_pad=True in PyTorch terms). A window that hangs off the edge is therefore smaller in the numerator than in the denominator. This is the Core contract, not avg_pool with count_include_pad=False.

Max pooling pads with -inf, so a pad cannot become the window max unless the whole window is padded (that geometry is rejected because O would be non-positive or the window would be empty of real taps only if pads fill it — the -inf then stays).

max_pool_1d

1
max_pool_1d(input: Value, kernel_size: Value, stride: Value | None = None, padding: Value = _POOL1D_PADDING_DEFAULT)

Channels-last 1D max pooling over [N*, X, C].

1
2
3
4
5
6
7
8
max_pool_1d(input, kernel_size, stride=None, padding=_POOL1D_PADDING_DEFAULT)

_POOL1D_PADDING_DEFAULT = (0, 0)     # (left, right)
# stride has no named constant: None means stride = kernel_size
# result [N*, Xout, C]
#   Xout = spatial_out(X, KX, stride[0], 1, padding[0], padding[1])
# max_pool_1d(x, (2,)) with x [8, 64]: stride becomes (2,)
#   Xout = (8 + 0 + 0 - 2) // 2 + 1 = 4  →  [4, 64]

avg_pool_1d

1
avg_pool_1d(input: Value, kernel_size: Value, stride: Value | None = None, padding: Value = _POOL1D_PADDING_DEFAULT)

Channels-last 1D average pooling over [N*, X, C].

1
2
3
4
5
6
avg_pool_1d(input, kernel_size, stride=None, padding=_POOL1D_PADDING_DEFAULT)

_POOL1D_PADDING_DEFAULT = (0, 0)     # (left, right)
# stride=None → kernel_size (same rule as max_pool_1d)
# result [N*, Xout, C]; divisor is KX, including pads
# avg_pool_1d(x, (2,)) with x [8, 64]: Xout = 4

max_pool_2d

1
max_pool_2d(input: Value, kernel_size: Value, stride: Value | None = None, padding: Value = _POOL2D_PADDING_DEFAULT)

Channels-last 2D max pooling over [N*, H, W, C].

1
2
3
4
5
6
7
max_pool_2d(input, kernel_size, stride=None, padding=_POOL2D_PADDING_DEFAULT)

_POOL2D_PADDING_DEFAULT = (0, 0, 0, 0)    # (top, bottom, left, right)
# stride=None → kernel_size
# result [N*, Hout, Wout, C]
#   Hout = spatial_out(H, KH, stride[0], 1, padding[0], padding[1])
#   Wout = spatial_out(W, KW, stride[1], 1, padding[2], padding[3])

avg_pool_2d

1
avg_pool_2d(input: Value, kernel_size: Value, stride: Value | None = None, padding: Value = _POOL2D_PADDING_DEFAULT)

Channels-last 2D average pooling over [N*, H, W, C].

1
2
3
4
avg_pool_2d(input, kernel_size, stride=None, padding=_POOL2D_PADDING_DEFAULT)

_POOL2D_PADDING_DEFAULT = (0, 0, 0, 0)    # (top, bottom, left, right)
# stride=None → kernel_size; divisor is KH * KW, including pads

Normalization

Both ops normalize one axis and keep the input shape. epsilon is an interpreted finite ≥ 0 scalar added under the square root for stability. The divisor is the logical axis length D, never the physically padded width. _AXIS_DEFAULT and _EPSILON_DEFAULT are listed with each API below.

Affine vectors are per-channel along that axis: rank-1 [D], or the same rank as input with leading dims 1 and last dim D. They broadcast across the non-reduced dims. Dtype must match input.

Last-axis tensors become rank-1 row leaves; see Broadcasting: last-axis rows.

RMSNorm

weight is required. There is no mean subtraction and no bias.

1
2
3
4
5
# input X, weight W, axis a, epsilon ε
D   = X.shape[a]                                # logical
ms  = reduce_sum(X * X, a, keepdim=True) / D    # mean of squares
rms = sqrt(ms + ε)
result = (X / rms) * W

This is x / sqrt(mean(x²) + ε) * weight.

LayerNorm

Mean-center, then scale by the biased standard deviation (/ D, not / (D-1)). weight and bias are independently optional.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
# input X, optional weight W, optional bias B, axis a, epsilon ε
D        = X.shape[a]
mean     = reduce_sum(X, a, keepdim=True) / D
centered = X - mean
var      = reduce_sum(centered * centered, a, keepdim=True) / D
std      = sqrt(var + ε)
result   = centered / std
if W is not None:
    result = result * W
if B is not None:
    result = result + B

Order is fixed: center → divide by sqrt(var + ε) → scale → shift. Omitting both affines is still LayerNorm (normalize only).

1
2
3
4
y = legato.math.rms_norm(x, gamma)                    # axis=-1, ε=1e-5
y = legato.math.layer_norm(x, gamma, beta)
y = legato.math.layer_norm(x)                         # normalize only
y = legato.math.layer_norm(x, None, None, -1, 1e-6)

rms_norm

1
rms_norm(input: Value, weight: Value, axis: Value = _AXIS_DEFAULT, epsilon: Value = _EPSILON_DEFAULT)

Root-mean-square normalization along axis with a required affine weight.

1
2
3
4
5
rms_norm(input, weight, axis=_AXIS_DEFAULT, epsilon=_EPSILON_DEFAULT)

_AXIS_DEFAULT    = -1
_EPSILON_DEFAULT = 1e-5
# weight is required; result.shape == input.shape

layer_norm

1
layer_norm(input: Value, weight: Value | None = None, bias: Value | None = None, axis: Value = _AXIS_DEFAULT, epsilon: Value = _EPSILON_DEFAULT)

Mean-centered layer normalization along axis with optional affine operands.

1
2
3
4
5
layer_norm(input, weight=None, bias=None, axis=_AXIS_DEFAULT, epsilon=_EPSILON_DEFAULT)

_AXIS_DEFAULT    = -1
_EPSILON_DEFAULT = 1e-5
# weight and bias are independently optional