Skip to content

Arithmetic Operations

Legato arithmetic uses Python operators inside compiled functions. Operators work on scalar values and tensors when their types and shapes are compatible.

Elementwise Arithmetic

1
2
3
4
5
6
add = a + b
sub = a - b
mul = a * b
div = a / b
floor_div = a // b
power = a**b

Tensor arithmetic follows the compiler's tensor shape rules, including broadcasting.

On EVT1, matching-shape BF16 tensor subtraction, multiplication and division now lower to the VPU alongside addition. The numerical coverage uses [16, 256] inputs. Select backend=bertha.evt1 explicitly; this does not imply EVT1 support for every broadcast, dtype or other operator in the list above. See Verified EVT1 paths.

Matrix Multiplication

Use Python's @ operator for matrix multiplication.

1
result = lhs @ rhs

On Bertha, matmul weights must use a storage kind accepted by the selected backend. The compiler prepares compatible SRAM weights for computation while the tensor remains in SRAM.

An SRAM weight may be read again for a batch prefix. Unit leading dimensions are squeezed, and varying batch dimensions are sliced per iteration. A "mpu_buffer" stream cannot be reused as a persistent SRAM tensor; unsupported replay reports LE0184. See Weight storage requirements.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
# Streamed one-shot matmul (no batch replay).
weight_type = legato.get_model().tensor_type(
    legato.types.float("bfloat16"), [256, 128], "mpu_weight"
)
loaded_weight = legato.tensor.receive_type(weight_type, 0, None, "mpu_buffer")
result = loaded_input @ loaded_weight

# SRAM path: compiler packs to MPU tensor layout and can reload across N*.
sram_w = legato.tensor.receive(sram_weight, 0, "sram")
batched = batched_lhs @ sram_w

Batch rules are in Broadcasting. See Bertha examples.

Bitwise Operations

Bitwise operators are supported on integer scalars and follow Python syntax. >> is always an arithmetic (sign-extending) shift; there is no separate unsigned right-shift operator.

1
2
3
4
5
6
left = a << b
right = a >> b
bit_and = a & b
bit_or = a | b
bit_xor = a ^ b
bit_not = ~a

Comparisons

Comparisons such as <, <=, >, >=, ==, and != produce boolean values or boolean tensors.

On the current EVT0 backend, scalar integer/index ordering comparisons must use nonnegative values: signed ordering for negative operands is not guaranteed. This restriction also applies to comparisons controlling if, for, and while. Equality, boolean handling, and the EVT1 implementation are separate paths.

1
mask = scores < threshold

Boolean tensors are often combined with legato.tensor.where.

1
2
filled = legato.tensor.full_like(scores, neg_inf)
masked_scores = legato.tensor.where(mask, scores, filled)

Casts

Use legato.language.arith.cast to convert scalar values to a target Legato type.

1
2
3
4
from legato.language import arith

zero = arith.cast(0, legato.types.index())
scale = arith.cast(8.0, legato.types.float("bfloat16"))

Floating-point and integer scalar conversions

Import arith from legato.language, then use arith.fp2int(value, policy) for an explicit rounding policy. "round_to_even" rounds to the nearest integer with ties to even; "towards_zero" discards the fractional part. The policy is a compile-time string and the result is int32. These are scalar operations; cast is not a replacement for choosing the rounding policy.

arith.int2fp(value) converts an int16 or int32 scalar to float32. Cast wider integer types to int32 first when the value fits that type. Runtime backend support for these conversions is currently EVT0; a successful frontend compile does not establish EVT1 support.

1
2
3
4
5
6
from legato.language import arith

# Inside an execution context, with a float32 scalar x and an int32 scalar n:
nearest = arith.fp2int(x, "round_to_even")
truncated = arith.fp2int(x, "towards_zero")
floating = arith.int2fp(n)

fp2int

1
fp2int(value: Value, policy: Value)

Convert a float scalar to an int32 scalar.

Parameters:

  • value (Value): The float scalar to convert.
  • policy (Value): Interpreted rounding policy string. "round_to_even" rounds to the nearest integer, and ties go to the even integer. "towards_zero" drops the fractional part.

Returns:

  • The int32 scalar.

Example:

1
2
3
4
5
6
7
from legato.language import arith

@legato.compile(backend=get_backend)
def fn(x: legato.types.float("float32")):
    device = legato.get_context().get_device()
    with device.get_core(0):
        i = arith.fp2int(x, "round_to_even")

int2fp

1
int2fp(value: Value)

Convert an int16 or int32 scalar to a float32 scalar.

Parameters:

  • value (Value): The integer scalar to convert. Cast other integer types to int32 first.

Returns:

  • The float32 scalar.

Example:

1
2
3
4
5
6
7
from legato.language import arith

@legato.compile(backend=get_backend)
def fn(x: legato.types.integer("int32")):
    device = legato.get_context().get_device()
    with device.get_core(0):
        f = arith.int2fp(x)

Math And Reductions

Elementwise math and reductions live under legato.math.

1
2
3
row_max = legato.math.reduce_max(scores, -1)
exp_scores = legato.math.exp(scores - row_max)
denom = legato.math.reduce_sum(exp_scores, -1)

See Math Operations for the full legato.math API.