Skip to content

Types

Legato is typed. Type annotations describe scalar widths, tensor element types, tensor shapes, and compile-time values.

Scalar Types

1
2
3
4
5
flag: legato.types.bool()
i64: legato.types.integer("int64")
idx: legato.types.index()
f32: legato.types.float("float32")
bf16: legato.types.float("bfloat16")

Python integer literals default to a signless integer type and Python floating point literals default to float32 unless a cast or annotation says otherwise.

Use legato.language.arith.cast when a value must become a specific Legato type.

1
2
3
4
from legato.language import arith

zero = arith.cast(0, legato.types.index())
scale = arith.cast(8.0, legato.types.float("bfloat16"))

Tensor Types

Generic tensor types use legato.types.tensor.

1
local_type = legato.types.tensor(legato.types.float("bfloat16"), [10, 512])

Backend-aware tensor types use legato.get_model().tensor_type.

1
2
3
4
5
dram_type = legato.get_model().tensor_type(
    legato.types.float("bfloat16"),
    ((-1, 10), 512),
    "shared_dram",
)

The third argument is a backend-specific kind. For Bertha, common kinds include "shared_dram", "row_major", "mpu_weight", and "sliced_dram". "value_dram" and "mpu_dram" select the same value-style DRAM layout; "key_dram" and "transposed_mpu_dram" select the same key-style DRAM layout. "instruction_table" marks a small host constant table whose data is embedded directly into the kernel's instruction table rather than living in device memory; at launch it takes a CPU torch.Tensor (see Instruction Table (Host) Arguments).

Shape Syntax

Shapes can contain:

  • fixed dimensions, such as [10, 512],
  • unconstrained dynamic dimensions, such as -1,
  • bounded dynamic dimensions, such as (-1, 10).
1
2
3
4
5
bounded = legato.get_model().tensor_type(
    legato.types.float("bfloat16"),
    ((-1, 10), 64),
    "shared_dram",
)

Use legato.tensor.get_dimension(tensor, index) to read a runtime dimension.

1
2
seq_len = legato.tensor.get_dimension(x, 0)
view = legato.tensor.view(x, (legato.range(0, seq_len), legato.range(0, 64)))

Logical shape and physical capacity

A dimension pair (logical, physical) describes a logical extent and a physical capacity/pitch, for example (4, (70, 256)) for four logical 70-element rows stored in 256-element rows. (-1, max_dim) is the bounded dynamic form. Use a tuple for mixed fixed and paired dimensions; a heterogeneous Python list is not a valid Legato list value.

These annotations do not allocate padding. The runtime tensor's backing storage and strides must match the declared physical dimensions. See Physical storage for host/device setup.

Tuples, Lists, And Ranges

Tuple and list types are useful for compile-time values and structured arguments.

1
2
3
4
5
pair_type = legato.types.tuple([
    legato.types.float("float32"),
    legato.types.float("float32"),
])
ids_type = legato.types.list(legato.types.integer("int64"))

legato.range(start, end, step) creates range values used by loops and tensor views.

1
2
3
rows = legato.range(0, seq_len)
cols = legato.range(0, 64)
subview = legato.tensor.view(x, (rows, cols))

Constexpr

Wrap a type with legato.types.constexpr(...) when a value must be known at compile time.

1
2
3
4
@legato.compile(backend=get_bertha)
def kernel(num_cores: legato.types.constexpr(legato.types.integer("int64"))):
    rng = legato.range(0, num_cores)
    ...

Constexpr values participate in compilation specialization. Calling the same compiled function with different constexpr values can produce different cached artifacts.

Interpreter And Device Values

Legato code operates in two worlds:

  • interpreted values are known while the compiler is generating IR,
  • compiled values are represented in IR and computed by the device.

Static helpers such as legato.static.fold require interpreted values. Tensor math and memory movement operate on compiled values.