Skip to content

Layouts And Memory Kinds

Legato tensor types describe both logical shape and physical placement. Generic tensor types can use explicit layout maps, while backend-aware tensor types use backend-specific memory/layout kind strings.

Generic Layout Maps

Layout maps describe how logical indices map to memory offsets.

1
2
3
4
5
6
7
8
import legato.language.types.layout as layout

tensor_type = legato.types.tensor(
    legato.types.float("float32"),
    [10, 10],
    None,
    layout.map("(d0, d1)[s0, s1] -> (d0 * s1 + d1)"),
)

In a layout expression:

  • d0, d1, ... are dimension indices.
  • s0, s1, ... are shape symbols.

Bertha Tensor Kinds

For Bertha kernels, prefer legato.get_model().tensor_type(dtype, shape, kind). The kind selects a predefined memory/layout combination.

Common kinds supported by the current implementation:

  • "shared_memory" and "shared_dram"
  • "row_major"
  • "value_dram" and "mpu_dram" for the same value-style DRAM layout
  • "key_dram" and "transposed_mpu_dram" for the same key-style DRAM layout
  • "mpu_weight"
  • "sliced_dram"
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
@legato.compile(backend=get_bertha)
def kernel(
    x: legato.get_model().tensor_type(
        legato.types.float("bfloat16"),
        ((-1, 10), 256),
        "shared_dram",
    ),
    out: legato.get_model().tensor_type(
        legato.types.float("bfloat16"),
        [10, 128],
        "shared_dram",
    ),
):
    ...

Use "row_major" when a Bertha tensor should use row-major layout in core-local SRAM. The older "row_major_sram" name has been replaced by "row_major", and "row_major_dram" is no longer an accepted Bertha kind.

For Bertha-specific layout details and alias pairs, see Bertha Layouts.

Dynamic Shapes

Dynamic dimensions can be written as (-1, max_dim).

1
2
3
4
dynamic_sram_type = legato.types.tensor(
    legato.types.float("bfloat16"),
    ((-1, 10), 512),
)

Runtime dimensions can be read from tensors and used in views or allocations.

1
2
dim0 = legato.tensor.get_dimension(input_tensor, 0)
tmp = legato.tensor.allocate(dynamic_sram_type, (dim0,))

When a result shape is fixed, pass dynamic_shape=None. For dynamic allocation, provide one runtime index per dynamic dimension. Do not add shape operands for fixed dimensions; constant shape operands can specialize a fill to a fixed result and cause a later operand-count mismatch.

The language can represent dynamic allocations; runtime support depends on the memory path. Tensor memory patterns states the verified stage of each example.

Views

Views create aliases with a new logical indexing region. They do not copy data.

1
2
3
4
front = legato.tensor.view(
    tensor,
    (legato.range(0, 5), legato.range(0, 256)),
)

Views are commonly used with memcpy to copy pieces of a tensor between memory kinds.