Bertha Layouts¶
Bertha tensors are usually declared with
legato.get_model().tensor_type(dtype, shape, kind). The kind string selects
both a Bertha memory object and a Bertha layout map.
This page describes the layout kinds supported by the current Bertha implementation. For generic Legato layout maps, see Layouts And Memory Kinds.
Supported Kind Strings¶
The following kind strings are accepted by Bertha.tensor_type (see
Target generations for backend limits):
"shared_memory"and"shared_dram""row_major""value_dram"and"mpu_dram""key_dram"and"transposed_mpu_dram""mpu_weight""mpu_weight_sram"for MPU-format weights resident in core SRAM"mpu_packed_buffer""sliced_dram""instruction_table"for host constants embedded in the instruction table
1 2 3 4 5 | |
"shared_memory" is an alias for the same shared-DRAM word-distributed layout
used by "shared_dram".
Row-Major Layouts¶
Use "row_major" for tensors in core-local SRAM with row-major indexing. This
kind replaces the older "row_major_sram" name.
Row-major layout maps logical indices to a flattened row-major offset:
1 | |
For a 2-D tensor this is equivalent to d0 * s1 + d1.
"row_major_dram" is no longer accepted by the Bertha constructor. Use "shared_dram" for shared DRAM tensors, or define an explicit
generic layout map with legato.types.tensor(...) if a custom row-major DRAM
type is required.
Shared DRAM Layouts¶
Use "shared_dram" or "shared_memory" for ordinary shared-DRAM tensors. These
kinds use a word-distributed DRAM layout across the configured channels.
The EVT0 defaults are 64-byte words and 8 DRAM channels.
The implementation computes:
- the row-major flattened element index,
- the element's position within a 64-byte word,
- the DRAM channel from the word index,
- and the local word offset inside the selected channel.
This layout is useful for tensors loaded from or stored to shared memory through standard Legato tensor movement operations.
Sliced DRAM¶
Use "sliced_dram" when a tensor should be split into equal contiguous slices
across the configured DRAM channels (8 by default on EVT0).
Constraints:
shapemust be available when the layout is built.- The total physical element count must be divisible by
params.num_memory_channels. - The dtype bit width must divide the Bertha channel size.
- Dynamic dimensions are allowed only when their maximum physical dimension is
known, as in
(-1, max_dim).
1 2 3 4 5 | |
Key And Value DRAM Aliases¶
"key_dram"and"transposed_mpu_dram"select the same key-style kind."value_dram"and"mpu_dram"select the same value-style kind.
Use names that communicate the tensor's role, such as key/value cache storage or matmul operands. Both kinds require rank at least 2, a dtype width of 4, 8, or 16 bits, and a shape accepted by the selected backend. Let the SDK construct the layout and check its constraints.
Channel address spacing is configured by params.dram_channel_stride_bytes
(64 GiB by default on EVT0); bytes_per_channel is channel capacity.
Overrides must match the selected device/runtime configuration.
Weight Layouts¶
Use "mpu_weight" for weights received through "mpu_buffer", or
"mpu_weight_sram" for compatible weights resident in core SRAM.
These kinds require rank at least 2 and a dtype width of 4, 8, or 16 bits.
| Tensor kind | Memory | Layout family |
|---|---|---|
"mpu_weight" |
MPU buffer | mpu_weight |
"mpu_weight_sram" |
Core SRAM | mpu_weight |
Construct these types with the Bertha kind so that they carry the layout-family tag used by executor assignment. MPU-format SRAM weights are supported as matmul RHS operands; they are rejected as matmul LHS operands or VPU arithmetic, reduction, or normalization inputs. The compiler can prepare compact row-major SRAM weights for the MPU while keeping the tensor in SRAM.
On EVT0, a value-style source [K, N] retains its logical shape when received
into SRAM; a key-style source [N, K] is received as [K, N].
Use a Bertha tensor kind to express this contract. A custom generic layout
does not necessarily satisfy the requirements for a weight operand.
1 2 3 4 5 6 7 | |
The compiler prepares compatible SRAM weights for matmul, convolution, and depthwise convolution while keeping their logical tensor shape. Applications should rely on the SDK's type checks and allocation/conversion helpers. See Weight storage requirements.
For mm_weight_quant / addmm_weight_quant, use "mpu_packed_buffer" for the
opaque buffer returned by the SDK weight converter. It requires rank 1 and an
i8 element type. Its shape is the converted byte count, not [C, M].
Keep the converted contents unchanged until all consumers have used them.
Memory Placement¶
Bertha.tensor_type maps layout kinds to memory objects as follows:
- Shared DRAM memory:
"sliced_dram","shared_dram","shared_memory", the key-style aliases"key_dram"and"transposed_mpu_dram", and the value-style aliases"value_dram"and"mpu_dram". - MPU buffer memory:
"mpu_weight"and"mpu_packed_buffer". - Core SRAM memory:
"row_major"and"mpu_weight_sram".
Memory placement and layout are selected together. If only a custom generic
layout map is needed, use legato.types.tensor(...) with
legato.language.types.layout.map(...) instead of a Bertha kind string.
Device Tensor Kinds (empty_with_kind)¶
The kind strings above are compile-time annotations. A device-resident input
tensor created on the host side carries its own layout kind, stamped by
torch.ops.lpu.empty_with_kind, which must match the parameter's annotation kind.
For the empty_with_kind name ↔ annotation kind matching table and the full
create → fill → launch workflow, see
PyTorch Integration.
Dynamic Shapes¶
Bertha tensor kinds use the same dynamic shape syntax as generic Legato tensor types:
1 2 3 4 5 | |
For layouts that need physical tensor size at compile time, such as
"sliced_dram", the maximum dimension is used as the physical dimension.
Physical alignment and shared loads¶
Constraints apply to physical element counts. On default EVT0 shared DRAM,
a physical row is aligned to 64 bytes: 32 BF16 elements or 16 FP32 elements.
SRAM rows use 128-byte words. A compact DRAM row may need padding/reformatting
when received into SRAM; the runnable examples use widths compatible with both.
An explicitly unaligned type reports LE0301. Its detailed alignment message
distinguishes the dtype-specific element multiple from the byte span: for
example, 64 BF16 elements or 32 FP32 elements occupy a 128-byte SRAM word.
See the compile diagnostics example.
Top-side shared-DRAM loads must cover whole DRAM lines. The line size is the configured channel count times the DRAM word size: 8 × 64 bytes = 512 bytes on default EVT0. For BF16 rows of width 64, ten rows occupy 1280 bytes and do not meet this condition; twelve rows occupy 1536 bytes and do. Padding only the physical first dimension does not change the logical transfer extent. Both the request and its backing allocation must be valid. Other transfer paths can have different requirements.
A view preserves its parent's physical pitch. Matmul and softmax lowering use that pitch while restricting math to logical elements. On the host, match the compiled layout with a real padded backing tensor, then pass its logical view; see Physical storage.