Memory Operations¶
Legato makes data movement explicit. A typical kernel issues loads from the top executor, receives the tensors in a core context, computes, and copies results back to the destination tensor.
Core Operations¶
request_load¶
1 | |
devices can be a single executor or memory, a list of executors/memories, or a
compiled bit mask for dynamic Bertha load requests. target_name identifies the
target buffer used by the receive side.
1 2 3 | |
receive¶
1 | |
Use receive when you already have a tensor value whose type and shape describe
the received data.
1 2 | |
receive_type¶
1 | |
Use receive_type when the received value is described by a type rather than an
existing tensor value. dynamic_shape is None for fully static shapes, or a
tuple of index-like values for dynamic dimensions.
1 2 3 4 5 6 | |
On Bertha, use static receive indices rather than forwarding the SSA handle
returned by request_load. receive(tensor, index, target) associates the
receive with that source tensor; receive_type(type, index, ...) selects a
receive slot in the target stream. For example, two weight requests consumed
with receive_type use slots 0 and 1. Target names must match on both sides.
The attention example demonstrates both forms.
Load Compute Store¶
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 | |
Copy, Move, And Memcpy¶
legato.tensor.copy(value, target)copies a tensor while preserving the source lifetime.legato.tensor.move(value, target)transfers ownership and invalidates the source value.legato.tensor.memcpy(dst, src)copies data into an existing destination tensor or view.legato.tensor.borrow_tuple_element(values, index)returns a tuple member without copying it. A tensor result aliases the producer's storage, so writes are visible through both values and the producer storage remains live until the borrowed value is no longer used. The index must be a compile-time, nonnegative integer within the tuple.
memcpy is the common store operation used in tests and examples.
Scalar int32 stores on EVT0¶
A core can store an int32 scalar with flags[row, 0] = step into a
shared_dram tensor. This path writes the entire containing 64-byte DRAM word;
the remaining 60 bytes are unspecified. Reserve a separate word for each flag,
and use a tensor view plus memcpy when neighboring elements must be
preserved. This operation alone does not establish host/device polling or
remote-memory visibility guarantees.
1 2 3 4 5 | |
Dynamic Shapes¶
Dynamic dimensions are represented in tensor types with (-1, max_dim).
Runtime sizes can be read with legato.tensor.get_dimension and passed to
allocation or receive operations.
1 2 3 4 5 6 | |
EVT1 Transfers¶
With an EVT1 backend, shared-DRAM requests and receives support runtime row counts and padded physical dimensions. The transfer size is rounded to whole DRAM lines, with the remaining dimensions copied at their physical sizes. This extends the request/receive path; destination SRAM physical extents still need to be static.
legato.tensor.memcpy(output, value) also supports writing core SRAM results
back to word-interleaved shared_dram on EVT1. BF16 output ranks 1 through 6
are numerically covered. Keep source and destination extents and transfer alignment
valid. The EVT1 coverage table
distinguishes these paths from the EVT0 examples.
Best Practices¶
- Keep load and receive order clear.
- Use explicit target names such as
"sram"and"mpu_buffer". - Use
receive_typefor MPU weights or dynamically shaped values. - Prefer
viewplusmemcpyfor partial tensor stores.