Channel Operations¶
A channel moves values from one device unit to another. It is a depth-1
rendezvous FIFO: a write blocks until the previous message on its lane has
been consumed, a read blocks until one is available, and messages arrive in
write order. The channel is parametrized by its endpoints alone — it carries no
payload type, so each write sends the value it is handed and successive
messages on one channel may differ in dtype and shape.
Each tensor message declares a typed LayoutMapAttr for the serving buffer
(after any staging copy) and receiving buffer on each lane. These contracts
currently require row-major layouts. The producer retains its own tensor
layout; supported sources that need staging are copied into the serving
buffer. Allocation padding is not transmitted as payload.
Fig. 1 — one channel, source to destination. legato.Channel(src, dst) pairs a source and destination unit into a channel before any device context opens. Inside the source's context, write enqueues a value; inside the destination's context, read dequeues the oldest unread value. The queue can be written and read repeatedly, in any order relative to other code, as long as each call runs on the correct side.
Fig. 2 — one write, many channels; one read per channel. Create a separate legato.Channel(src, dst) for each core-to-top pair. A single write call inside a multi-core context fans a value out to each core's channel in one statement, but the top side reads each channel with its own read call, since device.get_top() is always exactly one unit.
Creating Channels¶
Channel¶
1 | |
Create a depth-1 rendezvous FIFO between two device executors.
Must be called outside of any device context, so the channel can be shared into both the
source-side and the destination-side with blocks.
Parameters:
src: The source executor, or the channel's full lanes as anEndpointPairor a sequence of them whendstis omitted.dst: The destination executor. Omit it whensrcalready names the lanes. Default:None.
Returns:
- (
Callable[[CodeGenerator], Value | None]): One channel covering every requested lane.
Example:
1 2 3 4 5 6 7 8 9 | |
Write¶
write¶
1 | |
Send one message through the lanes of channels the active context owns.
Must be called from a device context whose units are source endpoints of channels. On a
cross-device channel the value's own element type and logical shape become the wire payload,
so successive messages need not agree; on a core-to-top channel the value must be an integer
scalar.
Parameters:
channels(Value): A channel created bylegato.Channel(), or a tuple of them.data(Value): The value to send.
Read¶
read¶
1 | |
Receive the next message on the lanes of channels the active context owns.
Must be called from a device context whose units are destination endpoints of channels.
The k-th read on a lane pairs with its k-th write.
Parameters:
channels(Value): A channel created bylegato.Channel(), or a tuple of them.payload_type: The expected payload, needed on a cross-device channel only when the matchingwritehas not been traced yet. Default:None.
Returns:
- (
Callable[[CodeGenerator], Value | None]): The received value, or a tuple of them whenchannelsis a tuple.
Example¶
1 2 3 4 5 6 7 8 9 10 11 | |
legato.Channel is called outside of any device context so the channel can be
shared into both the source-side and destination-side with blocks. The k-th
read on a lane pairs with its k-th write, so one channel carries a sequence
of messages rather than a single handshake.
Two Kinds of Lane¶
The current backend supports these endpoint and payload combinations:
| Endpoints | Payload | Runtime loops |
|---|---|---|
| A core and its own device's top context | Integer scalars | Allowed |
| Equal core IDs on different ranks | Static tensors | Not allowed |
Two sibling cores on one device cannot be paired by this API.
For an EVT0 tensor transfer, the receiver exports its SRAM destination address.
The runtime supplies that address to the sender through an implicit core
argument slot. Compiled artifacts record the peer-address bindings.
AddressSlotBinding.rank / core identify the sender; peer_rank /
peer_core identify the receiver. Descriptor ABI 2 uses this direction;
recompile cached binaries and manifests produced with descriptor ABI 1.
Cross-device lanes are statically scheduled¶
A cross-device message reserves its source SRAM at compile time, so in this release the software compiler requires:
- each lane's
writeandreadcounts must be compile-time constants and equal, and every lane of one channel must carry the same number of messages; - neither call may appear inside a runtime loop or branch;
- a
readtraced before its matchingwritehas no value to take a type from and must state the payload itself.
These are compiler constraints for the current lowering path. They describe what this software release can compile; they do not establish that the underlying hardware cannot support runtime-dependent loops or branches.
The current storage progress check follows tracked SRAM roots in each straight-line core function. It rejects writes to a source after its first send and checks that a static top-to-core supply matches receive bytes. It does not prove global progress or remote completion; source SRAM remains reserved for the transfer by the allocator.
1 2 3 4 5 6 7 8 9 10 11 | |
When the receiver is traced first, give read the payload it expects:
1 2 3 4 | |
Multiple Lanes¶
legato.Channel(src, dst) constructs one lane from one source endpoint to one
destination endpoint. To construct a multi-lane channel, pass a sequence of
EndpointPair values; one channel can then span every core of a rank pair.
mesh.pair_cores_by_id builds that pointwise set, and write and read select
the lanes the active context owns:
1 2 3 4 | |
For core-to-top lanes, pass the pairs explicitly:
1 2 3 4 5 6 7 | |
Multi-core tensor transfers¶
On Bertha EVT0, a tensor channel can compile with get_all_cores() or a
get_cores(...) group. Each selected core sends its own value to the
same-numbered core on the peer rank. This is pointwise communication across
ranks; it does not combine values across different core IDs.
Compatible cores can share one transfer program while retaining separate
per-core export and import records. Sparse groups such as [1, 3] are also
supported, and sender and receiver blocks may partition the selected cores
differently. If earlier transfers make their address slots differ, the compiler
separates the required transfer instructions and preserves each core's binding.
Slot numbers are local to a core: four sending cores can each use slot zero.
The following complete script compiles a four-core transfer between two ranks, checks both ELF binaries, and verifies every sender's peer core and byte count. It does not launch or claim hardware numerical verification. Download channel_multi_core.py; see Running the examples for SDK setup.
1 | |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 | |
Placement Constraints¶
writemust run in a device context whose units are source endpoints of the channel;readmust run in one whose units are destination endpoints.- A channel belongs to the compilation that created it and cannot be reused in another.