Skip to content

Channel Operations

A channel moves values from one device unit to another. It is a depth-1 rendezvous FIFO: a write blocks until the previous message on its lane has been consumed, a read blocks until one is available, and messages arrive in write order. The channel is parametrized by its endpoints alone — it carries no payload type, so each write sends the value it is handed and successive messages on one channel may differ in dtype and shape.

Each tensor message declares a typed LayoutMapAttr for the serving buffer (after any staging copy) and receiving buffer on each lane. These contracts currently require row-major layouts. The producer retains its own tensor layout; supported sources that need staging are copied into the serving buffer. Allocation padding is not transmitted as payload.

Inside the source context, core 0 holds a value that a write op moves into the channel. Inside the destination context, a read op pulls the value out of the channel into the top unit's result.

Fig. 1 — one channel, source to destination. legato.Channel(src, dst) pairs a source and destination unit into a channel before any device context opens. Inside the source's context, write enqueues a value; inside the destination's context, read dequeues the oldest unread value. The queue can be written and read repeatedly, in any order relative to other code, as long as each call runs on the correct side.

In the source context, core 0 and core 1 each hold a value that feeds one shared write op, since a single write call spans both active units; that op fans out into channel ch0 and channel ch1. In the destination context, each channel feeds its own separate read op, since read is called once per channel, and each op deposits into the top unit's result0 or result1.

Fig. 2 — one write, many channels; one read per channel. Create a separate legato.Channel(src, dst) for each core-to-top pair. A single write call inside a multi-core context fans a value out to each core's channel in one statement, but the top side reads each channel with its own read call, since device.get_top() is always exactly one unit.

Creating Channels

Channel

1
Channel(src, dst = None) -> Callable[[CodeGenerator], Value | None]

Create a depth-1 rendezvous FIFO between two device executors.

Must be called outside of any device context, so the channel can be shared into both the source-side and the destination-side with blocks.

Parameters:

  • src: The source executor, or the channel's full lanes as an EndpointPair or a sequence of them when dst is omitted.
  • dst: The destination executor. Omit it when src already names the lanes. Default: None.

Returns:

  • (Callable[[CodeGenerator], Value | None]): One channel covering every requested lane.

Example:

1
2
3
4
5
6
7
8
9
@legato.compile(backend=get_backend)
def fn(value: legato.types.integer("int64")):
    device = legato.get_context().get_device()
    channel = legato.Channel(device.get_core(0), device.get_top())
    with device.get_core(0):
        legato.write(channel, value)
    with device.get_top():
        result = legato.read(channel)
        yield result

Write

write

1
write(channels: Value, data: Value) -> Callable[[CodeGenerator], Value | None]

Send one message through the lanes of channels the active context owns.

Must be called from a device context whose units are source endpoints of channels. On a cross-device channel the value's own element type and logical shape become the wire payload, so successive messages need not agree; on a core-to-top channel the value must be an integer scalar.

Parameters:

  • channels (Value): A channel created by legato.Channel(), or a tuple of them.
  • data (Value): The value to send.

Read

read

1
read(channels: Value, payload_type = None) -> Callable[[CodeGenerator], Value | None]

Receive the next message on the lanes of channels the active context owns.

Must be called from a device context whose units are destination endpoints of channels. The k-th read on a lane pairs with its k-th write.

Parameters:

  • channels (Value): A channel created by legato.Channel(), or a tuple of them.
  • payload_type: The expected payload, needed on a cross-device channel only when the matching write has not been traced yet. Default: None.

Returns:

  • (Callable[[CodeGenerator], Value | None]): The received value, or a tuple of them when channels is a tuple.

Example

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
@legato.compile(backend=get_backend)
def fn(value: legato.types.integer("int64")):
    device = legato.get_context().get_device()
    channel = legato.Channel(device.get_core(0), device.get_top())

    with device.get_core(0):
        legato.write(channel, value)

    with device.get_top():
        result = legato.read(channel)
        yield result

legato.Channel is called outside of any device context so the channel can be shared into both the source-side and destination-side with blocks. The k-th read on a lane pairs with its k-th write, so one channel carries a sequence of messages rather than a single handshake.

Two Kinds of Lane

The current backend supports these endpoint and payload combinations:

Endpoints Payload Runtime loops
A core and its own device's top context Integer scalars Allowed
Equal core IDs on different ranks Static tensors Not allowed

Two sibling cores on one device cannot be paired by this API.

For an EVT0 tensor transfer, the receiver exports its SRAM destination address. The runtime supplies that address to the sender through an implicit core argument slot. Compiled artifacts record the peer-address bindings. AddressSlotBinding.rank / core identify the sender; peer_rank / peer_core identify the receiver. Descriptor ABI 2 uses this direction; recompile cached binaries and manifests produced with descriptor ABI 1.

Cross-device lanes are statically scheduled

A cross-device message reserves its source SRAM at compile time, so in this release the software compiler requires:

  • each lane's write and read counts must be compile-time constants and equal, and every lane of one channel must carry the same number of messages;
  • neither call may appear inside a runtime loop or branch;
  • a read traced before its matching write has no value to take a type from and must state the payload itself.

These are compiler constraints for the current lowering path. They describe what this software release can compile; they do not establish that the underlying hardware cannot support runtime-dependent loops or branches.

The current storage progress check follows tracked SRAM roots in each straight-line core function. It rejects writes to a source after its first send and checks that a static top-to-core supply matches receive bytes. It does not prove global progress or remote completion; source SRAM remains reserved for the transfer by the allocator.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
mesh = legato.get_context().get_mesh()
rank0, rank1 = mesh.get_device(0), mesh.get_device(1)
channel = legato.Channel(mesh.pair_cores_by_id(rank0, rank1))

with rank0.get_core(0):
    legato.write(channel, first)
    legato.write(channel, second)

with rank1.get_core(0):
    head = legato.read(channel)
    tail = legato.read(channel)

When the receiver is traced first, give read the payload it expects:

1
2
3
4
with rank1.get_core(0):
    received = legato.read(channel, legato.types.tensor(legato.types.float("bfloat16"), (1, 64)))
with rank0.get_core(0):
    legato.write(channel, value)

Multiple Lanes

legato.Channel(src, dst) constructs one lane from one source endpoint to one destination endpoint. To construct a multi-lane channel, pass a sequence of EndpointPair values; one channel can then span every core of a rank pair. mesh.pair_cores_by_id builds that pointwise set, and write and read select the lanes the active context owns:

1
2
3
4
channel = legato.Channel(mesh.pair_cores_by_id(rank0, rank1))

with rank0.get_cores([0, 1]):
    legato.write(channel, value)      # writes core 0's and core 1's lanes

For core-to-top lanes, pass the pairs explicitly:

1
2
3
4
5
6
7
channel = legato.Channel([
    legato.EndpointPair(device.get_core(0), top),
    legato.EndpointPair(device.get_core(1), top),
])

with device.get_cores([0, 1]):
    legato.write(channel, value)

Multi-core tensor transfers

On Bertha EVT0, a tensor channel can compile with get_all_cores() or a get_cores(...) group. Each selected core sends its own value to the same-numbered core on the peer rank. This is pointwise communication across ranks; it does not combine values across different core IDs.

Compatible cores can share one transfer program while retaining separate per-core export and import records. Sparse groups such as [1, 3] are also supported, and sender and receiver blocks may partition the selected cores differently. If earlier transfers make their address slots differ, the compiler separates the required transfer instructions and preserves each core's binding. Slot numbers are local to a core: four sending cores can each use slot zero.

The following complete script compiles a four-core transfer between two ranks, checks both ELF binaries, and verifies every sender's peer core and byte count. It does not launch or claim hardware numerical verification. Download channel_multi_core.py; see Running the examples for SDK setup.

1
HA_TOOLCHAIN=1 python "$DOCS_ROOT/docs/legato-language/tutorials/code/channel_multi_core.py"
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
"""Compile a four-core EVT0 tensor channel between two ranks without launching."""

import os
from pathlib import Path

import torch

import legato
from legato.model.bertha import BerthaDeviceMesh

OUTPUT = Path(os.environ.get("LEGATO_EXAMPLE_OUTPUT", "legato_example_output"))


@legato.compile(backend=lambda ctx: BerthaDeviceMesh(ctx, "channel_mesh", 2, num_cores=4))
def multi_core_transfer(value: legato.types.tensor(legato.types.float("bfloat16"), (1, 64))):
    """Send each core's tensor to the same-numbered core on the peer rank."""
    mesh = legato.get_context().get_mesh()
    source, peer = mesh.get_device(0), mesh.get_device(1)
    # Pair core N on the source rank with core N on the peer rank.
    channel = legato.Channel(mesh.pair_cores_by_id(source, peer))

    with source.get_all_cores():
        legato.write(channel, value)
    with peer.get_all_cores():
        received = legato.read(channel)
        yield received


def main():
    """Compile both ranks and verify their per-core address-slot bindings."""
    # CPU tensors describe the compiled payload; no device or simulator is initialized.
    value = torch.zeros((1, 64), dtype=torch.bfloat16)
    with legato.session(output_type=legato.OutputType.BINARY, output_path=str(OUTPUT), launch=False):
        # Only rank zero consumes the input argument. The receiver gets its value through the channel.
        multi_core_transfer(legato.RankArgs({0: value}))

    # Compilation produces one ELF binary for each logical rank.
    (artifacts,) = multi_core_transfer.mesh_artifacts.values()
    assert {artifact.rank for artifact in artifacts} == {0, 1}
    for artifact in artifacts:
        assert Path(artifact.path).read_bytes().startswith(b"\x7fELF")

    # Each sender core's slot zero points to the same-numbered receiver core's SRAM.
    bindings = tuple(binding for artifact in artifacts for binding in artifact.address_slot_bindings)
    assert {(b.rank, b.core, b.slot, b.peer_rank, b.peer_core, b.size_bytes) for b in bindings} == {
        (0, core, 0, 1, core, 128) for core in range(4)
    }
    assert {artifact.rank: artifact.address_slot_count for artifact in artifacts} == {0: 1, 1: 0}


if __name__ == "__main__":
    main()

Placement Constraints

  • write must run in a device context whose units are source endpoints of the channel; read must run in one whose units are destination endpoints.
  • A channel belongs to the compilation that created it and cannot be reused in another.