Skip to content

Matrix Multiplication

Document status

Owner: CL Team · Baseline: Legato 0.14.1+public, PyTorch 2.10.0+cpu, Bertha EVT0 · Last verified: 2026-10-19

Compute BF16 [16, 256] @ [256, 128] → [16, 128] on core 0. Save the complete executable in the SDK environment and run it:

1
HA_TOOLCHAIN=1 python3 matmul.py

Input and weight layout

Argument Kernel memory space Device allocation kind Core receive location
Input a shared_dram dram_word_interleaved sram
weight mpu_dram mpu_tensor mpu_buffer
Output out shared_dram dram_word_interleaved Output DRAM

Allocate the weight with mpu_tensor, then copy the CPU tensor using copy_. The input and weight use different memory layouts.

1
2
3
4
weight = torch.ops.lpu.empty_with_kind(
    [256, 128], "mpu_tensor", dtype=torch.bfloat16, device="lpu:0"
)
weight.copy_(w_cpu)

Kernel body

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
device = legato.get_context().get_device()
with device.get_top():
    legato.tensor.request_load(a, device.get_core(0), "sram")
    legato.tensor.request_load(weight, device.get_core(0), "mpu_buffer")
with device.get_core(0):
    x = legato.tensor.receive(a, 0, "sram")
    weight_type = legato.get_model().tensor_type(
        legato.types.float("bfloat16"), [256, 128], "mpu_weight"
    )
    w = legato.tensor.receive_type(weight_type, 0, None, "mpu_buffer")
    legato.tensor.memcpy(out, x @ w)

Receive the weight sent to mpu_buffer as mpu_weight, then multiply using @. Receive indices are separate for each destination memory, so both the SRAM input and MPU weight use index 0.

Execution and comparison

Call matmul(a, weight, out) inside legato.session(...). Arguments use the default universal declaration and tensors are passed directly. Retrieve with out.cpu() and compare all output elements with an FP32 CPU matmul converted to BF16. The example uses rtol=0.02, atol=0.002. Set appropriate validation criteria separately for changed input ranges or shapes. On success, the program prints PASS: matmul [16, 256] @ [256, 128], BF16, core 0.