Skip to content

Vector Addition: Running Your First Kernel

Document status

Owner: CL Team · Baseline: Legato 0.14.1+public, PyTorch 2.10.0+cpu, Bertha EVT0 · Last verified: 2026-10-19

Add two BF16 [1, 256] inputs on core 0 and store the result with the same shape. Save the complete executable in the SDK environment and run it:

1
2
export HA_TOOLCHAIN=1
python3 vector_add.py

Run in an HW runtime environment connected to a Bertha card. On success, the program prints PASS: vector_add [1, 256], BF16, core 0.

Complete code

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
"""BF16 vector addition on one Bertha EVT0 core."""

import torch
import torch_ha  # Registers the LPU device and torch.ops.lpu operations.
import legato
import legato.model.bertha as bertha


def get_bertha(ctx):
    return bertha.Bertha(ctx, "bertha", num_cores=32)


@legato.compile(backend=get_bertha)
def vector_add(
    a: legato.get_model().tensor_type(legato.types.float("bfloat16"), [1, 256], "shared_dram"),
    b: legato.get_model().tensor_type(legato.types.float("bfloat16"), [1, 256], "shared_dram"),
    out: legato.get_model().tensor_type(legato.types.float("bfloat16"), [1, 256], "shared_dram"),
):
    device = legato.get_context().get_device()
    with device.get_top():
        legato.tensor.request_load(a, device.get_core(0), "sram")
        legato.tensor.request_load(b, device.get_core(0), "sram")
    with device.get_core(0):
        tile_type = legato.types.tensor(legato.types.float("bfloat16"), [1, 256])
        x = legato.tensor.receive_type(tile_type, 0, None, "sram")
        y = legato.tensor.receive_type(tile_type, 1, None, "sram")
        legato.tensor.memcpy(out, x + y)


def main():
    shape = [1, 256]
    a_cpu = ((torch.arange(256).reshape(shape) % 17 - 8) / 8).to(torch.bfloat16)
    b_cpu = ((torch.arange(256).reshape(shape) % 13 - 6) / 8).to(torch.bfloat16)
    a, b, out = [
        torch.ops.lpu.empty_with_kind(
            shape, "dram_word_interleaved", dtype=torch.bfloat16, device="lpu:0"
        )
        for _ in range(3)
    ]
    a.copy_(a_cpu)
    b.copy_(b_cpu)
    with legato.session(output_type=legato.OutputType.BINARY, output_path="vector_add"):
        vector_add(a, b, out)
    actual = out.cpu()  # Synchronous readback waits for the result.
    torch.testing.assert_close(actual, a_cpu + b_cpu, rtol=0, atol=0)
    print("PASS: vector_add [1, 256], BF16, core 0")


if __name__ == "__main__":
    main()

Steps

  1. get_bertha: configure the target device. This example uses the 32-core Bertha EVT0 configuration.
  2. @legato.compile: connect the kernel function to the backend. Specify dtype, shape, and memory space in the argument types.
  3. Request input loads in get_top(), then receive, compute, and store the output in get_core(0).
  4. Copy CPU inputs to LPU memory and call the kernel inside legato.session(...).
  5. Retrieve the result with the synchronous out.cpu() copy and compare it with the CPU result.
Item Configuration in this example
Device memory Allocate with dram_word_interleaved, corresponding to shared_dram in the type
Argument declaration and passing Default universal when param_kinds is omitted; pass tensors directly
receive_type indices Order of SRAM inputs requested for the core: 0 for a, 1 for b

request_load transfers data from DRAM to SRAM; memcpy stores the result in the output. Keep input and output tensors alive until readback. Inputs in this example are exactly representable in binary, so the comparison uses zero tolerance.

Compile without launching

Set launch=False in the session to generate a binary only. This does not verify LPU execution or numerical correctness. If changing only this option in the complete example, the result comparison still runs. Omit the readback and comparison when checking compilation alone.

1
2
with legato.session(output_type=legato.OutputType.BINARY, output_path="vector_add", launch=False):
    vector_add(a, b, out)

Troubleshooting

Symptom Check first
Call outside a session Move the kernel call inside with legato.session(...):
Argument placement error Correspondence between param_kinds and the top or core locations in Arg
Compilation or memory layout error Dtype, shape, and allocation kind match between type declarations and tensors
Incorrect result Input receive order, output storage, and CPU reference

See Compiler Session for output paths and intermediate IR.