Skip to content

Compiler Session

legato.session(...) sets compilation options for the next calls to @legato.compile kernels. A session is required before JIT-compiling a kernel: calling a compiled kernel outside a session raises a runtime error.

1
2
with legato.session(output_type=legato.OutputType.BINARY, output_path="simple_matmul"):
    simple_matmul(input_tensor, weight_tensor, output_tensor)

What A Session Controls

A session stores these options in the current Python context:

  • output_type: which compiler artifact to produce.
  • output_path: where to write the artifact and the compiler log.
  • launch: whether calling a compiled kernel also launches it on the device (defaults to True). See Running.

When a compiled kernel is called, Legato reads those options before building the IR. The options determine which compiler passes run and what file is written.

Output Types

output_type must be either None or one of the legato.OutputType enum values. Strings such as "binary" are rejected.

Output type Extension Compiler behavior
legato.OutputType.MLIR .mlir Emits frontend MLIR before core/backend passes.
legato.OutputType.CORE_IR .mlir Runs core passes and emits Core IR.
legato.OutputType.BACKEND_IR .mlir Runs core and backend passes, then emits backend IR.
legato.OutputType.ASM .asm Runs passes and translates to assembly.
legato.OutputType.BINARY .bin Runs passes and translates to binary.
None .mlir or .bin Uses Legato defaults depending on output_path; see below.

Output Paths

When output_path is provided, Legato writes artifacts under a directory named after the compiled function.

1
2
with legato.session(output_type=legato.OutputType.CORE_IR, output_path="multiply_add"):
    multiply_add(a, b, c)

For a function named multiply_add, output files are written under:

1
multiply_add/multiply_add/

The final artifact name includes the kernel specialization hash, and a matching .log file is written next to it. The hash changes when constexpr arguments change, so different constexpr specializations get separate artifacts.

When both output_type and output_path are None, Legato uses its cache and log directories. In that mode, existing cached artifacts can skip code generation for the same specialization.

Default Behavior

legato.session(output_type=None, output_path="some-dir") requests a binary artifact. This is useful for normal compile-and-run style examples.

legato.session(output_type=None, output_path=None) uses the internal cache path and defaults the pass target to binary.

Running

By default (launch=True), calling a compiled kernel inside a session both compiles it and launches it on the device:

1
2
with legato.session(output_type=legato.OutputType.BINARY, output_path="simple_matmul", launch=True):
    simple_matmul(a, legato.Arg(b, bertha.top), legato.Arg(out, bertha.core(0)))

Pass launch=False to produce the requested artifact without invoking the launcher. This is useful for lowering-only flows and for environments without the optional hart runtime. Only OutputType.BINARY is launchable; the text-IR output types (MLIR, CORE_IR, BACKEND_IR, ASM) have nothing to launch, so launch has no effect for them.

Example: Binary Output

Compile simple_matmul to a binary artifact. a, b, and out are device-resident tensors; see Simple Matmul for the full kernel and the device-tensor setup.

1
2
3
4
5
6
with legato.session(output_type=legato.OutputType.BINARY, output_path="simple_matmul"):
    simple_matmul(
        a,                          # universal -> bare
        legato.Arg(b, bertha.top),
        legato.Arg(out, bertha.core(0)),
    )

Example: Inspect Core IR (Unavailable With Secure Binaries)

Use CORE_IR when you want to inspect the result after core compiler passes.

1
2
3
4
5
6
7
# Compile-only: CPU tensors describe the typed operands; no device launch occurs.
with legato.session(output_type=legato.OutputType.CORE_IR, output_path="simple_matmul", launch=False):
    simple_matmul(
        torch.randn(10, 256, dtype=torch.bfloat16),
        legato.Arg(torch.randn(256, 128, dtype=torch.bfloat16), bertha.top),
        legato.Arg(torch.empty(10, 128, dtype=torch.bfloat16), bertha.core(0)),
    )

Common Mistakes

  • Treating successful OutputType.MLIR emission as proof of backend support. Later outputs run executor, memory, and backend checks that frontend IR alone does not exercise. See the compile-only diagnostic example.
  • Calling a compiled kernel without with legato.session(...):.
  • Passing a string for output_type instead of legato.OutputType.
  • Expecting one artifact per function only; constexpr specializations produce distinct hashed artifact names.