Skip to content

Static Evaluation

Static evaluation in Legato

Legato provides a small set of "static" control-flow and data-construction utilities that execute entirely at compile time. Instead of emitting intermediate representation (IR) loops and branches, these utilities run in the host interpreter while the compiler is generating code, allowing you to:

  • Specialize kernels without runtime overhead by deciding and unrolling work at compile time.
  • Reuse ordinary Python control abstractions (lambdas and functions) to build code that is easy to read and refactor.
  • Simplify implementation by keeping compile-time decisions outside of the IR and inside a predictable, type-checked interpreter.

Static vs dynamic control flow

  • Dynamic control flow (ordinary for, while, if within kernel bodies over dynamic values) produces IR loops/branches that execute on the device.
  • Static control flow (the legato.static.* utilities) executes on the host compiler. Results of static constructs are constants, compile-time lists, or compiler-side actions that synthesize IR without introducing dynamic loops/branches.

See Runtime Control Flow for while, nearest-loop break, the supported carried-state types, and current backend limits.

Static constructs require their inputs to be compile-time (interpreted) values. Legato diagnoses misuse with clear errors when a dynamic value is passed where a static value is required.

Evaluation strategy: compile as much as possible

Legato's compiler aggressively evaluates any expression at compile time when possible and only produces a runtime (device) computation when required. This minimizes runtime cost:

  • If an input is known at compile time, the compiler folds and unrolls with that value.
  • If an input is dynamic, the compiler emits IR so the device computes it.

In other words, the compiler "pushes left" every operation it can, keeping runtime code lean.

Constexpr vs runtime values and the compile cache

  • Constexpr values are intended to be evaluated statically. You mark parameters as constexpr to signal that they should be available to the compiler during codegen.
  • Runtime values are computed on the device. They are not available to guide compile-time decisions.

Legato caches compiled variants for functions decorated with @legato.compile, keyed by their constexpr parameters. If a constexpr parameter changes, the specialization changes and the @legato.compile cache will refresh (recompile) for the new combination.

Functions decorated with @legato.primitive do not maintain a separate launch cache. They are compiled into their caller's module with constexpr values available while generating the body. Frontend MLIR can still contain a primitive function and a func.call; neither inlining nor removal of that call is guaranteed at every output stage.

Example: the arithmetic inside this fully constexpr primitive folds to a constant, while the frontend function call can remain:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
@legato.primitive(backend=get_bertha)
def add_primitive(a: lt.constexpr(lt.integer("int64")), b: lt.constexpr(lt.integer("int64"))) -> lt.integer("int64"):
    c = a + b   # computed by the compiler
    return c

@legato.compile(backend=get_bertha)
def call_primitive(lhs: lt.constexpr(lt.integer("int64")), rhs: lt.constexpr(lt.integer("int64"))):
    add_primitive(lhs, rhs)  # The callee's addition folds; the call can remain.

# First call compiles and caches the @legato.compile wrapper for (lhs=1, rhs=2),
# and emits the primitive body with the constexpr values available
with legato.session(output_type=legato.OutputType.MLIR, launch=False):
    call_primitive(1, 2)
# Different constexpr inputs (e.g., 3, 4) trigger a new cached variant of the
# @legato.compile function with the new values available in the primitive body
with legato.session(output_type=legato.OutputType.MLIR, launch=False):
    call_primitive(3, 4)

Example: mixed constexpr/runtime partially evaluates at compile time and leaves dynamic work for the device:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
@legato.primitive(backend=get_bertha)
def add_primitive(a: lt.constexpr(lt.integer("int64")), b: lt.integer("int64")) -> lt.integer("int64"):
    # The compiler bakes the constant `a` into the generated code
    c = a + b
    return c

@legato.compile(backend=get_bertha)
def call_primitive(lhs: lt.constexpr(lt.integer("int64")), rhs: lt.integer("int64")):
    add_primitive(lhs, rhs)

# Cache specialization is keyed by `lhs` on the @legato.compile function;
# each variant compiles the primitive body with that value of `lhs`.
with legato.session(output_type=legato.OutputType.MLIR, launch=False):
    call_primitive(1, 2)
    call_primitive(4, 2)

Quick start: compile-time iteration with static.for_each

Use legato.static.for_each(iterable, func) to iterate over an interpreted iterable at compile time. The loop is fully unrolled by the compiler; func is invoked per element while code is being generated.

Example adapted from tests (iterate over core ids computed as a range):

1
2
3
4
5
6
7
8
@legato.compile(backend=get_bertha)
def test_static_for_each(to_add: lt.integer("int64"), num_cores: lt.constexpr(lt.integer("int64"))):
    device = legato.get_context().get_device()
    with device.get_all_cores():
        # range is constructed from constexpr bounds and is interpreted
        range_value = legato.range(arith.cast(0, lt.index()), arith.cast(num_cores, lt.index()))
        # Body runs at compile time; the addition is baked into generated IR per element
        legato.static.for_each(range_value, lambda x: arith.cast(x, lt.integer("int64")) + to_add)

You can also pass a named function:

1
2
3
4
5
def add_one(x):
    return arith.cast(x, lt.integer("int64")) + to_add

with device.get_all_cores():
    legato.static.for_each(range_value, add_one)

Compile-time conditionals with static.if_else

legato.static.if_else(condition, on_true, on_false) chooses one of two zero-argument branches at compile time based on an interpreted boolean condition.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
@legato.compile(backend=get_bertha)
def test_static_if_else():
    device = legato.get_context().get_device()
    with device.get_all_cores():
        condition = True  # constexpr
        x = 0
        x = x - 1

        def on_true():
            return x + 1

        def on_false():
            return x + 2

        legato.static.if_else(condition, on_true, on_false)

        x = -100
        condition = False
        legato.static.if_else(condition, on_true, on_false)

Only the selected branch contributes to generated IR; the other branch is entirely omitted.

Compile-time folds with static.fold and static.fold_back

legato.static.fold(iterable, func, initial) reduces an interpreted iterable to a single interpreted value at compile time. The result can then be embedded into generated IR as a constant.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
@legato.compile(backend=get_bertha)
def test_fold():
    device = legato.get_context().get_device()
    total = legato.static.fold(
        legato.range(arith.cast(0, lt.index()), arith.cast(10, lt.index())),
        lambda x, acc: x + acc,
        0,
    )
    with device.get_all_cores():
        x = total

legato.static.fold_back performs the same reduction from right to left (useful when function is not associative or when order matters).

Working with compile-time lists: static.empty_list and static.append

Legato supports constructing and transforming lists entirely at compile time. These lists carry element type information and are validated by the compiler.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
@legato.compile(backend=get_bertha)
def test_static_for_each_with_list(to_add: lt.integer("int32"), core_ids: lt.constexpr(lt.list(lt.integer("int64")))):
    device = legato.get_context().get_device()

    # Create an empty constexpr list of int32
    empty_list = legato.static.empty_list(lt.integer("int32"))

    # Fold a constexpr Python list into an interpreted Legato list
    casted_core_ids = legato.static.fold(
        core_ids,
        lambda core_id, acc: legato.static.append(acc, arith.cast(core_id, lt.integer("int32"))),
        empty_list,
    )

    def add_one(x: lt.integer("int32")):
        return x + to_add

    with device.get_all_cores():
        legato.static.for_each(casted_core_ids, add_one)

API synopsis

  • legato.static.for_each(iterable, func): Unrolls a loop at compile time over an interpreted iterable. func(elem) must be an interpreted callable; its body is invoked by the compiler per element to synthesize IR.
  • legato.static.fold(iterable, func, initial): Left-fold at compile time, returning an interpreted result. func(elem, acc) returns the next interpreted accumulator.
  • legato.static.fold_back(iterable, func, initial): Right-fold variant (func(acc, elem)).
  • legato.static.empty_list(dtype): Creates an empty interpreted list with element type dtype (a compile-time type value).
  • legato.static.append(list_value, value): Appends value to an interpreted list, enforcing element type equality.
  • legato.static.if_else(condition, if_true, if_false): Chooses and executes one branch at compile time. Only the chosen branch contributes IR.

Each function requires that its arguments be interpreted (compile-time) values. When a dynamic value is passed, Legato emits a diagnostic such as "expected interpreted value" or a type mismatch notice.

Why static evaluation?

  • Zero-cost specialization: Decisions like enabling an epilogue or choosing between algorithmic variants can be made at compile time without introducing dynamic branches in device code.
  • Unrolled patterns: Common patterns like per-core setup, tiling across a fixed set, or emitting repeated IR fragments are expressed succinctly with for_each and are fully unrolled.
  • Safer metaprogramming: Legato validates types and interpretation at the boundary, catching errors early (e.g., appending a wrong-typed value to a compile-time list).

This mirrors CuTe's philosophy where compile-time control (e.g., cutlass.range_constexpr, cutlass.const_expr) produces specialized kernels without runtime overhead, while dynamic control produces IR branches/loops that execute on the device. See: CuTe DSL: Control Flow.

Best practices

  • Prefer static constructs when the bounds/data are known at compile time; prefer dynamic constructs when they depend on runtime inputs.
  • Keep static lambdas pure (return the value or IR-building callable); avoid relying on runtime-only state.
  • Use legato.static.empty_list with an explicit element type to seed compile-time list computations; append with legato.static.append inside fold to transform constexpr Python sequences into typed, interpreted lists.
  • Tests that exercise these APIs are in legato/src/tests/test_static_evaluation.py.
  • Implementations live in legato/src/legato/language/operations/static.py.

References