Static Evaluation¶
Static evaluation in Legato¶
Legato provides a small set of "static" control-flow and data-construction utilities that execute entirely at compile time. Instead of emitting intermediate representation (IR) loops and branches, these utilities run in the host interpreter while the compiler is generating code, allowing you to:
- Specialize kernels without runtime overhead by deciding and unrolling work at compile time.
- Reuse ordinary Python control abstractions (lambdas and functions) to build code that is easy to read and refactor.
- Simplify implementation by keeping compile-time decisions outside of the IR and inside a predictable, type-checked interpreter.
Static vs dynamic control flow¶
- Dynamic control flow (ordinary
for,while,ifwithin kernel bodies over dynamic values) produces IR loops/branches that execute on the device. - Static control flow (the
legato.static.*utilities) executes on the host compiler. Results of static constructs are constants, compile-time lists, or compiler-side actions that synthesize IR without introducing dynamic loops/branches.
See Runtime Control Flow for while, nearest-loop break,
the supported carried-state types, and current backend limits.
Static constructs require their inputs to be compile-time (interpreted) values. Legato diagnoses misuse with clear errors when a dynamic value is passed where a static value is required.
Evaluation strategy: compile as much as possible¶
Legato's compiler aggressively evaluates any expression at compile time when possible and only produces a runtime (device) computation when required. This minimizes runtime cost:
- If an input is known at compile time, the compiler folds and unrolls with that value.
- If an input is dynamic, the compiler emits IR so the device computes it.
In other words, the compiler "pushes left" every operation it can, keeping runtime code lean.
Constexpr vs runtime values and the compile cache¶
- Constexpr values are intended to be evaluated statically. You mark parameters as constexpr to signal that they should be available to the compiler during codegen.
- Runtime values are computed on the device. They are not available to guide compile-time decisions.
Legato caches compiled variants for functions decorated with @legato.compile, keyed by their constexpr parameters. If a constexpr parameter changes, the specialization changes and the @legato.compile cache will refresh (recompile) for the new combination.
Functions decorated with @legato.primitive do not maintain a separate launch
cache. They are compiled into their caller's module with constexpr values
available while generating the body. Frontend MLIR can still contain a
primitive function and a func.call; neither inlining nor removal of that call
is guaranteed at every output stage.
Example: the arithmetic inside this fully constexpr primitive folds to a constant, while the frontend function call can remain:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 | |
Example: mixed constexpr/runtime partially evaluates at compile time and leaves dynamic work for the device:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 | |
Quick start: compile-time iteration with static.for_each¶
Use legato.static.for_each(iterable, func) to iterate over an interpreted iterable at compile time. The loop is fully unrolled by the compiler; func is invoked per element while code is being generated.
Example adapted from tests (iterate over core ids computed as a range):
1 2 3 4 5 6 7 8 | |
You can also pass a named function:
1 2 3 4 5 | |
Compile-time conditionals with static.if_else¶
legato.static.if_else(condition, on_true, on_false) chooses one of two zero-argument branches at compile time based on an interpreted boolean condition.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 | |
Only the selected branch contributes to generated IR; the other branch is entirely omitted.
Compile-time folds with static.fold and static.fold_back¶
legato.static.fold(iterable, func, initial) reduces an interpreted iterable to a single interpreted value at compile time. The result can then be embedded into generated IR as a constant.
1 2 3 4 5 6 7 8 9 10 | |
legato.static.fold_back performs the same reduction from right to left (useful when function is not associative or when order matters).
Working with compile-time lists: static.empty_list and static.append¶
Legato supports constructing and transforming lists entirely at compile time. These lists carry element type information and are validated by the compiler.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 | |
API synopsis¶
legato.static.for_each(iterable, func): Unrolls a loop at compile time over an interpretediterable.func(elem)must be an interpreted callable; its body is invoked by the compiler per element to synthesize IR.legato.static.fold(iterable, func, initial): Left-fold at compile time, returning an interpreted result.func(elem, acc)returns the next interpreted accumulator.legato.static.fold_back(iterable, func, initial): Right-fold variant (func(acc, elem)).legato.static.empty_list(dtype): Creates an empty interpreted list with element typedtype(a compile-time type value).legato.static.append(list_value, value): Appendsvalueto an interpreted list, enforcing element type equality.legato.static.if_else(condition, if_true, if_false): Chooses and executes one branch at compile time. Only the chosen branch contributes IR.
Each function requires that its arguments be interpreted (compile-time) values. When a dynamic value is passed, Legato emits a diagnostic such as "expected interpreted value" or a type mismatch notice.
Why static evaluation?¶
- Zero-cost specialization: Decisions like enabling an epilogue or choosing between algorithmic variants can be made at compile time without introducing dynamic branches in device code.
- Unrolled patterns: Common patterns like per-core setup, tiling across a fixed set, or emitting repeated IR fragments are expressed succinctly with
for_eachand are fully unrolled. - Safer metaprogramming: Legato validates types and interpretation at the boundary, catching errors early (e.g., appending a wrong-typed value to a compile-time list).
This mirrors CuTe's philosophy where compile-time control (e.g., cutlass.range_constexpr, cutlass.const_expr) produces specialized kernels without runtime overhead, while dynamic control produces IR branches/loops that execute on the device. See: CuTe DSL: Control Flow.
Best practices¶
- Prefer static constructs when the bounds/data are known at compile time; prefer dynamic constructs when they depend on runtime inputs.
- Keep static lambdas pure (return the value or IR-building callable); avoid relying on runtime-only state.
- Use
legato.static.empty_listwith an explicit element type to seed compile-time list computations; append withlegato.static.appendinsidefoldto transform constexpr Python sequences into typed, interpreted lists.
Related examples in the repository¶
- Tests that exercise these APIs are in
legato/src/tests/test_static_evaluation.py. - Implementations live in
legato/src/legato/language/operations/static.py.
References¶
- NVIDIA CuTe DSL – Control Flow: docs.nvidia.com/cutlass/.../dsl_control_flow.html