Python Execution API¶
| Function | Signature | Notes |
|---|---|---|
launch_kernel_with_binary_path |
(binary_path: str, args: HaLaunchArgs) -> None |
Blocks until device execution completes |
launch_kernel_async_with_binary_path |
(binary_path: str, args: HaLaunchArgs, stream: int = HA_STREAM_DEFAULT) -> None |
Enqueues on the stream and returns immediately |
library_load_data |
(data: int) -> int |
Registers a precompiled kernel library and returns a library handle. Does not use the device |
library_get_kernel |
(library: int, name: str) -> int |
Returns the handle of the kernel named name in the library |
launch_kernel_with_handle |
(kernel: int, args: HaLaunchArgs) -> None |
Launches a library kernel and blocks until device execution completes |
launch_kernel_async_with_handle |
(kernel: int, args: HaLaunchArgs, stream: int = HA_STREAM_DEFAULT) -> None |
Enqueues a library kernel launch on the stream and returns immediately |
The args parameter must be a single HaLaunchArgs object. Passing a collection such as a Python list
directly raises TypeError.
The launch functions read all data in the argument list before returning control, so it is safe to release the argument object or change its values right after the call returns. The same holds for asynchronous launches.
Kernel Libraries¶
Register a library that contains precompiled kernels, then launch with a kernel handle obtained by name. Library handles and
kernel handles are opaque int values that stay valid until the process exits. For the corresponding C++ API, see
Execution API.
- The
dataargument oflibrary_load_data()is the address of aHaLibraryDataprovided by the extension module that embeds it. The data is not copied, so it must stay valid until the process exits. Each call registers a new library. library_get_kernel()always returns the same handle for the same name, regardless of which device later runs it.- Library kernels are Legato kernels. The kernel is loaded onto a device on its first launch on that device.
launch_kernel_async_with_handle()reads all arguments during the call and returns once the launch is enqueued. Device execution failures surface in later calls.
| Situation | Result |
|---|---|
0 passed to library_load_data() |
RuntimeError (HaInvalidArgument) |
| Invalid library data | RuntimeError (HaKernelInvalidArgument) |
Library in a format this hart cannot read |
RuntimeError (HaKernelUnsupported) |
0 passed to library_get_kernel() |
RuntimeError (HaInvalidArgument) |
Library handle not returned by library_load_data() |
RuntimeError (HaKernelInvalidHandle) |
| No kernel with that name in the library | RuntimeError (HaKernelNotFound) |
Launch with a kernel handle not returned by library_get_kernel() |
RuntimeError (HaKernelInvalidHandle) |
| Library has no kernel binary for the current device | RuntimeError (HaKernelNoBinaryForDevice) |
| Argument the kernel does not accept | RuntimeError (HaKernelInvalidArgument) |
HaLaunchArgs¶
An object that holds the arguments for one kernel launch, in the order they were added. The internal layout in which arguments are written to match the kernel ABI is not public; callers handle arguments only through the methods below.
1 | |
| Method | Signature | Purpose |
|---|---|---|
HaLaunchArgs() |
() -> HaLaunchArgs |
Creates an empty argument list object |
add_tensor |
(route: HaLaunchArgRoute, target_core: int, da: int, dims: list[int], offset: int = 0) -> None |
Adds a Tensor argument |
add_scalar |
(route: HaLaunchArgRoute, target_core: int, dtype: HaDataType, bits: int) -> None |
Adds a Scalar argument |
add_array |
(route: HaLaunchArgRoute, target_core: int, dims: list[int], raw_data: bytes \| bytearray \| memoryview, strides: list[int] \| None = None) -> None |
Adds an Array argument (the data is copied internally) |
set_tensor |
(index: int, da: int, dims: list[int]) -> None |
Replaces the address and dimensions of the index-th Tensor |
set_scalar_bits |
(index: int, bits: int) -> None |
Replaces the value of the index-th Scalar |
set_array |
(index: int, dims: list[int], raw_data: bytes \| bytearray \| memoryview) -> None |
Replaces the dimensions and data of the index-th Array |
- Thread safety: One
HaLaunchArgsobject can be shared safely across threads. A lock inside the object serializes argument additions and replacements with kernel launches so that they do not overlap. However, when a value replacement and a kernel launch overlap, it is unspecified whether the launch uses the value from before or after the replacement. - Device validation: When an argument is added, it is validated against the current device set with
set_device(), for example the core number given for theCOREroute. Callingadd_*with no current device set raisesRuntimeError(HaDeviceNotInitialized). Value replacement (set_*) only changes values of arguments already held, so it does not need a device. - Device addresses: Pass a device memory address (
da) directly as anint. Do not wrap it in another type such asPyCapsuleor a ctypes integer. - Value replacement constraints: The
indexofset_*is the order in which the argument was added withadd_*, counting from 0. The route,target_core, anddtypeset at the first addition are kept, and the rank of a Tensor or Array cannot change. - C++ API mapping: Each Python method maps 1:1 to the C++
HaLaunchArgsclass and theHaLaunchArgsAdd*()/HaLaunchArgsSet*()functions. For detailed parameter requirements, see C++ Execution API.
route and target_core¶
Every argument takes a route and a target_core that determine where the argument is written.
- route: Selects the ABI block to write the argument to.
UNIVERSALis the top block and every core block,TOPis the top block only, andCOREis the block of the single core given bytarget_core. - target_core: Meaningful only when the route is
CORE; pass the target core number. Any other route ignores it; pass-1by convention.
The route must match the one fixed when the kernel was compiled.
The bits argument of add_scalar is the bit pattern of the value. dtype is one of INT8, INT16, INT32, INT64,
FLOAT16, BFLOAT16, FLOAT32, FLOAT64, FLOAT8, and only the low-order bytes occupied by that data type are read. For
example, a negative INT32 is interpreted as the two's complement in the low 4 bytes, and FLOAT32 as the IEEE 754 bit
pattern in the low 4 bytes.
Exceptions and Error Handling¶
| Situation | Result |
|---|---|
| No current device set | RuntimeError (HaDeviceNotInitialized) at add_* |
| Device reset in progress | RuntimeError (HaDeviceNotReady) at add_* |
Violation of argument rules, such as an unsupported dtype, an invalid route, or a core number not on the current device |
RuntimeError (HaKernelInvalidArgument) at addition or replacement; the list is unchanged |
| Argument kinds, count, or routes differ from the kernel declaration, or point to a core not on the launch device | The kernel launch is rejected with RuntimeError (HaKernelInvalidArgument), and the expected and passed layouts are logged |
Example¶
launch_kernel_with_binary_path¶
Launches a kernel synchronously from a binary file path. The first launch on each device also loads the kernel.
Kernel metadata is cached per device, so in a benchmark the first launch includes load time, not just compute. For accurate measurements, always warm up before the measured runs.
Example
1 2 3 4 5 6 7 8 9 | |
launch_kernel_async_with_binary_path¶
Enqueue memory transfers and the kernel launch on the same stream; they run in order while the host continues without waiting.
Example
1 2 3 4 5 6 7 8 9 10 11 12 | |
launch_kernel_with_handle¶
Registers a library once, gets a kernel handle, and launches it. library_data_address is the HaLibraryData address
provided by the extension module that contains the library.
Example
1 2 3 4 5 6 7 8 9 10 11 | |
launch_kernel_async_with_handle¶
Enqueues a launch on a stream with a kernel handle.
Example
1 2 | |
Repeated Launches Reusing the Argument Object¶
When launching a kernel repeatedly with the same argument layout, reuse the existing argument list object and replace only the values, rather than creating a new object each time.
Example
1 2 3 4 5 6 7 8 | |