Skip to content

Python Execution API

Function Signature Notes
launch_kernel_with_binary_path (binary_path: str, args: HaLaunchArgs) -> None Blocks until device execution completes
launch_kernel_async_with_binary_path (binary_path: str, args: HaLaunchArgs, stream: int = HA_STREAM_DEFAULT) -> None Enqueues on the stream and returns immediately
library_load_data (data: int) -> int Registers a precompiled kernel library and returns a library handle. Does not use the device
library_get_kernel (library: int, name: str) -> int Returns the handle of the kernel named name in the library
launch_kernel_with_handle (kernel: int, args: HaLaunchArgs) -> None Launches a library kernel and blocks until device execution completes
launch_kernel_async_with_handle (kernel: int, args: HaLaunchArgs, stream: int = HA_STREAM_DEFAULT) -> None Enqueues a library kernel launch on the stream and returns immediately

The args parameter must be a single HaLaunchArgs object. Passing a collection such as a Python list directly raises TypeError.

The launch functions read all data in the argument list before returning control, so it is safe to release the argument object or change its values right after the call returns. The same holds for asynchronous launches.


Kernel Libraries

Register a library that contains precompiled kernels, then launch with a kernel handle obtained by name. Library handles and kernel handles are opaque int values that stay valid until the process exits. For the corresponding C++ API, see Execution API.

  • The data argument of library_load_data() is the address of a HaLibraryData provided by the extension module that embeds it. The data is not copied, so it must stay valid until the process exits. Each call registers a new library.
  • library_get_kernel() always returns the same handle for the same name, regardless of which device later runs it.
  • Library kernels are Legato kernels. The kernel is loaded onto a device on its first launch on that device.
  • launch_kernel_async_with_handle() reads all arguments during the call and returns once the launch is enqueued. Device execution failures surface in later calls.
Situation Result
0 passed to library_load_data() RuntimeError (HaInvalidArgument)
Invalid library data RuntimeError (HaKernelInvalidArgument)
Library in a format this hart cannot read RuntimeError (HaKernelUnsupported)
0 passed to library_get_kernel() RuntimeError (HaInvalidArgument)
Library handle not returned by library_load_data() RuntimeError (HaKernelInvalidHandle)
No kernel with that name in the library RuntimeError (HaKernelNotFound)
Launch with a kernel handle not returned by library_get_kernel() RuntimeError (HaKernelInvalidHandle)
Library has no kernel binary for the current device RuntimeError (HaKernelNoBinaryForDevice)
Argument the kernel does not accept RuntimeError (HaKernelInvalidArgument)

HaLaunchArgs

An object that holds the arguments for one kernel launch, in the order they were added. The internal layout in which arguments are written to match the kernel ABI is not public; callers handle arguments only through the methods below.

1
from hart import HaDataType, HaLaunchArgRoute, HaLaunchArgs
Method Signature Purpose
HaLaunchArgs() () -> HaLaunchArgs Creates an empty argument list object
add_tensor (route: HaLaunchArgRoute, target_core: int, da: int, dims: list[int], offset: int = 0) -> None Adds a Tensor argument
add_scalar (route: HaLaunchArgRoute, target_core: int, dtype: HaDataType, bits: int) -> None Adds a Scalar argument
add_array (route: HaLaunchArgRoute, target_core: int, dims: list[int], raw_data: bytes \| bytearray \| memoryview, strides: list[int] \| None = None) -> None Adds an Array argument (the data is copied internally)
set_tensor (index: int, da: int, dims: list[int]) -> None Replaces the address and dimensions of the index-th Tensor
set_scalar_bits (index: int, bits: int) -> None Replaces the value of the index-th Scalar
set_array (index: int, dims: list[int], raw_data: bytes \| bytearray \| memoryview) -> None Replaces the dimensions and data of the index-th Array
  • Thread safety: One HaLaunchArgs object can be shared safely across threads. A lock inside the object serializes argument additions and replacements with kernel launches so that they do not overlap. However, when a value replacement and a kernel launch overlap, it is unspecified whether the launch uses the value from before or after the replacement.
  • Device validation: When an argument is added, it is validated against the current device set with set_device(), for example the core number given for the CORE route. Calling add_* with no current device set raises RuntimeError (HaDeviceNotInitialized). Value replacement (set_*) only changes values of arguments already held, so it does not need a device.
  • Device addresses: Pass a device memory address (da) directly as an int. Do not wrap it in another type such as PyCapsule or a ctypes integer.
  • Value replacement constraints: The index of set_* is the order in which the argument was added with add_*, counting from 0. The route, target_core, and dtype set at the first addition are kept, and the rank of a Tensor or Array cannot change.
  • C++ API mapping: Each Python method maps 1:1 to the C++ HaLaunchArgs class and the HaLaunchArgsAdd*() / HaLaunchArgsSet*() functions. For detailed parameter requirements, see C++ Execution API.

route and target_core

Every argument takes a route and a target_core that determine where the argument is written.

  • route: Selects the ABI block to write the argument to. UNIVERSAL is the top block and every core block, TOP is the top block only, and CORE is the block of the single core given by target_core.
  • target_core: Meaningful only when the route is CORE; pass the target core number. Any other route ignores it; pass -1 by convention.

The route must match the one fixed when the kernel was compiled.

The bits argument of add_scalar is the bit pattern of the value. dtype is one of INT8, INT16, INT32, INT64, FLOAT16, BFLOAT16, FLOAT32, FLOAT64, FLOAT8, and only the low-order bytes occupied by that data type are read. For example, a negative INT32 is interpreted as the two's complement in the low 4 bytes, and FLOAT32 as the IEEE 754 bit pattern in the low 4 bytes.

Exceptions and Error Handling

Situation Result
No current device set RuntimeError (HaDeviceNotInitialized) at add_*
Device reset in progress RuntimeError (HaDeviceNotReady) at add_*
Violation of argument rules, such as an unsupported dtype, an invalid route, or a core number not on the current device RuntimeError (HaKernelInvalidArgument) at addition or replacement; the list is unchanged
Argument kinds, count, or routes differ from the kernel declaration, or point to a core not on the launch device The kernel launch is rejected with RuntimeError (HaKernelInvalidArgument), and the expected and passed layouts are logged

Example

launch_kernel_with_binary_path

Launches a kernel synchronously from a binary file path. The first launch on each device also loads the kernel.

Kernel metadata is cached per device, so in a benchmark the first launch includes load time, not just compute. For accurate measurements, always warm up before the measured runs.

Example
1
2
3
4
5
6
7
8
9
import hart
from hart import HaLaunchArgRoute, HaLaunchArgs

hart.set_device(0)  # arguments are validated against the current device
args = HaLaunchArgs()
args.add_tensor(HaLaunchArgRoute.UNIVERSAL, -1, input_ptr, [m, k])
args.add_tensor(HaLaunchArgRoute.UNIVERSAL, -1, weight_ptr, [k, n])
args.add_tensor(HaLaunchArgRoute.UNIVERSAL, -1, output_ptr, [m, n])
hart.launch_kernel_with_binary_path("/path/to/kernel.bin", args)

launch_kernel_async_with_binary_path

Enqueue memory transfers and the kernel launch on the same stream; they run in order while the host continues without waiting.

Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
import hart
from hart import HaDataType, HaLaunchArgRoute, HaLaunchArgs

args = HaLaunchArgs()
args.add_tensor(HaLaunchArgRoute.UNIVERSAL, -1, dev_in, [10, 512])
args.add_scalar(HaLaunchArgRoute.TOP, -1, HaDataType.INT64, bits=7)

hart.mem_cpy_async(dev_in, host_in, n, hart.HaMemCpyDirection.HostToDevice, stream)
hart.launch_kernel_async_with_binary_path(kernel_path, args, stream)
hart.mem_cpy_async(host_out, dev_out, n, hart.HaMemCpyDirection.DeviceToHost, stream)

hart.stream_synchronize(stream)  # transfers and the kernel launch have all completed at this point

launch_kernel_with_handle

Registers a library once, gets a kernel handle, and launches it. library_data_address is the HaLibraryData address provided by the extension module that contains the library.

Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
import hart
from hart import HaLaunchArgRoute, HaLaunchArgs

library = hart.library_load_data(library_data_address)  # once per process
matmul = hart.library_get_kernel(library, "matmul")

hart.set_device(0)
args = HaLaunchArgs()
args.add_tensor(HaLaunchArgRoute.UNIVERSAL, -1, input_ptr, [m, k])
args.add_tensor(HaLaunchArgRoute.UNIVERSAL, -1, output_ptr, [m, n])
hart.launch_kernel_with_handle(matmul, args)  # the kernel is loaded on the first launch on device 0

launch_kernel_async_with_handle

Enqueues a launch on a stream with a kernel handle.

Example
1
2
hart.launch_kernel_async_with_handle(matmul, args, stream)
hart.stream_synchronize(stream)

Repeated Launches Reusing the Argument Object

When launching a kernel repeatedly with the same argument layout, reuse the existing argument list object and replace only the values, rather than creating a new object each time.

Example
1
2
3
4
5
6
7
8
args = HaLaunchArgs()
args.add_tensor(HaLaunchArgRoute.UNIVERSAL, -1, buf_a, [10, 512])   # index 0 (Tensor)
args.add_scalar(HaLaunchArgRoute.TOP, -1, HaDataType.INT64, bits=7)  # index 1 (Scalar)

for step in range(steps):
    args.set_tensor(0, da=bufs[step], dims=[10, 512])
    args.set_scalar_bits(1, step)
    hart.launch_kernel_async_with_binary_path(kernel_path, args, stream)

See also