Skip to content

2. Execution Model

To use a device through the driver API, first bootstrap the device and open a handle, then load a kernel and launch it through its handle. Unlike the runtime API, the driver API has no separate synchronous execution functions. Kernel launches and memory copies all return as soon as they are enqueued on a stream. Check completion with ha_drv_wait, described in Chapter 4 Streams and Events.

This chapter covers the lifecycles of device and kernel handles, and when control returns from an asynchronous dispatch.


Contents

Section Objective
2.1 Device Management Bootstrap, handle open/close, attribute queries, and shutdown order
2.2 Kernel Loading and Execution Loading, asynchronous launch, and release of kernel handles (HaDrvKernel)
2.3 Asynchronous Dispatch When control returns, how to check completion, and host buffer validity

2.1 Device Management

Device management in the driver API is split into a per-process function pair (ha_drv_init / ha_drv_deinit) and a per-device-handle function pair (ha_drv_open / ha_drv_close).

Device Lifecycle

  1. Initialize (ha_drv_init): Scans and bootstraps all devices installed in the system. If no device is available, it returns HaDeviceNotAvailable; you can call it again once a device is ready.
  2. Open a handle (ha_drv_open): Call ha_drv_open(index, &dev) to take a reference to the target device and receive a HaDrvDev handle. This does not create a new device object; the handle refers to the already bootstrapped device.
  3. Use resources: All subsequent APIs, including memory, stream, and kernel control functions, take this handle as their first argument.
  4. Close the handle (ha_drv_close): Call ha_drv_close(dev) to release the reference. The device itself stays bootstrapped.
  5. Shut down (ha_drv_deinit): Releases all devices and their streams, events, kernels, and memory. If any open handle remains, it releases nothing and returns HaDeviceBusy.
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
ha_drv_init();                                   // Process: bootstrap all devices

int count = 0;
ha_drv_get_device_count(&count);

HaDrvDev dev = nullptr;
ha_drv_open(0, &dev);                            // Handle: take a reference to device 0

int64_t channels = 0;
ha_drv_get_attr(dev, HA_DEV_ATTR_DRAM_CHANNELS, &channels);

// ... memory, stream, and kernel work ...

ha_drv_close(dev);                               // Release the handle (device 0 stays bootstrapped)
ha_drv_deinit();                                 // Process: release all device resources
API Sync / Async Behavior
ha_drv_init() Sync Scans and bootstraps all devices. Does nothing if the devices are already bootstrapped
ha_drv_get_device_count() Sync Queries the number of bootstrapped devices
ha_drv_open() Sync Obtains a HaDrvDev handle to the target device
ha_drv_get_attr() Sync Reads the value of a single device attribute
ha_drv_close() Sync Releases the device handle reference
ha_drv_deinit() Sync Releases all device resources (HaDeviceBusy if open handles remain)

Note: Re-bootstrapping

ha_drv_init() returns success without doing anything if the devices are already bootstrapped. To bootstrap the devices again, close all open handles and call ha_drv_deinit() first.

Device Reset

ha_drv_reset(dev) forcibly reinitializes the device and invalidates all of its streams, events, memory allocations, and kernel handles at once. The top-level device handle (HaDrvDev) remains valid. For details, see 5.3 Device Reset.

2.2 Kernel Loading and Execution

The driver API does not launch a kernel directly from a binary file path. Instead, you load the kernel onto the device and launch it by passing the kernel handle (HaDrvKernel) returned by the load. A kernel is loaded from a file or from a precompiled kernel library.

Kernel Load

ha_drv_kernel_load_from_file(dev, path, &k) validates the kernel file and reserves device memory. This operation does not wait for device work to complete. Loading the same file again issues a new handle, but the device memory is shared.

Kernel Library

A kernel library is a single bundle of data (HaLibraryData) that contains kernels built for each device version. A library is handled independently of any device; when it is loaded onto a device, the build matching that device's chip is selected.

  1. Register the library with ha_drv_library_load_data(&lib, data). It does not need a device, so it can be called before ha_drv_init(). The data is not copied and must remain valid until the process exits.
  2. Get the kernel handle (HaKernel_t) for a name with ha_drv_library_get_kernel(&kernel, lib, name). This handle refers to a kernel that is not loaded on any device. The name can be read back with ha_drv_library_kernel_name().
  3. Load the build matching the device chip with ha_drv_kernel_load_from_library(dev, kernel, &k) and receive an HaDrvKernel. If the library has no build for that chip, it returns HaKernelNoBinaryForDevice. Launch and release then work the same as for a kernel loaded from a file.

Library handles (HaLibrary_t) and library kernel handles (HaKernel_t) remain valid until the process exits and are not affected by device reset. Only the HaDrvKernel loaded onto the device is invalidated by a reset, so after a reset you only need to call ha_drv_kernel_load_from_library() again.

Kernel Execution

ha_drv_kernel_launch(dev, stream, k, args) returns immediately after enqueuing the kernel launch on the stream. Kernel arguments are passed as an argument list (a HaLaunchArgs object). Constructing the object creates the list, and the list is released when the object goes out of scope. Arguments are added with ha_drv_launch_args_add_*(), which check each argument against the core configuration of the specified device (dev). To change only the values for repeated launches, use ha_drv_launch_args_set_*() (see Execution). If the list does not match the kernel's argument format, the launch is rejected with HaKernelInvalidArgument.

The argument list is consumed before the function returns, so it is safe to release the list or update it for the next launch immediately after the call. On the first launch of a kernel, a binary upload may be enqueued on that stream first.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
HaDrvKernel k = nullptr;
if (ha_drv_kernel_load_from_file(dev, "kernels/add.elf", &k) == HaError::HaKernelNotFound) {
  return;   // Check the kernel file path
}

uint32_t dims[] = {1024};
HaLaunchArgs args;                                       // Released when it goes out of scope
ha_drv_launch_args_add_tensor(dev, args, HaLaunchArgRoute::Universal, -1, a, dims, 1, 0);
ha_drv_launch_args_add_tensor(dev, args, HaLaunchArgRoute::Universal, -1, b, dims, 1, 0);
ha_drv_launch_args_add_tensor(dev, args, HaLaunchArgRoute::Universal, -1, out, dims, 1, 0);

ha_drv_kernel_launch(dev, HA_STREAM_DEFAULT, k, args);   // Returns immediately after enqueuing on the stream
ha_drv_kernel_launch(dev, HA_STREAM_DEFAULT, k, args);   // Can relaunch with the same handle and list

ha_drv_wait(dev, HA_STREAM_DEFAULT, nullptr, HA_WAIT_STREAM_SYNC, -1, nullptr);
ha_drv_kernel_release(dev, k);
API Sync / Async Behavior
ha_drv_library_load_data() Sync Registers a kernel library and issues a library handle (HaLibrary_t). No device required
ha_drv_library_get_kernel() Sync Looks up a library kernel handle (HaKernel_t) by name
ha_drv_kernel_load_from_file() Sync Validates the kernel file, reserves device memory, and returns a kernel handle (HaDrvKernel)
ha_drv_kernel_load_from_library() Sync Validates the library kernel build for the device chip, reserves device memory, and returns a kernel handle (HaDrvKernel)
ha_drv_kernel_launch() Async Enqueues a kernel launch on the stream (on the first launch, a binary upload may be enqueued first)
ha_drv_kernel_release() Sync Invalidates the kernel handle (launches already enqueued still run)

Note: Kernel handle safety

Launching a kernel with a handle that was already released, or that was returned before a device reset, returns HaKernelInvalidHandle; a different kernel is never launched by mistake. Device memory is not returned when ha_drv_kernel_release is called; it is released all at once at device reset or shutdown.

2.3 Asynchronous Dispatch

The driver API's kernel launch (ha_drv_kernel_launch) and memory copy (ha_drv_memcpy, ha_drv_memcpy_nd) functions are all asynchronous. On every stream, including HA_STREAM_DEFAULT, they return as soon as the operation is added to the queue. Returning only means that the operation was enqueued; it does not indicate that the device work has completed.

There are three cases in which some host-side processing happens before the call returns. In none of them does the return mean that the work enqueued by this call has completed.

  • First launch of a kernel: If a binary upload of the same kernel is in progress on another stream, ha_drv_kernel_launch may wait on the host for that upload to finish before returning.
  • N-D copy through staging: For a host-to-device copy in which ha_drv_memcpy_nd uses a host staging buffer, the source is copied into the staging buffer before the call returns. In this case you can release the source buffer immediately after the call returns.
  • Staging buffer limit exceeded: If the process-wide total of staging buffers not yet synchronized exceeds HA_MEMCPYND_ASYNC_STAGING_MAX_UNSYNCED_BYTES when ha_drv_memcpy_nd uses staging, it first synchronizes the stream that receives the copy, before enqueuing. In this case the call does not return until the work previously enqueued on that stream has finished.

If you need blocking behavior equivalent to the runtime API's synchronous functions (HaLaunchKernel, HaMemCpy), call ha_drv_wait(HA_WAIT_STREAM_SYNC) on the same stream immediately after enqueuing the operation.

Caution: Host buffer lifetime

Do not release, reallocate, or modify a host buffer passed to an asynchronous copy until you have synchronized on completion of that stream (ha_drv_wait). When the call returns, the host-device transfer may still be in progress.


Next Steps