2. Execution Model¶
To use a device through the driver API, first bootstrap the device and open a handle, then load a kernel and launch it
through its handle. Unlike the runtime API, the driver API has no separate synchronous execution functions. Kernel
launches and memory copies all return as soon as they are enqueued on a stream. Check completion with ha_drv_wait,
described in Chapter 4 Streams and Events.
This chapter covers the lifecycles of device and kernel handles, and when control returns from an asynchronous dispatch.
Contents¶
| Section | Objective |
|---|---|
| 2.1 Device Management | Bootstrap, handle open/close, attribute queries, and shutdown order |
| 2.2 Kernel Loading and Execution | Loading, asynchronous launch, and release of kernel handles (HaDrvKernel) |
| 2.3 Asynchronous Dispatch | When control returns, how to check completion, and host buffer validity |
2.1 Device Management¶
Device management in the driver API is split into a per-process function pair (ha_drv_init / ha_drv_deinit) and a
per-device-handle function pair (ha_drv_open / ha_drv_close).
Device Lifecycle¶
- Initialize (
ha_drv_init): Scans and bootstraps all devices installed in the system. If no device is available, it returnsHaDeviceNotAvailable; you can call it again once a device is ready. - Open a handle (
ha_drv_open): Callha_drv_open(index, &dev)to take a reference to the target device and receive aHaDrvDevhandle. This does not create a new device object; the handle refers to the already bootstrapped device. - Use resources: All subsequent APIs, including memory, stream, and kernel control functions, take this handle as their first argument.
- Close the handle (
ha_drv_close): Callha_drv_close(dev)to release the reference. The device itself stays bootstrapped. - Shut down (
ha_drv_deinit): Releases all devices and their streams, events, kernels, and memory. If any open handle remains, it releases nothing and returnsHaDeviceBusy.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 | |
| API | Sync / Async | Behavior |
|---|---|---|
ha_drv_init() |
Sync | Scans and bootstraps all devices. Does nothing if the devices are already bootstrapped |
ha_drv_get_device_count() |
Sync | Queries the number of bootstrapped devices |
ha_drv_open() |
Sync | Obtains a HaDrvDev handle to the target device |
ha_drv_get_attr() |
Sync | Reads the value of a single device attribute |
ha_drv_close() |
Sync | Releases the device handle reference |
ha_drv_deinit() |
Sync | Releases all device resources (HaDeviceBusy if open handles remain) |
Note: Re-bootstrapping
ha_drv_init() returns success without doing anything if the devices are already bootstrapped. To bootstrap the
devices again, close all open handles and call ha_drv_deinit() first.
Device Reset¶
ha_drv_reset(dev) forcibly reinitializes the device and invalidates all of its streams, events, memory allocations,
and kernel handles at once. The top-level device handle (HaDrvDev) remains valid. For details, see 5.3 Device Reset.
2.2 Kernel Loading and Execution¶
The driver API does not launch a kernel directly from a binary file path. Instead, you load the kernel onto the device
and launch it by passing the kernel handle (HaDrvKernel) returned by the load. A kernel is loaded from a file or from a
precompiled kernel library.
Kernel Load¶
ha_drv_kernel_load_from_file(dev, path, &k) validates the kernel file and reserves device memory. This operation does
not wait for device work to complete. Loading the same file again issues a new handle, but the device memory is
shared.
Kernel Library¶
A kernel library is a single bundle of data (HaLibraryData) that contains kernels built for each device version. A
library is handled independently of any device; when it is loaded onto a device, the build matching that device's chip
is selected.
- Register the library with
ha_drv_library_load_data(&lib, data). It does not need a device, so it can be called beforeha_drv_init(). The data is not copied and must remain valid until the process exits. - Get the kernel handle (
HaKernel_t) for a name withha_drv_library_get_kernel(&kernel, lib, name). This handle refers to a kernel that is not loaded on any device. The name can be read back withha_drv_library_kernel_name(). - Load the build matching the device chip with
ha_drv_kernel_load_from_library(dev, kernel, &k)and receive anHaDrvKernel. If the library has no build for that chip, it returnsHaKernelNoBinaryForDevice. Launch and release then work the same as for a kernel loaded from a file.
Library handles (HaLibrary_t) and library kernel handles (HaKernel_t) remain valid until the process exits and are
not affected by device reset. Only the HaDrvKernel loaded onto the device is invalidated by a reset, so after a reset
you only need to call ha_drv_kernel_load_from_library() again.
Kernel Execution¶
ha_drv_kernel_launch(dev, stream, k, args) returns immediately after enqueuing the kernel launch on the stream. Kernel
arguments are passed as an argument list (a HaLaunchArgs object). Constructing the object creates the list, and the list is released when the object goes out of scope. Arguments are added with ha_drv_launch_args_add_*(), which check
each argument against the core configuration of the specified device (dev). To change only the values for repeated
launches, use ha_drv_launch_args_set_*()
(see Execution). If the list does not match
the kernel's argument format, the launch is rejected with HaKernelInvalidArgument.
The argument list is consumed before the function returns, so it is safe to release the list or update it for the next launch immediately after the call. On the first launch of a kernel, a binary upload may be enqueued on that stream first.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 | |
| API | Sync / Async | Behavior |
|---|---|---|
ha_drv_library_load_data() |
Sync | Registers a kernel library and issues a library handle (HaLibrary_t). No device required |
ha_drv_library_get_kernel() |
Sync | Looks up a library kernel handle (HaKernel_t) by name |
ha_drv_kernel_load_from_file() |
Sync | Validates the kernel file, reserves device memory, and returns a kernel handle (HaDrvKernel) |
ha_drv_kernel_load_from_library() |
Sync | Validates the library kernel build for the device chip, reserves device memory, and returns a kernel handle (HaDrvKernel) |
ha_drv_kernel_launch() |
Async | Enqueues a kernel launch on the stream (on the first launch, a binary upload may be enqueued first) |
ha_drv_kernel_release() |
Sync | Invalidates the kernel handle (launches already enqueued still run) |
Note: Kernel handle safety
Launching a kernel with a handle that was already released, or that was returned before a device reset, returns
HaKernelInvalidHandle; a different kernel is never launched by mistake. Device memory is not returned when
ha_drv_kernel_release is called; it is released all at once at device reset or shutdown.
2.3 Asynchronous Dispatch¶
The driver API's kernel launch (ha_drv_kernel_launch) and memory copy (ha_drv_memcpy, ha_drv_memcpy_nd) functions
are all asynchronous. On every stream, including HA_STREAM_DEFAULT, they return as soon as the operation is added to
the queue. Returning only means that the operation was enqueued; it does not indicate that the device work has
completed.
There are three cases in which some host-side processing happens before the call returns. In none of them does the return mean that the work enqueued by this call has completed.
- First launch of a kernel: If a binary upload of the same kernel is in progress on another stream,
ha_drv_kernel_launchmay wait on the host for that upload to finish before returning. - N-D copy through staging: For a host-to-device copy in which
ha_drv_memcpy_nduses a host staging buffer, the source is copied into the staging buffer before the call returns. In this case you can release the source buffer immediately after the call returns. - Staging buffer limit exceeded: If the process-wide total of staging buffers not yet synchronized exceeds
HA_MEMCPYND_ASYNC_STAGING_MAX_UNSYNCED_BYTESwhenha_drv_memcpy_nduses staging, it first synchronizes the stream that receives the copy, before enqueuing. In this case the call does not return until the work previously enqueued on that stream has finished.
If you need blocking behavior equivalent to the runtime API's synchronous functions (HaLaunchKernel, HaMemCpy), call
ha_drv_wait(HA_WAIT_STREAM_SYNC) on the same stream immediately after enqueuing the operation.
Caution: Host buffer lifetime
Do not release, reallocate, or modify a host buffer passed to an asynchronous copy until you have synchronized on
completion of that stream (ha_drv_wait). When the call returns, the host-device transfer may still be in progress.
Next Steps¶
- Memory Model — Allocation, release, and data transfer
- Driver API: Device Handle · Driver API: Execution — Per-function signatures