Driver API: Execution¶
Kernel libraries, and kernel load, launch, and release by handle. See Driver API for the concepts.
Contents¶
| Name | Kind | Description |
|---|---|---|
ha_drv_library_load_data |
API | Register a precompiled kernel library (no device needed) |
ha_drv_library_get_kernel |
API | Look up a kernel handle in a library by name |
ha_drv_library_kernel_name |
API | Get the name of a library kernel |
ha_drv_kernel_load_from_file |
API | Load a kernel binary file |
ha_drv_kernel_load_from_library |
API | Load a library kernel as the build that matches the device chip |
ha_drv_kernel_launch |
API | Launch a kernel (asynchronous) |
ha_drv_launch_args_add_tensor |
API | Add a Tensor argument, checked against dev |
ha_drv_launch_args_add_scalar |
API | Add a Scalar argument, checked against dev |
ha_drv_launch_args_add_array |
API | Add an Array argument, checked against dev |
ha_drv_launch_args_set_tensor |
API | Replace the address and size of an added Tensor argument (no device needed) |
ha_drv_launch_args_set_scalar_bits |
API | Replace the value of an added Scalar argument |
ha_drv_launch_args_set_array |
API | Replace the size and data of an added Array argument |
ha_drv_kernel_release |
API | Release a kernel handle |
ha_drv_library_load_data¶
Register a kernel library (CUDA cuLibraryLoadData). Needs no device: callable before ha_drv_init.
Validates data (a HaLibraryData, ha_types.h) and issues a handle for it; nothing
is copied or loaded onto a device. data is only read during the call; the names and ELF bytes it points to are not
copied and must stay valid for the life of the process. Binary sets for a device version this runtime does not know
are ignored, unchecked. data is not a typed pointer, so only its magic and version are checked: it must point to a
complete HaLibraryData.
| Parameter | Direction | Default | Description |
|---|---|---|---|
library |
out | — | Receives the library handle on success; untouched on failure. |
data |
in | — | The library data (a HaLibraryData). |
Returns — HaSuccess; HaInvalidArgument if library or data is null; HaKernelUnsupported for a
HaLibraryData version this runtime does not read; HaKernelInvalidArgument for a malformed library or one with no
binary set for a known device version (nothing is registered).
Signature · Example
1 | |
Example
1 2 3 | |
ha_drv_library_get_kernel¶
The kernel named name in library (CUDA cuLibraryGetKernel). The handle is not bound to any device; load it
onto a device with ha_drv_kernel_load_from_library.
| Parameter | Direction | Default | Description |
|---|---|---|---|
library_kernel |
out | — | Receives the kernel handle on success; untouched on failure. |
library |
in | — | Handle from ha_drv_library_load_data. |
name |
in | — | Kernel name. |
Returns — HaSuccess; HaInvalidArgument if library_kernel, library or name is null;
HaKernelInvalidHandle if library was not issued by ha_drv_library_load_data; HaKernelNotFound if no set of
the library has name.
Signature · Example
1 | |
Example
1 2 3 4 | |
ha_drv_library_kernel_name¶
The name of a library kernel.
| Parameter | Direction | Default | Description |
|---|---|---|---|
library_kernel |
in | — | Any value. |
name |
out | — | Receives the name on success (valid for the life of the process); untouched on failure. |
Returns — HaSuccess; HaInvalidArgument if name is null; HaKernelInvalidHandle if library_kernel was
not issued by ha_drv_library_get_kernel.
Signature · Example
1 | |
Example
1 2 | |
ha_drv_kernel_load_from_file¶
Load the kernel binary at path onto dev (CUDA cuModuleLoad).
Reads and checks the file and reserves its device memory; does not wait on the device. Instead of being uploaded during the load, the binary may be uploaded on the stream of the kernel's first ha_drv_kernel_launch. Loading a
kernel already loaded on dev (same file: canonical path, inode, size and mtime) reuses the existing load: it returns a new handle but allocates no new device memory. ha_drv_kernel_release does not free that memory; only a device reset or device teardown does.
| Parameter | Direction | Default | Description |
|---|---|---|---|
dev |
in | — | Device handle from ha_drv_open. |
path |
in | — | Kernel binary (Legato ELF). |
out |
out | — | Receives the kernel handle on success. |
Returns — HaSuccess; HaInvalidArgument if dev/path/out is null; HaDeviceUnsupported if dev
cannot execute kernels; HaDeviceNotReady if a reset is in progress; HaKernelNotFound if path does not exist
or has no ABI generator; HaKernelInvalidArgument for a malformed or oversized file; HaKernelUnsupported for a
Legato ELF of another target or kernel ABI major; HaMemoryAllocationFailed when the device's kernel memory is full;
HaKernelLoadFailed when the device fails the load otherwise.
Signature · Example
1 | |
Example
1 2 3 4 5 | |
ha_drv_kernel_load_from_library¶
Load a library kernel onto dev, the build for the device's chip.
Reads and checks the ELF of library_kernel in the set for the device version of dev, and reserves its device
memory; does not wait on the device. If the binary is not uploaded during the load, it is uploaded on the stream of the kernel's first ha_drv_kernel_launch. The handle is launched with
ha_drv_kernel_launch, like one from ha_drv_kernel_load_from_file; a device reset invalidates it. Loading the same kernel onto the same device again returns a new handle to the copy made by the first load, until a device reset releases it.
| Parameter | Direction | Default | Description |
|---|---|---|---|
dev |
in | — | Device handle from ha_drv_open. |
library_kernel |
in | — | The kernel in its library, from ha_drv_library_get_kernel; it is not bound to any device. |
loaded_kernel |
out | — | Receives the kernel as loaded on dev on success; untouched on failure. |
Returns — HaSuccess; HaInvalidArgument if dev or loaded_kernel is null; HaKernelInvalidHandle if
library_kernel was not issued by ha_drv_library_get_kernel; HaKernelNoBinaryForDevice if the library has no
binary of the kernel for the chip of dev; HaDeviceUnsupported if dev cannot execute kernels; HaDeviceNotReady
if a reset is in progress; HaKernelInvalidArgument for a malformed ELF; HaKernelUnsupported for an ELF of another
target or kernel ABI major; HaMemoryAllocationFailed when the device's kernel memory is full; HaKernelLoadFailed
when the device fails the load otherwise. On any error nothing is loaded.
Signature · Example
1 | |
Example
1 2 3 4 | |
ha_drv_kernel_launch¶
Launch kernel on stream with the arguments of a HaLaunchArgs list (CUDA cuLaunchKernel). Asynchronous.
Build the list as a HaLaunchArgs object and fill it with
ha_drv_launch_args_add_*(), which check each argument against dev (see
Execution for the list itself). The list is consumed before this returns, so it may be
destroyed or updated right after.
When the binary is uploaded on the launch stream, the kernel's first launch posts the binary copy on stream ahead of
the launch; a copy failure is stream's sticky error. A first launch on another stream while that copy is in flight
waits on the host for it, so this call may block (and is woken by a device reset like ha_drv_wait).
Errors are reported device first, then kernel: a device or stream error takes precedence over a kernel handle error.
| Parameter | Direction | Default | Description |
|---|---|---|---|
dev |
in | — | Device handle from ha_drv_open. |
stream |
in | — | Stream to launch on, or HA_STREAM_DEFAULT. |
kernel |
in | — | Handle from ha_drv_kernel_load_from_file or ha_drv_kernel_load_from_library on dev. |
args |
in | — | Argument list. |
Returns — HaSuccess if queued; HaInvalidArgument if dev is null or kernel was loaded on another device;
HaKernelInvalidArgument if args was moved from or does not fit the kernel's ABI; HaDeviceUnsupported if dev cannot execute kernels; HaDeviceNotReady if a reset is in progress or completed
during the call; HaStreamInvalidHandle if stream is not owned by dev; HaKernelInvalidHandle if kernel was
released or invalidated by a reset; the binary copy's post or wait error; or the stream's sticky error. No seq is
consumed on failure.
Signature · Example
1 | |
Example
1 2 3 4 5 6 7 8 9 10 11 12 13 | |
ha_drv_launch_args_add_tensor¶
Append a tensor argument to args, checked against dev. The same as the runtime's HaLaunchArgsAddTensor(), for the device the caller names instead of the thread's current device: a Core route whose target_core is not one of dev's cores is rejected. The list is not bound to dev; the launch checks it again against the device it launches on.
| Parameter | Direction | Default | Description |
|---|---|---|---|
dev |
in | — | Device handle from ha_drv_open. |
args |
in, out | — | The list. See HaLaunchArgs. |
route |
in | — | Block the argument is written to. One of Universal, Top, Core (see HaLaunchArgRoute). |
target_core |
in | — | Core whose block receives a HaLaunchArgRoute::Core argument. Ignored otherwise. |
da |
in | — | Device address of the tensor's base. |
dims |
in | — | rank extents, in the order the kernel declares them. Copied. |
rank |
in | — | Number of entries in dims. |
offset |
in | — | Element offset from da. |
Returns — HaSuccess; HaInvalidArgument if dev is null; HaDeviceNotReady if a reset is in progress; or HaKernelInvalidArgument, appending nothing, under the conditions of the runtime's HaLaunchArgsAddTensor() that apply.
Signature
1 2 | |
ha_drv_launch_args_add_scalar¶
Append a scalar argument to args, checked against dev; see ha_drv_launch_args_add_tensor.
| Parameter | Direction | Default | Description |
|---|---|---|---|
dev |
in | — | Device handle from ha_drv_open. |
args |
in, out | — | The list. |
route |
in | — | Block the argument is written to. One of Universal, Top, Core (see HaLaunchArgRoute). |
target_core |
in | — | Core whose block receives a HaLaunchArgRoute::Core argument. Ignored otherwise. |
dtype |
in | — | Element type of the scalar, as for HaLaunchArgsAddScalar(). |
bits |
in | — | The value, bit for bit. |
Returns — HaSuccess; HaInvalidArgument if dev is null; HaDeviceNotReady if a reset is in progress; or HaKernelInvalidArgument, appending nothing, under the conditions of the runtime's HaLaunchArgsAddScalar().
Signature
1 2 | |
ha_drv_launch_args_add_array¶
Append an array argument to args, checked against dev; see ha_drv_launch_args_add_tensor.
| Parameter | Direction | Default | Description |
|---|---|---|---|
dev |
in | — | Device handle from ha_drv_open. |
args |
in, out | — | The list. |
route |
in | — | Block the argument is written to. One of Universal, Top, Core (see HaLaunchArgRoute). |
target_core |
in | — | Core whose block receives a HaLaunchArgRoute::Core argument. Ignored otherwise. |
dims |
in | — | rank extents. Copied. |
rank |
in | — | Number of entries in dims. |
strides |
in | — | rank strides, or nullptr for unit strides. Copied. |
raw_data |
in | — | The element bytes. Copied. |
raw_data_size |
in | — | Number of bytes in raw_data. |
Returns — HaSuccess; HaInvalidArgument if dev is null; HaDeviceNotReady if a reset is in progress; or HaKernelInvalidArgument, appending nothing, under the conditions of the runtime's HaLaunchArgsAddArray().
Signature
1 2 3 | |
ha_drv_launch_args_set_tensor¶
Replace the address and extents of the tensor at index; see the runtime's HaLaunchArgsSetTensor(). Unlike the add functions this needs no device: it changes records the list already holds, so it takes no device handle, is not held off by a reset, and works on a thread that has no current device.
| Parameter | Direction | Default | Description |
|---|---|---|---|
args |
in, out | — | The list. |
index |
in | — | Position the tensor was appended at, counting from 0. |
da |
in | — | New device address of the tensor's base. |
dims |
in | — | rank extents. Copied. |
rank |
in | — | Number of entries in dims; must equal the rank it was appended with. |
Returns — HaSuccess, or HaKernelInvalidArgument, changing nothing, under the conditions of the runtime's HaLaunchArgsSetTensor().
Signature
1 2 | |
ha_drv_launch_args_set_scalar_bits¶
Replace the value of the scalar at index; see the runtime's HaLaunchArgsSetScalarBits() and ha_drv_launch_args_set_tensor; like it, this needs no device.
| Parameter | Direction | Default | Description |
|---|---|---|---|
args |
in, out | — | The list. |
index |
in | — | Position the scalar was appended at, counting from 0. |
bits |
in | — | New value, bit for bit. |
Returns — HaSuccess, or HaKernelInvalidArgument, changing nothing, under the conditions of the runtime's HaLaunchArgsSetScalarBits().
Signature
1 | |
ha_drv_launch_args_set_array¶
Replace the extents and element bytes of the array at index; see the runtime's HaLaunchArgsSetArray() and ha_drv_launch_args_set_tensor; like it, this needs no device.
| Parameter | Direction | Default | Description |
|---|---|---|---|
args |
in, out | — | The list. |
index |
in | — | Position the array was appended at, counting from 0. |
dims |
in | — | rank extents. Copied. |
rank |
in | — | Number of entries in dims; must equal the rank it was appended with. |
raw_data |
in | — | The element bytes. Copied. |
raw_data_size |
in | — | Number of bytes in raw_data. |
Returns — HaSuccess, or HaKernelInvalidArgument, changing nothing, under the conditions of the runtime's HaLaunchArgsSetArray().
Signature
1 2 | |
ha_drv_kernel_release¶
Release kernel (CUDA cuModuleUnload). The handle is invalid on return; launches already queued run. The device
memory stays loaded (shared with other handles of the same kernel, and reused by a later load of
it) until a device reset or the device's teardown.
| Parameter | Direction | Default | Description |
|---|---|---|---|
dev |
in | — | Device handle from ha_drv_open. |
kernel |
in | — | Handle from ha_drv_kernel_load_from_file or ha_drv_kernel_load_from_library on dev. |
Returns — HaSuccess, also for a handle a device reset already invalidated; HaInvalidArgument if dev is
null or kernel was loaded on another device; HaDeviceUnsupported if dev cannot execute kernels;
HaKernelInvalidHandle if kernel was already released or never loaded.
Signature · Example
1 | |
Example
1 2 | |