Execution API¶
Kernel launch, kernel libraries, and the argument list a launch takes. The difference between the synchronous and asynchronous forms is described in Execution Model.
Contents¶
| API | 설명 |
|---|---|
HaLibraryLoadData |
미리 컴파일된 커널 라이브러리 등록 |
HaLibraryGetKernel |
라이브러리에서 이름으로 커널 핸들 조회 |
HaLaunchKernel |
커널 동기 실행 (커널 핸들 또는 바이너리 경로 지정) |
HaLaunchKernelAsync |
커널 비동기 실행 (커널 핸들 또는 바이너리 경로 지정) |
HaLaunchArgsAddTensor |
Tensor 인자 추가 |
HaLaunchArgsAddScalar |
Scalar 인자 추가 |
HaLaunchArgsAddArray |
Array 인자 추가 |
HaLaunchArgsSetTensor |
추가한 Tensor 인자의 주소·크기 교체 |
HaLaunchArgsSetScalarBits |
추가한 Scalar 인자의 값 교체 |
HaLaunchArgsSetArray |
추가한 Array 인자의 크기·데이터 교체 |
Launch arguments¶
HaLaunchKernel() and HaLaunchKernelAsync() take a HaLaunchArgs list: the arguments of one launch, in the order they were appended. The records are built inside the runtime in a layout that belongs to the kernel ABI and is not published, so a caller never depends on that layout.
- Object.
HaLaunchArgsis a C++ class that owns its records: construct it to get an empty list, and the records are released when the object is destroyed. It is move-only; a list that has been moved from is rejected by every function withHaKernelInvalidArgument. Construction allocates and throwsstd::bad_allocwhen memory is short. - Arguments. Tensor, scalar and array arguments, each with a
HaLaunchArgRoutethat picks the argument block (and a target core forCore). The route must be the one the kernel was compiled with. - Checked against the current device. Appending checks each argument against the device that is current on the calling thread (
HaSetDevice()): aCoreroute must name one of that device's cores, and the call returnsHaDeviceNotInitializedwhen the thread has no current device andHaDeviceNotReadywhen that device is not ready (a reset is in progress).HaLaunchArgsSet*()needs no device: it changes records the list already holds, so neither a missing current device nor a reset in progress affects it. The list does not remember which device it was built against: the launch checks it again against the device it launches on, so an argument naming a core that device does not have is rejected there instead of being dropped. To build a list for a device other than the current one, use the driver API'sha_drv_launch_args_*()functions. - Consumed before return. Both launch calls read every record on the calling thread before they return, so the list may be destroyed or updated right after either call; an asynchronous launch needs no different handling.
- Reuse. A caller that launches the same argument shape again can keep one list and update its records in place with
HaLaunchArgsSet*(), by the position each record was appended at (counting from 0). - Threading. Several threads may launch from one list at the same time, and may also append to it or update it: appends and updates take the list's lock exclusively and a launch reads under it shared, so racing calls are serialized in some order. Which update a racing launch sees is not defined. Moving or destroying a list is not synchronized with its use, so do not move or destroy a list while another thread uses it. Lists share no state.
A mismatch between the list and the kernel — a wrong type, a scalar where an array is read, a different argument count — is rejected at launch with HaKernelInvalidArgument, and the log names the argument.
예제
1 2 3 4 5 6 7 8 | |
HaLibraryLoadData¶
Registers a library of precompiled kernels. data points to an HaLibraryData (defined in ha_types.h).
| Parameter | Direction | Default | Description |
|---|---|---|---|
library |
out | — | Receives the library. |
data |
in | — | The library data (a HaLibraryData). |
Returns — HaSuccess; HaInvalidArgument if library or data is null; HaKernelInvalidArgument if data is not a valid HaLibraryData, or has no binary set for a device version this runtime knows; HaKernelUnsupported if the library was built against a version of this runtime whose library format this one does not read.
시그니처 · 예제
1 | |
예제
1 2 3 4 5 | |
Note
The HaLibraryData structs are only read during the call, but the kernel names and ELF bytes they point to are not copied and must stay valid for the life of the process. Call once per library per process: every call registers a new library.
Note
No device is touched: kernels are loaded onto a device when first launched there, and their code is checked then. Binary sets for device versions this runtime does not recognize are ignored without being checked; a kernel that appears only in such sets is treated as absent from the library. data is not a typed pointer, so only its magic and version are checked: it must point to a complete HaLibraryData.
HaLibraryGetKernel¶
Looks up a kernel of library by name. The same name returns the same kernel on every call, whatever device it later runs on.
| Parameter | Direction | Default | Description |
|---|---|---|---|
kernel |
out | — | Receives the kernel. |
library |
in | — | Library from HaLibraryLoadData(). |
name |
in | — | Kernel name. |
Returns — HaSuccess; HaInvalidArgument if an argument is null; HaKernelInvalidHandle if library did not come from HaLibraryLoadData(); HaKernelNotFound if the library has no kernel named name.
시그니처 · 예제
1 | |
예제
1 2 | |
HaLaunchKernel¶
Launches a kernel of a library on the current device and waits for it.
| Parameter | Direction | Default | Description |
|---|---|---|---|
kernel |
in | — | Kernel from HaLibraryGetKernel(). |
args |
in | — | Argument list (required); see Launch arguments. |
Returns — HaSuccess; HaKernelInvalidHandle if kernel did not come from HaLibraryGetKernel(); HaKernelInvalidArgument if args was moved from or does not match the kernel's arguments, or if the kernel's code is invalid; HaKernelUnsupported for a library kernel built for a kernel ABI this runtime does not support; HaKernelNoBinaryForDevice if the library has no binary of kernel for the device; or a load or launch error.
시그니처 · 예제 (kernel handle)
1 | |
예제
1 2 | |
Note
The first launch of a kernel on a device loads that kernel there; kernels never launched on a device take no device memory there. A kernel's code may be written to the device on the stream of its first launch there; if that write fails, launches of the kernel on any stream fail with that stream's error. Destroy that stream to recover (the next launch writes the kernel again), or reset the device if it is a default stream. The list is consumed before this returns, so it may be destroyed or updated right after.
Launches a device kernel from a kernel binary path (eager path). The list is consumed before this returns, so it may be destroyed or updated right after.
| Parameter | Direction | Default | Description |
|---|---|---|---|
binary_path |
in | — | Path to the kernel binary. |
args |
in | — | Argument list (required); see Launch arguments. |
Returns — HaSuccess; HaKernelInvalidArgument for an empty path, a list that was moved from, does not fit the kernel or that the kernel does not take, or a malformed binary; HaKernelNotFound if binary_path does not exist or names no known kernel; HaKernelUnsupported for a kernel built for another target or a kernel ABI this runtime does not support; or a load or launch error.
시그니처 · 예제 (binary path)
1 | |
예제
1 | |
HaLaunchKernelAsync¶
Asynchronously launches a kernel of a library on a stream. Loading and argument semantics are those of HaLaunchKernel(HaKernel_t, const HaLaunchArgs &).
| Parameter | Direction | Default | Description |
|---|---|---|---|
kernel |
in | — | Kernel from HaLibraryGetKernel(). |
args |
in | — | Argument list (required); see Launch arguments. |
stream |
in | HA_STREAM_DEFAULT |
Stream to enqueue into; omit or pass HA_STREAM_DEFAULT to use the calling thread's default stream on the current device. |
Returns — HaSuccess if enqueued, otherwise as HaLaunchKernel(HaKernel_t, const HaLaunchArgs &).
시그니처 · 예제 (kernel handle)
1 | |
예제
1 2 | |
Asynchronously launches a device kernel from a binary path on a stream (eager path). Enqueueing returns as soon as the work is queued, so overlapping a transfer with a launch is a matter of putting both on one stream and synchronizing once at the end.
| Parameter | Direction | Default | Description |
|---|---|---|---|
binary_path |
in | — | Path to the kernel binary. |
args |
in | — | Argument list (required); see Launch arguments. |
stream |
in | HA_STREAM_DEFAULT |
Stream to enqueue into; omit or pass HA_STREAM_DEFAULT to use the calling thread's default stream on the current device. |
Returns — HaSuccess if enqueued, otherwise as HaLaunchKernel(binary_path, ...).
시그니처 · 예제 (binary path)
1 2 | |
예제
1 2 3 4 | |
Note
Argument semantics are those of HaLaunchKernel(binary_path, const HaLaunchArgs &). The list is consumed before this returns.
HaLaunchArgsAddTensor¶
Appends a tensor argument.
| Parameter | Direction | Default | Description |
|---|---|---|---|
args |
in, out | — | The list. |
route |
in | — | Block the argument is written to. One of Universal, Top, Core (see HaLaunchArgRoute). |
target_core |
in | — | Core whose block receives a HaLaunchArgRoute::Core argument, from 0 to one less than the core count of the current device. Ignored otherwise. |
da |
in | — | Device address of the tensor's base. |
dims |
in | — | rank extents, in the order the kernel declares them. Copied, so dims need not outlive this call. |
rank |
in | — | Number of entries in dims. |
offset |
in | — | Element offset from da. |
Returns — HaSuccess; HaDeviceNotInitialized, appending nothing, when the calling thread has no current device; HaDeviceNotReady, appending nothing, when that device is not ready (a reset is in progress); or HaKernelInvalidArgument, appending nothing, when args was moved from, route is not a HaLaunchArgRoute, rank is negative, dims is nullptr with a positive rank, or route is HaLaunchArgRoute::Core and target_core names a core the current device does not have.
시그니처
1 2 | |
HaLaunchArgsAddScalar¶
Appends a scalar argument.
| Parameter | Direction | Default | Description |
|---|---|---|---|
args |
in, out | — | The list. |
route |
in | — | Block the argument is written to. One of Universal, Top, Core (see HaLaunchArgRoute). |
target_core |
in | — | Core whose block receives a HaLaunchArgRoute::Core argument. As for HaLaunchArgsAddTensor(). |
dtype |
in | — | Element type of the scalar: Int8, Int16, Int32, Int64, Float16, BFloat16, Float32, Float64 or Float8 (E4M3). Any other type is rejected. |
bits |
in | — | The value, bit for bit. Only the low bytes the type occupies are read, so a negative Int32 is its two's complement in the low 4 bytes and a Float32 is its IEEE pattern in the low 4 bytes. |
Returns — HaSuccess, or HaKernelInvalidArgument, appending nothing, under the conditions of HaLaunchArgsAddTensor() that apply, or when dtype is not one of the types above.
시그니처
1 2 | |
HaLaunchArgsAddArray¶
Appends an array argument: element bytes the kernel reads from its instruction table. Unlike a tensor, the data is not a device pointer; the bytes are embedded in the argument block.
| Parameter | Direction | Default | Description |
|---|---|---|---|
args |
in, out | — | The list. |
route |
in | — | Block the argument is written to. One of Universal, Top, Core (see HaLaunchArgRoute). |
target_core |
in | — | Core whose block receives a HaLaunchArgRoute::Core argument. As for HaLaunchArgsAddTensor(). |
dims |
in | — | rank extents, in the order the kernel declares them. Copied. |
rank |
in | — | Number of entries in dims. |
strides |
in | — | rank strides, or nullptr for unit strides. Copied. |
raw_data |
in | — | The element bytes. Copied. |
raw_data_size |
in | — | Number of bytes in raw_data. |
Returns — HaSuccess, or HaKernelInvalidArgument, appending nothing, under the conditions of HaLaunchArgsAddTensor() that apply, or when raw_data is nullptr with a positive raw_data_size.
시그니처 · 예제
1 2 | |
예제
1 2 3 4 5 | |
HaLaunchArgsSetTensor¶
Replaces the address and extents of the tensor appended at index. Everything else about the record (route, target core, offset, strides) stays as it was appended.
| Parameter | Direction | Default | Description |
|---|---|---|---|
args |
in, out | — | The list. |
index |
in | — | Position the tensor was appended at, counting from 0. |
da |
in | — | New device address of the tensor's base. |
dims |
in | — | rank extents. Copied. |
rank |
in | — | Number of entries in dims. Must equal the rank the tensor was appended with. |
Returns — HaSuccess, or HaKernelInvalidArgument, changing nothing, when args was moved from, index is out of range or names a record that is not a tensor, rank differs from the record's, or dims is nullptr with a positive rank.
시그니처
1 | |
HaLaunchArgsSetScalarBits¶
Replaces the value of the scalar appended at index.
| Parameter | Direction | Default | Description |
|---|---|---|---|
args |
in, out | — | The list. |
index |
in | — | Position the scalar was appended at, counting from 0. |
bits |
in | — | New value, bit for bit, as for HaLaunchArgsAddScalar(). |
Returns — HaSuccess, or HaKernelInvalidArgument, changing nothing, when args was moved from, index is out of range or names a record that is not a scalar.
시그니처
1 | |
HaLaunchArgsSetArray¶
Replaces the extents and element bytes of the array appended at index.
| Parameter | Direction | Default | Description |
|---|---|---|---|
args |
in, out | — | The list. |
index |
in | — | Position the array was appended at, counting from 0. |
dims |
in | — | rank extents. Copied. |
rank |
in | — | Number of entries in dims. Must equal the rank the array was appended with. |
raw_data |
in | — | The element bytes. Copied. |
raw_data_size |
in | — | Number of bytes in raw_data. |
Returns — HaSuccess, or HaKernelInvalidArgument, changing nothing, when args was moved from, index is out of range or names a record that is not an array, rank differs from the record's, dims is nullptr with a positive rank, or raw_data is nullptr with a positive raw_data_size.
시그니처
1 2 | |