콘텐츠로 이동

Driver API: Execution

Kernel libraries, and kernel load, launch, and release by handle. See Driver API for the concepts.

Contents

이름 분류 설명
ha_drv_library_load_data API 미리 컴파일된 커널 라이브러리 등록 (디바이스 불필요)
ha_drv_library_get_kernel API 라이브러리에서 이름으로 커널 핸들 조회
ha_drv_library_kernel_name API 라이브러리 커널의 이름 조회
ha_drv_kernel_load_from_file API 커널 바이너리 파일 로드
ha_drv_kernel_load_from_library API 라이브러리 커널을 디바이스 칩에 맞는 빌드로 로드
ha_drv_kernel_launch API 커널 실행 (비동기)
ha_drv_launch_args_add_tensor API dev 기준으로 검사해 Tensor 인자 추가
ha_drv_launch_args_add_scalar API dev 기준으로 검사해 Scalar 인자 추가
ha_drv_launch_args_add_array API dev 기준으로 검사해 Array 인자 추가
ha_drv_launch_args_set_tensor API 추가한 Tensor 인자의 주소·크기 교체 (디바이스 불필요)
ha_drv_launch_args_set_scalar_bits API 추가한 Scalar 인자의 값 교체
ha_drv_launch_args_set_array API 추가한 Array 인자의 크기·데이터 교체
ha_drv_kernel_release API 커널 핸들 해제

ha_drv_library_load_data

Register a kernel library (CUDA cuLibraryLoadData). Needs no device: callable before ha_drv_init.

Validates data (a HaLibraryData, ha_types.h) and issues a handle for it; nothing is copied or loaded onto a device. data is only read during the call; the names and ELF bytes it points to are not copied and must stay valid for the life of the process. Binary sets for a device version this runtime does not know are ignored, unchecked. data is not a typed pointer, so only its magic and version are checked: it must point to a complete HaLibraryData.

Parameter Direction Default Description
library out — Receives the library handle on success; untouched on failure.
data in — The library data (a HaLibraryData).

Returns — HaSuccess; HaInvalidArgument if library or data is null; HaKernelUnsupported for a HaLibraryData version this runtime does not read; HaKernelInvalidArgument for a malformed library or one with no binary set for a known device version (nothing is registered).

시그니처 · 예제
1
HaError ha_drv_library_load_data(HaLibrary_t *library, const void *data);

예제

1
2
3
// my_kernels_data: 이 바이너리에 들어 있는 HaLibraryData
HaLibrary_t lib = nullptr;
ha_drv_library_load_data(&lib, &my_kernels_data);   // ha_drv_init 전에도 호출 가능

ha_drv_library_get_kernel

The kernel named name in library (CUDA cuLibraryGetKernel). The handle is not bound to any device; load it onto a device with ha_drv_kernel_load_from_library.

Parameter Direction Default Description
library_kernel out — Receives the kernel handle on success; untouched on failure.
library in — Handle from ha_drv_library_load_data.
name in — Kernel name.

Returns — HaSuccess; HaInvalidArgument if library_kernel, library or name is null; HaKernelInvalidHandle if library was not issued by ha_drv_library_load_data; HaKernelNotFound if no set of the library has name.

시그니처 · 예제
1
HaError ha_drv_library_get_kernel(HaKernel_t *library_kernel, HaLibrary_t library, const char *name);

예제

1
2
3
4
HaKernel_t add = nullptr;
if (ha_drv_library_get_kernel(&add, lib, "add") == HaKernelNotFound) {
  // 라이브러리에 그 이름의 커널이 없음
}

ha_drv_library_kernel_name

The name of a library kernel.

Parameter Direction Default Description
library_kernel in — Any value.
name out — Receives the name on success (valid for the life of the process); untouched on failure.

Returns — HaSuccess; HaInvalidArgument if name is null; HaKernelInvalidHandle if library_kernel was not issued by ha_drv_library_get_kernel.

시그니처 · 예제
1
HaError ha_drv_library_kernel_name(HaKernel_t library_kernel, const char **name);

예제

1
2
const char *name = nullptr;
ha_drv_library_kernel_name(add, &name);   // "add"

ha_drv_kernel_load_from_file

Load the kernel binary at path onto dev (CUDA cuModuleLoad).

Reads and checks the file and reserves its device memory; does not wait on the device. Instead of being uploaded during the load, the binary may be uploaded on the stream of the kernel's first ha_drv_kernel_launch. Loading a kernel already loaded on dev (same file: canonical path, inode, size and mtime) reuses the existing load: it returns a new handle but allocates no new device memory. ha_drv_kernel_release does not free that memory; only a device reset or device teardown does.

Parameter Direction Default Description
dev in — Device handle from ha_drv_open.
path in — Kernel binary (Legato ELF).
out out — Receives the kernel handle on success.

Returns — HaSuccess; HaInvalidArgument if dev/path/out is null; HaDeviceUnsupported if dev cannot execute kernels; HaDeviceNotReady if a reset is in progress; HaKernelNotFound if path does not exist or has no ABI generator; HaKernelInvalidArgument for a malformed or oversized file; HaKernelUnsupported for a Legato ELF of another target or kernel ABI major; HaMemoryAllocationFailed when the device's kernel memory is full; HaKernelLoadFailed when the device fails the load otherwise.

시그니처 · 예제
1
HaError ha_drv_kernel_load_from_file(HaDrvDev dev, const char *path, HaDrvKernel *out);

예제

1
2
3
4
5
HaDrvKernel k = nullptr;
HaError err = ha_drv_kernel_load_from_file(dev, "kernels/add.elf", &k);
if (err == HaKernelNotFound) {
  // 경로 확인
}

ha_drv_kernel_load_from_library

Load a library kernel onto dev, the build for the device's chip.

Reads and checks the ELF of library_kernel in the set for the device version of dev, and reserves its device memory; does not wait on the device. If the binary is not uploaded during the load, it is uploaded on the stream of the kernel's first ha_drv_kernel_launch. The handle is launched with ha_drv_kernel_launch, like one from ha_drv_kernel_load_from_file; a device reset invalidates it. Loading the same kernel onto the same device again returns a new handle to the copy made by the first load, until a device reset releases it.

Parameter Direction Default Description
dev in — Device handle from ha_drv_open.
library_kernel in — The kernel in its library, from ha_drv_library_get_kernel; it is not bound to any device.
loaded_kernel out — Receives the kernel as loaded on dev on success; untouched on failure.

Returns — HaSuccess; HaInvalidArgument if dev or loaded_kernel is null; HaKernelInvalidHandle if library_kernel was not issued by ha_drv_library_get_kernel; HaKernelNoBinaryForDevice if the library has no binary of the kernel for the chip of dev; HaDeviceUnsupported if dev cannot execute kernels; HaDeviceNotReady if a reset is in progress; HaKernelInvalidArgument for a malformed ELF; HaKernelUnsupported for an ELF of another target or kernel ABI major; HaMemoryAllocationFailed when the device's kernel memory is full; HaKernelLoadFailed when the device fails the load otherwise. On any error nothing is loaded.

시그니처 · 예제
1
HaError ha_drv_kernel_load_from_library(HaDrvDev dev, HaKernel_t library_kernel, HaDrvKernel *loaded_kernel);

예제

1
2
3
4
HaDrvKernel k = nullptr;
if (ha_drv_kernel_load_from_library(dev, add, &k) == HaKernelNoBinaryForDevice) {
  // 라이브러리에 이 디바이스 칩용 빌드가 없음
}

ha_drv_kernel_launch

Launch kernel on stream with the arguments of a HaLaunchArgs list (CUDA cuLaunchKernel). Asynchronous.

Build the list as a HaLaunchArgs object and fill it with ha_drv_launch_args_add_*(), which check each argument against dev (see Execution for the list itself). The list is consumed before this returns, so it may be destroyed or updated right after.

When the binary is uploaded on the launch stream, the kernel's first launch posts the binary copy on stream ahead of the launch; a copy failure is stream's sticky error. A first launch on another stream while that copy is in flight waits on the host for it, so this call may block (and is woken by a device reset like ha_drv_wait).

Errors are reported device first, then kernel: a device or stream error takes precedence over a kernel handle error.

Parameter Direction Default Description
dev in — Device handle from ha_drv_open.
stream in — Stream to launch on, or HA_STREAM_DEFAULT.
kernel in — Handle from ha_drv_kernel_load_from_file or ha_drv_kernel_load_from_library on dev.
args in — Argument list.

Returns — HaSuccess if queued; HaInvalidArgument if dev is null or kernel was loaded on another device; HaKernelInvalidArgument if args was moved from or does not fit the kernel's ABI; HaDeviceUnsupported if dev cannot execute kernels; HaDeviceNotReady if a reset is in progress or completed during the call; HaStreamInvalidHandle if stream is not owned by dev; HaKernelInvalidHandle if kernel was released or invalidated by a reset; the binary copy's post or wait error; or the stream's sticky error. No seq is consumed on failure.

시그니처 · 예제
1
HaError ha_drv_kernel_launch(HaDrvDev dev, HaStream_t stream, HaDrvKernel kernel, const HaLaunchArgs &args);

예제

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
HaDrvKernel k = nullptr;
ha_drv_kernel_load_from_file(dev, "kernels/add.elf", &k);

uint32_t dims[] = {1024};
HaLaunchArgs args;                      // 스코프를 벗어나면 해제됨
ha_drv_launch_args_add_tensor(dev, args, HaLaunchArgRoute::Universal, -1, a, dims, 1, 0);
ha_drv_launch_args_add_tensor(dev, args, HaLaunchArgRoute::Universal, -1, b, dims, 1, 0);
ha_drv_launch_args_add_tensor(dev, args, HaLaunchArgRoute::Universal, -1, out, dims, 1, 0);

ha_drv_kernel_launch(dev, HA_STREAM_DEFAULT, k, args);   // 반환 시점에 인자는 이미 소비됨
ha_drv_wait(dev, HA_STREAM_DEFAULT, nullptr, HA_WAIT_STREAM_SYNC, -1, nullptr);

ha_drv_kernel_release(dev, k);

ha_drv_launch_args_add_tensor

Append a tensor argument to args, checked against dev. The same as the runtime's HaLaunchArgsAddTensor(), for the device the caller names instead of the thread's current device: a Core route whose target_core is not one of dev's cores is rejected. The list is not bound to dev; the launch checks it again against the device it launches on.

Parameter Direction Default Description
dev in — Device handle from ha_drv_open.
args in, out — The list. See HaLaunchArgs.
route in — Block the argument is written to. One of Universal, Top, Core (see HaLaunchArgRoute).
target_core in — Core whose block receives a HaLaunchArgRoute::Core argument. Ignored otherwise.
da in — Device address of the tensor's base.
dims in — rank extents, in the order the kernel declares them. Copied.
rank in — Number of entries in dims.
offset in — Element offset from da.

Returns — HaSuccess; HaInvalidArgument if dev is null; HaDeviceNotReady if a reset is in progress; or HaKernelInvalidArgument, appending nothing, under the conditions of the runtime's HaLaunchArgsAddTensor() that apply.

시그니처
1
2
HaError ha_drv_launch_args_add_tensor(HaDrvDev dev, HaLaunchArgs &args, HaLaunchArgRoute route, int target_core,
                                      HaDevicePtr da, const uint32_t *dims, int rank, uint64_t offset);

ha_drv_launch_args_add_scalar

Append a scalar argument to args, checked against dev; see ha_drv_launch_args_add_tensor.

Parameter Direction Default Description
dev in — Device handle from ha_drv_open.
args in, out — The list.
route in — Block the argument is written to. One of Universal, Top, Core (see HaLaunchArgRoute).
target_core in — Core whose block receives a HaLaunchArgRoute::Core argument. Ignored otherwise.
dtype in — Element type of the scalar, as for HaLaunchArgsAddScalar().
bits in — The value, bit for bit.

Returns — HaSuccess; HaInvalidArgument if dev is null; HaDeviceNotReady if a reset is in progress; or HaKernelInvalidArgument, appending nothing, under the conditions of the runtime's HaLaunchArgsAddScalar().

시그니처
1
2
HaError ha_drv_launch_args_add_scalar(HaDrvDev dev, HaLaunchArgs &args, HaLaunchArgRoute route, int target_core,
                                      HaDataType dtype, uint64_t bits);

ha_drv_launch_args_add_array

Append an array argument to args, checked against dev; see ha_drv_launch_args_add_tensor.

Parameter Direction Default Description
dev in — Device handle from ha_drv_open.
args in, out — The list.
route in — Block the argument is written to. One of Universal, Top, Core (see HaLaunchArgRoute).
target_core in — Core whose block receives a HaLaunchArgRoute::Core argument. Ignored otherwise.
dims in — rank extents. Copied.
rank in — Number of entries in dims.
strides in — rank strides, or nullptr for unit strides. Copied.
raw_data in — The element bytes. Copied.
raw_data_size in — Number of bytes in raw_data.

Returns — HaSuccess; HaInvalidArgument if dev is null; HaDeviceNotReady if a reset is in progress; or HaKernelInvalidArgument, appending nothing, under the conditions of the runtime's HaLaunchArgsAddArray().

시그니처
1
2
3
HaError ha_drv_launch_args_add_array(HaDrvDev dev, HaLaunchArgs &args, HaLaunchArgRoute route, int target_core,
                                     const uint32_t *dims, int rank, const uint32_t *strides, const uint8_t *raw_data,
                                     size_t raw_data_size);

ha_drv_launch_args_set_tensor

Replace the address and extents of the tensor at index; see the runtime's HaLaunchArgsSetTensor(). Unlike the add functions this needs no device: it changes records the list already holds, so it takes no device handle, is not held off by a reset, and works on a thread that has no current device.

Parameter Direction Default Description
args in, out — The list.
index in — Position the tensor was appended at, counting from 0.
da in — New device address of the tensor's base.
dims in — rank extents. Copied.
rank in — Number of entries in dims; must equal the rank it was appended with.

Returns — HaSuccess, or HaKernelInvalidArgument, changing nothing, under the conditions of the runtime's HaLaunchArgsSetTensor().

시그니처
1
2
HaError ha_drv_launch_args_set_tensor(HaLaunchArgs &args, int index, HaDevicePtr da, const uint32_t *dims,
                                      int rank);

ha_drv_launch_args_set_scalar_bits

Replace the value of the scalar at index; see the runtime's HaLaunchArgsSetScalarBits() and ha_drv_launch_args_set_tensor; like it, this needs no device.

Parameter Direction Default Description
args in, out — The list.
index in — Position the scalar was appended at, counting from 0.
bits in — New value, bit for bit.

Returns — HaSuccess, or HaKernelInvalidArgument, changing nothing, under the conditions of the runtime's HaLaunchArgsSetScalarBits().

시그니처
1
HaError ha_drv_launch_args_set_scalar_bits(HaLaunchArgs &args, int index, uint64_t bits);

ha_drv_launch_args_set_array

Replace the extents and element bytes of the array at index; see the runtime's HaLaunchArgsSetArray() and ha_drv_launch_args_set_tensor; like it, this needs no device.

Parameter Direction Default Description
args in, out — The list.
index in — Position the array was appended at, counting from 0.
dims in — rank extents. Copied.
rank in — Number of entries in dims; must equal the rank it was appended with.
raw_data in — The element bytes. Copied.
raw_data_size in — Number of bytes in raw_data.

Returns — HaSuccess, or HaKernelInvalidArgument, changing nothing, under the conditions of the runtime's HaLaunchArgsSetArray().

시그니처
1
2
HaError ha_drv_launch_args_set_array(HaLaunchArgs &args, int index, const uint32_t *dims, int rank,
                                     const uint8_t *raw_data, size_t raw_data_size);

ha_drv_kernel_release

Release kernel (CUDA cuModuleUnload). The handle is invalid on return; launches already queued run. The device memory stays loaded (shared with other handles of the same kernel, and reused by a later load of it) until a device reset or the device's teardown.

Parameter Direction Default Description
dev in — Device handle from ha_drv_open.
kernel in — Handle from ha_drv_kernel_load_from_file or ha_drv_kernel_load_from_library on dev.

Returns — HaSuccess, also for a handle a device reset already invalidated; HaInvalidArgument if dev is null or kernel was loaded on another device; HaDeviceUnsupported if dev cannot execute kernels; HaKernelInvalidHandle if kernel was already released or never loaded.

시그니처 · 예제
1
HaError ha_drv_kernel_release(HaDrvDev dev, HaDrvKernel kernel);

예제

1
2
ha_drv_kernel_release(dev, k);          // 이미 큐에 들어간 실행은 그대로 진행
k = nullptr;