1. Overview¶
The driver API (ha_drv_*) is a device-handle-based API: every function that operates on a device takes a device handle. Where the
runtime API (Ha*) corresponds to the CUDA runtime API, the driver API corresponds to the CUDA driver API.
The runtime API is sufficient for most applications. Use the driver API in lightweight environments (such as edge) where the device must be specified by handle, or when you implement your own runtime or framework backend and need to manage the lifetimes of devices, streams, and kernels yourself.
Note
For detailed per-function signatures, see the Driver API Reference. This guide follows the same chapter structure as the runtime API guide — execution (Chapter 2), memory (Chapter 3), ordering and synchronization (Chapter 4), failure handling (Chapter 5).
Contents¶
| Section | Objective |
|---|---|
| 1.1 Role of the Driver API | What the driver API does and how software built on it is structured |
| 1.2 Differences from the Runtime API | How devices are specified, numbering schemes, and why the two must not be mixed |
| 1.3 Key Concepts | Core vocabulary used throughout the following chapters |
| 1.4 Device Handle | Meaning, lifetime, sharing, and ownership of HaDrvDev |
| 1.5 Basic Usage Example | Basic flow from memory allocation and copy to kernel launch and synchronization |
1.1 Role of the Driver API¶
The driver API handles device management, memory, execution, and synchronization with each call targeting a single device handle.
The figure below shows the expected structure of software that uses the driver API. A layer the user builds (a custom runtime, a framework backend, and so on) sits between the application threads and the driver API, takes care of device handles, object lifetimes, and cross-device coordination, and passes a device handle on every call.
Layering of software that uses the driver API
The driver API covers the following five areas.
| Area | What it does | Main functions | Guide |
|---|---|---|---|
| Device management | Bootstrap, open and close handles, read attributes, reset | ha_drv_init, ha_drv_open, ha_drv_close, ha_drv_get_attr, ha_drv_reset |
Execution Model |
| Memory management | Allocate and free device memory, copy between the host and the device | ha_drv_malloc, ha_drv_free, ha_drv_memcpy, ha_drv_memcpy_nd |
Memory Model |
| Execution control | Load kernels from files or kernel libraries, enqueue launches on streams | ha_drv_kernel_load_from_file, ha_drv_kernel_load_from_library, ha_drv_kernel_launch |
Execution Model |
| Ordering and synchronization | Create streams and events, enqueue work on streams, wait for completion | ha_drv_obj_create, ha_drv_stream_post, ha_drv_wait |
Streams and Events |
| Failure handling | Check errors left on streams, reset devices | ha_drv_wait, ha_drv_reset |
Error Handling and Reset |
Every function that operates on a device takes a device handle (HaDrvDev) as its first argument; no function operates
on multiple devices at once. Functions that do not depend on a device (bootstrap, host memory, kernel libraries, and
updating values in an argument list) do not take a handle. The API is available only from C++ (with C linkage) and requires linking the CMake target hart::ha_umd_core.
1.2 Differences from the Runtime API¶
The two APIs differ as follows.
| Aspect | Runtime API (Ha*) |
Driver API (ha_drv_*) |
|---|---|---|
| How the device is specified | Per-thread current device (HaSetDevice) |
The device to use is specified with a HaDrvDev handle argument |
| Device numbering | Virtual IDs for successfully initialized devices (renumbered from 0) | Physical hardware device index |
| Multi-device handling | Automatically routes each call to the device that owns the pointer, and supports direct P2P transfer | Single-device control (multi-device setups are built by combining calls) |
| Kernel launch | Launches by binary path or by a kernel handle from a kernel library (per-device loading is automatic) | Explicit load (ha_drv_kernel_load_from_file, ha_drv_kernel_load_from_library), then launch by handle, enqueued asynchronously on a stream |
| Memory copy interface | Separate synchronous (HaMemCpy) and asynchronous (HaMemCpyAsync) functions |
Always enqueued asynchronously on a stream |
| CMake link target | hart::ha_hart |
hart::ha_umd_core |
| Explicit streams and events | HaStreamCreate, HaEventCreate (HaStream_t, HaEvent_t) |
ha_drv_obj_create (HaStream_t, HaDrvEvent). Driver events are separate objects from runtime events |
| Memory free | HaMemFree, and stream-ordered HaMemFreeAsync |
Immediate free (ha_drv_free), and stream-ordered HA_STREAM_OP_FREE |
| Synchronization | HaStreamSynchronize, HaEventSynchronize, HaDeviceSynchronize |
A single entry point, ha_drv_wait, waits on streams, events, and devices |
| Device reset | HaResetDevice, HaResetAllDevices |
ha_drv_reset (keeps the device handle, invalidates child objects) |
| Languages | C++, Python | C++ (C linkage) |
There is one default stream for each (thread, device) pair, and the two APIs share the default stream on the same thread and device.
Note: Device numbers do not match
The index passed to ha_drv_open is a hardware device index, whereas a runtime API device ID is a
virtual ID. If any device fails to initialize, the
two numbering schemes diverge, so the same number is not guaranteed to refer to the same device.
Do not mix the runtime API and the driver API on the same device
The two APIs share the same device state, but the information the runtime API manages separately (the global address table and stream/event bookkeeping) does not track driver API calls. Mixing the two APIs on the same device causes the following problems.
- Memory management conflicts: If memory allocated with one API is freed or copied with the other, the runtime does not recognize ownership of the address, or a freed address remains in the runtime's bookkeeping.
- Incompatible events: Driver events (
HaDrvEvent) and runtime events (HaEvent_t) are separate objects and cannot wait on each other. - Unexpected handle invalidation:
ha_drv_resetalso invalidates streams, events, and memory allocations created by the runtime API, but the runtime is not aware of this.
Recommendation: If a process needs both APIs, dedicate each device to only one of them.
1.3 Key Concepts¶
| Key concept | Description |
|---|---|
Device handle (HaDrvDev) |
A reference to one bootstrapped device. Obtained with ha_drv_open and released with ha_drv_close (see Device Handle). |
Default stream (HA_STREAM_DEFAULT) |
One stream per (calling thread, device). Created automatically on first use (see Stream). |
Stream operation (HaDrvStreamOp) |
An operation enqueued on a stream with ha_drv_stream_post. Includes copies, deferred frees, file loads, event records, and more (see Stream). |
Sequence number (seq) |
A number assigned to each operation enqueued on a stream. Used to tell which operations a synchronization covers (see Synchronization). |
Kernel handle (HaDrvKernel) |
A kernel loaded onto a device with ha_drv_kernel_load_from_file or ha_drv_kernel_load_from_library. Pass this handle as an argument when launching a kernel (see Kernel Loading and Execution). |
| Stream error (Sticky error) | The error from an operation that failed during execution. It stays on the stream and is returned by every later wait and enqueue (see Stream Errors). |
| Device reset | An operation that reinitializes a device and invalidates all of its child handles at once (see Device Reset). |
1.4 Device Handle¶
HaDrvDev is how every driver API call specifies its target device. Whereas the runtime API implicitly uses the
"current device" selected per thread, every driver API call takes this handle explicitly.
What a Handle Is¶
HaDrvDev is a reference to one already bootstrapped device. It is not the device itself, so opening or closing a
handle does not create or destroy a device. Devices are created and destroyed by the per-process ha_drv_init /
ha_drv_deinit; a handle only takes and drops a reference in between (Device Management).
- Open and close:
ha_drv_open(index, &dev)takes a reference andha_drv_close(dev)drops it. Each successful open needs exactly one close, and a closed handle must not be used again. - Shutdown condition:
ha_drv_deinitreleases devices only when no handle is open. If any remain, it returnsHaDeviceBusyand releases nothing. - Index: The
indexofha_drv_openis a hardware device index. To get the index back from a handle, useha_drv_get_device_id(dev). It is a different number from the runtime API's virtual device ID (Differences from the Runtime API).
Multiple Handles¶
Opening the same index multiple times creates an additional handle to the same device each time. These handles share
the device state. Memory allocated through one handle can be used by operations enqueued through another, and
HA_STREAM_DEFAULT refers to the same default stream for both handles, so operations enqueued through both handles
share a single sequence (seq). Each handle must be closed separately.
Device handle lifetime and two handles sharing the same device state
Threads¶
A handle is not bound to a particular thread. Multiple threads can use the same handle concurrently. However,
passing HA_STREAM_DEFAULT as the stream argument means the default stream of the calling thread, so each thread
uses a different default stream even with the same handle (Stream).
Owned Objects¶
Streams, events, kernels, and memory allocations belong to the device that created them. A child object passed to a function must belong to the same device as the handle passed with it; passing an object from another device is rejected.
| Object passed by mistake | Returns |
|---|---|
| Stream from another device | HaStreamInvalidHandle |
| Event from another device | HaEventInvalidHandle |
| Kernel loaded on another device | HaInvalidArgument |
Reset¶
The device handle remains valid after ha_drv_reset(dev). Only the device's child objects (streams, events,
allocations, kernels) are invalidated, so recreate them with the same handle (Device Reset).
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 | |
| API | Sync / Async | Behavior |
|---|---|---|
ha_drv_open() |
Sync | Obtains a handle to a bootstrapped device |
ha_drv_close() |
Sync | Releases the handle. The device stays bootstrapped |
ha_drv_get_device_id() |
Sync | Queries the hardware device index the handle refers to |
ha_drv_is_ready() |
Sync | Checks that the device has not been released and the handle is valid. Does not reflect whether a reset is in progress |
ha_drv_get_attr() |
Sync | Reads a single device attribute |
1.5 Basic Usage Example¶
This example opens device 0, allocates memory, copies data, launches a kernel, and reads back the result.
The copies and the launch are all enqueued in order on the calling thread's default stream, and a single ha_drv_wait at the end confirms that all of them completed.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 | |
Next Steps¶
- Execution Model — Device management and kernel launch
- Driver API Reference — Per-function signatures and return codes