Skip to content

1. Overview

The driver API (ha_drv_*) is a device-handle-based API: every function that operates on a device takes a device handle. Where the runtime API (Ha*) corresponds to the CUDA runtime API, the driver API corresponds to the CUDA driver API.

The runtime API is sufficient for most applications. Use the driver API in lightweight environments (such as edge) where the device must be specified by handle, or when you implement your own runtime or framework backend and need to manage the lifetimes of devices, streams, and kernels yourself.

Note

For detailed per-function signatures, see the Driver API Reference. This guide follows the same chapter structure as the runtime API guide — execution (Chapter 2), memory (Chapter 3), ordering and synchronization (Chapter 4), failure handling (Chapter 5).


Contents

Section Objective
1.1 Role of the Driver API What the driver API does and how software built on it is structured
1.2 Differences from the Runtime API How devices are specified, numbering schemes, and why the two must not be mixed
1.3 Key Concepts Core vocabulary used throughout the following chapters
1.4 Device Handle Meaning, lifetime, sharing, and ownership of HaDrvDev
1.5 Basic Usage Example Basic flow from memory allocation and copy to kernel launch and synchronization

1.1 Role of the Driver API

The driver API handles device management, memory, execution, and synchronization with each call targeting a single device handle.

The figure below shows the expected structure of software that uses the driver API. A layer the user builds (a custom runtime, a framework backend, and so on) sits between the application threads and the driver API, takes care of device handles, object lifetimes, and cross-device coordination, and passes a device handle on every call.

Layering of software that uses the driver API

Layering of software that uses the driver API

The driver API covers the following five areas.

Area What it does Main functions Guide
Device management Bootstrap, open and close handles, read attributes, reset ha_drv_init, ha_drv_open, ha_drv_close, ha_drv_get_attr, ha_drv_reset Execution Model
Memory management Allocate and free device memory, copy between the host and the device ha_drv_malloc, ha_drv_free, ha_drv_memcpy, ha_drv_memcpy_nd Memory Model
Execution control Load kernels from files or kernel libraries, enqueue launches on streams ha_drv_kernel_load_from_file, ha_drv_kernel_load_from_library, ha_drv_kernel_launch Execution Model
Ordering and synchronization Create streams and events, enqueue work on streams, wait for completion ha_drv_obj_create, ha_drv_stream_post, ha_drv_wait Streams and Events
Failure handling Check errors left on streams, reset devices ha_drv_wait, ha_drv_reset Error Handling and Reset

Every function that operates on a device takes a device handle (HaDrvDev) as its first argument; no function operates on multiple devices at once. Functions that do not depend on a device (bootstrap, host memory, kernel libraries, and updating values in an argument list) do not take a handle. The API is available only from C++ (with C linkage) and requires linking the CMake target hart::ha_umd_core.

1.2 Differences from the Runtime API

The two APIs differ as follows.

Aspect Runtime API (Ha*) Driver API (ha_drv_*)
How the device is specified Per-thread current device (HaSetDevice) The device to use is specified with a HaDrvDev handle argument
Device numbering Virtual IDs for successfully initialized devices (renumbered from 0) Physical hardware device index
Multi-device handling Automatically routes each call to the device that owns the pointer, and supports direct P2P transfer Single-device control (multi-device setups are built by combining calls)
Kernel launch Launches by binary path or by a kernel handle from a kernel library (per-device loading is automatic) Explicit load (ha_drv_kernel_load_from_file, ha_drv_kernel_load_from_library), then launch by handle, enqueued asynchronously on a stream
Memory copy interface Separate synchronous (HaMemCpy) and asynchronous (HaMemCpyAsync) functions Always enqueued asynchronously on a stream
CMake link target hart::ha_hart hart::ha_umd_core
Explicit streams and events HaStreamCreate, HaEventCreate (HaStream_t, HaEvent_t) ha_drv_obj_create (HaStream_t, HaDrvEvent). Driver events are separate objects from runtime events
Memory free HaMemFree, and stream-ordered HaMemFreeAsync Immediate free (ha_drv_free), and stream-ordered HA_STREAM_OP_FREE
Synchronization HaStreamSynchronize, HaEventSynchronize, HaDeviceSynchronize A single entry point, ha_drv_wait, waits on streams, events, and devices
Device reset HaResetDevice, HaResetAllDevices ha_drv_reset (keeps the device handle, invalidates child objects)
Languages C++, Python C++ (C linkage)

There is one default stream for each (thread, device) pair, and the two APIs share the default stream on the same thread and device.

Note: Device numbers do not match

The index passed to ha_drv_open is a hardware device index, whereas a runtime API device ID is a virtual ID. If any device fails to initialize, the two numbering schemes diverge, so the same number is not guaranteed to refer to the same device.

Do not mix the runtime API and the driver API on the same device

The two APIs share the same device state, but the information the runtime API manages separately (the global address table and stream/event bookkeeping) does not track driver API calls. Mixing the two APIs on the same device causes the following problems.

  • Memory management conflicts: If memory allocated with one API is freed or copied with the other, the runtime does not recognize ownership of the address, or a freed address remains in the runtime's bookkeeping.
  • Incompatible events: Driver events (HaDrvEvent) and runtime events (HaEvent_t) are separate objects and cannot wait on each other.
  • Unexpected handle invalidation: ha_drv_reset also invalidates streams, events, and memory allocations created by the runtime API, but the runtime is not aware of this.

Recommendation: If a process needs both APIs, dedicate each device to only one of them.

1.3 Key Concepts

Key concept Description
Device handle (HaDrvDev) A reference to one bootstrapped device. Obtained with ha_drv_open and released with ha_drv_close (see Device Handle).
Default stream (HA_STREAM_DEFAULT) One stream per (calling thread, device). Created automatically on first use (see Stream).
Stream operation (HaDrvStreamOp) An operation enqueued on a stream with ha_drv_stream_post. Includes copies, deferred frees, file loads, event records, and more (see Stream).
Sequence number (seq) A number assigned to each operation enqueued on a stream. Used to tell which operations a synchronization covers (see Synchronization).
Kernel handle (HaDrvKernel) A kernel loaded onto a device with ha_drv_kernel_load_from_file or ha_drv_kernel_load_from_library. Pass this handle as an argument when launching a kernel (see Kernel Loading and Execution).
Stream error (Sticky error) The error from an operation that failed during execution. It stays on the stream and is returned by every later wait and enqueue (see Stream Errors).
Device reset An operation that reinitializes a device and invalidates all of its child handles at once (see Device Reset).

1.4 Device Handle

HaDrvDev is how every driver API call specifies its target device. Whereas the runtime API implicitly uses the "current device" selected per thread, every driver API call takes this handle explicitly.

What a Handle Is

HaDrvDev is a reference to one already bootstrapped device. It is not the device itself, so opening or closing a handle does not create or destroy a device. Devices are created and destroyed by the per-process ha_drv_init / ha_drv_deinit; a handle only takes and drops a reference in between (Device Management).

  • Open and close: ha_drv_open(index, &dev) takes a reference and ha_drv_close(dev) drops it. Each successful open needs exactly one close, and a closed handle must not be used again.
  • Shutdown condition: ha_drv_deinit releases devices only when no handle is open. If any remain, it returns HaDeviceBusy and releases nothing.
  • Index: The index of ha_drv_open is a hardware device index. To get the index back from a handle, use ha_drv_get_device_id(dev). It is a different number from the runtime API's virtual device ID (Differences from the Runtime API).

Multiple Handles

Opening the same index multiple times creates an additional handle to the same device each time. These handles share the device state. Memory allocated through one handle can be used by operations enqueued through another, and HA_STREAM_DEFAULT refers to the same default stream for both handles, so operations enqueued through both handles share a single sequence (seq). Each handle must be closed separately.

Device handle lifetime and two handles to the same device

Device handle lifetime and two handles sharing the same device state

Threads

A handle is not bound to a particular thread. Multiple threads can use the same handle concurrently. However, passing HA_STREAM_DEFAULT as the stream argument means the default stream of the calling thread, so each thread uses a different default stream even with the same handle (Stream).

Owned Objects

Streams, events, kernels, and memory allocations belong to the device that created them. A child object passed to a function must belong to the same device as the handle passed with it; passing an object from another device is rejected.

Object passed by mistake Returns
Stream from another device HaStreamInvalidHandle
Event from another device HaEventInvalidHandle
Kernel loaded on another device HaInvalidArgument

Reset

The device handle remains valid after ha_drv_reset(dev). Only the device's child objects (streams, events, allocations, kernels) are invalidated, so recreate them with the same handle (Device Reset).

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
HaDrvDev a = nullptr, b = nullptr;
ha_drv_open(0, &a);
ha_drv_open(0, &b);                     // Second handle to the same device 0

HaDevicePtr buf = 0;
ha_drv_malloc(a, &buf, n, /*is_interleaved=*/false, HA_STREAM_DEFAULT);

// Memory allocated through a is used by a copy enqueued through b. Both handles refer to this thread's same default stream
HaDrvStreamOp h2d{HA_STREAM_OP_MEMCPY, {}};
h2d.memcpy = {buf, reinterpret_cast<HaDevicePtr>(src), n, HaMemCpyDirection::HostToDevice};
uint64_t seq_a = 0, seq_b = 0;
ha_drv_stream_post(a, HA_STREAM_DEFAULT, &h2d, &seq_a);
ha_drv_stream_post(b, HA_STREAM_DEFAULT, &h2d, &seq_b);   // seq_b == seq_a + 1 (same stream, one sequence)
ha_drv_wait(a, HA_STREAM_DEFAULT, nullptr, HA_WAIT_STREAM_SYNC, -1, nullptr);
ha_drv_free(a, buf, HA_STREAM_DEFAULT);

int index = ha_drv_get_device_id(a);    // 0

ha_drv_close(b);
ha_drv_close(a);                        // One close per open
API Sync / Async Behavior
ha_drv_open() Sync Obtains a handle to a bootstrapped device
ha_drv_close() Sync Releases the handle. The device stays bootstrapped
ha_drv_get_device_id() Sync Queries the hardware device index the handle refers to
ha_drv_is_ready() Sync Checks that the device has not been released and the handle is valid. Does not reflect whether a reset is in progress
ha_drv_get_attr() Sync Reads a single device attribute

1.5 Basic Usage Example

This example opens device 0, allocates memory, copies data, launches a kernel, and reads back the result. The copies and the launch are all enqueued in order on the calling thread's default stream, and a single ha_drv_wait at the end confirms that all of them completed.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
#include "ha_driver_api.h"

int main() {
  if (ha_drv_init() != HaError::HaSuccess) return 1;   // Bootstrap all devices

  HaDrvDev dev = nullptr;
  ha_drv_open(0, &dev);                                // Obtain a handle to device 0

  float in[1024] = {}, out[1024] = {};
  const size_t n = sizeof(in);
  HaDevicePtr da = 0;
  ha_drv_malloc(dev, &da, n, /*is_interleaved=*/false, HA_STREAM_DEFAULT);

  ha_drv_memcpy(dev, da, reinterpret_cast<HaDevicePtr>(in), n, HaMemCpyDirection::HostToDevice,
                HA_STREAM_DEFAULT);

  HaDrvKernel k = nullptr;
  ha_drv_kernel_load_from_file(dev, "kernels/scale.elf", &k);
  uint32_t dims[] = {1024};
  HaLaunchArgs args;                                   // Released when it goes out of scope
  ha_drv_launch_args_add_tensor(dev, args, HaLaunchArgRoute::Universal, -1, da, dims, 1, 0);
  ha_drv_kernel_launch(dev, HA_STREAM_DEFAULT, k, args);   // The list is already consumed when the launch call returns

  ha_drv_memcpy(dev, reinterpret_cast<HaDevicePtr>(out), da, n, HaMemCpyDirection::DeviceToHost,
                HA_STREAM_DEFAULT);

  // Do not release the in/out host buffers before this wait completes.
  ha_drv_wait(dev, HA_STREAM_DEFAULT, nullptr, HA_WAIT_STREAM_SYNC, -1, nullptr);

  ha_drv_kernel_release(dev, k);
  ha_drv_free(dev, da, HA_STREAM_DEFAULT);
  ha_drv_close(dev);
  ha_drv_deinit();                                     // Release everything after all handles are closed
  return 0;
}

Next Steps