C++ API¶
The C++ reference is written from the Doxygen comments in the public headers.
1 2 | |
Link the targets of the CMake package hart. The library file name depends on the build variant, so do not
specify it directly.
1 2 | |
The reference body is in English
The body of the pages below follows the header comments, so it is provided mostly in English, also on the Korean site: a separate translation would drift every time a header changes. For concepts and usage, see the Programming Guide.
Sections¶
Runtime API (Ha*)¶
| Page | Contents |
|---|---|
| Device | Device discovery, selection, attribute queries, reset |
| Memory | Allocation, transfers, tensor layout, device memory statistics, host memory |
| Execution | Kernel library loading, kernel launch and the argument list (HaLaunchArgs) |
| Streams | Ordered queues of asynchronous work |
| Events | Stream markers and cross-stream ordering |
| TLB | Static KV-cache TLB initialization and page management, TLB configuration and invalidation |
| VMM | Virtual memory for the Bertha DC EVT0 KV cache: pool allocation, address reservation, per-request page mapping |
| Sampling | Token sampling parameters |
| Types | Enums, type aliases, structs |
| Callback Params | Argument structs for API call instrumentation |
Weight Converter API¶
| Page | Contents |
|---|---|
| Weight Converter | Converts weight tensors to the device memory layout |
Driver API (ha_drv_*)¶
A device-handle-based API: every function that operates on a device takes a device handle. The common rules are in the Overview.
| Page | Contents |
|---|---|
| Device | Device bootstrap, opening and closing handles, attribute queries, reset |
| Memory | Device memory allocation and release, 1-D/N-D copies, device memory statistics, host memory |
| Execution | Kernel libraries, kernel load, launch, and release |
| Streams & Events | Stream and event creation and destruction, enqueueing work, host waits |
| TLB | Static KV-cache TLB and page-table TLB configuration |
| Types | Enums, handle types, structs |
Headers and CMake targets¶
The public headers to include and the CMake targets to link depend on the purpose. An application that uses the runtime API
only needs to link hart::ha_hart; the driver API library is linked along with it.
Relationship between the public headers and the CMake targets
| Header | Purpose | CMake target |
|---|---|---|
ha_runtime_api.h |
Runtime API (Ha*) |
hart::ha_hart |
ha_driver_api.h |
Driver API (ha_drv_*) |
hart::ha_umd_core (included in hart::ha_hart) |
ha_driver_types.h |
Driver API types (HaDrv*) and the kernel argument list HaLaunchArgs. Included by ha_driver_api.h and ha_runtime_api.h |
— |
ha_types.h |
Common types, TLB structs, kernel library format. Included by the two headers above | — |
ha_error_code.h |
Error codes (HaError). Included by ha_types.h |
— |
ha_error.h |
lpu::runtime::HaErrorName() and HaErrorToString(), which turn a HaError into a string. See Error Codes |
hart::ha_hart |
ha_callback_params.h |
Argument structs for API call instrumentation | hart::ha_hart |
weight_converter/ha_weight_converter.h, weight_converter/ha_weight_converter_types.h |
Weight Converter API | hart::ha_weight_converter |
ha_vmm.h |
KV-cache virtual memory (HaVmm*). Not included by ha_runtime_api.h, so include it directly |
hart::ha_hart |
How to read this reference¶
Two rules apply to every entry point and are not repeated on the individual pages.
outparameters are not written if the call fails. Values markedoutin a parameter table are valid only afterHaSuccess.- A
HaStream_tparameter with a default value means the per-thread default stream.HaStream_t stream = HA_STREAM_DEFAULTin a signature is not "no stream" but the calling thread's default stream.
Topics that span multiple entry points, such as asynchronous behavior, memory ownership, and error classification, are not repeated here; they are described once in the Programming Guide.
Scope¶
Only the headers under runtime/shared/public/ and the Weight Converter public headers (installed under include/weight_converter/)
are supported API. Other headers in the source tree are internal and carry no compatibility guarantee, so do not include them.
The Driver API (ha_driver_api.h) is documented separately for integrators who call with a device handle
specified directly.