Types¶
Enumerations, type aliases, and structures declared by the public headers.
Contents¶
| 이름 | 분류 | 설명 |
|---|---|---|
HaDataType |
Enum | HaLayout의 요소 타입 정의 |
HaDeviceAttr |
Enum | HaGetDeviceAttribute()로 읽는 디바이스 속성 |
HaDeviceVersion |
Enum | 디바이스 칩 세대 식별자 (HaDeviceAttr::Version의 값) |
HaError |
Enum | 런타임 API가 반환하는 공통 에러 코드 |
HaLaunchArgRoute |
Enum | Legato 커널 인자를 기록할 인자 블록 지정 |
HaMemCpyDirection |
Enum | HaMemCpy()의 데이터 복사 방향 지정 |
HaMemScope |
Enum | HaMemCpyNd() 포인터 인자가 가리키는 주소 공간 지정 |
HaTensorKind |
Enum | HaBuildDeviceTensorLayout()의 텐서 종류 판별자 |
PageType |
Enum | KV 캐시의 KEY·VALUE 페이지 구분 |
ValuePageMode |
Enum | VALUE 페이지 배치 방식 |
HaDevicePtr |
Type alias | 64비트 디바이스 주소 타입 |
HaEvent_t |
Type alias | 이벤트 공개 핸들 타입 |
HaKernel_t |
Type alias | 커널 라이브러리 안 커널의 공개 핸들 타입 |
HaLibrary_t |
Type alias | 커널 라이브러리 공개 핸들 타입 |
HaStream_t |
Type alias | 스트림 공개 핸들 타입 |
HaDeviceMemStats |
Struct | 현재 디바이스 메모리 통계 (HaDeviceMemGetStats()) |
HaHostMemStats |
Struct | 프로세스 호스트 메모리 통계 (HaHostMemGetStats()) |
HaLaunchArgs |
Class | 커널 launch 인자 목록 (소유권을 갖는 이동 전용 객체) |
HaLayout |
Struct | HaMemCpyNd()용 오퍼랜드별 N차원(N-D) Shape 기술자 |
HaLibraryData |
Struct | 커널 라이브러리의 메모리 내 형식 (HaLibraryLoadData()에 전달) |
HaShape |
Struct | 3D 논리 Shape (HaMemAlloc3D() 등에서 사용) |
HaVmmBlockSegment |
Struct | 풀의 연속 구간: pa_start, block_count, raw_ptr, raw_size |
HaVmmPool |
Struct | 세그먼트 배열과 개수 (HaVmmAllocatePool()에서 입력: 용량, 출력: 개수) |
TLBInitInfo |
Struct | HaInitTlb()의 정적 KV 캐시 TLB 설정 |
TLBAllocInfo |
Struct | HaAllocTlbPage()의 페이지 할당 요청 |
TLBDeallocInfo |
Struct | HaDeallocTlbPage()의 페이지 해제 요청 |
HaVmmBlockId |
Struct | 풀 번호와 풀 안의 블록 번호 (HaVmmMapBlocksBatchFromPools()) |
HA_DEVICE_PTR_NULL 외 |
Constant | HA_DEVICE_PTR_NULL, HA_STREAM_DEFAULT, HA_MAX_NDIM, HA_LAYOUT_INIT, HA_LIBRARY_DATA_*, kHa* 별칭 |
HaDataTypeBits |
Helper | 데이터 타입(dtype)의 요소당 비트 수 계산 |
HaDataTypeBytes |
Helper | 데이터 타입(dtype)의 요소당 바이트 크기 계산 |
Enumerations¶
HaDataType¶
Element type of a HaLayout — unit of shape[] / strides[].
| Value | Description |
|---|---|
Bytes |
Raw bytes (1 byte per element). |
Int8 |
Signed 8-bit integer. |
Int16 |
Signed 16-bit integer. |
Int32 |
Signed 32-bit integer. |
Int64 |
Signed 64-bit integer. |
Uint8 |
Unsigned 8-bit integer. |
Uint16 |
Unsigned 16-bit integer. |
Uint32 |
Unsigned 32-bit integer. |
Uint64 |
Unsigned 64-bit integer. |
Float16 |
IEEE 754 binary16 (half-precision float). |
Float32 |
IEEE 754 binary32 (single-precision float). |
BFloat16 |
bfloat16. |
Float8 |
8-bit float (1 byte). |
Float4 |
4-bit float (sub-byte: 0.5 byte). Two Float4 elements pack into one byte. |
Float64 |
IEEE 754 binary64 (double-precision float). Appended last so the values above keep their numbers. |
Note
Raw byte transfers use Bytes as an escape hatch. Values mirror the DLPack dtype family in intent but are runtime-local.
HaDeviceAttr¶
Scalar device attributes readable through HaGetDeviceAttribute(). Declared as enum class HaDeviceAttr : int; the values are part of the ABI, so new values are only appended and existing values are never renumbered.
| Value | Number | Description |
|---|---|---|
PciDomain |
0 | PCI domain (segment). |
PciBus |
1 | PCI bus number. |
PciSlot |
2 | PCI device number. |
PciFunc |
3 | PCI function number. |
PeerPresentMask |
4 | Peers the driver has registered; bit i = device i, own bit always 0. |
PeerMappedMask |
5 | Peers whose User BAR is mapped into this device; bit i = device i. Necessary but not sufficient for P2P: a set bit means an IOVA exists, not that the PCIe fabric routes the transfer peer-to-peer. |
DramChannels |
6 | Number of DRAM channels. |
DramChannelStride |
7 | Byte distance between consecutive DRAM channels' address ranges. |
Version |
10 | Chip generation, a HaDeviceVersion value. Packed weight layouts and ABI selection are chip-specific and the chip cannot be recovered from any buffer contents, so callers that need it must ask the runtime. |
HaDeviceVersion¶
Chip generation of a device. Read it with HaGetDeviceAttribute() and HaDeviceAttr::Version, then cast the value to this enum.
1 | |
| Value | Number | Description |
|---|---|---|
BerthaCloudEVT0 |
0 | Bertha Cloud EVT0. |
BerthaCloudEVT1 |
1 | Bertha Cloud EVT1. |
BerthaEdgeEVT0 |
2 | Bertha Edge EVT0. |
Note
Append only: the values are part of the ABI. Use this to select hardware revisions/targets or branch on supported features.
HaError¶
Common error codes returned by the runtime API.
Declared in ha_error_code.h, which ha_types.h includes.
| Value | Description |
|---|---|
HaSuccess |
Success. |
HaInvalidArgument |
Invalid non-device argument (range/null pointer, etc.). |
HaNotSupported |
Unsupported API, option, or execution path. |
HaTimeout |
Operation timed out. |
HaInternalError |
Internal error (unexpected state/bug). |
HaUnknownError |
Unknown error (unclassified failure). |
HaDeviceError |
— |
HaDeviceNotAvailable |
Device is unavailable or does not exist. |
HaDeviceNotInitialized |
Device is not initialized (e.g., HaSetDevice() not called). |
HaDeviceNotReady |
Device is not ready (initialization/reset/load in progress, etc.). |
HaDeviceInvalidArgument |
Invalid device-related argument (range/null pointer, etc.). |
HaDeviceAlreadyInitialized |
Device is already initialized. |
HaDeviceOpenFailed |
Failed to open the device. |
HaDeviceCloseFailed |
Failed to close the device. |
HaDeviceResetFailed |
Failed to reset the device. |
HaDeviceUnsupported |
Unsupported device/version/feature. |
HaDeviceRegisterSetFailed |
Failed to program device registers (e.g., SFR). |
HaDeviceInvalidValue |
The device returned an invalid value. |
HaDeviceBusy |
Device is still in use (e.g., ha_drv_deinit while a handle is open). |
HaMemoryError |
— |
HaMemoryInvalidAddress |
Invalid memory address (host/device pointer mismatch, etc.). |
HaMemoryAlignmentError |
Alignment constraint violation. |
HaMemoryInvalidArgument |
Invalid memory-related argument (size/null pointer, etc.). |
HaMemoryAllocationFailed |
Allocation failed (out-of-memory/constraints, etc.). |
HaMemoryFreeFailed |
Failed to free memory. |
HaMemoryCopyFailed |
Failed to copy memory. |
HaMemCpyNotSupported |
Memcpy operation not supported (e.g., cross-device D2D). |
HaHostMemLimitExceeded |
Allocation rejected by a configured limit (HA_HOST_MEM_MAX_BYTES, HA_MEMCPYND_STAGING_MAX_BYTES), not by the system. |
HaKernelError |
— |
HaKernelInvalidArgument |
Invalid kernel-related argument (arg count/pointer, etc.). |
HaKernelUnsupported |
Unsupported kernel or execution path. |
HaKernelExecutionFailed |
Kernel execution failed (including runtime/device errors). |
HaKernelNotFound |
Kernel not found (registration/binary load failure, etc.). |
HaKernelInvalidHandle |
Kernel handle released, invalidated by a device reset, or never loaded. |
HaKernelNoBinaryForDevice |
The library has no binary of the kernel for the device. |
HaKernelLoadFailed |
The device could not load the kernel (a failure other than running out of kernel memory). |
HaStreamError |
— |
HaStreamInvalidHandle |
Invalid stream handle. |
HaStreamInvalidArgument |
Invalid stream arguments. |
HaStreamCreationFailed |
Failed to create stream. |
HaStreamDestroyFailed |
Failed to destroy stream. |
HaStreamSyncFailed |
Stream synchronization failed. |
HaStreamNotReady |
Returned by HaStreamQuery() when work is still in progress. |
HaEventError |
— |
HaEventInvalidHandle |
Invalid event handle. |
HaEventInvalidArgument |
Invalid event arguments. |
HaEventCreationFailed |
Failed to create event. |
HaEventDestroyFailed |
Failed to destroy event. |
HaEventNotRecorded |
HaEventRecord() has not been called on this event. |
HaEventNotReady |
Returned by HaEventQuery() when the event has not yet completed. |
Note
Use these codes to classify failures (device/memory/kernel) and implement user-level error handling, retry, and logging.
Value encoding:
- Success: 0
- Common:
code(1..65535) - Device:
(1 << 16) \| code - Memory:
(2 << 16) \| code - Kernel:
(3 << 16) \| code - Stream:
(4 << 16) \| code - Event:
(5 << 16) \| code
Codes, causes and remedies are listed in Error Codes.
PageType¶
Distinguishes KEY pages and VALUE pages in the KV cache. Used by TLBAllocInfo and TLBDeallocInfo.
| Value | Description |
|---|---|
KEY |
Key page. |
VALUE |
Value page. |
ValuePageMode¶
Placement policy for value pages. Used by TLBInitInfo.
| Value | Description |
|---|---|
CONTIGUOUS |
Value pages allocated sequentially in memory. |
STRIDE |
Value pages allocated with fixed stride (for interleave). |
HaLaunchArgRoute¶
Which executor's argument block a Legato kernel argument is written to. A Legato kernel's ABI holds one argument block for the top executor and one per core, and each executor reads only its own. The route must be the one the kernel was compiled with: a different route puts the argument in a block the kernel does not read it from.
| Value | Description |
|---|---|
Universal |
The top block and every core block. |
Top |
The top block only. |
Core |
The block of core target_core only. A core that needs the argument needs its own. |
HaMemCpyDirection¶
Memory copy direction for HaMemCpy().
| Value | Description |
|---|---|
HostToDevice |
Host → Device. |
DeviceToHost |
Device → Host. |
DeviceToDevice |
Device → Device. |
Note
Select the appropriate direction based on where src and dst reside (host vs. device).
HaMemScope¶
Which address space the matching HaMemCpyNd() pointer argument refers to.
| Value | Description |
|---|---|
Host |
Host memory (process-visible pointer). |
Device |
Device pointer (resolved by the per-device memory agent). |
Note
The runtime uses (dst.scope, src.scope) to decide H2D / D2H / D2D dispatch for HaMemCpyNd(). HaMemCpyNd() performs H2H copies on the host. HaMemCpyNdAsync() rejects H2H because there is no host-to-host stream operation to order it against.
HaTensorKind¶
Tensor-kind discriminator for HaBuildDeviceTensorLayout().
| Value | Description |
|---|---|
KvCacheKey |
KV-cache K tensor; canonical 10-D storage layout. |
KvCacheValue |
KV-cache V tensor. Same logical dims as KvCacheKey; differs in dim-4 ↔ dim-7 stride swap. |
SingleChannel |
Whole matrix on a single channel; chunk = H × W × element_size. |
RowInterleaved |
Row-interleaved across channels; chunk = W × element_size (one row). |
MaxRowsInterleaved |
Multi-row-interleaved; chunk = max_reuse_num × cores_per_cg × W × element_size (HW MAC-reuse window, multiple rows per chunk). |
DramWordInterleaved |
DRAM-word-interleaved (generic primitive); chunk = dram_word_size_bytes (64 B on BerthaCloud, 32 B on BerthaEdge). |
MPUTensor |
Inner-6-D Value-pattern layout (block_size = ∞ limit of KvCacheValue). |
MpuTensorTransposed |
Inner-6-D Key-pattern layout (block_size = ∞ limit of KvCacheKey). The hardware reads it as the transposed view of the data, for transposed-weight / right-hand-side matmul operands. |
SinglePassInterleaved |
Single-pass interleave: each channel gets one contiguous slab of total/N; no cycling (the chunk is sized exactly to consume the tensor in one pass). |
NumOptions |
Sentinel (count of kinds), not a kind. Never pass it to HaBuildDeviceTensorLayout(). |
Each kind fixes the meaning, length, and order of the dims array passed to HaBuildDeviceTensorLayout (see that function's docs):
KvCacheKey/KvCacheValue:logical dims describe[num_blocks, num_layers, num_kv_heads, block_size, head_dim].SingleChannel/RowInterleaved/MaxRowsInterleaved:2 dims =[H, W]. All three interleave the allocation across all DRAM channels and follow the unified padding rulepadding = data_size mod (#channels × chunk_size).MPUTensor:2 dims =[H, W]. Inner-6-D of the Value-pattern layout — theblock_size = ∞limit ofKvCacheValuewith batching dims (num_blocks, num_layers, num_kv_heads, blocks_per_stripe) all collapsed to 1. Shares the storage formula withKvCacheValue.MpuTensorTransposed:2 dims =[H, W]. Inner-6-D of the Key-pattern layout — theblock_size = ∞limit ofKvCacheKeywith batching dims (num_blocks, num_layers, num_kv_heads, blocks_per_stripe) all collapsed to 1. Shares the storage formula withKvCacheKey; differs fromMPUTensoronly in the dim-0 ↔ dim-3 stride swap of the 6-D core (the Key/Value variant of the inner layout). Use this for a transposed weight / right-hand-side matmul operand that needs the Key wire pattern instead of the Value wire pattern.DramWordInterleaved:any rankndim >= 1. Storage formula only consumestotal = product(dims); caller may pass their natural contiguous logical shape (e.g.[B, S, D],[H, W], or[total]) without flattening. Storage is rank-3(chunks_per_channel, num_dram_channels, elements_per_word)with strides(elements_per_word, channel_stride_elements, 1).SinglePassInterleaved:any rankndim >= 1(dtype units;HaDataType::Bytesfor raw byte buffers). Single-pass interleave — each channel gets one contiguous slab oftotal/N; chunk is sized to consume the tensor in one pass. Storage formula only consumestotal = product(dims); caller may pass their natural contiguous logical shape without flattening. Storage is rank-2(num_dram_channels, total / num_dram_channels)with strides(channel_stride_elements, 1).
Kind contract:
- Layout is
(kind, dtype, dims, chip params) -> HaLayout, not kind alone; chip params come from the current device (8 ch / 64 B word on BerthaCloud, 2 / 32 B on BerthaEdge). - Host-side selector only:
HaBuildDeviceTensorLayout()expands it intoHaLayoutshape/strides, which carry no kind field. - Ordinals are a wire format between components, so treat the numbering as append-only.
Type aliases¶
HaDevicePtr¶
64-bit device address type (analogous to CUdeviceptr).
1 | |
Note
Primary role: holds device-side addresses for memory allocation / free / DA-to-device routing (HaMemAlloc, HaMemFree, VMM internals). CUDA uses CUdeviceptr = unsigned long long (integer, not void*); the driver API (cuMemMap, cuMemAddressReserve, cuMemcpyHtoD) uses CUdeviceptr while the runtime API (cudaMemcpy) uses void*. They're interoperable via cast — HaDevicePtr matches the driver-API convention.
Note
Secondary role (copy APIs): for parameters that may carry either a host pointer or a device address — disambiguated at runtime by the accompanying HaMemCpyDirection — HaDevicePtr is reused as the integer carrier. Numerically compatible with both void * and host integer addresses on 64-bit targets (sizeof(uintptr_t) == sizeof(void *) == 8).
Note
Conversion rules: Host pointer → HaDevicePtr : reinterpret_cast<HaDevicePtr>(host_ptr) Device address → HaDevicePtr : value-preserving (no cast needed) HaDevicePtr → host pointer : reinterpret_cast<void *>(addr) (H2D: src; D2H: dst) HaDevicePtr → device address : value-preserving (no cast needed)
HaEvent_t¶
Public handle type for an event.
1 | |
HaKernel_t¶
Public handle type for a kernel of a library, from HaLibraryGetKernel(). Valid until the process ends, across device resets.
1 | |
Note
A handle is an id the runtime looks up, not the address of an object: HaKernel is only forward-declared, so a handle cannot be dereferenced.
HaLibrary_t¶
Public handle type for a library of precompiled kernels, from HaLibraryLoadData(). Valid until the process ends.
1 | |
HaStream_t¶
Public handle type for a stream. Pass nullptr (HA_STREAM_DEFAULT) to use the calling thread's default stream on the current device.
1 | |
Note
Default streams follow CUDA per-thread-default-stream (PTDS) semantics: one backing stream per (current device, calling thread), created lazily on first use. Two threads that pass HA_STREAM_DEFAULT on the same device therefore get different streams and are NOT ordered against each other. To order work across threads, create an explicit stream and share it, or sequence them with events.
Structures¶
HaDeviceMemStats¶
Device memory statistics of the current device, filled by HaDeviceMemGetStats() (and the driver API's ha_drv_device_mem_get_stats()). Extensible like HaHostMemStats: set struct_size to sizeof(HaDeviceMemStats) before the call; the library fills every field that fits entirely and stores the number of bytes written. Each counter is read atomically, but the counters together are not one consistent snapshot.
Byte figures are the sizes occupied in all DRAM channels of the device, summed, so a contiguous allocation is counted as its requested size multiplied by the number of channels. The exception is requested_bytes and its peak, which are the sizes the callers asked for and are not multiplied by the channel count. free_bytes is the driver's own figure for the whole device (every process, memory the runtime's pools do not manage included); the other fields cover the runtime's pools only.
| Field | Type | Description |
|---|---|---|
struct_size |
uint32_t |
In: sizeof as compiled. Out: bytes written. |
reserved0 |
uint32_t |
Written as 0. |
total_bytes |
uint64_t |
DRAM capacity the device reports. |
free_bytes |
uint64_t |
Free DRAM of the device as the driver reports it at the call. |
releasable_bytes |
uint64_t |
Bytes of segments with no allocated block, which a trim returns to the driver. |
allocated_bytes |
uint64_t |
Bytes of the blocks currently handed out, rounded as the pool slots them. |
peak_allocated_bytes |
uint64_t |
Highest allocated_bytes since the last peak reset. |
reserved_bytes |
uint64_t |
Bytes of the segments the pools hold, summed over both runtime pools. |
peak_reserved_bytes |
uint64_t |
Highest reserved_bytes since the last peak reset. |
requested_bytes |
uint64_t |
Sizes the callers asked for, over both runtime pools; not scaled by channels. |
peak_requested_bytes |
uint64_t |
Highest requested_bytes since the last peak reset. |
cached_free_bytes |
uint64_t |
Freed bytes kept in the pools for reuse (reserved_bytes − allocated_bytes), summed over both runtime pools. |
active_blocks |
uint64_t |
Number of blocks currently handed out, summed over both runtime pools. |
peak_active_blocks |
uint64_t |
Highest active_blocks since the last peak reset. |
segments |
uint64_t |
Number of segments the pools hold, summed over both runtime pools. |
peak_segments |
uint64_t |
Highest segments since the last peak reset. |
device_allocs |
uint64_t |
Number of segments obtained from the driver so far, summed over both runtime pools. |
device_frees |
uint64_t |
Number of segments returned to the driver so far, summed over both runtime pools. |
HaHostMemStats¶
Host memory statistics of the whole process, filled by HaHostMemGetStats() (and the driver API's ha_drv_host_mem_get_stats()). Extensible: set struct_size to sizeof(HaHostMemStats) before the call; the library fills at most that many bytes and stores the count it wrote. Each counter is read atomically; the set is not one snapshot.
| Field | Type | Description |
|---|---|---|
struct_size |
uint32_t |
In: sizeof as compiled. Out: bytes written. |
reserved0 |
uint32_t |
Written as 0. |
allocated_bytes |
uint64_t |
Bytes currently allocated (as requested, not rounded). |
peak_allocated_bytes |
uint64_t |
Highest allocated_bytes since the last peak reset. |
reserved_bytes |
uint64_t |
Bytes obtained from the OS, including cached freed memory. |
peak_reserved_bytes |
uint64_t |
Highest reserved_bytes since the last peak reset. |
alloc_count |
uint64_t |
Allocations ever made. |
free_count |
uint64_t |
Allocations ever freed. |
HaLaunchArgs¶
The arguments of one kernel launch, in the order they were appended. Owns its records. Defined in ha_driver_types.h, which ha_runtime_api.h and ha_driver_api.h include; ha_types.h holds only a forward declaration.
Construct one to get an empty list, fill it with HaLaunchArgsAdd*() (or the driver API's ha_drv_launch_args_add_*()), and pass it to HaLaunchKernel() / HaLaunchKernelAsync(). The records are released when the object is destroyed. The record layout belongs to the kernel ABI and is not published. See Execution API.
| Member | Description |
|---|---|
HaLaunchArgs() |
Creates an empty argument list. Allocates, and throws std::bad_alloc when memory is short. |
~HaLaunchArgs() |
Releases the records. |
HaLaunchArgs(HaLaunchArgs &&other) noexcept |
Takes the records of other, which is left moved from. |
HaLaunchArgs &operator=(HaLaunchArgs &&other) noexcept |
Releases the records and takes those of other, which is left moved from. |
The class is move-only (copy construction and copy assignment are deleted). A moved-from list is rejected by every function with HaKernelInvalidArgument. Threads may share one list: appends and updates take its lock exclusively and a launch reads under it shared. Moving a list is not synchronized, so do not move or destroy a list while another thread uses it.
HaLayout¶
Per-operand N-D shape descriptor for HaMemCpyNd(): shape, strides, dtype, scope.
| Field | Type | Description |
|---|---|---|
shape |
uint64_t[HA_MAX_NDIM] |
Dimension sizes (elements of dtype). Only the first ndim entries are valid. |
strides |
uint64_t[HA_MAX_NDIM] |
Per-dimension strides (elements of dtype). Only the first ndim entries are valid. |
scope |
HaMemScope |
Address space the matching HaMemCpyNd() pointer argument refers to. |
dtype |
HaDataType |
Element type — unit of shape and strides. |
ndim |
uint32_t |
Number of valid entries in shape / strides; must satisfy 1 <= ndim <= HA_MAX_NDIM. |
Note
A pure shape spec — no buffer pointer. The matching HaDevicePtr is passed alongside the layout to HaMemCpyNd(), so a single layout (e.g. a KV-cache block layout) can describe many different buffers without rebuilding.
Note
The hot arrays (shape, strides) are placed first and the struct is 64 B aligned. At HA_MAX_NDIM 25 each array is 200 B. Start from HA_LAYOUT_INIT to zero-initialize a layout.
Note
shape / strides are expressed in elements of dtype, not bytes. strides[i] is the element distance between consecutive entries along dimension i. ndim counts how many entries of each array are valid; entries past ndim are ignored.
HaLibraryData¶
Root of a kernel library: what HaLibraryLoadData() is given.
| Field | Type | Description |
|---|---|---|
magic |
uint32_t |
HA_LIBRARY_DATA_MAGIC (0x424C4148u, "HALB" read as a little-endian uint32_t). |
version |
uint32_t |
HA_LIBRARY_DATA_VERSION (1u), bumped on any layout change. |
set_count |
uint32_t |
Number of entries in sets. |
sets |
const HaLibraryBinarySet * |
One set per device version. |
A HaLibraryData holds one HaLibraryBinarySet per device version, and each set lists the kernels built for that version as HaLibraryKernelBinary entries (name and Legato ELF).
HaLibraryBinarySet:
| Field | Type | Description |
|---|---|---|
device_version |
int32_t |
HaDeviceVersion value; unique within a HaLibraryData. |
binary_count |
uint32_t |
Number of entries in binaries. |
binaries |
const HaLibraryKernelBinary * |
The set's binaries. |
HaLibraryKernelBinary:
| Field | Type | Description |
|---|---|---|
name |
const char * |
NUL-terminated kernel name, unique within its set. |
elf |
const void * |
Legato ELF bytes. |
elf_size |
uint64_t |
Number of bytes at elf. |
Note
The device version of a set is stated once by the build input as a HaDeviceVersion value. A HaLibraryData of a version this runtime does not read is rejected. The structs are read only during HaLibraryLoadData(), but the kernel names and ELF bytes they point to are not copied and must stay valid for the life of the process.
HaShape¶
3D logical shape (e.g., for HaMemAlloc3D()); used to describe tensor-like buffers.
| Field | Type | Description |
|---|---|---|
w |
size_t |
Width (contiguous dimension). |
h |
size_t |
Height (split by channel/block). |
c |
size_t |
Channel. |
Concrete layout/packing rules are defined by the runtime/device.
Note
W = width (contiguous dimension), H = height (split by channel/block), C = channel.
HaVmmBlockSegment¶
One contiguous run of block slots: a start and a size.
| Field | Type | Description |
|---|---|---|
pa_start |
HaDevicePtr |
Physical base address of this segment. |
block_count |
int64_t |
Blocks the segment spans from pa_start. |
raw_ptr |
HaDevicePtr |
HaMemAlloc base this segment was carved from (HaMemFree target). |
raw_size |
uint64_t |
Full byte size of that allocation, alignment slack included. |
Note
A Bertha DC EVT0 pool is one of these. pa_start is raw_ptr rounded up to the requested alignment and is the base HaVmmInitBlocks() registers with the TLB; free raw_ptr, never pa_start. Declared in ha_types.h.
TLBInitInfo¶
Static configuration for TLB initialization at model load time, passed to HaInitTlb(). Has an equality operator that compares every field.
| Field | Type | Description |
|---|---|---|
page_size |
int |
Page size in bytes. |
base_addr |
size_t |
Base address of TLB. |
layer_num |
int |
Number of LLM layers. |
head_num |
int |
Number of LLM heads. |
max_page_num |
int |
Maximum pages the TLB can map. |
value_page_mode |
ValuePageMode |
Value page mode (CONTIGUOUS or STRIDE). |
value_stride |
int |
Stride between value pages when STRIDE mode. |
value_line_page_num |
int |
Pages per value line. |
log_value_stride |
int |
log2 of value_stride. |
TLBAllocInfo¶
Page allocation request, passed to HaAllocTlbPage(). Has an equality operator that compares every field.
| Field | Type | Description |
|---|---|---|
req_id |
int |
Request identifier. |
page_num |
int |
Number of pages to allocate. |
page_type |
PageType |
Page category (KEY / VALUE). |
TLBDeallocInfo¶
Page deallocation request, passed to HaDeallocTlbPage(). Has an equality operator that compares every field.
| Field | Type | Description |
|---|---|---|
req_id |
int |
Request identifier. |
page_num |
int |
Number of pages to free. |
page_type |
PageType |
Page category. |
HaVmmBlockId¶
A two-part block id: which pool, and which block inside it. HaVmmMapBlocksBatchFromPools() names every placement with one of these, so a call can draw from several pools at once. Declared in ha_vmm.h.
| Field | Type | Description |
|---|---|---|
pool_idx |
uint32_t |
Index into the call's pools array. |
block_idx |
int64_t |
Block index within that pool, from 0. |
HaVmmPool¶
A pool as one value: the segment list plus its count.
| Field | Type | Description |
|---|---|---|
segments |
HaVmmBlockSegment * |
Segment list. The map and unmap calls do not read it and accept null. |
num_segments |
uint32_t |
Segment count; for HaVmmAllocatePool() it is in: capacity / out: count (1). |
Note
The allocator that fills a pool (HaVmmAllocatePool()), the map that reads it (HaVmmMapBlocksBatchFromPool()), and the free that reclaims it (HaVmmFreePool()) all pass a single object. segments is non-const because HaVmmAllocatePool() writes the caller's buffer through it; readers take it by value and only read. Declared in ha_vmm.h.
Constants and aliases¶
| Name | Value | Description |
|---|---|---|
HA_DEVICE_PTR_NULL |
0x0 |
Null sentinel for HaDevicePtr: no valid device address. The memory pools never hand out address 0 as a successful allocation. |
HA_STREAM_DEFAULT |
nullptr |
Selects the calling thread's default stream on the current device (see HaStream_t). |
HA_MAX_NDIM |
25 |
Maximum rank of a HaLayout; the size of HaLayout::shape / strides. |
HA_LAYOUT_INIT |
{} |
Zero-initializer for a fresh HaLayout: HaLayout L = HA_LAYOUT_INIT;. |
HA_LIBRARY_DATA_MAGIC |
0x424C4148u |
"HALB" read as a little-endian uint32_t; the magic of a HaLibraryData. |
HA_LIBRARY_DATA_VERSION |
1u |
The version of a HaLibraryData; bumped on any layout change. |
k-prefixed aliases of HaDataType, equivalent to PyTorch's c10::ScalarType constants but with the Ha prefix so they do not collide with c10::k*:
| Alias | HaDataType |
|---|---|
kHaByte |
Bytes |
kHaChar |
Int8 |
kHaShort |
Int16 |
kHaInt |
Int32 |
kHaLong |
Int64 |
kHaUInt8 |
Uint8 |
kHaUInt16 |
Uint16 |
kHaUInt32 |
Uint32 |
kHaUInt64 |
Uint64 |
kHaHalf |
Float16 |
kHaFloat |
Float32 |
kHaDouble |
Float64 |
kHaBFloat16 |
BFloat16 |
kHaFloat8 |
Float8 |
kHaFloat4 |
Float4 |
Helpers¶
HaDataTypeBits¶
Bits per element for dtype.
| Parameter | Direction | Default | Description |
|---|---|---|---|
dtype |
in | — | Element type whose bit width is queried. |
Returns — Number of bits per element, or 0 if dtype is not a recognized enumerator.
시그니처 · 예제
1 | |
예제
1 2 3 4 5 | |
Note
Returns 0 for unrecognized values (caller treats 0 as an error). This is the authoritative size accessor — HaDataTypeBytes() is a convenience that returns HaDataTypeBits() / 8 rounded down (and so returns 0 for sub-byte dtypes like Float4).
HaDataTypeBytes¶
Byte size of one element of dtype.
| Parameter | Direction | Default | Description |
|---|---|---|---|
dtype |
in | — | Element type whose byte size is queried. |
Returns — Number of bytes per element (HaDataTypeBits(dtype) / 8), or 0 if dtype is sub-byte or unrecognized.
시그니처 · 예제
1 | |
예제
1 | |
Note
Returns 0 for sub-byte dtypes (e.g. Float4) and for unrecognized values. Sub-byte callers should use HaDataTypeBits() and handle byte-alignment explicitly.
Note
Sub-byte dtypes cannot be sized in bytes, so use HaDataTypeBits() and handle the byte alignment yourself rather than treating the 0 as an error.