콘텐츠로 이동

Types

Enumerations, type aliases, and structures declared by the public headers.

Contents

이름 분류 설명
HaDataType Enum HaLayout의 요소 타입 정의
HaDeviceAttr Enum HaGetDeviceAttribute()로 읽는 디바이스 속성
HaDeviceVersion Enum 디바이스 칩 세대 식별자 (HaDeviceAttr::Version의 값)
HaError Enum 런타임 API가 반환하는 공통 에러 코드
HaLaunchArgRoute Enum Legato 커널 인자를 기록할 인자 블록 지정
HaMemCpyDirection Enum HaMemCpy()의 데이터 복사 방향 지정
HaMemScope Enum HaMemCpyNd() 포인터 인자가 가리키는 주소 공간 지정
HaTensorKind Enum HaBuildDeviceTensorLayout()의 텐서 종류 판별자
PageType Enum KV 캐시의 KEY·VALUE 페이지 구분
ValuePageMode Enum VALUE 페이지 배치 방식
HaDevicePtr Type alias 64비트 디바이스 주소 타입
HaEvent_t Type alias 이벤트 공개 핸들 타입
HaKernel_t Type alias 커널 라이브러리 안 커널의 공개 핸들 타입
HaLibrary_t Type alias 커널 라이브러리 공개 핸들 타입
HaStream_t Type alias 스트림 공개 핸들 타입
HaDeviceMemStats Struct 현재 디바이스 메모리 통계 (HaDeviceMemGetStats())
HaHostMemStats Struct 프로세스 호스트 메모리 통계 (HaHostMemGetStats())
HaLaunchArgs Class 커널 launch 인자 목록 (소유권을 갖는 이동 전용 객체)
HaLayout Struct HaMemCpyNd()용 오퍼랜드별 N차원(N-D) Shape 기술자
HaLibraryData Struct 커널 라이브러리의 메모리 내 형식 (HaLibraryLoadData()에 전달)
HaShape Struct 3D 논리 Shape (HaMemAlloc3D() 등에서 사용)
HaVmmBlockSegment Struct 풀의 연속 구간: pa_start, block_count, raw_ptr, raw_size
HaVmmPool Struct 세그먼트 배열과 개수 (HaVmmAllocatePool()에서 입력: 용량, 출력: 개수)
TLBInitInfo Struct HaInitTlb()의 정적 KV 캐시 TLB 설정
TLBAllocInfo Struct HaAllocTlbPage()의 페이지 할당 요청
TLBDeallocInfo Struct HaDeallocTlbPage()의 페이지 해제 요청
HaVmmBlockId Struct 풀 번호와 풀 안의 블록 번호 (HaVmmMapBlocksBatchFromPools())
HA_DEVICE_PTR_NULL 외 Constant HA_DEVICE_PTR_NULL, HA_STREAM_DEFAULT, HA_MAX_NDIM, HA_LAYOUT_INIT, HA_LIBRARY_DATA_*, kHa* 별칭
HaDataTypeBits Helper 데이터 타입(dtype)의 요소당 비트 수 계산
HaDataTypeBytes Helper 데이터 타입(dtype)의 요소당 바이트 크기 계산

Enumerations

HaDataType

Element type of a HaLayout — unit of shape[] / strides[].

Value Description
Bytes Raw bytes (1 byte per element).
Int8 Signed 8-bit integer.
Int16 Signed 16-bit integer.
Int32 Signed 32-bit integer.
Int64 Signed 64-bit integer.
Uint8 Unsigned 8-bit integer.
Uint16 Unsigned 16-bit integer.
Uint32 Unsigned 32-bit integer.
Uint64 Unsigned 64-bit integer.
Float16 IEEE 754 binary16 (half-precision float).
Float32 IEEE 754 binary32 (single-precision float).
BFloat16 bfloat16.
Float8 8-bit float (1 byte).
Float4 4-bit float (sub-byte: 0.5 byte). Two Float4 elements pack into one byte.
Float64 IEEE 754 binary64 (double-precision float). Appended last so the values above keep their numbers.

Note

Raw byte transfers use Bytes as an escape hatch. Values mirror the DLPack dtype family in intent but are runtime-local.


HaDeviceAttr

Scalar device attributes readable through HaGetDeviceAttribute(). Declared as enum class HaDeviceAttr : int; the values are part of the ABI, so new values are only appended and existing values are never renumbered.

Value Number Description
PciDomain 0 PCI domain (segment).
PciBus 1 PCI bus number.
PciSlot 2 PCI device number.
PciFunc 3 PCI function number.
PeerPresentMask 4 Peers the driver has registered; bit i = device i, own bit always 0.
PeerMappedMask 5 Peers whose User BAR is mapped into this device; bit i = device i. Necessary but not sufficient for P2P: a set bit means an IOVA exists, not that the PCIe fabric routes the transfer peer-to-peer.
DramChannels 6 Number of DRAM channels.
DramChannelStride 7 Byte distance between consecutive DRAM channels' address ranges.
Version 10 Chip generation, a HaDeviceVersion value. Packed weight layouts and ABI selection are chip-specific and the chip cannot be recovered from any buffer contents, so callers that need it must ask the runtime.

HaDeviceVersion

Chip generation of a device. Read it with HaGetDeviceAttribute() and HaDeviceAttr::Version, then cast the value to this enum.

1
enum class HaDeviceVersion : int;
Value Number Description
BerthaCloudEVT0 0 Bertha Cloud EVT0.
BerthaCloudEVT1 1 Bertha Cloud EVT1.
BerthaEdgeEVT0 2 Bertha Edge EVT0.

Note

Append only: the values are part of the ABI. Use this to select hardware revisions/targets or branch on supported features.


HaError

Common error codes returned by the runtime API.

Declared in ha_error_code.h, which ha_types.h includes.

Value Description
HaSuccess Success.
HaInvalidArgument Invalid non-device argument (range/null pointer, etc.).
HaNotSupported Unsupported API, option, or execution path.
HaTimeout Operation timed out.
HaInternalError Internal error (unexpected state/bug).
HaUnknownError Unknown error (unclassified failure).
HaDeviceError —
HaDeviceNotAvailable Device is unavailable or does not exist.
HaDeviceNotInitialized Device is not initialized (e.g., HaSetDevice() not called).
HaDeviceNotReady Device is not ready (initialization/reset/load in progress, etc.).
HaDeviceInvalidArgument Invalid device-related argument (range/null pointer, etc.).
HaDeviceAlreadyInitialized Device is already initialized.
HaDeviceOpenFailed Failed to open the device.
HaDeviceCloseFailed Failed to close the device.
HaDeviceResetFailed Failed to reset the device.
HaDeviceUnsupported Unsupported device/version/feature.
HaDeviceRegisterSetFailed Failed to program device registers (e.g., SFR).
HaDeviceInvalidValue The device returned an invalid value.
HaDeviceBusy Device is still in use (e.g., ha_drv_deinit while a handle is open).
HaMemoryError —
HaMemoryInvalidAddress Invalid memory address (host/device pointer mismatch, etc.).
HaMemoryAlignmentError Alignment constraint violation.
HaMemoryInvalidArgument Invalid memory-related argument (size/null pointer, etc.).
HaMemoryAllocationFailed Allocation failed (out-of-memory/constraints, etc.).
HaMemoryFreeFailed Failed to free memory.
HaMemoryCopyFailed Failed to copy memory.
HaMemCpyNotSupported Memcpy operation not supported (e.g., cross-device D2D).
HaHostMemLimitExceeded Allocation rejected by a configured limit (HA_HOST_MEM_MAX_BYTES, HA_MEMCPYND_STAGING_MAX_BYTES), not by the system.
HaKernelError —
HaKernelInvalidArgument Invalid kernel-related argument (arg count/pointer, etc.).
HaKernelUnsupported Unsupported kernel or execution path.
HaKernelExecutionFailed Kernel execution failed (including runtime/device errors).
HaKernelNotFound Kernel not found (registration/binary load failure, etc.).
HaKernelInvalidHandle Kernel handle released, invalidated by a device reset, or never loaded.
HaKernelNoBinaryForDevice The library has no binary of the kernel for the device.
HaKernelLoadFailed The device could not load the kernel (a failure other than running out of kernel memory).
HaStreamError —
HaStreamInvalidHandle Invalid stream handle.
HaStreamInvalidArgument Invalid stream arguments.
HaStreamCreationFailed Failed to create stream.
HaStreamDestroyFailed Failed to destroy stream.
HaStreamSyncFailed Stream synchronization failed.
HaStreamNotReady Returned by HaStreamQuery() when work is still in progress.
HaEventError —
HaEventInvalidHandle Invalid event handle.
HaEventInvalidArgument Invalid event arguments.
HaEventCreationFailed Failed to create event.
HaEventDestroyFailed Failed to destroy event.
HaEventNotRecorded HaEventRecord() has not been called on this event.
HaEventNotReady Returned by HaEventQuery() when the event has not yet completed.

Note

Use these codes to classify failures (device/memory/kernel) and implement user-level error handling, retry, and logging.

Value encoding:

  • Success: 0
  • Common: code (1..65535)
  • Device: (1 << 16) \| code
  • Memory: (2 << 16) \| code
  • Kernel: (3 << 16) \| code
  • Stream: (4 << 16) \| code
  • Event: (5 << 16) \| code

Codes, causes and remedies are listed in Error Codes.


PageType

Distinguishes KEY pages and VALUE pages in the KV cache. Used by TLBAllocInfo and TLBDeallocInfo.

Value Description
KEY Key page.
VALUE Value page.

ValuePageMode

Placement policy for value pages. Used by TLBInitInfo.

Value Description
CONTIGUOUS Value pages allocated sequentially in memory.
STRIDE Value pages allocated with fixed stride (for interleave).

HaLaunchArgRoute

Which executor's argument block a Legato kernel argument is written to. A Legato kernel's ABI holds one argument block for the top executor and one per core, and each executor reads only its own. The route must be the one the kernel was compiled with: a different route puts the argument in a block the kernel does not read it from.

Value Description
Universal The top block and every core block.
Top The top block only.
Core The block of core target_core only. A core that needs the argument needs its own.

HaMemCpyDirection

Memory copy direction for HaMemCpy().

Value Description
HostToDevice Host → Device.
DeviceToHost Device → Host.
DeviceToDevice Device → Device.

Note

Select the appropriate direction based on where src and dst reside (host vs. device).


HaMemScope

Which address space the matching HaMemCpyNd() pointer argument refers to.

Value Description
Host Host memory (process-visible pointer).
Device Device pointer (resolved by the per-device memory agent).

Note

The runtime uses (dst.scope, src.scope) to decide H2D / D2H / D2D dispatch for HaMemCpyNd(). HaMemCpyNd() performs H2H copies on the host. HaMemCpyNdAsync() rejects H2H because there is no host-to-host stream operation to order it against.


HaTensorKind

Tensor-kind discriminator for HaBuildDeviceTensorLayout().

Value Description
KvCacheKey KV-cache K tensor; canonical 10-D storage layout.
KvCacheValue KV-cache V tensor. Same logical dims as KvCacheKey; differs in dim-4 ↔ dim-7 stride swap.
SingleChannel Whole matrix on a single channel; chunk = H × W × element_size.
RowInterleaved Row-interleaved across channels; chunk = W × element_size (one row).
MaxRowsInterleaved Multi-row-interleaved; chunk = max_reuse_num × cores_per_cg × W × element_size (HW MAC-reuse window, multiple rows per chunk).
DramWordInterleaved DRAM-word-interleaved (generic primitive); chunk = dram_word_size_bytes (64 B on BerthaCloud, 32 B on BerthaEdge).
MPUTensor Inner-6-D Value-pattern layout (block_size = ∞ limit of KvCacheValue).
MpuTensorTransposed Inner-6-D Key-pattern layout (block_size = ∞ limit of KvCacheKey). The hardware reads it as the transposed view of the data, for transposed-weight / right-hand-side matmul operands.
SinglePassInterleaved Single-pass interleave: each channel gets one contiguous slab of total/N; no cycling (the chunk is sized exactly to consume the tensor in one pass).
NumOptions Sentinel (count of kinds), not a kind. Never pass it to HaBuildDeviceTensorLayout().

Each kind fixes the meaning, length, and order of the dims array passed to HaBuildDeviceTensorLayout (see that function's docs):

  • KvCacheKey / KvCacheValue: logical dims describe [num_blocks, num_layers, num_kv_heads, block_size, head_dim].
  • SingleChannel / RowInterleaved / MaxRowsInterleaved: 2 dims = [H, W]. All three interleave the allocation across all DRAM channels and follow the unified padding rule padding = data_size mod (#channels × chunk_size).
  • MPUTensor: 2 dims = [H, W]. Inner-6-D of the Value-pattern layout — the block_size = ∞ limit of KvCacheValue with batching dims (num_blocks, num_layers, num_kv_heads, blocks_per_stripe) all collapsed to 1. Shares the storage formula with KvCacheValue.
  • MpuTensorTransposed: 2 dims = [H, W]. Inner-6-D of the Key-pattern layout — the block_size = ∞ limit of KvCacheKey with batching dims (num_blocks, num_layers, num_kv_heads, blocks_per_stripe) all collapsed to 1. Shares the storage formula with KvCacheKey; differs from MPUTensor only in the dim-0 ↔ dim-3 stride swap of the 6-D core (the Key/Value variant of the inner layout). Use this for a transposed weight / right-hand-side matmul operand that needs the Key wire pattern instead of the Value wire pattern.
  • DramWordInterleaved: any rank ndim >= 1. Storage formula only consumes total = product(dims); caller may pass their natural contiguous logical shape (e.g. [B, S, D], [H, W], or [total]) without flattening. Storage is rank-3 (chunks_per_channel, num_dram_channels, elements_per_word) with strides (elements_per_word, channel_stride_elements, 1).
  • SinglePassInterleaved: any rank ndim >= 1 (dtype units; HaDataType::Bytes for raw byte buffers). Single-pass interleave — each channel gets one contiguous slab of total/N; chunk is sized to consume the tensor in one pass. Storage formula only consumes total = product(dims); caller may pass their natural contiguous logical shape without flattening. Storage is rank-2 (num_dram_channels, total / num_dram_channels) with strides (channel_stride_elements, 1).

Kind contract:

  • Layout is (kind, dtype, dims, chip params) -> HaLayout, not kind alone; chip params come from the current device (8 ch / 64 B word on BerthaCloud, 2 / 32 B on BerthaEdge).
  • Host-side selector only: HaBuildDeviceTensorLayout() expands it into HaLayout shape/strides, which carry no kind field.
  • Ordinals are a wire format between components, so treat the numbering as append-only.

Type aliases

HaDevicePtr

64-bit device address type (analogous to CUdeviceptr).

1
using HaDevicePtr = uint64_t;

Note

Primary role: holds device-side addresses for memory allocation / free / DA-to-device routing (HaMemAlloc, HaMemFree, VMM internals). CUDA uses CUdeviceptr = unsigned long long (integer, not void*); the driver API (cuMemMap, cuMemAddressReserve, cuMemcpyHtoD) uses CUdeviceptr while the runtime API (cudaMemcpy) uses void*. They're interoperable via cast — HaDevicePtr matches the driver-API convention.

Note

Secondary role (copy APIs): for parameters that may carry either a host pointer or a device address — disambiguated at runtime by the accompanying HaMemCpyDirection — HaDevicePtr is reused as the integer carrier. Numerically compatible with both void * and host integer addresses on 64-bit targets (sizeof(uintptr_t) == sizeof(void *) == 8).

Note

Conversion rules: Host pointer → HaDevicePtr : reinterpret_cast<HaDevicePtr>(host_ptr) Device address → HaDevicePtr : value-preserving (no cast needed) HaDevicePtr → host pointer : reinterpret_cast<void *>(addr) (H2D: src; D2H: dst) HaDevicePtr → device address : value-preserving (no cast needed)


HaEvent_t

Public handle type for an event.

1
typedef HaEvent* HaEvent_t;

HaKernel_t

Public handle type for a kernel of a library, from HaLibraryGetKernel(). Valid until the process ends, across device resets.

1
typedef HaKernel *HaKernel_t;

Note

A handle is an id the runtime looks up, not the address of an object: HaKernel is only forward-declared, so a handle cannot be dereferenced.


HaLibrary_t

Public handle type for a library of precompiled kernels, from HaLibraryLoadData(). Valid until the process ends.

1
typedef HaLibrary *HaLibrary_t;

HaStream_t

Public handle type for a stream. Pass nullptr (HA_STREAM_DEFAULT) to use the calling thread's default stream on the current device.

1
using HaStream_t = HaStream *;

Note

Default streams follow CUDA per-thread-default-stream (PTDS) semantics: one backing stream per (current device, calling thread), created lazily on first use. Two threads that pass HA_STREAM_DEFAULT on the same device therefore get different streams and are NOT ordered against each other. To order work across threads, create an explicit stream and share it, or sequence them with events.


Structures

HaDeviceMemStats

Device memory statistics of the current device, filled by HaDeviceMemGetStats() (and the driver API's ha_drv_device_mem_get_stats()). Extensible like HaHostMemStats: set struct_size to sizeof(HaDeviceMemStats) before the call; the library fills every field that fits entirely and stores the number of bytes written. Each counter is read atomically, but the counters together are not one consistent snapshot.

Byte figures are the sizes occupied in all DRAM channels of the device, summed, so a contiguous allocation is counted as its requested size multiplied by the number of channels. The exception is requested_bytes and its peak, which are the sizes the callers asked for and are not multiplied by the channel count. free_bytes is the driver's own figure for the whole device (every process, memory the runtime's pools do not manage included); the other fields cover the runtime's pools only.

Field Type Description
struct_size uint32_t In: sizeof as compiled. Out: bytes written.
reserved0 uint32_t Written as 0.
total_bytes uint64_t DRAM capacity the device reports.
free_bytes uint64_t Free DRAM of the device as the driver reports it at the call.
releasable_bytes uint64_t Bytes of segments with no allocated block, which a trim returns to the driver.
allocated_bytes uint64_t Bytes of the blocks currently handed out, rounded as the pool slots them.
peak_allocated_bytes uint64_t Highest allocated_bytes since the last peak reset.
reserved_bytes uint64_t Bytes of the segments the pools hold, summed over both runtime pools.
peak_reserved_bytes uint64_t Highest reserved_bytes since the last peak reset.
requested_bytes uint64_t Sizes the callers asked for, over both runtime pools; not scaled by channels.
peak_requested_bytes uint64_t Highest requested_bytes since the last peak reset.
cached_free_bytes uint64_t Freed bytes kept in the pools for reuse (reserved_bytes − allocated_bytes), summed over both runtime pools.
active_blocks uint64_t Number of blocks currently handed out, summed over both runtime pools.
peak_active_blocks uint64_t Highest active_blocks since the last peak reset.
segments uint64_t Number of segments the pools hold, summed over both runtime pools.
peak_segments uint64_t Highest segments since the last peak reset.
device_allocs uint64_t Number of segments obtained from the driver so far, summed over both runtime pools.
device_frees uint64_t Number of segments returned to the driver so far, summed over both runtime pools.

HaHostMemStats

Host memory statistics of the whole process, filled by HaHostMemGetStats() (and the driver API's ha_drv_host_mem_get_stats()). Extensible: set struct_size to sizeof(HaHostMemStats) before the call; the library fills at most that many bytes and stores the count it wrote. Each counter is read atomically; the set is not one snapshot.

Field Type Description
struct_size uint32_t In: sizeof as compiled. Out: bytes written.
reserved0 uint32_t Written as 0.
allocated_bytes uint64_t Bytes currently allocated (as requested, not rounded).
peak_allocated_bytes uint64_t Highest allocated_bytes since the last peak reset.
reserved_bytes uint64_t Bytes obtained from the OS, including cached freed memory.
peak_reserved_bytes uint64_t Highest reserved_bytes since the last peak reset.
alloc_count uint64_t Allocations ever made.
free_count uint64_t Allocations ever freed.

HaLaunchArgs

The arguments of one kernel launch, in the order they were appended. Owns its records. Defined in ha_driver_types.h, which ha_runtime_api.h and ha_driver_api.h include; ha_types.h holds only a forward declaration.

Construct one to get an empty list, fill it with HaLaunchArgsAdd*() (or the driver API's ha_drv_launch_args_add_*()), and pass it to HaLaunchKernel() / HaLaunchKernelAsync(). The records are released when the object is destroyed. The record layout belongs to the kernel ABI and is not published. See Execution API.

Member Description
HaLaunchArgs() Creates an empty argument list. Allocates, and throws std::bad_alloc when memory is short.
~HaLaunchArgs() Releases the records.
HaLaunchArgs(HaLaunchArgs &&other) noexcept Takes the records of other, which is left moved from.
HaLaunchArgs &operator=(HaLaunchArgs &&other) noexcept Releases the records and takes those of other, which is left moved from.

The class is move-only (copy construction and copy assignment are deleted). A moved-from list is rejected by every function with HaKernelInvalidArgument. Threads may share one list: appends and updates take its lock exclusively and a launch reads under it shared. Moving a list is not synchronized, so do not move or destroy a list while another thread uses it.


HaLayout

Per-operand N-D shape descriptor for HaMemCpyNd(): shape, strides, dtype, scope.

Field Type Description
shape uint64_t[HA_MAX_NDIM] Dimension sizes (elements of dtype). Only the first ndim entries are valid.
strides uint64_t[HA_MAX_NDIM] Per-dimension strides (elements of dtype). Only the first ndim entries are valid.
scope HaMemScope Address space the matching HaMemCpyNd() pointer argument refers to.
dtype HaDataType Element type — unit of shape and strides.
ndim uint32_t Number of valid entries in shape / strides; must satisfy 1 <= ndim <= HA_MAX_NDIM.

Note

A pure shape spec — no buffer pointer. The matching HaDevicePtr is passed alongside the layout to HaMemCpyNd(), so a single layout (e.g. a KV-cache block layout) can describe many different buffers without rebuilding.

Note

The hot arrays (shape, strides) are placed first and the struct is 64 B aligned. At HA_MAX_NDIM 25 each array is 200 B. Start from HA_LAYOUT_INIT to zero-initialize a layout.

Note

shape / strides are expressed in elements of dtype, not bytes. strides[i] is the element distance between consecutive entries along dimension i. ndim counts how many entries of each array are valid; entries past ndim are ignored.


HaLibraryData

Root of a kernel library: what HaLibraryLoadData() is given.

Field Type Description
magic uint32_t HA_LIBRARY_DATA_MAGIC (0x424C4148u, "HALB" read as a little-endian uint32_t).
version uint32_t HA_LIBRARY_DATA_VERSION (1u), bumped on any layout change.
set_count uint32_t Number of entries in sets.
sets const HaLibraryBinarySet * One set per device version.

A HaLibraryData holds one HaLibraryBinarySet per device version, and each set lists the kernels built for that version as HaLibraryKernelBinary entries (name and Legato ELF).

HaLibraryBinarySet:

Field Type Description
device_version int32_t HaDeviceVersion value; unique within a HaLibraryData.
binary_count uint32_t Number of entries in binaries.
binaries const HaLibraryKernelBinary * The set's binaries.

HaLibraryKernelBinary:

Field Type Description
name const char * NUL-terminated kernel name, unique within its set.
elf const void * Legato ELF bytes.
elf_size uint64_t Number of bytes at elf.

Note

The device version of a set is stated once by the build input as a HaDeviceVersion value. A HaLibraryData of a version this runtime does not read is rejected. The structs are read only during HaLibraryLoadData(), but the kernel names and ELF bytes they point to are not copied and must stay valid for the life of the process.


HaShape

3D logical shape (e.g., for HaMemAlloc3D()); used to describe tensor-like buffers.

Field Type Description
w size_t Width (contiguous dimension).
h size_t Height (split by channel/block).
c size_t Channel.

Concrete layout/packing rules are defined by the runtime/device.

Note

W = width (contiguous dimension), H = height (split by channel/block), C = channel.


HaVmmBlockSegment

One contiguous run of block slots: a start and a size.

Field Type Description
pa_start HaDevicePtr Physical base address of this segment.
block_count int64_t Blocks the segment spans from pa_start.
raw_ptr HaDevicePtr HaMemAlloc base this segment was carved from (HaMemFree target).
raw_size uint64_t Full byte size of that allocation, alignment slack included.

Note

A Bertha DC EVT0 pool is one of these. pa_start is raw_ptr rounded up to the requested alignment and is the base HaVmmInitBlocks() registers with the TLB; free raw_ptr, never pa_start. Declared in ha_types.h.


TLBInitInfo

Static configuration for TLB initialization at model load time, passed to HaInitTlb(). Has an equality operator that compares every field.

Field Type Description
page_size int Page size in bytes.
base_addr size_t Base address of TLB.
layer_num int Number of LLM layers.
head_num int Number of LLM heads.
max_page_num int Maximum pages the TLB can map.
value_page_mode ValuePageMode Value page mode (CONTIGUOUS or STRIDE).
value_stride int Stride between value pages when STRIDE mode.
value_line_page_num int Pages per value line.
log_value_stride int log2 of value_stride.

TLBAllocInfo

Page allocation request, passed to HaAllocTlbPage(). Has an equality operator that compares every field.

Field Type Description
req_id int Request identifier.
page_num int Number of pages to allocate.
page_type PageType Page category (KEY / VALUE).

TLBDeallocInfo

Page deallocation request, passed to HaDeallocTlbPage(). Has an equality operator that compares every field.

Field Type Description
req_id int Request identifier.
page_num int Number of pages to free.
page_type PageType Page category.

HaVmmBlockId

A two-part block id: which pool, and which block inside it. HaVmmMapBlocksBatchFromPools() names every placement with one of these, so a call can draw from several pools at once. Declared in ha_vmm.h.

Field Type Description
pool_idx uint32_t Index into the call's pools array.
block_idx int64_t Block index within that pool, from 0.

HaVmmPool

A pool as one value: the segment list plus its count.

Field Type Description
segments HaVmmBlockSegment * Segment list. The map and unmap calls do not read it and accept null.
num_segments uint32_t Segment count; for HaVmmAllocatePool() it is in: capacity / out: count (1).

Note

The allocator that fills a pool (HaVmmAllocatePool()), the map that reads it (HaVmmMapBlocksBatchFromPool()), and the free that reclaims it (HaVmmFreePool()) all pass a single object. segments is non-const because HaVmmAllocatePool() writes the caller's buffer through it; readers take it by value and only read. Declared in ha_vmm.h.


Constants and aliases

Name Value Description
HA_DEVICE_PTR_NULL 0x0 Null sentinel for HaDevicePtr: no valid device address. The memory pools never hand out address 0 as a successful allocation.
HA_STREAM_DEFAULT nullptr Selects the calling thread's default stream on the current device (see HaStream_t).
HA_MAX_NDIM 25 Maximum rank of a HaLayout; the size of HaLayout::shape / strides.
HA_LAYOUT_INIT {} Zero-initializer for a fresh HaLayout: HaLayout L = HA_LAYOUT_INIT;.
HA_LIBRARY_DATA_MAGIC 0x424C4148u "HALB" read as a little-endian uint32_t; the magic of a HaLibraryData.
HA_LIBRARY_DATA_VERSION 1u The version of a HaLibraryData; bumped on any layout change.

k-prefixed aliases of HaDataType, equivalent to PyTorch's c10::ScalarType constants but with the Ha prefix so they do not collide with c10::k*:

Alias HaDataType
kHaByte Bytes
kHaChar Int8
kHaShort Int16
kHaInt Int32
kHaLong Int64
kHaUInt8 Uint8
kHaUInt16 Uint16
kHaUInt32 Uint32
kHaUInt64 Uint64
kHaHalf Float16
kHaFloat Float32
kHaDouble Float64
kHaBFloat16 BFloat16
kHaFloat8 Float8
kHaFloat4 Float4

Helpers

HaDataTypeBits

Bits per element for dtype.

Parameter Direction Default Description
dtype in — Element type whose bit width is queried.

Returns — Number of bits per element, or 0 if dtype is not a recognized enumerator.

시그니처 · 예제
1
constexpr size_t HaDataTypeBits(HaDataType dtype);

예제

1
2
3
4
5
const size_t bits = HaDataTypeBits(dtype);
if (bits == 0) {
  return UnsupportedDataType();   // 0 is the error, not a valid width
}
const size_t bytes_needed = (numel * bits + 7) / 8;

Note

Returns 0 for unrecognized values (caller treats 0 as an error). This is the authoritative size accessor — HaDataTypeBytes() is a convenience that returns HaDataTypeBits() / 8 rounded down (and so returns 0 for sub-byte dtypes like Float4).


HaDataTypeBytes

Byte size of one element of dtype.

Parameter Direction Default Description
dtype in — Element type whose byte size is queried.

Returns — Number of bytes per element (HaDataTypeBits(dtype) / 8), or 0 if dtype is sub-byte or unrecognized.

시그니처 · 예제
1
constexpr size_t HaDataTypeBytes(HaDataType dtype);

예제

1
const size_t bytes = HaDataTypeBytes(dtype);   // 0 for Float4 and other sub-byte types

Note

Returns 0 for sub-byte dtypes (e.g. Float4) and for unrecognized values. Sub-byte callers should use HaDataTypeBits() and handle byte-alignment explicitly.

Note

Sub-byte dtypes cannot be sized in bytes, so use HaDataTypeBits() and handle the byte alignment yourself rather than treating the 0 as an error.