Weight Converter API
Converts weight tensors (with optional bias, scale and zero-point metadata) into the byte layout the target
architecture reads from device memory. The API is stateless: there is no handle to create or destroy, and every
call is independent of the others. Use the two-call pattern — query the size, allocate, then convert.
Input conventions, the device placement flow and the per-architecture support table are explained in the
Weight Conversion guide.
| #include "ha_weight_converter.h" // also includes ha_weight_converter_types.h and ha_error_code.h
|
| find_package(hart REQUIRED COMPONENTS hw)
target_link_libraries(your_target PRIVATE hart::ha_weight_converter)
|
hart::ha_weight_converter carries both include directories it needs (${hart_INCLUDE_DIR} for ha_error_code.h
and ${hart_INCLUDE_DIR}/weight_converter). All names are at global scope with a HaWeight prefix.
Contents
Error handling
Both functions return HaError and never let an exception cross the boundary.
| Code |
When |
HaSuccess |
The call succeeded. |
HaInvalidArgument |
A required pointer is null, out_buffer_size is below the queried size, a tensor the quantization type needs is missing, or input_channel_group_size is not a multiple of the architecture's DRAM word size. |
HaNotSupported |
The quantization type is not supported on the target architecture. |
HaInternalError |
Any other converter failure. |
HaUnknownError |
An unexpected non-standard failure. |
The HaWeightTensor constructor and HaWeightTensor::CopyOf are the exception: they validate their arguments and
throw std::runtime_error (and CopyOf may throw std::bad_alloc), so construct tensors before calling the
converter.
Functions
HaWeightGetConvertedSize
Query the converted-buffer size in bytes. Use the returned size to allocate the buffer passed to
HaWeightConvert.
| Parameter |
Direction |
Default |
Description |
cfg |
in |
— |
Converter configuration. |
weight_tensor |
in |
— |
Weight tensor (required). |
bias_tensor |
in |
— |
Bias tensor (optional, may be nullptr). |
output_data_type |
in |
— |
Data type of the matrix operation's output (see HaWeightConvert). |
out_size |
out |
— |
Receives the total converted buffer size in bytes. |
Returns — HaSuccess; HaInvalidArgument (null cfg / weight / out_size, or unknown arch);
HaNotSupported (unsupported quant on this arch); HaInternalError (unexpected failure).
Signature
| HaError HaWeightGetConvertedSize(const HaWeightConfig *cfg, HaWeightTensor *weight_tensor,
HaWeightTensor *bias_tensor, HaWeightDataType output_data_type,
uint64_t *out_size);
|
HaWeightConvert
Convert weight (+ optional bias / quant metadata) into a caller-provided buffer. The buffer must be at least
HaWeightGetConvertedSize bytes. The converter writes directly into
it; no intermediate copy is made.
| Parameter |
Direction |
Default |
Description |
cfg |
in |
— |
Converter configuration. |
weight_tensor |
in |
— |
Weight tensor (required). |
bias_tensor |
in |
— |
Bias tensor (optional). |
scale_tensor |
in |
— |
Scale tensor (depends on the quantization method). |
mx_scale_tensor |
in |
— |
MX-scale tensor (MXFP only). |
wzp_tensor |
in |
— |
Weight zero-point tensor (depends on the quantization method). |
ozp_tensor |
in |
— |
Output zero-point tensor (depends on the quantization method). |
output_data_type |
in |
— |
Data type of the matrix operation's output (not of the converted weight): kFp32, kBf16, kFp8, kUint8 or kSint8. |
channel_wise_packing |
in |
— |
Whether to pack into DRAM channels. |
out_buffer |
out |
— |
Caller-allocated destination buffer (>= queried size). |
out_buffer_size |
in |
— |
Capacity of out_buffer in bytes. |
Returns — HaSuccess; HaInvalidArgument (null cfg / weight / out_buffer, or out_buffer_size below
the queried size); HaNotSupported (unsupported quant on this arch); HaInternalError (unexpected failure).
Which optional tensors are required depends on cfg->quant_type and the output data type; a missing one returns
HaInvalidArgument.
| Tensor |
Required when |
scale_tensor |
kOutputChannelQuant, kInputChannelWiseQuant, kMxfpQuant |
wzp_tensor |
kOutputChannelQuant, kInputChannelWiseQuant, with an integer weight data type |
ozp_tensor |
Any quantization type other than kNone with an integer output data type |
mx_scale_tensor |
kMxfpQuant |
Metadata tensors are shaped per output channel m of the weight. For kInputChannelWiseQuant, groups is the
number of input-channel groups (r * s * c / input_channel_group_size, rounded up).
| Tensor |
Shape |
bias_tensor, ozp_tensor |
(1, 1, 1, m) |
scale_tensor, wzp_tensor (kOutputChannelQuant) |
(1, 1, 1, m) |
scale_tensor, wzp_tensor (kInputChannelWiseQuant) |
(1, 1, groups, m) |
Signature · Example
| HaError HaWeightConvert(const HaWeightConfig *cfg, HaWeightTensor *weight_tensor, HaWeightTensor *bias_tensor,
HaWeightTensor *scale_tensor, HaWeightTensor *mx_scale_tensor, HaWeightTensor *wzp_tensor,
HaWeightTensor *ozp_tensor, HaWeightDataType output_data_type, bool channel_wise_packing,
void *out_buffer, uint64_t out_buffer_size);
|
Example
| HaWeightConfig cfg{HaWeightArchitectureType::kEvt1, HaWeightQuantType::kOutputChannelQuant};
uint64_t size = 0;
HaWeightGetConvertedSize(&cfg, &weight, &bias, HaWeightDataType::kUint8, &size);
std::vector<uint8_t> buf(size);
HaError err = HaWeightConvert(&cfg, &weight, &bias, &scale, nullptr, &wzp, &ozp,
HaWeightDataType::kUint8, /*channel_wise_packing=*/true,
buf.data(), buf.size());
|
Structs and classes
HaWeightConfig
Configuration passed on every call.
| Field |
Type |
Default |
Description |
arch_type |
HaWeightArchitectureType |
— |
Target architecture. |
quant_type |
HaWeightQuantType |
— |
Quantization method. |
input_channel_group_size |
int |
0 |
Input channel group size (0 if unused). |
input_channel_group_size applies to kInputChannelWiseQuant, where it must be a multiple of the architecture's
DRAM word size (128 for kEvt0 / kEvt1, 64 for kEdge). kMxfpQuant ignores it and uses the DRAM word size.
HaWeightTensor
An input tensor: data pointer, HaWeightShape, HaWeightDataType and HaWeightTensorType. Move-only.
| Member |
Description |
HaWeightTensor(void *data, HaWeightShape shape, HaWeightDataType dtype, HaWeightTensorType tensor_type) |
Borrows data: the caller must keep the buffer alive while the tensor is used. |
static HaWeightTensor CopyOf(const void *src, size_t bytes, HaWeightShape shape, HaWeightDataType dtype, HaWeightTensorType tensor_type) |
Owns a copy of bytes bytes at src, allocated from the runtime's host memory manager. |
void *GetData() const |
The data pointer. |
HaWeightShape GetShape() const |
The shape. |
HaWeightDataType GetDataType() const |
The data type. |
HaWeightTensorType GetTensorType() const |
The tensor's role. |
uint64_t GetSize() const |
Size in bytes (r * s * c * m elements at the element bit width, rounded up). |
Both constructors throw std::runtime_error for a null data pointer, a zero dimension, or a data type the tensor
type does not accept. CopyOf also throws std::runtime_error for zero bytes or bytes short of what the shape and data type need
(GetSize()), and std::bad_alloc when the host memory manager rejects the allocation (its HA_HOST_MEM_MAX_BYTES limit, or the system).
HaWeightTensorType |
Accepted HaWeightDataType |
kWeight |
kBf16, kFp8, kFp4, kUint8, kSint8, kUint4, kSint4, kMxfp8, kMxfp4 |
kBias |
kFp32 |
kScale |
kFp32 |
kMxScale |
kMxfp8, kMxfp4 |
kWzp |
kUint8, kSint8, kUint4, kSint4 |
kOzp |
kUint8, kSint8 |
HaWeightShape
Shape for representing 4 dimensional weight data. Every dimension must be non-zero.
| Field |
Type |
Description |
r |
uint64_t |
The number of rows. |
s |
uint64_t |
The number of columns. |
c |
uint64_t |
The number of input channels. |
m |
uint64_t |
The number of output channels. |
Enumerations
HaWeightArchitectureType
| Value |
Target |
kEvt0 |
Bertha Cloud EVT0 |
kEvt1 |
Bertha Cloud EVT1 |
kEdge |
Bertha Edge EVT0 |
HaWeightQuantType
| Value |
Description |
kNone |
No quantization. |
kOutputChannelQuant |
Per-output-channel quantization. |
kInputChannelWiseQuant |
Grouped input-channel-wise quantization (input_channel_group_size). |
kTokenWiseQuant |
Not supported on any architecture; returns HaNotSupported. |
kMxfpQuant |
MXFP quantization. Supported on kEvt1 and kEdge; kEvt0 returns HaNotSupported. |
HaWeightDataType
| Value |
Bit width |
kFp32 |
32 |
kBf16 |
16 |
kFp8 |
8 |
kFp4 |
4 |
kUint8 |
8 |
kSint8 |
8 |
kUint4 |
4 |
kSint4 |
4 |
kMxfp8 |
8 |
kMxfp4 |
4 |
HaWeightTensorType
| Value |
Description |
kWeight |
Weight. |
kBias |
Bias. |
kScale |
Scale. |
kMxScale |
MX scale (MXFP only). |
kWzp |
Weight zero point. |
kOzp |
Output zero point. |