Skip to content

Weight Converter API

Converts weight tensors (with optional bias, scale and zero-point metadata) into the byte layout the target architecture reads from device memory. The API is stateless: there is no handle to create or destroy, and every call is independent of the others. Use the two-call pattern — query the size, allocate, then convert.

Input conventions, the device placement flow and the per-architecture support table are explained in the Weight Conversion guide.

1
#include "ha_weight_converter.h"   // also includes ha_weight_converter_types.h and ha_error_code.h
1
2
find_package(hart REQUIRED COMPONENTS hw)
target_link_libraries(your_target PRIVATE hart::ha_weight_converter)

hart::ha_weight_converter carries both include directories it needs (${hart_INCLUDE_DIR} for ha_error_code.h and ${hart_INCLUDE_DIR}/weight_converter). All names are at global scope with a HaWeight prefix.

Contents

API Kind Description
HaWeightGetConvertedSize Function Get the size of the converted output buffer
HaWeightConvert Function Convert weights into a caller-allocated buffer
HaWeightConfig Struct Target architecture and quantization method
HaWeightTensor Class Input tensor (data, shape, data type, role)
HaWeightShape Struct 4-D weight shape
HaWeightArchitectureType Enum Target architecture
HaWeightQuantType Enum Quantization method
HaWeightDataType Enum Element data type
HaWeightTensorType Enum Role of the tensor

Error handling

Both functions return HaError and never let an exception cross the boundary.

Code When
HaSuccess The call succeeded.
HaInvalidArgument A required pointer is null, out_buffer_size is below the queried size, a tensor the quantization type needs is missing, or input_channel_group_size is not a multiple of the architecture's DRAM word size.
HaNotSupported The quantization type is not supported on the target architecture.
HaInternalError Any other converter failure.
HaUnknownError An unexpected non-standard failure.

The HaWeightTensor constructor and HaWeightTensor::CopyOf are the exception: they validate their arguments and throw std::runtime_error (and CopyOf may throw std::bad_alloc), so construct tensors before calling the converter.


Functions

HaWeightGetConvertedSize

Query the converted-buffer size in bytes. Use the returned size to allocate the buffer passed to HaWeightConvert.

Parameter Direction Default Description
cfg in — Converter configuration.
weight_tensor in — Weight tensor (required).
bias_tensor in — Bias tensor (optional, may be nullptr).
output_data_type in — Data type of the matrix operation's output (see HaWeightConvert).
out_size out — Receives the total converted buffer size in bytes.

Returns — HaSuccess; HaInvalidArgument (null cfg / weight / out_size, or unknown arch); HaNotSupported (unsupported quant on this arch); HaInternalError (unexpected failure).

Signature
1
2
3
HaError HaWeightGetConvertedSize(const HaWeightConfig *cfg, HaWeightTensor *weight_tensor,
                                 HaWeightTensor *bias_tensor, HaWeightDataType output_data_type,
                                 uint64_t *out_size);

HaWeightConvert

Convert weight (+ optional bias / quant metadata) into a caller-provided buffer. The buffer must be at least HaWeightGetConvertedSize bytes. The converter writes directly into it; no intermediate copy is made.

Parameter Direction Default Description
cfg in — Converter configuration.
weight_tensor in — Weight tensor (required).
bias_tensor in — Bias tensor (optional).
scale_tensor in — Scale tensor (depends on the quantization method).
mx_scale_tensor in — MX-scale tensor (MXFP only).
wzp_tensor in — Weight zero-point tensor (depends on the quantization method).
ozp_tensor in — Output zero-point tensor (depends on the quantization method).
output_data_type in — Data type of the matrix operation's output (not of the converted weight): kFp32, kBf16, kFp8, kUint8 or kSint8.
channel_wise_packing in — Whether to pack into DRAM channels.
out_buffer out — Caller-allocated destination buffer (>= queried size).
out_buffer_size in — Capacity of out_buffer in bytes.

Returns — HaSuccess; HaInvalidArgument (null cfg / weight / out_buffer, or out_buffer_size below the queried size); HaNotSupported (unsupported quant on this arch); HaInternalError (unexpected failure).

Which optional tensors are required depends on cfg->quant_type and the output data type; a missing one returns HaInvalidArgument.

Tensor Required when
scale_tensor kOutputChannelQuant, kInputChannelWiseQuant, kMxfpQuant
wzp_tensor kOutputChannelQuant, kInputChannelWiseQuant, with an integer weight data type
ozp_tensor Any quantization type other than kNone with an integer output data type
mx_scale_tensor kMxfpQuant

Metadata tensors are shaped per output channel m of the weight. For kInputChannelWiseQuant, groups is the number of input-channel groups (r * s * c / input_channel_group_size, rounded up).

Tensor Shape
bias_tensor, ozp_tensor (1, 1, 1, m)
scale_tensor, wzp_tensor (kOutputChannelQuant) (1, 1, 1, m)
scale_tensor, wzp_tensor (kInputChannelWiseQuant) (1, 1, groups, m)
Signature · Example
1
2
3
4
HaError HaWeightConvert(const HaWeightConfig *cfg, HaWeightTensor *weight_tensor, HaWeightTensor *bias_tensor,
                        HaWeightTensor *scale_tensor, HaWeightTensor *mx_scale_tensor, HaWeightTensor *wzp_tensor,
                        HaWeightTensor *ozp_tensor, HaWeightDataType output_data_type, bool channel_wise_packing,
                        void *out_buffer, uint64_t out_buffer_size);

Example

1
2
3
4
5
6
7
8
9
HaWeightConfig cfg{HaWeightArchitectureType::kEvt1, HaWeightQuantType::kOutputChannelQuant};

uint64_t size = 0;
HaWeightGetConvertedSize(&cfg, &weight, &bias, HaWeightDataType::kUint8, &size);

std::vector<uint8_t> buf(size);
HaError err = HaWeightConvert(&cfg, &weight, &bias, &scale, nullptr, &wzp, &ozp,
                              HaWeightDataType::kUint8, /*channel_wise_packing=*/true,
                              buf.data(), buf.size());

Structs and classes

HaWeightConfig

Configuration passed on every call.

Field Type Default Description
arch_type HaWeightArchitectureType — Target architecture.
quant_type HaWeightQuantType — Quantization method.
input_channel_group_size int 0 Input channel group size (0 if unused).

input_channel_group_size applies to kInputChannelWiseQuant, where it must be a multiple of the architecture's DRAM word size (128 for kEvt0 / kEvt1, 64 for kEdge). kMxfpQuant ignores it and uses the DRAM word size.


HaWeightTensor

An input tensor: data pointer, HaWeightShape, HaWeightDataType and HaWeightTensorType. Move-only.

Member Description
HaWeightTensor(void *data, HaWeightShape shape, HaWeightDataType dtype, HaWeightTensorType tensor_type) Borrows data: the caller must keep the buffer alive while the tensor is used.
static HaWeightTensor CopyOf(const void *src, size_t bytes, HaWeightShape shape, HaWeightDataType dtype, HaWeightTensorType tensor_type) Owns a copy of bytes bytes at src, allocated from the runtime's host memory manager.
void *GetData() const The data pointer.
HaWeightShape GetShape() const The shape.
HaWeightDataType GetDataType() const The data type.
HaWeightTensorType GetTensorType() const The tensor's role.
uint64_t GetSize() const Size in bytes (r * s * c * m elements at the element bit width, rounded up).

Both constructors throw std::runtime_error for a null data pointer, a zero dimension, or a data type the tensor type does not accept. CopyOf also throws std::runtime_error for zero bytes or bytes short of what the shape and data type need (GetSize()), and std::bad_alloc when the host memory manager rejects the allocation (its HA_HOST_MEM_MAX_BYTES limit, or the system).

HaWeightTensorType Accepted HaWeightDataType
kWeight kBf16, kFp8, kFp4, kUint8, kSint8, kUint4, kSint4, kMxfp8, kMxfp4
kBias kFp32
kScale kFp32
kMxScale kMxfp8, kMxfp4
kWzp kUint8, kSint8, kUint4, kSint4
kOzp kUint8, kSint8

HaWeightShape

Shape for representing 4 dimensional weight data. Every dimension must be non-zero.

Field Type Description
r uint64_t The number of rows.
s uint64_t The number of columns.
c uint64_t The number of input channels.
m uint64_t The number of output channels.

Enumerations

HaWeightArchitectureType

Value Target
kEvt0 Bertha Cloud EVT0
kEvt1 Bertha Cloud EVT1
kEdge Bertha Edge EVT0

HaWeightQuantType

Value Description
kNone No quantization.
kOutputChannelQuant Per-output-channel quantization.
kInputChannelWiseQuant Grouped input-channel-wise quantization (input_channel_group_size).
kTokenWiseQuant Not supported on any architecture; returns HaNotSupported.
kMxfpQuant MXFP quantization. Supported on kEvt1 and kEdge; kEvt0 returns HaNotSupported.

HaWeightDataType

Value Bit width
kFp32 32
kBf16 16
kFp8 8
kFp4 4
kUint8 8
kSint8 8
kUint4 4
kSint4 4
kMxfp8 8
kMxfp4 4

HaWeightTensorType

Value Description
kWeight Weight.
kBias Bias.
kScale Scale.
kMxScale MX scale (MXFP only).
kWzp Weight zero point.
kOzp Output zero point.