Skip to content

Python Weight Converter API

Converts weight tensors to the byte layout that the target architecture reads from device memory. The semantics are the same as the C++ Weight Converter API. It is provided as module functions that map 1:1 to the C++ functions; only the debugging function show() is added in Python. The HaWeight prefix of the C++ type names is dropped (for example, HaWeightTensor → Tensor). For input conventions and the overall flow, see the Weight Conversion guide. There are no handles or state, so each call is independent.

1
2
3
4
from hart.weight_converter import (
    WeightConverterConfig, get_converted_size, convert, show,
    ArchitectureType, QuantType, DataType, Shape, Tensor, TensorType,
)

Functions

Function Signature
get_converted_size (cfg: WeightConverterConfig, weight_tensor: Tensor, bias_tensor: Tensor \| None = None, *, output_data_type: DataType) -> int
convert (cfg: WeightConverterConfig, weight_tensor: Tensor, bias_tensor: Tensor \| None = None, scale_tensor: Tensor \| None = None, mx_scale_tensor: Tensor \| None = None, wzp_tensor: Tensor \| None = None, ozp_tensor: Tensor \| None = None, *, output_data_type: DataType, channel_wise_packing: bool = True) -> numpy.ndarray[numpy.uint8]
show (cfg: WeightConverterConfig, converted_data: buffer, weight_tensor: Tensor, bias_tensor: Tensor \| None = None, *, output_data_type: DataType) -> None

convert() handles both the size query and the buffer allocation, and returns a one-dimensional uint8 NumPy array that holds the converted result. The result buffer is allocated from the runtime host memory manager and exposed as the array without a copy. When the array is released, the buffer is returned as well. To get only the size in advance, use get_converted_size().

output_data_type and channel_wise_packing can be passed only as keyword arguments. C++ has no default, but in Python channel_wise_packing defaults to True.

show() is a debugging function that prints the converted result as a hex table; there is no corresponding public C++ API. It works only in debug builds. In release builds, it logs a single "not supported" warning on the first call and prints nothing.


WeightConverterConfig

Field Type Default Description
arch_type ArchitectureType — Target architecture
quant_type QuantType QuantType.NONE Quantization method
input_channel_group_size int 0 Input channel group size. Used only with InputChannelWiseQuant

The tensors required by each quantization method, the support range per architecture, and the constraints on input_channel_group_size are the same as in the C++ API.


Tensor and Shape

1
2
Tensor(data, shape: Shape, dtype: DataType, tensor_type: TensorType)
Shape(r: int, s: int, c: int, m: int)

Tensor takes a one-dimensional buffer (an object that supports the buffer protocol, such as bytes, bytearray, or a one-dimensional NumPy array) and keeps a copy. Modifying or releasing the original buffer after construction does not affect the tensor. The copy is allocated from the runtime host memory manager, so it counts toward the HA_HOST_MEM_MAX_BYTES limit. Flatten a multi-dimensional array to one dimension, for example with reshape(-1), before passing it.

Method Returns
get_data() Data buffer
get_shape() Shape
get_data_type() DataType
get_tensor_type() TensorType
get_size() Size in bytes

r, s, c, and m of Shape are the number of rows, columns, input channels, and output channels, and all must be greater than 0. For the data types allowed per tensor role, see Tensor in the C++ API.


Enumerations

The values are the same as the C++ enumerations; only the name spelling differs.

Enumeration Values
ArchitectureType EVT0, EVT1, Edge
QuantType NONE, OutputChannelQuant, InputChannelWiseQuant, TokenWiseQuant, MXFPQuant
DataType FP32, BF16, FP8, FP4, UINT8, SINT8, UINT4, SINT4, MXFP8, MXFP4
TensorType Weight, Bias, Scale, MXScale, WZP, OZP

Error Handling

When a conversion function fails, it raises RuntimeError. The message contains the name of the error code returned by the C++ API (for example, weight_converter: HaNotSupported). Invalid arguments to the Tensor constructor raise RuntimeError immediately.


Example

Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
import numpy as np
from hart.weight_converter import (
    WeightConverterConfig, convert,
    ArchitectureType, QuantType, DataType, Shape, Tensor, TensorType,
)

shape = Shape(1, 1, in_features, out_features)
weight = Tensor(w_u8.reshape(-1), shape, DataType.UINT8, TensorType.Weight)
scale = Tensor(scale_f32, Shape(1, 1, 1, out_features), DataType.FP32, TensorType.Scale)
wzp = Tensor(wzp_u8, Shape(1, 1, 1, out_features), DataType.UINT8, TensorType.WZP)
ozp = Tensor(ozp_u8, Shape(1, 1, 1, out_features), DataType.UINT8, TensorType.OZP)

cfg = WeightConverterConfig(arch_type=ArchitectureType.EVT1,
                            quant_type=QuantType.OutputChannelQuant)
buf = convert(cfg, weight, None, scale, None, wzp, ozp, output_data_type=DataType.UINT8)