Weight Converter API
가중치 텐서를 bias·scale·zero-point 같은 선택적 메타데이터와 함께, 대상 아키텍처가 디바이스 메모리에서 읽는
바이트 레이아웃으로 변환합니다. 상태가 없는 API라 만들거나 해제할 핸들이 없고, 호출은 서로 독립적입니다. 크기를
조회하고, 버퍼를 할당한 뒤, 변환하는 2단계 호출로 사용합니다.
입력 규약, 디바이스 배치 흐름, 아키텍처별 지원 범위는 가중치 변환
가이드에 있습니다.
| #include "ha_weight_converter.h" // also includes ha_weight_converter_types.h and ha_error_code.h
|
| find_package(hart REQUIRED COMPONENTS hw)
target_link_libraries(your_target PRIVATE hart::ha_weight_converter)
|
hart::ha_weight_converter는 필요한 include 디렉터리 두 개(ha_error_code.h가 있는 ${hart_INCLUDE_DIR}와
${hart_INCLUDE_DIR}/weight_converter)를 함께 지정합니다. 모든 이름은 HaWeight 접두사를 붙여 전역 범위에 있습니다.
Contents
에러 처리
두 함수는 HaError를 반환하며, 예외를 API 경계 밖으로 내보내지 않습니다.
| Code |
When |
HaSuccess |
The call succeeded. |
HaInvalidArgument |
A required pointer is null, out_buffer_size is below the queried size, a tensor the quantization type needs is missing, or input_channel_group_size is not a multiple of the architecture's DRAM word size. |
HaNotSupported |
The quantization type is not supported on the target architecture. |
HaInternalError |
Any other converter failure. |
HaUnknownError |
An unexpected non-standard failure. |
예외는 HaWeightTensor 생성자와 HaWeightTensor::CopyOf입니다. 이 둘은 인자를 검사해
std::runtime_error를 던지고, CopyOf는 std::bad_alloc도 던질 수 있으므로 변환 함수를 호출하기 전에 텐서를
만들어 두십시오.
함수
HaWeightGetConvertedSize
Query the converted-buffer size in bytes. Use the returned size to allocate the buffer passed to
HaWeightConvert.
| Parameter |
Direction |
Default |
Description |
cfg |
in |
— |
Converter configuration. |
weight_tensor |
in |
— |
Weight tensor (required). |
bias_tensor |
in |
— |
Bias tensor (optional, may be nullptr). |
output_data_type |
in |
— |
Data type of the matrix operation's output (see HaWeightConvert). |
out_size |
out |
— |
Receives the total converted buffer size in bytes. |
Returns — HaSuccess; HaInvalidArgument (null cfg / weight / out_size, or unknown arch);
HaNotSupported (unsupported quant on this arch); HaInternalError (unexpected failure).
시그니처
| HaError HaWeightGetConvertedSize(const HaWeightConfig *cfg, HaWeightTensor *weight_tensor,
HaWeightTensor *bias_tensor, HaWeightDataType output_data_type,
uint64_t *out_size);
|
HaWeightConvert
Convert weight (+ optional bias / quant metadata) into a caller-provided buffer. The buffer must be at least
HaWeightGetConvertedSize bytes. The converter writes directly into
it; no intermediate copy is made.
| Parameter |
Direction |
Default |
Description |
cfg |
in |
— |
Converter configuration. |
weight_tensor |
in |
— |
Weight tensor (required). |
bias_tensor |
in |
— |
Bias tensor (optional). |
scale_tensor |
in |
— |
Scale tensor (depends on the quantization method). |
mx_scale_tensor |
in |
— |
MX-scale tensor (MXFP only). |
wzp_tensor |
in |
— |
Weight zero-point tensor (depends on the quantization method). |
ozp_tensor |
in |
— |
Output zero-point tensor (depends on the quantization method). |
output_data_type |
in |
— |
Data type of the matrix operation's output (not of the converted weight): kFp32, kBf16, kFp8, kUint8 or kSint8. |
channel_wise_packing |
in |
— |
Whether to pack into DRAM channels. |
out_buffer |
out |
— |
Caller-allocated destination buffer (>= queried size). |
out_buffer_size |
in |
— |
Capacity of out_buffer in bytes. |
Returns — HaSuccess; HaInvalidArgument (null cfg / weight / out_buffer, or out_buffer_size below
the queried size); HaNotSupported (unsupported quant on this arch); HaInternalError (unexpected failure).
필요한 선택 텐서는 cfg->quant_type과 출력 데이터 타입에 따라 정해지며, 빠지면 HaInvalidArgument를 반환합니다.
| Tensor |
Required when |
scale_tensor |
kOutputChannelQuant, kInputChannelWiseQuant, kMxfpQuant |
wzp_tensor |
kOutputChannelQuant, kInputChannelWiseQuant, with an integer weight data type |
ozp_tensor |
Any quantization type other than kNone with an integer output data type |
mx_scale_tensor |
kMxfpQuant |
메타데이터 텐서의 형상은 가중치의 출력 채널 m을 기준으로 합니다. kInputChannelWiseQuant에서 groups는 입력
채널 그룹 수(r * s * c / input_channel_group_size, 올림)입니다.
| Tensor |
Shape |
bias_tensor, ozp_tensor |
(1, 1, 1, m) |
scale_tensor, wzp_tensor (kOutputChannelQuant) |
(1, 1, 1, m) |
scale_tensor, wzp_tensor (kInputChannelWiseQuant) |
(1, 1, groups, m) |
시그니처 · 예제
| HaError HaWeightConvert(const HaWeightConfig *cfg, HaWeightTensor *weight_tensor, HaWeightTensor *bias_tensor,
HaWeightTensor *scale_tensor, HaWeightTensor *mx_scale_tensor, HaWeightTensor *wzp_tensor,
HaWeightTensor *ozp_tensor, HaWeightDataType output_data_type, bool channel_wise_packing,
void *out_buffer, uint64_t out_buffer_size);
|
예제
| HaWeightConfig cfg{HaWeightArchitectureType::kEvt1, HaWeightQuantType::kOutputChannelQuant};
uint64_t size = 0;
HaWeightGetConvertedSize(&cfg, &weight, &bias, HaWeightDataType::kUint8, &size);
std::vector<uint8_t> buf(size);
HaError err = HaWeightConvert(&cfg, &weight, &bias, &scale, nullptr, &wzp, &ozp,
HaWeightDataType::kUint8, /*channel_wise_packing=*/true,
buf.data(), buf.size());
|
구조체와 클래스
HaWeightConfig
Configuration passed on every call.
| Field |
Type |
Default |
Description |
arch_type |
HaWeightArchitectureType |
— |
Target architecture. |
quant_type |
HaWeightQuantType |
— |
Quantization method. |
input_channel_group_size |
int |
0 |
Input channel group size (0 if unused). |
input_channel_group_size는 kInputChannelWiseQuant에만 쓰이며, 아키텍처의 DRAM 워드 크기(kEvt0·kEvt1은
128, kEdge는 64)의 배수여야 합니다. kMxfpQuant는 이 값을 무시하고 DRAM 워드 크기를 사용합니다.
HaWeightTensor
An input tensor: data pointer, HaWeightShape, HaWeightDataType and HaWeightTensorType. Move-only.
| Member |
Description |
HaWeightTensor(void *data, HaWeightShape shape, HaWeightDataType dtype, HaWeightTensorType tensor_type) |
Borrows data: the caller must keep the buffer alive while the tensor is used. |
static HaWeightTensor CopyOf(const void *src, size_t bytes, HaWeightShape shape, HaWeightDataType dtype, HaWeightTensorType tensor_type) |
Owns a copy of bytes bytes at src, allocated from the runtime's host memory manager. |
void *GetData() const |
The data pointer. |
HaWeightShape GetShape() const |
The shape. |
HaWeightDataType GetDataType() const |
The data type. |
HaWeightTensorType GetTensorType() const |
The tensor's role. |
uint64_t GetSize() const |
Size in bytes (r * s * c * m elements at the element bit width, rounded up). |
두 생성 방법 모두 데이터 포인터가 null이거나, 차원 중 0이 있거나, 텐서 역할이 받지 않는 데이터 타입이면
std::runtime_error를 던집니다. CopyOf는 bytes가 0이거나 형상과 데이터 타입에 필요한 크기(GetSize())보다
작을 때도 std::runtime_error를, 호스트 메모리 관리자가 할당을 거부하면(HA_HOST_MEM_MAX_BYTES 한도 또는 시스템)
std::bad_alloc을 던집니다.
HaWeightTensorType |
Accepted HaWeightDataType |
kWeight |
kBf16, kFp8, kFp4, kUint8, kSint8, kUint4, kSint4, kMxfp8, kMxfp4 |
kBias |
kFp32 |
kScale |
kFp32 |
kMxScale |
kMxfp8, kMxfp4 |
kWzp |
kUint8, kSint8, kUint4, kSint4 |
kOzp |
kUint8, kSint8 |
HaWeightShape
Shape for representing 4 dimensional weight data. Every dimension must be non-zero.
| Field |
Type |
Description |
r |
uint64_t |
The number of rows. |
s |
uint64_t |
The number of columns. |
c |
uint64_t |
The number of input channels. |
m |
uint64_t |
The number of output channels. |
열거형
HaWeightArchitectureType
| Value |
Target |
kEvt0 |
Bertha Cloud EVT0 |
kEvt1 |
Bertha Cloud EVT1 |
kEdge |
Bertha Edge EVT0 |
HaWeightQuantType
| Value |
Description |
kNone |
No quantization. |
kOutputChannelQuant |
Per-output-channel quantization. |
kInputChannelWiseQuant |
Grouped input-channel-wise quantization (input_channel_group_size). |
kTokenWiseQuant |
Not supported on any architecture; returns HaNotSupported. |
kMxfpQuant |
MXFP quantization. Supported on kEvt1 and kEdge; kEvt0 returns HaNotSupported. |
HaWeightDataType
| Value |
Bit width |
kFp32 |
32 |
kBf16 |
16 |
kFp8 |
8 |
kFp4 |
4 |
kUint8 |
8 |
kSint8 |
8 |
kUint4 |
4 |
kSint4 |
4 |
kMxfp8 |
8 |
kMxfp4 |
4 |
HaWeightTensorType
| Value |
Description |
kWeight |
Weight. |
kBias |
Bias. |
kScale |
Scale. |
kMxScale |
MX scale (MXFP only). |
kWzp |
Weight zero point. |
kOzp |
Output zero point. |