콘텐츠로 이동

Weight Converter API

가중치 텐서를 bias·scale·zero-point 같은 선택적 메타데이터와 함께, 대상 아키텍처가 디바이스 메모리에서 읽는 바이트 레이아웃으로 변환합니다. 상태가 없는 API라 만들거나 해제할 핸들이 없고, 호출은 서로 독립적입니다. 크기를 조회하고, 버퍼를 할당한 뒤, 변환하는 2단계 호출로 사용합니다.

입력 규약, 디바이스 배치 흐름, 아키텍처별 지원 범위는 가중치 변환 가이드에 있습니다.

1
#include "ha_weight_converter.h"   // also includes ha_weight_converter_types.h and ha_error_code.h
1
2
find_package(hart REQUIRED COMPONENTS hw)
target_link_libraries(your_target PRIVATE hart::ha_weight_converter)

hart::ha_weight_converter는 필요한 include 디렉터리 두 개(ha_error_code.h가 있는 ${hart_INCLUDE_DIR}와 ${hart_INCLUDE_DIR}/weight_converter)를 함께 지정합니다. 모든 이름은 HaWeight 접두사를 붙여 전역 범위에 있습니다.

Contents

API Kind 설명
HaWeightGetConvertedSize Function 변환 결과 버퍼 크기 조회
HaWeightConvert Function 호출자가 할당한 버퍼로 가중치 변환
HaWeightConfig Struct 대상 아키텍처와 양자화 방식
HaWeightTensor Class 입력 텐서 (데이터, 형상, 데이터 타입, 역할)
HaWeightShape Struct 4차원 가중치 형상
HaWeightArchitectureType Enum 대상 아키텍처
HaWeightQuantType Enum 양자화 방식
HaWeightDataType Enum 원소 데이터 타입
HaWeightTensorType Enum 텐서의 역할

에러 처리

두 함수는 HaError를 반환하며, 예외를 API 경계 밖으로 내보내지 않습니다.

Code When
HaSuccess The call succeeded.
HaInvalidArgument A required pointer is null, out_buffer_size is below the queried size, a tensor the quantization type needs is missing, or input_channel_group_size is not a multiple of the architecture's DRAM word size.
HaNotSupported The quantization type is not supported on the target architecture.
HaInternalError Any other converter failure.
HaUnknownError An unexpected non-standard failure.

예외는 HaWeightTensor 생성자와 HaWeightTensor::CopyOf입니다. 이 둘은 인자를 검사해 std::runtime_error를 던지고, CopyOf는 std::bad_alloc도 던질 수 있으므로 변환 함수를 호출하기 전에 텐서를 만들어 두십시오.


함수

HaWeightGetConvertedSize

Query the converted-buffer size in bytes. Use the returned size to allocate the buffer passed to HaWeightConvert.

Parameter Direction Default Description
cfg in — Converter configuration.
weight_tensor in — Weight tensor (required).
bias_tensor in — Bias tensor (optional, may be nullptr).
output_data_type in — Data type of the matrix operation's output (see HaWeightConvert).
out_size out — Receives the total converted buffer size in bytes.

Returns — HaSuccess; HaInvalidArgument (null cfg / weight / out_size, or unknown arch); HaNotSupported (unsupported quant on this arch); HaInternalError (unexpected failure).

시그니처
1
2
3
HaError HaWeightGetConvertedSize(const HaWeightConfig *cfg, HaWeightTensor *weight_tensor,
                                 HaWeightTensor *bias_tensor, HaWeightDataType output_data_type,
                                 uint64_t *out_size);

HaWeightConvert

Convert weight (+ optional bias / quant metadata) into a caller-provided buffer. The buffer must be at least HaWeightGetConvertedSize bytes. The converter writes directly into it; no intermediate copy is made.

Parameter Direction Default Description
cfg in — Converter configuration.
weight_tensor in — Weight tensor (required).
bias_tensor in — Bias tensor (optional).
scale_tensor in — Scale tensor (depends on the quantization method).
mx_scale_tensor in — MX-scale tensor (MXFP only).
wzp_tensor in — Weight zero-point tensor (depends on the quantization method).
ozp_tensor in — Output zero-point tensor (depends on the quantization method).
output_data_type in — Data type of the matrix operation's output (not of the converted weight): kFp32, kBf16, kFp8, kUint8 or kSint8.
channel_wise_packing in — Whether to pack into DRAM channels.
out_buffer out — Caller-allocated destination buffer (>= queried size).
out_buffer_size in — Capacity of out_buffer in bytes.

Returns — HaSuccess; HaInvalidArgument (null cfg / weight / out_buffer, or out_buffer_size below the queried size); HaNotSupported (unsupported quant on this arch); HaInternalError (unexpected failure).

필요한 선택 텐서는 cfg->quant_type과 출력 데이터 타입에 따라 정해지며, 빠지면 HaInvalidArgument를 반환합니다.

Tensor Required when
scale_tensor kOutputChannelQuant, kInputChannelWiseQuant, kMxfpQuant
wzp_tensor kOutputChannelQuant, kInputChannelWiseQuant, with an integer weight data type
ozp_tensor Any quantization type other than kNone with an integer output data type
mx_scale_tensor kMxfpQuant

메타데이터 텐서의 형상은 가중치의 출력 채널 m을 기준으로 합니다. kInputChannelWiseQuant에서 groups는 입력 채널 그룹 수(r * s * c / input_channel_group_size, 올림)입니다.

Tensor Shape
bias_tensor, ozp_tensor (1, 1, 1, m)
scale_tensor, wzp_tensor (kOutputChannelQuant) (1, 1, 1, m)
scale_tensor, wzp_tensor (kInputChannelWiseQuant) (1, 1, groups, m)
시그니처 · 예제
1
2
3
4
HaError HaWeightConvert(const HaWeightConfig *cfg, HaWeightTensor *weight_tensor, HaWeightTensor *bias_tensor,
                        HaWeightTensor *scale_tensor, HaWeightTensor *mx_scale_tensor, HaWeightTensor *wzp_tensor,
                        HaWeightTensor *ozp_tensor, HaWeightDataType output_data_type, bool channel_wise_packing,
                        void *out_buffer, uint64_t out_buffer_size);

예제

1
2
3
4
5
6
7
8
9
HaWeightConfig cfg{HaWeightArchitectureType::kEvt1, HaWeightQuantType::kOutputChannelQuant};

uint64_t size = 0;
HaWeightGetConvertedSize(&cfg, &weight, &bias, HaWeightDataType::kUint8, &size);

std::vector<uint8_t> buf(size);
HaError err = HaWeightConvert(&cfg, &weight, &bias, &scale, nullptr, &wzp, &ozp,
                              HaWeightDataType::kUint8, /*channel_wise_packing=*/true,
                              buf.data(), buf.size());

구조체와 클래스

HaWeightConfig

Configuration passed on every call.

Field Type Default Description
arch_type HaWeightArchitectureType — Target architecture.
quant_type HaWeightQuantType — Quantization method.
input_channel_group_size int 0 Input channel group size (0 if unused).

input_channel_group_size는 kInputChannelWiseQuant에만 쓰이며, 아키텍처의 DRAM 워드 크기(kEvt0·kEvt1은 128, kEdge는 64)의 배수여야 합니다. kMxfpQuant는 이 값을 무시하고 DRAM 워드 크기를 사용합니다.


HaWeightTensor

An input tensor: data pointer, HaWeightShape, HaWeightDataType and HaWeightTensorType. Move-only.

Member Description
HaWeightTensor(void *data, HaWeightShape shape, HaWeightDataType dtype, HaWeightTensorType tensor_type) Borrows data: the caller must keep the buffer alive while the tensor is used.
static HaWeightTensor CopyOf(const void *src, size_t bytes, HaWeightShape shape, HaWeightDataType dtype, HaWeightTensorType tensor_type) Owns a copy of bytes bytes at src, allocated from the runtime's host memory manager.
void *GetData() const The data pointer.
HaWeightShape GetShape() const The shape.
HaWeightDataType GetDataType() const The data type.
HaWeightTensorType GetTensorType() const The tensor's role.
uint64_t GetSize() const Size in bytes (r * s * c * m elements at the element bit width, rounded up).

두 생성 방법 모두 데이터 포인터가 null이거나, 차원 중 0이 있거나, 텐서 역할이 받지 않는 데이터 타입이면 std::runtime_error를 던집니다. CopyOf는 bytes가 0이거나 형상과 데이터 타입에 필요한 크기(GetSize())보다 작을 때도 std::runtime_error를, 호스트 메모리 관리자가 할당을 거부하면(HA_HOST_MEM_MAX_BYTES 한도 또는 시스템) std::bad_alloc을 던집니다.

HaWeightTensorType Accepted HaWeightDataType
kWeight kBf16, kFp8, kFp4, kUint8, kSint8, kUint4, kSint4, kMxfp8, kMxfp4
kBias kFp32
kScale kFp32
kMxScale kMxfp8, kMxfp4
kWzp kUint8, kSint8, kUint4, kSint4
kOzp kUint8, kSint8

HaWeightShape

Shape for representing 4 dimensional weight data. Every dimension must be non-zero.

Field Type Description
r uint64_t The number of rows.
s uint64_t The number of columns.
c uint64_t The number of input channels.
m uint64_t The number of output channels.

열거형

HaWeightArchitectureType

Value Target
kEvt0 Bertha Cloud EVT0
kEvt1 Bertha Cloud EVT1
kEdge Bertha Edge EVT0

HaWeightQuantType

Value Description
kNone No quantization.
kOutputChannelQuant Per-output-channel quantization.
kInputChannelWiseQuant Grouped input-channel-wise quantization (input_channel_group_size).
kTokenWiseQuant Not supported on any architecture; returns HaNotSupported.
kMxfpQuant MXFP quantization. Supported on kEvt1 and kEdge; kEvt0 returns HaNotSupported.

HaWeightDataType

Value Bit width
kFp32 32
kBf16 16
kFp8 8
kFp4 4
kUint8 8
kSint8 8
kUint4 4
kSint4 4
kMxfp8 8
kMxfp4 4

HaWeightTensorType

Value Description
kWeight Weight.
kBias Bias.
kScale Scale.
kMxScale MX scale (MXFP only).
kWzp Weight zero point.
kOzp Output zero point.