Skip to content

Driver API: TLB

Static KV-cache TLB and page-table TLB configuration. See Driver API for the concepts.

Contents

Name Kind Description
ha_drv_init_tlb API Initialize the static KV-cache TLB
ha_drv_alloc_tlb_page API Allocate a TLB page
ha_drv_dealloc_tlb_page API Free a TLB page
ha_drv_configure_tlb API Configure the page-table TLB
ha_drv_set_page_table_range API Set the page table address and size
ha_drv_flush_tlb API Invalidate the whole TLB cache

ha_drv_init_tlb

Initialize the static KV-cache TLB on dev from the geometry in info.

Parameter Direction Default Description
dev in — Device handle from ha_drv_open.
info in — TLB init info (non-null).

Returns — HaSuccess; HaInvalidArgument if dev/info is null; HaDeviceNotReady if a reset is in progress; or the device's TLB error.

Signature · Example
1
HaError ha_drv_init_tlb(HaDrvDev dev, const TLBInitInfo *info);

Example

1
2
3
4
5
6
7
TLBInitInfo info{};
info.page_size = page_size;
info.base_addr = kv_base;
info.layer_num = num_layers;
info.head_num = num_kv_heads;
info.max_page_num = max_pages;
ha_drv_init_tlb(dev, &info);

ha_drv_alloc_tlb_page

Allocate a TLB page on dev. Backs the runtime's TLB API.

Parameter Direction Default Description
dev in — Device handle from ha_drv_open.
tlb_alloc_info in — TLB alloc info.

Returns — HaSuccess; HaInvalidArgument if dev/tlb_alloc_info is null; HaDeviceNotReady if a reset is in progress; or the device's TLB error.

Signature · Example
1
HaError ha_drv_alloc_tlb_page(HaDrvDev dev, const TLBAllocInfo *tlb_alloc_info);

Example

1
2
3
4
5
TLBAllocInfo alloc{};
alloc.req_id = request_id;
alloc.page_num = pages_needed;
alloc.page_type = PageType::KEY;
ha_drv_alloc_tlb_page(dev, &alloc);

ha_drv_dealloc_tlb_page

Free a TLB page on dev. Backs the runtime's TLB API.

Parameter Direction Default Description
dev in — Device handle from ha_drv_open.
tlb_dealloc_info in — TLB dealloc info.

Returns — HaSuccess; HaInvalidArgument if dev/tlb_dealloc_info is null; HaDeviceNotReady if a reset is in progress; or the device's TLB error.

Signature · Example
1
HaError ha_drv_dealloc_tlb_page(HaDrvDev dev, const TLBDeallocInfo *tlb_dealloc_info);

Example

1
2
3
4
5
TLBDeallocInfo dealloc{};
dealloc.req_id = request_id;
dealloc.page_num = pages_to_release;
dealloc.page_type = PageType::KEY;
ha_drv_dealloc_tlb_page(dev, &dealloc);   // return it when the request finishes

ha_drv_configure_tlb

Configure the page-table TLB on dev. (Re)configuration always invalidates the TLB cache; use ha_drv_flush_tlb for standalone invalidation.

Parameter Direction Default Description
dev in — Device handle from ha_drv_open.
auto_prefetch_enable in — Non-zero to enable auto-prefetch.
auto_prefetch_len_x in — Auto-prefetch length along the X dimension.
auto_prefetch_len_y in — Auto-prefetch length along the Y dimension.
auto_prefetch_stride_y in — Auto-prefetch stride along the Y dimension.
transl_time_limit in — Address-translation time limit in cycles; 0 means no limit.
page_size in — Page size in bytes.
shared_channel_number in — Number of shared DRAM channels.

Returns — HaSuccess; HaInvalidArgument if dev is null; HaDeviceNotReady if a reset is in progress; or the device's TLB error.

Signature · Example
1
2
3
HaError ha_drv_configure_tlb(HaDrvDev dev, uint8_t auto_prefetch_enable, uint16_t auto_prefetch_len_x,
                             uint16_t auto_prefetch_len_y, uint16_t auto_prefetch_stride_y, uint32_t transl_time_limit,
                             uint32_t page_size, uint32_t shared_channel_number);

Example

1
2
3
ha_drv_configure_tlb(dev, /*auto_prefetch_enable=*/1, /*auto_prefetch_len_x=*/4, /*auto_prefetch_len_y=*/1,
                     /*auto_prefetch_stride_y=*/0, /*transl_time_limit=*/0, /*page_size=*/64 * 1024,
                     /*shared_channel_number=*/channels);

ha_drv_set_page_table_range

Set the page-table base address and size on dev (page-table TLB).

Parameter Direction Default Description
dev in — Device handle from ha_drv_open.
page_table_base_addr in — Physical base address of the page table.
page_table_size in — Size in bytes of the page-table region.
effective_vpn_width in — Number of significant bits in the virtual page number.
cache_invalidate in — Non-zero to invalidate the TLB cache once after updating the range.

Returns — HaSuccess; HaInvalidArgument if dev is null; HaDeviceNotReady if a reset is in progress; or the device's TLB error.

Signature · Example
1
2
HaError ha_drv_set_page_table_range(HaDrvDev dev, size_t page_table_base_addr, size_t page_table_size,
                                    uint32_t effective_vpn_width, uint8_t cache_invalidate);

Example

1
2
ha_drv_set_page_table_range(dev, page_table_base, page_table_bytes, /*effective_vpn_width=*/20,
                            /*cache_invalidate=*/1);

ha_drv_flush_tlb

Flush (invalidate) the entire TLB cache on dev (page-table TLB).

Parameter Direction Default Description
dev in — Device handle from ha_drv_open.

Returns — HaSuccess; HaInvalidArgument if dev is null; HaDeviceNotReady if a reset is in progress; or the device's TLB error.

Signature · Example
1
HaError ha_drv_flush_tlb(HaDrvDev dev);

Example

1
ha_drv_flush_tlb(dev);                  // invalidate the TLB cache after changing the page table