Python Sampling API
| Function |
Signature |
set_sampling_params |
(core_id: int, entry_index: int, entry_num: int, top_p: float, top_k: int, temperature: float, repetition_penalty: float, pseudo_random: float) -> None |
set_sampling_params_async |
(core_id: int, entry_index: int, entry_num: int, top_p: float, top_k: int, temperature: float, repetition_penalty: float, pseudo_random: float, stream: int = HA_STREAM_DEFAULT) -> None |
Example
set_sampling_params
The parameters are cached on the device, so writing unchanged values again every step is wasted work.
Example
| import hart
hart.set_sampling_params(
core_id, entry_index, entry_num,
top_p=0.9, top_k=50, temperature=0.8, repetition_penalty=1.1, pseudo_random=0.0,
)
|
set_sampling_params_async
Enqueues the setting on the stream and returns immediately. It is applied before any launch enqueued after it on the
same stream. Check completion with stream_synchronize().
Example
| hart.set_sampling_params_async(
core_id, entry_index, entry_num,
top_p=0.9, top_k=50, temperature=0.8, repetition_penalty=1.1, pseudo_random=0.0,
stream=stream,
)
hart.launch_kernel_async_with_binary_path(kernel_path, args, stream)
|
See also