Skip to content

Python API

1
import hart

The semantics are the same as the C++ API; only the differences are listed in Differences from the C++ API. Device pointers are int.


Sections

Page Contents
Device Device discovery, selection, and reset
Memory Allocation, transfers, tensor layout, host and device memory statistics
Execution Kernel launch, kernel library loading, and launch by handle
Streams Ordered queues of asynchronous work
Events Stream markers and cross-stream ordering
TLB Static KV-cache TLB initialization and page management
VMM KV-cache virtual memory. On Bertha DC EVT0, use the C++ API or PyTorch operators
Sampling Token sampling parameters
Weight Converter Converts weight tensors to the device memory layout (hart.weight_converter)
Types Enumerations and structs. Import the TLB types from hart.hart

Differences from the C++ API

C++ Python
Failure reporting Returns HaError Raises RuntimeError whose message ends in the HaError name and value (... : HaKernelNotFound (err=196612))
Device pointers HaDevicePtr int
Default stream HA_STREAM_DEFAULT (nullptr) HA_STREAM_DEFAULT (0)
Host buffers HaDevicePtr cast from a host pointer bytes or any buffer-protocol object
HaLaunchKernel overloads Overloaded on binary path vs kernel handle (HaKernel_t) Separate functions: launch_kernel_with_binary_path, launch_kernel_with_handle (and their _async_ forms)
Library and kernel handles HaLibrary_t, HaKernel_t int
Launch arguments HaLaunchArgs object filled with HaLaunchArgsAdd*(), released when destroyed HaLaunchArgs object (add_*), released with the object
Memory statistics HaHostMemStats / HaDeviceMemStats struct with struct_size dict[str, int] keyed by field name

Process Exit

hart registers an atexit hook that calls reset_all_devices(), but the hook runs only if the runtime was actually initialized. A process that imports hart but never uses a device exits without opening the device.

Do not register an atexit handler that runs after this hook and touches the runtime.