Skip to content

Python Streams API

Function Signature Notes
stream_create () -> int Created on the current device. Up to 10 per device
stream_destroy (stream: int) -> None Waits for the queued work, then destroys. If that work failed, raises after destroying the stream
stream_synchronize (stream: int = HA_STREAM_DEFAULT) -> None Blocks
stream_query (stream: int = HA_STREAM_DEFAULT) -> bool True when complete, False while in progress. Raises only on a runtime error
stream_wait_event (stream: int, event: int) -> None Device-side dependency. Returns immediately
stream_get_device (stream: int) -> int

HA_STREAM_DEFAULT is 0 and resolves to the default stream of the calling thread.


Example

stream_create

Each device allows up to 10 explicit streams.

Create as many as the pipeline needs at startup and reuse them. Creating one per request hits the limit. Per-thread default streams do not count toward this limit.

Example
1
2
3
4
import hart

hart.set_device(0)
streams = [hart.stream_create() for _ in range(pipeline_depth)]

stream_destroy

Waits for the work queued on the stream, then destroys it. The stream is destroyed even when that work failed; the failure is raised here, as stream_synchronize() would have raised it, and nothing after this call can report it.

Example
1
2
3
4
5
6
import hart

try:
    hart.stream_destroy(stream)  # waits for the queued work, then destroys
except RuntimeError:
    pass  # the queued work failed; the stream is destroyed anyway

stream_get_device

Checks which device a stream belongs to.

Example
1
2
3
import hart

device_id = hart.stream_get_device(stream)

stream_query

Non-blocking polling. A stream with work in progress returns False rather than raising; it raises only on a runtime error.

Example
1
2
3
4
5
6
7
import hart

stream = hart.stream_create()
hart.mem_cpy_async(dev, host, size, hart.HaMemCpyDirection.HostToDevice, stream)

while not hart.stream_query(stream):  # True: everything enqueued has completed
    do_unrelated_host_work()

stream_synchronize

The synchronization point that asynchronous calls rely on.

Synchronizing on a stream that holds the whole pipeline stalls the host until the whole pipeline finishes. To wait for a single point, record an event and wait on it.

Example
1
2
3
4
5
6
7
import hart

hart.mem_cpy_async(dev_in, host_in, n, hart.HaMemCpyDirection.HostToDevice, stream)
hart.launch_kernel_async_with_binary_path(kernel_path, args, stream)

hart.stream_synchronize(stream)
use(host_out)  # only valid below this line

stream_wait_event

Device-side dependency. It returns immediately, so the host keeps enqueuing.

Waiting on an event that is never recorded blocks the waiting stream forever.

Example
1
2
3
4
5
6
7
import hart

hart.launch_kernel_async_with_binary_path(producer_path, args, producer)
hart.event_record(done, producer)

hart.stream_wait_event(consumer, done)  # returns immediately
hart.launch_kernel_async_with_binary_path(consumer_path, args2, consumer)

See also