4. Streams and Events¶
Every asynchronous operation in the driver API is enqueued in order on a stream. To wait for completion or query status
from the host, use ha_drv_wait for streams, events, and devices alike. Streams and events are created and destroyed
with the same functions (ha_drv_obj_create / ha_drv_obj_destroy), and stream operations that have no dedicated
function are enqueued with ha_drv_stream_post.
Contents¶
| Section | Objective |
|---|---|
| 4.1 Stream | Stream creation, the default stream (HA_STREAM_DEFAULT), count limit, and stream operations |
| 4.2 Event | Event recording and the rules for cross-stream dependency waits |
| 4.3 Synchronization | The five synchronization modes of ha_drv_wait and completion checks based on sequence numbers (seq) |
4.1 Stream¶
A stream is an ordered queue in which work is processed serially on the device. Kernel launches and memory copies enqueued on the same stream run in the order they were enqueued. Across different streams, execution order is not guaranteed unless the streams are linked with events.
1 2 3 4 5 6 7 8 9 10 | |
| API | Sync / Async | Behavior |
|---|---|---|
ha_drv_obj_create() + HA_OBJ_STREAM |
Sync | Creates an explicit stream on the device |
ha_drv_obj_destroy() + HA_OBJ_STREAM |
Sync | Waits until the work left on the stream finishes, then destroys it |
ha_drv_stream_post() |
Async | Adds one stream operation (HaDrvStreamOp) to the queue |
ha_drv_wait() + HA_WAIT_STREAM_SYNC |
Sync | Waits on the host until all enqueued work completes |
ha_drv_wait() + HA_WAIT_STREAM_QUERY |
Sync | Returns the stream error if there is one; otherwise HaSuccess if idle, HaStreamNotReady if in progress |
Default Stream¶
Passing HA_STREAM_DEFAULT as the stream argument refers to the default stream that the calling thread uses on that
device (equivalent to CUDA's CU_STREAM_PER_THREAD). The default stream is created automatically on first use. When the
thread exits, the stream is reclaimed after its remaining work has finished; until then, HA_WAIT_DEVICE_SYNC still
synchronizes it.
- Per-thread independence: If two different threads each use
HA_STREAM_DEFAULT, they are different streams and execution order is not guaranteed. If you need ordering across threads, share an explicit stream or synchronize with events. - Relationship to the runtime API: It is the same stream as the default stream the runtime API uses for the same thread and device in the same process. However, do not mix the two APIs on one device (see Section 1.2).
Stream Count Limit¶
A device can hold up to 10 explicit streams at a time. Default streams do not count toward this limit; streams
created with the runtime API on the same device do. Once the limit is reached, ha_drv_obj_create returns
HaStreamCreationFailed.
Stream Operations¶
Stream operations other than kernel launches, including memory copies, are enqueued with ha_drv_stream_post by putting the
operation type and payload in a HaDrvStreamOp structure.
| Operation type | Behavior |
|---|---|
HA_STREAM_OP_MEMCPY |
Copies data between the host and the device, or within the device |
HA_STREAM_OP_FREE |
Returns memory asynchronously after the preceding work completes (see Section 3.1) |
HA_STREAM_OP_LOAD_FILE |
Loads directly by DMA from a file descriptor into device memory (see Section 3.3) |
HA_STREAM_OP_MARKER |
Consumes a sequence number (seq) and does nothing else. Used as a synchronization reference point |
HA_STREAM_OP_SET_SAMPLING |
Sets token sampling parameters in stream order, before subsequent kernel launches |
HA_STREAM_OP_RECORD_EVENT |
Records an event at the current point of the stream (see Section 4.2) |
HA_STREAM_OP_WAIT_EVENT |
Holds subsequent work on this stream until the event completes (see Section 4.2) |
1 2 3 4 5 | |
4.2 Event¶
An event is an object that marks a specific execution point in a stream. When one stream records an event, another
stream or the host can wait until the work up to that point completes. Driver events (HaDrvEvent) are separate objects from runtime
events (HaEvent_t).
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 | |
| API | Sync / Async | Behavior |
|---|---|---|
ha_drv_obj_create() + HA_OBJ_EVENT |
Sync | Creates an event object |
ha_drv_stream_post() + HA_STREAM_OP_RECORD_EVENT |
Async | Adds an event record operation to the stream |
ha_drv_stream_post() + HA_STREAM_OP_WAIT_EVENT |
Async | Makes the stream's later work wait for this event. The host does not wait |
ha_drv_wait() + HA_WAIT_EVENT_SYNC |
Sync | Waits on the host until the work up to the recorded point completes |
ha_drv_wait() + HA_WAIT_EVENT_QUERY |
Sync | HaSuccess if complete, HaEventNotReady if in progress |
ha_drv_obj_destroy() + HA_OBJ_EVENT |
Sync | Destroys the event. If it was recorded, waits until the record operation reaches the device, then releases it |
Event Record and Wait Rules¶
- Single record: An event can be recorded only once in its lifetime. Recording the same event again is rejected
with
HaEventInvalidArgument. - No self-wait: Enqueuing a wait on a stream for its own recorded event would never finish, so it is rejected
with
HaStreamInvalidArgument. - No wait before record: Waiting on an event that has never been recorded, with a stream operation or with
ha_drv_wait, returnsHaEventNotRecorded.
4.3 Synchronization¶
Host-side completion waits and status queries go through ha_drv_wait. The kind argument determines the wait target
and whether the call blocks. Stream modes take only stream, and event modes take only event; leave the other argument
as HA_STREAM_DEFAULT or nullptr. The device mode leaves both empty. Any other combination returns
HaInvalidArgument. timeout_ms applies only to the SYNC modes; a negative value waits without a host-side time limit. With
HA_WAIT_STREAM_SYNC, the last argument of ha_drv_wait(), out_synced_seq, optionally returns the sequence number this synchronization took.
Synchronization mode (kind) |
Target | Behavior |
|---|---|---|
HA_WAIT_STREAM_SYNC |
Stream | Blocks until all work enqueued before the call completes |
HA_WAIT_STREAM_QUERY |
Stream | Queries status without blocking. Returns the stream error if there is one; otherwise HaSuccess if complete, HaStreamNotReady if in progress |
HA_WAIT_EVENT_SYNC |
Event | Blocks until the work at the recorded point completes |
HA_WAIT_EVENT_QUERY |
Event | Queries status without blocking. HaSuccess if complete, HaEventNotReady if pending |
HA_WAIT_DEVICE_SYNC |
Device | Synchronizes every stream on the device at the time of the call: explicit streams, the default streams of all threads, and the default streams of exited threads. Returns the first error |
Note: Work enqueued after the call
In every synchronization mode, work newly enqueued after the ha_drv_wait call is not included in the wait.
Completion Checks with Sequence Numbers (seq)¶
Compare the sequence number returned by ha_drv_stream_post in out_seq with the sequence number returned by
ha_drv_wait(HA_WAIT_STREAM_SYNC) in out_synced_seq to determine whether a given stream operation has completed.
If the synchronization returned HaSuccess, every operation with a seq smaller than out_synced_seq has completed.
out_synced_seq is written before the wait, so it is filled in even when the synchronization fails. An operation that
fails to enqueue does not consume a sequence number.
Sequence numbers of stream operations and the out_synced_seq that ha_drv_wait returns
1 2 3 4 5 6 7 | |
Next Steps¶
- Error handling and reset — Stream errors and device reset
- Driver API: Streams & Events — Per-function signatures