Skip to content

Stream API

Ordered queues for asynchronous work. HA_STREAM_DEFAULT resolves to the calling thread's default stream; see Streams and Events.

Contents

API Description
HaStreamCreate Creates a stream bound to the current device
HaStreamDestroy Waits for the queued work, then destroys the stream
HaStreamGetDevice Returns the device the stream belongs to
HaStreamQuery Queries the completion status of the stream's work without blocking
HaStreamSynchronize Blocks the calling thread until all work in the stream completes
HaStreamWaitEvent Makes the stream wait on the specified event

HaStreamCreate

Creates a stream bound to the calling thread's current device.

Parameter Direction Default Description
stream out — Output handle.

Returns — HaSuccess if the stream was created successfully.

Signature · Example
1
HaError HaStreamCreate(HaStream_t *stream);

Example

1
2
HaStream_t stream = nullptr;
HaStreamCreate(&stream);   // on the calling thread's current device

Note

A device holds at most ten explicit streams. Create what a pipeline needs at startup and reuse them; creating one per request reaches the limit.


HaStreamDestroy

Destroys a stream, after waiting for the work queued on it. The stream is destroyed even when that work failed; in that case this call returns the failure, as HaStreamSynchronize() would, and no later call can report it.

Parameter Direction Default Description
stream in — Stream to destroy (must not be nullptr).

Returns — HaSuccess if the stream's work completed and it was destroyed, or if a device reset running concurrently owns the stream's teardown (the reset discards that work); that work's error otherwise.

Signature · Example
1
HaError HaStreamDestroy(HaStream_t stream);

Example

1
2
3
4
HaError err = HaStreamDestroy(stream);   // waits for the queued work first
if (err != HaError::HaSuccess) {
  // the queued work failed; the stream is destroyed anyway
}

Note

Do not pass nullptr (the default stream cannot be destroyed).


HaStreamGetDevice

Returns the device associated with a stream.

Parameter Direction Default Description
stream in — Stream to query; pass HA_STREAM_DEFAULT to use the calling thread's default stream on the current device.
device_id out — Output virtual device ID. If the default stream is used, it follows the same behavior as HaGetDevice().

Returns — HaSuccess if successful.

Signature · Example
1
HaError HaStreamGetDevice(HaStream_t stream, int *device_id);

Example

1
2
int device_id = -1;
HaStreamGetDevice(stream, &device_id);   // which device this stream belongs to

HaStreamQuery

Queries the completion status of a stream without blocking.

Parameter Direction Default Description
stream in HA_STREAM_DEFAULT Stream to query; omit or pass HA_STREAM_DEFAULT to use the calling thread's default stream on the current device.

Returns — HaSuccess if all enqueued work has completed; HaStreamNotReady if work is still in progress; HaStreamInvalidHandle or another HaError on failure.

Signature · Example
1
HaError HaStreamQuery(HaStream_t stream=HA_STREAM_DEFAULT);

Example

1
2
3
4
5
6
7
8
HaError status = HaStreamQuery(stream);
while (status == HaError::HaStreamNotReady) {
  DoUnrelatedHostWork();
  status = HaStreamQuery(stream);
}
if (status != HaError::HaSuccess) {
  // The stream's work failed; status carries the first error it hit.
}

HaStreamSynchronize

Blocks the calling thread until all work enqueued in the stream completes.

Parameter Direction Default Description
stream in HA_STREAM_DEFAULT Stream to synchronize; omit or pass HA_STREAM_DEFAULT to use the calling thread's default stream on the current device.

Returns — HaSuccess if all work completed successfully.

Signature · Example
1
HaError HaStreamSynchronize(HaStream_t stream=HA_STREAM_DEFAULT);

Example

1
2
3
4
5
6
HaMemCpyAsync(dev_in, host_in, n, HaMemCpyDirection::HostToDevice, stream);
HaLaunchKernelAsync(kernel_path, args, stream);
HaMemCpyAsync(host_out, dev_out, n, HaMemCpyDirection::DeviceToHost, stream);

HaStreamSynchronize(stream);
UseResult(host_out);   // only valid below this line

Note

Synchronizing a stream that holds a whole pipeline stalls the host for the whole pipeline. To wait for one point rather than all of it, record an event and wait on that.


HaStreamWaitEvent

Makes a stream wait for an event before proceeding.

Parameter Direction Default Description
stream in — Stream that will wait; pass HA_STREAM_DEFAULT to use the calling thread's default stream on the current device.
event in — Event to wait for (must have been recorded via HaEventRecord()).

Returns — HaSuccess if the dependency was inserted successfully.

Signature · Example
1
HaError HaStreamWaitEvent(HaStream_t stream, HaEvent_t event);

Example

1
2
3
4
5
HaLaunchKernelAsync(producer_path, args, producer);
HaEventRecord(done, producer);

HaStreamWaitEvent(consumer, done);   // returns at once; consumer waits on the device
HaLaunchKernelAsync(consumer_path, args2, consumer);

Note

Returns immediately on the host; only device-side execution is serialized.

Note

Waiting on an event that is never recorded blocks the waiting stream forever. Every HaStreamWaitEvent() needs a HaEventRecord() that will actually run.