콘텐츠로 이동

Device API

Device discovery, selection, and reset. The current device is set per thread; see Multi-Device Programming.

Contents

API 설명
HaDeviceSynchronize 현재 디바이스의 모든 스트림 작업이 끝날 때까지 대기
HaGetDevice 현재 선택된 디바이스의 가상 ID 조회
HaGetDeviceAttribute 디바이스 속성 하나 읽기 (PCI 위치, 피어, DRAM 채널, 칩 세대)
HaGetDeviceCount 시스템에서 사용 가능한 디바이스 개수 조회
HaIsRuntimeConstructed 런타임 인스턴스 생성 여부 확인
HaResetAllDevices 모든 디바이스의 런타임 상태 일괄 리셋
HaResetDevice 지정한 단일 디바이스의 런타임 상태 리셋
HaSetDevice 호출 스레드의 현재 디바이스 지정

HaDeviceSynchronize

Blocks the calling thread until all work enqueued on the current device completes.

Returns — HaSuccess if every stream completed successfully; otherwise the first error reported. Every stream is drained even after a failure, so a later stream's error is logged rather than returned; it remains retrievable from that stream itself. HaDeviceNotReady if a device reset ran during the call: the work waited on was discarded, not completed, and the reset freed the streams that would otherwise carry the error.

시그니처 · 예제
1
HaError HaDeviceSynchronize();

예제

1
2
3
4
HaSetDevice(0);
HaLaunchKernelAsync(path, args, stream_a);
HaLaunchKernelAsync(path, args, stream_b);
HaDeviceSynchronize();   // waits for every stream of device 0, including other threads' default streams

Note

This is not the same as HaStreamSynchronize(HA_STREAM_DEFAULT). The default stream is per-thread, so synchronizing it waits only for the calling thread's own work. Use this entry point when the requirement is "the device is idle" rather than "my thread is idle".

Note

Operates on the device selected with HaSetDevice() on this thread. It covers every stream the device has when the call starts -- explicit streams, every thread's default stream, and streams created through the driver API on the same device. A stream created after the call starts is not covered, as with cudaDeviceSynchronize.


HaGetDevice

Gets the currently selected user virtual device ID.

Parameter Direction Default Description
device_id out — Output pointer to receive the current virtual ID (required).

Returns — Success or an error code.

시그니처 · 예제
1
HaError HaGetDevice(int *device_id);

예제

1
2
int current = -1;
HaGetDevice(&current);   // fails with HaDeviceInvalidArgument if this thread never set one

HaGetDeviceAttribute

Reads one scalar attribute of a device. See HaDeviceAttr for the attributes.

Parameter Direction Default Description
value out — Receives the attribute; written only on success (required).
attr in — Which attribute to read. One of HaDeviceAttr::PciDomain, HaDeviceAttr::PciBus, HaDeviceAttr::PciSlot, HaDeviceAttr::PciFunc, HaDeviceAttr::PeerPresentMask, HaDeviceAttr::PeerMappedMask, HaDeviceAttr::DramChannels, HaDeviceAttr::DramChannelStride, HaDeviceAttr::Version (see HaDeviceAttr).
device_id in — Virtual device ID in the range [0, HaGetDeviceCount()).

Returns — HaSuccess; HaDeviceInvalidArgument if value is null, the ID is out of range, or attr is unknown; HaNotSupported if the device cannot report this attribute.

시그니처 · 예제
1
HaError HaGetDeviceAttribute(int64_t *value, HaDeviceAttr attr, int device_id);

예제

1
2
3
4
5
6
7
8
9
int64_t bus = 0;
if (HaGetDeviceAttribute(&bus, HaDeviceAttr::PciBus, /*device_id=*/0) == HaError::HaSuccess) {
  // bus is the PCI bus number of device 0
}

int64_t raw_version = 0;
if (HaGetDeviceAttribute(&raw_version, HaDeviceAttr::Version, /*device_id=*/0) == HaError::HaSuccess) {
  auto version = static_cast<HaDeviceVersion>(raw_version);   // select the weight layout for this chip
}

Note

An attribute the device cannot report returns HaNotSupported rather than a placeholder: a PCI address of 0000:00:00.0 is legal and would be indistinguishable from a real one. Treat HaNotSupported as "unknown", not "zero".

Note

HaDeviceAttr::Version reports the chip generation as a HaDeviceVersion value. Packed weight layouts and ABI selection are chip-specific and the chip cannot be recovered from any buffer contents, so a caller that needs it must ask the runtime rather than guess.


HaGetDeviceCount

Returns the number of available devices.

Parameter Direction Default Description
count out — Output pointer to receive the device count (required).

Returns — Success or an error code.

시그니처 · 예제
1
HaError HaGetDeviceCount(int *count);

예제

1
2
3
4
5
6
7
int count = 0;
if (HaGetDeviceCount(&count) != HaError::HaSuccess || count == 0) {
  return NoDeviceAvailable();
}
for (int id = 0; id < count; ++id) {   // valid IDs are [0, count)
  HaSetDevice(id);
}

Note

Use this to discover the valid device_id range before calling HaSetDevice().


HaIsRuntimeConstructed

Reports whether the runtime has been constructed yet. Unlike every other Ha* entry point, this does not construct the runtime as a side effect, so it is safe to call speculatively (e.g. from a process-exit cleanup hook) in a process that may never have used the runtime.

Parameter Direction Default Description
constructed out — Set to true if the runtime has already been constructed.

Returns — Success or an error code.

시그니처 · 예제
1
HaError HaIsRuntimeConstructed(bool *constructed);

예제

1
2
3
4
5
bool constructed = false;
HaIsRuntimeConstructed(&constructed);
if (constructed) {
  HaResetAllDevices();
}

HaResetAllDevices

Resets runtime state for all devices.

Returns — Success or an error code.

시그니처 · 예제
1
HaError HaResetAllDevices();

예제

1
2
StopWorkers();          // quiesce first, or a worker touches a device mid-reset
HaResetAllDevices();

Note

Useful for test/benchmark loops that need a clean slate between iterations.

Note

Process-wide: it also resets devices that other threads are using. The runtime does not stop those threads, so stop them first.


HaResetDevice

Resets runtime state for a single device.

Parameter Direction Default Description
device_id in — Virtual device ID in the range [0, HaGetDeviceCount()).

Returns — Success or an error code.

시그니처 · 예제
1
HaError HaResetDevice(const int device_id);

예제

1
2
3
4
5
HaStreamSynchronize(stream);
HaStreamDestroy(stream);
HaMemFree(buf);

HaResetDevice(0);   // any handle still held from before this line is now stale

Note

This may invalidate runtime-managed resources associated with the device. Ensure your code does not keep using device allocations/handles that are no longer valid after a reset.

Note

Synchronize every stream holding a cross-device copy to or from this device first: a stream on the other device is not waited for (see HaMemCpyAsync()).


HaSetDevice

Sets the current device for the calling thread.

Parameter Direction Default Description
device_id in — Virtual device ID in the range [0, HaGetDeviceCount()).

Returns — Success or an error code.

시그니처 · 예제
1
HaError HaSetDevice(const int device_id);

예제

1
2
3
4
5
void Worker(int device_id) {
  HaSetDevice(device_id);   // without this, the first call fails with HaDeviceInvalidArgument
  HaDevicePtr buf = 0;
  HaMemAlloc(&buf, size, false);
}

Note

This selects which device subsequent calls on this thread (e.g., HaMemAlloc(), HaLaunchKernel()) operate on.

Note

The current device applies only to the calling thread, so each thread that uses the runtime must set its own, including threads created by a framework or thread pool rather than by your code.