Python API¶
1 | |
The meaning is the same as in the C++ API; only the differences are summarized in Differences from the C++ API.
Things to know first
import lcclfinds thehartinstalled in the same Python environment. If you keephartin another directory, for example throughPYTHONPATH, runimport hartfirst.- The standalone
lcclpackage is released as an LPU hardware build (+hw).torch_haalso includes LCCL, which you can use throughtorch.distributed. See the Support matrix. - To use
lcclandtorch_hain one process, installhart,lccl, andtorch_hafrom the same release.lcclandtorch_haeach ship their own library with the same name (libha_lccl.so), and both use whichever is loaded first.
Communicators¶
| Function | Signature | Notes |
|---|---|---|
get_unique_id |
() -> LcclUniqueId |
Call on one rank only and distribute the result out of band. This process must stay alive until all ranks have connected |
comm_init_rank |
(nranks: int, comm_id: LcclUniqueId, myrank: int) -> LcclComm |
Blocks until all nranks ranks have joined. Call hart.set_device() first |
comm_init_all |
(ndev: int, devlist: list[int]) -> list[LcclComm] |
Single process, single server. Bind a device to the thread before the call. devlist must be a list whose length equals ndev, with no duplicate or out-of-range values. If it raises, stop using lccl in that process and start again in a new process |
comm_destroy |
(comm: LcclComm) -> None |
Releases the resources. The garbage collector does not do this for you, so always call it |
bootstrap_is_root_alive |
() -> int |
Whether ranks are still connecting to the unique ID created by this process (1 means it is still waiting). There is one value for the whole process, and it becomes 0 once all ranks have connected |
Groups¶
| Function | Signature | Notes |
|---|---|---|
group_start |
() -> None |
Can be nested. The depth is tracked per thread |
group_end |
() -> None |
Executes when the outermost region closes |
Collective operations¶
| Function | Signature |
|---|---|
all_reduce |
(send_buff, recv_buff, count: int, datatype: LcclDataType, op: LcclRedOp, comm: LcclComm, stream: int = 0) -> None |
reduce |
(send_buff, recv_buff, count, datatype, op, root: int, comm) -> None |
reduce_scatter |
(send_buff, recv_buff, recv_count, datatype, op, comm) -> None |
broadcast |
(send_buff, recv_buff, count, datatype, root: int, comm) -> None |
all_gather |
(send_buff, recv_buff, send_count, datatype, comm) -> None |
gather |
(send_buff, recv_buff, count, datatype, root: int, comm) -> None |
scatter |
(send_buff, recv_buff, count, datatype, root: int, comm) -> None |
all_to_all |
(send_buff, recv_buff, count, datatype, comm) -> None |
The receive buffer size differs for each operation. See Collective operations. The operations that run on the LPU are listed in the Support matrix.
Buffer arguments¶
send_buff and recv_buff accept the device address (an integer) returned by hart.mem_alloc(). An object that becomes an integer through __index__ (a NumPy or PyTorch integer scalar) is treated the same way. The size is not checked, and count is trusted. bool is rejected.
On a non-root rank, a buffer that is not used must still be given a value. None does not work; pass the integer 0.
Point-to-point¶
| Function | Signature | Notes |
|---|---|---|
send |
(send_buff, count, datatype, peer: int, comm) -> None |
Pair it with the corresponding recv inside a group |
recv |
(recv_buff, count, datatype, peer: int, comm) -> None |
For two ranks to exchange data with each other, both sides must put both calls in the same group; otherwise they deadlock. See Point-to-point.
1 2 3 4 5 6 | |
Types¶
LcclComm¶
Read-only attributes that describe the communicator. They are useful for diagnostics.
| Attribute | Meaning |
|---|---|
rank, n_ranks |
The index of this rank and the size of the communicator |
ha_dev |
The device this communicator drives |
bus_id |
The bus identifier of that device |
node, n_nodes |
The index of this server and the number of servers |
local_rank, local_ranks |
The rank index within the server and the number of ranks on this server |
n_channels |
The number of channels this communicator created |
n_nodes shows how many servers LCCL decided the ranks are spread over, and n_channels shows the number of channels actually created. A release build cannot provide this information through the INFO log, so check it with these attributes. See Environment variables.
LcclUniqueId¶
| Form | Meaning |
|---|---|
LcclUniqueId() |
An empty handle. Use the one that get_unique_id() fills in and returns |
LcclUniqueId(data: bytes) |
Reconstructs a handle from the internal value. It must be exactly 128 bytes, otherwise ValueError |
internal |
Read-only. The raw 128-byte handle, as bytes |
What you distribute to the other ranks is bytes(unique_id.internal), and the receiving side restores the handle with LcclUniqueId(data). The handle object itself cannot be serialized with pickle.
LcclDataType¶
LcclInt8 / LcclChar, LcclUint8, LcclInt32 / LcclInt, LcclUint32, LcclInt64, LcclUint64, LcclFloat16 / LcclHalf, LcclFloat32 / LcclFloat, LcclFloat64 / LcclDouble, LcclBfloat16.
Paired names are aliases with the same value and match the NCCL enumeration.
LcclRedOp¶
LcclSum, LcclProd, LcclMax, LcclMin, LcclAvg.
The enumeration is wider than the implementation
LcclMax and LcclMin have no kernel for any operation, so they raise a RuntimeError (kLcclNotSupported). LcclProd and LcclAvg work only with all_reduce and reduce. See the Support matrix.
Differences from the C++ API¶
| C++ | Python | |
|---|---|---|
| Failure reporting | Returns LcclResult |
RuntimeError. The message is <function name> failed: <message> |
| Invalid unique ID length | — | ValueError |
| Buffers | void * device pointer |
Integer (device address) |
| Streams | Most take an HaStream_t |
Only all_reduce has a stream keyword (integer handle, default 0). The rest use the default stream |
| Communicator queries | LcclCommCount(), LcclCommUserRank(), LcclCommHaDevice() |
Attributes of LcclComm |
LcclCommInitRankConfig, LcclCommFinalize, LcclCommAbort |
Available | Not exposed |
LcclCommGetAsyncError |
Available | Not exposed |
LcclCommRegister, LcclCommDeregister |
Available | Not exposed |
LcclConfig |
Available | Not available. Environment variables and configuration files are the only configuration path |
LcclResult values |
Enumeration | Not exposed. Present only as a sentence in the exception message |
| String helpers | LcclGetErrorToString() and others |
Not exposed |
| GIL | — | Blocking calls do not release the GIL |
Asynchronous errors cannot be checked from Python
LcclCommGetAsyncError() has no Python binding. In nonblocking mode (LCCL_BLOCKING=0), a call that returns kLcclInProgress turns into an exception, so use only the default blocking mode from Python. The cause code cannot be read either when kLcclKernelAborted is returned. If this matters in your deployment, drive the library from C++.
The GIL is not released, so splitting ranks across threads deadlocks
Collective operations and comm_init_rank block while waiting for the other ranks, and they do not release the GIL in the meantime, so other Python threads in the same process stall. Assigning one rank per thread within a single process deadlocks. Use a separate process for each rank.
Device selection¶
A communicator is tied to the device bound to the calling thread at the time of creation. Set the device before creating it. If you omit this, the error is LcclCommInitRank failed: Communicator unknown or unclassified error.
1 2 3 | |
The current device is thread-local. See hart. Before calling an operation or comm_destroy() from a thread other than the one that created the communicator, select the same device on that thread with hart.set_device(). A child created by fork from a process that has initialized the runtime is not safe, so start rank processes with spawn.