Communicator API¶
Creating, querying, and tearing down a communicator. The bootstrap sequence is described in Communicators.
LcclCommAbort¶
Aborts all pending operations on the communicator and releases its resources. The handle cannot be used after the call, so do not call LcclCommDestroy on it.
1 | |
| Parameter | Direction | Description |
|---|---|---|
comm |
in | Communicator to abort. |
Returns — kLcclSuccess on success, error code otherwise.
LcclCommCount¶
Returns the number of ranks in the communicator.
1 | |
| Parameter | Direction | Description |
|---|---|---|
comm |
in | Communicator to query. |
count |
out | Output rank count. |
Returns — kLcclSuccess on success, error code otherwise.
LcclCommDeregister¶
Releases one reference on a registration obtained from LcclCommRegister.
1 | |
| Parameter | Direction | Description |
|---|---|---|
comm |
in | Communicator the buffer was registered with. |
handle |
in | Handle returned by LcclCommRegister; may be null. |
Returns — kLcclSuccess on success or a null handle, kLcclCommInvalidArgs if handle is not a live registration, kLcclCommStateInvalid while a nonblocking group on comm is still in progress, or the recorded error if comm already has an asynchronous error (in these last two cases the reference is kept), error code otherwise.
The network registration is dropped and the cache entry freed only when the last reference goes. The handle is matched by identity and never dereferenced, so a handle that is not (or no longer) a live registration is rejected. A handle whose buffer was freed while registered stays accepted until its remaining references are released. A null handle, which is what LcclCommRegister returns when LCCL_LOCAL_REGISTER=0 disables the cache, is a no-op. Must not be called concurrently with collectives or group calls on the same comm.
LcclCommDestroy¶
Frees local resources associated with communicator object.
1 | |
| Parameter | Direction | Description |
|---|---|---|
comm |
in | Pointer to the LCCL communicator |
Returns — kLcclSuccess if the communicator is destroyed successfully, error code otherwise
LcclCommFinalize¶
Finalises a communicator, draining outstanding work without freeing it.
1 | |
| Parameter | Direction | Description |
|---|---|---|
comm |
in | Communicator to finalize. |
Returns — kLcclSuccess on success, error code otherwise.
LcclCommGetAsyncError¶
Retrieves the latest asynchronous error recorded on the communicator.
1 | |
| Parameter | Direction | Description |
|---|---|---|
comm |
in | Communicator to query. |
async_error |
out | Output: most recent async error code, or kLcclSuccess if none. |
Returns — kLcclSuccess on success, error code otherwise.
LcclCommHaDevice¶
Returns the HyperAccel device index associated with the communicator.
1 | |
| Parameter | Direction | Description |
|---|---|---|
comm |
in | Communicator to query. |
device |
out | Output device index. |
Returns — kLcclSuccess on success, error code otherwise.
LcclCommInitAll¶
Creates a clique of communicators (single process version).
1 | |
| Parameter | Direction | Description |
|---|---|---|
comm |
out | Pointer to the LCCL communicator |
ndev |
in | Number of devices |
devlist |
in | List of devices |
Returns — kLcclSuccess if the communicators are initialized successfully, error code otherwise
Note
This is a convenience function to create a single-process communicator clique. Returns an array of ndev newly initialized communicators in comm. comm should be pre-allocated with size at least ndev*sizeof(LcclComm). devlist must not be NULL; passing NULL returns kLcclCommNullPtr. Pass ndev valid, non-duplicate device indices. Order of devlist defines user-order of processors within the communicator. If LcclCommInitAll returns an error, stop using LCCL in that process; fix the cause and start again in a new process.
LcclCommInitRank¶
Creates a new communicator (Multi thread/process version).
1 | |
| Parameter | Direction | Description |
|---|---|---|
newcomm |
out | Pointer to the LCCL communicator |
nranks |
in | Number of ranks in the communicator |
comm_id |
in | LCCL unique identifier |
myrank |
in | Rank of the communicator |
Returns — kLcclSuccess if the communicator is initialized successfully, error code otherwise
Note
rank must be between 0 and nranks-1 and unique within a communicator clique. Each rank is associated to a HyperAccel device, which has to be set before calling LcclCommInitRank. LcclCommInitRank implicitly synchronizes with other ranks, so it must be called by different threads/processes or use LcclGroupStart/LcclGroupEnd.
LcclCommInitRankConfig¶
Like LcclCommInitRank but accepts an explicit LcclConfig for tunables.
1 | |
| Parameter | Direction | Description |
|---|---|---|
newcomm |
out | Output pointer to the new communicator handle. Check the return value to detect a failure, and do not use newcomm after a failed call. |
nranks |
in | Total number of ranks in the communicator. |
comm_id |
in | Bootstrap unique ID shared by all ranks. |
myrank |
in | Rank of the calling process within the communicator. |
config |
in | Configuration block (must be initialised with LCCL_CONFIG_INITIALIZER). |
Returns — kLcclSuccess on success, error code otherwise.
LcclCommRegister¶
Registers a device buffer with comm so network transports can reuse its registration.
1 | |
| Parameter | Direction | Description |
|---|---|---|
comm |
in | Communicator to register with. |
buff |
in | Device buffer to register. |
size |
in | Buffer size in bytes; must be non-zero. |
handle |
out | Opaque registration handle to pass to LcclCommDeregister. |
Returns — kLcclSuccess on success, kLcclCommInvalidArgs on a null buffer/handle or zero size, error code otherwise.
Registrations are cached per communicator at page granularity and reference counted: registering a range already covered by a cached entry returns the same handle and bumps its count, so every successful call must be paired with one LcclCommDeregister. Whether the network plugin actually registers the pages depends on the plugin (global registration and LCCL_PTR_HART support are required); when it does not, the call still succeeds and the buffer is simply treated as unregistered. Only LcclSend / LcclRecv over the NET transport use a registration, moving the bytes straight between the user buffers; the matching buffer on the peer must be registered as well. Otherwise the rank that detects the mismatch fails the transfer instead of mixing a user buffer with the peer's FIFO slots, and the peer may block until LcclCommAbort. Registrations still outstanding at LcclCommDestroy / LcclCommAbort are released there. Deregister before freeing the buffer: an HaMemFree / HaMemFreeAsync of a pointer inside a live registration is logged as an error and drops that registration; its handle stays valid only for the LcclCommDeregister calls still owed on it. A free is matched by the pointer passed to it, so freeing an allocation whose base lies below the registered range is not detected. Must not be called concurrently with collectives or group calls on the same comm.
LcclCommUserRank¶
Returns the user-assigned rank of the calling process within the communicator.
1 | |
| Parameter | Direction | Description |
|---|---|---|
comm |
in | Communicator to query. |
rank |
out | Output rank value. |
Returns — kLcclSuccess on success, error code otherwise.
LcclGetUniqueId¶
Generates an Id to be used in LcclCommInitRank.
1 | |
| Parameter | Direction | Description |
|---|---|---|
out |
out | Pointer to the unique ID |
Returns — kLcclSuccess if the unique ID is created successfully, error code otherwise
Note
LcclGetUniqueId should be called once and the Id should be distributed to all ranks in the communicator before calling LcclCommInitRank.