Skip to content

Communicator API

Creating, querying, and tearing down a communicator. The bootstrap sequence is described in Communicators.


LcclCommAbort

Aborts all pending operations on the communicator and releases its resources. The handle cannot be used after the call, so do not call LcclCommDestroy on it.

1
LcclResult LcclCommAbort(LcclComm comm);
Parameter Direction Description
comm in Communicator to abort.

Returns — kLcclSuccess on success, error code otherwise.


LcclCommCount

Returns the number of ranks in the communicator.

1
LcclResult LcclCommCount(LcclComm comm, int *count);
Parameter Direction Description
comm in Communicator to query.
count out Output rank count.

Returns — kLcclSuccess on success, error code otherwise.


LcclCommDeregister

Releases one reference on a registration obtained from LcclCommRegister.

1
LcclResult LcclCommDeregister(LcclComm comm, void *handle);
Parameter Direction Description
comm in Communicator the buffer was registered with.
handle in Handle returned by LcclCommRegister; may be null.

Returns — kLcclSuccess on success or a null handle, kLcclCommInvalidArgs if handle is not a live registration, kLcclCommStateInvalid while a nonblocking group on comm is still in progress, or the recorded error if comm already has an asynchronous error (in these last two cases the reference is kept), error code otherwise.

The network registration is dropped and the cache entry freed only when the last reference goes. The handle is matched by identity and never dereferenced, so a handle that is not (or no longer) a live registration is rejected. A handle whose buffer was freed while registered stays accepted until its remaining references are released. A null handle, which is what LcclCommRegister returns when LCCL_LOCAL_REGISTER=0 disables the cache, is a no-op. Must not be called concurrently with collectives or group calls on the same comm.


LcclCommDestroy

Frees local resources associated with communicator object.

1
LcclResult LcclCommDestroy(LcclComm comm);
Parameter Direction Description
comm in Pointer to the LCCL communicator

Returns — kLcclSuccess if the communicator is destroyed successfully, error code otherwise


LcclCommFinalize

Finalises a communicator, draining outstanding work without freeing it.

1
LcclResult LcclCommFinalize(LcclComm comm);
Parameter Direction Description
comm in Communicator to finalize.

Returns — kLcclSuccess on success, error code otherwise.


LcclCommGetAsyncError

Retrieves the latest asynchronous error recorded on the communicator.

1
LcclResult LcclCommGetAsyncError(LcclComm comm, LcclResult *async_error);
Parameter Direction Description
comm in Communicator to query.
async_error out Output: most recent async error code, or kLcclSuccess if none.

Returns — kLcclSuccess on success, error code otherwise.


LcclCommHaDevice

Returns the HyperAccel device index associated with the communicator.

1
LcclResult LcclCommHaDevice(LcclComm comm, int *device);
Parameter Direction Description
comm in Communicator to query.
device out Output device index.

Returns — kLcclSuccess on success, error code otherwise.


LcclCommInitAll

Creates a clique of communicators (single process version).

1
LcclResult LcclCommInitAll(LcclComm *comm, int ndev, const int *devlist);
Parameter Direction Description
comm out Pointer to the LCCL communicator
ndev in Number of devices
devlist in List of devices

Returns — kLcclSuccess if the communicators are initialized successfully, error code otherwise

Note

This is a convenience function to create a single-process communicator clique. Returns an array of ndev newly initialized communicators in comm. comm should be pre-allocated with size at least ndev*sizeof(LcclComm). devlist must not be NULL; passing NULL returns kLcclCommNullPtr. Pass ndev valid, non-duplicate device indices. Order of devlist defines user-order of processors within the communicator. If LcclCommInitAll returns an error, stop using LCCL in that process; fix the cause and start again in a new process.


LcclCommInitRank

Creates a new communicator (Multi thread/process version).

1
LcclResult LcclCommInitRank(LcclComm *newcomm, int nranks, LcclUniqueId comm_id, int myrank);
Parameter Direction Description
newcomm out Pointer to the LCCL communicator
nranks in Number of ranks in the communicator
comm_id in LCCL unique identifier
myrank in Rank of the communicator

Returns — kLcclSuccess if the communicator is initialized successfully, error code otherwise

Note

rank must be between 0 and nranks-1 and unique within a communicator clique. Each rank is associated to a HyperAccel device, which has to be set before calling LcclCommInitRank. LcclCommInitRank implicitly synchronizes with other ranks, so it must be called by different threads/processes or use LcclGroupStart/LcclGroupEnd.


LcclCommInitRankConfig

Like LcclCommInitRank but accepts an explicit LcclConfig for tunables.

1
LcclResult LcclCommInitRankConfig(LcclComm *newcomm, int nranks, LcclUniqueId comm_id, int myrank, LcclConfig *config);
Parameter Direction Description
newcomm out Output pointer to the new communicator handle. Check the return value to detect a failure, and do not use newcomm after a failed call.
nranks in Total number of ranks in the communicator.
comm_id in Bootstrap unique ID shared by all ranks.
myrank in Rank of the calling process within the communicator.
config in Configuration block (must be initialised with LCCL_CONFIG_INITIALIZER).

Returns — kLcclSuccess on success, error code otherwise.


LcclCommRegister

Registers a device buffer with comm so network transports can reuse its registration.

1
LcclResult LcclCommRegister(LcclComm comm, void *buff, size_t size, void **handle);
Parameter Direction Description
comm in Communicator to register with.
buff in Device buffer to register.
size in Buffer size in bytes; must be non-zero.
handle out Opaque registration handle to pass to LcclCommDeregister.

Returns — kLcclSuccess on success, kLcclCommInvalidArgs on a null buffer/handle or zero size, error code otherwise.

Registrations are cached per communicator at page granularity and reference counted: registering a range already covered by a cached entry returns the same handle and bumps its count, so every successful call must be paired with one LcclCommDeregister. Whether the network plugin actually registers the pages depends on the plugin (global registration and LCCL_PTR_HART support are required); when it does not, the call still succeeds and the buffer is simply treated as unregistered. Only LcclSend / LcclRecv over the NET transport use a registration, moving the bytes straight between the user buffers; the matching buffer on the peer must be registered as well. Otherwise the rank that detects the mismatch fails the transfer instead of mixing a user buffer with the peer's FIFO slots, and the peer may block until LcclCommAbort. Registrations still outstanding at LcclCommDestroy / LcclCommAbort are released there. Deregister before freeing the buffer: an HaMemFree / HaMemFreeAsync of a pointer inside a live registration is logged as an error and drops that registration; its handle stays valid only for the LcclCommDeregister calls still owed on it. A free is matched by the pointer passed to it, so freeing an allocation whose base lies below the registered range is not detected. Must not be called concurrently with collectives or group calls on the same comm.


LcclCommUserRank

Returns the user-assigned rank of the calling process within the communicator.

1
LcclResult LcclCommUserRank(LcclComm comm, int *rank);
Parameter Direction Description
comm in Communicator to query.
rank out Output rank value.

Returns — kLcclSuccess on success, error code otherwise.


LcclGetUniqueId

Generates an Id to be used in LcclCommInitRank.

1
LcclResult LcclGetUniqueId(LcclUniqueId *out);
Parameter Direction Description
out out Pointer to the unique ID

Returns — kLcclSuccess if the unique ID is created successfully, error code otherwise

Note

LcclGetUniqueId should be called once and the Id should be distributed to all ranks in the communicator before calling LcclCommInitRank.