Point-to-point communication

MPI.AbstractRequest

An abstract type for Julia objects wrapping MPI Requests objects, which represent non-blocking MPI communication operations. The following implementations provided in MPI.jl

• Request: this is the default request type.
• UnsafeRequest: similar to Request, but does not maintain a reference to the underlying communication buffer.
• MultiRequestItem: created by calling getindex on a MultiRequest / UnsafeMultiRequest object, which efficiently stores a collection of requests.

How request objects are used

A request object can be passed to non-blocking communication operations, such as MPI.Isend and MPI.Irecv!. If no object is provided, then an MPI.Request is used.

The status of a Request can be checked by the Wait and Test functions or their mœultiple-request variants, which will deallocate the request once it is determined to be complete.

Alternatively, it will be deallocated by calling MPI.free or at finalization, meaning that it is safe to ignore the request objects if the status of the communication can be checked by other means.

In certain cases, the operation can also be cancelled by Cancel!.

Implementing new request types

Subtypes R <: AbstractRequest should define the methods for the following functions:

• C conversion functions to MPI_Request and Ptr{MPI_Request}:
  • Base.cconvert(::Type{MPI_Request}, req::R) / Base.unsafe_convert(::Type{MPI_Request}, req::R)
  • Base.cconvert(::Type{Ptr{MPI_Request}}, req::R) / Base.unsafe_convert(::Type{Ptr{MPI_Request}}, req::R)`
• setbuffer!(req::R, val): keep a reference to the communication bufferval. Ifval == nothing`, then clear the reference.

MPI.Request()

The default MPI Request object, representing a non-blocking communication. This also contains a reference to the buffer used in the communication to ensure it isn't garbage-collected during communication.

See AbstractRequest for more information.

MPI.UnsafeRequest()

Similar to MPI.Request, but does not maintain a reference to the underlying communication buffer. This may have improve performance by reducing memory allocations.

MPI.MultiRequest(n::Integer=0)

A collection of MPI Requests. This is useful when operating on multiple MPI requests at the same time. MultiRequest objects can be passed directly to MPI.Waitall, MPI.Testall, etc.

req[i] will return a MultiRequestItem which adheres to the [AbstractRequest] interface.

Usage

reqs = MPI.MultiRequest(n)
for i = 1:n
    MPI.Isend(buf, comm, reqs[i]; rank=dest[i])
end
MPI.Waitall(reqs)

MPI.UnsafeMultiRequest(n::Integer=0)

Similar to MPI.MultiRequest, except that it does not maintain references to the underlying communication buffers. The same caveats apply as MPI.UnsafeRequest.

RequestSet(requests::Vector{Request})
RequestSet() # create an empty RequestSet

A wrapper for an array of Requests that can be used to reduce intermediate memory allocations in Waitall, Testall, Waitany, Testany, Waitsome or Testsome.

Consider using a MultiRequest or UnsafeMultiRequest instead.

MPI.Status

The status of an MPI receive communication. It has 3 accessible fields

• source: source of the received message
• tag: tag of the received message
• error: error code. This is only set if a function returns multiple statuses.

Additionally, the accessor function MPI.Get_count can be used to determine the number of entries received.

MPI.Get_count(status::Status, T)

The number of entries received. T should match the argument provided by the receive call that set the status variable.

If the number of entries received exceeds the limits of the count parameter, then it returns MPI_UNDEFINED.

External links

• MPI_Get_count man page: OpenMPI, MPICH

MPI.PROC_NULL

A dummy value that can be used instead of a rank wherever a source or a destination argument is required in a call. A send

MPI.ANY_SOURCE

A wild card value for receive or probe operations that matches any source rank.

MPI.ANY_TAG

A wild card value for receive or probe operations that matches any tag.

Send(buf, comm::Comm; dest::Integer, tag::Integer=0)

Perform a blocking send from the buffer buf to MPI rank dest of communicator comm using the message tag tag.

Send(obj, comm::Comm; dest::Integer, tag::Integer=0)

Complete a blocking send of an isbits object obj to MPI rank dest of communicator comm using with the message tag tag.

External links

• MPI_Send man page: OpenMPI, MPICH

send(obj, comm::Comm; dest::Integer, tag::Integer=0)

Complete a blocking send using a serialized version of obj to MPI rank dest of communicator comm using with the message tag tag.

data = Recv!(recvbuf, comm::Comm;
        source::Integer=MPI.ANY_SOURCE, tag::Integer=MPI.ANY_TAG)
data, status = Recv!(recvbuf, comm::Comm, MPI.Status;
        source::Integer=MPI.ANY_SOURCE, tag::Integer=MPI.ANY_TAG)

Completes a blocking receive into the buffer recvbuf from MPI rank source of communicator comm using with the message tag tag.

recvbuf can be a Buffer, or any object for which Buffer(recvbuf) is defined.

Optionally returns the Status object of the receive.

See also

• Recv
• recv

External links

• MPI_Recv man page: OpenMPI, MPICH

data = Recv(::Type{T}, comm::Comm;
        source::Integer=MPI.ANY_SOURCE, tag::Integer=MPI.ANY_TAG)
data, status = Recv(::Type{T}, comm::Comm, MPI.Status;
        source::Integer=MPI.ANY_SOURCE, tag::Integer=MPI.ANY_TAG)

Completes a blocking receive of a single isbits object of type T from MPI rank source of communicator comm using with the message tag tag.

Returns a tuple of the object of type T and optionally the Status of the receive.

See also

• Recv!
• recv

External links

• MPI_Recv man page: OpenMPI, MPICH

obj = recv(comm::Comm;
        source::Integer=MPI.ANY_SOURCE, tag::Integer=MPI.ANY_TAG)
obj, status = recv(comm::Comm, MPI.Status;
        source::Integer=MPI.ANY_SOURCE, tag::Integer=MPI.ANY_TAG)

Completes a blocking receive of a serialized object from MPI rank source of communicator comm using with the message tag tag.

Returns the deserialized object and optionally the Status of the receive.

data = Sendrecv!(sendbuf, recvbuf, comm;
        dest::Integer, sendtag::Integer=0, source::Integer=MPI.ANY_SOURCE, recvtag::Integer=MPI.ANY_TAG)
data, status = Sendrecv!(sendbuf, recvbuf, comm, MPI.Status;
        dest::Integer, sendtag::Integer=0, source::Integer=MPI.ANY_SOURCE, recvtag::Integer=MPI.ANY_TAG)

Complete a blocking send-receive operation over the MPI communicator comm. Send sendbuf to the MPI rank dest using message tag sendtag, and receive from MPI rank source into the buffer recvbuf using message tag recvtag. Return a Status object.

External links

• MPI_Sendrecv man page: OpenMPI, MPICH

Isend(data, comm::Comm[, req::AbstractRequest = Request()]; dest::Integer, tag::Integer=0)

Starts a nonblocking send of data to MPI rank dest of communicator comm using with the message tag tag.

data can be a Buffer, or any object for which Buffer_send is defined.

Returns the AbstractRequest object for the nonblocking send.

External links

• MPI_Isend man page: OpenMPI, MPICH

isend(obj, comm::Comm[, req::AbstractRequest = Request()]; dest::Integer, tag::Integer=0)

Starts a nonblocking send of using a serialized version of obj to MPI rank dest of communicator comm using with the message tag tag.

Returns the communication Request for the nonblocking send.

req = Irecv!(recvbuf, comm::Comm[, req::AbstractRequest = Request()];
        source::Integer=MPI.ANY_SOURCE, tag::Integer=MPI.ANY_TAG)

Starts a nonblocking receive into the buffer data from MPI rank source of communicator comm using with the message tag tag.

data can be a Buffer, or any object for which Buffer(data) is defined.

Returns the AbstractRequest object for the nonblocking receive.

External links

• MPI_Irecv man page: OpenMPI, MPICH

flag = Test(req::AbstractRequest)
flag, status = Test(req::AbstractRequest, Status)

Check if the request req is complete. If so, the request is deallocated and flag = true is returned. Otherwise flag = false.

The Status argument additionally returns the Status of the completed request.

External links

• MPI_Test man page: OpenMPI, MPICH

flag = Testall(reqs::AbstractVector{Request}[, statuses::Vector{Status}])
flag, statuses = Testall(reqs::AbstractVector{Request}, Status)

Check if all active requests in the array reqs are complete. If so, the requests are deallocated and true is returned. Otherwise no requests are modified, and false is returned.

The optional statuses or Status argument can be used to obtain the return Status of each request.

See also

• RequestSet can be used to minimize allocations

External links

• MPI_Testall man page: OpenMPI, MPICH

flag, idx = Testany(reqs::AbstractVector{Request}[, status::Ref{Status}])
flag, idx, status = Testany(reqs::AbstractVector{Request}, Status)

Checks if any one of the requests in the array reqs is complete.

If one or more requests are complete, then one is chosen arbitrarily, deallocated. flag = true and its (1-based) index idx is returned.

If there are no completed requests, then flag = false and idx = nothing is returned.

If there are no active requests, flag = true and idx = nothing.

The optional status argument can be used to obtain the return Status of the request.

See also

• RequestSet can be used to minimize allocations

External links

• MPI_Testany man page: OpenMPI, MPICH

inds = Testsome(reqs::AbstractVector{Request}[, statuses::Vector{Status}])

Similar to Waitsome except that if no operations have completed it will return an empty array.

If there are no active requests, then the function returns nothing.

The optional statuses argument can be used to obtain the return Status of each completed request.

See also

• RequestSet can be used to minimize allocations

External links

• MPI_Testsome man page: OpenMPI, MPICH

Wait(req::AbstractRequest)
status = Wait(req::AbstractRequest, Status)

Block until the request req is complete and deallocated.

The Status argument returns the Status of the completed request.

External links

• MPI_Wait man page: OpenMPI, MPICH

Base.wait(req::MPI.Request)

Wait for an MPI request to complete. Unlike MPI.Wait, it will yield to other Julia tasks resulting in a cooperative wait.

Waitall(reqs::AbstractVector{Request}[, statuses::Vector{Status}])
statuses = Waitall(reqs::AbstractVector{Request}, Status)

Block until all active requests in the array reqs are complete.

The optional statuses or Status argument can be used to obtain the return Status of each request.

See also

• RequestSet can be used to minimize allocations

External links

• MPI_Waitall man page: OpenMPI, MPICH

i = Waitany(reqs::AbstractVector{Request}[, status::Ref{Status}])
i, status = Waitany(reqs::AbstractVector{Request}, Status)

Blocks until one of the requests in the array reqs is complete: if more than one is complete, one is chosen arbitrarily. The request is deallocated and the (1-based) index i of the completed request is returned.

If there are no active requests, then i = nothing.

The optional status argument can be used to obtain the return Status of the request.

See also

• RequestSet can be used to minimize allocations

External links

• MPI_Waitany man page: OpenMPI, MPICH

inds = Waitsome(reqs::AbstractVector{Request}[, statuses::Vector{Status}])

Block until at least one of the active requests in the array reqs is complete. The completed requests are deallocated, and an array inds of their indices in reqs is returned.

If there are no active requests, then inds = nothing.

The optional statuses argument can be used to obtain the return Status of each completed request.

See also

• RequestSet can be used to minimize allocations

External links

• MPI_Waitsome man page: OpenMPI, MPICH

isnull(req::AbstractRequest)

Is req is a null request.

Cancel!(req::Request)

Marks a pending Irecv! operation for cancellation (cancelling a Isend, while supported in some implementations, is deprecated as of MPI 3.1). Note that the request is not deallocated, and can still be queried using the test or wait functions.

External links

• MPI_Cancel man page: OpenMPI, MPICH

ismsg = Iprobe(comm::Comm;
        source::Integer=MPI.ANY_SOURCE, tag::Integer=MPI.ANY_TAG)
ismsg, status = Iprobe(comm::Comm, MPI.Status;
        source::Integer=MPI.ANY_SOURCE, tag::Integer=MPI.ANY_TAG)

Checks if there is a message that can be received matching source, tag and comm. If so, returns ismsg = true. The Status argument additionally returns the Status of the completed request.

External links

• MPI_Iprobe man page: OpenMPI, MPICH

Probe(comm::Comm;
        source::Integer=MPI.ANY_SOURCE, tag::Integer=MPI.ANY_TAG)
status = Probe(comm::Comm, MPI.Status;
    source::Integer=MPI.ANY_SOURCE, tag::Integer=MPI.ANY_TAG)

Blocks until there is a message that can be received matching source, tag and comm. Optionally returns the corresponding Status object.

External links

• MPI_Probe man page: OpenMPI, MPICH

Send_init(buf, comm::MPI.Comm[, req::AbstractRequest = Request()];
    dest, tag=0)

Allocate a persistent send request, returning a AbstractRequest object. Use Start or Startall to start the communication operation, and free to deallocate the request.

External links

• MPI_Send_init man page: OpenMPI, MPICH

Recv_init(buf, comm::MPI.Comm[, req::AbstractRequest = Request()];
    source=MPI.ANY_SOURCE, tag=MPI.ANY_TAG)

Allocate a persistent receive request, returning a AbstractRequest object. Use Start or Startall to start the communication operation, and free to deallocate the request.

External links

• MPI_Recv_init man page: OpenMPI, MPICH

Start(request::AbstractRequest)

Start a persistent communication request created by Send_init or Recv_init. Call Wait to complete the request.

External links

• MPI_Start man page: OpenMPI, MPICH

Startall(reqs::AbstractVector{Request})

Start a set of persistent communication requests created by Send_init or Recv_init. Call Waitall to complete the requests.

External links

• MPI_Startall man page: OpenMPI, MPICH

MPI.Message

An MPI message handle object, used by matched receive operations. These are returned by MPI.Mprobe and MPI.Improbe operations, and must be received by either MPI.Mrecv! or MPI.Imrecv!.

msg = MPI.Mprobe(comm::MPI.Comm;
    source::Integer=MPI.ANY_SOURCE, tag::Integer=MPI.ANY_TAG)
msg, status = MPI.Mprobe(comm::MPI.Comm, MPI.Status;
    source::Integer=MPI.ANY_SOURCE, tag::Integer=MPI.ANY_TAG)

Matching blocking probe. Similar to MPI.Probe, except that it also returns msg, an MPI.Message object.

Blocks until a message that can be received matching source, tag and comm, returning a Message object msg, which must be received by either MPI.Mrecv! or MPI.Imrecv!.

The Status argument additionally returns the Status of the completed request.

External links

• MPI_Mprobe man page: OpenMPI, MPICH

ismsg, msg = MPI.Improbe(comm::MPI.Comm;
    source::Integer=MPI.ANY_SOURCE, tag::Integer=MPI.ANY_TAG)
ismsg, msg, status = MPI.Improbe(comm::MPI.Comm, MPI.Status;
    source::Integer=MPI.ANY_SOURCE, tag::Integer=MPI.ANY_TAG)

Matching non-blocking probe. Similar to MPI.Iprobe, except that it also returns msg, an MPI.Message object.

Checks if there is a message that can be received matching source, tag and comm. If so, returns ismsg = true, and a Message object msg, which must be received by either MPI.Mrecv! or MPI.Imrecv!. Otherwise msg is set to be a null Message.

The Status argument additionally returns the Status of the completed request.

External links

• MPI_Improbe man page: OpenMPI, MPICH

data = MPI.Mrecv!(recvbuf, msg::MPI.Message)
data, status = MPI.Mrecv!(recvbuf, msg::MPI.Message, MPI.Status)

Completes a blocking receive matched by a matching probe operation into the buffer recvbuf, and the Message msg.

recvbuf can be a Buffer, or any object for which Buffer(recvbuf) is defined.

Optionally returns the Status object of the receive.

External links

• MPI_Mrecv man page: OpenMPI, MPICH

req = MPI.Imrecv!(recvbuf, msg::MPI.Message[, req::AbstractRequest=Request()])

Starts a nonblocking receive matched by a matching probe operation into the buffer recvbuf, and the Message msg.

recvbuf can be a Buffer, or any object for which Buffer(recvbuf) is defined.

Returns req, an AbstractRequest object for the nonblocking receive.

External links

• MPI_Imrecv man page: OpenMPI, MPICH