Point-to-point communication

Types

MPI.AbstractRequestType
MPI.AbstractRequest

An abstract type for Julia objects wrapping MPI Requests objects, which represent non-blocking MPI communication operations. The following implementations provided in MPI.jl

  • Request: this is the default request type.
  • UnsafeRequest: similar to Request, but does not maintain a reference to the underlying communication buffer.
  • MultiRequestItem: created by calling getindex on a MultiRequest / UnsafeMultiRequest object, which efficiently stores a collection of requests.

How request objects are used

A request object can be passed to non-blocking communication operations, such as MPI.Isend and MPI.Irecv!. If no object is provided, then an MPI.Request is used.

The status of a Request can be checked by the Wait and Test functions or their mœultiple-request variants, which will deallocate the request once it is determined to be complete.

Alternatively, it will be deallocated by calling MPI.free or at finalization, meaning that it is safe to ignore the request objects if the status of the communication can be checked by other means.

In certain cases, the operation can also be cancelled by Cancel!.

Implementing new request types

Subtypes R <: AbstractRequest should define the methods for the following functions:

  • C conversion functions to MPI_Request and Ptr{MPI_Request}:
    • Base.cconvert(::Type{MPI_Request}, req::R) / Base.unsafe_convert(::Type{MPI_Request}, req::R)
    • Base.cconvert(::Type{Ptr{MPI_Request}}, req::R) / Base.unsafe_convert(::Type{Ptr{MPI_Request}}, req::R)`
  • setbuffer!(req::R, val): keep a reference to the communication bufferval. Ifval == nothing`, then clear the reference.
source
MPI.RequestType
MPI.Request()

The default MPI Request object, representing a non-blocking communication. This also contains a reference to the buffer used in the communication to ensure it isn't garbage-collected during communication.

See AbstractRequest for more information.

source
MPI.UnsafeRequestType
MPI.UnsafeRequest()

Similar to MPI.Request, but does not maintain a reference to the underlying communication buffer. This may have improve performance by reducing memory allocations.

Warning

The user should ensure that another reference to the communication buffer is maintained so that it is not cleaned up by the garbage collector before the communication operation is complete.

For example ```julia buf = MPI.Buffer(zeros(10)) GC.@preserve buf begin req = MPI.Isend(buf, comm, UnsafeRequest(); rank=1) # ... MPI.Wait(req) end

source
MPI.MultiRequestType
MPI.MultiRequest(n::Integer=0)

A collection of MPI Requests. This is useful when operating on multiple MPI requests at the same time. MultiRequest objects can be passed directly to MPI.Waitall, MPI.Testall, etc.

req[i] will return a MultiRequestItem which adheres to the [AbstractRequest] interface.

Usage

reqs = MPI.MultiRequest(n)
for i = 1:n
    MPI.Isend(buf, comm, reqs[i]; rank=dest[i])
end
MPI.Waitall(reqs)
source
MPI.StatusType
MPI.Status

The status of an MPI receive communication. It has 3 accessible fields

  • source: source of the received message
  • tag: tag of the received message
  • error: error code. This is only set if a function returns multiple statuses.

Additionally, the accessor function MPI.Get_count can be used to determine the number of entries received.

source

Accessors

MPI.Get_countFunction
MPI.Get_count(status::Status, T)

The number of entries received. T should match the argument provided by the receive call that set the status variable.

If the number of entries received exceeds the limits of the count parameter, then it returns MPI_UNDEFINED.

External links

source

Constants

MPI.PROC_NULLConstant
MPI.PROC_NULL

A dummy value that can be used instead of a rank wherever a source or a destination argument is required in a call. A send

source
MPI.ANY_SOURCEConstant
MPI.ANY_SOURCE

A wild card value for receive or probe operations that matches any source rank.

source
MPI.ANY_TAGConstant
MPI.ANY_TAG

A wild card value for receive or probe operations that matches any tag.

source

Blocking communication

MPI.SendFunction
Send(buf, comm::Comm; dest::Integer, tag::Integer=0)

Perform a blocking send from the buffer buf to MPI rank dest of communicator comm using the message tag tag.

Send(obj, comm::Comm; dest::Integer, tag::Integer=0)

Complete a blocking send of an isbits object obj to MPI rank dest of communicator comm using with the message tag tag.

External links

source
MPI.sendFunction
send(obj, comm::Comm; dest::Integer, tag::Integer=0)

Complete a blocking send using a serialized version of obj to MPI rank dest of communicator comm using with the message tag tag.

source
MPI.Recv!Function
data = Recv!(recvbuf, comm::Comm;
        source::Integer=MPI.ANY_SOURCE, tag::Integer=MPI.ANY_TAG)
data, status = Recv!(recvbuf, comm::Comm, MPI.Status;
        source::Integer=MPI.ANY_SOURCE, tag::Integer=MPI.ANY_TAG)

Completes a blocking receive into the buffer recvbuf from MPI rank source of communicator comm using with the message tag tag.

recvbuf can be a Buffer, or any object for which Buffer(recvbuf) is defined.

Optionally returns the Status object of the receive.

See also

External links

source
MPI.RecvFunction
data = Recv(::Type{T}, comm::Comm;
        source::Integer=MPI.ANY_SOURCE, tag::Integer=MPI.ANY_TAG)
data, status = Recv(::Type{T}, comm::Comm, MPI.Status;
        source::Integer=MPI.ANY_SOURCE, tag::Integer=MPI.ANY_TAG)

Completes a blocking receive of a single isbits object of type T from MPI rank source of communicator comm using with the message tag tag.

Returns a tuple of the object of type T and optionally the Status of the receive.

See also

External links

source
MPI.recvFunction
obj = recv(comm::Comm;
        source::Integer=MPI.ANY_SOURCE, tag::Integer=MPI.ANY_TAG)
obj, status = recv(comm::Comm, MPI.Status;
        source::Integer=MPI.ANY_SOURCE, tag::Integer=MPI.ANY_TAG)

Completes a blocking receive of a serialized object from MPI rank source of communicator comm using with the message tag tag.

Returns the deserialized object and optionally the Status of the receive.

source
MPI.Sendrecv!Function
data = Sendrecv!(sendbuf, recvbuf, comm;
        dest::Integer, sendtag::Integer=0, source::Integer=MPI.ANY_SOURCE, recvtag::Integer=MPI.ANY_TAG)
data, status = Sendrecv!(sendbuf, recvbuf, comm, MPI.Status;
        dest::Integer, sendtag::Integer=0, source::Integer=MPI.ANY_SOURCE, recvtag::Integer=MPI.ANY_TAG)

Complete a blocking send-receive operation over the MPI communicator comm. Send sendbuf to the MPI rank dest using message tag sendtag, and receive from MPI rank source into the buffer recvbuf using message tag recvtag. Return a Status object.

External links

source

Non-blocking communication

Initiation

MPI.IsendFunction
Isend(data, comm::Comm[, req::AbstractRequest = Request()]; dest::Integer, tag::Integer=0)

Starts a nonblocking send of data to MPI rank dest of communicator comm using with the message tag tag.

data can be a Buffer, or any object for which Buffer_send is defined.

Returns the AbstractRequest object for the nonblocking send.

External links

source
MPI.isendFunction
isend(obj, comm::Comm[, req::AbstractRequest = Request()]; dest::Integer, tag::Integer=0)

Starts a nonblocking send of using a serialized version of obj to MPI rank dest of communicator comm using with the message tag tag.

Returns the commication Request for the nonblocking send.

source
MPI.Irecv!Function
req = Irecv!(recvbuf, comm::Comm[, req::AbstractRequest = Request()];
        source::Integer=MPI.ANY_SOURCE, tag::Integer=MPI.ANY_TAG)

Starts a nonblocking receive into the buffer data from MPI rank source of communicator comm using with the message tag tag.

data can be a Buffer, or any object for which Buffer(data) is defined.

Returns the AbstractRequest object for the nonblocking receive.

External links

source

Completion

MPI.TestFunction
flag = Test(req::AbstractRequest)
flag, status = Test(req::AbstractRequest, Status)

Check if the request req is complete. If so, the request is deallocated and flag = true is returned. Otherwise flag = false.

The Status argument additionally returns the Status of the completed request.

External links

source
MPI.TestallFunction
flag = Testall(reqs::AbstractVector{Request}[, statuses::Vector{Status}])
flag, statuses = Testall(reqs::AbstractVector{Request}, Status)

Check if all active requests in the array reqs are complete. If so, the requests are deallocated and true is returned. Otherwise no requests are modified, and false is returned.

The optional statuses or Status argument can be used to obtain the return Status of each request.

See also

External links

source
MPI.TestanyFunction
flag, idx = Testany(reqs::AbstractVector{Request}[, status::Ref{Status}])
flag, idx, status = Testany(reqs::AbstractVector{Request}, Status)

Checks if any one of the requests in the array reqs is complete.

If one or more requests are complete, then one is chosen arbitrarily, deallocated. flag = true and its (1-based) index idx is returned.

If there are no completed requests, then flag = false and idx = nothing is returned.

If there are no active requests, flag = true and idx = nothing.

The optional status argument can be used to obtain the return Status of the request.

See also

External links

source
MPI.TestsomeFunction
inds = Testsome(reqs::AbstractVector{Request}[, statuses::Vector{Status}])

Similar to Waitsome except that if no operations have completed it will return an empty array.

If there are no active requests, then the function returns nothing.

The optional statuses argument can be used to obtain the return Status of each completed request.

See also

External links

source
MPI.WaitFunction
Wait(req::AbstractRequest)
status = Wait(req::AbstractRequest, Status)

Block until the request req is complete and deallocated.

The Status argument returns the Status of the completed request.

External links

source
Base.waitMethod
Base.wait(req::MPI.Request)

Wait for an MPI request to complete. Unlike MPI.Wait, it will yield to other Julia tasks resulting in a cooperative wait.

source
MPI.WaitallFunction
Waitall(reqs::AbstractVector{Request}[, statuses::Vector{Status}])
statuses = Waitall(reqs::AbstractVector{Request}, Status)

Block until all active requests in the array reqs are complete.

The optional statuses or Status argument can be used to obtain the return Status of each request.

See also

External links

source
MPI.WaitanyFunction
i = Waitany(reqs::AbstractVector{Request}[, status::Ref{Status}])
i, status = Waitany(reqs::AbstractVector{Request}, Status)

Blocks until one of the requests in the array reqs is complete: if more than one is complete, one is chosen arbitrarily. The request is deallocated and the (1-based) index i of the completed request is returned.

If there are no active requests, then i = nothing.

The optional status argument can be used to obtain the return Status of the request.

See also

External links

source
MPI.WaitsomeFunction
inds = Waitsome(reqs::AbstractVector{Request}[, statuses::Vector{Status}])

Block until at least one of the active requests in the array reqs is complete. The completed requests are deallocated, and an array inds of their indices in reqs is returned.

If there are no active requests, then inds = nothing.

The optional statuses argument can be used to obtain the return Status of each completed request.

See also

External links

source

Probe/Cancel

MPI.isnullFunction
isnull(req::AbstractRequest)

Is req is a null request.

source
MPI.Cancel!Function
Cancel!(req::Request)

Marks a pending Irecv! operation for cancellation (cancelling a Isend, while supported in some implementations, is deprecated as of MPI 3.1). Note that the request is not deallocated, and can still be queried using the test or wait functions.

External links

source
MPI.IprobeFunction
ismsg = Iprobe(comm::Comm;
        source::Integer=MPI.ANY_SOURCE, tag::Integer=MPI.ANY_TAG)
ismsg, status = Iprobe(comm::Comm, MPI.Status;
        source::Integer=MPI.ANY_SOURCE, tag::Integer=MPI.ANY_TAG)

Checks if there is a message that can be received matching source, tag and comm. If so, returns ismsg = true. The Status argument additionally returns the Status of the completed request.

External links

source
MPI.ProbeFunction
Probe(comm::Comm;
        source::Integer=MPI.ANY_SOURCE, tag::Integer=MPI.ANY_TAG)
status = Probe(comm::Comm, MPI.Status;
    source::Integer=MPI.ANY_SOURCE, tag::Integer=MPI.ANY_TAG)

Blocks until there is a message that can be received matching source, tag and comm. Optionally returns the corresponding Status object.

External links

source

Persistent requests

MPI.Send_initFunction
Send_init(buf, comm::MPI.Comm[, req::AbstractRequest = Request()];
    dest, tag=0)

Allocate a persistent send request, returning a AbstractRequest object. Use Start or Startall to start the communication operation, and free to deallocate the request.

External links

source
MPI.Recv_initFunction
Recv_init(buf, comm::MPI.Comm[, req::AbstractRequest = Request()];
    source=MPI.ANY_SOURCE, tag=MPI.ANY_TAG)

Allocate a persistent receive request, returning a AbstractRequest object. Use Start or Startall to start the communication operation, and free to deallocate the request.

External links

source

Matching probes and receives

MPI.MprobeFunction
msg = MPI.Mprobe(comm::MPI.Comm;
    source::Integer=MPI.ANY_SOURCE, tag::Integer=MPI.ANY_TAG)
msg, status = MPI.Mprobe(comm::MPI.Comm, MPI.Status;
    source::Integer=MPI.ANY_SOURCE, tag::Integer=MPI.ANY_TAG)

Matching blocking probe. Similar to MPI.Probe, except that it also returns msg, an MPI.Message object.

Blocks until a message that can be received matching source, tag and comm, returning a Message objec msg, which must be received by either MPI.Mrecv! or MPI.Imrecv!.

The Status argument additionally returns the Status of the completed request.

External links

source
MPI.ImprobeFunction
ismsg, msg = MPI.Improbe(comm::MPI.Comm;
    source::Integer=MPI.ANY_SOURCE, tag::Integer=MPI.ANY_TAG)
ismsg, msg, status = MPI.Improbe(comm::MPI.Comm, MPI.Status;
    source::Integer=MPI.ANY_SOURCE, tag::Integer=MPI.ANY_TAG)

Matching non-blocking probe. Similar to MPI.Iprobe, except that it also returns msg, an MPI.Message object.

Checks if there is a message that can be received matching source, tag and comm. If so, returns ismsg = true, and a Message objec msg, which must be received by either MPI.Mrecv! or MPI.Imrecv!. Otherwise msg is set to be a null Message.

The Status argument additionally returns the Status of the completed request.

External links

source
MPI.Mrecv!Function
data = MPI.Mrecv!(recvbuf, msg::MPI.Message)
data, status = MPI.Mrecv!(recvbuf, msg::MPI.Message, MPI.Status)

Completes a blocking receive matched by a matching probe operation into the buffer recvbuf, and the Message msg.

recvbuf can be a Buffer, or any object for which Buffer(recvbuf) is defined.

Optionally returns the Status object of the receive.

External links

source
MPI.Imrecv!Function
req = MPI.Imrecv!(recvbuf, msg::MPI.Message[, req::AbstractRequest=Request()])

Starts a nonblocking receive matched by a matching probe operation into the buffer recvbuf, and the Message msg.

recvbuf can be a Buffer, or any object for which Buffer(recvbuf) is defined.

Returns req, an AbstractRequest object for the nonblocking receive.

External links

source