API

DArray(init, dims, [procs, dist])

Construct a distributed array.

The parameter init is a function that accepts a tuple of index ranges. This function should allocate a local chunk of the distributed array and initialize it for the specified indices.

dims is the overall size of the distributed array.

procs optionally specifies a vector of process IDs to use. If unspecified, the array is distributed over all worker processes only. Typically, when running in distributed mode, i.e., nprocs() > 1, this would mean that no chunk of the distributed array exists on the process hosting the interactive julia prompt.

dist is an integer vector specifying how many chunks the distributed array should be divided into in each dimension.

For example, the dfill function that creates a distributed array and fills it with a value v is implemented as:

Example

dfill(v, args...) = DArray(I->fill(v, map(length,I)), args...)

sort(d::DVector; sample=true, kwargs...) -> DVector

Sorts and returns a new distributed vector.

The sorted vector may not have the same distribution as the original.

Keyword argument sample can take values:

• true: A sample of max size 512 is first taken from all nodes. This is used to balance the distribution of the sorted array on participating workers. Default is true.

• false: No sampling is done. Assumes a uniform distribution between min(d) and max(d)

• 2-element tuple of the form (min, max): No sampling is done. Assumes a uniform distribution between specified min and max values

• Array{T}: The passed array is assumed to be a sample of the distribution and is used to balance the sorted distribution.

Keyword argument alg takes the same options Base.sort

procs(d::DArray)

Get the vector of processes storing pieces of DArray d.

 dfill(x, dims, ...)

Construct a distributed array filled with value x. Trailing arguments are the same as those accepted by DArray.

distribute(A, DA)

Distribute a local array A like the distributed array DA.

 distribute(A[; procs, dist])

Convert a local array to distributed.

procs optionally specifies an array of process IDs to use. (defaults to all workers) dist optionally specifies a vector or tuple of the number of partitions in each dimension

dones(dims, ...)

Construct a distributed array of ones. Trailing arguments are the same as those accepted by DArray.

 drand(dims, ...)

Construct a distributed uniform random array. Trailing arguments are the same as those accepted by DArray.

 drandn(dims, ...)

Construct a distributed normal random array. Trailing arguments are the same as those accepted by DArray.

 dzeros(dims, ...)

Construct a distributed array of zeros. Trailing arguments are the same as those accepted by DArray.

localindices(d)

A tuple describing the indices owned by the local process. Returns a tuple with empty ranges if no local part exists on the calling process.

localpart(A)

The identity when input is not distributed

localpart(d::DArray)

Get the local piece of a distributed array. Returns an empty array if no local part exists on the calling process.

d[:L], d[:l], d[:LP], d[:lp] are an alternative means to get localparts. This syntaxt can also be used for assignment. For example, d[:L]=v will assign v to the localpart of d.

locate(d::DArray, I::Int...)

Determine the index of procs(d) that hold element I.

makelocal(A::DArray, I...)

Equivalent to Array(view(A, I...)) but optimised for the case that the data is local. Can return a view into localpart(A)

next_did()

Produces an incrementing ID that will be used for DArrays.

 ppeval(f, D...; dim::NTuple)

Evaluates the callable argument f on slices of the elements of the D tuple.

Arguments

f can be any callable object that accepts sliced or broadcasted elements of D. The result returned from f must be either an array or a scalar.

D has any number of elements and the elements can have any type. If an element of D is a distributed array along the dimension specified by dim. If an element of D is not distributed, the element is by default broadcasted and applied on all evaluations of f.

dim is a tuple of integers specifying the dimension over which the elements of D is slices. The length of the tuple must therefore be the same as the number of arguments D. By default distributed arrays are slides along the last dimension. If the value is less than or equal to zero the element are broadcasted to all evaluations of f.

Result

ppeval returns a distributed array of dimension p+1 where the first p sizes correspond to the sizes of return values of f. The last dimension of the return array from ppeval has the same length as the dimension over which the input arrays are sliced.

Examples

addprocs(Sys.CPU_THREADS)

using DistributedArrays

A = drandn((10, 10, Sys.CPU_THREADS), workers(), [1, 1, Sys.CPU_THREADS])

ppeval(eigvals, A)

ppeval(eigvals, A, randn(10,10)) # broadcasting second argument

B = drandn((10, Sys.CPU_THREADS), workers(), [1, Sys.CPU_THREADS])

ppeval(*, A, B)