Vec

PETSc vectors (Vec) are the fundamental building blocks for storing solution data, right-hand sides, and other distributed arrays. PETSc.jl provides a Julia-friendly interface that makes Vec objects behave like native Julia arrays.

Overview

PETSc vectors support:

  • Distributed parallel storage: Split across MPI processes
  • Sequential storage: For serial computations
  • Ghost points: For communication in stencil operations
  • Julia array interface: Use familiar indexing and broadcasting syntax

Creating Vectors

Sequential Vectors

# Create a sequential vector of length n
v = PetscVec(petsclib, n)

# Wrap an existing Julia array (no copy)
julia_array = zeros(100)
v = PetscVec(petsclib, julia_array)

# `destroy!` releases it; a vector on MPI.COMM_SELF also gets a finalizer
PETSc.destroy!(v)

From DM Objects

# Create global and local vectors from a DM
gvec = PETSc.global_vec(dm)
lvec = PETSc.local_vec(dm)

PetscVec replaces v0.4's VecSeq and as_petsc_vec: construction goes through the type (naming conventions, §6). The old spellings still work in v0.5 and warn once.

Julia Array Interface

PETSc vectors implement the Julia array interface:

v[1] = 1.0           # Set single element
v[1:10] .= 2.0       # Set range
x = v[5]             # Get element
length(v)            # Get length
size(v)              # Get size tuple

Linear Algebra

The vector operations of LinearAlgebra and the arithmetic operators call PETSc directly. All of them are collective, so every rank must make the call:

using LinearAlgebra

w = copy(v)            # VecDuplicate + VecCopy
dot(x, y)              # xᴴy, the global result on every rank
axpy!(α, x, y)         # y = αx + y
axpby!(α, x, β, y)     # y = αx + βy
rmul!(v, α)            # v = αv
z = x + y              # a new vector; so are x - y, -x, α * x and x / α

copy and the operators return a new PetscVec with the layout of their first vector argument. On more than one rank it has no finalizer, so call destroy! on it on every rank. Broadcasting is different for now: y .= α .* x .+ y updates y in place and works on any number of ranks, while an out-of-place broadcast such as 2 .* x returns a Julia Vector and needs an undistributed vector.

Assembly

After setting values, vectors must be assembled:

v[1] = 1.0
v[2] = 2.0
PETSc.assemble!(v)  # Finalize vector assembly

Ghost Point Updates

For vectors with ghost points (from DMDA/DMStag):

# Update ghost values from neighboring processes
PETSc.ghost_update!(vec; mode = PETSc.INSERT_VALUES, scatter = PETSc.SCATTER_FORWARD)

# Or use begin/end for non-blocking:
PETSc.ghost_update_begin!(vec; mode = PETSc.INSERT_VALUES, scatter = PETSc.SCATTER_FORWARD)
# ... do other work ...
PETSc.ghost_update_end!(vec; mode = PETSc.INSERT_VALUES, scatter = PETSc.SCATTER_FORWARD)

Functions

PETSc.LibPETSc.PetscVec — Method
PetscVec(petsclib, ptr::CVec; own::Bool = false)

Wrap a raw PETSc Vec handle.

own = false is the borrowed case of docs/src/man/naming.md §3.3: no finalizer is attached and destroy! is a no-op. The result is a VecPtr, the wrapper type that carries the ownership flag.

source
PETSc.LibPETSc.PetscVec — Method
PetscVec(v::AbstractPetscVec)

The plain PetscVec handle behind any high-level vector wrapper, borrowed from v: destroy! on it does nothing (see owns).

The autowrapped *AndMemType routines are typed x::PetscVec, while AbstractPetscVec also covers VecPtr; this converts transparently.

source
PETSc.LibPETSc.PetscVec — Method
v = PetscVec(petsclib, comm, array::Vector)

Creates a sequential PETSc vector on the communicator comm that uses array as its storage and keeps it alive. The element type of array must be the library's scalar type, which throws an ArgumentError otherwise: PETSc reads the memory as it is.

External Links

source
PETSc.LibPETSc.PetscVec — Method
PetscVec(petsclib, n::Integer)

A standard, sequentially-stored serial PETSc vector for petsclib.PetscScalar of length n.

Replaces v0.4's VecSeq: construction goes through the type (docs/src/man/naming.md §5.1).

External Links

source
PETSc.LibPETSc.PetscVec — Method
PetscVec(petsclib, comm, n::Integer; local_size = PETSC_DECIDE)

A standard PETSc vector of global length n distributed over comm, with local_size entries on this rank. Pass n = PETSC_DETERMINE to give only the local sizes. On a communicator of more than one rank the vector has no finalizer, since VecDestroy is collective: call destroy! on every rank.

External Links

source
PETSc.LibPETSc.PetscVec — Method
PetscVec(petsclib, v::Vector)

A standard, sequentially-stored serial PETSc vector, wrapping the Julia vector v.

This reuses the array v as storage, and so v should not be resize!-ed or otherwise have its length modified while the PETSc object exists. For the same reason v must be a Vector of exactly the library's scalar type; anything else throws an ArgumentError. Convert other arrays first: PetscVec(petsclib, Vector{PetscScalar}(w)). The vector keeps v alive, so PetscVec(petsclib, [1.0, 2.0]) is safe.

This should only be need to be called for more advanced uses, for most simple usecases, users should be able to pass Vectors directly and have the wrapping performed automatically

External Links

source
PETSc.VecPtr — Type
VecPtr(petsclib, ptr::CVec; own::Bool = false)

Container type for a PETSc Vec that is just a raw pointer.

If own is true a finalizer is set on the vector, but only on a serial communicator, since VecDestroy is collective and a GC finalizer runs at an arbitrary point. If own is false, the default, the handle belongs to PETSc and destroy! is a no-op, leaving the wrapper usable.

Calling VecPtr is deprecated and goes in v0.6: wrap a raw handle with PetscVec(petsclib, ptr; own).

source
PETSc.destroy! — Method
destroy!(v::AbstractPetscVec)

Destroy a PETSc vector and release its resources.

Safe to call more than once, and safe to reach as a GC finalizer after the library has been finalized or re-initialized: see isdestroyable. Does nothing on a vector that only borrows its handle: see owns.

External Links

source
PETSc.ownership_range — Method
ownership_range(vec::AbstractPetscVec)

The range of indices owned by this processor, assuming that the vec is laid out with the first n1 elements on the first processor, next n2 elements on the second, etc. For certain parallel layouts this range may not be well defined.

The range is 1-based, always: an index into Julia data is 1-based (docs/src/man/naming.md §12.1). v0.4 took base_one::Bool positionally and made the convention a runtime choice; ownership_range(v, false) warns in v0.5 and is a MethodError in v0.6.

Note

unlike the C function, the range returned is inclusive (idx_first:idx_last)

External Links

source
PETSc.type_name — Method
type_name(v::AbstractPetscVec)

The name PETSc knows this vector's implementation by, as a Symbol (:seq, :mpi, …), or nothing when no type has been set yet (docs/src/man/naming.md §3.1). v0.4 answered with a String; that is a break with no shim (§16).

External Links

source
PETSc.PetscVecStyle — Type
PetscVecStyle

The broadcast style of a PetscVec: an expression with one in it runs on the local arrays rather than entry by entry. It wins over an ordinary Vector, so x .+ v uses it too.

source
Base.copyto! — Method
copyto!(dst::AbstractPetscVec, src::AbstractPetscVec)

Copy the entries of src into dst and return dst. The two vectors must have the same global length, which throws a DimensionMismatch otherwise, and the same parallel layout, which PETSc checks.

External Links

source
Base.isapprox — Method
isapprox(v::AbstractPetscVec, w::AbstractPetscVec; atol, rtol, nans)

Whether every entry of v is approximately equal to the matching entry of w, with the keywords of isapprox for numbers applied entry by entry. Unlike isapprox on two Julia arrays, this does not compare the norm of the difference. It warns once: from v0.6 it compares norms, ‖v − w‖ ≤ max(atol, rtol·max(‖v‖, ‖w‖)), as for Julia arrays.

source
Base.similar — Method
similar(v::AbstractPetscVec)

A new vector with the same layout as v (size, parallel distribution, type), its entries not set. There is no similar(v, T) or similar(v, dims): every vector of a library holds that library's scalar type, and a different size has no layout to copy.

External Links

source
PETSc.AbstractPetscMemBackend — Type
AbstractPetscMemBackend

Abstract supertype for GPU memory backends used by PETSc extensions. Host memory is represented by nothing, not a subtype of this. GPU extensions define their own concrete subtype (e.g. CUDAMemBackend).

source
PETSc.acquire_local_array — Method
acquire_local_array(vec; read, write) -> (arr, cpu_arr, backend)

Get the local array from vec via VecGetArray*AndMemType without registering a Julia finalizer. Returns the user-visible array, the raw PETSc cpuarr needed for restore, and the backend singleton. Extensions overload `makelocalarray(cpuarr, backend)to wrap the raw array for their device (e.g.CUDAMemBackend→CuArray`).

source
PETSc.local_arrays — Method
local_arrays(petsclib, g_fx, l_x) -> (fx, lx, fx_arr, lx_arr, fx_bounce)

Check out the arrays of g_fx (read-write) and l_x (read-only) for a compute kernel; restore_local_arrays! must hand them back.

Deprecated: use with_local_arrays!, which restores the arrays also when the kernel throws.

source
PETSc.memtype — Method
memtype(vecs::AbstractPetscVec...)

Query the PetscMemType of each Vec and return the corresponding array type. Errors if the Vecs are on heterogeneous devices (different PetscMemType values), since a single with_local_array! call cannot handle mixed backends. Returns Vector when all Vecs are host-resident.

Extensions overload array_type(::Val{MT}) for a PetscMemType enum value MT to register the corresponding array type (e.g. PETSC_MEMTYPE_DEVICE → CuArray).

source
PETSc.memtype_backend — Method
memtype_backend(mtype::PetscMemType) → Nothing | AbstractPetscMemBackend

Convert a PetscMemType to a dispatch tag. Returns nothing for host memory; GPU extensions return their own singleton for device memory.

source
PETSc.release_local_array — Method
release_local_array(cpu_arr, backend, vec; read, write)

Restore a local array taken with acquire_local_array, after waiting for the device to finish with it. Called in finally blocks by with_local_array!.

source
PETSc.unsafe_local_array — Method
unsafe_local_array(vec::AbstractVec; read=true, write=true)

Return an Array{PetscScalar} containing local portion of the PETSc vec

Use read=false if the array is write-only; write=false if read-only.

The caller hands the array back: vec stays checked out, and the next VecGetArray or assembly on it fails, until Base.finalize runs on the array (or the garbage collector gets to it). Prefer with_local_array!, which hands it back when its block returns or throws; acquire_local_array and release_local_array are the explicit pair with no finalizer.

External Links

source
PETSc.with_local_array! — Method
with_local_array!(
    f!,
    vecs::NTuple{N, AbstractVec};
    read::Union{Bool, NTuple{N, Bool}} = true,
    write::Union{Bool, NTuple{N, Bool}} = true,
)
with_local_array!(::Type{A}, f!, vecs...; read, write) where {A <: AbstractArray}

Apply f! to local array views of vecs.

The optional ::Type{A} second argument (after the do-block function) asserts that every array returned from VecGetArray*AndMemType is of type A. Use it with do-block syntax:

with_local_array!(Vector, petsc_x; write=true) do x
    x .= 1
end
source
PETSc.with_local_arrays! — Method
with_local_arrays!(f!, g_fx::AbstractPetscVec, l_x::AbstractPetscVec)

Call f!(fx, lx) with the local array of g_fx (written) and of l_x (read), as arrays a compute kernel can use, and hand both back to PETSc afterwards, also when f! throws. Returns what f! returns.

Where each array lives is decided for the pair: with a GPU backend extension loaded and both Vecs on the device, fx and lx wrap the device memory. When g_fx is on the host, fx is a device scratch array whose contents are copied back into g_fx on return, so the kernel can run on the device either way. Without a GPU, both are plain host arrays.

with_local_arrays!(g_fx, l_x) do fx, lx
    residual_kernel!(fx, lx)
end

External Links

source
PETSc.ghost_update! — Method
ghost_update!(vec::AbstractPetscVec; mode = INSERT_VALUES, scatter = SCATTER_FORWARD)

Update the ghost values of vec and return it: ghost_update_begin! followed by ghost_update_end!. Use the two halves separately to overlap the communication with other work.

scatter = SCATTER_FORWARD copies the owned values into the ghost points; SCATTER_REVERSE sends the ghost values back to their owners, combined by mode (INSERT_VALUES or ADD_VALUES).

External Links

source
Base.:+ — Method
x + y
x - y
-x
α * x, x * α
x / α

Vector arithmetic on PetscVecs, each giving a new PetscVec with the layout of x. Collective. The result has a finalizer only on a one-rank communicator, as copy does. For an update in place use axpy!, axpby! and rmul!, or broadcast into an existing vector, y .= α .* x .+ y.

External Links

source
Base.copy — Method
copy(v::AbstractPetscVec)

A new PetscVec with the layout and entries of v. Collective. As from a constructor, the copy has a finalizer only on a one-rank communicator; on more ranks call destroy! on it on every rank.

External Links

source
LinearAlgebra.dot — Method
dot(x::AbstractPetscVec, y::AbstractPetscVec)

xᴴy, conjugating x as dot does for Julia arrays. Collective: every rank gets the global result.

External Links

source
LinearAlgebra.norm — Method
norm(v::AbstractPetscVec, p::Real = 2)
norm(v::AbstractPetscVec, normtype::NormType)

The p-norm of v, for p = 1, 2 or Inf; other values throw an ArgumentError. The second form passes a PETSc NormType straight to VecNorm.

External Links

source
LinearAlgebra.rmul! — Method
rmul!(v::AbstractPetscVec, α)
lmul!(α, v::AbstractPetscVec)

Scale v by α in place and return it. Collective.

External Links

source