DM

The DM module provides the base functionality for managing distributed data structures in PETSc. It serves as a foundation for various grid managers.

Overview

A DM object encapsulates the topology and data layout of a computational grid, enabling:

  • Parallel data distribution across MPI processes
  • Ghost point management for communication
  • Creation of vectors and matrices with appropriate parallel layouts
  • Multigrid hierarchy management

DM Types in PETSc

PETSc provides several DM implementations for different mesh types:

High-Level Interface Available in PETSc.jl

DM TypeDescriptionStatus
DMDADistributed arrays for structured grids (1D/2D/3D)✅ Full support
DMStagStaggered grids for finite volume/difference methods✅ Full support
DMPlexUnstructured meshes + full FEM workflow (Gmsh, FE spaces, callbacks, VTK)✅ Full support

Flavour is a type

DMDA, DMStag and DMPlex are Julia types, and constructing one returns that type:

da = PETSc.DMDA(petsclib, comm, (PETSc.DM_BOUNDARY_NONE,), (10,), 1, 1)
da isa PETSc.DMDA{typeof(petsclib), 1}     # true: flavour and dimension

All three are subtypes of LibPETSc.AbstractPetscDM, which is what a function working on any DM takes. corners, ghost_corners, local_indices, set_uniform_coordinates! and Base.size differ by flavour, so they are ordinary methods on DMDA and DMStag rather than one function comparing the string DMGetType returns.

LibPETSc.PetscDM remains the low-level handle: LibPETSc.DMCreate and friends have to return something before the flavour is known. Turn one into a typed handle with PETSc.narrow:

d = PETSc.narrow(dm)          # DMDA{L,N}, DMStag{L,N}, DMPlex{L}, or dm unchanged

narrow queries PETSc, so its return type is a wide Union and the call is a dynamic dispatch. One dispatch is cheap; propagating an abstractly-typed DM through a hot loop is not, so narrow once behind a function barrier. The accessors that hand back a DM PETSc owns — dm(ksp), dm(snes), dm(ts), coarse_dm — narrow for you, and what they return is a borrowed handle: it belongs to the object it was asked of, and destroy! on it is a no-op.

Low-Level Interface Only (via LibPETSc)

The following DM types are available through the low-level LibPETSc wrapper but do not yet have a convenient high-level Julia interface:

DM TypeDescriptionUse Case
DMForestAdaptive mesh refinement (AMR) via p4est/p8estOctree-based adaptivity
DMNetworkGraph/network structuresPower grids, pipe networks
DMSwarmParticle data managementPIC methods, Lagrangian particles
DMProductTensor product of DMsSemi-structured problems
DMSlicedSliced representationLegacy, specialized uses
DMShellUser-defined DMCustom implementations
DMCompositeComposition of multiple DMsMulti-physics coupling
DMRedundantRedundant storage on all ranksSmall coupled systems

Using Low-Level DM Types

For DM types without high-level wrappers, you can use the LibPETSc module directly:

using PETSc
using PETSc.LibPETSc

# Example: Create a DMPlex (low-level)
petsclib = PETSc.petsclibs[1]
dm = LibPETSc.DMPlexCreate(petsclib, MPI.COMM_WORLD)
# ... configure using LibPETSc functions ...
LibPETSc.DMDestroy(petsclib, dm)
Contributing

Contributions to add high-level interfaces for additional DM types are welcome! See the Contributing page for guidelines.

Functions

PETSc.DMPlex — Type
DMPlex{PetscLib}

A DMPLEX: an unstructured PETSc mesh.

Dimension is a runtime property here rather than a type parameter, because nothing dispatches on it and DMPlex(petsclib, comm) leaves it unset until setup. Ask for it with ndims(dm).

External Links

source
PETSc.LibPETSc.PetscMat — Method
PetscMat(da::AbstractPetscDM)

Create a sparse matrix (AIJ format) with sparsity pattern determined by the DM.

Replaces v0.4's MatAIJ: construction goes through the type (docs/src/man/naming.md §5.1).

Returns

A PetscMat object compatible with vectors from the DM.

External Links

source
Base.size — Method
size(dm::DMDA{PetscLib, N})
size(dm::DMStag{PetscLib, N})

Return the global size of the DM as an NTuple{N,Int}.

v0.4 returned (M, N, P) with the unused dimensions padded to 1; the result is now dimension-correct (§12), which is a break with no shim (§16).

External Links

source
PETSc.corners — Function
corners(dm)

Returns a NamedTuple with the global indices (excluding ghost points) of the lower and upper corners as well as the size. A DMStag also reports nextra, the number of extra partial elements in each direction.

Defined for DMDA and DMStag; the flavour is a type parameter, so any other DM is a MethodError rather than a runtime check.

source
PETSc.corners — Method
lower, upper, size = corners(da::DMDA{PetscLib, N})

Returns a NamedTuple with the global indices (excluding ghost points) of the lower and upper corners as well as the size.

The result is dimension-correct (§12): lower and upper are CartesianIndex{N} and size is an NTuple{N,Int}, with no padding to three entries. This is a break with no shim (§16).

External Links

source
PETSc.destroy! — Method
destroy!(dm::AbstractPetscDM)

Destroy a DM object and release associated resources.

This function is typically called automatically via finalizers when the object is garbage collected, but can be called explicitly to free resources immediately. Does nothing on a DM that only borrows its handle: see owns.

External Links

source
PETSc.ghost_corners — Function
ghost_corners(dm)

Returns a NamedTuple with the global indices (including ghost points) of the lower and upper corners as well as the size.

Defined for DMDA and DMStag.

source
PETSc.ghost_corners — Method
lower, upper, size = ghost_corners(da::DMDA{PetscLib, N})

Returns a NamedTuple with the global indices (including ghost points) of the lower and upper corners as well as the size of the local part of the domain.

Dimension-correct like corners: CartesianIndex{N} and NTuple{N,Int}.

External Links

source
PETSc.global_to_local! — Method
global_to_local!(lvec, dm, gvec, mode = INSERT_VALUES)

Transfer values from the global vector gvec to the local vector lvec associated with the dm object, including ghost point values from neighboring processes.

The written vector comes first and the dm follows it (§8); v0.4 had two spellings of this call, one taking the DM last and one taking it first, and they collapse onto this one. A shim forwards the v0.4 name dm_global_to_local! from the old order (§17.2).

Arguments

  • lvec::AbstractPetscVec: Local vector (destination, written)
  • dm::AbstractPetscDM: DM object
  • gvec::AbstractPetscVec: Global vector (source)
  • mode::InsertMode: Insert mode, either INSERT_VALUES or ADD_VALUES

External Links

source
PETSc.global_vec — Method
v::PetscVec = global_vec(dm::AbstractPetscDM{PetscLib}) where {PetscLib}

Returns a global vector v from the dm object.

source
PETSc.info — Method
info(dm::DMDA)

Get information about a DMDA.

Returns

A NamedTuple with the following fields, all of them N-dimensional for an N-dimensional DMDA (docs/src/man/naming.md §12):

  • dim: Dimension of the DMDA (1, 2, or 3)
  • global_size: NTuple{N,Int} with the global dimensions in each direction
  • procs: NTuple{N,Int} with the number of MPI processes in each direction
  • ndofs: Degrees of freedom per node
  • stencil_width: Width of the stencil
  • boundary_type: NTuple{N,DMBoundaryType} with the boundary type per direction
  • stencil_type: Stencil type, either DMDA_STENCIL_STAR or DMDA_STENCIL_BOX

v0.4 reported the stencil width twice, as s and as stencil_width, spelled the dof count dof and the process grid mpi_proc_size, and padded every tuple to three entries. All four are gone; this is a break with no shim (§16).

External Links

source
PETSc.local_coordinate_array — Method
local_coordinate_array(da::Union{DMDA, DMStag})

Return coordinate arrays for the local portion of the domain.

The returned arrays are OffsetArrays that can be addressed using global indices, accounting for ghost points.

External Links

source
PETSc.local_coordinates — Method
local_coordinates(dm::AbstractDM)

The coordinates of dm, ghost points included, as a local PetscVec.

Borrowed handle

The returned object is owned by the object it was asked of, not by the caller: it carries no finalizer and must not be destroyed. destroy! on it is a no-op (owns).

External Links

source
PETSc.local_to_global! — Method
local_to_global!(gvec, dm, lvec, mode = INSERT_VALUES)

Transfer values from the local vector lvec to the global vector gvec associated with the dm object.

The written vector comes first and the dm follows it (§8); v0.4 took the DM last. A shim forwards the v0.4 name dm_local_to_global! from the old order (§17.2); the new name only accepts the new one (§16).

Arguments

  • gvec::AbstractPetscVec: Global vector (destination, written)
  • dm::AbstractPetscDM: DM object
  • lvec::AbstractPetscVec: Local vector (source)
  • mode::InsertMode: Insert mode, either INSERT_VALUES or ADD_VALUES

External Links

source
PETSc.local_to_local! — Method
local_to_local!(dst::AbstractPetscVec, dm::AbstractPetscDM, src::AbstractPetscVec, mode = INSERT_VALUES)
local_to_local!(v::AbstractPetscVec, dm::AbstractPetscDM, mode = INSERT_VALUES)

Fill the ghost points of the local vector dst from the owned values of the local vector src on the neighbouring ranks, and return dst. dst may be src: the second form refreshes the ghost points of v in place.

External Links

source
PETSc.local_vec — Method
v::PetscVec = local_vec(dm::AbstractPetscDM{PetscLib}) where {PetscLib}

Returns a local vector v from the dm object.

source
PETSc.narrow — Method
narrow(dm::AbstractPetscDM; own = false)

A handle onto the same PETSc object whose Julia type carries the DM's flavour and, for a DMDA or a DMStag, its dimension.

LibPETSc.DMGetType and LibPETSc.DMGetDimension are queried, so the return type is a wide Union and the call is a dynamic dispatch. One dispatch is cheap; propagating an abstractly-typed DM through a hot loop is not, so narrow once behind a function barrier. A flavour with no type of its own comes back unchanged.

Borrowed handle

The returned object is owned by the object it was asked of, not by the caller: it carries no finalizer and must not be destroyed. destroy! on it is a no-op (owns).

The exception is own = true, which the callers that really do hand over a new object use: clone and distribute!.

External Links

source
PETSc.set_matrix_preallocate_only! — Method
set_matrix_preallocate_only!(dm::AbstractPetscDM, flag::Bool)

With flag set, a matrix made from dm by PetscMat is preallocated but its nonzero pattern is not inserted, so the first assembly defines the pattern instead of every coupling the stencil allows being stored as an explicit zero. The setting stays on dm for every later matrix. Returns dm.

External Links

source
PETSc.set_type! — Method
set_type!(dm::AbstractPetscDM, type::Symbol)

Set the DM flavour, for example :da, :stag or :plex.

This configures a raw PetscDM handle; the high-level constructors set the flavour themselves, and narrow is what puts it into the Julia type.

External Links

source
PETSc.set_uniform_coordinates! — Function
set_uniform_coordinates!(
    dm::AbstractPetscDM,
    xyzmin::NTuple{N, Real},
    xyzmax::NTuple{N, Real},
) where {N}

Set uniform coordinates on dm using the lower and upper corners defined by the NTuples xyzmin and xyzmax. If N is less than the dimension of the dm then the value of the trailing coordinates is set to 0.

Defined for DMDA and DMStag, which reach different PETSc calls; the flavour is a type parameter, so the two are ordinary methods.

External Links

source
PETSc.type_name — Method
type_name(dm::AbstractPetscDM)

The name PETSc knows this DM's flavour by, as a Symbol (:da, :stag, :plex, …), or nothing when no type has been set yet (docs/src/man/naming.md §3.1). v0.4 answered with a String; that is a break with no shim (§16).

The flavour is a type parameter in v0.5 (DMDA, DMStag, DMPlex), so dispatch, not this reader, is the way to branch on it.

External Links

source