GitHub

DM

The DM module provides the base functionality for managing distributed data structures in PETSc. It serves as a foundation for various grid managers.

Overview

A DM object encapsulates the topology and data layout of a computational grid, enabling:

  • Parallel data distribution across MPI processes
  • Ghost point management for communication
  • Creation of vectors and matrices with appropriate parallel layouts
  • Multigrid hierarchy management

DM Types in PETSc

PETSc provides several DM implementations for different mesh types:

High-Level Interface Available in PETSc.jl

DM TypeDescriptionStatus
DMDADistributed arrays for structured grids (1D/2D/3D)✅ Full support
DMStagStaggered grids for finite volume/difference methods✅ Full support
DMPlexUnstructured meshes + full FEM workflow (Gmsh, FE spaces, callbacks, VTK)✅ Full support

Flavour is a type

DMDA, DMStag and DMPlex are Julia types, and constructing one returns that type:

da = PETSc.DMDA(petsclib, comm, (PETSc.DM_BOUNDARY_NONE,), (10,), 1; stencil_width = 1)
da isa PETSc.DMDA{typeof(petsclib), 1}     # true: flavour and dimension

All three are subtypes of LibPETSc.AbstractPetscDM, which is what a function working on any DM takes. corners, ghost_corners, local_indices, set_uniform_coordinates! and Base.size differ by flavour, so they are ordinary methods on DMDA and DMStag rather than one function comparing the string DMGetType returns.

LibPETSc.PetscDM remains the low-level handle: LibPETSc.DMCreate and friends have to return something before the flavour is known. Turn one into a typed handle with PETSc.narrow:

d = PETSc.narrow(dm)          # DMDA{L,N}, DMStag{L,N}, DMPlex{L}, or dm unchanged

narrow queries PETSc, so its return type is a wide Union and the call is a dynamic dispatch. One dispatch is cheap; propagating an abstractly-typed DM through a hot loop is not, so narrow once behind a function barrier. The accessors that hand back a DM PETSc owns — dm(ksp), dm(snes), dm(ts), coarse_dm — narrow for you, and what they return is a borrowed handle: it belongs to the object it was asked of, and destroy! on it is a no-op.

Low-Level Interface Only (via LibPETSc)

The following DM types are available through the low-level LibPETSc wrapper but do not yet have a convenient high-level Julia interface:

DM TypeDescriptionUse Case
DMForestAdaptive mesh refinement (AMR) via p4est/p8estOctree-based adaptivity
DMNetworkGraph/network structuresPower grids, pipe networks
DMSwarmParticle data managementPIC methods, Lagrangian particles
DMProductTensor product of DMsSemi-structured problems
DMSlicedSliced representationLegacy, specialized uses
DMShellUser-defined DMCustom implementations
DMCompositeComposition of multiple DMsMulti-physics coupling
DMRedundantRedundant storage on all ranksSmall coupled systems

Using Low-Level DM Types

For DM types without high-level wrappers, you can use the LibPETSc module directly:

using PETSc
using PETSc.LibPETSc

# Example: Create a DMPlex (low-level)
petsclib = PETSc.petsclibs[1]
dm = LibPETSc.DMPlexCreate(petsclib, MPI.COMM_WORLD)
# ... configure using LibPETSc functions ...
LibPETSc.DMDestroy(petsclib, dm)
Contributing

Contributions to add high-level interfaces for additional DM types are welcome! See the Contributing page for guidelines.

Functions

PETSc.DMPlex — Type
DMPlex{PetscLib}

A DMPLEX: an unstructured PETSc mesh.

Dimension is a runtime property here rather than a type parameter, because nothing dispatches on it and DMPlex(petsclib, comm) leaves it unset until setup. Ask for it with ndims(dm).

External Links

source
PETSc.LibPETSc.PetscDM — Type
PetscDM{PetscLib}

Low-level handle to a PETSc DM of any flavour, as LibPETSc.DMCreate and the other generated functions return it. narrow gives the typed handle, DMDA, DMStag or DMPlex, that the high-level functions dispatch on.

source
PETSc.LibPETSc.PetscMat — Method
PetscMat(da::AbstractPetscDM)

Create a sparse matrix (AIJ format) with sparsity pattern determined by the DM.

Replaces v0.4's MatAIJ: construction goes through the type (docs/src/man/naming.md §5.1).

Returns

A PetscMat object compatible with vectors from the DM.

External Links

source
PETSc.add_coarsen_hook! — Function
add_coarsen_hook!(dm, coarsenhook, restricthook = C_NULL)

Register a C callback invoked each time dm is coarsened (e.g. during FAS hierarchy setup). The hooks are C function pointers, not Julia functions, so dm comes first as in set_residual!. The coarsenhook signature is:

hook(fine::Ptr{Cvoid}, coarse::Ptr{Cvoid}, ctx::Ptr{Cvoid}) → Cint

Use @cfunction(f, Cint, (Ptr{Cvoid}, Ptr{Cvoid}, Ptr{Cvoid})) to create the pointer. restricthook (optional) is called on each nonlinear solve restriction step.

source
PETSc.clone — Method
clone(dm::AbstractPetscDM) -> AbstractPetscDM

Return a new DM that is a clone of dm (same topology, no fields or DS).

The clone is a new object the caller owns, and comes back narrowed to its flavour.

External Links

source
PETSc.coarse_dm — Function
coarse_dm(dm::AbstractPetscDM) -> AbstractPetscDM

Return the coarse DM from which dm was obtained by refinement (e.g. via -dm_refine_hierarchy). Check convert(Ptr{Cvoid}, cdm) == C_NULL to detect when there is no coarser level.

Borrowed handle

The returned object is owned by the object it was asked of, not by the caller: it carries no finalizer and must not be destroyed. destroy! on it is a no-op (owns).

External Links

source
PETSc.corners — Function
corners(dm)

Returns a NamedTuple with the global indices (excluding ghost points) of the lower and upper corners as well as the size. A DMStag also reports nextra, the number of extra partial elements in each direction.

Defined for DMDA and DMStag; the flavour is a type parameter, so any other DM is a MethodError rather than a runtime check.

source
PETSc.destroy! — Method
destroy!(dm::AbstractPetscDM)

Destroy a DM object and release associated resources.

This function is typically called automatically via finalizers when the object is garbage collected, but can be called explicitly to free resources immediately. Does nothing on a DM that only borrows its handle: see owns.

External Links

source
PETSc.ghost_corners — Function
ghost_corners(dm)

Returns a NamedTuple with the global indices (including ghost points) of the lower and upper corners as well as the size.

Defined for DMDA and DMStag.

source
PETSc.global_to_local! — Method
global_to_local!(lvec, dm, gvec; mode = INSERT_VALUES)

Transfer values from the global vector gvec to the local vector lvec associated with the dm object, including ghost point values from neighboring processes.

The written vector comes first and the dm follows it (§8); v0.4 had two spellings of this call, one taking the DM last and one taking it first, and they collapse onto this one. A shim forwards the v0.4 name dm_global_to_local! from the old order (§17.2).

Arguments

  • lvec::AbstractPetscVec: Local vector (destination, written)
  • dm::AbstractPetscDM: DM object
  • gvec::AbstractPetscVec: Global vector (source)
  • mode::InsertMode (keyword): INSERT_VALUES or ADD_VALUES

External Links

source
PETSc.global_vec — Method
v::PetscVec = global_vec(dm::AbstractPetscDM{PetscLib}) where {PetscLib}

Returns a global vector v from the dm object.

source
PETSc.local_coordinates — Method
local_coordinates(dm::AbstractDM)

The coordinates of dm, ghost points included, as a local PetscVec.

Borrowed handle

The returned object is owned by the object it was asked of, not by the caller: it carries no finalizer and must not be destroyed. destroy! on it is a no-op (owns).

External Links

source
PETSc.local_to_global! — Method
local_to_global!(gvec, dm, lvec; mode = INSERT_VALUES)

Transfer values from the local vector lvec to the global vector gvec associated with the dm object.

The written vector comes first and the dm follows it (§8); v0.4 took the DM last. A shim forwards the v0.4 name dm_local_to_global! from the old order (§17.2); the new name only accepts the new one (§16).

Arguments

  • gvec::AbstractPetscVec: Global vector (destination, written)
  • dm::AbstractPetscDM: DM object
  • lvec::AbstractPetscVec: Local vector (source)
  • mode::InsertMode (keyword): INSERT_VALUES or ADD_VALUES

External Links

source
PETSc.local_to_local! — Method
local_to_local!(dst::AbstractPetscVec, dm::AbstractPetscDM, src::AbstractPetscVec; mode = INSERT_VALUES)
local_to_local!(v::AbstractPetscVec, dm::AbstractPetscDM; mode = INSERT_VALUES)

Fill the ghost points of the local vector dst from the owned values of the local vector src on the neighbouring ranks, and return dst. dst may be src: the second form refreshes the ghost points of v in place.

External Links

source
PETSc.local_vec — Method
v::PetscVec = local_vec(dm::AbstractPetscDM{PetscLib}) where {PetscLib}

Returns a local vector v from the dm object.

source
PETSc.narrow — Method
narrow(dm::AbstractPetscDM; own = false)

A handle onto the same PETSc object whose Julia type carries the DM's flavour and, for a DMDA or a DMStag, its dimension.

LibPETSc.DMGetType and LibPETSc.DMGetDimension are queried, so the return type is a wide Union and the call is a dynamic dispatch. One dispatch is cheap; propagating an abstractly-typed DM through a hot loop is not, so narrow once behind a function barrier. A flavour with no type of its own comes back unchanged.

Borrowed handle

The returned object is owned by the object it was asked of, not by the caller: it carries no finalizer and must not be destroyed. destroy! on it is a no-op (owns).

The exception is own = true, which the callers that really do hand over a new object use: clone and distribute.

External Links

source
PETSc.set_matrix_preallocate_only! — Method
set_matrix_preallocate_only!(dm::AbstractPetscDM, flag::Bool)

With flag set, a matrix made from dm by PetscMat is preallocated but its nonzero pattern is not inserted, so the first assembly defines the pattern instead of every coupling the stencil allows being stored as an explicit zero. The setting stays on dm for every later matrix. Returns dm.

External Links

source
PETSc.set_type! — Method
set_type!(dm::AbstractPetscDM, type::Symbol)

Set the DM flavour, for example :da, :stag or :plex.

This configures a raw PetscDM handle; the high-level constructors set the flavour themselves, and narrow is what puts it into the Julia type.

External Links

source
PETSc.set_uniform_coordinates! — Function
set_uniform_coordinates!(
    dm::AbstractPetscDM,
    xyzmin::NTuple{N, Real},
    xyzmax::NTuple{N, Real},
) where {N}

Set uniform coordinates on dm using the lower and upper corners defined by the NTuples xyzmin and xyzmax. If N is less than the dimension of the dm then the value of the trailing coordinates is set to 0.

Defined for DMDA and DMStag, which reach different PETSc calls; the flavour is a type parameter, so the two are ordinary methods.

External Links

source
PETSc.type_name — Method
type_name(dm::AbstractPetscDM)

The name PETSc knows this DM's flavour by, as a Symbol (:da, :stag, :plex, …), or nothing when no type has been set yet (docs/src/man/naming.md §3.1). v0.4 answered with a String; that is a break with no shim (§16).

The flavour is a type parameter in v0.5 (DMDA, DMStag, DMPlex), so dispatch, not this reader, is the way to branch on it.

External Links

source
PETSc.with_local_coordinate_array — Method
with_local_coordinate_array(f, dm::Union{DMDA, DMStag})

Call f(coord) with the coordinates of the local part of dm, ghost points included, and hand the array back to PETSc afterwards, also when f throws. Returns what f returns.

coord is an OffsetArray addressed with global indices, the coordinate direction first: coord[1, i, j] is the x coordinate of point (i, j). It reads PETSc's memory, so it is valid only inside f; return copy(coord) to keep the values.

coord = with_local_coordinate_array(copy, da)

External Links

source