GitHub

DMDA

The DMDA (Distributed Array) module provides functionality for creating and managing structured grids in 1D, 2D, or 3D.

Overview

DMDA is ideal for problems on regular structured grids where:

  • The grid is logically rectangular
  • Each grid point has the same number of degrees of freedom
  • Stencil operations follow a regular pattern (star or box stencils)

Creating a DMDA

# 2D grid example
da = DMDA(
    petsclib,
    MPI.COMM_WORLD,
    (PETSc.DM_BOUNDARY_NONE, PETSc.DM_BOUNDARY_NONE),  # boundary types
    (nx, ny),                                          # global dimensions
    1,                                                 # degrees of freedom per node
    1,                                                 # stencil width
    PETSc.DMDA_STENCIL_STAR,                           # stencil type
)

da isa PETSc.DMDA{typeof(petsclib), 2}   # true: the flavour and the dimension

DMDA is a Julia type rather than a factory function, so the dimension is in the type and the returns are dimension-correct: PETSc.corners(da).size is a 2-tuple for the grid above, and PETSc.info(da) reports ndofs and procs (naming conventions, §5.3 and §12).

Reading the local extent

c  = PETSc.corners(da)         # (; lower, upper, size), 1-based and inclusive
gc = PETSc.ghost_corners(da)   # the same, with the ghost points
i  = PETSc.info(da)            # (; dim, global_size, procs, ndofs, …)
ndims(da)                      # the dimension, as Base spells it

Colouring for a finite-difference Jacobian

# Takes no `petsclib`, and every index field says its base (naming.md §12.1)
c = PETSc.star_fd_coloring(da)
c.row_coo_local_0b     # straight to MatSetPreallocationCOOLocal
c.perturb_cols_1b      # indexes Julia arrays

Functions

PETSc.DMDA — Method
DMDA(
    petsclib::PetscLib
    comm::MPI.Comm,
    boundary_type::NTuple{D, DMBoundaryType},
    global_dim::NTuple{D, Integer},
    dof_per_node::Integer,
    stencil_width::Integer,
    stencil_type;
    points_per_proc::Tuple,
    processors::Tuple,
    setfromoptions = true,
    dmsetup = true,
    prefix = "",
    options...
)

Creates a D-dimensional distributed array with the options specified using keyword arguments.

If keyword argument points_per_proc[k] isa Vector{petsclib.PetscInt} then this specifies the points per processor in dimension k.

If keyword argument processors[k] isa Integer then this specifies the number of processors used in dimension k; ignored when D == 1.

If keyword argument setfromoptions == true then set_from_options! called.

If keyword argument dmsetup == true then setup! is called.

When D == 1 the stencil_type argument is not required and ignored if specified.

External Links

source
PETSc.local_interior_linear_index — Method
ind = local_interior_linear_index(dmda::DMDA)

Returns the linear indices associated with the degrees of freedom own by this MPI rank embedded in the ghost index space for the dmda

source
PETSc.reshape_local_array — Method
reshape_local_array(Arr, da::Union{DMDA, DMStag}, ndof = ndofs(da))

Returns an array with the same data as Arr but reshaped as an array that can be addressed with global indexing.

source
PETSc.star_fd_coloring — Method
star_fd_coloring(da::DMDA{PetscLib, 2})

Build all data needed for manual FD coloring of a 2-D DMDA with a STAR stencil, using IS_COLORING_LOCAL and ghost-local COO indexing.

petsclib is not an argument: the DMDA carries it as a type parameter (docs/src/man/naming.md §8). The dimension is a type parameter too, so a DMDA of any other dimension is a MethodError rather than a silently wrong answer.

Specifically, this function:

  1. Creates an IS_COLORING_LOCAL ISColoring via DMCreateColoring and extracts the per-DOF color vector (ghost-local layout).
  2. Enumerates all STAR-stencil (row, col) pairs for every owned node and records their ghost-local 0-based indices and colors.
  3. Builds per-color index arrays (perturb_cols_1b, coo_idxs_1b, local_rows_1b) ready for use in an FD coloring Newton loop.

Returns a NamedTuple. The index vectors carry the base they use in their name (§12.1), because this function returns both: the COO arrays go straight to LibPETSc.MatSetPreallocationCOOLocal and keep PETSc's base, while the rest index Julia arrays and are 1-based.

  • n_colors — number of colors
  • n_local_dofs — number of locally owned DOFs (owned nodes × dof/node)
  • nnz_coo — total number of COO entries
  • row_coo_local_0b — ghost-local 0-based row indices (Vector{PetscInt})
  • col_coo_local_0b — ghost-local 0-based column indices (Vector{PetscInt})
  • perturb_cols_1b — perturb_cols_1b[c]: 1-based owned-local column indices with color c-1; used to scatter +h perturbations.
  • coo_idxs_1b — coo_idxs_1b[c]: 1-based COO entry indices for color c-1
  • local_rows_1b — local_rows_1b[c]: corresponding 1-based owned-local residual-row indices; used to read (f1-f0)/h.
2-D DMDA STAR stencil only

Neighbor enumeration covers only ±x and ±y directions. The ghost-local flat-index formula is d + ix*dof + iy*dof*nx_g. For a 3-D DMDA:

  • add ±z neighbors guarded by kk > 1 / kk < mz,
  • extend the flat-index formula with + iz*dof*nx_g*ny_g,
  • reshape col_colors_mat to (dof, nx_g, ny_g, nz_g),
  • decode z_owned in the perturb_cols_1b loop.
source