GitHub

Low-Level Interface (LibPETSc)

PETSc.jl provides two ways to interact with the PETSc library:

  1. High-level interface: Julia-friendly wrappers that handle memory management, use 1-based indexing, and provide convenient syntax (e.g., A[1,2] = 3.0)
  2. Low-level interface (LibPETSc): Direct access to nearly the entire PETSc C API (~13,000+ functions)

This guide focuses on the low-level interface.

When to Use the Low-Level Interface

Use the low-level LibPETSc interface when:

  • You need PETSc functionality not yet wrapped in the high-level interface
  • You're porting existing PETSc C/Fortran code to Julia
  • You need fine-grained control over PETSc objects
  • You want to access advanced or experimental PETSc features
  • Performance-critical code requires eliminating Julia wrapper overhead

For most common tasks (creating vectors/matrices, solving linear systems), the high-level interface is recommended.

Basic Usage Pattern

The low-level interface follows the PETSc C API closely. All functions require a petsclib parameter as the first argument:

using PETSc

# Get a library instance (usually you'll use the first one)
petsclib = PETSc.petsclibs[1]

# Initialize PETSc
PETSc.initialize(petsclib)

# Create and use PETSc objects
vec = LibPETSc.VecCreate(petsclib, LibPETSc.PETSC_COMM_SELF)
LibPETSc.VecSetSizes(petsclib, vec, 10, 10)
LibPETSc.VecSetType(petsclib, vec, "seq")  # Set vector type
LibPETSc.VecSetFromOptions(petsclib, vec)

# Work with the vector
LibPETSc.VecSet(petsclib, vec, 1.0)
println("Vector size: ", LibPETSc.VecGetSize(petsclib, vec))

# Clean up
LibPETSc.VecDestroy(petsclib, vec)
PETSc.finalize(petsclib)

Key Concepts

1. The petsclib Parameter

Every low-level function requires a petsclib instance. This specifies which PETSc library variant to use (e.g., Float64 vs Float32, Int32 vs Int64):

# Default libraries available (uses prebuilt PETSc binaries)
petsclib = PETSc.petsclibs[1]  # Usually Float64, Int64

# Access the scalar and integer types
PetscScalar = petsclib.PetscScalar  # The floating-point type (e.g., Float64)
PetscInt = petsclib.PetscInt        # The integer type (e.g., Int64)
PetscReal = petsclib.PetscReal      # The real type (real part of PetscScalar)

Using a Custom PETSc Build

You can link to your own custom build of PETSc instead of using the prebuilt binaries. This is useful when you need:

  • Specific configure options not available in the default build
  • Custom optimizations for your hardware
  • Debug builds for troubleshooting
  • Integration with specific external packages

To persistently configure a custom PETSc installation (recommended):

using PETSc
PETSc.set_library!("/path/to/your/libpetsc.so"; PetscScalar=Float64, PetscInt=Int64)
# Restart Julia — PETSc_jll is not loaded and your library is used automatically.

To revert: PETSc.unset_library!(). To inspect the current config: PETSc.library_info().

For a one-off session without changing persistent settings, use the PetscLibType constructor:

petsclib = PETSc.LibPETSc.PetscLibType("/path/to/your/libpetsc.so";
                              PetscScalar=Float64, PetscInt=Int64)
PETSc.initialize(petsclib)
# ... your code using petsclib ...
PETSc.finalize(petsclib)

Important notes for custom builds:

  • The library must be compiled as a dynamic/shared library (libpetsc.so / .dylib)
  • PetscScalar and PetscInt must match how your PETSc was configured
  • Your PETSc must be linked against the same MPI that MPI.jl uses
  • You can check available precompiled libraries with [PETSc.petsclibs...]
PETSc.LibPETSc.getlib — Function
getlib(; PetscScalar = Float64, PetscInt = Int64)

Return the PetscLibType with the associated parameters

source

2. Zero-Based Indexing

Important: The low-level interface uses 0-based indexing (C convention), while Julia uses 1-based indexing:

# Low-level (0-based)
indices = PetscInt[0, 1, 2]  # First three elements
LibPETSc.VecSetValues(petsclib, vec, 3, indices, values, INSERT_VALUES)

# High-level (1-based)
vec[1] = value  # First element

3. Error Handling

Low-level functions return PetscErrorCode. Use the @chk macro to check for errors:

using PETSc.LibPETSc: @chk

err = LibPETSc.VecCreate(petsclib, MPI.COMM_SELF)
@chk err  # Throws an error if PETSc returned non-zero

4. String-valued types

PETSc "string enums" such as PCType, KSPType or MatType are C strings (typedef const char *PCType). Every wrapper that takes one accepts a Julia String, and every XGetType returns a String. The registered names are available as constants in LibPETSc, so both spellings below are equivalent:

LibPETSc.MatSetType(petsclib, mat, "seqaij")
LibPETSc.MatSetType(petsclib, mat, LibPETSc.MATSEQAIJ)

LibPETSc.KSPSetType(petsclib, ksp, LibPETSc.KSPGMRES)
LibPETSc.PCSetType(petsclib, pc, LibPETSc.PCILU)
LibPETSc.DMSetType(petsclib, dm, LibPETSc.DMPLEX)
LibPETSc.PetscViewerSetType(petsclib, viewer, LibPETSc.PETSCVIEWERASCII)

LibPETSc.KSPGetType(petsclib, ksp) == LibPETSc.KSPGMRES   # true

Most wrapper functions already include error checking, but when calling C functions directly, use @chk.

4. Memory Management

PETSc objects created with Create functions must be destroyed with corresponding Destroy functions:

# Create
mat = LibPETSc.MatCreate(petsclib, MPI.COMM_WORLD)
LibPETSc.MatSetSizes(petsclib, mat, m_local, n_local, m_global, n_global)

# ... use mat ...

# Destroy when done
LibPETSc.MatDestroy(petsclib, mat)

For serial (single-process) objects, the high-level interface handles this automatically via finalizers.

PETSc.destroy! works on every handle, including the ones with no high-level layer (IS, AO, PF, Tao). Unlike the raw Destroy call, it does nothing on a borrowed handle (see PETSc.owns), and it is safe to call twice or after a finalize/initialize cycle. length(is) is an index set's global size.

Some functions hand back memory PETSc allocated and say the caller frees it with PetscFree(). PetscFree is a C macro; LibPETSc.PetscFree(petsclib, ptr) does the same from Julia. DMCreateFieldIS and ISColoringGetIS (with PETSC_OWN_POINTER) copy what they return into Julia Vectors and free the C arrays themselves.

PETSc.destroy! — Method
destroy!(v::AbstractPetscVec)

Destroy a PETSc vector and release its resources.

Safe to call more than once, and safe to reach as a GC finalizer after the library has been finalized or re-initialized: see isdestroyable. Does nothing on a vector that only borrows its handle: see owns.

External Links

source
destroy!(m::AbstractPetscMat)

Destroy a Mat (matrix) object and release associated resources.

This function is typically called automatically via finalizers when the object is garbage collected, but can be called explicitly to free resources immediately. Does nothing on a matrix that only borrows its handle: see owns.

External Links

source
destroy!(opts::AbstractPetscOptions)

Free the options database opts holds, if this process is still allowed to.

Does nothing when the library has been finalized or re-initialized, or when opts was already destroyed: see isdestroyable. Does nothing on a borrowed handle either: see owns.

source
destroy!(ts::AbstractTS)

Destroy ts and release the options database attached to it.

The call is a no-op when the library has been finalized or when ts predates the current initialize/finalize cycle, so a stale handle never reaches TSDestroy. Does nothing on a borrowed handle: see owns.

External Links

source
destroy!(ksp::KSP)

Destroy the solver ksp holds. Does nothing on a borrowed handle, such as the one ksp(ts) hands back: see owns.

External Links

source
destroy!(p::AbstractPC)

Destroy the preconditioner p holds. Does nothing on the borrowed handle pc hands back, which its KSP destroys: see owns.

External Links

source
destroy!(is::LibPETSc.AbstractIS)

Destroy the index set is holds. Safe to call more than once. Does nothing on a borrowed index set, such as one ISColoringGetIS leaves with its coloring: see owns.

External Links

source
destroy!(ao::LibPETSc.AbstractAO)

Destroy the application ordering ao holds. Safe to call more than once; does nothing on a borrowed handle: see owns.

External Links

source
destroy!(pf::LibPETSc.AbstractPF)

Destroy the mathematical function pf holds. Safe to call more than once; does nothing on a borrowed handle: see owns.

External Links

source
destroy!(tao::LibPETSc.AbstractTao)

Destroy the optimization solver tao holds. Safe to call more than once; does nothing on a borrowed handle: see owns.

External Links

source
destroy!(snes::AbstractSNES)

Destroy a SNES (nonlinear solver) object and release associated resources.

This function is typically called automatically via finalizers when the object is garbage collected, but can be called explicitly to free resources immediately. Does nothing on a borrowed handle, such as the one snes(ts) hands back.

External Links

source
destroy!(dm::AbstractPetscDM)

Destroy a DM object and release associated resources.

This function is typically called automatically via finalizers when the object is garbage collected, but can be called explicitly to free resources immediately. Does nothing on a DM that only borrows its handle: see owns.

External Links

source
PETSc.destroy! — Method
destroy!(v::AbstractPetscVec)

Destroy a PETSc vector and release its resources.

Safe to call more than once, and safe to reach as a GC finalizer after the library has been finalized or re-initialized: see isdestroyable. Does nothing on a vector that only borrows its handle: see owns.

External Links

source
destroy!(m::AbstractPetscMat)

Destroy a Mat (matrix) object and release associated resources.

This function is typically called automatically via finalizers when the object is garbage collected, but can be called explicitly to free resources immediately. Does nothing on a matrix that only borrows its handle: see owns.

External Links

source
destroy!(opts::AbstractPetscOptions)

Free the options database opts holds, if this process is still allowed to.

Does nothing when the library has been finalized or re-initialized, or when opts was already destroyed: see isdestroyable. Does nothing on a borrowed handle either: see owns.

source
destroy!(ts::AbstractTS)

Destroy ts and release the options database attached to it.

The call is a no-op when the library has been finalized or when ts predates the current initialize/finalize cycle, so a stale handle never reaches TSDestroy. Does nothing on a borrowed handle: see owns.

External Links

source
destroy!(ksp::KSP)

Destroy the solver ksp holds. Does nothing on a borrowed handle, such as the one ksp(ts) hands back: see owns.

External Links

source
destroy!(p::AbstractPC)

Destroy the preconditioner p holds. Does nothing on the borrowed handle pc hands back, which its KSP destroys: see owns.

External Links

source
destroy!(is::LibPETSc.AbstractIS)

Destroy the index set is holds. Safe to call more than once. Does nothing on a borrowed index set, such as one ISColoringGetIS leaves with its coloring: see owns.

External Links

source
destroy!(ao::LibPETSc.AbstractAO)

Destroy the application ordering ao holds. Safe to call more than once; does nothing on a borrowed handle: see owns.

External Links

source
destroy!(pf::LibPETSc.AbstractPF)

Destroy the mathematical function pf holds. Safe to call more than once; does nothing on a borrowed handle: see owns.

External Links

source
destroy!(tao::LibPETSc.AbstractTao)

Destroy the optimization solver tao holds. Safe to call more than once; does nothing on a borrowed handle: see owns.

External Links

source
destroy!(snes::AbstractSNES)

Destroy a SNES (nonlinear solver) object and release associated resources.

This function is typically called automatically via finalizers when the object is garbage collected, but can be called explicitly to free resources immediately. Does nothing on a borrowed handle, such as the one snes(ts) hands back.

External Links

source
destroy!(dm::AbstractPetscDM)

Destroy a DM object and release associated resources.

This function is typically called automatically via finalizers when the object is garbage collected, but can be called explicitly to free resources immediately. Does nothing on a DM that only borrows its handle: see owns.

External Links

source
PETSc.destroy! — Method
destroy!(v::AbstractPetscVec)

Destroy a PETSc vector and release its resources.

Safe to call more than once, and safe to reach as a GC finalizer after the library has been finalized or re-initialized: see isdestroyable. Does nothing on a vector that only borrows its handle: see owns.

External Links

source
destroy!(m::AbstractPetscMat)

Destroy a Mat (matrix) object and release associated resources.

This function is typically called automatically via finalizers when the object is garbage collected, but can be called explicitly to free resources immediately. Does nothing on a matrix that only borrows its handle: see owns.

External Links

source
destroy!(opts::AbstractPetscOptions)

Free the options database opts holds, if this process is still allowed to.

Does nothing when the library has been finalized or re-initialized, or when opts was already destroyed: see isdestroyable. Does nothing on a borrowed handle either: see owns.

source
destroy!(ts::AbstractTS)

Destroy ts and release the options database attached to it.

The call is a no-op when the library has been finalized or when ts predates the current initialize/finalize cycle, so a stale handle never reaches TSDestroy. Does nothing on a borrowed handle: see owns.

External Links

source
destroy!(ksp::KSP)

Destroy the solver ksp holds. Does nothing on a borrowed handle, such as the one ksp(ts) hands back: see owns.

External Links

source
destroy!(p::AbstractPC)

Destroy the preconditioner p holds. Does nothing on the borrowed handle pc hands back, which its KSP destroys: see owns.

External Links

source
destroy!(is::LibPETSc.AbstractIS)

Destroy the index set is holds. Safe to call more than once. Does nothing on a borrowed index set, such as one ISColoringGetIS leaves with its coloring: see owns.

External Links

source
destroy!(ao::LibPETSc.AbstractAO)

Destroy the application ordering ao holds. Safe to call more than once; does nothing on a borrowed handle: see owns.

External Links

source
destroy!(pf::LibPETSc.AbstractPF)

Destroy the mathematical function pf holds. Safe to call more than once; does nothing on a borrowed handle: see owns.

External Links

source
destroy!(tao::LibPETSc.AbstractTao)

Destroy the optimization solver tao holds. Safe to call more than once; does nothing on a borrowed handle: see owns.

External Links

source
destroy!(snes::AbstractSNES)

Destroy a SNES (nonlinear solver) object and release associated resources.

This function is typically called automatically via finalizers when the object is garbage collected, but can be called explicitly to free resources immediately. Does nothing on a borrowed handle, such as the one snes(ts) hands back.

External Links

source
destroy!(dm::AbstractPetscDM)

Destroy a DM object and release associated resources.

This function is typically called automatically via finalizers when the object is garbage collected, but can be called explicitly to free resources immediately. Does nothing on a DM that only borrows its handle: see owns.

External Links

source
PETSc.destroy! — Method
destroy!(v::AbstractPetscVec)

Destroy a PETSc vector and release its resources.

Safe to call more than once, and safe to reach as a GC finalizer after the library has been finalized or re-initialized: see isdestroyable. Does nothing on a vector that only borrows its handle: see owns.

External Links

source
destroy!(m::AbstractPetscMat)

Destroy a Mat (matrix) object and release associated resources.

This function is typically called automatically via finalizers when the object is garbage collected, but can be called explicitly to free resources immediately. Does nothing on a matrix that only borrows its handle: see owns.

External Links

source
destroy!(opts::AbstractPetscOptions)

Free the options database opts holds, if this process is still allowed to.

Does nothing when the library has been finalized or re-initialized, or when opts was already destroyed: see isdestroyable. Does nothing on a borrowed handle either: see owns.

source
destroy!(ts::AbstractTS)

Destroy ts and release the options database attached to it.

The call is a no-op when the library has been finalized or when ts predates the current initialize/finalize cycle, so a stale handle never reaches TSDestroy. Does nothing on a borrowed handle: see owns.

External Links

source
destroy!(ksp::KSP)

Destroy the solver ksp holds. Does nothing on a borrowed handle, such as the one ksp(ts) hands back: see owns.

External Links

source
destroy!(p::AbstractPC)

Destroy the preconditioner p holds. Does nothing on the borrowed handle pc hands back, which its KSP destroys: see owns.

External Links

source
destroy!(is::LibPETSc.AbstractIS)

Destroy the index set is holds. Safe to call more than once. Does nothing on a borrowed index set, such as one ISColoringGetIS leaves with its coloring: see owns.

External Links

source
destroy!(ao::LibPETSc.AbstractAO)

Destroy the application ordering ao holds. Safe to call more than once; does nothing on a borrowed handle: see owns.

External Links

source
destroy!(pf::LibPETSc.AbstractPF)

Destroy the mathematical function pf holds. Safe to call more than once; does nothing on a borrowed handle: see owns.

External Links

source
destroy!(tao::LibPETSc.AbstractTao)

Destroy the optimization solver tao holds. Safe to call more than once; does nothing on a borrowed handle: see owns.

External Links

source
destroy!(snes::AbstractSNES)

Destroy a SNES (nonlinear solver) object and release associated resources.

This function is typically called automatically via finalizers when the object is garbage collected, but can be called explicitly to free resources immediately. Does nothing on a borrowed handle, such as the one snes(ts) hands back.

External Links

source
destroy!(dm::AbstractPetscDM)

Destroy a DM object and release associated resources.

This function is typically called automatically via finalizers when the object is garbage collected, but can be called explicitly to free resources immediately. Does nothing on a DM that only borrows its handle: see owns.

External Links

source
Base.length — Method
length(is::LibPETSc.AbstractIS)

The global number of indices in is, summed over its ranks, as length of a PetscVec is its global length.

External Links

source
PETSc.LibPETSc.PetscFree — Function
PetscFree(petsclib::PetscLibType, ptr::Ptr)

Frees memory that PETSc allocated with PetscMalloc() and handed to the caller, such as the arrays some Create and Get functions return with the note "the caller must free it with PetscFree()". Does nothing on C_NULL.

PetscFree() is a C macro, so this calls the allocator's free routine PetscTrFree directly, which also honours an allocator installed with PetscMallocSet().

Not Collective

Input Parameter:

  • ptr - memory allocated by PETSc

Level: beginner

See also: PetscMalloc(), PetscMallocSet()

External Links

source

The Get...Array functions return a PetscArray that points into PETSc's storage. Pass it back to the matching Restore...Array function when you are done with it.

PETSc.LibPETSc.PetscArray — Type
PetscArray{T,N} <: AbstractArray{T,N}

Behaves like a normal Julia array but holds a pointer to PETSc data, which can be destroyed

source

5. Assembly

After setting values in vectors or matrices, you must call assembly functions:

# Set values
LibPETSc.VecSetValues(petsclib, vec, n, indices, values, INSERT_VALUES)

# Assemble
LibPETSc.VecAssemblyBegin(petsclib, vec)
LibPETSc.VecAssemblyEnd(petsclib, vec)

Common Patterns

Creating and Filling a Vector

petsclib = PETSc.petsclibs[1]
PetscInt = petsclib.PetscInt
PetscScalar = petsclib.PetscScalar

# Create vector
vec = LibPETSc.VecCreate(petsclib, MPI.COMM_SELF)
LibPETSc.VecSetSizes(petsclib, vec, 10, 10)
LibPETSc.VecSetType(petsclib, vec, "seq")  # Set vector type
LibPETSc.VecSetFromOptions(petsclib, vec)

# Set values (0-based indices!)
indices = PetscInt[0, 1, 2, 3, 4]
values = PetscScalar[1.0, 2.0, 3.0, 4.0, 5.0]
LibPETSc.VecSetValues(petsclib, vec, 5, indices, values, INSERT_VALUES)

# Assemble
LibPETSc.VecAssemblyBegin(petsclib, vec)
LibPETSc.VecAssemblyEnd(petsclib, vec)

# View (print to stdout)
LibPETSc.VecView(petsclib, vec, C_NULL)

# Clean up
LibPETSc.VecDestroy(petsclib, vec)

Creating a Sparse Matrix

# Create matrix
mat = LibPETSc.MatCreate(petsclib, MPI.COMM_SELF)
LibPETSc.MatSetSizes(petsclib, mat, 5, 5, 5, 5)
LibPETSc.MatSetType(petsclib, mat, "seqaij")
LibPETSc.MatSetUp(petsclib, mat)

# Set values (0-based indexing!)
row = PetscInt[0]
cols = PetscInt[0, 1]
vals = PetscScalar[2.0, -1.0]
LibPETSc.MatSetValues(petsclib, mat, 1, row, 2, cols, vals, INSERT_VALUES)

# Assemble
LibPETSc.MatAssemblyBegin(petsclib, mat, LibPETSc.MAT_FINAL_ASSEMBLY)
LibPETSc.MatAssemblyEnd(petsclib, mat, LibPETSc.MAT_FINAL_ASSEMBLY)

# View
LibPETSc.MatView(petsclib, mat, C_NULL)

# Clean up
LibPETSc.MatDestroy(petsclib, mat)

Solving a Linear System

# Create KSP solver
ksp = LibPETSc.KSPCreate(petsclib, MPI.COMM_SELF)
LibPETSc.KSPSetOperators(petsclib, ksp, mat, mat)
LibPETSc.KSPSetFromOptions(petsclib, ksp)

# Solve Ax = b
LibPETSc.KSPSolve(petsclib, ksp, b, x)

# Get convergence info
reason = LibPETSc.KSPGetConvergedReason(petsclib, ksp)

iterations = LibPETSc.KSPGetIterationNumber(petsclib, ksp)

println("Converged in $(iterations) iterations, reason: $(reason)")

# Clean up
LibPETSc.KSPDestroy(petsclib, ksp)

Mixing High-Level and Low-Level Interfaces

You can mix both interfaces. High-level objects provide .ptr field to access the underlying C pointer:

# Create with high-level interface
vec_high = PetscVec(petsclib, 10)

# Use with low-level interface
LibPETSc.VecSet(petsclib, vec_high.ptr, 5.0)

# Or use the object directly (if it's a compatible type)
LibPETSc.VecView(petsclib, vec_high, C_NULL)

Finding Functions

The low-level interface provides wrappers for most PETSc functions. To find a function:

  1. Check the PETSc manual: https://petsc.org/release/docs/
  2. Use Julia's help system: Type ?LibPETSc.FunctionName
  3. Browse the documentation: See the reference pages for each PETSc class (Vec, Mat, KSP, etc.)
  4. Autocomplete: In the REPL, type LibPETSc.Vec and press Tab to see all Vec functions

Type Conventions

Low-level functions use these type patterns:

# PETSc objects (opaque pointers)
CVec = Ptr{Cvoid}        # Vector
CMat = Ptr{Cvoid}        # Matrix
CDM = Ptr{Cvoid}         # DM (domain management)
CKSP = Ptr{Cvoid}        # KSP (linear solver)
CSNES = Ptr{Cvoid}       # SNES (nonlinear solver)

# PETSc data types (depend on petsclib)
PetscInt                 # Integer type (Int32 or Int64)
PetscScalar              # Scalar type (Float64, Float32, ComplexF64, etc.)
PetscReal                # Real type (real part of PetscScalar)

Reference Pages

Detailed documentation for low-level functions by category:

Getting Help

If you encounter issues:

  1. Check the PETSc documentation
  2. Review the examples
  3. Ask questions on Julia Discourse with the petsc tag
  4. Open an issue on GitHub