Changes: 3.16#
General:
- Change - MPIU_Allreduce()to always returns a MPI error code that should be checked with- CHKERRMPI(ierr)
- Add support for A64FX and Cray compilers 
- Add support for ESSL 5.2 and later; drop support for ESSL <=5.1 
- Add support for NEC-SX Tsubasa Vector Engine 
- Add support for NVIDIA HPC SDK 
Configure/Build:
- Configure requires Python-2.7 or Python-3.4+. 
- Remove - --with-kokkos-cuda-arch. One can use- -with-cuda-gencodearchto specify the cuda arch for Kokkos. Usually not needed since PETSc auto detects that
- For - --download-hdf5, disable- --download-hdf5-fortran-bindingsby default
- Add OpenCascade package to PETSc and allow - --download-opencascade
- Add support for hypre in device mode for both NVIDIA and AMD GPUs 
- Extend detection of C++ dialect to C++17. Configure now also takes minimum and maximum required C++ dialect of packages into account when choosing the C++ dialect 
Sys:
- Add - PetscDeviceclass to manage discovered GPU devices
- Add - PetscDeviceKind
- Add - PetscDeviceCreate(),- PetscDeviceConfigure(), and- PetscDeviceDestroy()
- Add - PetscDeviceContextclass to manage asynchronous GPU compute support via a fork-join model
- Add - PetscDeviceContextCreate(),- PetscDeviceContextDestroy(),- PetscDeviceContextSetDevice(),- PetscDeviceContextGetDevice(),- PetscDeviceContextSetStreamType(),- PetscDeviceContextGetStreamType(),- PetscDeviceContextSetUp(),- PetscDeviceContextDuplicate(),- PetscDeviceContextQueryIdle(),- PetscDeviceContextWaitForContext(),- PetscDeviceContextFork(),- PetscDeviceContextJoin(),- PetscDeviceContextSynchronize(),- PetscDeviceContextGetCurrentContext(),- PetscDeviceContextSetCurrentContext(), and- PetscDeviceContextSetFromOptions()
- Deprecate - petsccublas.hand- petschipblas.hin favor of- petscdevice.hand- petscdevicetypes.h
- Add GPU event timers to capture kernel execution time accurately 
- Remove - WaitForCUDA()and- WaitForHIP()before- PetscLogGpuTimeEnd()
- Add MPIU_REAL_INT and MPIU_SCALAR_INT datatypes to be used for reduction operations 
- Add MPIU_MAXLOC and MPIU_MINLOC operations 
- Add - CHKERRCXX()to catch C++ exceptions and return a PETSc error code
- Remove - PetscStackroutines from public headers, this class should now be considered private
PetscViewer:
- PetscViewerHDF5PushGroup(): if input path begins with- /, it is taken as absolute, otherwise relative to the current group
- PetscViewerHDF5HasAttribute(),- PetscViewerHDF5ReadAttribute(),- PetscViewerHDF5WriteAttribute(),- PetscViewerHDF5HasDataset()and- PetscViewerHDF5HasGroup()support absolute paths (starting with- /) and paths relative to the current pushed group
- Add input argument to - PetscViewerHDF5ReadAttribute()for default value that is used if attribute is not found in the HDF5 file
- Add - PetscViewerHDF5PushTimestepping(),- PetscViewerHDF5PopTimestepping()and- PetscViewerHDF5IsTimestepping()to control timestepping mode.
- One can call - PetscViewerHDF5IncrementTimestep(),- PetscViewerHDF5SetTimestep()or- PetscViewerHDF5GetTimestep()only if timestepping mode is active
- Error if timestepped dataset is read/written out of timestepping mode, or vice-versa 
PetscDraw:
AO:
IS:
VecScatter / PetscSF:
PF:
Vec:
- Add - VecMean()to calculate arithmetic mean of elements of a vector
- Add - VecBoundToCPU()to query information set with- VecBindToCPU()
PetscSection:
- Extend - PetscSectionView()for section saving to HDF5
- Add - PetscSectionLoad()for section loading from HDF5
PetscPartitioner:
Mat:
- MATMPIKAIJnow tracks the object state of the AIJ matrix describing the blockwise action of the KAIJ matrix and automatically rebuilds internal data structures before executing operations with the KAIJ matrix if the state has changed
- Factorization types now provide their preferred ordering (which may be - MATORDERINGEXTERNAL) to prevent PETSc PCFactor from, by default, picking an ordering when it is not ideal
- Deprecate - MatFactorGetUseOrdering(); Use- MatFactorGetCanUseOrdering()instead
- Add - --download-htoolto use hierarchical matrices with the new type- MATHTOOL
- Add - MATCENTERINGspecial matrix type that implements action of the centering matrix
- Remove -mat_mumps_icntl_7 1 option, use -pc_factor_mat_ordering_type <type> to have PETSc perform the ordering (sequential only) 
- Add - MATSOLVERSPQR- interface to SuiteSparse QR factorization
- Add - MatSeqAIJKron()- Kronecker product of two- MatSeqAIJ
- Add - MatNormalGetMat()to retrieve the underlying- Matof a- MATNORMAL
- Add - MatNormalHermitianGetMat()to retrieve the underlying- Matof a- MATNORMALHERMITIAN
- Add - VecCreateMPICUDA()and- VecCreateMPIHIP()to create MPI device vectors
- Add accessor routines for device index data of - MATSEQAIJCUSPARSEmatrices:- MatSeqAIJCUSPARSEGetIJ()and- MatSeqAIJCUSPARSERestoreIJ()
- Add accessor routines for device data of - MATSEQAIJCUSPARSEmatrices:- MatSeqAIJCUSPARSEGetArray(),- MatSeqAIJCUSPARSERestoreArray(),- MatSeqAIJCUSPARSEGetArrayRead(),- MatSeqAIJCUSPARSERestoreArrayRead().- MatSeqAIJCUSPARSEGetArrayWrite(),- MatSeqAIJCUSPARSERestoreArrayWrite()
- Add support for - MATHYPREmatrices on NVIDIA and AMD GPUs
- MatPreallocatorPreallocateperformance significantly improved
- Add - MatGetColumnReductions()developer routine to calculate reductions over columns of a matrix
- Add - MatGetColumnSums(),- MatGetColumnSumsRealPart(),- MatGetColumnSumsImaginaryPart()to compute sums over matrix columns
- Add - MatGetColumnMeans(),- MatGetColumnMeansRealPart(),- MatGetColumnMeansImaginaryPart()to compute arithmetic means over matrix columns
- Add - MatBoundToCPU()to query information set with- MatBindToCPU()
- Rename - MATHARAin- MATH2OPUS, supporting distributed memory operations with hierarchical matrices
PC:
- Add - PCSetPreSolve()
- Add - PCQR- interface to SuiteSparse QR factorization for- MatSeqAIJ,- MATNORMAL, and- MATNORMALHERMITIAN
- Add support for BoomerAMG from - PCHYPREto run on NVIDIA and AMD GPUs
- PCShellGetContext()now takes- void*as return argument
- Rename - PCHARAin- PCH2OPUS, supporting distributed memory operations with hierarchical matrices
KSP:
- KSPGetMonitorContext()now takes- void*as return argument
- KSPGetConvergenceContext()now takes- void*as return argument
SNES:
- Add - SNESSetComputeMFFunction()
- Add support for - -snes_mf_operatorfor use with- SNESSetPicard()
- SNESShellGetContext()now takes- void*as return argument
SNESLineSearch:
TS:
- Add - -ts_type irk- fully implicit Runge-Kutta solvers
- Add - TSTrajectoryinterface to the CAMS library for optimal offline checkpointing for multistage time stepping schemes
- Add option - -ts_trajectory_memory_type <revolve | cams | petsc>to switch checkpointing schedule software
- Add option - -ts_trajectory_max_units_ramto specify the maximum number of allowed checkpointing units
TAO:
- TaoShellGetContext()now takes- void*as return argument
DM/DA:
- Change management of auxiliary data in DM from object composition to - DMGetAuxiliaryVec()/- DMSetAuxiliaryVec(),- DMCopyAuxiliaryVec()
- Remove - DMGetNumBoundary()and- DMGetBoundary()in favor of DS counterparts
- Remove - DMCopyBoundary()
- Change interface for - DMAddBoundary(),- PetscDSAddBoundary(),- PetscDSGetBoundary(),- PetscDSUpdateBoundary()
- Add - DMDAVecGetArrayDOFWrite()and- DMDAVecRestoreArrayDOFWrite()
- DMShellGetContext()now takes- void*as return argument
DMSwarm:
- Add - DMSwarmGetCellSwarm()and- DMSwarmRestoreCellSwarm()
DMPlex:
- Add a - PETSCVIEWEREXODUSIIviewer type for- DMView()/- DMLoad()and- VecView()/- VecLoad(). Note that not all DMPlex can be saved in exodusII format since this file format requires that the numbering of cell sets be compact
- Add - PetscViewerExodusIIOpen()convenience function
- Add - PetscViewerExodusIISetOrder()to generate “2nd order” elements (i.e. tri6, tet10, hex27) when using- DMViewwith a- PETSCVIEWEREXODUSIIviewer
- Change - DMPlexComputeBdResidualSingle()and- DMPlexComputeBdJacobianSingle()to take a form key
- Add - DMPlexTopologyLoad(),- DMPlexCoordinatesLoad(), and- DMPlexLabelsLoad()for incremental loading of a- DMPlexobject from an HDF5 file
- Add - DMPlexTopologyView(),- DMPlexCoordinatesView(), and- DMPlexLabelsView()for incremental saving of a- DMPlexobject to an HDF5 file
- Add - DMPlexSectionView()saving a- PetscSectionin association with a- DMPlexmesh
- Add - DMPlexSectionLoad()loading a- PetscSectionin association with a- DMPlexmesh
- Add - DMPlexGlobalVectorView()and- DMPlexLocalVectorView()saving global and local vectors in association with a data layout on a- DMPlexmesh
- Add - DMPlexGlobalVectorLoad()and- DMPlexLocalVectorLoad()loading global and local vectors in association with a data layout on a- DMPlexmesh
- Add - DMPlexIsSimplex()to check the shape of the first cell
- Add - DMPlexShapeto describe prebuilt mesh domains
- Add - DMPlexCreateCoordinateSpace()to make an FE space for the coordinates
- Add the automatic creation of a Plex from options, see - DMSetFromOptions()
- The old options for - DMPlexCreateBoxMesh()NO LONGER WORK. They have been changed to make the interface more uniform
- Replace - DMPlexCreateSquareBoundary()and- DMPlexCreateCubeBoundary()with- DMPlexCreateBoxSurfaceMesh()
- Remove - DMPlexCreateReferenceCellByType()
- The number of refinements is no longer an argument to - DMPlexCreateHexCylinderMesh()
- Add - DMSetLabel()
- Replace - DMPlexComputeJacobianAction()with- DMSNESComputeJacobianAction()
- Add - DMSNESCreateJacobianMF()
- Change - DMPlexComputeBdResidualSingle()to take- PetscFormKeyinstead of explicit label/value/field arguments
- Add - DMPlexInflateToGeomModel()which pushes refined points out to a geometric boundary
- Separate EGADS and EGADSLite functionality, add - DMPlexCreateEGADSLiteFromFile()
- Remove - DMPlexReverseCell()and- DMPlexOrientCell()in favor of- DMPlexOrientPoint()
- Remove - DMPlexCompareOrientations()in favor of- DMPolytopeMatchOrientation()
- Add - DMPlexGetCompressedClosure()and- DMPlexRestoreCompressedClosure()
- Add - DMPlexMetricCreateas a helper function for creating a (P1) Riemannian metric.
- Add - DMPlexMetricCreateUniformas a helper function for creating a uniform metric.
- Add - DMPlexMetricCreateIsotropicas a helper function for creating an isotropic metric.
- Add - DMPlexMetricEnforceSPDfor enforcing that a metric is symmetric positive-definite.
- Add - DMPlexMetricNormalizeto apply L-p metric normalization.
- Add - DMPlexMetricAverageto average an arbitrary number of metrics.
- Add - DMPlexMetricAverage2to average two metrics.
- Add - DMPlexMetricAverage3to average three metrics.
- Add - DMPlexMetricIntersectionto intersect an arbitrary number of metrics.
- Add - DMPlexMetricIntersection2to intersect two metrics.
- Add - DMPlexMetricIntersection3to intersect three metrics.
FE/FV:
- Change - PetscFEIntegrateBdResidual()and- PetscFEIntegrateBdJacobian()to take both- PetscWeakFormand form key
- Add - PetscFEGeomGetPoint()and- PetscFEGeomGetCellPointto package up geometry handling
DMNetwork:
- Add - DMNetworkCreateIS()and- DMNetworkCreateLocalIS()
- Remove nv from - DMNetworkAddSubnetwork()
DMStag:
DT:
- Add - PetscWeakFormCopy(),- PetscWeakFormClear(),- PetscWeakFormRewriteKeys()and- PetscWeakFormClearIndex()
- Add - PetscDSDestroyBoundary()and- PetscDSCopyExactSolutions()
- PetscDSGetContext()now takes- void*as return argument
- Add - PetscWeakFormReplaceLabel()to change labels after mesh modification
Fortran:
- Add support for - PetscInitialize(filename,help,ierr),- PetscInitialize(ierr)in addition to current- PetscInitialize(filename,ierr)