[petsc-users] Verifying matching PetscFunctionBeginUser() and PetscFunctionReturn()

[Aagaard, Brad T]

I created a small toy example (attached) that suggests that the verification of matching PetscFunctionBeginUser() and PetscFunctionReturn() fails when PetscFunctionReturn() is missing or in some cases when different functions are missing PetscFunctionBeginUser() or PetscFunctionReturn(). The cases where the code crashes are much more complex and not easily reproduced in a small toy example.

Here is the output for 4 cases of running my toy example:

CASE 1: Correct stack. No errors.

./check_petscstack
Testing correct stack...
Layer 0: Calling layer 1
Layer 1: Noop

CASE 2: Missing PetscFunctionBeginUser() in layer 0 is correctly detected and error message is generated.

./check_petscstack missing_begin
Testing missing PetscFunctionBeginUser() in layer 0...
Layer 0: Calling layer 1
Layer 1: Noop
[0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------
[0]PETSC ERROR: Petsc has generated inconsistent data
[0]PETSC ERROR: Invalid stack size 0, pop layer0 /home/baagaard/src/cig/pylith/tests/libtests/utils/TestPetscStack.cc:54.

[0]PETSC ERROR: See https://petsc.org/release/faq/ for trouble shooting.
[0]PETSC ERROR: Petsc Development GIT revision: v3.19.2-475-g71252c2aa25  GIT Date: 2023-06-15 13:06:55 -0600
[0]PETSC ERROR: /home/baagaard/scratch/build/gcc-9.3/cig/pylith-debug/tests/libtests/utils/.libs/check_petscstack on a arch-gcc-9.3_debug named igskcicguwarlng by baagaard Wed Jul 12 11:20:50 2023
[0]PETSC ERROR: Configure options --PETSC_ARCH=arch-gcc-9.3_debug --with-debugging=1 --with-clanguage=c --with-mpi-compilers=1 --with-shared-libraries=1 --with-64-bit-points=1 --with-large-file-io=1 --with-lgrind=0 --download-chaco=1 --download-parmetis=1 --download-metis=1 --download-triangle --download-ml=1 --download-superlu=1 --with-fc=0 --download-f2cblaslapack --with-hdf5=1 --with-hdf5-dir=/software/baagaard/hdf5-1.12.1/gcc-9.3 --with-zlib=1 CFLAGS=-g
[0]PETSC ERROR: #1 layer0() at /home/baagaard/src/cig/pylith/tests/libtests/utils/TestPetscStack.cc:54
--------------------------------------------------------------------------
MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_SELF
with errorcode 77.

NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.
You may or may not see output from other processes, depending on
exactly when Open MPI kills them.
--------------------------------------------------------------------------

CASE 3: Missing PetscFunctionReturn() in layer 1 is not detected.

./check_petscstack missing_return
Testing missing PetscFunctionReturn() in layer 1...
Layer 0: Calling layer 1
Layer 1: Noop

CASE 4: Missing PetscFunctionBeginUser() in layer 0 and missing PetscFunctionReturn() in layer 1 are not detected.

./check_petscstack missing_both
Testing missing PetsFunctionBeginUser() in layer 0 and PetscFunctionReturn() in layer 1...
Layer 0: Calling layer 1
Layer 1: Noop

________________________________________
From: Barry Smith <bsmith at petsc.dev>
Sent: Tuesday, July 11, 2023 10:45 PM
To: Aagaard, Brad T
Cc: petsc-users at mcs.anl.gov
Subject: [EXTERNAL] Re: [petsc-users] Verifying matching PetscFunctionBeginUser() and PetscFunctionReturn()



 This email has been received from outside of DOI - Use caution before clicking on links, opening attachments, or responding.



  #define PetscStackPop_Private(stack__, func__) \
    do { \
      PetscCheckAbort(!stack__.check || stack__.currentsize > 0, PETSC_COMM_SELF, PETSC_ERR_PLIB, "Invalid stack size %d, pop %s %s:%d.\n", stack__.currentsize, func__, __FILE__, __LINE__); \
      if (--stack__.currentsize < PETSCSTACKSIZE) { \
        PetscCheckAbort(!stack__.check || stack__.petscroutine[stack__.currentsize] != 1 || stack__.function[stack__.currentsize] == (const char *)(func__), PETSC_COMM_SELF, PETSC_ERR_PLIB, "Invalid stack: push from %s %s:%d. Pop from %s %s:%d.\n", \
                        stack__.function[stack__.currentsize], stack__.file[stack__.currentsize], stack__.line[stack__.currentsize], func__, __FILE__, __LINE__); \


  It is checking on each pop (return) that the current function it is in is the same function that the begin was called in.

  You do not have to call through PetscCall() to have this stuff work (though we strong recommend adding them).

  Even if you have a return and no outstanding begins it should not crash

  Maybe you need to see a couple of crashes so we try to figure out what is going on.


On Jul 11, 2023, at 11:29 PM, Aagaard, Brad T via petsc-users <petsc-users at mcs.anl.gov> wrote:

PETSc developers,

When I fail to have matching PetscFunctionBeginUser() and PetscFunctionReturn() in my code, I get segfaults and valgrind reports invalid writes at places in PETSc where memory is freed. As a result, it is difficult to track down the actual source of the error. I know there used to be a command line argument for checking for mismatches in PetscFunctionBeginUser() and PetscFunctionReturn(), but Matt said it is no longer implemented and the reporting is supposed to be automatic when debugging is turned on. This automatic reporting does not seem to be working for me.

Is there a test in PETSc of this automatic reporting that I can run to verify it works in my build?

Are there any additional requirements for this automatic reporting of mismatched PetscFunctionBeginUser() and PetscFunctionReturn() in my code? Must I wrap all PETSc function calls with PetscCall()?

Thanks,
Brad

-------------- next part --------------
A non-text attachment was scrubbed...
Name: TestPetscStack.cc
Type: text/x-c++src
Size: 2018 bytes
Desc: TestPetscStack.cc
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20230712/18bdbb74/attachment-0001.bin>