[petsc-users] Verifying matching PetscFunctionBeginUser() and PetscFunctionReturn()

[Barry Smith]

  Ok, I don't understand the logic behind this but if you use PetscFunctionBeginUser (not PetscFunctionBegin) the check on the exit to the function is not done.

 PetscCheckAbort(!stack__.check || stack__.petscroutine[stack__.currentsize] != 1 || stack__.function[stack__.currentsize] == (const char *)(func__), PETSC_COMM_SELF, PETSC_ERR_PLIB, "Invalid stack: push from %s %s:%d. Pop from %s %s:%d.\n", \

since petscroutine[] has been set to 2 the middle || is true and so PetscCheckAbort() is not triggered.

  Jacob, do you remember the logic behind skipping the checks here for user routines?  

  Seems like it should check it, if the user buys into trying to use PetscFunctionBeginUser/PetscFunctionReturn we should have checks to make sure they are doing it correctly.

  Barry




> On Jul 12, 2023, at 1:30 PM, Aagaard, Brad T <baagaard at usgs.gov> wrote:
> 
> I created a small toy example (attached) that suggests that the verification of matching PetscFunctionBeginUser() and PetscFunctionReturn() fails when PetscFunctionReturn() is missing or in some cases when different functions are missing PetscFunctionBeginUser() or PetscFunctionReturn(). The cases where the code crashes are much more complex and not easily reproduced in a small toy example.
> 
> Here is the output for 4 cases of running my toy example:
> 
> CASE 1: Correct stack. No errors.
> 
> ./check_petscstack
> Testing correct stack...
> Layer 0: Calling layer 1
> Layer 1: Noop
> 
> CASE 2: Missing PetscFunctionBeginUser() in layer 0 is correctly detected and error message is generated.
> 
> ./check_petscstack missing_begin
> Testing missing PetscFunctionBeginUser() in layer 0...
> Layer 0: Calling layer 1
> Layer 1: Noop
> [0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------
> [0]PETSC ERROR: Petsc has generated inconsistent data
> [0]PETSC ERROR: Invalid stack size 0, pop layer0 /home/baagaard/src/cig/pylith/tests/libtests/utils/TestPetscStack.cc:54.
> 
> [0]PETSC ERROR: See https://petsc.org/release/faq/ for trouble shooting.
> [0]PETSC ERROR: Petsc Development GIT revision: v3.19.2-475-g71252c2aa25  GIT Date: 2023-06-15 13:06:55 -0600
> [0]PETSC ERROR: /home/baagaard/scratch/build/gcc-9.3/cig/pylith-debug/tests/libtests/utils/.libs/check_petscstack on a arch-gcc-9.3_debug named igskcicguwarlng by baagaard Wed Jul 12 11:20:50 2023
> [0]PETSC ERROR: Configure options --PETSC_ARCH=arch-gcc-9.3_debug --with-debugging=1 --with-clanguage=c --with-mpi-compilers=1 --with-shared-libraries=1 --with-64-bit-points=1 --with-large-file-io=1 --with-lgrind=0 --download-chaco=1 --download-parmetis=1 --download-metis=1 --download-triangle --download-ml=1 --download-superlu=1 --with-fc=0 --download-f2cblaslapack --with-hdf5=1 --with-hdf5-dir=/software/baagaard/hdf5-1.12.1/gcc-9.3 --with-zlib=1 CFLAGS=-g
> [0]PETSC ERROR: #1 layer0() at /home/baagaard/src/cig/pylith/tests/libtests/utils/TestPetscStack.cc:54
> --------------------------------------------------------------------------
> MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_SELF
> with errorcode 77.
> 
> NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.
> You may or may not see output from other processes, depending on
> exactly when Open MPI kills them.
> --------------------------------------------------------------------------
> 
> CASE 3: Missing PetscFunctionReturn() in layer 1 is not detected.
> 
> ./check_petscstack missing_return
> Testing missing PetscFunctionReturn() in layer 1...
> Layer 0: Calling layer 1
> Layer 1: Noop
> 
> CASE 4: Missing PetscFunctionBeginUser() in layer 0 and missing PetscFunctionReturn() in layer 1 are not detected.
> 
> ./check_petscstack missing_both
> Testing missing PetsFunctionBeginUser() in layer 0 and PetscFunctionReturn() in layer 1...
> Layer 0: Calling layer 1
> Layer 1: Noop
> 
> ________________________________________
> From: Barry Smith <bsmith at petsc.dev>
> Sent: Tuesday, July 11, 2023 10:45 PM
> To: Aagaard, Brad T
> Cc: petsc-users at mcs.anl.gov
> Subject: [EXTERNAL] Re: [petsc-users] Verifying matching PetscFunctionBeginUser() and PetscFunctionReturn()
> 
> 
> 
> This email has been received from outside of DOI - Use caution before clicking on links, opening attachments, or responding.
> 
> 
> 
>  #define PetscStackPop_Private(stack__, func__) \
>    do { \
>      PetscCheckAbort(!stack__.check || stack__.currentsize > 0, PETSC_COMM_SELF, PETSC_ERR_PLIB, "Invalid stack size %d, pop %s %s:%d.\n", stack__.currentsize, func__, __FILE__, __LINE__); \
>      if (--stack__.currentsize < PETSCSTACKSIZE) { \
>        PetscCheckAbort(!stack__.check || stack__.petscroutine[stack__.currentsize] != 1 || stack__.function[stack__.currentsize] == (const char *)(func__), PETSC_COMM_SELF, PETSC_ERR_PLIB, "Invalid stack: push from %s %s:%d. Pop from %s %s:%d.\n", \
>                        stack__.function[stack__.currentsize], stack__.file[stack__.currentsize], stack__.line[stack__.currentsize], func__, __FILE__, __LINE__); \
> 
> 
>  It is checking on each pop (return) that the current function it is in is the same function that the begin was called in.
> 
>  You do not have to call through PetscCall() to have this stuff work (though we strong recommend adding them).
> 
>  Even if you have a return and no outstanding begins it should not crash
> 
>  Maybe you need to see a couple of crashes so we try to figure out what is going on.
> 
> 
> On Jul 11, 2023, at 11:29 PM, Aagaard, Brad T via petsc-users <petsc-users at mcs.anl.gov> wrote:
> 
> PETSc developers,
> 
> When I fail to have matching PetscFunctionBeginUser() and PetscFunctionReturn() in my code, I get segfaults and valgrind reports invalid writes at places in PETSc where memory is freed. As a result, it is difficult to track down the actual source of the error. I know there used to be a command line argument for checking for mismatches in PetscFunctionBeginUser() and PetscFunctionReturn(), but Matt said it is no longer implemented and the reporting is supposed to be automatic when debugging is turned on. This automatic reporting does not seem to be working for me.
> 
> Is there a test in PETSc of this automatic reporting that I can run to verify it works in my build?
> 
> Are there any additional requirements for this automatic reporting of mismatched PetscFunctionBeginUser() and PetscFunctionReturn() in my code? Must I wrap all PETSc function calls with PetscCall()?
> 
> Thanks,
> Brad
> 
> <TestPetscStack.cc>