[petsc-users] Does petsc duplicate the users communicator?

[Barry Smith]

   Whenever PETSc is handed a communicator it looks for an attribute inside of the communicator that contains the "PETSc" version of that communicator. If it does not find the attribute it adds an attribute with a new communicator in, if it does find one it increases its reference count by one. The routine it uses to perform this is PetscCommDuplicate(). We do it this was so that PETSc communication will never potentially interfere with the users use of their communicators. PetscCommDestroy() decreases the reference count of the inner communicator by one. So, for example, if you use "comm" to create two PETSc objects, PETSc will create an attribute on "comm" with a new communicator, when both objects are destroy then PetscCommDestroy() will have been called twice and the inner (PETSc) communicator will be destroyed. 

  If someone did 

      Use MPI to create a new communicator
      VecCreate(comm,...)
      Use MPI to destroy the new communicator
      ....
      VecDestroy() 

I am not sure what will happen since PETSc keeps a reference to the outer communicator from its own inner communicator. And destroying the user communicator will cause an attempt to destroy the attribute containing the inner PETSc communicator.  I had always just assumed the user would not be deleting any MPI communicators they made and pass to PETSc until they were done with PETSc. It may work correctly but may not.

The reality is very few MPI codes have complicated life cycles for MPI communicators. 

Barry


> On Jul 8, 2021, at 10:17 PM, Kozdon, Jeremy (CIV) <jekozdon at nps.edu> wrote:
> 
> Sorry if this is clearly stated somewhere in the docs, I'm still getting familiar with the petsc codebase and was also unable to find the answer searching (nor could I determine where this would be done in the source).
> 
> Does petsc duplicate MPI communicators? Or does the users program need to make sure that the communicator remains valid for the life of a petsc object?
> 
> The attached little test code seems to suggest that there is some duplication of MPI communicators behind the scenes.
> 
> This came up when working on Julia wrappers for petsc. (Julia has a garbage collector so we need to make sure that references are properly kept if needed.)
> 
> <try.c>