[petsc-users] MPI_Comm_spawn when using PETSc

[Jed Brown]

Suraj Prabhakaran <s.prabhakaran at grs-sim.de> writes:

> Dear all,
>
> I am completely new to PETSc. I would like to know if it is possible
> to use PETSc after increasing the number of processes in an MPI
> program through MPI_Comm_spawn.  The application is started with a set
> of MPI processes and the PETSC is initialized with this set. At a
> later point, the application could indeed increase the set of
> processes (through MPI_Comm_spawn) and create a intra communicator
> with the parent and the children. I would like to use these processes
> too with PETSC. Would this be possible (by providing the new
> communicator for instance)?

The logging and profiling features will need PETSC_COMM_WORLD to contain
all processes.  So for that to work properly after spawning new
processes, you would call PetscFinalize on the original comm, then set
PETSC_COMM_WORLD = new_comm, then PetscInitialize again.

It's possible that you can just call PetscInitialize on the spawned comm
containing only the new members, then create an object on the union.
I'm reluctant to recommend this, and it's not tested in the present test
suite, but I can't think of anything that will truly break.  If you
wanted logging information in that setting, you should send it to
different files so that it's doesn't arrive all jumbled up on stdout.