[petsc-users] how to use petsc4py with mpi subcommunicators?

[Jed Brown]

Rodrigo Felicio <Rodrigo.Felicio at iongeo.com> writes:

> Hello all,
>
> Sorry for the newbie question, but is there a way of making petsc4py work with an MPI group or subcommunicator?  I  saw a solution posted back in 2010 (http://lists.mcs.anl.gov/pipermail/petsc-users/2010-May/006382.html), but it does not work for me. Indeed, if I try to use petsc4py.init(comm=newcom), then my sample code prints a msg "Attempting to use an MPI routine before initializing MPI".  Below I attach both the output and the source of the python code.
>
> kind regards
> Rodrigo
>
>
> time mpirun -n 5 python split_comm_ex2.py
> Global: rank 0 of 5. New comm : rank  0 of  3
> Global: rank 1 of 5. New comm : rank  0 of  2
> Global: rank 2 of 5. New comm : rank  1 of  3
> Global: rank 3 of 5. New comm : rank  1 of  2
> Global: rank 4 of 5. New comm : rank  2 of  3
> Attempting to use an MPI routine before initializing MPI
> Attempting to use an MPI routine before initializing MPI
>
> real    0m0.655s
> user    0m1.122s
> sys     0m1.047s
>
> And the python code:
>
> from mpi4py import MPI
>
> comm = MPI.COMM_WORLD
> world_rank = comm.rank
> world_size = comm.size
>
> color = world_rank % 2
>
> newcomm = comm.Split(color)
> newcomm_rank = newcomm.rank
> newcomm_size = newcomm.size
>
> for i in range(world_size):
>     comm.Barrier()
>     if (world_rank == i):
>         print ("Global: rank %d of %d. New comm : rank % d of  %d" %
>                (world_rank, world_size, newcomm_rank, newcomm_size))
>
> if newcomm.rank == 0:
>     import petsc4py
>     petsc4py.init(comm=newcomm)

I don't know if it fixes your problem, but you definitely need to call
this collectively on newcomm.  For example, using "if color == 0" above
would have the right collective semantics.

>     from petsc4py import PETSc
>
>     pcomm = PETSc.COMM_WORLD
>     print('pcomm size is {}/{}'.format(pcomm.rank, pcomm.size))
>
> newcomm.Free()
>
> ________________________________
>
>
> This email and any files transmitted with it are confidential and are intended solely for the use of the individual or entity to whom they are addressed. If you are not the original recipient or the person responsible for delivering the email to the intended recipient, be advised that you have received this email in error, and that any use, dissemination, forwarding, printing, or copying of this email is strictly prohibited. If you received this email in error, please immediately notify the sender and delete the original.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20170410/af4493f9/attachment.pgp>