[MPICH] Changing the comm size at runtime

[Rajeev Thakur]

"Invalid buffer pointer" is an odd message to come from
MPI_Intercomm_create. Is that what is really printed? Can you send the
entire error message.

Rajeev 

> -----Original Message-----
> From: owner-mpich-discuss at mcs.anl.gov 
> [mailto:owner-mpich-discuss at mcs.anl.gov] On Behalf Of Patrick Gräbel
> Sent: Tuesday, April 10, 2007 11:15 AM
> To: mpich-discuss at mcs.anl.gov
> Subject: Re: [MPICH] Changing the comm size at runtime
> 
> 
> I tried to "fold" the accepted intercomm objects to a large single
> intracomm object as you suggested, but MPI_Intercomm_create 
> aborts with
> "Invalid buffer pointer". The code for a master "A" awaiting 
> two slaves
> "B" and "C" to connect looks like this:
> 
> ------------------
> MASTER (process A)
> 
>   // accept all slaves
>   MPI::Intercomm inter[num_slaves];
>   for(int i = 0; i < num_slaves; i++)
>     inter[i] = MPI::COMM_WORLD.Accept(port,MPI::INFO_NULL,0);
> 
>   // collective merging over A and B
>   MPI::Intracomm AB_intra = inter[0].Merge(false);
> 
>   // collective intercomm creation over A, B and C
>   MPI::Intercomm AB_C_inter = 
> AB_intra.Create_intercomm(0,inter[1],0,0);
> 
> -------------------
> SLAVE 1 (process B)
> 
>   // connect to master
>   MPI::Intercomm AB_inter = 
> MPI::COMM_WORLD.Connect(port,MPI::INFO_NULL,0);
> 
>   // collective merge over A and B
>   MPI::Intracomm AB_intra = AB_inter.Merge(true);
> 
>   // collective intercomm creation over A, B and C
>   MPI::Intercomm AB_C_inter = 
> AB_intra.Create_intercomm(0,AB_inter,0,0);
> 
> -------------------
> SLAVE 2 (process C)
> 
>   // connect to master
>   MPI::Intercomm AC_inter = 
> MPI::COMM_WORLD.Connect(port,MPI::INFO_NULL,0);
> 
>   // collective intercomm creation over A, B and C
>   MPI::Intercomm AB_C_inter =
> MPI::COMM_WORLD.Create_intercomm(0,AC_inter,0,0);
> ------------------
> 
> The MPI-Forum's guide says:
> 
> "The function MPI_INTERCOMM_CREATE can be used to create an
> inter-communicator from two existing intra-communicators"
> 
> I don't know if this excludes inter-communicators like those I am
> passing. Maybe the leader rank args are wrong but I could not find a
> solution.
> 
> Thanks
> Patrick
> 
> 
> Rajeev Thakur wrote:
> > If the slaves can connect to the master with a single collective
> > MPI_Comm_connect (i.e. the communicator includes all 
> slaves), then it is
> > easy: A single MPI_Intercomm_merge will create the giant 
> intracommunicator.
> > If the slaves are connecting one by one, it is much more 
> difficult. Here is
> > an example of how to do it:
> > 
> > Let's say Process A is the master and processes B, C, and D 
> are slaves. B,
> > C, D individually connect to A, resulting in 3 
> intercommunicators: AB_inter,
> > AC_inter, and AD_inter. To merge them all into a single 
> intracommunicator:
> > 
> > * begin by doing an MPI_Intercomm_merge on AB_inter, resulting in an
> > intracommunicator AB_intra.
> > 
> > * then create an intercommunicator between AB on one side 
> and C on the other
> > by using MPI_Intercomm_create. Pass AB_intra as the 
> local_comm on A and B,
> > MPI_COMM_WORLD as the intracomm on C, and AC_inter as the 
> peer_comm. This
> > results in the intercommunicator AB_C_inter. 
> > 
> > * then call MPI_Intercomm_merge on it to create the 
> intracommunicator
> > ABC_intra. 
> > 
> > * then call MPI_Intercomm_create to create an 
> intercommunicator between ABC
> > and D just as you did with AB and C above. 
> > 
> > * then call MPI_Intercomm_merge to get a single 
> intracommunicator containing
> > all.
> > 
> > Rajeev 
> > 
> > 
> >> -----Original Message-----
> >> From: owner-mpich-discuss at mcs.anl.gov 
> >> [mailto:owner-mpich-discuss at mcs.anl.gov] On Behalf Of 
> Patrick Gräbel
> >> Sent: Sunday, March 18, 2007 2:02 PM
> >> To: mpich-discuss at mcs.anl.gov
> >> Subject: Re: [MPICH] Changing the comm size at runtime
> >>
> >> At the moment the master iterates over a set of Intracomm 
> objects that
> >> are created by calling MPI_Intercomm_Merge on the accepted 
> Intercomm
> >> objects in order to communicate. This works but...
> >>
> >> ...is there a way to form a larger Intracomm (or Intercomm) 
> >> object that
> >> contains all slaves at once? This would make the programm 
> more elegant
> >> and smaller.
> >>
> >> Greetings
> >> Patrick
> >>
> >> Rajeev Thakur wrote:
> >>> I should point out that you can also create an 
> >> intracommunicator from an
> >>> intercommunicator by using MPI_Intercomm_merge and then use 
> >> the regular
> >>> intracommunicator collectives if that's what you need.
> >>>
> >>> Rajeev
> >>
> > 
> > 
> 
>