[mpich-discuss] Re: [MPICH] Problem setting up MPICH between a 32 bit INTEL and a 32 bit AMD machine
Darius Buntinas
buntinas at mcs.anl.gov
Mon Mar 3 11:04:25 CST 2008
You might have figured this out already, but all of the functions which
enqueue or dequeue on the posted recv queue and unexpected queues are in
ch3u_recvq.c . There are hooks for callbacks when requests are added to
the posted recv queue. You can just do the same for the unexpected
queue. There may be several functions there that modify the unexpected
queue, so check all of them.
I hope that helps.
-d
On 03/02/2008 07:58 AM, Krishna Chaitanya wrote:
> I think i have figured it out. Will post back if I have any further doubts.
> Thanks,
> Krishna Chaitanya K
>
> On Sun, Mar 2, 2008 at 1:02 AM, Krishna Chaitanya <kris.c1986 at gmail.com
> <mailto:kris.c1986 at gmail.com>> wrote:
>
> >Not sure I understand your question fully,
> Basically, I am trying to modify the MPICH library to invoke
> a callback when a message gets inserted into the unex queue and also
> monitor the unex queue's length.
>
> >So, if the message from
> the other send arrives in the meanwhile, MPID_Progress_Wait will
> cause it to
> be received and placed in the unexpected queue.
>
> Well, This is what I have understood :
> 1. MPI_Irecv calls MPIDI_CH3U_Recvq_FDU_or_AEP. This scans through
> the unex queue. In my case, there isnt one, so it creates a new
> request and inserts it into the posted queue.
> 2. MPI_Irecv returns.
> 3. MPI_Iwait calls MPIDI_CH3i_Progress_wait which calls
> MPIDI_CH3I_Progress_handle_sock_event in a loop.
> 4. If the event is MPIDU_SOCK_OP_READ and if the conn->state is
> connected, and MPIDI_CH3U_Recvq_FDP_or_AEU gets called. This
> function scans through the posted queue for a match. If a match is
> found, it gets dequeued. Else, it is placed into the unex queue.
>
> OK...
> A node can actually gather a new incoming message only when it is
> in the progress engine. The progress engine is invoked only by
> MPI_Wait. MPI_Wait gets called only after MPI_Irecv returns. (Am i
> right so far ? )
>
> From the traces that I have done so far, I have observed that a
> new message is not getting into the unex queue, as a matching
> MPI_Irecv has already been executed. ( Which has placed the recv
> request in the posted queue) . When MPIDI_CH3U_Recvq_FDP_or_AEU is
> called from the progress engine, a matching request in the posted
> queue is invariably found and the new message is not really
> "unexepected" from the receiver's standpoint.
>
> This is what my code looks like :
> sender side :
> MPI_Isend (buffer, 1, MPI_INTEGER, 1, 1001,
> MPI_COMM_WORLD, &req2[0]);
> MPI_Isend (buffer, 1, MPI_INTEGER, 1, 1002,
> MPI_COMM_WORLD, &req2[1]);
> printf(" [%d] MPI_Isend returns \n",rank);
> MPI_Wait( &req2[0] , &sts2[0]);
> MPI_Wait( &req2[1], &sts2[1]);
>
> receiver side :
> sleep(10);
> MPI_Irecv (buffer, 1, MPI_INTEGER, 0, 1001,
> MPI_COMM_WORLD, &(req1[0]));
> sleep(10);
> MPI_Irecv (buffer, 1, MPI_INTEGER, 1, 1002,
> MPI_COMM_WORLD, &(req1[1]));
> MPI_Wait(&req1[0], &sts1[0]);
> MPI_Wait(&req1[1], &sts1[1]);
>
> I tried to have those sleep()'s to make the reciever operate
> slower than the sender.
> Thanks for your time and please let me know if I am missing
> something .
>
>
> Krishna Chaitanya,
> Final Year B-Tech,
> National Institute of Technology,Karnataka.
>
>
>
> On Fri, Feb 29, 2008 at 7:44 PM, Rajeev Thakur <thakur at mcs.anl.gov
> <mailto:thakur at mcs.anl.gov>> wrote:
>
> > I just started looking into the non-blocking calls
> today. I am
> > not sure of whats happening in the call :
> MPIR_Grequest_progress_poke.
> > What exactly is the difference between a generalized request
> and a
> > native request?
>
> A generalized request is something defined in MPI-2's chapter on
> external
> interfaces. It is a way for a new library to define its own
> nonblocking
> operations and use MPI_Request objects to test or wait on.
> MPIR_Grequest_progress_poke uses an extension to MPI-2
> generalized requests,
> as explained in
> http://www-unix.mcs.anl.gov/~thakur/papers/grequest-redesign.pdf
> <http://www-unix.mcs.anl.gov/%7Ethakur/papers/grequest-redesign.pdf>.
> But you
> can just ignore generalized requests for now.
>
> > I am also keen on knowing how an unexpected message
> is handled
> > by MPICH2 and I have understood what happens in the function
> > MPIDI_CH3U_Recvq_FDU_or_AEP(). So, I had two ranks(ranks 0
> and 1)
> > executing MPI_Isend() and one rank(rank 2) executing the two
> matching
> > MPI_Irecv() calls, with the hope that the message that is
> arriving
> > from rank 1 would be unexpected from rank 2's standpoint.
> But, as it
> > turns out, MPI_Wait() actually calls MPID_Progress_Wait().
> This is a
> > blocking call and the MPI_Isend() that is being executed by
> rank1 is
> > blocked until rank2 executes the corresponding MPI_Irecv().
> > My question is, how can I get an incoming message at
> rank 2 to
> > get into the unexpected queue and wait there till the
> receiver scans
> > through the queue until a match is found and proceeds to
> transfer of
> > data?
> > To make this happen, MPID_Progress_Wait must never
> get called
> > and so the MPI_Wait must constantly. Is that correct?
>
> Not sure I understand your question fully, but
> MPID_Progress_wait will block
> until some (any) event occurs. That event need not be the event
> of interest,
> namely, the one for which the MPI_Wait was called. So, if the
> message from
> the other send arrives in the meanwhile, MPID_Progress_Wait will
> cause it to
> be received and placed in the unexpected queue.
>
> Rajeev
>
>
>
>
> --
> In the middle of difficulty, lies opportunity
>
>
>
>
> --
> In the middle of difficulty, lies opportunity
More information about the mpich-discuss
mailing list