[mpich-discuss] How expensive is MPI_Win_create() compared to memcpy()?

[Pavan Balaji]

On 12/27/2011 12:42 PM, Jed Brown wrote:
> Hmm, what about limiting scope, e.g. if I knew a priori which process
> groups I would use as arguments to MPI_Win_post() and MPI_Win_start()?
> Is there anything useful you could do if I only knew the process group
> for MPI_Win_start()?

Hmm... On second thought, it might be possible to make it not 
synchronize by passing an info argument (though it's not implemented 
yet).  In this case, the base address and other information can be 
queried on-demand when the next PUT/GET/ACCUMULATE operation occurs. 
When memory registration is involved, this will be even messier.  So I 
guess it's theoretically possible, though there's no plan to do it at 
present.  Also, from a portability perspective, you can never assume 
that there will not be synchronization (other MPI implementations might 
not respect this info argument).

In MPI-3, there's an option for dynamically attaching memory to windows. 
  That's probably the best option for you.  We are working on 
implementing it for our next major release (1.5).

> It's cumbersome for a user to always arrange to reuse some special
> memory, probably with different types at different times. I have a
> choice of (a) memcpy into existing window memory and (b) wrap a window
> around the user's memory. For short messages, the memcpy version will be
> faster.
>
> Sometimes I'm accessing parts of a buffer that is much larger than the
> send/receive volume (specified by composite MPI_Datatypes). The memcpy
> costs proportional to the size of the whole buffer rather than to the
> communication volume. I can pack, but this involves one more transaction
> during setup.
>
> Note that a single communication object of mine might have multiple
> in-flight operations at a given time, each using different windows, so I
> would be keeping a cache of windows.

The part I don't understand here is what benefit does wrapping an 
existing window object around new memory give you?  Why not create a new 
window object around your new memory?

  -- Pavan

-- 
Pavan Balaji
http://www.mcs.anl.gov/~balaji