itaps-parallel removal of

[Vitus Leung]

On Tue, 2008-03-18 at 00:48 -0600, Carl Ollivier-Gooch wrote:

> In this context, I could argue that the parallel read will have to do
> some outside-iMeshP communication so that the reading proc can have all
> the part handles.

In general, this is what I had concluded.

> Or I could argue (and this is the alternative I prefer) that it's
> possible to have part handles that:
> 
> a) are globally unique;
> b) allow identification of what process they live on;
> c) can be computed knowing only process rank and local part number; and
> d) still be be universally distinguished from pointers.
> 
> So here we go:
> 
> We know what the number of processors is (MPI communicator size or
> equivalent).  So we can easily compute the number of bits required to
> store the process rank.  Call this m.
> 
> This leaves n = 30-m bits for the number of parts per process.  That 30
> is deliberate.
> 
> Then the part handle can be an int that looks like this:
> 
> Bit
> 31 30 29 28 ... 31-m n n-1 n-2 ... 3 2 1 0
> 1  [process rank   ] [part # in process] 1
> 
> That 1 at the end is there because no pointer to a struct ever ends in
> anything but 00.  The 1 at the beginning disambiguates this from integer
> handles (which will surely be positive).  And having "only" 30 bits left
> over for rank and part # in process isn't that much of a restriction.
> Yes, I can imagine a million processes or a million parts on one
> process, but not as many as 1000 parts on each of a million processes.
> And of course 64 bit handles make questions of size moot.
> 
> Given this approach, one can compute (from the number of parts on a
> process and the process rank) the handles of all those parts.  This way,
> you "only" have to know the number of parts on a process (which should
> be very compressible data: most will be the same...) instead of all the
> part handles.
> 
> Having said all that, it's possible that this cure is worse than the
> disease.  Any thoughts on that?
> 
> Carl

But this might be better.

Vitus