[petsc-dev] ISRestoreNonlocalIS?

[Jed Brown]

On Thu, Nov 18, 2010 at 23:04, Barry Smith <bsmith at mcs.anl.gov> wrote:

> Jed, this is what started this entire process that you are unhappy with
> :-).
>

I know, but I was bothered by an IS interface that destroys structure
(necessarily flattens everything to scalar indices).

>  Currently it only produces serial matrices, in which case both isrow and
> iscol should be serial.  But to extract on subcomms, the user would pass in
> (isrow,iscol) living on subcomms.  This would actually make MatGetSubMatrix
> a redundant API because the user could pass a single global (isrow,iscol) to
> MatGetSubMatrices().  Along these lines, it is easy to implement the
> trivially cheap ISGetLocalIS() so that, e.g. MatGetSubMatrix_MPIAIJ_Private
> doesn't need to do anything weird.
>
>    We are getting closer.   But you still haven't resolved my big
> questions.
>
> (1)  Where do we cache ISGetAllIndices(IS,IS*) (replacement for
> ISAllGather()) and ISGetNonlocalIndices(IS,IS*)? I want to cache them in the
> original IS, is that ok with you?
>

Yes, perhaps with some way to free that memory later (we've already gotten
questions about why calling MatGetSubMatrix triples  the memory usage of the
original matrix, and they have to make another copy, and then destroy the
matrix they used to get the submatrix in order to reclaim that memory).  But
that can be a long-term API issue.

If IS is immutable, the I definitely agree with caching the gathered part on
the IS.  But MatGetSubMatrix_MPIAIJ should also cache the original IS so
that it doesn't produce garbage if the user tries to MAT_REUSE_MATRIX with a
different IS.


> (2) What is the meaning of the IS that is returned? It really bothers me
> that it is a sequential beast. What is it suppose to mean? To me it is
> information about a parallel object therefor belongs in that parallel
> object.  Am I just wrong to be bothered by it?
>

It doesn't bother me that it is sequential.  After all, you are requesting a
sequential matrix defined by extracting the part defined by sequential
(isrow,iscol) from the parallel matrix.  Other procs are requesting
different parts and will get back their own sequential parts.

Jed
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20101118/896dbb2d/attachment.html>