[petsc-dev] ISRestoreNonlocalIS?

[Barry Smith]

  It is possible there could be other methods added to IS that would be used by VecScatterCreate() and MatGet...Matrix() that "gather" needed information without everyone getting everything. What they are I don't know but by looking at the parallel VecScatterCreates  and MatGetSubMatrices() you might get some ideas.


   Barry

On Nov 18, 2010, at 4:41 PM, Jed Brown wrote:

> On Thu, Nov 18, 2010 at 23:12, Dmitry Karpeev <karpeev at mcs.anl.gov> wrote:
> I'm not disagreeing, but I also don't see a general way of figuring
> out the off-rank
> columns for a submatrix that is destined to live on a subcomm.
> 
> VecScatter figures out who is sending to who without allgathering everyone's indices.  I realize it's a slightly different problem and I don't have a complete solution right now.  And iscol often has some structure (even if just block).
> 
> Another point is that since we anticipate a submatrix is extracted on a subcomm,
> the size of the submatrix and the subcomm might not grow in the weak
> scaling case.
> 
> That depends whether the subdomains are different materials or just solver-friendly subdomains (e.g. to reduce iterations in ASM and make overlap less expensive in 3D).  Consider getting submatrices for Fluid and 
>  
> The names are definitely an stop-gap fix: for consistency we'd have to change
> ISGetIndices -->  ISGetLocalIndices, ISGetTotalIndices -->
> ISGetIndices, VecGetArray --> VecGetLocalArray, etc.
> Maybe this will be done one day :-)
> 
> I don't know, I like to endorse the domain decomposition philosophy and make the simple things (and in particular, storage) local by default, with parallel semantics where appropriate.  In particular, you should have to spell out a word with non-scalable connotations in order to call a blatantly non-scalable function.
> 
> Jed