[petsc-dev] Index sets in MatCreateNest

[Jed Brown]

On Mon, Apr 11, 2011 at 08:43, Thomas Witkowski <
thomas.witkowski at tu-dresden.de> wrote:

> Could some of you give me a short explanation, what the index sets in
> MatCreateNest can be used for? I already had a look to the examples and
> tests, but all of them just put PETSC_NULL for the index sets or create the
> standard contiguous index sets by hand (ksp/ksp/example/tests/ex11.c). Does
> the index sets influence the row/column indices for the final MatNest?
>

The primary purpose I had in mind was enabling a matrix-free method to
interlace fields in the residual and Krylov vectors (e.g. for better cache
reuse) while splitting them apart in the field-split preconditioner without
any copies.


> The reason why I'm asking for is that I want to have a special order in my
> MatNest. My scenario is as follows: I have a 2x2 MatNest. The first block
> corresponds to all interior nodes on all subdomains. The second block
> corresponds to the boundary nodes between all subdomains. This works fine,
> also with the fieldsplit preconditioner! Now I would like to permute the
> global index of the second block in that way, that all edge boundary nodes
> are listed first and finally all vertex boundary nodes (its a 2D problem).
>

This is almost certainly a bad ordering because it will not reuse cache
efficiently, thus decreasing the throughput that sparse matrix kernels
achieve.


> So each rank local matrix of the second block is no more contiguous. I'm
> not sure, if I need to change the creation of the second block matrix or the
> creation of the nested matrix. I thought may be it is possible to do it with
> the index sets when creating the MatNest?
>

If I understand what you want correctly, then you can certainly do this.
Order you unknowns however you want, give MatCreateNest the index sets for
interior and boundary nodes and give the same index sets to PCFieldSplit.

We have not yet implemented MatSetValuesLocal_Nest because our primary model
was to use MatGetLocalSubMatrix and use MatSetValuesLocal to set values on
each block separately. For your purposes, calling
MatSetValuesLocal(Anest,...) would probably be more convenient (but if you
are currently using MatNest, you must already be assembling the blocks
separately).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20110411/ccc6b4db/attachment.html>