[petsc-users] Some questions regarding VecNest type.

[Vijay S. Mahadevan]

> It would make sense to have VecNestSetSubVec(), but it is not implemented
> yet.

In src/vec/vec/examples/tests/ex37.c, I see a call to this function
but it is part of dead code.

I don't see a specialization VecGetSubVector/VecRestoreSubVector in
VecNest and so my related question is if I call VecGetSubVector with
multiple indices, do I expect a nested vector back ? And in the case
when the index set has only one entry, perhaps the returned vector is
in the parallel layout of the corresponding block entry. Or does the
returned Vector have a different parallel layout in these cases ?

> You get a lot more consistent support if you
> use a standard contiguous Vec type and call VecGetSubVector() or a
> VecScatter when you need to get part of it.

I used to take this route until I found the block extensions. I feel
it is elegant to handle different physics blocks using the block or
Nest interface specifically since dofs for each physics can be
distributed independently based on its own mesh and other
requirements. IMO, the blocking also makes the solve/preconditioning
for multi-physics problems natural (yes the sparsity pattern for
off-diagonal blocks are tricky). I will also investigate the Nest
independent interface to generalize this access.

I will look at the tests and look at the implementation again to
understand the details. The VecNestSetSubVec() would be nice to have
though. I will see if I can do this today and send a patch if it
succeeds. Anyway, thanks for the pointers !

Oh and an unrelated question: I couldn't find what replaced the
Vec/Mat-Valid routines that existed in 3.0. Currently is there a
routine to check if a Vec/Mat has called its setup yet ?

Vijay

On Tue, Oct 11, 2011 at 1:19 PM, Jed Brown <jedbrown at mcs.anl.gov> wrote:
> On Tue, Oct 11, 2011 at 00:16, Vijay S. Mahadevan <vijay.m at gmail.com> wrote:
>>
>> I understand that the Vec/Mat Nest types in the current release of
>> petsc are designed to be similar in philosophy to Dave May's petsc-ext
>> "block" vectors and matrices.
>
> They have a similar data structure, but have an interface that interoperates
> better, so they should replace petsc-ext.
>
>>
>> I have some existing code based on
>> petsc-ext and am trying to figure out the amount of work involved to
>> make a transition from petsc-ext to pure petsc (>=3.2).
>>
>> Here are a list of my questions after looking at the VecNest type:
>>
>> 1) Does VecDuplicate create a shallow or deep copy ? i.e., are all
>> leaf vectors/matrices recreated based on a recursive duplicate or does
>> it only replicate the higher level structure ?
>
> Deep copy. A shallow copy would not be semantically correct.
>
>>
>> 2) Do I need to restore the references after I get the vectors ? Say
>> VecNestGetSubVecs and something along the lines of
>> VecNestRestoreSubVecs (I dont see such a routine though) ? Or is this
>> unnecessary ?
>
> VecNestGetSubVecs() gives you direct access to the data structure. It does
> not increment a reference and does not need to be restored. You should
> prefer to use VecGetSubVector() which works for any Vec type and still gives
> you a cheap (no-copy) reference for VecNest.
>
>>
>> 3) Is there a way to lazily set the reference to one of the vectors
>> that is part of the nest structure ? Or does it always have to be
>> specified during creation of VECNEST ? I see that this is not the case
>> for matrices since there is a MatNestSetSubMats but I don't see
>> something similar for Vec. Can this be handled via VecNestGetSubVecs ?
>> In petsc-ext, there was a function that handled VecBlockSetValue and I
>> guess I am looking for a suitable replacement, if that shows better
>> light on the context.
>
> Even with Mat, you are only supposed to call this once (at setup time). If
> it doesn't check for this, it should. It could be modified to be safe to
> call more than once, but is not currently. The single-Mat function
> MatNestSetSubMat() does exist and is safe to call multiple times.
> It would make sense to have VecNestSetSubVec(), but it is not implemented
> yet.
>>
>> 4) Are there any examples that make use of the VecNest type ? Even a
>> trivial one might help clarify some trouble I'm having in
>> understanding calling sequence.
>
> In general, I don't recommend using VecNest unless it's very, very important
> to you that fields with some interlaced ordering actually be stored
> separately (or you have very specific requirements about in-place ghost
> updates for individual fields). You get a lot more consistent support if you
> use a standard contiguous Vec type and call VecGetSubVector() or a
> VecScatter when you need to get part of it. If you write code this way,
> you'll be able to use -dm_vec_type nest sometime and see if it helps
> performance.
> If after the above, you are still dedicated to needless pain, check the
> tests for usage of the internal interfaces.
> $ grep -l VecNest src/**/examples/**/*.c
> src/ksp/ksp/examples/tests/ex22.c
> src/snes/examples/tests/ex17.c
> src/vec/vec/examples/tests/ex37.c
>
> Note: MatNest is far more useful because the data structure tradeoffs are
> more significant when using PCFieldSplit. Again, you should try to avoid
> ever using the MatNest() interface directly. Instead, you can use
> MatGetLocalSubMatrix(). See src/snes/examples/tutorials/ex28.c, make
> runex28_3 for usage of MatNest without ever referencing it in the code.
> Note that there is still a preallocation interface challenge for
> off-diagonal blocks when using MatNest in a multiphysics application.