VecView behaviour

[Andreas Grassl]

Hi Jed,

the BNN-Algorithm in the literature distinguishes always between inner nodes and
interface nodes. The short question arising from your explanation for me is, if
owned DOF's is a synonym for the inner DOF's and ghosted DOF's for the interface
DOF's?

Below you find more extended thoughts and an example.

Jed Brown schrieb:
> Andreas Grassl wrote:
>> Barry Smith schrieb:
>>>    Hmm, it sounds like the difference between local "ghosted" vectors
>>> and the global parallel vectors. But I do not understand why any of the
>>> local vector entries would be zero.
>>> Doesn't the vector X that is passed into KSP (or SNES) have the global
>>> entries and uniquely define the solution? Why is viewing that not right?
>>>
>> I still don't understand fully the underlying processes of the whole PCNN
>> solution procedure, but trying around I substituted
>>
>> MatCreateIS(commw, ind_length, ind_length, PETSC_DECIDE, PETSC_DECIDE,
>> gridmapping, &A);
> 
> This creates a matrix that is bigger than you want, and gives you the
> dead values at the end (global dofs that are not in the range of the
> LocalToGlobalMapping.
> 
> This from the note on MatCreateIS:
> 
> | m and n are NOT related to the size of the map, they are the size of the part of the vector owned
> | by that process. m + nghosts (or n + nghosts) is the length of map since map maps all local points 
> | plus the ghost points to global indices.
> 
>> by
>>
>> MatCreateIS(commw, PETSC_DECIDE, PETSC_DECIDE, actdof, actdof, gridmapping, &A);
> 
> This creates a matrix of the correct size, but it looks like it could
> easily end up with the "wrong" dofs owned locally.  What you probably
> want to do is:
> 
> 1. Resolve ownership just like with any other DD method.  This
> partitions your dofs into n owned dofs and ngh ghosted dofs on each
> process.  The global sum of n is N, the size of the global vectors that
> the solver will interact with.

do I understand right, that owned dofs are the inner nodes and the ghosted dofs
are the interface dofs?

> 
> 2. Make an ISLocalToGlobalMapping where all the owned dofs come first,
> mapping (0..n-1) to (rstart..rstart+n-1), followed by the ghosted dofs
> (local index n..ngh-1) which map to remote processes.  (rstart is the
> global index of the first owned dof)

currently I set up my ISLocalToGlobalMapping by giving the processes all the
dofs in arbitrary order having the effect, that the interface dofs appear more
times. Attached I give you a small example with 2 subdomains and 270 DOF's.

> 
> One way to do this is to use MPI_Scan to find rstart, then number all
> the owned dofs and scatter the result.  The details will be dependent on
> how you store your mesh.  (I'm assuming it's unstructured, this step is
> trivial if you use a DA.)

Yes, the mesh is unstructured, I read out from the FE-package the partitioning
at element-basis, loop over all elements to find the belonging DOF's and
assemble the index vector for the ISLocalToGlobalMapping this way, without
regarding interface DOF's, thinking this would be done automatically by setting
up the mapping because by this some global DOF's appear more times.

> 
> 3. Call MatCreateIS(comm,n,n,PETSC_DECIDE,PETSC_DECIDE,mapping,&A);
> 

Seeing this function call and interpreting the owned DOF's as the subdomain
inner DOF's the Matrix A has not the full size?!

Given a 4x6 grid with 1 DOF per node divided into 4 subdomains I get 9 interface
DOF's.

0  o  o  O  o  5
         |
6  o  o  O  o  o
         |
O--O--O--O--O--O
         |
o  o  o  O  o  23

My first approach to create the Matrix would give a Matrix size of 35x35, with
11 dead entries at the end of the vector.

My second approach would give the "correct" Matrix size of 24x24.

By splitting up in n owned values and some ghosted values I would expect to
receive a Matrix of size 15x15. Otherwise I don't see how I could partition the
grid in a consistent way.

I would really appreciate, if you could show me, how the partition and ownership
of the DOF's in this little example work out.

cheers,

ando

-- 
 /"\                               Grassl Andreas
 \ /    ASCII Ribbon Campaign      Uni Innsbruck Institut f. Mathematik
  X      against HTML email        Technikerstr. 13 Zi 709
 / \                               +43 (0)512 507 6091