about ghosted vectors and there local representation

[Thomas Geenen]

oke a small example with 2 cpu's

local2global, 
[0] 0 0
[0] 1 1
[0] 2 2
[0] 3 3
[0] 4 4
[0] 5 5
[0] 6 6
[0] 7 7
[0] 8 8
[0] 9 9
[0] 10 10
[0] 11 11
[0] 12 12
[0] 13 13
[1] 0 0
[1] 1 1
[1] 2 14
[1] 3 15
[1] 4 16
[1] 5 17
[1] 6 18
[1] 7 19
[1] 8 20
[1] 9 21
[1] 10 22
[1] 11 23
[1] 12 24
[1] 13 25
on cpu0 nc = 14, nrows =14, nghosts=0
on cpu1 nc= 12, nrows=14, nghosts=2, ghosts=(0,1)
ierr = KSPGetSolution(ksp, &x);
ierr = VecGhostUpdateBegin(x,INSERT_VALUES,SCATTER_FORWARD);
ierr = VecGhostUpdateEnd(x,INSERT_VALUES,SCATTER_FORWARD);
ierr = VecGhostGetLocalForm(x, &lx);
ierr = VecView(lx, PETSC_NULL);
cpu0
-1.98507e-08
1.09281e-08
0.592588
0.160857
-0.666662
-34.895
8.93136
0.499999
4.22046e-08
0.2963
-0.0497465
0.666661
26.8951
-8.93131
cpu1
-0.2963
0.0497465
0.666662
-26.8951
8.93131
-0.499999
-1.70893e-08
-0.592588
-0.160857
-0.666661
34.895
-8.93136
-1.98507e-08
-1.98507e-08

1) ghost values are at the end of local solution array lx on cpu1 I do the 
renumering myself. would be nice if this could be done by petsc automaticly
2) the first two values on cpu0 are not equal to the last two at cpu1


thomas

On Sunday 25 June 2006 15:20, you wrote:
> ----------  Forwarded Message  ----------
>
> Subject: Re: Fwd: Re: about ghosted vectors and there local representation
> Date: Thursday 22 June 2006 22:13
> From: Barry Smith <bsmith at mcs.anl.gov>
> To: petsc-users at mcs.anl.gov
>
>    Send the values of local2global, nc, ghosts, nrows and nghosts for all
> processors (2 I hope). Also note that nrows should equal nc + nghosts.
>
>     Barry
>
> On Thu, 22 Jun 2006, Thomas Geenen wrote:
> > nrows is the global number of rows
> > local2global is the local to global renumbering
> > nghosts are the number of ghosts
> > ghosts is the global row number i want to have local access to after the
> > solve
> >
> >
> > ISLocalToGlobalMappingCreate(PETSC_COMM_WORLD, nrows, local2global,
> > &ltog); VecCreateGhost(PETSC_COMM_WORLD, nc,
> >                            	PETSC_DECIDE, nghosts[mype], ghosts, &x);
> > VecSetLocalToGlobalMapping(x,ltog);
> > VecSetValuesLocal(x, nc, indx, x_values, INSERT_VALUES);
> > VecAssemblyBegin(x);
> > VecAssemblyEnd(x);
> >
> > solve the system
> >
> > ierr = KSPGetSolution(ksp, &x);
> > ierr = VecGhostUpdateBegin(x,INSERT_VALUES,SCATTER_FORWARD);
> > ierr = VecGhostUpdateEnd(x,INSERT_VALUES,SCATTER_FORWARD);
> > ierr = VecGhostGetLocalForm(x, &lx);
> >
> > nrows is the global number of rows
> > local2global is the local to global renumbering
> > nghosts are the number of ghosts
> > ghosts is the global row number i want to have local access to after the
> > solve
> >
> >
> > thomas
> >
> > On Thursday 22 June 2006 09:20, you wrote:
> >> ----------  Forwarded Message  ----------
> >>
> >> Subject: Re: about ghosted vectors and there local representation
> >> Date: Tuesday 20 June 2006 20:39
> >> From: Barry Smith <bsmith at mcs.anl.gov>
> >> To: petsc-users at mcs.anl.gov
> >>
> >>    The ltog has to exactly match the way the ghosts are provided to
> >> VecCreateGhost(). Are they? Please send the exact example code.
> >>
> >>     Barry
> >>
> >> On Tue, 20 Jun 2006, Thomas Geenen wrote:
> >>> Dear Petsc users,
> >>>
> >>> I think that I do not use the ghosted vector concept in the right way.
> >>>
> >>> I create a vector with
> >>> ierr = VecCreateGhost(PETSC_COMM_WORLD, nc,
> >>>                            		 PETSC_DECIDE, nghosts, ghosts, &x);
> >>> The entries in ghost contain the global vector positions that I want to
> >>> have available locally.
> >>>
> >>> I have a local representation so i apply (after having constructed the
> >>> mapping of course)
> >>> ierr = VecSetLocalToGlobalMapping(x,ltog);
> >>> I now insert the values
> >>> ierr = VecSetValuesLocal(x, nc, indx, x_values, INSERT_VALUES);
> >>> VecAssemblyBegin(x);
> >>> VecAssemblyEnd(x);
> >>> I am now under the assumption that petsc divided the vector entries
> >>> among processes according to his global representation of the vector.
> >>>
> >>> I solve my system of equations
> >>> KSPSolve(ksp,b,x);
> >>> afterwards i do
> >>> ierr = KSPGetSolution(ksp, &x);
> >>> ierr = VecGhostUpdateBegin(x,INSERT_VALUES,SCATTER_FORWARD);
> >>> ierr = VecGhostUpdateEnd(x,INSERT_VALUES,SCATTER_FORWARD);
> >>> ierr = VecGhostGetLocalForm(x, &lx);
> >>>
> >>> I now expect the vector to be in the local form. That means that the
> >>> positions that I created as ghost points are filled with the solution
> >>> corresponding to the global solution. I expect that they are inserted
> >>> in the position in my local vector corresponding to there corresponding
> >>> global position.
> >>>
> >>> apparently this is not what happens.
> >>>
> >>> a small example
> >>> 2 cpu's
> >>> each cpu contains 3 vector entries however there is an overlap of 1
> >>> on cpu 1 the vector contains points (1, 2, 3)
> >>> on cpu 2 the vector contains points (3, 4, 5)
> >>>
> >>> I make position 1 on cpu 2 a ghost points with nghost=1 ghosts(3)
> >>> after solving and restoring I expect vector lx to contain the solution
> >>> as (3, 4, 5)
> >>>
> >>>
> >>> I hope I have explained my misunderstanding of the underlying concept
> >>> clear enough
> >>>
> >>> thanks for your help
> >>> Thomas Geenen
> >>
> >> -------------------------------------------------------
>
> -------------------------------------------------------