[petsc-users] Input and output distributions of vectors for PCShellSetApply 'apply' routine

[Jack Poulson]

On Mon, Mar 7, 2011 at 11:32 AM, Barry Smith <bsmith at mcs.anl.gov> wrote:

>
> On Mar 6, 2011, at 11:03 PM, Jack Poulson wrote:
>
> > Sorry to hit the list so quickly again, but the PCShellSetApply
> documentation doesn't say anything about how the 'xin' and 'xout' vectors
> will be distributed, or if it's up to the user to determine it.
> >
> > Given that a distributed Mat implies a right and left vector
> distribution, I would assume that the 'xin' should be distributed like the
> 'left' vector distribution for a left preconditioner.
> >
> > Is it up to the application routine to determine the 'xout' vector
> distribution? If so, is there a performance penalty for making it something
> other than the 'right' vector distribution implied by the matrix registered
> with the KSP? If not, what distribution does it need to be?
>
>     The PCShell doesn't determine the vector layouts at all. That is
> determined by the KSP that is using the PC. So basically your PC should
> handle any distribution; if your PCShell can only handle certain
> distributions then you need to make sure that the Vecs and Mats you provide
> to KSP or SNES (or TS) match what the PCShell needs.
>
>   Barry
>
>
It turns out that I had simply misunderstood the way PETSc was distributing
matrices under the hood. I was assuming that it was best to only distribute
the rows of the matrix using something along the lines of MatSetSizes( A,
N/nproc, N, N, N ), but Jed informed me that the local size arguments for
MatSetSizes only constrain the left and right vector distributions implied
by the matrix distribution (through rmap and cmap, respectively). Also, I
now know that KSP expects the right and left vector distributions to be the
same for square matrices, so I should instead create my matrix along the
lines of MatSetSizes( A, N/nproc, N/nproc, N, N ), and then both the in and
out vectors for the PCShell application routine should be distributed in the
same manner.

Thank you both for clearing up my misconceptions.

Jack
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20110307/47c4e859/attachment.htm>