[petsc-users] Coordinate format internal reordering

[Enrico]

Maybe I wasn't clear enough. I would like to completely get rid of Petsc 
ordering because I don't want extra communication between processes to 
construct the vector and the matrix (since I have to fill them every 
time step because I'm just using the linear solver with a Mat and a Vec 
data structure). I don't understand how I can do that.

My initial idea was to create another global index ordering within my 
application to use only for the Petsc interface but then I think that 
the ghost cells are wrong.

On 19/10/2023 14:50, Matthew Knepley wrote:
> On Thu, Oct 19, 2023 at 6:51 AM Enrico <degregori at dkrz.de 
> <mailto:degregori at dkrz.de>> wrote:
> 
>     Hello,
> 
>     if I create an application ordering using AOCreateBasic, should I
>     provide the same array for const PetscInt myapp[] and const PetscInt
>     mypetsc[] in order to get the same ordering of the application
>     within PETSC?
> 
> 
> Are you asking if the identity permutation can be constructed using the 
> same array twice? Yes.
> 
>     And once I define the ordering so that the local vector and matrix are
>     defined in PETSC as in my application, how can I use it to create the
>     actual vector and matrix?
> 
> 
> The vectors and matrices do not change. The AO is a permutation. You can 
> use it to permute
> a vector into another order, or to convert on index to another.
> 
>    Thanks,
> 
>        Matt
> 
>     Thanks in advance for the help.
> 
>     Cheers,
>     Enrico
> 
>     On 18/10/2023 13:39, Matthew Knepley wrote:
>      > On Wed, Oct 18, 2023 at 5:55 AM Enrico <degregori at dkrz.de
>     <mailto:degregori at dkrz.de>
>      > <mailto:degregori at dkrz.de <mailto:degregori at dkrz.de>>> wrote:
>      >
>      >     Hello,
>      >
>      >     I'm trying to use Petsc to solve a linear system in an
>     application. I'm
>      >     using the coordinate format to define the matrix and the
>     vector (it
>      >     should work better on GPU but at the moment every test is on
>     CPU).
>      >     After
>      >     the call to VecSetValuesCOO, I've noticed that the vector is
>     storing
>      >     the
>      >     data in a different way from my application. For example with two
>      >     processes in the application
>      >
>      >     process 0 owns cells 2, 3, 4
>      >
>      >     process 1 owns cells 0, 1, 5
>      >
>      >     But in the vector data structure of Petsc
>      >
>      >     process 0 owns cells 0, 1, 2
>      >
>      >     process 1 owns cells 3, 4, 5
>      >
>      >     This is in principle not a big issue, but after solving the
>     linear
>      >     system I get the solution vector x and I want to get the
>     values in the
>      >     correct processes. Is there a way to get vector values from other
>      >     processes or to get a mapping so that I can do it myself?
>      >
>      >
>      > By definition, PETSc vectors and matrices own contiguous row
>     blocks. If
>      > you want to have another,
>      > global ordering, we support that with
>      > https://petsc.org/main/manualpages/AO/
>     <https://petsc.org/main/manualpages/AO/>
>      > <https://petsc.org/main/manualpages/AO/
>     <https://petsc.org/main/manualpages/AO/>>
>      >
>      >    Thanks,
>      >
>      >       Matt
>      >
>      >     Cheers,
>      >     Enrico Degregori
>      >
>      >
>      >
>      > --
>      > What most experimenters take for granted before they begin their
>      > experiments is infinitely more interesting than any results to which
>      > their experiments lead.
>      > -- Norbert Wiener
>      >
>      > https://www.cse.buffalo.edu/~knepley/
>     <https://www.cse.buffalo.edu/~knepley/>
>     <http://www.cse.buffalo.edu/~knepley/
>     <http://www.cse.buffalo.edu/~knepley/>>
> 
> 
> 
> -- 
> What most experimenters take for granted before they begin their 
> experiments is infinitely more interesting than any results to which 
> their experiments lead.
> -- Norbert Wiener
> 
> https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>