[petsc-dev] How do I collect all the values from a sequential vector on the zeroth processor into a parallel PETSc vector ?
Franck Houssen
franck.houssen at inria.fr
Thu Jan 4 11:27:40 CST 2018
----- Mail original -----
> De: "Barry F. Smith" <bsmith at mcs.anl.gov>
> À: "Franck Houssen" <franck.houssen at inria.fr>
> Cc: "Barry F. Smith" <bsmith at mcs.anl.gov>, "For users of the development version of PETSc" <petsc-dev at mcs.anl.gov>
> Envoyé: Jeudi 4 Janvier 2018 18:13:32
> Objet: Re: [petsc-dev] How do I collect all the values from a sequential vector on the zeroth processor into a
> parallel PETSc vector ?
>
>
>
> > On Jan 4, 2018, at 11:10 AM, Franck Houssen <franck.houssen at inria.fr>
> > wrote:
> >
> > Do you reproduce the error at your side ? Or is the pattern I use wrong ?
>
> You should have received an email from me with a patch for PETSc code.
> Yes I could reproduce, clearly our bug your code was fine.
Didn't get the (other) mail yet: guess I'll get it tomorrow. The mail server seems to be messy/capricious... I also noticed some attachments may be "cut" if you open them from your web browser (?!). Anyway...
OK. bug fixed : good news ! I'm on maint, I guess you pushed on master and I need to apply the patch on maint: no problem.
Thanks Barry
>
> Barry
>
> >
> > Franck
> >
> > ----- Mail original -----
> >> De: "Franck Houssen" <franck.houssen at inria.fr>
> >> À: "Barry F. Smith" <bsmith at mcs.anl.gov>
> >> Cc: "For users of the development version of PETSc"
> >> <petsc-dev at mcs.anl.gov>
> >> Envoyé: Jeudi 4 Janvier 2018 09:31:05
> >> Objet: Re: [petsc-dev] How do I collect all the values from a sequential
> >> vector on the zeroth processor into a
> >> parallel PETSc vector ?
> >>
> >> I attached it in the very first mail.
> >>
> >> Franck
> >>
> >>>> more vecScatterGatherRoot.cpp
> >> // How do I collect all the values from a sequential vector on the zeroth
> >> processor into a parallel PETSc vector ?
> >> //
> >> // ~> g++ -o vecScatterGatherRoot.exe vecScatterGatherRoot.cpp -lpetsc
> >> -lm;
> >> mpirun -n X vecScatterGatherRoot.exe
> >>
> >> #include "petsc.h"
> >>
> >> int main(int argc,char **argv) {
> >> PetscInitialize(&argc, &argv, NULL, NULL);
> >> int size = 0; MPI_Comm_size(MPI_COMM_WORLD, &size);
> >> int rank = 0; MPI_Comm_rank(MPI_COMM_WORLD, &rank);
> >>
> >> PetscInt globSize = size;
> >> Vec globVec; VecCreateMPI(PETSC_COMM_WORLD, 1, globSize, &globVec);
> >> VecSetValue(globVec, rank, (PetscScalar) (1.+rank), INSERT_VALUES);
> >> VecAssemblyBegin(globVec); VecAssemblyEnd(globVec);
> >> VecView(globVec, PETSC_VIEWER_STDOUT_WORLD);
> >> PetscViewerFlush(PETSC_VIEWER_STDOUT_WORLD);
> >>
> >> // Collect all the values from a parallel PETSc vector into a vector on
> >> the
> >> zeroth processor.
> >>
> >> Vec locVec = NULL;
> >> if (rank == 0) {
> >> PetscInt locSize = globSize;
> >> VecCreateSeq(PETSC_COMM_SELF, locSize, &locVec); VecSet(locVec, -1.);
> >> }
> >> VecScatter scatCtx; VecScatterCreateToZero(globVec, &scatCtx, &locVec);
> >> VecScatterBegin(scatCtx, globVec, locVec, INSERT_VALUES,
> >> SCATTER_FORWARD);
> >> VecScatterEnd (scatCtx, globVec, locVec, INSERT_VALUES,
> >> SCATTER_FORWARD);
> >>
> >> // Modify sequential vector on the zeroth processor.
> >>
> >> if (rank == 0) {
> >> VecView(locVec, PETSC_VIEWER_STDOUT_SELF);
> >> PetscViewerFlush(PETSC_VIEWER_STDOUT_SELF);
> >> VecScale(locVec, -1.);
> >> VecView(locVec, PETSC_VIEWER_STDOUT_SELF);
> >> PetscViewerFlush(PETSC_VIEWER_STDOUT_SELF);
> >> }
> >> MPI_Barrier(MPI_COMM_WORLD);
> >>
> >> // How do I collect all the values from a sequential vector on the zeroth
> >> processor into a parallel PETSc vector ?
> >>
> >> VecSet(globVec, 0.);
> >> VecScatterBegin(scatCtx, locVec, globVec, ADD_VALUES, SCATTER_REVERSE);
> >> VecScatterEnd (scatCtx, locVec, globVec, ADD_VALUES, SCATTER_REVERSE);
> >> VecView(globVec, PETSC_VIEWER_STDOUT_WORLD);
> >> PetscViewerFlush(PETSC_VIEWER_STDOUT_WORLD);
> >>
> >> PetscFinalize();
> >> }
> >>
> >>
> >> ----- Mail original -----
> >>> De: "Barry F. Smith" <bsmith at mcs.anl.gov>
> >>> À: "Franck Houssen" <franck.houssen at inria.fr>
> >>> Cc: "Barry F. Smith" <bsmith at mcs.anl.gov>, "Matthew Knepley"
> >>> <knepley at gmail.com>, "For users of the development
> >>> version of PETSc" <petsc-dev at mcs.anl.gov>
> >>> Envoyé: Mercredi 3 Janvier 2018 18:20:21
> >>> Objet: Re: [petsc-dev] How do I collect all the values from a sequential
> >>> vector on the zeroth processor into a
> >>> parallel PETSc vector ?
> >>>
> >>>
> >>> Send the complete code as an attachment.
> >>>
> >>>
> >>>> On Jan 3, 2018, at 11:08 AM, Franck Houssen <franck.houssen at inria.fr>
> >>>> wrote:
> >>>>
> >>>> ----- Mail original -----
> >>>>> De: "Barry F. Smith" <bsmith at mcs.anl.gov>
> >>>>> À: "Franck Houssen" <franck.houssen at inria.fr>
> >>>>> Cc: "Matthew Knepley" <knepley at gmail.com>, "For users of the
> >>>>> development
> >>>>> version of PETSc" <petsc-dev at mcs.anl.gov>
> >>>>> Envoyé: Mercredi 3 Janvier 2018 18:01:35
> >>>>> Objet: Re: [petsc-dev] How do I collect all the values from a
> >>>>> sequential
> >>>>> vector on the zeroth processor into a
> >>>>> parallel PETSc vector ?
> >>>>>
> >>>>>
> >>>>>
> >>>>>> On Jan 3, 2018, at 10:59 AM, Franck Houssen <franck.houssen at inria.fr>
> >>>>>> wrote:
> >>>>>>
> >>>>>> I need the exact opposite operation of an entry in the FAQ called "How
> >>>>>> do
> >>>>>> I
> >>>>>> collect all the values from a parallel PETSc vector into a vector on
> >>>>>> the
> >>>>>> zeroth processor?"
> >>>>>
> >>>>> You can use VecScatterCreateToZero() then do the scatter with scatter
> >>>>> reverse.
> >>>>>
> >>>>
> >>>> That's what I tried but got an error. Did I miss something ?
> >>>>
> >>>>>> tail vecScatterGatherRoot.cpp
> >>>>
> >>>> // How do I collect all the values from a sequential vector on the
> >>>> zeroth
> >>>> processor into a parallel PETSc vector ?
> >>>>
> >>>> VecSet(globVec, 0.);
> >>>> VecScatterBegin(scatCtx, locVec, globVec, ADD_VALUES, SCATTER_REVERSE);
> >>>> VecScatterEnd (scatCtx, locVec, globVec, ADD_VALUES, SCATTER_REVERSE);
> >>>> VecView(globVec, PETSC_VIEWER_STDOUT_WORLD);
> >>>> PetscViewerFlush(PETSC_VIEWER_STDOUT_WORLD);
> >>>>
> >>>> PetscFinalize();
> >>>> }
> >>>>
> >>>>>> mpirun -n 2 ./vecScatterGatherRoot.exe
> >>>> Vec Object: 2 MPI processes
> >>>> type: mpi
> >>>> Process [0]
> >>>> 1.
> >>>> Process [1]
> >>>> 2.
> >>>> Vec Object: 1 MPI processes
> >>>> type: seq
> >>>> 1.
> >>>> 2.
> >>>> Vec Object: 1 MPI processes
> >>>> type: seq
> >>>> -1.
> >>>> -2.
> >>>> ...
> >>>> [1]PETSC ERROR: [1] VecScatterBegin_MPI_ToOne line 161
> >>>> /home/fghoussen/Documents/INRIA/petsc/src/vec/vec/utils/vscat.c
> >>>> [1]PETSC ERROR: [1] VecScatterBegin line 1698
> >>>> /home/fghoussen/Documents/INRIA/petsc/src/vec/vec/utils/vscat.c
> >>>> [1]PETSC ERROR: --------------------- Error Message
> >>>> --------------------------------------------------------------
> >>>> [1]PETSC ERROR: Signal received
> >>>> [1]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html
> >>>> for
> >>>> trouble shooting.
> >>>>
> >>>>
> >>>>>>
> >>>>>> Shall I use VecScatterCreateToZero "backward", or, use
> >>>>>> VecScatterCreate
> >>>>>> instead ? If yes, how ? (my understanding is that VecScatterCreate can
> >>>>>> take only parallel vector as input)
> >>>>>
> >>>>> This understanding is completely incorrect.
> >>>>>
> >>>>> Barry
> >>>>>
> >>>>>
> >>>>>> Not sure how to do this.
> >>>>>>
> >>>>>> Its not clear what you want. Do you want a seq vector duplicated on
> >>>>>> all
> >>>>>> procs?
> >>>>>>
> >>>>>> No. The seq vec should "live" only on the master proc. This seq master
> >>>>>> vec
> >>>>>> should be "converted" into a parallel vector. Not sure how to do that
> >>>>>>
> >>>>>> Do you want it split up? The short
> >>>>>> answer is, use the appropriate scatter.
> >>>>>>
> >>>>>> Matt
> >>>>>>
> >>>>>> Franck
> >>>>>>
> >>>>>> In this example, the final VecView of the parallel globVec vector
> >>>>>> should
> >>>>>> be
> >>>>>> [-1., -2.]
> >>>>>>
> >>>>>>>> mpirun -n 2 ./vecScatterGatherRoot.exe
> >>>>>> Vec Object: 2 MPI processes
> >>>>>> type: mpi
> >>>>>> Process [0]
> >>>>>> 1.
> >>>>>> Process [1]
> >>>>>> 2.
> >>>>>> Vec Object: 1 MPI processes
> >>>>>> type: seq
> >>>>>> 1.
> >>>>>> 2.
> >>>>>> Vec Object: 1 MPI processes
> >>>>>> type: seq
> >>>>>> -1.
> >>>>>> -2.
> >>>>>> [1]PETSC ERROR:
> >>>>>> ------------------------------------------------------------------------
> >>>>>> [1]PETSC ERROR: Caught signal number 11 SEGV: Segmentation Violation,
> >>>>>> probably memory access out of range
> >>>>>>
> >>>>>>>> more vecScatterGatherRoot.cpp
> >>>>>> // How do I collect all the values from a sequential vector on the
> >>>>>> zeroth
> >>>>>> processor into a parallel PETSc vector ?
> >>>>>> //
> >>>>>> // ~> g++ -o vecScatterGatherRoot.exe vecScatterGatherRoot.cpp -lpetsc
> >>>>>> -lm;
> >>>>>> mpirun -n X vecScatterGatherRoot.exe
> >>>>>>
> >>>>>> #include "petsc.h"
> >>>>>>
> >>>>>> int main(int argc,char **argv) {
> >>>>>> PetscInitialize(&argc, &argv, NULL, NULL);
> >>>>>> int size = 0; MPI_Comm_size(MPI_COMM_WORLD, &size);
> >>>>>> int rank = 0; MPI_Comm_rank(MPI_COMM_WORLD, &rank);
> >>>>>>
> >>>>>> PetscInt globSize = size;
> >>>>>> Vec globVec; VecCreateMPI(PETSC_COMM_WORLD, 1, globSize, &globVec);
> >>>>>> VecSetValue(globVec, rank, (PetscScalar) (1.+rank), INSERT_VALUES);
> >>>>>> VecAssemblyBegin(globVec); VecAssemblyEnd(globVec);
> >>>>>> VecView(globVec, PETSC_VIEWER_STDOUT_WORLD);
> >>>>>> PetscViewerFlush(PETSC_VIEWER_STDOUT_WORLD);
> >>>>>>
> >>>>>> // Collect all the values from a parallel PETSc vector into a vector
> >>>>>> on
> >>>>>> the zeroth processor.
> >>>>>>
> >>>>>> Vec locVec = NULL;
> >>>>>> if (rank == 0) {
> >>>>>> PetscInt locSize = globSize;
> >>>>>> VecCreateSeq(PETSC_COMM_SELF, locSize, &locVec); VecSet(locVec,
> >>>>>> -1.);
> >>>>>> }
> >>>>>> VecScatter scatCtx; VecScatterCreateToZero(globVec, &scatCtx,
> >>>>>> &locVec);
> >>>>>> VecScatterBegin(scatCtx, globVec, locVec, INSERT_VALUES,
> >>>>>> SCATTER_FORWARD);
> >>>>>> VecScatterEnd (scatCtx, globVec, locVec, INSERT_VALUES,
> >>>>>> SCATTER_FORWARD);
> >>>>>>
> >>>>>> // Modify sequential vector on the zeroth processor.
> >>>>>>
> >>>>>> if (rank == 0) {
> >>>>>> VecView(locVec, PETSC_VIEWER_STDOUT_SELF);
> >>>>>> PetscViewerFlush(PETSC_VIEWER_STDOUT_SELF);
> >>>>>> VecScale(locVec, -1.);
> >>>>>> VecView(locVec, PETSC_VIEWER_STDOUT_SELF);
> >>>>>> PetscViewerFlush(PETSC_VIEWER_STDOUT_SELF);
> >>>>>> }
> >>>>>> MPI_Barrier(MPI_COMM_WORLD);
> >>>>>>
> >>>>>> // How do I collect all the values from a sequential vector on the
> >>>>>> zeroth
> >>>>>> processor into a parallel PETSc vector ?
> >>>>>>
> >>>>>> VecSet(globVec, 0.);
> >>>>>> VecScatterBegin(scatCtx, locVec, globVec, ADD_VALUES,
> >>>>>> SCATTER_REVERSE);
> >>>>>> VecScatterEnd (scatCtx, locVec, globVec, ADD_VALUES,
> >>>>>> SCATTER_REVERSE);
> >>>>>> VecView(globVec, PETSC_VIEWER_STDOUT_WORLD);
> >>>>>> PetscViewerFlush(PETSC_VIEWER_STDOUT_WORLD);
> >>>>>>
> >>>>>> PetscFinalize();
> >>>>>> }
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> What most experimenters take for granted before they begin their
> >>>>>> experiments is infinitely more interesting than any results to which
> >>>>>> their
> >>>>>> experiments lead.
> >>>>>> -- Norbert Wiener
> >>>>>>
> >>>>>> https://www.cse.buffalo.edu/~knepley/
> >>>>>>
> >>>>>
> >>>>>
> >>>
> >>>
> >>
>
>
More information about the petsc-dev
mailing list