[petsc-dev] How do I collect all the values from a sequential vector on the zeroth processor into a parallel PETSc vector ?
Smith, Barry F.
bsmith at mcs.anl.gov
Thu Jan 4 11:10:03 CST 2018
Frank,
This is our bug. I have attached a patch and also fixed it in the PETSc repository master branch.
apply with
patch -p1 < franck.patch
if you are using the tarball version of PETSc.
Barry
> On Jan 4, 2018, at 2:31 AM, Franck Houssen <franck.houssen at inria.fr> wrote:
>
> I attached it in the very first mail.
>
> Franck
>
>>> more vecScatterGatherRoot.cpp
> // How do I collect all the values from a sequential vector on the zeroth processor into a parallel PETSc vector ?
> //
> // ~> g++ -o vecScatterGatherRoot.exe vecScatterGatherRoot.cpp -lpetsc -lm; mpirun -n X vecScatterGatherRoot.exe
>
> #include "petsc.h"
>
> int main(int argc,char **argv) {
> PetscInitialize(&argc, &argv, NULL, NULL);
> int size = 0; MPI_Comm_size(MPI_COMM_WORLD, &size);
> int rank = 0; MPI_Comm_rank(MPI_COMM_WORLD, &rank);
>
> PetscInt globSize = size;
> Vec globVec; VecCreateMPI(PETSC_COMM_WORLD, 1, globSize, &globVec);
> VecSetValue(globVec, rank, (PetscScalar) (1.+rank), INSERT_VALUES);
> VecAssemblyBegin(globVec); VecAssemblyEnd(globVec);
> VecView(globVec, PETSC_VIEWER_STDOUT_WORLD); PetscViewerFlush(PETSC_VIEWER_STDOUT_WORLD);
>
> // Collect all the values from a parallel PETSc vector into a vector on the zeroth processor.
>
> Vec locVec = NULL;
> if (rank == 0) {
> PetscInt locSize = globSize;
> VecCreateSeq(PETSC_COMM_SELF, locSize, &locVec); VecSet(locVec, -1.);
> }
> VecScatter scatCtx; VecScatterCreateToZero(globVec, &scatCtx, &locVec);
> VecScatterBegin(scatCtx, globVec, locVec, INSERT_VALUES, SCATTER_FORWARD);
> VecScatterEnd (scatCtx, globVec, locVec, INSERT_VALUES, SCATTER_FORWARD);
>
> // Modify sequential vector on the zeroth processor.
>
> if (rank == 0) {
> VecView(locVec, PETSC_VIEWER_STDOUT_SELF); PetscViewerFlush(PETSC_VIEWER_STDOUT_SELF);
> VecScale(locVec, -1.);
> VecView(locVec, PETSC_VIEWER_STDOUT_SELF); PetscViewerFlush(PETSC_VIEWER_STDOUT_SELF);
> }
> MPI_Barrier(MPI_COMM_WORLD);
>
> // How do I collect all the values from a sequential vector on the zeroth processor into a parallel PETSc vector ?
>
> VecSet(globVec, 0.);
> VecScatterBegin(scatCtx, locVec, globVec, ADD_VALUES, SCATTER_REVERSE);
> VecScatterEnd (scatCtx, locVec, globVec, ADD_VALUES, SCATTER_REVERSE);
> VecView(globVec, PETSC_VIEWER_STDOUT_WORLD); PetscViewerFlush(PETSC_VIEWER_STDOUT_WORLD);
>
> PetscFinalize();
> }
>
>
> ----- Mail original -----
>> De: "Barry F. Smith" <bsmith at mcs.anl.gov>
>> À: "Franck Houssen" <franck.houssen at inria.fr>
>> Cc: "Barry F. Smith" <bsmith at mcs.anl.gov>, "Matthew Knepley" <knepley at gmail.com>, "For users of the development
>> version of PETSc" <petsc-dev at mcs.anl.gov>
>> Envoyé: Mercredi 3 Janvier 2018 18:20:21
>> Objet: Re: [petsc-dev] How do I collect all the values from a sequential vector on the zeroth processor into a
>> parallel PETSc vector ?
>>
>>
>> Send the complete code as an attachment.
>>
>>
>>> On Jan 3, 2018, at 11:08 AM, Franck Houssen <franck.houssen at inria.fr>
>>> wrote:
>>>
>>> ----- Mail original -----
>>>> De: "Barry F. Smith" <bsmith at mcs.anl.gov>
>>>> À: "Franck Houssen" <franck.houssen at inria.fr>
>>>> Cc: "Matthew Knepley" <knepley at gmail.com>, "For users of the development
>>>> version of PETSc" <petsc-dev at mcs.anl.gov>
>>>> Envoyé: Mercredi 3 Janvier 2018 18:01:35
>>>> Objet: Re: [petsc-dev] How do I collect all the values from a sequential
>>>> vector on the zeroth processor into a
>>>> parallel PETSc vector ?
>>>>
>>>>
>>>>
>>>>> On Jan 3, 2018, at 10:59 AM, Franck Houssen <franck.houssen at inria.fr>
>>>>> wrote:
>>>>>
>>>>> I need the exact opposite operation of an entry in the FAQ called "How do
>>>>> I
>>>>> collect all the values from a parallel PETSc vector into a vector on the
>>>>> zeroth processor?"
>>>>
>>>> You can use VecScatterCreateToZero() then do the scatter with scatter
>>>> reverse.
>>>>
>>>
>>> That's what I tried but got an error. Did I miss something ?
>>>
>>>>> tail vecScatterGatherRoot.cpp
>>>
>>> // How do I collect all the values from a sequential vector on the zeroth
>>> processor into a parallel PETSc vector ?
>>>
>>> VecSet(globVec, 0.);
>>> VecScatterBegin(scatCtx, locVec, globVec, ADD_VALUES, SCATTER_REVERSE);
>>> VecScatterEnd (scatCtx, locVec, globVec, ADD_VALUES, SCATTER_REVERSE);
>>> VecView(globVec, PETSC_VIEWER_STDOUT_WORLD);
>>> PetscViewerFlush(PETSC_VIEWER_STDOUT_WORLD);
>>>
>>> PetscFinalize();
>>> }
>>>
>>>>> mpirun -n 2 ./vecScatterGatherRoot.exe
>>> Vec Object: 2 MPI processes
>>> type: mpi
>>> Process [0]
>>> 1.
>>> Process [1]
>>> 2.
>>> Vec Object: 1 MPI processes
>>> type: seq
>>> 1.
>>> 2.
>>> Vec Object: 1 MPI processes
>>> type: seq
>>> -1.
>>> -2.
>>> ...
>>> [1]PETSC ERROR: [1] VecScatterBegin_MPI_ToOne line 161
>>> /home/fghoussen/Documents/INRIA/petsc/src/vec/vec/utils/vscat.c
>>> [1]PETSC ERROR: [1] VecScatterBegin line 1698
>>> /home/fghoussen/Documents/INRIA/petsc/src/vec/vec/utils/vscat.c
>>> [1]PETSC ERROR: --------------------- Error Message
>>> --------------------------------------------------------------
>>> [1]PETSC ERROR: Signal received
>>> [1]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html for
>>> trouble shooting.
>>>
>>>
>>>>>
>>>>> Shall I use VecScatterCreateToZero "backward", or, use VecScatterCreate
>>>>> instead ? If yes, how ? (my understanding is that VecScatterCreate can
>>>>> take only parallel vector as input)
>>>>
>>>> This understanding is completely incorrect.
>>>>
>>>> Barry
>>>>
>>>>
>>>>> Not sure how to do this.
>>>>>
>>>>> Its not clear what you want. Do you want a seq vector duplicated on all
>>>>> procs?
>>>>>
>>>>> No. The seq vec should "live" only on the master proc. This seq master
>>>>> vec
>>>>> should be "converted" into a parallel vector. Not sure how to do that
>>>>>
>>>>> Do you want it split up? The short
>>>>> answer is, use the appropriate scatter.
>>>>>
>>>>> Matt
>>>>>
>>>>> Franck
>>>>>
>>>>> In this example, the final VecView of the parallel globVec vector should
>>>>> be
>>>>> [-1., -2.]
>>>>>
>>>>>>> mpirun -n 2 ./vecScatterGatherRoot.exe
>>>>> Vec Object: 2 MPI processes
>>>>> type: mpi
>>>>> Process [0]
>>>>> 1.
>>>>> Process [1]
>>>>> 2.
>>>>> Vec Object: 1 MPI processes
>>>>> type: seq
>>>>> 1.
>>>>> 2.
>>>>> Vec Object: 1 MPI processes
>>>>> type: seq
>>>>> -1.
>>>>> -2.
>>>>> [1]PETSC ERROR:
>>>>> ------------------------------------------------------------------------
>>>>> [1]PETSC ERROR: Caught signal number 11 SEGV: Segmentation Violation,
>>>>> probably memory access out of range
>>>>>
>>>>>>> more vecScatterGatherRoot.cpp
>>>>> // How do I collect all the values from a sequential vector on the zeroth
>>>>> processor into a parallel PETSc vector ?
>>>>> //
>>>>> // ~> g++ -o vecScatterGatherRoot.exe vecScatterGatherRoot.cpp -lpetsc
>>>>> -lm;
>>>>> mpirun -n X vecScatterGatherRoot.exe
>>>>>
>>>>> #include "petsc.h"
>>>>>
>>>>> int main(int argc,char **argv) {
>>>>> PetscInitialize(&argc, &argv, NULL, NULL);
>>>>> int size = 0; MPI_Comm_size(MPI_COMM_WORLD, &size);
>>>>> int rank = 0; MPI_Comm_rank(MPI_COMM_WORLD, &rank);
>>>>>
>>>>> PetscInt globSize = size;
>>>>> Vec globVec; VecCreateMPI(PETSC_COMM_WORLD, 1, globSize, &globVec);
>>>>> VecSetValue(globVec, rank, (PetscScalar) (1.+rank), INSERT_VALUES);
>>>>> VecAssemblyBegin(globVec); VecAssemblyEnd(globVec);
>>>>> VecView(globVec, PETSC_VIEWER_STDOUT_WORLD);
>>>>> PetscViewerFlush(PETSC_VIEWER_STDOUT_WORLD);
>>>>>
>>>>> // Collect all the values from a parallel PETSc vector into a vector on
>>>>> the zeroth processor.
>>>>>
>>>>> Vec locVec = NULL;
>>>>> if (rank == 0) {
>>>>> PetscInt locSize = globSize;
>>>>> VecCreateSeq(PETSC_COMM_SELF, locSize, &locVec); VecSet(locVec, -1.);
>>>>> }
>>>>> VecScatter scatCtx; VecScatterCreateToZero(globVec, &scatCtx, &locVec);
>>>>> VecScatterBegin(scatCtx, globVec, locVec, INSERT_VALUES,
>>>>> SCATTER_FORWARD);
>>>>> VecScatterEnd (scatCtx, globVec, locVec, INSERT_VALUES,
>>>>> SCATTER_FORWARD);
>>>>>
>>>>> // Modify sequential vector on the zeroth processor.
>>>>>
>>>>> if (rank == 0) {
>>>>> VecView(locVec, PETSC_VIEWER_STDOUT_SELF);
>>>>> PetscViewerFlush(PETSC_VIEWER_STDOUT_SELF);
>>>>> VecScale(locVec, -1.);
>>>>> VecView(locVec, PETSC_VIEWER_STDOUT_SELF);
>>>>> PetscViewerFlush(PETSC_VIEWER_STDOUT_SELF);
>>>>> }
>>>>> MPI_Barrier(MPI_COMM_WORLD);
>>>>>
>>>>> // How do I collect all the values from a sequential vector on the
>>>>> zeroth
>>>>> processor into a parallel PETSc vector ?
>>>>>
>>>>> VecSet(globVec, 0.);
>>>>> VecScatterBegin(scatCtx, locVec, globVec, ADD_VALUES, SCATTER_REVERSE);
>>>>> VecScatterEnd (scatCtx, locVec, globVec, ADD_VALUES, SCATTER_REVERSE);
>>>>> VecView(globVec, PETSC_VIEWER_STDOUT_WORLD);
>>>>> PetscViewerFlush(PETSC_VIEWER_STDOUT_WORLD);
>>>>>
>>>>> PetscFinalize();
>>>>> }
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> What most experimenters take for granted before they begin their
>>>>> experiments is infinitely more interesting than any results to which
>>>>> their
>>>>> experiments lead.
>>>>> -- Norbert Wiener
>>>>>
>>>>> https://www.cse.buffalo.edu/~knepley/
>>>>>
>>>>
>>>>
>>
>>
> <vecScatterGatherRoot.cpp>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20180104/2a8efa0f/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: franck.patch
Type: application/octet-stream
Size: 799 bytes
Desc: franck.patch
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20180104/2a8efa0f/attachment-0001.obj>
More information about the petsc-dev
mailing list