[petsc-dev] How do I collect all the values from a sequential vector on the zeroth processor into a parallel PETSc vector ?
Smith, Barry F.
bsmith at mcs.anl.gov
Thu Jan 4 11:13:32 CST 2018
> On Jan 4, 2018, at 11:10 AM, Franck Houssen <franck.houssen at inria.fr> wrote:
>
> Do you reproduce the error at your side ? Or is the pattern I use wrong ?
You should have received an email from me with a patch for PETSc code. Yes I could reproduce, clearly our bug your code was fine.
Barry
>
> Franck
>
> ----- Mail original -----
>> De: "Franck Houssen" <franck.houssen at inria.fr>
>> À: "Barry F. Smith" <bsmith at mcs.anl.gov>
>> Cc: "For users of the development version of PETSc" <petsc-dev at mcs.anl.gov>
>> Envoyé: Jeudi 4 Janvier 2018 09:31:05
>> Objet: Re: [petsc-dev] How do I collect all the values from a sequential vector on the zeroth processor into a
>> parallel PETSc vector ?
>>
>> I attached it in the very first mail.
>>
>> Franck
>>
>>>> more vecScatterGatherRoot.cpp
>> // How do I collect all the values from a sequential vector on the zeroth
>> processor into a parallel PETSc vector ?
>> //
>> // ~> g++ -o vecScatterGatherRoot.exe vecScatterGatherRoot.cpp -lpetsc -lm;
>> mpirun -n X vecScatterGatherRoot.exe
>>
>> #include "petsc.h"
>>
>> int main(int argc,char **argv) {
>> PetscInitialize(&argc, &argv, NULL, NULL);
>> int size = 0; MPI_Comm_size(MPI_COMM_WORLD, &size);
>> int rank = 0; MPI_Comm_rank(MPI_COMM_WORLD, &rank);
>>
>> PetscInt globSize = size;
>> Vec globVec; VecCreateMPI(PETSC_COMM_WORLD, 1, globSize, &globVec);
>> VecSetValue(globVec, rank, (PetscScalar) (1.+rank), INSERT_VALUES);
>> VecAssemblyBegin(globVec); VecAssemblyEnd(globVec);
>> VecView(globVec, PETSC_VIEWER_STDOUT_WORLD);
>> PetscViewerFlush(PETSC_VIEWER_STDOUT_WORLD);
>>
>> // Collect all the values from a parallel PETSc vector into a vector on the
>> zeroth processor.
>>
>> Vec locVec = NULL;
>> if (rank == 0) {
>> PetscInt locSize = globSize;
>> VecCreateSeq(PETSC_COMM_SELF, locSize, &locVec); VecSet(locVec, -1.);
>> }
>> VecScatter scatCtx; VecScatterCreateToZero(globVec, &scatCtx, &locVec);
>> VecScatterBegin(scatCtx, globVec, locVec, INSERT_VALUES, SCATTER_FORWARD);
>> VecScatterEnd (scatCtx, globVec, locVec, INSERT_VALUES, SCATTER_FORWARD);
>>
>> // Modify sequential vector on the zeroth processor.
>>
>> if (rank == 0) {
>> VecView(locVec, PETSC_VIEWER_STDOUT_SELF);
>> PetscViewerFlush(PETSC_VIEWER_STDOUT_SELF);
>> VecScale(locVec, -1.);
>> VecView(locVec, PETSC_VIEWER_STDOUT_SELF);
>> PetscViewerFlush(PETSC_VIEWER_STDOUT_SELF);
>> }
>> MPI_Barrier(MPI_COMM_WORLD);
>>
>> // How do I collect all the values from a sequential vector on the zeroth
>> processor into a parallel PETSc vector ?
>>
>> VecSet(globVec, 0.);
>> VecScatterBegin(scatCtx, locVec, globVec, ADD_VALUES, SCATTER_REVERSE);
>> VecScatterEnd (scatCtx, locVec, globVec, ADD_VALUES, SCATTER_REVERSE);
>> VecView(globVec, PETSC_VIEWER_STDOUT_WORLD);
>> PetscViewerFlush(PETSC_VIEWER_STDOUT_WORLD);
>>
>> PetscFinalize();
>> }
>>
>>
>> ----- Mail original -----
>>> De: "Barry F. Smith" <bsmith at mcs.anl.gov>
>>> À: "Franck Houssen" <franck.houssen at inria.fr>
>>> Cc: "Barry F. Smith" <bsmith at mcs.anl.gov>, "Matthew Knepley"
>>> <knepley at gmail.com>, "For users of the development
>>> version of PETSc" <petsc-dev at mcs.anl.gov>
>>> Envoyé: Mercredi 3 Janvier 2018 18:20:21
>>> Objet: Re: [petsc-dev] How do I collect all the values from a sequential
>>> vector on the zeroth processor into a
>>> parallel PETSc vector ?
>>>
>>>
>>> Send the complete code as an attachment.
>>>
>>>
>>>> On Jan 3, 2018, at 11:08 AM, Franck Houssen <franck.houssen at inria.fr>
>>>> wrote:
>>>>
>>>> ----- Mail original -----
>>>>> De: "Barry F. Smith" <bsmith at mcs.anl.gov>
>>>>> À: "Franck Houssen" <franck.houssen at inria.fr>
>>>>> Cc: "Matthew Knepley" <knepley at gmail.com>, "For users of the development
>>>>> version of PETSc" <petsc-dev at mcs.anl.gov>
>>>>> Envoyé: Mercredi 3 Janvier 2018 18:01:35
>>>>> Objet: Re: [petsc-dev] How do I collect all the values from a sequential
>>>>> vector on the zeroth processor into a
>>>>> parallel PETSc vector ?
>>>>>
>>>>>
>>>>>
>>>>>> On Jan 3, 2018, at 10:59 AM, Franck Houssen <franck.houssen at inria.fr>
>>>>>> wrote:
>>>>>>
>>>>>> I need the exact opposite operation of an entry in the FAQ called "How
>>>>>> do
>>>>>> I
>>>>>> collect all the values from a parallel PETSc vector into a vector on
>>>>>> the
>>>>>> zeroth processor?"
>>>>>
>>>>> You can use VecScatterCreateToZero() then do the scatter with scatter
>>>>> reverse.
>>>>>
>>>>
>>>> That's what I tried but got an error. Did I miss something ?
>>>>
>>>>>> tail vecScatterGatherRoot.cpp
>>>>
>>>> // How do I collect all the values from a sequential vector on the
>>>> zeroth
>>>> processor into a parallel PETSc vector ?
>>>>
>>>> VecSet(globVec, 0.);
>>>> VecScatterBegin(scatCtx, locVec, globVec, ADD_VALUES, SCATTER_REVERSE);
>>>> VecScatterEnd (scatCtx, locVec, globVec, ADD_VALUES, SCATTER_REVERSE);
>>>> VecView(globVec, PETSC_VIEWER_STDOUT_WORLD);
>>>> PetscViewerFlush(PETSC_VIEWER_STDOUT_WORLD);
>>>>
>>>> PetscFinalize();
>>>> }
>>>>
>>>>>> mpirun -n 2 ./vecScatterGatherRoot.exe
>>>> Vec Object: 2 MPI processes
>>>> type: mpi
>>>> Process [0]
>>>> 1.
>>>> Process [1]
>>>> 2.
>>>> Vec Object: 1 MPI processes
>>>> type: seq
>>>> 1.
>>>> 2.
>>>> Vec Object: 1 MPI processes
>>>> type: seq
>>>> -1.
>>>> -2.
>>>> ...
>>>> [1]PETSC ERROR: [1] VecScatterBegin_MPI_ToOne line 161
>>>> /home/fghoussen/Documents/INRIA/petsc/src/vec/vec/utils/vscat.c
>>>> [1]PETSC ERROR: [1] VecScatterBegin line 1698
>>>> /home/fghoussen/Documents/INRIA/petsc/src/vec/vec/utils/vscat.c
>>>> [1]PETSC ERROR: --------------------- Error Message
>>>> --------------------------------------------------------------
>>>> [1]PETSC ERROR: Signal received
>>>> [1]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html
>>>> for
>>>> trouble shooting.
>>>>
>>>>
>>>>>>
>>>>>> Shall I use VecScatterCreateToZero "backward", or, use VecScatterCreate
>>>>>> instead ? If yes, how ? (my understanding is that VecScatterCreate can
>>>>>> take only parallel vector as input)
>>>>>
>>>>> This understanding is completely incorrect.
>>>>>
>>>>> Barry
>>>>>
>>>>>
>>>>>> Not sure how to do this.
>>>>>>
>>>>>> Its not clear what you want. Do you want a seq vector duplicated on all
>>>>>> procs?
>>>>>>
>>>>>> No. The seq vec should "live" only on the master proc. This seq master
>>>>>> vec
>>>>>> should be "converted" into a parallel vector. Not sure how to do that
>>>>>>
>>>>>> Do you want it split up? The short
>>>>>> answer is, use the appropriate scatter.
>>>>>>
>>>>>> Matt
>>>>>>
>>>>>> Franck
>>>>>>
>>>>>> In this example, the final VecView of the parallel globVec vector
>>>>>> should
>>>>>> be
>>>>>> [-1., -2.]
>>>>>>
>>>>>>>> mpirun -n 2 ./vecScatterGatherRoot.exe
>>>>>> Vec Object: 2 MPI processes
>>>>>> type: mpi
>>>>>> Process [0]
>>>>>> 1.
>>>>>> Process [1]
>>>>>> 2.
>>>>>> Vec Object: 1 MPI processes
>>>>>> type: seq
>>>>>> 1.
>>>>>> 2.
>>>>>> Vec Object: 1 MPI processes
>>>>>> type: seq
>>>>>> -1.
>>>>>> -2.
>>>>>> [1]PETSC ERROR:
>>>>>> ------------------------------------------------------------------------
>>>>>> [1]PETSC ERROR: Caught signal number 11 SEGV: Segmentation Violation,
>>>>>> probably memory access out of range
>>>>>>
>>>>>>>> more vecScatterGatherRoot.cpp
>>>>>> // How do I collect all the values from a sequential vector on the
>>>>>> zeroth
>>>>>> processor into a parallel PETSc vector ?
>>>>>> //
>>>>>> // ~> g++ -o vecScatterGatherRoot.exe vecScatterGatherRoot.cpp -lpetsc
>>>>>> -lm;
>>>>>> mpirun -n X vecScatterGatherRoot.exe
>>>>>>
>>>>>> #include "petsc.h"
>>>>>>
>>>>>> int main(int argc,char **argv) {
>>>>>> PetscInitialize(&argc, &argv, NULL, NULL);
>>>>>> int size = 0; MPI_Comm_size(MPI_COMM_WORLD, &size);
>>>>>> int rank = 0; MPI_Comm_rank(MPI_COMM_WORLD, &rank);
>>>>>>
>>>>>> PetscInt globSize = size;
>>>>>> Vec globVec; VecCreateMPI(PETSC_COMM_WORLD, 1, globSize, &globVec);
>>>>>> VecSetValue(globVec, rank, (PetscScalar) (1.+rank), INSERT_VALUES);
>>>>>> VecAssemblyBegin(globVec); VecAssemblyEnd(globVec);
>>>>>> VecView(globVec, PETSC_VIEWER_STDOUT_WORLD);
>>>>>> PetscViewerFlush(PETSC_VIEWER_STDOUT_WORLD);
>>>>>>
>>>>>> // Collect all the values from a parallel PETSc vector into a vector
>>>>>> on
>>>>>> the zeroth processor.
>>>>>>
>>>>>> Vec locVec = NULL;
>>>>>> if (rank == 0) {
>>>>>> PetscInt locSize = globSize;
>>>>>> VecCreateSeq(PETSC_COMM_SELF, locSize, &locVec); VecSet(locVec,
>>>>>> -1.);
>>>>>> }
>>>>>> VecScatter scatCtx; VecScatterCreateToZero(globVec, &scatCtx,
>>>>>> &locVec);
>>>>>> VecScatterBegin(scatCtx, globVec, locVec, INSERT_VALUES,
>>>>>> SCATTER_FORWARD);
>>>>>> VecScatterEnd (scatCtx, globVec, locVec, INSERT_VALUES,
>>>>>> SCATTER_FORWARD);
>>>>>>
>>>>>> // Modify sequential vector on the zeroth processor.
>>>>>>
>>>>>> if (rank == 0) {
>>>>>> VecView(locVec, PETSC_VIEWER_STDOUT_SELF);
>>>>>> PetscViewerFlush(PETSC_VIEWER_STDOUT_SELF);
>>>>>> VecScale(locVec, -1.);
>>>>>> VecView(locVec, PETSC_VIEWER_STDOUT_SELF);
>>>>>> PetscViewerFlush(PETSC_VIEWER_STDOUT_SELF);
>>>>>> }
>>>>>> MPI_Barrier(MPI_COMM_WORLD);
>>>>>>
>>>>>> // How do I collect all the values from a sequential vector on the
>>>>>> zeroth
>>>>>> processor into a parallel PETSc vector ?
>>>>>>
>>>>>> VecSet(globVec, 0.);
>>>>>> VecScatterBegin(scatCtx, locVec, globVec, ADD_VALUES,
>>>>>> SCATTER_REVERSE);
>>>>>> VecScatterEnd (scatCtx, locVec, globVec, ADD_VALUES,
>>>>>> SCATTER_REVERSE);
>>>>>> VecView(globVec, PETSC_VIEWER_STDOUT_WORLD);
>>>>>> PetscViewerFlush(PETSC_VIEWER_STDOUT_WORLD);
>>>>>>
>>>>>> PetscFinalize();
>>>>>> }
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> What most experimenters take for granted before they begin their
>>>>>> experiments is infinitely more interesting than any results to which
>>>>>> their
>>>>>> experiments lead.
>>>>>> -- Norbert Wiener
>>>>>>
>>>>>> https://www.cse.buffalo.edu/~knepley/
>>>>>>
>>>>>
>>>>>
>>>
>>>
>>
More information about the petsc-dev
mailing list