[petsc-dev] How do I collect all the values from a sequential vector on the zeroth processor into a parallel PETSc vector ?

[Smith, Barry F.]

  Send the complete code as an attachment.


> On Jan 3, 2018, at 11:08 AM, Franck Houssen <franck.houssen at inria.fr> wrote:
> 
> ----- Mail original -----
>> De: "Barry F. Smith" <bsmith at mcs.anl.gov>
>> À: "Franck Houssen" <franck.houssen at inria.fr>
>> Cc: "Matthew Knepley" <knepley at gmail.com>, "For users of the development version of PETSc" <petsc-dev at mcs.anl.gov>
>> Envoyé: Mercredi 3 Janvier 2018 18:01:35
>> Objet: Re: [petsc-dev] How do I collect all the values from a sequential vector on the zeroth processor into a
>> parallel PETSc vector ?
>> 
>> 
>> 
>>> On Jan 3, 2018, at 10:59 AM, Franck Houssen <franck.houssen at inria.fr>
>>> wrote:
>>> 
>>> I need the exact opposite operation of an entry in the FAQ called "How do I
>>> collect all the values from a parallel PETSc vector into a vector on the
>>> zeroth processor?"
>> 
>>  You can use VecScatterCreateToZero() then do the scatter with scatter
>>  reverse.
>> 
> 
> That's what I tried but got an error. Did I miss something ?
> 
>>> tail vecScatterGatherRoot.cpp 
> 
>  // How do I collect all the values from a sequential vector on the zeroth processor into a parallel PETSc vector ?
> 
>  VecSet(globVec, 0.);
>  VecScatterBegin(scatCtx, locVec, globVec, ADD_VALUES, SCATTER_REVERSE);
>  VecScatterEnd  (scatCtx, locVec, globVec, ADD_VALUES, SCATTER_REVERSE);
>  VecView(globVec, PETSC_VIEWER_STDOUT_WORLD); PetscViewerFlush(PETSC_VIEWER_STDOUT_WORLD);
> 
>  PetscFinalize();
> }
> 
>>> mpirun -n 2 ./vecScatterGatherRoot.exe 
> Vec Object: 2 MPI processes
>  type: mpi
> Process [0]
> 1.
> Process [1]
> 2.
> Vec Object: 1 MPI processes
>  type: seq
> 1.
> 2.
> Vec Object: 1 MPI processes
>  type: seq
> -1.
> -2.
> ...
> [1]PETSC ERROR: [1] VecScatterBegin_MPI_ToOne line 161 /home/fghoussen/Documents/INRIA/petsc/src/vec/vec/utils/vscat.c
> [1]PETSC ERROR: [1] VecScatterBegin line 1698 /home/fghoussen/Documents/INRIA/petsc/src/vec/vec/utils/vscat.c
> [1]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------
> [1]PETSC ERROR: Signal received
> [1]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting.
> 
> 
>>> 
>>> Shall I use VecScatterCreateToZero "backward", or, use VecScatterCreate
>>> instead ? If yes, how ? (my understanding is that VecScatterCreate can
>>> take only parallel vector as input)
>> 
>>   This understanding is completely incorrect.
>> 
>>  Barry
>> 
>> 
>>> Not sure how to do this.
>>> 
>>> Its not clear what you want. Do you want a seq vector duplicated on all
>>> procs?
>>> 
>>> No. The seq vec should "live" only on the master proc. This seq master vec
>>> should be "converted" into a parallel vector. Not sure how to do that
>>> 
>>> Do you want it split up? The short
>>> answer is, use the appropriate scatter.
>>> 
>>>   Matt
>>> 
>>> Franck
>>> 
>>> In this example, the final VecView of the parallel globVec vector should be
>>> [-1., -2.]
>>> 
>>>>> mpirun -n 2 ./vecScatterGatherRoot.exe
>>> Vec Object: 2 MPI processes
>>>  type: mpi
>>> Process [0]
>>> 1.
>>> Process [1]
>>> 2.
>>> Vec Object: 1 MPI processes
>>>  type: seq
>>> 1.
>>> 2.
>>> Vec Object: 1 MPI processes
>>>  type: seq
>>> -1.
>>> -2.
>>> [1]PETSC ERROR:
>>> ------------------------------------------------------------------------
>>> [1]PETSC ERROR: Caught signal number 11 SEGV: Segmentation Violation,
>>> probably memory access out of range
>>> 
>>>>> more vecScatterGatherRoot.cpp
>>> // How do I collect all the values from a sequential vector on the zeroth
>>> processor into a parallel PETSc vector ?
>>> //
>>> // ~> g++ -o vecScatterGatherRoot.exe vecScatterGatherRoot.cpp -lpetsc -lm;
>>> mpirun -n X vecScatterGatherRoot.exe
>>> 
>>> #include "petsc.h"
>>> 
>>> int main(int argc,char **argv) {
>>>  PetscInitialize(&argc, &argv, NULL, NULL);
>>>  int size = 0; MPI_Comm_size(MPI_COMM_WORLD, &size);
>>>  int rank = 0; MPI_Comm_rank(MPI_COMM_WORLD, &rank);
>>> 
>>>  PetscInt globSize = size;
>>>  Vec globVec; VecCreateMPI(PETSC_COMM_WORLD, 1, globSize, &globVec);
>>>  VecSetValue(globVec, rank, (PetscScalar) (1.+rank), INSERT_VALUES);
>>>  VecAssemblyBegin(globVec); VecAssemblyEnd(globVec);
>>>  VecView(globVec, PETSC_VIEWER_STDOUT_WORLD);
>>>  PetscViewerFlush(PETSC_VIEWER_STDOUT_WORLD);
>>> 
>>>  // Collect all the values from a parallel PETSc vector into a vector on
>>>  the zeroth processor.
>>> 
>>>  Vec locVec = NULL;
>>>  if (rank == 0) {
>>>    PetscInt locSize = globSize;
>>>    VecCreateSeq(PETSC_COMM_SELF, locSize, &locVec); VecSet(locVec, -1.);
>>>  }
>>>  VecScatter scatCtx; VecScatterCreateToZero(globVec, &scatCtx, &locVec);
>>>  VecScatterBegin(scatCtx, globVec, locVec, INSERT_VALUES,
>>>  SCATTER_FORWARD);
>>>  VecScatterEnd  (scatCtx, globVec, locVec, INSERT_VALUES,
>>>  SCATTER_FORWARD);
>>> 
>>>  // Modify sequential vector on the zeroth processor.
>>> 
>>>  if (rank == 0) {
>>>    VecView(locVec, PETSC_VIEWER_STDOUT_SELF);
>>>    PetscViewerFlush(PETSC_VIEWER_STDOUT_SELF);
>>>    VecScale(locVec, -1.);
>>>    VecView(locVec, PETSC_VIEWER_STDOUT_SELF);
>>>    PetscViewerFlush(PETSC_VIEWER_STDOUT_SELF);
>>>  }
>>>  MPI_Barrier(MPI_COMM_WORLD);
>>> 
>>>  // How do I collect all the values from a sequential vector on the zeroth
>>>  processor into a parallel PETSc vector ?
>>> 
>>>  VecSet(globVec, 0.);
>>>  VecScatterBegin(scatCtx, locVec, globVec, ADD_VALUES, SCATTER_REVERSE);
>>>  VecScatterEnd  (scatCtx, locVec, globVec, ADD_VALUES, SCATTER_REVERSE);
>>>  VecView(globVec, PETSC_VIEWER_STDOUT_WORLD);
>>>  PetscViewerFlush(PETSC_VIEWER_STDOUT_WORLD);
>>> 
>>>  PetscFinalize();
>>> }
>>> 
>>> 
>>> 
>>> 
>>> --
>>> What most experimenters take for granted before they begin their
>>> experiments is infinitely more interesting than any results to which their
>>> experiments lead.
>>> -- Norbert Wiener
>>> 
>>> https://www.cse.buffalo.edu/~knepley/
>>> 
>> 
>>