[petsc-users] VecScatter Error w/Multiple Processors

[Barry Smith]

  There are several problems with this code.

On Jul 1, 2014, at 12:04 PM, Arthur Kurlej <akurlej at gmail.com> wrote:

> Hello,
> 
> I am trying to use VecScatter to grab values from nonlocal processors, and I've run into the following error when I try to run for 8+ processors. The error is as follows:
> 
> [0]PETSC ERROR: VecScatterCreate() line 1161 in src/vec/vec/utils/vscat.c
> 
> And the code that causes it is
> 
> Mat A
> MatGetOwnershipRange(A,&Istart,&Iend); 
> 
> for(rowcounter = Istart; rowcounter < Iend; rowcounter++){
> 
>  ISCreateGeneral(PETSC_COMM_WORLD,lSize,vecIndices,PETSC_COPY_VALUES,&isGlobal); 

   ISCreateGeneral(PETSC_COMM_WORLD) is a collective operation, all processes in PETSC_COMM_WORLD must call this each time. But you have this inside a loop for the number of local rows; not all processes will have the same number of local rows

>  VecScatterCreate(gVec,isGlobal,lVec, NULL,&ctx);
>  VecScatterBegin(ctx,gVec,lVec,INSERT_VALUES,SCATTER_FORWARD); 
>  VecScatterEnd(ctx,gVec,lVec,INSERT_VALUES,SCATTER_FORWARD); 
> 
>  //fill matrix
> }

   I am not sure what you want to accomplish but there is no way that doing a VecScatter for every row in a matrix is going to be scalable.  are you trying to move values to other processes that you then plan to set into the matrix locally? If so just use MatSetValues() where each process sets the values it currently has.

> 
> Any thoughts?
> 
>