Newbie question about synchronization

[Satish Balay]

On Thu, 29 Mar 2007, Knut Erik Teigen wrote:

> On Thu, 2007-03-29 at 03:42 -0400, Diego Nehab wrote:

> > So far, using only one process, everything is simple and
> > works (it took me longer to compile and test MPI and Petsc
> > then to write code that solves my problem :)).
> > 
> > When I move to multiple processes, the solver still works, but I
> > couldn't figure out how to select one of the processes to
> > consolidate the solution vector and use it to save a file on disk.
> > I always get an error of the form
> > 
> >     [0]PETSC ERROR: Argument out of range!
> >     [0]PETSC ERROR: Can only get local values, trying xxx!
> > 
> > I assume I must bring the solution vector back from the other
> > processes, right?
> > 
> > 2) If so, how do I do this?
> If you only want to save the results to disk, you can use the VecView
> function. Just create a viewer, like e.g.
> PetscViewerAsciiOpen(PETSC_COMM_WORLD,"filename",&viewer)
> VecView(solution,viewer)
> You can also output in binary format using BinaryOpen instead. Check the
> chapter on Viewers in the manual.
> If you need to gather the results on one processor for further
> computations, I use standard MPI calls, like this(in Fortran):
> 
>  call VecGetArray(sol,sol_ptr,sol_i,ierr)
> 
>  ! gather solution on process 0
>  if (rank==0) then 
>    do i=low,high !copy local solution to global solution
>      global_sol(i)=sol_ptr(sol_i+i)
>    enddo
>    do p=1,size-1 !recieve local solution from other processes
>      call
> MPI_Recv(local_sol,loc_n,MPI_REAL,p,1,PETSC_COMM_WORLD,istat,mpierr)
>      do i=1,loc_n !copy local part to correct position in global
>        global_sol(i+high)=local_sol(i)
>        high=high+1
>      enddo
>    enddo
>  else
>   do j=1,loc_n
>     local_sol(j)=sol_ptr(sol_i+j)
>   enddo  
>   call MPI_Send(local_sol,loc_n,MPI_REAL,0,1,PETSC_COMM_WORLD,mpierr)
>  endif
> 
>  call VecRestoreArray(sol,sol_ptr,sol_i,ierr)
> 
>  !copy global solution vector back to grid array
>  if (rank==0) then
>    do j=1,jmax
>      do i=1,imax
>        T(i,j)=global_sol((j-1)*imax+i)
>      end do
>    end do
>  endif
> 
> This is probably not the recommended way of doing things. I'm quite new
> at using PETSc myself, so if anyone has a better solution, please
> enlighten me! I should use PETSc data structures for everything, but I'm
> trying to integrate PETSc into already existing code, so it's not that
> easy to do.

VecView() is the correct thing to do for dumping the vec into a file.
However - if you need all the values of the vec on 1 proc - then the
use VecScatterCreateToZero()

http://www-unix.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/Vec/VecScatterCreateToZero.html

Satish