[petsc-users] general VecScatter from MPI to MPI
Ethan Coon
ecoon at lanl.gov
Thu Feb 17 11:31:06 CST 2011
>
> So effectively I need two scatters -- a scatter from the
> global Vec to
> the sequential local Vecs, then a scatter (which requires no
> communication) to inject the sequential Vecs into the new
> global Vec?
>
>
> No, just wrap up the pieces of your global Vec as local Vecs and
> scatter
> straight into that storage using VecCreateSeqWithArray().
>
Ah ha! Thanks,
Ethan
>
> Matt
>
> Why? Am I missing something that makes the MPI to MPI scatter
> ill-posed
> as long as the global sizes (but not local sizes) are equal?
>
> This is mostly curiosity on my part... I think I have to do
> two scatters
> anyway since I'm working with multiple comms -- scatter from
> an MPI Vec
> on one sub-comm into local, sequential Vecs, then scatter
> those
> sequential Vecs into an MPI Vec on PETSC_COMM_WORLD. That's
> the correct
> model for injecting an MPI Vec on one comm into an MPI Vec on
> PETSC_COMM_WORLD, correct?
>
> Ethan
>
>
> >
> > Matt
> >
> > See below snippet (and its errors):
> >
> > Ethan
> >
> >
> >
> > Vec vA
> > Vec vB
> > VecScatter scatter_AB
> >
> > PetscInt np
> > PetscInt rank
> > PetscErrorCode ierr
> >
> > if (rank.eq.0) np = 3
> > if (rank.eq.1) np = 1
> >
> > call VecCreateMPI(PETSC_COMM_WORLD, 2,
> PETSC_DETERMINE, vA,
> > ierr)
> > call VecCreateMPI(PETSC_COMM_WORLD, np,
> PETSC_DETERMINE, vB,
> > ierr)
> >
> > call VecScatterCreate(vA, PETSC_NULL_OBJECT, vB,
> > PETSC_NULL_OBJECT,
> > scatter_AB, ierr)
> >
> > ...
> >
> > $> mpiexec -n 2 ./test
> >
> > [0]PETSC ERROR: --------------------- Error Message
> > ------------------------------------
> > [0]PETSC ERROR: Nonconforming object sizes!
> > [0]PETSC ERROR: Local scatter sizes don't match!
> > [0]PETSC ERROR:
> >
> ------------------------------------------------------------------------
> > [1]PETSC ERROR: --------------------- Error Message
> > ------------------------------------
> > [1]PETSC ERROR: Nonconforming object sizes!
> > [1]PETSC ERROR: Local scatter sizes don't match!
> > [0]PETSC ERROR: Petsc Development HG revision:
> > 5dbe1264252fb9cb5d8e033d620d18f7b0e9111f HG Date:
> Fri Feb 11
> > 15:44:04
> > 2011 -0600
> > [0]PETSC ERROR: See docs/changes/index.html for
> recent
> > updates.
> > [0]PETSC ERROR: See docs/faq.html for hints about
> trouble
> > shooting.
> > [0]PETSC ERROR: See docs/index.html for manual
> pages.
> > [0]PETSC ERROR:
> >
> ------------------------------------------------------------------------
> > [0]PETSC ERROR: ./test on a linux-gnu named tama1 by
> ecoon Thu
> > Feb 17
> > 08:14:57 2011
> > [0]PETSC ERROR: Libraries linked
> >
> from /packages/petsc/petsc-dev3.0-mpich2-local-gcc-4.3.3/debug-shared/lib
> > [0]PETSC ERROR: Configure run at Fri Feb 11 16:15:14
> 2011
> > [0]PETSC ERROR: Configure options --with-debugging=1
> >
> --prefix=/packages/petsc/petsc-dev3.0-mpich2-local-gcc-4.3.3/debug-shared --download-mpich=1 --download-ml=1 --download-umfpack=1 --with-blas-lapack-dir=/usr/lib --download-parmetis=yes PETSC_ARCH=linux-gnu-c-debug-shared --with-clanguage=c --download-hypre=1 --with-shared-libraries=1 --download-hdf5=1
> > [0]PETSC ERROR:
> >
> ------------------------------------------------------------------------
> > [0]PETSC ERROR: VecScatterCreate() line 1432 in
> > src/vec/vec/utils/vscat.c
> > application called MPI_Abort(MPI_COMM_WORLD, 60) -
> process 0
> > [cli_0]: aborting job:
> > application called MPI_Abort(MPI_COMM_WORLD, 60) -
> process 0
> > [1]PETSC ERROR: APPLICATION TERMINATED WITH THE EXIT
> STRING:
> > Hangup
> > (signal 1)
> >
> >
> >
> >
> > --
> > ------------------------------------
> > Ethan Coon
> > Post-Doctoral Researcher
> > Applied Mathematics - T-5
> > Los Alamos National Laboratory
> > 505-665-8289
> >
> > http://www.ldeo.columbia.edu/~ecoon/
> > ------------------------------------
> >
> >
> >
> >
> > --
> > What most experimenters take for granted before they begin
> their
> > experiments is infinitely more interesting than any results
> to which
> > their experiments lead.
> > -- Norbert Wiener
>
>
> --
>
> ------------------------------------
> Ethan Coon
> Post-Doctoral Researcher
> Applied Mathematics - T-5
> Los Alamos National Laboratory
> 505-665-8289
>
> http://www.ldeo.columbia.edu/~ecoon/
> ------------------------------------
>
>
>
>
>
> --
> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which
> their experiments lead.
> -- Norbert Wiener
--
------------------------------------
Ethan Coon
Post-Doctoral Researcher
Applied Mathematics - T-5
Los Alamos National Laboratory
505-665-8289
http://www.ldeo.columbia.edu/~ecoon/
------------------------------------
More information about the petsc-users
mailing list