[petsc-users] Equivalent of all_reduce for sparse matrices

[Matthew Knepley]

On Fri, May 9, 2014 at 3:15 AM, marco restelli <mrestelli at gmail.com> wrote:

> 2014-05-09 5:47 GMT+0200, Jed Brown <jed at jedbrown.org>:
> > marco restelli <mrestelli at gmail.com> writes:
> >> Matt, thanks but this I don't understand. What I want is getting three
> >> arrays (i,j,coeff) with all the nonzero local coefficients, so that I
> >> can send them around with MPI.
> >>
> >> MatGetSubmatrices would give me some PETSc objects, which I can not
> >> pass to MPI, right?
> >
> > I'm not sure you want this, but you can use MatGetRowIJ and similar to
> > access the representation you're asking for if you are dead set on
> > depending on a specific data format rather than using generic
> > interfaces.
> >
>
> Jed, thank you. This is probably not the PETSc solution, but still it
> might a solution!
>
> I have found this example for MatGetRowIJ:
>

I really do not think you want to do this. It is complex, fragile and I
believe the performance
improvement to be non-existent. You can get the effect you want JUST by
using one function.
For example, suppose you want 2 procs to get rows [0,5] and two procs to
get rows [1,3], then

procs A.B

  MatGetSubmatrices(A, 2, [0,5], 2, [0,5], ..., &submat)

procs C, D

  MatGetSubmatrices(A, 2, [1,3], 2, [1,3], ..., &submat)

and its done. No MPI, no extraction which depends on the Mat data structure.

   Matt


>
> http://www.stce.rwth-aachen.de/trac/petsc/browser/src/mat/examples/tests/ex79f.F?rev=a52934f9a5da430fdd891fa538a66c376435ec4c
>
> My understanding is that I need to:
>
> 1) get the sequential part of the matrix, i.e. those rows stored on
> this processor
>       call MatMPIAIJGetSeqAIJ(A,Ad,Ao,icol,iicol,ierr)
>
> 2) get the indexes of these rows
>       call MatGetOwnershipRange(A,rstart,rend,ierr)
>
> 3) get the indexes i,j of the local portion of the matrix (compressed
> form)
>       call MatGetRowIJ(Ad,one,zero,zero,n,ia,iia,ja,jja,done,ierr)
>
> 4) get the corresponding elements
>       call MatGetArray(Ad,aa,aaa,ierr)
>
> 5) WARNING: the row indexes obtained with MatGetRowIJ are local to
> this processor, so they must be corrected with rstart to obtain the
> corresponding global indexes
>
> 6) clean-up
>
>
> Does this make sense?
>
>
> Marco
>



-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20140509/4cae87d5/attachment.html>