[petsc-users] MatGetSubMatrices or VecCreateGhost

[Bikash Kanungo]

Hi Barry,

I'm sorry, I should have provided more details.

Matrix A defined in my problem is of size NxN and I have N vectors v_i,
each of length N, which are to be multiplied by A. N is fairly small
10000-50000. Ideally speaking, it doesn't require parallel storage. But I
will soon exhaust memory by storing both A and v_i's sequentially. So I
planned to implement it through a sub-communiator as this Mat-Vec(s)
operation(s) is part of a bigger problem which requires a greater degree of
parallelization.

My ghost points are physical grid points, FEM nodal points to be exact. So
I have a list of nodes whose data is required on a given processor to
perform evaluation of any physical quantity. It is through this list that I
decide the ghost points (I check if the global ids of these nodes are
within the ownership range for the PETSc partioning or not).

Now depending upon the number of processors I put into my sub-communicator,
the percentage of non-local data I'll require will range from 5-10%.

I hope I'm able to put forth my problem properly and adequately.

Thanks,
Bikash
On Oct 16, 2015 11:49 AM, "Barry Smith" <bsmith at mcs.anl.gov> wrote:

>
> > On Oct 16, 2015, at 7:52 AM, Bikash Kanungo <bikash at umich.edu> wrote:
> >
> > Hi,
> >
> > I need to multiply a matrix A with several vectors v_i's and use the
> resultant vectors in my calculation which involves access to non-local
> nodes.
>
>    What percentage of the "non-local" results do you need in "your
> calculation", a few percent, all of them? How do you know which "non-local"
> results you will need for "your calculation"? Are they ghost points on a
> grid? Something else?
>
>     What is "several vectors"? 5? 100? 1000?
>
>   Barry
>
> > I can think of two possible ways to do it:
> >
> > 1. Store the vectors v_i's into an MPIDENSE matrix and perform
> MatMatMult with A to get a resultant dense matrix B. I can now use
> MatGetSubMatrices on B to access the non-local part required in my
> subsequent calculations.
> >
> > 2. Use MatMult to evaluate A*v_i for each of the vectors and provide
> ghost padding through VecCreateGhost for the resultant vector.
> >
> > Which of the two is a more efficient way to perform the task? Also, if
> there is a better way possible to achieve the same, then kindly let me know.
> >
> > Thanks,
> > Bikash
> >
> > --
> > Bikash S. Kanungo
> > PhD Student
> > Computational Materials Physics Group
> > Mechanical Engineering
> > University of Michigan
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20151016/6e1f0494/attachment-0001.html>