[petsc-users] matsetvalueslocal for aijcusparse matrix

[Xiangdong]

My Matrix setup workflow is like this:

1 MatCreate
2 MatSetSizes
3 MatSetType(A,MATAIJ)
4 MatMPIAIJSetPreallocation
5 MatSeqAIJSetPreallocation
6 MatSetLocalToGlobalMapping
7 MatSetOptionPrefix(A,"test_")
8 MatSetFromOptions
9 MatSetUp

10 loop all the nonzero entries by calling MatSetValuesLocal(A,1,&i,1,&j,
&val, ADD_VALUES);

11 MatAssemblyBegin
12 MatAssemblyEnd

For the AIJ format, it works fine. -info gives "Number of mallocs during
MatSetValues() is 0" and "Stash has 0 entries, uses 0 mallocs."

If I run the same code with -test_mat_type aijcusparse, it takes forever to
finish step 10. Does this step really involve moving data from host to
devices? Do I need to have more changes to use aijcusparse other than just
changing mat_type from command line?

Thank you.

Best,
Xiangdong


On Tue, Oct 22, 2019 at 1:53 AM Smith, Barry F. <bsmith at mcs.anl.gov> wrote:

>
>    The aijcusparse actually uses the same data structures and code for
> setting values as does aij. So it is not related directly to that format.
>
>   Barry
>
>
> > On Oct 21, 2019, at 6:26 PM, Xiangdong via petsc-users <
> petsc-users at mcs.anl.gov> wrote:
> >
> > Hello everyone,
> >
> > When I use matsetvalueslocal to form the matrix in aijcusparse format, I
> found that it takes forever to finish the loops of matsetvalueslocal. I am
> setting one entry a time, and there are about about 28 Million nonzeros.
> (It is fast if the matrix is aij, instead of aijcusparse).
> >
> > However, if I have the matrix ready in binary format and use matload to
> get it into aijcusparse format, it is fast.
> >
> > Is it the issue of matsetvalueslocal or my wrong way of using
> matsetvalueslocal? any suggestions to speed up the process?
> >
> > Thank you.
> >
> > Xiangdong
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20191022/8d4b71ff/attachment.html>