[petsc-users] [External] Re: MatVec on GPUs
Chang Liu
cliu at pppl.gov
Wed Oct 20 11:55:47 CDT 2021
Hi Junchao,
Thank you for the suggestion. I did some more tests and found that
MatConvert does not always work. In one of my tests, I did MatConvert to
convert the matrix to aijcusparse, then did a preonly ksp solver and it
works well. But then I tried a fgmres solver and it gave an error. It
only happen when the matrix is mpiaijcusparse and for seqaijcusparse it
works.
So I tried to create a new aijcusparse matrix and copy the data line by
line, then both solvers works. So I guess there are some tricky things
with MatConvert.
Chang
On 10/18/21 9:23 PM, Junchao Zhang wrote:
> MatSetOptionsPrefix(A,"mymat")
> VecSetOptionsPrefix(v,"myvec")
>
> --Junchao Zhang
>
>
> On Mon, Oct 18, 2021 at 8:04 PM Chang Liu <cliu at pppl.gov
> <mailto:cliu at pppl.gov>> wrote:
>
> Hi Junchao,
>
> Thank you for your answer. I tried MatConvert and it works. I didn't
> make it before because I forgot to convert a vector from mpi to mpicuda
> previously.
>
> For vector, there is no VecConvert to use, so I have to do
> VecDuplicate,
> VecSetType and VecCopy. Is there an easier option?
>
> As Matt suggested, you could single out the matrix and vector with
> options prefix and set their type on command line
>
> MatSetOptionsPrefix(A,"mymat");
> VecSetOptionsPrefix(v,"myvec");
>
> Then, -mymat_mat_type aijcusparse -myvec_vec_type cuda
> A simpler code is to have the vector type automatically set by
> MatCreateVecs(A,&v,NULL)
>
>
> Chang
>
> On 10/18/21 5:23 PM, Junchao Zhang wrote:
> >
> >
> > On Mon, Oct 18, 2021 at 3:42 PM Chang Liu via petsc-users
> > <petsc-users at mcs.anl.gov <mailto:petsc-users at mcs.anl.gov>
> <mailto:petsc-users at mcs.anl.gov <mailto:petsc-users at mcs.anl.gov>>>
> wrote:
> >
> > Hi Matt,
> >
> > I have a related question. In my code I have many matrices
> and I only
> > want to have one living on GPU, the others still staying on
> CPU mem.
> >
> > I wonder if there is an easier way to copy a mpiaij matrix to
> > mpiaijcusparse (in other words, copy data to GPUs). I can
> think of
> > creating a new mpiaijcusparse matrix, and copying the data
> line by
> > line.
> > But I wonder if there is a better option.
> >
> > I have tried MatCopy and MatConvert but neither work.
> >
> > Did you use MatConvert(mat,matype,MAT_INPLACE_MATRIX,&mat)?
> >
> >
> > Chang
> >
> > On 10/17/21 7:50 PM, Matthew Knepley wrote:
> > > On Sun, Oct 17, 2021 at 7:12 PM Swarnava Ghosh
> > <swarnava89 at gmail.com <mailto:swarnava89 at gmail.com>
> <mailto:swarnava89 at gmail.com <mailto:swarnava89 at gmail.com>>
> > > <mailto:swarnava89 at gmail.com <mailto:swarnava89 at gmail.com>
> <mailto:swarnava89 at gmail.com <mailto:swarnava89 at gmail.com>>>> wrote:
> > >
> > > Do I need convert the MATSEQBAIJ to a cuda matrix in code?
> > >
> > >
> > > You would need a call to MatSetFromOptions() to take that type
> > from the
> > > command line, and not have
> > > the type hard-coded in your application. It is generally a bad
> > idea to
> > > hard code the implementation type.
> > >
> > > If I do it from command line, then are the other
> MatVec calls are
> > > ported onto CUDA? I have many MatVec calls in my code,
> but I
> > > specifically want to port just one call.
> > >
> > >
> > > You can give that one matrix an options prefix to isolate it.
> > >
> > > Thanks,
> > >
> > > Matt
> > >
> > > Sincerely,
> > > Swarnava
> > >
> > > On Sun, Oct 17, 2021 at 7:07 PM Junchao Zhang
> > > <junchao.zhang at gmail.com
> <mailto:junchao.zhang at gmail.com> <mailto:junchao.zhang at gmail.com
> <mailto:junchao.zhang at gmail.com>>
> > <mailto:junchao.zhang at gmail.com
> <mailto:junchao.zhang at gmail.com> <mailto:junchao.zhang at gmail.com
> <mailto:junchao.zhang at gmail.com>>>>
> > wrote:
> > >
> > > You can do that with command line options -mat_type
> > aijcusparse
> > > -vec_type cuda
> > >
> > > On Sun, Oct 17, 2021, 5:32 PM Swarnava Ghosh
> > > <swarnava89 at gmail.com
> <mailto:swarnava89 at gmail.com> <mailto:swarnava89 at gmail.com
> <mailto:swarnava89 at gmail.com>>
> > <mailto:swarnava89 at gmail.com <mailto:swarnava89 at gmail.com>
> <mailto:swarnava89 at gmail.com <mailto:swarnava89 at gmail.com>>>> wrote:
> > >
> > > Dear Petsc team,
> > >
> > > I had a query regarding using CUDA to
> accelerate a matrix
> > > vector product.
> > > I have a sequential sparse matrix
> (MATSEQBAIJ type).
> > I want
> > > to port a MatVec call onto GPUs. Is there any
> > code/example I
> > > can look at?
> > >
> > > Sincerely,
> > > SG
> > >
> > >
> > >
> > > --
> > > What most experimenters take for granted before they begin
> their
> > > experiments is infinitely more interesting than any
> results to which
> > > their experiments lead.
> > > -- Norbert Wiener
> > >
> > > https://www.cse.buffalo.edu/~knepley/
> <https://www.cse.buffalo.edu/~knepley/>
> > <https://www.cse.buffalo.edu/~knepley/
> <https://www.cse.buffalo.edu/~knepley/>>
> > <http://www.cse.buffalo.edu/~knepley/
> <http://www.cse.buffalo.edu/~knepley/>
> > <http://www.cse.buffalo.edu/~knepley/
> <http://www.cse.buffalo.edu/~knepley/>>>
> >
> > --
> > Chang Liu
> > Staff Research Physicist
> > +1 609 243 3438
> > cliu at pppl.gov <mailto:cliu at pppl.gov> <mailto:cliu at pppl.gov
> <mailto:cliu at pppl.gov>>
> > Princeton Plasma Physics Laboratory
> > 100 Stellarator Rd, Princeton NJ 08540, USA
> >
>
> --
> Chang Liu
> Staff Research Physicist
> +1 609 243 3438
> cliu at pppl.gov <mailto:cliu at pppl.gov>
> Princeton Plasma Physics Laboratory
> 100 Stellarator Rd, Princeton NJ 08540, USA
>
--
Chang Liu
Staff Research Physicist
+1 609 243 3438
cliu at pppl.gov
Princeton Plasma Physics Laboratory
100 Stellarator Rd, Princeton NJ 08540, USA
More information about the petsc-users
mailing list