[petsc-users] Using PETSc GPU backend

[Barry Smith]

  Note that options like -mat_type aijcusparse  -vec_type cuda only work if the program is set up to allow runtime swapping of matrix and vector types. If you have a call to MatCreateMPIAIJ() or other specific types then then these options do nothing but because Mark had you use -options_left the program will tell you at the end that it did not use the option so you will know.

> On Jun 30, 2023, at 9:30 AM, Mark Adams <mfadams at lbl.gov> wrote:
> 
> PetscCall(PetscInitialize(&argc, &argv, NULL, help)); gives us the args and you run:
> 
> a.out -mat_type aijcusparse -vec_type cuda -log_view -options_left
> 
> Mark
> 
> On Fri, Jun 30, 2023 at 6:16 AM Matthew Knepley <knepley at gmail.com <mailto:knepley at gmail.com>> wrote:
>> On Fri, Jun 30, 2023 at 1:13 AM Ng, Cho-Kuen via petsc-users <petsc-users at mcs.anl.gov <mailto:petsc-users at mcs.anl.gov>> wrote:
>>> Mark,
>>> 
>>> The application code reads in parameters from an input file, where we can put the PETSc runtime options. Then we pass the options to PetscInitialize(...). Does that sounds right?
>> 
>> PETSc will read command line argument automatically in PetscInitialize() unless you shut it off.
>> 
>>   Thanks,
>> 
>>     Matt
>>  
>>> Cho
>>> From: Ng, Cho-Kuen <cho at slac.stanford.edu <mailto:cho at slac.stanford.edu>>
>>> Sent: Thursday, June 29, 2023 8:32 PM
>>> To: Mark Adams <mfadams at lbl.gov <mailto:mfadams at lbl.gov>>
>>> Cc: petsc-users at mcs.anl.gov <mailto:petsc-users at mcs.anl.gov> <petsc-users at mcs.anl.gov <mailto:petsc-users at mcs.anl.gov>>
>>> Subject: Re: [petsc-users] Using PETSc GPU backend
>>>  
>>> Mark,
>>> 
>>> Thanks for the information. How do I put the runtime options for the executable, say, a.out, which does not have the provision to append arguments? Do I need to change the C++ main to read in the options?
>>> 
>>> Cho
>>> From: Mark Adams <mfadams at lbl.gov <mailto:mfadams at lbl.gov>>
>>> Sent: Thursday, June 29, 2023 5:55 PM
>>> To: Ng, Cho-Kuen <cho at slac.stanford.edu <mailto:cho at slac.stanford.edu>>
>>> Cc: petsc-users at mcs.anl.gov <mailto:petsc-users at mcs.anl.gov> <petsc-users at mcs.anl.gov <mailto:petsc-users at mcs.anl.gov>>
>>> Subject: Re: [petsc-users] Using PETSc GPU backend
>>>  
>>> Run with options: -mat_type aijcusparse -vec_type cuda -log_view -options_left
>>> 
>>> The last column of the performance data (from -log_view) will be the percent flops on the GPU. Check that that is > 0.
>>> 
>>> The end of the output will list the options that were used and options that were _not_ used (if any). Check that there are no options left.
>>> 
>>> Mark
>>> 
>>> On Thu, Jun 29, 2023 at 7:50 PM Ng, Cho-Kuen via petsc-users <petsc-users at mcs.anl.gov <mailto:petsc-users at mcs.anl.gov>> wrote:
>>> I installed PETSc on Perlmutter using "spack install petsc+cuda+zoltan" and used it by "spack load petsc/fwge6pf". Then I compiled the application code (purely CPU code) linking to the petsc package, hoping that I can get performance improvement using the petsc GPU backend. However, the timing was the same using the same number of MPI tasks with and without GPU accelerators. Have I missed something in the process, for example, setting up PETSc options at runtime to use the GPU backend?
>>> 
>>> Thanks,
>>> Cho
>> 
>> 
>> -- 
>> What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.
>> -- Norbert Wiener
>> 
>> https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20230630/b67dd36f/attachment-0001.html>