[petsc-users] Error while running with -pc_type gamg on NVIDIA GPU

[Maruthi NH]

Hi Barry Smith,

Thanks for the quick response.

Here is the code I used to test PETSc on GPU.
This is the command I used to run
mpiexec.hydra -n 1 ./heat_diff_cu -Nx 10000000 -ksp_type gmres -mat_type
aijcusparse -vec_type cuda -use_gpu_aware_mpi 0 -pc_type gamg
-ksp_converged_reason

Regards,
Maruthi



On Sun, Sep 10, 2023 at 11:37 PM Barry Smith <bsmith at petsc.dev> wrote:

>
>
> On Sep 10, 2023, at 5:54 AM, Maruthi NH <maruthinh at gmail.com> wrote:
>
> Hi all,
>
> I am trying to accelerate the linear solver with PETSc GPU backend. For
> testing I have a simple 1D heat diffusion solver, here are some
> observations.
> 1. If I use -pc_type gamg it throws the following error
>  ** On entry to cusparseCreateCsr() parameter number 5 (csrRowOffsets) had
> an illegal value: NULL pointer
>
> [0]PETSC ERROR: --------------------- Error Message
> --------------------------------------------------------------
> [0]PETSC ERROR: GPU error
> [0]PETSC ERROR: cuSPARSE errorcode 3 (CUSPARSE_STATUS_INVALID_VALUE) :
> invalid value
> [0]PETSC ERROR: See https://petsc.org/release/faq/ for trouble shooting.
> [0]PETSC ERROR: Petsc Development GIT revision: v3.19.4-959-g92f1e92e88
>  GIT Date: 2023-08-13 19:43:04 +0000
>
>    Can you share the code that triggers this?
>
> 2. Default pc ilu takes about 1.2 seconds on a single CPU and it takes
> about 105.9 seconds on a GPU. Similar observations with pc_type asm
> I have NVIDIA RTX A2000 8GB Laptop GPU
>
>
>   This is expected. The triangular solves sequentialize on the GPU so
> naturally are extremely slow since they cannot take advantage of the
> massive parallelism of the GPU.
>
>
> 3. What I could be missing? Also, are there any general guidelines for
> better GPU performance using PETSc?
>
> Regards,
> Maruthi
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20230911/912edc25/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: heat_diff_cu.cu
Type: application/octet-stream
Size: 6772 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20230911/912edc25/attachment.obj>