[petsc-users] SuperLU + GPUs

[Satish Balay]

Please send configure.log

This is what I get on my linux build:

[balay at p1 petsc]$ ./configure --with-mpi-dir=/home/petsc/soft/openmpi-4.0.2-cuda --with-cuda=1 --with-openmp=1 --download-superlu-dist=1 && make && make check
<snip>
Running check examples to verify correct installation
Using PETSC_DIR=/home/balay/petsc and PETSC_ARCH=arch-linux-c-debug
C/C++ example src/snes/tutorials/ex19 run successfully with 1 MPI process
C/C++ example src/snes/tutorials/ex19 run successfully with 2 MPI processes
1a2,19
> CUDA version:   v 10020
> CUDA Devices: 
>  
> 0 : Quadro T2000 7 5
>   Global memory:   3911 mb 
>   Shared memory:   48 kb 
>   Constant memory: 64 kb 
>   Block registers: 65536 
> 
> CUDA version:   v 10020
> CUDA Devices: 
>  
> 0 : Quadro T2000 7 5
>   Global memory:   3911 mb 
>   Shared memory:   48 kb 
>   Constant memory: 64 kb 
>   Block registers: 65536 
> 
/home/balay/petsc/src/snes/tutorials
Possible problem with ex19 running with superlu_dist, diffs above
=========================================
Fortran example src/snes/tutorials/ex5f run successfully with 1 MPI process
Completed test examples


On Wed, 15 Apr 2020, Mark Adams wrote:

> On Wed, Apr 15, 2020 at 5:17 PM Satish Balay <balay at mcs.anl.gov> wrote:
> 
> > The build should work. It should give some verbose info [at runtime]
> > regarding GPUs - from the following code.
> >
> >
> I don't see that and I am running GPUs in my code and have gotten cusparse
> LU to run. Should I use '-info :sys:'  ?
> 
> 
> > >>>>> SRC/cublas_utils.c >>>>>>>>>>>
> >  void DisplayHeader()
> > {
> >     const int kb = 1024;
> >     const int mb = kb * kb;
> >     // cout << "NBody.GPU" << endl << "=========" << endl << endl;
> >
> >     printf("CUDA version:   v %d\n",CUDART_VERSION);
> >     //cout << "Thrust version: v" << THRUST_MAJOR_VERSION << "." <<
> > THRUST_MINOR_VERSION << endl << endl;
> >
> >     int devCount;
> >     cudaGetDeviceCount(&devCount);
> >     printf( "CUDA Devices: \n \n");
> > <snip>
> > <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
> >
> > Satish
> >
> > On Wed, 15 Apr 2020, Junchao Zhang wrote:
> >
> > > I remember Barry said superlu gpu support is broken.
> > > --Junchao Zhang
> > >
> > >
> > > On Wed, Apr 15, 2020 at 3:47 PM Mark Adams <mfadams at lbl.gov> wrote:
> > >
> > > > How does one use SuperLU with GPUs. I don't seem to get any GPU
> > > > performance data so I assume GPUs are not getting turned on. Am I wrong
> > > > about that?
> > > >
> > > > I configure with:
> > > > configure options: --with-fc=0 --COPTFLAGS="-g -O2 -fPIC -fopenmp"
> > > > --CXXOPTFLAGS="-g -O2 -fPIC -fopenmp" --FOPTFLAGS="-g -O2 -fPIC
> > -fopenmp"
> > > > --CUDAOPTFLAGS="-O2 -g" --with-ssl=0 --with-batch=0 --with-cxx=mpicxx
> > > > --with-mpiexec="jsrun -g1" --with-cuda=1 --with-cudac=nvcc
> > > > --download-p4est=1 --download-zlib --download-hdf5=1 --download-metis
> > > > --download-superlu --download-superlu_dist --with-make-np=16
> > > > --download-parmetis --download-triangle
> > > >
> > --with-blaslapack-lib="-L/autofs/nccs-svm1_sw/summit/.swci/1-compute/opt/spack/20180914/linux-rhel7-ppc64le/gcc-6.4.0/netlib-lapack-3.8.0-wcabdyqhdi5rooxbkqa6x5d7hxyxwdkm/lib64
> > > > -lblas -llapack" --with-cc=mpicc --with-shared-libraries=1 --with-x=0
> > > > --with-64-bit-indices=0 --with-debugging=0
> > > > PETSC_ARCH=arch-summit-opt-gnu-cuda-omp --with-openmp=1
> > > > --with-threadsaftey=1 --with-log=1
> > > >
> > > > Thanks,
> > > > Mark
> > > >
> > >
> >
> >
>