[petsc-dev] sm_70

[Mark Adams]

0:30 adams/feature-mat-cuda *= ~/petsc$ nvidia-smi
Sat Sep 26 12:42:37 2020
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 418.116.00   Driver Version: 418.116.00   CUDA Version: 10.1
  |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr.
ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute
M. |
|===============================+======================+======================|
|   0  Tesla V100-SXM2...  On   | 00000004:05:00.0 Off |
 0 |
| N/A   38C    P0    36W / 300W |      0MiB / 16130MiB |      0%   E.
Process |
+-------------------------------+----------------------+----------------------+
|   1  Tesla V100-SXM2...  On   | 00000035:04:00.0 Off |
 0 |
| N/A   40C    P0    38W / 300W |      0MiB / 16130MiB |      0%   E.
Process |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                       GPU
Memory |
|  GPU       PID   Type   Process name                             Usage
   |
|=============================================================================|
|  No running processes found
  |
+-----------------------------------------------------------------------------+

On Fri, Sep 25, 2020 at 9:36 PM Jacob Faibussowitsch <jacob.fai at gmail.com>
wrote:

>  Configure by default should find out the available GPU and build for that
> sm_*  it should not require the user to set this (how the heck is the user
> going to know what to set?)  If I remember correctly there is a utility
> available that gives this information.
>
> For CUDA I believe the tool is nvidia-smi. Should make sure this automatic
> detection works when configuring —with-batch though since login nodes might
> have different arch than compute.
>
> Best regards,
>
> Jacob Faibussowitsch
> (Jacob Fai - booss - oh - vitch)
> Cell: (312) 694-3391
>
> On Sep 25, 2020, at 21:09, Barry Smith <bsmith at petsc.dev> wrote:
>
>
>   Configure by default should find out the available GPU and build for
> that sm_*  it should not require the user to set this (how the heck is the
> user going to know what to set?)  If I remember correctly there is a
> utility available that gives this information.
>
>   For generic builds like in package distributions I don't know how it
> should work, ideally all the possibilities would be available in the
> library and at run time the correct one will be utilized.
>
>   Barry
>
>
> On Sep 25, 2020, at 5:49 PM, Mark Adams <mfadams at lbl.gov> wrote:
>
>    '--CUDAFLAGS=-arch=sm_70',
>
> seems to fix this.
>
> On Fri, Sep 25, 2020 at 6:31 PM Mark Adams <mfadams at lbl.gov> wrote:
>
>> I see kokkos and hyper have a sm_70 flag, but I don't see one for PETSc.
>>
>> It looks like you have to specify this to get modern atomics to work in
>> Cuda. I get:
>>
>> /ccs/home/adams/petsc/include/petscaijdevice.h(99): error: no instance of
>> overloaded function "atomicAdd" matches the argument list
>>             argument types are: (double *, double)
>>
>> I tried using a Kokkos configuration, thinking I could get these sm_70
>> flags, but that did not work.
>>
>> Any ideas?
>>
>> Mark
>>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20200926/69518327/attachment-0001.html>