[petsc-dev] MatPinToCPU

[Mark Adams]

On Mon, Jul 29, 2019 at 5:31 PM Smith, Barry F. <bsmith at mcs.anl.gov> wrote:

>
>   I don't understand the notation in the legend on the second page
>
> 12,288 cpus and no GPUs ?
>

Yes


>
> 24 GPUs?  or 6 GPUs
>

24 virtual, 6 real GPUs per node. The first case is one node, 24 cores/vGPUs


>
> 192 GPUs?
>
> 1536 GPUs?
>
> 12,288 GPUs?  or 12288/4 = 3072  GPUs?
>

All "GPUs" are one core/process/vGPU. So 12288 virtual GPUs and 3072
physical GPUs.

Maybe I should add 'virtual GPUs' and put (4 processes/SUMMIT GPU)


>
> So on the largest run using GPUs or not takes pretty much exactly the same
> amount of  time?
>

yes. The raw Mat-vec is about 3x faster with ~95K equations/process. I've
attached the data.


>
> What about 6 GPUs vs 24 CPUs ? Same equal amount of time.
>
> Can you send some log summaries
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20190729/c6eaa479/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: out_cpu_012288
Type: application/octet-stream
Size: 67119 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20190729/c6eaa479/attachment-0005.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: out_cuda_000024
Type: application/octet-stream
Size: 68761 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20190729/c6eaa479/attachment-0006.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: out_cuda_001536
Type: application/octet-stream
Size: 69610 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20190729/c6eaa479/attachment-0007.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: out_cuda_000192
Type: application/octet-stream
Size: 69109 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20190729/c6eaa479/attachment-0008.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: out_cuda_012288
Type: application/octet-stream
Size: 69994 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20190729/c6eaa479/attachment-0009.obj>