[petsc-dev] PETSc GPU example

[Matthew Knepley]

On Mon, Dec 6, 2021 at 9:04 PM Fande Kong <fdkong.jd at gmail.com> wrote:

> On Mon, Dec 6, 2021 at 5:59 PM Matthew Knepley <knepley at gmail.com> wrote:
>
>> On Mon, Dec 6, 2021 at 7:54 PM Fande Kong <fdkong.jd at gmail.com> wrote:
>>
>>> Thanks, Matt,
>>>
>>> Sorry, I still have more questions on this example. How to refine mesh
>>> to make the problem larger?
>>>
>>> I tried the following options, and none of them worked. I might do
>>> something wrong.
>>>
>>> -ex56_dm_refine 9
>>>
>>> and
>>>
>>> -dm_refine 4
>>>
>>
>> The mesh handling in this example does not conform to the others, but it
>> appears that
>>
>>   -ex56_dm_refine <k>
>>
>> should take effect at
>>
>>
>> https://gitlab.com/petsc/petsc/-/blob/main/src/snes/tutorials/ex56.c#L381
>>
>>
> I was puzzled about this because DMSetFromOptions does not seem to trigger
> -ex56_dm_refine.
>
> I did a search, and could not find where we call " -ex56_dm_refine" in
> PETSc.
>
> I got the same result by running the following two combinations:
>
> 1) ./ex56  -log_view  -snes_view  -max_conv_its 3 -ex56_dm_refine 10
>
> 2) ./ex56  -log_view  -snes_view  -max_conv_its 3 -ex56_dm_refine 0
>

It is refining:

nathawani/feature-1D-droplet-model
$:/PETSc3/petsc/petsc-dev/src/snes/tutorials$ ./ex56 -ex56_dm_view
-max_conv_its 1
DM Object: Mesh (ex56_) 1 MPI processes
  type: plex
Mesh in 3 dimensions:
  Number of 0-cells per rank: 8
  Number of 1-cells per rank: 12
  Number of 2-cells per rank: 6
  Number of 3-cells per rank: 1
Labels:
  marker: 1 strata with value/size (1 (18))
  Face Sets: 6 strata with value/size (6 (1), 5 (1), 3 (1), 4 (1), 1 (1), 2
(1))
  depth: 4 strata with value/size (0 (8), 1 (12), 2 (6), 3 (1))
  boundary: 1 strata with value/size (1 (26))
  celltype: 4 strata with value/size (7 (1), 0 (8), 4 (6), 1 (12))
Field deformation:
  adjacency FEM
nathawani/feature-1D-droplet-model
$:/PETSc3/petsc/petsc-dev/src/snes/tutorials$ ./ex56 -ex56_dm_view
-max_conv_its 1 -ex56_dm_refine 3
DM Object: Mesh (ex56_) 1 MPI processes
  type: plex
Mesh in 3 dimensions:
  Number of 0-cells per rank: 729
  Number of 1-cells per rank: 1944
  Number of 2-cells per rank: 1728
  Number of 3-cells per rank: 512
Labels:
  celltype: 4 strata with value/size (0 (729), 1 (1944), 4 (1728), 7 (512))
  depth: 4 strata with value/size (0 (729), 1 (1944), 2 (1728), 3 (512))
  marker: 1 strata with value/size (1 (1530))
  Face Sets: 6 strata with value/size (6 (225), 5 (225), 3 (225), 4 (225),
1 (225), 2 (225))
  boundary: 1 strata with value/size (1 (1538))
Field deformation:
  adjacency FEM

   Matt


> Thanks,
>
> Fande
>
>
> unless you are setting max_conv_its to 0 somehow.
>>
>>   Thanks,
>>
>>      Matt
>>
>>
>>> Thanks,
>>>
>>> Fande
>>>
>>> On Mon, Dec 6, 2021 at 5:04 PM Matthew Knepley <knepley at gmail.com>
>>> wrote:
>>>
>>>> On Mon, Dec 6, 2021 at 7:02 PM Fande Kong <fdkong.jd at gmail.com> wrote:
>>>>
>>>>> Thanks, Matt
>>>>>
>>>>> On Mon, Dec 6, 2021 at 4:47 PM Matthew Knepley <knepley at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> On Mon, Dec 6, 2021 at 6:40 PM Fande Kong <fdkong.jd at gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Dear PETSc team,
>>>>>>>
>>>>>>> I am interested in a careful evaluation of PETSc GPU performance in
>>>>>>> our INL cluster.
>>>>>>>
>>>>>>> Any example in PETSc that can show GPU speedup with solving a
>>>>>>> nonlinear equation?
>>>>>>>
>>>>>>> I talked to Junchao; he suggested that I try SNES/tutorial/ex56. I
>>>>>>> tried that, but I could not find any speedup using the GPU. I could attach
>>>>>>> some results of "log_view" later if we would like to see that.
>>>>>>>
>>>>>>
>>>>>> We should note that you will only see speedup in the solver, so that
>>>>>> problem has to be pretty large. I believe Mark has good results with it.
>>>>>> The assembly is still all on the CPU. I am working on this over
>>>>>> break, and hope to have a CEED version of it by the new year.
>>>>>>
>>>>>
>>>>> Are both function and matrix assmelies on CPU? Or just the matrix
>>>>> assembly?
>>>>>
>>>>
>>>> There is no GPU assembly right now.
>>>>
>>>>   Matt
>>>>
>>>>
>>>>> OK, I will try to check the solver part
>>>>>
>>>>> Thanks, again
>>>>>
>>>>> Fande
>>>>>
>>>>>
>>>>>
>>>>>>
>>>>>>   Thanks,
>>>>>>
>>>>>>      Matt
>>>>>>
>>>>>>
>>>>>>> Appreciate any instructions/comments about running a simple PETSc
>>>>>>> GPU example to get a speedup.
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> Fande
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> What most experimenters take for granted before they begin their
>>>>>> experiments is infinitely more interesting than any results to which their
>>>>>> experiments lead.
>>>>>> -- Norbert Wiener
>>>>>>
>>>>>> https://www.cse.buffalo.edu/~knepley/
>>>>>> <http://www.cse.buffalo.edu/~knepley/>
>>>>>>
>>>>>
>>>>
>>>> --
>>>> What most experimenters take for granted before they begin their
>>>> experiments is infinitely more interesting than any results to which their
>>>> experiments lead.
>>>> -- Norbert Wiener
>>>>
>>>> https://www.cse.buffalo.edu/~knepley/
>>>> <http://www.cse.buffalo.edu/~knepley/>
>>>>
>>>
>>
>> --
>> What most experimenters take for granted before they begin their
>> experiments is infinitely more interesting than any results to which their
>> experiments lead.
>> -- Norbert Wiener
>>
>> https://www.cse.buffalo.edu/~knepley/
>> <http://www.cse.buffalo.edu/~knepley/>
>>
>

-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20211206/8150e4db/attachment.html>