[petsc-users] PCMG questions

Tue May 9 21:43:32 CDT 2023

  Based on the error message, there is a DMSHELL. Run with -start_in_debugger and put a breakpoint in DMCreate_Shell, and you'll find out where it is being called

> On May 9, 2023, at 10:30 PM, Mark Adams <mfadams at lbl.gov> wrote:
> 
> No, there is no DMSHELL.
> 
> The code is in adams/snes-example-tokamac: src/dm/impls/plex/tests/ex96.c
> 
> With Galerkin coarse grids, the solver does solve but it is wrong so I am going to focus on that.
> I am manually doing what is in DMRefineHierarchy_Plex and there are a few things that I am not doing.
> This code does not work in parallel but I need to redo my MPI syntax strategy here (building a tree in MPI), so don't look at that.
> (the first few coarse grids are attached, wip)
> 
> Thanks,
> Mark
> 
> 
> On Tue, May 9, 2023 at 9:07 PM Barry Smith <bsmith at petsc.dev <mailto:bsmith at petsc.dev>> wrote:
>> 
>>> Must call DMShellSetGlobalVector() or DMShellSetCreateGlobalVector()
>>> [0]PETSC ERROR: #1 DMCreateGlobalVector_Shell() at /Users/markadams/Codes/petsc/src/dm/impls/shell/dmshell.c:210
>> 
>>   It looks like you have built a DMSHELL? You need to teach it how to generate global vectors since yours currently does not.
>> 
>>   Barry
>> 
>> 
>>> On May 9, 2023, at 5:40 PM, Mark Adams <mfadams at lbl.gov <mailto:mfadams at lbl.gov>> wrote:
>>> 
>>> 
>>> 
>>> On Tue, May 9, 2023 at 3:01 PM Barry Smith <bsmith at petsc.dev <mailto:bsmith at petsc.dev>> wrote:
>>>> 
>>>> 
>>>>> On May 9, 2023, at 12:32 PM, Mark Adams <mfadams at lbl.gov <mailto:mfadams at lbl.gov>> wrote:
>>>>> 
>>>>> I have a MG hierarchy that I construct manually with DMRefine and DMPlexExtrude.
>>>>> 
>>>>> * The solver works great with chevy/sor but with chevy/sor it converges slowly or I get indefinite PC errors from CG. And the eigen estimates in cheby are really high, like 10-15. 
>>>> 
>>>>    So with Cheby/SOR it works great but with the exact same options Cheby/SOR it behaves poorly? Are you using  some quantum computer and NERSc?
>>>  
>>> It turned out that I had the sign wrong on my Laplacian point function and so the matrix was negative definite. I'm not sure what really happened exactly but it is sort of behaving better.
>>> It looks like my prolongation operator is garbage, the coarse grid correction does nothing (cg/jacobi converges in a little less that the number of MG iterations times the sum of pre and post smoothing steps), and the rows sums of P are not 1.
>>> Not sure what is going on there, but it is probably related to the DM hierarchy not being constructed correctly....
>>>>> 
>>>>> * I tried turning galerkin=none and I got this error. 
>>>> 
>>>>   This is because without Garkin it needs to restrict the current solution and then compute the coarse grid Jacobian. Since you did not provide a DM that has the ability to even generate coarse grid vectors the process can work. You need a DM that can provide the coarse grid vectors and restrict solutions. Did you forget to pass a DM to the solver?
>>> 
>>> The DM does everything. Similar to many examples but I've been checking with snes/tutorials/ex12.c today. 
>>> I do call:
>>> PetscCall(DMSetCoarseDM(dmhierarchy[r], dmhierarchy[r-1]));
>>> But I am missing something else that goes on in DMRefineHierarchy, which I can't use because I am semi-coarsening.
>>> I probably have to build a section on each DM or something, but I have bigger fish to fry at this point.
>>> 
>>> (I construct a 2D coarse grid, refine that a number of times and DMPlexExtrude each one the same amount (number and distance), the extruded direction is wrapped around a torus and made periodic. 
>>> The fine grid now looks like, and will eventually be, the grids that tokamak codes use.)
>>> 
>>> Thanks,
>>> Mark
>>>  
>>>>> 
>>>>> Any thoughts on either of these issues?
>>>>> 
>>>>> Thanks,
>>>>> Mark
>>>>> 
>>>>> [0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------
>>>>> [0]PETSC ERROR: Must call DMShellSetGlobalVector() or DMShellSetCreateGlobalVector()
>>>>> [0]PETSC ERROR: WARNING! There are option(s) set that were not used! Could be the program crashed before they were used or a spelling mistake, etc!
>>>>> [0]PETSC ERROR:   Option left: name:-ksp_converged_reason (no value) source: command line
>>>>> [0]PETSC ERROR:   Option left: name:-mg_levels_esteig_ksp_type value: cg source: command line
>>>>> [0]PETSC ERROR:   Option left: name:-mg_levels_pc_type value: sor source: command line
>>>>> [0]PETSC ERROR:   Option left: name:-options_left (no value) source: command line
>>>>> [0]PETSC ERROR: See https://petsc.org/release/faq/ for trouble shooting.
>>>>> [0]PETSC ERROR: Petsc Development GIT revision: v3.19.1-224-g9ed82936d20  GIT Date: 2023-05-07 12:33:48 -0400
>>>>> [0]PETSC ERROR: ./ex96 on a arch-macosx-gnu-O named MarksMac-302.local by markadams Tue May  9 12:26:52 2023
>>>>> [0]PETSC ERROR: Configure options CFLAGS="-g  -Wall" CXXFLAGS="-g  -Wall" COPTFLAGS=-O CXXOPTFLAGS=-O --with-cc=/usr/local/opt/llvm/bin/clang --with-cxx=/usr/local/opt/llvm/bin/clang++ --download-mpich --with-strict-petscerrorcode --download-triangle=1 --with-x=0 --with-debugging=0 --download-hdf5=1 PETSC_ARCH=arch-macosx-gnu-O
>>>>> [0]PETSC ERROR: #1 DMCreateGlobalVector_Shell() at /Users/markadams/Codes/petsc/src/dm/impls/shell/dmshell.c:210
>>>>> [0]PETSC ERROR: #2 DMCreateGlobalVector() at /Users/markadams/Codes/petsc/src/dm/interface/dm.c:1022
>>>>> [0]PETSC ERROR: #3 DMGetNamedGlobalVector() at /Users/markadams/Codes/petsc/src/dm/interface/dmget.c:377
>>>>> [0]PETSC ERROR: #4 DMRestrictHook_SNESVecSol() at /Users/markadams/Codes/petsc/src/snes/interface/snes.c:649
>>>>> [0]PETSC ERROR: #5 DMRestrict() at /Users/markadams/Codes/petsc/src/dm/interface/dm.c:3407
>>>>> [0]PETSC ERROR: #6 PCSetUp_MG() at /Users/markadams/Codes/petsc/src/ksp/pc/impls/mg/mg.c:1074
>>>>> [0]PETSC ERROR: #7 PCSetUp() at /Users/markadams/Codes/petsc/src/ksp/pc/interface/precon.c:994
>>>>> [0]PETSC ERROR: #8 KSPSetUp() at /Users/markadams/Codes/petsc/src/ksp/ksp/interface/itfunc.c:406
>>>>> [0]PETSC ERROR: #9 KSPSolve_Private() at /Users/markadams/Codes/petsc/src/ksp/ksp/interface/itfunc.c:824
>>>>> [0]PETSC ERROR: #10 KSPSolve() at /Users/markadams/Codes/petsc/src/ksp/ksp/interface/itfunc.c:1070
>>>>> [0]PETSC ERROR: #11 SNESSolve_KSPONLY() at /Users/markadams/Codes/petsc/src/snes/impls/ksponly/ksponly.c:48
>>>>> [0]PETSC ERROR: #12 SNESSolve() at /Users/markadams/Codes/petsc/src/snes/interface/snes.c:4663
>>>>> [0]PETSC ERROR: #13 main() at ex96.c:433
>>>>> 
>>>>> 
>>>>> 
>>>> 
>> 
> <Screenshot 2023-05-09 at 7.49.16 AM.png>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20230509/36c4b0d9/attachment.html>