[petsc-dev] ASM for each field solve on GPUs

[Mark Adams]

PCFieldSplitSetDefaults is called from PCSetUpOnBlocks and there is only
one block. So it is called from a subpc, which does not have a DM and it's
too late anyway.

#14 main (argc=<optimized out>, argv=<optimized out>) at
/gpfs/alpine/csc314/scratch/adams/petsc/src/ts/utils/dmplexlandau/tutorials/ex2.c:774
(at 0x0000000010002a28)
#13 TSSolve (ts=0x83638480, u=0x8338de50) at
/gpfs/alpine/csc314/scratch/adams/petsc/src/ts/interface/ts.c:4168 (at
0x00002000015e60e4)
#12 TSStep (ts=0x83638480) at
/gpfs/alpine/csc314/scratch/adams/petsc/src/ts/interface/ts.c:3771 (at
0x00002000015e416c)
#11 TSStep_ARKIMEX (ts=0x83638480) at
/gpfs/alpine/csc314/scratch/adams/petsc/src/ts/impls/arkimex/arkimex.c:845
(at 0x000020000165e97c)
#10 SNESSolve (snes=0x83639d20, b=0x0, x=0x8338f320) at
/gpfs/alpine/csc314/scratch/adams/petsc/src/snes/interface/snes.c:4563 (at
0x00002000014f2b98)
#9 SNESSolve_NEWTONLS (snes=0x83639d20) at
/gpfs/alpine/csc314/scratch/adams/petsc/src/snes/impls/ls/ls.c:225 (at
0x0000200001553d34)
#8 KSPSolve (ksp=0x8363a610, b=0x8363db00, x=0x83642350) at
/gpfs/alpine/csc314/scratch/adams/petsc/src/ksp/ksp/interface/itfunc.c:963
(at 0x000020000138987c)
#7 KSPSolve_Private (ksp=0x8363a610, b=0x8363db00, x=0x83642350) at
/gpfs/alpine/csc314/scratch/adams/petsc/src/ksp/ksp/interface/itfunc.c:727
(at 0x0000200001386d70)
#6 KSPSetUpOnBlocks (ksp=0x8363a610) at
/gpfs/alpine/csc314/scratch/adams/petsc/src/ksp/ksp/interface/itfunc.c:214
(at 0x0000200001381a6c)
#5 PCSetUpOnBlocks (pc=0x8338afe0) at
/gpfs/alpine/csc314/scratch/adams/petsc/src/ksp/pc/interface/precon.c:1046
(at 0x0000200001333aa0)
#4 PCSetUpOnBlocks_ASM (pc=0x8338afe0) at
/gpfs/alpine/csc314/scratch/adams/petsc/src/ksp/pc/impls/asm/asm.c:437 (at
0x00002000012cb87c)
#3 KSPSetUp (ksp=0x836486c0) at
/gpfs/alpine/csc314/scratch/adams/petsc/src/ksp/ksp/interface/itfunc.c:406
(at 0x0000200001383258)
#2 PCSetUp (pc=0x83645200) at
/gpfs/alpine/csc314/scratch/adams/petsc/src/ksp/pc/interface/precon.c:1015
(at 0x00002000013335f0)
#1 PCSetUp_FieldSplit (pc=0x83645200) at
/gpfs/alpine/csc314/scratch/adams/petsc/src/ksp/pc/impls/fieldsplit/fieldsplit.c:614
(at 0x00002000012a27f4)
#0 PCFieldSplitSetDefaults (pc=0x83645200) at
/gpfs/alpine/csc314/scratch/adams/petsc/src/ksp/pc/impls/fieldsplit/fieldsplit.c:398
(at 0x000020000129f854)

On Thu, Dec 31, 2020 at 1:56 PM Matthew Knepley <knepley at gmail.com> wrote:

> On Thu, Dec 31, 2020 at 1:50 PM Mark Adams <mfadams at lbl.gov> wrote:
>
>> On Thu, Dec 31, 2020 at 1:12 PM Matthew Knepley <knepley at gmail.com>
>> wrote:
>>
>>> On Thu, Dec 31, 2020 at 12:46 PM Mark Adams <mfadams at lbl.gov> wrote:
>>>
>>>> Still missing something:
>>>>
>>>
>>> You can get in the debugger in PCFieldSplitSetDefaults(), but the most
>>> likely thing is that pc->dm == NULL here,
>>>
>>
>> the setuponblocks PC does have a DM and it is a cusparse mattype.
>>
>
> Then go in the debugger and see why that option is not being used.
>
>    Matt
>
>
>>
>>
>>> so the PC is not getting the DM.
>>> This does make some sense as you are doing ASM, which is ripping up the
>>> domain, so you might not want the same DM on the subproblem. I
>>> am not sure what should be done here.
>>>
>>>
>
> --
> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which their
> experiments lead.
> -- Norbert Wiener
>
> https://www.cse.buffalo.edu/~knepley/
> <http://www.cse.buffalo.edu/~knepley/>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20201231/9d472543/attachment.html>