bug ? number of processor, coarse and fine DA for DMMG

[nicolas aunai]

Hello and thanks for your answer, could you tell me what exactly is
the limitation, in terms of number of processors and grid size ?


Nico



2009/9/4 Barry Smith <bsmith at mcs.anl.gov>:
>
>  This is a limitation in the way the interpolation is computed.  It is not
> really an error. There is no easy way to extend the code for these other
> configurations.
>
>
>   Barry
>
> On Sep 4, 2009, at 7:39 AM, nicolas aunai wrote:
>
>> Hello,
>>
>>
>> I have a code that uses the DMMG objet for a linear problem. The DA I
>> create for the coarse grid is DA_XPERIODIC and 2D. I let petsc decide
>> for the number of processors in each direction.
>>
>> I have a problem which depends on the number of processors I use.
>> If I want a 1024x1025 grid with 4 levels, the coarsest grid will be
>> 128x129 with the default refinement factor.
>>
>> Everything seems to work fine when I run with 1, 2, 3, 4, 6, 8...
>> processors... but 5, 7, 11 proc fail with the following error :
>>
>>
>> nicolas at lx-nau:~/code/tests/petsc/dmmg$ mpiexec -n 5 ./inertia6
>> -ksp_type gmres -pc_type sor -ksp_rtol 1e-9
>>
>> size of the coarse level : 128 129
>> debug 1, number of proc in each dir. : 1 5
>>
>> [3]PETSC ERROR: --------------------- Error Message
>> ------------------------------------
>> [3]PETSC ERROR: Arguments are incompatible!
>> [3]PETSC ERROR: Processor's coarse DA must lie over fine DA
>>   j_start 615 j_c 307 j_start_ghost_c 308!
>> [3]PETSC ERROR:
>> ------------------------------------------------------------------------
>> [3]PETSC ERROR: Petsc Release Version 3.0.0, Patch 7, Mon Jul  6
>> 11:33:34 CDT 2009
>> [3]PETSC ERROR: See docs/changes/index.html for recent updates.
>> [3]PETSC ERROR: See docs/faq.html for hints about trouble shooting.
>> [3]PETSC ERROR: See docs/index.html for manual pages.
>> [3]PETSC ERROR:
>> ------------------------------------------------------------------------
>> [3]PETSC ERROR: ./inertia6 on a linux-gnu named lx-nau by nicolas Fri
>> Sep  4 14:39:27 2009
>> [3]PETSC ERROR: Libraries linked from
>> /home/nicolas/bin/petsc-3.0.0-p7/linux-gnu-debug/lib
>> [3]PETSC ERROR: Configure run at Mon Aug 17 11:34:27 2009
>> [3]PETSC ERROR: Configure options --with-cc=gcc --with-fc=gfortran
>> --download-f-blas-lapack=1 --download-mpich=1
>> PETSC_ARCH=linux-gnu-debug --with-shared=0
>> [3]PETSC ERROR:
>> ------------------------------------------------------------------------
>> [3]PETSC ERROR: DAGetInterpolation_2D_Q1() line 280 in
>> src/dm/da/src/dainterp.c
>> [3]PETSC ERROR: DAGetInterpolation() line 879 in src/dm/da/src/dainterp.c
>> [3]PETSC ERROR: DMGetInterpolation() line 144 in src/dm/da/utils/dm.c
>> [3]PETSC ERROR: DMMGSetDM() line 309 in src/snes/utils/damg.c
>>
>>
>>
>> it seems that the error come from the routine DMMGSetDM, when petsc
>> build the restriction/interpolation operators... but I can't figure
>> out why I couln't use 5 processors with this grid size ?
>>
>> if the code is needed, it is here : http://nico.aunai.free.fr/inertia6.c
>>
>>
>> Thx for your help
>>
>> Nico
>
>