bug ? number of processor, coarse and fine DA for DMMG

[Barry Smith]

     There is no simple formula.

    Barry

On Sep 7, 2009, at 1:33 AM, nicolas aunai wrote:

> Hello and thanks for your answer, could you tell me what exactly is
> the limitation, in terms of number of processors and grid size ?
>
>
> Nico
>
>
>
> 2009/9/4 Barry Smith <bsmith at mcs.anl.gov>:
>>
>>  This is a limitation in the way the interpolation is computed.  It  
>> is not
>> really an error. There is no easy way to extend the code for these  
>> other
>> configurations.
>>
>>
>>   Barry
>>
>> On Sep 4, 2009, at 7:39 AM, nicolas aunai wrote:
>>
>>> Hello,
>>>
>>>
>>> I have a code that uses the DMMG objet for a linear problem. The  
>>> DA I
>>> create for the coarse grid is DA_XPERIODIC and 2D. I let petsc  
>>> decide
>>> for the number of processors in each direction.
>>>
>>> I have a problem which depends on the number of processors I use.
>>> If I want a 1024x1025 grid with 4 levels, the coarsest grid will be
>>> 128x129 with the default refinement factor.
>>>
>>> Everything seems to work fine when I run with 1, 2, 3, 4, 6, 8...
>>> processors... but 5, 7, 11 proc fail with the following error :
>>>
>>>
>>> nicolas at lx-nau:~/code/tests/petsc/dmmg$ mpiexec -n 5 ./inertia6
>>> -ksp_type gmres -pc_type sor -ksp_rtol 1e-9
>>>
>>> size of the coarse level : 128 129
>>> debug 1, number of proc in each dir. : 1 5
>>>
>>> [3]PETSC ERROR: --------------------- Error Message
>>> ------------------------------------
>>> [3]PETSC ERROR: Arguments are incompatible!
>>> [3]PETSC ERROR: Processor's coarse DA must lie over fine DA
>>>   j_start 615 j_c 307 j_start_ghost_c 308!
>>> [3]PETSC ERROR:
>>> ------------------------------------------------------------------------
>>> [3]PETSC ERROR: Petsc Release Version 3.0.0, Patch 7, Mon Jul  6
>>> 11:33:34 CDT 2009
>>> [3]PETSC ERROR: See docs/changes/index.html for recent updates.
>>> [3]PETSC ERROR: See docs/faq.html for hints about trouble shooting.
>>> [3]PETSC ERROR: See docs/index.html for manual pages.
>>> [3]PETSC ERROR:
>>> ------------------------------------------------------------------------
>>> [3]PETSC ERROR: ./inertia6 on a linux-gnu named lx-nau by nicolas  
>>> Fri
>>> Sep  4 14:39:27 2009
>>> [3]PETSC ERROR: Libraries linked from
>>> /home/nicolas/bin/petsc-3.0.0-p7/linux-gnu-debug/lib
>>> [3]PETSC ERROR: Configure run at Mon Aug 17 11:34:27 2009
>>> [3]PETSC ERROR: Configure options --with-cc=gcc --with-fc=gfortran
>>> --download-f-blas-lapack=1 --download-mpich=1
>>> PETSC_ARCH=linux-gnu-debug --with-shared=0
>>> [3]PETSC ERROR:
>>> ------------------------------------------------------------------------
>>> [3]PETSC ERROR: DAGetInterpolation_2D_Q1() line 280 in
>>> src/dm/da/src/dainterp.c
>>> [3]PETSC ERROR: DAGetInterpolation() line 879 in src/dm/da/src/ 
>>> dainterp.c
>>> [3]PETSC ERROR: DMGetInterpolation() line 144 in src/dm/da/utils/ 
>>> dm.c
>>> [3]PETSC ERROR: DMMGSetDM() line 309 in src/snes/utils/damg.c
>>>
>>>
>>>
>>> it seems that the error come from the routine DMMGSetDM, when petsc
>>> build the restriction/interpolation operators... but I can't figure
>>> out why I couln't use 5 processors with this grid size ?
>>>
>>> if the code is needed, it is here : http://nico.aunai.free.fr/inertia6.c
>>>
>>>
>>> Thx for your help
>>>
>>> Nico
>>
>>