[petsc-dev] DARefine produces unusable partitions

[Jed Brown]

Fixed, I'm still working on the ex48 problem.

Jed

On Wed, Sep 22, 2010 at 20:55, Barry Smith <bsmith at mcs.anl.gov> wrote:
>
> 0a1,29
>> [0]PETSC ERROR: --------------------- Error Message ------------------------------------
>> [0]PETSC ERROR: Argument out of range!
>> [0]PETSC ERROR: Sum of lx across processors not equal to M 39 19!
>> [0]PETSC ERROR: ------------------------------------------------------------------------
>> [0]PETSC ERROR: Petsc Development HG revision: cd00cd8c8d87f4b93d5195df073a02522fc5c1a8  HG Date: Fri Sep 17 15:02:40 2010 -0500
>> [0]PETSC ERROR: See docs/changes/index.html for recent updates.
>> [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting.
>> [0]PETSC ERROR: See docs/index.html for manual pages.
>> [0]PETSC ERROR: ------------------------------------------------------------------------
>> [0]PETSC ERROR: ./ex22 on a arch-gnu named bsmith-laptop.mcs.anl.gov by barrysmith Wed Sep 22 13:41:50 2010
>> [0]PETSC ERROR: Libraries linked from /Users/barrysmith/Src/petsc-dev/arch-gnu/lib
>> [0]PETSC ERROR: Configure run at Fri Sep 17 15:06:36 2010
>> [0]PETSC ERROR: Configure options --download-mpich PETSC_ARCH=arch-gnu --with-server --with-ams-dir=/Users/barrysmith/Src/ams-dev --with-java --with-fc="gfortran -m64"
>> [0]PETSC ERROR: ------------------------------------------------------------------------
>> [0]PETSC ERROR: DACreate_1D() line 188 in src/dm/da/src/da1.c
>> [0]PETSC ERROR: DASetType() line 48 in src/dm/da/src/dareg.c
>> [0]PETSC ERROR: DASetTypeFromOptions_Private() line 67 in src/dm/da/src/dacreate.c
>> [0]PETSC ERROR: DASetFromOptions() line 128 in src/dm/da/src/dacreate.c
>> [0]PETSC ERROR: DACreate1d() line 393 in src/dm/da/src/da1.c
>> [0]PETSC ERROR: DARefine() line 690 in src/dm/da/src/da.c
>> [0]PETSC ERROR: DMRefine() line 259 in src/dm/da/utils/dm.c
>> [0]PETSC ERROR: DMCompositeRefine() line 1387 in src/dm/da/utils/pack.c
>> [0]PETSC ERROR: DMRefine() line 259 in src/dm/da/utils/dm.c
>> [0]PETSC ERROR: DMRefineHierarchy() line 444 in src/dm/da/utils/dm.c
>> [0]PETSC ERROR: DMMGSetDM() line 222 in src/snes/utils/damg.c
>> [0]PETSC ERROR: main() line 118 in src/snes/examples/tutorials/ex22.c
>> application called MPI_Abort(MPI_COMM_WORLD, 63) - process 0
>> [cli_0]: aborting job:
>> application called MPI_Abort(MPI_COMM_WORLD, 63) - process 0
> Possible problem with ex22_1, diffs above
> 3c3
> <     2 SNES Function norm 2.29297e-06
> ---
>>     2 SNES Function norm 2.29302e-06
> 6c6
> <   1 SNES Function norm 5.60828e-05
> ---
>>   1 SNES Function norm 5.6082e-05
> Possible problem with ex23_1, diffs above
>
>
> On Sep 22, 2010, at 11:41 AM, Jed Brown wrote:
>
>> $ cd src/dm/da/examples/tests
>> $ make ex3
>> $ mpiexec -n 5 ./ex3 -M 8
>>
>> The issue is that the partition [0,2,4,6,7,8] is refined to
>> [0,3,6,9,12,15].  Now on rank 3, the start indices (6 and 9) don't
>> line up (because 9/2=4 < 6-1).  I'm about to push a patch for this
>> that creates and always uses lx,ly,lz for refinement so that the
>> subdomains are always nested.  The downside of this is that when the
>> coarse level is imbalanced (not a serious problem), all refinements
>> will preserve the coarse fractional imbalance, which may be a big
>> problem on the fine meshes.
>>
>> It is probably not hard to redistribute lx,ly,lz within the
>> constraints that DAGetInterpolation has to work, so you can't move a
>> subdomain interface by more than one coarse stencil width.  But I'm
>> not sure that any redistribution is always desirable (especially if
>> the user used DASetVertexDivision).
>>
>> Jed
>
>