<div dir="ltr"><div dir="ltr">Please see PR <a href="https://gitlab.com/petsc/petsc/merge_requests/2294/diffs">https://gitlab.com/petsc/petsc/merge_requests/2294/diffs</a></div><div dir="ltr"><br></div><div>This may be a PETSc bug. </div><div><br></div><div>Fande,</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sun, Nov 10, 2019 at 7:28 PM Smith, Barry F. <<a href="mailto:bsmith@mcs.anl.gov">bsmith@mcs.anl.gov</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><br>
Nice. Anyway to add this exact reproducibility into a PETSc example that runs daily? <br>
<br>
The truism: all codes are buggy, even those that haven't been touched in 15 years, is definitely represented here.<br>
<br>
Barry<br>
<br>
<br>
> On Nov 10, 2019, at 7:31 PM, Fande Kong via petsc-dev <<a href="mailto:petsc-dev@mcs.anl.gov" target="_blank">petsc-dev@mcs.anl.gov</a>> wrote:<br>
> <br>
> Valgrind info:<br>
> <br>
> ==32155== Invalid read of size 4<br>
> ==32155== at 0x885F62F: libmetis__CreateCoarseGraphNoMask (coarsen.c:879)<br>
> ==32155== by 0x885E2B9: libmetis__CreateCoarseGraph (coarsen.c:636)<br>
> ==32155== by 0x885CA49: libmetis__Match_RM (coarsen.c:262)<br>
> ==32155== by 0x885BFBD: libmetis__CoarsenGraph (coarsen.c:55)<br>
> ==32155== by 0x8869A4C: libmetis__MultilevelBisect (pmetis.c:240)<br>
> ==32155== by 0x88696F6: libmetis__MlevelRecursiveBisection (pmetis.c:183)<br>
> ==32155== by 0x88698D7: libmetis__MlevelRecursiveBisection (pmetis.c:207)<br>
> ==32155== by 0x886951E: METIS_PartGraphRecursive (pmetis.c:133)<br>
> ==32155== by 0x885650B: libmetis__InitKWayPartitioning (kmetis.c:194)<br>
> ==32155== by 0x8856147: libmetis__MlevelKWayPartitioning (kmetis.c:121)<br>
> ==32155== by 0x8855FD3: METIS_PartGraphKway (kmetis.c:71)<br>
> ==32155== by 0x85E7611: libparmetis__PartitionSmallGraph (weird.c:478)<br>
> ==32155== by 0x85D00F3: ParMETIS_V3_PartKway (kmetis.c:91)<br>
> ==32155== by 0x5515C19: MatPartitioningApply_Parmetis_Private (pmetis.c:147)<br>
> ==32155== by 0x5516F7D: MatPartitioningApply_Parmetis (pmetis.c:221)<br>
> ==32155== by 0x550DD7F: MatPartitioningApply (partition.c:332)<br>
> ==32155== by 0x6387033: PCGAMGCreateLevel_GAMG (gamg.c:226)<br>
> ==32155== by 0x638BCA5: PCSetUp_GAMG (gamg.c:593)<br>
> ==32155== by 0x64746B1: PCSetUp (precon.c:894)<br>
> ==32155== by 0x65D388A: KSPSetUp (itfunc.c:377)<br>
> ==32155== Address 0xee2502c is 0 bytes after a block of size 284 alloc'd<br>
> ==32155== at 0x4C2AB80: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)<br>
> ==32155== by 0x8820179: gk_malloc (memory.c:147)<br>
> ==32155== by 0x883D92B: libmetis__imalloc (gklib.c:24)<br>
> ==32155== by 0x885BF6A: libmetis__CoarsenGraph (coarsen.c:46)<br>
> ==32155== by 0x8869A4C: libmetis__MultilevelBisect (pmetis.c:240)<br>
> ==32155== by 0x88696F6: libmetis__MlevelRecursiveBisection (pmetis.c:183)<br>
> ==32155== by 0x88698D7: libmetis__MlevelRecursiveBisection (pmetis.c:207)<br>
> ==32155== by 0x886951E: METIS_PartGraphRecursive (pmetis.c:133)<br>
> ==32155== by 0x885650B: libmetis__InitKWayPartitioning (kmetis.c:194)<br>
> ==32155== by 0x8856147: libmetis__MlevelKWayPartitioning (kmetis.c:121)<br>
> ==32155== by 0x8855FD3: METIS_PartGraphKway (kmetis.c:71)<br>
> ==32155== by 0x85E7611: libparmetis__PartitionSmallGraph (weird.c:478)<br>
> ==32155== by 0x85D00F3: ParMETIS_V3_PartKway (kmetis.c:91)<br>
> ==32155== by 0x5515C19: MatPartitioningApply_Parmetis_Private (pmetis.c:147)<br>
> ==32155== by 0x5516F7D: MatPartitioningApply_Parmetis (pmetis.c:221)<br>
> ==32155== by 0x550DD7F: MatPartitioningApply (partition.c:332)<br>
> ==32155== by 0x6387033: PCGAMGCreateLevel_GAMG (gamg.c:226)<br>
> ==32155== by 0x638BCA5: PCSetUp_GAMG (gamg.c:593)<br>
> ==32155== by 0x64746B1: PCSetUp (precon.c:894)<br>
> ==32155== by 0x65D388A: KSPSetUp (itfunc.c:377)<br>
> ==32155== <br>
> ==32155== Conditional jump or move depends on uninitialised value(s)<br>
> ==32155== at 0x885F651: libmetis__CreateCoarseGraphNoMask (coarsen.c:880)<br>
> ==32155== by 0x885E2B9: libmetis__CreateCoarseGraph (coarsen.c:636)<br>
> ==32155== by 0x885CA49: libmetis__Match_RM (coarsen.c:262)<br>
> ==32155== by 0x885BFBD: libmetis__CoarsenGraph (coarsen.c:55)<br>
> ==32155== by 0x8869A4C: libmetis__MultilevelBisect (pmetis.c:240)<br>
> ==32155== by 0x88696F6: libmetis__MlevelRecursiveBisection (pmetis.c:183)<br>
> ==32155== by 0x88698D7: libmetis__MlevelRecursiveBisection (pmetis.c:207)<br>
> ==32155== by 0x886951E: METIS_PartGraphRecursive (pmetis.c:133)<br>
> ==32155== by 0x885650B: libmetis__InitKWayPartitioning (kmetis.c:194)<br>
> ==32155== by 0x8856147: libmetis__MlevelKWayPartitioning (kmetis.c:121)<br>
> ==32155== by 0x8855FD3: METIS_PartGraphKway (kmetis.c:71)<br>
> ==32155== by 0x85E7611: libparmetis__PartitionSmallGraph (weird.c:478)<br>
> ==32155== by 0x85D00F3: ParMETIS_V3_PartKway (kmetis.c:91)<br>
> ==32155== by 0x5515C19: MatPartitioningApply_Parmetis_Private (pmetis.c:147)<br>
> ==32155== by 0x5516F7D: MatPartitioningApply_Parmetis (pmetis.c:221)<br>
> ==32155== by 0x550DD7F: MatPartitioningApply (partition.c:332)<br>
> ==32155== by 0x6387033: PCGAMGCreateLevel_GAMG (gamg.c:226)<br>
> ==32155== by 0x638BCA5: PCSetUp_GAMG (gamg.c:593)<br>
> ==32155== by 0x64746B1: PCSetUp (precon.c:894)<br>
> ==32155== by 0x65D388A: KSPSetUp (itfunc.c:377)<br>
> ==32155== <br>
> ==32155== Use of uninitialised value of size 8<br>
> ==32155== at 0x885F6F2: libmetis__CreateCoarseGraphNoMask (coarsen.c:886)<br>
> ==32155== by 0x885E2B9: libmetis__CreateCoarseGraph (coarsen.c:636)<br>
> ==32155== by 0x885CA49: libmetis__Match_RM (coarsen.c:262)<br>
> ==32155== by 0x885BFBD: libmetis__CoarsenGraph (coarsen.c:55)<br>
> ==32155== by 0x8869A4C: libmetis__MultilevelBisect (pmetis.c:240)<br>
> ==32155== by 0x88696F6: libmetis__MlevelRecursiveBisection (pmetis.c:183)<br>
> ==32155== by 0x88698D7: libmetis__MlevelRecursiveBisection (pmetis.c:207)<br>
> ==32155== by 0x886951E: METIS_PartGraphRecursive (pmetis.c:133)<br>
> ==32155== by 0x885650B: libmetis__InitKWayPartitioning (kmetis.c:194)<br>
> ==32155== by 0x8856147: libmetis__MlevelKWayPartitioning (kmetis.c:121)<br>
> ==32155== by 0x8855FD3: METIS_PartGraphKway (kmetis.c:71)<br>
> ==32155== by 0x85E7611: libparmetis__PartitionSmallGraph (weird.c:478)<br>
> ==32155== by 0x85D00F3: ParMETIS_V3_PartKway (kmetis.c:91)<br>
> ==32155== by 0x5515C19: MatPartitioningApply_Parmetis_Private (pmetis.c:147)<br>
> ==32155== by 0x5516F7D: MatPartitioningApply_Parmetis (pmetis.c:221)<br>
> ==32155== by 0x550DD7F: MatPartitioningApply (partition.c:332)<br>
> ==32155== by 0x6387033: PCGAMGCreateLevel_GAMG (gamg.c:226)<br>
> ==32155== by 0x638BCA5: PCSetUp_GAMG (gamg.c:593)<br>
> ==32155== by 0x64746B1: PCSetUp (precon.c:894)<br>
> ==32155== by 0x65D388A: KSPSetUp (itfunc.c:377)<br>
> ==32155== <br>
> ==32155== Use of uninitialised value of size 8<br>
> ==32155== at 0x885F710: libmetis__CreateCoarseGraphNoMask (coarsen.c:886)<br>
> ==32155== by 0x885E2B9: libmetis__CreateCoarseGraph (coarsen.c:636)<br>
> ==32155== by 0x885CA49: libmetis__Match_RM (coarsen.c:262)<br>
> ==32155== by 0x885BFBD: libmetis__CoarsenGraph (coarsen.c:55)<br>
> ==32155== by 0x8869A4C: libmetis__MultilevelBisect (pmetis.c:240)<br>
> ==32155== by 0x88696F6: libmetis__MlevelRecursiveBisection (pmetis.c:183)<br>
> ==32155== by 0x88698D7: libmetis__MlevelRecursiveBisection (pmetis.c:207)<br>
> ==32155== by 0x886951E: METIS_PartGraphRecursive (pmetis.c:133)<br>
> ==32155== by 0x885650B: libmetis__InitKWayPartitioning (kmetis.c:194)<br>
> ==32155== by 0x8856147: libmetis__MlevelKWayPartitioning (kmetis.c:121)<br>
> ==32155== by 0x8855FD3: METIS_PartGraphKway (kmetis.c:71)<br>
> ==32155== by 0x85E7611: libparmetis__PartitionSmallGraph (weird.c:478)<br>
> ==32155== by 0x85D00F3: ParMETIS_V3_PartKway (kmetis.c:91)<br>
> ==32155== by 0x5515C19: MatPartitioningApply_Parmetis_Private (pmetis.c:147)<br>
> ==32155== by 0x5516F7D: MatPartitioningApply_Parmetis (pmetis.c:221)<br>
> ==32155== by 0x550DD7F: MatPartitioningApply (partition.c:332)<br>
> ==32155== by 0x6387033: PCGAMGCreateLevel_GAMG (gamg.c:226)<br>
> ==32155== by 0x638BCA5: PCSetUp_GAMG (gamg.c:593)<br>
> ==32155== by 0x64746B1: PCSetUp (precon.c:894)<br>
> ==32155== by 0x65D388A: KSPSetUp (itfunc.c:377)<br>
> ==32155== <br>
> ==32155== Invalid read of size 4<br>
> ==32155== at 0x885F392: libmetis__CreateCoarseGraphNoMask (coarsen.c:856)<br>
> ==32155== by 0x885E2B9: libmetis__CreateCoarseGraph (coarsen.c:636)<br>
> ==32155== by 0x885CA49: libmetis__Match_RM (coarsen.c:262)<br>
> ==32155== by 0x885BFBD: libmetis__CoarsenGraph (coarsen.c:55)<br>
> ==32155== by 0x8869A4C: libmetis__MultilevelBisect (pmetis.c:240)<br>
> ==32155== by 0x88696F6: libmetis__MlevelRecursiveBisection (pmetis.c:183)<br>
> ==32155== by 0x88698D7: libmetis__MlevelRecursiveBisection (pmetis.c:207)<br>
> ==32155== by 0x886951E: METIS_PartGraphRecursive (pmetis.c:133)<br>
> ==32155== by 0x885650B: libmetis__InitKWayPartitioning (kmetis.c:194)<br>
> ==32155== by 0x8856147: libmetis__MlevelKWayPartitioning (kmetis.c:121)<br>
> ==32155== by 0x8855FD3: METIS_PartGraphKway (kmetis.c:71)<br>
> ==32155== by 0x85E7611: libparmetis__PartitionSmallGraph (weird.c:478)<br>
> ==32155== by 0x85D00F3: ParMETIS_V3_PartKway (kmetis.c:91)<br>
> ==32155== by 0x5515C19: MatPartitioningApply_Parmetis_Private (pmetis.c:147)<br>
> ==32155== by 0x5516F7D: MatPartitioningApply_Parmetis (pmetis.c:221)<br>
> ==32155== by 0x550DD7F: MatPartitioningApply (partition.c:332)<br>
> ==32155== by 0x6387033: PCGAMGCreateLevel_GAMG (gamg.c:226)<br>
> ==32155== by 0x638BCA5: PCSetUp_GAMG (gamg.c:593)<br>
> ==32155== by 0x64746B1: PCSetUp (precon.c:894)<br>
> ==32155== by 0x65D388A: KSPSetUp (itfunc.c:377)<br>
> ==32155== Address 0xee2502c is 0 bytes after a block of size 284 alloc'd<br>
> ==32155== at 0x4C2AB80: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)<br>
> ==32155== by 0x8820179: gk_malloc (memory.c:147)<br>
> ==32155== by 0x883D92B: libmetis__imalloc (gklib.c:24)<br>
> ==32155== by 0x885BF6A: libmetis__CoarsenGraph (coarsen.c:46)<br>
> ==32155== by 0x8869A4C: libmetis__MultilevelBisect (pmetis.c:240)<br>
> ==32155== by 0x88696F6: libmetis__MlevelRecursiveBisection (pmetis.c:183)<br>
> ==32155== by 0x88698D7: libmetis__MlevelRecursiveBisection (pmetis.c:207)<br>
> ==32155== by 0x886951E: METIS_PartGraphRecursive (pmetis.c:133)<br>
> ==32155== by 0x885650B: libmetis__InitKWayPartitioning (kmetis.c:194)<br>
> ==32155== by 0x8856147: libmetis__MlevelKWayPartitioning (kmetis.c:121)<br>
> ==32155== by 0x8855FD3: METIS_PartGraphKway (kmetis.c:71)<br>
> ==32155== by 0x85E7611: libparmetis__PartitionSmallGraph (weird.c:478)<br>
> ==32155== by 0x85D00F3: ParMETIS_V3_PartKway (kmetis.c:91)<br>
> ==32155== by 0x5515C19: MatPartitioningApply_Parmetis_Private (pmetis.c:147)<br>
> ==32155== by 0x5516F7D: MatPartitioningApply_Parmetis (pmetis.c:221)<br>
> ==32155== by 0x550DD7F: MatPartitioningApply (partition.c:332)<br>
> ==32155== by 0x6387033: PCGAMGCreateLevel_GAMG (gamg.c:226)<br>
> ==32155== by 0x638BCA5: PCSetUp_GAMG (gamg.c:593)<br>
> ==32155== by 0x64746B1: PCSetUp (precon.c:894)<br>
> ==32155== by 0x65D388A: KSPSetUp (itfunc.c:377)<br>
> ==32155== <br>
> ==32155== Invalid read of size 4<br>
> ==32155== at 0x885F62F: libmetis__CreateCoarseGraphNoMask (coarsen.c:879)<br>
> ==32155== by 0x885E2B9: libmetis__CreateCoarseGraph (coarsen.c:636)<br>
> ==32155== by 0x885D3CC: libmetis__Match_SHEM (coarsen.c:403)<br>
> ==32155== by 0x885BFD2: libmetis__CoarsenGraph (coarsen.c:57)<br>
> ==32155== by 0x8869A4C: libmetis__MultilevelBisect (pmetis.c:240)<br>
> ==32155== by 0x88696F6: libmetis__MlevelRecursiveBisection (pmetis.c:183)<br>
> ==32155== by 0x88698D7: libmetis__MlevelRecursiveBisection (pmetis.c:207)<br>
> ==32155== by 0x886951E: METIS_PartGraphRecursive (pmetis.c:133)<br>
> ==32155== by 0x885650B: libmetis__InitKWayPartitioning (kmetis.c:194)<br>
> ==32155== by 0x8856147: libmetis__MlevelKWayPartitioning (kmetis.c:121)<br>
> ==32155== by 0x8855FD3: METIS_PartGraphKway (kmetis.c:71)<br>
> ==32155== by 0x85E7611: libparmetis__PartitionSmallGraph (weird.c:478)<br>
> ==32155== by 0x85D00F3: ParMETIS_V3_PartKway (kmetis.c:91)<br>
> ==32155== by 0x5515C19: MatPartitioningApply_Parmetis_Private (pmetis.c:147)<br>
> ==32155== by 0x5516F7D: MatPartitioningApply_Parmetis (pmetis.c:221)<br>
> ==32155== by 0x550DD7F: MatPartitioningApply (partition.c:332)<br>
> ==32155== by 0x6387033: PCGAMGCreateLevel_GAMG (gamg.c:226)<br>
> ==32155== by 0x638BCA5: PCSetUp_GAMG (gamg.c:593)<br>
> ==32155== by 0x64746B1: PCSetUp (precon.c:894)<br>
> ==32155== by 0x65D388A: KSPSetUp (itfunc.c:377)<br>
> ==32155== Address 0xee27524 is 8 bytes after a block of size 76 alloc'd<br>
> ==32155== at 0x4C2AB80: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)<br>
> ==32155== by 0x8820179: gk_malloc (memory.c:147)<br>
> ==32155== by 0x883D92B: libmetis__imalloc (gklib.c:24)<br>
> ==32155== by 0x88607CF: libmetis__SetupCoarseGraph (coarsen.c:1107)<br>
> ==32155== by 0x885F16A: libmetis__CreateCoarseGraphNoMask (coarsen.c:826)<br>
> ==32155== by 0x885E2B9: libmetis__CreateCoarseGraph (coarsen.c:636)<br>
> ==32155== by 0x885D3CC: libmetis__Match_SHEM (coarsen.c:403)<br>
> ==32155== by 0x885BFD2: libmetis__CoarsenGraph (coarsen.c:57)<br>
> ==32155== by 0x8869A4C: libmetis__MultilevelBisect (pmetis.c:240)<br>
> ==32155== by 0x88696F6: libmetis__MlevelRecursiveBisection (pmetis.c:183)<br>
> ==32155== by 0x88698D7: libmetis__MlevelRecursiveBisection (pmetis.c:207)<br>
> ==32155== by 0x886951E: METIS_PartGraphRecursive (pmetis.c:133)<br>
> ==32155== by 0x885650B: libmetis__InitKWayPartitioning (kmetis.c:194)<br>
> ==32155== by 0x8856147: libmetis__MlevelKWayPartitioning (kmetis.c:121)<br>
> ==32155== by 0x8855FD3: METIS_PartGraphKway (kmetis.c:71)<br>
> ==32155== by 0x85E7611: libparmetis__PartitionSmallGraph (weird.c:478)<br>
> ==32155== by 0x85D00F3: ParMETIS_V3_PartKway (kmetis.c:91)<br>
> ==32155== by 0x5515C19: MatPartitioningApply_Parmetis_Private (pmetis.c:147)<br>
> ==32155== by 0x5516F7D: MatPartitioningApply_Parmetis (pmetis.c:221)<br>
> ==32155== by 0x550DD7F: MatPartitioningApply (partition.c:332)<br>
> ==32155== <br>
> ==32155== Invalid read of size 4<br>
> ==32155== at 0x885F392: libmetis__CreateCoarseGraphNoMask (coarsen.c:856)<br>
> ==32155== by 0x885E2B9: libmetis__CreateCoarseGraph (coarsen.c:636)<br>
> ==32155== by 0x885D3CC: libmetis__Match_SHEM (coarsen.c:403)<br>
> ==32155== by 0x885BFD2: libmetis__CoarsenGraph (coarsen.c:57)<br>
> ==32155== by 0x8869A4C: libmetis__MultilevelBisect (pmetis.c:240)<br>
> ==32155== by 0x88696F6: libmetis__MlevelRecursiveBisection (pmetis.c:183)<br>
> ==32155== by 0x88698D7: libmetis__MlevelRecursiveBisection (pmetis.c:207)<br>
> ==32155== by 0x886951E: METIS_PartGraphRecursive (pmetis.c:133)<br>
> ==32155== by 0x885650B: libmetis__InitKWayPartitioning (kmetis.c:194)<br>
> ==32155== by 0x8856147: libmetis__MlevelKWayPartitioning (kmetis.c:121)<br>
> ==32155== by 0x8855FD3: METIS_PartGraphKway (kmetis.c:71)<br>
> ==32155== by 0x85E7611: libparmetis__PartitionSmallGraph (weird.c:478)<br>
> ==32155== by 0x85D00F3: ParMETIS_V3_PartKway (kmetis.c:91)<br>
> ==32155== by 0x5515C19: MatPartitioningApply_Parmetis_Private (pmetis.c:147)<br>
> ==32155== by 0x5516F7D: MatPartitioningApply_Parmetis (pmetis.c:221)<br>
> ==32155== by 0x550DD7F: MatPartitioningApply (partition.c:332)<br>
> ==32155== by 0x6387033: PCGAMGCreateLevel_GAMG (gamg.c:226)<br>
> ==32155== by 0x638BCA5: PCSetUp_GAMG (gamg.c:593)<br>
> ==32155== by 0x64746B1: PCSetUp (precon.c:894)<br>
> ==32155== by 0x65D388A: KSPSetUp (itfunc.c:377)<br>
> ==32155== Address 0xee274cc is 4 bytes before a block of size 76 alloc'd<br>
> ==32155== at 0x4C2AB80: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)<br>
> ==32155== by 0x8820179: gk_malloc (memory.c:147)<br>
> ==32155== by 0x883D92B: libmetis__imalloc (gklib.c:24)<br>
> ==32155== by 0x88607CF: libmetis__SetupCoarseGraph (coarsen.c:1107)<br>
> ==32155== by 0x885F16A: libmetis__CreateCoarseGraphNoMask (coarsen.c:826)<br>
> ==32155== by 0x885E2B9: libmetis__CreateCoarseGraph (coarsen.c:636)<br>
> ==32155== by 0x885D3CC: libmetis__Match_SHEM (coarsen.c:403)<br>
> ==32155== by 0x885BFD2: libmetis__CoarsenGraph (coarsen.c:57)<br>
> ==32155== by 0x8869A4C: libmetis__MultilevelBisect (pmetis.c:240)<br>
> ==32155== by 0x88696F6: libmetis__MlevelRecursiveBisection (pmetis.c:183)<br>
> ==32155== by 0x88698D7: libmetis__MlevelRecursiveBisection (pmetis.c:207)<br>
> ==32155== by 0x886951E: METIS_PartGraphRecursive (pmetis.c:133)<br>
> ==32155== by 0x885650B: libmetis__InitKWayPartitioning (kmetis.c:194)<br>
> ==32155== by 0x8856147: libmetis__MlevelKWayPartitioning (kmetis.c:121)<br>
> ==32155== by 0x8855FD3: METIS_PartGraphKway (kmetis.c:71)<br>
> ==32155== by 0x85E7611: libparmetis__PartitionSmallGraph (weird.c:478)<br>
> ==32155== by 0x85D00F3: ParMETIS_V3_PartKway (kmetis.c:91)<br>
> ==32155== by 0x5515C19: MatPartitioningApply_Parmetis_Private (pmetis.c:147)<br>
> ==32155== by 0x5516F7D: MatPartitioningApply_Parmetis (pmetis.c:221)<br>
> ==32155== by 0x550DD7F: MatPartitioningApply (partition.c:332)<br>
> ==32155== <br>
> <br>
> <br>
> On Sun, Nov 10, 2019 at 2:24 PM Mark Adams <<a href="mailto:mfadams@lbl.gov" target="_blank">mfadams@lbl.gov</a>> wrote:<br>
> Fande, It looks to me like this branch in ParMetis must be taken to trigger this error. First Match_SHEM and then CreateCoarseGraphNoMask.<br>
> <br>
> /* determine which matching scheme you will use */<br>
> switch (ctrl->ctype) {<br>
> case METIS_CTYPE_RM:<br>
> Match_RM(ctrl, graph);<br>
> break;<br>
> case METIS_CTYPE_SHEM:<br>
> if (eqewgts || graph->nedges == 0)<br>
> Match_RM(ctrl, graph);<br>
> else<br>
> Match_SHEM(ctrl, graph);<br>
> break;<br>
> default:<br>
> gk_errexit(SIGERR, "Unknown ctype: %d\n", ctrl->ctype);<br>
> }<br>
> <br>
> -----------------------<br>
> <br>
> /* Check if the mask-version of the code is a good choice */<br>
> mask = HTLENGTH;<br>
> if (cnvtxs < 2*mask || graph->nedges/graph->nvtxs > mask/20) { <br>
> CreateCoarseGraphNoMask(ctrl, graph, cnvtxs, match);<br>
> return;<br>
> }<br>
> <br>
> ------------<br>
> <br>
> The actual error is in CreateCoarseGraphNoMask, graph->cmap is too small and this gets garbage. parmetis coarsen.c:856:<br>
> <br>
> istart = xadj[v];<br>
> iend = xadj[v+1];<br>
> for (j=istart; j<iend; j++) {<br>
> k = cmap[adjncy[j]];<br>
> if ((m = htable[k]) == -1) {<br>
> cadjncy[nedges] = k;<br>
> <br>
> Anyway, this is what I have so far,<br>
> Mark<br>
> <br>
> <br>
> On Sun, Nov 10, 2019 at 10:34 AM Mark Adams <<a href="mailto:mfadams@lbl.gov" target="_blank">mfadams@lbl.gov</a>> wrote:<br>
> Fande, the problem is k below seems to index beyond the end of htable, resulting in a crazy m and a segv on the last line below.<br>
> <br>
> I don't have a clean valgrind machine now, that is what is needed if no one has seen anything like this. I could add a test in a MR and get the pipeline to do it.<br>
> <br>
> void CreateCoarseGraphNoMask(ctrl_t *ctrl, graph_t *graph, idx_t cnvtxs, <br>
> idx_t *match)<br>
> {<br>
> idx_t j, k, m, istart, iend, nvtxs, nedges, ncon, cnedges, v, u, dovsize;<br>
> idx_t *xadj, *vwgt, *vsize, *adjncy, *adjwgt;<br>
> idx_t *cmap, *htable;<br>
> idx_t *cxadj, *cvwgt, *cvsize, *cadjncy, *cadjwgt;<br>
> graph_t *cgraph;<br>
> ine <br>
> WCOREPUSH;<br>
> <br>
> dovsize = (ctrl->objtype == METIS_OBJTYPE_VOL ? 1 : 0);<br>
> <br>
> IFSET(ctrl->dbglvl, METIS_DBG_TIME, gk_startcputimer(ctrl->ContractTmr));<br>
> <br>
> nvtxs = graph->nvtxs;<br>
> ncon = graph->ncon;<br>
> xadj = graph->xadj;<br>
> vwgt = graph->vwgt;<br>
> vsize = graph->vsize;<br>
> adjncy = graph->adjncy;<br>
> adjwgt = graph->adjwgt;<br>
> cmap = graph->cmap;<br>
> <br>
> <br>
> /* Initialize the coarser graph */<br>
> cgraph = SetupCoarseGraph(graph, cnvtxs, dovsize);<br>
> cxadj = cgraph->xadj;<br>
> cvwgt = cgraph->vwgt;<br>
> cvsize = cgraph->vsize;<br>
> cadjncy = cgraph->adjncy;<br>
> cadjwgt = cgraph->adjwgt;<br>
> <br>
> htable = iset(cnvtxs, -1, iwspacemalloc(ctrl, cnvtxs));<br>
> <br>
> cxadj[0] = cnvtxs = cnedges = 0;<br>
> for (v=0; v<nvtxs; v++) {<br>
> if ((u = match[v]) < v)<br>
> continue;<br>
> <br>
> ASSERT(cmap[v] == cnvtxs);<br>
> ASSERT(cmap[match[v]] == cnvtxs);<br>
> <br>
> if (ncon == 1)<br>
> cvwgt[cnvtxs] = vwgt[v];<br>
> else<br>
> icopy(ncon, vwgt+v*ncon, cvwgt+cnvtxs*ncon);<br>
> <br>
> if (dovsize)<br>
> cvsize[cnvtxs] = vsize[v];<br>
> <br>
> nedges = 0;<br>
> <br>
> istart = xadj[v];<br>
> iend = xadj[v+1];<br>
> for (j=istart; j<iend; j++) {<br>
> k = cmap[adjncy[j]];<br>
> if ((m = htable[k]) == -1) {<br>
> cadjncy[nedges] = k;<br>
> cadjwgt[nedges] = adjwgt[j];<br>
> htable[k] = nedges++;<br>
> }<br>
> else {<br>
> cadjwgt[m] += adjwgt[j];<br>
> <br>
> On Sun, Nov 10, 2019 at 1:35 AM Mark Adams <<a href="mailto:mfadams@lbl.gov" target="_blank">mfadams@lbl.gov</a>> wrote:<br>
> <br>
> <br>
> On Sat, Nov 9, 2019 at 10:51 PM Fande Kong <<a href="mailto:fdkong.jd@gmail.com" target="_blank">fdkong.jd@gmail.com</a>> wrote:<br>
> Hi Mark,<br>
> <br>
> Thanks for reporting this bug. I was surprised because we have sufficient heavy tests in moose using partition weights and do not have any issue so far.<br>
> <br>
> <br>
> I have been pounding on this code with elasticity and have not seen this issue. I am now looking at Lapacianas and I only see it with pretty large problems. The example below is pretty minimal (eg, it works with 16 cores and it works with -dm_refine 4). I have reproduced this on Cori, SUMMIT and my laptop.<br>
> <br>
> I will take a shot on this.<br>
> <br>
> Thanks, I'll try to take a look at it also. I have seen it in DDT, but did not dig further. It looked like a typical segv in ParMetis.<br>
> <br>
> <br>
> Fande,<br>
> <br>
> On Sat, Nov 9, 2019 at 3:08 PM Mark Adams <<a href="mailto:mfadams@lbl.gov" target="_blank">mfadams@lbl.gov</a>> wrote:<br>
> snes/ex13 is getting a ParMetis segv with GAMG and coarse grid repartitioning. Below shows the branch and how to run it.<br>
> <br>
> I've tried valgrind on Cori but it gives a lot of false positives. I've seen this error in DDT but I have not had a chance to dig and try to fix it. At least I know it has something to do with weights.<br>
> <br>
> If anyone wants to take a shot at it feel free. This bug rarely happens.<br>
> <br>
> The changes use weights and are just a few lines of code (from 1.5 years ago):<br>
> <br>
> 12:08 (0455fb9fec...)|BISECTING ~/Codes/petsc$ git bisect bad<br>
> 0455fb9fecf69cf5cf35948c84d3837e5a427e2e is the first bad commit<br>
> commit 0455fb9fecf69cf5cf35948c84d3837e5a427e2e<br>
> Author: Fande Kong <<a href="mailto:fdkong.jd@gmail.com" target="_blank">fdkong.jd@gmail.com</a>><br>
> Date: Thu Jun 21 18:21:19 2018 -0600<br>
> <br>
> Let parmetis and ptsotch take edge weights and vertex weights<br>
> <br>
> src/mat/partition/impls/pmetis/pmetis.c | 7 +++++++<br>
> src/mat/partition/impls/scotch/scotch.c | 6 +++---<br>
> 2 files changed, 10 insertions(+), 3 deletions(-)<br>
> <br>
> > mpiexec -n 32 ./ex13 -cells 2,4,4, -dm_refine 5 -simplex 0 -dim 3 -potential_petscspace_degree 1 -potential_petscspace_order 1 -pc_type gamg -petscpartitioner_type simple -pc_gamg_repartition true -check_pointer_intensity 0<br>
<br>
</blockquote></div>