<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body>
<div dir="ltr">Yes, it does. It is a bug.<br clear="all">
<div>
<div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature">
<div dir="ltr">--Junchao Zhang</div>
</div>
</div>
<br>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Thu, Mar 21, 2019 at 11:16 AM Balay, Satish <<a href="mailto:balay@mcs.anl.gov">balay@mcs.anl.gov</a>> wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
Does maint also need this fix?<br>
<br>
Satish<br>
<br>
On Thu, 21 Mar 2019, Stefano Zampini via petsc-users wrote:<br>
<br>
> Derek<br>
> <br>
> I have fixed the optimized plan few weeks ago<br>
> <br>
> <a href="https://bitbucket.org/petsc/petsc/commits/c3caad8634d376283f7053f3b388606b45b3122c" rel="noreferrer" target="_blank">
https://bitbucket.org/petsc/petsc/commits/c3caad8634d376283f7053f3b388606b45b3122c</a><br>
> <br>
> Maybe this will fix your problem too?<br>
> <br>
> Stefano<br>
> <br>
> <br>
> Il Gio 21 Mar 2019, 04:21 Zhang, Junchao via petsc-users <<br>
> <a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a>> ha scritto:<br>
> <br>
> > Hi, Derek,<br>
> > Try to apply this tiny (but dirty) patch on your version of PETSc to<br>
> > disable the VecScatterMemcpyPlan optimization to see if it helps.<br>
> > Thanks.<br>
> > --Junchao Zhang<br>
> ><br>
> > On Wed, Mar 20, 2019 at 6:33 PM Junchao Zhang <<a href="mailto:jczhang@mcs.anl.gov" target="_blank">jczhang@mcs.anl.gov</a>> wrote:<br>
> ><br>
> >> Did you see the warning with small scale runs? Is it possible to provide<br>
> >> a test code?<br>
> >> You mentioned "changing PETSc now would be pretty painful". Is it because<br>
> >> it will affect your performance (but not your code)? If yes, could you try<br>
> >> PETSc master and run you code with or without -vecscatter_type sf. I want<br>
> >> to isolate the problem and see if it is due to possible bugs in VecScatter.<br>
> >> If the above suggestion is not feasible, I will disable VecScatterMemcpy.<br>
> >> It is an optimization I added. Sorry I did not have an option to turn off<br>
> >> it because I thought it was always useful:) I will provide you a patch<br>
> >> later to disable it. With that you can run again to isolate possible bugs<br>
> >> in VecScatterMemcpy.<br>
> >> Thanks.<br>
> >> --Junchao Zhang<br>
> >><br>
> >><br>
> >> On Wed, Mar 20, 2019 at 5:40 PM Derek Gaston via petsc-users <<br>
> >> <a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a>> wrote:<br>
> >><br>
> >>> Trying to track down some memory corruption I'm seeing on larger scale<br>
> >>> runs (3.5B+ unknowns). Was able to run Valgrind on it... and I'm seeing<br>
> >>> quite a lot of uninitialized value errors coming from ghost updating. Here<br>
> >>> are some of the traces:<br>
> >>><br>
> >>> ==87695== Conditional jump or move depends on uninitialised value(s)<br>
> >>> ==87695== at 0x73236D3: PetscMallocAlign (mal.c:28)<br>
> >>> ==87695== by 0x7323C70: PetscMallocA (mal.c:390)<br>
> >>> ==87695== by 0x739048E: VecScatterMemcpyPlanCreate_Index (vscat.c:284)<br>
> >>> ==87695== by 0x73A5D97: VecScatterMemcpyPlanCreate_PtoP<br>
> >>> (vpscat_mpi1.c:312)<br>
> >>> ==64730== by 0x7393E8A: VecScatterSetUp_vectype_private (vscat.c:857)<br>
> >>> ==64730== by 0x7395E5D: VecScatterSetUp_MPI1 (vpscat_mpi1.c:2543)<br>
> >>> ==64730== by 0x73DDD39: VecScatterSetUp (vscatfce.c:212)<br>
> >>> ==64730== by 0x73DCD73: VecScatterCreateWithData (vscreate.c:333)<br>
> >>> ==64730== by 0x7444232: VecCreateGhostWithArray (pbvec.c:685)<br>
> >>> ==64730== by 0x744490D: VecCreateGhost (pbvec.c:741)<br>
> >>><br>
> >>> ==133582== Conditional jump or move depends on uninitialised value(s)<br>
> >>> ==133582== at 0x4030384: memcpy@@GLIBC_2.14<br>
> >>> (vg_replace_strmem.c:1034)<br>
> >>> ==133582== by 0x739E4F9: PetscMemcpy (petscsys.h:1649)<br>
> >>> ==133582== by 0x739E4F9: VecScatterMemcpyPlanExecute_Pack<br>
> >>> (vecscatterimpl.h:150)<br>
> >>> ==133582== by 0x739E4F9: VecScatterBeginMPI1_1 (vpscat_mpi1.h:69)<br>
> >>> ==133582== by 0x73DD964: VecScatterBegin (vscatfce.c:110)<br>
> >>> ==133582== by 0x744E195: VecGhostUpdateBegin (commonmpvec.c:225)<br>
> >>><br>
> >>> This is from a Git checkout of PETSc... the hash I branched from is:<br>
> >>> 0e667e8fea4aa from December 23rd (updating would be really hard at this<br>
> >>> point as I've completed 90% of my dissertation with this version... and<br>
> >>> changing PETSc now would be pretty painful!).<br>
> >>><br>
> >>> Any ideas? Is it possible it's in my code? Is it possible that there<br>
> >>> are later PETSc commits that already fix this?<br>
> >>><br>
> >>> Thanks for any help,<br>
> >>> Derek<br>
> >>><br>
> >>><br>
> <br>
<br>
</blockquote>
</div>
</body>
</html>