<html><head><meta http-equiv="Content-Type" content="text/html; charset=us-ascii"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div class=""><br class=""></div> valgrind?<br class=""><div><br class=""><blockquote type="cite" class=""><div class="">On Jul 14, 2020, at 11:29 AM, Fande Kong <<a href="mailto:fdkong.jd@gmail.com" class="">fdkong.jd@gmail.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class=""><div dir="ltr" class="">The petsc configuration log was attached.</div><div dir="ltr" class=""><br class=""></div><div dir="ltr" class="">Thanks,</div><div dir="ltr" class=""><br class=""></div><div dir="ltr" class="">Fande,<br class=""><div class=""><br class=""></div><div class=""><br class=""></div></div></div><br class=""><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Jul 14, 2020 at 9:16 AM Fande Kong <<a href="mailto:fdkong.jd@gmail.com" class="">fdkong.jd@gmail.com</a>> wrote:<br class=""></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div dir="ltr" class=""><div dir="ltr" class="">Hi All,</div><div dir="ltr" class=""><br class=""></div><div dir="ltr" class=""><br class=""></div><div class="">I was doing a large-scale simulation using 12288 cores and had the following error. The code ran fine using less than 12288 cores.</div><div class=""><br class=""></div><div class="">Any quick suggestions to track down this issue?</div><div class=""><br class=""></div><div class="">Thanks,</div><div class=""><br class=""></div><div class="">Fande,</div><div dir="ltr" class=""><br class=""><div class=""><br class=""></div><div class=""><div class="">[3342]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------</div><div class="">[3342]PETSC ERROR: Petsc has generated inconsistent data</div><div class="">[3342]PETSC ERROR: Received vector entry 0 out of local range [344829312,344964096)]</div><div class="">[3342]PETSC ERROR: See <a href="https://www.mcs.anl.gov/petsc/documentation/faq.html" target="_blank" class="">https://www.mcs.anl.gov/petsc/documentation/faq.html</a> for trouble shooting.</div><div class="">[3342]PETSC ERROR: Petsc Release Version 3.13.3, unknown </div><div class="">[3342]PETSC ERROR: /home/kongf/workhome/sawtooth/griffin/griffin-opt on a arch-moose named r1i4n34 by kongf Tue Jul 14 08:44:02 2020</div><div class="">[3342]PETSC ERROR: Configure options --download-hypre=1 --with-debugging=no --with-shared-libraries=1 --download-fblaslapack=1 --download-metis=1 --download-ptscotch=1 --download-parmetis=1 --download-superlu_dist=1 --download-mumps=1 --download-scalapack=1 --download-slepc=1 --with-mpi=1 --with-cxx-dialect=C++11 --with-fortran-bindings=0 --with-sowing=0 --with-64-bit-indices --download-mumps=0</div><div class="">[3342]PETSC ERROR: #1 VecAssemblyEnd_MPI_BTS() line 324 in /home/kongf/workhome/sawtooth/moosers/petsc/src/vec/vec/impls/mpi/pbvec.c</div><div class="">[3342]PETSC ERROR: #2 VecAssemblyEnd() line 171 in /home/kongf/workhome/sawtooth/moosers/petsc/src/vec/vec/interface/vector.c</div><div class="">[cli_3342]: aborting job:</div><div class="">application called MPI_Abort(MPI_COMM_WORLD, 1) - process 3342</div></div></div></div>
</blockquote></div>
<span id="cid:f_kcm5hlvv0"><configure.log.zip></span></div></blockquote></div><br class=""></body></html>