<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"Malgun Gothic";
panose-1:2 11 5 3 2 0 0 2 0 4;}
@font-face
{font-family:Menlo;
panose-1:2 11 6 9 3 8 4 2 2 4;}
@font-face
{font-family:"\@Malgun Gothic";}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
span.apple-converted-space
{mso-style-name:apple-converted-space;}
span.s1
{mso-style-name:s1;}
span.EmailStyle20
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body lang="EN-US" link="blue" vlink="purple" style="word-wrap:break-word;-webkit-nbsp-mode: space;line-break:after-white-space">
<div class="WordSection1">
<p class="MsoNormal">HI,<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Jenn: Perhaps, we need to have an option to use or not use SNESSetConvergenceTest. I think PFLOTRAN is set to use that function as default no matter what.<o:p></o:p></p>
<p class="MsoNormal">Barry: Thank you for your answers. Let me look into the second case deeper. I will have to test this in different machines and debug to see what is causing the issue.
<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="color:black">- Heeho Daniel Park</span><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:12.0pt;color:black">From: </span></b><span style="font-size:12.0pt;color:black">Barry Smith <bsmith@petsc.dev><br>
<b>Date: </b>Wednesday, February 16, 2022 at 4:06 PM<br>
<b>To: </b>"Park, Heeho" <heepark@sandia.gov><br>
<b>Cc: </b>"petsc-users@mcs.anl.gov" <petsc-users@mcs.anl.gov>, "LaForce, Tara" <tlaforc@sandia.gov><br>
<b>Subject: </b>[EXTERNAL] Re: [petsc-users] Changes in SNES convergence criteria<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal"><br>
<br>
<o:p></o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal">On Feb 16, 2022, at 6:28 PM, Park, Heeho via petsc-users <<a href="mailto:petsc-users@mcs.anl.gov">petsc-users@mcs.anl.gov</a>> wrote:<o:p></o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">Hi Petsc developers,<span style="font-size:12.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"> <span style="font-size:12.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal">I’m running into two issues related to SNES convergence criteria. First is setting convergence criteria to a large value. This used to work to effectively turn off RTOL, STOL, ATOL, and use PFLOTRAN’s Newton iteration convergence criteria
using INF norms, but this trick does not work anymore ( example 1 ).<span style="font-size:12.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal">So, we set the condition to PETSc convergence criteria (default) AND PFLOTRAN convergence criteria to declare convergence, and get this error ( example 2).<span style="font-size:12.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal">Is there a good way to bypass SNES convergence criteria and use our own?<span style="font-size:12.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal">What’s happening in the second example?<span style="font-size:12.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"> <span style="font-size:12.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal">Thank you,<span style="font-size:12.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"> <span style="font-size:12.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal">Example 1:<span style="font-size:12.0pt"><o:p></o:p></span></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal"> Do you use SNESSetConvergenceTest? If so, then unless your test uses the PETSc SNES object tolerance values (unlikely), you don't need to change the convergence tolerance values. So you can just not set it to that huge value or any particular
value.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><br>
<br>
<o:p></o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal"><span style="font-size:8.5pt;font-family:Menlo">[0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------</span><span style="font-size:12.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:8.5pt;font-family:Menlo">[0]PETSC ERROR: Argument out of range</span><span style="font-size:12.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:8.5pt;font-family:Menlo">[0]PETSC ERROR: Relative tolerance 1e+20 must be non-negative and less than 1.0</span><span style="font-size:12.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:8.5pt;font-family:Menlo">[0]PETSC ERROR: See<span class="apple-converted-space"> </span><a href="https://petsc.org/release/faq/"><span style="color:#00006A">https://petsc.org/release/faq/</span></a><span class="apple-converted-space"> </span>for
trouble shooting.</span><span style="font-size:12.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:8.5pt;font-family:Menlo">[0]PETSC ERROR: Petsc Development GIT revision: v3.16.4-756-g5b3d650 GIT Date: 2022-02-02 17:48:52 +0000</span><span style="font-size:12.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:8.5pt;font-family:Menlo">[0]PETSC ERROR: /data/home/heepark/software/pflotran-bcnew-trd/src/pflotran/pflotran on a intel-c-opt named cb007 by tlaforc</span><span style="font-size:12.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:8.5pt;font-family:Menlo"> Tue Feb 15 19:45:39 2022</span><span style="font-size:12.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:8.5pt;font-family:Menlo">[0]PETSC ERROR: Configure options --with-petsc-arch=intel-c-opt --with-cc=mpicc --with-cxx=mpicxx --with-fc=mpifort --with-c</span><span style="font-size:12.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:8.5pt;font-family:Menlo">language=c --with-shared-libraries=0 --with-debugging=0 --with-mpi=yes --download-sowing=yes --with-valgrind=yes --with-cmak</span><span style="font-size:12.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:8.5pt;font-family:Menlo">e=yes --with-make=yes --with-fblaslaback=yes --with-blaslapack=yes --FFLAGS="-diag-disable 5462" --CFLAGS="-O3 -g" --CXXFLAG</span><span style="font-size:12.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:8.5pt;font-family:Menlo">S="-O3 -g" --FFLAGS="-O3 -g" --with-hdf5-include=/data/home/heepark/software/hdf5/build/include --with-hdf5-lib="-L/data/hom</span><span style="font-size:12.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:8.5pt;font-family:Menlo">e/heepark/software/hdf5/build/lib -lhdf5hl_fortran -lhdf5_fortran -lhdf5_hl -lhdf5" --with-hypre-include=/data/home/heepark/</span><span style="font-size:12.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:8.5pt;font-family:Menlo">software/hypre/build/include --with-hypre-lib="-L/data/home/heepark/software/hypre/build/lib -lHYPRE" --with-metis-include=/</span><span style="font-size:12.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:8.5pt;font-family:Menlo">data/home/heepark/software/metis/include --with-metis-lib=/data/home/heepark/software/metis/build/libmetis/libmetis.a --with</span><span style="font-size:12.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:8.5pt;font-family:Menlo">-parmetis-include=/data/home/heepark/software/parmetis/include --with-parmetis-lib=/data/home/heepark/software/parmetis/buil</span><span style="font-size:12.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:8.5pt;font-family:Menlo">d/libparmetis/libparmetis.a --with-scalapack-lib=/data/home/heepark/software/scalapack/build/lib/libscalapack.a --with-hdf5-</span><span style="font-size:12.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:8.5pt;font-family:Menlo">fortran-bindings=yes</span><span style="font-size:12.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:8.5pt;font-family:Menlo">[0]PETSC ERROR: #1 SNESSetTolerances() at /data/home/heepark/software/petsc-main/src/snes/interface/snes.c:3833</span><span style="font-size:12.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:8.5pt;font-family:Menlo">[0]PETSC ERROR: #2 User provided function() at User file:0</span><span style="font-size:12.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"> <span style="font-size:12.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal">Example 2:<span style="font-size:12.0pt"><o:p></o:p></span></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal"> This should never be happening. Perhaps it is caused by memory corruption. Can you run with valgrind to verify no memory issues. If valgrind does not work on your system you can try with the PETSc option -malloc_debug <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><br>
<br>
<o:p></o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal"><span class="s1"><span style="font-size:8.5pt;font-family:Menlo">[0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------</span></span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span class="s1"><span style="font-size:8.5pt;font-family:Menlo">[0]PETSC ERROR: Argument out of range</span></span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span class="s1"><span style="font-size:8.5pt;font-family:Menlo">[0]PETSC ERROR: toranks[39] 40 not in comm size 40</span></span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span class="s1"><span style="font-size:8.5pt;font-family:Menlo">[0]PETSC ERROR: See</span></span><span class="apple-converted-space"><span style="font-size:8.5pt;font-family:Menlo"> </span></span><span class="s1"><span style="font-size:8.5pt;font-family:Menlo"><a href="https://petsc.org/release/faq/" title="https://petsc.org/release/faq/"><span style="color:#00006A">https://petsc.org/release/faq/</span></a></span></span><span class="apple-converted-space"><span style="font-size:8.5pt;font-family:Menlo"> </span></span><span class="s1"><span style="font-size:8.5pt;font-family:Menlo">for
trouble shooting.</span></span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span class="s1"><span style="font-size:8.5pt;font-family:Menlo">[0]PETSC ERROR: Petsc Development GIT revision: v3.16.4-756-g5b3d650</span></span><span class="apple-converted-space"><span style="font-size:8.5pt;font-family:Menlo"> </span></span><span class="s1"><span style="font-size:8.5pt;font-family:Menlo">GIT
Date: 2022-02-02 17:48:52 +0000</span></span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span class="s1"><span style="font-size:8.5pt;font-family:Menlo">[0]PETSC ERROR: /data/home/heepark/software/pflotran-bcnew-trd/src/pflotran/pflotran on a intel-c-opt named cb007 by tlaforc Wed Feb 16 14:13:12 2022</span></span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span class="s1"><span style="font-size:8.5pt;font-family:Menlo">[0]PETSC ERROR: Configure options --with-petsc-arch=intel-c-opt --with-cc=mpicc --with-cxx=mpicxx --with-fc=mpifort --with-clanguage=c --with-shared-libraries=0 --with-debugging=0
--with-mpi=yes --download-sowing=yes --with-valgrind=yes --with-cmake=yes --with-make=yes --with-fblaslaback=yes --with-blaslapack=yes --FFLAGS="-diag-disable 5462" --CFLAGS="-O3 -g" --CXXFLAGS="-O3 -g" --FFLAGS="-O3 -g" --with-hdf5-include=/data/home/heepark/software/hdf5/build/include
--with-hdf5-lib="-L/data/home/heepark/software/hdf5/build/lib -lhdf5hl_fortran -lhdf5_fortran -lhdf5_hl -lhdf5" --with-hypre-include=/data/home/heepark/software/hypre/build/include --with-hypre-lib="-L/data/home/heepark/software/hypre/build/lib -lHYPRE" --with-metis-include=/data/home/heepark/software/metis/include
--with-metis-lib=/data/home/heepark/software/metis/build/libmetis/libmetis.a --with-parmetis-include=/data/home/heepark/software/parmetis/include --with-parmetis-lib=/data/home/heepark/software/parmetis/build/libparmetis/libparmetis.a --with-scalapack-lib=/data/home/heepark/software/scalapack/build/lib/libscalapack.a
--with-hdf5-fortran-bindings=yes</span></span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span class="s1"><span style="font-size:8.5pt;font-family:Menlo">[0]PETSC ERROR: #1 PetscCommBuildTwoSidedFReq() at /data/home/heepark/software/petsc-main/src/sys/utils/mpits.c:555</span></span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span class="s1"><span style="font-size:8.5pt;font-family:Menlo">[0]PETSC ERROR: #2 MatStashScatterBegin_BTS() at /data/home/heepark/software/petsc-main/src/mat/utils/matstash.c:940</span></span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span class="s1"><span style="font-size:8.5pt;font-family:Menlo">[0]PETSC ERROR: #3 MatStashScatterBegin_Private() at /data/home/heepark/software/petsc-main/src/mat/utils/matstash.c:461</span></span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span class="s1"><span style="font-size:8.5pt;font-family:Menlo">[0]PETSC ERROR: #4 MatAssemblyBegin_MPIAIJ() at /data/home/heepark/software/petsc-main/src/mat/impls/aij/mpi/mpiaij.c:673</span></span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span class="s1"><span style="font-size:8.5pt;font-family:Menlo">[0]PETSC ERROR: #5 MatAssemblyBegin() at /data/home/heepark/software/petsc-main/src/mat/interface/matrix.c:5592</span></span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span class="s1"><span style="font-size:8.5pt;font-family:Menlo">[0]PETSC ERROR: #6 User provided function() at User file:0</span></span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span class="s1"><span style="font-size:8.5pt;font-family:Menlo">[0]PETSC ERROR: #7 oursnesjacobian() at /data/home/heepark/software/petsc-main/src/snes/interface/ftn-custom/zsnesf.c:173</span></span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span class="s1"><span style="font-size:8.5pt;font-family:Menlo">[0]PETSC ERROR: #8 SNESComputeJacobian() at /data/home/heepark/software/petsc-main/src/snes/interface/snes.c:2864</span></span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span class="s1"><span style="font-size:8.5pt;font-family:Menlo">[0]PETSC ERROR: #9 SNESSolve_NEWTONLS() at /data/home/heepark/software/petsc-main/src/snes/impls/ls/ls.c:222</span></span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span class="s1"><span style="font-size:8.5pt;font-family:Menlo">[0]PETSC ERROR: #10 SNESSolve() at /data/home/heepark/software/petsc-main/src/snes/interface/snes.c:4810</span></span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span class="s1"><span style="font-size:8.5pt;font-family:Menlo">[0]PETSC ERROR: #11 User provided function() at User file:0</span></span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <span style="font-size:12.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"> <span style="font-size:12.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal">Heeho Daniel Park<br>
<br>
! ------------------------------------ !<br>
Sandia National Laboratories<span style="font-size:12.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal">Org: 08844, R&D<span style="font-size:12.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal">Work: 505-844-1319<br>
! ------------------------------------ !<span style="font-size:12.0pt"><o:p></o:p></span></p>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><br>
<br>
<o:p></o:p></p>
</div>
</body>
</html>