<div dir="ltr"><div dir="ltr">On Tue, Sep 10, 2024 at 1:16 PM Karthikeyan Chockalingam - STFC UKRI <<a href="mailto:karthikeyan.chockalingam@stfc.ac.uk">karthikeyan.chockalingam@stfc.ac.uk</a>> wrote:<br></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div class="msg5103815541450882016">
<div lang="EN-GB" style="overflow-wrap: break-word;">
<div class="m_5103815541450882016WordSection1">
<p class="MsoNormal"><span style="font-size:11pt">Hi Matt,<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt">I am not sure, if I understand how to read the source code, let’s take the below line<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt">PetscCall(PetscOptionsReal("-pc_hypre_ams_relax_weight", "Relaxation weight for AMS smoother", "None", jac->as_relax_weight, &jac->as_relax_weight, &flag3))<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt">(Q1) Am I doing it right by setting as in the line below? How does the line below match up to the above? Since the above
line has more arguments <u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt">petscErr = PetscOptionsSetValue(NULL,"-pc_hypre_ams_relax_weight", "1.0")</span></p></div></div></div></blockquote><div><br></div><div>Yes. All options take a single string argument. The other arguments to the function help present it to the user (say through a GUI).</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div class="msg5103815541450882016"><div lang="EN-GB" style="overflow-wrap: break-word;"><div class="m_5103815541450882016WordSection1"><p class="MsoNormal"><span style="font-size:11pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt">(Q2)
</span><span style="font-size:11pt">How do I go about setting the four parameters below using PetscOptionsSetValue?</span></p></div></div></div></blockquote><div><br></div><div>It looks like you have 5 parameters, and you would call</div><div><br></div><div><span style="font-size:14.6667px">petscErr = PetscOptionsSetValue(NULL,"-</span><span style="font-size:14.6667px">pc_hypre_ams_amg_alpha_options", "10,21,32,43,54")</span><br></div><div><span style="font-size:14.6667px"><br></span></div><div><span style="font-size:14.6667px"> Thanks,</span></div><div><span style="font-size:14.6667px"><br></span></div><div><span style="font-size:14.6667px"> Matt</span></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div class="msg5103815541450882016"><div lang="EN-GB" style="overflow-wrap: break-word;"><div class="m_5103815541450882016WordSection1"><p class="MsoNormal"><span style="font-size:11pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt">PetscCall(PetscOptionsIntArray("-pc_hypre_ams_amg_alpha_options", "AMG options for vector Poisson", "None", jac->as_amg_alpha_opts, &n, &flag2));<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> if (flag || flag2) {<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> PetscCallExternal(HYPRE_AMSSetAlphaAMGOptions, jac->hsolver, jac->as_amg_alpha_opts[0], /* AMG coarsen type */<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> jac->as_amg_alpha_opts[1], /* AMG agg_levels */<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> jac->as_amg_alpha_opts[2], /* AMG relax_type */<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> jac->as_amg_alpha_theta, jac->as_amg_alpha_opts[3], /* AMG interp_type */<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> jac->as_amg_alpha_opts[4]); /* AMG Pmax */<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> }<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt">Kind regards,<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt">Karthik.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"><u></u> <u></u></span></p>
<div id="m_5103815541450882016mail-editor-reference-message-container">
<div>
<div>
<div style="border-right:none;border-bottom:none;border-left:none;border-top:1pt solid rgb(181,196,223);padding:3pt 0cm 0cm">
<p class="MsoNormal" style="margin-bottom:12pt"><b><span style="font-size:12pt;color:black">From:
</span></b><span style="font-size:12pt;color:black">Matthew Knepley <<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>><br>
<b>Date: </b>Monday, 9 September 2024 at 22:00<br>
<b>To: </b>Chockalingam, Karthikeyan (STFC,DL,HC) <<a href="mailto:karthikeyan.chockalingam@stfc.ac.uk" target="_blank">karthikeyan.chockalingam@stfc.ac.uk</a>><br>
<b>Cc: </b><a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a> <<a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a>><br>
<b>Subject: </b>Re: [petsc-users] Hypre AMS usage<u></u><u></u></span></p>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:12pt">On Mon, Sep 9, 2024 at 4:32</span><span style="font-size:12pt;font-family:Arial,sans-serif"> </span><span style="font-size:12pt">PM Karthikeyan Chockalingam - STFC UKRI <<a href="mailto:karthikeyan.chockalingam@stfc.ac.uk" target="_blank">karthikeyan.chockalingam@stfc.ac.uk</a>>
wrote:<u></u><u></u></span></p>
</div>
<div>
<blockquote style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin-left:4.8pt;margin-right:0cm">
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11pt">Great. Thank you for letting me know.</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt">I got the reference to KSP as well from libmesh and I am not creating a new KSP.</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt">This time around, I didn’t have to set the PC type and it seems to work.</span><span style="font-size:12pt"><u></u><u></u></span></p>
</div>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><span style="font-size:12pt"><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt">Excellent.<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt"><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt"> Thanks,<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt"><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt"> Matt <u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt"> <u></u><u></u></span></p>
</div>
<blockquote style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin-left:4.8pt;margin-right:0cm">
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11pt"> petscErr = PetscOptionsSetValue(NULL,"-ksp_type", "gmres");
</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> petscErr = PetscOptionsSetValue(NULL,"-pc_type", "hypre");</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> petscErr = PetscOptionsSetValue(NULL,"-pc_hypre_type", "ams");</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> petscErr = PetscOptionsSetValue(NULL,"-pc_hypre_ams_relax_type", "2");</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> petscErr = PetscOptionsSetValue(NULL,"-pc_hypre_ams_relax_times", "1");</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> petscErr = PetscOptionsSetValue(NULL,"-pc_hypre_ams_relax_weight", "1.0");</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> petscErr = PetscOptionsSetValue(NULL, "-pc_hypre_ams_omega", "1.0");</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> petscErr = PetscOptionsSetValue(NULL, "-ksp_view", NULL);</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> petscErr = PetscOptionsSetValue(NULL, "-ksp_monitor_true_residual", NULL);</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> petscErr = PetscOptionsSetValue(NULL, "-ksp_converged_reason", NULL);</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> petscErr = PetscOptionsSetValue(NULL, "-options_left", NULL);</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> petscErr = KSPSetFromOptions(ksp);
</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> </span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> // Set discrete gradient
</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> petscErr = PCHYPRESetDiscreteGradient(pc, par_G);</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> </span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> // Set vertex coordinates</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> petscErr = PCHYPRESetEdgeConstantVectors(pc, par_xvec, par_yvec, par_zvec);</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> </span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> </span><span style="font-size:12pt"><u></u><u></u></span></p>
<div id="m_5103815541450882016m_4947689944060802622mail-editor-reference-message-container">
<div>
<div>
<div style="border-right:none;border-bottom:none;border-left:none;border-top:1pt solid rgb(181,196,223);padding:3pt 0cm 0cm">
<p class="MsoNormal" style="margin-bottom:12pt"><b><span style="font-size:12pt;color:black">From:
</span></b><span style="font-size:12pt;color:black">Matthew Knepley <<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>><br>
<b>Date: </b>Monday, 9 September 2024 at 21:11<br>
<b>To: </b>Chockalingam, Karthikeyan (STFC,DL,HC) <<a href="mailto:karthikeyan.chockalingam@stfc.ac.uk" target="_blank">karthikeyan.chockalingam@stfc.ac.uk</a>><br>
<b>Cc: </b>Stefano Zampini <<a href="mailto:stefano.zampini@gmail.com" target="_blank">stefano.zampini@gmail.com</a>>,
<a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a> <<a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a>><br>
<b>Subject: </b>Re: [petsc-users] Hypre AMS usage</span><span style="font-size:12pt"><u></u><u></u></span></p>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:12pt">On Mon, Sep 9, 2024 at 4:03</span><span style="font-size:12pt;font-family:Arial,sans-serif"> </span><span style="font-size:12pt">PM Karthikeyan
Chockalingam - STFC UKRI <<a href="mailto:karthikeyan.chockalingam@stfc.ac.uk" target="_blank">karthikeyan.chockalingam@stfc.ac.uk</a>> wrote:<u></u><u></u></span></p>
</div>
<div>
<blockquote style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin:5pt 0cm 5pt 4.8pt">
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11pt">I never explicitly called KSPGetPC(). I am embedding AMS in a libmesh (fem) example. So, I only got the reference to pc from libmesh and set petscErr
= PCHYPRESetDiscreteGradient(pc, par_G);</span><span style="font-size:12pt"><u></u><u></u></span></p>
</div>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><span style="font-size:12pt"> <u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt">However, you are creating a _new_ KSP, so your PC will not match it. This is not doing what you want.<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt"> <u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt"> Thanks,<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt"> <u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt"> Matt<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt"> <u></u><u></u></span></p>
</div>
<blockquote style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin:5pt 0cm 5pt 4.8pt">
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11pt"> </span><span style="font-size:12pt"><u></u><u></u></span></p>
<div id="m_5103815541450882016m_4947689944060802622m_-2936368868008084915mail-editor-reference-message-container">
<div>
<div>
<div style="border-right:none;border-bottom:none;border-left:none;border-top:1pt solid rgb(181,196,223);padding:3pt 0cm 0cm">
<p class="MsoNormal" style="margin-bottom:12pt"><b><span style="font-size:12pt;color:black">From:
</span></b><span style="font-size:12pt;color:black">Matthew Knepley <<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>><br>
<b>Date: </b>Monday, 9 September 2024 at 20:57<br>
<b>To: </b>Chockalingam, Karthikeyan (STFC,DL,HC) <<a href="mailto:karthikeyan.chockalingam@stfc.ac.uk" target="_blank">karthikeyan.chockalingam@stfc.ac.uk</a>><br>
<b>Cc: </b>Stefano Zampini <<a href="mailto:stefano.zampini@gmail.com" target="_blank">stefano.zampini@gmail.com</a>>,
<a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a> <<a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a>><br>
<b>Subject: </b>Re: [petsc-users] Hypre AMS usage</span><span style="font-size:12pt"><u></u><u></u></span></p>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:12pt">On Mon, Sep 9, 2024 at 3:38</span><span style="font-size:12pt;font-family:Arial,sans-serif"> </span><span style="font-size:12pt">PM Karthikeyan
Chockalingam - STFC UKRI <<a href="mailto:karthikeyan.chockalingam@stfc.ac.uk" target="_blank">karthikeyan.chockalingam@stfc.ac.uk</a>> wrote:<u></u><u></u></span></p>
</div>
<div>
<blockquote style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin:5pt 0cm 5pt 4.8pt">
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11pt">I didn’t know how to check the pc type but adding </span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt">petscErr = PCSetType(pc, "hypre");</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt">before the two functions made it to work.</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> </span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt">But how does it work from the command line?</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> </span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt">For summary:</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> </span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> KSPCreate(mesh.comm().get(), &ksp);</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> petscErr = PetscOptionsSetValue(NULL,"-ksp_type", "gmres");
</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> petscErr = PetscOptionsSetValue(NULL,"-pc_type", "hypre");</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> petscErr = PetscOptionsSetValue(NULL,"-pc_hypre_type", "ams");</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> petscErr = PetscOptionsSetValue(NULL,"-pc_hypre_ams_relax_type", "2");</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> petscErr = PetscOptionsSetValue(NULL,"-pc_hypre_ams_relax_times", "1");</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> petscErr = PetscOptionsSetValue(NULL,"-pc_hypre_ams_relax_weigh", "1.0");</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> petscErr = PetscOptionsSetValue(NULL, "-pc_hypre_ams_omega", "1.0");</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> petscErr = PetscOptionsSetValue(NULL, "-ksp_view", NULL);</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> petscErr = PetscOptionsSetValue(NULL, "-ksp_monitor_true_residual", NULL);</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> petscErr = PetscOptionsSetValue(NULL, "-ksp_converged_reason", NULL);</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> petscErr = PetscOptionsSetValue(NULL, "-options_left", NULL);</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> petscErr = KSPSetFromOptions(ksp);</span><span style="font-size:12pt"><u></u><u></u></span></p>
</div>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><span style="font-size:12pt"> <u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt">Where is the KSPGetPC() call?<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt"> <u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt"> Thanks,<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt"> <u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt"> Matt<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt"> <u></u><u></u></span></p>
</div>
<blockquote style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin:5pt 0cm 5pt 4.8pt">
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11pt"> // Set pc type</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> petscErr = PCSetType(pc, "hypre");</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> </span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> // Set discrete gradient
</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> petscErr = PCHYPRESetDiscreteGradient(pc, par_G);</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> </span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> // Set vertex coordinates</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> petscErr = PCHYPRESetEdgeConstantVectors(pc, par_xvec, par_yvec, par_zvec);</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> </span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> </span><span style="font-size:12pt"><u></u><u></u></span></p>
<div id="m_5103815541450882016m_4947689944060802622m_-2936368868008084915m_-2834263137362592157mail-editor-reference-message-container">
<div>
<div>
<div style="border-right:none;border-bottom:none;border-left:none;border-top:1pt solid rgb(181,196,223);padding:3pt 0cm 0cm">
<p class="MsoNormal" style="margin-bottom:12pt"><b><span style="font-size:12pt;color:black">From:
</span></b><span style="font-size:12pt;color:black">Matthew Knepley <<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>><br>
<b>Date: </b>Monday, 9 September 2024 at 19:46<br>
<b>To: </b>Chockalingam, Karthikeyan (STFC,DL,HC) <<a href="mailto:karthikeyan.chockalingam@stfc.ac.uk" target="_blank">karthikeyan.chockalingam@stfc.ac.uk</a>><br>
<b>Cc: </b>Stefano Zampini <<a href="mailto:stefano.zampini@gmail.com" target="_blank">stefano.zampini@gmail.com</a>>,
<a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a> <<a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a>><br>
<b>Subject: </b>Re: [petsc-users] Hypre AMS usage</span><span style="font-size:12pt"><u></u><u></u></span></p>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:12pt">On Mon, Sep 9, 2024 at 2:18</span><span style="font-size:12pt;font-family:Arial,sans-serif"> </span><span style="font-size:12pt">PM Karthikeyan
Chockalingam - STFC UKRI <<a href="mailto:karthikeyan.chockalingam@stfc.ac.uk" target="_blank">karthikeyan.chockalingam@stfc.ac.uk</a>> wrote:<u></u><u></u></span></p>
</div>
<div>
<blockquote style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin:5pt 0cm 5pt 4.8pt">
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11pt">Calling the two functions after
</span><span style="font-size:12pt">KSPSetFromOptions() did not work either.<u></u><u></u></span></p>
</div>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><span style="font-size:12pt"> <u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt">Can you check that the PC has the correct type when you call it?<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt"> <u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt"> Thanks,<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt"> <u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt"> Matt<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt"> <u></u><u></u></span></p>
</div>
<blockquote style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin:5pt 0cm 5pt 4.8pt">
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:12pt">Everything works from the command line.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:12pt">I haven’t set KSPSetOperators, not sure if that is an issue.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> </span><span style="font-size:12pt"><u></u><u></u></span></p>
<div id="m_5103815541450882016m_4947689944060802622m_-2936368868008084915m_-2834263137362592157m_9051397168688990659mail-editor-reference-message-container">
<div>
<div>
<div style="border-right:none;border-bottom:none;border-left:none;border-top:1pt solid rgb(181,196,223);padding:3pt 0cm 0cm">
<p class="MsoNormal" style="margin-bottom:12pt"><b><span style="font-size:12pt;color:black">From:
</span></b><span style="font-size:12pt;color:black">Matthew Knepley <<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>><br>
<b>Date: </b>Monday, 9 September 2024 at 19:09<br>
<b>To: </b>Chockalingam, Karthikeyan (STFC,DL,HC) <<a href="mailto:karthikeyan.chockalingam@stfc.ac.uk" target="_blank">karthikeyan.chockalingam@stfc.ac.uk</a>><br>
<b>Cc: </b>Stefano Zampini <<a href="mailto:stefano.zampini@gmail.com" target="_blank">stefano.zampini@gmail.com</a>>,
<a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a> <<a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a>><br>
<b>Subject: </b>Re: [petsc-users] Hypre AMS usage</span><span style="font-size:12pt"><u></u><u></u></span></p>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:12pt">On Mon, Sep 9, 2024 at 1:21</span><span style="font-size:12pt;font-family:Arial,sans-serif"> </span><span style="font-size:12pt">PM Karthikeyan
Chockalingam - STFC UKRI <<a href="mailto:karthikeyan.chockalingam@stfc.ac.uk" target="_blank">karthikeyan.chockalingam@stfc.ac.uk</a>> wrote:<u></u><u></u></span></p>
</div>
<div>
<blockquote style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin:5pt 0cm 5pt 4.8pt">
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11pt">Hi Stefano,</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> </span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt">Thank you. That was helpful. I tried the following:</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> </span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> petscErr = PetscOptionsSetValue(NULL,"-ksp_type", "gmres");
</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> petscErr = PetscOptionsSetValue(NULL,"-pc_type", "hypre");</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> petscErr = PetscOptionsSetValue(NULL,"-pc_hypre_type", "ams");</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> </span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> // Set discrete gradient
</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> petscErr = PCHYPRESetDiscreteGradient(pc, par_G);</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> </span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> // Set vertex coordinates</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> petscErr = PCHYPRESetEdgeConstantVectors(pc, par_xvec, par_yvec, par_zvec);</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> </span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> petscErr = PetscOptionsSetValue(NULL, "-ksp_view", NULL);</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> petscErr = PetscOptionsSetValue(NULL, "-ksp_monitor_true_residual", NULL);</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> petscErr = PetscOptionsSetValue(NULL, "-ksp_converged_reason", NULL);</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> petscErr = PetscOptionsSetValue(NULL, "-options_left", NULL);</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> petscErr = KSPSetFromOptions(ksp);
</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> </span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> </span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt">It errored though I have set PCHYPRESetEdgeConstantVectors.</span><span style="font-size:12pt"><u></u><u></u></span></p>
</div>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><span style="font-size:12pt"> <u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt">My guess is that the "pc" was not yet Hypre, since that type was not set until KSPSetFromOptions() was called. So you need<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt">to call those two functions after that call.<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt"> <u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt"> Thanks,<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt"> <u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt"> Matt<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt"> <u></u><u></u></span></p>
</div>
<blockquote style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin:5pt 0cm 5pt 4.8pt">
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11pt">But my program works, without PetscOptionsSetValue and by passing everything on the command line.</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> </span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><b><span style="font-size:11pt">[0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------</span></b><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt">[0]PETSC ERROR: HYPRE AMS preconditioner needs either the coordinate vectors via PCSetCoordinates() or the edge constant vectors via PCHYPRESetEdgeConstantVectors()
or the interpolation matrix via PCHYPRESetInterpolations()</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt">[0]PETS C ERROR: WARNING! There are unused option(s) set! Could be the program crashed before usage or a spelling mistake, etc!</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt">[0]PETSC ERROR: Option left: name:-ksp_converged_reason (no value) source: code</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt">[0]PETSC ERROR: Option left: name:-options_left (no value) source: code</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt">[0]PETSC ERROR: See
<a href="https://urldefense.us/v3/__https://petsc.org/release/faq/__;!!G_uCfscf7eWS!fzNdjKwlSE6S1482kG8uLzgyvhIjtNTWlib31rKGN57mOsU51yP6t9MkwWHKKx5J_BOV6XeIS4ibNFOiY6cO$" target="_blank">https://petsc.org/release/faq/</a> for trouble shooting.</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt">[0]PETSC ERROR: Petsc Release Version 3.20.3, unknown </span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt">[0]PETSC ERROR: ./example-dbg on a arch-moose named HC20210312 by karthikeyan.chockalingam Mon Sep 9 18:10:13 2024</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt">[0]PETSC ERROR: Configure options --with-64-bit-indices --with-cxx-dialect=C++17 --with-debugging=no --with-fortran-bindings=0 --with-mpi=1 --with-openmp=1
--with-shared-libraries=1 --with-sowing=0 --download-fblaslapack=1 --download-hypre=1 --download-metis=1 --download-mumps=1 --download-ptscotch=1 --download-parmetis=1 --download-scalapack=1 --download-slepc=1 --download-strumpack=1 --download-superlu_dist=1
--download-bison=1 --download-hdf5=1 --with-hdf5-fortran-bindings=0 --download-hdf5-configure-arguments="--with-zlib" --with-make-np=4 </span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt">[0]PETSC ERROR: #1 PCSetUp_HYPRE() at /Users/karthikeyan.chockalingam/moose/petsc/src/ksp/pc/impls/hypre/hypre.c:295</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt">[0]PETSC ERROR: #2 PCSetUp() at /Users/karthikeyan.chockalingam/moose/petsc/src/ksp/pc/interface/precon.c:1080</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt">[0]PETSC ERROR: #3 KSPSetUp() at /Users/karthikeyan.chockalingam/moose/petsc/src/ksp/ksp/interface/itfunc.c:415</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt">[0]PETSC ERROR: #4 KSPSolve_Private() at /Users/karthikeyan.chockalingam/moose/petsc/src/ksp/ksp/interface/itfunc.c:833</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt">[0]PETSC ERROR: #5 KSPSolve() at /Users/karthikeyan.chockalingam/moose/petsc/src/ksp/ksp/interface/itfunc.c:1080</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt">libMesh terminating:</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt">HYPRE AMS preconditioner needs either the coordinate vectors via PCSetCoordinates() or the edge constant vectors via PCHYPRESetEdgeConstantVectors()
or the interpolation matrix via PCHYPRESetInterpolations()</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> </span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> </span><span style="font-size:12pt"><u></u><u></u></span></p>
<div id="m_5103815541450882016m_4947689944060802622m_-2936368868008084915m_-2834263137362592157m_9051397168688990659m_-8689017490874508934mail-editor-reference-message-container">
<div>
<div>
<div style="border-right:none;border-bottom:none;border-left:none;border-top:1pt solid rgb(181,196,223);padding:3pt 0cm 0cm">
<p class="MsoNormal" style="margin-bottom:12pt"><b><span style="font-size:12pt;color:black">From:
</span></b><span style="font-size:12pt;color:black">Stefano Zampini <<a href="mailto:stefano.zampini@gmail.com" target="_blank">stefano.zampini@gmail.com</a>><br>
<b>Date: </b>Monday, 9 September 2024 at 17:02<br>
<b>To: </b>Matthew Knepley <<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>><br>
<b>Cc: </b>Chockalingam, Karthikeyan (STFC,DL,HC) <<a href="mailto:karthikeyan.chockalingam@stfc.ac.uk" target="_blank">karthikeyan.chockalingam@stfc.ac.uk</a>>,
<a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a> <<a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a>><br>
<b>Subject: </b>Re: [petsc-users] Hypre AMS usage</span><span style="font-size:12pt"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt">I would say the best way is to look at the source code<u></u><u></u></span></p>
<div>
<p class="MsoNormal"><span style="font-size:12pt"> <u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt"><a href="https://urldefense.us/v3/__https://gitlab.com/petsc/petsc/-/blob/main/src/ksp/pc/impls/hypre/hypre.c?ref_type=heads*L2061__;Iw!!G_uCfscf7eWS!fzNdjKwlSE6S1482kG8uLzgyvhIjtNTWlib31rKGN57mOsU51yP6t9MkwWHKKx5J_BOV6XeIS4ibNPqq4coE$" target="_blank">https://gitlab.com/petsc/petsc/-/blob/main/src/ksp/pc/impls/hypre/hypre.c?ref_type=heads#L2061</a><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt"><a href="https://urldefense.us/v3/__https://gitlab.com/petsc/petsc/-/blob/main/src/ksp/pc/impls/hypre/hypre.c?ref_type=heads*L1239__;Iw!!G_uCfscf7eWS!fzNdjKwlSE6S1482kG8uLzgyvhIjtNTWlib31rKGN57mOsU51yP6t9MkwWHKKx5J_BOV6XeIS4ibNJC3V4RB$" target="_blank">https://gitlab.com/petsc/petsc/-/blob/main/src/ksp/pc/impls/hypre/hypre.c?ref_type=heads#L1239</a><u></u><u></u></span></p>
</div>
</div>
<p class="MsoNormal"><span style="font-size:12pt"> <u></u><u></u></span></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:12pt">Il giorno lun 9 set 2024 alle ore 18:50 Matthew Knepley <<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>> ha scritto:<u></u><u></u></span></p>
</div>
<blockquote style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin:5pt 0cm 5pt 4.8pt">
<div>
<div>
<p class="MsoNormal"><span style="font-size:12pt">On Mon, Sep 9, 2024 at 10:17</span><span style="font-size:12pt;font-family:Arial,sans-serif"> </span><span style="font-size:12pt">AM Karthikeyan
Chockalingam - STFC UKRI <<a href="mailto:karthikeyan.chockalingam@stfc.ac.uk" target="_blank">karthikeyan.chockalingam@stfc.ac.uk</a>> wrote:<u></u><u></u></span></p>
</div>
<div>
<blockquote style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin:5pt 0cm 5pt 4.8pt">
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11pt">Hi Matt,</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> </span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt">You mentioned it doesn’t hurt to set the smoothing flags</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> </span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"><a href="https://urldefense.us/v3/__https:/github.com/hypre-space/hypre/blob/3caa81955eb8d1b4e35d9b450e27cf6d07b50f6e/src/examples/ex15.c*L965__;Iw!!G_uCfscf7eWS!biuCOXkV-qD-e8iTXiwaD9XJ0EWuqr2PidZJdPqbnFzCshQFo-mclGSvGBOPuGSTkjPvMKnIfJzWos7QShcSitpwyrhL55yOMB-v$" target="_blank">https://github.com/hypre-space/hypre/blob/3caa81955eb8d1b4e35d9b450e27cf6d07b50f6e/src/examples/ex15.c#L965</a></span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt">Do you know where I can look for the equivalent PETSc commands? Thank you.</span><span style="font-size:12pt"><u></u><u></u></span></p>
</div>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><span style="font-size:12pt"> <u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt">Yes, this is described here: <a href="https://urldefense.us/v3/__https:/petsc.org/main/manualpages/PC/PCHYPRE/__;!!G_uCfscf7eWS!dM_jKT_vX0NCVNcY9XOTTUaTSW-TF2CgOkB2UDeZ2ztYXapdmLlSBT4XhV7PknN3ewygGWDbKFEP9e0RfePC$" target="_blank">https://petsc.org/main/manualpages/PC/PCHYPRE/</a><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt">The best way is to run with -help or look at the source.<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt"> <u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt"> Thanks,<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt"> <u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt"> Matt<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt"> <u></u><u></u></span></p>
</div>
<blockquote style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin:5pt 0cm 5pt 4.8pt">
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11pt"> </span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt">Kind regards,</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt">Karthik.</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> </span><span style="font-size:12pt"><u></u><u></u></span></p>
<div id="m_5103815541450882016m_4947689944060802622m_-2936368868008084915m_-2834263137362592157m_9051397168688990659m_-8689017490874508934m_7816721049659871032m_690867627511261516mail-editor-reference-message-container">
<div>
<div>
<div style="border-right:none;border-bottom:none;border-left:none;border-top:1pt solid rgb(181,196,223);padding:3pt 0cm 0cm">
<p class="MsoNormal" style="margin-bottom:12pt"><b><span style="font-size:12pt;color:black">From:
</span></b><span style="font-size:12pt;color:black">Matthew Knepley <<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>><br>
<b>Date: </b>Friday, 6 September 2024 at 17:57<br>
<b>To: </b>Chockalingam, Karthikeyan (STFC,DL,HC) <<a href="mailto:karthikeyan.chockalingam@stfc.ac.uk" target="_blank">karthikeyan.chockalingam@stfc.ac.uk</a>><br>
<b>Cc: </b><a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a> <<a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a>><br>
<b>Subject: </b>Re: [petsc-users] Hypre AMS usage</span><span style="font-size:12pt"><u></u><u></u></span></p>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:12pt">On Fri, Sep 6, 2024 at 11:37</span><span style="font-size:12pt;font-family:Arial,sans-serif"> </span><span style="font-size:12pt">AM Karthikeyan
Chockalingam - STFC UKRI via petsc-users <<a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a>> wrote:<u></u><u></u></span></p>
</div>
<div>
<blockquote style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin:5pt 0cm 5pt 4.8pt">
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11pt">Hello,</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> </span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt">I am trying to use the Hypre AMS preconditioner for the first time.
</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> </span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt">I am following the example problem from Hypre</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"><a href="https://urldefense.us/v3/__https:/github.com/hypre-space/hypre/blob/3caa81955eb8d1b4e35d9b450e27cf6d07b50f6e/src/examples/ex15.c*L954__;Iw!!G_uCfscf7eWS!biuCOXkV-qD-e8iTXiwaD9XJ0EWuqr2PidZJdPqbnFzCshQFo-mclGSvGBOPuGSTkjPvMKnIfJzWos7QShcSitpwyrhL5ySgA0XM$" target="_blank">https://github.com/hypre-space/hypre/blob/3caa81955eb8d1b4e35d9b450e27cf6d07b50f6e/src/examples/ex15.c#L954</a></span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> </span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt">I have so far successfully set the discrete gradient operator and vertex co-ordinates,</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> </span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt">// Set discrete gradient
</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> petscErr = PCHYPRESetDiscreteGradient(pc, par_G);</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> </span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> // Set vertex coordinates</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> petscErr = PCHYPRESetEdgeConstantVectors(pc, par_xvec, par_yvec, par_zvec);</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> </span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt">Do I need to set the following smoothing options?</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"><a href="https://urldefense.us/v3/__https:/github.com/hypre-space/hypre/blob/3caa81955eb8d1b4e35d9b450e27cf6d07b50f6e/src/examples/ex15.c*L965__;Iw!!G_uCfscf7eWS!biuCOXkV-qD-e8iTXiwaD9XJ0EWuqr2PidZJdPqbnFzCshQFo-mclGSvGBOPuGSTkjPvMKnIfJzWos7QShcSitpwyrhL55yOMB-v$" target="_blank">https://github.com/hypre-space/hypre/blob/3caa81955eb8d1b4e35d9b450e27cf6d07b50f6e/src/examples/ex15.c#L965</a></span><span style="font-size:12pt"><u></u><u></u></span></p>
</div>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><span style="font-size:12pt"> <u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt">It cannot hurt. I would set them to begin with.<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt"> <u></u><u></u></span></p>
</div>
<blockquote style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin:5pt 0cm 5pt 4.8pt">
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11pt">Also, do I need to convert from MATMPIAIJ to CSR?</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"><a href="https://urldefense.us/v3/__https:/github.com/hypre-space/hypre/blob/3caa81955eb8d1b4e35d9b450e27cf6d07b50f6e/src/examples/ex15.c*L984__;Iw!!G_uCfscf7eWS!biuCOXkV-qD-e8iTXiwaD9XJ0EWuqr2PidZJdPqbnFzCshQFo-mclGSvGBOPuGSTkjPvMKnIfJzWos7QShcSitpwyrhL53Trv1si$" target="_blank">https://github.com/hypre-space/hypre/blob/3caa81955eb8d1b4e35d9b450e27cf6d07b50f6e/src/examples/ex15.c#L984</a></span><span style="font-size:12pt"><u></u><u></u></span></p>
</div>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><span style="font-size:12pt"> <u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt">No.<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt"> <u></u><u></u></span></p>
</div>
<blockquote style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin:5pt 0cm 5pt 4.8pt">
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11pt">What are the other PETSc calls to invoke AMS? Is there an example problem I can look at?</span><span style="font-size:12pt"><u></u><u></u></span></p>
</div>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><span style="font-size:12pt"> <u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt">I do not know. I don't think we have an example.<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt"> <u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt"> Thanks,<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt"> <u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt"> Matt<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt"> <u></u><u></u></span></p>
</div>
<blockquote style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin:5pt 0cm 5pt 4.8pt">
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11pt">Thank you.</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> </span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt">Karthik.</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> </span><span style="font-size:12pt"><u></u><u></u></span></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11pt;font-family:Calibri,sans-serif">-- </span><span style="font-size:12pt"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><b><span style="font-size:10.5pt;font-family:Arial,sans-serif;color:rgb(45,45,98)">Karthik Chockalingam, Ph.D.</span></b><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:Arial,sans-serif;color:black">Senior Research Software Engineer</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:Arial,sans-serif;color:black">High Performance Systems Engineering Group</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:Arial,sans-serif;color:black">Hartree Centre | Science and Technology Facilities Council</span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:Arial,sans-serif;color:rgb(33,33,33)"><a href="mailto:karthikeyan.chockalingam@stfc.ac.uk" target="_blank"><span style="color:rgb(5,99,193)">karthikeyan.chockalingam@stfc.ac.uk</span></a></span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:12pt;font-family:"Times New Roman",serif;color:black"> </span><span style="font-size:10.5pt;font-family:Arial,sans-serif;color:rgb(65,65,65)"> </span><span style="font-size:12pt"><u></u><u></u></span></p>
<p class="MsoNormal"><b><span style="font-size:10.5pt;font-family:Arial,sans-serif;color:rgb(65,65,65)"> </span></b><span style="font-size:12pt;font-family:"Times New Roman",serif;color:rgb(31,73,125)"><img border="0" width="254" height="116" style="width: 2.6458in; height: 1.2083in;" id="m_5103815541450882016m_4947689944060802622m_-2936368868008084915m_-2834263137362592157m_9051397168688990659m_-8689017490874508934m_7816721049659871032m_690867627511261516m_-9159410701733724734Picture_x0020_1" src="cid:ii_191dd299bf24cff311" alt="signature_3970890138"></span><span style="font-size:12pt"><u></u><u></u></span></p>
</div>
</div>
<p class="MsoNormal"><span style="font-size:12pt"> <u></u><u></u></span></p>
</div>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><span style="font-size:12pt"><br clear="all">
<u></u><u></u></span></p>
<div>
<p class="MsoNormal"><span style="font-size:12pt"> <u></u><u></u></span></p>
</div>
<p class="MsoNormal"><span style="font-size:12pt">--
<u></u><u></u></span></p>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:12pt">What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments
lead.<br>
-- Norbert Wiener<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt"> <u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt"><a href="https://urldefense.us/v3/__http:/www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!dM_jKT_vX0NCVNcY9XOTTUaTSW-TF2CgOkB2UDeZ2ztYXapdmLlSBT4XhV7PknN3ewygGWDbKFEP9ZyhqOzc$" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><u></u><u></u></span></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><span style="font-size:12pt"><br clear="all">
<u></u><u></u></span></p>
<div>
<p class="MsoNormal"><span style="font-size:12pt"> <u></u><u></u></span></p>
</div>
<p class="MsoNormal"><span class="m_5103815541450882016m4947689944060802622m-2936368868008084915m-2834263137362592157m9051397168688990659m-8689017490874508934gmailsignatureprefix"><span style="font-size:12pt">--
</span></span><span style="font-size:12pt"><u></u><u></u></span></p>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:12pt">What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments
lead.<br>
-- Norbert Wiener<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt"> <u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt"><a href="https://urldefense.us/v3/__http:/www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!dM_jKT_vX0NCVNcY9XOTTUaTSW-TF2CgOkB2UDeZ2ztYXapdmLlSBT4XhV7PknN3ewygGWDbKFEP9ZyhqOzc$" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><u></u><u></u></span></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><span style="font-size:12pt"><br clear="all">
<u></u><u></u></span></p>
<div>
<p class="MsoNormal"><span style="font-size:12pt"> <u></u><u></u></span></p>
</div>
<p class="MsoNormal"><span class="m_5103815541450882016m4947689944060802622m-2936368868008084915m-2834263137362592157m9051397168688990659m-8689017490874508934gmailsignatureprefix"><span style="font-size:12pt">--
</span></span><span style="font-size:12pt"><u></u><u></u></span></p>
<div>
<p class="MsoNormal"><span style="font-size:12pt">Stefano<u></u><u></u></span></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><span style="font-size:12pt"><br clear="all">
<u></u><u></u></span></p>
<div>
<p class="MsoNormal"><span style="font-size:12pt"> <u></u><u></u></span></p>
</div>
<p class="MsoNormal"><span class="m_5103815541450882016m4947689944060802622m-2936368868008084915m-2834263137362592157m9051397168688990659gmailsignatureprefix"><span style="font-size:12pt">--
</span></span><span style="font-size:12pt"><u></u><u></u></span></p>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:12pt">What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments
lead.<br>
-- Norbert Wiener<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt"> <u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt"><a href="https://urldefense.us/v3/__http://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!fzNdjKwlSE6S1482kG8uLzgyvhIjtNTWlib31rKGN57mOsU51yP6t9MkwWHKKx5J_BOV6XeIS4ibNP14Ygti$" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><u></u><u></u></span></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><span style="font-size:12pt"><br clear="all">
<u></u><u></u></span></p>
<div>
<p class="MsoNormal"><span style="font-size:12pt"> <u></u><u></u></span></p>
</div>
<p class="MsoNormal"><span class="m_5103815541450882016m4947689944060802622m-2936368868008084915m-2834263137362592157gmailsignatureprefix"><span style="font-size:12pt">--
</span></span><span style="font-size:12pt"><u></u><u></u></span></p>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:12pt">What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments
lead.<br>
-- Norbert Wiener<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt"> <u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt"><a href="https://urldefense.us/v3/__http://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!fzNdjKwlSE6S1482kG8uLzgyvhIjtNTWlib31rKGN57mOsU51yP6t9MkwWHKKx5J_BOV6XeIS4ibNP14Ygti$" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><u></u><u></u></span></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><span style="font-size:12pt"><br clear="all">
<u></u><u></u></span></p>
<div>
<p class="MsoNormal"><span style="font-size:12pt"> <u></u><u></u></span></p>
</div>
<p class="MsoNormal"><span class="m_5103815541450882016m4947689944060802622m-2936368868008084915gmailsignatureprefix"><span style="font-size:12pt">--
</span></span><span style="font-size:12pt"><u></u><u></u></span></p>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:12pt">What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments
lead.<br>
-- Norbert Wiener<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt"> <u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt"><a href="https://urldefense.us/v3/__http://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!fzNdjKwlSE6S1482kG8uLzgyvhIjtNTWlib31rKGN57mOsU51yP6t9MkwWHKKx5J_BOV6XeIS4ibNP14Ygti$" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><u></u><u></u></span></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><span style="font-size:12pt"><br clear="all">
<u></u><u></u></span></p>
<div>
<p class="MsoNormal"><span style="font-size:12pt"> <u></u><u></u></span></p>
</div>
<p class="MsoNormal"><span class="m_5103815541450882016m4947689944060802622gmailsignatureprefix"><span style="font-size:12pt">--
</span></span><span style="font-size:12pt"><u></u><u></u></span></p>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:12pt">What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments
lead.<br>
-- Norbert Wiener<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt"> <u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt"><a href="https://urldefense.us/v3/__http://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!fzNdjKwlSE6S1482kG8uLzgyvhIjtNTWlib31rKGN57mOsU51yP6t9MkwWHKKx5J_BOV6XeIS4ibNP14Ygti$" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><u></u><u></u></span></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><span style="font-size:12pt"><br clear="all">
<u></u><u></u></span></p>
<div>
<p class="MsoNormal"><span style="font-size:12pt"><u></u> <u></u></span></p>
</div>
<p class="MsoNormal"><span class="m_5103815541450882016gmailsignatureprefix"><span style="font-size:12pt">--
</span></span><span style="font-size:12pt"><u></u><u></u></span></p>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:12pt">What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>
-- Norbert Wiener<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt"><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt"><a href="https://urldefense.us/v3/__http://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!fzNdjKwlSE6S1482kG8uLzgyvhIjtNTWlib31rKGN57mOsU51yP6t9MkwWHKKx5J_BOV6XeIS4ibNP14Ygti$" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><u></u><u></u></span></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div></blockquote></div><br clear="all"><div><br></div><span class="gmail_signature_prefix">-- </span><br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div><div><br></div><div><a href="https://urldefense.us/v3/__http://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!fzNdjKwlSE6S1482kG8uLzgyvhIjtNTWlib31rKGN57mOsU51yP6t9MkwWHKKx5J_BOV6XeIS4ibNP14Ygti$" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br></div></div></div></div></div></div></div></div>