<head><!-- BaNnErBlUrFlE-HeAdEr-start -->
<style>
#pfptBannertac677e { all: revert !important; display: block !important;
visibility: visible !important; opacity: 1 !important;
background-color: #D0D8DC !important;
max-width: none !important; max-height: none !important }
.pfptPrimaryButtontac677e:hover, .pfptPrimaryButtontac677e:focus {
background-color: #b4c1c7 !important; }
.pfptPrimaryButtontac677e:active {
background-color: #90a4ae !important; }
</style>
<!-- BaNnErBlUrFlE-HeAdEr-end -->
</head><!-- BaNnErBlUrFlE-BoDy-start -->
<!-- Preheader Text : BEGIN -->
<div style="display:none !important;display:none;visibility:hidden;mso-hide:all;font-size:1px;color:#ffffff;line-height:1px;height:0px;max-height:0px;opacity:0;overflow:hidden;">
On Wed, Jun 12, 2024 at 6: 36 PM Yongzhong Li <yongzhong. li@ mail. utoronto. ca> wrote: Dear PETSc’s developers, I hope this email finds you well. I am currently working on a project using PETSc and have encountered a performance issue with
</div>
<!-- Preheader Text : END -->
<!-- Email Banner : BEGIN -->
<div style="display:none !important;display:none;visibility:hidden;mso-hide:all;font-size:1px;color:#ffffff;line-height:1px;height:0px;max-height:0px;opacity:0;overflow:hidden;">ZjQcmQRYFpfptBannerStart</div>
<!--[if ((ie)|(mso))]>
<table border="0" cellspacing="0" cellpadding="0" width="100%" style="padding: 16px 0px 16px 0px; direction: ltr" ><tr><td>
<table border="0" cellspacing="0" cellpadding="0" style="padding: 0px 10px 5px 6px; width: 100%; border-radius:4px; border-top:4px solid #90a4ae;background-color:#D0D8DC;"><tr><td valign="top">
<table align="left" border="0" cellspacing="0" cellpadding="0" style="padding: 4px 8px 4px 8px">
<tr><td style="color:#000000; font-family: 'Arial', sans-serif; font-weight:bold; font-size:14px; direction: ltr">
This Message Is From an External Sender
</td></tr>
<tr><td style="color:#000000; font-weight:normal; font-family: 'Arial', sans-serif; font-size:12px; direction: ltr">
This message came from outside your organization.
</td></tr>
</table>
</td></tr></table>
</td></tr></table>
<![endif]-->
<![if !((ie)|(mso))]>
<div dir="ltr" id="pfptBannertac677e" style="all: revert !important; display:block !important; text-align: left !important; margin:16px 0px 16px 0px !important; padding:8px 16px 8px 16px !important; border-radius: 4px !important; min-width: 200px !important; background-color: #D0D8DC !important; background-color: #D0D8DC; border-top: 4px solid #90a4ae !important; border-top: 4px solid #90a4ae;">
<div id="pfptBannertac677e" style="all: unset !important; float:left !important; display:block !important; margin: 0px 0px 1px 0px !important; max-width: 600px !important;">
<div id="pfptBannertac677e" style="all: unset !important; display:block !important; visibility: visible !important; background-color: #D0D8DC !important; color:#000000 !important; color:#000000; font-family: 'Arial', sans-serif !important; font-family: 'Arial', sans-serif; font-weight:bold !important; font-weight:bold; font-size:14px !important; line-height:18px !important; line-height:18px">
This Message Is From an External Sender
</div>
<div id="pfptBannertac677e" style="all: unset !important; display:block !important; visibility: visible !important; background-color: #D0D8DC !important; color:#000000 !important; color:#000000; font-weight:normal; font-family: 'Arial', sans-serif !important; font-family: 'Arial', sans-serif; font-size:12px !important; line-height:18px !important; line-height:18px; margin-top:2px !important;">
This message came from outside your organization.
</div>
</div>
<div style="clear: both !important; display: block !important; visibility: hidden !important; line-height: 0 !important; font-size: 0.01px !important; height: 0px"> </div>
</div>
<![endif]>
<div style="display:none !important;display:none;visibility:hidden;mso-hide:all;font-size:1px;color:#ffffff;line-height:1px;height:0px;max-height:0px;opacity:0;overflow:hidden;">ZjQcmQRYFpfptBannerEnd</div>
<!-- Email Banner : END -->
<!-- BaNnErBlUrFlE-BoDy-end -->
<div dir="ltr"><div dir="ltr">On Wed, Jun 12, 2024 at 6:36 PM Yongzhong Li <<a href="mailto:yongzhong.li@mail.utoronto.ca">yongzhong.li@mail.utoronto.ca</a>> wrote:<br></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div class="msg2686255934643610028">
<div style="font-size:1px;color:rgb(255,255,255);line-height:1px;height:0px;max-height:0px;opacity:0;overflow:hidden;display:none">
Dear PETSc’s developers, I hope this email finds you well. I am currently working on a project using PETSc and have encountered a performance issue with the KSPSolve function. Specifically, I have noticed that the time taken by KSPSolve is
</div>
<div style="font-size:1px;color:rgb(255,255,255);line-height:1px;height:0px;max-height:0px;opacity:0;overflow:hidden;display:none">ZjQcmQRYFpfptBannerStart</div>
<u></u>
<div dir="ltr" id="m_5223539340070699115pfptBannerpeukvww" style="display:block;text-align:left;margin:16px 0px;padding:8px 16px;border-radius:4px;min-width:200px;background-color:rgb(208,216,220);border-top:4px solid rgb(144,164,174)">
<div id="m_5223539340070699115pfptBannerpeukvww" style="float:left;display:block;margin:0px 0px 1px;max-width:600px">
<div id="m_5223539340070699115pfptBannerpeukvww" style="display:block;background-color:rgb(208,216,220);color:rgb(0,0,0);font-family:Arial,sans-serif;font-weight:bold;font-size:14px;line-height:18px">
This Message Is From an External Sender
</div>
<div id="m_5223539340070699115pfptBannerpeukvww" style="font-weight:normal;display:block;background-color:rgb(208,216,220);color:rgb(0,0,0);font-family:Arial,sans-serif;font-size:12px;line-height:18px;margin-top:2px">
This message came from outside your organization.
</div>
</div>
<div style="height:0px;clear:both;display:block;line-height:0;font-size:0.01px"> </div>
</div>
<u></u>
<div style="font-size:1px;color:rgb(255,255,255);line-height:1px;height:0px;max-height:0px;opacity:0;overflow:hidden;display:none">ZjQcmQRYFpfptBannerEnd</div>
<div lang="en-CN" style="overflow-wrap: break-word;">
<div class="m_5223539340070699115WordSection1">
<p class="MsoNormal"><span style="color:black">Dear
</span><span lang="EN-US" style="color:black">PETSc’s developers</span><span style="color:black">,<u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:black">I hope this email finds you well.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:black">I am currently working on a project using PETSc and have encountered a performance issue with the </span><span style="font-size:10pt;font-family:"Courier New";color:black">KSPSolve</span><span style="color:black"> function.
Specifically, <b>I have noticed that the time taken by </b></span><b><span style="font-size:10pt;font-family:"Courier New";color:black">KSPSolve</span></b><b><span style="color:black"> is
</span></b><b><span lang="EN-US" style="color:black">almost two times
</span></b><b><span style="color:black">greater than the CPU time for matrix-vector product multiplied by the number of iteration steps</span></b><span style="color:black">.</span><span style="color:black">
<span lang="EN-US">I use C++ chrono to record CPU time.<u></u><u></u></span></span></p>
<p class="MsoNormal"><span style="color:black">For context, I am using a shell system matrix</span><span lang="EN-US" style="color:black"> A</span><span style="color:black">.
Despite my efforts to </span><span lang="EN-US" style="color:black">parallelize</span><span style="color:black"> the matrix-vector product</span><span lang="EN-US" style="color:black"> (Ax)</span><span style="color:black">,
the overall solve time remains higher</span><span lang="EN-US" style="color:black"> than the matrix vector product per iteration indicates when multiple threads were used</span><span style="color:black">. Here are a few
details of my setup:<u></u><u></u></span></p>
<ul type="disc">
<li class="MsoNormal" style="color:black">
<b><span>Matrix Type</span></b><span>: Shell system matrix<u></u><u></u></span></li><li class="MsoNormal" style="color:black">
<b><span>Preconditioner</span></b><span>:
</span><span lang="EN-US">Shell PC</span><span><u></u><u></u></span></li><li class="MsoNormal" style="color:black">
<b><span>Parallel Environment</span></b><span>:
</span><span lang="EN-US">Using Intel MKL as PETSc’s BLAS/LAPACK library, multithreading is enabled
</span><span><u></u><u></u></span></li></ul>
<p class="MsoNormal"><span lang="EN-US" style="color:black">I have considered several potential reasons, such as preconditioner setup, additional solver operations, and the inherent
overhead of using a shell system matrix. <b>However, since KSPSolve is a high-level API, I have been unable to pinpoint the exact cause of the increased solve time.<u></u><u></u></b></span></p>
<p class="MsoNormal"><span lang="EN-US" style="color:black">Have you observed the same issue?
</span><span style="color:black">Could you</span><span lang="EN-US" style="color:black"> please</span><span lang="EN-US" style="color:black">
</span><span style="color:black">provide some </span><span lang="EN-US" style="color:black">experience</span><span style="color:black"> on how to diagnose and address this performance discrepancy? Any
insights or recommendations you could offer would be greatly appreciated.</span></p></div></div></div></blockquote><div><br></div><div>For any performance question like this, we need to see the output of your code run with</div><div><br></div><div> -ksp_view -ksp_monitor_true_residual -ksp_converged_reason -log_view</div><div><br></div><div> Thanks,</div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div class="msg2686255934643610028"><div lang="en-CN" style="overflow-wrap: break-word;"><div class="m_5223539340070699115WordSection1"><p class="MsoNormal"><span style="color:black"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:black">Thank you for your time and assistance.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:black">Best</span><span lang="EN-US" style="color:black"> regards,<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="color:black">Yongzhong<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11pt;font-family:Calibri,sans-serif">-----------------------------------------------------------<u></u><u></u></span></p>
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:11pt;font-family:Calibri,sans-serif">Yongzhong Li</span></b><span lang="EN-US" style="font-size:11pt;font-family:Calibri,sans-serif"><u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11pt;font-family:Calibri,sans-serif">PhD student | Electromagnetics Group<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11pt;font-family:Calibri,sans-serif">Department of Electrical & Computer Engineering<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11pt;font-family:Calibri,sans-serif">University of Toronto<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11pt;font-family:Calibri,sans-serif"><a href="https://urldefense.us/v3/__http://www.modelics.org__;!!G_uCfscf7eWS!cuLttMJEcegaqu461Bt4QLsO4fASfLM5vjRbtyNhWJQiInbjgNwkGNdkFE1ebSbFjOUatYB0-jd2yQWMWzqkDFFjwMvNl3ZKAr8$" target="_blank"><span style="color:rgb(5,99,193)">http://www.modelics.org</span></a><u></u><u></u></span></p>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div>
</div></blockquote></div><br clear="all"><div><br></div><span class="gmail_signature_prefix">-- </span><br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div><div><br></div><div><a href="https://urldefense.us/v3/__http://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!eMuXWvayLIhrQweHZY95IfQMST6PiUiLEskCz9WUy0pb9bazMdyoLiAyZh_l80blSuxXwO5yN7vzdFPL_avf$" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br></div></div></div></div></div></div></div></div>