<!-- BaNnErBlUrFlE-BoDy-start -->
<!-- Preheader Text : BEGIN -->
<div style="display:none !important;display:none;visibility:hidden;mso-hide:all;font-size:1px;color:#ffffff;line-height:1px;height:0px;max-height:0px;opacity:0;overflow:hidden;">
Dear Barry, Thanks for your response. Now I am doing simple debugging for my customized mat_mult_op of my shell jacobian matrix. As far as I understand, because the input and output of shell jacobian are all global vectors, I need to do the
</div>
<!-- Preheader Text : END -->
<!-- Email Banner : BEGIN -->
<div style="display:none !important;display:none;visibility:hidden;mso-hide:all;font-size:1px;color:#ffffff;line-height:1px;height:0px;max-height:0px;opacity:0;overflow:hidden;">ZjQcmQRYFpfptBannerStart</div>
<!--[if ((ie)|(mso))]>
<table border="0" cellspacing="0" cellpadding="0" width="100%" style="padding: 16px 0px 16px 0px; direction: ltr" ><tr><td>
<table border="0" cellspacing="0" cellpadding="0" style="padding: 0px 10px 5px 6px; width: 100%; border-radius:4px; border-top:4px solid #90a4ae;background-color:#D0D8DC;"><tr><td valign="top">
<table align="left" border="0" cellspacing="0" cellpadding="0" style="padding: 4px 8px 4px 8px">
<tr><td style="color:#000000; font-family: 'Arial', sans-serif; font-weight:bold; font-size:14px; direction: ltr">
This Message Is From an External Sender
</td></tr>
<tr><td style="color:#000000; font-weight:normal; font-family: 'Arial', sans-serif; font-size:12px; direction: ltr">
This message came from outside your organization.
</td></tr>
</table>
</td></tr></table>
</td></tr></table>
<![endif]-->
<![if !((ie)|(mso))]>
<div dir="ltr" id="pfptBannersa9epfk" style="all: revert !important; display:block !important; text-align: left !important; margin:16px 0px 16px 0px !important; padding:8px 16px 8px 16px !important; border-radius: 4px !important; min-width: 200px !important; background-color: #D0D8DC !important; background-color: #D0D8DC; border-top: 4px solid #90a4ae !important; border-top: 4px solid #90a4ae;">
<div id="pfptBannersa9epfk" style="all: unset !important; float:left !important; display:block !important; margin: 0px 0px 1px 0px !important; max-width: 600px !important;">
<div id="pfptBannersa9epfk" style="all: unset !important; display:block !important; visibility: visible !important; background-color: #D0D8DC !important; color:#000000 !important; color:#000000; font-family: 'Arial', sans-serif !important; font-family: 'Arial', sans-serif; font-weight:bold !important; font-weight:bold; font-size:14px !important; line-height:18px !important; line-height:18px">
This Message Is From an External Sender
</div>
<div id="pfptBannersa9epfk" style="all: unset !important; display:block !important; visibility: visible !important; background-color: #D0D8DC !important; color:#000000 !important; color:#000000; font-weight:normal; font-family: 'Arial', sans-serif !important; font-family: 'Arial', sans-serif; font-size:12px !important; line-height:18px !important; line-height:18px; margin-top:2px !important;">
This message came from outside your organization.
</div>
</div>
<div style="clear: both !important; display: block !important; visibility: hidden !important; line-height: 0 !important; font-size: 0.01px !important; height: 0px"> </div>
</div>
<![endif]>
<div style="display:none !important;display:none;visibility:hidden;mso-hide:all;font-size:1px;color:#ffffff;line-height:1px;height:0px;max-height:0px;opacity:0;overflow:hidden;">ZjQcmQRYFpfptBannerEnd</div>
<!-- Email Banner : END -->
<!-- BaNnErBlUrFlE-BoDy-end -->
<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><!-- BaNnErBlUrFlE-HeAdEr-start -->
<style>
#pfptBannersa9epfk { all: revert !important; display: block !important;
visibility: visible !important; opacity: 1 !important;
background-color: #D0D8DC !important;
max-width: none !important; max-height: none !important }
.pfptPrimaryButtonsa9epfk:hover, .pfptPrimaryButtonsa9epfk:focus {
background-color: #b4c1c7 !important; }
.pfptPrimaryButtonsa9epfk:active {
background-color: #90a4ae !important; }
</style>
<!-- BaNnErBlUrFlE-HeAdEr-end -->
<meta http-equiv=Content-Type content="text/html; charset=utf-8"><meta name=Generator content="Microsoft Word 15 (filtered medium)"><!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:Helvetica;
panose-1:2 11 6 4 2 2 2 2 2 4;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:DengXian;
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
@font-face
{font-family:"\@DengXian";
panose-1:2 1 6 0 3 1 1 1 1 1;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0in;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:Consolas;}
span.apple-converted-space
{mso-style-name:apple-converted-space;}
span.EmailStyle23
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=en-DE link=blue vlink=purple><div class=WordSection1><p class=MsoNormal><span lang=EN-US>Dear Barry,<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>Thanks for your response. Now I am doing simple debugging for my customized mat_mult_op of my shell jacobian matrix. As far as I understand, because the input and output of shell jacobian are all global vectors, I need to do the global to local mapping of my input vector (dF) by myself. Before starting debugging the mapping, I first try to verify the size match of input and output.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>Because the input and output of a mat_mult_op of my shell matrix should have the same size. So I tried just equating them in my customized mat_mult_op. Basically like this <o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal style='line-height:14.25pt;background:#1F1F1F'><span style='font-size:10.5pt;font-family:Consolas;color:#569CD6'>subroutine</span><span style='font-size:10.5pt;font-family:Consolas;color:#CCCCCC'> </span><span style='font-size:10.5pt;font-family:Consolas;color:#DCDCAA'>GK_op</span><span style='font-size:10.5pt;font-family:Consolas;color:#CCCCCC'>(</span><span style='font-size:10.5pt;font-family:Consolas;color:#9CDCFE'>Jac</span><span style='font-size:10.5pt;font-family:Consolas;color:#CCCCCC'>,</span><span style='font-size:10.5pt;font-family:Consolas;color:#9CDCFE'>dF_global</span><span style='font-size:10.5pt;font-family:Consolas;color:#CCCCCC'>,</span><span style='font-size:10.5pt;font-family:Consolas;color:#9CDCFE'>output_global</span><span style='font-size:10.5pt;font-family:Consolas;color:#CCCCCC'>,</span><span style='font-size:10.5pt;font-family:Consolas;color:#9CDCFE'>err_PETSc</span><span style='font-size:10.5pt;font-family:Consolas;color:#CCCCCC'>)<o:p></o:p></span></p><p class=MsoNormal style='line-height:14.25pt;background:#1F1F1F'><span style='font-size:10.5pt;font-family:Consolas;color:#CCCCCC'><o:p> </o:p></span></p><p class=MsoNormal style='line-height:14.25pt;background:#1F1F1F'><span style='font-size:10.5pt;font-family:Consolas;color:#CCCCCC'> DM :: dm_local </span><span style='font-size:10.5pt;font-family:Consolas;color:#6A9955'>! Yi: later for is,ie</span><span style='font-size:10.5pt;font-family:Consolas;color:#CCCCCC'><o:p></o:p></span></p><p class=MsoNormal style='line-height:14.25pt;background:#1F1F1F'><span style='font-size:10.5pt;font-family:Consolas;color:#CCCCCC'> Vec :: dF_global<o:p></o:p></span></p><p class=MsoNormal style='line-height:14.25pt;background:#1F1F1F'><span style='font-size:10.5pt;font-family:Consolas;color:#CCCCCC'> Vec :: output_global<o:p></o:p></span></p><p class=MsoNormal style='line-height:14.25pt;background:#1F1F1F'><span style='font-size:10.5pt;font-family:Consolas;color:#CCCCCC'> PetscErrorCode :: err_PETSc<o:p></o:p></span></p><p class=MsoNormal style='line-height:14.25pt;background:#1F1F1F'><span style='font-size:10.5pt;font-family:Consolas;color:#CCCCCC'><o:p> </o:p></span></p><p class=MsoNormal style='line-height:14.25pt;background:#1F1F1F'><span style='font-size:10.5pt;font-family:Consolas;color:#CCCCCC'> </span><span style='font-size:10.5pt;font-family:Consolas;color:#569CD6'>real</span><span style='font-size:10.5pt;font-family:Consolas;color:#CCCCCC'>(pREAL), </span><span style='font-size:10.5pt;font-family:Consolas;color:#569CD6'>pointer</span><span style='font-size:10.5pt;font-family:Consolas;color:#CCCCCC'>,</span><span style='font-size:10.5pt;font-family:Consolas;color:#569CD6'>dimension</span><span style='font-size:10.5pt;font-family:Consolas;color:#CCCCCC'>(:,:,:,:) </span><span style='font-size:10.5pt;font-family:Consolas;color:#D4D4D4'>::</span><span style='font-size:10.5pt;font-family:Consolas;color:#CCCCCC'> dF_scal, output_scal<o:p></o:p></span></p><p class=MsoNormal style='line-height:14.25pt;background:#1F1F1F'><span style='font-size:10.5pt;font-family:Consolas;color:#CCCCCC'><o:p> </o:p></span></p><p class=MsoNormal style='line-height:14.25pt;background:#1F1F1F'><span style='font-size:10.5pt;font-family:Consolas;color:#CCCCCC'> </span><span style='font-size:10.5pt;font-family:Consolas;color:#C586C0'>call</span><span style='font-size:10.5pt;font-family:Consolas;color:#CCCCCC'> </span><span style='font-size:10.5pt;font-family:Consolas;color:#DCDCAA'>SNESGetDM</span><span style='font-size:10.5pt;font-family:Consolas;color:#CCCCCC'>(SNES_mech,dm_local,err_PETSc)<o:p></o:p></span></p><p class=MsoNormal style='line-height:14.25pt;background:#1F1F1F'><span style='font-size:10.5pt;font-family:Consolas;color:#CCCCCC'> CHKERRQ(err_PETSc)<o:p></o:p></span></p><p class=MsoNormal style='line-height:14.25pt;background:#1F1F1F'><span style='font-size:10.5pt;font-family:Consolas;color:#CCCCCC'><o:p> </o:p></span></p><p class=MsoNormal style='line-height:14.25pt;background:#1F1F1F'><span style='font-size:10.5pt;font-family:Consolas;color:#CCCCCC'> output_global </span><span style='font-size:10.5pt;font-family:Consolas;color:#D4D4D4'>=</span><span style='font-size:10.5pt;font-family:Consolas;color:#CCCCCC'> dF_global</span><span style='font-size:10.5pt;font-family:Consolas;color:#569CD6'> <o:p></o:p></span></p><p class=MsoNormal style='line-height:14.25pt;background:#1F1F1F'><span style='font-size:10.5pt;font-family:Consolas;color:#569CD6'><o:p> </o:p></span></p><p class=MsoNormal style='line-height:14.25pt;background:#1F1F1F'><span style='font-size:10.5pt;font-family:Consolas;color:#569CD6'>end subroutine</span><span style='font-size:10.5pt;font-family:Consolas;color:#CCCCCC'> </span><span style='font-size:10.5pt;font-family:Consolas;color:#DCDCAA'>GK_op</span><span style='font-size:10.5pt;font-family:Consolas;color:#CCCCCC'><o:p></o:p></span></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal><span lang=EN-US>When I run with mpirun -np 3, it gives me similar error like previous, “Preconditioner number of local rows 27 does not equal input vector size 81”, (I changed my problem size so the numbers are different). <o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>Maybe simple equating input and output is not valid (due to ownership of different portion of a global vector). Then it may give me different error message. In fact my global dF has size 9*3*3*3, when running on 3 processors, the local dF has size 9*3*3*1 (I split my domain in z direction). The error message seems to suggest I am using a local dF rather than a global dF. And the output and input vector sizes seems to be different. Do I miss something here? <o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>Best regards,<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>Yi<o:p></o:p></span></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal><o:p> </o:p></p><div><div style='border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in'><p class=MsoNormal><b><span lang=EN-US>From:</span></b><span lang=EN-US> Barry Smith <bsmith@petsc.dev> <br><b>Sent:</b> Sunday, March 10, 2024 6:45 PM<br><b>To:</b> Yi Hu <y.hu@mpie.de><br><b>Cc:</b> Mark Adams <mfadams@lbl.gov>; petsc-users <petsc-users@mcs.anl.gov><br><b>Subject:</b> Re: [petsc-users] snes matrix-free jacobian fails with preconditioner shape mismatch<o:p></o:p></span></p></div></div><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal><o:p> </o:p></p><div><p class=MsoNormal><br><br><o:p></o:p></p><blockquote style='margin-top:5.0pt;margin-bottom:5.0pt'><div><p class=MsoNormal>On Mar 10, 2024, at 10:16 AM, Yi Hu <<a href="mailto:y.hu@mpie.de">y.hu@mpie.de</a>> wrote:<o:p></o:p></p></div><p class=MsoNormal><o:p> </o:p></p><div><div id=pfptBannercw8qp5q><div id=pfptBannercw8qp5q><div id=pfptBannercw8qp5q><p class=MsoNormal>This Message Is From an External Sender<o:p></o:p></p></div><div id=pfptBannercw8qp5q><p class=MsoNormal>This message came from outside your organization.<o:p></o:p></p></div></div></div><p style='caret-color: rgb(0, 0, 0);font-variant-caps: normal;text-align:start;-webkit-text-stroke-width: 0px;word-spacing:0px'><span style='font-size:13.5pt;font-family:"Helvetica",sans-serif'>Dear Mark,<o:p></o:p></span></p><p style='caret-color: rgb(0, 0, 0);font-variant-caps: normal;text-align:start;-webkit-text-stroke-width: 0px;word-spacing:0px'><span style='font-size:13.5pt;font-family:"Helvetica",sans-serif'>Thanks for your reply. I see this mismatch. In fact my global DoF is 324. It seems like I always get the local size = global Dof / np^2, np is my processor number. By the way, I used DMDASNESsetFunctionLocal() to set my form function. Is it eligible to mix DMDASNESsetFunctionLocal() and a native SNESSetJacobian()?<o:p></o:p></span></p></div></blockquote><div><p class=MsoNormal><o:p> </o:p></p></div><p class=MsoNormal>Yes<o:p></o:p></p></div><div><p class=MsoNormal><br><br><o:p></o:p></p><blockquote style='margin-top:5.0pt;margin-bottom:5.0pt'><div><p style='caret-color: rgb(0, 0, 0);font-variant-caps: normal;text-align:start;-webkit-text-stroke-width: 0px;word-spacing:0px'><span style='font-size:13.5pt;font-family:"Helvetica",sans-serif'>Best,<o:p></o:p></span></p><p style='caret-color: rgb(0, 0, 0);font-variant-caps: normal;text-align:start;-webkit-text-stroke-width: 0px;word-spacing:0px'><span style='font-size:13.5pt;font-family:"Helvetica",sans-serif'>Yi<o:p></o:p></span></p><div><p class=MsoNormal><span style='font-size:13.5pt;font-family:"Helvetica",sans-serif'>On 3/10/24 13:55, Mark Adams wrote:<o:p></o:p></span></p></div><blockquote style='margin-top:5.0pt;margin-bottom:5.0pt;font-variant-caps: normal;orphans: auto;text-align:start;widows: auto;-webkit-text-stroke-width: 0px;word-spacing:0px'><div><div><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>It looks like your input vector is the global vector, size 162, and the local matrix size is 81.<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>Mark<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>[1]PETSC ERROR: Preconditioner number of local rows 81 does not equal <o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>input vector size 162<o:p></o:p></span></pre></div><p class=MsoNormal><span style='font-size:13.5pt;font-family:"Helvetica",sans-serif'><o:p> </o:p></span></p><div><div><p class=MsoNormal><span style='font-size:13.5pt;font-family:"Helvetica",sans-serif'>On Sun, Mar 10, 2024 at 7:21 AM Yi Hu <<a href="mailto:y.hu@mpie.de">y.hu@mpie.de</a>> wrote:<o:p></o:p></span></p></div><blockquote style='border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in'><div><div style='border:none;border-top:solid #90A4AE 3.0pt;padding:6.0pt 0in 0in 0in;margin-top:12.0pt;margin-bottom:12.0pt;border-radius: 4px;min-width: 200px' id="m_-4788138792073467353pfptBanner2satfg2"><div style='margin-bottom:.75pt;float:left' id="m_-4788138792073467353pfptBanner2satfg2"><div id="m_-4788138792073467353pfptBanner2satfg2"><p class=MsoNormal style='line-height:13.5pt;background:#D0D8DC'><b><span style='font-size:10.5pt;font-family:"Arial",sans-serif'>This Message Is From an External Sender<span class=apple-converted-space> </span><o:p></o:p></span></b></p></div><div style='margin-top:1.5pt' id="m_-4788138792073467353pfptBanner2satfg2"><p class=MsoNormal style='line-height:13.5pt;background:#D0D8DC'><span style='font-size:9.0pt;font-family:"Arial",sans-serif'>This message came from outside your organization.<span class=apple-converted-space> </span><o:p></o:p></span></p></div></div><div><p class=MsoNormal style='line-height:0%;background:#D0D8DC'><span style='font-size:1.0pt;font-family:"Helvetica",sans-serif'> <o:p></o:p></span></p></div></div><div><pre style='white-space:pre-wrap'><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>Dear petsc team,<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'><o:p> </o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>I implemented a matrix-free jacobian, and it can run sequentially. But <o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>running parallel I got the pc error like this (running with mpirun -np <o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>2, only error from rank1 is presented here)<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'><o:p> </o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>[1]PETSC ERROR: --------------------- Error Message <o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>--------------------------------------------------------------<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>[1]PETSC ERROR: Nonconforming object sizes<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>[1]PETSC ERROR: Preconditioner number of local rows 81 does not equal <o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>input vector size 162<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>[1]PETSC ERROR: See <a href="https://urldefense.us/v3/__https:/petsc.org/release/faq/__;!!G_uCfscf7eWS!ahmistzr4wD3TJ0OvI0JWxB9aVSIbP78Jcs2X_6KMb4LdoR8drLB_DkHvaguhrca22RgFer0PlyUrtdfCA$" target="_blank">https://urldefense.us/v3/__https://petsc.org/release/faq/__;!!G_uCfscf7eWS!ahmistzr4wD3TJ0OvI0JWxB9aVSIbP78Jcs2X_6KMb4LdoR8drLB_DkHvaguhrca22RgFer0PlyUrtdfCA$</a> for trouble shooting.<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>[1]PETSC ERROR: Petsc Release Version 3.17.3, Jun 29, 2022<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>[1]PETSC ERROR: /home/yi/workspace/DAMASK_yi/bin/DAMASK_grid on a <o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>arch-linux-c-opt named carbon-x1 by yi Sun Mar 10 12:01:46 2024<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>[1]PETSC ERROR: Configure options --download-fftw --download-hdf5 <o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>--with-hdf5-fortran-bindings --download-fblaslapack --download-chaco <o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>--download-hypre --download-metis --download-mumps --download-parmetis <o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>--download-scalapack --download-suitesparse --download-superlu <o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>--download-superlu_dist --download-triangle --download-zlib <o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>--download-cmake --with-cxx-dialect=C++11 --with-c2html=0 <o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>--with-debugging=0 --with-ssl=0 --with-x=0 COPTFLAGS=-O3 CXXOPTFLAGS=-O3 <o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>FOPTFLAGS=-O3<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>[1]PETSC ERROR: #1 PCApply() at <o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>/home/yi/App/petsc-3.17.3/src/ksp/pc/interface/precon.c:424<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>[1]PETSC ERROR: #2 KSP_PCApply() at <o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>/home/yi/App/petsc-3.17.3/include/petsc/private/kspimpl.h:376<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>[1]PETSC ERROR: #3 KSPInitialResidual() at <o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>/home/yi/App/petsc-3.17.3/src/ksp/ksp/interface/itres.c:64<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>[1]PETSC ERROR: #4 KSPSolve_GMRES() at <o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>/home/yi/App/petsc-3.17.3/src/ksp/ksp/impls/gmres/gmres.c:242<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>[1]PETSC ERROR: #5 KSPSolve_Private() at <o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>/home/yi/App/petsc-3.17.3/src/ksp/ksp/interface/itfunc.c:902<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>[1]PETSC ERROR: #6 KSPSolve() at <o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>/home/yi/App/petsc-3.17.3/src/ksp/ksp/interface/itfunc.c:1078<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>[1]PETSC ERROR: #7 SNESSolve_NEWTONLS() at <o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>/home/yi/App/petsc-3.17.3/src/snes/impls/ls/ls.c:222<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>[1]PETSC ERROR: #8 SNESSolve() at <o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>/home/yi/App/petsc-3.17.3/src/snes/interface/snes.c:4756<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>[1]PETSC ERROR: #9 User provided function() at User <a href="file:///0">file:0</a><o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'><o:p> </o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>However, from snes matrix-free documentation <o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>(<a href="https://urldefense.us/v3/__https:/petsc.org/release/manual/snes/*matrix-free-methods__;Iw!!G_uCfscf7eWS!ahmistzr4wD3TJ0OvI0JWxB9aVSIbP78Jcs2X_6KMb4LdoR8drLB_DkHvaguhrca22RgFer0Ply6ZbywOw$" target="_blank">https://urldefense.us/v3/__https://petsc.org/release/manual/snes/*matrix-free-methods__;Iw!!G_uCfscf7eWS!ahmistzr4wD3TJ0OvI0JWxB9aVSIbP78Jcs2X_6KMb4LdoR8drLB_DkHvaguhrca22RgFer0Ply6ZbywOw$</a>), it is said <o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>matrix-free is used with pcnone. So I assume it would not apply <o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>preconditioner, but it did use preconditioning probably the same as my <o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>matrix-free shell matrix. Here is how i initialize my shell matrix and <o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>the corresponding customized multiplication.<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'><o:p> </o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'> call MatCreateShell(PETSC_COMM_WORLD,PETSC_DECIDE,PETSC_DECIDE,&<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>int(9*product(cells(1:2))*cells3,pPETSCINT),&<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>int(9*product(cells(1:2))*cells3,pPETSCINT),&<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'> F_PETSc,Jac_PETSc,err_PETSc)<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'> call MatShellSetOperation(Jac_PETSc,MATOP_MULT,GK_op,err_PETSc)<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'> call SNESSetDM(SNES_mech,DM_mech,err_PETSc)<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'> call <o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>SNESSetJacobian(SNES_mech,Jac_PETSc,Jac_PETSc,PETSC_NULL_FUNCTION,0,err_PETSc)<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'> call SNESGetKSP(SNES_mech,ksp,err_PETSc)<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'> call PCSetType(pc,PCNONE,err_PETSc)<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'><o:p> </o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>And my GK_op is like<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'><o:p> </o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>subroutine GK_op(Jac,dF_global,output_local,err_PETSc)<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'><o:p> </o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'> DM :: dm_local<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'> Vec :: dF_global, dF_local, output_local<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'> Mat :: Jac<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'> PetscErrorCode :: err_PETSc<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'><o:p> </o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'> real(pREAL), pointer,dimension(:,:,:,:) :: dF_scal, output_scal<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'><o:p> </o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'> real(pREAL), dimension(3,3,cells(1),cells(2),cells3) :: &<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'> dF<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'> real(pREAL), dimension(3,3,cells(1),cells(2),cells3) :: &<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'> output<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'><o:p> </o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'> call SNESGetDM(SNES_mech,dm_local,err_PETSc)<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'><o:p> </o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'> call DMGetLocalVector(dm_local,dF_local,err_PETSc)<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'> call <o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>DMGlobalToLocalBegin(dm_local,dF_global,INSERT_VALUES,dF_local,err_PETSc)<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'> call <o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>DMGlobalToLocalEnd(dm_local,dF_global,INSERT_VALUES,dF_local,err_PETSc)<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'><o:p> </o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'> call DMDAVecGetArrayReadF90(dm_local,dF_local,dF_scal,err_PETSc)<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'> dF = reshape(dF_scal, [3,3,cells(1),cells(2),cells3])<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'><o:p> </o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>.......<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'><o:p> </o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'><o:p> </o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'> call DMDAVecRestoreArrayF90(dm_local,output_local,output_scal,err_PETSc)<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'> CHKERRQ(err_PETSc)<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'><o:p> </o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'> call DMDAVecRestoreArrayF90(dm_local,dF_local,dF_scal,err_PETSc)<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'> CHKERRQ(err_PETSc)<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'><o:p> </o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>end subroutine GK_op<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'><o:p> </o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>I checked my cells3, it corresponds to my local size, and it seems the <o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>local size of dF_local is ok.<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'><o:p> </o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>I am a bit lost here to find the reason for the preconditioner bug. <o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>Could you help me on this? Thanks.<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'><o:p> </o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>Best regards,<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'><o:p> </o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>Yi<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'><o:p> </o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'><o:p> </o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'><o:p> </o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'><o:p> </o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>-------------------------------------------------<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>Stay up to date and follow us on LinkedIn, Twitter and YouTube.<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'><o:p> </o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>Max-Planck-Institut für Eisenforschung GmbH<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>Max-Planck-Straße 1<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>D-40237 Düsseldorf<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'> <o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>Handelsregister B 2533 <o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>Amtsgericht Düsseldorf<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'> <o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>Geschäftsführung<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>Prof. Dr. Gerhard Dehm<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>Prof. Dr. Jörg Neugebauer<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>Prof. Dr. Dierk Raabe<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>Dr. Kai de Weldige<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'> <o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>Ust.-Id.-Nr.: DE 11 93 58 514 <o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>Steuernummer: 105 5891 1000<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'><o:p> </o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'><o:p> </o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>Please consider that invitations and e-mails of our institute are <o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>only valid if they end with …@<a href="https://urldefense.us/v3/__http:/mpie.de__;!!G_uCfscf7eWS!eslPb6ZEnby8M7NKRe_U_RT-95w2J8O2ngsc8rmrQVlMpEcB6ZTKlN2g65crXCWv7D2F9ubkzYI5nDpoeQ$" target="_blank">mpie.de</a>. <o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>If you are not sure of the validity please contact <a href="mailto:rco@mpie.de" target="_blank">rco@mpie.de</a><o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'><o:p> </o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>Bitte beachten Sie, dass Einladungen zu Veranstaltungen und E-Mails<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>aus unserem Haus nur mit der Endung …@<a href="https://urldefense.us/v3/__http:/mpie.de__;!!G_uCfscf7eWS!eslPb6ZEnby8M7NKRe_U_RT-95w2J8O2ngsc8rmrQVlMpEcB6ZTKlN2g65crXCWv7D2F9ubkzYI5nDpoeQ$" target="_blank">mpie.de</a> gültig sind. <o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>In Zweifelsfällen wenden Sie sich bitte an <a href="mailto:rco@mpie.de" target="_blank">rco@mpie.de</a><o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'>-------------------------------------------------<o:p></o:p></span></pre><pre><span style='font-size:13.5pt;font-family:"Arial",sans-serif'><o:p> </o:p></span></pre></div></div></blockquote></div></div></blockquote><p class=MsoNormal><span style='font-size:13.5pt;font-family:"Helvetica",sans-serif'><br style='caret-color: rgb(0, 0, 0);font-variant-caps: normal;text-align:start;-webkit-text-stroke-width: 0px;word-spacing:0px'><br></span><o:p></o:p></p><div class=MsoNormal align=center style='text-align:center'><hr size=2 width="100%" align=center></div><p class=MsoNormal><span style='font-size:13.5pt;font-family:"Helvetica",sans-serif'>-------------------------------------------------<br>Stay up to date and follow us on LinkedIn, Twitter and YouTube.<br><br>Max-Planck-Institut für Eisenforschung GmbH<br>Max-Planck-Straße 1<br>D-40237 Düsseldorf<br> <br>Handelsregister B 2533 <br>Amtsgericht Düsseldorf<br> <br>Geschäftsführung<br>Prof. Dr. Gerhard Dehm<br>Prof. Dr. Jörg Neugebauer<br>Prof. Dr. Dierk Raabe<br>Dr. Kai de Weldige<br> <br>Ust.-Id.-Nr.: DE 11 93 58 514 <br>Steuernummer: 105 5891 1000<br><br><br>Please consider that invitations and e-mails of our institute are <br>only valid if they end with …@mpie.de. <br>If you are not sure of the validity please contact </span><a href="mailto:rco@mpie.de"><span style='font-size:13.5pt;font-family:"Helvetica",sans-serif'>rco@mpie.de</span></a><span style='font-size:13.5pt;font-family:"Helvetica",sans-serif'><br><br>Bitte beachten Sie, dass Einladungen zu Veranstaltungen und E-Mails<br>aus unserem Haus nur mit der Endung …@mpie.de gültig sind. <br>In Zweifelsfällen wenden Sie sich bitte an </span><a href="mailto:rco@mpie.de"><span style='font-size:13.5pt;font-family:"Helvetica",sans-serif'>rco@mpie.de</span></a><span style='font-size:13.5pt;font-family:"Helvetica",sans-serif'><br>-------------------------------------------------</span><o:p></o:p></p></div></blockquote></div><p class=MsoNormal><o:p> </o:p></p></div><BR />
<BR />
<HR />
-------------------------------------------------<BR />
Stay up to date and follow us on LinkedIn, Twitter and YouTube.<BR />
<BR />
Max-Planck-Institut für Eisenforschung GmbH<BR />
Max-Planck-Straße 1<BR />
D-40237 Düsseldorf<BR />
<BR />
Handelsregister B 2533 <BR />
Amtsgericht Düsseldorf<BR />
<BR />
Geschäftsführung<BR />
Prof. Dr. Gerhard Dehm<BR />
Prof. Dr. Jörg Neugebauer<BR />
Prof. Dr. Dierk Raabe<BR />
Dr. Kai de Weldige<BR />
<BR />
Ust.-Id.-Nr.: DE 11 93 58 514 <BR />
Steuernummer: 105 5891 1000<BR />
<BR />
<BR />
Please consider that invitations and e-mails of our institute are <BR />
only valid if they end with …@mpie.de. <BR />
If you are not sure of the validity please contact rco@mpie.de<BR />
<BR />
Bitte beachten Sie, dass Einladungen zu Veranstaltungen und E-Mails<BR />
aus unserem Haus nur mit der Endung …@mpie.de gültig sind. <BR />
In Zweifelsfällen wenden Sie sich bitte an rco@mpie.de<BR />
-------------------------------------------------<BR />
</body></html>