<!DOCTYPE html>
<!-- BaNnErBlUrFlE-BoDy-start -->
<!-- Preheader Text : BEGIN -->
<div style="display:none !important;display:none;visibility:hidden;mso-hide:all;font-size:1px;color:#ffffff;line-height:1px;height:0px;max-height:0px;opacity:0;overflow:hidden;">
Dear Mark, Thanks for your response. I found the bug. I used wrong M N when MatCreateShell(). Now it works in both uniprocessor and mpirun cases. Best, Yi On 3/11/24 13: 10, Mark Adams wrote: I think you misunderstand "global" vector. The global
</div>
<!-- Preheader Text : END -->
<!-- Email Banner : BEGIN -->
<div style="display:none !important;display:none;visibility:hidden;mso-hide:all;font-size:1px;color:#ffffff;line-height:1px;height:0px;max-height:0px;opacity:0;overflow:hidden;">ZjQcmQRYFpfptBannerStart</div>
<!--[if ((ie)|(mso))]>
<table border="0" cellspacing="0" cellpadding="0" width="100%" style="padding: 16px 0px 16px 0px; direction: ltr" ><tr><td>
<table border="0" cellspacing="0" cellpadding="0" style="padding: 0px 10px 5px 6px; width: 100%; border-radius:4px; border-top:4px solid #90a4ae;background-color:#D0D8DC;"><tr><td valign="top">
<table align="left" border="0" cellspacing="0" cellpadding="0" style="padding: 4px 8px 4px 8px">
<tr><td style="color:#000000; font-family: 'Arial', sans-serif; font-weight:bold; font-size:14px; direction: ltr">
This Message Is From an External Sender
</td></tr>
<tr><td style="color:#000000; font-weight:normal; font-family: 'Arial', sans-serif; font-size:12px; direction: ltr">
This message came from outside your organization.
</td></tr>
</table>
</td></tr></table>
</td></tr></table>
<![endif]-->
<![if !((ie)|(mso))]>
<div dir="ltr" id="pfptBannernwlqhl4" style="all: revert !important; display:block !important; text-align: left !important; margin:16px 0px 16px 0px !important; padding:8px 16px 8px 16px !important; border-radius: 4px !important; min-width: 200px !important; background-color: #D0D8DC !important; background-color: #D0D8DC; border-top: 4px solid #90a4ae !important; border-top: 4px solid #90a4ae;">
<div id="pfptBannernwlqhl4" style="all: unset !important; float:left !important; display:block !important; margin: 0px 0px 1px 0px !important; max-width: 600px !important;">
<div id="pfptBannernwlqhl4" style="all: unset !important; display:block !important; visibility: visible !important; background-color: #D0D8DC !important; color:#000000 !important; color:#000000; font-family: 'Arial', sans-serif !important; font-family: 'Arial', sans-serif; font-weight:bold !important; font-weight:bold; font-size:14px !important; line-height:18px !important; line-height:18px">
This Message Is From an External Sender
</div>
<div id="pfptBannernwlqhl4" style="all: unset !important; display:block !important; visibility: visible !important; background-color: #D0D8DC !important; color:#000000 !important; color:#000000; font-weight:normal; font-family: 'Arial', sans-serif !important; font-family: 'Arial', sans-serif; font-size:12px !important; line-height:18px !important; line-height:18px; margin-top:2px !important;">
This message came from outside your organization.
</div>
</div>
<div style="clear: both !important; display: block !important; visibility: hidden !important; line-height: 0 !important; font-size: 0.01px !important; height: 0px"> </div>
</div>
<![endif]>
<div style="display:none !important;display:none;visibility:hidden;mso-hide:all;font-size:1px;color:#ffffff;line-height:1px;height:0px;max-height:0px;opacity:0;overflow:hidden;">ZjQcmQRYFpfptBannerEnd</div>
<!-- Email Banner : END -->
<!-- BaNnErBlUrFlE-BoDy-end -->
<html>
<head><!-- BaNnErBlUrFlE-HeAdEr-start -->
<style>
#pfptBannernwlqhl4 { all: revert !important; display: block !important;
visibility: visible !important; opacity: 1 !important;
background-color: #D0D8DC !important;
max-width: none !important; max-height: none !important }
.pfptPrimaryButtonnwlqhl4:hover, .pfptPrimaryButtonnwlqhl4:focus {
background-color: #b4c1c7 !important; }
.pfptPrimaryButtonnwlqhl4:active {
background-color: #90a4ae !important; }
</style>
<!-- BaNnErBlUrFlE-HeAdEr-end -->
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p>Dear Mark,</p>
<p>Thanks for your response. I found the bug. I used wrong M N when
MatCreateShell(). Now it works in both uniprocessor and mpirun
cases.</p>
<p>Best,<br>
Yi<br>
</p>
<div class="moz-cite-prefix">On 3/11/24 13:10, Mark Adams wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CADOhEh400e7KmgLDLFaN3tqD+vz=ndPTD5J1Y+DK4xYmjaP6HA@mail.gmail.com">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<div dir="ltr">I think you misunderstand "global" vector. The
global solution is never on one processor. A global vector in
this case seems to have 81 global values and 27 local values.
<div>It looks like you create a "global" vector that has 81
local values that should never be created other than for
debigging.</div>
<div>GlobalToLocal refers to a "local" vector with ghost cells,
so it would have > 27 values in this case and you use it
for local operations only (its communicator is
PETSC_COMM_SELF).</div>
<div>Hope this helps,</div>
<div>Mark</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Mon, Mar 11, 2024 at
4:48 AM Yi Hu <<a href="mailto:y.hu@mpie.de"
moz-do-not-send="true" class="moz-txt-link-freetext">y.hu@mpie.de</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote"
style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div class="msg6768418666828824524">
<div lang="en-DE">
<div class="m_6768418666828824524WordSection1">
<p class="MsoNormal"><span lang="EN-US">Dear Barry,</span></p>
<p class="MsoNormal"><span lang="EN-US"> </span></p>
<p class="MsoNormal"><span lang="EN-US">Thanks for your
response. Now I am doing simple debugging for my
customized mat_mult_op of my shell jacobian matrix.
As far as I understand, because the input and output
of shell jacobian are all global vectors, I need to
do the global to local mapping of my input vector
(dF) by myself. Before starting debugging the
mapping, I first try to verify the size match of
input and output.</span></p>
<p class="MsoNormal"><span lang="EN-US"> </span></p>
<p class="MsoNormal"><span lang="EN-US">Because the
input and output of a mat_mult_op of my shell matrix
should have the same size. So I tried just equating
them in my customized mat_mult_op. Basically like
this </span></p>
<p class="MsoNormal"><span lang="EN-US"> </span></p>
<p class="MsoNormal"
style="line-height:14.25pt;background:rgb(31,31,31)"><span
style="font-size:10.5pt;font-family:Consolas;color:rgb(86,156,214)">subroutine</span><span
style="font-size:10.5pt;font-family:Consolas;color:rgb(204,204,204)"> </span><span
style="font-size:10.5pt;font-family:Consolas;color:rgb(220,220,170)">GK_op</span><span
style="font-size:10.5pt;font-family:Consolas;color:rgb(204,204,204)">(</span><span
style="font-size:10.5pt;font-family:Consolas;color:rgb(156,220,254)">Jac</span><span
style="font-size:10.5pt;font-family:Consolas;color:rgb(204,204,204)">,</span><span
style="font-size:10.5pt;font-family:Consolas;color:rgb(156,220,254)">dF_global</span><span
style="font-size:10.5pt;font-family:Consolas;color:rgb(204,204,204)">,</span><span
style="font-size:10.5pt;font-family:Consolas;color:rgb(156,220,254)">output_global</span><span
style="font-size:10.5pt;font-family:Consolas;color:rgb(204,204,204)">,</span><span
style="font-size:10.5pt;font-family:Consolas;color:rgb(156,220,254)">err_PETSc</span><span
style="font-size:10.5pt;font-family:Consolas;color:rgb(204,204,204)">)</span></p>
<p class="MsoNormal"
style="line-height:14.25pt;background:rgb(31,31,31)"><span
style="font-size:10.5pt;font-family:Consolas;color:rgb(204,204,204)"> </span></p>
<p class="MsoNormal"
style="line-height:14.25pt;background:rgb(31,31,31)"><span
style="font-size:10.5pt;font-family:Consolas;color:rgb(204,204,204)">
DM :: dm_local </span><span
style="font-size:10.5pt;font-family:Consolas;color:rgb(106,153,85)">!
Yi: later for is,ie</span><span
style="font-size:10.5pt;font-family:Consolas;color:rgb(204,204,204)"></span></p>
<p class="MsoNormal"
style="line-height:14.25pt;background:rgb(31,31,31)"><span
style="font-size:10.5pt;font-family:Consolas;color:rgb(204,204,204)">
Vec :: dF_global</span></p>
<p class="MsoNormal"
style="line-height:14.25pt;background:rgb(31,31,31)"><span
style="font-size:10.5pt;font-family:Consolas;color:rgb(204,204,204)">
Vec ::
output_global</span></p>
<p class="MsoNormal"
style="line-height:14.25pt;background:rgb(31,31,31)"><span
style="font-size:10.5pt;font-family:Consolas;color:rgb(204,204,204)">
PetscErrorCode :: err_PETSc</span></p>
<p class="MsoNormal"
style="line-height:14.25pt;background:rgb(31,31,31)"><span
style="font-size:10.5pt;font-family:Consolas;color:rgb(204,204,204)"> </span></p>
<p class="MsoNormal"
style="line-height:14.25pt;background:rgb(31,31,31)"><span
style="font-size:10.5pt;font-family:Consolas;color:rgb(204,204,204)"> </span><span
style="font-size:10.5pt;font-family:Consolas;color:rgb(86,156,214)">real</span><span
style="font-size:10.5pt;font-family:Consolas;color:rgb(204,204,204)">(pREAL),
</span><span
style="font-size:10.5pt;font-family:Consolas;color:rgb(86,156,214)">pointer</span><span
style="font-size:10.5pt;font-family:Consolas;color:rgb(204,204,204)">,</span><span
style="font-size:10.5pt;font-family:Consolas;color:rgb(86,156,214)">dimension</span><span
style="font-size:10.5pt;font-family:Consolas;color:rgb(204,204,204)">(:,:,:,:)
</span><span
style="font-size:10.5pt;font-family:Consolas;color:rgb(212,212,212)">::</span><span
style="font-size:10.5pt;font-family:Consolas;color:rgb(204,204,204)">
dF_scal, output_scal</span></p>
<p class="MsoNormal"
style="line-height:14.25pt;background:rgb(31,31,31)"><span
style="font-size:10.5pt;font-family:Consolas;color:rgb(204,204,204)"> </span></p>
<p class="MsoNormal"
style="line-height:14.25pt;background:rgb(31,31,31)"><span
style="font-size:10.5pt;font-family:Consolas;color:rgb(204,204,204)"> </span><span
style="font-size:10.5pt;font-family:Consolas;color:rgb(197,134,192)">call</span><span
style="font-size:10.5pt;font-family:Consolas;color:rgb(204,204,204)"> </span><span
style="font-size:10.5pt;font-family:Consolas;color:rgb(220,220,170)">SNESGetDM</span><span
style="font-size:10.5pt;font-family:Consolas;color:rgb(204,204,204)">(SNES_mech,dm_local,err_PETSc)</span></p>
<p class="MsoNormal"
style="line-height:14.25pt;background:rgb(31,31,31)"><span
style="font-size:10.5pt;font-family:Consolas;color:rgb(204,204,204)">
CHKERRQ(err_PETSc)</span></p>
<p class="MsoNormal"
style="line-height:14.25pt;background:rgb(31,31,31)"><span
style="font-size:10.5pt;font-family:Consolas;color:rgb(204,204,204)"> </span></p>
<p class="MsoNormal"
style="line-height:14.25pt;background:rgb(31,31,31)"><span
style="font-size:10.5pt;font-family:Consolas;color:rgb(204,204,204)">
output_global </span><span
style="font-size:10.5pt;font-family:Consolas;color:rgb(212,212,212)">=</span><span
style="font-size:10.5pt;font-family:Consolas;color:rgb(204,204,204)">
dF_global</span><span
style="font-size:10.5pt;font-family:Consolas;color:rgb(86,156,214)"> </span></p>
<p class="MsoNormal"
style="line-height:14.25pt;background:rgb(31,31,31)"><span
style="font-size:10.5pt;font-family:Consolas;color:rgb(86,156,214)"> </span></p>
<p class="MsoNormal"
style="line-height:14.25pt;background:rgb(31,31,31)"><span
style="font-size:10.5pt;font-family:Consolas;color:rgb(86,156,214)">end
subroutine</span><span
style="font-size:10.5pt;font-family:Consolas;color:rgb(204,204,204)"> </span><span
style="font-size:10.5pt;font-family:Consolas;color:rgb(220,220,170)">GK_op</span><span
style="font-size:10.5pt;font-family:Consolas;color:rgb(204,204,204)"></span></p>
<p class="MsoNormal"> </p>
<p class="MsoNormal"><span lang="EN-US">When I run with
mpirun -np 3, it gives me similar error like
previous, “Preconditioner number of local rows 27
does not equal input vector size 81”, (I changed my
problem size so the numbers are different). </span></p>
<p class="MsoNormal"><span lang="EN-US"> </span></p>
<p class="MsoNormal"><span lang="EN-US">Maybe simple
equating input and output is not valid (due to
ownership of different portion of a global vector).
Then it may give me different error message. In fact
my global dF has size 9*3*3*3, when running on 3
processors, the local dF has size 9*3*3*1 (I split
my domain in z direction). The error message seems
to suggest I am using a local dF rather than a
global dF. And the output and input vector sizes
seems to be different. Do I miss something here? </span></p>
<p class="MsoNormal"><span lang="EN-US"> </span></p>
<p class="MsoNormal"><span lang="EN-US">Best regards,</span></p>
<p class="MsoNormal"><span lang="EN-US">Yi</span></p>
<p class="MsoNormal"> </p>
<p class="MsoNormal"> </p>
<div>
<div
style="border-right:none;border-bottom:none;border-left:none;border-top:1pt solid rgb(225,225,225);padding:3pt 0in 0in">
<p class="MsoNormal"><b><span lang="EN-US">From:</span></b><span
lang="EN-US"> Barry Smith <<a
href="mailto:bsmith@petsc.dev" target="_blank"
moz-do-not-send="true"
class="moz-txt-link-freetext">bsmith@petsc.dev</a>>
<br>
<b>Sent:</b> Sunday, March 10, 2024 6:45 PM<br>
<b>To:</b> Yi Hu <<a
href="mailto:y.hu@mpie.de" target="_blank"
moz-do-not-send="true"
class="moz-txt-link-freetext">y.hu@mpie.de</a>><br>
<b>Cc:</b> Mark Adams <<a
href="mailto:mfadams@lbl.gov" target="_blank"
moz-do-not-send="true"
class="moz-txt-link-freetext">mfadams@lbl.gov</a>>;
petsc-users <<a
href="mailto:petsc-users@mcs.anl.gov"
target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">petsc-users@mcs.anl.gov</a>><br>
<b>Subject:</b> Re: [petsc-users] snes
matrix-free jacobian fails with preconditioner
shape mismatch</span></p>
</div>
</div>
<p class="MsoNormal"> </p>
<p class="MsoNormal"> </p>
<div>
<p class="MsoNormal"><br>
<br>
</p>
<blockquote style="margin-top:5pt;margin-bottom:5pt">
<div>
<p class="MsoNormal">On Mar 10, 2024, at 10:16 AM,
Yi Hu <<a href="mailto:y.hu@mpie.de"
target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">y.hu@mpie.de</a>>
wrote:</p>
</div>
<p class="MsoNormal"> </p>
<div>
<div id="m_6768418666828824524pfptBannercw8qp5q">
<div id="m_6768418666828824524pfptBannercw8qp5q">
<div
id="m_6768418666828824524pfptBannercw8qp5q">
<p class="MsoNormal">This Message Is From an
External Sender</p>
</div>
<div
id="m_6768418666828824524pfptBannercw8qp5q">
<p class="MsoNormal">This message came from
outside your organization.</p>
</div>
</div>
</div>
<p
style="font-variant-caps:normal;text-align:start;word-spacing:0px"><span
style="font-size:13.5pt;font-family:Helvetica,sans-serif">Dear Mark,</span></p>
<p
style="font-variant-caps:normal;text-align:start;word-spacing:0px"><span
style="font-size:13.5pt;font-family:Helvetica,sans-serif">Thanks for
your reply. I see this mismatch. In fact my
global DoF is 324. It seems like I always get
the local size = global Dof / np^2, np is my
processor number. By the way, I used
DMDASNESsetFunctionLocal() to set my form
function. Is it eligible to mix
DMDASNESsetFunctionLocal() and a native
SNESSetJacobian()?</span></p>
</div>
</blockquote>
<div>
<p class="MsoNormal"> </p>
</div>
<p class="MsoNormal">Yes</p>
</div>
<div>
<p class="MsoNormal"><br>
<br>
</p>
<blockquote style="margin-top:5pt;margin-bottom:5pt">
<div>
<p
style="font-variant-caps:normal;text-align:start;word-spacing:0px"><span
style="font-size:13.5pt;font-family:Helvetica,sans-serif">Best,</span></p>
<p
style="font-variant-caps:normal;text-align:start;word-spacing:0px"><span
style="font-size:13.5pt;font-family:Helvetica,sans-serif">Yi</span></p>
<div>
<p class="MsoNormal"><span
style="font-size:13.5pt;font-family:Helvetica,sans-serif">On 3/10/24
13:55, Mark Adams wrote:</span></p>
</div>
<blockquote
style="margin-top:5pt;margin-bottom:5pt;font-variant-caps:normal;text-align:start;word-spacing:0px">
<div>
<div>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">It looks like your input vector is the global vector, size 162, and the local matrix size is 81.</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">Mark</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">[1]PETSC ERROR: Preconditioner number of local rows 81 does not equal </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">input vector size 162</span></pre>
</div>
<p class="MsoNormal"><span
style="font-size:13.5pt;font-family:Helvetica,sans-serif"> </span></p>
<div>
<div>
<p class="MsoNormal"><span
style="font-size:13.5pt;font-family:Helvetica,sans-serif">On Sun, Mar
10, 2024 at 7:21 AM Yi Hu <<a
href="mailto:y.hu@mpie.de"
target="_blank"
moz-do-not-send="true"
class="moz-txt-link-freetext">y.hu@mpie.de</a>>
wrote:</span></p>
</div>
<blockquote
style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0in 0in 0in 6pt;margin-left:4.8pt;margin-right:0in">
<div>
<div
style="border-right:none;border-bottom:none;border-left:none;border-top:3pt solid rgb(144,164,174);padding:6pt 0in 0in;margin-top:12pt;margin-bottom:12pt;border-radius:4px;min-width:200px"
id="m_6768418666828824524m_-4788138792073467353pfptBanner2satfg2">
<div
style="margin-bottom:0.75pt;float:left"
id="m_6768418666828824524m_-4788138792073467353pfptBanner2satfg2">
<div
id="m_6768418666828824524m_-4788138792073467353pfptBanner2satfg2">
<p class="MsoNormal"
style="line-height:13.5pt;background:rgb(208,216,220)"><b><span
style="font-size:10.5pt;font-family:Arial,sans-serif">This Message Is
From an External Sender<span
class="m_6768418666828824524apple-converted-space"> </span></span></b></p>
</div>
<div style="margin-top:1.5pt"
id="m_6768418666828824524m_-4788138792073467353pfptBanner2satfg2">
<p class="MsoNormal"
style="line-height:13.5pt;background:rgb(208,216,220)"><span
style="font-size:9pt;font-family:Arial,sans-serif">This message came
from outside your
organization.<span
class="m_6768418666828824524apple-converted-space"> </span></span></p>
</div>
</div>
<div>
<p class="MsoNormal"
style="line-height:0%;background:rgb(208,216,220)"><span
style="font-size:1pt;font-family:Helvetica,sans-serif"> </span></p>
</div>
</div>
<div>
<pre style="white-space:pre-wrap"><span
style="font-size:13.5pt;font-family:Arial,sans-serif">Dear petsc team,</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif"> </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">I implemented a matrix-free jacobian, and it can run sequentially. But </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">running parallel I got the pc error like this (running with mpirun -np </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">2, only error from rank1 is presented here)</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif"> </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">[1]PETSC ERROR: --------------------- Error Message </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">--------------------------------------------------------------</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">[1]PETSC ERROR: Nonconforming object sizes</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">[1]PETSC ERROR: Preconditioner number of local rows 81 does not equal </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">input vector size 162</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">[1]PETSC ERROR: See <a
href="https://urldefense.us/v3/__https:/petsc.org/release/faq/__;!!G_uCfscf7eWS!ahmistzr4wD3TJ0OvI0JWxB9aVSIbP78Jcs2X_6KMb4LdoR8drLB_DkHvaguhrca22RgFer0PlyUrtdfCA$"
target="_blank" moz-do-not-send="true">https://urldefense.us/v3/__https://petsc.org/release/faq/__;!!G_uCfscf7eWS!ahmistzr4wD3TJ0OvI0JWxB9aVSIbP78Jcs2X_6KMb4LdoR8drLB_DkHvaguhrca22RgFer0PlyUrtdfCA$</a> for trouble shooting.</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">[1]PETSC ERROR: Petsc Release Version 3.17.3, Jun 29, 2022</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">[1]PETSC ERROR: /home/yi/workspace/DAMASK_yi/bin/DAMASK_grid on a </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">arch-linux-c-opt named carbon-x1 by yi Sun Mar 10 12:01:46 2024</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">[1]PETSC ERROR: Configure options --download-fftw --download-hdf5 </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">--with-hdf5-fortran-bindings --download-fblaslapack --download-chaco </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">--download-hypre --download-metis --download-mumps --download-parmetis </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">--download-scalapack --download-suitesparse --download-superlu </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">--download-superlu_dist --download-triangle --download-zlib </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">--download-cmake --with-cxx-dialect=C++11 --with-c2html=0 </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">--with-debugging=0 --with-ssl=0 --with-x=0 COPTFLAGS=-O3 CXXOPTFLAGS=-O3 </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">FOPTFLAGS=-O3</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">[1]PETSC ERROR: #1 PCApply() at </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">/home/yi/App/petsc-3.17.3/src/ksp/pc/interface/precon.c:424</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">[1]PETSC ERROR: #2 KSP_PCApply() at </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">/home/yi/App/petsc-3.17.3/include/petsc/private/kspimpl.h:376</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">[1]PETSC ERROR: #3 KSPInitialResidual() at </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">/home/yi/App/petsc-3.17.3/src/ksp/ksp/interface/itres.c:64</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">[1]PETSC ERROR: #4 KSPSolve_GMRES() at </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">/home/yi/App/petsc-3.17.3/src/ksp/ksp/impls/gmres/gmres.c:242</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">[1]PETSC ERROR: #5 KSPSolve_Private() at </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">/home/yi/App/petsc-3.17.3/src/ksp/ksp/interface/itfunc.c:902</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">[1]PETSC ERROR: #6 KSPSolve() at </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">/home/yi/App/petsc-3.17.3/src/ksp/ksp/interface/itfunc.c:1078</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">[1]PETSC ERROR: #7 SNESSolve_NEWTONLS() at </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">/home/yi/App/petsc-3.17.3/src/snes/impls/ls/ls.c:222</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">[1]PETSC ERROR: #8 SNESSolve() at </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">/home/yi/App/petsc-3.17.3/src/snes/interface/snes.c:4756</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">[1]PETSC ERROR: #9 User provided function() at User <a
moz-do-not-send="true">file:0</a></span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif"> </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">However, from snes matrix-free documentation </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">(<a
href="https://urldefense.us/v3/__https:/petsc.org/release/manual/snes/*matrix-free-methods__;Iw!!G_uCfscf7eWS!ahmistzr4wD3TJ0OvI0JWxB9aVSIbP78Jcs2X_6KMb4LdoR8drLB_DkHvaguhrca22RgFer0Ply6ZbywOw$"
target="_blank" moz-do-not-send="true">https://urldefense.us/v3/__https://petsc.org/release/manual/snes/*matrix-free-methods__;Iw!!G_uCfscf7eWS!ahmistzr4wD3TJ0OvI0JWxB9aVSIbP78Jcs2X_6KMb4LdoR8drLB_DkHvaguhrca22RgFer0Ply6ZbywOw$</a>), it is said </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">matrix-free is used with pcnone. So I assume it would not apply </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">preconditioner, but it did use preconditioning probably the same as my </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">matrix-free shell matrix. Here is how i initialize my shell matrix and </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">the corresponding customized multiplication.</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif"> </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif"> call MatCreateShell(PETSC_COMM_WORLD,PETSC_DECIDE,PETSC_DECIDE,&</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">int(9*product(cells(1:2))*cells3,pPETSCINT),&</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">int(9*product(cells(1:2))*cells3,pPETSCINT),&</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif"> F_PETSc,Jac_PETSc,err_PETSc)</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif"> call MatShellSetOperation(Jac_PETSc,MATOP_MULT,GK_op,err_PETSc)</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif"> call SNESSetDM(SNES_mech,DM_mech,err_PETSc)</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif"> call </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">SNESSetJacobian(SNES_mech,Jac_PETSc,Jac_PETSc,PETSC_NULL_FUNCTION,0,err_PETSc)</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif"> call SNESGetKSP(SNES_mech,ksp,err_PETSc)</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif"> call PCSetType(pc,PCNONE,err_PETSc)</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif"> </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">And my GK_op is like</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif"> </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">subroutine GK_op(Jac,dF_global,output_local,err_PETSc)</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif"> </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif"> DM :: dm_local</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif"> Vec :: dF_global, dF_local, output_local</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif"> Mat :: Jac</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif"> PetscErrorCode :: err_PETSc</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif"> </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif"> real(pREAL), pointer,dimension(:,:,:,:) :: dF_scal, output_scal</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif"> </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif"> real(pREAL), dimension(3,3,cells(1),cells(2),cells3) :: &</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif"> dF</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif"> real(pREAL), dimension(3,3,cells(1),cells(2),cells3) :: &</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif"> output</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif"> </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif"> call SNESGetDM(SNES_mech,dm_local,err_PETSc)</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif"> </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif"> call DMGetLocalVector(dm_local,dF_local,err_PETSc)</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif"> call </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">DMGlobalToLocalBegin(dm_local,dF_global,INSERT_VALUES,dF_local,err_PETSc)</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif"> call </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">DMGlobalToLocalEnd(dm_local,dF_global,INSERT_VALUES,dF_local,err_PETSc)</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif"> </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif"> call DMDAVecGetArrayReadF90(dm_local,dF_local,dF_scal,err_PETSc)</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif"> dF = reshape(dF_scal, [3,3,cells(1),cells(2),cells3])</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif"> </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">.......</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif"> </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif"> </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif"> call DMDAVecRestoreArrayF90(dm_local,output_local,output_scal,err_PETSc)</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif"> CHKERRQ(err_PETSc)</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif"> </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif"> call DMDAVecRestoreArrayF90(dm_local,dF_local,dF_scal,err_PETSc)</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif"> CHKERRQ(err_PETSc)</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif"> </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">end subroutine GK_op</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif"> </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">I checked my cells3, it corresponds to my local size, and it seems the </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">local size of dF_local is ok.</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif"> </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">I am a bit lost here to find the reason for the preconditioner bug. </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">Could you help me on this? Thanks.</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif"> </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">Best regards,</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif"> </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">Yi</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif"> </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif"> </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif"> </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif"> </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">-------------------------------------------------</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">Stay up to date and follow us on LinkedIn, Twitter and YouTube.</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif"> </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">Max-Planck-Institut für Eisenforschung GmbH</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">Max-Planck-Straße 1</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">D-40237 Düsseldorf</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif"> </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">Handelsregister B 2533 </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">Amtsgericht Düsseldorf</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif"> </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">Geschäftsführung</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">Prof. Dr. Gerhard Dehm</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">Prof. Dr. Jörg Neugebauer</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">Prof. Dr. Dierk Raabe</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">Dr. Kai de Weldige</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif"> </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">Ust.-Id.-Nr.: DE 11 93 58 514 </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">Steuernummer: 105 5891 1000</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif"> </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif"> </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">Please consider that invitations and e-mails of our institute are </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">only valid if they end with …@<a
href="https://urldefense.us/v3/__http:/mpie.de__;!!G_uCfscf7eWS!eslPb6ZEnby8M7NKRe_U_RT-95w2J8O2ngsc8rmrQVlMpEcB6ZTKlN2g65crXCWv7D2F9ubkzYI5nDpoeQ$"
target="_blank" moz-do-not-send="true">mpie.de</a>. </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">If you are not sure of the validity please contact <a
href="mailto:rco@mpie.de"
target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">rco@mpie.de</a></span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif"> </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">Bitte beachten Sie, dass Einladungen zu Veranstaltungen und E-Mails</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">aus unserem Haus nur mit der Endung …@<a
href="https://urldefense.us/v3/__http:/mpie.de__;!!G_uCfscf7eWS!eslPb6ZEnby8M7NKRe_U_RT-95w2J8O2ngsc8rmrQVlMpEcB6ZTKlN2g65crXCWv7D2F9ubkzYI5nDpoeQ$"
target="_blank" moz-do-not-send="true">mpie.de</a> gültig sind. </span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">In Zweifelsfällen wenden Sie sich bitte an <a
href="mailto:rco@mpie.de"
target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">rco@mpie.de</a></span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif">-------------------------------------------------</span></pre>
<pre><span
style="font-size:13.5pt;font-family:Arial,sans-serif"> </span></pre>
</div>
</div>
</blockquote>
</div>
</div>
</blockquote>
<p class="MsoNormal"><span
style="font-size:13.5pt;font-family:Helvetica,sans-serif"><br
style="font-variant-caps:normal;text-align:start;word-spacing:0px">
<br>
</span></p>
<div class="MsoNormal" style="text-align:center"
align="center">
<hr width="100%" size="2" align="center"></div>
<p class="MsoNormal"><span
style="font-size:13.5pt;font-family:Helvetica,sans-serif">-------------------------------------------------<br>
Stay up to date and follow us on LinkedIn, Twitter and YouTube.<br>
<br>
Max-Planck-Institut für Eisenforschung GmbH<br>
Max-Planck-Straße 1<br>
D-40237 Düsseldorf<br>
<br>
Handelsregister B 2533 <br>
Amtsgericht Düsseldorf<br>
<br>
Geschäftsführung<br>
Prof. Dr. Gerhard Dehm<br>
Prof. Dr. Jörg Neugebauer<br>
Prof. Dr. Dierk Raabe<br>
Dr. Kai de Weldige<br>
<br>
Ust.-Id.-Nr.: DE 11 93 58 514 <br>
Steuernummer: 105 5891 1000<br>
<br>
<br>
Please consider that invitations and e-mails of our institute are <br>
only valid if they end with …@<a href="https://urldefense.us/v3/__http://mpie.de__;!!G_uCfscf7eWS!cggahkQVL4itzNc4xCIuB95w85Py0YzJ_d0ueBg3CxKFPTOpO7ANg7u8XPX1w9DBPF_tpI5-RXi12HTwVA$" target="_blank" moz-do-not-send="true">mpie.de</a>. <br>
If you are not sure of the validity please contact </span><a
href="mailto:rco@mpie.de" target="_blank"
moz-do-not-send="true"><span
style="font-size:13.5pt;font-family:Helvetica,sans-serif">rco@mpie.de</span></a><span
style="font-size:13.5pt;font-family:Helvetica,sans-serif"><br>
<br>
Bitte beachten Sie, dass Einladungen zu Veranstaltungen und E-Mails<br>
aus unserem Haus nur mit der Endung …@<a href="https://urldefense.us/v3/__http://mpie.de__;!!G_uCfscf7eWS!cggahkQVL4itzNc4xCIuB95w85Py0YzJ_d0ueBg3CxKFPTOpO7ANg7u8XPX1w9DBPF_tpI5-RXi12HTwVA$" target="_blank" moz-do-not-send="true">mpie.de</a> gültig sind. <br>
In Zweifelsfällen wenden Sie sich bitte an </span><a
href="mailto:rco@mpie.de" target="_blank"
moz-do-not-send="true"><span
style="font-size:13.5pt;font-family:Helvetica,sans-serif">rco@mpie.de</span></a><span
style="font-size:13.5pt;font-family:Helvetica,sans-serif"><br>
-------------------------------------------------</span></p>
</div>
</blockquote>
</div>
<p class="MsoNormal"> </p>
</div>
<br>
<br>
<hr>
-------------------------------------------------<br>
Stay up to date and follow us on LinkedIn, Twitter and YouTube.<br>
<br>
Max-Planck-Institut für Eisenforschung GmbH<br>
Max-Planck-Straße 1<br>
D-40237 Düsseldorf<br>
<br>
Handelsregister B 2533 <br>
Amtsgericht Düsseldorf<br>
<br>
Geschäftsführung<br>
Prof. Dr. Gerhard Dehm<br>
Prof. Dr. Jörg Neugebauer<br>
Prof. Dr. Dierk Raabe<br>
Dr. Kai de Weldige<br>
<br>
Ust.-Id.-Nr.: DE 11 93 58 514 <br>
Steuernummer: 105 5891 1000<br>
<br>
<br>
Please consider that invitations and e-mails of our institute are <br>
only valid if they end with …@<a href="https://urldefense.us/v3/__http://mpie.de__;!!G_uCfscf7eWS!cggahkQVL4itzNc4xCIuB95w85Py0YzJ_d0ueBg3CxKFPTOpO7ANg7u8XPX1w9DBPF_tpI5-RXi12HTwVA$" target="_blank" moz-do-not-send="true">mpie.de</a>. <br>
If you are not sure of the validity please contact <a
href="mailto:rco@mpie.de" target="_blank"
moz-do-not-send="true" class="moz-txt-link-freetext">rco@mpie.de</a><br>
<br>
Bitte beachten Sie, dass Einladungen zu Veranstaltungen und E-Mails<br>
aus unserem Haus nur mit der Endung …@<a href="https://urldefense.us/v3/__http://mpie.de__;!!G_uCfscf7eWS!cggahkQVL4itzNc4xCIuB95w85Py0YzJ_d0ueBg3CxKFPTOpO7ANg7u8XPX1w9DBPF_tpI5-RXi12HTwVA$" target="_blank" moz-do-not-send="true">mpie.de</a> gültig sind. <br>
In Zweifelsfällen wenden Sie sich bitte an <a
href="mailto:rco@mpie.de" target="_blank"
moz-do-not-send="true" class="moz-txt-link-freetext">rco@mpie.de</a><br>
-------------------------------------------------<br>
</div>
</div>
</blockquote>
</div>
</blockquote>
<BR />
<BR />
<HR />
-------------------------------------------------<BR />
Stay up to date and follow us on LinkedIn, Twitter and YouTube.<BR />
<BR />
Max-Planck-Institut für Eisenforschung GmbH<BR />
Max-Planck-Straße 1<BR />
D-40237 Düsseldorf<BR />
<BR />
Handelsregister B 2533 <BR />
Amtsgericht Düsseldorf<BR />
<BR />
Geschäftsführung<BR />
Prof. Dr. Gerhard Dehm<BR />
Prof. Dr. Jörg Neugebauer<BR />
Prof. Dr. Dierk Raabe<BR />
Dr. Kai de Weldige<BR />
<BR />
Ust.-Id.-Nr.: DE 11 93 58 514 <BR />
Steuernummer: 105 5891 1000<BR />
<BR />
<BR />
Please consider that invitations and e-mails of our institute are <BR />
only valid if they end with …@mpie.de. <BR />
If you are not sure of the validity please contact rco@mpie.de<BR />
<BR />
Bitte beachten Sie, dass Einladungen zu Veranstaltungen und E-Mails<BR />
aus unserem Haus nur mit der Endung …@mpie.de gültig sind. <BR />
In Zweifelsfällen wenden Sie sich bitte an rco@mpie.de<BR />
-------------------------------------------------<BR />
</body>
</html>