<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Menlo;
panose-1:2 11 6 9 3 8 4 2 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:10.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
p.m-3786694205175443202p1, li.m-3786694205175443202p1, div.m-3786694205175443202p1
{mso-style-name:m_-3786694205175443202p1;
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
p.m-3786694205175443202p2, li.m-3786694205175443202p2, div.m-3786694205175443202p2
{mso-style-name:m_-3786694205175443202p2;
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
p.m-3786694205175443202p3, li.m-3786694205175443202p3, div.m-3786694205175443202p3
{mso-style-name:m_-3786694205175443202p3;
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
p.m-3786694205175443202p4, li.m-3786694205175443202p4, div.m-3786694205175443202p4
{mso-style-name:m_-3786694205175443202p4;
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.m-3786694205175443202apple-converted-space
{mso-style-name:m_-3786694205175443202apple-converted-space;}
span.m-3786694205175443202s2
{mso-style-name:m_-3786694205175443202s2;}
span.m-3786694205175443202s1
{mso-style-name:m_-3786694205175443202s1;}
span.m-3786694205175443202s3
{mso-style-name:m_-3786694205175443202s3;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-GB" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">Thank you Mark and Matt for your response.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US"><a id="OWAAM548CF0EF1BA28E469FC02460310FFA80" href="mailto:mfadams@lbl.gov"><span style="font-family:"Calibri",sans-serif;text-decoration:none">@Mark Adams</span></a> that sounds
right because when I used cg and preconditioned with jacobi it worked in parallel.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">Best,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">Karthik.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal" style="margin-bottom:12.0pt"><b><span style="font-size:12.0pt;color:black">From:
</span></b><span style="font-size:12.0pt;color:black">Mark Adams <mfadams@lbl.gov><br>
<b>Date: </b>Monday, 28 November 2022 at 13:18<br>
<b>To: </b>Chockalingam, Karthikeyan (STFC,DL,HC) <karthikeyan.chockalingam@stfc.ac.uk><br>
<b>Cc: </b>Matthew Knepley <knepley@gmail.com>, Zhang, Hong <hzhang@mcs.anl.gov>, petsc-users@mcs.anl.gov <petsc-users@mcs.anl.gov><br>
<b>Subject: </b>Re: [petsc-users] Different solution while running in parallel<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt">Maybe I am missing something but I don't see MUMPS in the ksp_view. Just LU.<o:p></o:p></span></p>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt">The built-in LU is not parallel and so with > 1 MPI process you get one iteration of block Jacobi, so these results look fine to me, at least up to this point.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt">Mark<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt">On Thu, Nov 17, 2022 at 9:37 AM Karthikeyan Chockalingam - STFC UKRI via petsc-users <</span><a href="mailto:petsc-users@mcs.anl.gov"><span style="font-size:11.0pt">petsc-users@mcs.anl.gov</span></a><span style="font-size:11.0pt">>
wrote:<o:p></o:p></span></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt">Hi Matt and Hong,<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt"> <o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt">Thank you for your response.
<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt">I made the following changes, to get the desired output
<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt"> <o:p></o:p></span></p>
<p class="m-3786694205175443202p1"><span class="m-3786694205175443202apple-converted-space">
</span><span class="m-3786694205175443202s2">PetscReal</span><span class="m-3786694205175443202s1"> norm;
</span>/* norm of solution error */</p>
<p class="m-3786694205175443202p2"><span class="m-3786694205175443202apple-converted-space">
</span><span class="m-3786694205175443202s2">PetscInt</span><span class="m-3786694205175443202apple-converted-space">
</span>its;</p>
<p class="m-3786694205175443202p3"><span class="m-3786694205175443202apple-converted-space">
</span>KSPConvergedReason<span class="m-3786694205175443202s1"> reason;</span></p>
<p class="m-3786694205175443202p3"><span class="m-3786694205175443202apple-converted-space">
</span>PetscViewerAndFormat<span class="m-3786694205175443202s1"> *vf;</span></p>
<p class="m-3786694205175443202p4"><span class="m-3786694205175443202apple-converted-space">
</span>PetscViewerAndFormatCreate<span class="m-3786694205175443202s1">(</span><span class="m-3786694205175443202s3">PETSC_VIEWER_STDOUT_WORLD</span><span class="m-3786694205175443202s1">,
</span>PETSC_VIEWER_DEFAULT<span class="m-3786694205175443202s1">, &vf);</span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:9.0pt;font-family:Menlo;color:#643820"> </span><span style="font-size:11.0pt"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:9.0pt;font-family:Menlo;color:#643820"> ierr = </span><span style="font-size:9.0pt;font-family:Menlo;color:#6C36A9">KSPView</span><span style="font-size:9.0pt;font-family:Menlo;color:#643820">(ksp, PETSC_VIEWER_STDOUT_SELF);CHKERRQ(ierr);</span><span style="font-size:11.0pt"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:9.0pt;font-family:Menlo;color:black"> </span><span style="font-size:11.0pt"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:9.0pt;font-family:Menlo;color:black"> </span><span style="font-size:9.0pt;font-family:Menlo;color:#6C36A9">KSPSolve</span><span style="font-size:9.0pt;font-family:Menlo;color:black">(ksp, b, x);</span><span style="font-size:11.0pt"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:9.0pt;font-family:Menlo;color:black"> </span><span style="font-size:11.0pt"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:9.0pt;font-family:Menlo;color:black"> ierr = </span><span style="font-size:9.0pt;font-family:Menlo;color:#6C36A9">KSPMonitorTrueResidual</span><span style="font-size:9.0pt;font-family:Menlo;color:black">(ksp,its,norm,vf);</span><span style="font-size:9.0pt;font-family:Menlo;color:#643820">CHKERRQ</span><span style="font-size:9.0pt;font-family:Menlo;color:black">(ierr);</span><span style="font-size:11.0pt"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:9.0pt;font-family:Menlo;color:black"> ierr = </span><span style="font-size:9.0pt;font-family:Menlo;color:#6C36A9">KSPMonitorSingularValue</span><span style="font-size:9.0pt;font-family:Menlo;color:black">(ksp, its, norm, vf);</span><span style="font-size:9.0pt;font-family:Menlo;color:#643820">CHKERRQ</span><span style="font-size:9.0pt;font-family:Menlo;color:black">(ierr);</span><span style="font-size:11.0pt"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt"> <o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt">I have attached the outputs from both the runs. As before, I am also printing A, b, and x.<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt"> <o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt">I wonder if it is a memory issue related to mpi library employed. I am currently using openmpi – should I instead use mpich?<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt"> <o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt">Kind regards,<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt">Karthik.<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt"> <o:p></o:p></span></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal" style="mso-margin-top-alt:auto;margin-bottom:12.0pt"><b><span style="font-size:12.0pt;color:black">From:
</span></b><span style="font-size:12.0pt;color:black">Matthew Knepley <</span><a href="mailto:knepley@gmail.com" target="_blank"><span style="font-size:12.0pt">knepley@gmail.com</span></a><span style="font-size:12.0pt;color:black">><br>
<b>Date: </b>Thursday, 17 November 2022 at 12:19<br>
<b>To: </b>Zhang, Hong <</span><a href="mailto:hzhang@mcs.anl.gov" target="_blank"><span style="font-size:12.0pt">hzhang@mcs.anl.gov</span></a><span style="font-size:12.0pt;color:black">><br>
<b>Cc: </b></span><a href="mailto:petsc-users@mcs.anl.gov" target="_blank"><span style="font-size:12.0pt">petsc-users@mcs.anl.gov</span></a><span style="font-size:12.0pt;color:black"> <</span><a href="mailto:petsc-users@mcs.anl.gov" target="_blank"><span style="font-size:12.0pt">petsc-users@mcs.anl.gov</span></a><span style="font-size:12.0pt;color:black">>,
Chockalingam, Karthikeyan (STFC,DL,HC) <</span><a href="mailto:karthikeyan.chockalingam@stfc.ac.uk" target="_blank"><span style="font-size:12.0pt">karthikeyan.chockalingam@stfc.ac.uk</span></a><span style="font-size:12.0pt;color:black">><br>
<b>Subject: </b>Re: [petsc-users] Different solution while running in parallel</span><span style="font-size:11.0pt"><o:p></o:p></span></p>
</div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt">On Wed, Nov 16, 2022 at 9:07 PM Zhang, Hong via petsc-users <</span><a href="mailto:petsc-users@mcs.anl.gov" target="_blank"><span style="font-size:11.0pt">petsc-users@mcs.anl.gov</span></a><span style="font-size:11.0pt">>
wrote:<o:p></o:p></span></p>
</div>
<div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:9.0pt;font-family:Menlo;color:black;background:white">Karhik,</span><span style="font-size:11.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:9.0pt;font-family:Menlo;color:black;background:white">Can you find out the condition number of your matrix?</span><span style="font-size:11.0pt"><o:p></o:p></span></p>
</div>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt"> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt">Also, run using<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt"> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt"> -ksp_view -ksp_monitor_true_residual -ksp_converged_reason<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt"> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt">and send the two outputs.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt"> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt"> Thanks,<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt"> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt"> Matt<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt"> <o:p></o:p></span></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:9.0pt;font-family:Menlo;color:black;background:white">Hong</span><span style="font-size:11.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:12.0pt;color:black"> </span><span style="font-size:11.0pt"><o:p></o:p></span></p>
</div>
<div class="MsoNormal" align="center" style="text-align:center"><span style="font-size:11.0pt">
<hr size="0" width="36%" align="center">
</span></div>
<div id="m_-3786694205175443202m_-7607706715142363214divRplyFwdMsg">
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b><span style="font-size:11.0pt;color:black">From:</span></b><span style="font-size:11.0pt;color:black"> petsc-users <</span><a href="mailto:petsc-users-bounces@mcs.anl.gov" target="_blank"><span style="font-size:11.0pt">petsc-users-bounces@mcs.anl.gov</span></a><span style="font-size:11.0pt;color:black">>
on behalf of Karthikeyan Chockalingam - STFC UKRI via petsc-users <</span><a href="mailto:petsc-users@mcs.anl.gov" target="_blank"><span style="font-size:11.0pt">petsc-users@mcs.anl.gov</span></a><span style="font-size:11.0pt;color:black">><br>
<b>Sent:</b> Wednesday, November 16, 2022 6:04 PM<br>
<b>To:</b> </span><a href="mailto:petsc-users@mcs.anl.gov" target="_blank"><span style="font-size:11.0pt">petsc-users@mcs.anl.gov</span></a><span style="font-size:11.0pt;color:black"> <</span><a href="mailto:petsc-users@mcs.anl.gov" target="_blank"><span style="font-size:11.0pt">petsc-users@mcs.anl.gov</span></a><span style="font-size:11.0pt;color:black">><br>
<b>Subject:</b> [petsc-users] Different solution while running in parallel</span><span style="font-size:11.0pt">
<o:p></o:p></span></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt"> <o:p></o:p></span></p>
</div>
</div>
<div>
<div>
<p style="margin:0cm;background:white"><span style="font-size:9.0pt;font-family:Menlo;color:black"> Hello,</span></p>
<p style="margin:0cm;background:white"><span style="font-size:9.0pt;font-family:Menlo;color:black"> </span></p>
<p style="margin:0cm;background:white"><span style="font-size:9.0pt;font-family:Menlo;color:black"> I tried to solve a (FE discretized) Poisson equation using PCLU. For some reason I am getting different solutions while running the problem on one and two
cores. I have attached the output file (out.txt) from both the runs. I am printing A, b and x from both the runs – while A and b are the same but the solution seems is different.</span></p>
<p style="margin:0cm;background:white"><span style="font-size:9.0pt;font-family:Menlo;color:black"> </span></p>
<p style="margin:0cm;background:white"><span style="font-size:9.0pt;font-family:Menlo;color:black">I am not sure what I doing wrong.</span></p>
<p style="margin:0cm;background:white"><span style="font-size:9.0pt;font-family:Menlo;color:black"> </span></p>
<p style="margin:0cm;background:white"><span style="font-size:9.0pt;font-family:Menlo;color:black">Below is my matrix, vector, and solve setup.
</span></p>
<p style="margin:0cm;background:white"><span style="font-size:9.0pt;font-family:Menlo;color:black"> </span></p>
<p style="margin:0cm;background:white"><span style="font-size:9.0pt;font-family:Menlo;color:black"> </span></p>
<p style="margin:0cm;background:white"><span style="font-size:9.0pt;font-family:Menlo;color:black"> </span><span style="font-size:9.0pt;font-family:Menlo;color:#3900A0">Mat</span><span style="font-size:9.0pt;font-family:Menlo;color:black"> A;</span></p>
<p style="margin:0cm;background:white"><span style="font-size:9.0pt;font-family:Menlo;color:black">
</span><span style="font-size:9.0pt;font-family:Menlo;color:#3900A0">Vec</span><span style="font-size:9.0pt;font-family:Menlo;color:black"> b, x;</span></p>
<p style="margin:0cm;background:white"><span style="font-size:9.0pt;font-family:Menlo;color:black"> </span></p>
<p style="margin:0cm;background:white"><span style="font-size:9.0pt;font-family:Menlo;color:black"> ierr =
</span><span style="font-size:9.0pt;font-family:Menlo;color:#6C36A9">MatCreate</span><span style="font-size:9.0pt;font-family:Menlo;color:black">(</span><span style="font-size:9.0pt;font-family:Menlo;color:#6C36A9">PETSC_COMM_WORLD</span><span style="font-size:9.0pt;font-family:Menlo;color:black">,
&A); </span><span style="font-size:9.0pt;font-family:Menlo;color:#643820">CHKERRQ</span><span style="font-size:9.0pt;font-family:Menlo;color:black">(ierr);</span></p>
<p style="margin:0cm;background:white"><span style="font-size:9.0pt;font-family:Menlo;color:black"> ierr =
</span><span style="font-size:9.0pt;font-family:Menlo;color:#6C36A9">MatSetType</span><span style="font-size:9.0pt;font-family:Menlo;color:black">(A,
</span><span style="font-size:9.0pt;font-family:Menlo;color:#643820">MATMPIAIJ</span><span style="font-size:9.0pt;font-family:Menlo;color:black">);
</span><span style="font-size:9.0pt;font-family:Menlo;color:#643820">CHKERRQ</span><span style="font-size:9.0pt;font-family:Menlo;color:black">(ierr);</span></p>
<p style="margin:0cm;background:white"><span style="font-size:9.0pt;font-family:Menlo;color:black"> ierr =
</span><span style="font-size:9.0pt;font-family:Menlo;color:#6C36A9">MatSetSizes</span><span style="font-size:9.0pt;font-family:Menlo;color:black">(A,
</span><span style="font-size:9.0pt;font-family:Menlo;color:#643820">PETSC_DECIDE</span><span style="font-size:9.0pt;font-family:Menlo;color:black">,
</span><span style="font-size:9.0pt;font-family:Menlo;color:#643820">PETSC_DECIDE</span><span style="font-size:9.0pt;font-family:Menlo;color:black">, N, N);
</span><span style="font-size:9.0pt;font-family:Menlo;color:#643820">CHKERRQ</span><span style="font-size:9.0pt;font-family:Menlo;color:black">(ierr);</span></p>
<p style="margin:0cm;background:white"><span style="font-size:9.0pt;font-family:Menlo;color:black"> ierr =
</span><span style="font-size:9.0pt;font-family:Menlo;color:#6C36A9">MatMPIAIJSetPreallocation</span><span style="font-size:9.0pt;font-family:Menlo;color:black">(A,d_nz,
</span><b><span style="font-size:9.0pt;font-family:Menlo;color:#9B2393">NULL</span></b><span style="font-size:9.0pt;font-family:Menlo;color:black">, o_nz,
</span><b><span style="font-size:9.0pt;font-family:Menlo;color:#9B2393">NULL</span></b><span style="font-size:9.0pt;font-family:Menlo;color:black">);
</span><span style="font-size:9.0pt;font-family:Menlo;color:#643820">CHKERRQ</span><span style="font-size:9.0pt;font-family:Menlo;color:black">(ierr);</span></p>
<p style="margin:0cm;background:white"><span style="font-size:9.0pt;font-family:Menlo;color:#6C36A9"> ierr = MatSetOption(A,MAT_SYMMETRIC,PETSC_TRUE);
</span><span style="font-size:9.0pt;font-family:Menlo;color:#643820">CHKERRQ</span><span style="font-size:9.0pt;font-family:Menlo;color:#6C36A9">(ierr);</span></p>
<p style="margin:0cm;background:white"><span style="font-size:9.0pt;font-family:Menlo;color:black"> ierr =
</span><span style="font-size:9.0pt;font-family:Menlo;color:#6C36A9">MatCreateVecs</span><span style="font-size:9.0pt;font-family:Menlo;color:black">(A, &b, &x);
</span><span style="font-size:9.0pt;font-family:Menlo;color:#643820">CHKERRQ</span><span style="font-size:9.0pt;font-family:Menlo;color:black">(ierr);</span></p>
<p style="margin:0cm;background:white"><span style="font-size:9.0pt;font-family:Menlo;color:black"> </span></p>
<p style="margin:0cm;background:white"><span style="font-size:9.0pt;font-family:Menlo;color:black">
</span><span style="font-size:9.0pt;font-family:Menlo;color:#3900A0">KSP</span><span style="font-size:9.0pt;font-family:Menlo;color:black"> ksp;</span></p>
<p style="margin:0cm;background:white"><span style="font-size:9.0pt;font-family:Menlo;color:black">
</span><span style="font-size:9.0pt;font-family:Menlo;color:#3900A0">PC</span><span style="font-size:9.0pt;font-family:Menlo;color:black"> pc;</span></p>
<p style="margin:0cm;background:white"><span style="font-size:9.0pt;font-family:Menlo;color:#6C36A9"> KSPCreate(PETSC_COMM_WORLD, &ksp);</span></p>
<p style="margin:0cm;background:white"><span style="font-size:9.0pt;font-family:Menlo;color:black">
</span><span style="font-size:9.0pt;font-family:Menlo;color:#6C36A9">KSPSetOperators</span><span style="font-size:9.0pt;font-family:Menlo;color:black">(ksp, A, A);</span></p>
<p style="margin:0cm;background:white"><span style="font-size:9.0pt;font-family:Menlo;color:black"> ierr =
</span><span style="font-size:9.0pt;font-family:Menlo;color:#6C36A9">KSPSetType</span><span style="font-size:9.0pt;font-family:Menlo;color:black">(ksp,</span><span style="font-size:9.0pt;font-family:Menlo;color:#643820">KSPPREONLY</span><span style="font-size:9.0pt;font-family:Menlo;color:black">);</span><span style="font-size:9.0pt;font-family:Menlo;color:#643820">CHKERRQ</span><span style="font-size:9.0pt;font-family:Menlo;color:black">(ierr);</span></p>
<p style="margin:0cm;background:white"><span style="font-size:9.0pt;font-family:Menlo;color:black"> ierr =
</span><span style="font-size:9.0pt;font-family:Menlo;color:#6C36A9">KSPGetPC</span><span style="font-size:9.0pt;font-family:Menlo;color:black">(ksp,&pc);</span><span style="font-size:9.0pt;font-family:Menlo;color:#643820">CHKERRQ</span><span style="font-size:9.0pt;font-family:Menlo;color:black">(ierr);</span></p>
<p style="margin:0cm;background:white"><span style="font-size:9.0pt;font-family:Menlo;color:black"> ierr =
</span><span style="font-size:9.0pt;font-family:Menlo;color:#6C36A9">PCSetType</span><span style="font-size:9.0pt;font-family:Menlo;color:black">(pc,</span><span style="font-size:9.0pt;font-family:Menlo;color:#643820">PCLU</span><span style="font-size:9.0pt;font-family:Menlo;color:black">);</span><span style="font-size:9.0pt;font-family:Menlo;color:#643820">CHKERRQ</span><span style="font-size:9.0pt;font-family:Menlo;color:black">(ierr);</span></p>
<p style="margin:0cm;background:white"><span style="font-size:9.0pt;font-family:Menlo;color:#6C36A9"> ierr = PCFactorSetMatSolverType(pc,</span><span style="font-size:9.0pt;font-family:Menlo;color:#643820">MATSOLVERMUMPS</span><span style="font-size:9.0pt;font-family:Menlo;color:#6C36A9">);</span><span style="font-size:9.0pt;font-family:Menlo;color:#643820">CHKERRQ</span><span style="font-size:9.0pt;font-family:Menlo;color:#6C36A9">(ierr);</span></p>
<p style="margin:0cm;background:white"><span style="font-size:9.0pt;font-family:Menlo;color:black">
</span><span style="font-size:9.0pt;font-family:Menlo;color:#6C36A9">KSPSolve</span><span style="font-size:9.0pt;font-family:Menlo;color:black">(ksp, b, x);</span></p>
<p style="margin:0cm;background:white"><span style="font-size:9.0pt;font-family:Menlo;color:black"> </span></p>
<p style="margin:0cm;background:white"><span style="font-size:9.0pt;font-family:Menlo;color:black">Thank you for your help.</span></p>
<p style="margin:0cm;background:white"><span style="font-size:9.0pt;font-family:Menlo;color:black"> </span></p>
<p style="margin:0cm;background:white"><span style="font-size:9.0pt;font-family:Menlo;color:black">Karhik.</span></p>
<p style="margin:0cm"> </p>
</div>
<p><span style="font-size:6.0pt">This email and any attachments are intended solely for the use of the named recipients. If you are not the intended recipient you must not use, disclose, copy or distribute this email or any of its attachments and should notify
the sender immediately and delete this email from your system. UK Research and Innovation (UKRI) has taken every reasonable precaution to minimise risk of this email or any attachments containing viruses or malware but the recipient should carry out its own
virus and malware checks before opening the attachments. UKRI does not accept any liability for any losses or damages which the recipient may sustain due to presence of any viruses. </span></p>
</div>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt"><br clear="all">
<o:p></o:p></span></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt"> <o:p></o:p></span></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt">--
<o:p></o:p></span></p>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt">What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments
lead.<br>
-- Norbert Wiener<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt"> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><a href="http://www.cse.buffalo.edu/~knepley/" target="_blank"><span style="font-size:11.0pt">https://www.cse.buffalo.edu/~knepley/</span></a><span style="font-size:11.0pt"><o:p></o:p></span></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</body>
</html>