<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Menlo;
panose-1:2 11 6 9 3 8 4 2 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0cm;
margin-right:0cm;
margin-bottom:0cm;
margin-left:36.0pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle19
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
p.p1, li.p1, div.p1
{mso-style-name:p1;
margin:0cm;
font-size:13.5pt;
font-family:Menlo;
color:black;}
span.s1
{mso-style-name:s1;}
span.apple-converted-space
{mso-style-name:apple-converted-space;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:302974967;
mso-list-type:hybrid;
mso-list-template-ids:1698203250 -843546188 134807577 134807579 134807567 134807577 134807579 134807567 134807577 134807579;}
@list l0:level1
{mso-level-number-format:roman-lower;
mso-level-text:"\(%1\)";
mso-level-tab-stop:none;
mso-level-number-position:left;
margin-left:54.0pt;
text-indent:-36.0pt;}
@list l0:level2
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level3
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l0:level4
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level5
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level6
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l0:level7
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level8
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level9
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
ol
{margin-bottom:0cm;}
ul
{margin-bottom:0cm;}
--></style>
</head>
<body lang="EN-GB" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="p1"><span class="apple-converted-space"><span style="font-size:10.0pt">Thank you. I was able to confirm both the below options produced the same mesh<o:p></o:p></span></span></p>
<p class="p1"><span class="apple-converted-space"><span style="font-size:10.0pt"><o:p> </o:p></span></span></p>
<p class="p1"><span class="apple-converted-space"><span style="font-size:10.0pt"> </span></span><span class="s1"><span style="font-size:10.0pt">./ex56 -cells 2,2,1 -max_conv_its 2<o:p></o:p></span></span></p>
<p class="p1"><span class="s1"><span style="font-size:10.0pt">./ex56 -cells 4,4,2 -max_conv_its 1<o:p></o:p></span></span></p>
<p class="p1"><span class="s1"><span style="font-size:10.0pt"><o:p> </o:p></span></span></p>
<p class="p1"><span class="s1"><span style="font-size:10.0pt">But I didn’t get how is
</span></span><span style="font-size:10.0pt">-cells i,j,k <1,1,1><span class="s1"> is related to the number of MPI processes.</span><o:p></o:p></span></p>
<p class="MsoListParagraph" style="margin-left:54.0pt;text-indent:-36.0pt;mso-list:l0 level1 lfo1">
<![if !supportLists]><span class="s1"><span style="font-size:10.0pt;font-family:Menlo;mso-fareast-language:EN-US"><span style="mso-list:Ignore">(i)<span style="font:7.0pt "Times New Roman"">
</span></span></span></span><![endif]><span style="font-size:10.0pt;font-family:Menlo;mso-fareast-language:EN-US">Say I start with -cells 1,1,1 -max_conv its</span><span class="s1"><span style="font-size:10.0pt;font-family:Menlo"> 7; that would eventually leave
all refinement on level 7 running on 1 MPI process?</span></span><span class="s1"><span style="font-size:10.0pt;font-family:Menlo;mso-fareast-language:EN-US"><o:p></o:p></span></span></p>
<p class="MsoListParagraph" style="margin-left:54.0pt;text-indent:-36.0pt;mso-list:l0 level1 lfo1">
<![if !supportLists]><span style="font-size:10.0pt;font-family:Menlo;mso-fareast-language:EN-US"><span style="mso-list:Ignore">(ii)<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span style="font-size:10.0pt;font-family:Menlo;mso-fareast-language:EN-US">Say I start with -cells 2,2,1 -max_conv its</span><span class="s1"><span style="font-size:10.0pt;font-family:Menlo"> n</span></span><span style="font-size:10.0pt;font-family:Menlo;mso-fareast-language:EN-US">;
is it recommended to run on 4 MPI processes? <o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">I am running ex56 on gpu; I am looking at KSPSolve (or any other event) but no gpu flops are recorded in the -log_view?
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">For your reference I used the below flags:<o:p></o:p></span></p>
<p class="p1"><span class="s1"><span style="font-size:10.0pt">./ex56 -cells 1,1,1 -max_conv_its 3 -lx 1. -alpha .01 -petscspace_degree 1 -ksp_type cg -ksp_monitor -ksp_rtol 1.e-8 -pc_type asm -snes_monitor -use_mat_nearnullspace true -snes_rtol 1.e-10 -ex56_dm_view
-log_view</span></span><span style="font-size:10.0pt"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">Kind regards,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">Karthik.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span style="font-size:12.0pt;color:black">From: </span></b><span style="font-size:12.0pt;color:black">Mark Adams <mfadams@lbl.gov><br>
<b>Date: </b>Sunday, 12 December 2021 at 23:00<br>
<b>To: </b>"Chockalingam, Karthikeyan (STFC,DL,HC)" <karthikeyan.chockalingam@stfc.ac.uk><br>
<b>Cc: </b>Matthew Knepley <knepley@gmail.com>, "petsc-users@mcs.anl.gov" <petsc-users@mcs.anl.gov><br>
<b>Subject: </b>Re: [petsc-users] Unstructured mesh<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">On Sun, Dec 12, 2021 at 3:19 PM Karthikeyan Chockalingam - STFC UKRI <<a href="mailto:karthikeyan.chockalingam@stfc.ac.uk" target="_blank">karthikeyan.chockalingam@stfc.ac.uk</a>> wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Thank for your response that was helpful. I have a couple of questions:<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p style="margin-left:54.0pt">(i)<span style="font-size:7.0pt;font-family:"Times New Roman",serif">
</span>How can I control the level of refinement? I tried to pass the flag “-ex56_dm_refine 0” but that didn’t stop the refinement from 8 giving 32 cubes.<o:p></o:p></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I answered this question recently but ex56 clobbers ex56_dm_refine in the convergence loop. I have an MR that prints a warning if you provide a ex56_dm_refine. <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">* snes/ex56 runs a convergence study and confusingly sets the options manually, thus erasing your -ex56_dm_refine.<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">* To refine, use -max_conv_its N <3>, this sets the number of steps of refinement. That is, the length of the convergence study<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">* You can adjust where it starts from with -cells i,j,k <1,1,1><o:p></o:p></p>
</div>
<div>
<div>
<p class="MsoNormal">You do want to set this if you have multiple MPI processes so that the size of this mesh is the number of processes. That way it starts with one cell per process and refines from there.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<div>
<div>
<p style="margin-left:54.0pt">(ii)<span style="font-size:7.0pt;font-family:"Times New Roman",serif">
</span>What does -cell 2,2,1 correspond to? <o:p></o:p></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">The initial mesh or mesh_0. The convergence test uniformly refines this mesh. So if you want to refine this twice you could use -cells 8,8,4<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<div>
<div>
<p style="margin-left:54.0pt">How can I determine the total number of dofs? <o:p>
</o:p></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal">Unfortunately, that is not printed but you can calculate from the initial cell grid, the order of the element and the refinement in each iteration of the convergence tests.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<div>
<div>
<p style="margin-left:54.0pt">So that I can perform a scaling study by changing the input of the flag -cells.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><o:p> </o:p></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">You can and the convergence test gives you data for a strong speedup study in one run. Each solve is put in its own "stage" of the output and you want to look at KSPSolve lines in the log_view output. <o:p></o:p></p>
</div>
</div>
</div>
</div>
<p><span style="font-size: 6pt;">This email and any attachments are intended solely for the use of the named recipients. If you are not the intended recipient you must not use, disclose, copy or distribute this email or any of its attachments and should notify the sender immediately and delete this email from your system. UK Research and Innovation (UKRI) has taken every reasonable precaution to minimise risk of this email or any attachments containing viruses or malware but the recipient should carry out its own virus and malware checks before opening the attachments. UKRI does not accept any liability for any losses or damages which the recipient may sustain due to presence of any viruses. </span></p></body>
</html>