<!-- BaNnErBlUrFlE-BoDy-start -->
<!-- Preheader Text : BEGIN -->
<div style="display:none !important;display:none;visibility:hidden;mso-hide:all;font-size:1px;color:#ffffff;line-height:1px;height:0px;max-height:0px;opacity:0;overflow:hidden;">
Call MatSetOption(mat,MAT_KEEP_NONZERO_PATTERN,PETSC_TRUE); Before calling MatZeroRows() the first time, insert the periodic condition extra column value into the matrix for each periodic row so the matrix "has room" for that extra value. With
</div>
<!-- Preheader Text : END -->
<!-- Email Banner : BEGIN -->
<div style="display:none !important;display:none;visibility:hidden;mso-hide:all;font-size:1px;color:#ffffff;line-height:1px;height:0px;max-height:0px;opacity:0;overflow:hidden;">ZjQcmQRYFpfptBannerStart</div>
<!--[if ((ie)|(mso))]>
<table border="0" cellspacing="0" cellpadding="0" width="100%" style="padding: 16px 0px 16px 0px; direction: ltr" ><tr><td>
<table border="0" cellspacing="0" cellpadding="0" style="padding: 0px 10px 5px 6px; width: 100%; border-radius:4px; border-top:4px solid #90a4ae;background-color:#D0D8DC;"><tr><td valign="top">
<table align="left" border="0" cellspacing="0" cellpadding="0" style="padding: 4px 8px 4px 8px">
<tr><td style="color:#000000; font-family: 'Arial', sans-serif; font-weight:bold; font-size:14px; direction: ltr">
This Message Is From an External Sender
</td></tr>
<tr><td style="color:#000000; font-weight:normal; font-family: 'Arial', sans-serif; font-size:12px; direction: ltr">
This message came from outside your organization.
</td></tr>
</table>
</td></tr></table>
</td></tr></table>
<![endif]-->
<![if !((ie)|(mso))]>
<div dir="ltr" id="pfptBanner1xkmz0n" style="all: revert !important; display:block !important; text-align: left !important; margin:16px 0px 16px 0px !important; padding:8px 16px 8px 16px !important; border-radius: 4px !important; min-width: 200px !important; background-color: #D0D8DC !important; background-color: #D0D8DC; border-top: 4px solid #90a4ae !important; border-top: 4px solid #90a4ae;">
<div id="pfptBanner1xkmz0n" style="all: unset !important; float:left !important; display:block !important; margin: 0px 0px 1px 0px !important; max-width: 600px !important;">
<div id="pfptBanner1xkmz0n" style="all: unset !important; display:block !important; visibility: visible !important; background-color: #D0D8DC !important; color:#000000 !important; color:#000000; font-family: 'Arial', sans-serif !important; font-family: 'Arial', sans-serif; font-weight:bold !important; font-weight:bold; font-size:14px !important; line-height:18px !important; line-height:18px">
This Message Is From an External Sender
</div>
<div id="pfptBanner1xkmz0n" style="all: unset !important; display:block !important; visibility: visible !important; background-color: #D0D8DC !important; color:#000000 !important; color:#000000; font-weight:normal; font-family: 'Arial', sans-serif !important; font-family: 'Arial', sans-serif; font-size:12px !important; line-height:18px !important; line-height:18px; margin-top:2px !important;">
This message came from outside your organization.
</div>
</div>
<div style="clear: both !important; display: block !important; visibility: hidden !important; line-height: 0 !important; font-size: 0.01px !important; height: 0px"> </div>
</div>
<![endif]>
<div style="display:none !important;display:none;visibility:hidden;mso-hide:all;font-size:1px;color:#ffffff;line-height:1px;height:0px;max-height:0px;opacity:0;overflow:hidden;">ZjQcmQRYFpfptBannerEnd</div>
<!-- Email Banner : END -->
<!-- BaNnErBlUrFlE-BoDy-end -->
<html><head><!-- BaNnErBlUrFlE-HeAdEr-start -->
<style>
#pfptBanner1xkmz0n { all: revert !important; display: block !important;
visibility: visible !important; opacity: 1 !important;
background-color: #D0D8DC !important;
max-width: none !important; max-height: none !important }
.pfptPrimaryButton1xkmz0n:hover, .pfptPrimaryButton1xkmz0n:focus {
background-color: #b4c1c7 !important; }
.pfptPrimaryButton1xkmz0n:active {
background-color: #90a4ae !important; }
</style>
<!-- BaNnErBlUrFlE-HeAdEr-end -->
<meta http-equiv="content-type" content="text/html; charset=utf-8"></head><body style="overflow-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;"><div><br></div><div>Call MatSetOption(mat,MAT_KEEP_NONZERO_PATTERN,PETSC_TRUE);</div><div><br></div>Before calling MatZeroRows() the first time, insert the periodic condition extra column value into the matrix for each periodic row so the matrix "has room" for that extra value.<div><br></div><div>With this change, there should never be any mallocs, and the process should be pretty fast.</div><div><br></div><div> Barry</div><div><br><div><br></div><div><br><div><div><blockquote type="cite"><div>On Jul 15, 2024, at 5:19 AM, Alex Sp <alex.sp2408@gmail.com> wrote:</div><br class="Apple-interchange-newline"><div><div dir="ltr"><div>I am using versions 3.19 and 3.20.</div><div><br></div><div>Indeed I get extra mallocs during MatSetValues(). The problem is somewhere in the imposition of periodicity. Since the condition for any unknown u is: u(z=-W) - u(z=+W) = 0, what I do is to store in all processes the global node numbers in these boundaries. Then I call MatZeroRows() for the rows corresponding to (z=-W) and set 1 in the diagonal. Finally, I call MatSetValues() for the same rows and the columns corresponding to (z=+W) to insert the -1 value. I suspect that this creates the problem but why does it keep adding mallocs after the first assembly?</div><div>Here is part of the -info output:</div><div><br></div><div>[0] <sys> PetscCommDuplicate(): Using internal PETSc communicator 1140850688 -2080374781<br>[0] <sys> PetscCommDuplicate(): Using internal PETSc communicator 1140850689 -2080374778<br>[0] <sys> PetscCommDuplicate(): Using internal PETSc communicator 1140850689 -2080374778<br>[0] <vec> VecAssemblyBegin_MPI_BTS(): Stash has 11980 entries, uses 3 mallocs.<br>[0] <vec> VecAssemblyBegin_MPI_BTS(): Block-Stash has 0 entries, uses 0 mallocs.<br>[0] <mat> MatAssemblyBegin_MPIAIJ(): Stash has 5529600 entries, uses 0 mallocs.<br>[0] <vec> VecAssemblyBegin_MPI_BTS(): Stash has 0 entries, uses 0 mallocs.<br>[0] <vec> VecAssemblyBegin_MPI_BTS(): Block-Stash has 0 entries, uses 0 mallocs.<br>[0] <mat> MatAssemblyBegin_MPIAIJ(): Stash has 0 entries, uses 0 mallocs.<br>[0] <mat> MatAssemblyEnd_SeqAIJ(): Matrix size: 13835 X 36985; storage space: 1990 unneeded,2343675 used<br>[0] <mat> MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues() is 20339<br>[0] <mat> MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 330<br>[0] <mat> MatCheckCompressedRow(): Found the ratio (num_zerorows 0)/(num_localrows 13835) < 0.6. Do not use CompressedRow routines.</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, 15 Jul 2024 at 01:36, Barry Smith <<a href="mailto:bsmith@petsc.dev">bsmith@petsc.dev</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><br id="m_-9190178321667646020lineBreakAtBeginningOfMessage"><div><br><blockquote type="cite"><div>On Jul 14, 2024, at 6:04 PM, Alex Sp <<a href="mailto:alex.sp2408@gmail.com" target="_blank">alex.sp2408@gmail.com</a>> wrote:</div><br><div><div dir="ltr" id="m_-9190178321667646020pfptBannerxqgwx0i" style="writing-mode:revert;color:revert;font-family:revert;font-feature-settings:revert;font-kerning:revert;font-size:revert;font-size-adjust:revert;font-stretch:revert;font-variant-alternates:revert;font-variant-caps:revert;font-variant-east-asian:revert;font-variant-ligatures:revert;font-variant-numeric:revert;font-weight:revert;text-orientation:revert;zoom:revert;letter-spacing:revert;background-blend-mode:revert;background-image:revert;background-position:revert;background-repeat:revert;background-size:revert;border-collapse:revert;box-sizing:revert;break-after:revert;break-before:revert;break-inside:revert;caption-side:revert;clear:revert;columns:revert;column-fill:revert;column-gap:revert;column-rule:revert;column-span:revert;empty-cells:revert;float:revert;image-orientation:revert;isolation:revert;line-break:revert;line-height:revert;list-style:revert;mix-blend-mode:revert;object-fit:revert;object-position:revert;outline:revert;overflow:revert;quotes:revert;table-layout:revert;text-align:left;text-align-last:revert;text-decoration:revert;text-emphasis:revert;text-indent:revert;text-justify:revert;text-overflow:revert;text-transform:revert;text-underline-position:revert;vertical-align:revert;white-space:revert;word-break:revert;border-spacing:revert;word-spacing:revert;background-clip:revert;background-origin:revert;text-combine-upright:revert;display:block;opacity:revert;background-color:rgb(208,216,220);border-bottom:revert;border-left:revert;border-right:revert;height:revert;max-height:revert;max-width:revert;min-height:revert;width:revert;margin:16px 0px;padding:8px 16px;border-radius:4px;min-width:200px;border-top:4px solid rgb(144,164,174)"><div id="m_-9190178321667646020pfptBannerxqgwx0i" style="writing-mode:unset;color:unset;font-family:unset;font-feature-settings:unset;font-kerning:unset;font-size:unset;font-size-adjust:unset;font-stretch:unset;font-variant-alternates:unset;font-variant-caps:unset;font-variant-east-asian:unset;font-variant-ligatures:unset;font-variant-numeric:unset;font-weight:unset;text-orientation:unset;zoom:unset;letter-spacing:unset;background:unset;background-blend-mode:unset;border-collapse:unset;border:unset;box-sizing:unset;break-after:unset;break-before:unset;break-inside:unset;caption-side:unset;clear:unset;columns:unset;column-fill:unset;column-gap:unset;column-rule:unset;column-span:unset;empty-cells:unset;float:left;image-orientation:unset;isolation:unset;line-break:unset;line-height:unset;list-style:unset;mix-blend-mode:unset;object-fit:unset;object-position:unset;outline:unset;overflow:unset;quotes:unset;table-layout:unset;text-align:unset;text-align-last:unset;text-decoration:unset;text-emphasis:unset;text-indent:unset;text-justify:unset;text-overflow:unset;text-transform:unset;text-underline-position:unset;vertical-align:unset;white-space:unset;word-break:unset;border-spacing:unset;word-spacing:unset;text-combine-upright:unset;display:block;opacity:unset;border-radius:unset;height:unset;max-height:unset;min-height:unset;min-width:unset;padding:unset;width:unset;margin:0px 0px 1px;max-width:600px"><div id="m_-9190178321667646020pfptBannerxqgwx0i" style="writing-mode:unset;font-family:Arial,sans-serif;font-feature-settings:unset;font-kerning:unset;font-size:14px;font-size-adjust:unset;font-stretch:unset;font-variant-alternates:unset;font-variant-caps:unset;font-variant-east-asian:unset;font-variant-ligatures:unset;font-variant-numeric:unset;font-weight:bold;text-orientation:unset;zoom:unset;letter-spacing:unset;background-blend-mode:unset;background-image:unset;background-position:unset;background-repeat:unset;background-size:unset;border-collapse:unset;border:unset;box-sizing:unset;break-after:unset;break-before:unset;break-inside:unset;caption-side:unset;clear:unset;columns:unset;column-fill:unset;column-gap:unset;column-rule:unset;column-span:unset;empty-cells:unset;float:unset;image-orientation:unset;isolation:unset;line-break:unset;line-height:18px;list-style:unset;mix-blend-mode:unset;object-fit:unset;object-position:unset;outline:unset;overflow:unset;quotes:unset;table-layout:unset;text-align:unset;text-align-last:unset;text-decoration:unset;text-emphasis:unset;text-indent:unset;text-overflow:unset;text-transform:unset;text-underline-position:unset;vertical-align:unset;white-space:unset;word-break:unset;border-spacing:unset;word-spacing:unset;background-clip:unset;background-origin:unset;text-combine-upright:unset;display:block;opacity:unset;background-color:rgb(208,216,220);border-radius:unset;height:unset;margin:unset;max-height:unset;max-width:unset;min-height:unset;min-width:unset;padding:unset;width:unset">This Message Is From an External Sender</div><div id="m_-9190178321667646020pfptBannerxqgwx0i" style="writing-mode:unset;font-family:Arial,sans-serif;font-feature-settings:unset;font-kerning:unset;font-size:12px;font-size-adjust:unset;font-stretch:unset;font-variant-alternates:unset;font-variant-caps:unset;font-variant-east-asian:unset;font-variant-ligatures:unset;font-variant-numeric:unset;font-weight:unset;text-orientation:unset;zoom:unset;letter-spacing:unset;background-blend-mode:unset;background-image:unset;background-position:unset;background-repeat:unset;background-size:unset;border-collapse:unset;border:unset;box-sizing:unset;break-after:unset;break-before:unset;break-inside:unset;caption-side:unset;clear:unset;columns:unset;column-fill:unset;column-gap:unset;column-rule:unset;column-span:unset;empty-cells:unset;float:unset;image-orientation:unset;isolation:unset;line-break:unset;line-height:18px;list-style:unset;mix-blend-mode:unset;object-fit:unset;object-position:unset;outline:unset;overflow:unset;quotes:unset;table-layout:unset;text-align:unset;text-align-last:unset;text-decoration:unset;text-emphasis:unset;text-indent:unset;text-overflow:unset;text-transform:unset;text-underline-position:unset;vertical-align:unset;white-space:unset;word-break:unset;border-spacing:unset;word-spacing:unset;background-clip:unset;background-origin:unset;text-combine-upright:unset;display:block;opacity:unset;background-color:rgb(208,216,220);border-radius:unset;height:unset;margin-bottom:unset;margin-left:unset;margin-right:unset;max-height:unset;max-width:unset;min-height:unset;min-width:unset;padding:unset;width:unset;margin-top:2px">This message came from outside your organization.</div></div><div style="height:0px;clear:both;display:block;line-height:0;font-size:0.01px"></div></div><div dir="auto" style="font-family:Helvetica;font-size:18px;font-style:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none"><div dir="auto">Hello,</div><div dir="auto"><br></div><div dir="auto">I am trying to solve the problem of the 3D fluid flow around a cylinder using an in-house finite element Fortran code. I want to apply periodic boundary conditions in the neutral direction (the z direction which coincides with the axis of the cylinder). The method works fine and it gives correct results. However the assembly of the matrix for the Newton Raphson method takes too long. The whole process takes about 10 minutes for a single Newton iteration. For the same system size it takes only 10 seconds if instead of periodic conditions I set wall boundaries (no slip, no penetration). What is also strange is that the very first matrix assembly is quite normal (takes about 30 seconds). </div></div></div></blockquote><div><br></div> Does the matrix non-zero structure change between the first and later matrix assemblies? Your behavior is generally a result of new non-zero locations being entered into the matrix after the first assembly is done. </div><div><br></div><div> What version of PETSc are you using?</div><div><br></div><div> You can run with the option -info and grep for mallocs to see if additional mallocs are needed</div><div><br></div><div><br></div><div><br><blockquote type="cite"><div><div dir="auto" style="font-family:Helvetica;font-size:18px;font-style:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none"><div dir="auto">What could be the reason for this? It seems like some matrix entries are allocated anew every time. However this happens only when I apply periodicity. I do not do any matrix preallocation.</div><div dir="auto"><br></div><div dir="auto">Thanks in advance,</div><div dir="auto">Alex</div></div></div></blockquote></div><br></div></blockquote></div><br clear="all"><br><span class="gmail_signature_prefix">-- </span><br><div dir="ltr" class="gmail_signature"><div dir="ltr">Alexandros Spyridakis<div>PhD Candidate</div><div>Department of Chemical Engineering, University of Patras, Greece</div><div>Laboratory of Fluid Mechanics and Rheology</div></div></div>
</div></blockquote></div><br></div></div></div></body></html>