<!-- BaNnErBlUrFlE-BoDy-start -->
<!-- Preheader Text : BEGIN -->
<div style="display:none !important;display:none;visibility:hidden;mso-hide:all;font-size:1px;color:#ffffff;line-height:1px;height:0px;max-height:0px;opacity:0;overflow:hidden;">
Attaching the heaptrack files and some screenshots from one of the cores. Its most likely coming from MatStashSortCompress_Private, so if I understand correctly, it does some temporary allocations for mpi communication, but that memory gets
</div>
<!-- Preheader Text : END -->
<!-- Email Banner : BEGIN -->
<div style="display:none !important;display:none;visibility:hidden;mso-hide:all;font-size:1px;color:#ffffff;line-height:1px;height:0px;max-height:0px;opacity:0;overflow:hidden;">ZjQcmQRYFpfptBannerStart</div>
<!--[if ((ie)|(mso))]>
<table border="0" cellspacing="0" cellpadding="0" width="100%" style="padding: 16px 0px 16px 0px; direction: ltr" ><tr><td>
<table border="0" cellspacing="0" cellpadding="0" style="padding: 0px 10px 5px 6px; width: 100%; border-radius:4px; border-top:4px solid #90a4ae;background-color:#D0D8DC;"><tr><td valign="top">
<table align="left" border="0" cellspacing="0" cellpadding="0" style="padding: 4px 8px 4px 8px">
<tr><td style="color:#000000; font-family: 'Arial', sans-serif; font-weight:bold; font-size:14px; direction: ltr">
This Message Is From an External Sender
</td></tr>
<tr><td style="color:#000000; font-weight:normal; font-family: 'Arial', sans-serif; font-size:12px; direction: ltr">
This message came from outside your organization.
</td></tr>
</table>
</td></tr></table>
</td></tr></table>
<![endif]-->
<![if !((ie)|(mso))]>
<div dir="ltr" id="pfptBannerax0y7w8" style="all: revert !important; display:block !important; text-align: left !important; margin:16px 0px 16px 0px !important; padding:8px 16px 8px 16px !important; border-radius: 4px !important; min-width: 200px !important; background-color: #D0D8DC !important; background-color: #D0D8DC; border-top: 4px solid #90a4ae !important; border-top: 4px solid #90a4ae;">
<div id="pfptBannerax0y7w8" style="all: unset !important; float:left !important; display:block !important; margin: 0px 0px 1px 0px !important; max-width: 600px !important;">
<div id="pfptBannerax0y7w8" style="all: unset !important; display:block !important; visibility: visible !important; background-color: #D0D8DC !important; color:#000000 !important; color:#000000; font-family: 'Arial', sans-serif !important; font-family: 'Arial', sans-serif; font-weight:bold !important; font-weight:bold; font-size:14px !important; line-height:18px !important; line-height:18px">
This Message Is From an External Sender
</div>
<div id="pfptBannerax0y7w8" style="all: unset !important; display:block !important; visibility: visible !important; background-color: #D0D8DC !important; color:#000000 !important; color:#000000; font-weight:normal; font-family: 'Arial', sans-serif !important; font-family: 'Arial', sans-serif; font-size:12px !important; line-height:18px !important; line-height:18px; margin-top:2px !important;">
This message came from outside your organization.
</div>
</div>
<div style="clear: both !important; display: block !important; visibility: hidden !important; line-height: 0 !important; font-size: 0.01px !important; height: 0px"> </div>
</div>
<![endif]>
<div style="display:none !important;display:none;visibility:hidden;mso-hide:all;font-size:1px;color:#ffffff;line-height:1px;height:0px;max-height:0px;opacity:0;overflow:hidden;">ZjQcmQRYFpfptBannerEnd</div>
<!-- Email Banner : END -->
<!-- BaNnErBlUrFlE-BoDy-end -->
<html>
<head><!-- BaNnErBlUrFlE-HeAdEr-start -->
<style>
#pfptBannerax0y7w8 { all: revert !important; display: block !important;
visibility: visible !important; opacity: 1 !important;
background-color: #D0D8DC !important;
max-width: none !important; max-height: none !important }
.pfptPrimaryButtonax0y7w8:hover, .pfptPrimaryButtonax0y7w8:focus {
background-color: #b4c1c7 !important; }
.pfptPrimaryButtonax0y7w8:active {
background-color: #90a4ae !important; }
</style>
<!-- BaNnErBlUrFlE-HeAdEr-end -->
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Attaching the heaptrack files and some screenshots from one of the cores. Its most likely coming from
<span style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, BlinkMacSystemFont, Roboto, "Helvetica Neue", sans-serif; font-size: 14.6667px; color: rgb(36, 36, 36); background-color: rgb(255, 255, 255);">
MatStashSortCompress_Private, so if I understand correctly, it does some temporary allocations for mpi communication, but that memory gets released before it could be registered here
<a href="https://urldefense.us/v3/__https://gitlab.com/petsc/petsc/-/blob/main/src/sys/objects/inherit.c*L111__;Iw!!G_uCfscf7eWS!fEVnmR9ofeaju0KXzcErICtGBrtw9qPbE9Dw1Y7vq1h4BzpT0-V2ZDldrxNysEDfCAOCKcYITz5WyYyQkUyJkkhCFYc$" id="LPlnk" class="OWAAutoLink" data-loopstyle="linkonly">
https://gitlab.com/petsc/petsc/-/blob/main/src/sys/objects/inherit.c#L111</a></span></div>
<div class="_Entity _EType_OWALinkPreview _EId_OWALinkPreview _EReadonly_1">
<div id="LPBorder_GTaHR0cHM6Ly9naXRsYWIuY29tL3BldHNjL3BldHNjLy0vYmxvYi9tYWluL3NyYy9zeXMvb2JqZWN0cy9pbmhlcml0LmMjTDExMQ.." class="LPBorder265759" style="width:100%;margin-top:16px;margin-bottom:16px;max-width:800px;min-width:424px">
<table id="LPContainer265759" style="padding:12px 36px 12px 12px;width:100%;border-width:1px;border-style:solid;border-color:rgb(200, 200, 200);border-radius:2px">
<tbody>
<tr valign="top" style="border-spacing:0px">
<td>
<div id="LPImageContainer265759" style="margin-right:12px;height:160px;overflow:hidden">
<a target="_blank" id="LPImageAnchor265759" href="https://urldefense.us/v3/__https://gitlab.com/petsc/petsc/-/blob/main/src/sys/objects/inherit.c*L111__;Iw!!G_uCfscf7eWS!fEVnmR9ofeaju0KXzcErICtGBrtw9qPbE9Dw1Y7vq1h4BzpT0-V2ZDldrxNysEDfCAOCKcYITz5WyYyQkUyJkkhCFYc$" data-loopstyle="linkonly"><img id="LPThumbnailImageId265759" alt="" height="160" style="display:block" width="160" src="https://gitlab.com/uploads/-/system/project/avatar/13882401/PETSc_RBG-logo.png"></a></div>
</td>
<td style="width:100%">
<div id="LPTitle265759" style="font-size:21px;font-weight:300;margin-right:8px;font-family:wf_segoe-ui_light, "Segoe UI Light", "Segoe WP Light", "Segoe UI", "Segoe WP", Tahoma, Arial, sans-serif;margin-bottom:12px">
<a target="_blank" id="LPUrlAnchor265759" href="https://urldefense.us/v3/__https://gitlab.com/petsc/petsc/-/blob/main/src/sys/objects/inherit.c*L111__;Iw!!G_uCfscf7eWS!fEVnmR9ofeaju0KXzcErICtGBrtw9qPbE9Dw1Y7vq1h4BzpT0-V2ZDldrxNysEDfCAOCKcYITz5WyYyQkUyJkkhCFYc$" style="text-decoration:none" data-loopstyle="linkonly">src/sys/objects/inherit.c · main · PETSc / petsc · GitLab</a></div>
<div id="LPDescription265759" style="font-size:14px;max-height:100px;color:rgb(102, 102, 102);font-family:wf_segoe-ui_normal, "Segoe UI", "Segoe WP", Tahoma, Arial, sans-serif;margin-bottom:12px;margin-right:8px;overflow:hidden">
PETSc, pronounced PET-see (the S is silent), is a suite of data structures and routines for the scalable (parallel) solution of scientific applications modeled by partial differential equations.</div>
<div id="LPMetadata265759" style="font-size:14px;color:rgb(166, 166, 166);font-family:wf_segoe-ui_normal, "Segoe UI", "Segoe WP", Tahoma, Arial, sans-serif">
gitlab.com</div>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Ashish</div>
<div id="appendonsend"></div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> Jed Brown <jed@jedbrown.org><br>
<b>Sent:</b> Wednesday, April 24, 2024 10:25 AM<br>
<b>To:</b> Ashish Patel <ashish.patel@ansys.com>; Mark Adams <mfadams@lbl.gov>; PETSc users list <petsc-users@mcs.anl.gov><br>
<b>Cc:</b> Scott McClennan <scott.mcclennan@ansys.com>; Jeremy Theler (External) <jeremy.theler-ext@ansys.com><br>
<b>Subject:</b> Re: [petsc-users] About recent changes in GAMG</font>
<div> </div>
</div>
<div class="BodyFragment"><font size="2"><span style="font-size:11pt;">
<div class="PlainText">[External Sender]<br>
<br>
Ashish Patel <ashish.patel@ansys.com> writes:<br>
<br>
> Hi Jed,<br>
> VmRss is on a higher side and seems to match what PetscMallocGetMaximumUsage is reporting. HugetlbPages was 0 for me.<br>
><br>
> Mark, running without the near nullspace also gives similar results. I have attached the malloc_view and gamg info for serial and 2 core runs. Some of the standout functions on rank 0 for parallel run seems to be<br>
> 5.3 GB MatSeqAIJSetPreallocation_SeqAIJ<br>
> 7.7 GB MatStashSortCompress_Private<br>
> 10.1 GB PetscMatStashSpaceGet<br>
> 7.7 GB PetscSegBufferAlloc_Private<br>
><br>
> malloc_view also says the following<br>
> [0] Maximum memory PetscMalloc()ed 32387548912 maximum size of entire process 8270635008<br>
> which fits the PetscMallocGetMaximumUsage > PetscMemoryGetMaximumUsage output.<br>
<br>
This would occur if there was a large PetscMalloc'd block that did not get used (so only a portion of it is faulted and thus becomes resident).<br>
<br>
Can you run a heap profiler like heaptrack?<br>
<br>
<a href="https://urldefense.us/v3/__https://github.com/KDE/heaptrack__;!!G_uCfscf7eWS!fEVnmR9ofeaju0KXzcErICtGBrtw9qPbE9Dw1Y7vq1h4BzpT0-V2ZDldrxNysEDfCAOCKcYITz5WyYyQkUyJ9RdSDEA$">https://github.com/KDE/heaptrack</a><br>
</div>
</span></font></div>
</body>
</html>