<html><head></head><body><div style="color:#000; background-color:#fff; font-family:Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;font-size:13px"><div id="yui_3_16_0_ym19_1_1493033032302_28886" dir="ltr">Hmm, you are right. The case I did here is much more in line with our use case matrix sizes. Hmm. I need to do further experiments.<br></div><div id="yui_3_16_0_ym19_1_1493033032302_28871"><br></div><div dir="ltr" id="yui_3_16_0_ym19_1_1493033032302_28778">me@blah:~/src$ ./ex8 -n_row 200 -n_col 40000 -sparsity 0.03 -mat_type mpiaij<br id="yui_3_16_0_ym19_1_1493033032302_28766">Initialize<br id="yui_3_16_0_ym19_1_1493033032302_28767">Got options<br id="yui_3_16_0_ym19_1_1493033032302_28768">Create and assemble matrix<br id="yui_3_16_0_ym19_1_1493033032302_28769">Assembled<br id="yui_3_16_0_ym19_1_1493033032302_28770">Peak RSS 19 Mb<br id="yui_3_16_0_ym19_1_1493033032302_28771">me@blah:~/src$ ./ex8 -n_row 200 -n_col 40000 -sparsity 0.03 -mat_type mpidense<br id="yui_3_16_0_ym19_1_1493033032302_28772">Initialize<br id="yui_3_16_0_ym19_1_1493033032302_28773">Got options<br id="yui_3_16_0_ym19_1_1493033032302_28774">Create and assemble matrix<br id="yui_3_16_0_ym19_1_1493033032302_28775">Assembled<br id="yui_3_16_0_ym19_1_1493033032302_28776">Peak RSS 80 Mb<br id="yui_3_16_0_ym19_1_1493033032302_28777"><br></div><div id="yui_3_16_0_ym19_1_1493033032302_28741"><span>Thank you, Andrew McRae and Zhang Hong.</span></div> <div class="qtdSeparateBR"><br><br></div><div class="yahoo_quoted" style="display: block;"> <div style="font-family: Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif; font-size: 13px;"> <div style="font-family: HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif; font-size: 16px;"> <div dir="ltr"><font size="2" face="Arial"> On Monday, April 24, 2017 12:45 PM, Andrew McRae <A.T.T.McRae@bath.ac.uk> wrote:<br></font></div> <br><br> <div class="y_msg_container"><div id="yiv2665991566"><div dir="ltr"><div>This matrix is 2000 x 200? That's tiny. Even a 2000-by-200 dense matrix takes only (2000*200 entries)*(8 bytes per entry) to store, or about 3 MB. The sparse version might take 150KB. The 'peak RSS' differs by 2MB, so this seems consistent.<br><br></div>Try a 20,000 x 20,000 dense and sparse matrix, by which time the memory usage will be dominated by the matrix storage.<br></div><div class="yiv2665991566gmail_extra"><br><div class="yiv2665991566gmail_quote">On 24 April 2017 at 17:24, D D <span dir="ltr"><<a rel="nofollow" ymailto="mailto:driver.dan12@yahoo.com" target="_blank" href="mailto:driver.dan12@yahoo.com">driver.dan12@yahoo.com</a>></span> wrote:<br><blockquote class="yiv2665991566gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div>
<div style="color:#000;background-color:#fff;font-family:Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;font-size:13px;">
<div id="yiv2665991566m_-542945584979095556yui_3_16_0_ym19_1_1493033032302_23896">Unless, of course, my assumption is incorrect. But why should my assumption be incorrect?<br>
</div>
<div id="yiv2665991566m_-542945584979095556yui_3_16_0_ym19_1_1493033032302_23902"><br>
</div>
<div dir="ltr" id="yiv2665991566m_-542945584979095556yui_3_16_0_ym19_1_1493033032302_24344">I think I'm constructing my sparse matrix properly by calling MatSetFromOptions. The loop from line 52 - 57 in example1.cpp may be incorrect.<br>
</div>
<div id="yiv2665991566m_-542945584979095556yui_3_16_0_ym19_1_1493033032302_24345"><span><br>
</span></div>
<div id="yiv2665991566m_-542945584979095556yui_3_16_0_ym19_1_1493033032302_24585"><span id="yiv2665991566m_-542945584979095556yui_3_16_0_ym19_1_1493033032302_24584">How do you think I should measure the effect of the size of the sparse vs dense matrix structure to make sure I'm effectively using the PETSc sparse matrix structure
in my example code?</span></div><div><div class="yiv2665991566h5">
<div class="yiv2665991566m_-542945584979095556qtdSeparateBR"><br>
<br>
</div>
<div class="yiv2665991566m_-542945584979095556yahoo_quoted" style="display:block;">
<div style="font-family:Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;font-size:13px;">
<div style="font-family:HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;font-size:16px;">
<div dir="ltr"><font size="2" face="Arial">On Monday, April 24, 2017 12:10 PM, D D <<a rel="nofollow" ymailto="mailto:driver.dan12@yahoo.com" target="_blank" href="mailto:driver.dan12@yahoo.com">driver.dan12@yahoo.com</a>> wrote:<br>
</font></div>
<br>
<br>
<div class="yiv2665991566m_-542945584979095556y_msg_container">
<div id="yiv2665991566m_-542945584979095556yiv0567569497">
<div>
<div style="color:#000;background-color:#fff;font-family:Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;font-size:13px;">
You are correct, and that is why I'm using the peak RSS. The total memory should be lower to reflect the sparse versus dense structure.<br clear="none">
<div class="yiv2665991566m_-542945584979095556yiv0567569497qtdSeparateBR"><br clear="none">
<br clear="none">
</div>
<div class="yiv2665991566m_-542945584979095556yiv0567569497yqt7881792458" id="yiv2665991566m_-542945584979095556yiv0567569497yqt49326">
<div class="yiv2665991566m_-542945584979095556yiv0567569497yahoo_quoted" style="display:block;">
<div style="font-family:Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;font-size:13px;">
<div style="font-family:HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;font-size:16px;">
<div dir="ltr"><font size="2" face="Arial">On Monday, April 24, 2017 11:28 AM, "Zhang, Hong" <<a rel="nofollow" ymailto="mailto:hongzhang@anl.gov" target="_blank" href="mailto:hongzhang@anl.gov">hongzhang@anl.gov</a>> wrote:<br clear="none">
</font></div>
<br clear="none">
<br clear="none">
<div class="yiv2665991566m_-542945584979095556yiv0567569497y_msg_container">
<div id="yiv2665991566m_-542945584979095556yiv0567569497">
<div>The peak RSS does not tell you how much memory the matrix takes. It may include many things such as the binary, the libraries linked to it, and stack and heap memory.
<div class="yiv2665991566m_-542945584979095556yiv0567569497"><br class="yiv2665991566m_-542945584979095556yiv0567569497" clear="none">
</div>
<div class="yiv2665991566m_-542945584979095556yiv0567569497">Hong (Mr.)<br class="yiv2665991566m_-542945584979095556yiv0567569497" clear="none">
<div class="yiv2665991566m_-542945584979095556yiv0567569497yqt8050180386" id="yiv2665991566m_-542945584979095556yiv0567569497yqt78088">
<div class="yiv2665991566m_-542945584979095556yiv0567569497">
<div class="yiv2665991566m_-542945584979095556yiv0567569497"><br class="yiv2665991566m_-542945584979095556yiv0567569497" clear="none">
<div>
<blockquote class="yiv2665991566m_-542945584979095556yiv0567569497" type="cite">
<div class="yiv2665991566m_-542945584979095556yiv0567569497">On Apr 24, 2017, at 9:46 AM, D D <<a rel="nofollow" shape="rect" class="yiv2665991566m_-542945584979095556yiv0567569497" ymailto="mailto:driver.dan12@yahoo.com" target="_blank" href="mailto:driver.dan12@yahoo.com">driver.dan12@yahoo.com</a>> wrote:</div>
<br class="yiv2665991566m_-542945584979095556yiv0567569497Apple-interchange-newline" clear="none">
<div class="yiv2665991566m_-542945584979095556yiv0567569497">
<div class="yiv2665991566m_-542945584979095556yiv0567569497">
<div class="yiv2665991566m_-542945584979095556yiv0567569497" style="background-color:rgb(255,255,255);font-size:13px;">
<div class="yiv2665991566m_-542945584979095556yiv0567569497" id="yiv2665991566m_-542945584979095556yiv0567569497yui_3_16_0_ym19_1_1493033032302_14257">
Hello,</div>
<div class="yiv2665991566m_-542945584979095556yiv0567569497" id="yiv2665991566m_-542945584979095556yiv0567569497yui_3_16_0_ym19_1_1493033032302_13790">
<br class="yiv2665991566m_-542945584979095556yiv0567569497" clear="none">
</div>
<div class="yiv2665991566m_-542945584979095556yiv0567569497" id="yiv2665991566m_-542945584979095556yiv0567569497yui_3_16_0_ym19_1_1493033032302_13789">
I see memory usage that confuses me:</div>
<div class="yiv2665991566m_-542945584979095556yiv0567569497" id="yiv2665991566m_-542945584979095556yiv0567569497yui_3_16_0_ym19_1_1493033032302_13788">
<br class="yiv2665991566m_-542945584979095556yiv0567569497" clear="none">
</div>
<div class="yiv2665991566m_-542945584979095556yiv0567569497" dir="ltr" id="yiv2665991566m_-542945584979095556yiv0567569497yui_3_16_0_ym19_1_1493033032302_15260">
me@blah:src$ ./example1 -n_row 200 -n_col 2000 -sparsity 0.03 -mat_type mpidenseInitialize<br class="yiv2665991566m_-542945584979095556yiv0567569497" id="yiv2665991566m_-542945584979095556yiv0567569497yui_3_16_0_ym19_1_1493033032302_14304" clear="none">
Got options<br class="yiv2665991566m_-542945584979095556yiv0567569497" id="yiv2665991566m_-542945584979095556yiv0567569497yui_3_16_0_ym19_1_1493033032302_14305" clear="none">
Create and assemble matrix<br class="yiv2665991566m_-542945584979095556yiv0567569497" id="yiv2665991566m_-542945584979095556yiv0567569497yui_3_16_0_ym19_1_1493033032302_14306" clear="none">
Assembled<br class="yiv2665991566m_-542945584979095556yiv0567569497" id="yiv2665991566m_-542945584979095556yiv0567569497yui_3_16_0_ym19_1_1493033032302_14307" clear="none">
Peak RSS 21 Mb<br class="yiv2665991566m_-542945584979095556yiv0567569497" id="yiv2665991566m_-542945584979095556yiv0567569497yui_3_16_0_ym19_1_1493033032302_14308" clear="none">
me@blah:~/src$ ./example1 -n_row 200 -n_col 2000 -sparsity 0.03 -mat_type mpiaij<br class="yiv2665991566m_-542945584979095556yiv0567569497" id="yiv2665991566m_-542945584979095556yiv0567569497yui_3_16_0_ym19_1_1493033032302_14309" clear="none">
Initialize<br class="yiv2665991566m_-542945584979095556yiv0567569497" id="yiv2665991566m_-542945584979095556yiv0567569497yui_3_16_0_ym19_1_1493033032302_14310" clear="none">
Got options<br class="yiv2665991566m_-542945584979095556yiv0567569497" id="yiv2665991566m_-542945584979095556yiv0567569497yui_3_16_0_ym19_1_1493033032302_14311" clear="none">
Create and assemble matrix<br class="yiv2665991566m_-542945584979095556yiv0567569497" id="yiv2665991566m_-542945584979095556yiv0567569497yui_3_16_0_ym19_1_1493033032302_14312" clear="none">
Assembled<br class="yiv2665991566m_-542945584979095556yiv0567569497" id="yiv2665991566m_-542945584979095556yiv0567569497yui_3_16_0_ym19_1_1493033032302_14313" clear="none">
Peak RSS 19 Mb<br class="yiv2665991566m_-542945584979095556yiv0567569497" id="yiv2665991566m_-542945584979095556yiv0567569497yui_3_16_0_ym19_1_1493033032302_14314" clear="none">
<br class="yiv2665991566m_-542945584979095556yiv0567569497" clear="none">
</div>
<div class="yiv2665991566m_-542945584979095556yiv0567569497" dir="ltr" id="yiv2665991566m_-542945584979095556yiv0567569497yui_3_16_0_ym19_1_1493033032302_13786">
I put my example code on Github so I can more effectively communicate my question. And here is my question: why does the program as written use so much memory for the sparse case - matrix type mpiaij? Note that I'm creating a random dense matrix with at most
3% non-zero entries since this is my use case.<br class="yiv2665991566m_-542945584979095556yiv0567569497" clear="none">
</div>
<div class="yiv2665991566m_-542945584979095556yiv0567569497" dir="ltr" id="yiv2665991566m_-542945584979095556yiv0567569497yui_3_16_0_ym19_1_1493033032302_15688">
<br class="yiv2665991566m_-542945584979095556yiv0567569497" clear="none">
</div>
<div class="yiv2665991566m_-542945584979095556yiv0567569497" dir="ltr" id="yiv2665991566m_-542945584979095556yiv0567569497yui_3_16_0_ym19_1_1493033032302_15697">
I have read the relevant portions of the user's manual and searched for answers. Have I missed a resource that can answer my question?</div>
<div class="yiv2665991566m_-542945584979095556yiv0567569497" dir="ltr" id="yiv2665991566m_-542945584979095556yiv0567569497yui_3_16_0_ym19_1_1493033032302_15757">
<br class="yiv2665991566m_-542945584979095556yiv0567569497" clear="none">
</div>
<div class="yiv2665991566m_-542945584979095556yiv0567569497" dir="ltr" id="yiv2665991566m_-542945584979095556yiv0567569497yui_3_16_0_ym19_1_1493033032302_15737">
<a rel="nofollow" shape="rect" class="yiv2665991566m_-542945584979095556yiv0567569497enhancr2_091ed883-8ee2-c9df-4955-6e64f88122d6" id="yiv2665991566m_-542945584979095556yiv0567569497yui_3_16_0_ym19_1_1493033032302_16258" target="_blank" href="https://github.com/dtsmith2001/hpc">dtsmith2001/hpc</a><br class="yiv2665991566m_-542945584979095556yiv0567569497" clear="none">
</div>
<div class="yiv2665991566m_-542945584979095556yiv0567569497" id="yiv2665991566m_-542945584979095556yiv0567569497yui_3_16_0_ym19_1_1493033032302_16300">
<br class="yiv2665991566m_-542945584979095556yiv0567569497" clear="none">
</div>
<div class="yiv2665991566m_-542945584979095556yiv0567569497ymail-preserve-class yiv2665991566m_-542945584979095556yiv0567569497ymail-preserve-style yiv2665991566m_-542945584979095556yiv0567569497yahoo-link-enhancr-card" dir="ltr" id="yiv2665991566m_-542945584979095556yiv0567569497enhancr2_091ed883-8ee2-c9df-4955-6e64f88122d6" style="max-width:400px;">
<a rel="nofollow" shape="rect" class="yiv2665991566m_-542945584979095556yiv0567569497yahoo-enhancr-cardlink" id="yiv2665991566m_-542945584979095556yiv0567569497yui_3_16_0_ym19_1_1493033032302_16281" target="_blank" href="https://github.com/dtsmith2001/hpc" style="text-decoration:none;">
<table class="yiv2665991566m_-542945584979095556yiv0567569497card-wrapper yiv2665991566m_-542945584979095556yiv0567569497yahoo-ignore-table" id="yiv2665991566m_-542945584979095556yiv0567569497yui_3_16_0_ym19_1_1493033032302_16280" style="max-width:400px;" cellspacing="0" cellpadding="0" border="0">
<tbody class="yiv2665991566m_-542945584979095556yiv0567569497" id="yiv2665991566m_-542945584979095556yiv0567569497yui_3_16_0_ym19_1_1493033032302_16279">
<tr class="yiv2665991566m_-542945584979095556yiv0567569497" id="yiv2665991566m_-542945584979095556yiv0567569497yui_3_16_0_ym19_1_1493033032302_16278">
<td colspan="1" rowspan="1" class="yiv2665991566m_-542945584979095556yiv0567569497" id="yiv2665991566m_-542945584979095556yiv0567569497yui_3_16_0_ym19_1_1493033032302_16277" width="400">
<table class="yiv2665991566m_-542945584979095556yiv0567569497card yiv2665991566m_-542945584979095556yiv0567569497yahoo-ignore-table" id="yiv2665991566m_-542945584979095556yiv0567569497yui_3_16_0_ym19_1_1493033032302_16276" style="max-width:400px;" width="100%" cellspacing="0" cellpadding="0" border="0">
<tbody class="yiv2665991566m_-542945584979095556yiv0567569497" id="yiv2665991566m_-542945584979095556yiv0567569497yui_3_16_0_ym19_1_1493033032302_16275">
<tr class="yiv2665991566m_-542945584979095556yiv0567569497" id="yiv2665991566m_-542945584979095556yiv0567569497yui_3_16_0_ym19_1_1493033032302_16274">
<td colspan="1" rowspan="1" class="yiv2665991566m_-542945584979095556yiv0567569497card-primary-image-cell" id="yiv2665991566m_-542945584979095556yiv0567569497yui_3_16_0_ym19_1_1493033032302_16273" style="background:#000;background-size:cover;height:200px;" valign="top" bgcolor="#000000">
<table class="yiv2665991566m_-542945584979095556yiv0567569497yahoo-ignore-table" id="yiv2665991566m_-542945584979095556yiv0567569497yui_3_16_0_ym19_1_1493033032302_16272" style="width:100%;" cellspacing="0" cellpadding="0" border="0">
<tbody class="yiv2665991566m_-542945584979095556yiv0567569497" id="yiv2665991566m_-542945584979095556yiv0567569497yui_3_16_0_ym19_1_1493033032302_16271">
<tr class="yiv2665991566m_-542945584979095556yiv0567569497" id="yiv2665991566m_-542945584979095556yiv0567569497yui_3_16_0_ym19_1_1493033032302_16270">
<td colspan="1" rowspan="1" class="yiv2665991566m_-542945584979095556yiv0567569497" id="yiv2665991566m_-542945584979095556yiv0567569497yui_3_16_0_ym19_1_1493033032302_16269" style="background:transparent;height:200px;" valign="top" bgcolor="transparent">
<table class="yiv2665991566m_-542945584979095556yiv0567569497yahoo-ignore-table" id="yiv2665991566m_-542945584979095556yiv0567569497yui_3_16_0_ym19_1_1493033032302_16268" style="width:100%;height:185px;min-height:185px;" height="185">
<tbody class="yiv2665991566m_-542945584979095556yiv0567569497" id="yiv2665991566m_-542945584979095556yiv0567569497yui_3_16_0_ym19_1_1493033032302_16267">
<tr class="yiv2665991566m_-542945584979095556yiv0567569497" id="yiv2665991566m_-542945584979095556yiv0567569497yui_3_16_0_ym19_1_1493033032302_16266">
<td colspan="1" rowspan="1" class="yiv2665991566m_-542945584979095556yiv0567569497card-richInfo2" id="yiv2665991566m_-542945584979095556yiv0567569497yui_3_16_0_ym19_1_1493033032302_16768" style="text-align:left;text-align:left;padding:15px 0 0 15px;vertical-align:top;">
</td>
<td colspan="1" rowspan="1" class="yiv2665991566m_-542945584979095556yiv0567569497card-actions" id="yiv2665991566m_-542945584979095556yiv0567569497yui_3_16_0_ym19_1_1493033032302_16265" style="text-align:right;padding:15px 15px 0 0;vertical-align:top;">
<div class="yiv2665991566m_-542945584979095556yiv0567569497card-share-container"></div>
</td>
</tr>
</tbody>
</table>
</td>
</tr>
</tbody>
</table>
</td>
</tr>
<tr class="yiv2665991566m_-542945584979095556yiv0567569497">
<td colspan="1" rowspan="1" class="yiv2665991566m_-542945584979095556yiv0567569497">
<table class="yiv2665991566m_-542945584979095556yiv0567569497card-info yiv2665991566m_-542945584979095556yiv0567569497yahoo-ignore-table" style="background:#fff;width:95%;max-width:380px;border:1px solid #e0e4e9;border-bottom:3px solid #010101;margin-left:auto;margin-right:auto;" cellspacing="0" cellpadding="0" align="center" border="0">
<tbody class="yiv2665991566m_-542945584979095556yiv0567569497">
<tr class="yiv2665991566m_-542945584979095556yiv0567569497">
<td colspan="1" rowspan="1" class="yiv2665991566m_-542945584979095556yiv0567569497" style="background-color:#ffffff;padding:16px 0 16px 12px;vertical-align:top;">
<img class="yiv2665991566m_-542945584979095556yiv0567569497card-object-1 yiv2665991566m_-542945584979095556yiv0567569497yahoo-ignore-inline-image yiv2665991566m_-542945584979095556yiv0567569497ymail-preserve-class" style="min-width:32px;border:1px solid #e0e4e9;margin-top:3px;" height="32">
</td>
<td colspan="1" rowspan="1" class="yiv2665991566m_-542945584979095556yiv0567569497" style="vertical-align:middle;padding:16px 12px;width:99%;">
<h2 class="yiv2665991566m_-542945584979095556yiv0567569497card-title" style="font-size:16px;line-height:19px;margin:0 0 4px 0;">
dtsmith2001/hpc</h2>
<div class="yiv2665991566m_-542945584979095556yiv0567569497card-description" style="font-size:11px;line-height:15px;color:#999;">
hpc - High Performance Computing Explorations using PETSc and SLEPc</div>
</td>
<td colspan="1" rowspan="1" class="yiv2665991566m_-542945584979095556yiv0567569497" style="text-align:right;padding:16px 12px 16px 0;">
</td>
</tr>
</tbody>
</table>
</td>
</tr>
</tbody>
</table>
</td>
</tr>
</tbody>
</table>
</a></div>
<div class="yiv2665991566m_-542945584979095556yiv0567569497"><br class="yiv2665991566m_-542945584979095556yiv0567569497" clear="none">
</div>
<div class="yiv2665991566m_-542945584979095556yiv0567569497" id="yiv2665991566m_-542945584979095556yiv0567569497yui_3_16_0_ym19_1_1493033032302_13785">
<br class="yiv2665991566m_-542945584979095556yiv0567569497" clear="none">
</div>
<div class="yiv2665991566m_-542945584979095556yiv0567569497" id="yiv2665991566m_-542945584979095556yiv0567569497yui_3_16_0_ym19_1_1493033032302_16243">
Dale</div>
<div class="yiv2665991566m_-542945584979095556yiv0567569497" id="yiv2665991566m_-542945584979095556yiv0567569497yui_3_16_0_ym19_1_1493033032302_15791">
<br class="yiv2665991566m_-542945584979095556yiv0567569497" clear="none">
</div>
</div>
</div>
</div>
</blockquote>
</div>
<br class="yiv2665991566m_-542945584979095556yiv0567569497" clear="none">
</div>
</div>
</div>
</div>
</div>
</div>
<br clear="none">
<br clear="none">
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<br>
<br>
</div>
</div>
</div>
</div>
</div></div></div>
</div>
</blockquote></div><br></div></div><br><br></div> </div> </div> </div></div></body></html>