<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Mon, Jul 20, 2015 at 10:59 AM, <a href="mailto:Mahir.Ulker-Kaustell@tyrens.se">Mahir.Ulker-Kaustell@tyrens.se</a> <span dir="ltr"><<a href="mailto:Mahir.Ulker-Kaustell@tyrens.se" target="_blank">Mahir.Ulker-Kaustell@tyrens.se</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div lang="SV" link="blue" vlink="purple">
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Hong:<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Previous experiences with this equation have shown that it is very difficult to solve it iteratively. Hence the use of a direct solver.<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">The large test problem I am trying to solve has slightly less than 10^6 degrees of freedom. The matrices are derived from finite elements so they
are sparse.</span></p></div></div></blockquote><div><br></div><div>Estimated how? It is very difficult to estimate flll-in.</div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div lang="SV" link="blue" vlink="purple"><div><p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">The machine I am working on has 128GB ram. I have estimated the memory needed to less than 20GB, so if the solver needs twice or even three times
as much, it should still work well. Or have I completely misunderstood something here?<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Mahir<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif""><u></u> <u></u></span></b></p>
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Hong [mailto:<a href="mailto:hzhang@mcs.anl.gov" target="_blank">hzhang@mcs.anl.gov</a>]
<br>
<b>Sent:</b> den 20 juli 2015 17:39<br>
<b>To:</b> Ülker-Kaustell, Mahir<br>
<b>Cc:</b> petsc-users<br>
<b>Subject:</b> Re: [petsc-users] SuperLU MPI-problem<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US"><u></u> <u></u></span></p>
<div>
<div>
<div>
<p class="MsoNormal">Mahir:<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Direct solvers consume large amount of memory. Suggest to try followings:<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">1. A sparse iterative solver if <span style="font-size:11.0pt;font-family:"Calibri","sans-serif""> [-omega^2M + K] is not too ill-conditioned. You may test it using the small matrix.</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">2. Incrementally increase your matrix sizes. Try different matrix orderings.</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">Do you get memory crash in the 1st symbolic factorization? </span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">In your case, matrix data structure stays same when omega changes, so you only need to do one matrix symbolic factorization and reuse it.</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">3. Use a machine that gives larger memory.</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">Hong</span><u></u><u></u></p>
</div>
<div>
<blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<div>
<p class="MsoNormal">Dear Petsc-Users,<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">I am trying to use PETSc to solve a set of linear equations arising from Naviers equation (elastodynamics) in the frequency domain.</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">The frequency dependency of the problem requires that the system</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> [-omega^2M + K]u = F</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">where M and K are constant, square, positive definite matrices (mass and stiffness respectively) is solved for each frequency omega of interest.</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">K is a complex matrix, including material damping.</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">I have written a PETSc program which solves this problem for a small (1000 degrees of freedom) test problem on one or several processors, but it keeps crashing
when I try it on my full scale (in the order of 10^6 degrees of freedom) problem.</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">The program crashes at KSPSetUp() and from what I can see in the error messages, it appears as if it consumes too much memory.
</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">I would guess that similar problems have occurred in this mail-list, so I am hoping that someone can push me in the right direction…</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US" style="color:#888888"> </span><span style="color:#888888"><u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="color:#888888">Mahir</span><span style="color:#888888"><u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="color:#888888"> </span><span style="color:#888888"><u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="color:#888888"> </span><span style="color:#888888"><u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="color:#888888"> </span><span style="color:#888888"><u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="color:#888888"> </span><span style="color:#888888"><u></u><u></u></span></p>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div>
</div>
</div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature">What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div>
</div></div>