<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
{mso-style-priority:99;
mso-style-link:"Balloon Text Char";
margin:0cm;
margin-bottom:.0001pt;
font-size:8.0pt;
font-family:"Tahoma","sans-serif";}
span.EmailStyle17
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.BalloonTextChar
{mso-style-name:"Balloon Text Char";
mso-style-priority:99;
mso-style-link:"Balloon Text";
font-family:"Tahoma","sans-serif";
mso-fareast-language:SV;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";
mso-fareast-language:EN-US;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:70.85pt 70.85pt 70.85pt 70.85pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="SV" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span lang="EN-US">So, I tried to use MUMPS instead of SuperLU.
<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Still some problems with running out of memory, but I think that Sherry and Hong have poked me in an important direction:<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">I may have also underestimated the need for matrix reordering when computing the LU-factors. I have to look into this…
<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">One thing which is puzzling me now is that when I followed Hongs suggestion to try an iterative solver, I found that it solved my simple test problem after some testing with different settings.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">I am solving Ax = b with a sparse, indefinite, symmetric, complex matrix, can anything be said about the chances of success in using an iterative method?<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">/Mahir<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Matthew Knepley [mailto:knepley@gmail.com]
<br>
<b>Sent:</b> den 22 juli 2015 19:17<br>
<b>To:</b> Ülker-Kaustell, Mahir<br>
<b>Cc:</b> Barry Smith; petsc-users<br>
<b>Subject:</b> Re: [petsc-users] SuperLU MPI-problem<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<div>
<p class="MsoNormal">On Wed, Jul 22, 2015 at 11:11 AM, <a href="mailto:Mahir.Ulker-Kaustell@tyrens.se">
Mahir.Ulker-Kaustell@tyrens.se</a> <<a href="mailto:Mahir.Ulker-Kaustell@tyrens.se" target="_blank">Mahir.Ulker-Kaustell@tyrens.se</a>> wrote:<o:p></o:p></p>
<p class="MsoNormal">Thank you for your reply.<br>
<br>
As you have probably figured out already, I am not a computational scientist. I am a researcher in civil engineering (railways for high-speed traffic), trying to produce some, from my perspective, fairly large parametric studies based on finite element discretizations.<br>
<br>
I am working in a Windows-environment and have installed PETSc through Cygwin.<br>
Apparently, there is no support for Valgrind in this OS.<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">It is really worth any amount of time and effort to get away from Windows if you are doing computational science.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<p class="MsoNormal">If I have understood you correct, the memory issues are related to superLU and given my background, there is not much I can do. Is this correct?<o:p></o:p></p>
</blockquote>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">The next step is to run the problem using MUMPS (--download-mumps --download-scalapack).<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"> Thanks,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"> Matt<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<p class="MsoNormal" style="margin-bottom:12.0pt">Best regards,<br>
Mahir<br>
<br>
______________________________________________<br>
Mahir Ülker-Kaustell, Kompetenssamordnare, Brokonstruktör, Tekn. Dr, Tyréns AB<br>
010 452 30 82, <a href="mailto:Mahir.Ulker-Kaustell@tyrens.se">Mahir.Ulker-Kaustell@tyrens.se</a><br>
______________________________________________<br>
<br>
-----Original Message-----<br>
From: Barry Smith [mailto:<a href="mailto:bsmith@mcs.anl.gov">bsmith@mcs.anl.gov</a>]<br>
Sent: den 22 juli 2015 02:57<br>
To: Ülker-Kaustell, Mahir<br>
Cc: Xiaoye S. Li; petsc-users<br>
Subject: Re: [petsc-users] SuperLU MPI-problem<br>
<br>
<br>
Run the program under valgrind <a href="http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind" target="_blank">
http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind</a> . When I use the option -mat_superlu_dist_parsymbfact I get many scary memory problems some involving for example ddist_psymbtonum (pdsymbfact_distdata.c:1332)<br>
<br>
Note that I consider it unacceptable for running programs to EVER use uninitialized values; until these are all cleaned up I won't trust any runs like this.<br>
<br>
Barry<br>
<br>
<br>
<br>
<br>
==42050== Conditional jump or move depends on uninitialised value(s)<br>
==42050== at 0x10274C436: MPI_Allgatherv (allgatherv.c:1053)<br>
==42050== by 0x101557F60: get_perm_c_parmetis (get_perm_c_parmetis.c:285)<br>
==42050== by 0x101501192: pdgssvx (pdgssvx.c:934)<br>
==42050== by 0x1009CFE7A: MatLUFactorNumeric_SuperLU_DIST (superlu_dist.c:414)<br>
==42050== by 0x10046CC5C: MatLUFactorNumeric (matrix.c:2946)<br>
==42050== by 0x100F09F2C: PCSetUp_LU (lu.c:152)<br>
==42050== by 0x100FF9036: PCSetUp (precon.c:982)<br>
==42050== by 0x1010F54EB: KSPSetUp (itfunc.c:332)<br>
==42050== by 0x1010F7985: KSPSolve (itfunc.c:546)<br>
==42050== by 0x10125541E: SNESSolve_NEWTONLS (ls.c:233)<br>
==42050== by 0x1011C49B7: SNESSolve (snes.c:3906)<br>
==42050== by 0x100001B3C: main (in ./ex19)<br>
==42050== Uninitialised value was created by a stack allocation<br>
==42050== at 0x10155751B: get_perm_c_parmetis (get_perm_c_parmetis.c:96)<br>
==42050==<br>
==42050== Conditional jump or move depends on uninitialised value(s)<br>
==42050== at 0x102851C61: MPIR_Allgatherv_intra (allgatherv.c:651)<br>
==42050== by 0x102853EC7: MPIR_Allgatherv (allgatherv.c:903)<br>
==42050== by 0x102853F84: MPIR_Allgatherv_impl (allgatherv.c:944)<br>
==42050== by 0x10274CA41: MPI_Allgatherv (allgatherv.c:1107)<br>
==42050== by 0x101557F60: get_perm_c_parmetis (get_perm_c_parmetis.c:285)<br>
==42050== by 0x101501192: pdgssvx (pdgssvx.c:934)<br>
==42050== by 0x1009CFE7A: MatLUFactorNumeric_SuperLU_DIST (superlu_dist.c:414)<br>
==42050== by 0x10046CC5C: MatLUFactorNumeric (matrix.c:2946)<br>
==42050== by 0x100F09F2C: PCSetUp_LU (lu.c:152)<br>
==42050== by 0x100FF9036: PCSetUp (precon.c:982)<br>
==42050== by 0x1010F54EB: KSPSetUp (itfunc.c:332)<br>
==42050== by 0x1010F7985: KSPSolve (itfunc.c:546)<br>
==42050== by 0x10125541E: SNESSolve_NEWTONLS (ls.c:233)<br>
==42050== by 0x1011C49B7: SNESSolve (snes.c:3906)<br>
==42050== by 0x100001B3C: main (in ./ex19)<br>
==42050== Uninitialised value was created by a stack allocation<br>
==42050== at 0x10155751B: get_perm_c_parmetis (get_perm_c_parmetis.c:96)<br>
==42050==<br>
==42049== Syscall param writev(vector[...]) points to uninitialised byte(s)<br>
==42049== at 0x102DA1C3A: writev (in /usr/lib/system/libsystem_kernel.dylib)<br>
==42049== by 0x10296A0DC: MPL_large_writev (mplsock.c:32)<br>
==42049== by 0x10295F6AD: MPIDU_Sock_writev (sock_immed.i:610)<br>
==42049== by 0x102943FCA: MPIDI_CH3_iSendv (ch3_isendv.c:84)<br>
==42049== by 0x102934361: MPIDI_CH3_EagerContigIsend (ch3u_eager.c:556)<br>
==42049== by 0x102939531: MPID_Isend (mpid_isend.c:138)<br>
==42049== by 0x10277656E: MPI_Isend (isend.c:125)<br>
==42049== by 0x102088B66: libparmetis__gkMPI_Isend (gkmpi.c:63)<br>
==42049== by 0x10208140F: libparmetis__CommInterfaceData (comm.c:298)<br>
==42049== by 0x1020A8758: libparmetis__CompactGraph (ometis.c:553)<br>
==42049== by 0x1020A77BB: libparmetis__MultilevelOrder (ometis.c:225)<br>
==42049== by 0x1020A7493: ParMETIS_V32_NodeND (ometis.c:151)<br>
==42049== by 0x1020A6AFB: ParMETIS_V3_NodeND (ometis.c:34)<br>
==42049== by 0x101557CFC: get_perm_c_parmetis (get_perm_c_parmetis.c:241)<br>
==42049== by 0x101501192: pdgssvx (pdgssvx.c:934)<br>
==42049== by 0x1009CFE7A: MatLUFactorNumeric_SuperLU_DIST (superlu_dist.c:414)<br>
==42049== by 0x10046CC5C: MatLUFactorNumeric (matrix.c:2946)<br>
==42049== by 0x100F09F2C: PCSetUp_LU (lu.c:152)<br>
==42049== by 0x100FF9036: PCSetUp (precon.c:982)<br>
==42048== Syscall param writev(vector[...]) points to uninitialised byte(s)<br>
==42049== by 0x1010F54EB: KSPSetUp (itfunc.c:332)<br>
==42049== Address 0x105edff70 is 1,424 bytes inside a block of size 752,720 alloc'd<br>
==42049== at 0x1000183B1: malloc (vg_replace_malloc.c:303)<br>
==42049== by 0x1020EB90C: gk_malloc (memory.c:147)<br>
==42049== by 0x1020EAA28: gk_mcoreCreate (mcore.c:28)<br>
==42048== at 0x102DA1C3A: writev (in /usr/lib/system/libsystem_kernel.dylib)<br>
==42048== by 0x10296A0DC: MPL_large_writev (mplsock.c:32)<br>
==42049== by 0x1020BA5CF: libparmetis__AllocateWSpace (wspace.c:23)<br>
==42049== by 0x1020A6E84: ParMETIS_V32_NodeND (ometis.c:98)<br>
==42048== by 0x10295F6AD: MPIDU_Sock_writev (sock_immed.i:610)<br>
==42048== by 0x102943FCA: MPIDI_CH3_iSendv (ch3_isendv.c:84)<br>
==42048== by 0x102934361: MPIDI_CH3_EagerContigIsend (ch3u_eager.c:556)<br>
==42049== by 0x1020A6AFB: ParMETIS_V3_NodeND (ometis.c:34)<br>
==42049== by 0x101557CFC: get_perm_c_parmetis (get_perm_c_parmetis.c:241)<br>
==42049== by 0x101501192: pdgssvx (pdgssvx.c:934)<br>
==42048== by 0x102939531: MPID_Isend (mpid_isend.c:138)<br>
==42048== by 0x10277656E: MPI_Isend (isend.c:125)<br>
==42049== by 0x1009CFE7A: MatLUFactorNumeric_SuperLU_DIST (superlu_dist.c:414)<br>
==42049== by 0x10046CC5C: MatLUFactorNumeric (matrix.c:2946)<br>
==42049== by 0x100F09F2C: PCSetUp_LU (lu.c:152)<br>
==42049== by 0x100FF9036: PCSetUp (precon.c:982)<br>
==42049== by 0x1010F54EB: KSPSetUp (itfunc.c:332)<br>
==42048== by 0x102088B66: libparmetis__gkMPI_Isend (gkmpi.c:63)<br>
==42048== by 0x10208140F: libparmetis__CommInterfaceData (comm.c:298)<br>
==42049== by 0x1010F7985: KSPSolve (itfunc.c:546)<br>
==42049== by 0x10125541E: SNESSolve_NEWTONLS (ls.c:233)<br>
==42048== by 0x1020A8758: libparmetis__CompactGraph (ometis.c:553)<br>
==42048== by 0x1020A77BB: libparmetis__MultilevelOrder (ometis.c:225)<br>
==42048== by 0x1020A7493: ParMETIS_V32_NodeND (ometis.c:151)<br>
==42049== by 0x1011C49B7: SNESSolve (snes.c:3906)<br>
==42049== by 0x100001B3C: main (in ./ex19)<br>
==42049== Uninitialised value was created by a heap allocation<br>
==42049== at 0x1000183B1: malloc (vg_replace_malloc.c:303)<br>
==42049== by 0x1020EB90C: gk_malloc (memory.c:147)<br>
==42048== by 0x1020A6AFB: ParMETIS_V3_NodeND (ometis.c:34)<br>
==42048== by 0x101557CFC: get_perm_c_parmetis (get_perm_c_parmetis.c:241)<br>
==42048== by 0x101501192: pdgssvx (pdgssvx.c:934)<br>
==42048== by 0x1009CFE7A: MatLUFactorNumeric_SuperLU_DIST (superlu_dist.c:414)<br>
==42049== by 0x10211C50B: libmetis__imalloc (gklib.c:24)<br>
==42049== by 0x1020A8566: libparmetis__CompactGraph (ometis.c:519)<br>
==42049== by 0x1020A77BB: libparmetis__MultilevelOrder (ometis.c:225)<br>
==42048== by 0x10046CC5C: MatLUFactorNumeric (matrix.c:2946)<br>
==42049== by 0x1020A7493: ParMETIS_V32_NodeND (ometis.c:151)<br>
==42049== by 0x1020A6AFB: ParMETIS_V3_NodeND (ometis.c:34)<br>
==42049== by 0x101557CFC: get_perm_c_parmetis (get_perm_c_parmetis.c:241)<br>
==42049== by 0x101501192: pdgssvx (pdgssvx.c:934)<br>
==42049== by 0x1009CFE7A: MatLUFactorNumeric_SuperLU_DIST (superlu_dist.c:414)<br>
==42049== by 0x10046CC5C: MatLUFactorNumeric (matrix.c:2946)<br>
==42048== by 0x100F09F2C: PCSetUp_LU (lu.c:152)<br>
==42049== by 0x100F09F2C: PCSetUp_LU (lu.c:152)<br>
==42049== by 0x100FF9036: PCSetUp (precon.c:982)<br>
==42049== by 0x1010F54EB: KSPSetUp (itfunc.c:332)<br>
==42049== by 0x1010F7985: KSPSolve (itfunc.c:546)<br>
==42049== by 0x10125541E: SNESSolve_NEWTONLS (ls.c:233)<br>
==42048== by 0x100FF9036: PCSetUp (precon.c:982)<br>
==42048== by 0x1010F54EB: KSPSetUp (itfunc.c:332)<br>
==42048== Address 0x10597a860 is 1,408 bytes inside a block of size 752,720 alloc'd<br>
==42049== by 0x1011C49B7: SNESSolve (snes.c:3906)<br>
==42049== by 0x100001B3C: main (in ./ex19)<br>
==42049==<br>
==42048== at 0x1000183B1: malloc (vg_replace_malloc.c:303)<br>
==42048== by 0x1020EB90C: gk_malloc (memory.c:147)<br>
==42048== by 0x1020EAA28: gk_mcoreCreate (mcore.c:28)<br>
==42048== by 0x1020BA5CF: libparmetis__AllocateWSpace (wspace.c:23)<br>
==42048== by 0x1020A6E84: ParMETIS_V32_NodeND (ometis.c:98)<br>
==42048== by 0x1020A6AFB: ParMETIS_V3_NodeND (ometis.c:34)<br>
==42048== by 0x101557CFC: get_perm_c_parmetis (get_perm_c_parmetis.c:241)<br>
==42048== by 0x101501192: pdgssvx (pdgssvx.c:934)<br>
==42048== by 0x1009CFE7A: MatLUFactorNumeric_SuperLU_DIST (superlu_dist.c:414)<br>
==42048== by 0x10046CC5C: MatLUFactorNumeric (matrix.c:2946)<br>
==42048== by 0x100F09F2C: PCSetUp_LU (lu.c:152)<br>
==42048== by 0x100FF9036: PCSetUp (precon.c:982)<br>
==42048== by 0x1010F54EB: KSPSetUp (itfunc.c:332)<br>
==42048== by 0x1010F7985: KSPSolve (itfunc.c:546)<br>
==42048== by 0x10125541E: SNESSolve_NEWTONLS (ls.c:233)<br>
==42048== by 0x1011C49B7: SNESSolve (snes.c:3906)<br>
==42048== by 0x100001B3C: main (in ./ex19)<br>
==42048== Uninitialised value was created by a heap allocation<br>
==42048== at 0x1000183B1: malloc (vg_replace_malloc.c:303)<br>
==42048== by 0x1020EB90C: gk_malloc (memory.c:147)<br>
==42048== by 0x10211C50B: libmetis__imalloc (gklib.c:24)<br>
==42048== by 0x1020A8566: libparmetis__CompactGraph (ometis.c:519)<br>
==42048== by 0x1020A77BB: libparmetis__MultilevelOrder (ometis.c:225)<br>
==42048== by 0x1020A7493: ParMETIS_V32_NodeND (ometis.c:151)<br>
==42048== by 0x1020A6AFB: ParMETIS_V3_NodeND (ometis.c:34)<br>
==42048== by 0x101557CFC: get_perm_c_parmetis (get_perm_c_parmetis.c:241)<br>
==42048== by 0x101501192: pdgssvx (pdgssvx.c:934)<br>
==42048== by 0x1009CFE7A: MatLUFactorNumeric_SuperLU_DIST (superlu_dist.c:414)<br>
==42048== by 0x10046CC5C: MatLUFactorNumeric (matrix.c:2946)<br>
==42048== by 0x100F09F2C: PCSetUp_LU (lu.c:152)<br>
==42048== by 0x100FF9036: PCSetUp (precon.c:982)<br>
==42048== by 0x1010F54EB: KSPSetUp (itfunc.c:332)<br>
==42048== by 0x1010F7985: KSPSolve (itfunc.c:546)<br>
==42048== by 0x10125541E: SNESSolve_NEWTONLS (ls.c:233)<br>
==42048== by 0x1011C49B7: SNESSolve (snes.c:3906)<br>
==42048== by 0x100001B3C: main (in ./ex19)<br>
==42048==<br>
==42048== Syscall param write(buf) points to uninitialised byte(s)<br>
==42048== at 0x102DA1C22: write (in /usr/lib/system/libsystem_kernel.dylib)<br>
==42048== by 0x10295F5BD: MPIDU_Sock_write (sock_immed.i:525)<br>
==42048== by 0x102944839: MPIDI_CH3_iStartMsg (ch3_istartmsg.c:86)<br>
==42048== by 0x102933B80: MPIDI_CH3_EagerContigShortSend (ch3u_eager.c:257)<br>
==42048== by 0x10293ADBA: MPID_Send (mpid_send.c:130)<br>
==42048== by 0x10277A1FA: MPI_Send (send.c:127)<br>
==42048== by 0x10155802F: get_perm_c_parmetis (get_perm_c_parmetis.c:299)<br>
==42048== by 0x101501192: pdgssvx (pdgssvx.c:934)<br>
==42048== by 0x1009CFE7A: MatLUFactorNumeric_SuperLU_DIST (superlu_dist.c:414)<br>
==42048== by 0x10046CC5C: MatLUFactorNumeric (matrix.c:2946)<br>
==42048== by 0x100F09F2C: PCSetUp_LU (lu.c:152)<br>
==42048== by 0x100FF9036: PCSetUp (precon.c:982)<br>
==42048== by 0x1010F54EB: KSPSetUp (itfunc.c:332)<br>
==42048== by 0x1010F7985: KSPSolve (itfunc.c:546)<br>
==42048== by 0x10125541E: SNESSolve_NEWTONLS (ls.c:233)<br>
==42048== by 0x1011C49B7: SNESSolve (snes.c:3906)<br>
==42048== by 0x100001B3C: main (in ./ex19)<br>
==42048== Address 0x104810704 is on thread 1's stack<br>
==42048== in frame #3, created by MPIDI_CH3_EagerContigShortSend (ch3u_eager.c:218)<br>
==42048== Uninitialised value was created by a heap allocation<br>
==42048== at 0x1000183B1: malloc (vg_replace_malloc.c:303)<br>
==42048== by 0x10153B704: superlu_malloc_dist (memory.c:108)<br>
==42048== by 0x101557AB9: get_perm_c_parmetis (get_perm_c_parmetis.c:185)<br>
==42048== by 0x101501192: pdgssvx (pdgssvx.c:934)<br>
==42048== by 0x1009CFE7A: MatLUFactorNumeric_SuperLU_DIST (superlu_dist.c:414)<br>
==42048== by 0x10046CC5C: MatLUFactorNumeric (matrix.c:2946)<br>
==42048== by 0x100F09F2C: PCSetUp_LU (lu.c:152)<br>
==42048== by 0x100FF9036: PCSetUp (precon.c:982)<br>
==42048== by 0x1010F54EB: KSPSetUp (itfunc.c:332)<br>
==42048== by 0x1010F7985: KSPSolve (itfunc.c:546)<br>
==42048== by 0x10125541E: SNESSolve_NEWTONLS (ls.c:233)<br>
==42048== by 0x1011C49B7: SNESSolve (snes.c:3906)<br>
==42048== by 0x100001B3C: main (in ./ex19)<br>
==42048==<br>
==42050== Conditional jump or move depends on uninitialised value(s)<br>
==42050== at 0x102744CB8: MPI_Alltoallv (alltoallv.c:480)<br>
==42050== by 0x101510B3E: dist_symbLU (pdsymbfact_distdata.c:539)<br>
==42050== by 0x10150A5C6: ddist_psymbtonum (pdsymbfact_distdata.c:1275)<br>
==42050== by 0x1015018C2: pdgssvx (pdgssvx.c:1057)<br>
==42050== by 0x1009CFE7A: MatLUFactorNumeric_SuperLU_DIST (superlu_dist.c:414)<br>
==42050== by 0x10046CC5C: MatLUFactorNumeric (matrix.c:2946)<br>
==42050== by 0x100F09F2C: PCSetUp_LU (lu.c:152)<br>
==42050== by 0x100FF9036: PCSetUp (precon.c:982)<br>
==42050== by 0x1010F54EB: KSPSetUp (itfunc.c:332)<br>
==42050== by 0x1010F7985: KSPSolve (itfunc.c:546)<br>
==42050== by 0x10125541E: SNESSolve_NEWTONLS (ls.c:233)<br>
==42050== by 0x1011C49B7: SNESSolve (snes.c:3906)<br>
==42050== by 0x100001B3C: main (in ./ex19)<br>
==42050== Uninitialised value was created by a stack allocation<br>
==42050== at 0x10150E4C4: dist_symbLU (pdsymbfact_distdata.c:96)<br>
==42050==<br>
==42050== Conditional jump or move depends on uninitialised value(s)<br>
==42050== at 0x102744E43: MPI_Alltoallv (alltoallv.c:490)<br>
==42050== by 0x101510B3E: dist_symbLU (pdsymbfact_distdata.c:539)<br>
==42050== by 0x10150A5C6: ddist_psymbtonum (pdsymbfact_distdata.c:1275)<br>
==42050== by 0x1015018C2: pdgssvx (pdgssvx.c:1057)<br>
==42050== by 0x1009CFE7A: MatLUFactorNumeric_SuperLU_DIST (superlu_dist.c:414)<br>
==42050== by 0x10046CC5C: MatLUFactorNumeric (matrix.c:2946)<br>
==42050== by 0x100F09F2C: PCSetUp_LU (lu.c:152)<br>
==42050== by 0x100FF9036: PCSetUp (precon.c:982)<br>
==42050== by 0x1010F54EB: KSPSetUp (itfunc.c:332)<br>
==42050== by 0x1010F7985: KSPSolve (itfunc.c:546)<br>
==42050== by 0x10125541E: SNESSolve_NEWTONLS (ls.c:233)<br>
==42050== by 0x1011C49B7: SNESSolve (snes.c:3906)<br>
==42050== by 0x100001B3C: main (in ./ex19)<br>
==42050== Uninitialised value was created by a stack allocation<br>
==42050== at 0x10150E4C4: dist_symbLU (pdsymbfact_distdata.c:96)<br>
==42050==<br>
==42050== Conditional jump or move depends on uninitialised value(s)<br>
==42050== at 0x102744EBF: MPI_Alltoallv (alltoallv.c:497)<br>
==42050== by 0x101510B3E: dist_symbLU (pdsymbfact_distdata.c:539)<br>
==42050== by 0x10150A5C6: ddist_psymbtonum (pdsymbfact_distdata.c:1275)<br>
==42050== by 0x1015018C2: pdgssvx (pdgssvx.c:1057)<br>
==42050== by 0x1009CFE7A: MatLUFactorNumeric_SuperLU_DIST (superlu_dist.c:414)<br>
==42050== by 0x10046CC5C: MatLUFactorNumeric (matrix.c:2946)<br>
==42050== by 0x100F09F2C: PCSetUp_LU (lu.c:152)<br>
==42050== by 0x100FF9036: PCSetUp (precon.c:982)<br>
==42050== by 0x1010F54EB: KSPSetUp (itfunc.c:332)<br>
==42050== by 0x1010F7985: KSPSolve (itfunc.c:546)<br>
==42050== by 0x10125541E: SNESSolve_NEWTONLS (ls.c:233)<br>
==42050== by 0x1011C49B7: SNESSolve (snes.c:3906)<br>
==42050== by 0x100001B3C: main (in ./ex19)<br>
==42050== Uninitialised value was created by a stack allocation<br>
==42050== at 0x10150E4C4: dist_symbLU (pdsymbfact_distdata.c:96)<br>
==42050==<br>
==42050== Conditional jump or move depends on uninitialised value(s)<br>
==42050== at 0x1027450B1: MPI_Alltoallv (alltoallv.c:512)<br>
==42050== by 0x101510B3E: dist_symbLU (pdsymbfact_distdata.c:539)<br>
==42050== by 0x10150A5C6: ddist_psymbtonum (pdsymbfact_distdata.c:1275)<br>
==42050== by 0x1015018C2: pdgssvx (pdgssvx.c:1057)<br>
==42050== by 0x1009CFE7A: MatLUFactorNumeric_SuperLU_DIST (superlu_dist.c:414)<br>
==42050== by 0x10046CC5C: MatLUFactorNumeric (matrix.c:2946)<br>
==42050== by 0x100F09F2C: PCSetUp_LU (lu.c:152)<br>
==42050== by 0x100FF9036: PCSetUp (precon.c:982)<br>
==42050== by 0x1010F54EB: KSPSetUp (itfunc.c:332)<br>
==42050== by 0x1010F7985: KSPSolve (itfunc.c:546)<br>
==42050== by 0x10125541E: SNESSolve_NEWTONLS (ls.c:233)<br>
==42050== by 0x1011C49B7: SNESSolve (snes.c:3906)<br>
==42050== by 0x100001B3C: main (in ./ex19)<br>
==42050== Uninitialised value was created by a stack allocation<br>
==42050== at 0x10150E4C4: dist_symbLU (pdsymbfact_distdata.c:96)<br>
==42050==<br>
==42050== Conditional jump or move depends on uninitialised value(s)<br>
==42050== at 0x10283FB06: MPIR_Alltoallv_intra (alltoallv.c:92)<br>
==42050== by 0x1028407B6: MPIR_Alltoallv (alltoallv.c:343)<br>
==42050== by 0x102840884: MPIR_Alltoallv_impl (alltoallv.c:380)<br>
==42050== by 0x10274541B: MPI_Alltoallv (alltoallv.c:531)<br>
==42050== by 0x101510B3E: dist_symbLU (pdsymbfact_distdata.c:539)<br>
==42050== by 0x10150A5C6: ddist_psymbtonum (pdsymbfact_distdata.c:1275)<br>
==42050== by 0x1015018C2: pdgssvx (pdgssvx.c:1057)<br>
==42050== by 0x1009CFE7A: MatLUFactorNumeric_SuperLU_DIST (superlu_dist.c:414)<br>
==42050== by 0x10046CC5C: MatLUFactorNumeric (matrix.c:2946)<br>
==42050== by 0x100F09F2C: PCSetUp_LU (lu.c:152)<br>
==42050== by 0x100FF9036: PCSetUp (precon.c:982)<br>
==42050== by 0x1010F54EB: KSPSetUp (itfunc.c:332)<br>
==42050== by 0x1010F7985: KSPSolve (itfunc.c:546)<br>
==42050== by 0x10125541E: SNESSolve_NEWTONLS (ls.c:233)<br>
==42050== by 0x1011C49B7: SNESSolve (snes.c:3906)<br>
==42050== by 0x100001B3C: main (in ./ex19)<br>
==42050== Uninitialised value was created by a stack allocation<br>
==42050== at 0x10150E4C4: dist_symbLU (pdsymbfact_distdata.c:96)<br>
==42050==<br>
==42050== Syscall param writev(vector[...]) points to uninitialised byte(s)<br>
==42050== at 0x102DA1C3A: writev (in /usr/lib/system/libsystem_kernel.dylib)<br>
==42050== by 0x10296A0DC: MPL_large_writev (mplsock.c:32)<br>
==42050== by 0x10295F6AD: MPIDU_Sock_writev (sock_immed.i:610)<br>
==42050== by 0x102943FCA: MPIDI_CH3_iSendv (ch3_isendv.c:84)<br>
==42050== by 0x102934361: MPIDI_CH3_EagerContigIsend (ch3u_eager.c:556)<br>
==42050== by 0x102939531: MPID_Isend (mpid_isend.c:138)<br>
==42050== by 0x10277656E: MPI_Isend (isend.c:125)<br>
==42050== by 0x101524C41: pdgstrf2_trsm (pdgstrf2.c:201)<br>
==42050== by 0x10151ECBF: pdgstrf (pdgstrf.c:1082)<br>
==42050== by 0x1015019A5: pdgssvx (pdgssvx.c:1069)<br>
==42050== by 0x1009CFE7A: MatLUFactorNumeric_SuperLU_DIST (superlu_dist.c:414)<br>
==42050== by 0x10046CC5C: MatLUFactorNumeric (matrix.c:2946)<br>
==42050== by 0x100F09F2C: PCSetUp_LU (lu.c:152)<br>
==42050== by 0x100FF9036: PCSetUp (precon.c:982)<br>
==42050== by 0x1010F54EB: KSPSetUp (itfunc.c:332)<br>
==42050== by 0x1010F7985: KSPSolve (itfunc.c:546)<br>
==42050== by 0x10125541E: SNESSolve_NEWTONLS (ls.c:233)<br>
==42050== by 0x1011C49B7: SNESSolve (snes.c:3906)<br>
==42050== by 0x100001B3C: main (in ./ex19)<br>
==42050== Address 0x1060144d0 is 1,168 bytes inside a block of size 131,072 alloc'd<br>
==42050== at 0x1000183B1: malloc (vg_replace_malloc.c:303)<br>
==42050== by 0x10153B704: superlu_malloc_dist (memory.c:108)<br>
==42050== by 0x1014FD7AD: doubleMalloc_dist (dmemory.c:145)<br>
==42050== by 0x10151DA7D: pdgstrf (pdgstrf.c:735)<br>
==42050== by 0x1015019A5: pdgssvx (pdgssvx.c:1069)<br>
==42050== by 0x1009CFE7A: MatLUFactorNumeric_SuperLU_DIST (superlu_dist.c:414)<br>
==42050== by 0x10046CC5C: MatLUFactorNumeric (matrix.c:2946)<br>
==42050== by 0x100F09F2C: PCSetUp_LU (lu.c:152)<br>
==42050== by 0x100FF9036: PCSetUp (precon.c:982)<br>
==42050== by 0x1010F54EB: KSPSetUp (itfunc.c:332)<br>
==42050== by 0x1010F7985: KSPSolve (itfunc.c:546)<br>
==42050== by 0x10125541E: SNESSolve_NEWTONLS (ls.c:233)<br>
==42050== by 0x1011C49B7: SNESSolve (snes.c:3906)<br>
==42050== by 0x100001B3C: main (in ./ex19)<br>
==42050== Uninitialised value was created by a heap allocation<br>
==42050== at 0x1000183B1: malloc (vg_replace_malloc.c:303)<br>
==42050== by 0x10153B704: superlu_malloc_dist (memory.c:108)<br>
==42050== by 0x1014FD7AD: doubleMalloc_dist (dmemory.c:145)<br>
==42050== by 0x10151DA7D: pdgstrf (pdgstrf.c:735)<br>
==42050== by 0x1015019A5: pdgssvx (pdgssvx.c:1069)<br>
==42050== by 0x1009CFE7A: MatLUFactorNumeric_SuperLU_DIST (superlu_dist.c:414)<br>
==42050== by 0x10046CC5C: MatLUFactorNumeric (matrix.c:2946)<br>
==42050== by 0x100F09F2C: PCSetUp_LU (lu.c:152)<br>
==42050== by 0x100FF9036: PCSetUp (precon.c:982)<br>
==42050== by 0x1010F54EB: KSPSetUp (itfunc.c:332)<br>
==42050== by 0x1010F7985: KSPSolve (itfunc.c:546)<br>
==42050== by 0x10125541E: SNESSolve_NEWTONLS (ls.c:233)<br>
==42050== by 0x1011C49B7: SNESSolve (snes.c:3906)<br>
==42050== by 0x100001B3C: main (in ./ex19)<br>
==42050==<br>
==42048== Conditional jump or move depends on uninitialised value(s)<br>
==42048== at 0x10151F141: pdgstrf (pdgstrf.c:1139)<br>
==42048== by 0x1015019A5: pdgssvx (pdgssvx.c:1069)<br>
==42048== by 0x1009CFE7A: MatLUFactorNumeric_SuperLU_DIST (superlu_dist.c:414)<br>
==42048== by 0x10046CC5C: MatLUFactorNumeric (matrix.c:2946)<br>
==42048== by 0x100F09F2C: PCSetUp_LU (lu.c:152)<br>
==42048== by 0x100FF9036: PCSetUp (precon.c:982)<br>
==42048== by 0x1010F54EB: KSPSetUp (itfunc.c:332)<br>
==42048== by 0x1010F7985: KSPSolve (itfunc.c:546)<br>
==42048== by 0x10125541E: SNESSolve_NEWTONLS (ls.c:233)<br>
==42048== by 0x1011C49B7: SNESSolve (snes.c:3906)<br>
==42048== by 0x100001B3C: main (in ./ex19)<br>
==42048== Uninitialised value was created by a heap allocation<br>
==42048== at 0x1000183B1: malloc (vg_replace_malloc.c:303)<br>
==42048== by 0x10153B704: superlu_malloc_dist (memory.c:108)<br>
==42048== by 0x10150ABE2: ddist_psymbtonum (pdsymbfact_distdata.c:1332)<br>
==42048== by 0x1015018C2: pdgssvx (pdgssvx.c:1057)<br>
==42048== by 0x1009CFE7A: MatLUFactorNumeric_SuperLU_DIST (superlu_dist.c:414)<br>
==42048== by 0x10046CC5C: MatLUFactorNumeric (matrix.c:2946)<br>
==42048== by 0x100F09F2C: PCSetUp_LU (lu.c:152)<br>
==42048== by 0x100FF9036: PCSetUp (precon.c:982)<br>
==42048== by 0x1010F54EB: KSPSetUp (itfunc.c:332)<br>
==42048== by 0x1010F7985: KSPSolve (itfunc.c:546)<br>
==42048== by 0x10125541E: SNESSolve_NEWTONLS (ls.c:233)<br>
==42048== by 0x1011C49B7: SNESSolve (snes.c:3906)<br>
==42048== by 0x100001B3C: main (in ./ex19)<br>
==42048==<br>
==42049== Conditional jump or move depends on uninitialised value(s)<br>
==42049== at 0x10151F141: pdgstrf (pdgstrf.c:1139)<br>
==42049== by 0x1015019A5: pdgssvx (pdgssvx.c:1069)<br>
==42049== by 0x1009CFE7A: MatLUFactorNumeric_SuperLU_DIST (superlu_dist.c:414)<br>
==42049== by 0x10046CC5C: MatLUFactorNumeric (matrix.c:2946)<br>
==42049== by 0x100F09F2C: PCSetUp_LU (lu.c:152)<br>
==42049== by 0x100FF9036: PCSetUp (precon.c:982)<br>
==42049== by 0x1010F54EB: KSPSetUp (itfunc.c:332)<br>
==42049== by 0x1010F7985: KSPSolve (itfunc.c:546)<br>
==42049== by 0x10125541E: SNESSolve_NEWTONLS (ls.c:233)<br>
==42049== by 0x1011C49B7: SNESSolve (snes.c:3906)<br>
==42049== by 0x100001B3C: main (in ./ex19)<br>
==42049== Uninitialised value was created by a heap allocation<br>
==42049== at 0x1000183B1: malloc (vg_replace_malloc.c:303)<br>
==42049== by 0x10153B704: superlu_malloc_dist (memory.c:108)<br>
==42049== by 0x10150ABE2: ddist_psymbtonum (pdsymbfact_distdata.c:1332)<br>
==42049== by 0x1015018C2: pdgssvx (pdgssvx.c:1057)<br>
==42049== by 0x1009CFE7A: MatLUFactorNumeric_SuperLU_DIST (superlu_dist.c:414)<br>
==42049== by 0x10046CC5C: MatLUFactorNumeric (matrix.c:2946)<br>
==42049== by 0x100F09F2C: PCSetUp_LU (lu.c:152)<br>
==42049== by 0x100FF9036: PCSetUp (precon.c:982)<br>
==42049== by 0x1010F54EB: KSPSetUp (itfunc.c:332)<br>
==42049== by 0x1010F7985: KSPSolve (itfunc.c:546)<br>
==42049== by 0x10125541E: SNESSolve_NEWTONLS (ls.c:233)<br>
==42049== by 0x1011C49B7: SNESSolve (snes.c:3906)<br>
==42049== by 0x100001B3C: main (in ./ex19)<br>
==42049==<br>
==42048== Conditional jump or move depends on uninitialised value(s)<br>
==42048== at 0x101520054: pdgstrf (pdgstrf.c:1429)<br>
==42048== by 0x1015019A5: pdgssvx (pdgssvx.c:1069)<br>
==42048== by 0x1009CFE7A: MatLUFactorNumeric_SuperLU_DIST (superlu_dist.c:414)<br>
==42048== by 0x10046CC5C: MatLUFactorNumeric (matrix.c:2946)<br>
==42048== by 0x100F09F2C: PCSetUp_LU (lu.c:152)<br>
==42048== by 0x100FF9036: PCSetUp (precon.c:982)<br>
==42048== by 0x1010F54EB: KSPSetUp (itfunc.c:332)<br>
==42048== by 0x1010F7985: KSPSolve (itfunc.c:546)<br>
==42049== Conditional jump or move depends on uninitialised value(s)<br>
==42048== by 0x10125541E: SNESSolve_NEWTONLS (ls.c:233)<br>
==42048== by 0x1011C49B7: SNESSolve (snes.c:3906)<br>
==42048== by 0x100001B3C: main (in ./ex19)<br>
==42048== Uninitialised value was created by a heap allocation<br>
==42049== at 0x101520054: pdgstrf (pdgstrf.c:1429)<br>
==42048== at 0x1000183B1: malloc (vg_replace_malloc.c:303)<br>
==42048== by 0x10153B704: superlu_malloc_dist (memory.c:108)<br>
==42049== by 0x1015019A5: pdgssvx (pdgssvx.c:1069)<br>
==42049== by 0x1009CFE7A: MatLUFactorNumeric_SuperLU_DIST (superlu_dist.c:414)<br>
==42048== by 0x10150ABE2: ddist_psymbtonum (pdsymbfact_distdata.c:1332)<br>
==42048== by 0x1015018C2: pdgssvx (pdgssvx.c:1057)<br>
==42048== by 0x1009CFE7A: MatLUFactorNumeric_SuperLU_DIST (superlu_dist.c:414)<br>
==42049== by 0x10046CC5C: MatLUFactorNumeric (matrix.c:2946)<br>
==42049== by 0x100F09F2C: PCSetUp_LU (lu.c:152)<br>
==42048== by 0x10046CC5C: MatLUFactorNumeric (matrix.c:2946)<br>
==42048== by 0x100F09F2C: PCSetUp_LU (lu.c:152)<br>
==42049== by 0x100FF9036: PCSetUp (precon.c:982)<br>
==42049== by 0x1010F54EB: KSPSetUp (itfunc.c:332)<br>
==42049== by 0x1010F7985: KSPSolve (itfunc.c:546)<br>
==42048== by 0x100FF9036: PCSetUp (precon.c:982)<br>
==42048== by 0x1010F54EB: KSPSetUp (itfunc.c:332)<br>
==42048== by 0x1010F7985: KSPSolve (itfunc.c:546)<br>
==42049== by 0x10125541E: SNESSolve_NEWTONLS (ls.c:233)<br>
==42049== by 0x1011C49B7: SNESSolve (snes.c:3906)<br>
==42048== by 0x10125541E: SNESSolve_NEWTONLS (ls.c:233)<br>
==42048== by 0x1011C49B7: SNESSolve (snes.c:3906)<br>
==42049== by 0x100001B3C: main (in ./ex19)<br>
==42049== Uninitialised value was created by a heap allocation<br>
==42049== at 0x1000183B1: malloc (vg_replace_malloc.c:303)<br>
==42048== by 0x100001B3C: main (in ./ex19)<br>
==42048==<br>
==42049== by 0x10153B704: superlu_malloc_dist (memory.c:108)<br>
==42049== by 0x10150ABE2: ddist_psymbtonum (pdsymbfact_distdata.c:1332)<br>
==42049== by 0x1015018C2: pdgssvx (pdgssvx.c:1057)<br>
==42049== by 0x1009CFE7A: MatLUFactorNumeric_SuperLU_DIST (superlu_dist.c:414)<br>
==42049== by 0x10046CC5C: MatLUFactorNumeric (matrix.c:2946)<br>
==42049== by 0x100F09F2C: PCSetUp_LU (lu.c:152)<br>
==42049== by 0x100FF9036: PCSetUp (precon.c:982)<br>
==42049== by 0x1010F54EB: KSPSetUp (itfunc.c:332)<br>
==42049== by 0x1010F7985: KSPSolve (itfunc.c:546)<br>
==42049== by 0x10125541E: SNESSolve_NEWTONLS (ls.c:233)<br>
==42049== by 0x1011C49B7: SNESSolve (snes.c:3906)<br>
==42049== by 0x100001B3C: main (in ./ex19)<br>
==42049==<br>
==42050== Conditional jump or move depends on uninitialised value(s)<br>
==42050== at 0x10151FDE6: pdgstrf (pdgstrf.c:1382)<br>
==42050== by 0x1015019A5: pdgssvx (pdgssvx.c:1069)<br>
==42050== by 0x1009CFE7A: MatLUFactorNumeric_SuperLU_DIST (superlu_dist.c:414)<br>
==42050== by 0x10046CC5C: MatLUFactorNumeric (matrix.c:2946)<br>
==42050== by 0x100F09F2C: PCSetUp_LU (lu.c:152)<br>
==42050== by 0x100FF9036: PCSetUp (precon.c:982)<br>
==42050== by 0x1010F54EB: KSPSetUp (itfunc.c:332)<br>
==42050== by 0x1010F7985: KSPSolve (itfunc.c:546)<br>
==42050== by 0x10125541E: SNESSolve_NEWTONLS (ls.c:233)<br>
==42050== by 0x1011C49B7: SNESSolve (snes.c:3906)<br>
==42050== by 0x100001B3C: main (in ./ex19)<br>
==42050== Uninitialised value was created by a heap allocation<br>
==42050== at 0x1000183B1: malloc (vg_replace_malloc.c:303)<br>
==42050== by 0x10153B704: superlu_malloc_dist (memory.c:108)<br>
==42050== by 0x10150B241: ddist_psymbtonum (pdsymbfact_distdata.c:1389)<br>
==42050== by 0x1015018C2: pdgssvx (pdgssvx.c:1057)<br>
==42050== by 0x1009CFE7A: MatLUFactorNumeric_SuperLU_DIST (superlu_dist.c:414)<br>
==42050== by 0x10046CC5C: MatLUFactorNumeric (matrix.c:2946)<br>
==42050== by 0x100F09F2C: PCSetUp_LU (lu.c:152)<br>
==42050== by 0x100FF9036: PCSetUp (precon.c:982)<br>
==42050== by 0x1010F54EB: KSPSetUp (itfunc.c:332)<br>
==42050== by 0x1010F7985: KSPSolve (itfunc.c:546)<br>
==42050== by 0x10125541E: SNESSolve_NEWTONLS (ls.c:233)<br>
==42050== by 0x1011C49B7: SNESSolve (snes.c:3906)<br>
==42050== by 0x100001B3C: main (in ./ex19)<br>
==42050==<br>
<br>
<br>
> On Jul 20, 2015, at 12:03 PM, <a href="mailto:Mahir.Ulker-Kaustell@tyrens.se">Mahir.Ulker-Kaustell@tyrens.se</a> wrote:<br>
><br>
> Ok. So I have been creating the full factorization on each process. That gives me some hope!<br>
><br>
> I followed your suggestion and tried to use the runtime option ‘-mat_superlu_dist_parsymbfact’.<br>
> However, now the program crashes with:<br>
><br>
> Invalid ISPEC at line 484 in file get_perm_c.c<br>
><br>
> And so on…<br>
><br>
> From the SuperLU manual; I should give the option either YES or NO, however -mat_superlu_dist_parsymbfact YES makes the program crash in the same way as above.<br>
> Also I can’t find any reference to -mat_superlu_dist_parsymbfact in the PETSc documentation<br>
><br>
> Mahir<br>
><br>
> Mahir Ülker-Kaustell, Kompetenssamordnare, Brokonstruktör, Tekn. Dr, Tyréns AB<br>
> 010 452 30 82, <a href="mailto:Mahir.Ulker-Kaustell@tyrens.se">Mahir.Ulker-Kaustell@tyrens.se</a><br>
><br>
> From: Xiaoye S. Li [mailto:<a href="mailto:xsli@lbl.gov">xsli@lbl.gov</a>]<br>
> Sent: den 20 juli 2015 18:12<br>
> To: Ülker-Kaustell, Mahir<br>
> Cc: Hong; petsc-users<br>
> Subject: Re: [petsc-users] SuperLU MPI-problem<br>
><br>
> The default SuperLU_DIST setting is to serial symbolic factorization. Therefore, what matters is how much memory do you have per MPI task?<br>
><br>
> The code failed to malloc memory during redistribution of matrix A to {L\U} data struction (using result of serial symbolic factorization.)<br>
><br>
> You can use parallel symbolic factorization, by runtime option: '-mat_superlu_dist_parsymbfact'<br>
><br>
> Sherry Li<br>
><br>
><br>
> On Mon, Jul 20, 2015 at 8:59 AM, <a href="mailto:Mahir.Ulker-Kaustell@tyrens.se">
Mahir.Ulker-Kaustell@tyrens.se</a> <<a href="mailto:Mahir.Ulker-Kaustell@tyrens.se">Mahir.Ulker-Kaustell@tyrens.se</a>> wrote:<br>
> Hong:<br>
><br>
> Previous experiences with this equation have shown that it is very difficult to solve it iteratively. Hence the use of a direct solver.<br>
><br>
> The large test problem I am trying to solve has slightly less than 10^6 degrees of freedom. The matrices are derived from finite elements so they are sparse.<br>
> The machine I am working on has 128GB ram. I have estimated the memory needed to less than 20GB, so if the solver needs twice or even three times as much, it should still work well. Or have I completely misunderstood something here?<br>
><br>
> Mahir<br>
><br>
><br>
><br>
> From: Hong [mailto:<a href="mailto:hzhang@mcs.anl.gov">hzhang@mcs.anl.gov</a>]<br>
> Sent: den 20 juli 2015 17:39<br>
> To: Ülker-Kaustell, Mahir<br>
> Cc: petsc-users<br>
> Subject: Re: [petsc-users] SuperLU MPI-problem<br>
><br>
> Mahir:<br>
> Direct solvers consume large amount of memory. Suggest to try followings:<br>
><br>
> 1. A sparse iterative solver if [-omega^2M + K] is not too ill-conditioned. You may test it using the small matrix.<br>
><br>
> 2. Incrementally increase your matrix sizes. Try different matrix orderings.<br>
> Do you get memory crash in the 1st symbolic factorization?<br>
> In your case, matrix data structure stays same when omega changes, so you only need to do one matrix symbolic factorization and reuse it.<br>
><br>
> 3. Use a machine that gives larger memory.<br>
><br>
> Hong<br>
><br>
> Dear Petsc-Users,<br>
><br>
> I am trying to use PETSc to solve a set of linear equations arising from Naviers equation (elastodynamics) in the frequency domain.<br>
> The frequency dependency of the problem requires that the system<br>
><br>
> [-omega^2M + K]u = F<br>
><br>
> where M and K are constant, square, positive definite matrices (mass and stiffness respectively) is solved for each frequency omega of interest.<br>
> K is a complex matrix, including material damping.<br>
><br>
> I have written a PETSc program which solves this problem for a small (1000 degrees of freedom) test problem on one or several processors, but it keeps crashing when I try it on my full scale (in the order of 10^6 degrees of freedom) problem.<br>
><br>
> The program crashes at KSPSetUp() and from what I can see in the error messages, it appears as if it consumes too much memory.<br>
><br>
> I would guess that similar problems have occurred in this mail-list, so I am hoping that someone can push me in the right direction…<br>
><br>
> Mahir<br>
<br>
<o:p></o:p></p>
</blockquote>
</div>
<p class="MsoNormal"><br>
<br clear="all">
<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal">-- <o:p></o:p></p>
<div>
<p class="MsoNormal">What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>
-- Norbert Wiener<o:p></o:p></p>
</div>
</div>
</div>
</div>
</body>
</html>