<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"Times New Roman \(Body CS\)";
panose-1:2 11 6 4 2 2 2 2 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Times New Roman",serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Times New Roman",serif">I believe standalone UMFPACK uses appropriate minimum degree (amd) ordering by default provided by package AMD.
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Times New Roman",serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Times New Roman",serif">Use the option -pc_factor_mat_ordering_type amd to use the amd ordering. You can also try quotient minimum degree (qmd) ordering available in PETSc (-pc_factor_mat_ordering_type
qmd)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Times New Roman",serif"><o:p> </o:p></span></p>
<div>
<p class="MsoNormal">Thanks,<o:p></o:p></p>
<p class="MsoNormal">Shri<o:p></o:p></p>
</div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="margin-left:.5in"><b><span style="font-size:12.0pt;color:black">From:
</span></b><span style="font-size:12.0pt;color:black">petsc-users <petsc-users-bounces@mcs.anl.gov> on behalf of Jed Brown via petsc-users <petsc-users@mcs.anl.gov><br>
<b>Reply-To: </b>Jed Brown <jed@jedbrown.org><br>
<b>Date: </b>Friday, July 5, 2019 at 9:48 AM<br>
<b>To: </b>Stefano Zampini <stefano.zampini@gmail.com>, Jared Crean <jcrean01@gmail.com><br>
<b>Cc: </b>PETSc users list <petsc-users@mcs.anl.gov><br>
<b>Subject: </b>Re: [petsc-users] LU Performance<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">Stefano Zampini via petsc-users <<a href="mailto:petsc-users@mcs.anl.gov">petsc-users@mcs.anl.gov</a>> writes:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
</div>
<blockquote style="border:none;border-left:solid #B5C4DF 4.5pt;padding:0in 0in 0in 4.0pt;margin-left:3.75pt;margin-right:0in" id="MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE">
<div>
<p class="MsoNormal" style="margin-left:.5in">Jared,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">The petsc output shows<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">package used to perform factorization: petsc<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">You are not using umfpack, but the PETSc native LU. You can run with -options_left to see the options that are not processed from the PETSc options database.<o:p></o:p></p>
</div>
</blockquote>
<div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">-pc_factor_mat_solver_package was used until PETSc-3.9. You're using a<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">rather old (~2016) PETSc with new options.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><a href="https://www.mcs.anl.gov/petsc/documentation/changes/39.html">https://www.mcs.anl.gov/petsc/documentation/changes/39.html</a><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
</div>
<blockquote style="border:none;border-left:solid #B5C4DF 4.5pt;padding:0in 0in 0in 4.0pt;margin-left:3.75pt;margin-right:0in" id="MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE">
<div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
</div>
<blockquote style="border:none;border-left:solid #B5C4DF 4.5pt;padding:0in 0in 0in 4.0pt;margin-left:3.75pt;margin-right:0in" id="MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE">
<div>
<p class="MsoNormal" style="margin-left:.5in">On Jul 5, 2019, at 10:26 AM, Jared Crean via petsc-users <<a href="mailto:petsc-users@mcs.anl.gov">petsc-users@mcs.anl.gov</a>> wrote:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"> This is in reply to both David and Barry's emails.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"> I am using the Umfpack that Petsc built (--download-suitesparse=yes was passed to configure), so all the compiler flags and Blas/Lapack libraries are the same. I used OpenBlas for Blas and Lapack, with multi-threading
disabled. When calling Umfpack directly, the factorization takes about 4 second, compared to 135 seconds spent in MatLUFactorNum when using Umfpack via Petsc.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"> I added a call to umfpack_di_report_control() (which prints the Umfpack parameters) to my code, and also added -mat_umfpack_prl 2 to the Petsc options, which should cause Petsc to call the same function just
before doing the symbolic factorization (umfpack.c line 245 in Petsc 3.7.6). The output is attached (also with the -ksp_view option). My code did print the parameters, but Petsc did not, which makes me think MatLUFactorSymbolic_UMFPACK never got called. For
reference, here is how I am invoking the program:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">./test -ksp_type preonly -pc_type lu -pc_factor_mat_solver_type umfpack -log_view -ksp_view -mat_umfpack_prl 2 > fout_umfpacklu<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"> Jared Crean<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">On 7/5/19 4:02 AM, Smith, Barry F. wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #B5C4DF 4.5pt;padding:0in 0in 0in 4.0pt;margin-left:3.75pt;margin-right:0in" id="MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE">
<div>
<p class="MsoNormal" style="margin-left:.5in"> When you use Umfpack standalone do you use OpenMP threads? When you use umfpack alone do you us thread enabled BLAS/LAPACK? Perhaps OpenBLAS or MKL?<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"> You can run both cases with -ksp_view and it will print more details indicating indicating the solver used.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"> Do you use the same compiler and same options when compiling PETSc and Umfpack standalone. Is the Umfpack standalone time in the numerical factorization much smaller? Perhaps umfpack is using a much better
ordering then when used with PETSc (perhaps the default orderings are different).<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"> Does Umfpack has a routine that tiggers output of the parameters etc it is using? If you can trigger it you might see differences between standalone and not.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"> Barry<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #B5C4DF 4.5pt;padding:0in 0in 0in 4.0pt;margin-left:3.75pt;margin-right:0in" id="MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE">
<div>
<p class="MsoNormal" style="margin-left:.5in">On Jul 4, 2019, at 4:05 PM, Jared Crean via petsc-users <<a href="mailto:petsc-users@mcs.anl.gov">petsc-users@mcs.anl.gov</a>> wrote:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">Hello,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"> I am getting very bad performance from the Umfpack LU solver when I use it via Petsc compared to calling Umfpack directly. It takes about 5.5 seconds to factor and solve the matrix with Umfpack, but 140 seconds
when I use Petsc with -ksp_type preonly -pc_type lu -pc_factor_mat_solver_type umfpack.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"> I have attached a minimal example (test.c) that reads a matrix from a file, solves with Umfpack, and then solves with Petsc. The matrix data files are not included because they are about 250 megabytes. I
also attached the output of the program with -log_view for -pc_factor_mat_solver_type umfpack (fout_umfpacklu) and -pc_factor_mat_solver_type petsc (fout_petsclu). Both results show nearly all of the time is spent in MatLuFactorNum. The times are very similar,
so I am wondering if Petsc is really calling Umfpack or if the Petsc LU solver is getting called in both cases.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"> Jared Crean<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><test_files.tar.gz><o:p></o:p></p>
</div>
</blockquote>
</blockquote>
<div>
<p class="MsoNormal" style="margin-left:.5in"><fout_umfpacklu.txt><o:p></o:p></p>
</div>
</blockquote>
</blockquote>
<div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
</div>
</div>
</body>
</html>