[petsc-users] LU Performance

[Abhyankar, Shrirang G]

I believe standalone UMFPACK uses appropriate minimum degree (amd) ordering by default provided by package AMD.

Use the option -pc_factor_mat_ordering_type amd to use the amd ordering. You can also try quotient minimum degree (qmd) ordering available in PETSc (-pc_factor_mat_ordering_type qmd)

Thanks,
Shri
From: petsc-users <petsc-users-bounces at mcs.anl.gov> on behalf of Jed Brown via petsc-users <petsc-users at mcs.anl.gov>
Reply-To: Jed Brown <jed at jedbrown.org>
Date: Friday, July 5, 2019 at 9:48 AM
To: Stefano Zampini <stefano.zampini at gmail.com>, Jared Crean <jcrean01 at gmail.com>
Cc: PETSc users list <petsc-users at mcs.anl.gov>
Subject: Re: [petsc-users] LU Performance

Stefano Zampini via petsc-users <petsc-users at mcs.anl.gov<mailto:petsc-users at mcs.anl.gov>> writes:

Jared,

The petsc output shows

package used to perform factorization: petsc

You are not using umfpack, but the PETSc native LU. You can run with -options_left  to see the options that are not processed from the PETSc options database.

-pc_factor_mat_solver_package was used until PETSc-3.9.  You're using a
rather old (~2016) PETSc with new options.

https://www.mcs.anl.gov/petsc/documentation/changes/39.html



On Jul 5, 2019, at 10:26 AM, Jared Crean via petsc-users <petsc-users at mcs.anl.gov<mailto:petsc-users at mcs.anl.gov>> wrote:
     This is in reply to both David and Barry's emails.
     I am using the Umfpack that Petsc built (--download-suitesparse=yes was passed to configure), so all the compiler flags and Blas/Lapack libraries are the same.  I used OpenBlas for Blas and Lapack, with multi-threading disabled.  When calling Umfpack directly, the factorization takes about 4 second, compared to 135 seconds spent in MatLUFactorNum when using Umfpack via Petsc.
     I added a call to umfpack_di_report_control()  (which prints  the Umfpack parameters) to my code, and also added -mat_umfpack_prl 2 to the Petsc options, which should cause Petsc to call the same function just before doing the symbolic factorization (umfpack.c line 245 in Petsc 3.7.6). The output is attached (also with the -ksp_view option).  My code did print the parameters, but Petsc did not, which makes me think MatLUFactorSymbolic_UMFPACK never got called.  For reference, here is how I am invoking the program:
./test -ksp_type preonly -pc_type lu -pc_factor_mat_solver_type umfpack -log_view -ksp_view -mat_umfpack_prl 2 > fout_umfpacklu
     Jared Crean
On 7/5/19 4:02 AM, Smith, Barry F. wrote:
    When you use Umfpack standalone do you use OpenMP threads? When you use umfpack alone do you us thread enabled BLAS/LAPACK? Perhaps OpenBLAS or MKL?
    You can run both cases with -ksp_view and it will print more details indicating indicating the solver used.
     Do you use the same compiler and same options when compiling PETSc and Umfpack standalone. Is the Umfpack standalone time in the numerical factorization much smaller? Perhaps umfpack is using a much better ordering then when used with PETSc (perhaps the default orderings are different).
    Does Umfpack has a routine that tiggers output of the parameters etc it is using? If you can trigger it you might see differences between standalone and not.
    Barry
On Jul 4, 2019, at 4:05 PM, Jared Crean via petsc-users <petsc-users at mcs.anl.gov<mailto:petsc-users at mcs.anl.gov>> wrote:
Hello,
     I am getting very bad performance from the Umfpack LU solver when I use it via Petsc compared to calling Umfpack directly. It takes about 5.5 seconds to factor and solve the matrix with Umfpack, but 140 seconds when I use Petsc with -ksp_type preonly -pc_type lu -pc_factor_mat_solver_type umfpack.
     I have attached a minimal example (test.c) that reads a matrix from a file, solves with Umfpack, and then solves with Petsc.  The matrix data files are not included because they are about 250 megabytes.  I also attached the output of the program with -log_view for -pc_factor_mat_solver_type umfpack (fout_umfpacklu) and -pc_factor_mat_solver_type petsc (fout_petsclu).  Both results show nearly all of the time is spent in MatLuFactorNum.  The times are very similar, so I am wondering if Petsc is really calling Umfpack or if the Petsc LU solver is getting called in both cases.
     Jared Crean
<test_files.tar.gz>
<fout_umfpacklu.txt>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20190705/90e5bf2c/attachment-0001.html>