[petsc-users] solving saddle point problem in distributed memory

Wed Jul 29 06:53:49 CDT 2020

On Wed, Jul 29, 2020 at 6:30 AM Bin Liu <lbllm2018 at hotmail.com> wrote:

> Thanks for your suggestions. Sure. I will check my codes of MatZeroRows
> from a small problem. The program does not terminate during solving in
> distributed memory, but repeating this error in run time. In fact, this is
> the entire error message repeating itself during runtime.
>

Are you not checking the return codes for PETSc calls in your code?

  Matt

>
>
>
>
> *From:* Matthew Knepley [mailto:knepley at gmail.com]
> *Sent:* Wednesday, 29 July 2020 5:21 PM
> *To:* Bin Liu <lbllm2018 at hotmail.com>
> *Cc:* Barry Smith <bsmith at petsc.dev>; petsc-users at mcs.anl.gov
> *Subject:* Re: [petsc-users] solving saddle point problem in distributed
> memory
>
>
>
> On Tue, Jul 28, 2020 at 11:33 PM Bin Liu <lbllm2018 at hotmail.com> wrote:
>
> Dear Barry,
>
>
>
> Thanks for your suggestions. I just tested it. I still get the same error.
> I attached it below:
>
>
>
> [1]PETSC ERROR: --------------------- Error Message
> --------------------------------------------------------------
>
> [1]PETSC ERROR: Object is in wrong state
>
> [1]PETSC ERROR: Matrix is missing diagonal entry in the zeroed row 24214
>
> [1]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html
> for trouble shooting.
>
> [1]PETSC ERROR: Petsc Release Version 3.9.1, Apr, 29, 2018
>
> [1]PETSC ERROR: ./re100 on a  named kdp-MS-7850 by kdp Wed Jul 29 11:26:57
> 2020
>
> [1]PETSC ERROR: Configure options --prefix=/home/kdp/petsc --debugging=0
> --with-mpi-dir=/home/kdp/mpich --download-fblaslapack --download-metis
> --download-parmetis --download-hypre=/home/kdp/hypre-2.14.0.tar.gz
> --download-superlu_dist
>
> [1]PETSC ERROR: #1 MatZeroRows_SeqAIJ() line 1819 in
> /home/kdp/petsc-3.9.1/src/mat/impls/aij/seq/aij.c
>
> [1]PETSC ERROR: #2 MatZeroRows() line 5710 in
> /home/kdp/petsc-3.9.1/src/mat/interface/matrix.c
>
> [1]PETSC ERROR: #3 MatZeroRows_MPIAIJ() line 797 in
> /home/kdp/petsc-3.9.1/src/mat/impls/aij/mpi/mpiaij.c
>
> [1]PETSC ERROR: #4 MatZeroRows() line 5710 in
> /home/kdp/petsc-3.9.1/src/mat/interface/matrix.c
>
>
> It still report missing diagonal entry. However I have epxlicitly assign
> 0.0 to the diagonal element of the system matrix and solve it in sequential
> mode without error. Where could possibly my wrong set up be?
>
>
>
> 1) Please send the complete error message. The stack trace here is cut off.
>
>
>
> 2) It does not look like it is failing in the solver, but in the call to
> MatZeroRows(), which as it says, requires diagonal elements
>
>
>
> 3) It looks like you are not assembling the matrix correctly in parallel
>
>
>
> I recommend making as small a problem as possible, running on 1 and 2
> processes, and printing out the matrix using -mat_view to compare them
> entry-by-entry.
>
>
>
>   Thanks,
>
>
>
>      Matt
>
>
>
> Best
>
>
> ------------------------------
>
> *From:* Barry Smith <bsmith at petsc.dev>
> *Sent:* Wednesday, July 29, 2020 10:49 AM
> *To:* Bin Liu <lbllm2018 at hotmail.com>
> *Cc:* petsc-users at mcs.anl.gov <petsc-users at mcs.anl.gov>
> *Subject:* Re: [petsc-users] solving saddle point problem in distributed
> memory
>
>
>
>
>
>    No, PETSc direct solvers do not work in parallel. Only SuperLU_DIST,
> MUMPS, and PasTix.
>
>
>
>     -pc_type lu (or cholesky depending on the package)
> -pc_factor_mat_solver_type superlu_dist or mumps or pastix
>
>
>
>    to run the direct solver in parallel.
>
>
>
>
>
>
>
> On Jul 28, 2020, at 9:01 PM, Bin Liu <lbllm2018 at hotmail.com> wrote:
>
>
>
> Dear Barry,
>
>
>
> Thanks for your explanation. Does it mean the default director solver LU
> in PETSc can also run in the distributed memory mode? I only used iterative
> solvers with preconditioners in distributed memory before.  My experience
> in using director solver in parallel is indeed limited, especially the
> saddle point problem in parallel. I have done some search online, but I did
> not find a working set up at this moment. Could you give a sample setup of
> the direct solver for parallel run? It is really appreciated.
>
>
>
> *From:* Barry Smith [mailto:bsmith at petsc.dev <bsmith at petsc.dev>]
> *Sent:* Wednesday, 29 July 2020 9:48 AM
> *To:* Bin Liu <lbllm2018 at hotmail.com>
> *Cc:* petsc-users at mcs.anl.gov
> *Subject:* Re: [petsc-users] solving saddle point problem in distributed
> memory
>
>
>
>
>
>    SuperLU_DIST won't "magically" run on saddle point problems. It only
> does limited pivoting, realistically a parallel LU cannot always do
> complete pivoting or it becomes a sequential algorithm. For parallel you
> need to use PCFIELDSPLIT, for sequential you can use SuperLU (not
> SuperLU_DIST) since it can do more pivoting, being a sequential algorithm.
>
>
>
>   Barry
>
>
>
>
>
> On Jul 28, 2020, at 7:59 PM, Bin Liu <lbllm2018 at hotmail.com> wrote:
>
>
>
> Thanks for your tutorials. Yes. I tried PCFIELDSPLIT. However, it only
> works for sequential runs. When I run the code in distributed memory, it
> reports errors. In fact, the essence of my wonder is (a) how to set up
> superlu_dist in petsc for solving saddle point problem in distributed
> memory? (b) does the direct solvers in petsc can run in distributed memory
> for solving saddle point problem?
>
>
>
> *From:* Stefano Zampini [mailto:stefano.zampini at gmail.com
> <stefano.zampini at gmail.com>]
> *Sent:* Tuesday, 28 July 2020 6:55 PM
> *To:* Bin Liu <lbllm2018 at hotmail.com>
> *Cc:* petsc-users at mcs.anl.gov
> *Subject:* Re: [petsc-users] solving saddle point problem in distributed
> memory
>
>
>
> If you want advice you should post the error trace PETSc reports.
>
>
>
> Anyway, solving Stokes is not so trivial (without direct solvers, you may
> need mesh dependent information), but we have examples for it
>
>
>
> https://gitlab.com/petsc/petsc/-/blob/master/src/ksp/ksp/tutorials/ex42.c
>
> https://gitlab.com/petsc/petsc/-/blob/master/src/ksp/ksp/tutorials/ex43.c
>
> https://gitlab.com/petsc/petsc/-/blob/master/src/snes/tutorials/ex69.c
>
>
>
> If you scroll to the end of those files, you see a bunch of possible
> options either using PCFIELDSPLIT,  PCBDDC or KSPFETIDP.
>
>
>
>
>
> On Jul 28, 2020, at 12:37 PM, Bin Liu <lbllm2018 at hotmail.com> wrote:
>
>
>
> I would like to solve a saddle point problem arising from the stokes
> equation. I got successful to use the direct solvers in sequential runs.
> However, I would like to extend it for distributed memory computation. I
> tried to use superlu_dist, but the program returns errors. Is it possible
> to solve a saddle point problem in distributed memory using superlu_dist?
> Could anyone give a simple sample code to set up the parameters of the
> solver?
>
>
>
> Thanks
>
>
>
>
>
>
>
>
> --
>
> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which their
> experiments lead.
> -- Norbert Wiener
>
>
>
> https://www.cse.buffalo.edu/~knepley/
> <http://www.cse.buffalo.edu/~knepley/>
>

-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20200729/d4671a34/attachment.html>