[petsc-users] Setting up nonlinear preconditioning with petsc4py

Thu Dec 5 01:11:01 CST 2024

Sure, I can even share the relevant parts of the code as this is part of my project cashocs. The related PR can be found here: https://github.com/sblauth/cashocs/pull/530

The relevant part of the SNES solver setup with FEniCS is here: https://github.com/sblauth/cashocs/blob/main/cashocs/nonlinear_solvers/snes.py#L235

There, I realized that FEniCS by default generates empty matrices and vectors which will have the correct size and sparsity pattern only after they have been assembled at least once. But I noticed that if I did only assemble them during the call to snes.solve, then, e.g., using MUMPS as solver caused issues. If I preassemble the matrices, everything works fine. And the same goes for the residual, which is what caused the issues with the nonlinear preconditioning.

Best,

Sebastian

--

Dr. Sebastian Blauth

Fraunhofer-Institut für

Techno- und Wirtschaftsmathematik ITWM

Abteilung Transportvorgänge

Fraunhofer-Platz 1, 67663 Kaiserslautern

Telefon: +49 631 31600-4968

sebastian.blauth at itwm.fraunhofer.de <mailto:sebastian.blauth at itwm.fraunhofer.de> 

https://www.itwm.fraunhofer.de

From: Matthew Knepley <knepley at gmail.com> 
Sent: Wednesday, December 4, 2024 3:51 PM
To: Blauth, Sebastian <sebastian.blauth at itwm.fraunhofer.de>
Cc: Stefano Zampini <stefano.zampini at gmail.com>; PETSc users list <petsc-users at mcs.anl.gov>
Subject: Re: [petsc-users] Setting up nonlinear preconditioning with petsc4py

On Wed, Dec 4, 2024 at 8:09 AM Blauth, Sebastian <sebastian.blauth at itwm.fraunhofer.de <mailto:sebastian.blauth at itwm.fraunhofer.de> > wrote:

Thanks a lot for the hint. With this I could find the error myself: I had not initialized the residual vector correctly, which has to be duplicated, hence the error. Assembling this once with FEniCS fixed the error and I can now use nonlinear preconditioning. Thanks a lot!

I would be very interested to hear how it works if you have the time to mail us back.

  Thanks

     Matt

Best,

Sebastian

--

Dr. Sebastian Blauth

Fraunhofer-Institut für

Techno- und Wirtschaftsmathematik ITWM

Abteilung Transportvorgänge

Fraunhofer-Platz 1, 67663 Kaiserslautern

Telefon: +49 631 31600-4968

sebastian.blauth at itwm.fraunhofer.de <mailto:sebastian.blauth at itwm.fraunhofer.de> 

https://www.itwm.fraunhofer.de

From: Stefano Zampini <stefano.zampini at gmail.com <mailto:stefano.zampini at gmail.com> > 
Sent: Wednesday, December 4, 2024 1:56 PM
To: Matthew Knepley <knepley at gmail.com <mailto:knepley at gmail.com> >
Cc: Blauth, Sebastian <sebastian.blauth at itwm.fraunhofer.de <mailto:sebastian.blauth at itwm.fraunhofer.de> >; PETSc users list <petsc-users at mcs.anl.gov <mailto:petsc-users at mcs.anl.gov> >
Subject: Re: [petsc-users] Setting up nonlinear preconditioning with petsc4py

FENICS swallows any useful error information returned from PETSc. You can try using the below code snippet at the beginning of your script right after you loaded dolfin stuff

PETSc.Sys.pushErrorHandler('python')

On Wed, Dec 4, 2024, 15:48 Matthew Knepley <knepley at gmail.com <mailto:knepley at gmail.com> > wrote:

On Wed, Dec 4, 2024 at 4:44 AM Blauth, Sebastian <sebastian.blauth at itwm.fraunhofer.de <mailto:sebastian.blauth at itwm.fraunhofer.de> > wrote:

Hi everyone,

I wanted to try nonlinear preconditioning for solving nonlinear systems. The problems arise from PDEs discretized with FEniCS and I have successfully implemented a wrapper for SNES. However, once I want to try nonlinear preconditioning by adding the option “-npc_snes_type newtonls” (or other solvers such as nrichardson), I get the error code 56.

My approach looks something like this

from petsc4py import PETSc

snes = PETSc.SNES().create()

snes.setFunction(self.assemble_function, self.residual_petsc)

snes.setJacobian(self.assemble_jacobian, self.A_petsc, self.P_petsc)

snes.setFromOptions()

snes.solve(None, self.u.vector().vec())

snes.destroy()

I don’t think that my user defined functions are problematic. The code runs well with the options (just an example)

-snes_type newtonls

-snes_rtol 1e-6

-snes_monitor

-ksp_type gmres

-ksp_monitor_true_residual

-pc_type lu

-pc_factor_mat_solver_type mumps

-mat_mumps_icntl_24

Here, the gmres is just used to verify that the direct solver works as expected. Newton Krylov methods also work well.

However, once I use

-snes_type nrichardson

-npc_snes_type newtonls

-npc_snes_max_it 4

(as discussed e.g. in https://climatemodeling.org/~rmills/talks/PSU-ACM-seminar-2024.pdf <https://urldefense.us/v3/__https:/climatemodeling.org/*rmills/talks/PSU-ACM-seminar-2024.pdf__;fg!!G_uCfscf7eWS!ZpQxakVxVcyih0WUbW_MxFNHXR0tUbZAvis8z497q782yqbJraYBBCMst79xJm6nlGdBUqW3I2vIDAtJiUTR$> ) I get the error message

File "/p/tv/blauths/cashocs/cashocs/nonlinear_solvers/snes.py", line 250, in solve

    snes.solve(None, self.u.vector().vec())

  File "petsc4py/PETSc/SNES.pyx", line 1555, in petsc4py.PETSc.SNES.solve

petsc4py.PETSc.Error: error code 56

I would be really grateful if someone could point me to the right direction on how to use nonlinear precondition with petsc4py.

56 is PETSC_ERR_SUP, which means we hit an unsupported operation. petsc4py preserves the stack trace, so FEniCS should not be throwing it away (along with the error message). Maybe run in the debugger so we can see the error message and stack?

  Thanks,

      Matt

Thanks a lot in advance,

Sebastian

--

Dr. Sebastian Blauth

Fraunhofer-Institut für

Techno- und Wirtschaftsmathematik ITWM

Abteilung Transportvorgänge

Fraunhofer-Platz 1, 67663 Kaiserslautern <https://www.google.com/maps/search/Fraunhofer-Platz+1,+67663+Kaiserslautern?entry=gmail&source=g> 

Telefon: +49 631 31600-4968

sebastian.blauth at itwm.fraunhofer.de <mailto:sebastian.blauth at itwm.fraunhofer.de> 

https://www.itwm.fraunhofer.de <https://urldefense.us/v3/__https:/www.itwm.fraunhofer.de__;!!G_uCfscf7eWS!ZpQxakVxVcyih0WUbW_MxFNHXR0tUbZAvis8z497q782yqbJraYBBCMst79xJm6nlGdBUqW3I2vIDJ9nKGZK$> 

-- 

What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.
-- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/ <https://urldefense.us/v3/__http:/www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!ZpQxakVxVcyih0WUbW_MxFNHXR0tUbZAvis8z497q782yqbJraYBBCMst79xJm6nlGdBUqW3I2vIDIJYMNfe$> 

-- 

What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.
-- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20241205/b37954e1/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 7943 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20241205/b37954e1/attachment-0001.p7s>