From mail2amneet at gmail.com Sun May 1 00:03:14 2022 From: mail2amneet at gmail.com (Amneet Bhalla) Date: Sat, 30 Apr 2022 22:03:14 -0700 Subject: [petsc-users] GMRES for outer solver In-Reply-To: <87o80iym2c.fsf@jedbrown.org> References: <87o80iym2c.fsf@jedbrown.org> Message-ID: How about using a fixed number of Richardson iterations as a Krylov preconditioner to a GMRES solver? Would that lead to a linear operation? On Sat, Apr 30, 2022 at 8:21 PM Jed Brown wrote: > In general, no. A fixed number of Krylov iterations (CG, GMRES, etc.) is a > nonlinear operation. > > A fixed number of iterations of a method with a fixed polynomial, such as > Chebyshev, is a linear operation so you don't need a flexible outer method. > > Ramakrishnan Thirumalaisamy writes: > > > Hi, > > > > I have a Krylov solver with a preconditioner that is also a Krylov > solver. > > I know I can use "fgmres" for the outer solver but can I use gmres for > the > > outer solver with a fixed number of iterations in the Krylov > > preconditioners? > > > > > > Thanks, > > Rama > -- --Amneet -------------- next part -------------- An HTML attachment was scrubbed... URL: From dave.mayhem23 at gmail.com Sun May 1 02:12:28 2022 From: dave.mayhem23 at gmail.com (Dave May) Date: Sun, 1 May 2022 09:12:28 +0200 Subject: [petsc-users] GMRES for outer solver In-Reply-To: References: <87o80iym2c.fsf@jedbrown.org> Message-ID: On Sun 1. May 2022 at 07:03, Amneet Bhalla wrote: > How about using a fixed number of Richardson iterations as a Krylov > preconditioner to a GMRES solver? > That is fine. Would that lead to a linear operation? > Yes. > On Sat, Apr 30, 2022 at 8:21 PM Jed Brown wrote: > >> In general, no. A fixed number of Krylov iterations (CG, GMRES, etc.) is >> a nonlinear operation. >> >> A fixed number of iterations of a method with a fixed polynomial, such as >> Chebyshev, is a linear operation so you don't need a flexible outer method. >> >> Ramakrishnan Thirumalaisamy writes: >> >> > Hi, >> > >> > I have a Krylov solver with a preconditioner that is also a Krylov >> solver. >> > I know I can use "fgmres" for the outer solver but can I use gmres for >> the >> > outer solver with a fixed number of iterations in the Krylov >> > preconditioners? >> > >> > >> > Thanks, >> > Rama >> > -- > --Amneet > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rthirumalaisam1857 at sdsu.edu Sun May 1 23:23:29 2022 From: rthirumalaisam1857 at sdsu.edu (Ramakrishnan Thirumalaisamy) Date: Sun, 1 May 2022 21:23:29 -0700 Subject: [petsc-users] GMRES for outer solver In-Reply-To: References: <87o80iym2c.fsf@jedbrown.org> Message-ID: Thank you. I have a couple of questions. I am solving the low Mach Navier-Stokes system using a projection preconditioner (pc_shell type) with GMRES being the outer solver and Richardson being the Krylov preconditioner. The solver diverges when ksp_pc_type is "right?: Linear stokes_ solve did not converge due to DIVERGED_NANORINF iterations 0 and it converges when ksp_pc_type is "left": Residual norms for stokes_ solve. 0 KSP preconditioned resid norm 8.829128536017e+04 true resid norm -nan ||r(i)||/||b|| -nan 1 KSP preconditioned resid norm 1.219313641627e+00 true resid norm -nan ||r(i)||/||b|| -nan 2 KSP preconditioned resid norm 8.547033285706e-12 true resid norm -nan ||r(i)||/||b|| -nan Linear stokes_ solve converged due to CONVERGED_RTOL iterations 2 I am curious to know why this is happening. The solver also diverges with "FGMRES" as the outer solver (which supports only right preconditioning). 2. Is it also possible to not get "-nan" when || b || = 0? Regards, Rama On Sun, May 1, 2022 at 12:12 AM Dave May wrote: > > > On Sun 1. May 2022 at 07:03, Amneet Bhalla wrote: > >> How about using a fixed number of Richardson iterations as a Krylov >> preconditioner to a GMRES solver? >> > > That is fine. > > Would that lead to a linear operation? >> > > Yes. > > > >> On Sat, Apr 30, 2022 at 8:21 PM Jed Brown wrote: >> >>> In general, no. A fixed number of Krylov iterations (CG, GMRES, etc.) is >>> a nonlinear operation. >>> >>> A fixed number of iterations of a method with a fixed polynomial, such >>> as Chebyshev, is a linear operation so you don't need a flexible outer >>> method. >>> >>> Ramakrishnan Thirumalaisamy writes: >>> >>> > Hi, >>> > >>> > I have a Krylov solver with a preconditioner that is also a Krylov >>> solver. >>> > I know I can use "fgmres" for the outer solver but can I use gmres for >>> the >>> > outer solver with a fixed number of iterations in the Krylov >>> > preconditioners? >>> > >>> > >>> > Thanks, >>> > Rama >>> >> -- >> --Amneet >> >> >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From rthirumalaisam1857 at sdsu.edu Sun May 1 23:29:55 2022 From: rthirumalaisam1857 at sdsu.edu (Ramakrishnan Thirumalaisamy) Date: Sun, 1 May 2022 21:29:55 -0700 Subject: [petsc-users] GMRES for outer solver In-Reply-To: References: <87o80iym2c.fsf@jedbrown.org> Message-ID: Sorry. There is a typo in my previous email. It should be ksp_pc_side. Regards, Rama On Sun, May 1, 2022 at 9:23 PM Ramakrishnan Thirumalaisamy < rthirumalaisam1857 at sdsu.edu> wrote: > Thank you. I have a couple of questions. I am solving the low Mach > Navier-Stokes system using a projection preconditioner (pc_shell type) with > GMRES being the outer solver and Richardson being the Krylov > preconditioner. The solver diverges when ksp_pc_type is "right?: > > Linear stokes_ solve did not converge due to DIVERGED_NANORINF iterations 0 > > and it converges when ksp_pc_type is "left": > > Residual norms for stokes_ solve. > 0 KSP preconditioned resid norm 8.829128536017e+04 true resid norm > -nan ||r(i)||/||b|| -nan > 1 KSP preconditioned resid norm 1.219313641627e+00 true resid norm > -nan ||r(i)||/||b|| -nan > 2 KSP preconditioned resid norm 8.547033285706e-12 true resid norm > -nan ||r(i)||/||b|| -nan > Linear stokes_ solve converged due to CONVERGED_RTOL iterations 2 > > I am curious to know why this is happening. The solver also diverges with > "FGMRES" as the outer solver (which supports only right preconditioning). > > 2. Is it also possible to not get "-nan" when || b || = 0? > > > Regards, > Rama > > On Sun, May 1, 2022 at 12:12 AM Dave May wrote: > >> >> >> On Sun 1. May 2022 at 07:03, Amneet Bhalla wrote: >> >>> How about using a fixed number of Richardson iterations as a Krylov >>> preconditioner to a GMRES solver? >>> >> >> That is fine. >> >> Would that lead to a linear operation? >>> >> >> Yes. >> >> >> >>> On Sat, Apr 30, 2022 at 8:21 PM Jed Brown wrote: >>> >>>> In general, no. A fixed number of Krylov iterations (CG, GMRES, etc.) >>>> is a nonlinear operation. >>>> >>>> A fixed number of iterations of a method with a fixed polynomial, such >>>> as Chebyshev, is a linear operation so you don't need a flexible outer >>>> method. >>>> >>>> Ramakrishnan Thirumalaisamy writes: >>>> >>>> > Hi, >>>> > >>>> > I have a Krylov solver with a preconditioner that is also a Krylov >>>> solver. >>>> > I know I can use "fgmres" for the outer solver but can I use gmres >>>> for the >>>> > outer solver with a fixed number of iterations in the Krylov >>>> > preconditioners? >>>> > >>>> > >>>> > Thanks, >>>> > Rama >>>> >>> -- >>> --Amneet >>> >>> >>> >>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From rthirumalaisam1857 at sdsu.edu Mon May 2 00:01:10 2022 From: rthirumalaisam1857 at sdsu.edu (Ramakrishnan Thirumalaisamy) Date: Sun, 1 May 2022 22:01:10 -0700 Subject: [petsc-users] GMRES for outer solver In-Reply-To: References: <87o80iym2c.fsf@jedbrown.org> Message-ID: As given in my previous email, when ksp_pc_type is "left", the residual norms are: Residual norms for stokes_ solve. 0 KSP preconditioned resid norm 8.829128536017e+04 true resid norm -nan ||r(i)||/||b|| -nan 1 KSP preconditioned resid norm 1.219313641627e+00 true resid norm -nan ||r(i)||/||b|| -nan 2 KSP preconditioned resid norm 8.547033285706e-12 true resid norm -nan ||r(i)||/||b|| -nan Linear stokes_ solve converged due to CONVERGED_RTOL iterations 2 where true resid norm is -nan. Is this related to left preconditioning? The following thread suggests to use right preconditioning to avoid "-nan" https://www.mail-archive.com/petsc-users at mcs.anl.gov/msg34602.html Regards, Rama On Sun, May 1, 2022 at 9:29 PM Ramakrishnan Thirumalaisamy < rthirumalaisam1857 at sdsu.edu> wrote: > Sorry. There is a typo in my previous email. It should be ksp_pc_side. > > > Regards, > Rama > > On Sun, May 1, 2022 at 9:23 PM Ramakrishnan Thirumalaisamy < > rthirumalaisam1857 at sdsu.edu> wrote: > >> Thank you. I have a couple of questions. I am solving the low Mach >> Navier-Stokes system using a projection preconditioner (pc_shell type) with >> GMRES being the outer solver and Richardson being the Krylov >> preconditioner. The solver diverges when ksp_pc_type is "right?: >> >> Linear stokes_ solve did not converge due to DIVERGED_NANORINF iterations >> 0 >> >> and it converges when ksp_pc_type is "left": >> >> Residual norms for stokes_ solve. >> 0 KSP preconditioned resid norm 8.829128536017e+04 true resid norm >> -nan ||r(i)||/||b|| -nan >> 1 KSP preconditioned resid norm 1.219313641627e+00 true resid norm >> -nan ||r(i)||/||b|| -nan >> 2 KSP preconditioned resid norm 8.547033285706e-12 true resid norm >> -nan ||r(i)||/||b|| -nan >> Linear stokes_ solve converged due to CONVERGED_RTOL iterations 2 >> >> I am curious to know why this is happening. The solver also diverges >> with "FGMRES" as the outer solver (which supports only right >> preconditioning). >> >> 2. Is it also possible to not get "-nan" when || b || = 0? >> >> >> Regards, >> Rama >> >> On Sun, May 1, 2022 at 12:12 AM Dave May wrote: >> >>> >>> >>> On Sun 1. May 2022 at 07:03, Amneet Bhalla >>> wrote: >>> >>>> How about using a fixed number of Richardson iterations as a Krylov >>>> preconditioner to a GMRES solver? >>>> >>> >>> That is fine. >>> >>> Would that lead to a linear operation? >>>> >>> >>> Yes. >>> >>> >>> >>>> On Sat, Apr 30, 2022 at 8:21 PM Jed Brown wrote: >>>> >>>>> In general, no. A fixed number of Krylov iterations (CG, GMRES, etc.) >>>>> is a nonlinear operation. >>>>> >>>>> A fixed number of iterations of a method with a fixed polynomial, such >>>>> as Chebyshev, is a linear operation so you don't need a flexible outer >>>>> method. >>>>> >>>>> Ramakrishnan Thirumalaisamy writes: >>>>> >>>>> > Hi, >>>>> > >>>>> > I have a Krylov solver with a preconditioner that is also a Krylov >>>>> solver. >>>>> > I know I can use "fgmres" for the outer solver but can I use gmres >>>>> for the >>>>> > outer solver with a fixed number of iterations in the Krylov >>>>> > preconditioners? >>>>> > >>>>> > >>>>> > Thanks, >>>>> > Rama >>>>> >>>> -- >>>> --Amneet >>>> >>>> >>>> >>>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Mon May 2 07:12:03 2022 From: knepley at gmail.com (Matthew Knepley) Date: Mon, 2 May 2022 08:12:03 -0400 Subject: [petsc-users] GMRES for outer solver In-Reply-To: References: <87o80iym2c.fsf@jedbrown.org> Message-ID: On Mon, May 2, 2022 at 12:23 AM Ramakrishnan Thirumalaisamy < rthirumalaisam1857 at sdsu.edu> wrote: > Thank you. I have a couple of questions. I am solving the low Mach > Navier-Stokes system using a projection preconditioner (pc_shell type) with > GMRES being the outer solver and Richardson being the Krylov > preconditioner. The solver diverges when ksp_pc_type is "right?: > > Linear stokes_ solve did not converge due to DIVERGED_NANORINF iterations 0 > NaN can always be tracked back. I recommend tracing it back to the first NaN produced. My guess is that your equation of state if producing a NaN. Also, we have an example of low Mach flow in TS ex76. Thanks, Matt > and it converges when ksp_pc_type is "left": > > Residual norms for stokes_ solve. > 0 KSP preconditioned resid norm 8.829128536017e+04 true resid norm > -nan ||r(i)||/||b|| -nan > 1 KSP preconditioned resid norm 1.219313641627e+00 true resid norm > -nan ||r(i)||/||b|| -nan > 2 KSP preconditioned resid norm 8.547033285706e-12 true resid norm > -nan ||r(i)||/||b|| -nan > Linear stokes_ solve converged due to CONVERGED_RTOL iterations 2 > > I am curious to know why this is happening. The solver also diverges with > "FGMRES" as the outer solver (which supports only right preconditioning). > > 2. Is it also possible to not get "-nan" when || b || = 0? > > > Regards, > Rama > > On Sun, May 1, 2022 at 12:12 AM Dave May wrote: > >> >> >> On Sun 1. May 2022 at 07:03, Amneet Bhalla wrote: >> >>> How about using a fixed number of Richardson iterations as a Krylov >>> preconditioner to a GMRES solver? >>> >> >> That is fine. >> >> Would that lead to a linear operation? >>> >> >> Yes. >> >> >> >>> On Sat, Apr 30, 2022 at 8:21 PM Jed Brown wrote: >>> >>>> In general, no. A fixed number of Krylov iterations (CG, GMRES, etc.) >>>> is a nonlinear operation. >>>> >>>> A fixed number of iterations of a method with a fixed polynomial, such >>>> as Chebyshev, is a linear operation so you don't need a flexible outer >>>> method. >>>> >>>> Ramakrishnan Thirumalaisamy writes: >>>> >>>> > Hi, >>>> > >>>> > I have a Krylov solver with a preconditioner that is also a Krylov >>>> solver. >>>> > I know I can use "fgmres" for the outer solver but can I use gmres >>>> for the >>>> > outer solver with a fixed number of iterations in the Krylov >>>> > preconditioners? >>>> > >>>> > >>>> > Thanks, >>>> > Rama >>>> >>> -- >>> --Amneet >>> >>> >>> >>> -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at petsc.dev Mon May 2 09:55:50 2022 From: bsmith at petsc.dev (Barry Smith) Date: Mon, 2 May 2022 10:55:50 -0400 Subject: [petsc-users] GMRES for outer solver In-Reply-To: References: <87o80iym2c.fsf@jedbrown.org> Message-ID: <74FAE9BF-2FEC-4858-90CC-A768E1EC4F34@petsc.dev> > On May 2, 2022, at 8:12 AM, Matthew Knepley wrote: > > On Mon, May 2, 2022 at 12:23 AM Ramakrishnan Thirumalaisamy > wrote: > Thank you. I have a couple of questions. I am solving the low Mach Navier-Stokes system using a projection preconditioner (pc_shell type) with GMRES being the outer solver and Richardson being the Krylov preconditioner. The solver diverges when ksp_pc_type is "right?: > > Linear stokes_ solve did not converge due to DIVERGED_NANORINF iterations 0 > > NaN can always be tracked back. I recommend tracing it back to the first NaN produced. My guess is that your equation of state if producing a NaN. You can run in the debugger with -ksp_error_if_not_converged or -fp_trap to see when the Nan first appears. If the problem does not appear on one rank or you need to use mpiexec to start the program you can use the option -start_in_debugger to have the program started up in the debugger https://petsc.org/main/docs/manualpages/Sys/PetscAttachDebugger.html > > Also, we have an example of low Mach flow in TS ex76. > > Thanks, > > Matt > > and it converges when ksp_pc_type is "left": > > Residual norms for stokes_ solve. > 0 KSP preconditioned resid norm 8.829128536017e+04 true resid norm -nan ||r(i)||/||b|| -nan > 1 KSP preconditioned resid norm 1.219313641627e+00 true resid norm -nan ||r(i)||/||b|| -nan > 2 KSP preconditioned resid norm 8.547033285706e-12 true resid norm -nan ||r(i)||/||b|| -nan > Linear stokes_ solve converged due to CONVERGED_RTOL iterations 2 > > I am curious to know why this is happening. The solver also diverges with "FGMRES" as the outer solver (which supports only right preconditioning). > > 2. Is it also possible to not get "-nan" when || b || = 0? > > > Regards, > Rama > > On Sun, May 1, 2022 at 12:12 AM Dave May > wrote: > > > On Sun 1. May 2022 at 07:03, Amneet Bhalla > wrote: > How about using a fixed number of Richardson iterations as a Krylov preconditioner to a GMRES solver? > > That is fine. > > Would that lead to a linear operation? > > Yes. > > > > On Sat, Apr 30, 2022 at 8:21 PM Jed Brown > wrote: > In general, no. A fixed number of Krylov iterations (CG, GMRES, etc.) is a nonlinear operation. > > A fixed number of iterations of a method with a fixed polynomial, such as Chebyshev, is a linear operation so you don't need a flexible outer method. > > Ramakrishnan Thirumalaisamy > writes: > > > Hi, > > > > I have a Krylov solver with a preconditioner that is also a Krylov solver. > > I know I can use "fgmres" for the outer solver but can I use gmres for the > > outer solver with a fixed number of iterations in the Krylov > > preconditioners? > > > > > > Thanks, > > Rama > -- > --Amneet > > > > > > -- > What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. > -- Norbert Wiener > > https://www.cse.buffalo.edu/~knepley/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From matteo.semplice at uninsubria.it Mon May 2 10:25:25 2022 From: matteo.semplice at uninsubria.it (Matteo Semplice) Date: Mon, 2 May 2022 17:25:25 +0200 Subject: [petsc-users] different periodicity per dof in DMDA? Message-ID: Hi. I know that when I create a DMDA I can select periodic b.c. per grid direction. I am facing a PDE with 2 dofs per node in which one dof has periodic b.c. in the x direction and the other one periodic b.c. in the y direction. Is there a (possibly quick-and-dirty) solution to represent this in a DM (not necessarily a DMDA)? I understand the request it's strange, but I should add that we are experimenting numerically with this toy model, so in fact the b.c. may change in the future... just to stress once more that I am not after a perfect solution, but anything that would at least allow parallel runs with few processors would do for now. Thanks in advance ??? Matteo From knepley at gmail.com Mon May 2 11:07:28 2022 From: knepley at gmail.com (Matthew Knepley) Date: Mon, 2 May 2022 12:07:28 -0400 Subject: [petsc-users] different periodicity per dof in DMDA? In-Reply-To: References: Message-ID: On Mon, May 2, 2022 at 11:25 AM Matteo Semplice < matteo.semplice at uninsubria.it> wrote: > Hi. > > I know that when I create a DMDA I can select periodic b.c. per grid > direction. > > I am facing a PDE with 2 dofs per node in which one dof has periodic > b.c. in the x direction and the other one periodic b.c. in the y > direction. Is there a (possibly quick-and-dirty) solution to represent > this in a DM (not necessarily a DMDA)? > I am trying to understand what this means. Usually we think of periodicity as arising from the domain, not the field. I think it would be easiest to: a) Use two different DMDA for the fields that "match up" where needed or b) Use a DMStag since it sounds like these should live on horizontal and vertical edges Thanks, Matt > I understand the request it's strange, but I should add that we are > experimenting numerically with this toy model, so in fact the b.c. may > change in the future... just to stress once more that I am not after a > perfect solution, but anything that would at least allow parallel runs > with few processors would do for now. > > Thanks in advance > > Matteo > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From matteo.semplice at uninsubria.it Mon May 2 11:22:55 2022 From: matteo.semplice at uninsubria.it (Matteo Semplice) Date: Mon, 2 May 2022 18:22:55 +0200 Subject: [petsc-users] different periodicity per dof in DMDA? In-Reply-To: References: Message-ID: <76bb23e2-e923-8121-5a02-5b760cd39d06@uninsubria.it> Thanks! On 02/05/2022 18:07, Matthew Knepley wrote: > On Mon, May 2, 2022 at 11:25 AM Matteo Semplice > wrote: > > Hi. > > I am facing a PDE with 2 dofs per node in which one dof has periodic > b.c. in the x direction and the other one periodic b.c. in the y > direction. Is there a (possibly quick-and-dirty) solution to > represent > this in a DM (not necessarily a DMDA)? > > > I am trying to understand what this means. It comes from a toy model for more complicated quantum field theory model that's hard for me to understand. > Usually we think of periodicity as arising from the domain, not the field. > > I think it would be easiest to: > > ? a) Use two different DMDA for the fields that "match up" where needed > This is kind of what I was thinking at. So I would create two DMDA, then a DMComposite with the two and finally create Vecs and Matrices from the DMComposite? If so, then how do I assemble matrices for linear systems? If I extract submatrices, I could use MatSetValuesStencil on diagonal blocks, but how about the off-diagonal ones? These latter would have rows/cols indexed by different DMDAs. Matteo > or > > ? b) Use a DMStag since it sounds like these should live on horizontal > and vertical edges > > ? Thanks, > > ? ? ?Matt > > I understand the request it's strange, but I should add that we are > experimenting numerically with this toy model, so in fact the b.c. > may > change in the future... just to stress once more that I am not > after a > perfect solution, but anything that would at least allow parallel > runs > with few processors would do for now. > > Thanks in advance > > ???? Matteo > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which > their experiments lead. > -- Norbert Wiener > > https://www.cse.buffalo.edu/~knepley/ > -- Prof. Matteo Semplice Universit? degli Studi dell?Insubria Dipartimento di Scienza e Alta Tecnologia ? DiSAT Professore Associato Via Valleggio, 11 ? 22100 Como (CO) ? Italia tel.: +39 031 2386316 -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Mon May 2 11:28:54 2022 From: knepley at gmail.com (Matthew Knepley) Date: Mon, 2 May 2022 12:28:54 -0400 Subject: [petsc-users] different periodicity per dof in DMDA? In-Reply-To: <76bb23e2-e923-8121-5a02-5b760cd39d06@uninsubria.it> References: <76bb23e2-e923-8121-5a02-5b760cd39d06@uninsubria.it> Message-ID: On Mon, May 2, 2022 at 12:23 PM Matteo Semplice < matteo.semplice at uninsubria.it> wrote: > Thanks! > > On 02/05/2022 18:07, Matthew Knepley wrote: > > On Mon, May 2, 2022 at 11:25 AM Matteo Semplice < > matteo.semplice at uninsubria.it> wrote: > >> Hi. >> >> I am facing a PDE with 2 dofs per node in which one dof has periodic >> b.c. in the x direction and the other one periodic b.c. in the y >> direction. Is there a (possibly quick-and-dirty) solution to represent >> this in a DM (not necessarily a DMDA)? >> > > I am trying to understand what this means. > > It comes from a toy model for more complicated quantum field theory model > that's hard for me to understand. > > Usually we think of periodicity as arising from the domain, not the field. > > I think it would be easiest to: > > a) Use two different DMDA for the fields that "match up" where needed > > This is kind of what I was thinking at. So I would create two DMDA, then a > DMComposite with the two and finally create Vecs and Matrices from the > DMComposite? > > Yes. The drawback here is that DMDA will not help you make the proper Jacobian. I'm not sure how important this is to you. > If so, then how do I assemble matrices for linear systems? If I extract > submatrices, I could use MatSetValuesStencil on diagonal blocks, but how > about the off-diagonal ones? These latter would have rows/cols indexed by > different DMDAs. > > Yes, you would need to manage it. Here is an example doing this: https://petsc.org/main/src/snes/tutorials/ex28.c.html I have never loved it, but sometimes it makes the most sense. As a third option, you could manually construct the local to global map. Periodicity is encoded here. However, this is pretty intrusive. Thanks, Matt > Matteo > > > or > > b) Use a DMStag since it sounds like these should live on horizontal and > vertical edges > > Thanks, > > Matt > > >> I understand the request it's strange, but I should add that we are >> experimenting numerically with this toy model, so in fact the b.c. may >> change in the future... just to stress once more that I am not after a >> perfect solution, but anything that would at least allow parallel runs >> with few processors would do for now. >> >> Thanks in advance >> >> Matteo >> >> > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > > https://www.cse.buffalo.edu/~knepley/ > > > -- > Prof. Matteo Semplice > Universit? degli Studi dell?Insubria > Dipartimento di Scienza e Alta Tecnologia ? DiSAT > Professore Associato > Via Valleggio, 11 ? 22100 Como (CO) ? Italia > tel.: +39 031 2386316 > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From mail2amneet at gmail.com Mon May 2 12:56:32 2022 From: mail2amneet at gmail.com (Amneet Bhalla) Date: Mon, 2 May 2022 10:56:32 -0700 Subject: [petsc-users] GMRES for outer solver In-Reply-To: <74FAE9BF-2FEC-4858-90CC-A768E1EC4F34@petsc.dev> References: <87o80iym2c.fsf@jedbrown.org> <74FAE9BF-2FEC-4858-90CC-A768E1EC4F34@petsc.dev> Message-ID: Thanks Matt and Barry. Since we are using MATSHELL, we printed max norm of || b || and || A*x ||, the two things that have all the physics of the problem. Both are coming out to be finite (no NANs). Perhaps there is a NaN in PCSHELL. But this is counter-intuitive because the true residual norm || b - A*x || has a NaN where there is no PC application. We checked the velocity and pressure field in VisIt and they seem to be reasonable and matching the analytical solution well. Need to try it in a debugger or -fp_trap next. The behavior is similar to what is discussed in this thread: https://www.mail-archive.com/petsc-users at mcs.anl.gov/msg34602.html On Mon, May 2, 2022 at 7:56 AM Barry Smith wrote: > > > On May 2, 2022, at 8:12 AM, Matthew Knepley wrote: > > On Mon, May 2, 2022 at 12:23 AM Ramakrishnan Thirumalaisamy < > rthirumalaisam1857 at sdsu.edu> wrote: > >> Thank you. I have a couple of questions. I am solving the low Mach >> Navier-Stokes system using a projection preconditioner (pc_shell type) with >> GMRES being the outer solver and Richardson being the Krylov >> preconditioner. The solver diverges when ksp_pc_type is "right?: >> >> Linear stokes_ solve did not converge due to DIVERGED_NANORINF iterations >> 0 >> > > NaN can always be tracked back. I recommend tracing it back to the first > NaN produced. My guess is that your equation of state if producing a NaN. > > > You can run in the debugger with -ksp_error_if_not_converged or > -fp_trap to see when the Nan first appears. If the problem does not appear > on one rank or you need to use mpiexec to start the program you can use the > option -start_in_debugger to have the program started up in the debugger > https://petsc.org/main/docs/manualpages/Sys/PetscAttachDebugger.html > > > > > > > Also, we have an example of low Mach flow in TS ex76. > > Thanks, > > Matt > > >> and it converges when ksp_pc_type is "left": >> >> Residual norms for stokes_ solve. >> 0 KSP preconditioned resid norm 8.829128536017e+04 true resid norm >> -nan ||r(i)||/||b|| -nan >> 1 KSP preconditioned resid norm 1.219313641627e+00 true resid norm >> -nan ||r(i)||/||b|| -nan >> 2 KSP preconditioned resid norm 8.547033285706e-12 true resid norm >> -nan ||r(i)||/||b|| -nan >> Linear stokes_ solve converged due to CONVERGED_RTOL iterations 2 >> >> I am curious to know why this is happening. The solver also diverges >> with "FGMRES" as the outer solver (which supports only right >> preconditioning). >> >> 2. Is it also possible to not get "-nan" when || b || = 0? >> >> >> Regards, >> Rama >> >> On Sun, May 1, 2022 at 12:12 AM Dave May wrote: >> >>> >>> >>> On Sun 1. May 2022 at 07:03, Amneet Bhalla >>> wrote: >>> >>>> How about using a fixed number of Richardson iterations as a Krylov >>>> preconditioner to a GMRES solver? >>>> >>> >>> That is fine. >>> >>> Would that lead to a linear operation? >>>> >>> >>> Yes. >>> >>> >>> >>>> On Sat, Apr 30, 2022 at 8:21 PM Jed Brown wrote: >>>> >>>>> In general, no. A fixed number of Krylov iterations (CG, GMRES, etc.) >>>>> is a nonlinear operation. >>>>> >>>>> A fixed number of iterations of a method with a fixed polynomial, such >>>>> as Chebyshev, is a linear operation so you don't need a flexible outer >>>>> method. >>>>> >>>>> Ramakrishnan Thirumalaisamy writes: >>>>> >>>>> > Hi, >>>>> > >>>>> > I have a Krylov solver with a preconditioner that is also a Krylov >>>>> solver. >>>>> > I know I can use "fgmres" for the outer solver but can I use gmres >>>>> for the >>>>> > outer solver with a fixed number of iterations in the Krylov >>>>> > preconditioners? >>>>> > >>>>> > >>>>> > Thanks, >>>>> > Rama >>>>> >>>> -- >>>> --Amneet >>>> >>>> >>>> >>>> > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > > https://www.cse.buffalo.edu/~knepley/ > > > > -- --Amneet -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Mon May 2 13:11:33 2022 From: knepley at gmail.com (Matthew Knepley) Date: Mon, 2 May 2022 14:11:33 -0400 Subject: [petsc-users] GMRES for outer solver In-Reply-To: References: <87o80iym2c.fsf@jedbrown.org> <74FAE9BF-2FEC-4858-90CC-A768E1EC4F34@petsc.dev> Message-ID: On Mon, May 2, 2022 at 1:56 PM Amneet Bhalla wrote: > Thanks Matt and Barry. Since we are using MATSHELL, we printed max norm of > || b || and || A*x ||, the two things that have all the physics of the > problem. Both are coming out to be finite (no NANs). Perhaps there is a NaN > in PCSHELL. But this is counter-intuitive because the true residual norm || > b - A*x || has a NaN where there is no PC application. We checked the > velocity and pressure field in VisIt and they seem to be reasonable and > matching the analytical solution well. Need to try it in a debugger or > -fp_trap next. > I would recommend -fp_trap. It will just fail at the first NaN so it is really easy to find. You can also send the code and I can run it. > The behavior is similar to what is discussed in this thread: > https://www.mail-archive.com/petsc-users at mcs.anl.gov/msg34602.html > That turned out to be a bug in their code. Thanks, Matt > On Mon, May 2, 2022 at 7:56 AM Barry Smith wrote: > >> >> >> On May 2, 2022, at 8:12 AM, Matthew Knepley wrote: >> >> On Mon, May 2, 2022 at 12:23 AM Ramakrishnan Thirumalaisamy < >> rthirumalaisam1857 at sdsu.edu> wrote: >> >>> Thank you. I have a couple of questions. I am solving the low Mach >>> Navier-Stokes system using a projection preconditioner (pc_shell type) with >>> GMRES being the outer solver and Richardson being the Krylov >>> preconditioner. The solver diverges when ksp_pc_type is "right?: >>> >>> Linear stokes_ solve did not converge due to DIVERGED_NANORINF >>> iterations 0 >>> >> >> NaN can always be tracked back. I recommend tracing it back to the first >> NaN produced. My guess is that your equation of state if producing a NaN. >> >> >> You can run in the debugger with -ksp_error_if_not_converged or >> -fp_trap to see when the Nan first appears. If the problem does not appear >> on one rank or you need to use mpiexec to start the program you can use the >> option -start_in_debugger to have the program started up in the debugger >> https://petsc.org/main/docs/manualpages/Sys/PetscAttachDebugger.html >> >> >> >> >> >> >> Also, we have an example of low Mach flow in TS ex76. >> >> Thanks, >> >> Matt >> >> >>> and it converges when ksp_pc_type is "left": >>> >>> Residual norms for stokes_ solve. >>> 0 KSP preconditioned resid norm 8.829128536017e+04 true resid norm >>> -nan ||r(i)||/||b|| -nan >>> 1 KSP preconditioned resid norm 1.219313641627e+00 true resid norm >>> -nan ||r(i)||/||b|| -nan >>> 2 KSP preconditioned resid norm 8.547033285706e-12 true resid norm >>> -nan ||r(i)||/||b|| -nan >>> Linear stokes_ solve converged due to CONVERGED_RTOL iterations 2 >>> >>> I am curious to know why this is happening. The solver also diverges >>> with "FGMRES" as the outer solver (which supports only right >>> preconditioning). >>> >>> 2. Is it also possible to not get "-nan" when || b || = 0? >>> >>> >>> Regards, >>> Rama >>> >>> On Sun, May 1, 2022 at 12:12 AM Dave May >>> wrote: >>> >>>> >>>> >>>> On Sun 1. May 2022 at 07:03, Amneet Bhalla >>>> wrote: >>>> >>>>> How about using a fixed number of Richardson iterations as a Krylov >>>>> preconditioner to a GMRES solver? >>>>> >>>> >>>> That is fine. >>>> >>>> Would that lead to a linear operation? >>>>> >>>> >>>> Yes. >>>> >>>> >>>> >>>>> On Sat, Apr 30, 2022 at 8:21 PM Jed Brown wrote: >>>>> >>>>>> In general, no. A fixed number of Krylov iterations (CG, GMRES, etc.) >>>>>> is a nonlinear operation. >>>>>> >>>>>> A fixed number of iterations of a method with a fixed polynomial, >>>>>> such as Chebyshev, is a linear operation so you don't need a flexible outer >>>>>> method. >>>>>> >>>>>> Ramakrishnan Thirumalaisamy writes: >>>>>> >>>>>> > Hi, >>>>>> > >>>>>> > I have a Krylov solver with a preconditioner that is also a Krylov >>>>>> solver. >>>>>> > I know I can use "fgmres" for the outer solver but can I use gmres >>>>>> for the >>>>>> > outer solver with a fixed number of iterations in the Krylov >>>>>> > preconditioners? >>>>>> > >>>>>> > >>>>>> > Thanks, >>>>>> > Rama >>>>>> >>>>> -- >>>>> --Amneet >>>>> >>>>> >>>>> >>>>> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which their >> experiments lead. >> -- Norbert Wiener >> >> https://www.cse.buffalo.edu/~knepley/ >> >> >> >> > > -- > --Amneet > > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From mail2amneet at gmail.com Mon May 2 14:09:21 2022 From: mail2amneet at gmail.com (Amneet Bhalla) Date: Mon, 2 May 2022 12:09:21 -0700 Subject: [petsc-users] GMRES for outer solver In-Reply-To: References: <87o80iym2c.fsf@jedbrown.org> <74FAE9BF-2FEC-4858-90CC-A768E1EC4F34@petsc.dev> Message-ID: While trying to use -fp_trap with a debugger on a local system we used the debug build of our and PETSc code and all solvers are working fine (left and right PC and fgmres also) and no NaNs either. Interrogating the optimized version of the code now? On Mon, May 2, 2022 at 11:11 AM Matthew Knepley wrote: > On Mon, May 2, 2022 at 1:56 PM Amneet Bhalla > wrote: > >> Thanks Matt and Barry. Since we are using MATSHELL, we printed max norm >> of || b || and || A*x ||, the two things that have all the physics of the >> problem. Both are coming out to be finite (no NANs). Perhaps there is a NaN >> in PCSHELL. But this is counter-intuitive because the true residual norm || >> b - A*x || has a NaN where there is no PC application. We checked the >> velocity and pressure field in VisIt and they seem to be reasonable and >> matching the analytical solution well. Need to try it in a debugger or >> -fp_trap next. >> > > I would recommend -fp_trap. It will just fail at the first NaN so it is > really easy to find. You can also send the code and I can run it. > > >> The behavior is similar to what is discussed in this thread: >> https://www.mail-archive.com/petsc-users at mcs.anl.gov/msg34602.html >> > > That turned out to be a bug in their code. > > Thanks, > > Matt > > >> On Mon, May 2, 2022 at 7:56 AM Barry Smith wrote: >> >>> >>> >>> On May 2, 2022, at 8:12 AM, Matthew Knepley wrote: >>> >>> On Mon, May 2, 2022 at 12:23 AM Ramakrishnan Thirumalaisamy < >>> rthirumalaisam1857 at sdsu.edu> wrote: >>> >>>> Thank you. I have a couple of questions. I am solving the low Mach >>>> Navier-Stokes system using a projection preconditioner (pc_shell type) with >>>> GMRES being the outer solver and Richardson being the Krylov >>>> preconditioner. The solver diverges when ksp_pc_type is "right?: >>>> >>>> Linear stokes_ solve did not converge due to DIVERGED_NANORINF >>>> iterations 0 >>>> >>> >>> NaN can always be tracked back. I recommend tracing it back to the first >>> NaN produced. My guess is that your equation of state if producing a NaN. >>> >>> >>> You can run in the debugger with -ksp_error_if_not_converged or >>> -fp_trap to see when the Nan first appears. If the problem does not appear >>> on one rank or you need to use mpiexec to start the program you can use the >>> option -start_in_debugger to have the program started up in the debugger >>> https://petsc.org/main/docs/manualpages/Sys/PetscAttachDebugger.html >>> >>> >>> >>> >>> >>> >>> Also, we have an example of low Mach flow in TS ex76. >>> >>> Thanks, >>> >>> Matt >>> >>> >>>> and it converges when ksp_pc_type is "left": >>>> >>>> Residual norms for stokes_ solve. >>>> 0 KSP preconditioned resid norm 8.829128536017e+04 true resid norm >>>> -nan ||r(i)||/||b|| -nan >>>> 1 KSP preconditioned resid norm 1.219313641627e+00 true resid norm >>>> -nan ||r(i)||/||b|| -nan >>>> 2 KSP preconditioned resid norm 8.547033285706e-12 true resid norm >>>> -nan ||r(i)||/||b|| -nan >>>> Linear stokes_ solve converged due to CONVERGED_RTOL iterations 2 >>>> >>>> I am curious to know why this is happening. The solver also diverges >>>> with "FGMRES" as the outer solver (which supports only right >>>> preconditioning). >>>> >>>> 2. Is it also possible to not get "-nan" when || b || = 0? >>>> >>>> >>>> Regards, >>>> Rama >>>> >>>> On Sun, May 1, 2022 at 12:12 AM Dave May >>>> wrote: >>>> >>>>> >>>>> >>>>> On Sun 1. May 2022 at 07:03, Amneet Bhalla >>>>> wrote: >>>>> >>>>>> How about using a fixed number of Richardson iterations as a Krylov >>>>>> preconditioner to a GMRES solver? >>>>>> >>>>> >>>>> That is fine. >>>>> >>>>> Would that lead to a linear operation? >>>>>> >>>>> >>>>> Yes. >>>>> >>>>> >>>>> >>>>>> On Sat, Apr 30, 2022 at 8:21 PM Jed Brown wrote: >>>>>> >>>>>>> In general, no. A fixed number of Krylov iterations (CG, GMRES, >>>>>>> etc.) is a nonlinear operation. >>>>>>> >>>>>>> A fixed number of iterations of a method with a fixed polynomial, >>>>>>> such as Chebyshev, is a linear operation so you don't need a flexible outer >>>>>>> method. >>>>>>> >>>>>>> Ramakrishnan Thirumalaisamy writes: >>>>>>> >>>>>>> > Hi, >>>>>>> > >>>>>>> > I have a Krylov solver with a preconditioner that is also a Krylov >>>>>>> solver. >>>>>>> > I know I can use "fgmres" for the outer solver but can I use gmres >>>>>>> for the >>>>>>> > outer solver with a fixed number of iterations in the Krylov >>>>>>> > preconditioners? >>>>>>> > >>>>>>> > >>>>>>> > Thanks, >>>>>>> > Rama >>>>>>> >>>>>> -- >>>>>> --Amneet >>>>>> >>>>>> >>>>>> >>>>>> >>> >>> -- >>> What most experimenters take for granted before they begin their >>> experiments is infinitely more interesting than any results to which their >>> experiments lead. >>> -- Norbert Wiener >>> >>> https://www.cse.buffalo.edu/~knepley/ >>> >>> >>> >>> >> >> -- >> --Amneet >> >> >> >> > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > > https://www.cse.buffalo.edu/~knepley/ > > -- --Amneet -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Mon May 2 14:16:37 2022 From: knepley at gmail.com (Matthew Knepley) Date: Mon, 2 May 2022 15:16:37 -0400 Subject: [petsc-users] GMRES for outer solver In-Reply-To: References: <87o80iym2c.fsf@jedbrown.org> <74FAE9BF-2FEC-4858-90CC-A768E1EC4F34@petsc.dev> Message-ID: On Mon, May 2, 2022 at 3:09 PM Amneet Bhalla wrote: > > While trying to use -fp_trap with a debugger on a local system we used the > debug build of our and PETSc code and all solvers are working fine (left > and right PC and fgmres also) and no NaNs either. > Debugging code tends to initial all variables to 0. Uninitialized vars can be NaN. That is the first place I would look. You can usually find that with compiler warnings. Thanks, Matt > Interrogating the optimized version of the code now? > > > On Mon, May 2, 2022 at 11:11 AM Matthew Knepley wrote: > >> On Mon, May 2, 2022 at 1:56 PM Amneet Bhalla >> wrote: >> >>> Thanks Matt and Barry. Since we are using MATSHELL, we printed max norm >>> of || b || and || A*x ||, the two things that have all the physics of the >>> problem. Both are coming out to be finite (no NANs). Perhaps there is a NaN >>> in PCSHELL. But this is counter-intuitive because the true residual norm || >>> b - A*x || has a NaN where there is no PC application. We checked the >>> velocity and pressure field in VisIt and they seem to be reasonable and >>> matching the analytical solution well. Need to try it in a debugger or >>> -fp_trap next. >>> >> >> I would recommend -fp_trap. It will just fail at the first NaN so it is >> really easy to find. You can also send the code and I can run it. >> >> >>> The behavior is similar to what is discussed in this thread: >>> https://www.mail-archive.com/petsc-users at mcs.anl.gov/msg34602.html >>> >>> >> >> That turned out to be a bug in their code. >> >> Thanks, >> >> Matt >> >> >>> On Mon, May 2, 2022 at 7:56 AM Barry Smith wrote: >>> >>>> >>>> >>>> On May 2, 2022, at 8:12 AM, Matthew Knepley wrote: >>>> >>>> On Mon, May 2, 2022 at 12:23 AM Ramakrishnan Thirumalaisamy < >>>> rthirumalaisam1857 at sdsu.edu> wrote: >>>> >>>>> Thank you. I have a couple of questions. I am solving the low Mach >>>>> Navier-Stokes system using a projection preconditioner (pc_shell type) with >>>>> GMRES being the outer solver and Richardson being the Krylov >>>>> preconditioner. The solver diverges when ksp_pc_type is "right?: >>>>> >>>>> Linear stokes_ solve did not converge due to DIVERGED_NANORINF >>>>> iterations 0 >>>>> >>>> >>>> NaN can always be tracked back. I recommend tracing it back to the >>>> first NaN produced. My guess is that your equation of state if producing a >>>> NaN. >>>> >>>> >>>> You can run in the debugger with -ksp_error_if_not_converged or >>>> -fp_trap to see when the Nan first appears. If the problem does not appear >>>> on one rank or you need to use mpiexec to start the program you can use the >>>> option -start_in_debugger to have the program started up in the debugger >>>> https://petsc.org/main/docs/manualpages/Sys/PetscAttachDebugger.html >>>> >>>> >>>> >>>> >>>> >>>> >>>> Also, we have an example of low Mach flow in TS ex76. >>>> >>>> Thanks, >>>> >>>> Matt >>>> >>>> >>>>> and it converges when ksp_pc_type is "left": >>>>> >>>>> Residual norms for stokes_ solve. >>>>> 0 KSP preconditioned resid norm 8.829128536017e+04 true resid norm >>>>> -nan ||r(i)||/||b|| -nan >>>>> 1 KSP preconditioned resid norm 1.219313641627e+00 true resid norm >>>>> -nan ||r(i)||/||b|| -nan >>>>> 2 KSP preconditioned resid norm 8.547033285706e-12 true resid norm >>>>> -nan ||r(i)||/||b|| -nan >>>>> Linear stokes_ solve converged due to CONVERGED_RTOL iterations 2 >>>>> >>>>> I am curious to know why this is happening. The solver also diverges >>>>> with "FGMRES" as the outer solver (which supports only right >>>>> preconditioning). >>>>> >>>>> 2. Is it also possible to not get "-nan" when || b || = 0? >>>>> >>>>> >>>>> Regards, >>>>> Rama >>>>> >>>>> On Sun, May 1, 2022 at 12:12 AM Dave May >>>>> wrote: >>>>> >>>>>> >>>>>> >>>>>> On Sun 1. May 2022 at 07:03, Amneet Bhalla >>>>>> wrote: >>>>>> >>>>>>> How about using a fixed number of Richardson iterations as a Krylov >>>>>>> preconditioner to a GMRES solver? >>>>>>> >>>>>> >>>>>> That is fine. >>>>>> >>>>>> Would that lead to a linear operation? >>>>>>> >>>>>> >>>>>> Yes. >>>>>> >>>>>> >>>>>> >>>>>>> On Sat, Apr 30, 2022 at 8:21 PM Jed Brown wrote: >>>>>>> >>>>>>>> In general, no. A fixed number of Krylov iterations (CG, GMRES, >>>>>>>> etc.) is a nonlinear operation. >>>>>>>> >>>>>>>> A fixed number of iterations of a method with a fixed polynomial, >>>>>>>> such as Chebyshev, is a linear operation so you don't need a flexible outer >>>>>>>> method. >>>>>>>> >>>>>>>> Ramakrishnan Thirumalaisamy writes: >>>>>>>> >>>>>>>> > Hi, >>>>>>>> > >>>>>>>> > I have a Krylov solver with a preconditioner that is also a >>>>>>>> Krylov solver. >>>>>>>> > I know I can use "fgmres" for the outer solver but can I use >>>>>>>> gmres for the >>>>>>>> > outer solver with a fixed number of iterations in the Krylov >>>>>>>> > preconditioners? >>>>>>>> > >>>>>>>> > >>>>>>>> > Thanks, >>>>>>>> > Rama >>>>>>>> >>>>>>> -- >>>>>>> --Amneet >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>> >>>> -- >>>> What most experimenters take for granted before they begin their >>>> experiments is infinitely more interesting than any results to which their >>>> experiments lead. >>>> -- Norbert Wiener >>>> >>>> https://www.cse.buffalo.edu/~knepley/ >>>> >>>> >>>> >>>> >>> >>> -- >>> --Amneet >>> >>> >>> >>> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which their >> experiments lead. >> -- Norbert Wiener >> >> https://www.cse.buffalo.edu/~knepley/ >> >> > -- > --Amneet > > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From tangqi at msu.edu Mon May 2 23:21:12 2022 From: tangqi at msu.edu (Tang, Qi) Date: Tue, 3 May 2022 04:21:12 +0000 Subject: [petsc-users] Quasi newton Message-ID: <083AA6CC-6E69-4AFB-8330-DABADC3F860B@msu.edu> Hi, Our code uses FDcoloring to compute Jacobian. The log file indicates most of time is spent in evaluating residual (2600 times in one Newton solve) while it only needs 3 nonlinear iterations and 6 total linear iterations thanks to the fieldsplit pc. As a temporary solution, is it possible to evaluate Jacobian only once in one Newton solve? This should work well based on my other experience if pc is very efficient. But I cannot find such a flag. Is there any other solution, other than implementing the analytical Jacobian? Thanks, Qi From pierre.seize at onera.fr Tue May 3 01:58:14 2022 From: pierre.seize at onera.fr (Pierre Seize) Date: Tue, 3 May 2022 08:58:14 +0200 Subject: [petsc-users] Quasi newton In-Reply-To: <083AA6CC-6E69-4AFB-8330-DABADC3F860B@msu.edu> References: <083AA6CC-6E69-4AFB-8330-DABADC3F860B@msu.edu> Message-ID: <32b34537-9948-1614-cb66-fa5b12697552@onera.fr> Hi, If I may, is this what you want ? https://petsc.org/main/docs/manualpages/SNES/SNESSetLagJacobian.html Pierre On 03/05/2022 06:21, Tang, Qi wrote: > Hi, > Our code uses FDcoloring to compute Jacobian. The log file indicates most of time is spent in evaluating residual (2600 times in one Newton solve) while it only needs 3 nonlinear iterations and 6 total linear iterations thanks to the fieldsplit pc. > > As a temporary solution, is it possible to evaluate Jacobian only once in one Newton solve? This should work well based on my other experience if pc is very efficient. But I cannot find such a flag. > > Is there any other solution, other than implementing the analytical Jacobian? > > Thanks, > Qi From knepley at gmail.com Tue May 3 05:51:18 2022 From: knepley at gmail.com (Matthew Knepley) Date: Tue, 3 May 2022 06:51:18 -0400 Subject: [petsc-users] Quasi newton In-Reply-To: <32b34537-9948-1614-cb66-fa5b12697552@onera.fr> References: <083AA6CC-6E69-4AFB-8330-DABADC3F860B@msu.edu> <32b34537-9948-1614-cb66-fa5b12697552@onera.fr> Message-ID: On Tue, May 3, 2022 at 2:58 AM Pierre Seize wrote: > Hi, > > If I may, is this what you want ? > > https://petsc.org/main/docs/manualpages/SNES/SNESSetLagJacobian.html Yes, this is a good suggestion. Also, you could implement an approximation to the Jacobian. You could then improve it at each iteration using a secant update. This is what the Generalized Broyden methods do. We call them NGMRES. Thanks, Matt > Pierre > > On 03/05/2022 06:21, Tang, Qi wrote: > > Hi, > > Our code uses FDcoloring to compute Jacobian. The log file indicates > most of time is spent in evaluating residual (2600 times in one Newton > solve) while it only needs 3 nonlinear iterations and 6 total linear > iterations thanks to the fieldsplit pc. > > > > As a temporary solution, is it possible to evaluate Jacobian only once > in one Newton solve? This should work well based on my other experience if > pc is very efficient. But I cannot find such a flag. > > > > Is there any other solution, other than implementing the analytical > Jacobian? > > > > Thanks, > > Qi > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From tangqi at msu.edu Tue May 3 08:08:34 2022 From: tangqi at msu.edu (Tang, Qi) Date: Tue, 3 May 2022 13:08:34 +0000 Subject: [petsc-users] Quasi newton In-Reply-To: References: <083AA6CC-6E69-4AFB-8330-DABADC3F860B@msu.edu> <32b34537-9948-1614-cb66-fa5b12697552@onera.fr> Message-ID: <4EB6F5A2-8AEB-4856-BB34-D3F66EFA7677@msu.edu> Pierre and Matt, Thanks a lot for the suggestion. It looks like lag Jacobian is exactly what I need. We will try that. I always thought ngmres is a fancy version of Anderson. Is there any reference or example related to what you said in which one actually implemented an approximated Jacobian through ngmres? This sounds very interesting. Qi On May 3, 2022, at 4:51 AM, Matthew Knepley wrote: ? On Tue, May 3, 2022 at 2:58 AM Pierre Seize > wrote: Hi, If I may, is this what you want ? https://petsc.org/main/docs/manualpages/SNES/SNESSetLagJacobian.html Yes, this is a good suggestion. Also, you could implement an approximation to the Jacobian. You could then improve it at each iteration using a secant update. This is what the Generalized Broyden methods do. We call them NGMRES. Thanks, Matt Pierre On 03/05/2022 06:21, Tang, Qi wrote: > Hi, > Our code uses FDcoloring to compute Jacobian. The log file indicates most of time is spent in evaluating residual (2600 times in one Newton solve) while it only needs 3 nonlinear iterations and 6 total linear iterations thanks to the fieldsplit pc. > > As a temporary solution, is it possible to evaluate Jacobian only once in one Newton solve? This should work well based on my other experience if pc is very efficient. But I cannot find such a flag. > > Is there any other solution, other than implementing the analytical Jacobian? > > Thanks, > Qi -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Tue May 3 08:27:11 2022 From: knepley at gmail.com (Matthew Knepley) Date: Tue, 3 May 2022 09:27:11 -0400 Subject: [petsc-users] Quasi newton In-Reply-To: <4EB6F5A2-8AEB-4856-BB34-D3F66EFA7677@msu.edu> References: <083AA6CC-6E69-4AFB-8330-DABADC3F860B@msu.edu> <32b34537-9948-1614-cb66-fa5b12697552@onera.fr> <4EB6F5A2-8AEB-4856-BB34-D3F66EFA7677@msu.edu> Message-ID: On Tue, May 3, 2022 at 9:08 AM Tang, Qi wrote: > Pierre and Matt, > > Thanks a lot for the suggestion. It looks like lag Jacobian is exactly > what I need. We will try that. > > I always thought ngmres is a fancy version of Anderson. Is there any > reference or example related to what you said in which one actually > implemented an approximated Jacobian through ngmres? This sounds very > interesting. > Jed and Peter do it here: @inproceedings{brown2013quasinewton, author = {Jed Brown and Peter Brune}, title = {Low-rank quasi-{N}ewton updates for robust {J}acobian lagging in {N}ewton-type methods}, year = {2013}, booktitle = {International Conference on Mathematics and Computational Methods Applied to Nuclear Science and Engineering}, pages = {2554--2565}, petsc_uses={KSP}, } Thanks, Matt > Qi > > > On May 3, 2022, at 4:51 AM, Matthew Knepley wrote: > > ? > On Tue, May 3, 2022 at 2:58 AM Pierre Seize wrote: > >> Hi, >> >> If I may, is this what you want ? >> >> https://petsc.org/main/docs/manualpages/SNES/SNESSetLagJacobian.html >> > > > Yes, this is a good suggestion. > > Also, you could implement an approximation to the Jacobian. > > You could then improve it at each iteration using a secant update. This is > what the Generalized Broyden methods do. We call them NGMRES. > > Thanks, > > Matt > > >> Pierre >> >> On 03/05/2022 06:21, Tang, Qi wrote: >> > Hi, >> > Our code uses FDcoloring to compute Jacobian. The log file indicates >> most of time is spent in evaluating residual (2600 times in one Newton >> solve) while it only needs 3 nonlinear iterations and 6 total linear >> iterations thanks to the fieldsplit pc. >> > >> > As a temporary solution, is it possible to evaluate Jacobian only once >> in one Newton solve? This should work well based on my other experience if >> pc is very efficient. But I cannot find such a flag. >> > >> > Is there any other solution, other than implementing the analytical >> Jacobian? >> > >> > Thanks, >> > Qi >> > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > > https://www.cse.buffalo.edu/~knepley/ > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From rlmackie862 at gmail.com Tue May 3 10:39:49 2022 From: rlmackie862 at gmail.com (Randall Mackie) Date: Tue, 3 May 2022 08:39:49 -0700 Subject: [petsc-users] error with version 3.17.1 Message-ID: Dear PETSc team: A part of our code that has worked for years and previous versions is now failing with the latest version 3.17.1, on the KSP solve with the following error: [0]PETSC ERROR: --------------------- Error Message -------------------------------------------------------------- [0]PETSC ERROR: Invalid argument [0]PETSC ERROR: Block size 3 is incompatible with the indices: non consecutive indices 153055 153124 [0]PETSC ERROR: See https://petsc.org/release/faq/ for trouble shooting. [0]PETSC ERROR: Petsc Release Version 3.17.1, Apr 28, 2022 [0]PETSC ERROR: ./test on a linux-gfortran-complex-debug named rmackie-VirtualBox by rmackie Tue May 3 08:12:15 2022 [0]PETSC ERROR: Configure options --with-clean=1 --with-scalar-type=complex --with-debugging=1 --with-fortran=1 --download-mpich=../external/mpich-4.0.1.tar.gz [0]PETSC ERROR: #1 ISSetBlockSize() at /home/rmackie/PETSc/petsc-3.17.1/src/vec/is/is/interface/index.c:1898 [0]PETSC ERROR: #2 MatIncreaseOverlap() at /home/rmackie/PETSc/petsc-3.17.1/src/mat/interface/matrix.c:7086 [0]PETSC ERROR: #3 PCSetUp_ASM() at /home/rmackie/PETSc/petsc-3.17.1/src/ksp/pc/impls/asm/asm.c:238 [0]PETSC ERROR: #4 PCSetUp() at /home/rmackie/PETSc/petsc-3.17.1/src/ksp/pc/interface/precon.c:990 [0]PETSC ERROR: #5 KSPSetUp() at /home/rmackie/PETSc/petsc-3.17.1/src/ksp/ksp/interface/itfunc.c:407 [0]PETSC ERROR: #6 KSPSolve_Private() at /home/rmackie/PETSc/petsc-3.17.1/src/ksp/ksp/interface/itfunc.c:843 [0]PETSC ERROR: #7 KSPSolve() at /home/rmackie/PETSc/petsc-3.17.1/src/ksp/ksp/interface/itfunc.c:1078 I have a small test program with binary matrix and right hand side that will show the problem and I can send it as a zip file, please advise what email address to use or where to send it. Thanks, Randy M. -------------- next part -------------- An HTML attachment was scrubbed... URL: From aduarteg at utexas.edu Tue May 3 10:58:53 2022 From: aduarteg at utexas.edu (Alfredo J Duarte Gomez) Date: Tue, 3 May 2022 10:58:53 -0500 Subject: [petsc-users] Preconditioning Diagnostics Message-ID: Good morning PETSC team, I have a bit of an open question on diagnosing preconditioner performance. For a bit of background, I am using the TS object in combination with the matrix-free snes, and a custom user defined preconditioner (PCSHELL). Everything is implemented with the help of a DMDA. Smallest problem size that I can get away with is a grid with 2.1 million points with 4 fields each, for a total of 8.4 million equations. The preconditioner works very well overall, but at some stages of the solution it performs poorly, evidenced by significant increases in the number of GMRES iterations and the maximum/minimum eigenvalue computed using KSPComputeExtremeSingularValues(). I am trying to understand the locations where the preconditioner is not working well, so for example, is there any way to map the maximum eigenvalue to a particular location/field in the DMDA. Alternatively, are there any other ways of diagnosing where the preconditioner is not doing a good job? GMRES iterations and the max/min eigenvalue provide a good overall picture, but I am struggling to get preconditioner metrics that are specific to a location and field. So far I have taken a close look at fields such as the residual, and Newton updates, but it is difficult to tell how to assess these in combination with the preconditioner. I appreciate any suggestions. Thank you and have a good day. -Alfredo -- Alfredo Duarte Graduate Research Assistant The University of Texas at Austin -------------- next part -------------- An HTML attachment was scrubbed... URL: From pierre at joliv.et Tue May 3 11:14:45 2022 From: pierre at joliv.et (Pierre Jolivet) Date: Tue, 3 May 2022 18:14:45 +0200 Subject: [petsc-users] error with version 3.17.1 In-Reply-To: References: Message-ID: Please send to petsc-maint at mcs.anl.gov I?ll have a look, I introduced that check. Thanks, Pierre > On 3 May 2022, at 5:39 PM, Randall Mackie wrote: > > Dear PETSc team: > > A part of our code that has worked for years and previous versions is now failing with the latest version 3.17.1, on the KSP solve with the following error: > > [0]PETSC ERROR: --------------------- Error Message -------------------------------------------------------------- > [0]PETSC ERROR: Invalid argument > [0]PETSC ERROR: Block size 3 is incompatible with the indices: non consecutive indices 153055 153124 > [0]PETSC ERROR: See https://petsc.org/release/faq/ for trouble shooting. > [0]PETSC ERROR: Petsc Release Version 3.17.1, Apr 28, 2022 > [0]PETSC ERROR: ./test on a linux-gfortran-complex-debug named rmackie-VirtualBox by rmackie Tue May 3 08:12:15 2022 > [0]PETSC ERROR: Configure options --with-clean=1 --with-scalar-type=complex --with-debugging=1 --with-fortran=1 --download-mpich=../external/mpich-4.0.1.tar.gz > [0]PETSC ERROR: #1 ISSetBlockSize() at /home/rmackie/PETSc/petsc-3.17.1/src/vec/is/is/interface/index.c:1898 > [0]PETSC ERROR: #2 MatIncreaseOverlap() at /home/rmackie/PETSc/petsc-3.17.1/src/mat/interface/matrix.c:7086 > [0]PETSC ERROR: #3 PCSetUp_ASM() at /home/rmackie/PETSc/petsc-3.17.1/src/ksp/pc/impls/asm/asm.c:238 > [0]PETSC ERROR: #4 PCSetUp() at /home/rmackie/PETSc/petsc-3.17.1/src/ksp/pc/interface/precon.c:990 > [0]PETSC ERROR: #5 KSPSetUp() at /home/rmackie/PETSc/petsc-3.17.1/src/ksp/ksp/interface/itfunc.c:407 > [0]PETSC ERROR: #6 KSPSolve_Private() at /home/rmackie/PETSc/petsc-3.17.1/src/ksp/ksp/interface/itfunc.c:843 > [0]PETSC ERROR: #7 KSPSolve() at /home/rmackie/PETSc/petsc-3.17.1/src/ksp/ksp/interface/itfunc.c:1078 > > > I have a small test program with binary matrix and right hand side that will show the problem and I can send it as a zip file, please advise what email address to use or where to send it. > > Thanks, > > Randy M. -------------- next part -------------- An HTML attachment was scrubbed... URL: From tangqi at msu.edu Tue May 3 11:31:53 2022 From: tangqi at msu.edu (Tang, Qi) Date: Tue, 3 May 2022 16:31:53 +0000 Subject: [petsc-users] Quasi newton In-Reply-To: References: <083AA6CC-6E69-4AFB-8330-DABADC3F860B@msu.edu> <32b34537-9948-1614-cb66-fa5b12697552@onera.fr> <4EB6F5A2-8AEB-4856-BB34-D3F66EFA7677@msu.edu> Message-ID: This is very helpful. JFNK + Qn in that paper sounds really promising. Jed, is there a flag I can quickly try to recover some performance related to that paper (snes ex48.c)? Thanks. Qi On May 3, 2022, at 7:27 AM, Matthew Knepley wrote: ? On Tue, May 3, 2022 at 9:08 AM Tang, Qi > wrote: Pierre and Matt, Thanks a lot for the suggestion. It looks like lag Jacobian is exactly what I need. We will try that. I always thought ngmres is a fancy version of Anderson. Is there any reference or example related to what you said in which one actually implemented an approximated Jacobian through ngmres? This sounds very interesting. Jed and Peter do it here: @inproceedings{brown2013quasinewton, author = {Jed Brown and Peter Brune}, title = {Low-rank quasi-{N}ewton updates for robust {J}acobian lagging in {N}ewton-type methods}, year = {2013}, booktitle = {International Conference on Mathematics and Computational Methods Applied to Nuclear Science and Engineering}, pages = {2554--2565}, petsc_uses={KSP}, } Thanks, Matt Qi On May 3, 2022, at 4:51 AM, Matthew Knepley > wrote: ? On Tue, May 3, 2022 at 2:58 AM Pierre Seize > wrote: Hi, If I may, is this what you want ? https://petsc.org/main/docs/manualpages/SNES/SNESSetLagJacobian.html Yes, this is a good suggestion. Also, you could implement an approximation to the Jacobian. You could then improve it at each iteration using a secant update. This is what the Generalized Broyden methods do. We call them NGMRES. Thanks, Matt Pierre On 03/05/2022 06:21, Tang, Qi wrote: > Hi, > Our code uses FDcoloring to compute Jacobian. The log file indicates most of time is spent in evaluating residual (2600 times in one Newton solve) while it only needs 3 nonlinear iterations and 6 total linear iterations thanks to the fieldsplit pc. > > As a temporary solution, is it possible to evaluate Jacobian only once in one Newton solve? This should work well based on my other experience if pc is very efficient. But I cannot find such a flag. > > Is there any other solution, other than implementing the analytical Jacobian? > > Thanks, > Qi -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From pierre at joliv.et Tue May 3 11:37:15 2022 From: pierre at joliv.et (Pierre Jolivet) Date: Tue, 3 May 2022 18:37:15 +0200 Subject: [petsc-users] error with version 3.17.1 In-Reply-To: References: Message-ID: <34013906-D91A-46F5-90A2-7FF48BC7B3FC@joliv.et> Thanks for the reproducer. My guess is that your AIJ matrix has not a block size of 3. A simple call such as: call MatConvert(A,MATBAIJ,MAT_INITIAL_MATRIX,C,ierr) is also failing, while it shouldn?t if your AIJ Mat is truly made of 3x3 dense blocks. How did you determine the block size of your Mat? Are you allocating 3x3 dense blocks everywhere or are you skipping zero coefficients in your AIJ Mat? In the meantime, you can bypass the issue by not setting a block size of 3 on your Mat, or by setting different block size for the column and row distributions, see MatSetBlockSizes(). Thanks, Pierre > On 3 May 2022, at 5:39 PM, Randall Mackie wrote: > > Dear PETSc team: > > A part of our code that has worked for years and previous versions is now failing with the latest version 3.17.1, on the KSP solve with the following error: > > [0]PETSC ERROR: --------------------- Error Message -------------------------------------------------------------- > [0]PETSC ERROR: Invalid argument > [0]PETSC ERROR: Block size 3 is incompatible with the indices: non consecutive indices 153055 153124 > [0]PETSC ERROR: See https://petsc.org/release/faq/ for trouble shooting. > [0]PETSC ERROR: Petsc Release Version 3.17.1, Apr 28, 2022 > [0]PETSC ERROR: ./test on a linux-gfortran-complex-debug named rmackie-VirtualBox by rmackie Tue May 3 08:12:15 2022 > [0]PETSC ERROR: Configure options --with-clean=1 --with-scalar-type=complex --with-debugging=1 --with-fortran=1 --download-mpich=../external/mpich-4.0.1.tar.gz > [0]PETSC ERROR: #1 ISSetBlockSize() at /home/rmackie/PETSc/petsc-3.17.1/src/vec/is/is/interface/index.c:1898 > [0]PETSC ERROR: #2 MatIncreaseOverlap() at /home/rmackie/PETSc/petsc-3.17.1/src/mat/interface/matrix.c:7086 > [0]PETSC ERROR: #3 PCSetUp_ASM() at /home/rmackie/PETSc/petsc-3.17.1/src/ksp/pc/impls/asm/asm.c:238 > [0]PETSC ERROR: #4 PCSetUp() at /home/rmackie/PETSc/petsc-3.17.1/src/ksp/pc/interface/precon.c:990 > [0]PETSC ERROR: #5 KSPSetUp() at /home/rmackie/PETSc/petsc-3.17.1/src/ksp/ksp/interface/itfunc.c:407 > [0]PETSC ERROR: #6 KSPSolve_Private() at /home/rmackie/PETSc/petsc-3.17.1/src/ksp/ksp/interface/itfunc.c:843 > [0]PETSC ERROR: #7 KSPSolve() at /home/rmackie/PETSc/petsc-3.17.1/src/ksp/ksp/interface/itfunc.c:1078 > > > I have a small test program with binary matrix and right hand side that will show the problem and I can send it as a zip file, please advise what email address to use or where to send it. > > Thanks, > > Randy M. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rlmackie862 at gmail.com Tue May 3 11:54:50 2022 From: rlmackie862 at gmail.com (Randall Mackie) Date: Tue, 3 May 2022 09:54:50 -0700 Subject: [petsc-users] error with version 3.17.1 In-Reply-To: <34013906-D91A-46F5-90A2-7FF48BC7B3FC@joliv.et> References: <34013906-D91A-46F5-90A2-7FF48BC7B3FC@joliv.et> Message-ID: <931DFFC4-52B8-4E74-8C55-7EB3ECD8408F@gmail.com> Hi Pierre, Here is how I create the matrix, and how I?ve done it for many years: ! Create global matrix to hold system of equations resulting from finite discretization ! of the Maxwell equations. ngrow3=ginfo%nx*ginfo%ny*ginfo%nz*3 call MatCreate(comm,A,ierr) call MatSetSizes(A,mloc3,mloc3,ngrow3,ngrow3,ierr) call MatSetBlockSize(A,i3,ierr) call MatSetType(A,MATAIJ,ierr) call MatSeqAIJSetPreallocation(A,i15,PETSC_NULL_INTEGER,ierr) call MatMPIAIJSetPreallocation(A,i15,PETSC_NULL_INTEGER,i7,PETSC_NULL_INTEGER,ierr) call MatSetOption(A,MAT_NEW_NONZERO_ALLOCATION_ERR,PETSC_FALSE,ierr) This is a staggered grid formulation (but derived many years before PETSc had a DMStag class) and so I use regular DMDA?.for those locations that are not involved, I put a 1 on the diagonal. Randy > On May 3, 2022, at 9:37 AM, Pierre Jolivet wrote: > > Thanks for the reproducer. > My guess is that your AIJ matrix has not a block size of 3. > A simple call such as: call MatConvert(A,MATBAIJ,MAT_INITIAL_MATRIX,C,ierr) is also failing, while it shouldn?t if your AIJ Mat is truly made of 3x3 dense blocks. > How did you determine the block size of your Mat? > Are you allocating 3x3 dense blocks everywhere or are you skipping zero coefficients in your AIJ Mat? > In the meantime, you can bypass the issue by not setting a block size of 3 on your Mat, or by setting different block size for the column and row distributions, see MatSetBlockSizes(). > > Thanks, > Pierre > >> On 3 May 2022, at 5:39 PM, Randall Mackie > wrote: >> >> Dear PETSc team: >> >> A part of our code that has worked for years and previous versions is now failing with the latest version 3.17.1, on the KSP solve with the following error: >> >> [0]PETSC ERROR: --------------------- Error Message -------------------------------------------------------------- >> [0]PETSC ERROR: Invalid argument >> [0]PETSC ERROR: Block size 3 is incompatible with the indices: non consecutive indices 153055 153124 >> [0]PETSC ERROR: See https://petsc.org/release/faq/ for trouble shooting. >> [0]PETSC ERROR: Petsc Release Version 3.17.1, Apr 28, 2022 >> [0]PETSC ERROR: ./test on a linux-gfortran-complex-debug named rmackie-VirtualBox by rmackie Tue May 3 08:12:15 2022 >> [0]PETSC ERROR: Configure options --with-clean=1 --with-scalar-type=complex --with-debugging=1 --with-fortran=1 --download-mpich=../external/mpich-4.0.1.tar.gz >> [0]PETSC ERROR: #1 ISSetBlockSize() at /home/rmackie/PETSc/petsc-3.17.1/src/vec/is/is/interface/index.c:1898 >> [0]PETSC ERROR: #2 MatIncreaseOverlap() at /home/rmackie/PETSc/petsc-3.17.1/src/mat/interface/matrix.c:7086 >> [0]PETSC ERROR: #3 PCSetUp_ASM() at /home/rmackie/PETSc/petsc-3.17.1/src/ksp/pc/impls/asm/asm.c:238 >> [0]PETSC ERROR: #4 PCSetUp() at /home/rmackie/PETSc/petsc-3.17.1/src/ksp/pc/interface/precon.c:990 >> [0]PETSC ERROR: #5 KSPSetUp() at /home/rmackie/PETSc/petsc-3.17.1/src/ksp/ksp/interface/itfunc.c:407 >> [0]PETSC ERROR: #6 KSPSolve_Private() at /home/rmackie/PETSc/petsc-3.17.1/src/ksp/ksp/interface/itfunc.c:843 >> [0]PETSC ERROR: #7 KSPSolve() at /home/rmackie/PETSc/petsc-3.17.1/src/ksp/ksp/interface/itfunc.c:1078 >> >> >> I have a small test program with binary matrix and right hand side that will show the problem and I can send it as a zip file, please advise what email address to use or where to send it. >> >> Thanks, >> >> Randy M. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From pierre at joliv.et Tue May 3 12:09:35 2022 From: pierre at joliv.et (Pierre Jolivet) Date: Tue, 3 May 2022 19:09:35 +0200 Subject: [petsc-users] error with version 3.17.1 In-Reply-To: <931DFFC4-52B8-4E74-8C55-7EB3ECD8408F@gmail.com> References: <34013906-D91A-46F5-90A2-7FF48BC7B3FC@joliv.et> <931DFFC4-52B8-4E74-8C55-7EB3ECD8408F@gmail.com> Message-ID: <8A8CEED0-E1A7-4BA1-8584-E07AA856AE69@joliv.et> > On 3 May 2022, at 6:54 PM, Randall Mackie wrote: > > Hi Pierre, > > Here is how I create the matrix, and how I?ve done it for many years: > > ! Create global matrix to hold system of equations resulting from finite discretization > ! of the Maxwell equations. > ngrow3=ginfo%nx*ginfo%ny*ginfo%nz*3 > call MatCreate(comm,A,ierr) > call MatSetSizes(A,mloc3,mloc3,ngrow3,ngrow3,ierr) > call MatSetBlockSize(A,i3,ierr) I don?t know enough about your discretization stencil, but again, since a simple call to MatConvert(A,MATBAIJ,MAT_INITIAL_MATRIX,C,ierr) fails in your MWE, I doubt that this line is correct. > call MatSetType(A,MATAIJ,ierr) > call MatSeqAIJSetPreallocation(A,i15,PETSC_NULL_INTEGER,ierr) > call MatMPIAIJSetPreallocation(A,i15,PETSC_NULL_INTEGER,i7,PETSC_NULL_INTEGER,ierr) > call MatSetOption(A,MAT_NEW_NONZERO_ALLOCATION_ERR,PETSC_FALSE,ierr) > > This is a staggered grid formulation (but derived many years before PETSc had a DMStag class) and so I use regular DMDA?.for those locations that are not involved, I put a 1 on the diagonal. If you want to use a block size of dimension 3, you also need to explicitly MatSetValue (or MatSetValues or MatSetValuesBlocked) zeros for all 3x3 dense blocks. Thanks, Pierre > Randy > >> On May 3, 2022, at 9:37 AM, Pierre Jolivet > wrote: >> >> Thanks for the reproducer. >> My guess is that your AIJ matrix has not a block size of 3. >> A simple call such as: call MatConvert(A,MATBAIJ,MAT_INITIAL_MATRIX,C,ierr) is also failing, while it shouldn?t if your AIJ Mat is truly made of 3x3 dense blocks. >> How did you determine the block size of your Mat? >> Are you allocating 3x3 dense blocks everywhere or are you skipping zero coefficients in your AIJ Mat? >> In the meantime, you can bypass the issue by not setting a block size of 3 on your Mat, or by setting different block size for the column and row distributions, see MatSetBlockSizes(). >> >> Thanks, >> Pierre >> >>> On 3 May 2022, at 5:39 PM, Randall Mackie > wrote: >>> >>> Dear PETSc team: >>> >>> A part of our code that has worked for years and previous versions is now failing with the latest version 3.17.1, on the KSP solve with the following error: >>> >>> [0]PETSC ERROR: --------------------- Error Message -------------------------------------------------------------- >>> [0]PETSC ERROR: Invalid argument >>> [0]PETSC ERROR: Block size 3 is incompatible with the indices: non consecutive indices 153055 153124 >>> [0]PETSC ERROR: See https://petsc.org/release/faq/ for trouble shooting. >>> [0]PETSC ERROR: Petsc Release Version 3.17.1, Apr 28, 2022 >>> [0]PETSC ERROR: ./test on a linux-gfortran-complex-debug named rmackie-VirtualBox by rmackie Tue May 3 08:12:15 2022 >>> [0]PETSC ERROR: Configure options --with-clean=1 --with-scalar-type=complex --with-debugging=1 --with-fortran=1 --download-mpich=../external/mpich-4.0.1.tar.gz >>> [0]PETSC ERROR: #1 ISSetBlockSize() at /home/rmackie/PETSc/petsc-3.17.1/src/vec/is/is/interface/index.c:1898 >>> [0]PETSC ERROR: #2 MatIncreaseOverlap() at /home/rmackie/PETSc/petsc-3.17.1/src/mat/interface/matrix.c:7086 >>> [0]PETSC ERROR: #3 PCSetUp_ASM() at /home/rmackie/PETSc/petsc-3.17.1/src/ksp/pc/impls/asm/asm.c:238 >>> [0]PETSC ERROR: #4 PCSetUp() at /home/rmackie/PETSc/petsc-3.17.1/src/ksp/pc/interface/precon.c:990 >>> [0]PETSC ERROR: #5 KSPSetUp() at /home/rmackie/PETSc/petsc-3.17.1/src/ksp/ksp/interface/itfunc.c:407 >>> [0]PETSC ERROR: #6 KSPSolve_Private() at /home/rmackie/PETSc/petsc-3.17.1/src/ksp/ksp/interface/itfunc.c:843 >>> [0]PETSC ERROR: #7 KSPSolve() at /home/rmackie/PETSc/petsc-3.17.1/src/ksp/ksp/interface/itfunc.c:1078 >>> >>> >>> I have a small test program with binary matrix and right hand side that will show the problem and I can send it as a zip file, please advise what email address to use or where to send it. >>> >>> Thanks, >>> >>> Randy M. >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rlmackie862 at gmail.com Tue May 3 12:33:01 2022 From: rlmackie862 at gmail.com (Randall Mackie) Date: Tue, 3 May 2022 10:33:01 -0700 Subject: [petsc-users] error with version 3.17.1 In-Reply-To: <8A8CEED0-E1A7-4BA1-8584-E07AA856AE69@joliv.et> References: <34013906-D91A-46F5-90A2-7FF48BC7B3FC@joliv.et> <931DFFC4-52B8-4E74-8C55-7EB3ECD8408F@gmail.com> <8A8CEED0-E1A7-4BA1-8584-E07AA856AE69@joliv.et> Message-ID: <13911040-CA99-4F30-806D-EE960C8F8B02@gmail.com> Hi Pierre, I do not discount that I might not be doing something right. However, this code has not changed and worked fine for versions up to and including 3.16. Can you please explain what changed in 3.17 that is causing this? Are you saying that now you have to explicitly set each 3x3 dense block, even if they are not used and that was not the case before? Randy > On May 3, 2022, at 10:09 AM, Pierre Jolivet wrote: > > > >> On 3 May 2022, at 6:54 PM, Randall Mackie > wrote: >> >> Hi Pierre, >> >> Here is how I create the matrix, and how I?ve done it for many years: >> >> ! Create global matrix to hold system of equations resulting from finite discretization >> ! of the Maxwell equations. >> ngrow3=ginfo%nx*ginfo%ny*ginfo%nz*3 >> call MatCreate(comm,A,ierr) >> call MatSetSizes(A,mloc3,mloc3,ngrow3,ngrow3,ierr) >> call MatSetBlockSize(A,i3,ierr) > > I don?t know enough about your discretization stencil, but again, since a simple call to MatConvert(A,MATBAIJ,MAT_INITIAL_MATRIX,C,ierr) fails in your MWE, I doubt that this line is correct. > >> call MatSetType(A,MATAIJ,ierr) >> call MatSeqAIJSetPreallocation(A,i15,PETSC_NULL_INTEGER,ierr) >> call MatMPIAIJSetPreallocation(A,i15,PETSC_NULL_INTEGER,i7,PETSC_NULL_INTEGER,ierr) >> call MatSetOption(A,MAT_NEW_NONZERO_ALLOCATION_ERR,PETSC_FALSE,ierr) >> >> This is a staggered grid formulation (but derived many years before PETSc had a DMStag class) and so I use regular DMDA?.for those locations that are not involved, I put a 1 on the diagonal. > > If you want to use a block size of dimension 3, you also need to explicitly MatSetValue (or MatSetValues or MatSetValuesBlocked) zeros for all 3x3 dense blocks. > > Thanks, > Pierre > >> Randy >> >>> On May 3, 2022, at 9:37 AM, Pierre Jolivet > wrote: >>> >>> Thanks for the reproducer. >>> My guess is that your AIJ matrix has not a block size of 3. >>> A simple call such as: call MatConvert(A,MATBAIJ,MAT_INITIAL_MATRIX,C,ierr) is also failing, while it shouldn?t if your AIJ Mat is truly made of 3x3 dense blocks. >>> How did you determine the block size of your Mat? >>> Are you allocating 3x3 dense blocks everywhere or are you skipping zero coefficients in your AIJ Mat? >>> In the meantime, you can bypass the issue by not setting a block size of 3 on your Mat, or by setting different block size for the column and row distributions, see MatSetBlockSizes(). >>> >>> Thanks, >>> Pierre >>> >>>> On 3 May 2022, at 5:39 PM, Randall Mackie > wrote: >>>> >>>> Dear PETSc team: >>>> >>>> A part of our code that has worked for years and previous versions is now failing with the latest version 3.17.1, on the KSP solve with the following error: >>>> >>>> [0]PETSC ERROR: --------------------- Error Message -------------------------------------------------------------- >>>> [0]PETSC ERROR: Invalid argument >>>> [0]PETSC ERROR: Block size 3 is incompatible with the indices: non consecutive indices 153055 153124 >>>> [0]PETSC ERROR: See https://petsc.org/release/faq/ for trouble shooting. >>>> [0]PETSC ERROR: Petsc Release Version 3.17.1, Apr 28, 2022 >>>> [0]PETSC ERROR: ./test on a linux-gfortran-complex-debug named rmackie-VirtualBox by rmackie Tue May 3 08:12:15 2022 >>>> [0]PETSC ERROR: Configure options --with-clean=1 --with-scalar-type=complex --with-debugging=1 --with-fortran=1 --download-mpich=../external/mpich-4.0.1.tar.gz >>>> [0]PETSC ERROR: #1 ISSetBlockSize() at /home/rmackie/PETSc/petsc-3.17.1/src/vec/is/is/interface/index.c:1898 >>>> [0]PETSC ERROR: #2 MatIncreaseOverlap() at /home/rmackie/PETSc/petsc-3.17.1/src/mat/interface/matrix.c:7086 >>>> [0]PETSC ERROR: #3 PCSetUp_ASM() at /home/rmackie/PETSc/petsc-3.17.1/src/ksp/pc/impls/asm/asm.c:238 >>>> [0]PETSC ERROR: #4 PCSetUp() at /home/rmackie/PETSc/petsc-3.17.1/src/ksp/pc/interface/precon.c:990 >>>> [0]PETSC ERROR: #5 KSPSetUp() at /home/rmackie/PETSc/petsc-3.17.1/src/ksp/ksp/interface/itfunc.c:407 >>>> [0]PETSC ERROR: #6 KSPSolve_Private() at /home/rmackie/PETSc/petsc-3.17.1/src/ksp/ksp/interface/itfunc.c:843 >>>> [0]PETSC ERROR: #7 KSPSolve() at /home/rmackie/PETSc/petsc-3.17.1/src/ksp/ksp/interface/itfunc.c:1078 >>>> >>>> >>>> I have a small test program with binary matrix and right hand side that will show the problem and I can send it as a zip file, please advise what email address to use or where to send it. >>>> >>>> Thanks, >>>> >>>> Randy M. >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From pierre at joliv.et Tue May 3 13:36:02 2022 From: pierre at joliv.et (Pierre Jolivet) Date: Tue, 3 May 2022 20:36:02 +0200 Subject: [petsc-users] error with version 3.17.1 In-Reply-To: <13911040-CA99-4F30-806D-EE960C8F8B02@gmail.com> References: <34013906-D91A-46F5-90A2-7FF48BC7B3FC@joliv.et> <931DFFC4-52B8-4E74-8C55-7EB3ECD8408F@gmail.com> <8A8CEED0-E1A7-4BA1-8584-E07AA856AE69@joliv.et> <13911040-CA99-4F30-806D-EE960C8F8B02@gmail.com> Message-ID: <414B82D5-81AD-4223-8012-837FE2E986B3@joliv.et> > On 3 May 2022, at 7:33 PM, Randall Mackie wrote: > > Hi Pierre, > > I do not discount that I might not be doing something right. > > However, this code has not changed and worked fine for versions up to and including 3.16. What are you implying? It is not because a code works that it is not bugged, that would be too simple. > Can you please explain what changed in 3.17 that is causing this? First, let me tell you that now, MatSetBlockSize() and ISSetBlockSize() work in rather different ways. The latter is not permissive and performs lots of checks, while the former is a little bit looser and does not check the pattern of the matrix. What you are seeing is a side effect of two commits. 1) https://gitlab.com/petsc/petsc/-/commit/30f30de87881d4fa2944681a7d91a8e1d429a77c check for valid block size of IS 2) https://gitlab.com/petsc/petsc/-/commit/9dd3beda480d3fbab2ab495f81e1e366d04021c6 propagation of Mat bs to overlapping IS PCASM uses MatIncreaseOverlap(). You use this PC with a Mat that you tell PETSc has a bs of 3. We don?t check this for you, but then when setting the bs of the IS, we do check this, and this errors out. > Are you saying that now you have to explicitly set each 3x3 dense block, even if they are not used and that was not the case before? That was always the case before, you may have misinterpreted the meaning of a Mat block size? Again, with version 3.XY (XY < 17), try MatConvert(A,MATBAIJ,MAT_INITIAL_MATRIX,C,ierr) and you will still get an error. You have two options: either do not set a block size or set a block size but make sure that the pattern of the Mat is indeed only made of blocks of size bs x bs. Thanks, Pierre > Randy > >> On May 3, 2022, at 10:09 AM, Pierre Jolivet > wrote: >> >> >> >>> On 3 May 2022, at 6:54 PM, Randall Mackie > wrote: >>> >>> Hi Pierre, >>> >>> Here is how I create the matrix, and how I?ve done it for many years: >>> >>> ! Create global matrix to hold system of equations resulting from finite discretization >>> ! of the Maxwell equations. >>> ngrow3=ginfo%nx*ginfo%ny*ginfo%nz*3 >>> call MatCreate(comm,A,ierr) >>> call MatSetSizes(A,mloc3,mloc3,ngrow3,ngrow3,ierr) >>> call MatSetBlockSize(A,i3,ierr) >> >> I don?t know enough about your discretization stencil, but again, since a simple call to MatConvert(A,MATBAIJ,MAT_INITIAL_MATRIX,C,ierr) fails in your MWE, I doubt that this line is correct. >> >>> call MatSetType(A,MATAIJ,ierr) >>> call MatSeqAIJSetPreallocation(A,i15,PETSC_NULL_INTEGER,ierr) >>> call MatMPIAIJSetPreallocation(A,i15,PETSC_NULL_INTEGER,i7,PETSC_NULL_INTEGER,ierr) >>> call MatSetOption(A,MAT_NEW_NONZERO_ALLOCATION_ERR,PETSC_FALSE,ierr) >>> >>> This is a staggered grid formulation (but derived many years before PETSc had a DMStag class) and so I use regular DMDA?.for those locations that are not involved, I put a 1 on the diagonal. >> >> If you want to use a block size of dimension 3, you also need to explicitly MatSetValue (or MatSetValues or MatSetValuesBlocked) zeros for all 3x3 dense blocks. >> >> Thanks, >> Pierre >> >>> Randy >>> >>>> On May 3, 2022, at 9:37 AM, Pierre Jolivet > wrote: >>>> >>>> Thanks for the reproducer. >>>> My guess is that your AIJ matrix has not a block size of 3. >>>> A simple call such as: call MatConvert(A,MATBAIJ,MAT_INITIAL_MATRIX,C,ierr) is also failing, while it shouldn?t if your AIJ Mat is truly made of 3x3 dense blocks. >>>> How did you determine the block size of your Mat? >>>> Are you allocating 3x3 dense blocks everywhere or are you skipping zero coefficients in your AIJ Mat? >>>> In the meantime, you can bypass the issue by not setting a block size of 3 on your Mat, or by setting different block size for the column and row distributions, see MatSetBlockSizes(). >>>> >>>> Thanks, >>>> Pierre >>>> >>>>> On 3 May 2022, at 5:39 PM, Randall Mackie > wrote: >>>>> >>>>> Dear PETSc team: >>>>> >>>>> A part of our code that has worked for years and previous versions is now failing with the latest version 3.17.1, on the KSP solve with the following error: >>>>> >>>>> [0]PETSC ERROR: --------------------- Error Message -------------------------------------------------------------- >>>>> [0]PETSC ERROR: Invalid argument >>>>> [0]PETSC ERROR: Block size 3 is incompatible with the indices: non consecutive indices 153055 153124 >>>>> [0]PETSC ERROR: See https://petsc.org/release/faq/ for trouble shooting. >>>>> [0]PETSC ERROR: Petsc Release Version 3.17.1, Apr 28, 2022 >>>>> [0]PETSC ERROR: ./test on a linux-gfortran-complex-debug named rmackie-VirtualBox by rmackie Tue May 3 08:12:15 2022 >>>>> [0]PETSC ERROR: Configure options --with-clean=1 --with-scalar-type=complex --with-debugging=1 --with-fortran=1 --download-mpich=../external/mpich-4.0.1.tar.gz >>>>> [0]PETSC ERROR: #1 ISSetBlockSize() at /home/rmackie/PETSc/petsc-3.17.1/src/vec/is/is/interface/index.c:1898 >>>>> [0]PETSC ERROR: #2 MatIncreaseOverlap() at /home/rmackie/PETSc/petsc-3.17.1/src/mat/interface/matrix.c:7086 >>>>> [0]PETSC ERROR: #3 PCSetUp_ASM() at /home/rmackie/PETSc/petsc-3.17.1/src/ksp/pc/impls/asm/asm.c:238 >>>>> [0]PETSC ERROR: #4 PCSetUp() at /home/rmackie/PETSc/petsc-3.17.1/src/ksp/pc/interface/precon.c:990 >>>>> [0]PETSC ERROR: #5 KSPSetUp() at /home/rmackie/PETSc/petsc-3.17.1/src/ksp/ksp/interface/itfunc.c:407 >>>>> [0]PETSC ERROR: #6 KSPSolve_Private() at /home/rmackie/PETSc/petsc-3.17.1/src/ksp/ksp/interface/itfunc.c:843 >>>>> [0]PETSC ERROR: #7 KSPSolve() at /home/rmackie/PETSc/petsc-3.17.1/src/ksp/ksp/interface/itfunc.c:1078 >>>>> >>>>> >>>>> I have a small test program with binary matrix and right hand side that will show the problem and I can send it as a zip file, please advise what email address to use or where to send it. >>>>> >>>>> Thanks, >>>>> >>>>> Randy M. >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at petsc.dev Tue May 3 14:27:07 2022 From: bsmith at petsc.dev (Barry Smith) Date: Tue, 3 May 2022 15:27:07 -0400 Subject: [petsc-users] Preconditioning Diagnostics In-Reply-To: References: Message-ID: <788632DC-1973-43EC-B5CB-97A72B9B62CA@petsc.dev> A difficult question with no easy answers. First, do you have a restart system so you can save your state just before your "bad behavior" and run experiments easily at the bad point? You could try to use SLEPc to compute the first few eigenmodes (presumably associated with excessively small eigenvalues) and visualize them? You could restart with and without preconditioning to see how badly the conditioning becomes for both the system and the preconditioned system to help see if the problem comes from just the preconditioner starting to behavior poorly or because the operator starts to behave poorly. There should be a way to allow doing this directly from within the KSPSolve trivially with appropriate monitors but I suspect that does not exist because calling SLEPc trivially from PETSc is a nightmare because of the dependency diamond. Barry > On May 3, 2022, at 11:58 AM, Alfredo J Duarte Gomez wrote: > > Good morning PETSC team, > > I have a bit of an open question on diagnosing preconditioner performance. > > For a bit of background, I am using the TS object in combination with the matrix-free snes, and a custom user defined preconditioner (PCSHELL). Everything is implemented with the help of a DMDA. Smallest problem size that I can get away with is a grid with 2.1 million points with 4 fields each, for a total of 8.4 million equations. > > The preconditioner works very well overall, but at some stages of the solution it performs poorly, evidenced by significant increases in the number of GMRES iterations and the maximum/minimum eigenvalue computed using KSPComputeExtremeSingularValues(). > > I am trying to understand the locations where the preconditioner is not working well, so for example, is there any way to map the maximum eigenvalue to a particular location/field in the DMDA. Alternatively, are there any other ways of diagnosing where the preconditioner is not doing a good job? > > GMRES iterations and the max/min eigenvalue provide a good overall picture, but I am struggling to get preconditioner metrics that are specific to a location and field. > > So far I have taken a close look at fields such as the residual, and Newton updates, but it is difficult to tell how to assess these in combination with the preconditioner. > > I appreciate any suggestions. > > Thank you and have a good day. > > -Alfredo > > > > -- > Alfredo Duarte > Graduate Research Assistant > The University of Texas at Austin -------------- next part -------------- An HTML attachment was scrubbed... URL: From mfadams at lbl.gov Tue May 3 14:37:39 2022 From: mfadams at lbl.gov (Mark Adams) Date: Tue, 3 May 2022 15:37:39 -0400 Subject: [petsc-users] error with version 3.17.1 In-Reply-To: <414B82D5-81AD-4223-8012-837FE2E986B3@joliv.et> References: <34013906-D91A-46F5-90A2-7FF48BC7B3FC@joliv.et> <931DFFC4-52B8-4E74-8C55-7EB3ECD8408F@gmail.com> <8A8CEED0-E1A7-4BA1-8584-E07AA856AE69@joliv.et> <13911040-CA99-4F30-806D-EE960C8F8B02@gmail.com> <414B82D5-81AD-4223-8012-837FE2E986B3@joliv.et> Message-ID: > > > Are you saying that now you have to explicitly set each 3x3 dense block, > even if they are not used and that was not the case before? > > > That was always the case before, you may have misinterpreted the meaning > of a Mat block size? > Actually block size is really more of a hint in that you don't have to set 3x3 dense blocks and thus any AIJ matrix can have any block size essentially. At least that is my understanding. There is a CI test that has sparse blocks and I ran into this issue with GAMG optimizations. (I had to add complicated code that Pierre actually found a bug in.) I don't know what changed in PETSc to make ASM fail for you, but if MatConvert and ASM fail then PETSc is broken and always has been. I did not follow this whole thread, but Randall could you change your code to add dense blocks or not use block size? Sorry, but I just don't think we should support this (Pierre seems to think that we do not) and we should "depreciate" this. This needs to be discussed of course. Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Tue May 3 15:43:29 2022 From: knepley at gmail.com (Matthew Knepley) Date: Tue, 3 May 2022 16:43:29 -0400 Subject: [petsc-users] Preconditioning Diagnostics In-Reply-To: <788632DC-1973-43EC-B5CB-97A72B9B62CA@petsc.dev> References: <788632DC-1973-43EC-B5CB-97A72B9B62CA@petsc.dev> Message-ID: On Tue, May 3, 2022 at 3:28 PM Barry Smith wrote: > > A difficult question with no easy answers. > > First, do you have a restart system so you can save your state just > before your "bad behavior" and run experiments easily at the bad point? > > You could try to use SLEPc to compute the first few eigenmodes > (presumably associated with excessively small eigenvalues) and visualize > them? You could restart with and without preconditioning to see how badly > the conditioning becomes for both the system and the preconditioned system > to help see if the problem comes from just the preconditioner starting to > behavior poorly or because the operator starts to behave poorly. > > There should be a way to allow doing this directly from within the > KSPSolve trivially with appropriate monitors but I suspect that does not > exist because calling SLEPc trivially from PETSc is a nightmare because of > the dependency diamond. > A quick and dirty thing is the plot the residual when your system does not converge. You can get an idea where the algebraic error is largest. There is already a -ksp_monitor_range, and Barry had code to cut out the region of high residual in a DMDA woth a halo, solve that, and project it back in, but I cannot remember where it is. Barry? Thanks, Matt > Barry > > > On May 3, 2022, at 11:58 AM, Alfredo J Duarte Gomez > wrote: > > Good morning PETSC team, > > I have a bit of an open question on diagnosing preconditioner performance. > > For a bit of background, I am using the TS object in combination with the > matrix-free snes, and a custom user defined preconditioner (PCSHELL). > Everything is implemented with the help of a DMDA. Smallest problem size > that I can get away with is a grid with 2.1 million points with 4 fields > each, for a total of 8.4 million equations. > > The preconditioner works very well overall, but at some stages of the > solution it performs poorly, evidenced by significant increases in the > number of GMRES iterations and the maximum/minimum eigenvalue computed > using KSPComputeExtremeSingularValues(). > > I am trying to understand the locations where the preconditioner is not > working well, so for example, is there any way to map the maximum > eigenvalue to a particular location/field in the DMDA. Alternatively, are > there any other ways of diagnosing where the preconditioner is not doing a > good job? > > GMRES iterations and the max/min eigenvalue provide a good overall > picture, but I am struggling to get preconditioner metrics that are > specific to a location and field. > > So far I have taken a close look at fields such as the residual, and > Newton updates, but it is difficult to tell how to assess these in > combination with the preconditioner. > > I appreciate any suggestions. > > Thank you and have a good day. > > -Alfredo > > > > -- > Alfredo Duarte > Graduate Research Assistant > The University of Texas at Austin > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From rlmackie862 at gmail.com Tue May 3 16:55:48 2022 From: rlmackie862 at gmail.com (Randall Mackie) Date: Tue, 3 May 2022 14:55:48 -0700 Subject: [petsc-users] error with version 3.17.1 In-Reply-To: References: <34013906-D91A-46F5-90A2-7FF48BC7B3FC@joliv.et> <931DFFC4-52B8-4E74-8C55-7EB3ECD8408F@gmail.com> <8A8CEED0-E1A7-4BA1-8584-E07AA856AE69@joliv.et> <13911040-CA99-4F30-806D-EE960C8F8B02@gmail.com> <414B82D5-81AD-4223-8012-837FE2E986B3@joliv.et> Message-ID: <3D83D6C0-D5CD-4EFB-8117-E510549CD2AA@gmail.com> > On May 3, 2022, at 12:37 PM, Mark Adams wrote: > > >> Are you saying that now you have to explicitly set each 3x3 dense block, even if they are not used and that was not the case before? > > That was always the case before, you may have misinterpreted the meaning of a Mat block size? > > Actually block size is really more of a hint in that you don't have to set 3x3 dense blocks and thus any AIJ matrix can have any block size essentially. > At least that is my understanding. > There is a CI test that has sparse blocks and I ran into this issue with GAMG optimizations. > (I had to add complicated code that Pierre actually found a bug in.) > > I don't know what changed in PETSc to make ASM fail for you, but if MatConvert and ASM fail then PETSc is broken and always has been. > > I did not follow this whole thread, but Randall could you change your code to add dense blocks or not use block size? > Sorry, but I just don't think we should support this (Pierre seems to think that we do not) and we should "depreciate" this. > This needs to be discussed of course. > > Mark Hi Mark and Pierre, You are correct that it is not necessary to use the block size. I had done that many many years ago because for some reason I thought it was necessary when creating a matrix for a 3D grid with more than 1 degree of freedom per node. But as long as the matrix entries are set correctly, block size doesn?t really matter. I think part of the issue in my situation is that there are parts of the matrix where not all 3x3 dense blocks are set due to representing a staggered grid system using a 3D DMDA (but like I say this was done many years before DMStag was developed). Thanks for the help and the clarifications, Randy -------------- next part -------------- An HTML attachment was scrubbed... URL: From mfadams at lbl.gov Tue May 3 20:10:46 2022 From: mfadams at lbl.gov (Mark Adams) Date: Tue, 3 May 2022 21:10:46 -0400 Subject: [petsc-users] error with version 3.17.1 In-Reply-To: <3D83D6C0-D5CD-4EFB-8117-E510549CD2AA@gmail.com> References: <34013906-D91A-46F5-90A2-7FF48BC7B3FC@joliv.et> <931DFFC4-52B8-4E74-8C55-7EB3ECD8408F@gmail.com> <8A8CEED0-E1A7-4BA1-8584-E07AA856AE69@joliv.et> <13911040-CA99-4F30-806D-EE960C8F8B02@gmail.com> <414B82D5-81AD-4223-8012-837FE2E986B3@joliv.et> <3D83D6C0-D5CD-4EFB-8117-E510549CD2AA@gmail.com> Message-ID: I'm glad that was easy to deal with. We need to clarify this. Thanks, Mark On Tue, May 3, 2022 at 5:55 PM Randall Mackie wrote: > > On May 3, 2022, at 12:37 PM, Mark Adams wrote: > > >> Are you saying that now you have to explicitly set each 3x3 dense block, >> even if they are not used and that was not the case before? >> >> >> That was always the case before, you may have misinterpreted the meaning >> of a Mat block size? >> > > Actually block size is really more of a hint in that you don't have to set > 3x3 dense blocks and thus any AIJ matrix can have any block size > essentially. > At least that is my understanding. > There is a CI test that has sparse blocks and I ran into this issue with > GAMG optimizations. > (I had to add complicated code that Pierre actually found a bug in.) > > I don't know what changed in PETSc to make ASM fail for you, but if > MatConvert and ASM fail then PETSc is broken and always has been. > > I did not follow this whole thread, but Randall could you change your code > to add dense blocks or not use block size? > Sorry, but I just don't think we should support this (Pierre seems to > think that we do not) and we should "depreciate" this. > This needs to be discussed of course. > > Mark > > > Hi Mark and Pierre, > > You are correct that it is not necessary to use the block size. I had done > that many many years ago because for some reason I thought it was necessary > when creating a matrix for a 3D grid with more than 1 degree of freedom per > node. > > But as long as the matrix entries are set correctly, block size doesn?t > really matter. > > I think part of the issue in my situation is that there are parts of the > matrix where not all 3x3 dense blocks are set due to representing a > staggered grid system using a 3D DMDA (but like I say this was done many > years before DMStag was developed). > > > Thanks for the help and the clarifications, > > Randy > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jed at jedbrown.org Tue May 3 22:33:09 2022 From: jed at jedbrown.org (Jed Brown) Date: Tue, 03 May 2022 21:33:09 -0600 Subject: [petsc-users] Quasi newton In-Reply-To: References: <083AA6CC-6E69-4AFB-8330-DABADC3F860B@msu.edu> <32b34537-9948-1614-cb66-fa5b12697552@onera.fr> <4EB6F5A2-8AEB-4856-BB34-D3F66EFA7677@msu.edu> Message-ID: <87o80evuoa.fsf@jedbrown.org> If you have a problem that's close to a variational minimization, you can start with this -snes_type qn -snes_qn_scale_type jacobian -snes_linesearch_type cp It's L-BFGS and Jacobian scaling with a Powell update and critical point line search. Such methods are mentioned in some nonlinear finite element books, like Wriggers or Bathe. What sort of problems are you solving? "Tang, Qi" writes: > This is very helpful. JFNK + Qn in that paper sounds really promising. > > Jed, is there a flag I can quickly try to recover some performance related to that paper (snes ex48.c)? Thanks. > > Qi > > > > > On May 3, 2022, at 7:27 AM, Matthew Knepley wrote: > > ? > On Tue, May 3, 2022 at 9:08 AM Tang, Qi > wrote: > Pierre and Matt, > > Thanks a lot for the suggestion. It looks like lag Jacobian is exactly what I need. We will try that. > > I always thought ngmres is a fancy version of Anderson. Is there any reference or example related to what you said in which one actually implemented an approximated Jacobian through ngmres? This sounds very interesting. > > Jed and Peter do it here: > > @inproceedings{brown2013quasinewton, > author = {Jed Brown and Peter Brune}, > title = {Low-rank quasi-{N}ewton updates for robust {J}acobian lagging in {N}ewton-type methods}, > year = {2013}, > booktitle = {International Conference on Mathematics and Computational Methods Applied to Nuclear Science and Engineering}, > pages = {2554--2565}, > petsc_uses={KSP}, > } > Thanks, > > Matt > > Qi > > > On May 3, 2022, at 4:51 AM, Matthew Knepley > wrote: > > ? > On Tue, May 3, 2022 at 2:58 AM Pierre Seize > wrote: > Hi, > > If I may, is this what you want ? > > https://petsc.org/main/docs/manualpages/SNES/SNESSetLagJacobian.html > > Yes, this is a good suggestion. > > Also, you could implement an approximation to the Jacobian. > > You could then improve it at each iteration using a secant update. This is what the Generalized Broyden methods do. We call them NGMRES. > > Thanks, > > Matt > > Pierre > > On 03/05/2022 06:21, Tang, Qi wrote: >> Hi, >> Our code uses FDcoloring to compute Jacobian. The log file indicates most of time is spent in evaluating residual (2600 times in one Newton solve) while it only needs 3 nonlinear iterations and 6 total linear iterations thanks to the fieldsplit pc. >> >> As a temporary solution, is it possible to evaluate Jacobian only once in one Newton solve? This should work well based on my other experience if pc is very efficient. But I cannot find such a flag. >> >> Is there any other solution, other than implementing the analytical Jacobian? >> >> Thanks, >> Qi > > > -- > What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. > -- Norbert Wiener > > https://www.cse.buffalo.edu/~knepley/ > > > -- > What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. > -- Norbert Wiener > > https://www.cse.buffalo.edu/~knepley/ From quentin.chevalier at polytechnique.edu Thu May 5 03:34:15 2022 From: quentin.chevalier at polytechnique.edu (Quentin Chevalier) Date: Thu, 5 May 2022 10:34:15 +0200 Subject: [petsc-users] Mysterious error code 77 Message-ID: Hello all and thanks for your great work in bringing this very helpful package to the community ! That said, I wouldn't need this mailing list if everything was running smoothly. I have a rather involved eigenvalue problem that I've been working on that's been throwing a mysterious error : > petsc4py.PETSc.Error: error code 77 > [1] EPSSolve() at /usr/local/slepc/src/eps/interface/epssolve.c:149 > [1] EPSSolve_KrylovSchur_Default() at > /usr/local/slepc/src/eps/impls/krylov/krylovschur/krylovschur.c:289 > [1] EPSGetStartVector() at > /usr/local/slepc/src/eps/interface/epssolve.c:824 > [1] Petsc has generated inconsistent data > [1] Initial vector is zero or belongs to the deflation space This problem occurs in parallel with two processors, using the petsc4py library using the dolfinx/dolfinx docker container . I have PETSc version 3.16.0, in complex mode, python 3, and I'm running all of that on a OpenSUSE Leap 15.2 machine (but I think the docker container has a Ubuntu OS). I wrote a minimal working example below, but I'm afraid the process for building the matrices is involved, so I decided to directly share the matrices instead : https://seminaris.polytechnique.fr/share/s/ryJ6L2nR4ketDwP They are in binary format, but inside the container I hope someone could reproduce my issue. A word on the structure and intent behind these matrices : - QE is a diagonal rectangular real matrix. Think of it as some sort of preconditioner - L is the least dense of them all, the only one that is complex, and in order to avoid inverting it I'm using two KSPs to compute solve problems on the fly - Mf is a diagonal square real matrix, its on the right-hand side of the Generalised Hermitian Eigenvalue problem (I'm solving QE^H*L^-1H*L^-1*QE*x=lambda*Mf*x Full MWE is below : from petsc4py import PETSc as pet from slepc4py import SLEPc as slp from mpi4py.MPI import COMM_WORLD # Global sizes m_local=COMM_WORLD.rank*(490363-489780)+489780 n_local=COMM_WORLD.rank*(452259-451743)+451743 m=980143 n=904002 QE=pet.Mat().createAIJ([[m_local,m],[n_local,n]]) L=pet.Mat().createAIJ([[m_local,m],[m_local,m]]) Mf=pet.Mat().createAIJ([[n_local,n],[n_local,n]]) viewerQE = pet.Viewer().createMPIIO("QE.dat", 'r', COMM_WORLD) QE.load(viewerQE) viewerL = pet.Viewer().createMPIIO("L.dat", 'r', COMM_WORLD) L.load(viewerL) viewerMf = pet.Viewer().createMPIIO("Mf.dat", 'r', COMM_WORLD) Mf.load(viewerMf) QE.assemble() L.assemble() KSPs = [] # Useful solvers (here to put options for computing a smart R) for Mat in [L,L.hermitianTranspose()]: KSP = pet.KSP().create() KSP.setOperators(Mat) KSP.setFromOptions() KSPs.append(KSP) class LHS_class: def mult(self,A,x,y): w=pet.Vec().createMPI([m_local,m],comm=COMM_WORLD) z=pet.Vec().createMPI([m_local,m],comm=COMM_WORLD) QE.mult(x,w) KSPs[0].solve(w,z) KSPs[1].solve(z,w) QE.multTranspose(w,y) # Matrix free operator LHS=pet.Mat() LHS.create(comm=COMM_WORLD) LHS.setSizes([[n_local,n],[n_local,n]]) LHS.setType(pet.Mat.Type.PYTHON) LHS.setPythonContext(LHS_class()) LHS.setUp() # Eigensolver EPS = slp.EPS(); EPS.create() EPS.setOperators(LHS,Mf) # Solve QE^T*L^-1H*L^-1*QE*x=lambda*Mf*x EPS.setProblemType(slp.EPS.ProblemType.GHEP) # Specify that A is hermitian (by construction), and B is semi-positive EPS.setWhichEigenpairs(EPS.Which.LARGEST_MAGNITUDE) # Find largest eigenvalues EPS.setFromOptions() EPS.solve() Quentin CHEVALIER ? IA parcours recherche LadHyX - Ecole polytechnique __________ -------------- next part -------------- An HTML attachment was scrubbed... URL: From quentin.chevalier at polytechnique.edu Thu May 5 03:46:13 2022 From: quentin.chevalier at polytechnique.edu (Quentin Chevalier) Date: Thu, 5 May 2022 10:46:13 +0200 Subject: [petsc-users] Mysterious error code 77 In-Reply-To: References: Message-ID: Just a quick amend on the previous statement ; the problem arises in sequential and parallel. The MWE as is is provided for the parallel case, but imposing m_local=m makes it go sequential. Cheers, Quentin CHEVALIER ? IA parcours recherche LadHyX - Ecole polytechnique __________ On Thu, 5 May 2022 at 10:34, Quentin Chevalier wrote: > > Hello all and thanks for your great work in bringing this very helpful package to the community ! > > That said, I wouldn't need this mailing list if everything was running smoothly. I have a rather involved eigenvalue problem that I've been working on that's been throwing a mysterious error : >> >> petsc4py.PETSc.Error: error code 77 >> >> [1] EPSSolve() at /usr/local/slepc/src/eps/interface/epssolve.c:149 >> [1] EPSSolve_KrylovSchur_Default() at /usr/local/slepc/src/eps/impls/krylov/krylovschur/krylovschur.c:289 >> [1] EPSGetStartVector() at /usr/local/slepc/src/eps/interface/epssolve.c:824 >> [1] Petsc has generated inconsistent data >> [1] Initial vector is zero or belongs to the deflation space > > > This problem occurs in parallel with two processors, using the petsc4py library using the dolfinx/dolfinx docker container. I have PETSc version 3.16.0, in complex mode, python 3, and I'm running all of that on a OpenSUSE Leap 15.2 machine (but I think the docker container has a Ubuntu OS). > > I wrote a minimal working example below, but I'm afraid the process for building the matrices is involved, so I decided to directly share the matrices instead : https://seminaris.polytechnique.fr/share/s/ryJ6L2nR4ketDwP > > They are in binary format, but inside the container I hope someone could reproduce my issue. A word on the structure and intent behind these matrices : > > QE is a diagonal rectangular real matrix. Think of it as some sort of preconditioner > L is the least dense of them all, the only one that is complex, and in order to avoid inverting it I'm using two KSPs to compute solve problems on the fly > Mf is a diagonal square real matrix, its on the right-hand side of the Generalised Hermitian Eigenvalue problem (I'm solving QE^H*L^-1H*L^-1*QE*x=lambda*Mf*x > > Full MWE is below : > > from petsc4py import PETSc as pet > from slepc4py import SLEPc as slp > from mpi4py.MPI import COMM_WORLD > > # Global sizes > m_local=COMM_WORLD.rank*(490363-489780)+489780 > n_local=COMM_WORLD.rank*(452259-451743)+451743 > m=980143 > n=904002 > > QE=pet.Mat().createAIJ([[m_local,m],[n_local,n]]) > L=pet.Mat().createAIJ([[m_local,m],[m_local,m]]) > Mf=pet.Mat().createAIJ([[n_local,n],[n_local,n]]) > > viewerQE = pet.Viewer().createMPIIO("QE.dat", 'r', COMM_WORLD) > QE.load(viewerQE) > viewerL = pet.Viewer().createMPIIO("L.dat", 'r', COMM_WORLD) > L.load(viewerL) > viewerMf = pet.Viewer().createMPIIO("Mf.dat", 'r', COMM_WORLD) > Mf.load(viewerMf) > > QE.assemble() > L.assemble() > > KSPs = [] > # Useful solvers (here to put options for computing a smart R) > for Mat in [L,L.hermitianTranspose()]: > KSP = pet.KSP().create() > KSP.setOperators(Mat) > KSP.setFromOptions() > KSPs.append(KSP) > class LHS_class: > def mult(self,A,x,y): > w=pet.Vec().createMPI([m_local,m],comm=COMM_WORLD) > z=pet.Vec().createMPI([m_local,m],comm=COMM_WORLD) > QE.mult(x,w) > KSPs[0].solve(w,z) > KSPs[1].solve(z,w) > QE.multTranspose(w,y) > > # Matrix free operator > LHS=pet.Mat() > LHS.create(comm=COMM_WORLD) > LHS.setSizes([[n_local,n],[n_local,n]]) > LHS.setType(pet.Mat.Type.PYTHON) > LHS.setPythonContext(LHS_class()) > LHS.setUp() > > # Eigensolver > EPS = slp.EPS(); EPS.create() > EPS.setOperators(LHS,Mf) # Solve QE^T*L^-1H*L^-1*QE*x=lambda*Mf*x > EPS.setProblemType(slp.EPS.ProblemType.GHEP) # Specify that A is hermitian (by construction), and B is semi-positive > EPS.setWhichEigenpairs(EPS.Which.LARGEST_MAGNITUDE) # Find largest eigenvalues > EPS.setFromOptions() > EPS.solve() > > Quentin CHEVALIER ? IA parcours recherche > > LadHyX - Ecole polytechnique > > __________ From jroman at dsic.upv.es Thu May 5 05:05:08 2022 From: jroman at dsic.upv.es (Jose E. Roman) Date: Thu, 5 May 2022 12:05:08 +0200 Subject: [petsc-users] Mysterious error code 77 In-Reply-To: References: Message-ID: Your operator is not well formed. If you do this: x, y = LHS.createVecs() x.set(1) LHS.mult(x,y) y.view() you will see that the output is all zeros. That is why SLEPc complains that "Initial vector is zero or belongs to the deflation space". Jose > El 5 may 2022, a las 10:46, Quentin Chevalier escribi?: > > Just a quick amend on the previous statement ; the problem arises in > sequential and parallel. The MWE as is is provided for the parallel > case, but imposing m_local=m makes it go sequential. > > Cheers, > > Quentin CHEVALIER ? IA parcours recherche > LadHyX - Ecole polytechnique > __________ > > > > On Thu, 5 May 2022 at 10:34, Quentin Chevalier > wrote: >> >> Hello all and thanks for your great work in bringing this very helpful package to the community ! >> >> That said, I wouldn't need this mailing list if everything was running smoothly. I have a rather involved eigenvalue problem that I've been working on that's been throwing a mysterious error : >>> >>> petsc4py.PETSc.Error: error code 77 >>> >>> [1] EPSSolve() at /usr/local/slepc/src/eps/interface/epssolve.c:149 >>> [1] EPSSolve_KrylovSchur_Default() at /usr/local/slepc/src/eps/impls/krylov/krylovschur/krylovschur.c:289 >>> [1] EPSGetStartVector() at /usr/local/slepc/src/eps/interface/epssolve.c:824 >>> [1] Petsc has generated inconsistent data >>> [1] Initial vector is zero or belongs to the deflation space >> >> >> This problem occurs in parallel with two processors, using the petsc4py library using the dolfinx/dolfinx docker container. I have PETSc version 3.16.0, in complex mode, python 3, and I'm running all of that on a OpenSUSE Leap 15.2 machine (but I think the docker container has a Ubuntu OS). >> >> I wrote a minimal working example below, but I'm afraid the process for building the matrices is involved, so I decided to directly share the matrices instead : https://seminaris.polytechnique.fr/share/s/ryJ6L2nR4ketDwP >> >> They are in binary format, but inside the container I hope someone could reproduce my issue. A word on the structure and intent behind these matrices : >> >> QE is a diagonal rectangular real matrix. Think of it as some sort of preconditioner >> L is the least dense of them all, the only one that is complex, and in order to avoid inverting it I'm using two KSPs to compute solve problems on the fly >> Mf is a diagonal square real matrix, its on the right-hand side of the Generalised Hermitian Eigenvalue problem (I'm solving QE^H*L^-1H*L^-1*QE*x=lambda*Mf*x >> >> Full MWE is below : >> >> from petsc4py import PETSc as pet >> from slepc4py import SLEPc as slp >> from mpi4py.MPI import COMM_WORLD >> >> # Global sizes >> m_local=COMM_WORLD.rank*(490363-489780)+489780 >> n_local=COMM_WORLD.rank*(452259-451743)+451743 >> m=980143 >> n=904002 >> >> QE=pet.Mat().createAIJ([[m_local,m],[n_local,n]]) >> L=pet.Mat().createAIJ([[m_local,m],[m_local,m]]) >> Mf=pet.Mat().createAIJ([[n_local,n],[n_local,n]]) >> >> viewerQE = pet.Viewer().createMPIIO("QE.dat", 'r', COMM_WORLD) >> QE.load(viewerQE) >> viewerL = pet.Viewer().createMPIIO("L.dat", 'r', COMM_WORLD) >> L.load(viewerL) >> viewerMf = pet.Viewer().createMPIIO("Mf.dat", 'r', COMM_WORLD) >> Mf.load(viewerMf) >> >> QE.assemble() >> L.assemble() >> >> KSPs = [] >> # Useful solvers (here to put options for computing a smart R) >> for Mat in [L,L.hermitianTranspose()]: >> KSP = pet.KSP().create() >> KSP.setOperators(Mat) >> KSP.setFromOptions() >> KSPs.append(KSP) >> class LHS_class: >> def mult(self,A,x,y): >> w=pet.Vec().createMPI([m_local,m],comm=COMM_WORLD) >> z=pet.Vec().createMPI([m_local,m],comm=COMM_WORLD) >> QE.mult(x,w) >> KSPs[0].solve(w,z) >> KSPs[1].solve(z,w) >> QE.multTranspose(w,y) >> >> # Matrix free operator >> LHS=pet.Mat() >> LHS.create(comm=COMM_WORLD) >> LHS.setSizes([[n_local,n],[n_local,n]]) >> LHS.setType(pet.Mat.Type.PYTHON) >> LHS.setPythonContext(LHS_class()) >> LHS.setUp() >> >> # Eigensolver >> EPS = slp.EPS(); EPS.create() >> EPS.setOperators(LHS,Mf) # Solve QE^T*L^-1H*L^-1*QE*x=lambda*Mf*x >> EPS.setProblemType(slp.EPS.ProblemType.GHEP) # Specify that A is hermitian (by construction), and B is semi-positive >> EPS.setWhichEigenpairs(EPS.Which.LARGEST_MAGNITUDE) # Find largest eigenvalues >> EPS.setFromOptions() >> EPS.solve() >> >> Quentin CHEVALIER ? IA parcours recherche >> >> LadHyX - Ecole polytechnique >> >> __________ From jroman at dsic.upv.es Fri May 6 07:47:48 2022 From: jroman at dsic.upv.es (Jose E. Roman) Date: Fri, 6 May 2022 14:47:48 +0200 Subject: [petsc-users] Mysterious error code 77 In-Reply-To: References: Message-ID: <3AFCE82E-EC31-4DBB-A80E-D5499D2076EB@dsic.upv.es> Please respond to the list. Why would you need different matrices for sequential and parallel? PETSc takes care of loading matrices from binary files in parallel codes. If I use the 'seq' matrices for both sequential and parallel runs it works. I get the eigenvalue 128662.745858 Take into account that in parallel you need to use a parallel linear solver, e.g., configure PETSc with MUMPS. See the FAQ #10 https://slepc.upv.es/documentation/faq.htm#faq10 Jose > El 5 may 2022, a las 14:57, Quentin Chevalier escribi?: > > Thank you for your answer, Jose. > > It would appear my problem is slightly more complicated than it appears. With slight modification of the original MWE to account for functioning in serial or parallel, and computing the matrices in either sequential or parallel (apologies, it's a large file), and including a slightly modified version of your test case, I obtain two different results in serial and parallel (commands python3 MWE.py and mpirun -n 2 python3 MWE.py). > > Serial gives my a finite norm (around 28) and parallel gives me two norms of 0 and a code 77. > > This is still a problem as I would really like my code to be parallel. I saved my matrices using : > viewerQE = pet.Viewer().createMPIIO("QE.dat", 'w', COMM_WORLD) > QE.view(viewerQE) > viewerL = pet.Viewer().createMPIIO("L.dat", 'w', COMM_WORLD) > L.view(viewerL) > viewerMq = pet.Viewer().createMPIIO("Mq.dat", 'w', COMM_WORLD) > Mq.view(viewerMq) > viewerMf = pet.Viewer().createMPIIO("Mf.dat", 'w', COMM_WORLD) > Mf.view(viewerMf) > > New MWE (also attached for convenience) : > from petsc4py import PETSc as pet > from slepc4py import SLEPc as slp > from mpi4py.MPI import COMM_WORLD > > dir="./mats/" > > # Global sizes > m=980143; m_local=m > n=904002; n_local=n > if COMM_WORLD.size>1: > m_local=COMM_WORLD.rank*(490363-489780)+489780 > n_local=COMM_WORLD.rank*(452259-451743)+451743 > dir+="par/" > else: > m_local=m > n_local=n > dir+="seq/" > > QE=pet.Mat().createAIJ([[m_local,m],[n_local,n]]) > L=pet.Mat().createAIJ([[m_local,m],[m_local,m]]) > Mf=pet.Mat().createAIJ([[n_local,n],[n_local,n]]) > > viewerQE = pet.Viewer().createMPIIO(dir+"QE.dat", 'r', COMM_WORLD) > QE.load(viewerQE) > viewerL = pet.Viewer().createMPIIO(dir+"L.dat", 'r', COMM_WORLD) > L.load(viewerL) > viewerMf = pet.Viewer().createMPIIO(dir+"Mf.dat", 'r', COMM_WORLD) > Mf.load(viewerMf) > > QE.assemble() > L.assemble() > Mf.assemble() > > KSPs = [] > # Useful solvers (here to put options for computing a smart R) > for Mat in [L,L.hermitianTranspose()]: > KSP = pet.KSP().create() > KSP.setOperators(Mat) > KSP.setFromOptions() > KSPs.append(KSP) > > class LHS_class: > def mult(self,A,x,y): > w=pet.Vec().createMPI([m_local,m],comm=COMM_WORLD) > z=pet.Vec().createMPI([m_local,m],comm=COMM_WORLD) > QE.mult(x,w) > KSPs[0].solve(w,z) > KSPs[1].solve(z,w) > QE.multTranspose(w,y) > > # Matrix free operator > LHS=pet.Mat() > LHS.create(comm=COMM_WORLD) > LHS.setSizes([[n_local,n],[n_local,n]]) > LHS.setType(pet.Mat.Type.PYTHON) > LHS.setPythonContext(LHS_class()) > LHS.setUp() > > x, y = LHS.createVecs() > x.set(1) > LHS.mult(x,y) > print(y.norm()) > > # Eigensolver > EPS = slp.EPS(); EPS.create() > EPS.setOperators(LHS,Mf) # Solve QE^T*L^-1H*L^-1*QE*f=sigma^2*Mf*f (cheaper than a proper SVD) > EPS.setProblemType(slp.EPS.ProblemType.GHEP) # Specify that A is hermitian (by construction), but M is semi-positive > EPS.setWhichEigenpairs(EPS.Which.LARGEST_MAGNITUDE) # Find largest eigenvalues > EPS.setFromOptions() > EPS.solve() > > > Quentin CHEVALIER ? IA parcours recherche > LadHyX - Ecole polytechnique > __________ > > > > > > Quentin CHEVALIER ? IA parcours recherche > LadHyX - Ecole polytechnique > > __________ > > > > On Thu, 5 May 2022 at 12:05, Jose E. Roman wrote: > Your operator is not well formed. If you do this: > > x, y = LHS.createVecs() > x.set(1) > LHS.mult(x,y) > y.view() > > you will see that the output is all zeros. That is why SLEPc complains that "Initial vector is zero or belongs to the deflation space". > > Jose > > > > El 5 may 2022, a las 10:46, Quentin Chevalier escribi?: > > > > Just a quick amend on the previous statement ; the problem arises in > > sequential and parallel. The MWE as is is provided for the parallel > > case, but imposing m_local=m makes it go sequential. > > > > Cheers, > > > > Quentin CHEVALIER ? IA parcours recherche > > LadHyX - Ecole polytechnique > > __________ > > > > > > > > On Thu, 5 May 2022 at 10:34, Quentin Chevalier > > wrote: > >> > >> Hello all and thanks for your great work in bringing this very helpful package to the community ! > >> > >> That said, I wouldn't need this mailing list if everything was running smoothly. I have a rather involved eigenvalue problem that I've been working on that's been throwing a mysterious error : > >>> > >>> petsc4py.PETSc.Error: error code 77 > >>> > >>> [1] EPSSolve() at /usr/local/slepc/src/eps/interface/epssolve.c:149 > >>> [1] EPSSolve_KrylovSchur_Default() at /usr/local/slepc/src/eps/impls/krylov/krylovschur/krylovschur.c:289 > >>> [1] EPSGetStartVector() at /usr/local/slepc/src/eps/interface/epssolve.c:824 > >>> [1] Petsc has generated inconsistent data > >>> [1] Initial vector is zero or belongs to the deflation space > >> > >> > >> This problem occurs in parallel with two processors, using the petsc4py library using the dolfinx/dolfinx docker container. I have PETSc version 3.16.0, in complex mode, python 3, and I'm running all of that on a OpenSUSE Leap 15.2 machine (but I think the docker container has a Ubuntu OS). > >> > >> I wrote a minimal working example below, but I'm afraid the process for building the matrices is involved, so I decided to directly share the matrices instead : https://seminaris.polytechnique.fr/share/s/ryJ6L2nR4ketDwP > >> > >> They are in binary format, but inside the container I hope someone could reproduce my issue. A word on the structure and intent behind these matrices : > >> > >> QE is a diagonal rectangular real matrix. Think of it as some sort of preconditioner > >> L is the least dense of them all, the only one that is complex, and in order to avoid inverting it I'm using two KSPs to compute solve problems on the fly > >> Mf is a diagonal square real matrix, its on the right-hand side of the Generalised Hermitian Eigenvalue problem (I'm solving QE^H*L^-1H*L^-1*QE*x=lambda*Mf*x > >> > >> Full MWE is below : > >> > >> from petsc4py import PETSc as pet > >> from slepc4py import SLEPc as slp > >> from mpi4py.MPI import COMM_WORLD > >> > >> # Global sizes > >> m_local=COMM_WORLD.rank*(490363-489780)+489780 > >> n_local=COMM_WORLD.rank*(452259-451743)+451743 > >> m=980143 > >> n=904002 > >> > >> QE=pet.Mat().createAIJ([[m_local,m],[n_local,n]]) > >> L=pet.Mat().createAIJ([[m_local,m],[m_local,m]]) > >> Mf=pet.Mat().createAIJ([[n_local,n],[n_local,n]]) > >> > >> viewerQE = pet.Viewer().createMPIIO("QE.dat", 'r', COMM_WORLD) > >> QE.load(viewerQE) > >> viewerL = pet.Viewer().createMPIIO("L.dat", 'r', COMM_WORLD) > >> L.load(viewerL) > >> viewerMf = pet.Viewer().createMPIIO("Mf.dat", 'r', COMM_WORLD) > >> Mf.load(viewerMf) > >> > >> QE.assemble() > >> L.assemble() > >> > >> KSPs = [] > >> # Useful solvers (here to put options for computing a smart R) > >> for Mat in [L,L.hermitianTranspose()]: > >> KSP = pet.KSP().create() > >> KSP.setOperators(Mat) > >> KSP.setFromOptions() > >> KSPs.append(KSP) > >> class LHS_class: > >> def mult(self,A,x,y): > >> w=pet.Vec().createMPI([m_local,m],comm=COMM_WORLD) > >> z=pet.Vec().createMPI([m_local,m],comm=COMM_WORLD) > >> QE.mult(x,w) > >> KSPs[0].solve(w,z) > >> KSPs[1].solve(z,w) > >> QE.multTranspose(w,y) > >> > >> # Matrix free operator > >> LHS=pet.Mat() > >> LHS.create(comm=COMM_WORLD) > >> LHS.setSizes([[n_local,n],[n_local,n]]) > >> LHS.setType(pet.Mat.Type.PYTHON) > >> LHS.setPythonContext(LHS_class()) > >> LHS.setUp() > >> > >> # Eigensolver > >> EPS = slp.EPS(); EPS.create() > >> EPS.setOperators(LHS,Mf) # Solve QE^T*L^-1H*L^-1*QE*x=lambda*Mf*x > >> EPS.setProblemType(slp.EPS.ProblemType.GHEP) # Specify that A is hermitian (by construction), and B is semi-positive > >> EPS.setWhichEigenpairs(EPS.Which.LARGEST_MAGNITUDE) # Find largest eigenvalues > >> EPS.setFromOptions() > >> EPS.solve() > >> > >> Quentin CHEVALIER ? IA parcours recherche > >> > >> LadHyX - Ecole polytechnique > >> > >> __________ > > From quentin.chevalier at polytechnique.edu Fri May 6 08:28:13 2022 From: quentin.chevalier at polytechnique.edu (Quentin Chevalier) Date: Fri, 6 May 2022 15:28:13 +0200 Subject: [petsc-users] Mysterious error code 77 In-Reply-To: <3AFCE82E-EC31-4DBB-A80E-D5499D2076EB@dsic.upv.es> References: <3AFCE82E-EC31-4DBB-A80E-D5499D2076EB@dsic.upv.es> Message-ID: Sorry for forgetting the list. Making two matrices was more of a precaution then a carefully thought strategy. It would seem the MWE as I provided it above (with a setDimensions to reduce calculation time) does work in sequential and gives the eigenvalue 118897.88711586884. I had a working MUMPS solver config for a similar problem so I simply put it there. This gives the following code. However, running it even in sequential gives a "MatSolverType mumps does not support matrix type python" error. petsc4py.PETSc.Error: error code 92 [0] EPSSolve() at /usr/local/slepc/src/eps/interface/epssolve.c:136 [0] EPSSetUp() at /usr/local/slepc/src/eps/interface/epssetup.c:350 [0] STSetUp() at /usr/local/slepc/src/sys/classes/st/interface/stsolve.c:582 [0] STSetUp_Sinvert() at /usr/local/slepc/src/sys/classes/st/impls/sinvert/sinvert.c:123 [0] KSPSetUp() at /usr/local/petsc/src/ksp/ksp/interface/itfunc.c:408 [0] PCSetUp() at /usr/local/petsc/src/ksp/pc/interface/precon.c:1016 [0] PCSetUp_LU() at /usr/local/petsc/src/ksp/pc/impls/factor/lu/lu.c:82 [0] MatGetFactor() at /usr/local/petsc/src/mat/interface/matrix.c:4779 [0] See https://petsc.org/release/overview/linear_solve_table/ for possible LU and Cholesky solvers [0] MatSolverType mumps does not support matrix type python Did I do something wrong ? I'm a bit confused by your FAQ entry, I never had a problem with this configuration in parallel. from petsc4py import PETSc as pet from slepc4py import SLEPc as slp from mpi4py.MPI import COMM_WORLD dir="./sanity_check_mats/" # Global sizes m=980143; m_local=m n=904002; n_local=n if COMM_WORLD.size>1: m_local=COMM_WORLD.rank*(490363-489780)+489780 n_local=COMM_WORLD.rank*(452259-451743)+451743 dir+="par/" else: m_local=m n_local=n dir+="seq/" QE=pet.Mat().createAIJ([[m_local,m],[n_local,n]]) L=pet.Mat().createAIJ([[m_local,m],[m_local,m]]) Mf=pet.Mat().createAIJ([[n_local,n],[n_local,n]]) viewerQE = pet.Viewer().createMPIIO(dir+"QE.dat", 'r', COMM_WORLD) QE.load(viewerQE) viewerL = pet.Viewer().createMPIIO(dir+"L.dat", 'r', COMM_WORLD) L.load(viewerL) viewerMf = pet.Viewer().createMPIIO(dir+"Mf.dat", 'r', COMM_WORLD) Mf.load(viewerMf) QE.assemble() L.assemble() Mf.assemble() KSPs = [] # Useful solvers (here to put options for computing a smart R) for Mat in [L,L.hermitianTranspose()]: KSP = pet.KSP().create() KSP.setOperators(Mat) KSP.setFromOptions() KSPs.append(KSP) class LHS_class: def mult(self,A,x,y): w=pet.Vec().createMPI([m_local,m],comm=COMM_WORLD) z=pet.Vec().createMPI([m_local,m],comm=COMM_WORLD) QE.mult(x,w) KSPs[0].solve(w,z) KSPs[1].solve(z,w) QE.multTranspose(w,y) # Matrix free operator LHS=pet.Mat() LHS.create(comm=COMM_WORLD) LHS.setSizes([[n_local,n],[n_local,n]]) LHS.setType(pet.Mat.Type.PYTHON) LHS.setPythonContext(LHS_class()) LHS.setUp() x, y = LHS.createVecs() x.set(1) LHS.mult(x,y) print(y.norm()) # Eigensolver EPS = slp.EPS(); EPS.create() EPS.setOperators(LHS,Mf) # Solve QE^T*L^-1H*L^-1*QE*f=sigma^2*Mf*f (cheaper than a proper SVD) EPS.setProblemType(slp.EPS.ProblemType.GHEP) # Specify that A is hermitian (by construction), but M is semi-positive EPS.setWhichEigenpairs(EPS.Which.LARGEST_MAGNITUDE) # Find largest eigenvalues # Spectral transform ST = EPS.getST(); ST.setType('sinvert') # Krylov subspace KSP = ST.getKSP(); KSP.setType('preonly') # Preconditioner PC = KSP.getPC(); PC.setType('lu') PC.setFactorSolverType('mumps') EPS.setDimensions(1,5) EPS.setFromOptions() EPS.solve() print(EPS.getEigenvalue(0)) Quentin CHEVALIER ? IA parcours recherche LadHyX - Ecole polytechnique __________ On Fri, 6 May 2022 at 14:47, Jose E. Roman wrote: > > Please respond to the list. > > Why would you need different matrices for sequential and parallel? PETSc takes care of loading matrices from binary files in parallel codes. If I use the 'seq' matrices for both sequential and parallel runs it works. I get the eigenvalue 128662.745858 > > Take into account that in parallel you need to use a parallel linear solver, e.g., configure PETSc with MUMPS. See the FAQ #10 https://slepc.upv.es/documentation/faq.htm#faq10 > > Jose > > > El 5 may 2022, a las 14:57, Quentin Chevalier < quentin.chevalier at polytechnique.edu> escribi?: > > > > Thank you for your answer, Jose. > > > > It would appear my problem is slightly more complicated than it appears. With slight modification of the original MWE to account for functioning in serial or parallel, and computing the matrices in either sequential or parallel (apologies, it's a large file), and including a slightly modified version of your test case, I obtain two different results in serial and parallel (commands python3 MWE.py and mpirun -n 2 python3 MWE.py). > > > > Serial gives my a finite norm (around 28) and parallel gives me two norms of 0 and a code 77. > > > > This is still a problem as I would really like my code to be parallel. I saved my matrices using : > > viewerQE = pet.Viewer().createMPIIO("QE.dat", 'w', COMM_WORLD) > > QE.view(viewerQE) > > viewerL = pet.Viewer().createMPIIO("L.dat", 'w', COMM_WORLD) > > L.view(viewerL) > > viewerMq = pet.Viewer().createMPIIO("Mq.dat", 'w', COMM_WORLD) > > Mq.view(viewerMq) > > viewerMf = pet.Viewer().createMPIIO("Mf.dat", 'w', COMM_WORLD) > > Mf.view(viewerMf) > > > > New MWE (also attached for convenience) : > > from petsc4py import PETSc as pet > > from slepc4py import SLEPc as slp > > from mpi4py.MPI import COMM_WORLD > > > > dir="./mats/" > > > > # Global sizes > > m=980143; m_local=m > > n=904002; n_local=n > > if COMM_WORLD.size>1: > > m_local=COMM_WORLD.rank*(490363-489780)+489780 > > n_local=COMM_WORLD.rank*(452259-451743)+451743 > > dir+="par/" > > else: > > m_local=m > > n_local=n > > dir+="seq/" > > > > QE=pet.Mat().createAIJ([[m_local,m],[n_local,n]]) > > L=pet.Mat().createAIJ([[m_local,m],[m_local,m]]) > > Mf=pet.Mat().createAIJ([[n_local,n],[n_local,n]]) > > > > viewerQE = pet.Viewer().createMPIIO(dir+"QE.dat", 'r', COMM_WORLD) > > QE.load(viewerQE) > > viewerL = pet.Viewer().createMPIIO(dir+"L.dat", 'r', COMM_WORLD) > > L.load(viewerL) > > viewerMf = pet.Viewer().createMPIIO(dir+"Mf.dat", 'r', COMM_WORLD) > > Mf.load(viewerMf) > > > > QE.assemble() > > L.assemble() > > Mf.assemble() > > > > KSPs = [] > > # Useful solvers (here to put options for computing a smart R) > > for Mat in [L,L.hermitianTranspose()]: > > KSP = pet.KSP().create() > > KSP.setOperators(Mat) > > KSP.setFromOptions() > > KSPs.append(KSP) > > > > class LHS_class: > > def mult(self,A,x,y): > > w=pet.Vec().createMPI([m_local,m],comm=COMM_WORLD) > > z=pet.Vec().createMPI([m_local,m],comm=COMM_WORLD) > > QE.mult(x,w) > > KSPs[0].solve(w,z) > > KSPs[1].solve(z,w) > > QE.multTranspose(w,y) > > > > # Matrix free operator > > LHS=pet.Mat() > > LHS.create(comm=COMM_WORLD) > > LHS.setSizes([[n_local,n],[n_local,n]]) > > LHS.setType(pet.Mat.Type.PYTHON) > > LHS.setPythonContext(LHS_class()) > > LHS.setUp() > > > > x, y = LHS.createVecs() > > x.set(1) > > LHS.mult(x,y) > > print(y.norm()) > > > > # Eigensolver > > EPS = slp.EPS(); EPS.create() > > EPS.setOperators(LHS,Mf) # Solve QE^T*L^-1H*L^-1*QE*f=sigma^2*Mf*f (cheaper than a proper SVD) > > EPS.setProblemType(slp.EPS.ProblemType.GHEP) # Specify that A is hermitian (by construction), but M is semi-positive > > EPS.setWhichEigenpairs(EPS.Which.LARGEST_MAGNITUDE) # Find largest eigenvalues > > EPS.setFromOptions() > > EPS.solve() > > > > > > Quentin CHEVALIER ? IA parcours recherche > > LadHyX - Ecole polytechnique > > __________ > > > > > > > > > > > > Quentin CHEVALIER ? IA parcours recherche > > LadHyX - Ecole polytechnique > > > > __________ > > > > > > > > On Thu, 5 May 2022 at 12:05, Jose E. Roman wrote: > > Your operator is not well formed. If you do this: > > > > x, y = LHS.createVecs() > > x.set(1) > > LHS.mult(x,y) > > y.view() > > > > you will see that the output is all zeros. That is why SLEPc complains that "Initial vector is zero or belongs to the deflation space". > > > > Jose > > > > > > > El 5 may 2022, a las 10:46, Quentin Chevalier < quentin.chevalier at polytechnique.edu> escribi?: > > > > > > Just a quick amend on the previous statement ; the problem arises in > > > sequential and parallel. The MWE as is is provided for the parallel > > > case, but imposing m_local=m makes it go sequential. > > > > > > Cheers, > > > > > > Quentin CHEVALIER ? IA parcours recherche > > > LadHyX - Ecole polytechnique > > > __________ > > > > > > > > > > > > On Thu, 5 May 2022 at 10:34, Quentin Chevalier > > > wrote: > > >> > > >> Hello all and thanks for your great work in bringing this very helpful package to the community ! > > >> > > >> That said, I wouldn't need this mailing list if everything was running smoothly. I have a rather involved eigenvalue problem that I've been working on that's been throwing a mysterious error : > > >>> > > >>> petsc4py.PETSc.Error: error code 77 > > >>> > > >>> [1] EPSSolve() at /usr/local/slepc/src/eps/interface/epssolve.c:149 > > >>> [1] EPSSolve_KrylovSchur_Default() at /usr/local/slepc/src/eps/impls/krylov/krylovschur/krylovschur.c:289 > > >>> [1] EPSGetStartVector() at /usr/local/slepc/src/eps/interface/epssolve.c:824 > > >>> [1] Petsc has generated inconsistent data > > >>> [1] Initial vector is zero or belongs to the deflation space > > >> > > >> > > >> This problem occurs in parallel with two processors, using the petsc4py library using the dolfinx/dolfinx docker container. I have PETSc version 3.16.0, in complex mode, python 3, and I'm running all of that on a OpenSUSE Leap 15.2 machine (but I think the docker container has a Ubuntu OS). > > >> > > >> I wrote a minimal working example below, but I'm afraid the process for building the matrices is involved, so I decided to directly share the matrices instead : https://seminaris.polytechnique.fr/share/s/ryJ6L2nR4ketDwP > > >> > > >> They are in binary format, but inside the container I hope someone could reproduce my issue. A word on the structure and intent behind these matrices : > > >> > > >> QE is a diagonal rectangular real matrix. Think of it as some sort of preconditioner > > >> L is the least dense of them all, the only one that is complex, and in order to avoid inverting it I'm using two KSPs to compute solve problems on the fly > > >> Mf is a diagonal square real matrix, its on the right-hand side of the Generalised Hermitian Eigenvalue problem (I'm solving QE^H*L^-1H*L^-1*QE*x=lambda*Mf*x > > >> > > >> Full MWE is below : > > >> > > >> from petsc4py import PETSc as pet > > >> from slepc4py import SLEPc as slp > > >> from mpi4py.MPI import COMM_WORLD > > >> > > >> # Global sizes > > >> m_local=COMM_WORLD.rank*(490363-489780)+489780 > > >> n_local=COMM_WORLD.rank*(452259-451743)+451743 > > >> m=980143 > > >> n=904002 > > >> > > >> QE=pet.Mat().createAIJ([[m_local,m],[n_local,n]]) > > >> L=pet.Mat().createAIJ([[m_local,m],[m_local,m]]) > > >> Mf=pet.Mat().createAIJ([[n_local,n],[n_local,n]]) > > >> > > >> viewerQE = pet.Viewer().createMPIIO("QE.dat", 'r', COMM_WORLD) > > >> QE.load(viewerQE) > > >> viewerL = pet.Viewer().createMPIIO("L.dat", 'r', COMM_WORLD) > > >> L.load(viewerL) > > >> viewerMf = pet.Viewer().createMPIIO("Mf.dat", 'r', COMM_WORLD) > > >> Mf.load(viewerMf) > > >> > > >> QE.assemble() > > >> L.assemble() > > >> > > >> KSPs = [] > > >> # Useful solvers (here to put options for computing a smart R) > > >> for Mat in [L,L.hermitianTranspose()]: > > >> KSP = pet.KSP().create() > > >> KSP.setOperators(Mat) > > >> KSP.setFromOptions() > > >> KSPs.append(KSP) > > >> class LHS_class: > > >> def mult(self,A,x,y): > > >> w=pet.Vec().createMPI([m_local,m],comm=COMM_WORLD) > > >> z=pet.Vec().createMPI([m_local,m],comm=COMM_WORLD) > > >> QE.mult(x,w) > > >> KSPs[0].solve(w,z) > > >> KSPs[1].solve(z,w) > > >> QE.multTranspose(w,y) > > >> > > >> # Matrix free operator > > >> LHS=pet.Mat() > > >> LHS.create(comm=COMM_WORLD) > > >> LHS.setSizes([[n_local,n],[n_local,n]]) > > >> LHS.setType(pet.Mat.Type.PYTHON) > > >> LHS.setPythonContext(LHS_class()) > > >> LHS.setUp() > > >> > > >> # Eigensolver > > >> EPS = slp.EPS(); EPS.create() > > >> EPS.setOperators(LHS,Mf) # Solve QE^T*L^-1H*L^-1*QE*x=lambda*Mf*x > > >> EPS.setProblemType(slp.EPS.ProblemType.GHEP) # Specify that A is hermitian (by construction), and B is semi-positive > > >> EPS.setWhichEigenpairs(EPS.Which.LARGEST_MAGNITUDE) # Find largest eigenvalues > > >> EPS.setFromOptions() > > >> EPS.solve() > > >> > > >> Quentin CHEVALIER ? IA parcours recherche > > >> > > >> LadHyX - Ecole polytechnique > > >> > > >> __________ > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jroman at dsic.upv.es Fri May 6 08:34:33 2022 From: jroman at dsic.upv.es (Jose E. Roman) Date: Fri, 6 May 2022 15:34:33 +0200 Subject: [petsc-users] Mysterious error code 77 In-Reply-To: References: <3AFCE82E-EC31-4DBB-A80E-D5499D2076EB@dsic.upv.es> Message-ID: MUMPS is for factorizing matrix Mf, i.e., EPS.Which.LARGEST_MAGNITUDE. But if you do shift-and-invert you have to do it in a different way. LHS cannot be factorized because it is a shell matrix. > El 6 may 2022, a las 15:28, Quentin Chevalier escribi?: > > Sorry for forgetting the list. Making two matrices was more of a precaution then a carefully thought strategy. > > It would seem the MWE as I provided it above (with a setDimensions to reduce calculation time) does work in sequential and gives the eigenvalue 118897.88711586884. > > I had a working MUMPS solver config for a similar problem so I simply put it there. This gives the following code. > > However, running it even in sequential gives a "MatSolverType mumps does not support matrix type python" error. > petsc4py.PETSc.Error: error code 92 > [0] EPSSolve() at /usr/local/slepc/src/eps/interface/epssolve.c:136 > [0] EPSSetUp() at /usr/local/slepc/src/eps/interface/epssetup.c:350 > [0] STSetUp() at /usr/local/slepc/src/sys/classes/st/interface/stsolve.c:582 > [0] STSetUp_Sinvert() at /usr/local/slepc/src/sys/classes/st/impls/sinvert/sinvert.c:123 > [0] KSPSetUp() at /usr/local/petsc/src/ksp/ksp/interface/itfunc.c:408 > [0] PCSetUp() at /usr/local/petsc/src/ksp/pc/interface/precon.c:1016 > [0] PCSetUp_LU() at /usr/local/petsc/src/ksp/pc/impls/factor/lu/lu.c:82 > [0] MatGetFactor() at /usr/local/petsc/src/mat/interface/matrix.c:4779 > [0] See https://petsc.org/release/overview/linear_solve_table/ for possible LU and Cholesky solvers > [0] MatSolverType mumps does not support matrix type python > > Did I do something wrong ? I'm a bit confused by your FAQ entry, I never had a problem with this configuration in parallel. > > from petsc4py import PETSc as pet > from slepc4py import SLEPc as slp > from mpi4py.MPI import COMM_WORLD > > dir="./sanity_check_mats/" > > # Global sizes > m=980143; m_local=m > n=904002; n_local=n > if COMM_WORLD.size>1: > m_local=COMM_WORLD.rank*(490363-489780)+489780 > n_local=COMM_WORLD.rank*(452259-451743)+451743 > dir+="par/" > else: > m_local=m > n_local=n > dir+="seq/" > > QE=pet.Mat().createAIJ([[m_local,m],[n_local,n]]) > L=pet.Mat().createAIJ([[m_local,m],[m_local,m]]) > Mf=pet.Mat().createAIJ([[n_local,n],[n_local,n]]) > > viewerQE = pet.Viewer().createMPIIO(dir+"QE.dat", 'r', COMM_WORLD) > QE.load(viewerQE) > viewerL = pet.Viewer().createMPIIO(dir+"L.dat", 'r', COMM_WORLD) > L.load(viewerL) > viewerMf = pet.Viewer().createMPIIO(dir+"Mf.dat", 'r', COMM_WORLD) > Mf.load(viewerMf) > > QE.assemble() > L.assemble() > Mf.assemble() > > KSPs = [] > # Useful solvers (here to put options for computing a smart R) > for Mat in [L,L.hermitianTranspose()]: > KSP = pet.KSP().create() > KSP.setOperators(Mat) > KSP.setFromOptions() > KSPs.append(KSP) > > class LHS_class: > def mult(self,A,x,y): > w=pet.Vec().createMPI([m_local,m],comm=COMM_WORLD) > z=pet.Vec().createMPI([m_local,m],comm=COMM_WORLD) > QE.mult(x,w) > KSPs[0].solve(w,z) > KSPs[1].solve(z,w) > QE.multTranspose(w,y) > > # Matrix free operator > LHS=pet.Mat() > LHS.create(comm=COMM_WORLD) > LHS.setSizes([[n_local,n],[n_local,n]]) > LHS.setType(pet.Mat.Type.PYTHON) > LHS.setPythonContext(LHS_class()) > LHS.setUp() > > x, y = LHS.createVecs() > x.set(1) > LHS.mult(x,y) > print(y.norm()) > > # Eigensolver > EPS = slp.EPS(); EPS.create() > EPS.setOperators(LHS,Mf) # Solve QE^T*L^-1H*L^-1*QE*f=sigma^2*Mf*f (cheaper than a proper SVD) > EPS.setProblemType(slp.EPS.ProblemType.GHEP) # Specify that A is hermitian (by construction), but M is semi-positive > EPS.setWhichEigenpairs(EPS.Which.LARGEST_MAGNITUDE) # Find largest eigenvalues > # Spectral transform > ST = EPS.getST(); ST.setType('sinvert') > # Krylov subspace > KSP = ST.getKSP(); KSP.setType('preonly') > # Preconditioner > PC = KSP.getPC(); PC.setType('lu') > PC.setFactorSolverType('mumps') > EPS.setDimensions(1,5) > EPS.setFromOptions() > EPS.solve() > print(EPS.getEigenvalue(0)) > > Quentin CHEVALIER ? IA parcours recherche > LadHyX - Ecole polytechnique > __________ > > > > On Fri, 6 May 2022 at 14:47, Jose E. Roman wrote: > > > > Please respond to the list. > > > > Why would you need different matrices for sequential and parallel? PETSc takes care of loading matrices from binary files in parallel codes. If I use the 'seq' matrices for both sequential and parallel runs it works. I get the eigenvalue 128662.745858 > > > > Take into account that in parallel you need to use a parallel linear solver, e.g., configure PETSc with MUMPS. See the FAQ #10 https://slepc.upv.es/documentation/faq.htm#faq10 > > > > Jose > > > > > El 5 may 2022, a las 14:57, Quentin Chevalier escribi?: > > > > > > Thank you for your answer, Jose. > > > > > > It would appear my problem is slightly more complicated than it appears. With slight modification of the original MWE to account for functioning in serial or parallel, and computing the matrices in either sequential or parallel (apologies, it's a large file), and including a slightly modified version of your test case, I obtain two different results in serial and parallel (commands python3 MWE.py and mpirun -n 2 python3 MWE.py). > > > > > > Serial gives my a finite norm (around 28) and parallel gives me two norms of 0 and a code 77. > > > > > > This is still a problem as I would really like my code to be parallel. I saved my matrices using : > > > viewerQE = pet.Viewer().createMPIIO("QE.dat", 'w', COMM_WORLD) > > > QE.view(viewerQE) > > > viewerL = pet.Viewer().createMPIIO("L.dat", 'w', COMM_WORLD) > > > L.view(viewerL) > > > viewerMq = pet.Viewer().createMPIIO("Mq.dat", 'w', COMM_WORLD) > > > Mq.view(viewerMq) > > > viewerMf = pet.Viewer().createMPIIO("Mf.dat", 'w', COMM_WORLD) > > > Mf.view(viewerMf) > > > > > > New MWE (also attached for convenience) : > > > from petsc4py import PETSc as pet > > > from slepc4py import SLEPc as slp > > > from mpi4py.MPI import COMM_WORLD > > > > > > dir="./mats/" > > > > > > # Global sizes > > > m=980143; m_local=m > > > n=904002; n_local=n > > > if COMM_WORLD.size>1: > > > m_local=COMM_WORLD.rank*(490363-489780)+489780 > > > n_local=COMM_WORLD.rank*(452259-451743)+451743 > > > dir+="par/" > > > else: > > > m_local=m > > > n_local=n > > > dir+="seq/" > > > > > > QE=pet.Mat().createAIJ([[m_local,m],[n_local,n]]) > > > L=pet.Mat().createAIJ([[m_local,m],[m_local,m]]) > > > Mf=pet.Mat().createAIJ([[n_local,n],[n_local,n]]) > > > > > > viewerQE = pet.Viewer().createMPIIO(dir+"QE.dat", 'r', COMM_WORLD) > > > QE.load(viewerQE) > > > viewerL = pet.Viewer().createMPIIO(dir+"L.dat", 'r', COMM_WORLD) > > > L.load(viewerL) > > > viewerMf = pet.Viewer().createMPIIO(dir+"Mf.dat", 'r', COMM_WORLD) > > > Mf.load(viewerMf) > > > > > > QE.assemble() > > > L.assemble() > > > Mf.assemble() > > > > > > KSPs = [] > > > # Useful solvers (here to put options for computing a smart R) > > > for Mat in [L,L.hermitianTranspose()]: > > > KSP = pet.KSP().create() > > > KSP.setOperators(Mat) > > > KSP.setFromOptions() > > > KSPs.append(KSP) > > > > > > class LHS_class: > > > def mult(self,A,x,y): > > > w=pet.Vec().createMPI([m_local,m],comm=COMM_WORLD) > > > z=pet.Vec().createMPI([m_local,m],comm=COMM_WORLD) > > > QE.mult(x,w) > > > KSPs[0].solve(w,z) > > > KSPs[1].solve(z,w) > > > QE.multTranspose(w,y) > > > > > > # Matrix free operator > > > LHS=pet.Mat() > > > LHS.create(comm=COMM_WORLD) > > > LHS.setSizes([[n_local,n],[n_local,n]]) > > > LHS.setType(pet.Mat.Type.PYTHON) > > > LHS.setPythonContext(LHS_class()) > > > LHS.setUp() > > > > > > x, y = LHS.createVecs() > > > x.set(1) > > > LHS.mult(x,y) > > > print(y.norm()) > > > > > > # Eigensolver > > > EPS = slp.EPS(); EPS.create() > > > EPS.setOperators(LHS,Mf) # Solve QE^T*L^-1H*L^-1*QE*f=sigma^2*Mf*f (cheaper than a proper SVD) > > > EPS.setProblemType(slp.EPS.ProblemType.GHEP) # Specify that A is hermitian (by construction), but M is semi-positive > > > EPS.setWhichEigenpairs(EPS.Which.LARGEST_MAGNITUDE) # Find largest eigenvalues > > > EPS.setFromOptions() > > > EPS.solve() > > > > > > > > > Quentin CHEVALIER ? IA parcours recherche > > > LadHyX - Ecole polytechnique > > > __________ > > > > > > > > > > > > > > > > > > Quentin CHEVALIER ? IA parcours recherche > > > LadHyX - Ecole polytechnique > > > > > > __________ > > > > > > > > > > > > On Thu, 5 May 2022 at 12:05, Jose E. Roman wrote: > > > Your operator is not well formed. If you do this: > > > > > > x, y = LHS.createVecs() > > > x.set(1) > > > LHS.mult(x,y) > > > y.view() > > > > > > you will see that the output is all zeros. That is why SLEPc complains that "Initial vector is zero or belongs to the deflation space". > > > > > > Jose > > > > > > > > > > El 5 may 2022, a las 10:46, Quentin Chevalier escribi?: > > > > > > > > Just a quick amend on the previous statement ; the problem arises in > > > > sequential and parallel. The MWE as is is provided for the parallel > > > > case, but imposing m_local=m makes it go sequential. > > > > > > > > Cheers, > > > > > > > > Quentin CHEVALIER ? IA parcours recherche > > > > LadHyX - Ecole polytechnique > > > > __________ > > > > > > > > > > > > > > > > On Thu, 5 May 2022 at 10:34, Quentin Chevalier > > > > wrote: > > > >> > > > >> Hello all and thanks for your great work in bringing this very helpful package to the community ! > > > >> > > > >> That said, I wouldn't need this mailing list if everything was running smoothly. I have a rather involved eigenvalue problem that I've been working on that's been throwing a mysterious error : > > > >>> > > > >>> petsc4py.PETSc.Error: error code 77 > > > >>> > > > >>> [1] EPSSolve() at /usr/local/slepc/src/eps/interface/epssolve.c:149 > > > >>> [1] EPSSolve_KrylovSchur_Default() at /usr/local/slepc/src/eps/impls/krylov/krylovschur/krylovschur.c:289 > > > >>> [1] EPSGetStartVector() at /usr/local/slepc/src/eps/interface/epssolve.c:824 > > > >>> [1] Petsc has generated inconsistent data > > > >>> [1] Initial vector is zero or belongs to the deflation space > > > >> > > > >> > > > >> This problem occurs in parallel with two processors, using the petsc4py library using the dolfinx/dolfinx docker container. I have PETSc version 3.16.0, in complex mode, python 3, and I'm running all of that on a OpenSUSE Leap 15.2 machine (but I think the docker container has a Ubuntu OS). > > > >> > > > >> I wrote a minimal working example below, but I'm afraid the process for building the matrices is involved, so I decided to directly share the matrices instead : https://seminaris.polytechnique.fr/share/s/ryJ6L2nR4ketDwP > > > >> > > > >> They are in binary format, but inside the container I hope someone could reproduce my issue. A word on the structure and intent behind these matrices : > > > >> > > > >> QE is a diagonal rectangular real matrix. Think of it as some sort of preconditioner > > > >> L is the least dense of them all, the only one that is complex, and in order to avoid inverting it I'm using two KSPs to compute solve problems on the fly > > > >> Mf is a diagonal square real matrix, its on the right-hand side of the Generalised Hermitian Eigenvalue problem (I'm solving QE^H*L^-1H*L^-1*QE*x=lambda*Mf*x > > > >> > > > >> Full MWE is below : > > > >> > > > >> from petsc4py import PETSc as pet > > > >> from slepc4py import SLEPc as slp > > > >> from mpi4py.MPI import COMM_WORLD > > > >> > > > >> # Global sizes > > > >> m_local=COMM_WORLD.rank*(490363-489780)+489780 > > > >> n_local=COMM_WORLD.rank*(452259-451743)+451743 > > > >> m=980143 > > > >> n=904002 > > > >> > > > >> QE=pet.Mat().createAIJ([[m_local,m],[n_local,n]]) > > > >> L=pet.Mat().createAIJ([[m_local,m],[m_local,m]]) > > > >> Mf=pet.Mat().createAIJ([[n_local,n],[n_local,n]]) > > > >> > > > >> viewerQE = pet.Viewer().createMPIIO("QE.dat", 'r', COMM_WORLD) > > > >> QE.load(viewerQE) > > > >> viewerL = pet.Viewer().createMPIIO("L.dat", 'r', COMM_WORLD) > > > >> L.load(viewerL) > > > >> viewerMf = pet.Viewer().createMPIIO("Mf.dat", 'r', COMM_WORLD) > > > >> Mf.load(viewerMf) > > > >> > > > >> QE.assemble() > > > >> L.assemble() > > > >> > > > >> KSPs = [] > > > >> # Useful solvers (here to put options for computing a smart R) > > > >> for Mat in [L,L.hermitianTranspose()]: > > > >> KSP = pet.KSP().create() > > > >> KSP.setOperators(Mat) > > > >> KSP.setFromOptions() > > > >> KSPs.append(KSP) > > > >> class LHS_class: > > > >> def mult(self,A,x,y): > > > >> w=pet.Vec().createMPI([m_local,m],comm=COMM_WORLD) > > > >> z=pet.Vec().createMPI([m_local,m],comm=COMM_WORLD) > > > >> QE.mult(x,w) > > > >> KSPs[0].solve(w,z) > > > >> KSPs[1].solve(z,w) > > > >> QE.multTranspose(w,y) > > > >> > > > >> # Matrix free operator > > > >> LHS=pet.Mat() > > > >> LHS.create(comm=COMM_WORLD) > > > >> LHS.setSizes([[n_local,n],[n_local,n]]) > > > >> LHS.setType(pet.Mat.Type.PYTHON) > > > >> LHS.setPythonContext(LHS_class()) > > > >> LHS.setUp() > > > >> > > > >> # Eigensolver > > > >> EPS = slp.EPS(); EPS.create() > > > >> EPS.setOperators(LHS,Mf) # Solve QE^T*L^-1H*L^-1*QE*x=lambda*Mf*x > > > >> EPS.setProblemType(slp.EPS.ProblemType.GHEP) # Specify that A is hermitian (by construction), and B is semi-positive > > > >> EPS.setWhichEigenpairs(EPS.Which.LARGEST_MAGNITUDE) # Find largest eigenvalues > > > >> EPS.setFromOptions() > > > >> EPS.solve() > > > >> > > > >> Quentin CHEVALIER ? IA parcours recherche > > > >> > > > >> LadHyX - Ecole polytechnique > > > >> > > > >> __________ > > > > > > > > From knepley at gmail.com Fri May 6 08:34:59 2022 From: knepley at gmail.com (Matthew Knepley) Date: Fri, 6 May 2022 09:34:59 -0400 Subject: [petsc-users] Mysterious error code 77 In-Reply-To: References: <3AFCE82E-EC31-4DBB-A80E-D5499D2076EB@dsic.upv.es> Message-ID: On Fri, May 6, 2022 at 9:28 AM Quentin Chevalier < quentin.chevalier at polytechnique.edu> wrote: > Sorry for forgetting the list. Making two matrices was more of a > precaution then a carefully thought strategy. > > It would seem the MWE as I provided it above (with a setDimensions to > reduce calculation time) does work in sequential and gives the eigenvalue > 118897.88711586884. > > I had a working MUMPS solver config for a similar problem so I simply put > it there. This gives the following code. > > However, running it even in sequential gives a "MatSolverType mumps does > not support matrix type python" error. > petsc4py.PETSc.Error: error code 92 > [0] EPSSolve() at /usr/local/slepc/src/eps/interface/epssolve.c:136 > [0] EPSSetUp() at /usr/local/slepc/src/eps/interface/epssetup.c:350 > [0] STSetUp() at > /usr/local/slepc/src/sys/classes/st/interface/stsolve.c:582 > [0] STSetUp_Sinvert() at > /usr/local/slepc/src/sys/classes/st/impls/sinvert/sinvert.c:123 > [0] KSPSetUp() at /usr/local/petsc/src/ksp/ksp/interface/itfunc.c:408 > [0] PCSetUp() at /usr/local/petsc/src/ksp/pc/interface/precon.c:1016 > [0] PCSetUp_LU() at /usr/local/petsc/src/ksp/pc/impls/factor/lu/lu.c:82 > [0] MatGetFactor() at /usr/local/petsc/src/mat/interface/matrix.c:4779 > [0] See https://petsc.org/release/overview/linear_solve_table/ for > possible LU and Cholesky solvers > [0] MatSolverType mumps does not support matrix type python > You would need type AIJ for MUMPS. Thanks, Matt > Did I do something wrong ? I'm a bit confused by your FAQ entry, I never > had a problem with this configuration in parallel. > > from petsc4py import PETSc as pet > from slepc4py import SLEPc as slp > from mpi4py.MPI import COMM_WORLD > > dir="./sanity_check_mats/" > > # Global sizes > m=980143; m_local=m > n=904002; n_local=n > if COMM_WORLD.size>1: > m_local=COMM_WORLD.rank*(490363-489780)+489780 > n_local=COMM_WORLD.rank*(452259-451743)+451743 > dir+="par/" > else: > m_local=m > n_local=n > dir+="seq/" > > QE=pet.Mat().createAIJ([[m_local,m],[n_local,n]]) > L=pet.Mat().createAIJ([[m_local,m],[m_local,m]]) > Mf=pet.Mat().createAIJ([[n_local,n],[n_local,n]]) > > viewerQE = pet.Viewer().createMPIIO(dir+"QE.dat", 'r', COMM_WORLD) > QE.load(viewerQE) > viewerL = pet.Viewer().createMPIIO(dir+"L.dat", 'r', COMM_WORLD) > L.load(viewerL) > viewerMf = pet.Viewer().createMPIIO(dir+"Mf.dat", 'r', COMM_WORLD) > Mf.load(viewerMf) > > QE.assemble() > L.assemble() > Mf.assemble() > > KSPs = [] > # Useful solvers (here to put options for computing a smart R) > for Mat in [L,L.hermitianTranspose()]: > KSP = pet.KSP().create() > KSP.setOperators(Mat) > KSP.setFromOptions() > KSPs.append(KSP) > class LHS_class: > def mult(self,A,x,y): > w=pet.Vec().createMPI([m_local,m],comm=COMM_WORLD) > z=pet.Vec().createMPI([m_local,m],comm=COMM_WORLD) > QE.mult(x,w) > KSPs[0].solve(w,z) > KSPs[1].solve(z,w) > QE.multTranspose(w,y) > > # Matrix free operator > LHS=pet.Mat() > LHS.create(comm=COMM_WORLD) > LHS.setSizes([[n_local,n],[n_local,n]]) > LHS.setType(pet.Mat.Type.PYTHON) > LHS.setPythonContext(LHS_class()) > LHS.setUp() > > x, y = LHS.createVecs() > x.set(1) > LHS.mult(x,y) > print(y.norm()) > > # Eigensolver > EPS = slp.EPS(); EPS.create() > EPS.setOperators(LHS,Mf) # Solve QE^T*L^-1H*L^-1*QE*f=sigma^2*Mf*f > (cheaper than a proper SVD) > EPS.setProblemType(slp.EPS.ProblemType.GHEP) # Specify that A is > hermitian (by construction), but M is semi-positive > EPS.setWhichEigenpairs(EPS.Which.LARGEST_MAGNITUDE) # Find largest > eigenvalues > # Spectral transform > ST = EPS.getST(); ST.setType('sinvert') > # Krylov subspace > KSP = ST.getKSP(); KSP.setType('preonly') > # Preconditioner > PC = KSP.getPC(); PC.setType('lu') > PC.setFactorSolverType('mumps') > EPS.setDimensions(1,5) > EPS.setFromOptions() > EPS.solve() > print(EPS.getEigenvalue(0)) > > Quentin CHEVALIER ? IA parcours recherche > LadHyX - Ecole polytechnique > __________ > > > > On Fri, 6 May 2022 at 14:47, Jose E. Roman wrote: > > > > Please respond to the list. > > > > Why would you need different matrices for sequential and parallel? PETSc > takes care of loading matrices from binary files in parallel codes. If I > use the 'seq' matrices for both sequential and parallel runs it works. I > get the eigenvalue 128662.745858 > > > > Take into account that in parallel you need to use a parallel linear > solver, e.g., configure PETSc with MUMPS. See the FAQ #10 > https://slepc.upv.es/documentation/faq.htm#faq10 > > > > Jose > > > > > El 5 may 2022, a las 14:57, Quentin Chevalier < > quentin.chevalier at polytechnique.edu> escribi?: > > > > > > Thank you for your answer, Jose. > > > > > > It would appear my problem is slightly more complicated than it > appears. With slight modification of the original MWE to account for > functioning in serial or parallel, and computing the matrices in either > sequential or parallel (apologies, it's a large file), and including a > slightly modified version of your test case, I obtain two different results > in serial and parallel (commands python3 MWE.py and mpirun -n 2 python3 > MWE.py). > > > > > > Serial gives my a finite norm (around 28) and parallel gives me two > norms of 0 and a code 77. > > > > > > This is still a problem as I would really like my code to be parallel. > I saved my matrices using : > > > viewerQE = pet.Viewer().createMPIIO("QE.dat", 'w', COMM_WORLD) > > > QE.view(viewerQE) > > > viewerL = pet.Viewer().createMPIIO("L.dat", 'w', COMM_WORLD) > > > L.view(viewerL) > > > viewerMq = pet.Viewer().createMPIIO("Mq.dat", 'w', COMM_WORLD) > > > Mq.view(viewerMq) > > > viewerMf = pet.Viewer().createMPIIO("Mf.dat", 'w', COMM_WORLD) > > > Mf.view(viewerMf) > > > > > > New MWE (also attached for convenience) : > > > from petsc4py import PETSc as pet > > > from slepc4py import SLEPc as slp > > > from mpi4py.MPI import COMM_WORLD > > > > > > dir="./mats/" > > > > > > # Global sizes > > > m=980143; m_local=m > > > n=904002; n_local=n > > > if COMM_WORLD.size>1: > > > m_local=COMM_WORLD.rank*(490363-489780)+489780 > > > n_local=COMM_WORLD.rank*(452259-451743)+451743 > > > dir+="par/" > > > else: > > > m_local=m > > > n_local=n > > > dir+="seq/" > > > > > > QE=pet.Mat().createAIJ([[m_local,m],[n_local,n]]) > > > L=pet.Mat().createAIJ([[m_local,m],[m_local,m]]) > > > Mf=pet.Mat().createAIJ([[n_local,n],[n_local,n]]) > > > > > > viewerQE = pet.Viewer().createMPIIO(dir+"QE.dat", 'r', COMM_WORLD) > > > QE.load(viewerQE) > > > viewerL = pet.Viewer().createMPIIO(dir+"L.dat", 'r', COMM_WORLD) > > > L.load(viewerL) > > > viewerMf = pet.Viewer().createMPIIO(dir+"Mf.dat", 'r', COMM_WORLD) > > > Mf.load(viewerMf) > > > > > > QE.assemble() > > > L.assemble() > > > Mf.assemble() > > > > > > KSPs = [] > > > # Useful solvers (here to put options for computing a smart R) > > > for Mat in [L,L.hermitianTranspose()]: > > > KSP = pet.KSP().create() > > > KSP.setOperators(Mat) > > > KSP.setFromOptions() > > > KSPs.append(KSP) > > > > > > class LHS_class: > > > def mult(self,A,x,y): > > > w=pet.Vec().createMPI([m_local,m],comm=COMM_WORLD) > > > z=pet.Vec().createMPI([m_local,m],comm=COMM_WORLD) > > > QE.mult(x,w) > > > KSPs[0].solve(w,z) > > > KSPs[1].solve(z,w) > > > QE.multTranspose(w,y) > > > > > > # Matrix free operator > > > LHS=pet.Mat() > > > LHS.create(comm=COMM_WORLD) > > > LHS.setSizes([[n_local,n],[n_local,n]]) > > > LHS.setType(pet.Mat.Type.PYTHON) > > > LHS.setPythonContext(LHS_class()) > > > LHS.setUp() > > > > > > x, y = LHS.createVecs() > > > x.set(1) > > > LHS.mult(x,y) > > > print(y.norm()) > > > > > > # Eigensolver > > > EPS = slp.EPS(); EPS.create() > > > EPS.setOperators(LHS,Mf) # Solve QE^T*L^-1H*L^-1*QE*f=sigma^2*Mf*f > (cheaper than a proper SVD) > > > EPS.setProblemType(slp.EPS.ProblemType.GHEP) # Specify that A is > hermitian (by construction), but M is semi-positive > > > EPS.setWhichEigenpairs(EPS.Which.LARGEST_MAGNITUDE) # Find largest > eigenvalues > > > EPS.setFromOptions() > > > EPS.solve() > > > > > > > > > Quentin CHEVALIER ? IA parcours recherche > > > LadHyX - Ecole polytechnique > > > __________ > > > > > > > > > > > > > > > > > > Quentin CHEVALIER ? IA parcours recherche > > > LadHyX - Ecole polytechnique > > > > > > __________ > > > > > > > > > > > > On Thu, 5 May 2022 at 12:05, Jose E. Roman wrote: > > > Your operator is not well formed. If you do this: > > > > > > x, y = LHS.createVecs() > > > x.set(1) > > > LHS.mult(x,y) > > > y.view() > > > > > > you will see that the output is all zeros. That is why SLEPc complains > that "Initial vector is zero or belongs to the deflation space". > > > > > > Jose > > > > > > > > > > El 5 may 2022, a las 10:46, Quentin Chevalier < > quentin.chevalier at polytechnique.edu> escribi?: > > > > > > > > Just a quick amend on the previous statement ; the problem arises in > > > > sequential and parallel. The MWE as is is provided for the parallel > > > > case, but imposing m_local=m makes it go sequential. > > > > > > > > Cheers, > > > > > > > > Quentin CHEVALIER ? IA parcours recherche > > > > LadHyX - Ecole polytechnique > > > > __________ > > > > > > > > > > > > > > > > On Thu, 5 May 2022 at 10:34, Quentin Chevalier > > > > wrote: > > > >> > > > >> Hello all and thanks for your great work in bringing this very > helpful package to the community ! > > > >> > > > >> That said, I wouldn't need this mailing list if everything was > running smoothly. I have a rather involved eigenvalue problem that I've > been working on that's been throwing a mysterious error : > > > >>> > > > >>> petsc4py.PETSc.Error: error code 77 > > > >>> > > > >>> [1] EPSSolve() at /usr/local/slepc/src/eps/interface/epssolve.c:149 > > > >>> [1] EPSSolve_KrylovSchur_Default() at > /usr/local/slepc/src/eps/impls/krylov/krylovschur/krylovschur.c:289 > > > >>> [1] EPSGetStartVector() at > /usr/local/slepc/src/eps/interface/epssolve.c:824 > > > >>> [1] Petsc has generated inconsistent data > > > >>> [1] Initial vector is zero or belongs to the deflation space > > > >> > > > >> > > > >> This problem occurs in parallel with two processors, using the > petsc4py library using the dolfinx/dolfinx docker container. I have PETSc > version 3.16.0, in complex mode, python 3, and I'm running all of that on a > OpenSUSE Leap 15.2 machine (but I think the docker container has a Ubuntu > OS). > > > >> > > > >> I wrote a minimal working example below, but I'm afraid the process > for building the matrices is involved, so I decided to directly share the > matrices instead : > https://seminaris.polytechnique.fr/share/s/ryJ6L2nR4ketDwP > > > >> > > > >> They are in binary format, but inside the container I hope someone > could reproduce my issue. A word on the structure and intent behind these > matrices : > > > >> > > > >> QE is a diagonal rectangular real matrix. Think of it as some sort > of preconditioner > > > >> L is the least dense of them all, the only one that is complex, and > in order to avoid inverting it I'm using two KSPs to compute solve problems > on the fly > > > >> Mf is a diagonal square real matrix, its on the right-hand side of > the Generalised Hermitian Eigenvalue problem (I'm solving > QE^H*L^-1H*L^-1*QE*x=lambda*Mf*x > > > >> > > > >> Full MWE is below : > > > >> > > > >> from petsc4py import PETSc as pet > > > >> from slepc4py import SLEPc as slp > > > >> from mpi4py.MPI import COMM_WORLD > > > >> > > > >> # Global sizes > > > >> m_local=COMM_WORLD.rank*(490363-489780)+489780 > > > >> n_local=COMM_WORLD.rank*(452259-451743)+451743 > > > >> m=980143 > > > >> n=904002 > > > >> > > > >> QE=pet.Mat().createAIJ([[m_local,m],[n_local,n]]) > > > >> L=pet.Mat().createAIJ([[m_local,m],[m_local,m]]) > > > >> Mf=pet.Mat().createAIJ([[n_local,n],[n_local,n]]) > > > >> > > > >> viewerQE = pet.Viewer().createMPIIO("QE.dat", 'r', COMM_WORLD) > > > >> QE.load(viewerQE) > > > >> viewerL = pet.Viewer().createMPIIO("L.dat", 'r', COMM_WORLD) > > > >> L.load(viewerL) > > > >> viewerMf = pet.Viewer().createMPIIO("Mf.dat", 'r', COMM_WORLD) > > > >> Mf.load(viewerMf) > > > >> > > > >> QE.assemble() > > > >> L.assemble() > > > >> > > > >> KSPs = [] > > > >> # Useful solvers (here to put options for computing a smart R) > > > >> for Mat in [L,L.hermitianTranspose()]: > > > >> KSP = pet.KSP().create() > > > >> KSP.setOperators(Mat) > > > >> KSP.setFromOptions() > > > >> KSPs.append(KSP) > > > >> class LHS_class: > > > >> def mult(self,A,x,y): > > > >> w=pet.Vec().createMPI([m_local,m],comm=COMM_WORLD) > > > >> z=pet.Vec().createMPI([m_local,m],comm=COMM_WORLD) > > > >> QE.mult(x,w) > > > >> KSPs[0].solve(w,z) > > > >> KSPs[1].solve(z,w) > > > >> QE.multTranspose(w,y) > > > >> > > > >> # Matrix free operator > > > >> LHS=pet.Mat() > > > >> LHS.create(comm=COMM_WORLD) > > > >> LHS.setSizes([[n_local,n],[n_local,n]]) > > > >> LHS.setType(pet.Mat.Type.PYTHON) > > > >> LHS.setPythonContext(LHS_class()) > > > >> LHS.setUp() > > > >> > > > >> # Eigensolver > > > >> EPS = slp.EPS(); EPS.create() > > > >> EPS.setOperators(LHS,Mf) # Solve QE^T*L^-1H*L^-1*QE*x=lambda*Mf*x > > > >> EPS.setProblemType(slp.EPS.ProblemType.GHEP) # Specify that A is > hermitian (by construction), and B is semi-positive > > > >> EPS.setWhichEigenpairs(EPS.Which.LARGEST_MAGNITUDE) # Find largest > eigenvalues > > > >> EPS.setFromOptions() > > > >> EPS.solve() > > > >> > > > >> Quentin CHEVALIER ? IA parcours recherche > > > >> > > > >> LadHyX - Ecole polytechnique > > > >> > > > >> __________ > > > > > > > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From quentin.chevalier at polytechnique.edu Fri May 6 08:46:24 2022 From: quentin.chevalier at polytechnique.edu (Quentin Chevalier) Date: Fri, 6 May 2022 15:46:24 +0200 Subject: [petsc-users] Mysterious error code 77 In-Reply-To: References: <3AFCE82E-EC31-4DBB-A80E-D5499D2076EB@dsic.upv.es> Message-ID: @Matthew, I'm sure you've noticed the L.solve inside the LHS class. That's preventing me from going AIJ ; I would have to invert L, which sounds like a bad idea. @Jose, I've had troubles reading the matrix files in parallel when they were created in sequential or parallel with a different number of processors - that's why I resorted to 2, and hard-coded the local sizes. If I naively try to create AIJ with global sizes only, I get Incompatible sizes every time. Would you care to enlighten me as to how to read the same matrix files with an arbitrary number of processors ? More importantly, you managed to run the MWE sent on May 5th without any change uin the same config as me and couldn't reproduce a code 77 ? You're also telling me that Mf needs to be factorised with MUMPS, but that LHS can't. So did you specify an additional solver/preconditioner inside the MWE or did it work as is ? Cheers, Quentin CHEVALIER ? IA parcours recherche LadHyX - Ecole polytechnique __________ On Fri, 6 May 2022 at 15:35, Matthew Knepley wrote: > > On Fri, May 6, 2022 at 9:28 AM Quentin Chevalier wrote: >> >> Sorry for forgetting the list. Making two matrices was more of a precaution then a carefully thought strategy. >> >> It would seem the MWE as I provided it above (with a setDimensions to reduce calculation time) does work in sequential and gives the eigenvalue 118897.88711586884. >> >> I had a working MUMPS solver config for a similar problem so I simply put it there. This gives the following code. >> >> However, running it even in sequential gives a "MatSolverType mumps does not support matrix type python" error. >> petsc4py.PETSc.Error: error code 92 >> [0] EPSSolve() at /usr/local/slepc/src/eps/interface/epssolve.c:136 >> [0] EPSSetUp() at /usr/local/slepc/src/eps/interface/epssetup.c:350 >> [0] STSetUp() at /usr/local/slepc/src/sys/classes/st/interface/stsolve.c:582 >> [0] STSetUp_Sinvert() at /usr/local/slepc/src/sys/classes/st/impls/sinvert/sinvert.c:123 >> [0] KSPSetUp() at /usr/local/petsc/src/ksp/ksp/interface/itfunc.c:408 >> [0] PCSetUp() at /usr/local/petsc/src/ksp/pc/interface/precon.c:1016 >> [0] PCSetUp_LU() at /usr/local/petsc/src/ksp/pc/impls/factor/lu/lu.c:82 >> [0] MatGetFactor() at /usr/local/petsc/src/mat/interface/matrix.c:4779 >> [0] See https://petsc.org/release/overview/linear_solve_table/ for possible LU and Cholesky solvers >> [0] MatSolverType mumps does not support matrix type python > > > You would need type AIJ for MUMPS. > > Thanks, > > Matt > >> >> Did I do something wrong ? I'm a bit confused by your FAQ entry, I never had a problem with this configuration in parallel. >> >> from petsc4py import PETSc as pet >> from slepc4py import SLEPc as slp >> from mpi4py.MPI import COMM_WORLD >> >> dir="./sanity_check_mats/" >> >> # Global sizes >> m=980143; m_local=m >> n=904002; n_local=n >> if COMM_WORLD.size>1: >> m_local=COMM_WORLD.rank*(490363-489780)+489780 >> n_local=COMM_WORLD.rank*(452259-451743)+451743 >> dir+="par/" >> else: >> m_local=m >> n_local=n >> dir+="seq/" >> >> QE=pet.Mat().createAIJ([[m_local,m],[n_local,n]]) >> L=pet.Mat().createAIJ([[m_local,m],[m_local,m]]) >> Mf=pet.Mat().createAIJ([[n_local,n],[n_local,n]]) >> >> viewerQE = pet.Viewer().createMPIIO(dir+"QE.dat", 'r', COMM_WORLD) >> QE.load(viewerQE) >> viewerL = pet.Viewer().createMPIIO(dir+"L.dat", 'r', COMM_WORLD) >> L.load(viewerL) >> viewerMf = pet.Viewer().createMPIIO(dir+"Mf.dat", 'r', COMM_WORLD) >> Mf.load(viewerMf) >> >> QE.assemble() >> L.assemble() >> Mf.assemble() >> >> KSPs = [] >> # Useful solvers (here to put options for computing a smart R) >> for Mat in [L,L.hermitianTranspose()]: >> KSP = pet.KSP().create() >> KSP.setOperators(Mat) >> KSP.setFromOptions() >> KSPs.append(KSP) >> class LHS_class: >> def mult(self,A,x,y): >> w=pet.Vec().createMPI([m_local,m],comm=COMM_WORLD) >> z=pet.Vec().createMPI([m_local,m],comm=COMM_WORLD) >> QE.mult(x,w) >> KSPs[0].solve(w,z) >> KSPs[1].solve(z,w) >> QE.multTranspose(w,y) >> >> # Matrix free operator >> LHS=pet.Mat() >> LHS.create(comm=COMM_WORLD) >> LHS.setSizes([[n_local,n],[n_local,n]]) >> LHS.setType(pet.Mat.Type.PYTHON) >> LHS.setPythonContext(LHS_class()) >> LHS.setUp() >> >> x, y = LHS.createVecs() >> x.set(1) >> LHS.mult(x,y) >> print(y.norm()) >> >> # Eigensolver >> EPS = slp.EPS(); EPS.create() >> EPS.setOperators(LHS,Mf) # Solve QE^T*L^-1H*L^-1*QE*f=sigma^2*Mf*f (cheaper than a proper SVD) >> EPS.setProblemType(slp.EPS.ProblemType.GHEP) # Specify that A is hermitian (by construction), but M is semi-positive >> EPS.setWhichEigenpairs(EPS.Which.LARGEST_MAGNITUDE) # Find largest eigenvalues >> # Spectral transform >> ST = EPS.getST(); ST.setType('sinvert') >> # Krylov subspace >> KSP = ST.getKSP(); KSP.setType('preonly') >> # Preconditioner >> PC = KSP.getPC(); PC.setType('lu') >> PC.setFactorSolverType('mumps') >> EPS.setDimensions(1,5) >> EPS.setFromOptions() >> EPS.solve() >> print(EPS.getEigenvalue(0)) >> >> Quentin CHEVALIER ? IA parcours recherche >> LadHyX - Ecole polytechnique >> __________ >> >> >> >> On Fri, 6 May 2022 at 14:47, Jose E. Roman wrote: >> > >> > Please respond to the list. >> > >> > Why would you need different matrices for sequential and parallel? PETSc takes care of loading matrices from binary files in parallel codes. If I use the 'seq' matrices for both sequential and parallel runs it works. I get the eigenvalue 128662.745858 >> > >> > Take into account that in parallel you need to use a parallel linear solver, e.g., configure PETSc with MUMPS. See the FAQ #10 https://slepc.upv.es/documentation/faq.htm#faq10 >> > >> > Jose >> > >> > > El 5 may 2022, a las 14:57, Quentin Chevalier escribi?: >> > > >> > > Thank you for your answer, Jose. >> > > >> > > It would appear my problem is slightly more complicated than it appears. With slight modification of the original MWE to account for functioning in serial or parallel, and computing the matrices in either sequential or parallel (apologies, it's a large file), and including a slightly modified version of your test case, I obtain two different results in serial and parallel (commands python3 MWE.py and mpirun -n 2 python3 MWE.py). >> > > >> > > Serial gives my a finite norm (around 28) and parallel gives me two norms of 0 and a code 77. >> > > >> > > This is still a problem as I would really like my code to be parallel. I saved my matrices using : >> > > viewerQE = pet.Viewer().createMPIIO("QE.dat", 'w', COMM_WORLD) >> > > QE.view(viewerQE) >> > > viewerL = pet.Viewer().createMPIIO("L.dat", 'w', COMM_WORLD) >> > > L.view(viewerL) >> > > viewerMq = pet.Viewer().createMPIIO("Mq.dat", 'w', COMM_WORLD) >> > > Mq.view(viewerMq) >> > > viewerMf = pet.Viewer().createMPIIO("Mf.dat", 'w', COMM_WORLD) >> > > Mf.view(viewerMf) >> > > >> > > New MWE (also attached for convenience) : >> > > from petsc4py import PETSc as pet >> > > from slepc4py import SLEPc as slp >> > > from mpi4py.MPI import COMM_WORLD >> > > >> > > dir="./mats/" >> > > >> > > # Global sizes >> > > m=980143; m_local=m >> > > n=904002; n_local=n >> > > if COMM_WORLD.size>1: >> > > m_local=COMM_WORLD.rank*(490363-489780)+489780 >> > > n_local=COMM_WORLD.rank*(452259-451743)+451743 >> > > dir+="par/" >> > > else: >> > > m_local=m >> > > n_local=n >> > > dir+="seq/" >> > > >> > > QE=pet.Mat().createAIJ([[m_local,m],[n_local,n]]) >> > > L=pet.Mat().createAIJ([[m_local,m],[m_local,m]]) >> > > Mf=pet.Mat().createAIJ([[n_local,n],[n_local,n]]) >> > > >> > > viewerQE = pet.Viewer().createMPIIO(dir+"QE.dat", 'r', COMM_WORLD) >> > > QE.load(viewerQE) >> > > viewerL = pet.Viewer().createMPIIO(dir+"L.dat", 'r', COMM_WORLD) >> > > L.load(viewerL) >> > > viewerMf = pet.Viewer().createMPIIO(dir+"Mf.dat", 'r', COMM_WORLD) >> > > Mf.load(viewerMf) >> > > >> > > QE.assemble() >> > > L.assemble() >> > > Mf.assemble() >> > > >> > > KSPs = [] >> > > # Useful solvers (here to put options for computing a smart R) >> > > for Mat in [L,L.hermitianTranspose()]: >> > > KSP = pet.KSP().create() >> > > KSP.setOperators(Mat) >> > > KSP.setFromOptions() >> > > KSPs.append(KSP) >> > > >> > > class LHS_class: >> > > def mult(self,A,x,y): >> > > w=pet.Vec().createMPI([m_local,m],comm=COMM_WORLD) >> > > z=pet.Vec().createMPI([m_local,m],comm=COMM_WORLD) >> > > QE.mult(x,w) >> > > KSPs[0].solve(w,z) >> > > KSPs[1].solve(z,w) >> > > QE.multTranspose(w,y) >> > > >> > > # Matrix free operator >> > > LHS=pet.Mat() >> > > LHS.create(comm=COMM_WORLD) >> > > LHS.setSizes([[n_local,n],[n_local,n]]) >> > > LHS.setType(pet.Mat.Type.PYTHON) >> > > LHS.setPythonContext(LHS_class()) >> > > LHS.setUp() >> > > >> > > x, y = LHS.createVecs() >> > > x.set(1) >> > > LHS.mult(x,y) >> > > print(y.norm()) >> > > >> > > # Eigensolver >> > > EPS = slp.EPS(); EPS.create() >> > > EPS.setOperators(LHS,Mf) # Solve QE^T*L^-1H*L^-1*QE*f=sigma^2*Mf*f (cheaper than a proper SVD) >> > > EPS.setProblemType(slp.EPS.ProblemType.GHEP) # Specify that A is hermitian (by construction), but M is semi-positive >> > > EPS.setWhichEigenpairs(EPS.Which.LARGEST_MAGNITUDE) # Find largest eigenvalues >> > > EPS.setFromOptions() >> > > EPS.solve() >> > > >> > > >> > > Quentin CHEVALIER ? IA parcours recherche >> > > LadHyX - Ecole polytechnique >> > > __________ >> > > >> > > >> > > >> > > >> > > >> > > Quentin CHEVALIER ? IA parcours recherche >> > > LadHyX - Ecole polytechnique >> > > >> > > __________ >> > > >> > > >> > > >> > > On Thu, 5 May 2022 at 12:05, Jose E. Roman wrote: >> > > Your operator is not well formed. If you do this: >> > > >> > > x, y = LHS.createVecs() >> > > x.set(1) >> > > LHS.mult(x,y) >> > > y.view() >> > > >> > > you will see that the output is all zeros. That is why SLEPc complains that "Initial vector is zero or belongs to the deflation space". >> > > >> > > Jose >> > > >> > > >> > > > El 5 may 2022, a las 10:46, Quentin Chevalier escribi?: >> > > > >> > > > Just a quick amend on the previous statement ; the problem arises in >> > > > sequential and parallel. The MWE as is is provided for the parallel >> > > > case, but imposing m_local=m makes it go sequential. >> > > > >> > > > Cheers, >> > > > >> > > > Quentin CHEVALIER ? IA parcours recherche >> > > > LadHyX - Ecole polytechnique >> > > > __________ >> > > > >> > > > >> > > > >> > > > On Thu, 5 May 2022 at 10:34, Quentin Chevalier >> > > > wrote: >> > > >> >> > > >> Hello all and thanks for your great work in bringing this very helpful package to the community ! >> > > >> >> > > >> That said, I wouldn't need this mailing list if everything was running smoothly. I have a rather involved eigenvalue problem that I've been working on that's been throwing a mysterious error : >> > > >>> >> > > >>> petsc4py.PETSc.Error: error code 77 >> > > >>> >> > > >>> [1] EPSSolve() at /usr/local/slepc/src/eps/interface/epssolve.c:149 >> > > >>> [1] EPSSolve_KrylovSchur_Default() at /usr/local/slepc/src/eps/impls/krylov/krylovschur/krylovschur.c:289 >> > > >>> [1] EPSGetStartVector() at /usr/local/slepc/src/eps/interface/epssolve.c:824 >> > > >>> [1] Petsc has generated inconsistent data >> > > >>> [1] Initial vector is zero or belongs to the deflation space >> > > >> >> > > >> >> > > >> This problem occurs in parallel with two processors, using the petsc4py library using the dolfinx/dolfinx docker container. I have PETSc version 3.16.0, in complex mode, python 3, and I'm running all of that on a OpenSUSE Leap 15.2 machine (but I think the docker container has a Ubuntu OS). >> > > >> >> > > >> I wrote a minimal working example below, but I'm afraid the process for building the matrices is involved, so I decided to directly share the matrices instead : https://seminaris.polytechnique.fr/share/s/ryJ6L2nR4ketDwP >> > > >> >> > > >> They are in binary format, but inside the container I hope someone could reproduce my issue. A word on the structure and intent behind these matrices : >> > > >> >> > > >> QE is a diagonal rectangular real matrix. Think of it as some sort of preconditioner >> > > >> L is the least dense of them all, the only one that is complex, and in order to avoid inverting it I'm using two KSPs to compute solve problems on the fly >> > > >> Mf is a diagonal square real matrix, its on the right-hand side of the Generalised Hermitian Eigenvalue problem (I'm solving QE^H*L^-1H*L^-1*QE*x=lambda*Mf*x >> > > >> >> > > >> Full MWE is below : >> > > >> >> > > >> from petsc4py import PETSc as pet >> > > >> from slepc4py import SLEPc as slp >> > > >> from mpi4py.MPI import COMM_WORLD >> > > >> >> > > >> # Global sizes >> > > >> m_local=COMM_WORLD.rank*(490363-489780)+489780 >> > > >> n_local=COMM_WORLD.rank*(452259-451743)+451743 >> > > >> m=980143 >> > > >> n=904002 >> > > >> >> > > >> QE=pet.Mat().createAIJ([[m_local,m],[n_local,n]]) >> > > >> L=pet.Mat().createAIJ([[m_local,m],[m_local,m]]) >> > > >> Mf=pet.Mat().createAIJ([[n_local,n],[n_local,n]]) >> > > >> >> > > >> viewerQE = pet.Viewer().createMPIIO("QE.dat", 'r', COMM_WORLD) >> > > >> QE.load(viewerQE) >> > > >> viewerL = pet.Viewer().createMPIIO("L.dat", 'r', COMM_WORLD) >> > > >> L.load(viewerL) >> > > >> viewerMf = pet.Viewer().createMPIIO("Mf.dat", 'r', COMM_WORLD) >> > > >> Mf.load(viewerMf) >> > > >> >> > > >> QE.assemble() >> > > >> L.assemble() >> > > >> >> > > >> KSPs = [] >> > > >> # Useful solvers (here to put options for computing a smart R) >> > > >> for Mat in [L,L.hermitianTranspose()]: >> > > >> KSP = pet.KSP().create() >> > > >> KSP.setOperators(Mat) >> > > >> KSP.setFromOptions() >> > > >> KSPs.append(KSP) >> > > >> class LHS_class: >> > > >> def mult(self,A,x,y): >> > > >> w=pet.Vec().createMPI([m_local,m],comm=COMM_WORLD) >> > > >> z=pet.Vec().createMPI([m_local,m],comm=COMM_WORLD) >> > > >> QE.mult(x,w) >> > > >> KSPs[0].solve(w,z) >> > > >> KSPs[1].solve(z,w) >> > > >> QE.multTranspose(w,y) >> > > >> >> > > >> # Matrix free operator >> > > >> LHS=pet.Mat() >> > > >> LHS.create(comm=COMM_WORLD) >> > > >> LHS.setSizes([[n_local,n],[n_local,n]]) >> > > >> LHS.setType(pet.Mat.Type.PYTHON) >> > > >> LHS.setPythonContext(LHS_class()) >> > > >> LHS.setUp() >> > > >> >> > > >> # Eigensolver >> > > >> EPS = slp.EPS(); EPS.create() >> > > >> EPS.setOperators(LHS,Mf) # Solve QE^T*L^-1H*L^-1*QE*x=lambda*Mf*x >> > > >> EPS.setProblemType(slp.EPS.ProblemType.GHEP) # Specify that A is hermitian (by construction), and B is semi-positive >> > > >> EPS.setWhichEigenpairs(EPS.Which.LARGEST_MAGNITUDE) # Find largest eigenvalues >> > > >> EPS.setFromOptions() >> > > >> EPS.solve() >> > > >> >> > > >> Quentin CHEVALIER ? IA parcours recherche >> > > >> >> > > >> LadHyX - Ecole polytechnique >> > > >> >> > > >> __________ >> > > >> > > >> > > > > > -- > What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. > -- Norbert Wiener > > https://www.cse.buffalo.edu/~knepley/ From jroman at dsic.upv.es Fri May 6 08:55:18 2022 From: jroman at dsic.upv.es (Jose E. Roman) Date: Fri, 6 May 2022 15:55:18 +0200 Subject: [petsc-users] Mysterious error code 77 In-Reply-To: References: <3AFCE82E-EC31-4DBB-A80E-D5499D2076EB@dsic.upv.es> Message-ID: <824E2E14-4543-418A-9289-C6EF84105A04@dsic.upv.es> I used your script. The only change I did was dir+="seq/" instead of dir+="par/" That is, read the sequential matrices. I would suggest writing matrix files from a sequential program. I assume that matrix files are for testing only, for production runs you should generate the matrices and solve in the same program, not save to file. The original script is doing EPS.setWhichEigenpairs(EPS.Which.LARGEST_MAGNITUDE). This computes largest eigenvalues. You cannot do ST.setType('sinvert') in this example. This would compute eigenvalues close to zero. Read chapter 3 of the manual to understand how shift-and-invert works. Jose > El 6 may 2022, a las 15:46, Quentin Chevalier escribi?: > > @Matthew, I'm sure you've noticed the L.solve inside the LHS class. > That's preventing me from going AIJ ; I would have to invert L, which > sounds like a bad idea. > > @Jose, I've had troubles reading the matrix files in parallel when > they were created in sequential or parallel with a different number of > processors - that's why I resorted to 2, and hard-coded the local > sizes. If I naively try to create AIJ with global sizes only, I get > Incompatible sizes every time. Would you care to enlighten me as to > how to read the same matrix files with an arbitrary number of > processors ? > > More importantly, you managed to run the MWE sent on May 5th without > any change uin the same config as me and couldn't reproduce a code 77 > ? > > You're also telling me that Mf needs to be factorised with MUMPS, but > that LHS can't. So did you specify an additional solver/preconditioner > inside the MWE or did it work as is ? > > Cheers, > > Quentin CHEVALIER ? IA parcours recherche > LadHyX - Ecole polytechnique > __________ > > > On Fri, 6 May 2022 at 15:35, Matthew Knepley wrote: >> >> On Fri, May 6, 2022 at 9:28 AM Quentin Chevalier wrote: >>> >>> Sorry for forgetting the list. Making two matrices was more of a precaution then a carefully thought strategy. >>> >>> It would seem the MWE as I provided it above (with a setDimensions to reduce calculation time) does work in sequential and gives the eigenvalue 118897.88711586884. >>> >>> I had a working MUMPS solver config for a similar problem so I simply put it there. This gives the following code. >>> >>> However, running it even in sequential gives a "MatSolverType mumps does not support matrix type python" error. >>> petsc4py.PETSc.Error: error code 92 >>> [0] EPSSolve() at /usr/local/slepc/src/eps/interface/epssolve.c:136 >>> [0] EPSSetUp() at /usr/local/slepc/src/eps/interface/epssetup.c:350 >>> [0] STSetUp() at /usr/local/slepc/src/sys/classes/st/interface/stsolve.c:582 >>> [0] STSetUp_Sinvert() at /usr/local/slepc/src/sys/classes/st/impls/sinvert/sinvert.c:123 >>> [0] KSPSetUp() at /usr/local/petsc/src/ksp/ksp/interface/itfunc.c:408 >>> [0] PCSetUp() at /usr/local/petsc/src/ksp/pc/interface/precon.c:1016 >>> [0] PCSetUp_LU() at /usr/local/petsc/src/ksp/pc/impls/factor/lu/lu.c:82 >>> [0] MatGetFactor() at /usr/local/petsc/src/mat/interface/matrix.c:4779 >>> [0] See https://petsc.org/release/overview/linear_solve_table/ for possible LU and Cholesky solvers >>> [0] MatSolverType mumps does not support matrix type python >> >> >> You would need type AIJ for MUMPS. >> >> Thanks, >> >> Matt >> >>> >>> Did I do something wrong ? I'm a bit confused by your FAQ entry, I never had a problem with this configuration in parallel. >>> >>> from petsc4py import PETSc as pet >>> from slepc4py import SLEPc as slp >>> from mpi4py.MPI import COMM_WORLD >>> >>> dir="./sanity_check_mats/" >>> >>> # Global sizes >>> m=980143; m_local=m >>> n=904002; n_local=n >>> if COMM_WORLD.size>1: >>> m_local=COMM_WORLD.rank*(490363-489780)+489780 >>> n_local=COMM_WORLD.rank*(452259-451743)+451743 >>> dir+="par/" >>> else: >>> m_local=m >>> n_local=n >>> dir+="seq/" >>> >>> QE=pet.Mat().createAIJ([[m_local,m],[n_local,n]]) >>> L=pet.Mat().createAIJ([[m_local,m],[m_local,m]]) >>> Mf=pet.Mat().createAIJ([[n_local,n],[n_local,n]]) >>> >>> viewerQE = pet.Viewer().createMPIIO(dir+"QE.dat", 'r', COMM_WORLD) >>> QE.load(viewerQE) >>> viewerL = pet.Viewer().createMPIIO(dir+"L.dat", 'r', COMM_WORLD) >>> L.load(viewerL) >>> viewerMf = pet.Viewer().createMPIIO(dir+"Mf.dat", 'r', COMM_WORLD) >>> Mf.load(viewerMf) >>> >>> QE.assemble() >>> L.assemble() >>> Mf.assemble() >>> >>> KSPs = [] >>> # Useful solvers (here to put options for computing a smart R) >>> for Mat in [L,L.hermitianTranspose()]: >>> KSP = pet.KSP().create() >>> KSP.setOperators(Mat) >>> KSP.setFromOptions() >>> KSPs.append(KSP) >>> class LHS_class: >>> def mult(self,A,x,y): >>> w=pet.Vec().createMPI([m_local,m],comm=COMM_WORLD) >>> z=pet.Vec().createMPI([m_local,m],comm=COMM_WORLD) >>> QE.mult(x,w) >>> KSPs[0].solve(w,z) >>> KSPs[1].solve(z,w) >>> QE.multTranspose(w,y) >>> >>> # Matrix free operator >>> LHS=pet.Mat() >>> LHS.create(comm=COMM_WORLD) >>> LHS.setSizes([[n_local,n],[n_local,n]]) >>> LHS.setType(pet.Mat.Type.PYTHON) >>> LHS.setPythonContext(LHS_class()) >>> LHS.setUp() >>> >>> x, y = LHS.createVecs() >>> x.set(1) >>> LHS.mult(x,y) >>> print(y.norm()) >>> >>> # Eigensolver >>> EPS = slp.EPS(); EPS.create() >>> EPS.setOperators(LHS,Mf) # Solve QE^T*L^-1H*L^-1*QE*f=sigma^2*Mf*f (cheaper than a proper SVD) >>> EPS.setProblemType(slp.EPS.ProblemType.GHEP) # Specify that A is hermitian (by construction), but M is semi-positive >>> EPS.setWhichEigenpairs(EPS.Which.LARGEST_MAGNITUDE) # Find largest eigenvalues >>> # Spectral transform >>> ST = EPS.getST(); ST.setType('sinvert') >>> # Krylov subspace >>> KSP = ST.getKSP(); KSP.setType('preonly') >>> # Preconditioner >>> PC = KSP.getPC(); PC.setType('lu') >>> PC.setFactorSolverType('mumps') >>> EPS.setDimensions(1,5) >>> EPS.setFromOptions() >>> EPS.solve() >>> print(EPS.getEigenvalue(0)) >>> >>> Quentin CHEVALIER ? IA parcours recherche >>> LadHyX - Ecole polytechnique >>> __________ >>> >>> >>> >>> On Fri, 6 May 2022 at 14:47, Jose E. Roman wrote: >>>> >>>> Please respond to the list. >>>> >>>> Why would you need different matrices for sequential and parallel? PETSc takes care of loading matrices from binary files in parallel codes. If I use the 'seq' matrices for both sequential and parallel runs it works. I get the eigenvalue 128662.745858 >>>> >>>> Take into account that in parallel you need to use a parallel linear solver, e.g., configure PETSc with MUMPS. See the FAQ #10 https://slepc.upv.es/documentation/faq.htm#faq10 >>>> >>>> Jose >>>> >>>>> El 5 may 2022, a las 14:57, Quentin Chevalier escribi?: >>>>> >>>>> Thank you for your answer, Jose. >>>>> >>>>> It would appear my problem is slightly more complicated than it appears. With slight modification of the original MWE to account for functioning in serial or parallel, and computing the matrices in either sequential or parallel (apologies, it's a large file), and including a slightly modified version of your test case, I obtain two different results in serial and parallel (commands python3 MWE.py and mpirun -n 2 python3 MWE.py). >>>>> >>>>> Serial gives my a finite norm (around 28) and parallel gives me two norms of 0 and a code 77. >>>>> >>>>> This is still a problem as I would really like my code to be parallel. I saved my matrices using : >>>>> viewerQE = pet.Viewer().createMPIIO("QE.dat", 'w', COMM_WORLD) >>>>> QE.view(viewerQE) >>>>> viewerL = pet.Viewer().createMPIIO("L.dat", 'w', COMM_WORLD) >>>>> L.view(viewerL) >>>>> viewerMq = pet.Viewer().createMPIIO("Mq.dat", 'w', COMM_WORLD) >>>>> Mq.view(viewerMq) >>>>> viewerMf = pet.Viewer().createMPIIO("Mf.dat", 'w', COMM_WORLD) >>>>> Mf.view(viewerMf) >>>>> >>>>> New MWE (also attached for convenience) : >>>>> from petsc4py import PETSc as pet >>>>> from slepc4py import SLEPc as slp >>>>> from mpi4py.MPI import COMM_WORLD >>>>> >>>>> dir="./mats/" >>>>> >>>>> # Global sizes >>>>> m=980143; m_local=m >>>>> n=904002; n_local=n >>>>> if COMM_WORLD.size>1: >>>>> m_local=COMM_WORLD.rank*(490363-489780)+489780 >>>>> n_local=COMM_WORLD.rank*(452259-451743)+451743 >>>>> dir+="par/" >>>>> else: >>>>> m_local=m >>>>> n_local=n >>>>> dir+="seq/" >>>>> >>>>> QE=pet.Mat().createAIJ([[m_local,m],[n_local,n]]) >>>>> L=pet.Mat().createAIJ([[m_local,m],[m_local,m]]) >>>>> Mf=pet.Mat().createAIJ([[n_local,n],[n_local,n]]) >>>>> >>>>> viewerQE = pet.Viewer().createMPIIO(dir+"QE.dat", 'r', COMM_WORLD) >>>>> QE.load(viewerQE) >>>>> viewerL = pet.Viewer().createMPIIO(dir+"L.dat", 'r', COMM_WORLD) >>>>> L.load(viewerL) >>>>> viewerMf = pet.Viewer().createMPIIO(dir+"Mf.dat", 'r', COMM_WORLD) >>>>> Mf.load(viewerMf) >>>>> >>>>> QE.assemble() >>>>> L.assemble() >>>>> Mf.assemble() >>>>> >>>>> KSPs = [] >>>>> # Useful solvers (here to put options for computing a smart R) >>>>> for Mat in [L,L.hermitianTranspose()]: >>>>> KSP = pet.KSP().create() >>>>> KSP.setOperators(Mat) >>>>> KSP.setFromOptions() >>>>> KSPs.append(KSP) >>>>> >>>>> class LHS_class: >>>>> def mult(self,A,x,y): >>>>> w=pet.Vec().createMPI([m_local,m],comm=COMM_WORLD) >>>>> z=pet.Vec().createMPI([m_local,m],comm=COMM_WORLD) >>>>> QE.mult(x,w) >>>>> KSPs[0].solve(w,z) >>>>> KSPs[1].solve(z,w) >>>>> QE.multTranspose(w,y) >>>>> >>>>> # Matrix free operator >>>>> LHS=pet.Mat() >>>>> LHS.create(comm=COMM_WORLD) >>>>> LHS.setSizes([[n_local,n],[n_local,n]]) >>>>> LHS.setType(pet.Mat.Type.PYTHON) >>>>> LHS.setPythonContext(LHS_class()) >>>>> LHS.setUp() >>>>> >>>>> x, y = LHS.createVecs() >>>>> x.set(1) >>>>> LHS.mult(x,y) >>>>> print(y.norm()) >>>>> >>>>> # Eigensolver >>>>> EPS = slp.EPS(); EPS.create() >>>>> EPS.setOperators(LHS,Mf) # Solve QE^T*L^-1H*L^-1*QE*f=sigma^2*Mf*f (cheaper than a proper SVD) >>>>> EPS.setProblemType(slp.EPS.ProblemType.GHEP) # Specify that A is hermitian (by construction), but M is semi-positive >>>>> EPS.setWhichEigenpairs(EPS.Which.LARGEST_MAGNITUDE) # Find largest eigenvalues >>>>> EPS.setFromOptions() >>>>> EPS.solve() >>>>> >>>>> >>>>> Quentin CHEVALIER ? IA parcours recherche >>>>> LadHyX - Ecole polytechnique >>>>> __________ >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> Quentin CHEVALIER ? IA parcours recherche >>>>> LadHyX - Ecole polytechnique >>>>> >>>>> __________ >>>>> >>>>> >>>>> >>>>> On Thu, 5 May 2022 at 12:05, Jose E. Roman wrote: >>>>> Your operator is not well formed. If you do this: >>>>> >>>>> x, y = LHS.createVecs() >>>>> x.set(1) >>>>> LHS.mult(x,y) >>>>> y.view() >>>>> >>>>> you will see that the output is all zeros. That is why SLEPc complains that "Initial vector is zero or belongs to the deflation space". >>>>> >>>>> Jose >>>>> >>>>> >>>>>> El 5 may 2022, a las 10:46, Quentin Chevalier escribi?: >>>>>> >>>>>> Just a quick amend on the previous statement ; the problem arises in >>>>>> sequential and parallel. The MWE as is is provided for the parallel >>>>>> case, but imposing m_local=m makes it go sequential. >>>>>> >>>>>> Cheers, >>>>>> >>>>>> Quentin CHEVALIER ? IA parcours recherche >>>>>> LadHyX - Ecole polytechnique >>>>>> __________ >>>>>> >>>>>> >>>>>> >>>>>> On Thu, 5 May 2022 at 10:34, Quentin Chevalier >>>>>> wrote: >>>>>>> >>>>>>> Hello all and thanks for your great work in bringing this very helpful package to the community ! >>>>>>> >>>>>>> That said, I wouldn't need this mailing list if everything was running smoothly. I have a rather involved eigenvalue problem that I've been working on that's been throwing a mysterious error : >>>>>>>> >>>>>>>> petsc4py.PETSc.Error: error code 77 >>>>>>>> >>>>>>>> [1] EPSSolve() at /usr/local/slepc/src/eps/interface/epssolve.c:149 >>>>>>>> [1] EPSSolve_KrylovSchur_Default() at /usr/local/slepc/src/eps/impls/krylov/krylovschur/krylovschur.c:289 >>>>>>>> [1] EPSGetStartVector() at /usr/local/slepc/src/eps/interface/epssolve.c:824 >>>>>>>> [1] Petsc has generated inconsistent data >>>>>>>> [1] Initial vector is zero or belongs to the deflation space >>>>>>> >>>>>>> >>>>>>> This problem occurs in parallel with two processors, using the petsc4py library using the dolfinx/dolfinx docker container. I have PETSc version 3.16.0, in complex mode, python 3, and I'm running all of that on a OpenSUSE Leap 15.2 machine (but I think the docker container has a Ubuntu OS). >>>>>>> >>>>>>> I wrote a minimal working example below, but I'm afraid the process for building the matrices is involved, so I decided to directly share the matrices instead : https://seminaris.polytechnique.fr/share/s/ryJ6L2nR4ketDwP >>>>>>> >>>>>>> They are in binary format, but inside the container I hope someone could reproduce my issue. A word on the structure and intent behind these matrices : >>>>>>> >>>>>>> QE is a diagonal rectangular real matrix. Think of it as some sort of preconditioner >>>>>>> L is the least dense of them all, the only one that is complex, and in order to avoid inverting it I'm using two KSPs to compute solve problems on the fly >>>>>>> Mf is a diagonal square real matrix, its on the right-hand side of the Generalised Hermitian Eigenvalue problem (I'm solving QE^H*L^-1H*L^-1*QE*x=lambda*Mf*x >>>>>>> >>>>>>> Full MWE is below : >>>>>>> >>>>>>> from petsc4py import PETSc as pet >>>>>>> from slepc4py import SLEPc as slp >>>>>>> from mpi4py.MPI import COMM_WORLD >>>>>>> >>>>>>> # Global sizes >>>>>>> m_local=COMM_WORLD.rank*(490363-489780)+489780 >>>>>>> n_local=COMM_WORLD.rank*(452259-451743)+451743 >>>>>>> m=980143 >>>>>>> n=904002 >>>>>>> >>>>>>> QE=pet.Mat().createAIJ([[m_local,m],[n_local,n]]) >>>>>>> L=pet.Mat().createAIJ([[m_local,m],[m_local,m]]) >>>>>>> Mf=pet.Mat().createAIJ([[n_local,n],[n_local,n]]) >>>>>>> >>>>>>> viewerQE = pet.Viewer().createMPIIO("QE.dat", 'r', COMM_WORLD) >>>>>>> QE.load(viewerQE) >>>>>>> viewerL = pet.Viewer().createMPIIO("L.dat", 'r', COMM_WORLD) >>>>>>> L.load(viewerL) >>>>>>> viewerMf = pet.Viewer().createMPIIO("Mf.dat", 'r', COMM_WORLD) >>>>>>> Mf.load(viewerMf) >>>>>>> >>>>>>> QE.assemble() >>>>>>> L.assemble() >>>>>>> >>>>>>> KSPs = [] >>>>>>> # Useful solvers (here to put options for computing a smart R) >>>>>>> for Mat in [L,L.hermitianTranspose()]: >>>>>>> KSP = pet.KSP().create() >>>>>>> KSP.setOperators(Mat) >>>>>>> KSP.setFromOptions() >>>>>>> KSPs.append(KSP) >>>>>>> class LHS_class: >>>>>>> def mult(self,A,x,y): >>>>>>> w=pet.Vec().createMPI([m_local,m],comm=COMM_WORLD) >>>>>>> z=pet.Vec().createMPI([m_local,m],comm=COMM_WORLD) >>>>>>> QE.mult(x,w) >>>>>>> KSPs[0].solve(w,z) >>>>>>> KSPs[1].solve(z,w) >>>>>>> QE.multTranspose(w,y) >>>>>>> >>>>>>> # Matrix free operator >>>>>>> LHS=pet.Mat() >>>>>>> LHS.create(comm=COMM_WORLD) >>>>>>> LHS.setSizes([[n_local,n],[n_local,n]]) >>>>>>> LHS.setType(pet.Mat.Type.PYTHON) >>>>>>> LHS.setPythonContext(LHS_class()) >>>>>>> LHS.setUp() >>>>>>> >>>>>>> # Eigensolver >>>>>>> EPS = slp.EPS(); EPS.create() >>>>>>> EPS.setOperators(LHS,Mf) # Solve QE^T*L^-1H*L^-1*QE*x=lambda*Mf*x >>>>>>> EPS.setProblemType(slp.EPS.ProblemType.GHEP) # Specify that A is hermitian (by construction), and B is semi-positive >>>>>>> EPS.setWhichEigenpairs(EPS.Which.LARGEST_MAGNITUDE) # Find largest eigenvalues >>>>>>> EPS.setFromOptions() >>>>>>> EPS.solve() >>>>>>> >>>>>>> Quentin CHEVALIER ? IA parcours recherche >>>>>>> >>>>>>> LadHyX - Ecole polytechnique >>>>>>> >>>>>>> __________ >>>>> >>>>> >>>> >> >> >> >> -- >> What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. >> -- Norbert Wiener >> >> https://www.cse.buffalo.edu/~knepley/ From quentin.chevalier at polytechnique.edu Fri May 6 09:43:31 2022 From: quentin.chevalier at polytechnique.edu (Quentin Chevalier) Date: Fri, 6 May 2022 16:43:31 +0200 Subject: [petsc-users] Mysterious error code 77 In-Reply-To: <824E2E14-4543-418A-9289-C6EF84105A04@dsic.upv.es> References: <3AFCE82E-EC31-4DBB-A80E-D5499D2076EB@dsic.upv.es> <824E2E14-4543-418A-9289-C6EF84105A04@dsic.upv.es> Message-ID: Ok thank you for the clarification. It would appear my MWE was poorly chosen indeed, I got it to work too. I still get code 77 inside my main code, I'll keep you posted. Indeed the matrices are stored for testing purposes only. Of course you are completely right about shift invert, I was confusing with another case where I had set a target eigenvalue. Thanks for your time, Quentin CHEVALIER ? IA parcours recherche LadHyX - Ecole polytechnique __________ On Fri, 6 May 2022 at 15:55, Jose E. Roman wrote: > > I used your script. The only change I did was > > dir+="seq/" > > instead of > > dir+="par/" > > That is, read the sequential matrices. I would suggest writing matrix files from a sequential program. I assume that matrix files are for testing only, for production runs you should generate the matrices and solve in the same program, not save to file. > > The original script is doing EPS.setWhichEigenpairs(EPS.Which.LARGEST_MAGNITUDE). This computes largest eigenvalues. > You cannot do ST.setType('sinvert') in this example. This would compute eigenvalues close to zero. Read chapter 3 of the manual to understand how shift-and-invert works. > > Jose > > > > El 6 may 2022, a las 15:46, Quentin Chevalier escribi?: > > > > @Matthew, I'm sure you've noticed the L.solve inside the LHS class. > > That's preventing me from going AIJ ; I would have to invert L, which > > sounds like a bad idea. > > > > @Jose, I've had troubles reading the matrix files in parallel when > > they were created in sequential or parallel with a different number of > > processors - that's why I resorted to 2, and hard-coded the local > > sizes. If I naively try to create AIJ with global sizes only, I get > > Incompatible sizes every time. Would you care to enlighten me as to > > how to read the same matrix files with an arbitrary number of > > processors ? > > > > More importantly, you managed to run the MWE sent on May 5th without > > any change uin the same config as me and couldn't reproduce a code 77 > > ? > > > > You're also telling me that Mf needs to be factorised with MUMPS, but > > that LHS can't. So did you specify an additional solver/preconditioner > > inside the MWE or did it work as is ? > > > > Cheers, > > > > Quentin CHEVALIER ? IA parcours recherche > > LadHyX - Ecole polytechnique > > __________ > > > > > > On Fri, 6 May 2022 at 15:35, Matthew Knepley wrote: > >> > >> On Fri, May 6, 2022 at 9:28 AM Quentin Chevalier wrote: > >>> > >>> Sorry for forgetting the list. Making two matrices was more of a precaution then a carefully thought strategy. > >>> > >>> It would seem the MWE as I provided it above (with a setDimensions to reduce calculation time) does work in sequential and gives the eigenvalue 118897.88711586884. > >>> > >>> I had a working MUMPS solver config for a similar problem so I simply put it there. This gives the following code. > >>> > >>> However, running it even in sequential gives a "MatSolverType mumps does not support matrix type python" error. > >>> petsc4py.PETSc.Error: error code 92 > >>> [0] EPSSolve() at /usr/local/slepc/src/eps/interface/epssolve.c:136 > >>> [0] EPSSetUp() at /usr/local/slepc/src/eps/interface/epssetup.c:350 > >>> [0] STSetUp() at /usr/local/slepc/src/sys/classes/st/interface/stsolve.c:582 > >>> [0] STSetUp_Sinvert() at /usr/local/slepc/src/sys/classes/st/impls/sinvert/sinvert.c:123 > >>> [0] KSPSetUp() at /usr/local/petsc/src/ksp/ksp/interface/itfunc.c:408 > >>> [0] PCSetUp() at /usr/local/petsc/src/ksp/pc/interface/precon.c:1016 > >>> [0] PCSetUp_LU() at /usr/local/petsc/src/ksp/pc/impls/factor/lu/lu.c:82 > >>> [0] MatGetFactor() at /usr/local/petsc/src/mat/interface/matrix.c:4779 > >>> [0] See https://petsc.org/release/overview/linear_solve_table/ for possible LU and Cholesky solvers > >>> [0] MatSolverType mumps does not support matrix type python > >> > >> > >> You would need type AIJ for MUMPS. > >> > >> Thanks, > >> > >> Matt > >> > >>> > >>> Did I do something wrong ? I'm a bit confused by your FAQ entry, I never had a problem with this configuration in parallel. > >>> > >>> from petsc4py import PETSc as pet > >>> from slepc4py import SLEPc as slp > >>> from mpi4py.MPI import COMM_WORLD > >>> > >>> dir="./sanity_check_mats/" > >>> > >>> # Global sizes > >>> m=980143; m_local=m > >>> n=904002; n_local=n > >>> if COMM_WORLD.size>1: > >>> m_local=COMM_WORLD.rank*(490363-489780)+489780 > >>> n_local=COMM_WORLD.rank*(452259-451743)+451743 > >>> dir+="par/" > >>> else: > >>> m_local=m > >>> n_local=n > >>> dir+="seq/" > >>> > >>> QE=pet.Mat().createAIJ([[m_local,m],[n_local,n]]) > >>> L=pet.Mat().createAIJ([[m_local,m],[m_local,m]]) > >>> Mf=pet.Mat().createAIJ([[n_local,n],[n_local,n]]) > >>> > >>> viewerQE = pet.Viewer().createMPIIO(dir+"QE.dat", 'r', COMM_WORLD) > >>> QE.load(viewerQE) > >>> viewerL = pet.Viewer().createMPIIO(dir+"L.dat", 'r', COMM_WORLD) > >>> L.load(viewerL) > >>> viewerMf = pet.Viewer().createMPIIO(dir+"Mf.dat", 'r', COMM_WORLD) > >>> Mf.load(viewerMf) > >>> > >>> QE.assemble() > >>> L.assemble() > >>> Mf.assemble() > >>> > >>> KSPs = [] > >>> # Useful solvers (here to put options for computing a smart R) > >>> for Mat in [L,L.hermitianTranspose()]: > >>> KSP = pet.KSP().create() > >>> KSP.setOperators(Mat) > >>> KSP.setFromOptions() > >>> KSPs.append(KSP) > >>> class LHS_class: > >>> def mult(self,A,x,y): > >>> w=pet.Vec().createMPI([m_local,m],comm=COMM_WORLD) > >>> z=pet.Vec().createMPI([m_local,m],comm=COMM_WORLD) > >>> QE.mult(x,w) > >>> KSPs[0].solve(w,z) > >>> KSPs[1].solve(z,w) > >>> QE.multTranspose(w,y) > >>> > >>> # Matrix free operator > >>> LHS=pet.Mat() > >>> LHS.create(comm=COMM_WORLD) > >>> LHS.setSizes([[n_local,n],[n_local,n]]) > >>> LHS.setType(pet.Mat.Type.PYTHON) > >>> LHS.setPythonContext(LHS_class()) > >>> LHS.setUp() > >>> > >>> x, y = LHS.createVecs() > >>> x.set(1) > >>> LHS.mult(x,y) > >>> print(y.norm()) > >>> > >>> # Eigensolver > >>> EPS = slp.EPS(); EPS.create() > >>> EPS.setOperators(LHS,Mf) # Solve QE^T*L^-1H*L^-1*QE*f=sigma^2*Mf*f (cheaper than a proper SVD) > >>> EPS.setProblemType(slp.EPS.ProblemType.GHEP) # Specify that A is hermitian (by construction), but M is semi-positive > >>> EPS.setWhichEigenpairs(EPS.Which.LARGEST_MAGNITUDE) # Find largest eigenvalues > >>> # Spectral transform > >>> ST = EPS.getST(); ST.setType('sinvert') > >>> # Krylov subspace > >>> KSP = ST.getKSP(); KSP.setType('preonly') > >>> # Preconditioner > >>> PC = KSP.getPC(); PC.setType('lu') > >>> PC.setFactorSolverType('mumps') > >>> EPS.setDimensions(1,5) > >>> EPS.setFromOptions() > >>> EPS.solve() > >>> print(EPS.getEigenvalue(0)) > >>> > >>> Quentin CHEVALIER ? IA parcours recherche > >>> LadHyX - Ecole polytechnique > >>> __________ > >>> > >>> > >>> > >>> On Fri, 6 May 2022 at 14:47, Jose E. Roman wrote: > >>>> > >>>> Please respond to the list. > >>>> > >>>> Why would you need different matrices for sequential and parallel? PETSc takes care of loading matrices from binary files in parallel codes. If I use the 'seq' matrices for both sequential and parallel runs it works. I get the eigenvalue 128662.745858 > >>>> > >>>> Take into account that in parallel you need to use a parallel linear solver, e.g., configure PETSc with MUMPS. See the FAQ #10 https://slepc.upv.es/documentation/faq.htm#faq10 > >>>> > >>>> Jose > >>>> > >>>>> El 5 may 2022, a las 14:57, Quentin Chevalier escribi?: > >>>>> > >>>>> Thank you for your answer, Jose. > >>>>> > >>>>> It would appear my problem is slightly more complicated than it appears. With slight modification of the original MWE to account for functioning in serial or parallel, and computing the matrices in either sequential or parallel (apologies, it's a large file), and including a slightly modified version of your test case, I obtain two different results in serial and parallel (commands python3 MWE.py and mpirun -n 2 python3 MWE.py). > >>>>> > >>>>> Serial gives my a finite norm (around 28) and parallel gives me two norms of 0 and a code 77. > >>>>> > >>>>> This is still a problem as I would really like my code to be parallel. I saved my matrices using : > >>>>> viewerQE = pet.Viewer().createMPIIO("QE.dat", 'w', COMM_WORLD) > >>>>> QE.view(viewerQE) > >>>>> viewerL = pet.Viewer().createMPIIO("L.dat", 'w', COMM_WORLD) > >>>>> L.view(viewerL) > >>>>> viewerMq = pet.Viewer().createMPIIO("Mq.dat", 'w', COMM_WORLD) > >>>>> Mq.view(viewerMq) > >>>>> viewerMf = pet.Viewer().createMPIIO("Mf.dat", 'w', COMM_WORLD) > >>>>> Mf.view(viewerMf) > >>>>> > >>>>> New MWE (also attached for convenience) : > >>>>> from petsc4py import PETSc as pet > >>>>> from slepc4py import SLEPc as slp > >>>>> from mpi4py.MPI import COMM_WORLD > >>>>> > >>>>> dir="./mats/" > >>>>> > >>>>> # Global sizes > >>>>> m=980143; m_local=m > >>>>> n=904002; n_local=n > >>>>> if COMM_WORLD.size>1: > >>>>> m_local=COMM_WORLD.rank*(490363-489780)+489780 > >>>>> n_local=COMM_WORLD.rank*(452259-451743)+451743 > >>>>> dir+="par/" > >>>>> else: > >>>>> m_local=m > >>>>> n_local=n > >>>>> dir+="seq/" > >>>>> > >>>>> QE=pet.Mat().createAIJ([[m_local,m],[n_local,n]]) > >>>>> L=pet.Mat().createAIJ([[m_local,m],[m_local,m]]) > >>>>> Mf=pet.Mat().createAIJ([[n_local,n],[n_local,n]]) > >>>>> > >>>>> viewerQE = pet.Viewer().createMPIIO(dir+"QE.dat", 'r', COMM_WORLD) > >>>>> QE.load(viewerQE) > >>>>> viewerL = pet.Viewer().createMPIIO(dir+"L.dat", 'r', COMM_WORLD) > >>>>> L.load(viewerL) > >>>>> viewerMf = pet.Viewer().createMPIIO(dir+"Mf.dat", 'r', COMM_WORLD) > >>>>> Mf.load(viewerMf) > >>>>> > >>>>> QE.assemble() > >>>>> L.assemble() > >>>>> Mf.assemble() > >>>>> > >>>>> KSPs = [] > >>>>> # Useful solvers (here to put options for computing a smart R) > >>>>> for Mat in [L,L.hermitianTranspose()]: > >>>>> KSP = pet.KSP().create() > >>>>> KSP.setOperators(Mat) > >>>>> KSP.setFromOptions() > >>>>> KSPs.append(KSP) > >>>>> > >>>>> class LHS_class: > >>>>> def mult(self,A,x,y): > >>>>> w=pet.Vec().createMPI([m_local,m],comm=COMM_WORLD) > >>>>> z=pet.Vec().createMPI([m_local,m],comm=COMM_WORLD) > >>>>> QE.mult(x,w) > >>>>> KSPs[0].solve(w,z) > >>>>> KSPs[1].solve(z,w) > >>>>> QE.multTranspose(w,y) > >>>>> > >>>>> # Matrix free operator > >>>>> LHS=pet.Mat() > >>>>> LHS.create(comm=COMM_WORLD) > >>>>> LHS.setSizes([[n_local,n],[n_local,n]]) > >>>>> LHS.setType(pet.Mat.Type.PYTHON) > >>>>> LHS.setPythonContext(LHS_class()) > >>>>> LHS.setUp() > >>>>> > >>>>> x, y = LHS.createVecs() > >>>>> x.set(1) > >>>>> LHS.mult(x,y) > >>>>> print(y.norm()) > >>>>> > >>>>> # Eigensolver > >>>>> EPS = slp.EPS(); EPS.create() > >>>>> EPS.setOperators(LHS,Mf) # Solve QE^T*L^-1H*L^-1*QE*f=sigma^2*Mf*f (cheaper than a proper SVD) > >>>>> EPS.setProblemType(slp.EPS.ProblemType.GHEP) # Specify that A is hermitian (by construction), but M is semi-positive > >>>>> EPS.setWhichEigenpairs(EPS.Which.LARGEST_MAGNITUDE) # Find largest eigenvalues > >>>>> EPS.setFromOptions() > >>>>> EPS.solve() > >>>>> > >>>>> > >>>>> Quentin CHEVALIER ? IA parcours recherche > >>>>> LadHyX - Ecole polytechnique > >>>>> __________ > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> Quentin CHEVALIER ? IA parcours recherche > >>>>> LadHyX - Ecole polytechnique > >>>>> > >>>>> __________ > >>>>> > >>>>> > >>>>> > >>>>> On Thu, 5 May 2022 at 12:05, Jose E. Roman wrote: > >>>>> Your operator is not well formed. If you do this: > >>>>> > >>>>> x, y = LHS.createVecs() > >>>>> x.set(1) > >>>>> LHS.mult(x,y) > >>>>> y.view() > >>>>> > >>>>> you will see that the output is all zeros. That is why SLEPc complains that "Initial vector is zero or belongs to the deflation space". > >>>>> > >>>>> Jose > >>>>> > >>>>> > >>>>>> El 5 may 2022, a las 10:46, Quentin Chevalier escribi?: > >>>>>> > >>>>>> Just a quick amend on the previous statement ; the problem arises in > >>>>>> sequential and parallel. The MWE as is is provided for the parallel > >>>>>> case, but imposing m_local=m makes it go sequential. > >>>>>> > >>>>>> Cheers, > >>>>>> > >>>>>> Quentin CHEVALIER ? IA parcours recherche > >>>>>> LadHyX - Ecole polytechnique > >>>>>> __________ > >>>>>> > >>>>>> > >>>>>> > >>>>>> On Thu, 5 May 2022 at 10:34, Quentin Chevalier > >>>>>> wrote: > >>>>>>> > >>>>>>> Hello all and thanks for your great work in bringing this very helpful package to the community ! > >>>>>>> > >>>>>>> That said, I wouldn't need this mailing list if everything was running smoothly. I have a rather involved eigenvalue problem that I've been working on that's been throwing a mysterious error : > >>>>>>>> > >>>>>>>> petsc4py.PETSc.Error: error code 77 > >>>>>>>> > >>>>>>>> [1] EPSSolve() at /usr/local/slepc/src/eps/interface/epssolve.c:149 > >>>>>>>> [1] EPSSolve_KrylovSchur_Default() at /usr/local/slepc/src/eps/impls/krylov/krylovschur/krylovschur.c:289 > >>>>>>>> [1] EPSGetStartVector() at /usr/local/slepc/src/eps/interface/epssolve.c:824 > >>>>>>>> [1] Petsc has generated inconsistent data > >>>>>>>> [1] Initial vector is zero or belongs to the deflation space > >>>>>>> > >>>>>>> > >>>>>>> This problem occurs in parallel with two processors, using the petsc4py library using the dolfinx/dolfinx docker container. I have PETSc version 3.16.0, in complex mode, python 3, and I'm running all of that on a OpenSUSE Leap 15.2 machine (but I think the docker container has a Ubuntu OS). > >>>>>>> > >>>>>>> I wrote a minimal working example below, but I'm afraid the process for building the matrices is involved, so I decided to directly share the matrices instead : https://seminaris.polytechnique.fr/share/s/ryJ6L2nR4ketDwP > >>>>>>> > >>>>>>> They are in binary format, but inside the container I hope someone could reproduce my issue. A word on the structure and intent behind these matrices : > >>>>>>> > >>>>>>> QE is a diagonal rectangular real matrix. Think of it as some sort of preconditioner > >>>>>>> L is the least dense of them all, the only one that is complex, and in order to avoid inverting it I'm using two KSPs to compute solve problems on the fly > >>>>>>> Mf is a diagonal square real matrix, its on the right-hand side of the Generalised Hermitian Eigenvalue problem (I'm solving QE^H*L^-1H*L^-1*QE*x=lambda*Mf*x > >>>>>>> > >>>>>>> Full MWE is below : > >>>>>>> > >>>>>>> from petsc4py import PETSc as pet > >>>>>>> from slepc4py import SLEPc as slp > >>>>>>> from mpi4py.MPI import COMM_WORLD > >>>>>>> > >>>>>>> # Global sizes > >>>>>>> m_local=COMM_WORLD.rank*(490363-489780)+489780 > >>>>>>> n_local=COMM_WORLD.rank*(452259-451743)+451743 > >>>>>>> m=980143 > >>>>>>> n=904002 > >>>>>>> > >>>>>>> QE=pet.Mat().createAIJ([[m_local,m],[n_local,n]]) > >>>>>>> L=pet.Mat().createAIJ([[m_local,m],[m_local,m]]) > >>>>>>> Mf=pet.Mat().createAIJ([[n_local,n],[n_local,n]]) > >>>>>>> > >>>>>>> viewerQE = pet.Viewer().createMPIIO("QE.dat", 'r', COMM_WORLD) > >>>>>>> QE.load(viewerQE) > >>>>>>> viewerL = pet.Viewer().createMPIIO("L.dat", 'r', COMM_WORLD) > >>>>>>> L.load(viewerL) > >>>>>>> viewerMf = pet.Viewer().createMPIIO("Mf.dat", 'r', COMM_WORLD) > >>>>>>> Mf.load(viewerMf) > >>>>>>> > >>>>>>> QE.assemble() > >>>>>>> L.assemble() > >>>>>>> > >>>>>>> KSPs = [] > >>>>>>> # Useful solvers (here to put options for computing a smart R) > >>>>>>> for Mat in [L,L.hermitianTranspose()]: > >>>>>>> KSP = pet.KSP().create() > >>>>>>> KSP.setOperators(Mat) > >>>>>>> KSP.setFromOptions() > >>>>>>> KSPs.append(KSP) > >>>>>>> class LHS_class: > >>>>>>> def mult(self,A,x,y): > >>>>>>> w=pet.Vec().createMPI([m_local,m],comm=COMM_WORLD) > >>>>>>> z=pet.Vec().createMPI([m_local,m],comm=COMM_WORLD) > >>>>>>> QE.mult(x,w) > >>>>>>> KSPs[0].solve(w,z) > >>>>>>> KSPs[1].solve(z,w) > >>>>>>> QE.multTranspose(w,y) > >>>>>>> > >>>>>>> # Matrix free operator > >>>>>>> LHS=pet.Mat() > >>>>>>> LHS.create(comm=COMM_WORLD) > >>>>>>> LHS.setSizes([[n_local,n],[n_local,n]]) > >>>>>>> LHS.setType(pet.Mat.Type.PYTHON) > >>>>>>> LHS.setPythonContext(LHS_class()) > >>>>>>> LHS.setUp() > >>>>>>> > >>>>>>> # Eigensolver > >>>>>>> EPS = slp.EPS(); EPS.create() > >>>>>>> EPS.setOperators(LHS,Mf) # Solve QE^T*L^-1H*L^-1*QE*x=lambda*Mf*x > >>>>>>> EPS.setProblemType(slp.EPS.ProblemType.GHEP) # Specify that A is hermitian (by construction), and B is semi-positive > >>>>>>> EPS.setWhichEigenpairs(EPS.Which.LARGEST_MAGNITUDE) # Find largest eigenvalues > >>>>>>> EPS.setFromOptions() > >>>>>>> EPS.solve() > >>>>>>> > >>>>>>> Quentin CHEVALIER ? IA parcours recherche > >>>>>>> > >>>>>>> LadHyX - Ecole polytechnique > >>>>>>> > >>>>>>> __________ > >>>>> > >>>>> > >>>> > >> > >> > >> > >> -- > >> What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. > >> -- Norbert Wiener > >> > >> https://www.cse.buffalo.edu/~knepley/ > From zjorti at lanl.gov Tue May 10 10:40:39 2022 From: zjorti at lanl.gov (Jorti, Zakariae) Date: Tue, 10 May 2022 15:40:39 +0000 Subject: [petsc-users] MatColoring Message-ID: <48152f97d9814c7cbee756147d847458@lanl.gov> Hi, I am solving a non-linear problem and using a finite difference approximation with coloring to compute the Jacobian matrix. There are several coloring algorithms available in PETSc as indicated here: https://petsc.org/release/docs/manualpages/Mat/MatColoring.html And I was wondering how to switch from one to another in the Jacobian setup routine and also how to check which coloring algorithm I am currently using. Thank you. Zakariae Jorti -------------- next part -------------- An HTML attachment was scrubbed... URL: From mfadams at lbl.gov Tue May 10 11:31:37 2022 From: mfadams at lbl.gov (Mark Adams) Date: Tue, 10 May 2022 12:31:37 -0400 Subject: [petsc-users] MatColoring In-Reply-To: <48152f97d9814c7cbee756147d847458@lanl.gov> References: <48152f97d9814c7cbee756147d847458@lanl.gov> Message-ID: This should do it: https://petsc.org/release/docs/manualpages/MatOrderings/MatColoringSetType.html#MatColoringSetType Also, this page has MATCOLORINGMIS, but that is not a type: typedef const char* MatColoringType ; #define MATCOLORINGJP "jp" #define MATCOLORINGPOWER "power" #define MATCOLORINGNATURAL "natural" #define MATCOLORINGSL "sl" #define MATCOLORINGLF "lf" #define MATCOLORINGID "id" #define MATCOLORINGGREEDY "greedy" I guess they mean MATCOLORINGGREEDY but not sure. Mark On Tue, May 10, 2022 at 11:40 AM Jorti, Zakariae via petsc-users < petsc-users at mcs.anl.gov> wrote: > Hi, > > I am solving a non-linear problem and using a finite difference > approximation with coloring to compute the Jacobian matrix. > > There are several coloring algorithms available in PETSc as indicated here: > https://petsc.org/release/docs/manualpages/Mat/MatColoring.html > > And I was wondering how to switch from one to another in the Jacobian > setup routine and also how to check which coloring algorithm I am currently > using. > > Thank you. > > > Zakariae Jorti > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at petsc.dev Tue May 10 11:35:54 2022 From: bsmith at petsc.dev (Barry Smith) Date: Tue, 10 May 2022 12:35:54 -0400 Subject: [petsc-users] MatColoring In-Reply-To: <48152f97d9814c7cbee756147d847458@lanl.gov> References: <48152f97d9814c7cbee756147d847458@lanl.gov> Message-ID: <651F493C-2595-4C1E-8F84-968A19176185@petsc.dev> This depends to some degree on how you are accessing applying the Jacobian construction process. If you are using SNES or TS then the SNES object handles most of the work of organizing the PETSc objects needed to compute the Jacobian and you can control the choices via the options database. With SNES/TS using a DM you can skip calling SNESSetJacobian() and simply use the option -snes_fd_color to have SNES compute the Jacobian for you. By default, the coloring is obtained from the DM (which is generally the best available coloring), you can have it use a coloring computed directly from the matrix structure with the options -snes_fd_color_use_mat -mat_coloring_type jp power sl lf ld or greedy (see MatColoringType, MatColoringSetFromOptions) Use -mat_fd_coloring_view to get information on the computation of the Jacobian from the coloring. Use -mat_coloring_view to get information on the coloring used. (see MatFDColoringSetFromOptions) If you are using SNES/TS but not using a DM you need to compute the nonzero structure of the matrix yourself into J and call SNESSetJacobian(snes,J,J,SNESComputeJacobianDefaultColor,matfdcoloring); you need to create the matfdcoloring object with MatFDColoringCreate() using an iscoloring you obtained with MatColoringCreate() etc. The same command line arguments as above allow you to control the coloring algorithm used and to view them etc. Barry > On May 10, 2022, at 11:40 AM, Jorti, Zakariae via petsc-users wrote: > > Hi, > > I am solving a non-linear problem and using a finite difference approximation with coloring to compute the Jacobian matrix. > > There are several coloring algorithms available in PETSc as indicated here: > https://petsc.org/release/docs/manualpages/Mat/MatColoring.html > > And I was wondering how to switch from one to another in the Jacobian setup routine and also how to check which coloring algorithm I am currently using. > > Thank you. > > Zakariae Jorti -------------- next part -------------- An HTML attachment was scrubbed... URL: From tangqi at msu.edu Tue May 10 11:51:07 2022 From: tangqi at msu.edu (Tang, Qi) Date: Tue, 10 May 2022 16:51:07 +0000 Subject: [petsc-users] MatColoring In-Reply-To: <651F493C-2595-4C1E-8F84-968A19176185@petsc.dev> References: <48152f97d9814c7cbee756147d847458@lanl.gov> <651F493C-2595-4C1E-8F84-968A19176185@petsc.dev> Message-ID: We are using SNES + TS + dmstag. The current bottleneck is the number of residual evaluation (more than 300 per Jacobian building using the default coloring from dmstag). We talked to Patrick and we are not sure how to improve further. So it looks like we should play with mat_coloring_type and see if others give us better performance. If there is anything else we can play with, please also let us know. We also lag Jacobian and only build once every three Newton iterations, which works well. Thanks, Qi On May 10, 2022, at 10:35 AM, Barry Smith > wrote: This depends to some degree on how you are accessing applying the Jacobian construction process. If you are using SNES or TS then the SNES object handles most of the work of organizing the PETSc objects needed to compute the Jacobian and you can control the choices via the options database. With SNES/TS using a DM you can skip calling SNESSetJacobian() and simply use the option -snes_fd_color to have SNES compute the Jacobian for you. By default, the coloring is obtained from the DM (which is generally the best available coloring), you can have it use a coloring computed directly from the matrix structure with the options -snes_fd_color_use_mat -mat_coloring_type jp power sl lf ld or greedy (see MatColoringType, MatColoringSetFromOptions) Use -mat_fd_coloring_view to get information on the computation of the Jacobian from the coloring. Use -mat_coloring_view to get information on the coloring used. (see MatFDColoringSetFromOptions) If you are using SNES/TS but not using a DM you need to compute the nonzero structure of the matrix yourself into J and call SNESSetJacobian(snes,J,J,SNESComputeJacobianDefaultColor,matfdcoloring); you need to create the matfdcoloring object with MatFDColoringCreate() using an iscoloring you obtained with MatColoringCreate() etc. The same command line arguments as above allow you to control the coloring algorithm used and to view them etc. Barry On May 10, 2022, at 11:40 AM, Jorti, Zakariae via petsc-users > wrote: Hi, I am solving a non-linear problem and using a finite difference approximation with coloring to compute the Jacobian matrix. There are several coloring algorithms available in PETSc as indicated here: https://petsc.org/release/docs/manualpages/Mat/MatColoring.html And I was wondering how to switch from one to another in the Jacobian setup routine and also how to check which coloring algorithm I am currently using. Thank you. Zakariae Jorti -------------- next part -------------- An HTML attachment was scrubbed... URL: From dave.mayhem23 at gmail.com Tue May 10 13:03:48 2022 From: dave.mayhem23 at gmail.com (Dave May) Date: Tue, 10 May 2022 20:03:48 +0200 Subject: [petsc-users] MatColoring In-Reply-To: References: <48152f97d9814c7cbee756147d847458@lanl.gov> <651F493C-2595-4C1E-8F84-968A19176185@petsc.dev> Message-ID: On Tue 10. May 2022 at 18:51, Tang, Qi wrote: > We are using SNES + TS + dmstag. The current bottleneck is the number of > residual evaluation (more than 300 per Jacobian building using the default > coloring from dmstag). > I suspect that this high count stems from the fact that non zero pattern defined by MatCreate for dmstag is not specialized for your particular stencil. Ie it just considers the stencil width, shape and assumes all cell, face, vertex dofs are connected. Is that correct? Would a simple solution to drop the 300 residual evals be just to define your own Amat with a non-zero pattern which is specific to your actually stencil? The code to define such a non zero pattern is probably not too hard to write using MatPreallocator. Just a thought. Thanks, Dave We talked to Patrick and we are not sure how to improve further. > > So it looks like we should play with mat_coloring_type and see if others > give us better performance. > > If there is anything else we can play with, please also let us know. We > also lag Jacobian and only build once every three Newton iterations, which > works well. Thanks, > > Qi > > > > On May 10, 2022, at 10:35 AM, Barry Smith wrote: > > > This depends to some degree on how you are accessing applying the > Jacobian construction process. > > If you are using SNES or TS then the SNES object handles most of the > work of organizing the PETSc objects needed to compute the Jacobian and you > can control the choices via the options database. > > With SNES/TS using a DM you can skip calling SNESSetJacobian() and > simply use the option -snes_fd_color to have SNES compute the Jacobian for > you. By default, the coloring is obtained from the DM (which is generally > the best available coloring), you can have it use a coloring computed > directly from the matrix structure with the > options -snes_fd_color_use_mat -mat_coloring_type jp power sl lf ld or > greedy (see MatColoringType, MatColoringSetFromOptions) > Use -mat_fd_coloring_view to get information on the computation of the > Jacobian from the coloring. Use -mat_coloring_view to get information on > the coloring used. (see MatFDColoringSetFromOptions) > > If you are using SNES/TS but not using a DM you need to compute the > nonzero structure of the matrix yourself into J and call > SNESSetJacobian(snes,J,J,SNESComputeJacobianDefaultColor,matfdcoloring); > you need to create the matfdcoloring object with MatFDColoringCreate() > using an iscoloring you obtained with MatColoringCreate() etc. The same > command line arguments as above allow you to control the coloring algorithm > used and to view them etc. > > Barry > > > > > > > > > On May 10, 2022, at 11:40 AM, Jorti, Zakariae via petsc-users < > petsc-users at mcs.anl.gov> wrote: > > Hi, > > I am solving a non-linear problem and using a finite difference > approximation with coloring to compute the Jacobian matrix. > > There are several coloring algorithms available in PETSc as indicated here: > https://petsc.org/release/docs/manualpages/Mat/MatColoring.html > > > And I was wondering how to switch from one to another in the Jacobian > setup routine and also how to check which coloring algorithm I am currently > using. > > Thank you. > > Zakariae Jorti > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tt73 at njit.edu Tue May 10 14:12:38 2022 From: tt73 at njit.edu (Takahashi, Tadanaga) Date: Tue, 10 May 2022 15:12:38 -0400 Subject: [petsc-users] Convergence issues for SNES NASM Message-ID: Hello, We are working on a finite difference solver for a 2D nonlinear PDE with Dirichlet Boundary conditions on a rectangular domain. Our goal is to solve the problem with parallel nonlinear additive Schwarz (NASM) as the outer solver. Our code is similar to SNES example 5 . In example 5, the parallel NASM can be executed with a command like `mpiexec -n 4 ./ex5 -mms 3 -snes_type nasm -snes_nasm_type restrict -da_overlap 2` which gives a convergent result. We assume this is the correct usage. A comment in the source code for NASM mentions that NASM should be a preconditioner but there's no documentation on the usage. The Brune paper does not cover parallel NASM either. We observed that increasing the overlap leads to fewer Schwarz iterations. The parallelization works seamlessly for an arbitrary number of subdomains. This is the type of behavior we were expecting from our code. Our method uses box-style stencil width d = ceil(N^(1/3)) on a N by N DMDA. The finite difference stencil consists of 4d+1 points spread out in a diamond formation. If a stencil point is out of bounds, then it is projected onto the boundary curve. Since the nodes on the boundary curve would result in an irregular mesh, we chose not treat boundary nodes as unknowns as in Example 5. We use DMDACreate2d to create the DA for the interior points and DMDASNESSetFunctionLocal to associate the residue function to the SNES object. Our code works serially. We have also tested our code with Newton-Krylov-Schwarz (NKS) by running something akin to `mpiexec -n ./solve -snes_type newtonls`. We have tested the NKS for several quantities of subdomains and overlap and the code works as expected. We have some confidence in the correctness of our code. The overlapping NASM was implemented in MATLAB so we know the method converges. However, the parallel NASM will not converge with our PETSc code. We don't understand why NKS works while NASM does not. The F-norm residue monotonically decreases and then stagnates. Here is an example of the output when attempting to run NASM in parallel: takahashi at ubuntu:~/Desktop/MA-DDM/Cpp/Rectangle$ mpiexec -n 4 ./test1 -t1_N 20 -snes_max_it 50 -snes_monitor -snes_view -da_overlap 3 -snes_type nasm -snes_nasm_type restrict 0 SNES Function norm 7.244681057908e+02 1 SNES Function norm 1.237688062971e+02 2 SNES Function norm 1.068926073552e+02 3 SNES Function norm 1.027563237834e+02 4 SNES Function norm 1.022184806736e+02 5 SNES Function norm 1.020818227640e+02 6 SNES Function norm 1.020325629121e+02 7 SNES Function norm 1.020149036595e+02 8 SNES Function norm 1.020088110545e+02 9 SNES Function norm 1.020067198030e+02 10 SNES Function norm 1.020060034469e+02 11 SNES Function norm 1.020057582380e+02 12 SNES Function norm 1.020056743241e+02 13 SNES Function norm 1.020056456101e+02 14 SNES Function norm 1.020056357849e+02 15 SNES Function norm 1.020056324231e+02 16 SNES Function norm 1.020056312727e+02 17 SNES Function norm 1.020056308791e+02 18 SNES Function norm 1.020056307444e+02 19 SNES Function norm 1.020056306983e+02 20 SNES Function norm 1.020056306826e+02 21 SNES Function norm 1.020056306772e+02 22 SNES Function norm 1.020056306753e+02 23 SNES Function norm 1.020056306747e+02 24 SNES Function norm 1.020056306745e+02 25 SNES Function norm 1.020056306744e+02 26 SNES Function norm 1.020056306744e+02 27 SNES Function norm 1.020056306744e+02 28 SNES Function norm 1.020056306744e+02 29 SNES Function norm 1.020056306744e+02 30 SNES Function norm 1.020056306744e+02 31 SNES Function norm 1.020056306744e+02 32 SNES Function norm 1.020056306744e+02 33 SNES Function norm 1.020056306744e+02 34 SNES Function norm 1.020056306744e+02 35 SNES Function norm 1.020056306744e+02 36 SNES Function norm 1.020056306744e+02 37 SNES Function norm 1.020056306744e+02 38 SNES Function norm 1.020056306744e+02 39 SNES Function norm 1.020056306744e+02 40 SNES Function norm 1.020056306744e+02 41 SNES Function norm 1.020056306744e+02 42 SNES Function norm 1.020056306744e+02 43 SNES Function norm 1.020056306744e+02 44 SNES Function norm 1.020056306744e+02 45 SNES Function norm 1.020056306744e+02 46 SNES Function norm 1.020056306744e+02 47 SNES Function norm 1.020056306744e+02 48 SNES Function norm 1.020056306744e+02 49 SNES Function norm 1.020056306744e+02 50 SNES Function norm 1.020056306744e+02 SNES Object: 4 MPI processes type: nasm total subdomain blocks = 4 Local solver information for first block on rank 0: Use -snes_view ::ascii_info_detail to display information for all blocks SNES Object: (sub_) 1 MPI processes type: newtonls maximum iterations=50, maximum function evaluations=10000 tolerances: relative=1e-08, absolute=1e-50, solution=1e-08 total number of linear solver iterations=22 total number of function evaluations=40 norm schedule ALWAYS Jacobian is built using a DMDA local Jacobian SNESLineSearch Object: (sub_) 1 MPI processes type: bt interpolation: cubic alpha=1.000000e-04 maxstep=1.000000e+08, minlambda=1.000000e-12 tolerances: relative=1.000000e-08, absolute=1.000000e-15, lambda=1.000000e-08 maximum iterations=40 KSP Object: (sub_) 1 MPI processes type: preonly maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using NONE norm type for convergence test PC Object: (sub_) 1 MPI processes type: lu out-of-place factorization tolerance for zero pivot 2.22045e-14 matrix ordering: nd factor fill ratio given 5., needed 2.13732 Factored matrix follows: Mat Object: 1 MPI processes type: seqaij rows=169, cols=169 package used to perform factorization: petsc total: nonzeros=13339, allocated nonzeros=13339 using I-node routines: found 104 nodes, limit used is 5 linear system matrix = precond matrix: Mat Object: 1 MPI processes type: seqaij rows=169, cols=169 total: nonzeros=6241, allocated nonzeros=6241 total number of mallocs used during MatSetValues calls=0 not using I-node routines maximum iterations=50, maximum function evaluations=10000 tolerances: relative=1e-08, absolute=1e-50, solution=1e-08 total number of function evaluations=51 norm schedule ALWAYS Jacobian is built using a DMDA local Jacobian problem ex10 on 20 x 20 point 2D grid with d = 3, and eps = 0.082: error |u-uexact|_inf = 3.996e-01, |u-uexact|_h = 2.837e-01 We have been stuck on this for a while now. We do not know how to debug this issue. Please let us know if you have any insights. -------------- next part -------------- An HTML attachment was scrubbed... URL: From zjorti at lanl.gov Wed May 11 13:25:55 2022 From: zjorti at lanl.gov (Jorti, Zakariae) Date: Wed, 11 May 2022 18:25:55 +0000 Subject: [petsc-users] MatColoring In-Reply-To: References: <48152f97d9814c7cbee756147d847458@lanl.gov> <651F493C-2595-4C1E-8F84-968A19176185@petsc.dev>, Message-ID: <7aa2305cf42d4d88b7e5236602cdc9a6@lanl.gov> Hello, I have used the -mat_coloring_view and -mat_fd_coloring_view flags as you suggested and it turns out that the coloring used was MATCOLORINGSL. I have also tried other methods and obtained the following numbers of function evaluations per Jacobian building: - MATCOLORINGJP : 593 - MATCOLORINGID : 327 - MATCOLORINGSL : 321 - MATCOLORINGGREEDY : 316 - MATCOLORINGLF : 230 Is there any parameter/trick we can try to further cut down the number of function evaluations? And also how could we use MATCOLORINGMIS as this latter does not appear in the MatColoringType options? Many thanks. Zakariae ________________________________ From: Tang, Qi Sent: Tuesday, May 10, 2022 10:51:07 AM To: Barry Smith; petsc-users at mcs.anl.gov Cc: Jorti, Zakariae; Tang, Xianzhu Subject: [EXTERNAL] Re: [petsc-users] MatColoring We are using SNES + TS + dmstag. The current bottleneck is the number of residual evaluation (more than 300 per Jacobian building using the default coloring from dmstag). We talked to Patrick and we are not sure how to improve further. So it looks like we should play with mat_coloring_type and see if others give us better performance. If there is anything else we can play with, please also let us know. We also lag Jacobian and only build once every three Newton iterations, which works well. Thanks, Qi On May 10, 2022, at 10:35 AM, Barry Smith > wrote: This depends to some degree on how you are accessing applying the Jacobian construction process. If you are using SNES or TS then the SNES object handles most of the work of organizing the PETSc objects needed to compute the Jacobian and you can control the choices via the options database. With SNES/TS using a DM you can skip calling SNESSetJacobian() and simply use the option -snes_fd_color to have SNES compute the Jacobian for you. By default, the coloring is obtained from the DM (which is generally the best available coloring), you can have it use a coloring computed directly from the matrix structure with the options -snes_fd_color_use_mat -mat_coloring_type jp power sl lf ld or greedy (see MatColoringType, MatColoringSetFromOptions) Use -mat_fd_coloring_view to get information on the computation of the Jacobian from the coloring. Use -mat_coloring_view to get information on the coloring used. (see MatFDColoringSetFromOptions) If you are using SNES/TS but not using a DM you need to compute the nonzero structure of the matrix yourself into J and call SNESSetJacobian(snes,J,J,SNESComputeJacobianDefaultColor,matfdcoloring); you need to create the matfdcoloring object with MatFDColoringCreate() using an iscoloring you obtained with MatColoringCreate() etc. The same command line arguments as above allow you to control the coloring algorithm used and to view them etc. Barry On May 10, 2022, at 11:40 AM, Jorti, Zakariae via petsc-users > wrote: Hi, I am solving a non-linear problem and using a finite difference approximation with coloring to compute the Jacobian matrix. There are several coloring algorithms available in PETSc as indicated here: https://petsc.org/release/docs/manualpages/Mat/MatColoring.html And I was wondering how to switch from one to another in the Jacobian setup routine and also how to check which coloring algorithm I am currently using. Thank you. Zakariae Jorti -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at petsc.dev Wed May 11 14:28:11 2022 From: bsmith at petsc.dev (Barry Smith) Date: Wed, 11 May 2022 15:28:11 -0400 Subject: [petsc-users] MatColoring In-Reply-To: <7aa2305cf42d4d88b7e5236602cdc9a6@lanl.gov> References: <48152f97d9814c7cbee756147d847458@lanl.gov> <651F493C-2595-4C1E-8F84-968A19176185@petsc.dev> <7aa2305cf42d4d88b7e5236602cdc9a6@lanl.gov> Message-ID: It is unlikely that the true number of colors is much less than 230 so I don't think you can save by reducing the number of colors more. Are you using IS_COLORING_LOCAL it eliminates much parallel communication so you would definitely benefit by using this, depending on how your FormFunction is written you may need to refactor your code slightly. There is also a hidden parameter which is a tradeoff of memory usage vs speed. If memory is not an issue for you then you could change this parameter. It is computed in MatFDColoringCreate_MPIXAIJ() called bcols You can just hardware bcols to iscoloring->n This may help a bit. The only way to drastically improve the time after the above is to reorganize the PETSc code and your code to evaluate multiple function evaluations "at the same time" but this is a major project (it has to be done for GPUs). MATCOLORINGMIS doesn't exist, it is a typo in the docs. Barry > On May 11, 2022, at 2:25 PM, Jorti, Zakariae wrote: > > Hello, > > I have used the -mat_coloring_view and -mat_fd_coloring_view flags as you suggested and it turns out that the coloring used was MATCOLORINGSL. > I have also tried other methods and obtained the following numbers of function evaluations per Jacobian building: > - MATCOLORINGJP : 593 > - MATCOLORINGID : 327 > - MATCOLORINGSL : 321 > - MATCOLORINGGREEDY : 316 > - MATCOLORINGLF : 230 > > Is there any parameter/trick we can try to further cut down the number of function evaluations? And also how could we use MATCOLORINGMIS as this latter does not appear in the MatColoringType options? > Many thanks. > > Zakariae > From: Tang, Qi > > Sent: Tuesday, May 10, 2022 10:51:07 AM > To: Barry Smith; petsc-users at mcs.anl.gov > Cc: Jorti, Zakariae; Tang, Xianzhu > Subject: [EXTERNAL] Re: [petsc-users] MatColoring > > We are using SNES + TS + dmstag. The current bottleneck is the number of residual evaluation (more than 300 per Jacobian building using the default coloring from dmstag). We talked to Patrick and we are not sure how to improve further. > > So it looks like we should play with mat_coloring_type and see if others give us better performance. > > If there is anything else we can play with, please also let us know. We also lag Jacobian and only build once every three Newton iterations, which works well. Thanks, > > Qi > > >> On May 10, 2022, at 10:35 AM, Barry Smith > wrote: >> >> >> This depends to some degree on how you are accessing applying the Jacobian construction process. >> >> If you are using SNES or TS then the SNES object handles most of the work of organizing the PETSc objects needed to compute the Jacobian and you can control the choices via the options database. >> >> With SNES/TS using a DM you can skip calling SNESSetJacobian() and simply use the option -snes_fd_color to have SNES compute the Jacobian for you. By default, the coloring is obtained from the DM (which is generally the best available coloring), you can have it use a coloring computed directly from the matrix structure with the options -snes_fd_color_use_mat -mat_coloring_type jp power sl lf ld or greedy (see MatColoringType, MatColoringSetFromOptions) Use -mat_fd_coloring_view to get information on the computation of the Jacobian from the coloring. Use -mat_coloring_view to get information on the coloring used. (see MatFDColoringSetFromOptions) >> >> If you are using SNES/TS but not using a DM you need to compute the nonzero structure of the matrix yourself into J and call SNESSetJacobian(snes,J,J,SNESComputeJacobianDefaultColor,matfdcoloring); you need to create the matfdcoloring object with MatFDColoringCreate() using an iscoloring you obtained with MatColoringCreate() etc. The same command line arguments as above allow you to control the coloring algorithm used and to view them etc. >> >> Barry >> >> >> >> >> >> >> >> >>> On May 10, 2022, at 11:40 AM, Jorti, Zakariae via petsc-users > wrote: >>> >>> Hi, >>> >>> I am solving a non-linear problem and using a finite difference approximation with coloring to compute the Jacobian matrix. >>> >>> There are several coloring algorithms available in PETSc as indicated here: >>> https://petsc.org/release/docs/manualpages/Mat/MatColoring.html >>> >>> And I was wondering how to switch from one to another in the Jacobian setup routine and also how to check which coloring algorithm I am currently using. >>> >>> Thank you. >>> >>> Zakariae Jorti -------------- next part -------------- An HTML attachment was scrubbed... URL: From jed at jedbrown.org Wed May 11 22:43:56 2022 From: jed at jedbrown.org (Jed Brown) Date: Wed, 11 May 2022 21:43:56 -0600 Subject: [petsc-users] Convergence issues for SNES NASM In-Reply-To: References: Message-ID: <87lev75sar.fsf@jedbrown.org> Can you add -snes_linesearch_monitor -sub_snes_linesearch_monitor -ksp_converged_reason and send the output?? "Takahashi, Tadanaga" writes: > Hello, > > We are working on a finite difference solver for a 2D nonlinear PDE with > Dirichlet Boundary conditions on a rectangular domain. Our goal is to solve > the problem with parallel nonlinear additive Schwarz (NASM) as the outer > solver. Our code is similar to SNES example 5 > . In example 5, > the parallel NASM can be executed with a command like `mpiexec -n 4 ./ex5 > -mms 3 -snes_type nasm -snes_nasm_type restrict -da_overlap 2` which gives > a convergent result. We assume this is the correct usage. A comment in the > source code for NASM mentions that NASM should be a preconditioner but > there's no documentation on the usage. The Brune paper does not cover > parallel NASM either. We observed that increasing the overlap leads to > fewer Schwarz iterations. The parallelization works seamlessly for an > arbitrary number of subdomains. This is the type of behavior we were > expecting from our code. > > Our method uses box-style stencil width d = ceil(N^(1/3)) on a N by N DMDA. > The finite difference stencil consists of 4d+1 points spread out in a > diamond formation. If a stencil point is out of bounds, then it is > projected onto the boundary curve. Since the nodes on the boundary curve > would result in an irregular mesh, we chose not treat boundary nodes as > unknowns as in Example 5. We use DMDACreate2d to create the DA for the > interior points and DMDASNESSetFunctionLocal to associate the residue > function to the SNES object. > > Our code works serially. We have also tested our code > with Newton-Krylov-Schwarz (NKS) by running something akin to `mpiexec -n > ./solve -snes_type newtonls`. We have tested the NKS for several > quantities of subdomains and overlap and the code works as expected. We > have some confidence in the correctness of our code. The overlapping NASM > was implemented in MATLAB so we know the method converges. However, the > parallel NASM will not converge with our PETSc code. We don't understand > why NKS works while NASM does not. The F-norm residue monotonically > decreases and then stagnates. > > Here is an example of the output when attempting to run NASM in parallel: > takahashi at ubuntu:~/Desktop/MA-DDM/Cpp/Rectangle$ mpiexec -n 4 ./test1 -t1_N > 20 -snes_max_it 50 -snes_monitor -snes_view -da_overlap 3 -snes_type nasm > -snes_nasm_type restrict > 0 SNES Function norm 7.244681057908e+02 > 1 SNES Function norm 1.237688062971e+02 > 2 SNES Function norm 1.068926073552e+02 > 3 SNES Function norm 1.027563237834e+02 > 4 SNES Function norm 1.022184806736e+02 > 5 SNES Function norm 1.020818227640e+02 > 6 SNES Function norm 1.020325629121e+02 > 7 SNES Function norm 1.020149036595e+02 > 8 SNES Function norm 1.020088110545e+02 > 9 SNES Function norm 1.020067198030e+02 > 10 SNES Function norm 1.020060034469e+02 > 11 SNES Function norm 1.020057582380e+02 > 12 SNES Function norm 1.020056743241e+02 > 13 SNES Function norm 1.020056456101e+02 > 14 SNES Function norm 1.020056357849e+02 > 15 SNES Function norm 1.020056324231e+02 > 16 SNES Function norm 1.020056312727e+02 > 17 SNES Function norm 1.020056308791e+02 > 18 SNES Function norm 1.020056307444e+02 > 19 SNES Function norm 1.020056306983e+02 > 20 SNES Function norm 1.020056306826e+02 > 21 SNES Function norm 1.020056306772e+02 > 22 SNES Function norm 1.020056306753e+02 > 23 SNES Function norm 1.020056306747e+02 > 24 SNES Function norm 1.020056306745e+02 > 25 SNES Function norm 1.020056306744e+02 > 26 SNES Function norm 1.020056306744e+02 > 27 SNES Function norm 1.020056306744e+02 > 28 SNES Function norm 1.020056306744e+02 > 29 SNES Function norm 1.020056306744e+02 > 30 SNES Function norm 1.020056306744e+02 > 31 SNES Function norm 1.020056306744e+02 > 32 SNES Function norm 1.020056306744e+02 > 33 SNES Function norm 1.020056306744e+02 > 34 SNES Function norm 1.020056306744e+02 > 35 SNES Function norm 1.020056306744e+02 > 36 SNES Function norm 1.020056306744e+02 > 37 SNES Function norm 1.020056306744e+02 > 38 SNES Function norm 1.020056306744e+02 > 39 SNES Function norm 1.020056306744e+02 > 40 SNES Function norm 1.020056306744e+02 > 41 SNES Function norm 1.020056306744e+02 > 42 SNES Function norm 1.020056306744e+02 > 43 SNES Function norm 1.020056306744e+02 > 44 SNES Function norm 1.020056306744e+02 > 45 SNES Function norm 1.020056306744e+02 > 46 SNES Function norm 1.020056306744e+02 > 47 SNES Function norm 1.020056306744e+02 > 48 SNES Function norm 1.020056306744e+02 > 49 SNES Function norm 1.020056306744e+02 > 50 SNES Function norm 1.020056306744e+02 > SNES Object: 4 MPI processes > type: nasm > total subdomain blocks = 4 > Local solver information for first block on rank 0: > Use -snes_view ::ascii_info_detail to display information for all blocks > SNES Object: (sub_) 1 MPI processes > type: newtonls > maximum iterations=50, maximum function evaluations=10000 > tolerances: relative=1e-08, absolute=1e-50, solution=1e-08 > total number of linear solver iterations=22 > total number of function evaluations=40 > norm schedule ALWAYS > Jacobian is built using a DMDA local Jacobian > SNESLineSearch Object: (sub_) 1 MPI processes > type: bt > interpolation: cubic > alpha=1.000000e-04 > maxstep=1.000000e+08, minlambda=1.000000e-12 > tolerances: relative=1.000000e-08, absolute=1.000000e-15, > lambda=1.000000e-08 > maximum iterations=40 > KSP Object: (sub_) 1 MPI processes > type: preonly > maximum iterations=10000, initial guess is zero > tolerances: relative=1e-05, absolute=1e-50, divergence=10000. > left preconditioning > using NONE norm type for convergence test > PC Object: (sub_) 1 MPI processes > type: lu > out-of-place factorization > tolerance for zero pivot 2.22045e-14 > matrix ordering: nd > factor fill ratio given 5., needed 2.13732 > Factored matrix follows: > Mat Object: 1 MPI processes > type: seqaij > rows=169, cols=169 > package used to perform factorization: petsc > total: nonzeros=13339, allocated nonzeros=13339 > using I-node routines: found 104 nodes, limit used is 5 > linear system matrix = precond matrix: > Mat Object: 1 MPI processes > type: seqaij > rows=169, cols=169 > total: nonzeros=6241, allocated nonzeros=6241 > total number of mallocs used during MatSetValues calls=0 > not using I-node routines > maximum iterations=50, maximum function evaluations=10000 > tolerances: relative=1e-08, absolute=1e-50, solution=1e-08 > total number of function evaluations=51 > norm schedule ALWAYS > Jacobian is built using a DMDA local Jacobian > problem ex10 on 20 x 20 point 2D grid with d = 3, and eps = 0.082: > error |u-uexact|_inf = 3.996e-01, |u-uexact|_h = 2.837e-01 > > We have been stuck on this for a while now. We do not know how to debug > this issue. Please let us know if you have any insights. From tt73 at njit.edu Thu May 12 08:01:47 2022 From: tt73 at njit.edu (Takahashi, Tadanaga) Date: Thu, 12 May 2022 09:01:47 -0400 Subject: [petsc-users] Convergence issues for SNES NASM In-Reply-To: <87lev75sar.fsf@jedbrown.org> References: <87lev75sar.fsf@jedbrown.org> Message-ID: I ran the code with the additional options but the raw output is about 75,000 lines. I cannot paste it directly in the email. The output is in the attached file. On Wed, May 11, 2022 at 11:44 PM Jed Brown wrote: > Can you add -snes_linesearch_monitor -sub_snes_linesearch_monitor > -ksp_converged_reason and send the output?? > > "Takahashi, Tadanaga" writes: > > > Hello, > > > > We are working on a finite difference solver for a 2D nonlinear PDE with > > Dirichlet Boundary conditions on a rectangular domain. Our goal is to > solve > > the problem with parallel nonlinear additive Schwarz (NASM) as the outer > > solver. Our code is similar to SNES example 5 > > . In example 5, > > the parallel NASM can be executed with a command like `mpiexec -n 4 ./ex5 > > -mms 3 -snes_type nasm -snes_nasm_type restrict -da_overlap 2` which > gives > > a convergent result. We assume this is the correct usage. A comment in > the > > source code for NASM mentions that NASM should be a preconditioner but > > there's no documentation on the usage. The Brune paper does not cover > > parallel NASM either. We observed that increasing the overlap leads to > > fewer Schwarz iterations. The parallelization works seamlessly for an > > arbitrary number of subdomains. This is the type of behavior we were > > expecting from our code. > > > > Our method uses box-style stencil width d = ceil(N^(1/3)) on a N by N > DMDA. > > The finite difference stencil consists of 4d+1 points spread out in a > > diamond formation. If a stencil point is out of bounds, then it is > > projected onto the boundary curve. Since the nodes on the boundary curve > > would result in an irregular mesh, we chose not treat boundary nodes as > > unknowns as in Example 5. We use DMDACreate2d to create the DA for the > > interior points and DMDASNESSetFunctionLocal to associate the residue > > function to the SNES object. > > > > Our code works serially. We have also tested our code > > with Newton-Krylov-Schwarz (NKS) by running something akin to `mpiexec -n > > ./solve -snes_type newtonls`. We have tested the NKS for several > > quantities of subdomains and overlap and the code works as expected. We > > have some confidence in the correctness of our code. The overlapping NASM > > was implemented in MATLAB so we know the method converges. However, the > > parallel NASM will not converge with our PETSc code. We don't understand > > why NKS works while NASM does not. The F-norm residue monotonically > > decreases and then stagnates. > > > > Here is an example of the output when attempting to run NASM in parallel: > > takahashi at ubuntu:~/Desktop/MA-DDM/Cpp/Rectangle$ mpiexec -n 4 ./test1 > -t1_N > > 20 -snes_max_it 50 -snes_monitor -snes_view -da_overlap 3 -snes_type nasm > > -snes_nasm_type restrict > > 0 SNES Function norm 7.244681057908e+02 > > 1 SNES Function norm 1.237688062971e+02 > > 2 SNES Function norm 1.068926073552e+02 > > 3 SNES Function norm 1.027563237834e+02 > > 4 SNES Function norm 1.022184806736e+02 > > 5 SNES Function norm 1.020818227640e+02 > > 6 SNES Function norm 1.020325629121e+02 > > 7 SNES Function norm 1.020149036595e+02 > > 8 SNES Function norm 1.020088110545e+02 > > 9 SNES Function norm 1.020067198030e+02 > > 10 SNES Function norm 1.020060034469e+02 > > 11 SNES Function norm 1.020057582380e+02 > > 12 SNES Function norm 1.020056743241e+02 > > 13 SNES Function norm 1.020056456101e+02 > > 14 SNES Function norm 1.020056357849e+02 > > 15 SNES Function norm 1.020056324231e+02 > > 16 SNES Function norm 1.020056312727e+02 > > 17 SNES Function norm 1.020056308791e+02 > > 18 SNES Function norm 1.020056307444e+02 > > 19 SNES Function norm 1.020056306983e+02 > > 20 SNES Function norm 1.020056306826e+02 > > 21 SNES Function norm 1.020056306772e+02 > > 22 SNES Function norm 1.020056306753e+02 > > 23 SNES Function norm 1.020056306747e+02 > > 24 SNES Function norm 1.020056306745e+02 > > 25 SNES Function norm 1.020056306744e+02 > > 26 SNES Function norm 1.020056306744e+02 > > 27 SNES Function norm 1.020056306744e+02 > > 28 SNES Function norm 1.020056306744e+02 > > 29 SNES Function norm 1.020056306744e+02 > > 30 SNES Function norm 1.020056306744e+02 > > 31 SNES Function norm 1.020056306744e+02 > > 32 SNES Function norm 1.020056306744e+02 > > 33 SNES Function norm 1.020056306744e+02 > > 34 SNES Function norm 1.020056306744e+02 > > 35 SNES Function norm 1.020056306744e+02 > > 36 SNES Function norm 1.020056306744e+02 > > 37 SNES Function norm 1.020056306744e+02 > > 38 SNES Function norm 1.020056306744e+02 > > 39 SNES Function norm 1.020056306744e+02 > > 40 SNES Function norm 1.020056306744e+02 > > 41 SNES Function norm 1.020056306744e+02 > > 42 SNES Function norm 1.020056306744e+02 > > 43 SNES Function norm 1.020056306744e+02 > > 44 SNES Function norm 1.020056306744e+02 > > 45 SNES Function norm 1.020056306744e+02 > > 46 SNES Function norm 1.020056306744e+02 > > 47 SNES Function norm 1.020056306744e+02 > > 48 SNES Function norm 1.020056306744e+02 > > 49 SNES Function norm 1.020056306744e+02 > > 50 SNES Function norm 1.020056306744e+02 > > SNES Object: 4 MPI processes > > type: nasm > > total subdomain blocks = 4 > > Local solver information for first block on rank 0: > > Use -snes_view ::ascii_info_detail to display information for all > blocks > > SNES Object: (sub_) 1 MPI processes > > type: newtonls > > maximum iterations=50, maximum function evaluations=10000 > > tolerances: relative=1e-08, absolute=1e-50, solution=1e-08 > > total number of linear solver iterations=22 > > total number of function evaluations=40 > > norm schedule ALWAYS > > Jacobian is built using a DMDA local Jacobian > > SNESLineSearch Object: (sub_) 1 MPI processes > > type: bt > > interpolation: cubic > > alpha=1.000000e-04 > > maxstep=1.000000e+08, minlambda=1.000000e-12 > > tolerances: relative=1.000000e-08, absolute=1.000000e-15, > > lambda=1.000000e-08 > > maximum iterations=40 > > KSP Object: (sub_) 1 MPI processes > > type: preonly > > maximum iterations=10000, initial guess is zero > > tolerances: relative=1e-05, absolute=1e-50, divergence=10000. > > left preconditioning > > using NONE norm type for convergence test > > PC Object: (sub_) 1 MPI processes > > type: lu > > out-of-place factorization > > tolerance for zero pivot 2.22045e-14 > > matrix ordering: nd > > factor fill ratio given 5., needed 2.13732 > > Factored matrix follows: > > Mat Object: 1 MPI processes > > type: seqaij > > rows=169, cols=169 > > package used to perform factorization: petsc > > total: nonzeros=13339, allocated nonzeros=13339 > > using I-node routines: found 104 nodes, limit used is 5 > > linear system matrix = precond matrix: > > Mat Object: 1 MPI processes > > type: seqaij > > rows=169, cols=169 > > total: nonzeros=6241, allocated nonzeros=6241 > > total number of mallocs used during MatSetValues calls=0 > > not using I-node routines > > maximum iterations=50, maximum function evaluations=10000 > > tolerances: relative=1e-08, absolute=1e-50, solution=1e-08 > > total number of function evaluations=51 > > norm schedule ALWAYS > > Jacobian is built using a DMDA local Jacobian > > problem ex10 on 20 x 20 point 2D grid with d = 3, and eps = 0.082: > > error |u-uexact|_inf = 3.996e-01, |u-uexact|_h = 2.837e-01 > > > > We have been stuck on this for a while now. We do not know how to debug > > this issue. Please let us know if you have any insights. > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- 0 SNES Function norm 7.244681057908e+02 Line search: Using full step: fnorm 2.366767922608e+03 gnorm 7.324644673012e+02 Line search: Using full step: fnorm 2.366767922608e+03 gnorm 7.324644673012e+02 Line search: Using full step: fnorm 2.366767922608e+03 gnorm 7.368257945624e+02 Line search: Using full step: fnorm 2.366767922608e+03 gnorm 7.368257945624e+02 Line search: Using full step: fnorm 7.324644673012e+02 gnorm 3.030339094711e+02 Line search: Using full step: fnorm 7.324644673012e+02 gnorm 3.030339094711e+02 Line search: Using full step: fnorm 7.368257945624e+02 gnorm 3.036925065488e+02 Line search: Using full step: fnorm 7.368257945624e+02 gnorm 3.036925065488e+02 Line search: Using full step: fnorm 3.030339094711e+02 gnorm 1.710013987307e+02 Line search: Using full step: fnorm 3.030339094711e+02 gnorm 1.710013987307e+02 Line search: Using full step: fnorm 3.036925065488e+02 gnorm 1.710672486390e+02 Line search: Using full step: fnorm 3.036925065488e+02 gnorm 1.710672486390e+02 Line search: gnorm after quadratic fit 1.064831535546e+02 Line search: Quadratically determined step, lambda=4.8598042423461241e-01 Line search: gnorm after quadratic fit 1.064831535546e+02 Line search: Quadratically determined step, lambda=4.8598042423461130e-01 Line search: gnorm after quadratic fit 1.064869986827e+02 Line search: Quadratically determined step, lambda=4.8628691897289167e-01 Line search: gnorm after quadratic fit 1.064869986827e+02 Line search: Quadratically determined step, lambda=4.8628691897289139e-01 Line search: Using full step: fnorm 1.064831535546e+02 gnorm 2.437895828773e+01 Line search: Using full step: fnorm 1.064831535546e+02 gnorm 2.437895828773e+01 Line search: Using full step: fnorm 1.064869986827e+02 gnorm 2.437541131039e+01 Line search: Using full step: fnorm 1.064869986827e+02 gnorm 2.437541131039e+01 Line search: gnorm after quadratic fit 1.904344322941e+01 Line search: Quadratically determined step, lambda=2.3856317774238858e-01 Line search: gnorm after quadratic fit 1.904344322942e+01 Line search: Quadratically determined step, lambda=2.3856317774239236e-01 Line search: gnorm after quadratic fit 1.904019307986e+01 Line search: Quadratically determined step, lambda=2.3849940999249908e-01 Line search: gnorm after quadratic fit 1.904019307986e+01 Line search: Quadratically determined step, lambda=2.3849940999251007e-01 Line search: Using full step: fnorm 1.904344322942e+01 gnorm 1.534182314959e+00 Line search: Using full step: fnorm 1.904344322941e+01 gnorm 1.534182314959e+00 Line search: Using full step: fnorm 1.904019307986e+01 gnorm 1.533532574103e+00 Line search: Using full step: fnorm 1.904019307986e+01 gnorm 1.533532574104e+00 Line search: Using full step: fnorm 1.534182314959e+00 gnorm 4.230505579099e-02 Line search: Using full step: fnorm 1.534182314959e+00 gnorm 4.230505579101e-02 Line search: Using full step: fnorm 1.533532574103e+00 gnorm 4.226729088952e-02 Line search: Using full step: fnorm 1.533532574104e+00 gnorm 4.226729088953e-02 Line search: Using full step: fnorm 4.230505579101e-02 gnorm 4.234652015899e-05 Line search: Using full step: fnorm 4.230505579099e-02 gnorm 4.234652021221e-05 Line search: Using full step: fnorm 4.226729088952e-02 gnorm 4.226810598217e-05 Line search: Using full step: fnorm 4.226729088953e-02 gnorm 4.226810597805e-05 Line search: Using full step: fnorm 4.234652021221e-05 gnorm 4.372642285455e-11 Line search: Using full step: fnorm 4.234652015899e-05 gnorm 4.372470547882e-11 Line search: Using full step: fnorm 4.226810597805e-05 gnorm 4.356893319390e-11 Line search: Using full step: fnorm 4.226810598217e-05 gnorm 4.352576718601e-11 1 SNES Function norm 1.237688062971e+02 Line search: Using full step: fnorm 4.440079063783e+02 gnorm 1.591389440443e+02 Line search: Using full step: fnorm 4.440079063783e+02 gnorm 1.591389440443e+02 Line search: Using full step: fnorm 4.440079063783e+02 gnorm 1.591389440443e+02 Line search: Using full step: fnorm 4.440079063783e+02 gnorm 1.591389440443e+02 Line search: Using full step: fnorm 1.591389440443e+02 gnorm 6.251095458324e+01 Line search: Using full step: fnorm 1.591389440443e+02 gnorm 6.251095458324e+01 Line search: Using full step: fnorm 1.591389440443e+02 gnorm 6.251095458324e+01 Line search: Using full step: fnorm 1.591389440443e+02 gnorm 6.251095458324e+01 Line search: Using full step: fnorm 6.251095458324e+01 gnorm 4.464370931338e+01 Line search: Using full step: fnorm 6.251095458324e+01 gnorm 4.464370931338e+01 Line search: Using full step: fnorm 6.251095458324e+01 gnorm 4.464370931338e+01 Line search: Using full step: fnorm 6.251095458324e+01 gnorm 4.464370931338e+01 Line search: Using full step: fnorm 4.464370931338e+01 gnorm 2.153664033989e+01 Line search: Using full step: fnorm 4.464370931338e+01 gnorm 2.153664033989e+01 Line search: Using full step: fnorm 4.464370931338e+01 gnorm 2.153664033989e+01 Line search: Using full step: fnorm 4.464370931338e+01 gnorm 2.153664033989e+01 Line search: Using full step: fnorm 2.153664033989e+01 gnorm 2.367280398205e+00 Line search: Using full step: fnorm 2.153664033989e+01 gnorm 2.367280398205e+00 Line search: Using full step: fnorm 2.153664033989e+01 gnorm 2.367280398205e+00 Line search: Using full step: fnorm 2.153664033989e+01 gnorm 2.367280398205e+00 Line search: Using full step: fnorm 2.367280398205e+00 gnorm 1.607012542246e-01 Line search: Using full step: fnorm 2.367280398205e+00 gnorm 1.607012542246e-01 Line search: Using full step: fnorm 2.367280398205e+00 gnorm 1.607012542248e-01 Line search: Using full step: fnorm 2.367280398205e+00 gnorm 1.607012542246e-01 Line search: Using full step: fnorm 1.607012542246e-01 gnorm 1.008059167344e-03 Line search: Using full step: fnorm 1.607012542246e-01 gnorm 1.008059167299e-03 Line search: Using full step: fnorm 1.607012542248e-01 gnorm 1.008059167106e-03 Line search: Using full step: fnorm 1.607012542246e-01 gnorm 1.008059167242e-03 Line search: Using full step: fnorm 1.008059167344e-03 gnorm 5.435658530662e-08 Line search: Using full step: fnorm 1.008059167299e-03 gnorm 5.435653068089e-08 Line search: Using full step: fnorm 1.008059167106e-03 gnorm 5.435660600498e-08 Line search: Using full step: fnorm 1.008059167242e-03 gnorm 5.435649103726e-08 2 SNES Function norm 1.068926073552e+02 Line search: Using full step: fnorm 2.068615558922e+02 gnorm 1.393723341438e+02 Line search: Using full step: fnorm 2.068615558922e+02 gnorm 1.393723341438e+02 Line search: Using full step: fnorm 2.068615558922e+02 gnorm 1.393723341438e+02 Line search: Using full step: fnorm 2.068615558922e+02 gnorm 1.393723341438e+02 Line search: Using full step: fnorm 1.393723341438e+02 gnorm 5.409936246836e+01 Line search: Using full step: fnorm 1.393723341438e+02 gnorm 5.409936246836e+01 Line search: Using full step: fnorm 1.393723341438e+02 gnorm 5.409936246836e+01 Line search: Using full step: fnorm 1.393723341438e+02 gnorm 5.409936246836e+01 Line search: gnorm after quadratic fit 3.800259660005e+01 Line search: Quadratically determined step, lambda=3.8911460926523889e-01 Line search: gnorm after quadratic fit 3.800259660005e+01 Line search: Quadratically determined step, lambda=3.8911460926523883e-01 Line search: gnorm after quadratic fit 3.800259660005e+01 Line search: Quadratically determined step, lambda=3.8911460926523928e-01 Line search: gnorm after quadratic fit 3.800259660005e+01 Line search: Quadratically determined step, lambda=3.8911460926523894e-01 Line search: Using full step: fnorm 3.800259660005e+01 gnorm 9.463901192391e+00 Line search: Using full step: fnorm 3.800259660005e+01 gnorm 9.463901192391e+00 Line search: Using full step: fnorm 3.800259660005e+01 gnorm 9.463901192391e+00 Line search: Using full step: fnorm 3.800259660005e+01 gnorm 9.463901192391e+00 Line search: Using full step: fnorm 9.463901192391e+00 gnorm 4.333340417561e+00 Line search: Using full step: fnorm 9.463901192391e+00 gnorm 4.333340417561e+00 Line search: Using full step: fnorm 9.463901192391e+00 gnorm 4.333340417561e+00 Line search: Using full step: fnorm 9.463901192391e+00 gnorm 4.333340417561e+00 Line search: Using full step: fnorm 4.333340417561e+00 gnorm 3.981910153224e-01 Line search: Using full step: fnorm 4.333340417561e+00 gnorm 3.981910153224e-01 Line search: Using full step: fnorm 4.333340417561e+00 gnorm 3.981910153224e-01 Line search: Using full step: fnorm 4.333340417561e+00 gnorm 3.981910153224e-01 Line search: Using full step: fnorm 3.981910153224e-01 gnorm 1.168795270601e-03 Line search: Using full step: fnorm 3.981910153224e-01 gnorm 1.168795270614e-03 Line search: Using full step: fnorm 3.981910153224e-01 gnorm 1.168795270615e-03 Line search: Using full step: fnorm 3.981910153224e-01 gnorm 1.168795270605e-03 Line search: Using full step: fnorm 1.168795270601e-03 gnorm 5.836712980412e-08 Line search: Using full step: fnorm 1.168795270614e-03 gnorm 5.836712173140e-08 Line search: Using full step: fnorm 1.168795270615e-03 gnorm 5.836709729918e-08 Line search: Using full step: fnorm 1.168795270605e-03 gnorm 5.836712171400e-08 3 SNES Function norm 1.027563237834e+02 Line search: gnorm after quadratic fit 5.896621528793e+01 Line search: Quadratically determined step, lambda=3.3417411759577009e-01 Line search: gnorm after quadratic fit 5.896621528793e+01 Line search: Quadratically determined step, lambda=3.3417411759577059e-01 Line search: gnorm after quadratic fit 5.896621528793e+01 Line search: Quadratically determined step, lambda=3.3417411759577054e-01 Line search: gnorm after quadratic fit 5.896621528793e+01 Line search: Quadratically determined step, lambda=3.3417411759577015e-01 Line search: gnorm after quadratic fit 4.833412049945e+01 Line search: Quadratically determined step, lambda=1.8153887761288992e-01 Line search: gnorm after quadratic fit 4.833412049945e+01 Line search: Quadratically determined step, lambda=1.8153887761288856e-01 Line search: gnorm after quadratic fit 4.833412049945e+01 Line search: Quadratically determined step, lambda=1.8153887761288998e-01 Line search: gnorm after quadratic fit 4.833412049945e+01 Line search: Quadratically determined step, lambda=1.8153887761288900e-01 Line search: gnorm after quadratic fit 4.211065287968e+01 Line search: Quadratically determined step, lambda=1.2777212868296678e-01 Line search: gnorm after quadratic fit 4.211065287968e+01 Line search: Quadratically determined step, lambda=1.2777212868296670e-01 Line search: gnorm after quadratic fit 4.211065287968e+01 Line search: Quadratically determined step, lambda=1.2777212868296631e-01 Line search: gnorm after quadratic fit 4.211065287968e+01 Line search: Quadratically determined step, lambda=1.2777212868296728e-01 Line search: gnorm after quadratic fit 3.737772762235e+01 Line search: Quadratically determined step, lambda=1.1268327473970392e-01 Line search: gnorm after quadratic fit 3.737772762235e+01 Line search: Quadratically determined step, lambda=1.1268327473970409e-01 Line search: gnorm after quadratic fit 3.737772762235e+01 Line search: Quadratically determined step, lambda=1.1268327473970415e-01 Line search: gnorm after quadratic fit 3.737772762235e+01 Line search: Quadratically determined step, lambda=1.1268327473970390e-01 Line search: gnorm after quadratic fit 3.358979454151e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.358979454151e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.358979454151e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.358979454151e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.997444184268e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.997444184268e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.997444184268e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.997444184268e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.689133513643e+01 Line search: Quadratically determined step, lambda=1.0370873563461103e-01 Line search: gnorm after quadratic fit 2.689133513643e+01 Line search: Quadratically determined step, lambda=1.0370873563460987e-01 Line search: gnorm after quadratic fit 2.689133513643e+01 Line search: Quadratically determined step, lambda=1.0370873563461043e-01 Line search: gnorm after quadratic fit 2.689133513643e+01 Line search: Quadratically determined step, lambda=1.0370873563461103e-01 Line search: gnorm after quadratic fit 2.422821965493e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.422821965493e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.422821965493e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.422821965493e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.170508461365e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.170508461365e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.170508461365e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.170508461365e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.869338837974e+01 Line search: Quadratically determined step, lambda=1.4354823991582735e-01 Line search: gnorm after quadratic fit 1.869338837974e+01 Line search: Quadratically determined step, lambda=1.4354823991582763e-01 Line search: gnorm after quadratic fit 1.869338837974e+01 Line search: Quadratically determined step, lambda=1.4354823991582627e-01 Line search: gnorm after quadratic fit 1.869338837974e+01 Line search: Quadratically determined step, lambda=1.4354823991582696e-01 Line search: gnorm after quadratic fit 1.649244414319e+01 Line search: Quadratically determined step, lambda=1.1558407038803790e-01 Line search: gnorm after quadratic fit 1.649244414318e+01 Line search: Quadratically determined step, lambda=1.1558407038805768e-01 Line search: gnorm after quadratic fit 1.649244414318e+01 Line search: Quadratically determined step, lambda=1.1558407038806513e-01 Line search: gnorm after quadratic fit 1.649244414319e+01 Line search: Quadratically determined step, lambda=1.1558407038803363e-01 Line search: gnorm after quadratic fit 1.408701910902e+01 Line search: Quadratically determined step, lambda=1.4400542250644904e-01 Line search: gnorm after quadratic fit 1.408701910902e+01 Line search: Quadratically determined step, lambda=1.4400542250644563e-01 Line search: gnorm after quadratic fit 1.408701910902e+01 Line search: Quadratically determined step, lambda=1.4400542250644327e-01 Line search: gnorm after quadratic fit 1.408701910902e+01 Line search: Quadratically determined step, lambda=1.4400542250645118e-01 Line search: gnorm after quadratic fit 1.252374072903e+01 Line search: Quadratically determined step, lambda=1.1638377830367309e-01 Line search: gnorm after quadratic fit 1.252374072903e+01 Line search: Quadratically determined step, lambda=1.1638377830366374e-01 Line search: gnorm after quadratic fit 1.252374072903e+01 Line search: Quadratically determined step, lambda=1.1638377830366450e-01 Line search: gnorm after quadratic fit 1.252374072903e+01 Line search: Quadratically determined step, lambda=1.1638377830366908e-01 Line search: gnorm after quadratic fit 1.073719373988e+01 Line search: gnorm after quadratic fit 1.073719373988e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.073719373988e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.073719373988e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 1.073719373988e+01 gnorm 3.796196824527e+00 Line search: Using full step: fnorm 1.073719373988e+01 gnorm 3.796196824527e+00 Line search: Using full step: fnorm 1.073719373988e+01 gnorm 3.796196824527e+00 Line search: Using full step: fnorm 1.073719373988e+01 gnorm 3.796196824527e+00 Line search: Using full step: fnorm 3.796196824527e+00 gnorm 3.100229870781e-01 Line search: Using full step: fnorm 3.796196824527e+00 gnorm 3.100229870781e-01 Line search: Using full step: fnorm 3.796196824527e+00 gnorm 3.100229870781e-01 Line search: Using full step: fnorm 3.796196824527e+00 gnorm 3.100229870782e-01 Line search: Using full step: fnorm 3.100229870781e-01 gnorm 3.833811946146e-03 Line search: Using full step: fnorm 3.100229870781e-01 gnorm 3.833811945925e-03 Line search: Using full step: fnorm 3.100229870782e-01 gnorm 3.833811946143e-03 Line search: Using full step: fnorm 3.100229870781e-01 gnorm 3.833811946166e-03 Line search: Using full step: fnorm 3.833811946146e-03 gnorm 6.463398868533e-07 Line search: Using full step: fnorm 3.833811945925e-03 gnorm 6.463398761639e-07 Line search: Using full step: fnorm 3.833811946143e-03 gnorm 6.463398805254e-07 Line search: Using full step: fnorm 3.833811946166e-03 gnorm 6.463398843583e-07 4 SNES Function norm 1.022184806736e+02 Line search: gnorm after quadratic fit 5.051193144184e+01 Line search: Quadratically determined step, lambda=2.3451615293273340e-01 Line search: gnorm after quadratic fit 5.051193144184e+01 Line search: Quadratically determined step, lambda=2.3451615293273262e-01 Line search: gnorm after quadratic fit 5.051193144184e+01 Line search: Quadratically determined step, lambda=2.3451615293273320e-01 Line search: gnorm after quadratic fit 5.051193144184e+01 Line search: Quadratically determined step, lambda=2.3451615293273356e-01 Line search: gnorm after quadratic fit 4.269992085591e+01 Line search: Quadratically determined step, lambda=1.5271197767584788e-01 Line search: gnorm after quadratic fit 4.269992085591e+01 Line search: Quadratically determined step, lambda=1.5271197767584857e-01 Line search: gnorm after quadratic fit 4.269992085591e+01 Line search: Quadratically determined step, lambda=1.5271197767584768e-01 Line search: gnorm after quadratic fit 4.269992085591e+01 Line search: Quadratically determined step, lambda=1.5271197767584765e-01 Line search: gnorm after quadratic fit 3.798269131216e+01 Line search: Quadratically determined step, lambda=1.0906531365480246e-01 Line search: gnorm after quadratic fit 3.798269131216e+01 Line search: Quadratically determined step, lambda=1.0906531365480227e-01 Line search: gnorm after quadratic fit 3.798269131216e+01 Line search: Quadratically determined step, lambda=1.0906531365480203e-01 Line search: gnorm after quadratic fit 3.798269131216e+01 Line search: Quadratically determined step, lambda=1.0906531365480178e-01 Line search: gnorm after quadratic fit 3.417393882200e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.417393882200e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.417393882200e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.417393882200e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.058425475891e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.058425475891e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.058425475891e+01 Line search: gnorm after quadratic fit 3.058425475891e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.749352490002e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.749352490002e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.749352490002e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.749352490002e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.459358093530e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.459358093530e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.459358093530e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.459358093530e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.213608601165e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.213608601165e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.213608601165e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.213608601165e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.991264702666e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.991264702666e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.991264702666e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.991264702666e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.792317241133e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.792317241133e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.792317241133e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.792317241133e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.573510323244e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.573510323244e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.573510323244e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.573510323244e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.419005399634e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.419005399634e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.419005399634e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.419005399634e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.203189063298e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.203189063298e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.203189063298e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.203189063298e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.083620093528e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.083620093528e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.083620093528e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.083620093528e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.761171525567e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.761171525567e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.761171525567e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.761171525567e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.829619997117e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.829619997117e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.829619997117e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.829619997117e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 7.349100833091e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 7.349100833091e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 7.349100833091e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 7.349100833091e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 7.349100833091e+00 gnorm 3.047940208239e+00 Line search: Using full step: fnorm 7.349100833091e+00 gnorm 3.047940208238e+00 Line search: Using full step: fnorm 7.349100833091e+00 gnorm 3.047940208238e+00 Line search: Using full step: fnorm 7.349100833091e+00 gnorm 3.047940208238e+00 Line search: Using full step: fnorm 3.047940208239e+00 gnorm 1.931726734637e-01 Line search: Using full step: fnorm 3.047940208238e+00 gnorm 1.931726734637e-01 Line search: Using full step: fnorm 3.047940208238e+00 gnorm 1.931726734637e-01 Line search: Using full step: fnorm 3.047940208238e+00 gnorm 1.931726734637e-01 Line search: Using full step: fnorm 1.931726734637e-01 gnorm 1.178710338200e-03 Line search: Using full step: fnorm 1.931726734637e-01 gnorm 1.178710338211e-03 Line search: Using full step: fnorm 1.931726734637e-01 gnorm 1.178710338199e-03 Line search: Using full step: fnorm 1.931726734637e-01 gnorm 1.178710338187e-03 Line search: Using full step: fnorm 1.178710338200e-03 gnorm 5.190129168255e-08 Line search: Using full step: fnorm 1.178710338211e-03 gnorm 5.190120960392e-08 Line search: Using full step: fnorm 1.178710338199e-03 gnorm 5.190130947964e-08 Line search: Using full step: fnorm 1.178710338187e-03 gnorm 5.190130139428e-08 5 SNES Function norm 1.020818227640e+02 Line search: gnorm after quadratic fit 4.886628019956e+01 Line search: Quadratically determined step, lambda=2.2353485288897126e-01 Line search: gnorm after quadratic fit 4.886628019956e+01 Line search: Quadratically determined step, lambda=2.2353485288897157e-01 Line search: gnorm after quadratic fit 4.886628019956e+01 Line search: Quadratically determined step, lambda=2.2353485288897193e-01 Line search: gnorm after quadratic fit 4.886628019956e+01 Line search: Quadratically determined step, lambda=2.2353485288897082e-01 Line search: gnorm after quadratic fit 4.156511898210e+01 Line search: Quadratically determined step, lambda=1.4738191720329524e-01 Line search: gnorm after quadratic fit 4.156511898210e+01 Line search: Quadratically determined step, lambda=1.4738191720329444e-01 Line search: gnorm after quadratic fit 4.156511898210e+01 Line search: Quadratically determined step, lambda=1.4738191720329460e-01 Line search: gnorm after quadratic fit 4.156511898210e+01 Line search: Quadratically determined step, lambda=1.4738191720329588e-01 Line search: gnorm after quadratic fit 3.629487597696e+01 Line search: Quadratically determined step, lambda=1.2653580820893240e-01 Line search: gnorm after quadratic fit 3.629487597696e+01 Line search: Quadratically determined step, lambda=1.2653580820893176e-01 Line search: gnorm after quadratic fit 3.629487597696e+01 Line search: Quadratically determined step, lambda=1.2653580820893223e-01 Line search: gnorm after quadratic fit 3.629487597696e+01 Line search: Quadratically determined step, lambda=1.2653580820893293e-01 Line search: gnorm after quadratic fit 3.255668624221e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.255668624221e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.255668624221e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.255668624221e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.923824703249e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.923824703249e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.923824703249e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.923824703249e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.622891008102e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.622891008102e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.622891008102e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.622891008102e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.346428241159e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.346428241159e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.346428241159e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.346428241159e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.087798992161e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.087798992161e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.087798992161e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.087798992161e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.879053026538e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.879053026538e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.879053026538e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.879053026538e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.691064054715e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.691064054715e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.691064054715e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.691064054715e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473972706400e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473972706400e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473972706400e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473972706400e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.331644381674e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.331644381674e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.331644381674e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.331644381674e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.120935946848e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.120935946848e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.120935946848e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.120935946848e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.009411971383e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.009411971383e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.009411971383e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.009411971383e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.139993160275e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.139993160275e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.139993160275e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.139993160274e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.295651769362e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.295651769362e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.295651769362e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.295651769362e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.929821666362e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.929821666362e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.929821666362e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.929821666362e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 6.929821666362e+00 gnorm 3.811156538807e+00 Line search: Using full step: fnorm 6.929821666362e+00 gnorm 3.811156538808e+00 Line search: Using full step: fnorm 6.929821666362e+00 gnorm 3.811156538808e+00 Line search: Using full step: fnorm 6.929821666362e+00 gnorm 3.811156538807e+00 Line search: Using full step: fnorm 3.811156538807e+00 gnorm 2.895412800626e-01 Line search: Using full step: fnorm 3.811156538808e+00 gnorm 2.895412800628e-01 Line search: Using full step: fnorm 3.811156538808e+00 gnorm 2.895412800627e-01 Line search: Using full step: fnorm 3.811156538807e+00 gnorm 2.895412800629e-01 Line search: Using full step: fnorm 2.895412800626e-01 gnorm 2.877381403695e-03 Line search: Using full step: fnorm 2.895412800628e-01 gnorm 2.877381403684e-03 Line search: Using full step: fnorm 2.895412800627e-01 gnorm 2.877381403700e-03 Line search: Using full step: fnorm 2.895412800629e-01 gnorm 2.877381403684e-03 Line search: Using full step: fnorm 2.877381403695e-03 gnorm 3.430505065554e-07 Line search: Using full step: fnorm 2.877381403684e-03 gnorm 3.430506224103e-07 Line search: Using full step: fnorm 2.877381403700e-03 gnorm 3.430505172022e-07 Line search: Using full step: fnorm 2.877381403684e-03 gnorm 3.430506209398e-07 6 SNES Function norm 1.020325629121e+02 Line search: gnorm after quadratic fit 4.849065472094e+01 Line search: Quadratically determined step, lambda=2.2227399338709344e-01 Line search: gnorm after quadratic fit 4.849065472094e+01 Line search: Quadratically determined step, lambda=2.2227399338709419e-01 Line search: gnorm after quadratic fit 4.849065472094e+01 Line search: Quadratically determined step, lambda=2.2227399338709405e-01 Line search: gnorm after quadratic fit 4.849065472094e+01 Line search: Quadratically determined step, lambda=2.2227399338709330e-01 Line search: gnorm after quadratic fit 4.129572063278e+01 Line search: Quadratically determined step, lambda=1.4630139507222042e-01 Line search: gnorm after quadratic fit 4.129572063278e+01 Line search: Quadratically determined step, lambda=1.4630139507221984e-01 Line search: gnorm after quadratic fit 4.129572063278e+01 Line search: Quadratically determined step, lambda=1.4630139507222006e-01 Line search: gnorm after quadratic fit 4.129572063278e+01 Line search: Quadratically determined step, lambda=1.4630139507222079e-01 Line search: gnorm after quadratic fit 3.609599739550e+01 Line search: Quadratically determined step, lambda=1.2573103389295170e-01 Line search: gnorm after quadratic fit 3.609599739550e+01 Line search: Quadratically determined step, lambda=1.2573103389295126e-01 Line search: gnorm after quadratic fit 3.609599739550e+01 Line search: Quadratically determined step, lambda=1.2573103389295212e-01 Line search: gnorm after quadratic fit 3.609599739550e+01 Line search: Quadratically determined step, lambda=1.2573103389295126e-01 Line search: gnorm after quadratic fit 3.237486508254e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.237486508254e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.237486508254e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.237486508254e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.907078033076e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.907078033076e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.907078033076e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.907078033076e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.606250910062e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.606250910062e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.606250910062e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.606250910062e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.329251388799e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.329251388799e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.329251388799e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.329251388799e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.095123167273e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.095123167273e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.095123167273e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.095123167273e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.885175444728e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.885175444728e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.885175444728e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.885175444728e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.696386123167e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.696386123167e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.696386123167e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.696386123167e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.479780029534e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.479780029534e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.479780029534e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.479780029534e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.322264859603e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.322264859603e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.322264859603e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.322264859603e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.112379529750e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.112379529750e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.112379529750e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.112379529750e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.001772385067e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.001772385067e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.001772385067e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.001772385067e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.086635603164e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.086635603164e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.086635603164e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.086635603164e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.253113508182e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.253113508182e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.253113508182e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.253113508182e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.978398714791e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.978398714791e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.978398714791e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.978398714791e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 6.978398714791e+00 gnorm 4.701073111337e+00 Line search: Using full step: fnorm 6.978398714791e+00 gnorm 4.701073111337e+00 Line search: Using full step: fnorm 6.978398714791e+00 gnorm 4.701073111336e+00 Line search: Using full step: fnorm 6.978398714791e+00 gnorm 4.701073111337e+00 Line search: Using full step: fnorm 4.701073111337e+00 gnorm 4.125054503988e-01 Line search: Using full step: fnorm 4.701073111337e+00 gnorm 4.125054503987e-01 Line search: Using full step: fnorm 4.701073111336e+00 gnorm 4.125054503987e-01 Line search: Using full step: fnorm 4.701073111337e+00 gnorm 4.125054503989e-01 Line search: Using full step: fnorm 4.125054503988e-01 gnorm 5.939967797068e-03 Line search: Using full step: fnorm 4.125054503987e-01 gnorm 5.939967797220e-03 Line search: Using full step: fnorm 4.125054503987e-01 gnorm 5.939967797213e-03 Line search: Using full step: fnorm 4.125054503989e-01 gnorm 5.939967797204e-03 Line search: Using full step: fnorm 5.939967797068e-03 gnorm 1.486255672053e-06 Line search: Using full step: fnorm 5.939967797220e-03 gnorm 1.486255777495e-06 Line search: Using full step: fnorm 5.939967797213e-03 gnorm 1.486255775476e-06 Line search: Using full step: fnorm 5.939967797204e-03 gnorm 1.486255670687e-06 Line search: Using full step: fnorm 1.486255672053e-06 gnorm 1.166495899486e-12 Line search: Using full step: fnorm 1.486255777495e-06 gnorm 1.108596666737e-12 Line search: Using full step: fnorm 1.486255775476e-06 gnorm 1.003003229493e-12 Line search: Using full step: fnorm 1.486255670687e-06 gnorm 9.972379925769e-13 7 SNES Function norm 1.020149036595e+02 Line search: gnorm after quadratic fit 4.838147275872e+01 Line search: Quadratically determined step, lambda=2.2207338891545284e-01 Line search: gnorm after quadratic fit 4.838147275872e+01 Line search: Quadratically determined step, lambda=2.2207338891545234e-01 Line search: gnorm after quadratic fit 4.838147275872e+01 Line search: Quadratically determined step, lambda=2.2207338891545256e-01 Line search: gnorm after quadratic fit 4.838147275872e+01 Line search: Quadratically determined step, lambda=2.2207338891545136e-01 Line search: gnorm after quadratic fit 4.121442580568e+01 Line search: Quadratically determined step, lambda=1.4603704675067056e-01 Line search: gnorm after quadratic fit 4.121442580568e+01 Line search: Quadratically determined step, lambda=1.4603704675066992e-01 Line search: gnorm after quadratic fit 4.121442580568e+01 Line search: Quadratically determined step, lambda=1.4603704675067067e-01 Line search: gnorm after quadratic fit 4.121442580568e+01 Line search: Quadratically determined step, lambda=1.4603704675066970e-01 Line search: gnorm after quadratic fit 3.604303174308e+01 Line search: Quadratically determined step, lambda=1.2531846663251531e-01 Line search: gnorm after quadratic fit 3.604303174308e+01 Line search: Quadratically determined step, lambda=1.2531846663251558e-01 Line search: gnorm after quadratic fit 3.604303174308e+01 Line search: Quadratically determined step, lambda=1.2531846663251572e-01 Line search: gnorm after quadratic fit 3.604303174308e+01 Line search: Quadratically determined step, lambda=1.2531846663251631e-01 Line search: gnorm after quadratic fit 3.232563641995e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.232563641995e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.232563641995e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.232563641995e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.902451491358e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.902451491358e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.902451491358e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.902451491358e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.601539366454e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.601539366454e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.601539366454e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.601539366454e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.322301611665e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.322301611665e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.322301611665e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.322301611665e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.089046648609e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.089046648609e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.089046648609e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.089046648609e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.879672311405e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.879672311405e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.879672311405e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.879672311405e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.691425389869e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.691425389869e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.691425389869e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.691425389869e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.475592577765e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.475592577765e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.475592577765e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.475592577765e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.318368517224e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.318368517224e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.318368517224e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.318368517224e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.109051115021e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.109051115021e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.109051115021e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.109051115021e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.987518649316e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.987518649316e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.987518649316e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.987518649316e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.065233700103e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.065233700103e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.065233700103e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.065233700103e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.235976027732e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.235976027732e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.235976027732e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.235976027732e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.990239949430e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.990239949430e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.990239949430e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.990239949430e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 6.990239949430e+00 gnorm 5.066433100375e+00 Line search: Using full step: fnorm 6.990239949430e+00 gnorm 5.066433100374e+00 Line search: Using full step: fnorm 6.990239949430e+00 gnorm 5.066433100374e+00 Line search: Using full step: fnorm 6.990239949430e+00 gnorm 5.066433100374e+00 Line search: Using full step: fnorm 5.066433100375e+00 gnorm 4.665323835740e-01 Line search: Using full step: fnorm 5.066433100374e+00 gnorm 4.665323835740e-01 Line search: Using full step: fnorm 5.066433100374e+00 gnorm 4.665323835741e-01 Line search: Using full step: fnorm 5.066433100374e+00 gnorm 4.665323835739e-01 Line search: Using full step: fnorm 4.665323835740e-01 gnorm 7.619474682764e-03 Line search: Using full step: fnorm 4.665323835740e-01 gnorm 7.619474682879e-03 Line search: Using full step: fnorm 4.665323835741e-01 gnorm 7.619474682768e-03 Line search: Using full step: fnorm 4.665323835739e-01 gnorm 7.619474682920e-03 Line search: Using full step: fnorm 7.619474682764e-03 gnorm 2.454074436388e-06 Line search: Using full step: fnorm 7.619474682879e-03 gnorm 2.454074554498e-06 Line search: Using full step: fnorm 7.619474682768e-03 gnorm 2.454074425433e-06 Line search: Using full step: fnorm 7.619474682920e-03 gnorm 2.454074566500e-06 Line search: Using full step: fnorm 2.454074436388e-06 gnorm 9.474867051356e-13 Line search: Using full step: fnorm 2.454074554498e-06 gnorm 9.363644684245e-13 Line search: Using full step: fnorm 2.454074425433e-06 gnorm 9.754856446921e-13 Line search: Using full step: fnorm 2.454074566500e-06 gnorm 1.010630744432e-12 8 SNES Function norm 1.020088110545e+02 Line search: gnorm after quadratic fit 4.834620808075e+01 Line search: Quadratically determined step, lambda=2.2202847256548619e-01 Line search: gnorm after quadratic fit 4.834620808075e+01 Line search: Quadratically determined step, lambda=2.2202847256548550e-01 Line search: gnorm after quadratic fit 4.834620808075e+01 Line search: Quadratically determined step, lambda=2.2202847256548586e-01 Line search: gnorm after quadratic fit 4.834620808075e+01 Line search: Quadratically determined step, lambda=2.2202847256548650e-01 Line search: gnorm after quadratic fit 4.118773186956e+01 Line search: Quadratically determined step, lambda=1.4595922116342475e-01 Line search: gnorm after quadratic fit 4.118773186956e+01 Line search: Quadratically determined step, lambda=1.4595922116342439e-01 Line search: gnorm after quadratic fit 4.118773186956e+01 Line search: Quadratically determined step, lambda=1.4595922116342333e-01 Line search: gnorm after quadratic fit 4.118773186956e+01 Line search: Quadratically determined step, lambda=1.4595922116342477e-01 Line search: gnorm after quadratic fit 3.602654711105e+01 Line search: Quadratically determined step, lambda=1.2516120471922076e-01 Line search: gnorm after quadratic fit 3.602654711105e+01 Line search: Quadratically determined step, lambda=1.2516120471922076e-01 Line search: gnorm after quadratic fit 3.602654711105e+01 Line search: Quadratically determined step, lambda=1.2516120471922093e-01 Line search: gnorm after quadratic fit 3.602654711105e+01 Line search: Quadratically determined step, lambda=1.2516120471922143e-01 Line search: gnorm after quadratic fit 3.231021309634e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.231021309634e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.231021309634e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.231021309634e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900988169561e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900988169561e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900988169561e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900988169561e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.600037510624e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.600037510624e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.600037510624e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.600037510624e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.319949106997e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.319949106997e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.319949106997e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.319949106997e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.086960153433e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.086960153433e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.086960153433e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.086960153433e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.877780200491e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.877780200491e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.877780200491e+01 Line search: gnorm after quadratic fit 1.877780200491e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.689718550586e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.689718550586e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.689718550586e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.689718550586e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.474145583527e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.474145583527e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.474145583527e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.474145583527e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.317020586350e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.317020586350e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.317020586350e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.317020586350e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107877955050e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107877955050e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107877955050e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107877955050e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.976870270188e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.976870270188e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.976870270188e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.976870270188e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.057717676260e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.057717676261e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.057717676260e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.057717676261e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.229992184597e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.229992184597e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.229992184597e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.229992184597e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.993972344600e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.993972344600e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.993972344600e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.993972344600e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 6.993972344600e+00 gnorm 5.200500460154e+00 Line search: Using full step: fnorm 6.993972344600e+00 gnorm 5.200500460155e+00 Line search: Using full step: fnorm 6.993972344600e+00 gnorm 5.200500460156e+00 Line search: Using full step: fnorm 6.993972344600e+00 gnorm 5.200500460154e+00 Line search: Using full step: fnorm 5.200500460155e+00 gnorm 4.868170351610e-01 Line search: Using full step: fnorm 5.200500460154e+00 gnorm 4.868170351610e-01 Line search: Using full step: fnorm 5.200500460156e+00 gnorm 4.868170351611e-01 Line search: Using full step: fnorm 5.200500460154e+00 gnorm 4.868170351610e-01 Line search: Using full step: fnorm 4.868170351610e-01 gnorm 8.300473416952e-03 Line search: Using full step: fnorm 4.868170351610e-01 gnorm 8.300473417242e-03 Line search: Using full step: fnorm 4.868170351610e-01 gnorm 8.300473417001e-03 Line search: Using full step: fnorm 4.868170351611e-01 gnorm 8.300473417216e-03 Line search: Using full step: fnorm 8.300473417242e-03 gnorm 2.915141326738e-06 Line search: Using full step: fnorm 8.300473416952e-03 gnorm 2.915141337979e-06 Line search: Using full step: fnorm 8.300473417216e-03 gnorm 2.915141336751e-06 Line search: Using full step: fnorm 8.300473417001e-03 gnorm 2.915141336084e-06 Line search: Using full step: fnorm 2.915141326738e-06 gnorm 9.451456610105e-13 Line search: Using full step: fnorm 2.915141337979e-06 gnorm 9.562150532625e-13 Line search: Using full step: fnorm 2.915141336751e-06 gnorm 1.149936274118e-12 Line search: Using full step: fnorm 2.915141336084e-06 gnorm 1.005610647929e-12 9 SNES Function norm 1.020067198030e+02 Line search: gnorm after quadratic fit 4.833437996082e+01 Line search: Quadratically determined step, lambda=2.2201572160626992e-01 Line search: gnorm after quadratic fit 4.833437996082e+01 Line search: Quadratically determined step, lambda=2.2201572160627017e-01 Line search: gnorm after quadratic fit 4.833437996082e+01 Line search: Quadratically determined step, lambda=2.2201572160627131e-01 Line search: gnorm after quadratic fit 4.833437996082e+01 Line search: Quadratically determined step, lambda=2.2201572160626959e-01 Line search: gnorm after quadratic fit 4.117873640134e+01 Line search: Quadratically determined step, lambda=1.4593382049868897e-01 Line search: gnorm after quadratic fit 4.117873640134e+01 Line search: Quadratically determined step, lambda=1.4593382049868900e-01 Line search: gnorm after quadratic fit 4.117873640134e+01 Line search: Quadratically determined step, lambda=1.4593382049868855e-01 Line search: gnorm after quadratic fit 4.117873640134e+01 Line search: Quadratically determined step, lambda=1.4593382049868839e-01 Line search: gnorm after quadratic fit 3.602109608542e+01 Line search: Quadratically determined step, lambda=1.2510571913079091e-01 Line search: gnorm after quadratic fit 3.602109608542e+01 Line search: Quadratically determined step, lambda=1.2510571913078886e-01 Line search: gnorm after quadratic fit 3.602109608542e+01 Line search: Quadratically determined step, lambda=1.2510571913078999e-01 Line search: gnorm after quadratic fit 3.602109608542e+01 Line search: Quadratically determined step, lambda=1.2510571913078999e-01 Line search: gnorm after quadratic fit 3.230510088731e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230510088731e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230510088731e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230510088731e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900501376759e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900501376759e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900501376759e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900501376759e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599536618562e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599536618562e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599536618562e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599536618562e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.319149175882e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.319149175882e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.319149175882e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.319149175882e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.086248145425e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.086248145425e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.086248145425e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.086248145425e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.877134137174e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.877134137174e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.877134137174e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.877134137174e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.689135541955e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.689135541955e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.689135541955e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.689135541955e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473650975678e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473650975678e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473650975678e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473650975678e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316559591149e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316559591149e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316559591149e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316559591149e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107474579395e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107474579395e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107474579395e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107474579395e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.973208486499e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.973208486499e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.973208486499e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.973208486499e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.055139344534e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.055139344534e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.055139344534e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.055139344534e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.227944769087e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.227944769087e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.227944769087e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.227944769087e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995228493045e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995228493045e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995228493045e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995228493045e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 6.995228493045e+00 gnorm 5.247557070544e+00 Line search: Using full step: fnorm 6.995228493045e+00 gnorm 5.247557070545e+00 Line search: Using full step: fnorm 6.995228493045e+00 gnorm 5.247557070546e+00 Line search: Using full step: fnorm 6.995228493045e+00 gnorm 5.247557070549e+00 Line search: Using full step: fnorm 5.247557070544e+00 gnorm 4.939915297322e-01 Line search: Using full step: fnorm 5.247557070545e+00 gnorm 4.939915297324e-01 Line search: Using full step: fnorm 5.247557070546e+00 gnorm 4.939915297324e-01 Line search: Using full step: fnorm 5.247557070549e+00 gnorm 4.939915297329e-01 Line search: Using full step: fnorm 4.939915297322e-01 gnorm 8.547804569160e-03 Line search: Using full step: fnorm 4.939915297324e-01 gnorm 8.547804569122e-03 Line search: Using full step: fnorm 4.939915297324e-01 gnorm 8.547804569148e-03 Line search: Using full step: fnorm 4.939915297329e-01 gnorm 8.547804569149e-03 Line search: Using full step: fnorm 8.547804569160e-03 gnorm 3.092388832677e-06 Line search: Using full step: fnorm 8.547804569122e-03 gnorm 3.092388954504e-06 Line search: Using full step: fnorm 8.547804569148e-03 gnorm 3.092388844518e-06 Line search: Using full step: fnorm 8.547804569149e-03 gnorm 3.092388828405e-06 Line search: Using full step: fnorm 3.092388832677e-06 gnorm 1.116920140958e-12 Line search: Using full step: fnorm 3.092388954504e-06 gnorm 1.158324684515e-12 Line search: Using full step: fnorm 3.092388844518e-06 gnorm 1.154103999264e-12 Line search: Using full step: fnorm 3.092388828405e-06 gnorm 9.964411900313e-13 10 SNES Function norm 1.020060034469e+02 Line search: gnorm after quadratic fit 4.833036038367e+01 Line search: Quadratically determined step, lambda=2.2201165829644065e-01 Line search: gnorm after quadratic fit 4.833036038367e+01 Line search: Quadratically determined step, lambda=2.2201165829644057e-01 Line search: gnorm after quadratic fit 4.833036038367e+01 Line search: Quadratically determined step, lambda=2.2201165829644057e-01 Line search: gnorm after quadratic fit 4.833036038367e+01 Line search: Quadratically determined step, lambda=2.2201165829644057e-01 Line search: gnorm after quadratic fit 4.117567446937e+01 Line search: Quadratically determined step, lambda=1.4592527185938717e-01 Line search: gnorm after quadratic fit 4.117567446937e+01 Line search: Quadratically determined step, lambda=1.4592527185938817e-01 Line search: gnorm after quadratic fit 4.117567446937e+01 Line search: Quadratically determined step, lambda=1.4592527185938814e-01 Line search: gnorm after quadratic fit 4.117567446937e+01 Line search: Quadratically determined step, lambda=1.4592527185938775e-01 Line search: gnorm after quadratic fit 3.601925279133e+01 Line search: Quadratically determined step, lambda=1.2508654181788828e-01 Line search: gnorm after quadratic fit 3.601925279133e+01 Line search: Quadratically determined step, lambda=1.2508654181788725e-01 Line search: gnorm after quadratic fit 3.601925279133e+01 Line search: Quadratically determined step, lambda=1.2508654181788845e-01 Line search: gnorm after quadratic fit 3.601925279133e+01 Line search: Quadratically determined step, lambda=1.2508654181788781e-01 Line search: gnorm after quadratic fit 3.230337073226e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230337073226e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230337073226e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230337073226e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900336416379e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900336416379e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900336416379e+01 Line search: gnorm after quadratic fit 2.900336416379e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599366735613e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599366735613e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599366735613e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599366735613e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318876112868e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318876112868e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318876112868e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318876112868e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.086004827527e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.086004827527e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.086004827527e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.086004827527e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876913307771e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876913307771e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876913307771e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876913307771e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688936239243e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688936239243e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688936239243e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688936239243e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473481863182e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473481863182e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473481863182e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473481863182e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316401939741e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316401939741e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316401939742e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316401939741e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107336388589e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107336388589e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107336388589e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107336388589e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971953947924e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971953947924e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971953947924e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971953947924e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.054256825598e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.054256825598e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.054256825598e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.054256825598e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.227244639061e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.227244639061e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.227244639061e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.227244639061e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995656311421e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995656311421e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995656311421e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995656311421e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 6.995656311421e+00 gnorm 5.263801566550e+00 Line search: Using full step: fnorm 6.995656311421e+00 gnorm 5.263801566549e+00 Line search: Using full step: fnorm 6.995656311421e+00 gnorm 5.263801566550e+00 Line search: Using full step: fnorm 6.995656311421e+00 gnorm 5.263801566549e+00 Line search: Using full step: fnorm 5.263801566550e+00 gnorm 4.964747029726e-01 Line search: Using full step: fnorm 5.263801566549e+00 gnorm 4.964747029723e-01 Line search: Using full step: fnorm 5.263801566550e+00 gnorm 4.964747029727e-01 Line search: Using full step: fnorm 5.263801566549e+00 gnorm 4.964747029724e-01 Line search: Using full step: fnorm 4.964747029726e-01 gnorm 8.634190866152e-03 Line search: Using full step: fnorm 4.964747029723e-01 gnorm 8.634190866021e-03 Line search: Using full step: fnorm 4.964747029727e-01 gnorm 8.634190866163e-03 Line search: Using full step: fnorm 4.964747029724e-01 gnorm 8.634190866125e-03 Line search: Using full step: fnorm 8.634190866021e-03 gnorm 3.155526025939e-06 Line search: Using full step: fnorm 8.634190866152e-03 gnorm 3.155526138874e-06 Line search: Using full step: fnorm 8.634190866163e-03 gnorm 3.155526028600e-06 Line search: Using full step: fnorm 8.634190866125e-03 gnorm 3.155526024493e-06 Line search: Using full step: fnorm 3.155526025939e-06 gnorm 1.254288877198e-12 Line search: Using full step: fnorm 3.155526138874e-06 gnorm 1.106442718554e-12 Line search: Using full step: fnorm 3.155526028600e-06 gnorm 1.467758020724e-12 Line search: Using full step: fnorm 3.155526024493e-06 gnorm 1.051927357847e-12 11 SNES Function norm 1.020057582380e+02 Line search: gnorm after quadratic fit 4.832898823490e+01 Line search: Quadratically determined step, lambda=2.2201030276178771e-01 Line search: gnorm after quadratic fit 4.832898823490e+01 Line search: Quadratically determined step, lambda=2.2201030276178749e-01 Line search: gnorm after quadratic fit 4.832898823490e+01 Line search: Quadratically determined step, lambda=2.2201030276178796e-01 Line search: gnorm after quadratic fit 4.832898823490e+01 Line search: Quadratically determined step, lambda=2.2201030276178751e-01 Line search: gnorm after quadratic fit 4.117462864308e+01 Line search: Quadratically determined step, lambda=1.4592236343740411e-01 Line search: gnorm after quadratic fit 4.117462864308e+01 Line search: Quadratically determined step, lambda=1.4592236343740494e-01 Line search: gnorm after quadratic fit 4.117462864308e+01 Line search: Quadratically determined step, lambda=1.4592236343740536e-01 Line search: gnorm after quadratic fit 4.117462864308e+01 Line search: Quadratically determined step, lambda=1.4592236343740489e-01 Line search: gnorm after quadratic fit 3.601862462218e+01 Line search: Quadratically determined step, lambda=1.2507995766039934e-01 Line search: gnorm after quadratic fit 3.601862462218e+01 Line search: Quadratically determined step, lambda=1.2507995766040070e-01 Line search: gnorm after quadratic fit 3.601862462218e+01 Line search: Quadratically determined step, lambda=1.2507995766039950e-01 Line search: gnorm after quadratic fit 3.601862462218e+01 Line search: Quadratically determined step, lambda=1.2507995766039895e-01 Line search: gnorm after quadratic fit 3.230278095095e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230278095095e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230278095095e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230278095095e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900280159014e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900280159014e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900280159014e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900280159014e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599308782754e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599308782754e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599308782754e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599308782754e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318782757579e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318782757579e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318782757579e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318782757579e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085921611175e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085921611175e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085921611175e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085921611175e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876837777073e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876837777073e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876837777073e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876837777073e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688868068270e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688868068270e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688868068270e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688868068270e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473424015688e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473424015688e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473424015688e+01 Line search: gnorm after quadratic fit 1.473424015688e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316348008907e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316348008907e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316348008907e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316348008907e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107289086725e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107289086725e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107289086725e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107289086725e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971524519400e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971524519400e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971524519400e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971524519400e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053954839583e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053954839583e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053954839583e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053954839583e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.227005143736e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.227005143736e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.227005143736e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.227005143736e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995802481762e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995802481762e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995802481762e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995802481762e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 6.995802481762e+00 gnorm 5.269376882978e+00 Line search: Using full step: fnorm 6.995802481762e+00 gnorm 5.269376882979e+00 Line search: Using full step: fnorm 6.995802481762e+00 gnorm 5.269376882979e+00 Line search: Using full step: fnorm 6.995802481762e+00 gnorm 5.269376882979e+00 Line search: Using full step: fnorm 5.269376882978e+00 gnorm 4.973277167643e-01 Line search: Using full step: fnorm 5.269376882979e+00 gnorm 4.973277167645e-01 Line search: Using full step: fnorm 5.269376882979e+00 gnorm 4.973277167646e-01 Line search: Using full step: fnorm 5.269376882979e+00 gnorm 4.973277167645e-01 Line search: Using full step: fnorm 4.973277167645e-01 gnorm 8.663958689434e-03 Line search: Using full step: fnorm 4.973277167643e-01 gnorm 8.663958689456e-03 Line search: Using full step: fnorm 4.973277167646e-01 gnorm 8.663958689441e-03 Line search: Using full step: fnorm 4.973277167645e-01 gnorm 8.663958689428e-03 Line search: Using full step: fnorm 8.663958689456e-03 gnorm 3.177429826711e-06 Line search: Using full step: fnorm 8.663958689434e-03 gnorm 3.177429938007e-06 Line search: Using full step: fnorm 8.663958689441e-03 gnorm 3.177429938244e-06 Line search: Using full step: fnorm 8.663958689428e-03 gnorm 3.177429826098e-06 Line search: Using full step: fnorm 3.177429826711e-06 gnorm 9.463284211891e-13 Line search: Using full step: fnorm 3.177429938007e-06 gnorm 8.269832139458e-13 Line search: Using full step: fnorm 3.177429938244e-06 gnorm 8.927195024229e-13 Line search: Using full step: fnorm 3.177429826098e-06 gnorm 1.161973653290e-12 12 SNES Function norm 1.020056743241e+02 Line search: gnorm after quadratic fit 4.832851910567e+01 Line search: Quadratically determined step, lambda=2.2200984300356646e-01 Line search: gnorm after quadratic fit 4.832851910567e+01 Line search: Quadratically determined step, lambda=2.2200984300356605e-01 Line search: gnorm after quadratic fit 4.832851910567e+01 Line search: Quadratically determined step, lambda=2.2200984300356666e-01 Line search: gnorm after quadratic fit 4.832851910567e+01 Line search: Quadratically determined step, lambda=2.2200984300356502e-01 Line search: gnorm after quadratic fit 4.117427101286e+01 Line search: Quadratically determined step, lambda=1.4592137021444229e-01 Line search: gnorm after quadratic fit 4.117427101286e+01 Line search: Quadratically determined step, lambda=1.4592137021444096e-01 Line search: gnorm after quadratic fit 4.117427101286e+01 Line search: Quadratically determined step, lambda=1.4592137021444224e-01 Line search: gnorm after quadratic fit 4.117427101286e+01 Line search: Quadratically determined step, lambda=1.4592137021444271e-01 Line search: gnorm after quadratic fit 3.601840998022e+01 Line search: Quadratically determined step, lambda=1.2507770216668498e-01 Line search: gnorm after quadratic fit 3.601840998022e+01 Line search: Quadratically determined step, lambda=1.2507770216668360e-01 Line search: gnorm after quadratic fit 3.601840998022e+01 Line search: Quadratically determined step, lambda=1.2507770216668432e-01 Line search: gnorm after quadratic fit 3.601840998022e+01 Line search: Quadratically determined step, lambda=1.2507770216668518e-01 Line search: gnorm after quadratic fit 3.230257940617e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230257940617e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230257940617e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230257940617e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900260931349e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900260931349e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900260931349e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900260931349e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599288973654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599288973654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599288973654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599288973654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318750823590e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318750823590e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318750823590e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318750823590e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085893141889e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085893141889e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085893141889e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085893141889e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876811936486e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876811936486e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876811936486e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876811936486e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688844745230e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688844745230e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688844745230e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688844745230e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473404224243e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473404224243e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473404224243e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473404224243e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316329557028e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316329557028e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316329557028e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316329557028e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107272899579e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107272899579e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107272899579e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107272899579e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971377563985e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971377563985e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971377563985e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971377563985e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053851508344e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053851508344e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053851508344e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053851508344e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226923204477e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226923204477e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226923204477e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226923204477e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995852472138e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995852472138e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995852472138e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995852472138e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 6.995852472138e+00 gnorm 5.271286574972e+00 Line search: Using full step: fnorm 6.995852472138e+00 gnorm 5.271286574971e+00 Line search: Using full step: fnorm 6.995852472138e+00 gnorm 5.271286574971e+00 Line search: Using full step: fnorm 6.995852472138e+00 gnorm 5.271286574971e+00 Line search: Using full step: fnorm 5.271286574971e+00 gnorm 4.976199850840e-01 Line search: Using full step: fnorm 5.271286574972e+00 gnorm 4.976199850844e-01 Line search: Using full step: fnorm 5.271286574971e+00 gnorm 4.976199850841e-01 Line search: Using full step: fnorm 5.271286574971e+00 gnorm 4.976199850840e-01 Line search: Using full step: fnorm 4.976199850840e-01 gnorm 8.674168929552e-03 Line search: Using full step: fnorm 4.976199850844e-01 gnorm 8.674168929736e-03 Line search: Using full step: fnorm 4.976199850841e-01 gnorm 8.674168929760e-03 Line search: Using full step: fnorm 4.976199850840e-01 gnorm 8.674168929544e-03 Line search: Using full step: fnorm 8.674168929552e-03 gnorm 3.184960277854e-06 Line search: Using full step: fnorm 8.674168929736e-03 gnorm 3.184960046439e-06 Line search: Using full step: fnorm 8.674168929760e-03 gnorm 3.184960296072e-06 Line search: Using full step: fnorm 8.674168929544e-03 gnorm 3.184960269391e-06 Line search: Using full step: fnorm 3.184960277854e-06 gnorm 1.107419101754e-12 Line search: Using full step: fnorm 3.184960046439e-06 gnorm 1.119351343697e-12 Line search: Using full step: fnorm 3.184960296072e-06 gnorm 1.213284109156e-12 Line search: Using full step: fnorm 3.184960269391e-06 gnorm 1.024080237536e-12 13 SNES Function norm 1.020056456101e+02 Line search: gnorm after quadratic fit 4.832835862848e+01 Line search: Quadratically determined step, lambda=2.2200968616397104e-01 Line search: gnorm after quadratic fit 4.832835862848e+01 Line search: Quadratically determined step, lambda=2.2200968616397129e-01 Line search: gnorm after quadratic fit 4.832835862848e+01 Line search: Quadratically determined step, lambda=2.2200968616397057e-01 Line search: gnorm after quadratic fit 4.832835862848e+01 Line search: Quadratically determined step, lambda=2.2200968616397151e-01 Line search: gnorm after quadratic fit 4.117414866861e+01 Line search: Quadratically determined step, lambda=1.4592103059263803e-01 Line search: gnorm after quadratic fit 4.117414866861e+01 Line search: Quadratically determined step, lambda=1.4592103059263892e-01 Line search: gnorm after quadratic fit 4.117414866861e+01 Line search: Quadratically determined step, lambda=1.4592103059263775e-01 Line search: gnorm after quadratic fit 4.117414866861e+01 Line search: Quadratically determined step, lambda=1.4592103059263878e-01 Line search: gnorm after quadratic fit 3.601833657131e+01 Line search: Quadratically determined step, lambda=1.2507693010313473e-01 Line search: gnorm after quadratic fit 3.601833657131e+01 Line search: Quadratically determined step, lambda=1.2507693010313392e-01 Line search: gnorm after quadratic fit 3.601833657131e+01 Line search: Quadratically determined step, lambda=1.2507693010313473e-01 Line search: gnorm after quadratic fit 3.601833657131e+01 Line search: Quadratically determined step, lambda=1.2507693010313536e-01 Line search: gnorm after quadratic fit 3.230251047427e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230251047427e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230251047427e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230251047427e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900254354800e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900254354800e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900254354800e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900254354800e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599282198005e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599282198005e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599282198005e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599282198005e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318739897868e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318739897868e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318739897868e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318739897868e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085883401152e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085883401152e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085883401152e+01 Line search: gnorm after quadratic fit 2.085883401152e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876803095079e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876803095079e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876803095079e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876803095079e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688836765163e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688836765163e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688836765163e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688836765163e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473397452487e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473397452487e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473397452487e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473397452487e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316323243560e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316323243560e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316323243560e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316323243560e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107267360623e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107267360623e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107267360623e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107267360623e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971327278322e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971327278322e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971327278322e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971327278322e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053816151534e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053816151534e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053816151534e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053816151534e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226895168456e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226895168456e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226895168456e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226895168456e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995869574435e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995869574435e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995869574435e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995869574436e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 6.995869574435e+00 gnorm 5.271940245448e+00 Line search: Using full step: fnorm 6.995869574435e+00 gnorm 5.271940245447e+00 Line search: Using full step: fnorm 6.995869574435e+00 gnorm 5.271940245447e+00 Line search: Using full step: fnorm 6.995869574436e+00 gnorm 5.271940245447e+00 Line search: Using full step: fnorm 5.271940245448e+00 gnorm 4.977200363068e-01 Line search: Using full step: fnorm 5.271940245447e+00 gnorm 4.977200363068e-01 Line search: Using full step: fnorm 5.271940245447e+00 gnorm 4.977200363069e-01 Line search: Using full step: fnorm 5.271940245447e+00 gnorm 4.977200363069e-01 Line search: Using full step: fnorm 4.977200363068e-01 gnorm 8.677665442639e-03 Line search: Using full step: fnorm 4.977200363068e-01 gnorm 8.677665442525e-03 Line search: Using full step: fnorm 4.977200363069e-01 gnorm 8.677665442755e-03 Line search: Using full step: fnorm 4.977200363069e-01 gnorm 8.677665442509e-03 Line search: Using full step: fnorm 8.677665442639e-03 gnorm 3.187541023916e-06 Line search: Using full step: fnorm 8.677665442525e-03 gnorm 3.187540901092e-06 Line search: Using full step: fnorm 8.677665442755e-03 gnorm 3.187541253705e-06 Line search: Using full step: fnorm 8.677665442509e-03 gnorm 3.187541022068e-06 Line search: Using full step: fnorm 3.187541023916e-06 gnorm 1.265117838248e-12 Line search: Using full step: fnorm 3.187540901092e-06 gnorm 1.314734637256e-12 Line search: Using full step: fnorm 3.187541253705e-06 gnorm 1.028877982204e-12 Line search: Using full step: fnorm 3.187541022068e-06 gnorm 1.234551948975e-12 14 SNES Function norm 1.020056357849e+02 Line search: gnorm after quadratic fit 4.832830372338e+01 Line search: Quadratically determined step, lambda=2.2200963255400011e-01 Line search: gnorm after quadratic fit 4.832830372338e+01 Line search: Quadratically determined step, lambda=2.2200963255400094e-01 Line search: gnorm after quadratic fit 4.832830372338e+01 Line search: Quadratically determined step, lambda=2.2200963255400044e-01 Line search: gnorm after quadratic fit 4.832830372338e+01 Line search: Quadratically determined step, lambda=2.2200963255400161e-01 Line search: gnorm after quadratic fit 4.117410680924e+01 Line search: Quadratically determined step, lambda=1.4592091441138491e-01 Line search: gnorm after quadratic fit 4.117410680924e+01 Line search: Quadratically determined step, lambda=1.4592091441138377e-01 Line search: gnorm after quadratic fit 4.117410680924e+01 Line search: Quadratically determined step, lambda=1.4592091441138466e-01 Line search: gnorm after quadratic fit 4.117410680924e+01 Line search: Quadratically determined step, lambda=1.4592091441138474e-01 Line search: gnorm after quadratic fit 3.601831145716e+01 Line search: Quadratically determined step, lambda=1.2507666589156130e-01 Line search: gnorm after quadratic fit 3.601831145716e+01 Line search: Quadratically determined step, lambda=1.2507666589156277e-01 Line search: gnorm after quadratic fit 3.601831145716e+01 Line search: Quadratically determined step, lambda=1.2507666589156224e-01 Line search: gnorm after quadratic fit 3.601831145716e+01 Line search: Quadratically determined step, lambda=1.2507666589156205e-01 Line search: gnorm after quadratic fit 3.230248689149e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230248689149e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230248689149e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230248689149e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900252104810e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900252104810e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900252104810e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900252104810e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599279879872e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599279879872e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599279879872e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599279879872e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318736159556e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318736159556e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318736159556e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318736159556e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085880068242e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085880068242e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085880068242e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085880068242e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876800069877e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876800069877e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876800069877e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876800069877e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688834034675e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688834034675e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473395135434e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688834034675e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688834034675e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473395135434e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473395135434e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473395135434e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316321083311e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316321083311e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107265465340e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316321083311e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316321083311e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107265465340e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971310071896e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971310071896e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053804053530e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107265465340e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107265465340e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226885575529e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053804053530e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971310071896e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971310071896e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995875425974e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226885575529e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 6.995875425974e+00 gnorm 5.272163938385e+00 Line search: gnorm after quadratic fit 9.053804053530e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053804053530e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995875425974e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 5.272163938385e+00 gnorm 4.977542761082e-01 Line search: Using full step: fnorm 6.995875425974e+00 gnorm 5.272163938384e+00 Line search: Using full step: fnorm 4.977542761082e-01 gnorm 8.678862178167e-03 Line search: gnorm after quadratic fit 8.226885575529e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226885575529e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 5.272163938384e+00 gnorm 4.977542761081e-01 Line search: Using full step: fnorm 8.678862178167e-03 gnorm 3.188424713670e-06 Line search: gnorm after quadratic fit 6.995875425974e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995875425974e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 3.188424713670e-06 gnorm 1.118744170541e-12 Line search: Using full step: fnorm 4.977542761081e-01 gnorm 8.678862178151e-03 Line search: Using full step: fnorm 6.995875425974e+00 gnorm 5.272163938385e+00 Line search: Using full step: fnorm 6.995875425974e+00 gnorm 5.272163938383e+00 Line search: Using full step: fnorm 8.678862178151e-03 gnorm 3.188424679462e-06 Line search: Using full step: fnorm 5.272163938385e+00 gnorm 4.977542761082e-01 Line search: Using full step: fnorm 5.272163938383e+00 gnorm 4.977542761080e-01 Line search: Using full step: fnorm 3.188424679462e-06 gnorm 1.286038589946e-12 Line search: Using full step: fnorm 4.977542761082e-01 gnorm 8.678862178115e-03 Line search: Using full step: fnorm 4.977542761080e-01 gnorm 8.678862178113e-03 Line search: Using full step: fnorm 8.678862178115e-03 gnorm 3.188424717345e-06 Line search: Using full step: fnorm 8.678862178113e-03 gnorm 3.188424680425e-06 Line search: Using full step: fnorm 3.188424717345e-06 gnorm 1.055049607597e-12 Line search: Using full step: fnorm 3.188424680425e-06 gnorm 1.150237197570e-12 15 SNES Function norm 1.020056324231e+02 Line search: gnorm after quadratic fit 4.832828493718e+01 Line search: Quadratically determined step, lambda=2.2200961421685994e-01 Line search: gnorm after quadratic fit 4.832828493718e+01 Line search: Quadratically determined step, lambda=2.2200961421685822e-01 Line search: gnorm after quadratic fit 4.832828493718e+01 Line search: Quadratically determined step, lambda=2.2200961421685750e-01 Line search: gnorm after quadratic fit 4.832828493718e+01 Line search: Quadratically determined step, lambda=2.2200961421685864e-01 Line search: gnorm after quadratic fit 4.117409248662e+01 Line search: Quadratically determined step, lambda=1.4592087466092307e-01 Line search: gnorm after quadratic fit 4.117409248662e+01 Line search: Quadratically determined step, lambda=1.4592087466092382e-01 Line search: gnorm after quadratic fit 4.117409248662e+01 Line search: Quadratically determined step, lambda=1.4592087466092435e-01 Line search: gnorm after quadratic fit 4.117409248662e+01 Line search: Quadratically determined step, lambda=1.4592087466092346e-01 Line search: gnorm after quadratic fit 3.601830286436e+01 Line search: Quadratically determined step, lambda=1.2507657548246989e-01 Line search: gnorm after quadratic fit 3.601830286436e+01 Line search: Quadratically determined step, lambda=1.2507657548246978e-01 Line search: gnorm after quadratic fit 3.601830286436e+01 Line search: Quadratically determined step, lambda=1.2507657548247064e-01 Line search: gnorm after quadratic fit 3.601830286436e+01 Line search: Quadratically determined step, lambda=1.2507657548247050e-01 Line search: gnorm after quadratic fit 3.230247882262e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247882262e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247882262e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247882262e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900251334969e+01 Line search: gnorm after quadratic fit 2.900251334969e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900251334969e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900251334969e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599279086713e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599279086713e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599279086713e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599279086713e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734880438e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734880438e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734880438e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878927833e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878927833e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734880438e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878927833e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876799034754e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876799034754e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876799034754e+01 Line search: gnorm after quadratic fit 2.085878927833e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688833100393e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688833100393e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688833100393e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876799034754e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473394342615e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473394342615e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473394342615e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688833100393e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316320344144e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316320344144e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316320344144e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473394342615e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264816830e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264816830e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264816830e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316320344144e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971304184367e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971304184367e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971304184367e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264816830e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053799913971e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053799913971e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053799913971e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971304184367e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226882293144e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226882293144e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226882293144e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053799913971e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995877428148e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995877428148e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995877428148e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 6.995877428148e+00 gnorm 5.272240482302e+00 Line search: gnorm after quadratic fit 8.226882293144e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 6.995877428148e+00 gnorm 5.272240482303e+00 Line search: Using full step: fnorm 6.995877428148e+00 gnorm 5.272240482302e+00 Line search: Using full step: fnorm 5.272240482302e+00 gnorm 4.977659925280e-01 Line search: Using full step: fnorm 5.272240482303e+00 gnorm 4.977659925286e-01 Line search: gnorm after quadratic fit 6.995877428148e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 5.272240482302e+00 gnorm 4.977659925281e-01 Line search: Using full step: fnorm 4.977659925280e-01 gnorm 8.679271703488e-03 Line search: Using full step: fnorm 4.977659925286e-01 gnorm 8.679271703389e-03 Line search: Using full step: fnorm 6.995877428148e+00 gnorm 5.272240482303e+00 Line search: Using full step: fnorm 4.977659925281e-01 gnorm 8.679271703503e-03 Line search: Using full step: fnorm 8.679271703488e-03 gnorm 3.188726788899e-06 Line search: Using full step: fnorm 8.679271703389e-03 gnorm 3.188727014031e-06 Line search: Using full step: fnorm 5.272240482303e+00 gnorm 4.977659925286e-01 Line search: Using full step: fnorm 8.679271703503e-03 gnorm 3.188726786965e-06 Line search: Using full step: fnorm 3.188726788899e-06 gnorm 9.994290072819e-13 Line search: Using full step: fnorm 3.188727014031e-06 gnorm 9.282300833904e-13 Line search: Using full step: fnorm 4.977659925286e-01 gnorm 8.679271703386e-03 Line search: Using full step: fnorm 3.188726786965e-06 gnorm 9.658559512996e-13 Line search: Using full step: fnorm 8.679271703386e-03 gnorm 3.188726990242e-06 Line search: Using full step: fnorm 3.188726990242e-06 gnorm 1.075961851620e-12 16 SNES Function norm 1.020056312727e+02 Line search: gnorm after quadratic fit 4.832827850920e+01 Line search: Quadratically determined step, lambda=2.2200960794322513e-01 Line search: gnorm after quadratic fit 4.832827850920e+01 Line search: Quadratically determined step, lambda=2.2200960794322586e-01 Line search: gnorm after quadratic fit 4.832827850920e+01 Line search: Quadratically determined step, lambda=2.2200960794322505e-01 Line search: gnorm after quadratic fit 4.832827850920e+01 Line search: Quadratically determined step, lambda=2.2200960794322458e-01 Line search: gnorm after quadratic fit 4.117408758591e+01 Line search: Quadratically determined step, lambda=1.4592086105992663e-01 Line search: gnorm after quadratic fit 4.117408758591e+01 Line search: Quadratically determined step, lambda=1.4592086105992710e-01 Line search: gnorm after quadratic fit 4.117408758591e+01 Line search: Quadratically determined step, lambda=1.4592086105992705e-01 Line search: gnorm after quadratic fit 4.117408758591e+01 Line search: Quadratically determined step, lambda=1.4592086105992755e-01 Line search: gnorm after quadratic fit 3.601829992423e+01 Line search: Quadratically determined step, lambda=1.2507654454682243e-01 Line search: gnorm after quadratic fit 3.601829992423e+01 Line search: Quadratically determined step, lambda=1.2507654454682246e-01 Line search: gnorm after quadratic fit 3.601829992423e+01 Line search: Quadratically determined step, lambda=1.2507654454682252e-01 Line search: gnorm after quadratic fit 3.601829992423e+01 Line search: Quadratically determined step, lambda=1.2507654454682321e-01 Line search: gnorm after quadratic fit 3.230247606175e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247606175e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247606175e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247606175e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900251071558e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900251071558e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900251071558e+01 Line search: gnorm after quadratic fit 2.900251071558e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278815322e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278815322e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278815322e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278815322e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734442765e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734442765e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734442765e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734442765e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878537621e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878537621e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878537621e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878537621e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798680568e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798680568e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798680568e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798680568e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832780711e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832780711e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832780711e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832780711e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473394071337e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473394071337e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473394071337e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473394071337e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316320091224e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316320091224e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316320091224e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316320091224e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264594930e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264594930e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264594930e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264594930e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971302169836e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971302169836e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971302169836e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971302169836e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053798497544e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053798497544e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053798497544e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053798497544e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226881170017e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226881170017e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226881170017e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226881170016e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878113225e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878113225e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878113225e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878113225e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 6.995878113225e+00 gnorm 5.272266673611e+00 Line search: Using full step: fnorm 6.995878113225e+00 gnorm 5.272266673612e+00 Line search: Using full step: fnorm 6.995878113225e+00 gnorm 5.272266673611e+00 Line search: Using full step: fnorm 6.995878113225e+00 gnorm 5.272266673611e+00 Line search: Using full step: fnorm 5.272266673611e+00 gnorm 4.977700015949e-01 Line search: Using full step: fnorm 5.272266673612e+00 gnorm 4.977700015949e-01 Line search: Using full step: fnorm 5.272266673611e+00 gnorm 4.977700015948e-01 Line search: Using full step: fnorm 5.272266673611e+00 gnorm 4.977700015948e-01 Line search: Using full step: fnorm 4.977700015949e-01 gnorm 8.679411834380e-03 Line search: Using full step: fnorm 4.977700015949e-01 gnorm 8.679411834510e-03 Line search: Using full step: fnorm 4.977700015948e-01 gnorm 8.679411834384e-03 Line search: Using full step: fnorm 4.977700015948e-01 gnorm 8.679411834490e-03 Line search: Using full step: fnorm 8.679411834380e-03 gnorm 3.188830404071e-06 Line search: Using full step: fnorm 8.679411834510e-03 gnorm 3.188830549655e-06 Line search: Using full step: fnorm 8.679411834384e-03 gnorm 3.188830402970e-06 Line search: Using full step: fnorm 8.679411834490e-03 gnorm 3.188830297368e-06 Line search: Using full step: fnorm 3.188830404071e-06 gnorm 1.113218384642e-12 Line search: Using full step: fnorm 3.188830549655e-06 gnorm 1.193135894174e-12 Line search: Using full step: fnorm 3.188830402970e-06 gnorm 1.381740312219e-12 Line search: Using full step: fnorm 3.188830297368e-06 gnorm 9.630731360221e-13 17 SNES Function norm 1.020056308791e+02 Line search: gnorm after quadratic fit 4.832827630976e+01 Line search: Quadratically determined step, lambda=2.2200960579667506e-01 Line search: gnorm after quadratic fit 4.832827630976e+01 Line search: Quadratically determined step, lambda=2.2200960579667400e-01 Line search: gnorm after quadratic fit 4.832827630976e+01 Line search: Quadratically determined step, lambda=2.2200960579667534e-01 Line search: gnorm after quadratic fit 4.832827630976e+01 Line search: Quadratically determined step, lambda=2.2200960579667428e-01 Line search: gnorm after quadratic fit 4.117408590905e+01 Line search: Quadratically determined step, lambda=1.4592085640613406e-01 Line search: gnorm after quadratic fit 4.117408590905e+01 Line search: Quadratically determined step, lambda=1.4592085640613436e-01 Line search: gnorm after quadratic fit 4.117408590905e+01 Line search: Quadratically determined step, lambda=1.4592085640613436e-01 Line search: gnorm after quadratic fit 4.117408590905e+01 Line search: Quadratically determined step, lambda=1.4592085640613439e-01 Line search: gnorm after quadratic fit 3.601829891822e+01 Line search: Quadratically determined step, lambda=1.2507653396155582e-01 Line search: gnorm after quadratic fit 3.601829891822e+01 Line search: Quadratically determined step, lambda=1.2507653396155582e-01 Line search: gnorm after quadratic fit 3.601829891822e+01 Line search: Quadratically determined step, lambda=1.2507653396155602e-01 Line search: gnorm after quadratic fit 3.601829891822e+01 Line search: Quadratically determined step, lambda=1.2507653396155555e-01 Line search: gnorm after quadratic fit 3.230247511708e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247511708e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247511708e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247511708e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250981427e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250981427e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250981427e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250981427e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278722461e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278722461e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278722461e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278722461e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734293008e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734293008e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734293008e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734293008e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878404103e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878404103e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878404103e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878404103e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798559376e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798559376e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798559376e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798559376e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832671326e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832671326e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832671326e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832671326e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393978515e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393978515e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393978515e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393978515e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316320004683e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316320004683e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316320004683e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316320004683e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264519003e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264519003e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264519003e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264519003e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301480526e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301480526e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301480526e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301480526e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053798012887e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053798012887e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053798012887e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053798012887e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880785718e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880785718e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880785718e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880785718e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878347636e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878347636e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878347636e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878347636e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 6.995878347636e+00 gnorm 5.272275635500e+00 Line search: Using full step: fnorm 6.995878347636e+00 gnorm 5.272275635500e+00 Line search: Using full step: fnorm 6.995878347636e+00 gnorm 5.272275635499e+00 Line search: Using full step: fnorm 6.995878347636e+00 gnorm 5.272275635500e+00 Line search: Using full step: fnorm 5.272275635500e+00 gnorm 4.977713733802e-01 Line search: Using full step: fnorm 5.272275635500e+00 gnorm 4.977713733805e-01 Line search: Using full step: fnorm 5.272275635499e+00 gnorm 4.977713733800e-01 Line search: Using full step: fnorm 5.272275635500e+00 gnorm 4.977713733800e-01 Line search: Using full step: fnorm 4.977713733802e-01 gnorm 8.679459783722e-03 Line search: Using full step: fnorm 4.977713733805e-01 gnorm 8.679459783607e-03 Line search: Using full step: fnorm 4.977713733800e-01 gnorm 8.679459783589e-03 Line search: Using full step: fnorm 4.977713733800e-01 gnorm 8.679459783569e-03 Line search: Using full step: fnorm 8.679459783607e-03 gnorm 3.188866091111e-06 Line search: Using full step: fnorm 8.679459783722e-03 gnorm 3.188865975930e-06 Line search: Using full step: fnorm 8.679459783589e-03 gnorm 3.188865985170e-06 Line search: Using full step: fnorm 8.679459783569e-03 gnorm 3.188865956015e-06 Line search: Using full step: fnorm 3.188865975930e-06 gnorm 1.110726041543e-12 Line search: Using full step: fnorm 3.188866091111e-06 gnorm 1.093395320892e-12 Line search: Using full step: fnorm 3.188865985170e-06 gnorm 1.070900135705e-12 Line search: Using full step: fnorm 3.188865956015e-06 gnorm 1.182828525900e-12 18 SNES Function norm 1.020056307444e+02 Line search: gnorm after quadratic fit 4.832827555718e+01 Line search: Quadratically determined step, lambda=2.2200960506220271e-01 Line search: gnorm after quadratic fit 4.832827555718e+01 Line search: Quadratically determined step, lambda=2.2200960506220294e-01 Line search: gnorm after quadratic fit 4.832827555718e+01 Line search: Quadratically determined step, lambda=2.2200960506220352e-01 Line search: gnorm after quadratic fit 4.832827555718e+01 Line search: Quadratically determined step, lambda=2.2200960506220277e-01 Line search: gnorm after quadratic fit 4.117408533528e+01 Line search: Quadratically determined step, lambda=1.4592085481375763e-01 Line search: gnorm after quadratic fit 4.117408533528e+01 Line search: Quadratically determined step, lambda=1.4592085481375783e-01 Line search: gnorm after quadratic fit 4.117408533528e+01 Line search: Quadratically determined step, lambda=1.4592085481375824e-01 Line search: gnorm after quadratic fit 4.117408533528e+01 Line search: Quadratically determined step, lambda=1.4592085481375727e-01 Line search: gnorm after quadratic fit 3.601829857399e+01 Line search: Quadratically determined step, lambda=1.2507653033960256e-01 Line search: gnorm after quadratic fit 3.601829857399e+01 Line search: Quadratically determined step, lambda=1.2507653033960139e-01 Line search: gnorm after quadratic fit 3.601829857399e+01 Line search: Quadratically determined step, lambda=1.2507653033960253e-01 Line search: gnorm after quadratic fit 3.601829857399e+01 Line search: Quadratically determined step, lambda=1.2507653033960356e-01 Line search: gnorm after quadratic fit 3.230247479384e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247479384e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247479384e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247479384e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250950587e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250950587e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250950587e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250950587e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278690687e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278690687e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278690687e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278690687e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734241765e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734241765e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734241765e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734241765e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878358418e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878358418e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878358418e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878358418e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798517908e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798517908e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798517908e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798517908e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832633898e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832633898e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832633898e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832633898e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393946754e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393946754e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393946754e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393946754e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319975071e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319975071e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319975071e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319975071e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264493023e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264493023e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264493023e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264493023e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301244666e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301244666e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301244666e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301244666e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797847052e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797847052e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797847052e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797847052e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880654223e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880654223e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880654223e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878427845e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880654223e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878427845e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 6.995878427845e+00 gnorm 5.272278701982e+00 Line search: gnorm after quadratic fit 6.995878427845e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 6.995878427845e+00 gnorm 5.272278701982e+00 Line search: gnorm after quadratic fit 6.995878427844e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 5.272278701982e+00 gnorm 4.977718427631e-01 Line search: Using full step: fnorm 6.995878427845e+00 gnorm 5.272278701982e+00 Line search: Using full step: fnorm 5.272278701982e+00 gnorm 4.977718427633e-01 Line search: Using full step: fnorm 6.995878427844e+00 gnorm 5.272278701982e+00 Line search: Using full step: fnorm 4.977718427631e-01 gnorm 8.679476190618e-03 Line search: Using full step: fnorm 5.272278701982e+00 gnorm 4.977718427631e-01 Line search: Using full step: fnorm 4.977718427633e-01 gnorm 8.679476190755e-03 Line search: Using full step: fnorm 5.272278701982e+00 gnorm 4.977718427631e-01 Line search: Using full step: fnorm 8.679476190618e-03 gnorm 3.188877939196e-06 Line search: Using full step: fnorm 4.977718427631e-01 gnorm 8.679476190507e-03 Line search: Using full step: fnorm 8.679476190755e-03 gnorm 3.188877936875e-06 Line search: Using full step: fnorm 3.188877939196e-06 gnorm 1.097409539338e-12 Line search: Using full step: fnorm 4.977718427631e-01 gnorm 8.679476190475e-03 Line search: Using full step: fnorm 8.679476190507e-03 gnorm 3.188877951083e-06 Line search: Using full step: fnorm 3.188877936875e-06 gnorm 1.054800126589e-12 Line search: Using full step: fnorm 3.188877951083e-06 gnorm 1.158829754197e-12 Line search: Using full step: fnorm 8.679476190475e-03 gnorm 3.188877938713e-06 Line search: Using full step: fnorm 3.188877938713e-06 gnorm 1.227184346060e-12 19 SNES Function norm 1.020056306983e+02 Line search: gnorm after quadratic fit 4.832827529967e+01 Line search: Quadratically determined step, lambda=2.2200960481088952e-01 Line search: gnorm after quadratic fit 4.832827529967e+01 Line search: Quadratically determined step, lambda=2.2200960481088952e-01 Line search: gnorm after quadratic fit 4.832827529967e+01 Line search: Quadratically determined step, lambda=2.2200960481089002e-01 Line search: gnorm after quadratic fit 4.832827529967e+01 Line search: Quadratically determined step, lambda=2.2200960481089002e-01 Line search: gnorm after quadratic fit 4.117408513895e+01 Line search: Quadratically determined step, lambda=1.4592085426889648e-01 Line search: gnorm after quadratic fit 4.117408513895e+01 Line search: Quadratically determined step, lambda=1.4592085426889642e-01 Line search: gnorm after quadratic fit 4.117408513895e+01 Line search: Quadratically determined step, lambda=1.4592085426889678e-01 Line search: gnorm after quadratic fit 3.601829845621e+01 Line search: Quadratically determined step, lambda=1.2507652910028233e-01 Line search: gnorm after quadratic fit 3.601829845621e+01 Line search: Quadratically determined step, lambda=1.2507652910028280e-01 Line search: gnorm after quadratic fit 4.117408513895e+01 Line search: Quadratically determined step, lambda=1.4592085426889739e-01 Line search: gnorm after quadratic fit 3.230247468324e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247468324e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.601829845621e+01 Line search: Quadratically determined step, lambda=1.2507652910028194e-01 Line search: gnorm after quadratic fit 3.601829845621e+01 Line search: Quadratically determined step, lambda=1.2507652910028258e-01 Line search: gnorm after quadratic fit 2.900250940035e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250940035e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278679815e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278679815e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247468324e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247468324e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734224232e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734224232e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250940035e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250940035e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878342785e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878342785e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278679815e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278679815e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798503719e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798503719e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734224232e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734224232e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832621091e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832621091e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878342785e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393935886e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878342785e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393935886e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798503719e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319964939e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319964939e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832621091e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798503719e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264484133e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264484133e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393935886e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832621091e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301163962e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301163962e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319964939e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393935886e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797790309e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797790309e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264484133e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319964939e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880609229e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880609229e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301163962e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264484133e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878455289e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797790309e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878455289e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 6.995878455289e+00 gnorm 5.272279751237e+00 Line search: Using full step: fnorm 6.995878455289e+00 gnorm 5.272279751237e+00 Line search: gnorm after quadratic fit 9.971301163962e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880609229e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 5.272279751237e+00 gnorm 4.977720033716e-01 Line search: Using full step: fnorm 5.272279751237e+00 gnorm 4.977720033718e-01 Line search: gnorm after quadratic fit 6.995878455289e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797790309e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 4.977720033716e-01 gnorm 8.679481804351e-03 Line search: Using full step: fnorm 4.977720033718e-01 gnorm 8.679481804479e-03 Line search: Using full step: fnorm 6.995878455289e+00 gnorm 5.272279751237e+00 Line search: Using full step: fnorm 8.679481804351e-03 gnorm 3.188882021255e-06 Line search: gnorm after quadratic fit 8.226880609229e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 8.679481804479e-03 gnorm 3.188882019351e-06 Line search: Using full step: fnorm 5.272279751237e+00 gnorm 4.977720033715e-01 Line search: Using full step: fnorm 3.188882021255e-06 gnorm 9.749912104957e-13 Line search: Using full step: fnorm 3.188882019351e-06 gnorm 1.118893472013e-12 Line search: Using full step: fnorm 4.977720033715e-01 gnorm 8.679481804239e-03 Line search: gnorm after quadratic fit 6.995878455289e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 8.679481804239e-03 gnorm 3.188882021445e-06 Line search: Using full step: fnorm 6.995878455289e+00 gnorm 5.272279751237e+00 Line search: Using full step: fnorm 3.188882021445e-06 gnorm 1.177327797150e-12 Line search: Using full step: fnorm 5.272279751237e+00 gnorm 4.977720033718e-01 Line search: Using full step: fnorm 4.977720033718e-01 gnorm 8.679481804440e-03 Line search: Using full step: fnorm 8.679481804440e-03 gnorm 3.188882027659e-06 Line search: Using full step: fnorm 3.188882027659e-06 gnorm 1.118986643133e-12 20 SNES Function norm 1.020056306826e+02 Line search: gnorm after quadratic fit 4.832827521156e+01 Line search: Quadratically determined step, lambda=2.2200960472489900e-01 Line search: gnorm after quadratic fit 4.832827521156e+01 Line search: Quadratically determined step, lambda=2.2200960472489945e-01 Line search: gnorm after quadratic fit 4.832827521156e+01 Line search: Quadratically determined step, lambda=2.2200960472489942e-01 Line search: gnorm after quadratic fit 4.832827521156e+01 Line search: Quadratically determined step, lambda=2.2200960472489872e-01 Line search: gnorm after quadratic fit 4.117408507178e+01 Line search: Quadratically determined step, lambda=1.4592085408246327e-01 Line search: gnorm after quadratic fit 4.117408507178e+01 Line search: Quadratically determined step, lambda=1.4592085408246272e-01 Line search: gnorm after quadratic fit 4.117408507178e+01 Line search: Quadratically determined step, lambda=1.4592085408246239e-01 Line search: gnorm after quadratic fit 4.117408507178e+01 Line search: Quadratically determined step, lambda=1.4592085408246341e-01 Line search: gnorm after quadratic fit 3.601829841591e+01 Line search: Quadratically determined step, lambda=1.2507652867622648e-01 Line search: gnorm after quadratic fit 3.601829841591e+01 Line search: Quadratically determined step, lambda=1.2507652867622626e-01 Line search: gnorm after quadratic fit 3.601829841591e+01 Line search: Quadratically determined step, lambda=1.2507652867622723e-01 Line search: gnorm after quadratic fit 3.601829841591e+01 Line search: Quadratically determined step, lambda=1.2507652867622598e-01 Line search: gnorm after quadratic fit 3.230247464539e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247464539e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247464539e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247464539e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250936424e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250936424e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250936424e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250936424e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278676095e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278676095e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278676095e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278676095e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734218232e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734218232e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734218232e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734218232e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878337437e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878337437e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878337437e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878337437e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798498864e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798498864e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798498864e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798498864e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832616709e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832616709e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832616709e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832616709e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393932167e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393932167e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393932167e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393932167e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319961472e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319961472e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319961472e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319961472e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264481092e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264481091e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264481091e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264481092e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301136347e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301136347e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301136347e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301136347e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797770893e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797770893e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797770893e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797770893e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880593834e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880593834e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880593834e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880593834e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878464680e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878464680e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878464680e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878464680e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 6.995878464680e+00 gnorm 5.272280110258e+00 Line search: Using full step: fnorm 6.995878464680e+00 gnorm 5.272280110258e+00 Line search: Using full step: fnorm 6.995878464680e+00 gnorm 5.272280110258e+00 Line search: Using full step: fnorm 6.995878464680e+00 gnorm 5.272280110258e+00 Line search: Using full step: fnorm 5.272280110258e+00 gnorm 4.977720583264e-01 Line search: Using full step: fnorm 5.272280110258e+00 gnorm 4.977720583265e-01 Line search: Using full step: fnorm 5.272280110258e+00 gnorm 4.977720583268e-01 Line search: Using full step: fnorm 5.272280110258e+00 gnorm 4.977720583266e-01 Line search: Using full step: fnorm 4.977720583264e-01 gnorm 8.679483724927e-03 Line search: Using full step: fnorm 4.977720583265e-01 gnorm 8.679483724911e-03 Line search: Using full step: fnorm 4.977720583268e-01 gnorm 8.679483725154e-03 Line search: Using full step: fnorm 4.977720583266e-01 gnorm 8.679483724907e-03 Line search: Using full step: fnorm 8.679483724927e-03 gnorm 3.188883648710e-06 Line search: Using full step: fnorm 8.679483724911e-03 gnorm 3.188883521668e-06 Line search: Using full step: fnorm 8.679483725154e-03 gnorm 3.188883528412e-06 Line search: Using full step: fnorm 8.679483724907e-03 gnorm 3.188883646614e-06 Line search: Using full step: fnorm 3.188883648710e-06 gnorm 1.136497020689e-12 Line search: Using full step: fnorm 3.188883521668e-06 gnorm 1.105469910677e-12 Line search: Using full step: fnorm 3.188883528412e-06 gnorm 9.183316572053e-13 Line search: Using full step: fnorm 3.188883646614e-06 gnorm 1.077095213730e-12 21 SNES Function norm 1.020056306772e+02 Line search: gnorm after quadratic fit 4.832827518141e+01 Line search: Quadratically determined step, lambda=2.2200960469547532e-01 Line search: gnorm after quadratic fit 4.832827518141e+01 Line search: Quadratically determined step, lambda=2.2200960469547551e-01 Line search: gnorm after quadratic fit 4.832827518141e+01 Line search: Quadratically determined step, lambda=2.2200960469547676e-01 Line search: gnorm after quadratic fit 4.832827518141e+01 Line search: Quadratically determined step, lambda=2.2200960469547534e-01 Line search: gnorm after quadratic fit 4.117408504879e+01 Line search: Quadratically determined step, lambda=1.4592085401866978e-01 Line search: gnorm after quadratic fit 4.117408504879e+01 Line search: Quadratically determined step, lambda=1.4592085401867055e-01 Line search: gnorm after quadratic fit 4.117408504879e+01 Line search: Quadratically determined step, lambda=1.4592085401867064e-01 Line search: gnorm after quadratic fit 4.117408504879e+01 Line search: Quadratically determined step, lambda=1.4592085401867111e-01 Line search: gnorm after quadratic fit 3.601829840212e+01 Line search: Quadratically determined step, lambda=1.2507652853112647e-01 Line search: gnorm after quadratic fit 3.601829840212e+01 Line search: Quadratically determined step, lambda=1.2507652853112664e-01 Line search: gnorm after quadratic fit 3.601829840212e+01 Line search: Quadratically determined step, lambda=1.2507652853112694e-01 Line search: gnorm after quadratic fit 3.601829840212e+01 Line search: Quadratically determined step, lambda=1.2507652853112772e-01 Line search: gnorm after quadratic fit 3.230247463244e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247463244e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247463244e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247463244e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250935189e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250935189e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250935189e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250935189e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674822e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674822e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674822e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674822e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734216180e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734216180e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734216180e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734216180e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878335606e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878335606e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878335606e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878335606e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798497203e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798497203e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798497203e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798497203e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832615210e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832615210e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832615210e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832615210e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930895e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930895e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930895e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930895e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319960286e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319960286e+01 Line search: gnorm after quadratic fit 1.316319960286e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319960286e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264480051e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264480051e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264480051e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264480051e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301126899e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301126898e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301126898e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301126899e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797764250e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797764250e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797764249e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797764250e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880588566e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880588566e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880588566e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880588566e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878467893e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878467893e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878467893e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878467893e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 6.995878467893e+00 gnorm 5.272280233103e+00 Line search: Using full step: fnorm 6.995878467893e+00 gnorm 5.272280233104e+00 Line search: Using full step: fnorm 6.995878467893e+00 gnorm 5.272280233103e+00 Line search: Using full step: fnorm 6.995878467893e+00 gnorm 5.272280233103e+00 Line search: Using full step: fnorm 5.272280233103e+00 gnorm 4.977720771303e-01 Line search: Using full step: fnorm 5.272280233104e+00 gnorm 4.977720771305e-01 Line search: Using full step: fnorm 5.272280233103e+00 gnorm 4.977720771304e-01 Line search: Using full step: fnorm 5.272280233103e+00 gnorm 4.977720771302e-01 Line search: Using full step: fnorm 4.977720771303e-01 gnorm 8.679484382280e-03 Line search: Using full step: fnorm 4.977720771305e-01 gnorm 8.679484382273e-03 Line search: Using full step: fnorm 4.977720771304e-01 gnorm 8.679484382276e-03 Line search: Using full step: fnorm 4.977720771302e-01 gnorm 8.679484382248e-03 Line search: Using full step: fnorm 8.679484382280e-03 gnorm 3.188884140450e-06 Line search: Using full step: fnorm 8.679484382273e-03 gnorm 3.188884366514e-06 Line search: Using full step: fnorm 8.679484382276e-03 gnorm 3.188884259091e-06 Line search: Using full step: fnorm 8.679484382248e-03 gnorm 3.188884387373e-06 Line search: Using full step: fnorm 3.188884140450e-06 gnorm 1.139508012229e-12 Line search: Using full step: fnorm 3.188884366514e-06 gnorm 9.749688842638e-13 Line search: Using full step: fnorm 3.188884259091e-06 gnorm 1.097515360832e-12 Line search: Using full step: fnorm 3.188884387373e-06 gnorm 1.085131098271e-12 22 SNES Function norm 1.020056306753e+02 Line search: gnorm after quadratic fit 4.832827517109e+01 Line search: Quadratically determined step, lambda=2.2200960468540748e-01 Line search: gnorm after quadratic fit 4.832827517109e+01 Line search: Quadratically determined step, lambda=2.2200960468540770e-01 Line search: gnorm after quadratic fit 4.832827517109e+01 Line search: Quadratically determined step, lambda=2.2200960468540795e-01 Line search: gnorm after quadratic fit 4.832827517109e+01 Line search: Quadratically determined step, lambda=2.2200960468540881e-01 Line search: gnorm after quadratic fit 4.117408504092e+01 Line search: Quadratically determined step, lambda=1.4592085399684199e-01 Line search: gnorm after quadratic fit 4.117408504092e+01 Line search: Quadratically determined step, lambda=1.4592085399684288e-01 Line search: gnorm after quadratic fit 4.117408504092e+01 Line search: Quadratically determined step, lambda=1.4592085399684221e-01 Line search: gnorm after quadratic fit 4.117408504092e+01 Line search: Quadratically determined step, lambda=1.4592085399684218e-01 Line search: gnorm after quadratic fit 3.601829839740e+01 Line search: Quadratically determined step, lambda=1.2507652848147843e-01 Line search: gnorm after quadratic fit 3.601829839740e+01 Line search: Quadratically determined step, lambda=1.2507652848147813e-01 Line search: gnorm after quadratic fit 3.601829839740e+01 Line search: Quadratically determined step, lambda=1.2507652848147965e-01 Line search: gnorm after quadratic fit 3.601829839740e+01 Line search: Quadratically determined step, lambda=1.2507652848147932e-01 Line search: gnorm after quadratic fit 3.230247462801e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462801e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462801e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462801e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934766e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934766e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934766e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934766e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674387e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674387e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674387e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674387e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215477e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215477e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215477e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215477e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334980e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334980e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334980e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334980e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496635e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496635e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496635e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496635e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614697e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614697e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614697e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614697e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930460e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930460e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930460e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930460e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959880e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959880e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959880e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959880e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479695e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479695e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479695e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479695e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301123666e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301123665e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301123665e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301123666e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797761976e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797761976e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797761976e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880586764e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797761976e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880586764e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880586764e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878468993e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878468993e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880586764e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878468993e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 6.995878468993e+00 gnorm 5.272280275139e+00 Line search: Using full step: fnorm 6.995878468993e+00 gnorm 5.272280275140e+00 Line search: Using full step: fnorm 6.995878468993e+00 gnorm 5.272280275140e+00 Line search: gnorm after quadratic fit 6.995878468993e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 5.272280275139e+00 gnorm 4.977720835648e-01 Line search: Using full step: fnorm 5.272280275140e+00 gnorm 4.977720835646e-01 Line search: Using full step: fnorm 5.272280275140e+00 gnorm 4.977720835648e-01 Line search: Using full step: fnorm 6.995878468993e+00 gnorm 5.272280275139e+00 Line search: Using full step: fnorm 4.977720835648e-01 gnorm 8.679484607319e-03 Line search: Using full step: fnorm 4.977720835646e-01 gnorm 8.679484607225e-03 Line search: Using full step: fnorm 4.977720835648e-01 gnorm 8.679484607215e-03 Line search: Using full step: fnorm 5.272280275139e+00 gnorm 4.977720835649e-01 Line search: Using full step: fnorm 8.679484607319e-03 gnorm 3.188884219030e-06 Line search: Using full step: fnorm 8.679484607225e-03 gnorm 3.188884360507e-06 Line search: Using full step: fnorm 8.679484607215e-03 gnorm 3.188884222691e-06 Line search: Using full step: fnorm 3.188884219030e-06 gnorm 1.093327184170e-12 Line search: Using full step: fnorm 4.977720835649e-01 gnorm 8.679484607214e-03 Line search: Using full step: fnorm 3.188884360507e-06 gnorm 1.024164823888e-12 Line search: Using full step: fnorm 3.188884222691e-06 gnorm 1.185699041916e-12 Line search: Using full step: fnorm 8.679484607214e-03 gnorm 3.188884218962e-06 Line search: Using full step: fnorm 3.188884218962e-06 gnorm 1.148385566901e-12 23 SNES Function norm 1.020056306747e+02 Line search: gnorm after quadratic fit 4.832827516756e+01 Line search: Quadratically determined step, lambda=2.2200960468196251e-01 Line search: gnorm after quadratic fit 4.832827516756e+01 Line search: Quadratically determined step, lambda=2.2200960468196260e-01 Line search: gnorm after quadratic fit 4.832827516756e+01 Line search: Quadratically determined step, lambda=2.2200960468196221e-01 Line search: gnorm after quadratic fit 4.832827516756e+01 Line search: Quadratically determined step, lambda=2.2200960468196276e-01 Line search: gnorm after quadratic fit 4.117408503823e+01 Line search: Quadratically determined step, lambda=1.4592085398937416e-01 Line search: gnorm after quadratic fit 4.117408503823e+01 Line search: Quadratically determined step, lambda=1.4592085398937443e-01 Line search: gnorm after quadratic fit 4.117408503823e+01 Line search: Quadratically determined step, lambda=1.4592085398937479e-01 Line search: gnorm after quadratic fit 4.117408503823e+01 Line search: Quadratically determined step, lambda=1.4592085398937393e-01 Line search: gnorm after quadratic fit 3.601829839578e+01 Line search: Quadratically determined step, lambda=1.2507652846449116e-01 Line search: gnorm after quadratic fit 3.601829839578e+01 Line search: Quadratically determined step, lambda=1.2507652846449069e-01 Line search: gnorm after quadratic fit 3.601829839578e+01 Line search: Quadratically determined step, lambda=1.2507652846449149e-01 Line search: gnorm after quadratic fit 3.601829839578e+01 Line search: Quadratically determined step, lambda=1.2507652846449049e-01 Line search: gnorm after quadratic fit 3.230247462649e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462649e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462649e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462649e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934621e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934621e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934621e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934621e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674238e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674238e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674238e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674238e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215237e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215237e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215237e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215237e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334766e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334766e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334766e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334766e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496440e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496440e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496440e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496440e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614521e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614521e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614521e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614521e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930311e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930311e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930311e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930311e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959741e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959741e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959741e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959741e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479573e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479573e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479573e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479573e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301122559e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301122559e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301122559e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301122559e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797761199e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797761198e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797761199e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797761199e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880586147e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880586147e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880586147e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880586147e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469369e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469369e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469369e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469369e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 6.995878469369e+00 gnorm 5.272280289524e+00 Line search: Using full step: fnorm 6.995878469369e+00 gnorm 5.272280289524e+00 Line search: Using full step: fnorm 6.995878469369e+00 gnorm 5.272280289524e+00 Line search: Using full step: fnorm 6.995878469369e+00 gnorm 5.272280289524e+00 Line search: Using full step: fnorm 5.272280289524e+00 gnorm 4.977720857667e-01 Line search: Using full step: fnorm 5.272280289524e+00 gnorm 4.977720857667e-01 Line search: Using full step: fnorm 5.272280289524e+00 gnorm 4.977720857667e-01 Line search: Using full step: fnorm 5.272280289524e+00 gnorm 4.977720857667e-01 Line search: Using full step: fnorm 4.977720857667e-01 gnorm 8.679484684267e-03 Line search: Using full step: fnorm 4.977720857667e-01 gnorm 8.679484684266e-03 Line search: Using full step: fnorm 4.977720857667e-01 gnorm 8.679484684145e-03 Line search: Using full step: fnorm 4.977720857667e-01 gnorm 8.679484684148e-03 Line search: Using full step: fnorm 8.679484684267e-03 gnorm 3.188884230032e-06 Line search: Using full step: fnorm 8.679484684266e-03 gnorm 3.188884478684e-06 Line search: Using full step: fnorm 8.679484684145e-03 gnorm 3.188884374833e-06 Line search: Using full step: fnorm 8.679484684148e-03 gnorm 3.188884373244e-06 Line search: Using full step: fnorm 3.188884230032e-06 gnorm 1.267011225998e-12 Line search: Using full step: fnorm 3.188884478684e-06 gnorm 1.114503519977e-12 Line search: Using full step: fnorm 3.188884374833e-06 gnorm 1.050489646649e-12 Line search: Using full step: fnorm 3.188884373244e-06 gnorm 1.042181154901e-12 24 SNES Function norm 1.020056306745e+02 Line search: gnorm after quadratic fit 4.832827516635e+01 Line search: Quadratically determined step, lambda=2.2200960468078432e-01 Line search: gnorm after quadratic fit 4.832827516635e+01 Line search: Quadratically determined step, lambda=2.2200960468078410e-01 Line search: gnorm after quadratic fit 4.832827516635e+01 Line search: Quadratically determined step, lambda=2.2200960468078454e-01 Line search: gnorm after quadratic fit 4.832827516635e+01 Line search: Quadratically determined step, lambda=2.2200960468078373e-01 Line search: gnorm after quadratic fit 4.117408503731e+01 Line search: gnorm after quadratic fit 4.117408503731e+01 Line search: Quadratically determined step, lambda=1.4592085398681848e-01 Line search: gnorm after quadratic fit 4.117408503731e+01 Line search: Quadratically determined step, lambda=1.4592085398681900e-01 Line search: Quadratically determined step, lambda=1.4592085398681842e-01 Line search: gnorm after quadratic fit 4.117408503731e+01 Line search: Quadratically determined step, lambda=1.4592085398681834e-01 Line search: gnorm after quadratic fit 3.601829839523e+01 Line search: Quadratically determined step, lambda=1.2507652845867751e-01 Line search: gnorm after quadratic fit 3.601829839523e+01 Line search: Quadratically determined step, lambda=1.2507652845867870e-01 Line search: gnorm after quadratic fit 3.601829839523e+01 Line search: Quadratically determined step, lambda=1.2507652845867862e-01 Line search: gnorm after quadratic fit 3.601829839523e+01 Line search: Quadratically determined step, lambda=1.2507652845867884e-01 Line search: gnorm after quadratic fit 3.230247462598e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462598e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462598e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462598e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934572e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934572e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934572e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934572e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674187e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674187e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674187e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674187e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215155e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215155e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215155e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215155e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334692e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334693e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334693e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334693e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496374e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496374e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496374e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496374e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614461e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614461e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614461e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614461e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930260e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930260e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930260e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930260e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959694e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959694e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959694e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959694e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479531e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479531e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479531e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479531e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301122181e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301122181e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301122181e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301122181e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760933e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760933e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760933e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760933e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585936e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585936e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585936e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585936e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469497e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469497e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469497e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469497e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 6.995878469497e+00 gnorm 5.272280294443e+00 Line search: Using full step: fnorm 6.995878469497e+00 gnorm 5.272280294443e+00 Line search: Using full step: fnorm 6.995878469497e+00 gnorm 5.272280294443e+00 Line search: Using full step: fnorm 6.995878469497e+00 gnorm 5.272280294443e+00 Line search: Using full step: fnorm 5.272280294443e+00 gnorm 4.977720865195e-01 Line search: Using full step: fnorm 5.272280294443e+00 gnorm 4.977720865197e-01 Line search: Using full step: fnorm 5.272280294443e+00 gnorm 4.977720865194e-01 Line search: Using full step: fnorm 5.272280294443e+00 gnorm 4.977720865196e-01 Line search: Using full step: fnorm 4.977720865195e-01 gnorm 8.679484710459e-03 Line search: Using full step: fnorm 4.977720865197e-01 gnorm 8.679484710686e-03 Line search: Using full step: fnorm 4.977720865194e-01 gnorm 8.679484710425e-03 Line search: Using full step: fnorm 4.977720865196e-01 gnorm 8.679484710323e-03 Line search: Using full step: fnorm 8.679484710459e-03 gnorm 3.188884306284e-06 Line search: Using full step: fnorm 8.679484710686e-03 gnorm 3.188884302433e-06 Line search: Using full step: fnorm 8.679484710425e-03 gnorm 3.188884312227e-06 Line search: Using full step: fnorm 8.679484710323e-03 gnorm 3.188884309895e-06 Line search: Using full step: fnorm 3.188884306284e-06 gnorm 1.193975548164e-12 Line search: Using full step: fnorm 3.188884302433e-06 gnorm 1.415378968891e-12 Line search: Using full step: fnorm 3.188884312227e-06 gnorm 1.244666413159e-12 Line search: Using full step: fnorm 3.188884309895e-06 gnorm 1.332296272996e-12 25 SNES Function norm 1.020056306744e+02 Line search: gnorm after quadratic fit 4.832827516594e+01 Line search: gnorm after quadratic fit 4.832827516594e+01 Line search: Quadratically determined step, lambda=2.2200960468038117e-01 Line search: Quadratically determined step, lambda=2.2200960468038125e-01 Line search: gnorm after quadratic fit 4.832827516594e+01 Line search: Quadratically determined step, lambda=2.2200960468038197e-01 Line search: gnorm after quadratic fit 4.832827516594e+01 Line search: Quadratically determined step, lambda=2.2200960468038181e-01 Line search: gnorm after quadratic fit 4.117408503700e+01 Line search: Quadratically determined step, lambda=1.4592085398594351e-01 Line search: gnorm after quadratic fit 4.117408503700e+01 Line search: Quadratically determined step, lambda=1.4592085398594404e-01 Line search: gnorm after quadratic fit 4.117408503700e+01 Line search: Quadratically determined step, lambda=1.4592085398594420e-01 Line search: gnorm after quadratic fit 4.117408503700e+01 Line search: Quadratically determined step, lambda=1.4592085398594357e-01 Line search: gnorm after quadratic fit 3.601829839504e+01 Line search: Quadratically determined step, lambda=1.2507652845668907e-01 Line search: gnorm after quadratic fit 3.601829839504e+01 Line search: Quadratically determined step, lambda=1.2507652845669001e-01 Line search: gnorm after quadratic fit 3.601829839504e+01 Line search: Quadratically determined step, lambda=1.2507652845668812e-01 Line search: gnorm after quadratic fit 3.601829839504e+01 Line search: Quadratically determined step, lambda=1.2507652845668857e-01 Line search: gnorm after quadratic fit 3.230247462580e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462580e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462580e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462580e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934555e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934555e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934555e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934555e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674169e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674169e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674169e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674169e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215127e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215127e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215126e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215127e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334667e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334667e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334667e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334667e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496351e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496351e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496351e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496351e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614441e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614441e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614441e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614441e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930242e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930242e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930242e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930242e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959678e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959678e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959678e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959678e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479517e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479517e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479517e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479517e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301122051e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301122051e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301122051e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301122051e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760842e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760841e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760841e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760841e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585864e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585863e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585864e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585864e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469542e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469541e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469541e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469541e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 6.995878469542e+00 gnorm 5.272280296129e+00 Line search: Using full step: fnorm 6.995878469541e+00 gnorm 5.272280296129e+00 Line search: Using full step: fnorm 6.995878469541e+00 gnorm 5.272280296129e+00 Line search: Using full step: fnorm 6.995878469541e+00 gnorm 5.272280296128e+00 Line search: Using full step: fnorm 5.272280296129e+00 gnorm 4.977720867777e-01 Line search: Using full step: fnorm 5.272280296129e+00 gnorm 4.977720867777e-01 Line search: Using full step: fnorm 5.272280296129e+00 gnorm 4.977720867772e-01 Line search: Using full step: fnorm 5.272280296128e+00 gnorm 4.977720867773e-01 Line search: Using full step: fnorm 4.977720867777e-01 gnorm 8.679484719684e-03 Line search: Using full step: fnorm 4.977720867777e-01 gnorm 8.679484719430e-03 Line search: Using full step: fnorm 4.977720867772e-01 gnorm 8.679484719803e-03 Line search: Using full step: fnorm 4.977720867773e-01 gnorm 8.679484719791e-03 Line search: Using full step: fnorm 8.679484719684e-03 gnorm 3.188884185305e-06 Line search: Using full step: fnorm 8.679484719430e-03 gnorm 3.188884297937e-06 Line search: Using full step: fnorm 8.679484719803e-03 gnorm 3.188884195770e-06 Line search: Using full step: fnorm 8.679484719791e-03 gnorm 3.188884192446e-06 Line search: Using full step: fnorm 3.188884185305e-06 gnorm 1.325924665877e-12 Line search: Using full step: fnorm 3.188884297937e-06 gnorm 1.041648871691e-12 Line search: Using full step: fnorm 3.188884195770e-06 gnorm 1.187227631816e-12 Line search: Using full step: fnorm 3.188884192446e-06 gnorm 1.287319276045e-12 26 SNES Function norm 1.020056306744e+02 Line search: gnorm after quadratic fit 4.832827516580e+01 Line search: Quadratically determined step, lambda=2.2200960468024339e-01 Line search: gnorm after quadratic fit 4.832827516580e+01 Line search: Quadratically determined step, lambda=2.2200960468024344e-01 Line search: gnorm after quadratic fit 4.832827516580e+01 Line search: Quadratically determined step, lambda=2.2200960468024236e-01 Line search: gnorm after quadratic fit 4.832827516580e+01 Line search: Quadratically determined step, lambda=2.2200960468024236e-01 Line search: gnorm after quadratic fit 4.117408503689e+01 Line search: Quadratically determined step, lambda=1.4592085398564555e-01 Line search: gnorm after quadratic fit 4.117408503689e+01 Line search: Quadratically determined step, lambda=1.4592085398564564e-01 Line search: gnorm after quadratic fit 4.117408503689e+01 Line search: Quadratically determined step, lambda=1.4592085398564547e-01 Line search: gnorm after quadratic fit 4.117408503689e+01 Line search: Quadratically determined step, lambda=1.4592085398564503e-01 Line search: gnorm after quadratic fit 3.601829839498e+01 Line search: Quadratically determined step, lambda=1.2507652845600861e-01 Line search: gnorm after quadratic fit 3.601829839498e+01 Line search: Quadratically determined step, lambda=1.2507652845600820e-01 Line search: gnorm after quadratic fit 3.601829839498e+01 Line search: Quadratically determined step, lambda=1.2507652845600975e-01 Line search: gnorm after quadratic fit 3.601829839498e+01 Line search: Quadratically determined step, lambda=1.2507652845600933e-01 Line search: gnorm after quadratic fit 3.230247462574e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462574e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462574e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462574e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934549e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934549e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934549e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934549e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674163e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674163e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674163e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674163e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215117e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215117e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215117e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215117e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334659e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334659e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334659e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334659e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496343e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496343e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496343e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496343e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614434e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614434e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614434e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614434e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930236e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930236e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930236e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930236e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959672e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959672e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959672e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959672e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479512e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479512e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479512e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479512e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301122007e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301122007e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301122007e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301122007e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760810e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760810e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760810e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760810e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585839e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585839e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585839e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585839e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469557e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469557e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469557e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469557e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 6.995878469557e+00 gnorm 5.272280296704e+00 Line search: Using full step: fnorm 6.995878469557e+00 gnorm 5.272280296705e+00 Line search: Using full step: fnorm 6.995878469557e+00 gnorm 5.272280296705e+00 Line search: Using full step: fnorm 6.995878469557e+00 gnorm 5.272280296705e+00 Line search: Using full step: fnorm 5.272280296704e+00 gnorm 4.977720868656e-01 Line search: Using full step: fnorm 5.272280296705e+00 gnorm 4.977720868657e-01 Line search: Using full step: fnorm 5.272280296705e+00 gnorm 4.977720868658e-01 Line search: Using full step: fnorm 5.272280296705e+00 gnorm 4.977720868661e-01 Line search: Using full step: fnorm 4.977720868656e-01 gnorm 8.679484722829e-03 Line search: Using full step: fnorm 4.977720868657e-01 gnorm 8.679484722812e-03 Line search: Using full step: fnorm 4.977720868658e-01 gnorm 8.679484722840e-03 Line search: Using full step: fnorm 4.977720868661e-01 gnorm 8.679484722723e-03 Line search: Using full step: fnorm 8.679484722829e-03 gnorm 3.188884364524e-06 Line search: Using full step: fnorm 8.679484722812e-03 gnorm 3.188884366224e-06 Line search: Using full step: fnorm 8.679484722840e-03 gnorm 3.188884375985e-06 Line search: Using full step: fnorm 8.679484722723e-03 gnorm 3.188884265122e-06 Line search: Using full step: fnorm 3.188884364524e-06 gnorm 9.942455994581e-13 Line search: Using full step: fnorm 3.188884366224e-06 gnorm 1.108631689515e-12 Line search: Using full step: fnorm 3.188884375985e-06 gnorm 1.151899806314e-12 Line search: Using full step: fnorm 3.188884265122e-06 gnorm 1.185037869177e-12 27 SNES Function norm 1.020056306744e+02 Line search: gnorm after quadratic fit 4.832827516575e+01 Line search: Quadratically determined step, lambda=2.2200960468019579e-01 Line search: gnorm after quadratic fit 4.832827516575e+01 Line search: Quadratically determined step, lambda=2.2200960468019632e-01 Line search: gnorm after quadratic fit 4.832827516575e+01 Line search: Quadratically determined step, lambda=2.2200960468019643e-01 Line search: gnorm after quadratic fit 4.832827516575e+01 Line search: Quadratically determined step, lambda=2.2200960468019604e-01 Line search: gnorm after quadratic fit 4.117408503685e+01 Line search: Quadratically determined step, lambda=1.4592085398554283e-01 Line search: gnorm after quadratic fit 4.117408503685e+01 Line search: Quadratically determined step, lambda=1.4592085398554214e-01 Line search: gnorm after quadratic fit 4.117408503685e+01 Line search: Quadratically determined step, lambda=1.4592085398554283e-01 Line search: gnorm after quadratic fit 4.117408503685e+01 Line search: Quadratically determined step, lambda=1.4592085398554278e-01 Line search: gnorm after quadratic fit 3.601829839496e+01 Line search: Quadratically determined step, lambda=1.2507652845577485e-01 Line search: gnorm after quadratic fit 3.601829839496e+01 Line search: Quadratically determined step, lambda=1.2507652845577591e-01 Line search: gnorm after quadratic fit 3.601829839496e+01 Line search: Quadratically determined step, lambda=1.2507652845577641e-01 Line search: gnorm after quadratic fit 3.601829839496e+01 Line search: Quadratically determined step, lambda=1.2507652845577621e-01 Line search: gnorm after quadratic fit 3.230247462572e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462572e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462572e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462572e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934547e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934547e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934547e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934547e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674161e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674161e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674161e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674161e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215114e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215114e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215114e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215114e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334656e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334656e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334656e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334656e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496340e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496340e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496340e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496340e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614431e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614431e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614431e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614431e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930234e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930234e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930234e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930234e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959670e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959670e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959670e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959670e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479510e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479510e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479510e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479510e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121992e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121992e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121992e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121992e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760800e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760799e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760799e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585830e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760800e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585830e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585830e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585830e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469562e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469562e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469562e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469562e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 6.995878469562e+00 gnorm 5.272280296903e+00 Line search: Using full step: fnorm 6.995878469562e+00 gnorm 5.272280296902e+00 Line search: Using full step: fnorm 6.995878469562e+00 gnorm 5.272280296901e+00 Line search: Using full step: fnorm 6.995878469562e+00 gnorm 5.272280296902e+00 Line search: Using full step: fnorm 5.272280296903e+00 gnorm 4.977720868958e-01 Line search: Using full step: fnorm 5.272280296902e+00 gnorm 4.977720868959e-01 Line search: Using full step: fnorm 5.272280296902e+00 gnorm 4.977720868960e-01 Line search: Using full step: fnorm 5.272280296901e+00 gnorm 4.977720868959e-01 Line search: Using full step: fnorm 4.977720868958e-01 gnorm 8.679484723649e-03 Line search: Using full step: fnorm 4.977720868960e-01 gnorm 8.679484723635e-03 Line search: Using full step: fnorm 4.977720868959e-01 gnorm 8.679484723636e-03 Line search: Using full step: fnorm 8.679484723649e-03 gnorm 3.188884476249e-06 Line search: Using full step: fnorm 4.977720868959e-01 gnorm 8.679484723634e-03 Line search: Using full step: fnorm 8.679484723635e-03 gnorm 3.188884487112e-06 Line search: Using full step: fnorm 3.188884476249e-06 gnorm 1.015090019563e-12 Line search: Using full step: fnorm 8.679484723636e-03 gnorm 3.188884480508e-06 Line search: Using full step: fnorm 3.188884487112e-06 gnorm 1.006813583605e-12 Line search: Using full step: fnorm 8.679484723634e-03 gnorm 3.188884485399e-06 Line search: Using full step: fnorm 3.188884480508e-06 gnorm 1.124708288428e-12 Line search: Using full step: fnorm 3.188884485399e-06 gnorm 1.017053128759e-12 28 SNES Function norm 1.020056306744e+02 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017888e-01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017938e-01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017927e-01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017894e-01 Line search: gnorm after quadratic fit 4.117408503684e+01 Line search: gnorm after quadratic fit 4.117408503684e+01 Line search: Quadratically determined step, lambda=1.4592085398550789e-01 Line search: Quadratically determined step, lambda=1.4592085398550833e-01 Line search: gnorm after quadratic fit 4.117408503684e+01 Line search: Quadratically determined step, lambda=1.4592085398550825e-01 Line search: gnorm after quadratic fit 4.117408503684e+01 Line search: Quadratically determined step, lambda=1.4592085398550717e-01 Line search: gnorm after quadratic fit 3.601829839495e+01 Line search: Quadratically determined step, lambda=1.2507652845569578e-01 Line search: gnorm after quadratic fit 3.601829839495e+01 Line search: Quadratically determined step, lambda=1.2507652845569625e-01 Line search: gnorm after quadratic fit 3.601829839495e+01 Line search: Quadratically determined step, lambda=1.2507652845569606e-01 Line search: gnorm after quadratic fit 3.601829839495e+01 Line search: Quadratically determined step, lambda=1.2507652845569575e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934547e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934547e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934547e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674161e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674161e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934547e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674161e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674161e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334655e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334655e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334655e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334655e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930234e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930234e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930234e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930234e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479510e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479510e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121987e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121987e+00 Line search: gnorm after quadratic fit 1.107264479510e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760796e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760796e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121987e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479510e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585827e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585828e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760796e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121987e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469564e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469564e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585827e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760796e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 6.995878469564e+00 gnorm 5.272280296970e+00 Line search: Using full step: fnorm 6.995878469564e+00 gnorm 5.272280296970e+00 Line search: gnorm after quadratic fit 6.995878469564e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585827e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 5.272280296970e+00 gnorm 4.977720869065e-01 Line search: Using full step: fnorm 5.272280296970e+00 gnorm 4.977720869063e-01 Line search: Using full step: fnorm 6.995878469564e+00 gnorm 5.272280296970e+00 Line search: Using full step: fnorm 4.977720869065e-01 gnorm 8.679484724049e-03 Line search: Using full step: fnorm 4.977720869063e-01 gnorm 8.679484724057e-03 Line search: gnorm after quadratic fit 6.995878469564e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 5.272280296970e+00 gnorm 4.977720869064e-01 Line search: Using full step: fnorm 8.679484724049e-03 gnorm 3.188884387631e-06 Line search: Using full step: fnorm 8.679484724057e-03 gnorm 3.188884490220e-06 Line search: Using full step: fnorm 6.995878469564e+00 gnorm 5.272280296969e+00 Line search: Using full step: fnorm 4.977720869064e-01 gnorm 8.679484724168e-03 Line search: Using full step: fnorm 3.188884387631e-06 gnorm 1.011705276682e-12 Line search: Using full step: fnorm 3.188884490220e-06 gnorm 1.075925650836e-12 Line search: Using full step: fnorm 5.272280296969e+00 gnorm 4.977720869062e-01 Line search: Using full step: fnorm 8.679484724168e-03 gnorm 3.188884391122e-06 Line search: Using full step: fnorm 4.977720869062e-01 gnorm 8.679484724057e-03 Line search: Using full step: fnorm 3.188884391122e-06 gnorm 1.218742722311e-12 Line search: Using full step: fnorm 8.679484724057e-03 gnorm 3.188884508673e-06 Line search: Using full step: fnorm 3.188884508673e-06 gnorm 9.396637003174e-13 29 SNES Function norm 1.020056306744e+02 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017292e-01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017283e-01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017339e-01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017403e-01 Line search: gnorm after quadratic fit 4.117408503684e+01 Line search: Quadratically determined step, lambda=1.4592085398549656e-01 Line search: gnorm after quadratic fit 4.117408503684e+01 Line search: Quadratically determined step, lambda=1.4592085398549590e-01 Line search: gnorm after quadratic fit 4.117408503684e+01 Line search: Quadratically determined step, lambda=1.4592085398549634e-01 Line search: gnorm after quadratic fit 4.117408503684e+01 Line search: Quadratically determined step, lambda=1.4592085398549598e-01 Line search: gnorm after quadratic fit 3.601829839495e+01 Line search: Quadratically determined step, lambda=1.2507652845566977e-01 Line search: gnorm after quadratic fit 3.601829839495e+01 Line search: Quadratically determined step, lambda=1.2507652845566869e-01 Line search: gnorm after quadratic fit 3.601829839495e+01 Line search: Quadratically determined step, lambda=1.2507652845566875e-01 Line search: gnorm after quadratic fit 3.601829839495e+01 Line search: Quadratically determined step, lambda=1.2507652845566877e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334655e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334655e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334655e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334655e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121985e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121985e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121985e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121985e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760795e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760795e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760795e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760795e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585827e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585827e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585827e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585827e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469564e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469564e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469564e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 6.995878469564e+00 gnorm 5.272280296992e+00 Line search: Using full step: fnorm 6.995878469564e+00 gnorm 5.272280296993e+00 Line search: Using full step: fnorm 6.995878469564e+00 gnorm 5.272280296992e+00 Line search: gnorm after quadratic fit 6.995878469564e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 5.272280296992e+00 gnorm 4.977720869098e-01 Line search: Using full step: fnorm 5.272280296993e+00 gnorm 4.977720869102e-01 Line search: Using full step: fnorm 5.272280296992e+00 gnorm 4.977720869101e-01 Line search: Using full step: fnorm 6.995878469564e+00 gnorm 5.272280296994e+00 Line search: Using full step: fnorm 4.977720869098e-01 gnorm 8.679484724109e-03 Line search: Using full step: fnorm 4.977720869102e-01 gnorm 8.679484724127e-03 Line search: Using full step: fnorm 4.977720869101e-01 gnorm 8.679484724332e-03 Line search: Using full step: fnorm 5.272280296994e+00 gnorm 4.977720869099e-01 Line search: Using full step: fnorm 8.679484724109e-03 gnorm 3.188884366954e-06 Line search: Using full step: fnorm 8.679484724332e-03 gnorm 3.188884481493e-06 Line search: Using full step: fnorm 8.679484724127e-03 gnorm 3.188884140213e-06 Line search: Using full step: fnorm 4.977720869099e-01 gnorm 8.679484724103e-03 Line search: Using full step: fnorm 3.188884366954e-06 gnorm 1.081069327575e-12 Line search: Using full step: fnorm 3.188884140213e-06 gnorm 1.140126765994e-12 Line search: Using full step: fnorm 3.188884481493e-06 gnorm 1.098104504104e-12 Line search: Using full step: fnorm 8.679484724103e-03 gnorm 3.188884614242e-06 Line search: Using full step: fnorm 3.188884614242e-06 gnorm 9.611554868156e-13 30 SNES Function norm 1.020056306744e+02 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017247e-01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017197e-01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017300e-01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017267e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398549140e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398549115e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398549104e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398549104e-01 Line search: gnorm after quadratic fit 3.601829839495e+01 Line search: Quadratically determined step, lambda=1.2507652845565936e-01 Line search: gnorm after quadratic fit 3.601829839495e+01 Line search: Quadratically determined step, lambda=1.2507652845565825e-01 Line search: gnorm after quadratic fit 3.601829839495e+01 Line search: Quadratically determined step, lambda=1.2507652845565906e-01 Line search: gnorm after quadratic fit 3.601829839495e+01 Line search: Quadratically determined step, lambda=1.2507652845565939e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760795e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760795e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760795e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760795e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469564e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 6.995878469564e+00 gnorm 5.272280296999e+00 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280296999e+00 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280296999e+00 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 5.272280296999e+00 gnorm 4.977720869105e-01 Line search: Using full step: fnorm 5.272280296999e+00 gnorm 4.977720869105e-01 Line search: Using full step: fnorm 5.272280296999e+00 gnorm 4.977720869107e-01 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280296998e+00 Line search: Using full step: fnorm 4.977720869105e-01 gnorm 8.679484724219e-03 Line search: Using full step: fnorm 4.977720869105e-01 gnorm 8.679484724324e-03 Line search: Using full step: fnorm 4.977720869107e-01 gnorm 8.679484724192e-03 Line search: Using full step: fnorm 5.272280296998e+00 gnorm 4.977720869106e-01 Line search: Using full step: fnorm 8.679484724219e-03 gnorm 3.188884468866e-06 Line search: Using full step: fnorm 8.679484724324e-03 gnorm 3.188884583014e-06 Line search: Using full step: fnorm 8.679484724192e-03 gnorm 3.188884469700e-06 Line search: Using full step: fnorm 4.977720869106e-01 gnorm 8.679484724245e-03 Line search: Using full step: fnorm 3.188884468866e-06 gnorm 1.204416932363e-12 Line search: Using full step: fnorm 3.188884583014e-06 gnorm 1.166994961184e-12 Line search: Using full step: fnorm 3.188884469700e-06 gnorm 1.062333793343e-12 Line search: Using full step: fnorm 8.679484724245e-03 gnorm 3.188884465264e-06 Line search: Using full step: fnorm 3.188884465264e-06 gnorm 1.294230480350e-12 31 SNES Function norm 1.020056306744e+02 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017092e-01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017156e-01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017134e-01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017139e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398549032e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398549090e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398549079e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398549109e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565576e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565570e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565634e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565650e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469564e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469564e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297002e+00 Line search: Using full step: fnorm 6.995878469564e+00 gnorm 5.272280297002e+00 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297002e+00 Line search: Using full step: fnorm 6.995878469564e+00 gnorm 5.272280297001e+00 Line search: Using full step: fnorm 5.272280297002e+00 gnorm 4.977720869109e-01 Line search: Using full step: fnorm 5.272280297002e+00 gnorm 4.977720869113e-01 Line search: Using full step: fnorm 5.272280297002e+00 gnorm 4.977720869114e-01 Line search: Using full step: fnorm 5.272280297001e+00 gnorm 4.977720869109e-01 Line search: Using full step: fnorm 4.977720869109e-01 gnorm 8.679484724167e-03 Line search: Using full step: fnorm 4.977720869113e-01 gnorm 8.679484724425e-03 Line search: Using full step: fnorm 4.977720869114e-01 gnorm 8.679484724422e-03 Line search: Using full step: fnorm 4.977720869109e-01 gnorm 8.679484724429e-03 Line search: Using full step: fnorm 8.679484724167e-03 gnorm 3.188884272156e-06 Line search: Using full step: fnorm 8.679484724425e-03 gnorm 3.188884063436e-06 Line search: Using full step: fnorm 8.679484724422e-03 gnorm 3.188884054884e-06 Line search: Using full step: fnorm 8.679484724429e-03 gnorm 3.188884272302e-06 Line search: Using full step: fnorm 3.188884272156e-06 gnorm 1.063520435978e-12 Line search: Using full step: fnorm 3.188884063436e-06 gnorm 1.279189487429e-12 Line search: Using full step: fnorm 3.188884054884e-06 gnorm 1.155874150726e-12 Line search: Using full step: fnorm 3.188884272302e-06 gnorm 1.219191504218e-12 32 SNES Function norm 1.020056306744e+02 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017078e-01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017067e-01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017086e-01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017131e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398549004e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398549021e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398549023e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398549004e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565548e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565556e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565542e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565623e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297004e+00 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297004e+00 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297004e+00 Line search: Using full step: fnorm 5.272280297004e+00 gnorm 4.977720869116e-01 Line search: Using full step: fnorm 5.272280297004e+00 gnorm 4.977720869112e-01 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297005e+00 Line search: Using full step: fnorm 5.272280297004e+00 gnorm 4.977720869112e-01 Line search: Using full step: fnorm 4.977720869116e-01 gnorm 8.679484724414e-03 Line search: Using full step: fnorm 4.977720869112e-01 gnorm 8.679484724309e-03 Line search: Using full step: fnorm 5.272280297005e+00 gnorm 4.977720869117e-01 Line search: Using full step: fnorm 4.977720869112e-01 gnorm 8.679484724169e-03 Line search: Using full step: fnorm 8.679484724414e-03 gnorm 3.188884190373e-06 Line search: Using full step: fnorm 8.679484724309e-03 gnorm 3.188884308390e-06 Line search: Using full step: fnorm 4.977720869117e-01 gnorm 8.679484724411e-03 Line search: Using full step: fnorm 8.679484724169e-03 gnorm 3.188884194371e-06 Line search: Using full step: fnorm 3.188884308390e-06 gnorm 1.247866045127e-12 Line search: Using full step: fnorm 3.188884190373e-06 gnorm 1.228645313556e-12 Line search: Using full step: fnorm 3.188884194371e-06 gnorm 1.472066558456e-12 Line search: Using full step: fnorm 8.679484724411e-03 gnorm 3.188884229520e-06 Line search: Using full step: fnorm 3.188884229520e-06 gnorm 1.216794771148e-12 33 SNES Function norm 1.020056306744e+02 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017220e-01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017195e-01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017170e-01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017211e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548887e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548968e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548868e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548890e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565406e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565534e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565473e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565448e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297004e+00 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297004e+00 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297005e+00 Line search: Using full step: fnorm 5.272280297004e+00 gnorm 4.977720869116e-01 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297004e+00 Line search: Using full step: fnorm 5.272280297004e+00 gnorm 4.977720869114e-01 Line search: Using full step: fnorm 5.272280297005e+00 gnorm 4.977720869116e-01 Line search: Using full step: fnorm 4.977720869116e-01 gnorm 8.679484724296e-03 Line search: Using full step: fnorm 5.272280297004e+00 gnorm 4.977720869117e-01 Line search: Using full step: fnorm 4.977720869114e-01 gnorm 8.679484724304e-03 Line search: Using full step: fnorm 4.977720869116e-01 gnorm 8.679484724322e-03 Line search: Using full step: fnorm 8.679484724296e-03 gnorm 3.188884108496e-06 Line search: Using full step: fnorm 8.679484724304e-03 gnorm 3.188884326655e-06 Line search: Using full step: fnorm 4.977720869117e-01 gnorm 8.679484724315e-03 Line search: Using full step: fnorm 8.679484724322e-03 gnorm 3.188884335183e-06 Line search: Using full step: fnorm 3.188884108496e-06 gnorm 1.269563130991e-12 Line search: Using full step: fnorm 3.188884326655e-06 gnorm 1.384732858637e-12 Line search: Using full step: fnorm 8.679484724315e-03 gnorm 3.188884334185e-06 Line search: Using full step: fnorm 3.188884335183e-06 gnorm 1.131705806116e-12 Line search: Using full step: fnorm 3.188884334185e-06 gnorm 1.166660051381e-12 34 SNES Function norm 1.020056306744e+02 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017172e-01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017239e-01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017272e-01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017131e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548860e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548854e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548876e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548860e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565406e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565426e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565506e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565478e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297005e+00 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297005e+00 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297004e+00 Line search: Using full step: fnorm 5.272280297005e+00 gnorm 4.977720869116e-01 Line search: Using full step: fnorm 5.272280297005e+00 gnorm 4.977720869117e-01 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297004e+00 Line search: Using full step: fnorm 5.272280297004e+00 gnorm 4.977720869112e-01 Line search: Using full step: fnorm 4.977720869116e-01 gnorm 8.679484724437e-03 Line search: Using full step: fnorm 4.977720869117e-01 gnorm 8.679484724329e-03 Line search: Using full step: fnorm 4.977720869112e-01 gnorm 8.679484724326e-03 Line search: Using full step: fnorm 5.272280297004e+00 gnorm 4.977720869115e-01 Line search: Using full step: fnorm 8.679484724437e-03 gnorm 3.188884219161e-06 Line search: Using full step: fnorm 8.679484724329e-03 gnorm 3.188884339566e-06 Line search: Using full step: fnorm 4.977720869115e-01 gnorm 8.679484724330e-03 Line search: Using full step: fnorm 8.679484724326e-03 gnorm 3.188884098513e-06 Line search: Using full step: fnorm 3.188884219161e-06 gnorm 1.238443985503e-12 Line search: Using full step: fnorm 3.188884339566e-06 gnorm 1.270488082810e-12 Line search: Using full step: fnorm 3.188884098513e-06 gnorm 1.401136100534e-12 Line search: Using full step: fnorm 8.679484724330e-03 gnorm 3.188884087471e-06 Line search: Using full step: fnorm 3.188884087471e-06 gnorm 1.201771434924e-12 35 SNES Function norm 1.020056306744e+02 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017122e-01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017211e-01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017245e-01 Line search: Quadratically determined step, lambda=2.2200960468017236e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548912e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548871e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548879e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548801e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565442e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565417e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565470e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565348e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297004e+00 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297004e+00 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 5.272280297004e+00 gnorm 4.977720869115e-01 Line search: Using full step: fnorm 5.272280297004e+00 gnorm 4.977720869116e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297004e+00 Line search: Using full step: fnorm 4.977720869115e-01 gnorm 8.679484724350e-03 Line search: Using full step: fnorm 4.977720869116e-01 gnorm 8.679484724440e-03 Line search: gnorm after quadratic fit 6.995878469564e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 5.272280297004e+00 gnorm 4.977720869115e-01 Line search: Using full step: fnorm 8.679484724350e-03 gnorm 3.188884099350e-06 Line search: Using full step: fnorm 8.679484724440e-03 gnorm 3.188884217874e-06 Line search: Using full step: fnorm 6.995878469564e+00 gnorm 5.272280297003e+00 Line search: Using full step: fnorm 3.188884099350e-06 gnorm 1.273610826522e-12 Line search: Using full step: fnorm 4.977720869115e-01 gnorm 8.679484724440e-03 Line search: Using full step: fnorm 3.188884217874e-06 gnorm 1.206943870198e-12 Line search: Using full step: fnorm 5.272280297003e+00 gnorm 4.977720869114e-01 Line search: Using full step: fnorm 8.679484724440e-03 gnorm 3.188884218766e-06 Line search: Using full step: fnorm 4.977720869114e-01 gnorm 8.679484724326e-03 Line search: Using full step: fnorm 3.188884218766e-06 gnorm 1.435480418399e-12 Line search: Using full step: fnorm 8.679484724326e-03 gnorm 3.188884098224e-06 Line search: Using full step: fnorm 3.188884098224e-06 gnorm 1.090002762591e-12 36 SNES Function norm 1.020056306744e+02 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017128e-01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017211e-01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017192e-01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017128e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548940e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548824e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548924e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548851e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565412e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565440e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565459e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565512e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297004e+00 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297003e+00 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297004e+00 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297005e+00 Line search: Using full step: fnorm 5.272280297003e+00 gnorm 4.977720869114e-01 Line search: Using full step: fnorm 5.272280297004e+00 gnorm 4.977720869115e-01 Line search: Using full step: fnorm 5.272280297004e+00 gnorm 4.977720869116e-01 Line search: Using full step: fnorm 5.272280297005e+00 gnorm 4.977720869115e-01 Line search: Using full step: fnorm 4.977720869114e-01 gnorm 8.679484724432e-03 Line search: Using full step: fnorm 4.977720869115e-01 gnorm 8.679484724311e-03 Line search: Using full step: fnorm 4.977720869116e-01 gnorm 8.679484724438e-03 Line search: Using full step: fnorm 4.977720869115e-01 gnorm 8.679484724442e-03 Line search: Using full step: fnorm 8.679484724311e-03 gnorm 3.188884098538e-06 Line search: Using full step: fnorm 8.679484724432e-03 gnorm 3.188884220070e-06 Line search: Using full step: fnorm 8.679484724438e-03 gnorm 3.188884097464e-06 Line search: Using full step: fnorm 8.679484724442e-03 gnorm 3.188884218614e-06 Line search: Using full step: fnorm 3.188884098538e-06 gnorm 1.240359149945e-12 Line search: Using full step: fnorm 3.188884220070e-06 gnorm 1.255043917216e-12 Line search: Using full step: fnorm 3.188884097464e-06 gnorm 1.110796462156e-12 Line search: Using full step: fnorm 3.188884218614e-06 gnorm 1.383304064781e-12 37 SNES Function norm 1.020056306744e+02 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017128e-01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017183e-01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017122e-01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017131e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548940e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548840e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548882e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548924e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565406e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565534e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565478e-01 Line search: Quadratically determined step, lambda=1.2507652845565392e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297004e+00 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297004e+00 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297004e+00 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297003e+00 Line search: Using full step: fnorm 5.272280297004e+00 gnorm 4.977720869115e-01 Line search: Using full step: fnorm 5.272280297004e+00 gnorm 4.977720869114e-01 Line search: Using full step: fnorm 5.272280297004e+00 gnorm 4.977720869116e-01 Line search: Using full step: fnorm 5.272280297003e+00 gnorm 4.977720869115e-01 Line search: Using full step: fnorm 4.977720869115e-01 gnorm 8.679484724437e-03 Line search: Using full step: fnorm 4.977720869114e-01 gnorm 8.679484724311e-03 Line search: Using full step: fnorm 4.977720869116e-01 gnorm 8.679484724438e-03 Line search: Using full step: fnorm 4.977720869115e-01 gnorm 8.679484724456e-03 Line search: Using full step: fnorm 8.679484724437e-03 gnorm 3.188884217511e-06 Line search: Using full step: fnorm 8.679484724311e-03 gnorm 3.188884098591e-06 Line search: Using full step: fnorm 8.679484724438e-03 gnorm 3.188884097392e-06 Line search: Using full step: fnorm 8.679484724456e-03 gnorm 3.188884218797e-06 Line search: Using full step: fnorm 3.188884217511e-06 gnorm 1.268226786200e-12 Line search: Using full step: fnorm 3.188884098591e-06 gnorm 1.253055271389e-12 Line search: Using full step: fnorm 3.188884097392e-06 gnorm 1.115079465187e-12 Line search: Using full step: fnorm 3.188884218797e-06 gnorm 1.420277433111e-12 38 SNES Function norm 1.020056306744e+02 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017128e-01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017164e-01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017178e-01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017120e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548890e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548899e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548899e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548929e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565392e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565423e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565415e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565473e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297004e+00 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297003e+00 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 5.272280297004e+00 gnorm 4.977720869114e-01 Line search: Using full step: fnorm 5.272280297003e+00 gnorm 4.977720869113e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 4.977720869114e-01 gnorm 8.679484724311e-03 Line search: Using full step: fnorm 4.977720869113e-01 gnorm 8.679484724436e-03 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297004e+00 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297004e+00 Line search: Using full step: fnorm 8.679484724311e-03 gnorm 3.188884098554e-06 Line search: Using full step: fnorm 8.679484724436e-03 gnorm 3.188884217757e-06 Line search: Using full step: fnorm 5.272280297004e+00 gnorm 4.977720869116e-01 Line search: Using full step: fnorm 5.272280297004e+00 gnorm 4.977720869115e-01 Line search: Using full step: fnorm 3.188884098554e-06 gnorm 1.252075052491e-12 Line search: Using full step: fnorm 3.188884217757e-06 gnorm 1.289827237421e-12 Line search: Using full step: fnorm 4.977720869116e-01 gnorm 8.679484724438e-03 Line search: Using full step: fnorm 4.977720869115e-01 gnorm 8.679484724442e-03 Line search: Using full step: fnorm 8.679484724438e-03 gnorm 3.188884097138e-06 Line search: Using full step: fnorm 8.679484724442e-03 gnorm 3.188884218809e-06 Line search: Using full step: fnorm 3.188884097138e-06 gnorm 1.115369901153e-12 Line search: Using full step: fnorm 3.188884218809e-06 gnorm 1.375476144695e-12 39 SNES Function norm 1.020056306744e+02 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017114e-01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017172e-01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017128e-01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017159e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548943e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548882e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548901e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548996e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565381e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565415e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565387e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565481e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297004e+00 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297003e+00 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297004e+00 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297003e+00 Line search: Using full step: fnorm 5.272280297004e+00 gnorm 4.977720869114e-01 Line search: Using full step: fnorm 5.272280297003e+00 gnorm 4.977720869113e-01 Line search: Using full step: fnorm 5.272280297004e+00 gnorm 4.977720869116e-01 Line search: Using full step: fnorm 5.272280297003e+00 gnorm 4.977720869115e-01 Line search: Using full step: fnorm 4.977720869114e-01 gnorm 8.679484724311e-03 Line search: Using full step: fnorm 4.977720869113e-01 gnorm 8.679484724436e-03 Line search: Using full step: fnorm 4.977720869116e-01 gnorm 8.679484724438e-03 Line search: Using full step: fnorm 4.977720869115e-01 gnorm 8.679484724456e-03 Line search: Using full step: fnorm 8.679484724311e-03 gnorm 3.188884098470e-06 Line search: Using full step: fnorm 8.679484724436e-03 gnorm 3.188884217747e-06 Line search: Using full step: fnorm 8.679484724438e-03 gnorm 3.188884097116e-06 Line search: Using full step: fnorm 8.679484724456e-03 gnorm 3.188884218797e-06 Line search: Using full step: fnorm 3.188884217747e-06 gnorm 1.258821430469e-12 Line search: Using full step: fnorm 3.188884098470e-06 gnorm 1.242709087863e-12 Line search: Using full step: fnorm 3.188884097116e-06 gnorm 1.115278549854e-12 Line search: Using full step: fnorm 3.188884218797e-06 gnorm 1.420284150295e-12 40 SNES Function norm 1.020056306744e+02 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017111e-01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017172e-01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017125e-01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017159e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548976e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548910e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548996e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548901e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565376e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565442e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565390e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565481e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297003e+00 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297003e+00 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297004e+00 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297003e+00 Line search: Using full step: fnorm 5.272280297003e+00 gnorm 4.977720869113e-01 Line search: Using full step: fnorm 5.272280297003e+00 gnorm 4.977720869113e-01 Line search: Using full step: fnorm 5.272280297004e+00 gnorm 4.977720869116e-01 Line search: Using full step: fnorm 5.272280297003e+00 gnorm 4.977720869115e-01 Line search: Using full step: fnorm 4.977720869113e-01 gnorm 8.679484724310e-03 Line search: Using full step: fnorm 4.977720869113e-01 gnorm 8.679484724436e-03 Line search: Using full step: fnorm 4.977720869116e-01 gnorm 8.679484724438e-03 Line search: Using full step: fnorm 4.977720869115e-01 gnorm 8.679484724456e-03 Line search: Using full step: fnorm 8.679484724310e-03 gnorm 3.188884094770e-06 Line search: Using full step: fnorm 8.679484724436e-03 gnorm 3.188884217574e-06 Line search: Using full step: fnorm 8.679484724438e-03 gnorm 3.188884097116e-06 Line search: Using full step: fnorm 8.679484724456e-03 gnorm 3.188884218792e-06 Line search: Using full step: fnorm 3.188884094770e-06 gnorm 1.244255786179e-12 Line search: Using full step: fnorm 3.188884217574e-06 gnorm 1.241013276105e-12 Line search: Using full step: fnorm 3.188884097116e-06 gnorm 1.115278549854e-12 Line search: Using full step: fnorm 3.188884218792e-06 gnorm 1.419691596810e-12 41 SNES Function norm 1.020056306744e+02 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017111e-01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017181e-01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017145e-01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017114e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548976e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548907e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548943e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548910e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565378e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565398e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565478e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565370e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297003e+00 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297003e+00 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297003e+00 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297004e+00 Line search: Using full step: fnorm 5.272280297003e+00 gnorm 4.977720869113e-01 Line search: Using full step: fnorm 5.272280297003e+00 gnorm 4.977720869113e-01 Line search: Using full step: fnorm 5.272280297003e+00 gnorm 4.977720869115e-01 Line search: Using full step: fnorm 5.272280297004e+00 gnorm 4.977720869116e-01 Line search: Using full step: fnorm 4.977720869113e-01 gnorm 8.679484724310e-03 Line search: Using full step: fnorm 4.977720869113e-01 gnorm 8.679484724436e-03 Line search: Using full step: fnorm 4.977720869115e-01 gnorm 8.679484724456e-03 Line search: Using full step: fnorm 4.977720869116e-01 gnorm 8.679484724438e-03 Line search: Using full step: fnorm 8.679484724310e-03 gnorm 3.188884094770e-06 Line search: Using full step: fnorm 8.679484724436e-03 gnorm 3.188884217747e-06 Line search: Using full step: fnorm 8.679484724456e-03 gnorm 3.188884218792e-06 Line search: Using full step: fnorm 8.679484724438e-03 gnorm 3.188884097116e-06 Line search: Using full step: fnorm 3.188884094770e-06 gnorm 1.241955401726e-12 Line search: Using full step: fnorm 3.188884217747e-06 gnorm 1.258821430469e-12 Line search: Using full step: fnorm 3.188884218792e-06 gnorm 1.419691596810e-12 Line search: Using full step: fnorm 3.188884097116e-06 gnorm 1.115294552897e-12 42 SNES Function norm 1.020056306744e+02 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017111e-01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017183e-01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017125e-01 Line search: Quadratically determined step, lambda=2.2200960468017145e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548976e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548843e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548940e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548901e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565378e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565512e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565390e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565478e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297003e+00 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297004e+00 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 5.272280297003e+00 gnorm 4.977720869113e-01 Line search: Using full step: fnorm 5.272280297004e+00 gnorm 4.977720869116e-01 Line search: Using full step: fnorm 4.977720869113e-01 gnorm 8.679484724310e-03 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297004e+00 Line search: Using full step: fnorm 4.977720869116e-01 gnorm 8.679484724438e-03 Line search: Using full step: fnorm 8.679484724438e-03 gnorm 3.188884097116e-06 Line search: Using full step: fnorm 8.679484724310e-03 gnorm 3.188884094770e-06 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 5.272280297004e+00 gnorm 4.977720869115e-01 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297003e+00 Line search: Using full step: fnorm 3.188884094770e-06 gnorm 1.241955401726e-12 Line search: Using full step: fnorm 3.188884097116e-06 gnorm 1.115294552897e-12 Line search: Using full step: fnorm 4.977720869115e-01 gnorm 8.679484724436e-03 Line search: Using full step: fnorm 5.272280297003e+00 gnorm 4.977720869115e-01 Line search: Using full step: fnorm 8.679484724436e-03 gnorm 3.188884217741e-06 Line search: Using full step: fnorm 4.977720869115e-01 gnorm 8.679484724456e-03 Line search: Using full step: fnorm 3.188884217741e-06 gnorm 1.251783426833e-12 Line search: Using full step: fnorm 8.679484724456e-03 gnorm 3.188884218792e-06 Line search: Using full step: fnorm 3.188884218792e-06 gnorm 1.419691596810e-12 43 SNES Function norm 1.020056306744e+02 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017111e-01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017128e-01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017164e-01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017145e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548976e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548901e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565376e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565390e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548876e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548940e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565478e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565378e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297003e+00 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297004e+00 Line search: Using full step: fnorm 5.272280297003e+00 gnorm 4.977720869113e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 5.272280297004e+00 gnorm 4.977720869116e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 4.977720869113e-01 gnorm 8.679484724309e-03 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297003e+00 Line search: Using full step: fnorm 4.977720869116e-01 gnorm 8.679484724438e-03 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 8.679484724309e-03 gnorm 3.188884094767e-06 Line search: Using full step: fnorm 5.272280297003e+00 gnorm 4.977720869115e-01 Line search: Using full step: fnorm 8.679484724438e-03 gnorm 3.188884097116e-06 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297003e+00 Line search: Using full step: fnorm 3.188884094767e-06 gnorm 1.243433253077e-12 Line search: Using full step: fnorm 4.977720869115e-01 gnorm 8.679484724456e-03 Line search: Using full step: fnorm 3.188884097116e-06 gnorm 1.115294552897e-12 Line search: Using full step: fnorm 5.272280297003e+00 gnorm 4.977720869113e-01 Line search: Using full step: fnorm 8.679484724456e-03 gnorm 3.188884219331e-06 Line search: Using full step: fnorm 4.977720869113e-01 gnorm 8.679484724436e-03 Line search: Using full step: fnorm 3.188884219331e-06 gnorm 1.425470073162e-12 Line search: Using full step: fnorm 8.679484724436e-03 gnorm 3.188884217747e-06 Line search: Using full step: fnorm 3.188884217747e-06 gnorm 1.258821430469e-12 44 SNES Function norm 1.020056306744e+02 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017125e-01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017111e-01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017183e-01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017145e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548843e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548943e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548901e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548976e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565512e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565478e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565390e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565378e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297004e+00 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297003e+00 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297004e+00 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297003e+00 Line search: Using full step: fnorm 5.272280297004e+00 gnorm 4.977720869115e-01 Line search: Using full step: fnorm 5.272280297003e+00 gnorm 4.977720869115e-01 Line search: Using full step: fnorm 5.272280297003e+00 gnorm 4.977720869113e-01 Line search: Using full step: fnorm 5.272280297004e+00 gnorm 4.977720869116e-01 Line search: Using full step: fnorm 4.977720869115e-01 gnorm 8.679484724436e-03 Line search: Using full step: fnorm 4.977720869115e-01 gnorm 8.679484724456e-03 Line search: Using full step: fnorm 4.977720869113e-01 gnorm 8.679484724310e-03 Line search: Using full step: fnorm 4.977720869116e-01 gnorm 8.679484724438e-03 Line search: Using full step: fnorm 8.679484724436e-03 gnorm 3.188884217741e-06 Line search: Using full step: fnorm 8.679484724456e-03 gnorm 3.188884218792e-06 Line search: Using full step: fnorm 8.679484724438e-03 gnorm 3.188884097116e-06 Line search: Using full step: fnorm 8.679484724310e-03 gnorm 3.188884094770e-06 Line search: Using full step: fnorm 3.188884217741e-06 gnorm 1.251783426833e-12 Line search: Using full step: fnorm 3.188884218792e-06 gnorm 1.419691596810e-12 Line search: Using full step: fnorm 3.188884097116e-06 gnorm 1.115294552897e-12 Line search: Using full step: fnorm 3.188884094770e-06 gnorm 1.241955401726e-12 45 SNES Function norm 1.020056306744e+02 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017128e-01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017164e-01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017145e-01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017111e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548901e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548876e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548940e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548976e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565378e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565390e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565478e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565376e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297003e+00 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297004e+00 Line search: Using full step: fnorm 5.272280297003e+00 gnorm 4.977720869113e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 5.272280297004e+00 gnorm 4.977720869116e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 4.977720869113e-01 gnorm 8.679484724436e-03 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297003e+00 Line search: Using full step: fnorm 4.977720869116e-01 gnorm 8.679484724438e-03 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297003e+00 Line search: Using full step: fnorm 8.679484724436e-03 gnorm 3.188884217747e-06 Line search: Using full step: fnorm 5.272280297003e+00 gnorm 4.977720869113e-01 Line search: Using full step: fnorm 8.679484724438e-03 gnorm 3.188884097116e-06 Line search: Using full step: fnorm 5.272280297003e+00 gnorm 4.977720869115e-01 Line search: Using full step: fnorm 3.188884217747e-06 gnorm 1.258821430469e-12 Line search: Using full step: fnorm 4.977720869113e-01 gnorm 8.679484724309e-03 Line search: Using full step: fnorm 3.188884097116e-06 gnorm 1.115294552897e-12 Line search: Using full step: fnorm 4.977720869115e-01 gnorm 8.679484724456e-03 Line search: Using full step: fnorm 8.679484724309e-03 gnorm 3.188884094767e-06 Line search: Using full step: fnorm 8.679484724456e-03 gnorm 3.188884219331e-06 Line search: Using full step: fnorm 3.188884094767e-06 gnorm 1.243433253077e-12 Line search: Using full step: fnorm 3.188884219331e-06 gnorm 1.425470073162e-12 46 SNES Function norm 1.020056306744e+02 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017111e-01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017183e-01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017125e-01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017145e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548976e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548843e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548901e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548943e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565378e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565512e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565390e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565478e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297003e+00 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297004e+00 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297004e+00 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297003e+00 Line search: Using full step: fnorm 5.272280297003e+00 gnorm 4.977720869113e-01 Line search: Using full step: fnorm 5.272280297004e+00 gnorm 4.977720869115e-01 Line search: Using full step: fnorm 5.272280297004e+00 gnorm 4.977720869116e-01 Line search: Using full step: fnorm 5.272280297003e+00 gnorm 4.977720869115e-01 Line search: Using full step: fnorm 4.977720869113e-01 gnorm 8.679484724310e-03 Line search: Using full step: fnorm 4.977720869115e-01 gnorm 8.679484724436e-03 Line search: Using full step: fnorm 4.977720869116e-01 gnorm 8.679484724438e-03 Line search: Using full step: fnorm 4.977720869115e-01 gnorm 8.679484724456e-03 Line search: Using full step: fnorm 8.679484724310e-03 gnorm 3.188884094770e-06 Line search: Using full step: fnorm 8.679484724436e-03 gnorm 3.188884217741e-06 Line search: Using full step: fnorm 8.679484724438e-03 gnorm 3.188884097116e-06 Line search: Using full step: fnorm 8.679484724456e-03 gnorm 3.188884218792e-06 Line search: Using full step: fnorm 3.188884094770e-06 gnorm 1.241955401726e-12 Line search: Using full step: fnorm 3.188884217741e-06 gnorm 1.251783426833e-12 Line search: Using full step: fnorm 3.188884097116e-06 gnorm 1.115294552897e-12 Line search: Using full step: fnorm 3.188884218792e-06 gnorm 1.419691596810e-12 47 SNES Function norm 1.020056306744e+02 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017111e-01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017164e-01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017145e-01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017128e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548976e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548876e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548940e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548901e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565376e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565378e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565478e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565390e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297003e+00 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297003e+00 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297004e+00 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297003e+00 Line search: Using full step: fnorm 5.272280297003e+00 gnorm 4.977720869113e-01 Line search: Using full step: fnorm 5.272280297003e+00 gnorm 4.977720869113e-01 Line search: Using full step: fnorm 5.272280297004e+00 gnorm 4.977720869116e-01 Line search: Using full step: fnorm 5.272280297003e+00 gnorm 4.977720869115e-01 Line search: Using full step: fnorm 4.977720869113e-01 gnorm 8.679484724309e-03 Line search: Using full step: fnorm 4.977720869113e-01 gnorm 8.679484724436e-03 Line search: Using full step: fnorm 4.977720869116e-01 gnorm 8.679484724438e-03 Line search: Using full step: fnorm 4.977720869115e-01 gnorm 8.679484724456e-03 Line search: Using full step: fnorm 8.679484724309e-03 gnorm 3.188884094767e-06 Line search: Using full step: fnorm 8.679484724436e-03 gnorm 3.188884217747e-06 Line search: Using full step: fnorm 8.679484724438e-03 gnorm 3.188884097116e-06 Line search: Using full step: fnorm 8.679484724456e-03 gnorm 3.188884219331e-06 Line search: Using full step: fnorm 3.188884094767e-06 gnorm 1.243433253077e-12 Line search: Using full step: fnorm 3.188884217747e-06 gnorm 1.258821430469e-12 Line search: Using full step: fnorm 3.188884097116e-06 gnorm 1.115294552897e-12 Line search: Using full step: fnorm 3.188884219331e-06 gnorm 1.425470073162e-12 48 SNES Function norm 1.020056306744e+02 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017111e-01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017183e-01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017145e-01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017125e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548976e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548843e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548943e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548901e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565512e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565390e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565378e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565478e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297004e+00 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297003e+00 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297004e+00 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297003e+00 Line search: Using full step: fnorm 5.272280297004e+00 gnorm 4.977720869115e-01 Line search: Using full step: fnorm 5.272280297003e+00 gnorm 4.977720869115e-01 Line search: Using full step: fnorm 5.272280297004e+00 gnorm 4.977720869116e-01 Line search: Using full step: fnorm 5.272280297003e+00 gnorm 4.977720869113e-01 Line search: Using full step: fnorm 4.977720869115e-01 gnorm 8.679484724436e-03 Line search: Using full step: fnorm 4.977720869115e-01 gnorm 8.679484724456e-03 Line search: Using full step: fnorm 4.977720869116e-01 gnorm 8.679484724438e-03 Line search: Using full step: fnorm 4.977720869113e-01 gnorm 8.679484724310e-03 Line search: Using full step: fnorm 8.679484724436e-03 gnorm 3.188884217741e-06 Line search: Using full step: fnorm 8.679484724456e-03 gnorm 3.188884218792e-06 Line search: Using full step: fnorm 8.679484724438e-03 gnorm 3.188884097116e-06 Line search: Using full step: fnorm 8.679484724310e-03 gnorm 3.188884094770e-06 Line search: Using full step: fnorm 3.188884217741e-06 gnorm 1.251783426833e-12 Line search: Using full step: fnorm 3.188884218792e-06 gnorm 1.419691596810e-12 Line search: Using full step: fnorm 3.188884097116e-06 gnorm 1.115294552897e-12 Line search: Using full step: fnorm 3.188884094770e-06 gnorm 1.241955401726e-12 49 SNES Function norm 1.020056306744e+02 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017164e-01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017145e-01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017128e-01 Line search: gnorm after quadratic fit 4.832827516573e+01 Line search: Quadratically determined step, lambda=2.2200960468017111e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548876e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548940e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548901e-01 Line search: gnorm after quadratic fit 4.117408503683e+01 Line search: Quadratically determined step, lambda=1.4592085398548976e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565378e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565478e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565390e-01 Line search: gnorm after quadratic fit 3.601829839494e+01 Line search: Quadratically determined step, lambda=1.2507652845565376e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 3.230247462571e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.900250934546e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.599278674160e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.318734215112e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 2.085878334654e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.876798496339e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.688832614430e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.473393930233e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.316319959669e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 1.107264479509e+01 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.971301121984e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 9.053797760794e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 8.226880585826e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297003e+00 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297003e+00 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297004e+00 Line search: gnorm after quadratic fit 6.995878469565e+00 Line search: Quadratically determined step, lambda=1.0000000000000001e-01 Line search: Using full step: fnorm 5.272280297003e+00 gnorm 4.977720869113e-01 Line search: Using full step: fnorm 5.272280297003e+00 gnorm 4.977720869115e-01 Line search: Using full step: fnorm 5.272280297004e+00 gnorm 4.977720869116e-01 Line search: Using full step: fnorm 6.995878469565e+00 gnorm 5.272280297003e+00 Line search: Using full step: fnorm 4.977720869113e-01 gnorm 8.679484724436e-03 Line search: Using full step: fnorm 4.977720869115e-01 gnorm 8.679484724456e-03 Line search: Using full step: fnorm 4.977720869116e-01 gnorm 8.679484724438e-03 Line search: Using full step: fnorm 5.272280297003e+00 gnorm 4.977720869113e-01 Line search: Using full step: fnorm 8.679484724436e-03 gnorm 3.188884217747e-06 Line search: Using full step: fnorm 8.679484724456e-03 gnorm 3.188884219331e-06 Line search: Using full step: fnorm 8.679484724438e-03 gnorm 3.188884097116e-06 Line search: Using full step: fnorm 4.977720869113e-01 gnorm 8.679484724309e-03 Line search: Using full step: fnorm 3.188884217747e-06 gnorm 1.258821430469e-12 Line search: Using full step: fnorm 3.188884219331e-06 gnorm 1.425470073162e-12 Line search: Using full step: fnorm 3.188884097116e-06 gnorm 1.115294552897e-12 Line search: Using full step: fnorm 8.679484724309e-03 gnorm 3.188884094767e-06 Line search: Using full step: fnorm 3.188884094767e-06 gnorm 1.243433253077e-12 50 SNES Function norm 1.020056306744e+02 SNES Object: 4 MPI processes type: nasm total subdomain blocks = 4 Local solver information for first block on rank 0: Use -snes_view ::ascii_info_detail to display information for all blocks SNES Object: (sub_) 1 MPI processes type: newtonls maximum iterations=50, maximum function evaluations=10000 tolerances: relative=1e-08, absolute=1e-50, solution=1e-08 total number of linear solver iterations=22 total number of function evaluations=40 norm schedule ALWAYS Jacobian is built using a DMDA local Jacobian SNESLineSearch Object: (sub_) 1 MPI processes type: bt interpolation: cubic alpha=1.000000e-04 maxstep=1.000000e+08, minlambda=1.000000e-12 tolerances: relative=1.000000e-08, absolute=1.000000e-15, lambda=1.000000e-08 maximum iterations=40 KSP Object: (sub_) 1 MPI processes type: preonly maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using NONE norm type for convergence test PC Object: (sub_) 1 MPI processes type: lu out-of-place factorization tolerance for zero pivot 2.22045e-14 matrix ordering: nd factor fill ratio given 5., needed 2.13732 Factored matrix follows: Mat Object: 1 MPI processes type: seqaij rows=169, cols=169 package used to perform factorization: petsc total: nonzeros=13339, allocated nonzeros=13339 using I-node routines: found 104 nodes, limit used is 5 linear system matrix = precond matrix: Mat Object: 1 MPI processes type: seqaij rows=169, cols=169 total: nonzeros=6241, allocated nonzeros=6241 total number of mallocs used during MatSetValues calls=0 not using I-node routines maximum iterations=50, maximum function evaluations=10000 tolerances: relative=1e-08, absolute=1e-50, solution=1e-08 total number of function evaluations=51 norm schedule ALWAYS Jacobian is built using a DMDA local Jacobian problem ex10 on 20 x 20 point 2D grid with d = 3, and eps = 0.082: error |u-uexact|_inf = 3.996e-01, |u-uexact|_h = 2.837e-01 WARNING! There are options you set that were not used! WARNING! could be spelling mistake, etc! There are 2 unused database options. They are: Option left: name:-ksp_converged_reason (no value) Option left: name:-snes_linesearch_monitor (no value) From elias.karabelas at uni-graz.at Thu May 12 02:13:10 2022 From: elias.karabelas at uni-graz.at (Karabelas, Elias (elias.karabelas@uni-graz.at)) Date: Thu, 12 May 2022 07:13:10 +0000 Subject: [petsc-users] [MEMORY LEAK, INTERFACE BUG] Petsc + Boomeramg + Fieldsplit on ARCHER2 Message-ID: Dear Team, I ran into some issues using Petsc with Boomeramg and FieldSplit as PC on the ARCHER2 cluster. These are my options for solving a Navier-Stokes-like system and it ran fine on other clusters. -ksp_type fgmres -ksp_pc_side right -ksp_initial_guess_nonzero 0 -ksp_lag_norm -ksp_gmres_restart 100 -ksp_monitor :block_system_ksp_its.log -pc_type fieldsplit -pc_fieldsplit_type schur -pc_fieldsplit_schur_fact_type full -pc_fieldsplit_schur_precondition selfp -fieldsplit_0_ksp_type preonly -fieldsplit_0_pc_type hypre -fieldsplit_0_pc_hypre_type boomeramg -fieldsplit_0_pc_hypre_boomeramg_max_iter 1 -fieldsplit_0_pc_hypre_boomeramg_coarsen_type HMIS -fieldsplit_0_pc_hypre_boomeramg_tol 0.0 -fieldsplit_0_pc_hypre_boomeramg_interp_type ext+i -fieldsplit_0_pc_hypre_boomeramg_strong_threshold 0.7 -fieldsplit_0_pc_hypre_boomeramg_relax_type_all l1-Gauss-Seidel -fieldsplit_1_mat_schur_complement_ainv_type blockdiag -fieldsplit_1_ksp_type gmres -fieldsplit_1_pc_type sor -fieldsplit_1_ksp_max_it 30 -fieldsplit_1_ksp_rtol 1e-1 -fieldsplit_1_ksp_atol 0.0 -fieldsplit_1_ksp_pc_side right However on ARCHER2 I ended up with OOM errors and the SLURM scheduler quit my jobs. I did some digging with valgrind I found out that in PCApply when BoomerAMG is called and HYPRE is compiled with the option "--enable-persistent 1" there are a lot of calloc's on HYPRE's side that are not free'd even after the PC gets destroyed from Petsc side. This seems to be the case on the vendor-shipped Petsc on ARCHER2. Building my own Petsc and Hypre on the cluster resolves this issue with the downside of loosing around 15% of speed. I thought the developers might be interested in this bug, but I do not know who is to be adressed here, the hypre developers, the petsc developers, or the ARCHER2 admins. Best regards Elias -- Dr. Elias Karabelas Research Associate University of Graz Institute of Mathematics and Scientific Computing Heinrichstra?e 36 A-8010 Graz Austria Phone: +43 316 380 8546 Email: elias.karabelas at uni-graz.at Web: https://ccl.medunigraz.at/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Thu May 12 08:21:27 2022 From: knepley at gmail.com (Matthew Knepley) Date: Thu, 12 May 2022 09:21:27 -0400 Subject: [petsc-users] [MEMORY LEAK, INTERFACE BUG] Petsc + Boomeramg + Fieldsplit on ARCHER2 In-Reply-To: References: Message-ID: On Thu, May 12, 2022 at 9:09 AM Karabelas, Elias ( elias.karabelas at uni-graz.at) wrote: > Dear Team, > > I ran into some issues using Petsc with Boomeramg and FieldSplit as PC on > the ARCHER2 cluster. > > These are my options for solving a Navier-Stokes-like system and it ran > fine on other clusters. > > -ksp_type fgmres > -ksp_pc_side right > -ksp_initial_guess_nonzero 0 > -ksp_lag_norm > -ksp_gmres_restart 100 > -ksp_monitor :block_system_ksp_its.log > -pc_type fieldsplit > -pc_fieldsplit_type schur > -pc_fieldsplit_schur_fact_type full > -pc_fieldsplit_schur_precondition selfp > -fieldsplit_0_ksp_type preonly > -fieldsplit_0_pc_type hypre > -fieldsplit_0_pc_hypre_type boomeramg > -fieldsplit_0_pc_hypre_boomeramg_max_iter 1 > -fieldsplit_0_pc_hypre_boomeramg_coarsen_type HMIS > -fieldsplit_0_pc_hypre_boomeramg_tol 0.0 > -fieldsplit_0_pc_hypre_boomeramg_interp_type ext+i > -fieldsplit_0_pc_hypre_boomeramg_strong_threshold 0.7 > -fieldsplit_0_pc_hypre_boomeramg_relax_type_all l1-Gauss-Seidel > -fieldsplit_1_mat_schur_complement_ainv_type blockdiag > -fieldsplit_1_ksp_type gmres > -fieldsplit_1_pc_type sor > -fieldsplit_1_ksp_max_it 30 > -fieldsplit_1_ksp_rtol 1e-1 > -fieldsplit_1_ksp_atol 0.0 > -fieldsplit_1_ksp_pc_side right > > > However on ARCHER2 I ended up with OOM errors and the SLURM scheduler quit > my jobs. I did some digging with valgrind I found out that in PCApply when > BoomerAMG is called and HYPRE is compiled with the option > "--enable-persistent 1" there are a lot of calloc's on HYPRE's side that > are not free'd even after the PC gets destroyed from Petsc side. This seems > to be the case on the vendor-shipped Petsc on ARCHER2. Building my own > Petsc and Hypre on the cluster resolves this issue with the downside of > loosing around 15% of speed. > Hi Elias, Is your self-compiled PETSc a newer version? If so, it sounds like this can be resolved by getting the admins on ARCHER to update the PETSc. They have been pretty responsive for the Firedrake people, so I don't think this will be hard. Thanks, Matt > I thought the developers might be interested in this bug, but I do not > know who is to be adressed here, the hypre developers, the petsc > developers, or the ARCHER2 admins. > > Best regards > > Elias > > -- > Dr. Elias Karabelas > Research Associate > University of Graz > Institute of Mathematics and Scientific Computing > Heinrichstra?e 36 > A-8010 Graz > Austria > > Phone: +43 316 380 8546 > Email: elias.karabelas at uni-graz.at > Web: https://ccl.medunigraz.at/ > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Thu May 12 08:36:56 2022 From: knepley at gmail.com (Matthew Knepley) Date: Thu, 12 May 2022 09:36:56 -0400 Subject: [petsc-users] Convergence issues for SNES NASM In-Reply-To: References: <87lev75sar.fsf@jedbrown.org> Message-ID: Your subdomain solves do not appear to be producing descent whatsoever. Possible reasons: 1) Your subdomain Jacobians are wrong (this is usually the problem) 2) You have some global coupling field for which local solves give no descent. (For this you want nonlinear elimination I think) Thanks, Matt On Thu, May 12, 2022 at 9:02 AM Takahashi, Tadanaga wrote: > I ran the code with the additional options but the raw output is about > 75,000 lines. I cannot paste it directly in the email. The output is in the > attached file. > > On Wed, May 11, 2022 at 11:44 PM Jed Brown wrote: > >> Can you add -snes_linesearch_monitor -sub_snes_linesearch_monitor >> -ksp_converged_reason and send the output?? >> >> "Takahashi, Tadanaga" writes: >> >> > Hello, >> > >> > We are working on a finite difference solver for a 2D nonlinear PDE with >> > Dirichlet Boundary conditions on a rectangular domain. Our goal is to >> solve >> > the problem with parallel nonlinear additive Schwarz (NASM) as the outer >> > solver. Our code is similar to SNES example 5 >> > . In example >> 5, >> > the parallel NASM can be executed with a command like `mpiexec -n 4 >> ./ex5 >> > -mms 3 -snes_type nasm -snes_nasm_type restrict -da_overlap 2` which >> gives >> > a convergent result. We assume this is the correct usage. A comment in >> the >> > source code for NASM mentions that NASM should be a preconditioner but >> > there's no documentation on the usage. The Brune paper does not cover >> > parallel NASM either. We observed that increasing the overlap leads to >> > fewer Schwarz iterations. The parallelization works seamlessly for an >> > arbitrary number of subdomains. This is the type of behavior we were >> > expecting from our code. >> > >> > Our method uses box-style stencil width d = ceil(N^(1/3)) on a N by N >> DMDA. >> > The finite difference stencil consists of 4d+1 points spread out in a >> > diamond formation. If a stencil point is out of bounds, then it is >> > projected onto the boundary curve. Since the nodes on the boundary curve >> > would result in an irregular mesh, we chose not treat boundary nodes as >> > unknowns as in Example 5. We use DMDACreate2d to create the DA for the >> > interior points and DMDASNESSetFunctionLocal to associate the residue >> > function to the SNES object. >> > >> > Our code works serially. We have also tested our code >> > with Newton-Krylov-Schwarz (NKS) by running something akin to `mpiexec >> -n >> > ./solve -snes_type newtonls`. We have tested the NKS for several >> > quantities of subdomains and overlap and the code works as expected. We >> > have some confidence in the correctness of our code. The overlapping >> NASM >> > was implemented in MATLAB so we know the method converges. However, the >> > parallel NASM will not converge with our PETSc code. We don't understand >> > why NKS works while NASM does not. The F-norm residue monotonically >> > decreases and then stagnates. >> > >> > Here is an example of the output when attempting to run NASM in >> parallel: >> > takahashi at ubuntu:~/Desktop/MA-DDM/Cpp/Rectangle$ mpiexec -n 4 ./test1 >> -t1_N >> > 20 -snes_max_it 50 -snes_monitor -snes_view -da_overlap 3 -snes_type >> nasm >> > -snes_nasm_type restrict >> > 0 SNES Function norm 7.244681057908e+02 >> > 1 SNES Function norm 1.237688062971e+02 >> > 2 SNES Function norm 1.068926073552e+02 >> > 3 SNES Function norm 1.027563237834e+02 >> > 4 SNES Function norm 1.022184806736e+02 >> > 5 SNES Function norm 1.020818227640e+02 >> > 6 SNES Function norm 1.020325629121e+02 >> > 7 SNES Function norm 1.020149036595e+02 >> > 8 SNES Function norm 1.020088110545e+02 >> > 9 SNES Function norm 1.020067198030e+02 >> > 10 SNES Function norm 1.020060034469e+02 >> > 11 SNES Function norm 1.020057582380e+02 >> > 12 SNES Function norm 1.020056743241e+02 >> > 13 SNES Function norm 1.020056456101e+02 >> > 14 SNES Function norm 1.020056357849e+02 >> > 15 SNES Function norm 1.020056324231e+02 >> > 16 SNES Function norm 1.020056312727e+02 >> > 17 SNES Function norm 1.020056308791e+02 >> > 18 SNES Function norm 1.020056307444e+02 >> > 19 SNES Function norm 1.020056306983e+02 >> > 20 SNES Function norm 1.020056306826e+02 >> > 21 SNES Function norm 1.020056306772e+02 >> > 22 SNES Function norm 1.020056306753e+02 >> > 23 SNES Function norm 1.020056306747e+02 >> > 24 SNES Function norm 1.020056306745e+02 >> > 25 SNES Function norm 1.020056306744e+02 >> > 26 SNES Function norm 1.020056306744e+02 >> > 27 SNES Function norm 1.020056306744e+02 >> > 28 SNES Function norm 1.020056306744e+02 >> > 29 SNES Function norm 1.020056306744e+02 >> > 30 SNES Function norm 1.020056306744e+02 >> > 31 SNES Function norm 1.020056306744e+02 >> > 32 SNES Function norm 1.020056306744e+02 >> > 33 SNES Function norm 1.020056306744e+02 >> > 34 SNES Function norm 1.020056306744e+02 >> > 35 SNES Function norm 1.020056306744e+02 >> > 36 SNES Function norm 1.020056306744e+02 >> > 37 SNES Function norm 1.020056306744e+02 >> > 38 SNES Function norm 1.020056306744e+02 >> > 39 SNES Function norm 1.020056306744e+02 >> > 40 SNES Function norm 1.020056306744e+02 >> > 41 SNES Function norm 1.020056306744e+02 >> > 42 SNES Function norm 1.020056306744e+02 >> > 43 SNES Function norm 1.020056306744e+02 >> > 44 SNES Function norm 1.020056306744e+02 >> > 45 SNES Function norm 1.020056306744e+02 >> > 46 SNES Function norm 1.020056306744e+02 >> > 47 SNES Function norm 1.020056306744e+02 >> > 48 SNES Function norm 1.020056306744e+02 >> > 49 SNES Function norm 1.020056306744e+02 >> > 50 SNES Function norm 1.020056306744e+02 >> > SNES Object: 4 MPI processes >> > type: nasm >> > total subdomain blocks = 4 >> > Local solver information for first block on rank 0: >> > Use -snes_view ::ascii_info_detail to display information for all >> blocks >> > SNES Object: (sub_) 1 MPI processes >> > type: newtonls >> > maximum iterations=50, maximum function evaluations=10000 >> > tolerances: relative=1e-08, absolute=1e-50, solution=1e-08 >> > total number of linear solver iterations=22 >> > total number of function evaluations=40 >> > norm schedule ALWAYS >> > Jacobian is built using a DMDA local Jacobian >> > SNESLineSearch Object: (sub_) 1 MPI processes >> > type: bt >> > interpolation: cubic >> > alpha=1.000000e-04 >> > maxstep=1.000000e+08, minlambda=1.000000e-12 >> > tolerances: relative=1.000000e-08, absolute=1.000000e-15, >> > lambda=1.000000e-08 >> > maximum iterations=40 >> > KSP Object: (sub_) 1 MPI processes >> > type: preonly >> > maximum iterations=10000, initial guess is zero >> > tolerances: relative=1e-05, absolute=1e-50, divergence=10000. >> > left preconditioning >> > using NONE norm type for convergence test >> > PC Object: (sub_) 1 MPI processes >> > type: lu >> > out-of-place factorization >> > tolerance for zero pivot 2.22045e-14 >> > matrix ordering: nd >> > factor fill ratio given 5., needed 2.13732 >> > Factored matrix follows: >> > Mat Object: 1 MPI processes >> > type: seqaij >> > rows=169, cols=169 >> > package used to perform factorization: petsc >> > total: nonzeros=13339, allocated nonzeros=13339 >> > using I-node routines: found 104 nodes, limit used is >> 5 >> > linear system matrix = precond matrix: >> > Mat Object: 1 MPI processes >> > type: seqaij >> > rows=169, cols=169 >> > total: nonzeros=6241, allocated nonzeros=6241 >> > total number of mallocs used during MatSetValues calls=0 >> > not using I-node routines >> > maximum iterations=50, maximum function evaluations=10000 >> > tolerances: relative=1e-08, absolute=1e-50, solution=1e-08 >> > total number of function evaluations=51 >> > norm schedule ALWAYS >> > Jacobian is built using a DMDA local Jacobian >> > problem ex10 on 20 x 20 point 2D grid with d = 3, and eps = 0.082: >> > error |u-uexact|_inf = 3.996e-01, |u-uexact|_h = 2.837e-01 >> > >> > We have been stuck on this for a while now. We do not know how to debug >> > this issue. Please let us know if you have any insights. >> > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From tt73 at njit.edu Thu May 12 12:03:31 2022 From: tt73 at njit.edu (Takahashi, Tadanaga) Date: Thu, 12 May 2022 13:03:31 -0400 Subject: [petsc-users] Convergence issues for SNES NASM In-Reply-To: References: <87lev75sar.fsf@jedbrown.org> Message-ID: Thank you for the feedback. We figured out what was causing the issue. We were using DMGetBoundingBox in order to get the limits of the global domain, but gmin and gmax contained limits for the local subdomains when we ran the code with NASM. Hence, our local coordinates xi and yj were completely wrong. The documentation states that DMGetBoundingBox gets the global limits. I believe this is a mistake. This is our new output: $ mpiexec -n 4 ./test1 -t1_N 20 -snes_max_it 50 -snes_monitor -snes_view -da_overlap 3 -snes_type nasm -snes_nasm_type restrict 0 SNES Function norm 7.244681057908e+02 1 SNES Function norm 4.394913250889e+01 2 SNES Function norm 1.823326663029e+01 3 SNES Function norm 7.033938512358e+00 4 SNES Function norm 2.797351504285e+00 5 SNES Function norm 1.130613777736e+00 6 SNES Function norm 4.605418417192e-01 7 SNES Function norm 1.882307001920e-01 8 SNES Function norm 7.704148683921e-02 9 SNES Function norm 3.155090858782e-02 10 SNES Function norm 1.292418188473e-02 11 SNES Function norm 5.294645671797e-03 12 SNES Function norm 2.169143207557e-03 13 SNES Function norm 8.886826738192e-04 14 SNES Function norm 3.640894847145e-04 15 SNES Function norm 1.491663153414e-04 16 SNES Function norm 6.111303899450e-05 17 SNES Function norm 2.503785968501e-05 18 SNES Function norm 1.025795062417e-05 19 SNES Function norm 4.202657921479e-06 SNES Object: 4 MPI processes type: nasm total subdomain blocks = 4 Local solver information for first block on rank 0: Use -snes_view ::ascii_info_detail to display information for all blocks SNES Object: (sub_) 1 MPI processes type: newtonls maximum iterations=50, maximum function evaluations=10000 tolerances: relative=1e-08, absolute=1e-50, solution=1e-08 total number of linear solver iterations=2 total number of function evaluations=3 norm schedule ALWAYS Jacobian is built using a DMDA local Jacobian SNESLineSearch Object: (sub_) 1 MPI processes type: bt interpolation: cubic alpha=1.000000e-04 maxstep=1.000000e+08, minlambda=1.000000e-12 tolerances: relative=1.000000e-08, absolute=1.000000e-15, lambda=1.000000e-08 maximum iterations=40 KSP Object: (sub_) 1 MPI processes type: preonly maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using NONE norm type for convergence test PC Object: (sub_) 1 MPI processes type: lu out-of-place factorization tolerance for zero pivot 2.22045e-14 matrix ordering: nd factor fill ratio given 5., needed 2.13732 Factored matrix follows: Mat Object: 1 MPI processes type: seqaij rows=169, cols=169 package used to perform factorization: petsc total: nonzeros=13339, allocated nonzeros=13339 using I-node routines: found 104 nodes, limit used is 5 linear system matrix = precond matrix: Mat Object: 1 MPI processes type: seqaij rows=169, cols=169 total: nonzeros=6241, allocated nonzeros=6241 total number of mallocs used during MatSetValues calls=0 not using I-node routines maximum iterations=50, maximum function evaluations=10000 tolerances: relative=1e-08, absolute=1e-50, solution=1e-08 total number of function evaluations=20 norm schedule ALWAYS Jacobian is built using a DMDA local Jacobian problem ex10 on 20 x 20 point 2D grid with d = 3, and eps = 0.082: error |u-uexact|_inf = 2.879e-02, |u-uexact|_h = 1.707e-02 On Thu, May 12, 2022 at 9:37 AM Matthew Knepley wrote: > Your subdomain solves do not appear to be producing descent whatsoever. > Possible reasons: > > 1) Your subdomain Jacobians are wrong (this is usually the problem) > > 2) You have some global coupling field for which local solves give no > descent. (For this you want nonlinear elimination I think) > > Thanks, > > Matt > > On Thu, May 12, 2022 at 9:02 AM Takahashi, Tadanaga wrote: > >> I ran the code with the additional options but the raw output is about >> 75,000 lines. I cannot paste it directly in the email. The output is in the >> attached file. >> >> On Wed, May 11, 2022 at 11:44 PM Jed Brown wrote: >> >>> Can you add -snes_linesearch_monitor -sub_snes_linesearch_monitor >>> -ksp_converged_reason and send the output?? >>> >>> "Takahashi, Tadanaga" writes: >>> >>> > Hello, >>> > >>> > We are working on a finite difference solver for a 2D nonlinear PDE >>> with >>> > Dirichlet Boundary conditions on a rectangular domain. Our goal is to >>> solve >>> > the problem with parallel nonlinear additive Schwarz (NASM) as the >>> outer >>> > solver. Our code is similar to SNES example 5 >>> > . In example >>> 5, >>> > the parallel NASM can be executed with a command like `mpiexec -n 4 >>> ./ex5 >>> > -mms 3 -snes_type nasm -snes_nasm_type restrict -da_overlap 2` which >>> gives >>> > a convergent result. We assume this is the correct usage. A comment in >>> the >>> > source code for NASM mentions that NASM should be a preconditioner but >>> > there's no documentation on the usage. The Brune paper does not cover >>> > parallel NASM either. We observed that increasing the overlap leads to >>> > fewer Schwarz iterations. The parallelization works seamlessly for an >>> > arbitrary number of subdomains. This is the type of behavior we were >>> > expecting from our code. >>> > >>> > Our method uses box-style stencil width d = ceil(N^(1/3)) on a N by N >>> DMDA. >>> > The finite difference stencil consists of 4d+1 points spread out in a >>> > diamond formation. If a stencil point is out of bounds, then it is >>> > projected onto the boundary curve. Since the nodes on the boundary >>> curve >>> > would result in an irregular mesh, we chose not treat boundary nodes as >>> > unknowns as in Example 5. We use DMDACreate2d to create the DA for the >>> > interior points and DMDASNESSetFunctionLocal to associate the residue >>> > function to the SNES object. >>> > >>> > Our code works serially. We have also tested our code >>> > with Newton-Krylov-Schwarz (NKS) by running something akin to `mpiexec >>> -n >>> > ./solve -snes_type newtonls`. We have tested the NKS for several >>> > quantities of subdomains and overlap and the code works as expected. We >>> > have some confidence in the correctness of our code. The overlapping >>> NASM >>> > was implemented in MATLAB so we know the method converges. However, the >>> > parallel NASM will not converge with our PETSc code. We don't >>> understand >>> > why NKS works while NASM does not. The F-norm residue monotonically >>> > decreases and then stagnates. >>> > >>> > Here is an example of the output when attempting to run NASM in >>> parallel: >>> > takahashi at ubuntu:~/Desktop/MA-DDM/Cpp/Rectangle$ mpiexec -n 4 ./test1 >>> -t1_N >>> > 20 -snes_max_it 50 -snes_monitor -snes_view -da_overlap 3 -snes_type >>> nasm >>> > -snes_nasm_type restrict >>> > 0 SNES Function norm 7.244681057908e+02 >>> > 1 SNES Function norm 1.237688062971e+02 >>> > 2 SNES Function norm 1.068926073552e+02 >>> > 3 SNES Function norm 1.027563237834e+02 >>> > 4 SNES Function norm 1.022184806736e+02 >>> > 5 SNES Function norm 1.020818227640e+02 >>> > 6 SNES Function norm 1.020325629121e+02 >>> > 7 SNES Function norm 1.020149036595e+02 >>> > 8 SNES Function norm 1.020088110545e+02 >>> > 9 SNES Function norm 1.020067198030e+02 >>> > 10 SNES Function norm 1.020060034469e+02 >>> > 11 SNES Function norm 1.020057582380e+02 >>> > 12 SNES Function norm 1.020056743241e+02 >>> > 13 SNES Function norm 1.020056456101e+02 >>> > 14 SNES Function norm 1.020056357849e+02 >>> > 15 SNES Function norm 1.020056324231e+02 >>> > 16 SNES Function norm 1.020056312727e+02 >>> > 17 SNES Function norm 1.020056308791e+02 >>> > 18 SNES Function norm 1.020056307444e+02 >>> > 19 SNES Function norm 1.020056306983e+02 >>> > 20 SNES Function norm 1.020056306826e+02 >>> > 21 SNES Function norm 1.020056306772e+02 >>> > 22 SNES Function norm 1.020056306753e+02 >>> > 23 SNES Function norm 1.020056306747e+02 >>> > 24 SNES Function norm 1.020056306745e+02 >>> > 25 SNES Function norm 1.020056306744e+02 >>> > 26 SNES Function norm 1.020056306744e+02 >>> > 27 SNES Function norm 1.020056306744e+02 >>> > 28 SNES Function norm 1.020056306744e+02 >>> > 29 SNES Function norm 1.020056306744e+02 >>> > 30 SNES Function norm 1.020056306744e+02 >>> > 31 SNES Function norm 1.020056306744e+02 >>> > 32 SNES Function norm 1.020056306744e+02 >>> > 33 SNES Function norm 1.020056306744e+02 >>> > 34 SNES Function norm 1.020056306744e+02 >>> > 35 SNES Function norm 1.020056306744e+02 >>> > 36 SNES Function norm 1.020056306744e+02 >>> > 37 SNES Function norm 1.020056306744e+02 >>> > 38 SNES Function norm 1.020056306744e+02 >>> > 39 SNES Function norm 1.020056306744e+02 >>> > 40 SNES Function norm 1.020056306744e+02 >>> > 41 SNES Function norm 1.020056306744e+02 >>> > 42 SNES Function norm 1.020056306744e+02 >>> > 43 SNES Function norm 1.020056306744e+02 >>> > 44 SNES Function norm 1.020056306744e+02 >>> > 45 SNES Function norm 1.020056306744e+02 >>> > 46 SNES Function norm 1.020056306744e+02 >>> > 47 SNES Function norm 1.020056306744e+02 >>> > 48 SNES Function norm 1.020056306744e+02 >>> > 49 SNES Function norm 1.020056306744e+02 >>> > 50 SNES Function norm 1.020056306744e+02 >>> > SNES Object: 4 MPI processes >>> > type: nasm >>> > total subdomain blocks = 4 >>> > Local solver information for first block on rank 0: >>> > Use -snes_view ::ascii_info_detail to display information for all >>> blocks >>> > SNES Object: (sub_) 1 MPI processes >>> > type: newtonls >>> > maximum iterations=50, maximum function evaluations=10000 >>> > tolerances: relative=1e-08, absolute=1e-50, solution=1e-08 >>> > total number of linear solver iterations=22 >>> > total number of function evaluations=40 >>> > norm schedule ALWAYS >>> > Jacobian is built using a DMDA local Jacobian >>> > SNESLineSearch Object: (sub_) 1 MPI processes >>> > type: bt >>> > interpolation: cubic >>> > alpha=1.000000e-04 >>> > maxstep=1.000000e+08, minlambda=1.000000e-12 >>> > tolerances: relative=1.000000e-08, absolute=1.000000e-15, >>> > lambda=1.000000e-08 >>> > maximum iterations=40 >>> > KSP Object: (sub_) 1 MPI processes >>> > type: preonly >>> > maximum iterations=10000, initial guess is zero >>> > tolerances: relative=1e-05, absolute=1e-50, divergence=10000. >>> > left preconditioning >>> > using NONE norm type for convergence test >>> > PC Object: (sub_) 1 MPI processes >>> > type: lu >>> > out-of-place factorization >>> > tolerance for zero pivot 2.22045e-14 >>> > matrix ordering: nd >>> > factor fill ratio given 5., needed 2.13732 >>> > Factored matrix follows: >>> > Mat Object: 1 MPI processes >>> > type: seqaij >>> > rows=169, cols=169 >>> > package used to perform factorization: petsc >>> > total: nonzeros=13339, allocated nonzeros=13339 >>> > using I-node routines: found 104 nodes, limit used >>> is 5 >>> > linear system matrix = precond matrix: >>> > Mat Object: 1 MPI processes >>> > type: seqaij >>> > rows=169, cols=169 >>> > total: nonzeros=6241, allocated nonzeros=6241 >>> > total number of mallocs used during MatSetValues calls=0 >>> > not using I-node routines >>> > maximum iterations=50, maximum function evaluations=10000 >>> > tolerances: relative=1e-08, absolute=1e-50, solution=1e-08 >>> > total number of function evaluations=51 >>> > norm schedule ALWAYS >>> > Jacobian is built using a DMDA local Jacobian >>> > problem ex10 on 20 x 20 point 2D grid with d = 3, and eps = 0.082: >>> > error |u-uexact|_inf = 3.996e-01, |u-uexact|_h = 2.837e-01 >>> > >>> > We have been stuck on this for a while now. We do not know how to debug >>> > this issue. Please let us know if you have any insights. >>> >> > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > > https://www.cse.buffalo.edu/~knepley/ > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Thu May 12 13:02:13 2022 From: knepley at gmail.com (Matthew Knepley) Date: Thu, 12 May 2022 14:02:13 -0400 Subject: [petsc-users] Convergence issues for SNES NASM In-Reply-To: References: <87lev75sar.fsf@jedbrown.org> Message-ID: On Thu, May 12, 2022 at 1:03 PM Takahashi, Tadanaga wrote: > Thank you for the feedback. We figured out what was causing the issue. We > were using DMGetBoundingBox > in order > to get the limits of the global domain, but gmin and gmax contained limits > for the local subdomains when we ran the code with NASM. Hence, our local > coordinates xi and yj were completely wrong. The documentation states > that DMGetBoundingBox gets the global limits. I believe this is a mistake. > I think I can explain this, and maybe you can tell us how to improve the documentation. I believe we make a new DM that comprises only the subdomain. Then the bounding box for this subdomain will only contain itself, not the original domain. Where should we say this? Thanks, Matt > This is our new output: > $ mpiexec -n 4 ./test1 -t1_N 20 -snes_max_it 50 -snes_monitor -snes_view > -da_overlap 3 -snes_type nasm -snes_nasm_type restrict > 0 SNES Function norm 7.244681057908e+02 > 1 SNES Function norm 4.394913250889e+01 > 2 SNES Function norm 1.823326663029e+01 > 3 SNES Function norm 7.033938512358e+00 > 4 SNES Function norm 2.797351504285e+00 > 5 SNES Function norm 1.130613777736e+00 > 6 SNES Function norm 4.605418417192e-01 > 7 SNES Function norm 1.882307001920e-01 > 8 SNES Function norm 7.704148683921e-02 > 9 SNES Function norm 3.155090858782e-02 > 10 SNES Function norm 1.292418188473e-02 > 11 SNES Function norm 5.294645671797e-03 > 12 SNES Function norm 2.169143207557e-03 > 13 SNES Function norm 8.886826738192e-04 > 14 SNES Function norm 3.640894847145e-04 > 15 SNES Function norm 1.491663153414e-04 > 16 SNES Function norm 6.111303899450e-05 > 17 SNES Function norm 2.503785968501e-05 > 18 SNES Function norm 1.025795062417e-05 > 19 SNES Function norm 4.202657921479e-06 > SNES Object: 4 MPI processes > type: nasm > total subdomain blocks = 4 > Local solver information for first block on rank 0: > Use -snes_view ::ascii_info_detail to display information for all > blocks > SNES Object: (sub_) 1 MPI processes > type: newtonls > maximum iterations=50, maximum function evaluations=10000 > tolerances: relative=1e-08, absolute=1e-50, solution=1e-08 > total number of linear solver iterations=2 > total number of function evaluations=3 > norm schedule ALWAYS > Jacobian is built using a DMDA local Jacobian > SNESLineSearch Object: (sub_) 1 MPI processes > type: bt > interpolation: cubic > alpha=1.000000e-04 > maxstep=1.000000e+08, minlambda=1.000000e-12 > tolerances: relative=1.000000e-08, absolute=1.000000e-15, > lambda=1.000000e-08 > maximum iterations=40 > KSP Object: (sub_) 1 MPI processes > type: preonly > maximum iterations=10000, initial guess is zero > tolerances: relative=1e-05, absolute=1e-50, divergence=10000. > left preconditioning > using NONE norm type for convergence test > PC Object: (sub_) 1 MPI processes > type: lu > out-of-place factorization > tolerance for zero pivot 2.22045e-14 > matrix ordering: nd > factor fill ratio given 5., needed 2.13732 > Factored matrix follows: > Mat Object: 1 MPI processes > type: seqaij > rows=169, cols=169 > package used to perform factorization: petsc > total: nonzeros=13339, allocated nonzeros=13339 > using I-node routines: found 104 nodes, limit used is 5 > linear system matrix = precond matrix: > Mat Object: 1 MPI processes > type: seqaij > rows=169, cols=169 > total: nonzeros=6241, allocated nonzeros=6241 > total number of mallocs used during MatSetValues calls=0 > not using I-node routines > maximum iterations=50, maximum function evaluations=10000 > tolerances: relative=1e-08, absolute=1e-50, solution=1e-08 > total number of function evaluations=20 > norm schedule ALWAYS > Jacobian is built using a DMDA local Jacobian > problem ex10 on 20 x 20 point 2D grid with d = 3, and eps = 0.082: > error |u-uexact|_inf = 2.879e-02, |u-uexact|_h = 1.707e-02 > > On Thu, May 12, 2022 at 9:37 AM Matthew Knepley wrote: > >> Your subdomain solves do not appear to be producing descent whatsoever. >> Possible reasons: >> >> 1) Your subdomain Jacobians are wrong (this is usually the problem) >> >> 2) You have some global coupling field for which local solves give no >> descent. (For this you want nonlinear elimination I think) >> >> Thanks, >> >> Matt >> >> On Thu, May 12, 2022 at 9:02 AM Takahashi, Tadanaga >> wrote: >> >>> I ran the code with the additional options but the raw output is about >>> 75,000 lines. I cannot paste it directly in the email. The output is in the >>> attached file. >>> >>> On Wed, May 11, 2022 at 11:44 PM Jed Brown wrote: >>> >>>> Can you add -snes_linesearch_monitor -sub_snes_linesearch_monitor >>>> -ksp_converged_reason and send the output?? >>>> >>>> "Takahashi, Tadanaga" writes: >>>> >>>> > Hello, >>>> > >>>> > We are working on a finite difference solver for a 2D nonlinear PDE >>>> with >>>> > Dirichlet Boundary conditions on a rectangular domain. Our goal is to >>>> solve >>>> > the problem with parallel nonlinear additive Schwarz (NASM) as the >>>> outer >>>> > solver. Our code is similar to SNES example 5 >>>> > . In >>>> example 5, >>>> > the parallel NASM can be executed with a command like `mpiexec -n 4 >>>> ./ex5 >>>> > -mms 3 -snes_type nasm -snes_nasm_type restrict -da_overlap 2` which >>>> gives >>>> > a convergent result. We assume this is the correct usage. A comment >>>> in the >>>> > source code for NASM mentions that NASM should be a preconditioner but >>>> > there's no documentation on the usage. The Brune paper does not cover >>>> > parallel NASM either. We observed that increasing the overlap leads to >>>> > fewer Schwarz iterations. The parallelization works seamlessly for an >>>> > arbitrary number of subdomains. This is the type of behavior we were >>>> > expecting from our code. >>>> > >>>> > Our method uses box-style stencil width d = ceil(N^(1/3)) on a N by N >>>> DMDA. >>>> > The finite difference stencil consists of 4d+1 points spread out in a >>>> > diamond formation. If a stencil point is out of bounds, then it is >>>> > projected onto the boundary curve. Since the nodes on the boundary >>>> curve >>>> > would result in an irregular mesh, we chose not treat boundary nodes >>>> as >>>> > unknowns as in Example 5. We use DMDACreate2d to create the DA for the >>>> > interior points and DMDASNESSetFunctionLocal to associate the residue >>>> > function to the SNES object. >>>> > >>>> > Our code works serially. We have also tested our code >>>> > with Newton-Krylov-Schwarz (NKS) by running something akin to >>>> `mpiexec -n >>>> > ./solve -snes_type newtonls`. We have tested the NKS for several >>>> > quantities of subdomains and overlap and the code works as expected. >>>> We >>>> > have some confidence in the correctness of our code. The overlapping >>>> NASM >>>> > was implemented in MATLAB so we know the method converges. However, >>>> the >>>> > parallel NASM will not converge with our PETSc code. We don't >>>> understand >>>> > why NKS works while NASM does not. The F-norm residue monotonically >>>> > decreases and then stagnates. >>>> > >>>> > Here is an example of the output when attempting to run NASM in >>>> parallel: >>>> > takahashi at ubuntu:~/Desktop/MA-DDM/Cpp/Rectangle$ mpiexec -n 4 >>>> ./test1 -t1_N >>>> > 20 -snes_max_it 50 -snes_monitor -snes_view -da_overlap 3 -snes_type >>>> nasm >>>> > -snes_nasm_type restrict >>>> > 0 SNES Function norm 7.244681057908e+02 >>>> > 1 SNES Function norm 1.237688062971e+02 >>>> > 2 SNES Function norm 1.068926073552e+02 >>>> > 3 SNES Function norm 1.027563237834e+02 >>>> > 4 SNES Function norm 1.022184806736e+02 >>>> > 5 SNES Function norm 1.020818227640e+02 >>>> > 6 SNES Function norm 1.020325629121e+02 >>>> > 7 SNES Function norm 1.020149036595e+02 >>>> > 8 SNES Function norm 1.020088110545e+02 >>>> > 9 SNES Function norm 1.020067198030e+02 >>>> > 10 SNES Function norm 1.020060034469e+02 >>>> > 11 SNES Function norm 1.020057582380e+02 >>>> > 12 SNES Function norm 1.020056743241e+02 >>>> > 13 SNES Function norm 1.020056456101e+02 >>>> > 14 SNES Function norm 1.020056357849e+02 >>>> > 15 SNES Function norm 1.020056324231e+02 >>>> > 16 SNES Function norm 1.020056312727e+02 >>>> > 17 SNES Function norm 1.020056308791e+02 >>>> > 18 SNES Function norm 1.020056307444e+02 >>>> > 19 SNES Function norm 1.020056306983e+02 >>>> > 20 SNES Function norm 1.020056306826e+02 >>>> > 21 SNES Function norm 1.020056306772e+02 >>>> > 22 SNES Function norm 1.020056306753e+02 >>>> > 23 SNES Function norm 1.020056306747e+02 >>>> > 24 SNES Function norm 1.020056306745e+02 >>>> > 25 SNES Function norm 1.020056306744e+02 >>>> > 26 SNES Function norm 1.020056306744e+02 >>>> > 27 SNES Function norm 1.020056306744e+02 >>>> > 28 SNES Function norm 1.020056306744e+02 >>>> > 29 SNES Function norm 1.020056306744e+02 >>>> > 30 SNES Function norm 1.020056306744e+02 >>>> > 31 SNES Function norm 1.020056306744e+02 >>>> > 32 SNES Function norm 1.020056306744e+02 >>>> > 33 SNES Function norm 1.020056306744e+02 >>>> > 34 SNES Function norm 1.020056306744e+02 >>>> > 35 SNES Function norm 1.020056306744e+02 >>>> > 36 SNES Function norm 1.020056306744e+02 >>>> > 37 SNES Function norm 1.020056306744e+02 >>>> > 38 SNES Function norm 1.020056306744e+02 >>>> > 39 SNES Function norm 1.020056306744e+02 >>>> > 40 SNES Function norm 1.020056306744e+02 >>>> > 41 SNES Function norm 1.020056306744e+02 >>>> > 42 SNES Function norm 1.020056306744e+02 >>>> > 43 SNES Function norm 1.020056306744e+02 >>>> > 44 SNES Function norm 1.020056306744e+02 >>>> > 45 SNES Function norm 1.020056306744e+02 >>>> > 46 SNES Function norm 1.020056306744e+02 >>>> > 47 SNES Function norm 1.020056306744e+02 >>>> > 48 SNES Function norm 1.020056306744e+02 >>>> > 49 SNES Function norm 1.020056306744e+02 >>>> > 50 SNES Function norm 1.020056306744e+02 >>>> > SNES Object: 4 MPI processes >>>> > type: nasm >>>> > total subdomain blocks = 4 >>>> > Local solver information for first block on rank 0: >>>> > Use -snes_view ::ascii_info_detail to display information for all >>>> blocks >>>> > SNES Object: (sub_) 1 MPI processes >>>> > type: newtonls >>>> > maximum iterations=50, maximum function evaluations=10000 >>>> > tolerances: relative=1e-08, absolute=1e-50, solution=1e-08 >>>> > total number of linear solver iterations=22 >>>> > total number of function evaluations=40 >>>> > norm schedule ALWAYS >>>> > Jacobian is built using a DMDA local Jacobian >>>> > SNESLineSearch Object: (sub_) 1 MPI processes >>>> > type: bt >>>> > interpolation: cubic >>>> > alpha=1.000000e-04 >>>> > maxstep=1.000000e+08, minlambda=1.000000e-12 >>>> > tolerances: relative=1.000000e-08, absolute=1.000000e-15, >>>> > lambda=1.000000e-08 >>>> > maximum iterations=40 >>>> > KSP Object: (sub_) 1 MPI processes >>>> > type: preonly >>>> > maximum iterations=10000, initial guess is zero >>>> > tolerances: relative=1e-05, absolute=1e-50, divergence=10000. >>>> > left preconditioning >>>> > using NONE norm type for convergence test >>>> > PC Object: (sub_) 1 MPI processes >>>> > type: lu >>>> > out-of-place factorization >>>> > tolerance for zero pivot 2.22045e-14 >>>> > matrix ordering: nd >>>> > factor fill ratio given 5., needed 2.13732 >>>> > Factored matrix follows: >>>> > Mat Object: 1 MPI processes >>>> > type: seqaij >>>> > rows=169, cols=169 >>>> > package used to perform factorization: petsc >>>> > total: nonzeros=13339, allocated nonzeros=13339 >>>> > using I-node routines: found 104 nodes, limit used >>>> is 5 >>>> > linear system matrix = precond matrix: >>>> > Mat Object: 1 MPI processes >>>> > type: seqaij >>>> > rows=169, cols=169 >>>> > total: nonzeros=6241, allocated nonzeros=6241 >>>> > total number of mallocs used during MatSetValues calls=0 >>>> > not using I-node routines >>>> > maximum iterations=50, maximum function evaluations=10000 >>>> > tolerances: relative=1e-08, absolute=1e-50, solution=1e-08 >>>> > total number of function evaluations=51 >>>> > norm schedule ALWAYS >>>> > Jacobian is built using a DMDA local Jacobian >>>> > problem ex10 on 20 x 20 point 2D grid with d = 3, and eps = 0.082: >>>> > error |u-uexact|_inf = 3.996e-01, |u-uexact|_h = 2.837e-01 >>>> > >>>> > We have been stuck on this for a while now. We do not know how to >>>> debug >>>> > this issue. Please let us know if you have any insights. >>>> >>> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which their >> experiments lead. >> -- Norbert Wiener >> >> https://www.cse.buffalo.edu/~knepley/ >> >> > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From tt73 at njit.edu Thu May 12 15:03:30 2022 From: tt73 at njit.edu (Takahashi, Tadanaga) Date: Thu, 12 May 2022 16:03:30 -0400 Subject: [petsc-users] Convergence issues for SNES NASM In-Reply-To: References: <87lev75sar.fsf@jedbrown.org> Message-ID: I'm still relatively new to PETSc. I was using DMDASetUniformCoordinates and DMGetBoundingBox together. In hindsight, it was a very unnecessary thing to do. I think the simplest way to prevent anyone else from making the same mistake is to add a caveat to the DMGetBoundingBox documentation page. On Thu, May 12, 2022 at 2:02 PM Matthew Knepley wrote: > On Thu, May 12, 2022 at 1:03 PM Takahashi, Tadanaga wrote: > >> Thank you for the feedback. We figured out what was causing the issue. We >> were using DMGetBoundingBox >> in order >> to get the limits of the global domain, but gmin and gmax contained limits >> for the local subdomains when we ran the code with NASM. Hence, our local >> coordinates xi and yj were completely wrong. The documentation states >> that DMGetBoundingBox gets the global limits. I believe this is a mistake. >> > > I think I can explain this, and maybe you can tell us how to improve the > documentation. > > I believe we make a new DM that comprises only the subdomain. Then the > bounding box for this subdomain will only contain itself, not the original > domain. > Where should we say this? > > Thanks, > > Matt > > >> This is our new output: >> $ mpiexec -n 4 ./test1 -t1_N 20 -snes_max_it 50 -snes_monitor -snes_view >> -da_overlap 3 -snes_type nasm -snes_nasm_type restrict >> 0 SNES Function norm 7.244681057908e+02 >> 1 SNES Function norm 4.394913250889e+01 >> 2 SNES Function norm 1.823326663029e+01 >> 3 SNES Function norm 7.033938512358e+00 >> 4 SNES Function norm 2.797351504285e+00 >> 5 SNES Function norm 1.130613777736e+00 >> 6 SNES Function norm 4.605418417192e-01 >> 7 SNES Function norm 1.882307001920e-01 >> 8 SNES Function norm 7.704148683921e-02 >> 9 SNES Function norm 3.155090858782e-02 >> 10 SNES Function norm 1.292418188473e-02 >> 11 SNES Function norm 5.294645671797e-03 >> 12 SNES Function norm 2.169143207557e-03 >> 13 SNES Function norm 8.886826738192e-04 >> 14 SNES Function norm 3.640894847145e-04 >> 15 SNES Function norm 1.491663153414e-04 >> 16 SNES Function norm 6.111303899450e-05 >> 17 SNES Function norm 2.503785968501e-05 >> 18 SNES Function norm 1.025795062417e-05 >> 19 SNES Function norm 4.202657921479e-06 >> SNES Object: 4 MPI processes >> type: nasm >> total subdomain blocks = 4 >> Local solver information for first block on rank 0: >> Use -snes_view ::ascii_info_detail to display information for all >> blocks >> SNES Object: (sub_) 1 MPI processes >> type: newtonls >> maximum iterations=50, maximum function evaluations=10000 >> tolerances: relative=1e-08, absolute=1e-50, solution=1e-08 >> total number of linear solver iterations=2 >> total number of function evaluations=3 >> norm schedule ALWAYS >> Jacobian is built using a DMDA local Jacobian >> SNESLineSearch Object: (sub_) 1 MPI processes >> type: bt >> interpolation: cubic >> alpha=1.000000e-04 >> maxstep=1.000000e+08, minlambda=1.000000e-12 >> tolerances: relative=1.000000e-08, absolute=1.000000e-15, >> lambda=1.000000e-08 >> maximum iterations=40 >> KSP Object: (sub_) 1 MPI processes >> type: preonly >> maximum iterations=10000, initial guess is zero >> tolerances: relative=1e-05, absolute=1e-50, divergence=10000. >> left preconditioning >> using NONE norm type for convergence test >> PC Object: (sub_) 1 MPI processes >> type: lu >> out-of-place factorization >> tolerance for zero pivot 2.22045e-14 >> matrix ordering: nd >> factor fill ratio given 5., needed 2.13732 >> Factored matrix follows: >> Mat Object: 1 MPI processes >> type: seqaij >> rows=169, cols=169 >> package used to perform factorization: petsc >> total: nonzeros=13339, allocated nonzeros=13339 >> using I-node routines: found 104 nodes, limit used is 5 >> linear system matrix = precond matrix: >> Mat Object: 1 MPI processes >> type: seqaij >> rows=169, cols=169 >> total: nonzeros=6241, allocated nonzeros=6241 >> total number of mallocs used during MatSetValues calls=0 >> not using I-node routines >> maximum iterations=50, maximum function evaluations=10000 >> tolerances: relative=1e-08, absolute=1e-50, solution=1e-08 >> total number of function evaluations=20 >> norm schedule ALWAYS >> Jacobian is built using a DMDA local Jacobian >> problem ex10 on 20 x 20 point 2D grid with d = 3, and eps = 0.082: >> error |u-uexact|_inf = 2.879e-02, |u-uexact|_h = 1.707e-02 >> >> On Thu, May 12, 2022 at 9:37 AM Matthew Knepley >> wrote: >> >>> Your subdomain solves do not appear to be producing descent whatsoever. >>> Possible reasons: >>> >>> 1) Your subdomain Jacobians are wrong (this is usually the problem) >>> >>> 2) You have some global coupling field for which local solves give no >>> descent. (For this you want nonlinear elimination I think) >>> >>> Thanks, >>> >>> Matt >>> >>> On Thu, May 12, 2022 at 9:02 AM Takahashi, Tadanaga >>> wrote: >>> >>>> I ran the code with the additional options but the raw output is about >>>> 75,000 lines. I cannot paste it directly in the email. The output is in the >>>> attached file. >>>> >>>> On Wed, May 11, 2022 at 11:44 PM Jed Brown wrote: >>>> >>>>> Can you add -snes_linesearch_monitor -sub_snes_linesearch_monitor >>>>> -ksp_converged_reason and send the output?? >>>>> >>>>> "Takahashi, Tadanaga" writes: >>>>> >>>>> > Hello, >>>>> > >>>>> > We are working on a finite difference solver for a 2D nonlinear PDE >>>>> with >>>>> > Dirichlet Boundary conditions on a rectangular domain. Our goal is >>>>> to solve >>>>> > the problem with parallel nonlinear additive Schwarz (NASM) as the >>>>> outer >>>>> > solver. Our code is similar to SNES example 5 >>>>> > . In >>>>> example 5, >>>>> > the parallel NASM can be executed with a command like `mpiexec -n 4 >>>>> ./ex5 >>>>> > -mms 3 -snes_type nasm -snes_nasm_type restrict -da_overlap 2` which >>>>> gives >>>>> > a convergent result. We assume this is the correct usage. A comment >>>>> in the >>>>> > source code for NASM mentions that NASM should be a preconditioner >>>>> but >>>>> > there's no documentation on the usage. The Brune paper does not cover >>>>> > parallel NASM either. We observed that increasing the overlap leads >>>>> to >>>>> > fewer Schwarz iterations. The parallelization works seamlessly for an >>>>> > arbitrary number of subdomains. This is the type of behavior we were >>>>> > expecting from our code. >>>>> > >>>>> > Our method uses box-style stencil width d = ceil(N^(1/3)) on a N by >>>>> N DMDA. >>>>> > The finite difference stencil consists of 4d+1 points spread out in a >>>>> > diamond formation. If a stencil point is out of bounds, then it is >>>>> > projected onto the boundary curve. Since the nodes on the boundary >>>>> curve >>>>> > would result in an irregular mesh, we chose not treat boundary nodes >>>>> as >>>>> > unknowns as in Example 5. We use DMDACreate2d to create the DA for >>>>> the >>>>> > interior points and DMDASNESSetFunctionLocal to associate the residue >>>>> > function to the SNES object. >>>>> > >>>>> > Our code works serially. We have also tested our code >>>>> > with Newton-Krylov-Schwarz (NKS) by running something akin to >>>>> `mpiexec -n >>>>> > ./solve -snes_type newtonls`. We have tested the NKS for several >>>>> > quantities of subdomains and overlap and the code works as expected. >>>>> We >>>>> > have some confidence in the correctness of our code. The overlapping >>>>> NASM >>>>> > was implemented in MATLAB so we know the method converges. However, >>>>> the >>>>> > parallel NASM will not converge with our PETSc code. We don't >>>>> understand >>>>> > why NKS works while NASM does not. The F-norm residue monotonically >>>>> > decreases and then stagnates. >>>>> > >>>>> > Here is an example of the output when attempting to run NASM in >>>>> parallel: >>>>> > takahashi at ubuntu:~/Desktop/MA-DDM/Cpp/Rectangle$ mpiexec -n 4 >>>>> ./test1 -t1_N >>>>> > 20 -snes_max_it 50 -snes_monitor -snes_view -da_overlap 3 -snes_type >>>>> nasm >>>>> > -snes_nasm_type restrict >>>>> > 0 SNES Function norm 7.244681057908e+02 >>>>> > 1 SNES Function norm 1.237688062971e+02 >>>>> > 2 SNES Function norm 1.068926073552e+02 >>>>> > 3 SNES Function norm 1.027563237834e+02 >>>>> > 4 SNES Function norm 1.022184806736e+02 >>>>> > 5 SNES Function norm 1.020818227640e+02 >>>>> > 6 SNES Function norm 1.020325629121e+02 >>>>> > 7 SNES Function norm 1.020149036595e+02 >>>>> > 8 SNES Function norm 1.020088110545e+02 >>>>> > 9 SNES Function norm 1.020067198030e+02 >>>>> > 10 SNES Function norm 1.020060034469e+02 >>>>> > 11 SNES Function norm 1.020057582380e+02 >>>>> > 12 SNES Function norm 1.020056743241e+02 >>>>> > 13 SNES Function norm 1.020056456101e+02 >>>>> > 14 SNES Function norm 1.020056357849e+02 >>>>> > 15 SNES Function norm 1.020056324231e+02 >>>>> > 16 SNES Function norm 1.020056312727e+02 >>>>> > 17 SNES Function norm 1.020056308791e+02 >>>>> > 18 SNES Function norm 1.020056307444e+02 >>>>> > 19 SNES Function norm 1.020056306983e+02 >>>>> > 20 SNES Function norm 1.020056306826e+02 >>>>> > 21 SNES Function norm 1.020056306772e+02 >>>>> > 22 SNES Function norm 1.020056306753e+02 >>>>> > 23 SNES Function norm 1.020056306747e+02 >>>>> > 24 SNES Function norm 1.020056306745e+02 >>>>> > 25 SNES Function norm 1.020056306744e+02 >>>>> > 26 SNES Function norm 1.020056306744e+02 >>>>> > 27 SNES Function norm 1.020056306744e+02 >>>>> > 28 SNES Function norm 1.020056306744e+02 >>>>> > 29 SNES Function norm 1.020056306744e+02 >>>>> > 30 SNES Function norm 1.020056306744e+02 >>>>> > 31 SNES Function norm 1.020056306744e+02 >>>>> > 32 SNES Function norm 1.020056306744e+02 >>>>> > 33 SNES Function norm 1.020056306744e+02 >>>>> > 34 SNES Function norm 1.020056306744e+02 >>>>> > 35 SNES Function norm 1.020056306744e+02 >>>>> > 36 SNES Function norm 1.020056306744e+02 >>>>> > 37 SNES Function norm 1.020056306744e+02 >>>>> > 38 SNES Function norm 1.020056306744e+02 >>>>> > 39 SNES Function norm 1.020056306744e+02 >>>>> > 40 SNES Function norm 1.020056306744e+02 >>>>> > 41 SNES Function norm 1.020056306744e+02 >>>>> > 42 SNES Function norm 1.020056306744e+02 >>>>> > 43 SNES Function norm 1.020056306744e+02 >>>>> > 44 SNES Function norm 1.020056306744e+02 >>>>> > 45 SNES Function norm 1.020056306744e+02 >>>>> > 46 SNES Function norm 1.020056306744e+02 >>>>> > 47 SNES Function norm 1.020056306744e+02 >>>>> > 48 SNES Function norm 1.020056306744e+02 >>>>> > 49 SNES Function norm 1.020056306744e+02 >>>>> > 50 SNES Function norm 1.020056306744e+02 >>>>> > SNES Object: 4 MPI processes >>>>> > type: nasm >>>>> > total subdomain blocks = 4 >>>>> > Local solver information for first block on rank 0: >>>>> > Use -snes_view ::ascii_info_detail to display information for >>>>> all blocks >>>>> > SNES Object: (sub_) 1 MPI processes >>>>> > type: newtonls >>>>> > maximum iterations=50, maximum function evaluations=10000 >>>>> > tolerances: relative=1e-08, absolute=1e-50, solution=1e-08 >>>>> > total number of linear solver iterations=22 >>>>> > total number of function evaluations=40 >>>>> > norm schedule ALWAYS >>>>> > Jacobian is built using a DMDA local Jacobian >>>>> > SNESLineSearch Object: (sub_) 1 MPI processes >>>>> > type: bt >>>>> > interpolation: cubic >>>>> > alpha=1.000000e-04 >>>>> > maxstep=1.000000e+08, minlambda=1.000000e-12 >>>>> > tolerances: relative=1.000000e-08, absolute=1.000000e-15, >>>>> > lambda=1.000000e-08 >>>>> > maximum iterations=40 >>>>> > KSP Object: (sub_) 1 MPI processes >>>>> > type: preonly >>>>> > maximum iterations=10000, initial guess is zero >>>>> > tolerances: relative=1e-05, absolute=1e-50, >>>>> divergence=10000. >>>>> > left preconditioning >>>>> > using NONE norm type for convergence test >>>>> > PC Object: (sub_) 1 MPI processes >>>>> > type: lu >>>>> > out-of-place factorization >>>>> > tolerance for zero pivot 2.22045e-14 >>>>> > matrix ordering: nd >>>>> > factor fill ratio given 5., needed 2.13732 >>>>> > Factored matrix follows: >>>>> > Mat Object: 1 MPI processes >>>>> > type: seqaij >>>>> > rows=169, cols=169 >>>>> > package used to perform factorization: petsc >>>>> > total: nonzeros=13339, allocated nonzeros=13339 >>>>> > using I-node routines: found 104 nodes, limit used >>>>> is 5 >>>>> > linear system matrix = precond matrix: >>>>> > Mat Object: 1 MPI processes >>>>> > type: seqaij >>>>> > rows=169, cols=169 >>>>> > total: nonzeros=6241, allocated nonzeros=6241 >>>>> > total number of mallocs used during MatSetValues calls=0 >>>>> > not using I-node routines >>>>> > maximum iterations=50, maximum function evaluations=10000 >>>>> > tolerances: relative=1e-08, absolute=1e-50, solution=1e-08 >>>>> > total number of function evaluations=51 >>>>> > norm schedule ALWAYS >>>>> > Jacobian is built using a DMDA local Jacobian >>>>> > problem ex10 on 20 x 20 point 2D grid with d = 3, and eps = 0.082: >>>>> > error |u-uexact|_inf = 3.996e-01, |u-uexact|_h = 2.837e-01 >>>>> > >>>>> > We have been stuck on this for a while now. We do not know how to >>>>> debug >>>>> > this issue. Please let us know if you have any insights. >>>>> >>>> >>> >>> -- >>> What most experimenters take for granted before they begin their >>> experiments is infinitely more interesting than any results to which their >>> experiments lead. >>> -- Norbert Wiener >>> >>> https://www.cse.buffalo.edu/~knepley/ >>> >>> >> > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > > https://www.cse.buffalo.edu/~knepley/ > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From patrick.sanan at gmail.com Fri May 13 02:04:12 2022 From: patrick.sanan at gmail.com (Patrick Sanan) Date: Fri, 13 May 2022 09:04:12 +0200 Subject: [petsc-users] MatColoring In-Reply-To: References: <48152f97d9814c7cbee756147d847458@lanl.gov> <651F493C-2595-4C1E-8F84-968A19176185@petsc.dev> <7aa2305cf42d4d88b7e5236602cdc9a6@lanl.gov> Message-ID: Am Mi., 11. Mai 2022 um 21:28 Uhr schrieb Barry Smith : > > It is unlikely that the true number of colors is much less than 230 so I > don't think you can save by reducing the number of colors more. > > Are you using IS_COLORING_LOCAL it eliminates much parallel > communication so you would definitely benefit by using this, depending on > how your FormFunction is written you may need to refactor your code > slightly. > > There is also a hidden parameter which is a tradeoff of memory usage vs > speed. If memory is not an issue for you then you could change this > parameter. It is computed in MatFDColoringCreate_MPIXAIJ() called bcols You > can just hardware bcols to iscoloring->n This may help a bit. > > The only way to drastically improve the time after the above is to > reorganize the PETSc code and your code to evaluate multiple function > evaluations "at the same time" but this is a major project (it has to be > done for GPUs). > > MATCOLORINGMIS doesn't exist, it is a typo in the docs. > https://gitlab.com/petsc/petsc/-/merge_requests/5249 > > Barry > > > On May 11, 2022, at 2:25 PM, Jorti, Zakariae wrote: > > Hello, > > I have used the -mat_coloring_view and -mat_fd_coloring_view flags as you > suggested and it turns out that the coloring used was MATCOLORINGSL. > I have also tried other methods and obtained the following numbers of > function evaluations per Jacobian building: > - MATCOLORINGJP : 593 > - MATCOLORINGID : 327 > - MATCOLORINGSL : 321 > - MATCOLORINGGREEDY : 316 > - MATCOLORINGLF : 230 > > Is there any parameter/trick we can try to further cut down the number of > function evaluations? And also how could we use MATCOLORINGMIS as this > latter does not appear in the MatColoringType options? > Many thanks. > > Zakariae > ------------------------------ > *From:* Tang, Qi > *Sent:* Tuesday, May 10, 2022 10:51:07 AM > *To:* Barry Smith; petsc-users at mcs.anl.gov > *Cc:* Jorti, Zakariae; Tang, Xianzhu > *Subject:* [EXTERNAL] Re: [petsc-users] MatColoring > > We are using SNES + TS + dmstag. The current bottleneck is the number of > residual evaluation (more than 300 per Jacobian building using the default > coloring from dmstag). We talked to Patrick and we are not sure how to > improve further. > > So it looks like we should play with mat_coloring_type and see if others > give us better performance. > > If there is anything else we can play with, please also let us know. We > also lag Jacobian and only build once every three Newton iterations, which > works well. Thanks, > > Qi > > > On May 10, 2022, at 10:35 AM, Barry Smith wrote: > > > This depends to some degree on how you are accessing applying the > Jacobian construction process. > > If you are using SNES or TS then the SNES object handles most of the > work of organizing the PETSc objects needed to compute the Jacobian and you > can control the choices via the options database. > > With SNES/TS using a DM you can skip calling SNESSetJacobian() and > simply use the option -snes_fd_color to have SNES compute the Jacobian for > you. By default, the coloring is obtained from the DM (which is generally > the best available coloring), you can have it use a coloring computed > directly from the matrix structure with the > options -snes_fd_color_use_mat -mat_coloring_type jp power sl lf ld or > greedy (see MatColoringType, MatColoringSetFromOptions) > Use -mat_fd_coloring_view to get information on the computation of the > Jacobian from the coloring. Use -mat_coloring_view to get information on > the coloring used. (see MatFDColoringSetFromOptions) > > If you are using SNES/TS but not using a DM you need to compute the > nonzero structure of the matrix yourself into J and call > SNESSetJacobian(snes,J,J,SNESComputeJacobianDefaultColor,matfdcoloring); > you need to create the matfdcoloring object with MatFDColoringCreate() > using an iscoloring you obtained with MatColoringCreate() etc. The same > command line arguments as above allow you to control the coloring algorithm > used and to view them etc. > > Barry > > > > > > > > > On May 10, 2022, at 11:40 AM, Jorti, Zakariae via petsc-users < > petsc-users at mcs.anl.gov> wrote: > > Hi, > > I am solving a non-linear problem and using a finite difference > approximation with coloring to compute the Jacobian matrix. > > There are several coloring algorithms available in PETSc as indicated here: > https://petsc.org/release/docs/manualpages/Mat/MatColoring.html > > > And I was wondering how to switch from one to another in the Jacobian > setup routine and also how to check which coloring algorithm I am currently > using. > > Thank you. > > Zakariae Jorti > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rolekst095 at gmail.com Sun May 15 07:39:08 2022 From: rolekst095 at gmail.com (=?UTF-8?B?0JzQuNGI0LAg0KHQsNC70YzQvdC40LrQvtCy?=) Date: Sun, 15 May 2022 15:39:08 +0300 Subject: [petsc-users] Matrix free generalized eigenvalue problem Message-ID: Hi, I am developing a program for solving generalized eigenvalue problem Ax = k Bx when both operators A and B are matrix-free. I found some examples on SLEPc documentation but and there was and example where problem Ax = k x are solving matrix free. So I tried to change this example a little bit and added operator B as identity matrix-free operator. But for now I am getting an error. My source code is attached. And the error message is below Can you please explain to me what I am doing wrong? [0]PETSC ERROR: --------------------- Error Message -------------------------------------------------------------- [0]PETSC ERROR: See https://petsc.org/release/overview/linear_solve_table/ for possible LU and Cholesky solvers [0]PETSC ERROR: Could not locate a solver type for factorization type LU and matrix type shell. [0]PETSC ERROR: See https://petsc.org/release/faq/ for trouble shooting. [0]PETSC ERROR: Petsc Release Version 3.15.5, Sep 29, 2021 [0]PETSC ERROR: ./exe_eps_impl_matr on a named LAPTOP-7DN2DH7N by misha Sun May 15 15:21:59 2022 [0]PETSC ERROR: Configure options --build=x86_64-linux-gnu --prefix=/usr --includedir=${prefix}/include --mandir=${prefix}/share/man --infodir=${prefix}/share/info --sysconfdir=/etc --localstatedir=/var --with-option-checking=0 --with-silent-rules=0 --libdir=${prefix}/lib/x86_64-linux-gnu --runstatedir=/run --with-maintainer-mode=0 --with-dependency-tracking=0 --with-64-bit-indices --with-debugging=0 --shared-library-extension=64_real --with-shared-libraries --with-pic=1 --with-cc=mpicc --with-cxx=mpicxx --with-fc=mpif90 --with-cxx-dialect=C++11 --with-opencl=1 --with-blas-lib=-lblas --with-lapack-lib=-llapack --with-scalapack=1 --with-scalapack-lib=-lscalapack-openmpi --with-ptscotch=1 --with-ptscotch-include=/usr/include/scotch --with-ptscotch-lib="-lptesmumps -lptscotch -lptscotcherr" --with-fftw=1 --with-fftw-include="[]" --with-fftw-lib="-lfftw3 -lfftw3_mpi" --with-yaml=1 --with-hdf5-include=/usr/include/hdf5/openmpi --with-hdf5-lib="-L/usr/lib/x86_64-linux-gnu/hdf5/openmpi -L/usr/lib/x86_64-linux-gnu/openmpi/lib -lhdf5 -lmpi" --CXX_LINKER_FLAGS=-Wl,--no-as-needed --with-hypre=1 --with-hypre-include=/usr/include/hypre64m --with-hypre-lib=-lHYPRE64m --with-mumps=1 --with-mumps-include="[]" --with-mumps-lib="-ldmumps_64 -lzmumps_64 -lsmumps_64 -lcmumps_64 -lmumps_common_64 -lpord_64" --with-suitesparse=1 --with-suitesparse-include=/usr/include/suitesparse --with-suitesparse-lib="-lumfpack -lamd -lcholmod -lklu" --with-zoltan=1 --with-zoltan-include=/usr/include/trilinos --with-zoltan-lib=-ltrilinos_zoltan --prefix=/usr/lib/petscdir/petsc64-3.15/x86_64-linux-gnu-real --PETSC_ARCH=x86_64-linux-gnu-real-64 CFLAGS="-g -O2 -ffile-prefix-map=/build/petsc-tFq6Xk/petsc-3.15.5+dfsg1=. -flto=auto -ffat-lto-objects -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -fPIC" CXXFLAGS="-g -O2 -ffile-prefix-map=/build/petsc-tFq6Xk/petsc-3.15.5+dfsg1=. -flto=auto -ffat-lto-objects -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -fPIC" FCFLAGS="-g -O2 -ffile-prefix-map=/build/petsc-tFq6Xk/petsc-3.15.5+dfsg1=. -flto=auto -ffat-lto-objects -flto=auto -ffat-lto-objects -fstack-protector-strong -fPIC -ffree-line-length-0" FFLAGS="-g -O2 -ffile-prefix-map=/build/petsc-tFq6Xk/petsc-3.15.5+dfsg1=. -flto=auto -ffat-lto-objects -flto=auto -ffat-lto-objects -fstack-protector-strong -fPIC -ffree-line-length-0" CPPFLAGS="-Wdate-time -D_FORTIFY_SOURCE=2" LDFLAGS="-Wl,-Bsymbolic-functions -flto=auto -ffat-lto-objects -flto=auto -Wl,-z,relro -fPIC" MAKEFLAGS=w [0]PETSC ERROR: #1 MatGetFactor() at ./src/mat/interface/matrix.c:4756 [0]PETSC ERROR: #2 PCSetUp_LU() at ./src/ksp/pc/impls/factor/lu/lu.c:82 [0]PETSC ERROR: #3 PCSetUp() at ./src/ksp/pc/interface/precon.c:1017 [0]PETSC ERROR: #4 KSPSetUp() at ./src/ksp/ksp/interface/itfunc.c:406 [0]PETSC ERROR: #5 STSetUp_Shift() at ./src/sys/classes/st/impls/shift/shift.c:107 [0]PETSC ERROR: #6 STSetUp() at ./src/sys/classes/st/interface/stsolve.c:582 [0]PETSC ERROR: #7 EPSSetUp() at ./src/eps/interface/epssetup.c:350 [0]PETSC ERROR: #8 EPSSolve() at ./src/eps/interface/epssolve.c:136 [0]PETSC ERROR: #9 main() at ex_implicit_matr.cpp:95 [0]PETSC ERROR: No PETSc Option Table entries [0]PETSC ERROR: ----------------End of Error Message -------send entire error message to petsc-maint at mcs.anl.gov---------- -------------------------------------------------------------------------- MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD with errorcode 92. NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes. You may or may not see output from other processes, depending on exactly when Open MPI kills them. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- /* - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - SLEPc - Scalable Library for Eigenvalue Problem Computations Copyright (c) 2002-, Universitat Politecnica de Valencia, Spain This file is part of SLEPc. SLEPc is distributed under a 2-clause BSD license (see LICENSE). - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - */ // 2-D Laplacian Eigenproblem (matrix-free version), N=100 (10x10 grid) // Solution method: krylovschur // Number of requested eigenvalues: 1 // ---------------------- -------------------- // k ||Ax-kx||/||kx|| // ---------------------- -------------------- // 7.837972 1.84054e-09 // 7.601493 6.49209e-09 // ---------------------- -------------------- static char help[] = "Solves the same eigenproblem as in example ex2, but using a shell matrix. " "The problem is a standard symmetric eigenproblem corresponding to the 2-D Laplacian operator.\n\n" "The command line options are:\n" " -n , where = number of grid subdivisions in both x and y dimensions.\n\n"; #include /* User-defined routines */ PetscErrorCode MatMult_Laplacian2D(Mat A, Vec x, Vec y); PetscErrorCode MatMult_Laplacian2D_right(Mat A, Vec x, Vec y); PetscErrorCode MatGetDiagonal_Laplacian2D(Mat A, Vec diag); int main(int argc, char **argv) { Mat A; /* operator matrix */ Mat B; /* operator matrix */ EPS eps; /* eigenproblem solver context */ EPSType type; PetscMPIInt size; PetscInt N, n = 10, nev; PetscBool terse; CHKERRQ(SlepcInitialize(&argc, &argv, (char *)0, help)); CHKERRMPI(MPI_Comm_size(PETSC_COMM_WORLD, &size)); CHKERRQ(PetscOptionsGetInt(NULL, NULL, "-n", &n, NULL)); N = n * n; CHKERRQ(PetscPrintf(PETSC_COMM_WORLD, "\n2-D Laplacian Eigenproblem (matrix-free version), N=%" PetscInt_FMT " (%" PetscInt_FMT "x%" PetscInt_FMT " grid)\n\n", N, n, n)); /* - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Create the operator matrix that defines the eigensystem, Ax=kx - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - */ CHKERRQ(MatCreateShell(PETSC_COMM_WORLD, N, N, N, N, &n, &A)); CHKERRQ(MatShellSetOperation(A, MATOP_MULT, (void (*)(void))MatMult_Laplacian2D)); CHKERRQ(MatCreateShell(PETSC_COMM_WORLD, N, N, N, N, &n, &B)); CHKERRQ(MatShellSetOperation(B, MATOP_MULT, (void (*)(void))MatMult_Laplacian2D_right)); CHKERRQ(MatSetFromOptions(A)); CHKERRQ(MatSetFromOptions(B)); /* - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Create the eigensolver and set various options - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - */ /* Create eigensolver context */ CHKERRQ(EPSCreate(PETSC_COMM_WORLD, &eps)); /* Set operators. In this case, it is a standard eigenvalue problem */ CHKERRQ(EPSSetOperators(eps, A, B)); // CHKERRQ(EPSSetOperators(eps, A, NULL)); /* Set solver parameters at runtime */ CHKERRQ(EPSSetFromOptions(eps)); /* - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Solve the eigensystem - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - */ CHKERRQ(EPSSolve(eps)); /* Optional: Get some information from the solver and display it */ CHKERRQ(EPSGetType(eps, &type)); CHKERRQ(PetscPrintf(PETSC_COMM_WORLD, " Solution method: %s\n\n", type)); CHKERRQ(EPSGetDimensions(eps, &nev, NULL, NULL)); CHKERRQ(PetscPrintf(PETSC_COMM_WORLD, " Number of requested eigenvalues: %" PetscInt_FMT "\n", nev)); /* - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Display solution and clean up - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - */ /* show detailed info unless -terse option is given by user */ CHKERRQ(PetscOptionsHasName(NULL, NULL, "-terse", &terse)); if (terse) CHKERRQ(EPSErrorView(eps, EPS_ERROR_RELATIVE, NULL)); else { CHKERRQ(PetscViewerPushFormat(PETSC_VIEWER_STDOUT_WORLD, PETSC_VIEWER_ASCII_INFO_DETAIL)); CHKERRQ(EPSErrorView(eps, EPS_ERROR_RELATIVE, PETSC_VIEWER_STDOUT_WORLD)); CHKERRQ(PetscViewerPopFormat(PETSC_VIEWER_STDOUT_WORLD)); } CHKERRQ(EPSDestroy(&eps)); CHKERRQ(MatDestroy(&A)); CHKERRQ(SlepcFinalize()); return 0; } /* Compute the matrix vector multiplication y<---T*x where T is a nx by nx tridiagonal matrix with DD on the diagonal, DL on the subdiagonal, and DU on the superdiagonal. */ static void tv(int nx, const PetscScalar *x, PetscScalar *y) { PetscScalar dd, dl, du; int j; dd = 4.0; dl = -1.0; du = -1.0; y[0] = dd * x[0] + du * x[1]; for (j = 1; j < nx - 1; j++) y[j] = dl * x[j - 1] + dd * x[j] + du * x[j + 1]; y[nx - 1] = dl * x[nx - 2] + dd * x[nx - 1]; } /* Matrix-vector product subroutine for the 2D Laplacian. The matrix used is the 2 dimensional discrete Laplacian on unit square with zero Dirichlet boundary condition. Computes y <-- A*x, where A is the block tridiagonal matrix | T -I | |-I T -I | A = | -I T | | ... -I| | -I T| The subroutine TV is called to compute y<--T*x. */ PetscErrorCode MatMult_Laplacian2D(Mat A, Vec x, Vec y) { void *ctx; int nx, lo, i, j; const PetscScalar *px; PetscScalar *py; PetscFunctionBeginUser; CHKERRQ(MatShellGetContext(A, &ctx)); nx = *(int *)ctx; CHKERRQ(VecGetArrayRead(x, &px)); CHKERRQ(VecGetArray(y, &py)); tv(nx, &px[0], &py[0]); for (i = 0; i < nx; i++) py[i] -= px[nx + i]; for (j = 2; j < nx; j++) { lo = (j - 1) * nx; tv(nx, &px[lo], &py[lo]); for (i = 0; i < nx; i++) py[lo + i] -= px[lo - nx + i] + px[lo + nx + i]; } lo = (nx - 1) * nx; tv(nx, &px[lo], &py[lo]); for (i = 0; i < nx; i++) py[lo + i] -= px[lo - nx + i]; CHKERRQ(VecRestoreArrayRead(x, &px)); CHKERRQ(VecRestoreArray(y, &py)); PetscFunctionReturn(0); } PetscErrorCode MatMult_Laplacian2D_right(Mat A, Vec x, Vec y) { void *ctx; int nx, lo, i, j; const PetscScalar *px; PetscScalar *py; PetscFunctionBeginUser; CHKERRQ(MatShellGetContext(A, &ctx)); nx = *(int *)ctx; CHKERRQ(VecGetArrayRead(x, &px)); CHKERRQ(VecGetArray(y, &py)); for (i = 0; i < nx*nx; i++) py[i] = px[i]; CHKERRQ(VecRestoreArrayRead(x, &px)); CHKERRQ(VecRestoreArray(y, &py)); PetscFunctionReturn(0); } /*TEST test: suffix: 1 args: -n 72 -eps_nev 4 -eps_ncv 20 -terse requires: !single TEST*/ From jroman at dsic.upv.es Sun May 15 09:52:21 2022 From: jroman at dsic.upv.es (Jose E. Roman) Date: Sun, 15 May 2022 16:52:21 +0200 Subject: [petsc-users] Matrix free generalized eigenvalue problem In-Reply-To: References: Message-ID: You cannot compute the LU factorization of a shell matrix. You will have to use a preconditioned iterative linear solver, see section 3.4.1 of the SLEPc users guide. Jose > El 15 may 2022, a las 14:39, ???? ????????? escribi?: > > Hi, I am developing a program for solving generalized eigenvalue problem Ax = k Bx when both operators A and B are matrix-free. I found some examples on SLEPc documentation but and there was and example where problem Ax = k x are solving matrix free. So I tried to change this example a little bit and added operator B as identity matrix-free operator. But for now I am getting an error. My source code is attached. And the error message is below > Can you please explain to me what I am doing wrong? > > [0]PETSC ERROR: --------------------- Error Message -------------------------------------------------------------- > [0]PETSC ERROR: See https://petsc.org/release/overview/linear_solve_table/ for possible LU and Cholesky solvers > [0]PETSC ERROR: Could not locate a solver type for factorization type LU and matrix type shell. > [0]PETSC ERROR: See https://petsc.org/release/faq/ for trouble shooting. > [0]PETSC ERROR: Petsc Release Version 3.15.5, Sep 29, 2021 > [0]PETSC ERROR: ./exe_eps_impl_matr on a named LAPTOP-7DN2DH7N by misha Sun May 15 15:21:59 2022 > [0]PETSC ERROR: Configure options --build=x86_64-linux-gnu --prefix=/usr --includedir=${prefix}/include --mandir=${prefix}/share/man --infodir=${prefix}/share/info --sysconfdir=/etc --localstatedir=/var --with-option-checking=0 --with-silent-rules=0 --libdir=${prefix}/lib/x86_64-linux-gnu --runstatedir=/run --with-maintainer-mode=0 --with-dependency-tracking=0 --with-64-bit-indices --with-debugging=0 --shared-library-extension=64_real --with-shared-libraries --with-pic=1 --with-cc=mpicc --with-cxx=mpicxx --with-fc=mpif90 --with-cxx-dialect=C++11 --with-opencl=1 --with-blas-lib=-lblas --with-lapack-lib=-llapack --with-scalapack=1 --with-scalapack-lib=-lscalapack-openmpi --with-ptscotch=1 --with-ptscotch-include=/usr/include/scotch --with-ptscotch-lib="-lptesmumps -lptscotch -lptscotcherr" --with-fftw=1 --with-fftw-include="[]" --with-fftw-lib="-lfftw3 -lfftw3_mpi" --with-yaml=1 --with-hdf5-include=/usr/include/hdf5/openmpi --with-hdf5-lib="-L/usr/lib/x86_64-linux-gnu/hdf5/openmpi -L/usr/lib/x86_64-linux-gnu/openmpi/lib -lhdf5 -lmpi" --CXX_LINKER_FLAGS=-Wl,--no-as-needed --with-hypre=1 --with-hypre-include=/usr/include/hypre64m --with-hypre-lib=-lHYPRE64m --with-mumps=1 --with-mumps-include="[]" --with-mumps-lib="-ldmumps_64 -lzmumps_64 -lsmumps_64 -lcmumps_64 -lmumps_common_64 -lpord_64" --with-suitesparse=1 --with-suitesparse-include=/usr/include/suitesparse --with-suitesparse-lib="-lumfpack -lamd -lcholmod -lklu" --with-zoltan=1 --with-zoltan-include=/usr/include/trilinos --with-zoltan-lib=-ltrilinos_zoltan --prefix=/usr/lib/petscdir/petsc64-3.15/x86_64-linux-gnu-real --PETSC_ARCH=x86_64-linux-gnu-real-64 CFLAGS="-g -O2 -ffile-prefix-map=/build/petsc-tFq6Xk/petsc-3.15.5+dfsg1=. -flto=auto -ffat-lto-objects -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -fPIC" CXXFLAGS="-g -O2 -ffile-prefix-map=/build/petsc-tFq6Xk/petsc-3.15.5+dfsg1=. -flto=auto -ffat-lto-objects -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -fPIC" FCFLAGS="-g -O2 -ffile-prefix-map=/build/petsc-tFq6Xk/petsc-3.15.5+dfsg1=. -flto=auto -ffat-lto-objects -flto=auto -ffat-lto-objects -fstack-protector-strong -fPIC -ffree-line-length-0" FFLAGS="-g -O2 -ffile-prefix-map=/build/petsc-tFq6Xk/petsc-3.15.5+dfsg1=. -flto=auto -ffat-lto-objects -flto=auto -ffat-lto-objects -fstack-protector-strong -fPIC -ffree-line-length-0" CPPFLAGS="-Wdate-time -D_FORTIFY_SOURCE=2" LDFLAGS="-Wl,-Bsymbolic-functions -flto=auto -ffat-lto-objects -flto=auto -Wl,-z,relro -fPIC" MAKEFLAGS=w > [0]PETSC ERROR: #1 MatGetFactor() at ./src/mat/interface/matrix.c:4756 > [0]PETSC ERROR: #2 PCSetUp_LU() at ./src/ksp/pc/impls/factor/lu/lu.c:82 > [0]PETSC ERROR: #3 PCSetUp() at ./src/ksp/pc/interface/precon.c:1017 > [0]PETSC ERROR: #4 KSPSetUp() at ./src/ksp/ksp/interface/itfunc.c:406 > [0]PETSC ERROR: #5 STSetUp_Shift() at ./src/sys/classes/st/impls/shift/shift.c:107 > [0]PETSC ERROR: #6 STSetUp() at ./src/sys/classes/st/interface/stsolve.c:582 > [0]PETSC ERROR: #7 EPSSetUp() at ./src/eps/interface/epssetup.c:350 > [0]PETSC ERROR: #8 EPSSolve() at ./src/eps/interface/epssolve.c:136 > [0]PETSC ERROR: #9 main() at ex_implicit_matr.cpp:95 > [0]PETSC ERROR: No PETSc Option Table entries > [0]PETSC ERROR: ----------------End of Error Message -------send entire error message to petsc-maint at mcs.anl.gov---------- > -------------------------------------------------------------------------- > MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD > with errorcode 92. > > NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes. > You may or may not see output from other processes, depending on > exactly when Open MPI kills them. > From rolekst095 at gmail.com Sun May 15 11:54:43 2022 From: rolekst095 at gmail.com (=?UTF-8?B?0JzQuNGI0LAg0KHQsNC70YzQvdC40LrQvtCy?=) Date: Sun, 15 May 2022 19:54:43 +0300 Subject: [petsc-users] Matrix free generalized eigenvalue problem In-Reply-To: References: Message-ID: Thanks a lot for your answer. I found a solution in the documentation. So if I understand correctly SLEPc during eigenvalue iterations of Ax = k Bx is trying to calculate the inverse of B or y = Bx. So the question is don't you know maybe there exist methods or ways for generalized eigenproblems without inverting of B? Again thanks a lot for the answer ??, 15 ??? 2022 ?. ? 17:52, Jose E. Roman : > You cannot compute the LU factorization of a shell matrix. You will have > to use a preconditioned iterative linear solver, see section 3.4.1 of the > SLEPc users guide. > > Jose > > > > El 15 may 2022, a las 14:39, ???? ????????? > escribi?: > > > > Hi, I am developing a program for solving generalized eigenvalue problem > Ax = k Bx when both operators A and B are matrix-free. I found some > examples on SLEPc documentation but and there was and example where problem > Ax = k x are solving matrix free. So I tried to change this example a > little bit and added operator B as identity matrix-free operator. But for > now I am getting an error. My source code is attached. And the error > message is below > > Can you please explain to me what I am doing wrong? > > > > [0]PETSC ERROR: --------------------- Error Message > -------------------------------------------------------------- > > [0]PETSC ERROR: See > https://petsc.org/release/overview/linear_solve_table/ for possible LU > and Cholesky solvers > > [0]PETSC ERROR: Could not locate a solver type for factorization type LU > and matrix type shell. > > [0]PETSC ERROR: See https://petsc.org/release/faq/ for trouble shooting. > > [0]PETSC ERROR: Petsc Release Version 3.15.5, Sep 29, 2021 > > [0]PETSC ERROR: ./exe_eps_impl_matr on a named LAPTOP-7DN2DH7N by misha > Sun May 15 15:21:59 2022 > > [0]PETSC ERROR: Configure options --build=x86_64-linux-gnu --prefix=/usr > --includedir=${prefix}/include --mandir=${prefix}/share/man > --infodir=${prefix}/share/info --sysconfdir=/etc --localstatedir=/var > --with-option-checking=0 --with-silent-rules=0 > --libdir=${prefix}/lib/x86_64-linux-gnu --runstatedir=/run > --with-maintainer-mode=0 --with-dependency-tracking=0 --with-64-bit-indices > --with-debugging=0 --shared-library-extension=64_real > --with-shared-libraries --with-pic=1 --with-cc=mpicc --with-cxx=mpicxx > --with-fc=mpif90 --with-cxx-dialect=C++11 --with-opencl=1 > --with-blas-lib=-lblas --with-lapack-lib=-llapack --with-scalapack=1 > --with-scalapack-lib=-lscalapack-openmpi --with-ptscotch=1 > --with-ptscotch-include=/usr/include/scotch > --with-ptscotch-lib="-lptesmumps -lptscotch -lptscotcherr" --with-fftw=1 > --with-fftw-include="[]" --with-fftw-lib="-lfftw3 -lfftw3_mpi" > --with-yaml=1 --with-hdf5-include=/usr/include/hdf5/openmpi > --with-hdf5-lib="-L/usr/lib/x86_64-linux-gnu/hdf5/openmpi > -L/usr/lib/x86_64-linux-gnu/openmpi/lib -lhdf5 -lmpi" > --CXX_LINKER_FLAGS=-Wl,--no-as-needed --with-hypre=1 > --with-hypre-include=/usr/include/hypre64m --with-hypre-lib=-lHYPRE64m > --with-mumps=1 --with-mumps-include="[]" --with-mumps-lib="-ldmumps_64 > -lzmumps_64 -lsmumps_64 -lcmumps_64 -lmumps_common_64 -lpord_64" > --with-suitesparse=1 --with-suitesparse-include=/usr/include/suitesparse > --with-suitesparse-lib="-lumfpack -lamd -lcholmod -lklu" --with-zoltan=1 > --with-zoltan-include=/usr/include/trilinos > --with-zoltan-lib=-ltrilinos_zoltan > --prefix=/usr/lib/petscdir/petsc64-3.15/x86_64-linux-gnu-real > --PETSC_ARCH=x86_64-linux-gnu-real-64 CFLAGS="-g -O2 > -ffile-prefix-map=/build/petsc-tFq6Xk/petsc-3.15.5+dfsg1=. -flto=auto > -ffat-lto-objects -flto=auto -ffat-lto-objects -fstack-protector-strong > -Wformat -Werror=format-security -fPIC" CXXFLAGS="-g -O2 > -ffile-prefix-map=/build/petsc-tFq6Xk/petsc-3.15.5+dfsg1=. -flto=auto > -ffat-lto-objects -flto=auto -ffat-lto-objects -fstack-protector-strong > -Wformat -Werror=format-security -fPIC" FCFLAGS="-g -O2 > -ffile-prefix-map=/build/petsc-tFq6Xk/petsc-3.15.5+dfsg1=. -flto=auto > -ffat-lto-objects -flto=auto -ffat-lto-objects -fstack-protector-strong > -fPIC -ffree-line-length-0" FFLAGS="-g -O2 > -ffile-prefix-map=/build/petsc-tFq6Xk/petsc-3.15.5+dfsg1=. -flto=auto > -ffat-lto-objects -flto=auto -ffat-lto-objects -fstack-protector-strong > -fPIC -ffree-line-length-0" CPPFLAGS="-Wdate-time -D_FORTIFY_SOURCE=2" > LDFLAGS="-Wl,-Bsymbolic-functions -flto=auto -ffat-lto-objects -flto=auto > -Wl,-z,relro -fPIC" MAKEFLAGS=w > > [0]PETSC ERROR: #1 MatGetFactor() at ./src/mat/interface/matrix.c:4756 > > [0]PETSC ERROR: #2 PCSetUp_LU() at ./src/ksp/pc/impls/factor/lu/lu.c:82 > > [0]PETSC ERROR: #3 PCSetUp() at ./src/ksp/pc/interface/precon.c:1017 > > [0]PETSC ERROR: #4 KSPSetUp() at ./src/ksp/ksp/interface/itfunc.c:406 > > [0]PETSC ERROR: #5 STSetUp_Shift() at > ./src/sys/classes/st/impls/shift/shift.c:107 > > [0]PETSC ERROR: #6 STSetUp() at > ./src/sys/classes/st/interface/stsolve.c:582 > > [0]PETSC ERROR: #7 EPSSetUp() at ./src/eps/interface/epssetup.c:350 > > [0]PETSC ERROR: #8 EPSSolve() at ./src/eps/interface/epssolve.c:136 > > [0]PETSC ERROR: #9 main() at ex_implicit_matr.cpp:95 > > [0]PETSC ERROR: No PETSc Option Table entries > > [0]PETSC ERROR: ----------------End of Error Message -------send entire > error message to petsc-maint at mcs.anl.gov---------- > > > -------------------------------------------------------------------------- > > MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD > > with errorcode 92. > > > > NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes. > > You may or may not see output from other processes, depending on > > exactly when Open MPI kills them. > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jroman at dsic.upv.es Sun May 15 12:02:15 2022 From: jroman at dsic.upv.es (Jose E. Roman) Date: Sun, 15 May 2022 19:02:15 +0200 Subject: [petsc-users] Matrix free generalized eigenvalue problem In-Reply-To: References: Message-ID: Preconditioned eigensolvers avoid an explicit inversion. You can try Davidson methods https://doi.org/10.1145/2543696 or LOBPCG if the problem is symmetric-definite. However, for reasonable convergence these methods usually require a good preconditioner, so if your matrix is implicit you still need to explicitly build some type of approximation that can be used to build the preconditioner. Jose > El 15 may 2022, a las 18:54, ???? ????????? escribi?: > > Thanks a lot for your answer. I found a solution in the documentation. So if I understand correctly SLEPc during eigenvalue iterations of Ax = k Bx is trying to calculate the inverse of B or y = Bx. So the question is don't you know maybe there exist methods or ways for generalized eigenproblems without inverting of B? Again thanks a lot for the answer > > ??, 15 ??? 2022 ?. ? 17:52, Jose E. Roman : > You cannot compute the LU factorization of a shell matrix. You will have to use a preconditioned iterative linear solver, see section 3.4.1 of the SLEPc users guide. > > Jose > > > > El 15 may 2022, a las 14:39, ???? ????????? escribi?: > > > > Hi, I am developing a program for solving generalized eigenvalue problem Ax = k Bx when both operators A and B are matrix-free. I found some examples on SLEPc documentation but and there was and example where problem Ax = k x are solving matrix free. So I tried to change this example a little bit and added operator B as identity matrix-free operator. But for now I am getting an error. My source code is attached. And the error message is below > > Can you please explain to me what I am doing wrong? > > > > [0]PETSC ERROR: --------------------- Error Message -------------------------------------------------------------- > > [0]PETSC ERROR: See https://petsc.org/release/overview/linear_solve_table/ for possible LU and Cholesky solvers > > [0]PETSC ERROR: Could not locate a solver type for factorization type LU and matrix type shell. > > [0]PETSC ERROR: See https://petsc.org/release/faq/ for trouble shooting. > > [0]PETSC ERROR: Petsc Release Version 3.15.5, Sep 29, 2021 > > [0]PETSC ERROR: ./exe_eps_impl_matr on a named LAPTOP-7DN2DH7N by misha Sun May 15 15:21:59 2022 > > [0]PETSC ERROR: Configure options --build=x86_64-linux-gnu --prefix=/usr --includedir=${prefix}/include --mandir=${prefix}/share/man --infodir=${prefix}/share/info --sysconfdir=/etc --localstatedir=/var --with-option-checking=0 --with-silent-rules=0 --libdir=${prefix}/lib/x86_64-linux-gnu --runstatedir=/run --with-maintainer-mode=0 --with-dependency-tracking=0 --with-64-bit-indices --with-debugging=0 --shared-library-extension=64_real --with-shared-libraries --with-pic=1 --with-cc=mpicc --with-cxx=mpicxx --with-fc=mpif90 --with-cxx-dialect=C++11 --with-opencl=1 --with-blas-lib=-lblas --with-lapack-lib=-llapack --with-scalapack=1 --with-scalapack-lib=-lscalapack-openmpi --with-ptscotch=1 --with-ptscotch-include=/usr/include/scotch --with-ptscotch-lib="-lptesmumps -lptscotch -lptscotcherr" --with-fftw=1 --with-fftw-include="[]" --with-fftw-lib="-lfftw3 -lfftw3_mpi" --with-yaml=1 --with-hdf5-include=/usr/include/hdf5/openmpi --with-hdf5-lib="-L/usr/lib/x86_64-linux-gnu/hdf5/openmpi -L/usr/lib/x86_64-linux-gnu/openmpi/lib -lhdf5 -lmpi" --CXX_LINKER_FLAGS=-Wl,--no-as-needed --with-hypre=1 --with-hypre-include=/usr/include/hypre64m --with-hypre-lib=-lHYPRE64m --with-mumps=1 --with-mumps-include="[]" --with-mumps-lib="-ldmumps_64 -lzmumps_64 -lsmumps_64 -lcmumps_64 -lmumps_common_64 -lpord_64" --with-suitesparse=1 --with-suitesparse-include=/usr/include/suitesparse --with-suitesparse-lib="-lumfpack -lamd -lcholmod -lklu" --with-zoltan=1 --with-zoltan-include=/usr/include/trilinos --with-zoltan-lib=-ltrilinos_zoltan --prefix=/usr/lib/petscdir/petsc64-3.15/x86_64-linux-gnu-real --PETSC_ARCH=x86_64-linux-gnu-real-64 CFLAGS="-g -O2 -ffile-prefix-map=/build/petsc-tFq6Xk/petsc-3.15.5+dfsg1=. -flto=auto -ffat-lto-objects -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -fPIC" CXXFLAGS="-g -O2 -ffile-prefix-map=/build/petsc-tFq6Xk/petsc-3.15.5+dfsg1=. -flto=auto -ffat-lto-objects -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -fPIC" FCFLAGS="-g -O2 -ffile-prefix-map=/build/petsc-tFq6Xk/petsc-3.15.5+dfsg1=. -flto=auto -ffat-lto-objects -flto=auto -ffat-lto-objects -fstack-protector-strong -fPIC -ffree-line-length-0" FFLAGS="-g -O2 -ffile-prefix-map=/build/petsc-tFq6Xk/petsc-3.15.5+dfsg1=. -flto=auto -ffat-lto-objects -flto=auto -ffat-lto-objects -fstack-protector-strong -fPIC -ffree-line-length-0" CPPFLAGS="-Wdate-time -D_FORTIFY_SOURCE=2" LDFLAGS="-Wl,-Bsymbolic-functions -flto=auto -ffat-lto-objects -flto=auto -Wl,-z,relro -fPIC" MAKEFLAGS=w > > [0]PETSC ERROR: #1 MatGetFactor() at ./src/mat/interface/matrix.c:4756 > > [0]PETSC ERROR: #2 PCSetUp_LU() at ./src/ksp/pc/impls/factor/lu/lu.c:82 > > [0]PETSC ERROR: #3 PCSetUp() at ./src/ksp/pc/interface/precon.c:1017 > > [0]PETSC ERROR: #4 KSPSetUp() at ./src/ksp/ksp/interface/itfunc.c:406 > > [0]PETSC ERROR: #5 STSetUp_Shift() at ./src/sys/classes/st/impls/shift/shift.c:107 > > [0]PETSC ERROR: #6 STSetUp() at ./src/sys/classes/st/interface/stsolve.c:582 > > [0]PETSC ERROR: #7 EPSSetUp() at ./src/eps/interface/epssetup.c:350 > > [0]PETSC ERROR: #8 EPSSolve() at ./src/eps/interface/epssolve.c:136 > > [0]PETSC ERROR: #9 main() at ex_implicit_matr.cpp:95 > > [0]PETSC ERROR: No PETSc Option Table entries > > [0]PETSC ERROR: ----------------End of Error Message -------send entire error message to petsc-maint at mcs.anl.gov---------- > > -------------------------------------------------------------------------- > > MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD > > with errorcode 92. > > > > NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes. > > You may or may not see output from other processes, depending on > > exactly when Open MPI kills them. > > > From rolekst095 at gmail.com Sun May 15 12:15:31 2022 From: rolekst095 at gmail.com (=?UTF-8?B?0JzQuNGI0LAg0KHQsNC70YzQvdC40LrQvtCy?=) Date: Sun, 15 May 2022 20:15:31 +0300 Subject: [petsc-users] Matrix free generalized eigenvalue problem In-Reply-To: References: Message-ID: Ok, thank you very much for your help and links to articles. ??, 15 ??? 2022 ?. ? 20:02, Jose E. Roman : > Preconditioned eigensolvers avoid an explicit inversion. You can try > Davidson methods https://doi.org/10.1145/2543696 or LOBPCG if the problem > is symmetric-definite. However, for reasonable convergence these methods > usually require a good preconditioner, so if your matrix is implicit you > still need to explicitly build some type of approximation that can be used > to build the preconditioner. > > Jose > > > > El 15 may 2022, a las 18:54, ???? ????????? > escribi?: > > > > Thanks a lot for your answer. I found a solution in the documentation. > So if I understand correctly SLEPc during eigenvalue iterations of Ax = k > Bx is trying to calculate the inverse of B or y = Bx. So the question is > don't you know maybe there exist methods or ways for generalized > eigenproblems without inverting of B? Again thanks a lot for the answer > > > > ??, 15 ??? 2022 ?. ? 17:52, Jose E. Roman : > > You cannot compute the LU factorization of a shell matrix. You will have > to use a preconditioned iterative linear solver, see section 3.4.1 of the > SLEPc users guide. > > > > Jose > > > > > > > El 15 may 2022, a las 14:39, ???? ????????? > escribi?: > > > > > > Hi, I am developing a program for solving generalized eigenvalue > problem Ax = k Bx when both operators A and B are matrix-free. I found some > examples on SLEPc documentation but and there was and example where problem > Ax = k x are solving matrix free. So I tried to change this example a > little bit and added operator B as identity matrix-free operator. But for > now I am getting an error. My source code is attached. And the error > message is below > > > Can you please explain to me what I am doing wrong? > > > > > > [0]PETSC ERROR: --------------------- Error Message > -------------------------------------------------------------- > > > [0]PETSC ERROR: See > https://petsc.org/release/overview/linear_solve_table/ for possible LU > and Cholesky solvers > > > [0]PETSC ERROR: Could not locate a solver type for factorization type > LU and matrix type shell. > > > [0]PETSC ERROR: See https://petsc.org/release/faq/ for trouble > shooting. > > > [0]PETSC ERROR: Petsc Release Version 3.15.5, Sep 29, 2021 > > > [0]PETSC ERROR: ./exe_eps_impl_matr on a named LAPTOP-7DN2DH7N by > misha Sun May 15 15:21:59 2022 > > > [0]PETSC ERROR: Configure options --build=x86_64-linux-gnu > --prefix=/usr --includedir=${prefix}/include --mandir=${prefix}/share/man > --infodir=${prefix}/share/info --sysconfdir=/etc --localstatedir=/var > --with-option-checking=0 --with-silent-rules=0 > --libdir=${prefix}/lib/x86_64-linux-gnu --runstatedir=/run > --with-maintainer-mode=0 --with-dependency-tracking=0 --with-64-bit-indices > --with-debugging=0 --shared-library-extension=64_real > --with-shared-libraries --with-pic=1 --with-cc=mpicc --with-cxx=mpicxx > --with-fc=mpif90 --with-cxx-dialect=C++11 --with-opencl=1 > --with-blas-lib=-lblas --with-lapack-lib=-llapack --with-scalapack=1 > --with-scalapack-lib=-lscalapack-openmpi --with-ptscotch=1 > --with-ptscotch-include=/usr/include/scotch > --with-ptscotch-lib="-lptesmumps -lptscotch -lptscotcherr" --with-fftw=1 > --with-fftw-include="[]" --with-fftw-lib="-lfftw3 -lfftw3_mpi" > --with-yaml=1 --with-hdf5-include=/usr/include/hdf5/openmpi > --with-hdf5-lib="-L/usr/lib/x86_64-linux-gnu/hdf5/openmpi > -L/usr/lib/x86_64-linux-gnu/openmpi/lib -lhdf5 -lmpi" > --CXX_LINKER_FLAGS=-Wl,--no-as-needed --with-hypre=1 > --with-hypre-include=/usr/include/hypre64m --with-hypre-lib=-lHYPRE64m > --with-mumps=1 --with-mumps-include="[]" --with-mumps-lib="-ldmumps_64 > -lzmumps_64 -lsmumps_64 -lcmumps_64 -lmumps_common_64 -lpord_64" > --with-suitesparse=1 --with-suitesparse-include=/usr/include/suitesparse > --with-suitesparse-lib="-lumfpack -lamd -lcholmod -lklu" --with-zoltan=1 > --with-zoltan-include=/usr/include/trilinos > --with-zoltan-lib=-ltrilinos_zoltan > --prefix=/usr/lib/petscdir/petsc64-3.15/x86_64-linux-gnu-real > --PETSC_ARCH=x86_64-linux-gnu-real-64 CFLAGS="-g -O2 > -ffile-prefix-map=/build/petsc-tFq6Xk/petsc-3.15.5+dfsg1=. -flto=auto > -ffat-lto-objects -flto=auto -ffat-lto-objects -fstack-protector-strong > -Wformat -Werror=format-security -fPIC" CXXFLAGS="-g -O2 > -ffile-prefix-map=/build/petsc-tFq6Xk/petsc-3.15.5+dfsg1=. -flto=auto > -ffat-lto-objects -flto=auto -ffat-lto-objects -fstack-protector-strong > -Wformat -Werror=format-security -fPIC" FCFLAGS="-g -O2 > -ffile-prefix-map=/build/petsc-tFq6Xk/petsc-3.15.5+dfsg1=. -flto=auto > -ffat-lto-objects -flto=auto -ffat-lto-objects -fstack-protector-strong > -fPIC -ffree-line-length-0" FFLAGS="-g -O2 > -ffile-prefix-map=/build/petsc-tFq6Xk/petsc-3.15.5+dfsg1=. -flto=auto > -ffat-lto-objects -flto=auto -ffat-lto-objects -fstack-protector-strong > -fPIC -ffree-line-length-0" CPPFLAGS="-Wdate-time -D_FORTIFY_SOURCE=2" > LDFLAGS="-Wl,-Bsymbolic-functions -flto=auto -ffat-lto-objects -flto=auto > -Wl,-z,relro -fPIC" MAKEFLAGS=w > > > [0]PETSC ERROR: #1 MatGetFactor() at ./src/mat/interface/matrix.c:4756 > > > [0]PETSC ERROR: #2 PCSetUp_LU() at ./src/ksp/pc/impls/factor/lu/lu.c:82 > > > [0]PETSC ERROR: #3 PCSetUp() at ./src/ksp/pc/interface/precon.c:1017 > > > [0]PETSC ERROR: #4 KSPSetUp() at ./src/ksp/ksp/interface/itfunc.c:406 > > > [0]PETSC ERROR: #5 STSetUp_Shift() at > ./src/sys/classes/st/impls/shift/shift.c:107 > > > [0]PETSC ERROR: #6 STSetUp() at > ./src/sys/classes/st/interface/stsolve.c:582 > > > [0]PETSC ERROR: #7 EPSSetUp() at ./src/eps/interface/epssetup.c:350 > > > [0]PETSC ERROR: #8 EPSSolve() at ./src/eps/interface/epssolve.c:136 > > > [0]PETSC ERROR: #9 main() at ex_implicit_matr.cpp:95 > > > [0]PETSC ERROR: No PETSc Option Table entries > > > [0]PETSC ERROR: ----------------End of Error Message -------send > entire error message to petsc-maint at mcs.anl.gov---------- > > > > -------------------------------------------------------------------------- > > > MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD > > > with errorcode 92. > > > > > > NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes. > > > You may or may not see output from other processes, depending on > > > exactly when Open MPI kills them. > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From yangzongze at gmail.com Mon May 16 00:31:45 2022 From: yangzongze at gmail.com (Yang Zongze) Date: Mon, 16 May 2022 13:31:45 +0800 Subject: [petsc-users] An internal abort when perform LU decomposition. Message-ID: <4EC59E5F-AFB9-4A28-8A48-D94F448F0614@hxcore.ol> An HTML attachment was scrubbed... URL: From Karol.Lewandowski at glasgow.ac.uk Mon May 16 04:58:08 2022 From: Karol.Lewandowski at glasgow.ac.uk (Karol Lewandowski) Date: Mon, 16 May 2022 09:58:08 +0000 Subject: [petsc-users] TS Time Derivative after solved step Message-ID: <2028A125-8E18-4B6B-82D3-AE9A024F17C7@glasgow.ac.uk> Hi, I am developing a program for solving a time-dependent problem using TS solver. At some point, after a successful step, I would like to compute the error indicator which involves time derivatives. I tried to use routines like TSSetPostStep(), however, they do not give me access to vector ts_u_t. Is there a way to compute the post-step time derivative without copying the vectors (like ts_u_t, ts_u_tt) at each step? Kind regards, Karol -------------- next part -------------- An HTML attachment was scrubbed... URL: From maschi at mek.dtu.dk Mon May 16 09:40:23 2022 From: maschi at mek.dtu.dk (=?UTF-8?Q?Martin_Schi=c3=b8dt?=) Date: Mon, 16 May 2022 16:40:23 +0200 Subject: [petsc-users] Loading contents of hdf5 file into Mat object Message-ID: <4998bf34-0b8c-58c1-e39f-3f1187a9ea8b@mek.dtu.dk> Dear petsc-expert! I am having trouble loading the contents of my hdf5 dataset - "Vp" - into the Mat object Vp using MatLoad. When I try to do this, I get the error message displayed below. The hdf5 file I am trying to read, was originally created in Matlab with their built in function h5write. I am wondering whether this is the source of the problem? Nonetheless, a screendump of the file's structure can be seen here: I have tried following a few of your examples as inspiration, however there may of course still be a problem in my code. I have appended the code below here (it is a short program, so I hope it is ok! - and please disregard the red error-squiggles - these are not true). I hope you are able to help with my problem. Best regards, Martin staticcharhelp[]= "Reads a HDF5 into a PETSC matrix.\n\n"; #include"../../petsc/include/petscsys.h" #include"../../petsc/include/petscmat.h" #include"../../petsc/include/petscviewerhdf5.h" intmain(intargc, char**argv) { MatVp; PetscViewerviewer; charfile[PETSC_MAX_PATH_LEN]="../../Desktop/OnewaySimulations/Realization_1.h5"; charVp_name[128]="Vp"; PetscBoolflg; PetscInitialize(&argc, &argv, (char*)0, help); // Determine from which file we read the matrix PetscOptionsGetString(NULL,NULL,"-f",file,sizeof(file),&flg); // Read PetscViewerHDF5Open(PETSC_COMM_WORLD,file,FILE_MODE_READ,&viewer); // Load matrix MatCreate(PETSC_COMM_WORLD, &Vp); PetscObjectSetName((PetscObject)Vp, Vp_name); MatSetFromOptions(Vp); MatLoad(Vp,viewer); // Destroy PetscViewerDestroy(&viewer); MatDestroy(&Vp); PetscFinalize(); return0; } -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: AA0qWUjvJt0LdFH6.png Type: image/png Size: 69771 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: e6nBGT0GcsizHRnE.png Type: image/png Size: 8314 bytes Desc: not available URL: From mfadams at lbl.gov Mon May 16 11:47:47 2022 From: mfadams at lbl.gov (Mark Adams) Date: Mon, 16 May 2022 12:47:47 -0400 Subject: [petsc-users] TS Time Derivative after solved step In-Reply-To: <2028A125-8E18-4B6B-82D3-AE9A024F17C7@glasgow.ac.uk> References: <2028A125-8E18-4B6B-82D3-AE9A024F17C7@glasgow.ac.uk> Message-ID: You generally want to use https://petsc.org/main/docs/manualpages/TS/TSMonitorSet/ for something like this. TSSetPostStep is for diagnostics. There are differences between the two but I don't recall them. And you can get the timer derivative with https://petsc.org/release/docs/manualpages/TS/TS2GetSolution.html On Mon, May 16, 2022 at 11:03 AM Karol Lewandowski < Karol.Lewandowski at glasgow.ac.uk> wrote: > Hi, I am developing a program for solving a time-dependent problem using TS solver. At some point, after a successful step, I would like to compute the error indicator which involves time derivatives. I tried to use routines like TSSetPostStep(), however, they do not give me access to vector ts_u_t. > > Is there a way to compute the post-step time derivative without copying the vectors (like ts_u_t, ts_u_tt) at each step? > > > Kind regards, > > > Karol > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mfadams at lbl.gov Mon May 16 11:51:47 2022 From: mfadams at lbl.gov (Mark Adams) Date: Mon, 16 May 2022 12:51:47 -0400 Subject: [petsc-users] Loading contents of hdf5 file into Mat object In-Reply-To: <4998bf34-0b8c-58c1-e39f-3f1187a9ea8b@mek.dtu.dk> References: <4998bf34-0b8c-58c1-e39f-3f1187a9ea8b@mek.dtu.dk> Message-ID: https://petsc.org/main/docs/manualpages/Mat/MatLoad/ "Loads a matrix that has been stored in binary/HDF5 format with MatView()." But there is a discussion on matlab format here. On Mon, May 16, 2022 at 11:03 AM Martin Schi?dt via petsc-users < petsc-users at mcs.anl.gov> wrote: > Dear petsc-expert! > > I am having trouble loading the contents of my hdf5 dataset - "Vp" - into > the Mat object Vp using MatLoad. When I try to do this, I get the error > message displayed below. > > The hdf5 file I am trying to read, was originally created in Matlab with > their built in function h5write. I am wondering whether this is the source > of the problem? Nonetheless, a screendump of the file's structure can be > seen here: > > I have tried following a few of your examples as inspiration, however > there may of course still be a problem in my code. I have appended the code > below here (it is a short program, so I hope it is ok! - and please > disregard the red error-squiggles - these are not true). > > I hope you are able to help with my problem. > > Best regards, > > Martin > static char help[] = "Reads a HDF5 into a PETSC matrix.\n\n"; > #include "../../petsc/include/petscsys.h" > #include "../../petsc/include/petscmat.h" > #include "../../petsc/include/petscviewerhdf5.h" > int main (int argc, char **argv) > { > Mat Vp; > PetscViewer viewer; > char file[PETSC_MAX_PATH_LEN]= > "../../Desktop/OnewaySimulations/Realization_1.h5"; > char Vp_name[128]="Vp"; > PetscBool flg; > PetscInitialize(&argc, &argv, (char*)0, help); > // Determine from which file we read the matrix > PetscOptionsGetString(NULL,NULL,"-f",file,sizeof(file),&flg); > // Read > PetscViewerHDF5Open(PETSC_COMM_WORLD,file,FILE_MODE_READ,&viewer); > // Load matrix > MatCreate(PETSC_COMM_WORLD, &Vp); > PetscObjectSetName((PetscObject)Vp, Vp_name); > MatSetFromOptions(Vp); > MatLoad(Vp,viewer); > // Destroy > PetscViewerDestroy(&viewer); > MatDestroy(&Vp); > PetscFinalize(); > return 0; > } > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: AA0qWUjvJt0LdFH6.png Type: image/png Size: 69771 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: e6nBGT0GcsizHRnE.png Type: image/png Size: 8314 bytes Desc: not available URL: From knepley at gmail.com Mon May 16 12:20:46 2022 From: knepley at gmail.com (Matthew Knepley) Date: Mon, 16 May 2022 07:20:46 -1000 Subject: [petsc-users] An internal abort when perform LU decomposition. In-Reply-To: <4EC59E5F-AFB9-4A28-8A48-D94F448F0614@hxcore.ol> References: <4EC59E5F-AFB9-4A28-8A48-D94F448F0614@hxcore.ol> Message-ID: On Mon, May 16, 2022 at 5:03 AM Yang Zongze wrote: > Hi, > > > > I am solving a Linear system with LU factorization. But failed with the > following error. > > Is there some suggestions on debugging this error? Thanks! > This appears to be inside MUMPS. I would recommend two things: 1) Get a stack trace for this error using the debugger 2) With that, also send a report to the MUMPS mailing list Thanks, Matt > The configure.log of PETSc is attached! > > > > ``` > > Assertion failed in file ./src/include/mpir_request.h at line 446: req != > NULL > > /home/z2yang/opt/firedrake-env/firedrake-mini-petsc/src/petsc/default/lib/libmpi.so.12(+0x2e6072) > [0x7fd51f801072] > > /home/z2yang/opt/firedrake-env/firedrake-mini-petsc/src/petsc/default/lib/libmpi.so.12(+0x20a8a8) > [0x7fd51f7258a8] > > /home/z2yang/opt/firedrake-env/firedrake-mini-petsc/src/petsc/default/lib/libmpi.so.12(+0x2a0318) > [0x7fd51f7bb318] > > /home/z2yang/opt/firedrake-env/firedrake-mini-petsc/src/petsc/default/lib/libmpi.so.12(MPI_Isend+0x498) > [0x7fd51f600ae8] > > /home/z2yang/opt/firedrake-env/firedrake-mini-petsc/src/petsc/default/lib/libmpifort.so.12(PMPI_ISEND+0x51) > [0x7fd51cf0acd1] > > /home/z2yang/opt/firedrake-env/firedrake-mini-petsc/src/petsc/default/lib/libpetsc.so.3.016(__dmumps_buf_MOD_dmumps_buf_send_contrib_type3+0x145b) > [0x7fd520b1011d] > > /home/z2yang/opt/firedrake-env/firedrake-mini-petsc/src/petsc/default/lib/libpetsc.so.3.016(dmumps_buf_send_contrib_type3_i_+0xb8) > [0x7fd520af584c] > > /home/z2yang/opt/firedrake-env/firedrake-mini-petsc/src/petsc/default/lib/libpetsc.so.3.016(dmumps_build_and_send_cb_root_+0x167e) > [0x7fd520b6252c] > > /home/z2yang/opt/firedrake-env/firedrake-mini-petsc/src/petsc/default/lib/libpetsc.so.3.016(dmumps_fac_stack_+0xf00) > [0x7fd520c60c88] > > /home/z2yang/opt/firedrake-env/firedrake-mini-petsc/src/petsc/default/lib/libpetsc.so.3.016(__dmumps_fac_par_m_MOD_dmumps_fac_par+0xade) > [0x7fd520c56e27] > > /home/z2yang/opt/firedrake-env/firedrake-mini-petsc/src/petsc/default/lib/libpetsc.so.3.016(dmumps_fac_par_i_+0x216) > [0x7fd520c000c2] > > /home/z2yang/opt/firedrake-env/firedrake-mini-petsc/src/petsc/default/lib/libpetsc.so.3.016(dmumps_fac_b_+0x975) > [0x7fd520c00a3f] > > /home/z2yang/opt/firedrake-env/firedrake-mini-petsc/src/petsc/default/lib/libpetsc.so.3.016(dmumps_fac_driver_+0x5d2f) > [0x7fd520bcd6cf] > > /home/z2yang/opt/firedrake-env/firedrake-mini-petsc/src/petsc/default/lib/libpetsc.so.3.016(dmumps_+0x2c6a) > [0x7fd520b5c499] > > /home/z2yang/opt/firedrake-env/firedrake-mini-petsc/src/petsc/default/lib/libpetsc.so.3.016(dmumps_f77_+0x190b) > [0x7fd520aef9f7] > > /home/z2yang/opt/firedrake-env/firedrake-mini-petsc/src/petsc/default/lib/libpetsc.so.3.016(dmumps_c+0xe10) > [0x7fd520aedfb4] > > /home/z2yang/opt/firedrake-env/firedrake-mini-petsc/src/petsc/default/lib/libpetsc.so.3.016(+0x50d7a3) > [0x7fd5200f77a3] > > /home/z2yang/opt/firedrake-env/firedrake-mini-petsc/src/petsc/default/lib/libpetsc.so.3.016(MatLUFactorNumeric+0x369) > [0x7fd520272431] > > /home/z2yang/opt/firedrake-env/firedrake-mini-petsc/src/petsc/default/lib/libpetsc.so.3.016(+0xc67f73) > [0x7fd520851f73] > > /home/z2yang/opt/firedrake-env/firedrake-mini-petsc/src/petsc/default/lib/libpetsc.so.3.016(PCSetUp+0x621) > [0x7fd5208d0029] > > /home/z2yang/opt/firedrake-env/firedrake-mini-petsc/src/petsc/default/lib/libpetsc.so.3.016(KSPSetUp+0xd57) > [0x7fd5206ffcf2] > > /home/z2yang/opt/firedrake-env/firedrake-mini-petsc/src/petsc/default/lib/libpetsc.so.3.016(+0xb17d4c) > [0x7fd520701d4c] > > /home/z2yang/opt/firedrake-env/firedrake-mini-petsc/src/petsc/default/lib/libpetsc.so.3.016(KSPSolve+0x13) > [0x7fd520704f13] > > > /home/z2yang/opt/firedrake-env/firedrake-mini-petsc/lib64/python3.6/site-packages/petsc4py/lib/default/ > PETSc.cpython-36m-x86_64-linux-gnu.so(+0xe18eb) [0x7fd52169c8eb] > > /lib64/libpython3.6m.so.1.0(_PyCFunction_FastCallDict+0x353) > [0x7fd5307a3a73] > > /lib64/libpython3.6m.so.1.0(+0x167dd0) [0x7fd5307a3dd0] > > /lib64/libpython3.6m.so.1.0(_PyEval_EvalFrameDefault+0x474) > [0x7fd5307cc5b4] > > /lib64/libpython3.6m.so.1.0(PyEval_EvalCodeEx+0x337) [0x7fd5307ae727] > > /lib64/libpython3.6m.so.1.0(+0x17369e) [0x7fd5307af69e] > > > /home/z2yang/opt/firedrake-env/firedrake-mini-petsc/lib64/python3.6/site-packages/petsc4py/lib/default/ > PETSc.cpython-36m-x86_64-linux-gnu.so(+0x71ed3) [0x7fd52162ced3] > > > /home/z2yang/opt/firedrake-env/firedrake-mini-petsc/lib64/python3.6/site-packages/petsc4py/lib/default/ > PETSc.cpython-36m-x86_64-linux-gnu.so(+0xd75f6) [0x7fd5216925f6] > > > /home/z2yang/opt/firedrake-env/firedrake-mini-petsc/lib64/python3.6/site-packages/petsc4py/lib/default/ > PETSc.cpython-36m-x86_64-linux-gnu.so(+0x717bf) [0x7fd52162c7bf] > > internal ABORT - process 0 > > ``` > > > > Best wishes, > > Zongze > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Mon May 16 12:26:48 2022 From: knepley at gmail.com (Matthew Knepley) Date: Mon, 16 May 2022 07:26:48 -1000 Subject: [petsc-users] TS Time Derivative after solved step In-Reply-To: References: <2028A125-8E18-4B6B-82D3-AE9A024F17C7@glasgow.ac.uk> Message-ID: On Mon, May 16, 2022 at 6:48 AM Mark Adams wrote: > You generally want to use > https://petsc.org/main/docs/manualpages/TS/TSMonitorSet/ for > something like this. > TSSetPostStep is for diagnostics. > There are differences between the two but I don't recall them. > Yes, I think this belongs in a monitor. > And you can get the timer derivative with > https://petsc.org/release/docs/manualpages/TS/TS2GetSolution.html > This only applies to a restricted set of TS implementations. In general, you have to compute approximations to the derivatives yourself, since it is not guaranteed to be represented in any given integration method. If you want a first order approximation, you would save the last solution, or maybe more for higher order reconstruction. PETSc provides the TSTrajectory object to automate this process (it is also used for adjoint computations). Thanks, Matt > On Mon, May 16, 2022 at 11:03 AM Karol Lewandowski < > Karol.Lewandowski at glasgow.ac.uk> wrote: > >> Hi, I am developing a program for solving a time-dependent problem using TS solver. At some point, after a successful step, I would like to compute the error indicator which involves time derivatives. I tried to use routines like TSSetPostStep(), however, they do not give me access to vector ts_u_t. >> >> Is there a way to compute the post-step time derivative without copying the vectors (like ts_u_t, ts_u_tt) at each step? >> >> >> Kind regards, >> >> >> Karol >> >> >> -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From hzhang at mcs.anl.gov Tue May 17 02:17:13 2022 From: hzhang at mcs.anl.gov (Zhang, Hong) Date: Tue, 17 May 2022 07:17:13 +0000 Subject: [petsc-users] An internal abort when perform LU decomposition. In-Reply-To: References: <4EC59E5F-AFB9-4A28-8A48-D94F448F0614@hxcore.ol> Message-ID: It crashes at /home/z2yang/opt/firedrake-env/firedrake-mini-petsc/src/petsc/default/lib/libpetsc.so.3.016(MatLUFactorNumeric+0x369) [0x7fd520272431] Please run it in debugging mode and get error info. Hong ________________________________ From: petsc-users on behalf of Matthew Knepley Sent: Monday, May 16, 2022 12:20 PM To: Yang Zongze Cc: petsc-users at mcs.anl.gov Subject: Re: [petsc-users] An internal abort when perform LU decomposition. On Mon, May 16, 2022 at 5:03 AM Yang Zongze > wrote: Hi, I am solving a Linear system with LU factorization. But failed with the following error. Is there some suggestions on debugging this error? Thanks! This appears to be inside MUMPS. I would recommend two things: 1) Get a stack trace for this error using the debugger 2) With that, also send a report to the MUMPS mailing list Thanks, Matt The configure.log of PETSc is attached! ``` Assertion failed in file ./src/include/mpir_request.h at line 446: req != NULL /home/z2yang/opt/firedrake-env/firedrake-mini-petsc/src/petsc/default/lib/libmpi.so.12(+0x2e6072) [0x7fd51f801072] /home/z2yang/opt/firedrake-env/firedrake-mini-petsc/src/petsc/default/lib/libmpi.so.12(+0x20a8a8) [0x7fd51f7258a8] /home/z2yang/opt/firedrake-env/firedrake-mini-petsc/src/petsc/default/lib/libmpi.so.12(+0x2a0318) [0x7fd51f7bb318] /home/z2yang/opt/firedrake-env/firedrake-mini-petsc/src/petsc/default/lib/libmpi.so.12(MPI_Isend+0x498) [0x7fd51f600ae8] /home/z2yang/opt/firedrake-env/firedrake-mini-petsc/src/petsc/default/lib/libmpifort.so.12(PMPI_ISEND+0x51) [0x7fd51cf0acd1] /home/z2yang/opt/firedrake-env/firedrake-mini-petsc/src/petsc/default/lib/libpetsc.so.3.016(__dmumps_buf_MOD_dmumps_buf_send_contrib_type3+0x145b) [0x7fd520b1011d] /home/z2yang/opt/firedrake-env/firedrake-mini-petsc/src/petsc/default/lib/libpetsc.so.3.016(dmumps_buf_send_contrib_type3_i_+0xb8) [0x7fd520af584c] /home/z2yang/opt/firedrake-env/firedrake-mini-petsc/src/petsc/default/lib/libpetsc.so.3.016(dmumps_build_and_send_cb_root_+0x167e) [0x7fd520b6252c] /home/z2yang/opt/firedrake-env/firedrake-mini-petsc/src/petsc/default/lib/libpetsc.so.3.016(dmumps_fac_stack_+0xf00) [0x7fd520c60c88] /home/z2yang/opt/firedrake-env/firedrake-mini-petsc/src/petsc/default/lib/libpetsc.so.3.016(__dmumps_fac_par_m_MOD_dmumps_fac_par+0xade) [0x7fd520c56e27] /home/z2yang/opt/firedrake-env/firedrake-mini-petsc/src/petsc/default/lib/libpetsc.so.3.016(dmumps_fac_par_i_+0x216) [0x7fd520c000c2] /home/z2yang/opt/firedrake-env/firedrake-mini-petsc/src/petsc/default/lib/libpetsc.so.3.016(dmumps_fac_b_+0x975) [0x7fd520c00a3f] /home/z2yang/opt/firedrake-env/firedrake-mini-petsc/src/petsc/default/lib/libpetsc.so.3.016(dmumps_fac_driver_+0x5d2f) [0x7fd520bcd6cf] /home/z2yang/opt/firedrake-env/firedrake-mini-petsc/src/petsc/default/lib/libpetsc.so.3.016(dmumps_+0x2c6a) [0x7fd520b5c499] /home/z2yang/opt/firedrake-env/firedrake-mini-petsc/src/petsc/default/lib/libpetsc.so.3.016(dmumps_f77_+0x190b) [0x7fd520aef9f7] /home/z2yang/opt/firedrake-env/firedrake-mini-petsc/src/petsc/default/lib/libpetsc.so.3.016(dmumps_c+0xe10) [0x7fd520aedfb4] /home/z2yang/opt/firedrake-env/firedrake-mini-petsc/src/petsc/default/lib/libpetsc.so.3.016(+0x50d7a3) [0x7fd5200f77a3] /home/z2yang/opt/firedrake-env/firedrake-mini-petsc/src/petsc/default/lib/libpetsc.so.3.016(MatLUFactorNumeric+0x369) [0x7fd520272431] /home/z2yang/opt/firedrake-env/firedrake-mini-petsc/src/petsc/default/lib/libpetsc.so.3.016(+0xc67f73) [0x7fd520851f73] /home/z2yang/opt/firedrake-env/firedrake-mini-petsc/src/petsc/default/lib/libpetsc.so.3.016(PCSetUp+0x621) [0x7fd5208d0029] /home/z2yang/opt/firedrake-env/firedrake-mini-petsc/src/petsc/default/lib/libpetsc.so.3.016(KSPSetUp+0xd57) [0x7fd5206ffcf2] /home/z2yang/opt/firedrake-env/firedrake-mini-petsc/src/petsc/default/lib/libpetsc.so.3.016(+0xb17d4c) [0x7fd520701d4c] /home/z2yang/opt/firedrake-env/firedrake-mini-petsc/src/petsc/default/lib/libpetsc.so.3.016(KSPSolve+0x13) [0x7fd520704f13] /home/z2yang/opt/firedrake-env/firedrake-mini-petsc/lib64/python3.6/site-packages/petsc4py/lib/default/PETSc.cpython-36m-x86_64-linux-gnu.so(+0xe18eb) [0x7fd52169c8eb] /lib64/libpython3.6m.so.1.0(_PyCFunction_FastCallDict+0x353) [0x7fd5307a3a73] /lib64/libpython3.6m.so.1.0(+0x167dd0) [0x7fd5307a3dd0] /lib64/libpython3.6m.so.1.0(_PyEval_EvalFrameDefault+0x474) [0x7fd5307cc5b4] /lib64/libpython3.6m.so.1.0(PyEval_EvalCodeEx+0x337) [0x7fd5307ae727] /lib64/libpython3.6m.so.1.0(+0x17369e) [0x7fd5307af69e] /home/z2yang/opt/firedrake-env/firedrake-mini-petsc/lib64/python3.6/site-packages/petsc4py/lib/default/PETSc.cpython-36m-x86_64-linux-gnu.so(+0x71ed3) [0x7fd52162ced3] /home/z2yang/opt/firedrake-env/firedrake-mini-petsc/lib64/python3.6/site-packages/petsc4py/lib/default/PETSc.cpython-36m-x86_64-linux-gnu.so(+0xd75f6) [0x7fd5216925f6] /home/z2yang/opt/firedrake-env/firedrake-mini-petsc/lib64/python3.6/site-packages/petsc4py/lib/default/PETSc.cpython-36m-x86_64-linux-gnu.so(+0x717bf) [0x7fd52162c7bf] internal ABORT - process 0 ``` Best wishes, Zongze -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From FERRANJ2 at my.erau.edu Tue May 17 18:26:40 2022 From: FERRANJ2 at my.erau.edu (Ferrand, Jesus A.) Date: Tue, 17 May 2022 23:26:40 +0000 Subject: [petsc-users] DMPlex/PetscSF How to determine if local topology is other rank's ghost? Message-ID: Dear PETSc team: I am working with a non-overlapping distributed plex (i.e., when I call DMPlexDistribute(), I input overlap = 0), so only vertices and edges appear as ghosts to the local ranks. For preallocation of a parallel global stiffness matrix for FEA, I want to determine which locally owned vertices are ghosts to another rank. From reading the paper on PetscSF (https://ieeexplore.ieee.org/document/9442258) I think I can answer my question by inspecting the PetscSF returned by DMPlexDistribute() with PetscSFGetGraph(). I am just confused by the root/leaf and ilocal/iremote terminology. I read the manual page on PetscSFGetGraph() (https://petsc.org/release/docs/manualpages/PetscSF/PetscSFGetGraph.html) and that gave me the impression that I need to PetscSFBcast() the point IDs from foreign ranks to the local ones. Is this correct? [https://ieeexplore.ieee.org/assets/img/ieee_logo_smedia_200X200.png] The PetscSF Scalable Communication Layer | IEEE Journals & Magazine | IEEE Xplore PetscSF, the communication component of the Portable, Extensible Toolkit for Scientific Computation (PETSc), is designed to provide PETSc's communication infrastructure suitable for exascale computers that utilize GPUs and other accelerators. PetscSF provides a simple application programming interface (API) for managing common communication patterns in scientific computations by using a star ... ieeexplore.ieee.org ? Sincerely: J.A. Ferrand Embry-Riddle Aeronautical University - Daytona Beach FL M.Sc. Aerospace Engineering | May 2022 B.Sc. Aerospace Engineering B.Sc. Computational Mathematics Sigma Gamma Tau Tau Beta Pi Phone: (386)-843-1829 Email(s): ferranj2 at my.erau.edu jesus.ferrand at gmail.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From toby.isaac at gmail.com Tue May 17 23:46:56 2022 From: toby.isaac at gmail.com (Toby Isaac) Date: Wed, 18 May 2022 00:46:56 -0400 Subject: [petsc-users] DMPlex/PetscSF How to determine if local topology is other rank's ghost? In-Reply-To: References: Message-ID: A leaf point is attached to a root point (in a star forest there are only leaves and roots), so that means that a root point would be the point that owns a degree of freedom and a leaf point would have a ghost value. For a "point SF" of a DMPlex: - Each process has a local numbering of mesh points (cells + edges + faces + vertices): they are all potential roots, so the number of these is what is returned by `nroots`. - The number of ghost mesh points is `nleaves`. - `ilocal` would be a list of the mesh points that are leaves (using the local numbering). - For each leaf in `ilocal`, `iremote` describes the root it is attached to: which process it belongs to, and its id in *that* process's local numbering. If you're trying to create dof numberings on your own, please consider PetscSectionCreateGlobalSection: < https://petsc.org/main/docs/manualpages/PetscSection/PetscSectionCreateGlobalSection/>. You supply the PetscSF and a PetscSection which says how many dofs there are for each point and whether any have essential boundary conditions, and it computes a global PetscSection that tells you what the global id is for each dof on this process. On Tue, May 17, 2022 at 7:26 PM Ferrand, Jesus A. wrote: > Dear PETSc team: > > I am working with a non-overlapping distributed plex (i.e., when I call > DMPlexDistribute(), I input overlap = 0), so only vertices and edges appear > as ghosts to the local ranks. > For preallocation of a parallel global stiffness matrix for FEA, I want to > determine which locally owned vertices are ghosts to another rank. > > From reading the paper on PetscSF ( > https://ieeexplore.ieee.org/document/9442258) I think I can answer my > question by inspecting the PetscSF returned by DMPlexDistribute() with > PetscSFGetGraph(). I am just confused by the root/leaf and ilocal/iremote > terminology. > > I read the manual page on PetscSFGetGraph() ( > https://petsc.org/release/docs/manualpages/PetscSF/PetscSFGetGraph.html) > and that gave me the impression that I need to PetscSFBcast() the point IDs > from foreign ranks to the local ones. > > Is this correct? > > > > The PetscSF Scalable Communication Layer | IEEE Journals & Magazine | IEEE > Xplore > PetscSF, the communication component of the Portable, Extensible Toolkit > for Scientific Computation (PETSc), is designed to provide PETSc's > communication infrastructure suitable for exascale computers that utilize > GPUs and other accelerators. PetscSF provides a simple application > programming interface (API) for managing common communication patterns in > scientific computations by using a star ... > ieeexplore.ieee.org > *?* > > > > Sincerely: > > *J.A. Ferrand* > > Embry-Riddle Aeronautical University - Daytona Beach FL > > M.Sc. Aerospace Engineering | May 2022 > > B.Sc. Aerospace Engineering > > B.Sc. Computational Mathematics > > > > Sigma Gamma Tau > > Tau Beta Pi > > > > *Phone:* (386)-843-1829 > > *Email(s):* ferranj2 at my.erau.edu > > jesus.ferrand at gmail.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Wed May 18 01:09:27 2022 From: knepley at gmail.com (Matthew Knepley) Date: Tue, 17 May 2022 20:09:27 -1000 Subject: [petsc-users] DMPlex/PetscSF How to determine if local topology is other rank's ghost? In-Reply-To: References: Message-ID: On Tue, May 17, 2022 at 6:47 PM Toby Isaac wrote: > A leaf point is attached to a root point (in a star forest there are only > leaves and roots), so that means that a root point would be the point that > owns a degree of freedom and a leaf point would have a ghost value. > > For a "point SF" of a DMPlex: > > - Each process has a local numbering of mesh points (cells + edges + faces > + vertices): they are all potential roots, so the number of these is what > is returned by `nroots`. > > - The number of ghost mesh points is `nleaves`. > > - `ilocal` would be a list of the mesh points that are leaves (using the > local numbering). > > - For each leaf in `ilocal`, `iremote` describes the root it is attached > to: which process it belongs to, and its id in *that* process's local > numbering. > > If you're trying to create dof numberings on your own, please consider > PetscSectionCreateGlobalSection: < > https://petsc.org/main/docs/manualpages/PetscSection/PetscSectionCreateGlobalSection/>. > You supply the PetscSF and a PetscSection which says how many dofs there > are for each point and whether any have essential boundary conditions, and > it computes a global PetscSection that tells you what the global id is for > each dof on this process. > Toby is exactly right. Also, if you want global numbering of points you can use https://petsc.org/main/docs/manualpages/DMPLEX/DMPlexCreatePointNumbering/ and there is a similar thing for jsut cells or vertices. Thanks, Matt > On Tue, May 17, 2022 at 7:26 PM Ferrand, Jesus A. > wrote: > >> Dear PETSc team: >> >> I am working with a non-overlapping distributed plex (i.e., when I call >> DMPlexDistribute(), I input overlap = 0), so only vertices and edges appear >> as ghosts to the local ranks. >> For preallocation of a parallel global stiffness matrix for FEA, I want >> to determine which locally owned vertices are ghosts to another rank. >> >> From reading the paper on PetscSF ( >> https://ieeexplore.ieee.org/document/9442258) I think I can answer my >> question by inspecting the PetscSF returned by DMPlexDistribute() with >> PetscSFGetGraph(). I am just confused by the root/leaf and ilocal/iremote >> terminology. >> >> I read the manual page on PetscSFGetGraph() ( >> https://petsc.org/release/docs/manualpages/PetscSF/PetscSFGetGraph.html) >> and that gave me the impression that I need to PetscSFBcast() the point IDs >> from foreign ranks to the local ones. >> >> Is this correct? >> >> >> >> The PetscSF Scalable Communication Layer | IEEE Journals & Magazine | >> IEEE Xplore >> PetscSF, the communication component of the Portable, Extensible Toolkit >> for Scientific Computation (PETSc), is designed to provide PETSc's >> communication infrastructure suitable for exascale computers that utilize >> GPUs and other accelerators. PetscSF provides a simple application >> programming interface (API) for managing common communication patterns in >> scientific computations by using a star ... >> ieeexplore.ieee.org >> *?* >> >> >> >> Sincerely: >> >> *J.A. Ferrand* >> >> Embry-Riddle Aeronautical University - Daytona Beach FL >> >> M.Sc. Aerospace Engineering | May 2022 >> >> B.Sc. Aerospace Engineering >> >> B.Sc. Computational Mathematics >> >> >> >> Sigma Gamma Tau >> >> Tau Beta Pi >> >> >> >> *Phone:* (386)-843-1829 >> >> *Email(s):* ferranj2 at my.erau.edu >> >> jesus.ferrand at gmail.com >> > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From zhangsijie at gatech.edu Wed May 18 22:49:38 2022 From: zhangsijie at gatech.edu (Zhang, Sijie) Date: Thu, 19 May 2022 03:49:38 +0000 Subject: [petsc-users] Error when using DMSTAG_UP for array indexing Message-ID: Hi, I got an error when using DMSTAG_UP for DMStagVec array indexing in a 2D DMStag Mesh. I used the following code for accessing the getting the index for local vector data stored at the left and up locations (edge). PetscInt idFup[4], dFleft[4]; for (ii = 0; ii < 4; ii++) DMStagGetLocationSlot(ctx->dmF, LEFT, ii, &idFleft[ii]); for (ii = 0; ii < 4; ii++) DMStagGetLocationSlot(ctx->dmF, RIGHT, ii, &idFright[ii]); And I used the following sample code for getting the data: a = UArr[ey][ex][idFright[ii]]) However, after the calculation, I found there are weird fluctuations at the boundary of the mesh blocks assigned to each processor as shown in the Figure below.(The yellow lines, 5 processors were used in this case.) There seemed to be a communication error when using DMSTAG_UP. And I also found the value of idFright[ii] was much larger than the index for the locations of left, right, or elements. It's on a level of 400+, while the other was only like 1 to 10. Was there any I did wrong when setting up the 2D DMStag? Thank you and best regards. Sijie Zhang [cid:94a295bd-6851-489d-83d8-44d2a6d80314] -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 20901 bytes Desc: image.png URL: From aduarteg at utexas.edu Thu May 19 12:27:12 2022 From: aduarteg at utexas.edu (Alfredo J Duarte Gomez) Date: Thu, 19 May 2022 12:27:12 -0500 Subject: [petsc-users] Solver/Preconditioner suggestions Message-ID: Good afternoon PETSC users, I am looking for some suggestions on preconditioners/solvers. Currently, I have a custom preconditioner that solves 4 independent systems, let's call them A,B,C, and D. A is an advective, diffusive, reactive system, although due to some coefficient it is the system with the highest condition number and therefore the most difficult to solve. B and C, are more "standard" advective, diffusion, reactive systems. The condition number is not as high as A. D is simply the laplacian used to solve an elliptic Poisson equation. For more context, A,B, and C need to be recomputed about once every time step, while D is a one time cost. The problem is 2-D, sizes are on the order of 1-10 million grid points, and I am using a structured grid. These usually run on somewhere between 100-400 processors. Currently I am solving A,B, and C with the HYPRE Euclid algorithm ILU(1), and D is solved with the direct solver MUMPS. While these were very useful to get the code working, I am now trying to get better parallel scaling/efficiencies and performance. HYPRE euclid does not seem to scale super well beyond like 60 procs, and MUMPS has very large memory requirements. Does anyone have suggestions on more scalable ILU algorithms for A,B, and C, or any other good alternatives? >From what I have read, multigrid methods are probably the best alternative for D, but I have very little experience with these and they seem to require a lot of parameters. Does anyone have pointers on a good setup for a multigrid preconditioner? Thank you and have a good day, -Alfredo -- Alfredo Duarte Graduate Research Assistant The University of Texas at Austin -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Thu May 19 12:30:52 2022 From: knepley at gmail.com (Matthew Knepley) Date: Thu, 19 May 2022 07:30:52 -1000 Subject: [petsc-users] Solver/Preconditioner suggestions In-Reply-To: References: Message-ID: On Thu, May 19, 2022 at 7:27 AM Alfredo J Duarte Gomez wrote: > Good afternoon PETSC users, > > I am looking for some suggestions on preconditioners/solvers. > > Currently, I have a custom preconditioner that solves 4 independent > systems, let's call them A,B,C, and D. > > A is an advective, diffusive, reactive system, although due to some > coefficient it is the system with the highest condition number and > therefore the most difficult to solve. > > B and C, are more "standard" advective, diffusion, reactive systems. The > condition number is not as high as A. > > D is simply the laplacian used to solve an elliptic Poisson equation. > > For more context, A,B, and C need to be recomputed about once every time > step, while D is a one time cost. > > The problem is 2-D, sizes are on the order of 1-10 million grid points, > and I am using a structured grid. These usually run on somewhere between > 100-400 processors. > > Currently I am solving A,B, and C with the HYPRE Euclid algorithm ILU(1), > and D is solved with the direct solver MUMPS. > > While these were very useful to get the code working, I am now trying to > get better parallel scaling/efficiencies and performance. HYPRE euclid does > not seem to scale super well beyond like 60 procs, and MUMPS has very large > memory requirements. > > Does anyone have suggestions on more scalable ILU algorithms for A,B, and > C, or any other good alternatives? > Are these advectively dominated? > From what I have read, multigrid methods are probably the best alternative > for D, but I have very little experience with these and they seem to > require a lot of parameters. Does anyone have pointers on a good setup for > a multigrid preconditioner? > This is the easy one. If you are using DMDA, just turn on -pc_type mg and give a number of levels and it should be fine. If not, then use GAMG and it should be fine, You could also use Hypre Boomeramg for this since it is optimized for the 2D Laplacian. Thanks, Matt > Thank you and have a good day, > > -Alfredo > > -- > Alfredo Duarte > Graduate Research Assistant > The University of Texas at Austin > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From aduarteg at utexas.edu Thu May 19 12:59:55 2022 From: aduarteg at utexas.edu (Alfredo J Duarte Gomez) Date: Thu, 19 May 2022 12:59:55 -0500 Subject: [petsc-users] Solver/Preconditioner suggestions In-Reply-To: References: Message-ID: Hello Matthew, Thank you for your suggestion on the Laplace solver. About the other systems I would say B and C are mostly dominated by advection, except for diffusive layers at the boundaries. A is a bit more difficult to judge, although it can be expected to have a very large advective component, it also has a large reactive component, and the diffusion dominated layers at the boundaries. Thank you, -Alfredo On Thu, May 19, 2022 at 12:31 PM Matthew Knepley wrote: > On Thu, May 19, 2022 at 7:27 AM Alfredo J Duarte Gomez < > aduarteg at utexas.edu> wrote: > >> Good afternoon PETSC users, >> >> I am looking for some suggestions on preconditioners/solvers. >> >> Currently, I have a custom preconditioner that solves 4 independent >> systems, let's call them A,B,C, and D. >> >> A is an advective, diffusive, reactive system, although due to some >> coefficient it is the system with the highest condition number and >> therefore the most difficult to solve. >> >> B and C, are more "standard" advective, diffusion, reactive systems. The >> condition number is not as high as A. >> >> D is simply the laplacian used to solve an elliptic Poisson equation. >> >> For more context, A,B, and C need to be recomputed about once every time >> step, while D is a one time cost. >> >> The problem is 2-D, sizes are on the order of 1-10 million grid points, >> and I am using a structured grid. These usually run on somewhere between >> 100-400 processors. >> >> Currently I am solving A,B, and C with the HYPRE Euclid algorithm >> ILU(1), and D is solved with the direct solver MUMPS. >> >> While these were very useful to get the code working, I am now trying to >> get better parallel scaling/efficiencies and performance. HYPRE euclid does >> not seem to scale super well beyond like 60 procs, and MUMPS has very large >> memory requirements. >> >> Does anyone have suggestions on more scalable ILU algorithms for A,B, and >> C, or any other good alternatives? >> > > Are these advectively dominated? > > >> From what I have read, multigrid methods are probably the best >> alternative for D, but I have very little experience with these and they >> seem to require a lot of parameters. Does anyone have pointers on a good >> setup for a multigrid preconditioner? >> > > This is the easy one. If you are using DMDA, just turn on -pc_type mg and > give a number of levels and it should be fine. If not, then use GAMG and it > should be fine, You could also use Hypre Boomeramg for this since it is > optimized for the 2D Laplacian. > > Thanks, > > Matt > > >> Thank you and have a good day, >> >> -Alfredo >> >> -- >> Alfredo Duarte >> Graduate Research Assistant >> The University of Texas at Austin >> > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > > https://www.cse.buffalo.edu/~knepley/ > > -- Alfredo Duarte Graduate Research Assistant The University of Texas at Austin -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Thu May 19 13:02:57 2022 From: knepley at gmail.com (Matthew Knepley) Date: Thu, 19 May 2022 08:02:57 -1000 Subject: [petsc-users] Solver/Preconditioner suggestions In-Reply-To: References: Message-ID: On Thu, May 19, 2022 at 8:00 AM Alfredo J Duarte Gomez wrote: > Hello Matthew, > > Thank you for your suggestion on the Laplace solver. > > About the other systems I would say B and C are mostly dominated by > advection, except for diffusive layers at the boundaries. > > A is a bit more difficult to judge, although it can be expected to have a > very large advective component, it also has a large reactive component, and > the diffusion dominated layers at the boundaries. > Is your mesh resolving the boundary layers? Thanks, Matt > Thank you, > > -Alfredo > > On Thu, May 19, 2022 at 12:31 PM Matthew Knepley > wrote: > >> On Thu, May 19, 2022 at 7:27 AM Alfredo J Duarte Gomez < >> aduarteg at utexas.edu> wrote: >> >>> Good afternoon PETSC users, >>> >>> I am looking for some suggestions on preconditioners/solvers. >>> >>> Currently, I have a custom preconditioner that solves 4 independent >>> systems, let's call them A,B,C, and D. >>> >>> A is an advective, diffusive, reactive system, although due to some >>> coefficient it is the system with the highest condition number and >>> therefore the most difficult to solve. >>> >>> B and C, are more "standard" advective, diffusion, reactive systems. The >>> condition number is not as high as A. >>> >>> D is simply the laplacian used to solve an elliptic Poisson equation. >>> >>> For more context, A,B, and C need to be recomputed about once every time >>> step, while D is a one time cost. >>> >>> The problem is 2-D, sizes are on the order of 1-10 million grid points, >>> and I am using a structured grid. These usually run on somewhere between >>> 100-400 processors. >>> >>> Currently I am solving A,B, and C with the HYPRE Euclid algorithm >>> ILU(1), and D is solved with the direct solver MUMPS. >>> >>> While these were very useful to get the code working, I am now trying to >>> get better parallel scaling/efficiencies and performance. HYPRE euclid does >>> not seem to scale super well beyond like 60 procs, and MUMPS has very large >>> memory requirements. >>> >>> Does anyone have suggestions on more scalable ILU algorithms for A,B, >>> and C, or any other good alternatives? >>> >> >> Are these advectively dominated? >> >> >>> From what I have read, multigrid methods are probably the best >>> alternative for D, but I have very little experience with these and they >>> seem to require a lot of parameters. Does anyone have pointers on a good >>> setup for a multigrid preconditioner? >>> >> >> This is the easy one. If you are using DMDA, just turn on -pc_type mg and >> give a number of levels and it should be fine. If not, then use GAMG and it >> should be fine, You could also use Hypre Boomeramg for this since it is >> optimized for the 2D Laplacian. >> >> Thanks, >> >> Matt >> >> >>> Thank you and have a good day, >>> >>> -Alfredo >>> >>> -- >>> Alfredo Duarte >>> Graduate Research Assistant >>> The University of Texas at Austin >>> >> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which their >> experiments lead. >> -- Norbert Wiener >> >> https://www.cse.buffalo.edu/~knepley/ >> >> > > > -- > Alfredo Duarte > Graduate Research Assistant > The University of Texas at Austin > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From aduarteg at utexas.edu Thu May 19 13:08:43 2022 From: aduarteg at utexas.edu (Alfredo J Duarte Gomez) Date: Thu, 19 May 2022 13:08:43 -0500 Subject: [petsc-users] Solver/Preconditioner suggestions In-Reply-To: References: Message-ID: Hello, Yes, the boundary layer is important to the problem and is resolved by the mesh, although it is a very small part of the mesh. Thanks, -Alfredo On Thu, May 19, 2022 at 1:03 PM Matthew Knepley wrote: > On Thu, May 19, 2022 at 8:00 AM Alfredo J Duarte Gomez < > aduarteg at utexas.edu> wrote: > >> Hello Matthew, >> >> Thank you for your suggestion on the Laplace solver. >> >> About the other systems I would say B and C are mostly dominated by >> advection, except for diffusive layers at the boundaries. >> >> A is a bit more difficult to judge, although it can be expected to have a >> very large advective component, it also has a large reactive component, and >> the diffusion dominated layers at the boundaries. >> > > Is your mesh resolving the boundary layers? > > Thanks, > > Matt > > >> Thank you, >> >> -Alfredo >> >> On Thu, May 19, 2022 at 12:31 PM Matthew Knepley >> wrote: >> >>> On Thu, May 19, 2022 at 7:27 AM Alfredo J Duarte Gomez < >>> aduarteg at utexas.edu> wrote: >>> >>>> Good afternoon PETSC users, >>>> >>>> I am looking for some suggestions on preconditioners/solvers. >>>> >>>> Currently, I have a custom preconditioner that solves 4 independent >>>> systems, let's call them A,B,C, and D. >>>> >>>> A is an advective, diffusive, reactive system, although due to some >>>> coefficient it is the system with the highest condition number and >>>> therefore the most difficult to solve. >>>> >>>> B and C, are more "standard" advective, diffusion, reactive systems. >>>> The condition number is not as high as A. >>>> >>>> D is simply the laplacian used to solve an elliptic Poisson equation. >>>> >>>> For more context, A,B, and C need to be recomputed about once every >>>> time step, while D is a one time cost. >>>> >>>> The problem is 2-D, sizes are on the order of 1-10 million grid points, >>>> and I am using a structured grid. These usually run on somewhere between >>>> 100-400 processors. >>>> >>>> Currently I am solving A,B, and C with the HYPRE Euclid algorithm >>>> ILU(1), and D is solved with the direct solver MUMPS. >>>> >>>> While these were very useful to get the code working, I am now trying >>>> to get better parallel scaling/efficiencies and performance. HYPRE euclid >>>> does not seem to scale super well beyond like 60 procs, and MUMPS has very >>>> large memory requirements. >>>> >>>> Does anyone have suggestions on more scalable ILU algorithms for A,B, >>>> and C, or any other good alternatives? >>>> >>> >>> Are these advectively dominated? >>> >>> >>>> From what I have read, multigrid methods are probably the best >>>> alternative for D, but I have very little experience with these and they >>>> seem to require a lot of parameters. Does anyone have pointers on a good >>>> setup for a multigrid preconditioner? >>>> >>> >>> This is the easy one. If you are using DMDA, just turn on -pc_type mg >>> and give a number of levels and it should be fine. If not, then use GAMG >>> and it >>> should be fine, You could also use Hypre Boomeramg for this since it is >>> optimized for the 2D Laplacian. >>> >>> Thanks, >>> >>> Matt >>> >>> >>>> Thank you and have a good day, >>>> >>>> -Alfredo >>>> >>>> -- >>>> Alfredo Duarte >>>> Graduate Research Assistant >>>> The University of Texas at Austin >>>> >>> >>> >>> -- >>> What most experimenters take for granted before they begin their >>> experiments is infinitely more interesting than any results to which their >>> experiments lead. >>> -- Norbert Wiener >>> >>> https://www.cse.buffalo.edu/~knepley/ >>> >>> >> >> >> -- >> Alfredo Duarte >> Graduate Research Assistant >> The University of Texas at Austin >> > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > > https://www.cse.buffalo.edu/~knepley/ > > -- Alfredo Duarte Graduate Research Assistant The University of Texas at Austin -------------- next part -------------- An HTML attachment was scrubbed... URL: From EPrudencio at slb.com Fri May 20 12:23:21 2022 From: EPrudencio at slb.com (Ernesto Prudencio) Date: Fri, 20 May 2022 17:23:21 +0000 Subject: [petsc-users] Very slow VecDot operations Message-ID: I am using LSQR to minimize || L x - b ||_2, where L is a sparse rectangular matrix with 145,253,395 rows, 209,423,775 columns, and around 54 billion non zeros. The numbers reported below are for a run with 27 compute nodes, each compute node with 4 MPI ranks, so a total of 108 ranks. Throughout the run, I assess the runtime taken by all dot products during the LSQR iterations, and I differentiate between dot products involving vectors of the size of the solution vector "x", and dot products involving vectors of the size of the rhs "b". Here are the numbers I get (we have an implementation of LSQR that performs some extra vector dot products for our needs): 236 VecDotSol take 1.523 seconds 226 VecDotRhs take 326.008 seconds Regarding the partition of rows and columns among the 108 MPI ranks: Rows: min = 838,529 ; avg = 1.34494e+06 ; max = 2,437,206 Columns: min = 1,903,500 ; avg = 1.93911e+06 ; max = 1,946,270 Regarding the partition of rows and columns among the 27 compute nodes: Rows: min = 3,575,584 ; avg = 5.37976e+06 ; max = 8,788,062 Columns: min = 7,637,500 ; avg = 7.75644e+06 ; max = 7,785,080 Questions: 1. Why the average run times are so different between VecDotSol and VecDotRhs? 2. Could the much bigger unbalancing among the number of rows per rank (compared to the very well balanced distribution of columns per rank) be the cause? 3. Have you ever observed such situation? 4. Could it be because of a bad MPI configuration / parametrization with respect to the underlying network? 5. But, if yes, why the VecDotSol dot products are so much faster than VecDotRhs? Thank you in advance, Ernesto. Schlumberger-Private -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at petsc.dev Fri May 20 12:32:58 2022 From: bsmith at petsc.dev (Barry Smith) Date: Fri, 20 May 2022 13:32:58 -0400 Subject: [petsc-users] Very slow VecDot operations In-Reply-To: References: Message-ID: <0427963F-87B2-401B-A798-BD499D1D1A3B@petsc.dev> Ernesto, If you ran (or can run) with -log_view you could see the time "ratio" in the output that tells how much time the "fastest" rank spent on the dot product versus the "slowest". Based on the different counts per rank you report that ratio might be around 3. But based on the times you report is around 200! My guess is that for the VecDotRhs() some ranks are arriving at the vec dot long before other ranks and have to wait there an extremely long amount of time making it appear that the dot product is very slow. While, in reality, the large time credited to the vecdot is due to a misbalance in time for the operation before the VecDot. Barry > On May 20, 2022, at 1:23 PM, Ernesto Prudencio via petsc-users wrote: > > I am using LSQR to minimize || L x ? b ||_2, where L is a sparse rectangular matrix with 145,253,395 rows, 209,423,775 columns, and around 54 billion non zeros. > > The numbers reported below are for a run with 27 compute nodes, each compute node with 4 MPI ranks, so a total of 108 ranks. > > Throughout the run, I assess the runtime taken by all dot products during the LSQR iterations, and I differentiate between dot products involving vectors of the size of the solution vector ?x?, and dot products involving vectors of the size of the rhs ?b?. Here are the numbers I get (we have an implementation of LSQR that performs some extra vector dot products for our needs): > > 236 VecDotSol take 1.523 seconds > 226 VecDotRhs take 326.008 seconds > > Regarding the partition of rows and columns among the 108 MPI ranks: > > Rows: min = 838,529 ; avg = 1.34494e+06 ; max = 2,437,206 > Columns: min = 1,903,500 ; avg = 1.93911e+06 ; max = 1,946,270 > > Regarding the partition of rows and columns among the 27 compute nodes: > > Rows: min = 3,575,584 ; avg = 5.37976e+06 ; max = 8,788,062 > Columns: min = 7,637,500 ; avg = 7.75644e+06 ; max = 7,785,080 > > Questions: > Why the average run times are so different between VecDotSol and VecDotRhs? > Could the much bigger unbalancing among the number of rows per rank (compared to the very well balanced distribution of columns per rank) be the cause? > Have you ever observed such situation? > Could it be because of a bad MPI configuration / parametrization with respect to the underlying network? > But, if yes, why the VecDotSol dot products are so much faster than VecDotRhs? > > Thank you in advance, > > Ernesto. > > > Schlumberger-Private -------------- next part -------------- An HTML attachment was scrubbed... URL: From EPrudencio at slb.com Fri May 20 12:37:02 2022 From: EPrudencio at slb.com (Ernesto Prudencio) Date: Fri, 20 May 2022 17:37:02 +0000 Subject: [petsc-users] [Ext] Re: Very slow VecDot operations In-Reply-To: <0427963F-87B2-401B-A798-BD499D1D1A3B@petsc.dev> References: <0427963F-87B2-401B-A798-BD499D1D1A3B@petsc.dev> Message-ID: Thank you, Barry. I will dig more on the issue with your suggestions. Schlumberger-Private From: Barry Smith Sent: Friday, May 20, 2022 12:33 PM To: Ernesto Prudencio Cc: PETSc users list Subject: [Ext] Re: [petsc-users] Very slow VecDot operations Ernesto, If you ran (or can run) with -log_view you could see the time "ratio" in the output that tells how much time the "fastest" rank spent on the dot product versus the "slowest". Based on the different counts per rank you report that ratio might be around 3. But based on the times you report is around 200! My guess is that for the VecDotRhs() some ranks are arriving at the vec dot long before other ranks and have to wait there an extremely long amount of time making it appear that the dot product is very slow. While, in reality, the large time credited to the vecdot is due to a misbalance in time for the operation before the VecDot. Barry On May 20, 2022, at 1:23 PM, Ernesto Prudencio via petsc-users > wrote: I am using LSQR to minimize || L x - b ||_2, where L is a sparse rectangular matrix with 145,253,395 rows, 209,423,775 columns, and around 54 billion non zeros. The numbers reported below are for a run with 27 compute nodes, each compute node with 4 MPI ranks, so a total of 108 ranks. Throughout the run, I assess the runtime taken by all dot products during the LSQR iterations, and I differentiate between dot products involving vectors of the size of the solution vector "x", and dot products involving vectors of the size of the rhs "b". Here are the numbers I get (we have an implementation of LSQR that performs some extra vector dot products for our needs): 236 VecDotSol take 1.523 seconds 226 VecDotRhs take 326.008 seconds Regarding the partition of rows and columns among the 108 MPI ranks: Rows: min = 838,529 ; avg = 1.34494e+06 ; max = 2,437,206 Columns: min = 1,903,500 ; avg = 1.93911e+06 ; max = 1,946,270 Regarding the partition of rows and columns among the 27 compute nodes: Rows: min = 3,575,584 ; avg = 5.37976e+06 ; max = 8,788,062 Columns: min = 7,637,500 ; avg = 7.75644e+06 ; max = 7,785,080 Questions: 1. Why the average run times are so different between VecDotSol and VecDotRhs? 2. Could the much bigger unbalancing among the number of rows per rank (compared to the very well balanced distribution of columns per rank) be the cause? 3. Have you ever observed such situation? 4. Could it be because of a bad MPI configuration / parametrization with respect to the underlying network? 5. But, if yes, why the VecDotSol dot products are so much faster than VecDotRhs? Thank you in advance, Ernesto. Schlumberger-Private -------------- next part -------------- An HTML attachment was scrubbed... URL: From jeremy at seamplex.com Fri May 20 13:39:27 2022 From: jeremy at seamplex.com (Jeremy Theler) Date: Fri, 20 May 2022 15:39:27 -0300 Subject: [petsc-users] API cal to set mg_levels_pc_type Message-ID: <9f4ae46bc6412181f5286cdbae8d1f1cbd2f20c5.camel@seamplex.com> The default smoothing PC changed from sor to jacobi in 3.17. The Changelog says the old behavior can be recovered by using -mg_levels_pc_type sor. 1. Is there a way to set the mg_levels_pc_type via an API call? 2. Are there any changes in efficiency expected with this new PC? Regards -- jeremy theler From bsmith at petsc.dev Fri May 20 13:48:53 2022 From: bsmith at petsc.dev (Barry Smith) Date: Fri, 20 May 2022 14:48:53 -0400 Subject: [petsc-users] API cal to set mg_levels_pc_type In-Reply-To: <9f4ae46bc6412181f5286cdbae8d1f1cbd2f20c5.camel@seamplex.com> References: <9f4ae46bc6412181f5286cdbae8d1f1cbd2f20c5.camel@seamplex.com> Message-ID: <8401A708-0646-47F5-B07F-83749C0ECF26@petsc.dev> > On May 20, 2022, at 2:39 PM, Jeremy Theler wrote: > > The default smoothing PC changed from sor to jacobi in 3.17. Note that this is only for GAMG, it is not for geometric multigrid (using PCMG directly). > The > Changelog says the old behavior can be recovered by using > -mg_levels_pc_type sor. The easiest is just doing PetscOptionsSetValue(NULL,"-mg_levels_pc_type","sor"); Unfortunately, there is no good way to set it with calls directly on the KSP or MG object. One needs to loop through the levels and set it for the PC on each level. > > 1. Is there a way to set the mg_levels_pc_type via an API call? > 2. Are there any changes in efficiency expected with this new PC? This was changed mainly because PCSOR is not effective (currently in PETSc) for GPUs. For CPUs it will definitely be problem dependent, you need to try both smoothers at scale to determine the most effective. Barry > > > Regards > -- > jeremy theler > From jed at jedbrown.org Fri May 20 14:10:23 2022 From: jed at jedbrown.org (Jed Brown) Date: Fri, 20 May 2022 13:10:23 -0600 Subject: [petsc-users] API cal to set mg_levels_pc_type In-Reply-To: <8401A708-0646-47F5-B07F-83749C0ECF26@petsc.dev> References: <9f4ae46bc6412181f5286cdbae8d1f1cbd2f20c5.camel@seamplex.com> <8401A708-0646-47F5-B07F-83749C0ECF26@petsc.dev> Message-ID: <87bkvs10m8.fsf@jedbrown.org> Barry Smith writes: >> 1. Is there a way to set the mg_levels_pc_type via an API call? >> 2. Are there any changes in efficiency expected with this new PC? > > This was changed mainly because PCSOR is not effective (currently in PETSc) for GPUs. For CPUs > it will definitely be problem dependent, you need to try both smoothers at scale to determine the most effective. When running on CPU for elliptic problems, -mg_levels_ksp_type richardson -mg_levels_ksp_max_it 1 -mg_levels_pc_type sor is often a good choice. The new default (Chebyshev/Jacobi) is efficient on both CPU and GPU. The setup cost for this configuration is lower in 3.17 because the GAMG setup needs the same eigenvalue estimates. From junchao.zhang at gmail.com Fri May 20 15:03:44 2022 From: junchao.zhang at gmail.com (Junchao Zhang) Date: Fri, 20 May 2022 15:03:44 -0500 Subject: [petsc-users] [Ext] Re: Very slow VecDot operations In-Reply-To: References: <0427963F-87B2-401B-A798-BD499D1D1A3B@petsc.dev> Message-ID: You can also use -log_view -log_sync to sync before timing so that you can clearly see which operations are really imbalanced. --Junchao Zhang On Fri, May 20, 2022 at 12:37 PM Ernesto Prudencio via petsc-users < petsc-users at mcs.anl.gov> wrote: > Thank you, Barry. I will dig more on the issue with your suggestions. > > > > > > Schlumberger-Private > > *From:* Barry Smith > *Sent:* Friday, May 20, 2022 12:33 PM > *To:* Ernesto Prudencio > *Cc:* PETSc users list > *Subject:* [Ext] Re: [petsc-users] Very slow VecDot operations > > > > > > Ernesto, > > > > If you ran (or can run) with -log_view you could see the time "ratio" > in the output that tells how much time the "fastest" rank spent on the dot > product versus the "slowest". Based on the different counts per rank you > report that ratio might be around 3. But based on the times you report is > around 200! > > > > My guess is that for the VecDotRhs() some ranks are arriving at the > vec dot long before other ranks and have to wait there an extremely long > amount of time making it appear that the dot product is very slow. While, > in reality, the large time credited to the vecdot is due to a misbalance in > time for the operation before the VecDot. > > > > Barry > > > > > > On May 20, 2022, at 1:23 PM, Ernesto Prudencio via petsc-users < > petsc-users at mcs.anl.gov> wrote: > > > > I am using LSQR to minimize || L x ? b ||_2, where L is a sparse > rectangular matrix with 145,253,395 rows, 209,423,775 columns, and around > 54 billion non zeros. > > > > The numbers reported below are for a run with 27 compute nodes, each > compute node with 4 MPI ranks, so a total of 108 ranks. > > > > Throughout the run, I assess the runtime taken by all dot products during > the LSQR iterations, and I differentiate between dot products involving > vectors of the size of the solution vector ?x?, and dot products involving > vectors of the size of the rhs ?b?. Here are the numbers I get (we have an > implementation of LSQR that performs some extra vector dot products for our > needs): > > > > 236 VecDotSol take 1.523 seconds > > 226 VecDotRhs take 326.008 seconds > > > > Regarding the partition of rows and columns among the 108 MPI ranks: > > > > Rows: min = 838,529 ; avg = 1.34494e+06 ; max = 2,437,206 > > Columns: min = 1,903,500 ; avg = 1.93911e+06 ; max = 1,946,270 > > > > Regarding the partition of rows and columns among the 27 compute nodes: > > > > Rows: min = 3,575,584 ; avg = 5.37976e+06 ; max = 8,788,062 > > Columns: min = 7,637,500 ; avg = 7.75644e+06 ; max = 7,785,080 > > > > Questions: > > 1. Why the average run times are so different between VecDotSol and > VecDotRhs? > 2. Could the much bigger unbalancing among the number of rows per rank > (compared to the very well balanced distribution of columns per rank) be > the cause? > 3. Have you ever observed such situation? > 4. Could it be because of a bad MPI configuration / parametrization > with respect to the underlying network? > 5. But, if yes, why the VecDotSol dot products are so much faster than > VecDotRhs? > > > > Thank you in advance, > > > > Ernesto. > > > > > > Schlumberger-Private > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at petsc.dev Fri May 20 15:11:29 2022 From: bsmith at petsc.dev (Barry Smith) Date: Fri, 20 May 2022 16:11:29 -0400 Subject: [petsc-users] API cal to set mg_levels_pc_type In-Reply-To: <5eda8d4b40b865d3c570ca7465135e73fa43e572.camel@seamplex.com> References: <9f4ae46bc6412181f5286cdbae8d1f1cbd2f20c5.camel@seamplex.com> <8401A708-0646-47F5-B07F-83749C0ECF26@petsc.dev> <5eda8d4b40b865d3c570ca7465135e73fa43e572.camel@seamplex.com> Message-ID: <4A00F64F-7D7E-4C6C-8832-2605683021CF@petsc.dev> > Yes, this is what I did. > But I have a function that sets up the preconditioner by taking a KSP > as an argument. Depending on the problem being solved, it can be a > single KSP or a sub-KSP coming from a field split (which if there are > lagrange multipliers, can be further split again). So I was looking for > a call that would work on the KSP passed as an argument. The option > string won't work in the general field-split case. You can provide multiple options with PetscOptionsSetValue() for example >> PetscOptionsSetValue(NULL,"-mg_levels_pc_type","sor"); >> PetscOptionsSetValue(NULL,"-fieldsplit_0_mg_levels_pc_type","sor"); etc and any "extra" ones not appropriate for your particular case will just be ignored. > On May 20, 2022, at 3:25 PM, Jeremy Theler wrote: > > On Fri, 2022-05-20 at 14:48 -0400, Barry Smith wrote: >>> >>> The default smoothing PC changed from sor to jacobi in 3.17. >> >> Note that this is only for GAMG, it is not for geometric multigrid >> (using PCMG directly). > > Indeed, I was asking about GAMG specifically. > >> >>> The >>> Changelog says the old behavior can be recovered by using >>> -mg_levels_pc_type sor. >> >> The easiest is just doing >> >> PetscOptionsSetValue(NULL,"-mg_levels_pc_type","sor"); > > Yes, this is what I did. > But I have a function that sets up the preconditioner by taking a KSP > as an argument. Depending on the problem being solved, it can be a > single KSP or a sub-KSP coming from a field split (which if there are > lagrange multipliers, can be further split again). So I was looking for > a call that would work on the KSP passed as an argument. The option > string won't work in the general field-split case. > >> >> Unfortunately, there is no good way to set it with calls directly >> on the KSP or MG object. One >> needs to loop through the levels and set it for the PC on each level. > > Maybe this is doable. I'll take a stab with this approach. > Thanks Barry. > > -- > jeremy From jgaray at cct.lsu.edu Fri May 20 15:52:08 2022 From: jgaray at cct.lsu.edu (jgaray) Date: Fri, 20 May 2022 15:52:08 -0500 Subject: [petsc-users] Problem with MUMPS package when running ex1 from src/tao/constrained/tutorials Message-ID: <91765358ec2bf3434ce265c56418cda9@cct.lsu.edu> Hello PETSc users, I have been trying to run example ex1 from src/tao/constrained/tutorials, which requires the use of the MUMPS package. I have downloaded MUMPS and configured PETSc with it using ./configure --download-make --download-cmake --download-mumps --download-metis --download-parmetis --download-scalapack then I have built the PETSc libraries and checked that they were working properly. But when I run ex1 with more than one MPI rank, I get the following error [0]PETSC ERROR: --------------------- Error Message -------------------------------------------------------------- [0]PETSC ERROR: No support for this operation for this object type [0]PETSC ERROR: Requires an external package that supports parallel PCCHOLESKY, e.g., MUMPS. [0]PETSC ERROR: See https://petsc.org/release/faq/ for trouble shooting. [0]PETSC ERROR: Petsc Release Version 3.17.1, unknown [0]PETSC ERROR: ./ex1 on a arch-linux-c-debug named smic104 by jgaray Fri May 20 14:51:05 2022 [0]PETSC ERROR: Configure options --download-make --download-cmake --download-mumps --download-metis --download-parmetis --download-scalapack [0]PETSC ERROR: #1 main() at ex1.c:100 [0]PETSC ERROR: No PETSc Option Table entries [0]PETSC ERROR: ----------------End of Error Message -------send entire error message to petsc-maint at mcs.anl.gov---------- Does anyone have an idea of why this is happening? Also, I tried to download MUMPS together with PTScotch using ./configure --download-make --download-cmake --download-mumps --download-metis --download-parmetis --download-scalapack --download-ptscotch and I got the following error Configuring PETSc to compile on your system ============================================================================================= ============================================================================================= Warning: Intel compiler being used with gcc 4.8.5 compatibility, failures may occur. Recommend having a newer gcc version in your path. ============================================================================================= ============================================================================================= Running arch-linux-c-debug/bin/bfort to generate fortran stubs ============================================================================================= ============================================================================================= Compiling PTScotch; this may take several minutes ============================================================================================= ******************************************************************************* UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log for details): ------------------------------------------------------------------------------- Error running make on PTScotch: Could not execute "['cd /home/jgaray/petsc/arch-linux-c-debug/externalpackages/git.ptscotch/src && make clean ptesmumps esmumps']": /usr/bin/mkdir -p ../bin /usr/bin/mkdir -p ../include /usr/bin/mkdir -p ../lib (cd libscotch ; make clean) make[1]: Entering directory `/home/jgaray/petsc/arch-linux-c-debug/externalpackages/git.ptscotch/src/libscotch' rm -f *~ *.o lib*.a parser_yy.c parser_ly.h parser_ll.c *scotch.h *scotchf.h y.output *dummysizes make[1]: Leaving directory `/home/jgaray/petsc/arch-linux-c-debug/externalpackages/git.ptscotch/src/libscotch' (cd scotch ; make clean) make[1]: Entering directory `/home/jgaray/petsc/arch-linux-c-debug/externalpackages/git.ptscotch/src/scotch' rm -f *~ *.o acpl amk_ccc amk_fft2 amk_grf amk_hy amk_m2 amk_p2 atst gbase gcv gdump *ggath *gmap gmk_hy gmk_m2 gmk_m3 gmk_msh gmk_ub2 gmtst *gord gotst gout *gpart *gscat *gtst mcv mmk_m2 mmk_m3 mord mtst make[1]: Leaving directory `/home/jgaray/petsc/arch-linux-c-debug/externalpackages/git.ptscotch/src/scotch' (cd libscotchmetis ; make clean) make[1]: Entering directory `/home/jgaray/petsc/arch-linux-c-debug/externalpackages/git.ptscotch/src/libscotchmetis' rm -f *~ *.o lib*.a metis.h metisf.h parmetis.h make[1]: Leaving directory `/home/jgaray/petsc/arch-linux-c-debug/externalpackages/git.ptscotch/src/libscotchmetis' (cd check ; make clean) make[1]: Entering directory `/home/jgaray/petsc/arch-linux-c-debug/externalpackages/git.ptscotch/src/check' rm -f *~ *.o make[1]: Leaving directory `/home/jgaray/petsc/arch-linux-c-debug/externalpackages/git.ptscotch/src/check' (cd esmumps ; make clean) make[1]: Entering directory `/home/jgaray/petsc/arch-linux-c-debug/externalpackages/git.ptscotch/src/esmumps' rm -f *~ common.h *.o lib*.a main_esmumps make[1]: Leaving directory `/home/jgaray/petsc/arch-linux-c-debug/externalpackages/git.ptscotch/src/esmumps' (cd libscotch ; make VERSION=6 RELEASE=1 PATCHLEVEL=2 scotch && make install) make[1]: Entering directory `/home/jgaray/petsc/arch-linux-c-debug/externalpackages/git.ptscotch/src/libscotch' make \ CC="mpicc" \ CCD="mpicc" \ scotch.h \ scotchf.h \ libscotch.a \ libscotcherr.a \ libscotcherrexit.a make[2]: Entering directory `/home/jgaray/petsc/arch-linux-c-debug/externalpackages/git.ptscotch/src/libscotch' mpicc -fPIC -wd1572 -g -O0 -std=c99 -DCOMMON_RANDOM_FIXED_SEED -DSCOTCH_RENAME -Drestrict="restrict" -DINTSIZE32 -DSCOTCH_METIS_PREFIX -c arch.c -o arch.o -DSCOTCH_VERSION_NUM=6 -DSCOTCH_RELEASE_NUM=1 -DSCOTCH_PATCHLEVEL_NUM=2 mpicc -fPIC -wd1572 -g -O0 -std=c99 -DCOMMON_RANDOM_FIXED_SEED -DSCOTCH_RENAME -Drestrict="restrict" -DINTSIZE32 -DSCOTCH_METIS_PREFIX -DSCOTCH_VERSION_NUM=6 -DSCOTCH_RELEASE_NUM=1 -DSCOTCH_PATCHLEVEL_NUM=2 dummysizes.c -o dummysizes -lpthread -lrt ./dummysizes "-s" library.h scotch.h ./dummysizes "-s" library_f.h scotchf.h mpicc -fPIC -wd1572 -g -O0 -std=c99 -DCOMMON_RANDOM_FIXED_SEED -DSCOTCH_RENAME -Drestrict="restrict" -DINTSIZE32 -DSCOTCH_METIS_PREFIX -c arch_build.c -o arch_build.o -DSCOTCH_VERSION_NUM=6 -DSCOTCH_RELEASE_NUM=1 -DSCOTCH_PATCHLEVEL_NUM=2 mpicc -fPIC -wd1572 -g -O0 -std=c99 -DCOMMON_RANDOM_FIXED_SEED -DSCOTCH_RENAME -Drestrict="restrict" -DINTSIZE32 -DSCOTCH_METIS_PREFIX -c arch_build2.c -o arch_build2.o -DSCOTCH_VERSION_NUM=6 -DSCOTCH_RELEASE_NUM=1 -DSCOTCH_PATCHLEVEL_NUM=2 mpicc -fPIC -wd1572 -g -O0 -std=c99 -DCOMMON_RANDOM_FIXED_SEED -DSCOTCH_RENAME -Drestrict="restrict" -DINTSIZE32 -DSCOTCH_METIS_PREFIX -c arch_cmplt.c -o arch_cmplt.o -DSCOTCH_VERSION_NUM=6 -DSCOTCH_RELEASE_NUM=1 -DSCOTCH_PATCHLEVEL_NUM=2 mpicc -fPIC -wd1572 -g -O0 -std=c99 -DCOMMON_RANDOM_FIXED_SEED -DSCOTCH_RENAME -Drestrict="restrict" -DINTSIZE32 -DSCOTCH_METIS_PREFIX -c arch_cmpltw.c -o arch_cmpltw.o -DSCOTCH_VERSION_NUM=6 -DSCOTCH_RELEASE_NUM=1 -DSCOTCH_PATCHLEVEL_NUM=2 mpicc -fPIC -wd1572 -g -O0 -std=c99 -DCOMMON_RANDOM_FIXED_SEED -DSCOTCH_RENAME -Drestrict="restrict" -DINTSIZE32 -DSCOTCH_METIS_PREFIX -c arch_deco.c -o arch_deco.o -DSCOTCH_VERSION_NUM=6 -DSCOTCH_RELEASE_NUM=1 -DSCOTCH_PATCHLEVEL_NUM=2 mpicc -fPIC -wd1572 -g -O0 -std=c99 -DCOMMON_RANDOM_FIXED_SEED -DSCOTCH_RENAME -Drestrict="restrict" -DINTSIZE32 -DSCOTCH_METIS_PREFIX -c arch_deco2.c -o arch_deco2.o -DSCOTCH_VERSION_NUM=6 -DSCOTCH_RELEASE_NUM=1 -DSCOTCH_PATCHLEVEL_NUM=2 mpicc -fPIC -wd1572 -g -O0 -std=c99 -DCOMMON_RANDOM_FIXED_SEED -DSCOTCH_RENAME -Drestrict="restrict" -DINTSIZE32 -DSCOTCH_METIS_PREFIX -c arch_dist.c -o arch_dist.o -DSCOTCH_VERSION_NUM=6 -DSCOTCH_RELEASE_NUM=1 -DSCOTCH_PATCHLEVEL_NUM=2 mpicc -fPIC -wd1572 -g -O0 -std=c99 -DCOMMON_RANDOM_FIXED_SEED -DSCOTCH_RENAME -Drestrict="restrict" -DINTSIZE32 -DSCOTCH_METIS_PREFIX -c arch_hcub.c -o arch_hcub.o -DSCOTCH_VERSION_NUM=6 -DSCOTCH_RELEASE_NUM=1 -DSCOTCH_PATCHLEVEL_NUM=2 mpicc -fPIC -wd1572 -g -O0 -std=c99 -DCOMMON_RANDOM_FIXED_SEED -DSCOTCH_RENAME -Drestrict="restrict" -DINTSIZE32 -DSCOTCH_METIS_PREFIX -c arch_mesh.c -o arch_mesh.o -DSCOTCH_VERSION_NUM=6 -DSCOTCH_RELEASE_NUM=1 -DSCOTCH_PATCHLEVEL_NUM=2 mpicc -fPIC -wd1572 -g -O0 -std=c99 -DCOMMON_RANDOM_FIXED_SEED -DSCOTCH_RENAME -Drestrict="restrict" -DINTSIZE32 -DSCOTCH_METIS_PREFIX -c arch_sub.c -o arch_sub.o -DSCOTCH_VERSION_NUM=6 -DSCOTCH_RELEASE_NUM=1 -DSCOTCH_PATCHLEVEL_NUM=2 mpicc -fPIC -wd1572 -g -O0 -std=c99 -DCOMMON_RANDOM_FIXED_SEED -DSCOTCH_RENAME -Drestrict="restrict" -DINTSIZE32 -DSCOTCH_METIS_PREFIX -c arch_tleaf.c -o arch_tleaf.o -DSCOTCH_VERSION_NUM=6 -DSCOTCH_RELEASE_NUM=1 -DSCOTCH_PATCHLEVEL_NUM=2 mpicc -fPIC -wd1572 -g -O0 -std=c99 -DCOMMON_RANDOM_FIXED_SEED -DSCOTCH_RENAME -Drestrict="restrict" -DINTSIZE32 -DSCOTCH_METIS_PREFIX -c arch_torus.c -o arch_torus.o -DSCOTCH_VERSION_NUM=6 -DSCOTCH_RELEASE_NUM=1 -DSCOTCH_PATCHLEVEL_NUM=2 mpicc -fPIC -wd1572 -g -O0 -std=c99 -DCOMMON_RANDOM_FIXED_SEED -DSCOTCH_RENAME -Drestrict="restrict" -DINTSIZE32 -DSCOTCH_METIS_PREFIX -c arch_vcmplt.c -o arch_vcmplt.o -DSCOTCH_VERSION_NUM=6 -DSCOTCH_RELEASE_NUM=1 -DSCOTCH_PATCHLEVEL_NUM=2 mpicc -fPIC -wd1572 -g -O0 -std=c99 -DCOMMON_RANDOM_FIXED_SEED -DSCOTCH_RENAME -Drestrict="restrict" -DINTSIZE32 -DSCOTCH_METIS_PREFIX -c arch_vhcub.c -o arch_vhcub.o -DSCOTCH_VERSION_NUM=6 -DSCOTCH_RELEASE_NUM=1 -DSCOTCH_PATCHLEVEL_NUM=2 mpicc -fPIC -wd1572 -g -O0 -std=c99 -DCOMMON_RANDOM_FIXED_SEED -DSCOTCH_RENAME -Drestrict="restrict" -DINTSIZE32 -DSCOTCH_METIS_PREFIX -c bgraph.c -o bgraph.o -DSCOTCH_VERSION_NUM=6 -DSCOTCH_RELEASE_NUM=1 -DSCOTCH_PATCHLEVEL_NUM=2 mpicc -fPIC -wd1572 -g -O0 -std=c99 -DCOMMON_RANDOM_FIXED_SEED -DSCOTCH_RENAME -Drestrict="restrict" -DINTSIZE32 -DSCOTCH_METIS_PREFIX -c bgraph_bipart_bd.c -o bgraph_bipart_bd.o -DSCOTCH_VERSION_NUM=6 -DSCOTCH_RELEASE_NUM=1 -DSCOTCH_PATCHLEVEL_NUM=2 mpicc -fPIC -wd1572 -g -O0 -std=c99 -DCOMMON_RANDOM_FIXED_SEED -DSCOTCH_RENAME -Drestrict="restrict" -DINTSIZE32 -DSCOTCH_METIS_PREFIX -c bgraph_bipart_df.c -o bgraph_bipart_df.o -DSCOTCH_VERSION_NUM=6 -DSCOTCH_RELEASE_NUM=1 -DSCOTCH_PATCHLEVEL_NUM=2 mpicc -fPIC -wd1572 -g -O0 -std=c99 -DCOMMON_RANDOM_FIXED_SEED -DSCOTCH_RENAME -Drestrict="restrict" -DINTSIZE32 -DSCOTCH_METIS_PREFIX -c bgraph_bipart_ex.c -o bgraph_bipart_ex.o -DSCOTCH_VERSION_NUM=6 -DSCOTCH_RELEASE_NUM=1 -DSCOTCH_PATCHLEVEL_NUM=2 mpicc -fPIC -wd1572 -g -O0 -std=c99 -DCOMMON_RANDOM_FIXED_SEED -DSCOTCH_RENAME -Drestrict="restrict" -DINTSIZE32 -DSCOTCH_METIS_PREFIX -c bgraph_bipart_fm.c -o bgraph_bipart_fm.o -DSCOTCH_VERSION_NUM=6 -DSCOTCH_RELEASE_NUM=1 -DSCOTCH_PATCHLEVEL_NUM=2 mpicc -fPIC -wd1572 -g -O0 -std=c99 -DCOMMON_RANDOM_FIXED_SEED -DSCOTCH_RENAME -Drestrict="restrict" -DINTSIZE32 -DSCOTCH_METIS_PREFIX -c bgraph_bipart_gg.c -o bgraph_bipart_gg.o -DSCOTCH_VERSION_NUM=6 -DSCOTCH_RELEASE_NUM=1 -DSCOTCH_PATCHLEVEL_NUM=2 mpicc -fPIC -wd1572 -g -O0 -std=c99 -DCOMMON_RANDOM_FIXED_SEED -DSCOTCH_RENAME -Drestrict="restrict" -DINTSIZE32 -DSCOTCH_METIS_PREFIX -c bgraph_bipart_gp.c -o bgraph_bipart_gp.o -DSCOTCH_VERSION_NUM=6 -DSCOTCH_RELEASE_NUM=1 -DSCOTCH_PATCHLEVEL_NUM=2 mpicc -fPIC -wd1572 -g -O0 -std=c99 -DCOMMON_RANDOM_FIXED_SEED -DSCOTCH_RENAME -Drestrict="restrict" -DINTSIZE32 -DSCOTCH_METIS_PREFIX -c bgraph_bipart_ml.c -o bgraph_bipart_ml.o -DSCOTCH_VERSION_NUM=6 -DSCOTCH_RELEASE_NUM=1 -DSCOTCH_PATCHLEVEL_NUM=2 mpicc -fPIC -wd1572 -g -O0 -std=c99 -DCOMMON_RANDOM_FIXED_SEED -DSCOTCH_RENAME -Drestrict="restrict" -DINTSIZE32 -DSCOTCH_METIS_PREFIX -c bgraph_bipart_st.c -o bgraph_bipart_st.o -DSCOTCH_VERSION_NUM=6 -DSCOTCH_RELEASE_NUM=1 -DSCOTCH_PATCHLEVEL_NUM=2 mpicc -fPIC -wd1572 -g -O0 -std=c99 -DCOMMON_RANDOM_FIXED_SEED -DSCOTCH_RENAME -Drestrict="restrict" -DINTSIZE32 -DSCOTCH_METIS_PREFIX -c bgraph_bipart_zr.c -o bgraph_bipart_zr.o -DSCOTCH_VERSION_NUM=6 -DSCOTCH_RELEASE_NUM=1 -DSCOTCH_PATCHLEVEL_NUM=2 mpicc -fPIC -wd1572 -g -O0 -std=c99 -DCOMMON_RANDOM_FIXED_SEED -DSCOTCH_RENAME -Drestrict="restrict" -DINTSIZE32 -DSCOTCH_METIS_PREFIX -c bgraph_check.c -o bgraph_check.o -DSCOTCH_VERSION_NUM=6 -DSCOTCH_RELEASE_NUM=1 -DSCOTCH_PATCHLEVEL_NUM=2 mpicc -fPIC -wd1572 -g -O0 -std=c99 -DCOMMON_RANDOM_FIXED_SEED -DSCOTCH_RENAME -Drestrict="restrict" -DINTSIZE32 -DSCOTCH_METIS_PREFIX -c bgraph_store.c -o bgraph_store.o -DSCOTCH_VERSION_NUM=6 -DSCOTCH_RELEASE_NUM=1 -DSCOTCH_PATCHLEVEL_NUM=2 mpicc -fPIC -wd1572 -g -O0 -std=c99 -DCOMMON_RANDOM_FIXED_SEED -DSCOTCH_RENAME -Drestrict="restrict" -DINTSIZE32 -DSCOTCH_METIS_PREFIX -c common.c -DSCOTCH_COMMON_RENAME -o common.o mpicc -fPIC -wd1572 -g -O0 -std=c99 -DCOMMON_RANDOM_FIXED_SEED -DSCOTCH_RENAME -Drestrict="restrict" -DINTSIZE32 -DSCOTCH_METIS_PREFIX -c common_file.c -DSCOTCH_COMMON_RENAME -o common_file.o mpicc -fPIC -wd1572 -g -O0 -std=c99 -DCOMMON_RANDOM_FIXED_SEED -DSCOTCH_RENAME -Drestrict="restrict" -DINTSIZE32 -DSCOTCH_METIS_PREFIX -c common_file_compress.c -DSCOTCH_COMMON_RENAME -o common_file_compress.o mpicc -fPIC -wd1572 -g -O0 -std=c99 -DCOMMON_RANDOM_FIXED_SEED -DSCOTCH_RENAME -Drestrict="restrict" -DINTSIZE32 -DSCOTCH_METIS_PREFIX -c common_file_decompress.c -DSCOTCH_COMMON_RENAME -o common_file_decompress.o mpicc -fPIC -wd1572 -g -O0 -std=c99 -DCOMMON_RANDOM_FIXED_SEED -DSCOTCH_RENAME -Drestrict="restrict" -DINTSIZE32 -DSCOTCH_METIS_PREFIX -c common_integer.c -DSCOTCH_COMMON_RENAME -o common_integer.o mpicc -fPIC -wd1572 -g -O0 -std=c99 -DCOMMON_RANDOM_FIXED_SEED -DSCOTCH_RENAME -Drestrict="restrict" -DINTSIZE32 -DSCOTCH_METIS_PREFIX -c common_memory.c -DSCOTCH_COMMON_RENAME -o common_memory.o mpicc -fPIC -wd1572 -g -O0 -std=c99 -DCOMMON_RANDOM_FIXED_SEED -DSCOTCH_RENAME -Drestrict="restrict" -DINTSIZE32 -DSCOTCH_METIS_PREFIX -c common_string.c -DSCOTCH_COMMON_RENAME -o common_string.o make[2]: Leaving directory `/home/jgaray/petsc/arch-linux-c-debug/externalpackages/git.ptscotch/src/libscotch' make[1]: Leaving directory `/home/jgaray/petsc/arch-linux-c-debug/externalpackages/git.ptscotch/src/libscotch'In file included from /usr/include/sys/wait.h(30), from common.h(132), from common_string.c(57): /usr/include/signal.h(156): error: identifier "siginfo_t" is undefined extern void psiginfo (const siginfo_t *__pinfo, const char *__s); ^ compilation aborted for common_string.c (code 2) make[2]: *** [common_string.o] Error 2 make[1]: *** [scotch] Error 2 make: *** [libscotch] Error 2 Any ideas of how to resolve these issues will be much appreciated! Best, Jose From mi.mike1021 at gmail.com Fri May 20 19:35:27 2022 From: mi.mike1021 at gmail.com (Mike Michell) Date: Fri, 20 May 2022 19:35:27 -0500 Subject: [petsc-users] PetscSF Object on Distributed DMPlex for Halo Data Exchange Message-ID: Dear PETSc developer team, Hi, I am using DMPlex for a finite-volume code and trying to understand the usage of PetscSF. What is a typical procedure for doing halo data exchange at parallel boundary using PetscSF object on DMPlex? Is there any example that I can refer to usage of PetscSF with distributed DMPlex? Assuming to use the attached mock-up code and mesh, if I give "-dm_distribute_overlap 1 -over_dm_view" to run the code, I can see a PetscSF object is already created, although I have not called "PetscSFCreate" in the code. How can I import & use that PetscSF already created by the code to do the halo data exchange? What I want to do is to exchange data (probably just MPI_Reduce) in a parallel boundary region using PetscSF and its functions. I might need to have an overlapping layer or not. Thanks, Mike -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Q_PetscSF.tar Type: application/x-tar Size: 133120 bytes Desc: not available URL: From toby.isaac at gmail.com Fri May 20 20:07:46 2022 From: toby.isaac at gmail.com (Toby Isaac) Date: Fri, 20 May 2022 21:07:46 -0400 Subject: [petsc-users] PetscSF Object on Distributed DMPlex for Halo Data Exchange In-Reply-To: References: Message-ID: The PetscSF that is created automatically is the "point sf" ( https://petsc.org/main/docs/manualpages/DM/DMGetPointSF/): it says which mesh points (cells, faces, edges and vertices) are duplicates of others. In a finite volume application we typically want to assign degrees of freedom just to cells: some applications may only have one degree of freedom, others may have multiple. You encode where you want degrees of freedom in a PetscSection and set that as the section for the DM in DMSetLocalSection() ( https://petsc.org/release/docs/manualpages/DM/DMSetLocalSection.html) (A c example of these steps that sets degrees of freedom for *vertices* instead of cells is `src/dm/impls/plex/tutorials/ex7.c`) After that you can call DMGetSectionSF() ( https://petsc.org/main/docs/manualpages/DM/DMGetSectionSF/) to the the PetscSF that you want for halo exchange: the one for your solution variables. After that, the only calls you typically need in a finite volume code is PetscSFBcastBegin() to start a halo exchange and PetscSFBcastEnd() to complete it. You say > What I want to do is to exchange data (probably just MPI_Reduce) which confuses me, because halo exchange is a point-to-point exchange and not a reduction. Can you clarify? On Fri, May 20, 2022 at 8:35 PM Mike Michell wrote: > Dear PETSc developer team, > > Hi, I am using DMPlex for a finite-volume code and trying to understand > the usage of PetscSF. What is a typical procedure for doing halo data > exchange at parallel boundary using PetscSF object on DMPlex? Is there any > example that I can refer to usage of PetscSF with distributed DMPlex? > > Assuming to use the attached mock-up code and mesh, if I give > "-dm_distribute_overlap 1 -over_dm_view" to run the code, I can see a > PetscSF object is already created, although I have not called > "PetscSFCreate" in the code. How can I import & use that PetscSF already > created by the code to do the halo data exchange? > > What I want to do is to exchange data (probably just MPI_Reduce) in a > parallel boundary region using PetscSF and its functions. I might need to > have an overlapping layer or not. > > Thanks, > Mike > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mi.mike1021 at gmail.com Fri May 20 21:45:00 2022 From: mi.mike1021 at gmail.com (Mike Michell) Date: Fri, 20 May 2022 21:45:00 -0500 Subject: [petsc-users] PetscSF Object on Distributed DMPlex for Halo Data Exchange In-Reply-To: References: Message-ID: Thanks for the reply. > "What I want to do is to exchange data (probably just MPI_Reduce)" which confuses me, because halo exchange is a point-to-point exchange and not a reduction. Can you clarify? PetscSFReduceBegin/End seems to be the function that do reduction for PetscSF object. What I intended to mention was either reduction or exchange, not specifically intended "reduction". As a follow-up question: Assuming that the code has its own local solution arrays (not Petsc type), and if the plex's DAG indices belong to the halo region are the only information that I want to know (not the detailed section description, such as degree of freedom on vertex, cells, etc.). I have another PetscSection for printing out my solution. Also if I can convert that DAG indices into my local cell/vertex index, can I just use the PetscSF object created from DMGetPointSF(), instead of "creating PetscSection + DMGetSectionSF()"? In other words, can I use the PetscSF object declared from DMGetPointSF() for the halo communication? Thanks, Mike The PetscSF that is created automatically is the "point sf" ( > https://petsc.org/main/docs/manualpages/DM/DMGetPointSF/): it says which > mesh points (cells, faces, edges and vertices) are duplicates of others. > > In a finite volume application we typically want to assign degrees of > freedom just to cells: some applications may only have one degree of > freedom, others may have multiple. > > You encode where you want degrees of freedom in a PetscSection and set > that as the section for the DM in DMSetLocalSection() ( > https://petsc.org/release/docs/manualpages/DM/DMSetLocalSection.html) > > (A c example of these steps that sets degrees of freedom for *vertices* > instead of cells is `src/dm/impls/plex/tutorials/ex7.c`) > > After that you can call DMGetSectionSF() ( > https://petsc.org/main/docs/manualpages/DM/DMGetSectionSF/) to the the > PetscSF that you want for halo exchange: the one for your solution > variables. > > After that, the only calls you typically need in a finite volume code is > PetscSFBcastBegin() to start a halo exchange and PetscSFBcastEnd() to > complete it. > > You say > > > What I want to do is to exchange data (probably just MPI_Reduce) > > which confuses me, because halo exchange is a point-to-point exchange and > not a reduction. Can you clarify? > > > > On Fri, May 20, 2022 at 8:35 PM Mike Michell > wrote: > >> Dear PETSc developer team, >> >> Hi, I am using DMPlex for a finite-volume code and trying to understand >> the usage of PetscSF. What is a typical procedure for doing halo data >> exchange at parallel boundary using PetscSF object on DMPlex? Is there any >> example that I can refer to usage of PetscSF with distributed DMPlex? >> >> Assuming to use the attached mock-up code and mesh, if I give >> "-dm_distribute_overlap 1 -over_dm_view" to run the code, I can see a >> PetscSF object is already created, although I have not called >> "PetscSFCreate" in the code. How can I import & use that PetscSF already >> created by the code to do the halo data exchange? >> >> What I want to do is to exchange data (probably just MPI_Reduce) in a >> parallel boundary region using PetscSF and its functions. I might need to >> have an overlapping layer or not. >> >> Thanks, >> Mike >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Sat May 21 09:33:56 2022 From: knepley at gmail.com (Matthew Knepley) Date: Sat, 21 May 2022 04:33:56 -1000 Subject: [petsc-users] PetscSF Object on Distributed DMPlex for Halo Data Exchange In-Reply-To: References: Message-ID: On Fri, May 20, 2022 at 4:45 PM Mike Michell wrote: > Thanks for the reply. > > > "What I want to do is to exchange data (probably just MPI_Reduce)" which > confuses me, because halo exchange is a point-to-point exchange and not a > reduction. Can you clarify? > PetscSFReduceBegin/End seems to be the function that do reduction for > PetscSF object. What I intended to mention was either reduction or > exchange, not specifically intended "reduction". > > As a follow-up question: > Assuming that the code has its own local solution arrays (not Petsc type), > and if the plex's DAG indices belong to the halo region are the only > information that I want to know (not the detailed section description, such > as degree of freedom on vertex, cells, etc.). I have another PetscSection > for printing out my solution. > Also if I can convert that DAG indices into my local cell/vertex index, > can I just use the PetscSF object created from DMGetPointSF(), instead of > "creating PetscSection + DMGetSectionSF()"? In other words, can I use the > PetscSF object declared from DMGetPointSF() for the halo communication? > No, because that point SF will index information by point number. You would need to build a new SF that indexes your dofs. The steps you would go through are exactly the same as you would if you just told us what the Section is that indexes your data. Thanks, Matt > Thanks, > Mike > > > The PetscSF that is created automatically is the "point sf" ( >> https://petsc.org/main/docs/manualpages/DM/DMGetPointSF/): it says which >> mesh points (cells, faces, edges and vertices) are duplicates of others. >> >> In a finite volume application we typically want to assign degrees of >> freedom just to cells: some applications may only have one degree of >> freedom, others may have multiple. >> >> You encode where you want degrees of freedom in a PetscSection and set >> that as the section for the DM in DMSetLocalSection() ( >> https://petsc.org/release/docs/manualpages/DM/DMSetLocalSection.html) >> >> (A c example of these steps that sets degrees of freedom for *vertices* >> instead of cells is `src/dm/impls/plex/tutorials/ex7.c`) >> >> After that you can call DMGetSectionSF() ( >> https://petsc.org/main/docs/manualpages/DM/DMGetSectionSF/) to the the >> PetscSF that you want for halo exchange: the one for your solution >> variables. >> >> After that, the only calls you typically need in a finite volume code is >> PetscSFBcastBegin() to start a halo exchange and PetscSFBcastEnd() to >> complete it. >> >> You say >> >> > What I want to do is to exchange data (probably just MPI_Reduce) >> >> which confuses me, because halo exchange is a point-to-point exchange and >> not a reduction. Can you clarify? >> >> >> >> On Fri, May 20, 2022 at 8:35 PM Mike Michell >> wrote: >> >>> Dear PETSc developer team, >>> >>> Hi, I am using DMPlex for a finite-volume code and trying to understand >>> the usage of PetscSF. What is a typical procedure for doing halo data >>> exchange at parallel boundary using PetscSF object on DMPlex? Is there any >>> example that I can refer to usage of PetscSF with distributed DMPlex? >>> >>> Assuming to use the attached mock-up code and mesh, if I give >>> "-dm_distribute_overlap 1 -over_dm_view" to run the code, I can see a >>> PetscSF object is already created, although I have not called >>> "PetscSFCreate" in the code. How can I import & use that PetscSF already >>> created by the code to do the halo data exchange? >>> >>> What I want to do is to exchange data (probably just MPI_Reduce) in a >>> parallel boundary region using PetscSF and its functions. I might need to >>> have an overlapping layer or not. >>> >>> Thanks, >>> Mike >>> >> -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From mateojmoinelo at gmail.com Sat May 21 11:41:16 2022 From: mateojmoinelo at gmail.com (=?UTF-8?Q?Mateo_Jos=C3=A9_Moinelo?=) Date: Sat, 21 May 2022 18:41:16 +0200 Subject: [petsc-users] Solving a linear system with sparse matrices Message-ID: Hi. I am working on a Petsc program, and I have a problem. I have two sparse matrices A and B, and I want to compute inv(A)*B. I understand that computing the inverse of a matrix directly is not effective, and that in this case, the best way to do the operation is instead solving the system A*X = B, being X the result. I reviewed the documentation, and found some interesting options, like the function MatMatSolve, or using a KSP. The problem I found is that these options seem to work only for dense matrices, and I do not know how to convert my sparce matrices to dense. What can I do in this case?. Thanks in advance. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jroman at dsic.upv.es Sat May 21 12:50:08 2022 From: jroman at dsic.upv.es (Jose E. Roman) Date: Sat, 21 May 2022 19:50:08 +0200 Subject: [petsc-users] Solving a linear system with sparse matrices In-Reply-To: References: Message-ID: <32955944-CF31-45D4-8118-C1C882CD2819@dsic.upv.es> The matrix inv(A)*B can be represented with a shell matrix that calls MatMult() followed by KSPSolve(), if you only need to apply this operator to a vector. It depends on what you are trying to do. Jose > El 21 may 2022, a las 18:41, Mateo Jos? Moinelo escribi?: > > Hi. I am working on a Petsc program, and I have a problem. I have two sparse matrices A and B, and I want to compute inv(A)*B. I understand that computing the inverse of a matrix directly is not effective, and that in this case, the best way to do the operation is instead solving the system A*X = B, being X the result. I reviewed the documentation, and found some interesting options, like the function MatMatSolve, or using a KSP. > > The problem I found is that these options seem to work only for dense matrices, and I do not know how to convert my sparce matrices to dense. What can I do in this case?. > > Thanks in advance. From jed at jedbrown.org Sat May 21 12:50:47 2022 From: jed at jedbrown.org (Jed Brown) Date: Sat, 21 May 2022 11:50:47 -0600 Subject: [petsc-users] Solving a linear system with sparse matrices In-Reply-To: References: Message-ID: <87o7zqzseg.fsf@jedbrown.org> You can MatConvert your sparse matrix to dense, but there's almost always a better way. Why do you want this product (which is dense) represented explicitly rather than via the sequence (multiply by B, then solve with A)? What will you use it for? Mateo Jos? Moinelo writes: > Hi. I am working on a Petsc program, and I have a problem. I have two > sparse matrices A and B, and I want to compute inv(A)*B. I understand that > computing the inverse of a matrix directly is not effective, and that in > this case, the best way to do the operation is instead solving the system > A*X = B, being X the result. I reviewed the documentation, and found some > interesting options, like the function MatMatSolve, or using a KSP. > > The problem I found is that these options seem to work only for dense > matrices, and I do not know how to convert my sparce matrices to dense. > What can I do in this case?. > > Thanks in advance. From stefano.zampini at gmail.com Sat May 21 13:42:48 2022 From: stefano.zampini at gmail.com (Stefano Zampini) Date: Sat, 21 May 2022 21:42:48 +0300 Subject: [petsc-users] Solving a linear system with sparse matrices In-Reply-To: References: Message-ID: MatMatSolve with sparse rhs works with MUMPS On Sat, May 21, 2022, 19:41 Mateo Jos? Moinelo wrote: > Hi. I am working on a Petsc program, and I have a problem. I have two > sparse matrices A and B, and I want to compute inv(A)*B. I understand that > computing the inverse of a matrix directly is not effective, and that in > this case, the best way to do the operation is instead solving the system > A*X = B, being X the result. I reviewed the documentation, and found some > interesting options, like the function MatMatSolve, or using a KSP. > > The problem I found is that these options seem to work only for dense > matrices, and I do not know how to convert my sparce matrices to dense. > What can I do in this case?. > > Thanks in advance. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From yu1299885905 at outlook.com Sat May 21 08:23:20 2022 From: yu1299885905 at outlook.com (wang yuqi) Date: Sat, 21 May 2022 13:23:20 +0000 Subject: [petsc-users] A compilation error about VecGetValues Message-ID: ? Windows ????? Hi, Dear developer: I had compilation problems when using VecGetValues. My procedure is as follows: program main #include use petscvec Vec x PetscInt N,i,ione PetscErrorCode ierr PetscMPIInt rank PetscScalar one call PetscInitialize(PETSC_NULL_CHARACTER,ierr) if (ierr .ne. 0) then print*,'PetscInitialize failed' stop endif one = 1.0 call MPI_Comm_rank(PETSC_COMM_WORLD,rank,ierr) N = rank + 1 call VecCreateMPI(PETSC_COMM_WORLD,N,PETSC_DECIDE,x,ierr) call VecGetSize(x,N,ierr) call VecSet(x,one,ierr) ione = 1 do 100 i=0,N-rank-1 call VecSetValues(x,ione,i,one,ADD_VALUES,ierr) 100 continue call VecAssemblyBegin(x,ierr) call VecAssemblyEnd(x,ierr) if (rank .eq. 0) then ione = 0 call VecGetValues(x,ione,i,one,ierr) endif call VecView(x,PETSC_VIEWER_STDOUT_WORLD,ierr) call VecDestroy(x,ierr) call PetscFinalize(ierr) end PROGRAM main The compilation error is as follows: mpif90 -I/root/soft/petsc/3.8.0/include -o ex2f.o ex2f.F90 ex2f.F90(62): error #8284: If the actual argument is scalar, the dummy argument shall be scalar unless the actual argument is of type character or is an element of an array that is not assumed shape, pointer, or polymorphic. [C] call VecGetValues(x,ione,i,one,ierr) -------------^ ex2f.F90(62): error #8284: If the actual argument is scalar, the dummy argument shall be scalar unless the actual argument is of type character or is an element of an array that is not assumed shape, pointer, or polymorphic. [D] call VecGetValues(x,ione,i,one,ierr) -------------^ compilation aborted for ex2f.F90 (code 1) And this error only appears when the Petsc version is higher than 3.8. I look forward to hearing from you. Thanks! -------------- next part -------------- An HTML attachment was scrubbed... URL: From mfadams at lbl.gov Sun May 22 07:40:24 2022 From: mfadams at lbl.gov (Mark Adams) Date: Sun, 22 May 2022 08:40:24 -0400 Subject: [petsc-users] A compilation error about VecGetValues In-Reply-To: References: Message-ID: The last two arguments, other than the error code, to https://petsc.org/main/docs/manualpages/Vec/VecGetValues/ are arrays not scalars. PETSc started emabling type checking after 3.8. https://petsc.org/main/src/vec/vec/tutorials/ex2f.F.html does this. This is a terrible example of code. It should be removed. Why does this example work? It has 'implicit none' You might add that to see if that fixes this problem, but this is a terrible "tutorial". Mark On Sun, May 22, 2022 at 12:17 AM wang yuqi wrote: > > > > > ? Windows ??? ?? > > Hi, Dear developer: > > I had compilation problems when using VecGetValues. My procedure is as > follows: > > program main > > #include > > use petscvec > > Vec x > > PetscInt N,i,ione > > PetscErrorCode ierr > > PetscMPIInt rank > > PetscScalar one > > > > call PetscInitialize(PETSC_NULL_CHARACTER,ierr) > > if (ierr .ne. 0) then > > print*,'PetscInitialize failed' > > stop > > endif > > one = 1.0 > > call MPI_Comm_rank(PETSC_COMM_WORLD,rank,ierr) > > N = rank + 1 > > call VecCreateMPI(PETSC_COMM_WORLD,N,PETSC_DECIDE,x,ierr) > > call VecGetSize(x,N,ierr) > > call VecSet(x,one,ierr) > > ione = 1 > > do 100 i=0,N-rank-1 > > call VecSetValues(x,ione,i,one,ADD_VALUES,ierr) > > 100 continue > > call VecAssemblyBegin(x,ierr) > > call VecAssemblyEnd(x,ierr) > > > > if (rank .eq. 0) then > > ione = 0 > > call VecGetValues(x,ione,i,one,ierr) > > endif > > call VecView(x,PETSC_VIEWER_STDOUT_WORLD,ierr) > > call VecDestroy(x,ierr) > > > > call PetscFinalize(ierr) > > end PROGRAM main > > > > The compilation error is as follows: > > mpif90 -I/root/soft/petsc/3.8.0/include -o ex2f.o ex2f.F90 > > ex2f.F90(62): error #8284: If the actual argument is scalar, the dummy > argument shall be scalar unless the actual argument is of type character or > is an element of an array that is not assumed shape, pointer, or > polymorphic. [C] > > call VecGetValues(x,ione,i,one,ierr) > > -------------^ > > ex2f.F90(62): error #8284: If the actual argument is scalar, the dummy > argument shall be scalar unless the actual argument is of type character or > is an element of an array that is not assumed shape, pointer, or > polymorphic. [D] > > call VecGetValues(x,ione,i,one,ierr) > > -------------^ > > compilation aborted for ex2f.F90 (code 1) > > And this error only appears when the Petsc version is higher than 3.8. > > I look forward to hearing from you. > > Thanks! > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mi.mike1021 at gmail.com Sun May 22 09:47:25 2022 From: mi.mike1021 at gmail.com (Mike Michell) Date: Sun, 22 May 2022 09:47:25 -0500 Subject: [petsc-users] PetscSF Object on Distributed DMPlex for Halo Data Exchange In-Reply-To: References: Message-ID: Thank you for the reply. The PetscSection and PetscSF objects are defined as in the attached mock-up code (Q_PetscSF_1.tar). 1-DOF is defined on vertex as my solution is determined on each vertex with 1-DOF from a finite-volume method. As follow up questions: (1) It seems that the "roots" means the number of vertex not considering overlap layer, and "leaves" seems the number of distributed vertex for each processor that includes overlap layer. Can you acknowledge that this is correct understanding? I have tried to find clearer examples from PETSc team's articles relevant to Star Forest, but I am still unclear about the exact relation & graphical notation of roots & leaves in SF if it's the case of DMPlex solution arrays. (2) If it is so, there is an issue that I cannot define "root data" and "leave data" generally. I am trying to following "src/vec/is/sf/tutorials/ex1f.F90", however, in that example, size of roots and leaves are predefined as 6. How can I generalize that? Because I can get size of leaves using DAG depth(or height), which is equal to number of vertices each proc has. But, how can I get the size of my "roots" region from SF? Any example about that? This question is connected to how can I define "rootdata" for "PetscSFBcastBegin/End()". (3) More importantly, with the attached PetscSection & SF layout, my vector is only resolved for the size equal to "number of roots" for each proc, but not for the overlapping area(i.e., "leaves"). What I wish to do is to exchange (or reduce) the solution data between each proc, in the overlapping region. Can I get some advices why my vector does not encompass the "leaves" regime? Is there any example doing similar things? Thanks, Mike > On Fri, May 20, 2022 at 4:45 PM Mike Michell > wrote: > >> Thanks for the reply. >> >> > "What I want to do is to exchange data (probably just MPI_Reduce)" >> which confuses me, because halo exchange is a point-to-point exchange and >> not a reduction. Can you clarify? >> PetscSFReduceBegin/End seems to be the function that do reduction for >> PetscSF object. What I intended to mention was either reduction or >> exchange, not specifically intended "reduction". >> >> As a follow-up question: >> Assuming that the code has its own local solution arrays (not Petsc >> type), and if the plex's DAG indices belong to the halo region are the only >> information that I want to know (not the detailed section description, such >> as degree of freedom on vertex, cells, etc.). I have another PetscSection >> for printing out my solution. >> Also if I can convert that DAG indices into my local cell/vertex index, >> can I just use the PetscSF object created from DMGetPointSF(), instead of >> "creating PetscSection + DMGetSectionSF()"? In other words, can I use the >> PetscSF object declared from DMGetPointSF() for the halo communication? >> > > No, because that point SF will index information by point number. You > would need to build a new SF that indexes your dofs. The steps you would > go through are exactly the same as you would if you just told us what the > Section is that indexes your data. > > Thanks, > > Matt > > >> Thanks, >> Mike >> >> >> The PetscSF that is created automatically is the "point sf" ( >>> https://petsc.org/main/docs/manualpages/DM/DMGetPointSF/): it says >>> which mesh points (cells, faces, edges and vertices) are duplicates of >>> others. >>> >>> In a finite volume application we typically want to assign degrees of >>> freedom just to cells: some applications may only have one degree of >>> freedom, others may have multiple. >>> >>> You encode where you want degrees of freedom in a PetscSection and set >>> that as the section for the DM in DMSetLocalSection() ( >>> https://petsc.org/release/docs/manualpages/DM/DMSetLocalSection.html) >>> >>> (A c example of these steps that sets degrees of freedom for *vertices* >>> instead of cells is `src/dm/impls/plex/tutorials/ex7.c`) >>> >>> After that you can call DMGetSectionSF() ( >>> https://petsc.org/main/docs/manualpages/DM/DMGetSectionSF/) to the the >>> PetscSF that you want for halo exchange: the one for your solution >>> variables. >>> >>> After that, the only calls you typically need in a finite volume code is >>> PetscSFBcastBegin() to start a halo exchange and PetscSFBcastEnd() to >>> complete it. >>> >>> You say >>> >>> > What I want to do is to exchange data (probably just MPI_Reduce) >>> >>> which confuses me, because halo exchange is a point-to-point exchange >>> and not a reduction. Can you clarify? >>> >>> >>> >>> On Fri, May 20, 2022 at 8:35 PM Mike Michell >>> wrote: >>> >>>> Dear PETSc developer team, >>>> >>>> Hi, I am using DMPlex for a finite-volume code and trying to understand >>>> the usage of PetscSF. What is a typical procedure for doing halo data >>>> exchange at parallel boundary using PetscSF object on DMPlex? Is there any >>>> example that I can refer to usage of PetscSF with distributed DMPlex? >>>> >>>> Assuming to use the attached mock-up code and mesh, if I give >>>> "-dm_distribute_overlap 1 -over_dm_view" to run the code, I can see a >>>> PetscSF object is already created, although I have not called >>>> "PetscSFCreate" in the code. How can I import & use that PetscSF already >>>> created by the code to do the halo data exchange? >>>> >>>> What I want to do is to exchange data (probably just MPI_Reduce) in a >>>> parallel boundary region using PetscSF and its functions. I might need to >>>> have an overlapping layer or not. >>>> >>>> Thanks, >>>> Mike >>>> >>> > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > > https://www.cse.buffalo.edu/~knepley/ > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Q_PetscSF_1.tar Type: application/x-tar Size: 112640 bytes Desc: not available URL: From bsmith at petsc.dev Sun May 22 11:44:13 2022 From: bsmith at petsc.dev (Barry Smith) Date: Sun, 22 May 2022 12:44:13 -0400 Subject: [petsc-users] A compilation error about VecGetValues In-Reply-To: References: Message-ID: <54D1F3FD-BADF-4ED4-9FB0-ECA0ABD73DBE@petsc.dev> This should work for at least the last three versions of PETSc. Please upgrade to the latest PETSc version and let us know if you continue to get this error message. Yes, for some PETSc versions between 3.8 and 3.14 it may generate this error message with some Fortran compilers. Barry > On May 21, 2022, at 9:23 AM, wang yuqi wrote: > > > > ? Windows ??? ?? > Hi, Dear developer: > I had compilation problems when using VecGetValues. My procedure is as follows: > program main > #include > use petscvec > Vec x > PetscInt N,i,ione > PetscErrorCode ierr > PetscMPIInt rank > PetscScalar one > > call PetscInitialize(PETSC_NULL_CHARACTER,ierr) > if (ierr .ne. 0) then > print*,'PetscInitialize failed' > stop > endif > one = 1.0 > call MPI_Comm_rank(PETSC_COMM_WORLD,rank,ierr) > N = rank + 1 > call VecCreateMPI(PETSC_COMM_WORLD,N,PETSC_DECIDE,x,ierr) > call VecGetSize(x,N,ierr) > call VecSet(x,one,ierr) > ione = 1 > do 100 i=0,N-rank-1 > call VecSetValues(x,ione,i,one,ADD_VALUES,ierr) > 100 continue > call VecAssemblyBegin(x,ierr) > call VecAssemblyEnd(x,ierr) > > if (rank .eq. 0) then > ione = 0 > call VecGetValues(x,ione,i,one,ierr) > endif > call VecView(x,PETSC_VIEWER_STDOUT_WORLD,ierr) > call VecDestroy(x,ierr) > > call PetscFinalize(ierr) > end PROGRAM main > > The compilation error is as follows: > mpif90 -I/root/soft/petsc/3.8.0/include -o ex2f.o ex2f.F90 > ex2f.F90(62): error #8284: If the actual argument is scalar, the dummy argument shall be scalar unless the actual argument is of type character or is an element of an array that is not assumed shape, pointer, or polymorphic. [C] > call VecGetValues(x,ione,i,one,ierr) > -------------^ > ex2f.F90(62): error #8284: If the actual argument is scalar, the dummy argument shall be scalar unless the actual argument is of type character or is an element of an array that is not assumed shape, pointer, or polymorphic. [D] > call VecGetValues(x,ione,i,one,ierr) > -------------^ > compilation aborted for ex2f.F90 (code 1) > And this error only appears when the Petsc version is higher than 3.8. > I look forward to hearing from you. > Thanks! -------------- next part -------------- An HTML attachment was scrubbed... URL: From toby.isaac at gmail.com Sun May 22 13:56:29 2022 From: toby.isaac at gmail.com (Toby Isaac) Date: Sun, 22 May 2022 14:56:29 -0400 Subject: [petsc-users] PetscSF Object on Distributed DMPlex for Halo Data Exchange In-Reply-To: References: Message-ID: Here's a diagram of a 1D mesh with overlap and 3 partitions, showing what the petscsf data is for each. The number of roots is the number of mesh points in the local representation, and the number of leaves is the number of mesh points that are duplicates of mesh points on other processes. With that in mind, answering your questions > (1) It seems that the "roots" means the number of vertex not considering overlap layer, and "leaves" seems the number of distributed vertex for each processor that includes overlap layer. Can you acknowledge that this is correct understanding? I have tried to find clearer examples from PETSc team's articles relevant to Star Forest, but I am still unclear about the exact relation & graphical notation of roots & leaves in SF if it's the case of DMPlex solution arrays. No, the number of roots for a DMPlex is the number of mesh points in the local portion of the mesh > (2) If it is so, there is an issue that I cannot define "root data" and "leave data" generally. I am trying to following "src/vec/is/sf/tutorials/ex1f.F90", however, in that example, size of roots and leaves are predefined as 6. How can I generalize that? Because I can get size of leaves using DAG depth(or height), which is equal to number of vertices each proc has. But, how can I get the size of my "roots" region from SF? Any example about that? This question is connected to how can I define "rootdata" for "PetscSFBcastBegin/End()". Does the diagram help you generalize? > (3) More importantly, with the attached PetscSection & SF layout, my vector is only resolved for the size equal to "number of roots" for each proc, but not for the overlapping area(i.e., "leaves"). What I wish to do is to exchange (or reduce) the solution data between each proc, in the overlapping region. Can I get some advices why my vector does not encompass the "leaves" regime? Is there any example doing similar things? Going back to my first response: if you use a section to say how many pieces of data are associated with each local mesh point, then a PetscSF is constructed that requires no more manipulation from you. On Sun, May 22, 2022 at 10:47 AM Mike Michell wrote: > Thank you for the reply. > The PetscSection and PetscSF objects are defined as in the attached > mock-up code (Q_PetscSF_1.tar). 1-DOF is defined on vertex as my solution > is determined on each vertex with 1-DOF from a finite-volume method. > > As follow up questions: > (1) It seems that the "roots" means the number of vertex not considering > overlap layer, and "leaves" seems the number of distributed vertex for each > processor that includes overlap layer. Can you acknowledge that this is > correct understanding? I have tried to find clearer examples from PETSc > team's articles relevant to Star Forest, but I am still unclear about the > exact relation & graphical notation of roots & leaves in SF if it's the > case of DMPlex solution arrays. > > (2) If it is so, there is an issue that I cannot define "root data" and > "leave data" generally. I am trying to following > "src/vec/is/sf/tutorials/ex1f.F90", however, in that example, size of roots > and leaves are predefined as 6. How can I generalize that? Because I can > get size of leaves using DAG depth(or height), which is equal to number of > vertices each proc has. But, how can I get the size of my "roots" region > from SF? Any example about that? This question is connected to how can I > define "rootdata" for "PetscSFBcastBegin/End()". > > (3) More importantly, with the attached PetscSection & SF layout, my > vector is only resolved for the size equal to "number of roots" for each > proc, but not for the overlapping area(i.e., "leaves"). What I wish to do > is to exchange (or reduce) the solution data between each proc, in the > overlapping region. Can I get some advices why my vector does not encompass > the "leaves" regime? Is there any example doing similar things? > > Thanks, > Mike > > >> On Fri, May 20, 2022 at 4:45 PM Mike Michell >> wrote: >> >>> Thanks for the reply. >>> >>> > "What I want to do is to exchange data (probably just MPI_Reduce)" >>> which confuses me, because halo exchange is a point-to-point exchange and >>> not a reduction. Can you clarify? >>> PetscSFReduceBegin/End seems to be the function that do reduction for >>> PetscSF object. What I intended to mention was either reduction or >>> exchange, not specifically intended "reduction". >>> >>> As a follow-up question: >>> Assuming that the code has its own local solution arrays (not Petsc >>> type), and if the plex's DAG indices belong to the halo region are the only >>> information that I want to know (not the detailed section description, such >>> as degree of freedom on vertex, cells, etc.). I have another PetscSection >>> for printing out my solution. >>> Also if I can convert that DAG indices into my local cell/vertex index, >>> can I just use the PetscSF object created from DMGetPointSF(), instead of >>> "creating PetscSection + DMGetSectionSF()"? In other words, can I use the >>> PetscSF object declared from DMGetPointSF() for the halo communication? >>> >> >> No, because that point SF will index information by point number. You >> would need to build a new SF that indexes your dofs. The steps you would >> go through are exactly the same as you would if you just told us what the >> Section is that indexes your data. >> >> Thanks, >> >> Matt >> >> >>> Thanks, >>> Mike >>> >>> >>> The PetscSF that is created automatically is the "point sf" ( >>>> https://petsc.org/main/docs/manualpages/DM/DMGetPointSF/): it says >>>> which mesh points (cells, faces, edges and vertices) are duplicates of >>>> others. >>>> >>>> In a finite volume application we typically want to assign degrees of >>>> freedom just to cells: some applications may only have one degree of >>>> freedom, others may have multiple. >>>> >>>> You encode where you want degrees of freedom in a PetscSection and set >>>> that as the section for the DM in DMSetLocalSection() ( >>>> https://petsc.org/release/docs/manualpages/DM/DMSetLocalSection.html) >>>> >>>> (A c example of these steps that sets degrees of freedom for *vertices* >>>> instead of cells is `src/dm/impls/plex/tutorials/ex7.c`) >>>> >>>> After that you can call DMGetSectionSF() ( >>>> https://petsc.org/main/docs/manualpages/DM/DMGetSectionSF/) to the the >>>> PetscSF that you want for halo exchange: the one for your solution >>>> variables. >>>> >>>> After that, the only calls you typically need in a finite volume code >>>> is PetscSFBcastBegin() to start a halo exchange and PetscSFBcastEnd() to >>>> complete it. >>>> >>>> You say >>>> >>>> > What I want to do is to exchange data (probably just MPI_Reduce) >>>> >>>> which confuses me, because halo exchange is a point-to-point exchange >>>> and not a reduction. Can you clarify? >>>> >>>> >>>> >>>> On Fri, May 20, 2022 at 8:35 PM Mike Michell >>>> wrote: >>>> >>>>> Dear PETSc developer team, >>>>> >>>>> Hi, I am using DMPlex for a finite-volume code and trying to >>>>> understand the usage of PetscSF. What is a typical procedure for doing halo >>>>> data exchange at parallel boundary using PetscSF object on DMPlex? Is there >>>>> any example that I can refer to usage of PetscSF with distributed DMPlex? >>>>> >>>>> Assuming to use the attached mock-up code and mesh, if I give >>>>> "-dm_distribute_overlap 1 -over_dm_view" to run the code, I can see a >>>>> PetscSF object is already created, although I have not called >>>>> "PetscSFCreate" in the code. How can I import & use that PetscSF already >>>>> created by the code to do the halo data exchange? >>>>> >>>>> What I want to do is to exchange data (probably just MPI_Reduce) in a >>>>> parallel boundary region using PetscSF and its functions. I might need to >>>>> have an overlapping layer or not. >>>>> >>>>> Thanks, >>>>> Mike >>>>> >>>> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which their >> experiments lead. >> -- Norbert Wiener >> >> https://www.cse.buffalo.edu/~knepley/ >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: overlap-svg-diagram.png Type: image/png Size: 197226 bytes Desc: not available URL: From mi.mike1021 at gmail.com Sun May 22 15:28:08 2022 From: mi.mike1021 at gmail.com (Mike Michell) Date: Sun, 22 May 2022 15:28:08 -0500 Subject: [petsc-users] PetscSF Object on Distributed DMPlex for Halo Data Exchange In-Reply-To: References: Message-ID: Thanks for the reply. The diagram makes sense and is helpful for understanding 1D representation. However, something is still unclear. From your diagram, the number of roots per process seems to vary according to run arguments, such as "-dm_distribute_overlap", because "the number of roots for a DMPlex is the number of mesh points in the local portion of the mesh (cited from your answer to my question (1))" will end up change according to that argument. However, from my mock-up code, number of roots is independent to -dm_distribute_overlap argument. The summation of "number of roots" through processes was always equal to number of physical vertex on my mesh, if I define the section layout on vertex with 1DOF. But in your diagram example, the summation of "nroots" is larger than the actual number of mesh points, which is 13. Also, it is still unclear how to get the size of "roots" from the PetscSection & PetscSF on distributed DMPlex? In your diagram, how can you tell your code and make it allocate the "nroots=7 for P0, nroots=9 for P1, and nroots=7 for P2" arrays before you call PetscSFBcastBegin/End()? It seems that we need to define arrays having the size of nroots & nleaves before calling PetscSFBcastBegin/End(). Thanks, Mike Here's a diagram of a 1D mesh with overlap and 3 partitions, showing what > the petscsf data is for each. The number of roots is the number of mesh > points in the local representation, and the number of leaves is the number > of mesh points that are duplicates of mesh points on other processes. With > that in mind, answering your questions > > > (1) It seems that the "roots" means the number of vertex not considering > overlap layer, and "leaves" seems the number of distributed vertex for each > processor that includes overlap layer. Can you acknowledge that this is > correct understanding? I have tried to find clearer examples from PETSc > team's articles relevant to Star Forest, but I am still unclear about the > exact relation & graphical notation of roots & leaves in SF if it's the > case of DMPlex solution arrays. > > No, the number of roots for a DMPlex is the number of mesh points in the > local portion of the mesh > > > (2) If it is so, there is an issue that I cannot define "root data" and > "leave data" generally. I am trying to following > "src/vec/is/sf/tutorials/ex1f.F90", however, in that example, size of roots > and leaves are predefined as 6. How can I generalize that? Because I can > get size of leaves using DAG depth(or height), which is equal to number of > vertices each proc has. But, how can I get the size of my "roots" region > from SF? Any example about that? This question is connected to how can I > define "rootdata" for "PetscSFBcastBegin/End()". > > Does the diagram help you generalize? > > > (3) More importantly, with the attached PetscSection & SF layout, my > vector is only resolved for the size equal to "number of roots" for each > proc, but not for the overlapping area(i.e., "leaves"). What I wish to do > is to exchange (or reduce) the solution data between each proc, in the > overlapping region. Can I get some advices why my vector does not encompass > the "leaves" regime? Is there any example doing similar things? > Going back to my first response: if you use a section to say how many > pieces of data are associated with each local mesh point, then a PetscSF is > constructed that requires no more manipulation from you. > > > On Sun, May 22, 2022 at 10:47 AM Mike Michell > wrote: > >> Thank you for the reply. >> The PetscSection and PetscSF objects are defined as in the attached >> mock-up code (Q_PetscSF_1.tar). 1-DOF is defined on vertex as my solution >> is determined on each vertex with 1-DOF from a finite-volume method. >> >> As follow up questions: >> (1) It seems that the "roots" means the number of vertex not considering >> overlap layer, and "leaves" seems the number of distributed vertex for each >> processor that includes overlap layer. Can you acknowledge that this is >> correct understanding? I have tried to find clearer examples from PETSc >> team's articles relevant to Star Forest, but I am still unclear about the >> exact relation & graphical notation of roots & leaves in SF if it's the >> case of DMPlex solution arrays. >> >> (2) If it is so, there is an issue that I cannot define "root data" and >> "leave data" generally. I am trying to following >> "src/vec/is/sf/tutorials/ex1f.F90", however, in that example, size of roots >> and leaves are predefined as 6. How can I generalize that? Because I can >> get size of leaves using DAG depth(or height), which is equal to number of >> vertices each proc has. But, how can I get the size of my "roots" region >> from SF? Any example about that? This question is connected to how can I >> define "rootdata" for "PetscSFBcastBegin/End()". >> >> (3) More importantly, with the attached PetscSection & SF layout, my >> vector is only resolved for the size equal to "number of roots" for each >> proc, but not for the overlapping area(i.e., "leaves"). What I wish to do >> is to exchange (or reduce) the solution data between each proc, in the >> overlapping region. Can I get some advices why my vector does not encompass >> the "leaves" regime? Is there any example doing similar things? >> >> Thanks, >> Mike >> >> >>> On Fri, May 20, 2022 at 4:45 PM Mike Michell >>> wrote: >>> >>>> Thanks for the reply. >>>> >>>> > "What I want to do is to exchange data (probably just MPI_Reduce)" >>>> which confuses me, because halo exchange is a point-to-point exchange and >>>> not a reduction. Can you clarify? >>>> PetscSFReduceBegin/End seems to be the function that do reduction for >>>> PetscSF object. What I intended to mention was either reduction or >>>> exchange, not specifically intended "reduction". >>>> >>>> As a follow-up question: >>>> Assuming that the code has its own local solution arrays (not Petsc >>>> type), and if the plex's DAG indices belong to the halo region are the only >>>> information that I want to know (not the detailed section description, such >>>> as degree of freedom on vertex, cells, etc.). I have another PetscSection >>>> for printing out my solution. >>>> Also if I can convert that DAG indices into my local cell/vertex index, >>>> can I just use the PetscSF object created from DMGetPointSF(), instead of >>>> "creating PetscSection + DMGetSectionSF()"? In other words, can I use the >>>> PetscSF object declared from DMGetPointSF() for the halo communication? >>>> >>> >>> No, because that point SF will index information by point number. You >>> would need to build a new SF that indexes your dofs. The steps you would >>> go through are exactly the same as you would if you just told us what >>> the Section is that indexes your data. >>> >>> Thanks, >>> >>> Matt >>> >>> >>>> Thanks, >>>> Mike >>>> >>>> >>>> The PetscSF that is created automatically is the "point sf" ( >>>>> https://petsc.org/main/docs/manualpages/DM/DMGetPointSF/): it says >>>>> which mesh points (cells, faces, edges and vertices) are duplicates of >>>>> others. >>>>> >>>>> In a finite volume application we typically want to assign degrees of >>>>> freedom just to cells: some applications may only have one degree of >>>>> freedom, others may have multiple. >>>>> >>>>> You encode where you want degrees of freedom in a PetscSection and set >>>>> that as the section for the DM in DMSetLocalSection() ( >>>>> https://petsc.org/release/docs/manualpages/DM/DMSetLocalSection.html) >>>>> >>>>> (A c example of these steps that sets degrees of freedom for >>>>> *vertices* instead of cells is `src/dm/impls/plex/tutorials/ex7.c`) >>>>> >>>>> After that you can call DMGetSectionSF() ( >>>>> https://petsc.org/main/docs/manualpages/DM/DMGetSectionSF/) to the >>>>> the PetscSF that you want for halo exchange: the one for your solution >>>>> variables. >>>>> >>>>> After that, the only calls you typically need in a finite volume code >>>>> is PetscSFBcastBegin() to start a halo exchange and PetscSFBcastEnd() to >>>>> complete it. >>>>> >>>>> You say >>>>> >>>>> > What I want to do is to exchange data (probably just MPI_Reduce) >>>>> >>>>> which confuses me, because halo exchange is a point-to-point exchange >>>>> and not a reduction. Can you clarify? >>>>> >>>>> >>>>> >>>>> On Fri, May 20, 2022 at 8:35 PM Mike Michell >>>>> wrote: >>>>> >>>>>> Dear PETSc developer team, >>>>>> >>>>>> Hi, I am using DMPlex for a finite-volume code and trying to >>>>>> understand the usage of PetscSF. What is a typical procedure for doing halo >>>>>> data exchange at parallel boundary using PetscSF object on DMPlex? Is there >>>>>> any example that I can refer to usage of PetscSF with distributed DMPlex? >>>>>> >>>>>> Assuming to use the attached mock-up code and mesh, if I give >>>>>> "-dm_distribute_overlap 1 -over_dm_view" to run the code, I can see a >>>>>> PetscSF object is already created, although I have not called >>>>>> "PetscSFCreate" in the code. How can I import & use that PetscSF already >>>>>> created by the code to do the halo data exchange? >>>>>> >>>>>> What I want to do is to exchange data (probably just MPI_Reduce) in a >>>>>> parallel boundary region using PetscSF and its functions. I might need to >>>>>> have an overlapping layer or not. >>>>>> >>>>>> Thanks, >>>>>> Mike >>>>>> >>>>> >>> >>> -- >>> What most experimenters take for granted before they begin their >>> experiments is infinitely more interesting than any results to which their >>> experiments lead. >>> -- Norbert Wiener >>> >>> https://www.cse.buffalo.edu/~knepley/ >>> >>> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From s_g at berkeley.edu Sun May 22 15:46:04 2022 From: s_g at berkeley.edu (Sanjay GOVINDJEE) Date: Sun, 22 May 2022 13:46:04 -0700 Subject: [petsc-users] Quad precision In-Reply-To: References: <73abc7fe-ad63-356d-12f3-7c9bd99eb690@berkeley.edu> Message-ID: Informational update on this thread, quadmath looks like it is now available on the M1 with the release of (homebrew) GCC 11.3 and GCC at 12. On Thu, Apr 28, 2022 at 12:16 PM Satish Balay wrote: > Hm - I don't find quadmath.h on M1 > > compute-macos-240-02:petsc svcpetsc$ which gcc-11 > /opt/homebrew/bin/gcc-11 > compute-macos-240-02:petsc svcpetsc$ find /opt/homebrew -name quadmath.h > -print > compute-macos-240-02:petsc svcpetsc$ > > But I see it on intel box > > compute-macos-240-01:~ balay$ which gcc-11 > /usr/local/bin/gcc-11 > compute-macos-240-01:~ balay$ find /usr/local -name quadmath.h > > /usr/local/Cellar/gcc/11.3.0/lib/gcc/11/gcc/x86_64-apple-darwin21/11/include/quadmath.h > compute-macos-240-01:~ balay$ > > > So perhaps this feature is missing with gcc on M1? > > Satish > > On Thu, 28 Apr 2022, Sanjay Govindjee wrote: > > > I'm trying to set up some computations in quad precision on a > > new Mac (OS 12.3.1, M1 2020) and my homebrew gfortran (11.3.0) is > complaining > > about real (kind=16). In fact selected_real_kind(30,300) returns -1. > > > > Has anyone else encountered this issue? and have a work around? > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Sun May 22 20:47:17 2022 From: knepley at gmail.com (Matthew Knepley) Date: Sun, 22 May 2022 21:47:17 -0400 Subject: [petsc-users] PetscSF Object on Distributed DMPlex for Halo Data Exchange In-Reply-To: References: Message-ID: On Sun, May 22, 2022 at 4:28 PM Mike Michell wrote: > Thanks for the reply. The diagram makes sense and is helpful for > understanding 1D representation. > > However, something is still unclear. From your diagram, the number of > roots per process seems to vary according to run arguments, such as > "-dm_distribute_overlap", because "the number of roots for a DMPlex is the > number of mesh points in the local portion of the mesh (cited from your > answer to my question (1))" will end up change according to that argument. > However, from my mock-up code, number of roots is independent to > -dm_distribute_overlap argument. The summation of "number of roots" through > processes was always equal to number of physical vertex on my mesh, if I > define the section layout on vertex with 1DOF. But in your diagram example, > the summation of "nroots" is larger than the actual number of mesh points, > which is 13. > I do not understand your question. Notice the -dm_distribute_overlap does _not_ change the owned points for any process. It only puts in new leaves, so it also never changes the roots for this way of using the SF. > Also, it is still unclear how to get the size of "roots" from the > PetscSection & PetscSF on distributed DMPlex? > For an SF mapping ghost dofs in a global vector, the number of roots is just the size of the local portion of the vector. > In your diagram, how can you tell your code and make it allocate the > "nroots=7 for P0, nroots=9 for P1, and nroots=7 for P2" arrays before you > call PetscSFBcastBegin/End()? It seems that we need to define arrays having > the size of nroots & nleaves before calling PetscSFBcastBegin/End(). > I just want to note that this usage is different from the canonical usage in Plex. It is fine to do this, but this will not match what I do in the library if you look. In Plex, I distinguish two linear spaces: 1) Global space: This is the vector space for the solvers. Each point is uniquely represented and owned by some process 2) Local space: This is the vector space for assembly. Some points are represented multiple times. I create an SF that maps from the global space (roots) to the local space (leaves), and it is called in DMGlobalToLocal() (and associated functions). This is more natural in FEM. You seem to want an SF that maps between global vectors. This will also work. The roots would be the local dofs, and the leaves would be shared dofs. Does this make sense? Thanks, Matt > Thanks, > Mike > > Here's a diagram of a 1D mesh with overlap and 3 partitions, showing what >> the petscsf data is for each. The number of roots is the number of mesh >> points in the local representation, and the number of leaves is the number >> of mesh points that are duplicates of mesh points on other processes. With >> that in mind, answering your questions >> >> > (1) It seems that the "roots" means the number of vertex not >> considering overlap layer, and "leaves" seems the number of distributed >> vertex for each processor that includes overlap layer. Can you acknowledge >> that this is correct understanding? I have tried to find clearer examples >> from PETSc team's articles relevant to Star Forest, but I am still unclear >> about the exact relation & graphical notation of roots & leaves in SF if >> it's the case of DMPlex solution arrays. >> >> No, the number of roots for a DMPlex is the number of mesh points in the >> local portion of the mesh >> >> > (2) If it is so, there is an issue that I cannot define "root data" and >> "leave data" generally. I am trying to following >> "src/vec/is/sf/tutorials/ex1f.F90", however, in that example, size of roots >> and leaves are predefined as 6. How can I generalize that? Because I can >> get size of leaves using DAG depth(or height), which is equal to number of >> vertices each proc has. But, how can I get the size of my "roots" region >> from SF? Any example about that? This question is connected to how can I >> define "rootdata" for "PetscSFBcastBegin/End()". >> >> Does the diagram help you generalize? >> >> > (3) More importantly, with the attached PetscSection & SF layout, my >> vector is only resolved for the size equal to "number of roots" for each >> proc, but not for the overlapping area(i.e., "leaves"). What I wish to do >> is to exchange (or reduce) the solution data between each proc, in the >> overlapping region. Can I get some advices why my vector does not encompass >> the "leaves" regime? Is there any example doing similar things? >> Going back to my first response: if you use a section to say how many >> pieces of data are associated with each local mesh point, then a PetscSF is >> constructed that requires no more manipulation from you. >> >> >> On Sun, May 22, 2022 at 10:47 AM Mike Michell >> wrote: >> >>> Thank you for the reply. >>> The PetscSection and PetscSF objects are defined as in the attached >>> mock-up code (Q_PetscSF_1.tar). 1-DOF is defined on vertex as my solution >>> is determined on each vertex with 1-DOF from a finite-volume method. >>> >>> As follow up questions: >>> (1) It seems that the "roots" means the number of vertex not considering >>> overlap layer, and "leaves" seems the number of distributed vertex for each >>> processor that includes overlap layer. Can you acknowledge that this is >>> correct understanding? I have tried to find clearer examples from PETSc >>> team's articles relevant to Star Forest, but I am still unclear about the >>> exact relation & graphical notation of roots & leaves in SF if it's the >>> case of DMPlex solution arrays. >>> >>> (2) If it is so, there is an issue that I cannot define "root data" and >>> "leave data" generally. I am trying to following >>> "src/vec/is/sf/tutorials/ex1f.F90", however, in that example, size of roots >>> and leaves are predefined as 6. How can I generalize that? Because I can >>> get size of leaves using DAG depth(or height), which is equal to number of >>> vertices each proc has. But, how can I get the size of my "roots" region >>> from SF? Any example about that? This question is connected to how can I >>> define "rootdata" for "PetscSFBcastBegin/End()". >>> >>> (3) More importantly, with the attached PetscSection & SF layout, my >>> vector is only resolved for the size equal to "number of roots" for each >>> proc, but not for the overlapping area(i.e., "leaves"). What I wish to do >>> is to exchange (or reduce) the solution data between each proc, in the >>> overlapping region. Can I get some advices why my vector does not encompass >>> the "leaves" regime? Is there any example doing similar things? >>> >>> Thanks, >>> Mike >>> >>> >>>> On Fri, May 20, 2022 at 4:45 PM Mike Michell >>>> wrote: >>>> >>>>> Thanks for the reply. >>>>> >>>>> > "What I want to do is to exchange data (probably just MPI_Reduce)" >>>>> which confuses me, because halo exchange is a point-to-point exchange and >>>>> not a reduction. Can you clarify? >>>>> PetscSFReduceBegin/End seems to be the function that do reduction for >>>>> PetscSF object. What I intended to mention was either reduction or >>>>> exchange, not specifically intended "reduction". >>>>> >>>>> As a follow-up question: >>>>> Assuming that the code has its own local solution arrays (not Petsc >>>>> type), and if the plex's DAG indices belong to the halo region are the only >>>>> information that I want to know (not the detailed section description, such >>>>> as degree of freedom on vertex, cells, etc.). I have another PetscSection >>>>> for printing out my solution. >>>>> Also if I can convert that DAG indices into my local cell/vertex >>>>> index, can I just use the PetscSF object created from DMGetPointSF(), >>>>> instead of "creating PetscSection + DMGetSectionSF()"? In other words, can >>>>> I use the PetscSF object declared from DMGetPointSF() for the halo >>>>> communication? >>>>> >>>> >>>> No, because that point SF will index information by point number. You >>>> would need to build a new SF that indexes your dofs. The steps you would >>>> go through are exactly the same as you would if you just told us what >>>> the Section is that indexes your data. >>>> >>>> Thanks, >>>> >>>> Matt >>>> >>>> >>>>> Thanks, >>>>> Mike >>>>> >>>>> >>>>> The PetscSF that is created automatically is the "point sf" ( >>>>>> https://petsc.org/main/docs/manualpages/DM/DMGetPointSF/): it says >>>>>> which mesh points (cells, faces, edges and vertices) are duplicates of >>>>>> others. >>>>>> >>>>>> In a finite volume application we typically want to assign degrees of >>>>>> freedom just to cells: some applications may only have one degree of >>>>>> freedom, others may have multiple. >>>>>> >>>>>> You encode where you want degrees of freedom in a PetscSection and >>>>>> set that as the section for the DM in DMSetLocalSection() ( >>>>>> https://petsc.org/release/docs/manualpages/DM/DMSetLocalSection.html) >>>>>> >>>>>> (A c example of these steps that sets degrees of freedom for >>>>>> *vertices* instead of cells is `src/dm/impls/plex/tutorials/ex7.c`) >>>>>> >>>>>> After that you can call DMGetSectionSF() ( >>>>>> https://petsc.org/main/docs/manualpages/DM/DMGetSectionSF/) to the >>>>>> the PetscSF that you want for halo exchange: the one for your solution >>>>>> variables. >>>>>> >>>>>> After that, the only calls you typically need in a finite volume code >>>>>> is PetscSFBcastBegin() to start a halo exchange and PetscSFBcastEnd() to >>>>>> complete it. >>>>>> >>>>>> You say >>>>>> >>>>>> > What I want to do is to exchange data (probably just MPI_Reduce) >>>>>> >>>>>> which confuses me, because halo exchange is a point-to-point exchange >>>>>> and not a reduction. Can you clarify? >>>>>> >>>>>> >>>>>> >>>>>> On Fri, May 20, 2022 at 8:35 PM Mike Michell >>>>>> wrote: >>>>>> >>>>>>> Dear PETSc developer team, >>>>>>> >>>>>>> Hi, I am using DMPlex for a finite-volume code and trying to >>>>>>> understand the usage of PetscSF. What is a typical procedure for doing halo >>>>>>> data exchange at parallel boundary using PetscSF object on DMPlex? Is there >>>>>>> any example that I can refer to usage of PetscSF with distributed DMPlex? >>>>>>> >>>>>>> Assuming to use the attached mock-up code and mesh, if I give >>>>>>> "-dm_distribute_overlap 1 -over_dm_view" to run the code, I can see a >>>>>>> PetscSF object is already created, although I have not called >>>>>>> "PetscSFCreate" in the code. How can I import & use that PetscSF already >>>>>>> created by the code to do the halo data exchange? >>>>>>> >>>>>>> What I want to do is to exchange data (probably just MPI_Reduce) in >>>>>>> a parallel boundary region using PetscSF and its functions. I might need to >>>>>>> have an overlapping layer or not. >>>>>>> >>>>>>> Thanks, >>>>>>> Mike >>>>>>> >>>>>> >>>> >>>> -- >>>> What most experimenters take for granted before they begin their >>>> experiments is infinitely more interesting than any results to which their >>>> experiments lead. >>>> -- Norbert Wiener >>>> >>>> https://www.cse.buffalo.edu/~knepley/ >>>> >>>> >>> -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at petsc.dev Mon May 23 09:14:07 2022 From: bsmith at petsc.dev (Barry Smith) Date: Mon, 23 May 2022 10:14:07 -0400 Subject: [petsc-users] A compilation error about VecGetValues In-Reply-To: References: <54D1F3FD-BADF-4ED4-9FB0-ECA0ABD73DBE@petsc.dev> Message-ID: <4D02A1CA-63F2-4DFC-BAF1-149912D47EF7@petsc.dev> Was PETSc ./configure with --download-hypre ? Send configure.log make.log and the full error message if you still have this failure. > On May 23, 2022, at 9:35 AM, wang yuqi wrote: > > Thank you very much. The problem has been solved. But I have a new problem, when I use the PCHYPRESetType as follows: > call PCHYPRESetType(pc,'boomeramg',ierr) > The problem occurs at compile time: > > undefined reference to `pchypresettype_' > > I checked the PETSC manual, but couldn't find a reason. > I look forward to hearing from you. > Thanks again! > ? Windows ??? ?? > > ???: Barry Smith > ????: 2022?5?23? 0:44 > ???: wang yuqi > ??: petsc-users at mcs.anl.gov > ??: Re: [petsc-users] A compilation error about VecGetValues > > > This should work for at least the last three versions of PETSc. Please upgrade to the latest PETSc version and let us know if you continue to get this error message. > > Yes, for some PETSc versions between 3.8 and 3.14 it may generate this error message with some Fortran compilers. > > Barry > > > > > On May 21, 2022, at 9:23 AM, wang yuqi > wrote: > > > > ? Windows ??? ?? > Hi, Dear developer: > I had compilation problems when using VecGetValues. My procedure is as follows: > program main > #include > use petscvec > Vec x > PetscInt N,i,ione > PetscErrorCode ierr > PetscMPIInt rank > PetscScalar one > > call PetscInitialize(PETSC_NULL_CHARACTER,ierr) > if (ierr .ne. 0) then > print*,'PetscInitialize failed' > stop > endif > one = 1.0 > call MPI_Comm_rank(PETSC_COMM_WORLD,rank,ierr) > N = rank + 1 > call VecCreateMPI(PETSC_COMM_WORLD,N,PETSC_DECIDE,x,ierr) > call VecGetSize(x,N,ierr) > call VecSet(x,one,ierr) > ione = 1 > do 100 i=0,N-rank-1 > call VecSetValues(x,ione,i,one,ADD_VALUES,ierr) > 100 continue > call VecAssemblyBegin(x,ierr) > call VecAssemblyEnd(x,ierr) > > if (rank .eq. 0) then > ione = 0 > call VecGetValues(x,ione,i,one,ierr) > endif > call VecView(x,PETSC_VIEWER_STDOUT_WORLD,ierr) > call VecDestroy(x,ierr) > > call PetscFinalize(ierr) > end PROGRAM main > > The compilation error is as follows: > mpif90 -I/root/soft/petsc/3.8.0/include -o ex2f.o ex2f.F90 > ex2f.F90(62): error #8284: If the actual argument is scalar, the dummy argument shall be scalar unless the actual argument is of type character or is an element of an array that is not assumed shape, pointer, or polymorphic. [C] > call VecGetValues(x,ione,i,one,ierr) > -------------^ > ex2f.F90(62): error #8284: If the actual argument is scalar, the dummy argument shall be scalar unless the actual argument is of type character or is an element of an array that is not assumed shape, pointer, or polymorphic. [D] > call VecGetValues(x,ione,i,one,ierr) > -------------^ > compilation aborted for ex2f.F90 (code 1) > And this error only appears when the Petsc version is higher than 3.8. > I look forward to hearing from you. > Thanks! -------------- next part -------------- An HTML attachment was scrubbed... URL: From rlmackie862 at gmail.com Mon May 23 12:59:41 2022 From: rlmackie862 at gmail.com (Randall Mackie) Date: Mon, 23 May 2022 10:59:41 -0700 Subject: [petsc-users] strumpack in ilu mode Message-ID: <7F3F0713-D79B-41D3-A12A-A1350AD4246A@gmail.com> Dear PETSc team: I am trying to use Strumpack in ILU mode, which is suppose to activate it?s low rank approximation as described on the man page: https://petsc.org/release/docs/manualpages/Mat/MATSOLVERSSTRUMPACK.html and in this exchange a few years ago: https://lists.mcs.anl.gov/pipermail/petsc-users/2017-October/033560.html However, regardless of whether I specify ILU or LU, the behavior is the same, and the solution is in 1 step, meaning (to me) it?s doing the LU direct solve. Furthermore, when I set verbose mode, it doesn?t seem to be activating low rank mode (that is, there are HSS or BLR frontal matrices created). I?m running this example: https://petsc.org/release/src/ksp/ksp/tutorials/ex2.c.html using options: mpiexec -n 4 ./ex2 -pc_type ilu -pc_factor_mat_solver_package strumpack I?m likely doing something wrong, but I can?t figure out what, and would appreciate advice on how to use the strumpack ILU mode. Thanks, Randy M. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at petsc.dev Mon May 23 13:40:35 2022 From: bsmith at petsc.dev (Barry Smith) Date: Mon, 23 May 2022 14:40:35 -0400 Subject: [petsc-users] strumpack in ilu mode In-Reply-To: <7F3F0713-D79B-41D3-A12A-A1350AD4246A@gmail.com> References: <7F3F0713-D79B-41D3-A12A-A1350AD4246A@gmail.com> Message-ID: Randy, Please take a look at https://gitlab.com/petsc/petsc/-/merge_requests/4543/ and try its branch. It has more features and may provide more of what you need. Barry > On May 23, 2022, at 1:59 PM, Randall Mackie wrote: > > Dear PETSc team: > > I am trying to use Strumpack in ILU mode, which is suppose to activate it?s low rank approximation as described on the man page: > > https://petsc.org/release/docs/manualpages/Mat/MATSOLVERSSTRUMPACK.html > > and in this exchange a few years ago: > > https://lists.mcs.anl.gov/pipermail/petsc-users/2017-October/033560.html > > > However, regardless of whether I specify ILU or LU, the behavior is the same, and the solution is in 1 step, meaning (to me) it?s doing the LU direct solve. > Furthermore, when I set verbose mode, it doesn?t seem to be activating low rank mode (that is, there are HSS or BLR frontal matrices created). > > > I?m running this example: > > https://petsc.org/release/src/ksp/ksp/tutorials/ex2.c.html > > using options: > > mpiexec -n 4 ./ex2 -pc_type ilu -pc_factor_mat_solver_package strumpack > > I?m likely doing something wrong, but I can?t figure out what, and would appreciate advice on how to use the strumpack ILU mode. > > Thanks, Randy M. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From pumpkinpieter at gmail.com Mon May 23 20:21:55 2022 From: pumpkinpieter at gmail.com (Pieter VandenBerge) Date: Mon, 23 May 2022 18:21:55 -0700 Subject: [petsc-users] hypre github location change Message-ID: Hi there, I was trying to install petsc on my login in the cluster at my school (Portland State University). I noticed that I wasn't able to download hypre, and that when I looked into why it seems to be because the location of hypre changed on github to https://github.com/hypre-space/hypre, whereas the current configuration is looking for it under Lawrence Livermore github. I went ahead and installed it without hypre for now, and I might be able to make it work on my own by altering the configuration file for hypre, but thought it should more generally be known about. Thanks, Pieter Vandenberge -------------- next part -------------- An HTML attachment was scrubbed... URL: From balay at mcs.anl.gov Mon May 23 20:28:07 2022 From: balay at mcs.anl.gov (Satish Balay) Date: Mon, 23 May 2022 20:28:07 -0500 (CDT) Subject: [petsc-users] hypre github location change In-Reply-To: References: Message-ID: current petsc release [3.17] has [in config/BuildSystem/config/packages/hypre.py]: self.download = ['git://https://github.com/hypre-space/hypre','https://github.com/hypre-space/hypre/archive/'+self.gitcommit+'.tar.gz'] What version of PETSc are you attempting to install? One can always use --download-hypre=TarBallURL - or --download-hypre=git://GitURL - but you would need to figure out the compatible version of hypre to use [with the corresponding PETSc version] However its best to install/use the latest PETSc release. Satish On Mon, 23 May 2022, Pieter VandenBerge wrote: > Hi there, > > I was trying to install petsc on my login in the cluster at my school > (Portland State University). I noticed that I wasn't able to download > hypre, and that when I looked into why it seems to be because the location > of hypre changed on github to https://github.com/hypre-space/hypre, whereas > the current configuration is looking for it under Lawrence Livermore > github. > > I went ahead and installed it without hypre for now, and I might be able to > make it work on my own by altering the configuration file for hypre, but > thought it should more generally be known about. > > Thanks, > Pieter Vandenberge > From jacob.fai at gmail.com Mon May 23 20:28:37 2022 From: jacob.fai at gmail.com (Jacob Faibussowitsch) Date: Mon, 23 May 2022 21:28:37 -0400 Subject: [petsc-users] hypre github location change In-Reply-To: References: Message-ID: <1B55800B-C218-4932-96CB-E9B90DB4B334@gmail.com> What version of petsc are you using? Current main https://gitlab.com/petsc/petsc/-/blob/main/config/BuildSystem/config/packages/hypre.py downloads from the link you cite (see the self.download member variable). Best regards, Jacob Faibussowitsch (Jacob Fai - booss - oh - vitch) > On May 23, 2022, at 21:24, Pieter VandenBerge wrote: > > ? > Hi there, > > I was trying to install petsc on my login in the cluster at my school (Portland State University). I noticed that I wasn't able to download hypre, and that when I looked into why it seems to be because the location of hypre changed on github to https://github.com/hypre-space/hypre, whereas the current configuration is looking for it under Lawrence Livermore github. > > I went ahead and installed it without hypre for now, and I might be able to make it work on my own by altering the configuration file for hypre, but thought it should more generally be known about. > > Thanks, > Pieter Vandenberge -------------- next part -------------- An HTML attachment was scrubbed... URL: From Jiannan_Tu at uml.edu Tue May 24 12:21:04 2022 From: Jiannan_Tu at uml.edu (Tu, Jiannan) Date: Tue, 24 May 2022 17:21:04 +0000 Subject: [petsc-users] Using matrix-free with KSP Message-ID: I want to use a matrix-free matrix to solve a large linear equation system because the matrix is too large to be stored. Petsc user manual describes matrix-free method for SNES with examples. The matrix-free matrices section explains how to set up such a matrix, but I can't find any example of matrix-free method with KSP. I am wondering how to apply the method to KSP iterative solver in parallel. I greatly appreciate your help for me to understand this topic. Jiannan Tu -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at petsc.dev Tue May 24 13:12:46 2022 From: bsmith at petsc.dev (Barry Smith) Date: Tue, 24 May 2022 14:12:46 -0400 Subject: [petsc-users] Using matrix-free with KSP In-Reply-To: References: Message-ID: You can use MatCreateMFFD https://petsc.org/main/docs/manualpages/Mat/MatCreateMFFD/ MatMFFDSetFunction MatSetFromOptions MatMFFDSetBase and provide the matrix to KSP. Note you will need to use -pc_type none or provide your own custom preconditioner with https://petsc.org/main/docs/manualpages/PC/PCSHELL/ > On May 24, 2022, at 1:21 PM, Tu, Jiannan wrote: > > I want to use a matrix-free matrix to solve a large linear equation system because the matrix is too large to be stored. Petsc user manual describes matrix-free method for SNES with examples. The matrix-free matrices section explains how to set up such a matrix, but I can't find any example of matrix-free method with KSP. I am wondering how to apply the method to KSP iterative solver in parallel. > > I greatly appreciate your help for me to understand this topic. > > Jiannan Tu -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Tue May 24 20:46:33 2022 From: knepley at gmail.com (Matthew Knepley) Date: Tue, 24 May 2022 21:46:33 -0400 Subject: [petsc-users] PetscSF Object on Distributed DMPlex for Halo Data Exchange In-Reply-To: References: Message-ID: I will also point out that Toby has created a nice example showing how to create an SF for halo exchange between local vectors. https://gitlab.com/petsc/petsc/-/merge_requests/5267 Thanks, Matt On Sun, May 22, 2022 at 9:47 PM Matthew Knepley wrote: > On Sun, May 22, 2022 at 4:28 PM Mike Michell > wrote: > >> Thanks for the reply. The diagram makes sense and is helpful for >> understanding 1D representation. >> >> However, something is still unclear. From your diagram, the number of >> roots per process seems to vary according to run arguments, such as >> "-dm_distribute_overlap", because "the number of roots for a DMPlex is the >> number of mesh points in the local portion of the mesh (cited from your >> answer to my question (1))" will end up change according to that argument. >> However, from my mock-up code, number of roots is independent to >> -dm_distribute_overlap argument. The summation of "number of roots" through >> processes was always equal to number of physical vertex on my mesh, if I >> define the section layout on vertex with 1DOF. But in your diagram example, >> the summation of "nroots" is larger than the actual number of mesh points, >> which is 13. >> > > I do not understand your question. Notice the -dm_distribute_overlap does > _not_ change the owned points for any process. It only puts in new leaves, > so it also never > changes the roots for this way of using the SF. > > >> Also, it is still unclear how to get the size of "roots" from the >> PetscSection & PetscSF on distributed DMPlex? >> > > For an SF mapping ghost dofs in a global vector, the number of roots is > just the size of the local portion of the vector. > > >> In your diagram, how can you tell your code and make it allocate the >> "nroots=7 for P0, nroots=9 for P1, and nroots=7 for P2" arrays before you >> call PetscSFBcastBegin/End()? It seems that we need to define arrays having >> the size of nroots & nleaves before calling PetscSFBcastBegin/End(). >> > > I just want to note that this usage is different from the canonical usage > in Plex. It is fine to do this, but this will not match what I do in the > library if you look. > In Plex, I distinguish two linear spaces: > > 1) Global space: This is the vector space for the solvers. Each point is > uniquely represented and owned by some process > > 2) Local space: This is the vector space for assembly. Some points are > represented multiple times. > > I create an SF that maps from the global space (roots) to the local space > (leaves), and it is called in DMGlobalToLocal() (and associated functions). > This > is more natural in FEM. You seem to want an SF that maps between global > vectors. This will also work. The roots would be the local dofs, and the > leaves > would be shared dofs. > > Does this make sense? > > Thanks, > > Matt > > >> Thanks, >> Mike >> >> Here's a diagram of a 1D mesh with overlap and 3 partitions, showing what >>> the petscsf data is for each. The number of roots is the number of mesh >>> points in the local representation, and the number of leaves is the number >>> of mesh points that are duplicates of mesh points on other processes. With >>> that in mind, answering your questions >>> >>> > (1) It seems that the "roots" means the number of vertex not >>> considering overlap layer, and "leaves" seems the number of distributed >>> vertex for each processor that includes overlap layer. Can you acknowledge >>> that this is correct understanding? I have tried to find clearer examples >>> from PETSc team's articles relevant to Star Forest, but I am still unclear >>> about the exact relation & graphical notation of roots & leaves in SF if >>> it's the case of DMPlex solution arrays. >>> >>> No, the number of roots for a DMPlex is the number of mesh points in the >>> local portion of the mesh >>> >>> > (2) If it is so, there is an issue that I cannot define "root data" >>> and "leave data" generally. I am trying to following >>> "src/vec/is/sf/tutorials/ex1f.F90", however, in that example, size of roots >>> and leaves are predefined as 6. How can I generalize that? Because I can >>> get size of leaves using DAG depth(or height), which is equal to number of >>> vertices each proc has. But, how can I get the size of my "roots" region >>> from SF? Any example about that? This question is connected to how can I >>> define "rootdata" for "PetscSFBcastBegin/End()". >>> >>> Does the diagram help you generalize? >>> >>> > (3) More importantly, with the attached PetscSection & SF layout, my >>> vector is only resolved for the size equal to "number of roots" for each >>> proc, but not for the overlapping area(i.e., "leaves"). What I wish to do >>> is to exchange (or reduce) the solution data between each proc, in the >>> overlapping region. Can I get some advices why my vector does not encompass >>> the "leaves" regime? Is there any example doing similar things? >>> Going back to my first response: if you use a section to say how many >>> pieces of data are associated with each local mesh point, then a PetscSF is >>> constructed that requires no more manipulation from you. >>> >>> >>> On Sun, May 22, 2022 at 10:47 AM Mike Michell >>> wrote: >>> >>>> Thank you for the reply. >>>> The PetscSection and PetscSF objects are defined as in the attached >>>> mock-up code (Q_PetscSF_1.tar). 1-DOF is defined on vertex as my solution >>>> is determined on each vertex with 1-DOF from a finite-volume method. >>>> >>>> As follow up questions: >>>> (1) It seems that the "roots" means the number of vertex not >>>> considering overlap layer, and "leaves" seems the number of distributed >>>> vertex for each processor that includes overlap layer. Can you acknowledge >>>> that this is correct understanding? I have tried to find clearer examples >>>> from PETSc team's articles relevant to Star Forest, but I am still unclear >>>> about the exact relation & graphical notation of roots & leaves in SF if >>>> it's the case of DMPlex solution arrays. >>>> >>>> (2) If it is so, there is an issue that I cannot define "root data" and >>>> "leave data" generally. I am trying to following >>>> "src/vec/is/sf/tutorials/ex1f.F90", however, in that example, size of roots >>>> and leaves are predefined as 6. How can I generalize that? Because I can >>>> get size of leaves using DAG depth(or height), which is equal to number of >>>> vertices each proc has. But, how can I get the size of my "roots" region >>>> from SF? Any example about that? This question is connected to how can I >>>> define "rootdata" for "PetscSFBcastBegin/End()". >>>> >>>> (3) More importantly, with the attached PetscSection & SF layout, my >>>> vector is only resolved for the size equal to "number of roots" for each >>>> proc, but not for the overlapping area(i.e., "leaves"). What I wish to do >>>> is to exchange (or reduce) the solution data between each proc, in the >>>> overlapping region. Can I get some advices why my vector does not encompass >>>> the "leaves" regime? Is there any example doing similar things? >>>> >>>> Thanks, >>>> Mike >>>> >>>> >>>>> On Fri, May 20, 2022 at 4:45 PM Mike Michell >>>>> wrote: >>>>> >>>>>> Thanks for the reply. >>>>>> >>>>>> > "What I want to do is to exchange data (probably just MPI_Reduce)" >>>>>> which confuses me, because halo exchange is a point-to-point exchange and >>>>>> not a reduction. Can you clarify? >>>>>> PetscSFReduceBegin/End seems to be the function that do reduction for >>>>>> PetscSF object. What I intended to mention was either reduction or >>>>>> exchange, not specifically intended "reduction". >>>>>> >>>>>> As a follow-up question: >>>>>> Assuming that the code has its own local solution arrays (not Petsc >>>>>> type), and if the plex's DAG indices belong to the halo region are the only >>>>>> information that I want to know (not the detailed section description, such >>>>>> as degree of freedom on vertex, cells, etc.). I have another PetscSection >>>>>> for printing out my solution. >>>>>> Also if I can convert that DAG indices into my local cell/vertex >>>>>> index, can I just use the PetscSF object created from DMGetPointSF(), >>>>>> instead of "creating PetscSection + DMGetSectionSF()"? In other words, can >>>>>> I use the PetscSF object declared from DMGetPointSF() for the halo >>>>>> communication? >>>>>> >>>>> >>>>> No, because that point SF will index information by point number. You >>>>> would need to build a new SF that indexes your dofs. The steps you would >>>>> go through are exactly the same as you would if you just told us what >>>>> the Section is that indexes your data. >>>>> >>>>> Thanks, >>>>> >>>>> Matt >>>>> >>>>> >>>>>> Thanks, >>>>>> Mike >>>>>> >>>>>> >>>>>> The PetscSF that is created automatically is the "point sf" ( >>>>>>> https://petsc.org/main/docs/manualpages/DM/DMGetPointSF/): it says >>>>>>> which mesh points (cells, faces, edges and vertices) are duplicates of >>>>>>> others. >>>>>>> >>>>>>> In a finite volume application we typically want to assign degrees >>>>>>> of freedom just to cells: some applications may only have one degree of >>>>>>> freedom, others may have multiple. >>>>>>> >>>>>>> You encode where you want degrees of freedom in a PetscSection and >>>>>>> set that as the section for the DM in DMSetLocalSection() ( >>>>>>> https://petsc.org/release/docs/manualpages/DM/DMSetLocalSection.html >>>>>>> ) >>>>>>> >>>>>>> (A c example of these steps that sets degrees of freedom for >>>>>>> *vertices* instead of cells is `src/dm/impls/plex/tutorials/ex7.c`) >>>>>>> >>>>>>> After that you can call DMGetSectionSF() ( >>>>>>> https://petsc.org/main/docs/manualpages/DM/DMGetSectionSF/) to the >>>>>>> the PetscSF that you want for halo exchange: the one for your solution >>>>>>> variables. >>>>>>> >>>>>>> After that, the only calls you typically need in a finite volume >>>>>>> code is PetscSFBcastBegin() to start a halo exchange and PetscSFBcastEnd() >>>>>>> to complete it. >>>>>>> >>>>>>> You say >>>>>>> >>>>>>> > What I want to do is to exchange data (probably just MPI_Reduce) >>>>>>> >>>>>>> which confuses me, because halo exchange is a point-to-point >>>>>>> exchange and not a reduction. Can you clarify? >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Fri, May 20, 2022 at 8:35 PM Mike Michell >>>>>>> wrote: >>>>>>> >>>>>>>> Dear PETSc developer team, >>>>>>>> >>>>>>>> Hi, I am using DMPlex for a finite-volume code and trying to >>>>>>>> understand the usage of PetscSF. What is a typical procedure for doing halo >>>>>>>> data exchange at parallel boundary using PetscSF object on DMPlex? Is there >>>>>>>> any example that I can refer to usage of PetscSF with distributed DMPlex? >>>>>>>> >>>>>>>> Assuming to use the attached mock-up code and mesh, if I give >>>>>>>> "-dm_distribute_overlap 1 -over_dm_view" to run the code, I can see a >>>>>>>> PetscSF object is already created, although I have not called >>>>>>>> "PetscSFCreate" in the code. How can I import & use that PetscSF already >>>>>>>> created by the code to do the halo data exchange? >>>>>>>> >>>>>>>> What I want to do is to exchange data (probably just MPI_Reduce) in >>>>>>>> a parallel boundary region using PetscSF and its functions. I might need to >>>>>>>> have an overlapping layer or not. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Mike >>>>>>>> >>>>>>> >>>>> >>>>> -- >>>>> What most experimenters take for granted before they begin their >>>>> experiments is infinitely more interesting than any results to which their >>>>> experiments lead. >>>>> -- Norbert Wiener >>>>> >>>>> https://www.cse.buffalo.edu/~knepley/ >>>>> >>>>> >>>> > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > > https://www.cse.buffalo.edu/~knepley/ > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From sami.ben-elhaj-salah at ensma.fr Wed May 25 12:16:46 2022 From: sami.ben-elhaj-salah at ensma.fr (Sami BEN ELHAJ SALAH) Date: Wed, 25 May 2022 19:16:46 +0200 Subject: [petsc-users] Question about DMPlexDistribute & distribute mesh over processes Message-ID: <7E83D8A1-E911-4B48-98C9-BCFA9429AA11@ensma.fr> Dear PETSc developer team, I m trying to create ? jacobian_matrix from distributed DM. I have followed the two examples (snes/tests/ex2.c and ex56.c). So I wrote this routine: PetscDM orig_dm; PetscDM dist_dm = NULL; PetscPartitioner part; DMPlexGetPartitioner(orig_dm, &part); PetscPartitionerSetType(part, PETSCPARTITIONERPARMETIS); DMPlexDistribute(orig_dm, 0, NULL, &dist_dm); PetscErrorCode err = DMCreateMatrix(dist_dm, &jacobian_matrix); PetscInt M, N, m, n; MatGetSize(jacobian_matrix, &M, &N); MatGetLocalSize(jacobian_matrix, &m, &n); Then I run my code with 2 processes and I obtained this result: Size from jacobian_matrix: M=0 m =0 (this result is the same in all processes). I didn't understand if I forgot something in my code to obtain the correct values for the local and global sizes for the jacobean_matrix? (I mean if my code is missing something to obtain a distributed mesh over processes ?) Thank you in advance for any help! Sami, -- Dr. Sami BEN ELHAJ SALAH Ing?nieur de Recherche (CNRS) Institut Pprime - ISAE - ENSMA Mobile: 06.62.51.26.74 Email: sami.ben-elhaj-salah at ensma.fr www.samibenelhajsalah.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From toby.isaac at gmail.com Wed May 25 12:41:30 2022 From: toby.isaac at gmail.com (Toby Isaac) Date: Wed, 25 May 2022 13:41:30 -0400 Subject: [petsc-users] Question about DMPlexDistribute & distribute mesh over processes In-Reply-To: <7E83D8A1-E911-4B48-98C9-BCFA9429AA11@ensma.fr> References: <7E83D8A1-E911-4B48-98C9-BCFA9429AA11@ensma.fr> Message-ID: Hi Sami, Just to verify: if you call DMCreateMatrix() on orig_dm, do you get a Jacobian matrix? The DMPlex must be told what kind of discretized fields you want a matrix for and what equations you are discretizing. This is handled by the PetscDS object. In snes/tutorials/ex59.c, see the code after DMGetDS() for an example. - Toby On Wed, May 25, 2022 at 1:17 PM Sami BEN ELHAJ SALAH wrote: > > Dear PETSc developer team, > > I m trying to create ? jacobian_matrix from distributed DM. I have followed the two examples (snes/tests/ex2.c and ex56.c). So I wrote this routine: > > PetscDM orig_dm; > PetscDM dist_dm = NULL; > PetscPartitioner part; > DMPlexGetPartitioner(orig_dm, &part); > PetscPartitionerSetType(part, PETSCPARTITIONERPARMETIS); > DMPlexDistribute(orig_dm, 0, NULL, &dist_dm); > > PetscErrorCode err = DMCreateMatrix(dist_dm, &jacobian_matrix); > PetscInt M, N, m, n; > MatGetSize(jacobian_matrix, &M, &N); > MatGetLocalSize(jacobian_matrix, &m, &n); > > Then I run my code with 2 processes and I obtained this result: > Size from jacobian_matrix: M=0 m =0 (this result is the same in all processes). > > I didn't understand if I forgot something in my code to obtain the correct values for the local and global sizes for the jacobean_matrix? (I mean if my code is missing something to obtain a distributed mesh over processes ?) > > Thank you in advance for any help! > Sami, > > > -- > Dr. Sami BEN ELHAJ SALAH > Ing?nieur de Recherche (CNRS) > Institut Pprime - ISAE - ENSMA > Mobile: 06.62.51.26.74 > Email: sami.ben-elhaj-salah at ensma.fr > www.samibenelhajsalah.com > > > From rlmackie862 at gmail.com Wed May 25 12:49:31 2022 From: rlmackie862 at gmail.com (Randall Mackie) Date: Wed, 25 May 2022 10:49:31 -0700 Subject: [petsc-users] strumpack in ilu mode In-Reply-To: References: <7F3F0713-D79B-41D3-A12A-A1350AD4246A@gmail.com> Message-ID: <344779AF-74B4-453C-A733-0B6C454D91AD@gmail.com> Thanks Barry, This branch works and indeed opens up more features that I am trying out. Randy > On May 23, 2022, at 11:40 AM, Barry Smith wrote: > > > Randy, > > Please take a look at https://gitlab.com/petsc/petsc/-/merge_requests/4543/ and try its branch. It has more features and may provide more of what you need. > > Barry > > >> On May 23, 2022, at 1:59 PM, Randall Mackie > wrote: >> >> Dear PETSc team: >> >> I am trying to use Strumpack in ILU mode, which is suppose to activate it?s low rank approximation as described on the man page: >> >> https://petsc.org/release/docs/manualpages/Mat/MATSOLVERSSTRUMPACK.html >> >> and in this exchange a few years ago: >> >> https://lists.mcs.anl.gov/pipermail/petsc-users/2017-October/033560.html >> >> >> However, regardless of whether I specify ILU or LU, the behavior is the same, and the solution is in 1 step, meaning (to me) it?s doing the LU direct solve. >> Furthermore, when I set verbose mode, it doesn?t seem to be activating low rank mode (that is, there are HSS or BLR frontal matrices created). >> >> >> I?m running this example: >> >> https://petsc.org/release/src/ksp/ksp/tutorials/ex2.c.html >> >> using options: >> >> mpiexec -n 4 ./ex2 -pc_type ilu -pc_factor_mat_solver_package strumpack >> >> I?m likely doing something wrong, but I can?t figure out what, and would appreciate advice on how to use the strumpack ILU mode. >> >> Thanks, Randy M. >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sami.ben-elhaj-salah at ensma.fr Fri May 27 08:40:09 2022 From: sami.ben-elhaj-salah at ensma.fr (Sami BEN ELHAJ SALAH) Date: Fri, 27 May 2022 15:40:09 +0200 Subject: [petsc-users] Question about DMPlexDistribute & distribute mesh over processes In-Reply-To: References: <7E83D8A1-E911-4B48-98C9-BCFA9429AA11@ensma.fr> Message-ID: <15043588-F026-46C0-8334-E7ABA28F44FF@ensma.fr> Hello Isaac, Thank you for your reply! Let me confirm that when I use DMCreateMatrix() with the orig_dm, I got my jacobian_matrix. Also, I have succeeded to solve my system and my solution was converged by using one process. Let me give you some other information about my code. Currently, I am using my own discretization system and not the PetscDS object. Considering the nonlinear solver SNES, I use the following routines (like the basic snes usage): - SNESSetFunction(snes,residual_vector,compute_residual ,(void*) _finite_element_formulation) - SNESSetJacobian(snes, jacobian.matrix(), _jacobian_precond_matrix, compute_jacobian, (void*) _finite_element_formulation) - SNESSolve(snes,NULL,x) Regarding your last answer, I will try to reformulate my question as follows: Using a distributed dm instead of the original dm (not distributed dm) and my own discretization system (not the PetscDS object), should I add something specific to get a distributed jacobian_matrix over processes? I precise that I just replaced the orig_dm by a distributed mesh with the routine that I wrote in my first mail. So is it enough ? Thank you and have a good day, Sami, -- Dr. Sami BEN ELHAJ SALAH Ing?nieur de Recherche (CNRS) Institut Pprime - ISAE - ENSMA Mobile: 06.62.51.26.74 Email: sami.ben-elhaj-salah at ensma.fr www.samibenelhajsalah.com > Le 25 mai 2022 ? 19:41, Toby Isaac a ?crit : > > Hi Sami, > > Just to verify: if you call DMCreateMatrix() on orig_dm, do you get a > Jacobian matrix? > > The DMPlex must be told what kind of discretized fields you want a > matrix for and what equations you are discretizing. This is handled > by the PetscDS object. In snes/tutorials/ex59.c, see the code after > DMGetDS() for an example. > > - Toby > > On Wed, May 25, 2022 at 1:17 PM Sami BEN ELHAJ SALAH > wrote: >> >> Dear PETSc developer team, >> >> I m trying to create ? jacobian_matrix from distributed DM. I have followed the two examples (snes/tests/ex2.c and ex56.c). So I wrote this routine: >> >> PetscDM orig_dm; >> PetscDM dist_dm = NULL; >> PetscPartitioner part; >> DMPlexGetPartitioner(orig_dm, &part); >> PetscPartitionerSetType(part, PETSCPARTITIONERPARMETIS); >> DMPlexDistribute(orig_dm, 0, NULL, &dist_dm); >> >> PetscErrorCode err = DMCreateMatrix(dist_dm, &jacobian_matrix); >> PetscInt M, N, m, n; >> MatGetSize(jacobian_matrix, &M, &N); >> MatGetLocalSize(jacobian_matrix, &m, &n); >> >> Then I run my code with 2 processes and I obtained this result: >> Size from jacobian_matrix: M=0 m =0 (this result is the same in all processes). >> >> I didn't understand if I forgot something in my code to obtain the correct values for the local and global sizes for the jacobean_matrix? (I mean if my code is missing something to obtain a distributed mesh over processes ?) >> >> Thank you in advance for any help! >> Sami, >> >> >> -- >> Dr. Sami BEN ELHAJ SALAH >> Ing?nieur de Recherche (CNRS) >> Institut Pprime - ISAE - ENSMA >> Mobile: 06.62.51.26.74 >> Email: sami.ben-elhaj-salah at ensma.fr >> www.samibenelhajsalah.com >> >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Fri May 27 13:45:03 2022 From: knepley at gmail.com (Matthew Knepley) Date: Fri, 27 May 2022 14:45:03 -0400 Subject: [petsc-users] Question about DMPlexDistribute & distribute mesh over processes In-Reply-To: <15043588-F026-46C0-8334-E7ABA28F44FF@ensma.fr> References: <7E83D8A1-E911-4B48-98C9-BCFA9429AA11@ensma.fr> <15043588-F026-46C0-8334-E7ABA28F44FF@ensma.fr> Message-ID: On Fri, May 27, 2022 at 9:42 AM Sami BEN ELHAJ SALAH < sami.ben-elhaj-salah at ensma.fr> wrote: > Hello Isaac, > > Thank you for your reply! > > Let me confirm that when I use DMCreateMatrix() with the orig_dm, I got my > jacobian_matrix. Also, I have succeeded to solve my system and my solution > was converged* by using one process*. > Let me give you some other information about my code. Currently, I am > using my own discretization system and not the PetscDS object. Considering > the nonlinear solver SNES, I use the following routines (like the basic > snes usage): > - SNESSetFunction(snes,residual_vector,compute_residual ,(void*) > _finite_element_formulation) > - SNESSetJacobian(snes, jacobian.matrix(), _jacobian_precond_matrix, > compute_jacobian, (void*) _finite_element_formulation) > - SNESSolve(snes,NULL,x) > > Regarding your last answer, I will try to reformulate my question as > follows: > Using a distributed dm instead of the original dm (not distributed dm) and > my own discretization system (not the PetscDS object), > You do not have to use PetscDS, but the DM does need to a PetscSection in order to compute sizes and sparsity patterns. Do you have that? Thanks, Matt > should I add something specific to get a distributed jacobian_matrix over > processes? > I precise that I just replaced the orig_dm by a distributed mesh with the > routine that I wrote in my first mail. So is it enough ? > > Thank you and have a good day, > Sami, > > > -- > Dr. Sami BEN ELHAJ SALAH > Ing?nieur de Recherche (CNRS) > Institut Pprime - ISAE - ENSMA > Mobile: 06.62.51.26.74 > Email: sami.ben-elhaj-salah at ensma.fr > www.samibenelhajsalah.com > > > > > Le 25 mai 2022 ? 19:41, Toby Isaac a ?crit : > > Hi Sami, > > Just to verify: if you call DMCreateMatrix() on orig_dm, do you get a > Jacobian matrix? > > The DMPlex must be told what kind of discretized fields you want a > matrix for and what equations you are discretizing. This is handled > by the PetscDS object. In snes/tutorials/ex59.c, see the code after > DMGetDS() for an example. > > - Toby > > On Wed, May 25, 2022 at 1:17 PM Sami BEN ELHAJ SALAH > wrote: > > > Dear PETSc developer team, > > I m trying to create ? jacobian_matrix from distributed DM. I have > followed the two examples (snes/tests/ex2.c and ex56.c). So I wrote this > routine: > > PetscDM orig_dm; > PetscDM dist_dm = NULL; > PetscPartitioner part; > DMPlexGetPartitioner(orig_dm, &part); > PetscPartitionerSetType(part, PETSCPARTITIONERPARMETIS); > DMPlexDistribute(orig_dm, 0, NULL, &dist_dm); > > PetscErrorCode err = DMCreateMatrix(dist_dm, &jacobian_matrix); > PetscInt M, N, m, n; > MatGetSize(jacobian_matrix, &M, &N); > MatGetLocalSize(jacobian_matrix, &m, &n); > > Then I run my code with 2 processes and I obtained this result: > Size from jacobian_matrix: M=0 m =0 (this result is the same in all > processes). > > I didn't understand if I forgot something in my code to obtain the correct > values for the local and global sizes for the jacobean_matrix? (I mean if > my code is missing something to obtain a distributed mesh over processes ?) > > Thank you in advance for any help! > Sami, > > > -- > Dr. Sami BEN ELHAJ SALAH > Ing?nieur de Recherche (CNRS) > Institut Pprime - ISAE - ENSMA > Mobile: 06.62.51.26.74 > Email: sami.ben-elhaj-salah at ensma.fr > www.samibenelhajsalah.com > > > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From sami.ben-elhaj-salah at ensma.fr Sat May 28 12:35:28 2022 From: sami.ben-elhaj-salah at ensma.fr (Sami BEN ELHAJ SALAH) Date: Sat, 28 May 2022 19:35:28 +0200 Subject: [petsc-users] Question about DMPlexDistribute & distribute mesh over processes In-Reply-To: References: <7E83D8A1-E911-4B48-98C9-BCFA9429AA11@ensma.fr> <15043588-F026-46C0-8334-E7ABA28F44FF@ensma.fr> Message-ID: Hi Matthew, Thank you for your response. I don't have that. My DM object is not linked to PetscSection yet. I'll try that. Is there any example that manages this case? (DMPlexDistribute & PetscSection) or any guideline will be helpful. Thanks in advance, Sami, -- Dr. Sami BEN ELHAJ SALAH Ing?nieur de Recherche (CNRS) Institut Pprime - ISAE - ENSMA Mobile: 06.62.51.26.74 Email: sami.ben-elhaj-salah at ensma.fr www.samibenelhajsalah.com > Le 27 mai 2022 ? 20:45, Matthew Knepley a ?crit : > > On Fri, May 27, 2022 at 9:42 AM Sami BEN ELHAJ SALAH > wrote: > Hello Isaac, > > Thank you for your reply! > > Let me confirm that when I use DMCreateMatrix() with the orig_dm, I got my jacobian_matrix. Also, I have succeeded to solve my system and my solution was converged by using one process. > Let me give you some other information about my code. Currently, I am using my own discretization system and not the PetscDS object. Considering the nonlinear solver SNES, I use the following routines (like the basic snes usage): > - SNESSetFunction(snes,residual_vector,compute_residual ,(void*) _finite_element_formulation) > - SNESSetJacobian(snes, jacobian.matrix(), _jacobian_precond_matrix, compute_jacobian, (void*) _finite_element_formulation) > - SNESSolve(snes,NULL,x) > > Regarding your last answer, I will try to reformulate my question as follows: > Using a distributed dm instead of the original dm (not distributed dm) and my own discretization system (not the PetscDS object), > > You do not have to use PetscDS, but the DM does need to a PetscSection in order to compute sizes and sparsity patterns. Do you have that? > > Thanks, > > Matt > > should I add something specific to get a distributed jacobian_matrix over processes? > I precise that I just replaced the orig_dm by a distributed mesh with the routine that I wrote in my first mail. So is it enough ? > > Thank you and have a good day, > Sami, > > > -- > Dr. Sami BEN ELHAJ SALAH > Ing?nieur de Recherche (CNRS) > Institut Pprime - ISAE - ENSMA > Mobile: 06.62.51.26.74 > Email: sami.ben-elhaj-salah at ensma.fr > www.samibenelhajsalah.com > > > >> Le 25 mai 2022 ? 19:41, Toby Isaac > a ?crit : >> >> Hi Sami, >> >> Just to verify: if you call DMCreateMatrix() on orig_dm, do you get a >> Jacobian matrix? >> >> The DMPlex must be told what kind of discretized fields you want a >> matrix for and what equations you are discretizing. This is handled >> by the PetscDS object. In snes/tutorials/ex59.c, see the code after >> DMGetDS() for an example. >> >> - Toby >> >> On Wed, May 25, 2022 at 1:17 PM Sami BEN ELHAJ SALAH >> > wrote: >>> >>> Dear PETSc developer team, >>> >>> I m trying to create ? jacobian_matrix from distributed DM. I have followed the two examples (snes/tests/ex2.c and ex56.c). So I wrote this routine: >>> >>> PetscDM orig_dm; >>> PetscDM dist_dm = NULL; >>> PetscPartitioner part; >>> DMPlexGetPartitioner(orig_dm, &part); >>> PetscPartitionerSetType(part, PETSCPARTITIONERPARMETIS); >>> DMPlexDistribute(orig_dm, 0, NULL, &dist_dm); >>> >>> PetscErrorCode err = DMCreateMatrix(dist_dm, &jacobian_matrix); >>> PetscInt M, N, m, n; >>> MatGetSize(jacobian_matrix, &M, &N); >>> MatGetLocalSize(jacobian_matrix, &m, &n); >>> >>> Then I run my code with 2 processes and I obtained this result: >>> Size from jacobian_matrix: M=0 m =0 (this result is the same in all processes). >>> >>> I didn't understand if I forgot something in my code to obtain the correct values for the local and global sizes for the jacobean_matrix? (I mean if my code is missing something to obtain a distributed mesh over processes ?) >>> >>> Thank you in advance for any help! >>> Sami, >>> >>> >>> -- >>> Dr. Sami BEN ELHAJ SALAH >>> Ing?nieur de Recherche (CNRS) >>> Institut Pprime - ISAE - ENSMA >>> Mobile: 06.62.51.26.74 >>> Email: sami.ben-elhaj-salah at ensma.fr >>> www.samibenelhajsalah.com >>> >>> >>> > > > > -- > What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. > -- Norbert Wiener > > https://www.cse.buffalo.edu/~knepley/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Sat May 28 13:19:53 2022 From: knepley at gmail.com (Matthew Knepley) Date: Sat, 28 May 2022 14:19:53 -0400 Subject: [petsc-users] Question about DMPlexDistribute & distribute mesh over processes In-Reply-To: References: <7E83D8A1-E911-4B48-98C9-BCFA9429AA11@ensma.fr> <15043588-F026-46C0-8334-E7ABA28F44FF@ensma.fr> Message-ID: On Sat, May 28, 2022 at 1:35 PM Sami BEN ELHAJ SALAH < sami.ben-elhaj-salah at ensma.fr> wrote: > Hi Matthew, > > Thank you for your response. > > I don't have that. My DM object is not linked to PetscSection yet. I'll > try that. > Is there any example that manages this case? (DMPlexDistribute & > PetscSection) or any guideline will be helpful. > Here is an example where we create a section without DS. THanks, Matt > Thanks in advance, > > Sami, > > -- > Dr. Sami BEN ELHAJ SALAH > Ing?nieur de Recherche (CNRS) > Institut Pprime - ISAE - ENSMA > Mobile: 06.62.51.26.74 > Email: sami.ben-elhaj-salah at ensma.fr > www.samibenelhajsalah.com > > > > > Le 27 mai 2022 ? 20:45, Matthew Knepley a ?crit : > > On Fri, May 27, 2022 at 9:42 AM Sami BEN ELHAJ SALAH < > sami.ben-elhaj-salah at ensma.fr> wrote: > >> Hello Isaac, >> >> Thank you for your reply! >> >> Let me confirm that when I use DMCreateMatrix() with the orig_dm, I got >> my jacobian_matrix. Also, I have succeeded to solve my system and my >> solution was converged* by using one process*. >> Let me give you some other information about my code. Currently, I am >> using my own discretization system and not the PetscDS object. Considering >> the nonlinear solver SNES, I use the following routines (like the basic >> snes usage): >> - SNESSetFunction(snes,residual_vector,compute_residual ,(void*) >> _finite_element_formulation) >> - SNESSetJacobian(snes, jacobian.matrix(), _jacobian_precond_matrix, >> compute_jacobian, (void*) _finite_element_formulation) >> - SNESSolve(snes,NULL,x) >> >> Regarding your last answer, I will try to reformulate my question as >> follows: >> Using a distributed dm instead of the original dm (not distributed dm) >> and my own discretization system (not the PetscDS object), >> > > You do not have to use PetscDS, but the DM does need to a PetscSection in > order to compute sizes and sparsity patterns. Do you have that? > > Thanks, > > Matt > > >> should I add something specific to get a distributed jacobian_matrix over >> processes? >> I precise that I just replaced the orig_dm by a distributed mesh with the >> routine that I wrote in my first mail. So is it enough ? >> >> Thank you and have a good day, >> Sami, >> >> >> -- >> Dr. Sami BEN ELHAJ SALAH >> Ing?nieur de Recherche (CNRS) >> Institut Pprime - ISAE - ENSMA >> Mobile: 06.62.51.26.74 >> Email: sami.ben-elhaj-salah at ensma.fr >> www.samibenelhajsalah.com >> >> >> >> >> Le 25 mai 2022 ? 19:41, Toby Isaac a ?crit : >> >> Hi Sami, >> >> Just to verify: if you call DMCreateMatrix() on orig_dm, do you get a >> Jacobian matrix? >> >> The DMPlex must be told what kind of discretized fields you want a >> matrix for and what equations you are discretizing. This is handled >> by the PetscDS object. In snes/tutorials/ex59.c, see the code after >> DMGetDS() for an example. >> >> - Toby >> >> On Wed, May 25, 2022 at 1:17 PM Sami BEN ELHAJ SALAH >> wrote: >> >> >> Dear PETSc developer team, >> >> I m trying to create ? jacobian_matrix from distributed DM. I have >> followed the two examples (snes/tests/ex2.c and ex56.c). So I wrote this >> routine: >> >> PetscDM orig_dm; >> PetscDM dist_dm = NULL; >> PetscPartitioner part; >> DMPlexGetPartitioner(orig_dm, &part); >> PetscPartitionerSetType(part, PETSCPARTITIONERPARMETIS); >> DMPlexDistribute(orig_dm, 0, NULL, &dist_dm); >> >> PetscErrorCode err = DMCreateMatrix(dist_dm, &jacobian_matrix); >> PetscInt M, N, m, n; >> MatGetSize(jacobian_matrix, &M, &N); >> MatGetLocalSize(jacobian_matrix, &m, &n); >> >> Then I run my code with 2 processes and I obtained this result: >> Size from jacobian_matrix: M=0 m =0 (this result is the same in all >> processes). >> >> I didn't understand if I forgot something in my code to obtain the >> correct values for the local and global sizes for the jacobean_matrix? (I >> mean if my code is missing something to obtain a distributed mesh over >> processes ?) >> >> Thank you in advance for any help! >> Sami, >> >> >> -- >> Dr. Sami BEN ELHAJ SALAH >> Ing?nieur de Recherche (CNRS) >> Institut Pprime - ISAE - ENSMA >> Mobile: 06.62.51.26.74 >> Email: sami.ben-elhaj-salah at ensma.fr >> www.samibenelhajsalah.com >> >> >> >> >> > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > > https://www.cse.buffalo.edu/~knepley/ > > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Sat May 28 13:20:05 2022 From: knepley at gmail.com (Matthew Knepley) Date: Sat, 28 May 2022 14:20:05 -0400 Subject: [petsc-users] Question about DMPlexDistribute & distribute mesh over processes In-Reply-To: References: <7E83D8A1-E911-4B48-98C9-BCFA9429AA11@ensma.fr> <15043588-F026-46C0-8334-E7ABA28F44FF@ensma.fr> Message-ID: On Sat, May 28, 2022 at 2:19 PM Matthew Knepley wrote: > On Sat, May 28, 2022 at 1:35 PM Sami BEN ELHAJ SALAH < > sami.ben-elhaj-salah at ensma.fr> wrote: > >> Hi Matthew, >> >> Thank you for your response. >> >> I don't have that. My DM object is not linked to PetscSection yet. I'll >> try that. >> Is there any example that manages this case? (DMPlexDistribute & >> PetscSection) or any guideline will be helpful. >> > > Here is an example where we create a section without DS. > Forgot the link: https://gitlab.com/petsc/petsc/-/blob/main/src/dm/impls/plex/tutorials/ex14.c > THanks, > > Matt > > >> Thanks in advance, >> >> Sami, >> >> -- >> Dr. Sami BEN ELHAJ SALAH >> Ing?nieur de Recherche (CNRS) >> Institut Pprime - ISAE - ENSMA >> Mobile: 06.62.51.26.74 >> Email: sami.ben-elhaj-salah at ensma.fr >> www.samibenelhajsalah.com >> >> >> >> >> Le 27 mai 2022 ? 20:45, Matthew Knepley a ?crit : >> >> On Fri, May 27, 2022 at 9:42 AM Sami BEN ELHAJ SALAH < >> sami.ben-elhaj-salah at ensma.fr> wrote: >> >>> Hello Isaac, >>> >>> Thank you for your reply! >>> >>> Let me confirm that when I use DMCreateMatrix() with the orig_dm, I got >>> my jacobian_matrix. Also, I have succeeded to solve my system and my >>> solution was converged* by using one process*. >>> Let me give you some other information about my code. Currently, I am >>> using my own discretization system and not the PetscDS object. Considering >>> the nonlinear solver SNES, I use the following routines (like the basic >>> snes usage): >>> - SNESSetFunction(snes,residual_vector,compute_residual ,(void*) >>> _finite_element_formulation) >>> - SNESSetJacobian(snes, jacobian.matrix(), _jacobian_precond_matrix, >>> compute_jacobian, (void*) _finite_element_formulation) >>> - SNESSolve(snes,NULL,x) >>> >>> Regarding your last answer, I will try to reformulate my question as >>> follows: >>> Using a distributed dm instead of the original dm (not distributed dm) >>> and my own discretization system (not the PetscDS object), >>> >> >> You do not have to use PetscDS, but the DM does need to a PetscSection in >> order to compute sizes and sparsity patterns. Do you have that? >> >> Thanks, >> >> Matt >> >> >>> should I add something specific to get a distributed jacobian_matrix >>> over processes? >>> I precise that I just replaced the orig_dm by a distributed mesh with >>> the routine that I wrote in my first mail. So is it enough ? >>> >>> Thank you and have a good day, >>> Sami, >>> >>> >>> -- >>> Dr. Sami BEN ELHAJ SALAH >>> Ing?nieur de Recherche (CNRS) >>> Institut Pprime - ISAE - ENSMA >>> Mobile: 06.62.51.26.74 >>> Email: sami.ben-elhaj-salah at ensma.fr >>> www.samibenelhajsalah.com >>> >>> >>> >>> >>> Le 25 mai 2022 ? 19:41, Toby Isaac a ?crit : >>> >>> Hi Sami, >>> >>> Just to verify: if you call DMCreateMatrix() on orig_dm, do you get a >>> Jacobian matrix? >>> >>> The DMPlex must be told what kind of discretized fields you want a >>> matrix for and what equations you are discretizing. This is handled >>> by the PetscDS object. In snes/tutorials/ex59.c, see the code after >>> DMGetDS() for an example. >>> >>> - Toby >>> >>> On Wed, May 25, 2022 at 1:17 PM Sami BEN ELHAJ SALAH >>> wrote: >>> >>> >>> Dear PETSc developer team, >>> >>> I m trying to create ? jacobian_matrix from distributed DM. I have >>> followed the two examples (snes/tests/ex2.c and ex56.c). So I wrote this >>> routine: >>> >>> PetscDM orig_dm; >>> PetscDM dist_dm = NULL; >>> PetscPartitioner part; >>> DMPlexGetPartitioner(orig_dm, &part); >>> PetscPartitionerSetType(part, PETSCPARTITIONERPARMETIS); >>> DMPlexDistribute(orig_dm, 0, NULL, &dist_dm); >>> >>> PetscErrorCode err = DMCreateMatrix(dist_dm, &jacobian_matrix); >>> PetscInt M, N, m, n; >>> MatGetSize(jacobian_matrix, &M, &N); >>> MatGetLocalSize(jacobian_matrix, &m, &n); >>> >>> Then I run my code with 2 processes and I obtained this result: >>> Size from jacobian_matrix: M=0 m =0 (this result is the same in all >>> processes). >>> >>> I didn't understand if I forgot something in my code to obtain the >>> correct values for the local and global sizes for the jacobean_matrix? (I >>> mean if my code is missing something to obtain a distributed mesh over >>> processes ?) >>> >>> Thank you in advance for any help! >>> Sami, >>> >>> >>> -- >>> Dr. Sami BEN ELHAJ SALAH >>> Ing?nieur de Recherche (CNRS) >>> Institut Pprime - ISAE - ENSMA >>> Mobile: 06.62.51.26.74 >>> Email: sami.ben-elhaj-salah at ensma.fr >>> www.samibenelhajsalah.com >>> >>> >>> >>> >>> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which their >> experiments lead. >> -- Norbert Wiener >> >> https://www.cse.buffalo.edu/~knepley/ >> >> >> >> > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > > https://www.cse.buffalo.edu/~knepley/ > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From mi.mike1021 at gmail.com Sat May 28 21:41:27 2022 From: mi.mike1021 at gmail.com (Mike Michell) Date: Sat, 28 May 2022 21:41:27 -0500 Subject: [petsc-users] PetscSF Object on Distributed DMPlex for Halo Data Exchange In-Reply-To: References: Message-ID: Matt and Toby, Thanks a lot for all the comments and a clear example. Mike 2022? 5? 24? (?) ?? 8:46, Matthew Knepley ?? ??: > I will also point out that Toby has created a nice example showing how to > create an SF for halo exchange between local vectors. > > https://gitlab.com/petsc/petsc/-/merge_requests/5267 > > Thanks, > > Matt > > On Sun, May 22, 2022 at 9:47 PM Matthew Knepley wrote: > >> On Sun, May 22, 2022 at 4:28 PM Mike Michell >> wrote: >> >>> Thanks for the reply. The diagram makes sense and is helpful for >>> understanding 1D representation. >>> >>> However, something is still unclear. From your diagram, the number of >>> roots per process seems to vary according to run arguments, such as >>> "-dm_distribute_overlap", because "the number of roots for a DMPlex is the >>> number of mesh points in the local portion of the mesh (cited from your >>> answer to my question (1))" will end up change according to that argument. >>> However, from my mock-up code, number of roots is independent to >>> -dm_distribute_overlap argument. The summation of "number of roots" through >>> processes was always equal to number of physical vertex on my mesh, if I >>> define the section layout on vertex with 1DOF. But in your diagram example, >>> the summation of "nroots" is larger than the actual number of mesh points, >>> which is 13. >>> >> >> I do not understand your question. Notice the -dm_distribute_overlap does >> _not_ change the owned points for any process. It only puts in new leaves, >> so it also never >> changes the roots for this way of using the SF. >> >> >>> Also, it is still unclear how to get the size of "roots" from the >>> PetscSection & PetscSF on distributed DMPlex? >>> >> >> For an SF mapping ghost dofs in a global vector, the number of roots is >> just the size of the local portion of the vector. >> >> >>> In your diagram, how can you tell your code and make it allocate the >>> "nroots=7 for P0, nroots=9 for P1, and nroots=7 for P2" arrays before you >>> call PetscSFBcastBegin/End()? It seems that we need to define arrays having >>> the size of nroots & nleaves before calling PetscSFBcastBegin/End(). >>> >> >> I just want to note that this usage is different from the canonical usage >> in Plex. It is fine to do this, but this will not match what I do in the >> library if you look. >> In Plex, I distinguish two linear spaces: >> >> 1) Global space: This is the vector space for the solvers. Each point >> is uniquely represented and owned by some process >> >> 2) Local space: This is the vector space for assembly. Some points are >> represented multiple times. >> >> I create an SF that maps from the global space (roots) to the local space >> (leaves), and it is called in DMGlobalToLocal() (and associated functions). >> This >> is more natural in FEM. You seem to want an SF that maps between global >> vectors. This will also work. The roots would be the local dofs, and the >> leaves >> would be shared dofs. >> >> Does this make sense? >> >> Thanks, >> >> Matt >> >> >>> Thanks, >>> Mike >>> >>> Here's a diagram of a 1D mesh with overlap and 3 partitions, showing >>>> what the petscsf data is for each. The number of roots is the number of >>>> mesh points in the local representation, and the number of leaves is the >>>> number of mesh points that are duplicates of mesh points on other >>>> processes. With that in mind, answering your questions >>>> >>>> > (1) It seems that the "roots" means the number of vertex not >>>> considering overlap layer, and "leaves" seems the number of distributed >>>> vertex for each processor that includes overlap layer. Can you acknowledge >>>> that this is correct understanding? I have tried to find clearer examples >>>> from PETSc team's articles relevant to Star Forest, but I am still unclear >>>> about the exact relation & graphical notation of roots & leaves in SF if >>>> it's the case of DMPlex solution arrays. >>>> >>>> No, the number of roots for a DMPlex is the number of mesh points in >>>> the local portion of the mesh >>>> >>>> > (2) If it is so, there is an issue that I cannot define "root data" >>>> and "leave data" generally. I am trying to following >>>> "src/vec/is/sf/tutorials/ex1f.F90", however, in that example, size of roots >>>> and leaves are predefined as 6. How can I generalize that? Because I can >>>> get size of leaves using DAG depth(or height), which is equal to number of >>>> vertices each proc has. But, how can I get the size of my "roots" region >>>> from SF? Any example about that? This question is connected to how can I >>>> define "rootdata" for "PetscSFBcastBegin/End()". >>>> >>>> Does the diagram help you generalize? >>>> >>>> > (3) More importantly, with the attached PetscSection & SF layout, my >>>> vector is only resolved for the size equal to "number of roots" for each >>>> proc, but not for the overlapping area(i.e., "leaves"). What I wish to do >>>> is to exchange (or reduce) the solution data between each proc, in the >>>> overlapping region. Can I get some advices why my vector does not encompass >>>> the "leaves" regime? Is there any example doing similar things? >>>> Going back to my first response: if you use a section to say how many >>>> pieces of data are associated with each local mesh point, then a PetscSF is >>>> constructed that requires no more manipulation from you. >>>> >>>> >>>> On Sun, May 22, 2022 at 10:47 AM Mike Michell >>>> wrote: >>>> >>>>> Thank you for the reply. >>>>> The PetscSection and PetscSF objects are defined as in the attached >>>>> mock-up code (Q_PetscSF_1.tar). 1-DOF is defined on vertex as my solution >>>>> is determined on each vertex with 1-DOF from a finite-volume method. >>>>> >>>>> As follow up questions: >>>>> (1) It seems that the "roots" means the number of vertex not >>>>> considering overlap layer, and "leaves" seems the number of distributed >>>>> vertex for each processor that includes overlap layer. Can you acknowledge >>>>> that this is correct understanding? I have tried to find clearer examples >>>>> from PETSc team's articles relevant to Star Forest, but I am still unclear >>>>> about the exact relation & graphical notation of roots & leaves in SF if >>>>> it's the case of DMPlex solution arrays. >>>>> >>>>> (2) If it is so, there is an issue that I cannot define "root data" >>>>> and "leave data" generally. I am trying to following >>>>> "src/vec/is/sf/tutorials/ex1f.F90", however, in that example, size of roots >>>>> and leaves are predefined as 6. How can I generalize that? Because I can >>>>> get size of leaves using DAG depth(or height), which is equal to number of >>>>> vertices each proc has. But, how can I get the size of my "roots" region >>>>> from SF? Any example about that? This question is connected to how can I >>>>> define "rootdata" for "PetscSFBcastBegin/End()". >>>>> >>>>> (3) More importantly, with the attached PetscSection & SF layout, my >>>>> vector is only resolved for the size equal to "number of roots" for each >>>>> proc, but not for the overlapping area(i.e., "leaves"). What I wish to do >>>>> is to exchange (or reduce) the solution data between each proc, in the >>>>> overlapping region. Can I get some advices why my vector does not encompass >>>>> the "leaves" regime? Is there any example doing similar things? >>>>> >>>>> Thanks, >>>>> Mike >>>>> >>>>> >>>>>> On Fri, May 20, 2022 at 4:45 PM Mike Michell >>>>>> wrote: >>>>>> >>>>>>> Thanks for the reply. >>>>>>> >>>>>>> > "What I want to do is to exchange data (probably just MPI_Reduce)" >>>>>>> which confuses me, because halo exchange is a point-to-point exchange and >>>>>>> not a reduction. Can you clarify? >>>>>>> PetscSFReduceBegin/End seems to be the function that do reduction >>>>>>> for PetscSF object. What I intended to mention was either reduction or >>>>>>> exchange, not specifically intended "reduction". >>>>>>> >>>>>>> As a follow-up question: >>>>>>> Assuming that the code has its own local solution arrays (not Petsc >>>>>>> type), and if the plex's DAG indices belong to the halo region are the only >>>>>>> information that I want to know (not the detailed section description, such >>>>>>> as degree of freedom on vertex, cells, etc.). I have another PetscSection >>>>>>> for printing out my solution. >>>>>>> Also if I can convert that DAG indices into my local cell/vertex >>>>>>> index, can I just use the PetscSF object created from DMGetPointSF(), >>>>>>> instead of "creating PetscSection + DMGetSectionSF()"? In other words, can >>>>>>> I use the PetscSF object declared from DMGetPointSF() for the halo >>>>>>> communication? >>>>>>> >>>>>> >>>>>> No, because that point SF will index information by point number. You >>>>>> would need to build a new SF that indexes your dofs. The steps you would >>>>>> go through are exactly the same as you would if you just told us what >>>>>> the Section is that indexes your data. >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Matt >>>>>> >>>>>> >>>>>>> Thanks, >>>>>>> Mike >>>>>>> >>>>>>> >>>>>>> The PetscSF that is created automatically is the "point sf" ( >>>>>>>> https://petsc.org/main/docs/manualpages/DM/DMGetPointSF/): it says >>>>>>>> which mesh points (cells, faces, edges and vertices) are duplicates of >>>>>>>> others. >>>>>>>> >>>>>>>> In a finite volume application we typically want to assign degrees >>>>>>>> of freedom just to cells: some applications may only have one degree of >>>>>>>> freedom, others may have multiple. >>>>>>>> >>>>>>>> You encode where you want degrees of freedom in a PetscSection and >>>>>>>> set that as the section for the DM in DMSetLocalSection() ( >>>>>>>> https://petsc.org/release/docs/manualpages/DM/DMSetLocalSection.html >>>>>>>> ) >>>>>>>> >>>>>>>> (A c example of these steps that sets degrees of freedom for >>>>>>>> *vertices* instead of cells is `src/dm/impls/plex/tutorials/ex7.c`) >>>>>>>> >>>>>>>> After that you can call DMGetSectionSF() ( >>>>>>>> https://petsc.org/main/docs/manualpages/DM/DMGetSectionSF/) to the >>>>>>>> the PetscSF that you want for halo exchange: the one for your solution >>>>>>>> variables. >>>>>>>> >>>>>>>> After that, the only calls you typically need in a finite volume >>>>>>>> code is PetscSFBcastBegin() to start a halo exchange and PetscSFBcastEnd() >>>>>>>> to complete it. >>>>>>>> >>>>>>>> You say >>>>>>>> >>>>>>>> > What I want to do is to exchange data (probably just MPI_Reduce) >>>>>>>> >>>>>>>> which confuses me, because halo exchange is a point-to-point >>>>>>>> exchange and not a reduction. Can you clarify? >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Fri, May 20, 2022 at 8:35 PM Mike Michell >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Dear PETSc developer team, >>>>>>>>> >>>>>>>>> Hi, I am using DMPlex for a finite-volume code and trying to >>>>>>>>> understand the usage of PetscSF. What is a typical procedure for doing halo >>>>>>>>> data exchange at parallel boundary using PetscSF object on DMPlex? Is there >>>>>>>>> any example that I can refer to usage of PetscSF with distributed DMPlex? >>>>>>>>> >>>>>>>>> Assuming to use the attached mock-up code and mesh, if I give >>>>>>>>> "-dm_distribute_overlap 1 -over_dm_view" to run the code, I can see a >>>>>>>>> PetscSF object is already created, although I have not called >>>>>>>>> "PetscSFCreate" in the code. How can I import & use that PetscSF already >>>>>>>>> created by the code to do the halo data exchange? >>>>>>>>> >>>>>>>>> What I want to do is to exchange data (probably just MPI_Reduce) >>>>>>>>> in a parallel boundary region using PetscSF and its functions. I might need >>>>>>>>> to have an overlapping layer or not. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Mike >>>>>>>>> >>>>>>>> >>>>>> >>>>>> -- >>>>>> What most experimenters take for granted before they begin their >>>>>> experiments is infinitely more interesting than any results to which their >>>>>> experiments lead. >>>>>> -- Norbert Wiener >>>>>> >>>>>> https://www.cse.buffalo.edu/~knepley/ >>>>>> >>>>>> >>>>> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which their >> experiments lead. >> -- Norbert Wiener >> >> https://www.cse.buffalo.edu/~knepley/ >> >> > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > > https://www.cse.buffalo.edu/~knepley/ > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sami.ben-elhaj-salah at ensma.fr Sun May 29 11:02:07 2022 From: sami.ben-elhaj-salah at ensma.fr (Sami BEN ELHAJ SALAH) Date: Sun, 29 May 2022 18:02:07 +0200 Subject: [petsc-users] Question about DMPlexDistribute & distribute mesh over processes In-Reply-To: References: <7E83D8A1-E911-4B48-98C9-BCFA9429AA11@ensma.fr> <15043588-F026-46C0-8334-E7ABA28F44FF@ensma.fr> Message-ID: Hi Matthew, Thank you for this example. It seems exactly what I am looking for. Thank you again for your help and have a good day. Sami, -- Dr. Sami BEN ELHAJ SALAH Ing?nieur de Recherche (CNRS) Institut Pprime - ISAE - ENSMA Mobile: 06.62.51.26.74 Email: sami.ben-elhaj-salah at ensma.fr www.samibenelhajsalah.com > Le 28 mai 2022 ? 20:20, Matthew Knepley a ?crit : > > On Sat, May 28, 2022 at 2:19 PM Matthew Knepley > wrote: > On Sat, May 28, 2022 at 1:35 PM Sami BEN ELHAJ SALAH > wrote: > Hi Matthew, > > Thank you for your response. > > I don't have that. My DM object is not linked to PetscSection yet. I'll try that. > Is there any example that manages this case? (DMPlexDistribute & PetscSection) or any guideline will be helpful. > > Here is an example where we create a section without DS. > > Forgot the link: https://gitlab.com/petsc/petsc/-/blob/main/src/dm/impls/plex/tutorials/ex14.c > > THanks, > > Matt > > Thanks in advance, > > Sami, > > -- > Dr. Sami BEN ELHAJ SALAH > Ing?nieur de Recherche (CNRS) > Institut Pprime - ISAE - ENSMA > Mobile: 06.62.51.26.74 > Email: sami.ben-elhaj-salah at ensma.fr > www.samibenelhajsalah.com > > > >> Le 27 mai 2022 ? 20:45, Matthew Knepley > a ?crit : >> >> On Fri, May 27, 2022 at 9:42 AM Sami BEN ELHAJ SALAH > wrote: >> Hello Isaac, >> >> Thank you for your reply! >> >> Let me confirm that when I use DMCreateMatrix() with the orig_dm, I got my jacobian_matrix. Also, I have succeeded to solve my system and my solution was converged by using one process. >> Let me give you some other information about my code. Currently, I am using my own discretization system and not the PetscDS object. Considering the nonlinear solver SNES, I use the following routines (like the basic snes usage): >> - SNESSetFunction(snes,residual_vector,compute_residual ,(void*) _finite_element_formulation) >> - SNESSetJacobian(snes, jacobian.matrix(), _jacobian_precond_matrix, compute_jacobian, (void*) _finite_element_formulation) >> - SNESSolve(snes,NULL,x) >> >> Regarding your last answer, I will try to reformulate my question as follows: >> Using a distributed dm instead of the original dm (not distributed dm) and my own discretization system (not the PetscDS object), >> >> You do not have to use PetscDS, but the DM does need to a PetscSection in order to compute sizes and sparsity patterns. Do you have that? >> >> Thanks, >> >> Matt >> >> should I add something specific to get a distributed jacobian_matrix over processes? >> I precise that I just replaced the orig_dm by a distributed mesh with the routine that I wrote in my first mail. So is it enough ? >> >> Thank you and have a good day, >> Sami, >> >> >> -- >> Dr. Sami BEN ELHAJ SALAH >> Ing?nieur de Recherche (CNRS) >> Institut Pprime - ISAE - ENSMA >> Mobile: 06.62.51.26.74 >> Email: sami.ben-elhaj-salah at ensma.fr >> www.samibenelhajsalah.com >> >> >> >>> Le 25 mai 2022 ? 19:41, Toby Isaac > a ?crit : >>> >>> Hi Sami, >>> >>> Just to verify: if you call DMCreateMatrix() on orig_dm, do you get a >>> Jacobian matrix? >>> >>> The DMPlex must be told what kind of discretized fields you want a >>> matrix for and what equations you are discretizing. This is handled >>> by the PetscDS object. In snes/tutorials/ex59.c, see the code after >>> DMGetDS() for an example. >>> >>> - Toby >>> >>> On Wed, May 25, 2022 at 1:17 PM Sami BEN ELHAJ SALAH >>> > wrote: >>>> >>>> Dear PETSc developer team, >>>> >>>> I m trying to create ? jacobian_matrix from distributed DM. I have followed the two examples (snes/tests/ex2.c and ex56.c). So I wrote this routine: >>>> >>>> PetscDM orig_dm; >>>> PetscDM dist_dm = NULL; >>>> PetscPartitioner part; >>>> DMPlexGetPartitioner(orig_dm, &part); >>>> PetscPartitionerSetType(part, PETSCPARTITIONERPARMETIS); >>>> DMPlexDistribute(orig_dm, 0, NULL, &dist_dm); >>>> >>>> PetscErrorCode err = DMCreateMatrix(dist_dm, &jacobian_matrix); >>>> PetscInt M, N, m, n; >>>> MatGetSize(jacobian_matrix, &M, &N); >>>> MatGetLocalSize(jacobian_matrix, &m, &n); >>>> >>>> Then I run my code with 2 processes and I obtained this result: >>>> Size from jacobian_matrix: M=0 m =0 (this result is the same in all processes). >>>> >>>> I didn't understand if I forgot something in my code to obtain the correct values for the local and global sizes for the jacobean_matrix? (I mean if my code is missing something to obtain a distributed mesh over processes ?) >>>> >>>> Thank you in advance for any help! >>>> Sami, >>>> >>>> >>>> -- >>>> Dr. Sami BEN ELHAJ SALAH >>>> Ing?nieur de Recherche (CNRS) >>>> Institut Pprime - ISAE - ENSMA >>>> Mobile: 06.62.51.26.74 >>>> Email: sami.ben-elhaj-salah at ensma.fr >>>> www.samibenelhajsalah.com >>>> >>>> >>>> >> >> >> >> -- >> What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. >> -- Norbert Wiener >> >> https://www.cse.buffalo.edu/~knepley/ > > > > -- > What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. > -- Norbert Wiener > > https://www.cse.buffalo.edu/~knepley/ > > > -- > What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. > -- Norbert Wiener > > https://www.cse.buffalo.edu/~knepley/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From Jiannan_Tu at uml.edu Mon May 30 18:33:30 2022 From: Jiannan_Tu at uml.edu (Tu, Jiannan) Date: Mon, 30 May 2022 23:33:30 +0000 Subject: [petsc-users] Using matrix-free with KSP In-Reply-To: References: Message-ID: I tried the matrix-free method. Since the number of the unknown is large, say 50000, then the number of matrix elements is 50000^2, which is greater than the max number represented by the 32-bit int. PETSC gives an error message: "Mesh of 50000 by 50000 by 1 (dof) is too large for 32 bit indices." I know using the 64-bit int option when compiling PETSC can avoid this error. However, it seems that 64-bit int makes the PETSC relatively slow. I wonder why the matrix-free method still counts the total number of matrix elements even though they are not stored. Any other ways around it? Thank you, Jiannan From: Barry Smith Sent: Tuesday, May 24, 2022 2:13 PM To: Tu, Jiannan Cc: petsc-users at mcs.anl.gov Subject: Re: [petsc-users] Using matrix-free with KSP This e-mail originated from outside the UMass Lowell network. ________________________________ You can use MatCreateMFFD https://petsc.org/main/docs/manualpages/Mat/MatCreateMFFD/ MatMFFDSetFunction MatSetFromOptions MatMFFDSetBase and provide the matrix to KSP. Note you will need to use -pc_type none or provide your own custom preconditioner with https://petsc.org/main/docs/manualpages/PC/PCSHELL/ On May 24, 2022, at 1:21 PM, Tu, Jiannan > wrote: I want to use a matrix-free matrix to solve a large linear equation system because the matrix is too large to be stored. Petsc user manual describes matrix-free method for SNES with examples. The matrix-free matrices section explains how to set up such a matrix, but I can't find any example of matrix-free method with KSP. I am wondering how to apply the method to KSP iterative solver in parallel. I greatly appreciate your help for me to understand this topic. Jiannan Tu -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at petsc.dev Mon May 30 18:49:06 2022 From: bsmith at petsc.dev (Barry Smith) Date: Mon, 30 May 2022 19:49:06 -0400 Subject: [petsc-users] Using matrix-free with KSP In-Reply-To: References: Message-ID: 50,000 * 50,000 > 2^31 (roughly 2 billion) hence we cannot represent even the length of a vector using 32 bit integers; just to do "book keeping" of dimensions requires using 64 bit integers. Using 64 bit integers should not slow PETSc significantly, especially in matrix-free mode. > On May 30, 2022, at 7:33 PM, Tu, Jiannan wrote: > > I tried the matrix-free method. Since the number of the unknown is large, say 50000, then the number of matrix elements is 50000^2, which is greater than the max number represented by the 32-bit int. PETSC gives an error message: ?Mesh of 50,000 by 50,000 by 1 (dof) is too large for 32 bit indices.? > > I know using the 64-bit int option when compiling PETSC can avoid this error. However, it seems that 64-bit int makes the PETSC relatively slow. I wonder why the matrix-free method still counts the total number of matrix elements even though they are not stored. Any other ways around it? > > Thank you, > Jiannan > > From: Barry Smith > > Sent: Tuesday, May 24, 2022 2:13 PM > To: Tu, Jiannan > > Cc: petsc-users at mcs.anl.gov > Subject: Re: [petsc-users] Using matrix-free with KSP > > This e-mail originated from outside the UMass Lowell network. > > You can use MatCreateMFFD https://petsc.org/main/docs/manualpages/Mat/MatCreateMFFD/ MatMFFDSetFunction MatSetFromOptions MatMFFDSetBase and provide the matrix to KSP. Note you will need to use -pc_type none or provide your own custom preconditioner with https://petsc.org/main/docs/manualpages/PC/PCSHELL/ > > > > > On May 24, 2022, at 1:21 PM, Tu, Jiannan > wrote: > > I want to use a matrix-free matrix to solve a large linear equation system because the matrix is too large to be stored. Petsc user manual describes matrix-free method for SNES with examples. The matrix-free matrices section explains how to set up such a matrix, but I can't find any example of matrix-free method with KSP. I am wondering how to apply the method to KSP iterative solver in parallel. > > I greatly appreciate your help for me to understand this topic. > > Jiannan Tu -------------- next part -------------- An HTML attachment was scrubbed... URL: From Jiannan_Tu at uml.edu Mon May 30 19:29:11 2022 From: Jiannan_Tu at uml.edu (Tu, Jiannan) Date: Tue, 31 May 2022 00:29:11 +0000 Subject: [petsc-users] Using matrix-free with KSP In-Reply-To: References: Message-ID: Barry, thanks for the clarification. Jiannan From: Barry Smith Sent: Monday, May 30, 2022 7:49 PM To: Tu, Jiannan Cc: petsc-users at mcs.anl.gov Subject: Re: [petsc-users] Using matrix-free with KSP This e-mail originated from outside the UMass Lowell network. ________________________________ 50,000 * 50,000 > 2^31 (roughly 2 billion) hence we cannot represent even the length of a vector using 32 bit integers; just to do "book keeping" of dimensions requires using 64 bit integers. Using 64 bit integers should not slow PETSc significantly, especially in matrix-free mode. On May 30, 2022, at 7:33 PM, Tu, Jiannan > wrote: I tried the matrix-free method. Since the number of the unknown is large, say 50000, then the number of matrix elements is 50000^2, which is greater than the max number represented by the 32-bit int. PETSC gives an error message: ?Mesh of 50,000 by 50,000 by 1 (dof) is too large for 32 bit indices.? I know using the 64-bit int option when compiling PETSC can avoid this error. However, it seems that 64-bit int makes the PETSC relatively slow. I wonder why the matrix-free method still counts the total number of matrix elements even though they are not stored. Any other ways around it? Thank you, Jiannan From: Barry Smith > Sent: Tuesday, May 24, 2022 2:13 PM To: Tu, Jiannan > Cc: petsc-users at mcs.anl.gov Subject: Re: [petsc-users] Using matrix-free with KSP This e-mail originated from outside the UMass Lowell network. ________________________________ You can use MatCreateMFFD https://petsc.org/main/docs/manualpages/Mat/MatCreateMFFD/ MatMFFDSetFunction MatSetFromOptions MatMFFDSetBase and provide the matrix to KSP. Note you will need to use -pc_type none or provide your own custom preconditioner with https://petsc.org/main/docs/manualpages/PC/PCSHELL/ On May 24, 2022, at 1:21 PM, Tu, Jiannan > wrote: I want to use a matrix-free matrix to solve a large linear equation system because the matrix is too large to be stored. Petsc user manual describes matrix-free method for SNES with examples. The matrix-free matrices section explains how to set up such a matrix, but I can't find any example of matrix-free method with KSP. I am wondering how to apply the method to KSP iterative solver in parallel. I greatly appreciate your help for me to understand this topic. Jiannan Tu -------------- next part -------------- An HTML attachment was scrubbed... URL: From lidia.varsh at mail.ioffe.ru Mon May 30 09:33:33 2022 From: lidia.varsh at mail.ioffe.ru (Lidia) Date: Mon, 30 May 2022 17:33:33 +0300 Subject: [petsc-users] Sparse linear system solving Message-ID: <026d55af-c978-81da-f571-46519e5e6f8e@mail.ioffe.ru> Dear colleagues, Is here anyone who have solved big sparse linear matrices using PETSC? We have found NO performance improvement while using more and more mpi processes (1-2-3) and open-mp threads (from 1 to 72 threads). Did anyone faced to this problem? Does anyone know any possible reasons of such behaviour? We use AMG preconditioner and GMRES solver from KSP package, as our matrix is large (from 100 000 to 1e+6 rows and columns), sparse, non-symmetric and includes both positive and negative values. But performance problems also exist while using CG solvers with symmetric matrices. Could anyone help us to set appropriate options of the preconditioner and solver? Now we use default parameters, maybe they are not the best, but we do not know a good combination. Or maybe you could suggest any other pairs of preconditioner+solver for such tasks? I can provide more information: the matrices that we solve, c++ script to run solving using petsc and any statistics obtained by our runs. Thank you in advance! Best regards, Lidiia Varshavchik, Ioffe Institute, St. Petersburg, Russia From 12132391 at mail.sustech.edu.cn Mon May 30 20:47:22 2022 From: 12132391 at mail.sustech.edu.cn (=?utf-8?B?5Yav5a6P56OK?=) Date: Tue, 31 May 2022 09:47:22 +0800 Subject: [petsc-users] How to read and write HDF5 in parallel withPETSc Message-ID: Hey there I'm a new user of PETSc. In use, I want to read and write an HDF5 file in parallel, but I only found an example of serial reading and writing. How can I read and write HDF5 in parallel? Can you give me a code sample? I'm using C for monogramming by the way. ??? ??????/???/???2021? ?????????????1088?   -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Mon May 30 21:17:45 2022 From: knepley at gmail.com (Matthew Knepley) Date: Mon, 30 May 2022 22:17:45 -0400 Subject: [petsc-users] Sparse linear system solving In-Reply-To: <026d55af-c978-81da-f571-46519e5e6f8e@mail.ioffe.ru> References: <026d55af-c978-81da-f571-46519e5e6f8e@mail.ioffe.ru> Message-ID: On Mon, May 30, 2022 at 10:12 PM Lidia wrote: > Dear colleagues, > > Is here anyone who have solved big sparse linear matrices using PETSC? > There are lots of publications with this kind of data. Here is one recent one: https://arxiv.org/abs/2204.01722 > We have found NO performance improvement while using more and more mpi > processes (1-2-3) and open-mp threads (from 1 to 72 threads). Did anyone > faced to this problem? Does anyone know any possible reasons of such > behaviour? > Solver behavior is dependent on the input matrix. The only general-purpose solvers are direct, but they do not scale linearly and have high memory requirements. Thus, in order to make progress you will have to be specific about your matrices. > We use AMG preconditioner and GMRES solver from KSP package, as our > matrix is large (from 100 000 to 1e+6 rows and columns), sparse, > non-symmetric and includes both positive and negative values. But > performance problems also exist while using CG solvers with symmetric > matrices. > There are many PETSc examples, such as example 5 for the Laplacian, that exhibit good scaling with both AMG and GMG. > Could anyone help us to set appropriate options of the preconditioner > and solver? Now we use default parameters, maybe they are not the best, > but we do not know a good combination. Or maybe you could suggest any > other pairs of preconditioner+solver for such tasks? > > I can provide more information: the matrices that we solve, c++ script > to run solving using petsc and any statistics obtained by our runs. > First, please provide a description of the linear system, and the output of -ksp_view -ksp_monitor_true_residual -ksp_converged_reason -log_view for each test case. Thanks, Matt > Thank you in advance! > > Best regards, > Lidiia Varshavchik, > Ioffe Institute, St. Petersburg, Russia > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Mon May 30 21:18:55 2022 From: knepley at gmail.com (Matthew Knepley) Date: Mon, 30 May 2022 22:18:55 -0400 Subject: [petsc-users] How to read and write HDF5 in parallel withPETSc In-Reply-To: References: Message-ID: On Mon, May 30, 2022 at 10:12 PM ??? <12132391 at mail.sustech.edu.cn> wrote: > Hey there > I'm a new user of PETSc. In use, I want to read and write an HDF5 file in > parallel, but I only found an example of serial reading and writing. How > can I read and write HDF5 in parallel? Can you give me a code sample? > > I'm using C for monogramming by the way. > > PETSc HDF5 output, such as VecView(), will automatically use parallelism if available. Are you trying to do something else? Thanks, Matt > ??? > > ??????/???/???2021? > > ?????????????1088? > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From mfadams at lbl.gov Mon May 30 21:42:05 2022 From: mfadams at lbl.gov (Mark Adams) Date: Mon, 30 May 2022 22:42:05 -0400 Subject: [petsc-users] Sparse linear system solving In-Reply-To: References: <026d55af-c978-81da-f571-46519e5e6f8e@mail.ioffe.ru> Message-ID: And if you see "NO" change in performance I suspect the solver/matrix is all on one processor. (PETSc does not use threads by default so threads should not change anything). As Matt said, it is best to start with a PETSc example that does something like what you want (parallel linear solve, see src/ksp/ksp/tutorials for examples), and then add your code to it. That way you get the basic infrastructure in place for you, which is pretty obscure to the uninitiated. Mark On Mon, May 30, 2022 at 10:18 PM Matthew Knepley wrote: > On Mon, May 30, 2022 at 10:12 PM Lidia wrote: > >> Dear colleagues, >> >> Is here anyone who have solved big sparse linear matrices using PETSC? >> > > There are lots of publications with this kind of data. Here is one recent > one: https://arxiv.org/abs/2204.01722 > > >> We have found NO performance improvement while using more and more mpi >> processes (1-2-3) and open-mp threads (from 1 to 72 threads). Did anyone >> faced to this problem? Does anyone know any possible reasons of such >> behaviour? >> > > Solver behavior is dependent on the input matrix. The only general-purpose > solvers > are direct, but they do not scale linearly and have high memory > requirements. > > Thus, in order to make progress you will have to be specific about your > matrices. > > >> We use AMG preconditioner and GMRES solver from KSP package, as our >> matrix is large (from 100 000 to 1e+6 rows and columns), sparse, >> non-symmetric and includes both positive and negative values. But >> performance problems also exist while using CG solvers with symmetric >> matrices. >> > > There are many PETSc examples, such as example 5 for the Laplacian, that > exhibit > good scaling with both AMG and GMG. > > >> Could anyone help us to set appropriate options of the preconditioner >> and solver? Now we use default parameters, maybe they are not the best, >> but we do not know a good combination. Or maybe you could suggest any >> other pairs of preconditioner+solver for such tasks? >> >> I can provide more information: the matrices that we solve, c++ script >> to run solving using petsc and any statistics obtained by our runs. >> > > First, please provide a description of the linear system, and the output of > > -ksp_view -ksp_monitor_true_residual -ksp_converged_reason -log_view > > for each test case. > > Thanks, > > Matt > > >> Thank you in advance! >> >> Best regards, >> Lidiia Varshavchik, >> Ioffe Institute, St. Petersburg, Russia >> > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > > https://www.cse.buffalo.edu/~knepley/ > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From 12132391 at mail.sustech.edu.cn Tue May 31 00:01:49 2022 From: 12132391 at mail.sustech.edu.cn (=?utf-8?B?5Yav5a6P56OK?=) Date: Tue, 31 May 2022 13:01:49 +0800 Subject: [petsc-users] How to read and write HDF5 in parallel withPETSc Message-ID: my code is below: ierr = PetscViewerCreate(PETSC_COMM_WORLD,&h5);CHKERRQ(ierr); ierr = PetscViewerHDF5Open(PETSC_COMM_WORLD,"explicit.h5", FILE_MODE_WRITE, &h5);CHKERRQ(ierr);  ierr = PetscObjectSetName((PetscObject) z, "explicit-vector");CHKERRQ(ierr);  ierr = PetscObjectSetName((PetscObject) tem, "explicit-necess-data");CHKERRQ(ierr);  ierr = VecView(tem, h5);CHKERRQ(ierr);  ierr = VecView(z, h5);CHKERRQ(ierr);  ierr = PetscViewerDestroy(&h5);CHKERRQ(ierr); when I use 1 core run this, it can save the right answer. when I use 3 cores run this, it prints that : [0]PETSC ERROR: No support for this operation for this object type [0]PETSC ERROR: Cannot use parallel HDF5 viewer since the given HDF5 does not support parallel I/O (H5_HAVE_PARALLEL is unset) I don't know how to solve this problem. By the way, I use PETSc-3.16.6, HDF5-1.12.1, C for programming. ??? ??????/???/???2021? ?????????????1088?   -------------- next part -------------- An HTML attachment was scrubbed... URL: From lidia.varsh at mail.ioffe.ru Tue May 31 06:34:15 2022 From: lidia.varsh at mail.ioffe.ru (Lidia) Date: Tue, 31 May 2022 14:34:15 +0300 Subject: [petsc-users] Sparse linear system solving In-Reply-To: References: <026d55af-c978-81da-f571-46519e5e6f8e@mail.ioffe.ru> Message-ID: <2e7ebbf1-511a-7055-ff92-131d3bf73f1e@mail.ioffe.ru> Matt, Mark, thank you much for your answers! Now we have run example # 5 on our computer cluster and on the local server and also have not seen any performance increase, but by unclear reason running times on the local server are much better than on the cluster. Now we will try to run petsc #5 example inside a docker container on our server and see if the problem is in our environment. I'll write you the results of this test as soon as we get it. The ksp_monitor outs for the 5th test at the current local server configuration (for 2 and 4 mpi processes) and for the cluster (for 1 and 3 mpi processes) are attached . And one more question. Potentially we can use 10 nodes and 96 threads at each node on our cluster. What do you think, which combination of numbers of mpi processes and openmp threads may be the best for the 5th example? Thank you! Best, Lidiia On 31.05.2022 05:42, Mark Adams wrote: > And if you see "NO" change in performance I suspect the solver/matrix > is all on one processor. > (PETSc does not use threads by default so threads should not change > anything). > > As Matt said, it is best to start with a PETSc example?that does > something like what you want (parallel linear solve, see > src/ksp/ksp/tutorials for examples), and then add your code to it. > That way you get the basic infrastructure?in place for you, which is > pretty obscure to the uninitiated. > > Mark > > On Mon, May 30, 2022 at 10:18 PM Matthew Knepley > wrote: > > On Mon, May 30, 2022 at 10:12 PM Lidia > wrote: > > Dear colleagues, > > Is here anyone who have solved big sparse linear matrices > using PETSC? > > > There are lots of publications with this kind of data. Here is one > recent one: https://arxiv.org/abs/2204.01722 > > We have found NO performance improvement while using more and > more mpi > processes (1-2-3) and open-mp threads (from 1 to 72 threads). > Did anyone > faced to this problem? Does anyone know any possible reasons > of such > behaviour? > > > Solver behavior is dependent on the input matrix. The only > general-purpose solvers > are direct, but they do not scale linearly and have high memory > requirements. > > Thus, in order to make progress you will have to be specific about > your matrices. > > We use AMG preconditioner and GMRES solver from KSP package, > as our > matrix is large (from 100 000 to 1e+6 rows and columns), sparse, > non-symmetric and includes both positive and negative values. But > performance problems also exist while using CG solvers with > symmetric > matrices. > > > There are many PETSc examples, such as example 5 for the > Laplacian, that exhibit > good scaling with both AMG and GMG. > > Could anyone help us to set appropriate options of the > preconditioner > and solver? Now we use default parameters, maybe they are not > the best, > but we do not know a good combination. Or maybe you could > suggest any > other pairs of preconditioner+solver for such tasks? > > I can provide more information: the matrices that we solve, > c++ script > to run solving using petsc and any statistics obtained by our > runs. > > > First, please provide a description of the linear system, and the > output of > > ? -ksp_view -ksp_monitor_true_residual -ksp_converged_reason -log_view > > for each test case. > > ? Thanks, > > ? ? ?Matt > > Thank you in advance! > > Best regards, > Lidiia Varshavchik, > Ioffe Institute, St. Petersburg, Russia > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to > which their experiments lead. > -- Norbert Wiener > > https://www.cse.buffalo.edu/~knepley/ > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- user at kitesrv:/data/raid1/tmp/petsc/src/ksp/ksp/tutorials$ LD_LIBRARY_PATH=/data/raid1/tmp/petsc/install/lib time /data/raid1/tmp/petsc/install/bin/mpirun -n 4 ./ex5 -ksp_view -ksp_monitor_true_residual -ksp_converged_reason -log_view 0 KSP preconditioned resid norm 5.925618307774e+02 true resid norm 1.489143042155e+03 ||r(i)||/||b|| 1.000000000000e+00 1 KSP preconditioned resid norm 2.022444592869e+02 true resid norm 5.516335411837e+02 ||r(i)||/||b|| 3.704369060378e-01 2 KSP preconditioned resid norm 1.058407283487e+02 true resid norm 2.759478800294e+02 ||r(i)||/||b|| 1.853064965673e-01 3 KSP preconditioned resid norm 3.693395834711e+01 true resid norm 1.275254917089e+02 ||r(i)||/||b|| 8.563683145197e-02 4 KSP preconditioned resid norm 1.408997772215e+01 true resid norm 4.766518770804e+01 ||r(i)||/||b|| 3.200846819863e-02 5 KSP preconditioned resid norm 5.361143512330e+00 true resid norm 2.037934685918e+01 ||r(i)||/||b|| 1.368528494730e-02 6 KSP preconditioned resid norm 1.510583748885e+00 true resid norm 6.398081426940e+00 ||r(i)||/||b|| 4.296485458965e-03 7 KSP preconditioned resid norm 5.309564077280e-01 true resid norm 2.119763900049e+00 ||r(i)||/||b|| 1.423479034614e-03 8 KSP preconditioned resid norm 1.241952019288e-01 true resid norm 4.237260240924e-01 ||r(i)||/||b|| 2.845435341652e-04 9 KSP preconditioned resid norm 5.821568520561e-02 true resid norm 2.020452441170e-01 ||r(i)||/||b|| 1.356788692539e-04 10 KSP preconditioned resid norm 1.821252523719e-02 true resid norm 7.650904617862e-02 ||r(i)||/||b|| 5.137790260087e-05 11 KSP preconditioned resid norm 4.368507888148e-03 true resid norm 1.808437158965e-02 ||r(i)||/||b|| 1.214414671909e-05 Linear solve converged due to CONVERGED_RTOL iterations 11 KSP Object: 4 MPI processes type: gmres restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement happy breakdown tolerance 1e-30 maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using PRECONDITIONED norm type for convergence test PC Object: 4 MPI processes type: bjacobi number of blocks = 4 Local solver information for first block is in the following KSP and PC objects on rank 0: Use -ksp_view ::ascii_info_detail to display information for all blocks KSP Object: (sub_) 1 MPI processes type: preonly maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using NONE norm type for convergence test PC Object: (sub_) 1 MPI processes type: ilu out-of-place factorization 0 levels of fill tolerance for zero pivot 2.22045e-14 matrix ordering: natural factor fill ratio given 1., needed 1. Factored matrix follows: Mat Object: 1 MPI processes type: seqaij rows=6, cols=6 package used to perform factorization: petsc total: nonzeros=16, allocated nonzeros=16 not using I-node routines linear system matrix = precond matrix: Mat Object: (sub_) 1 MPI processes type: seqaij rows=6, cols=6 total: nonzeros=16, allocated nonzeros=30 total number of mallocs used during MatSetValues calls=0 not using I-node routines linear system matrix = precond matrix: Mat Object: 4 MPI processes type: mpiaij rows=24, cols=24 total: nonzeros=98, allocated nonzeros=240 total number of mallocs used during MatSetValues calls=0 not using I-node (on process 0) routines Norm of error 0.00593779, Iterations 11 0 KSP preconditioned resid norm 7.186228668401e+02 true resid norm 3.204758181205e+03 ||r(i)||/||b|| 1.000000000000e+00 1 KSP preconditioned resid norm 1.689435569629e+02 true resid norm 8.091360976081e+02 ||r(i)||/||b|| 2.524796105845e-01 2 KSP preconditioned resid norm 4.515826460465e+01 true resid norm 2.188560284713e+02 ||r(i)||/||b|| 6.829096490175e-02 3 KSP preconditioned resid norm 6.171620381119e+00 true resid norm 3.403822057849e+01 ||r(i)||/||b|| 1.062115100544e-02 4 KSP preconditioned resid norm 1.386278591005e+00 true resid norm 7.417896285993e+00 ||r(i)||/||b|| 2.314650861802e-03 5 KSP preconditioned resid norm 2.592583219309e-01 true resid norm 1.555506429237e+00 ||r(i)||/||b|| 4.853740411242e-04 6 KSP preconditioned resid norm 4.031309698756e-02 true resid norm 2.532670674279e-01 ||r(i)||/||b|| 7.902844867150e-05 7 KSP preconditioned resid norm 8.307351448472e-03 true resid norm 5.128232470272e-02 ||r(i)||/||b|| 1.600193268980e-05 8 KSP preconditioned resid norm 7.297751517013e-04 true resid norm 4.388588867606e-03 ||r(i)||/||b|| 1.369397820199e-06 Linear solve converged due to CONVERGED_RTOL iterations 8 KSP Object: 4 MPI processes type: gmres restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement happy breakdown tolerance 1e-30 maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using PRECONDITIONED norm type for convergence test PC Object: 4 MPI processes type: bjacobi number of blocks = 4 Local solver information for first block is in the following KSP and PC objects on rank 0: Use -ksp_view ::ascii_info_detail to display information for all blocks KSP Object: (sub_) 1 MPI processes type: preonly maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using NONE norm type for convergence test PC Object: (sub_) 1 MPI processes type: ilu out-of-place factorization 0 levels of fill tolerance for zero pivot 2.22045e-14 matrix ordering: natural factor fill ratio given 1., needed 1. Factored matrix follows: Mat Object: 1 MPI processes type: seqaij rows=6, cols=6 package used to perform factorization: petsc total: nonzeros=16, allocated nonzeros=16 not using I-node routines linear system matrix = precond matrix: Mat Object: (sub_) 1 MPI processes type: seqaij rows=6, cols=6 total: nonzeros=16, allocated nonzeros=30 total number of mallocs used during MatSetValues calls=0 not using I-node routines linear system matrix = precond matrix: Mat Object: 4 MPI processes type: mpiaij rows=24, cols=24 total: nonzeros=98, allocated nonzeros=240 total number of mallocs used during MatSetValues calls=0 not using I-node (on process 0) routines Norm of error 0.000783616, Iterations 8 **************************************** *********************************************************************************************************************** *** WIDEN YOUR WINDOW TO 160 CHARACTERS. Use 'enscript -r -fCourier9' to print this document *** **************************************************************************************************************************************************************** ------------------------------------------------------------------ PETSc Performance Summary: ------------------------------------------------------------------- ./ex5 on a named kitesrv with 4 processors, by user Tue May 31 13:41:49 2022 Using 3 OpenMP threads Using Petsc Release Version 3.17.1, unknown Max Max/Min Avg Total Time (sec): 3.720e-03 1.002 3.718e-03 Objects: 8.900e+01 1.000 8.900e+01 Flops: 7.334e+03 1.023 7.252e+03 2.901e+04 Flops/sec: 1.975e+06 1.025 1.950e+06 7.801e+06 MPI Msg Count: 1.380e+02 1.468 1.170e+02 4.680e+02 MPI Msg Len (bytes): 4.364e+03 1.641 3.005e+01 1.406e+04 MPI Reductions: 1.050e+02 1.000 Flop counting convention: 1 flop = 1 real number operation of type (multiply/divide/add/subtract) e.g., VecAXPY() for real vectors of length N --> 2N flops and VecAXPY() for complex vectors of length N --> 8N flops Summary of Stages: ----- Time ------ ----- Flop ------ --- Messages --- -- Message Lengths -- -- Reductions -- Avg %Total Avg %Total Count %Total Avg %Total Count %Total 0: Main Stage: 1.2380e-04 3.3% 0.0000e+00 0.0% 0.000e+00 0.0% 0.000e+00 0.0% 2.000e+00 1.9% 1: Original Solve: 2.9587e-03 79.6% 1.7574e+04 60.6% 2.660e+02 56.8% 2.824e+01 53.4% 5.300e+01 50.5% 2: Second Solve: 6.3269e-04 17.0% 1.1432e+04 39.4% 2.020e+02 43.2% 3.244e+01 46.6% 3.200e+01 30.5% ------------------------------------------------------------------------------------------------------------------------ See the 'Profiling' chapter of the users' manual for details on interpreting output. Phase summary info: Count: number of times phase was executed Time and Flop: Max - maximum over all processors Ratio - ratio of maximum to minimum over all processors Mess: number of messages sent AvgLen: average message length (bytes) Reduct: number of global reductions Global: entire computation Stage: stages of a computation. Set stages with PetscLogStagePush() and PetscLogStagePop(). %T - percent time in this phase %F - percent flop in this phase %M - percent messages in this phase %L - percent message lengths in this phase %R - percent reductions in this phase Total Mflop/s: 10e-6 * (sum of flop over all processors)/(max time over all processors) ------------------------------------------------------------------------------------------------------------------------ Event Count Time (sec) Flop --- Global --- --- Stage ---- Total Max Ratio Max Ratio Max Ratio Mess AvgLen Reduct %T %F %M %L %R %T %F %M %L %R Mflop/s ------------------------------------------------------------------------------------------------------------------------ --- Event Stage 0: Main Stage --- Event Stage 1: Original Solve BuildTwoSided 3 1.0 4.6371e-05 2.1 0.00e+00 0.0 1.0e+01 4.0e+00 3.0e+00 1 0 2 0 3 1 0 4 1 6 0 BuildTwoSidedF 2 1.0 3.5755e-05 2.1 0.00e+00 0.0 0.0e+00 0.0e+00 2.0e+00 1 0 0 0 2 1 0 0 0 4 0 MatMult 24 1.0 4.3441e-04 2.8 1.10e+03 1.1 2.6e+02 2.9e+01 1.0e+00 8 14 56 53 1 10 23 98100 2 10 MatSolve 12 1.0 2.7930e-06 1.7 3.12e+02 1.2 0.0e+00 0.0e+00 0.0e+00 0 4 0 0 0 0 7 0 0 0 412 MatLUFactorNum 1 1.0 2.2540e-06 1.3 2.10e+01 1.2 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 35 MatILUFactorSym 1 1.0 9.7690e-06 1.3 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatAssemblyBegin 1 1.0 2.9545e-05 2.1 0.00e+00 0.0 0.0e+00 0.0e+00 1.0e+00 1 0 0 0 1 1 0 0 0 2 0 MatAssemblyEnd 1 1.0 1.8802e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 5.0e+00 5 0 0 0 5 6 0 0 0 9 0 MatGetRowIJ 1 1.0 2.3500e-07 2.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatGetOrdering 1 1.0 9.4980e-06 1.6 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatView 3 3.0 8.1759e-05 4.0 0.00e+00 0.0 0.0e+00 0.0e+00 1.0e+00 1 0 0 0 1 1 0 0 0 2 0 VecMDot 11 1.0 8.3470e-05 3.4 7.26e+02 1.0 0.0e+00 0.0e+00 1.1e+01 1 10 0 0 10 1 17 0 0 21 35 VecNorm 26 1.0 9.3217e-05 1.3 3.12e+02 1.0 0.0e+00 0.0e+00 2.6e+01 2 4 0 0 25 3 7 0 0 49 13 VecScale 12 1.0 4.7450e-06 1.9 7.20e+01 1.0 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 2 0 0 0 61 VecCopy 13 1.0 4.1490e-06 1.6 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecSet 25 1.0 4.0900e-06 1.4 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecAXPY 13 1.0 5.6990e-06 1.3 1.56e+02 1.0 0.0e+00 0.0e+00 0.0e+00 0 2 0 0 0 0 4 0 0 0 109 VecAYPX 12 1.0 2.0830e-06 1.5 7.20e+01 1.0 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 2 0 0 0 138 VecMAXPY 23 1.0 5.4380e-06 1.6 1.72e+03 1.0 0.0e+00 0.0e+00 0.0e+00 0 24 0 0 0 0 39 0 0 0 1262 VecAssemblyBegin 1 1.0 1.6203e-05 1.2 0.00e+00 0.0 0.0e+00 0.0e+00 1.0e+00 0 0 0 0 1 0 0 0 0 2 0 VecAssemblyEnd 1 1.0 1.0170e-06 1.6 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecScatterBegin 24 1.0 8.1684e-05 1.1 0.00e+00 0.0 2.6e+02 2.9e+01 1.0e+00 2 0 56 53 1 3 0 98100 2 0 VecScatterEnd 24 1.0 3.4411e-04 7.7 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 5 0 0 0 0 7 0 0 0 0 0 VecNormalize 12 1.0 5.7443e-05 1.7 2.16e+02 1.0 0.0e+00 0.0e+00 1.2e+01 1 3 0 0 11 2 5 0 0 23 15 SFSetGraph 1 1.0 1.0780e-06 1.3 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 SFSetUp 1 1.0 3.1964e-05 1.1 0.00e+00 0.0 2.0e+01 9.6e+00 1.0e+00 1 0 4 1 1 1 0 8 3 2 0 SFPack 24 1.0 7.5830e-06 1.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 SFUnpack 24 1.0 1.6541e-05 9.4 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 KSPSetUp 2 1.0 3.2793e-0413.8 0.00e+00 0.0 0.0e+00 0.0e+00 2.0e+00 3 0 0 0 2 3 0 0 0 4 0 KSPSolve 1 1.0 1.0170e-03 1.0 4.35e+03 1.0 2.3e+02 3.0e+01 3.6e+01 27 59 49 50 34 34 98 86 93 68 17 KSPGMRESOrthog 11 1.0 9.3328e-05 2.7 1.52e+03 1.0 0.0e+00 0.0e+00 1.1e+01 1 21 0 0 10 2 35 0 0 21 65 PCSetUp 2 1.0 1.6209e-04 1.1 2.10e+01 1.2 0.0e+00 0.0e+00 0.0e+00 4 0 0 0 0 5 0 0 0 0 0 PCSetUpOnBlocks 1 1.0 5.6611e-05 1.2 2.10e+01 1.2 0.0e+00 0.0e+00 0.0e+00 1 0 0 0 0 2 0 0 0 0 1 PCApply 12 1.0 4.0174e-05 1.4 3.12e+02 1.2 0.0e+00 0.0e+00 0.0e+00 1 4 0 0 0 1 7 0 0 0 29 --- Event Stage 2: Second Solve BuildTwoSided 1 1.0 1.9497e-05 2.0 0.00e+00 0.0 8.0e+00 4.0e+00 1.0e+00 0 0 2 0 1 2 0 4 0 3 0 BuildTwoSidedF 1 1.0 2.1054e-05 1.9 0.00e+00 0.0 1.6e+01 6.6e+01 1.0e+00 0 0 3 8 1 2 0 8 16 3 0 MatMult 18 1.0 1.1072e-04 1.7 8.28e+02 1.1 1.8e+02 3.0e+01 0.0e+00 2 11 38 39 0 14 27 89 84 0 28 MatSolve 9 1.0 1.8810e-06 1.5 2.34e+02 1.2 0.0e+00 0.0e+00 0.0e+00 0 3 0 0 0 0 8 0 0 0 459 MatLUFactorNum 1 1.0 1.3380e-06 1.4 2.10e+01 1.2 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 1 0 0 0 58 MatAssemblyBegin 1 1.0 3.0984e-05 1.6 0.00e+00 0.0 1.6e+01 6.6e+01 1.0e+00 1 0 3 8 1 4 0 8 16 3 0 MatAssemblyEnd 1 1.0 2.4243e-05 1.1 1.60e+01 1.0 0.0e+00 0.0e+00 2.0e+00 1 0 0 0 2 4 1 0 0 6 3 MatZeroEntries 1 1.0 2.1270e-06 2.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatView 3 3.0 5.9168e-05 3.5 0.00e+00 0.0 0.0e+00 0.0e+00 1.0e+00 1 0 0 0 1 5 0 0 0 3 0 VecMDot 8 1.0 4.9608e-05 3.0 3.96e+02 1.0 0.0e+00 0.0e+00 8.0e+00 1 5 0 0 8 4 14 0 0 25 32 VecNorm 20 1.0 5.5958e-05 1.1 2.40e+02 1.0 0.0e+00 0.0e+00 2.0e+01 1 3 0 0 19 8 8 0 0 62 17 VecScale 9 1.0 2.6620e-06 1.6 5.40e+01 1.0 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 2 0 0 0 81 VecCopy 10 1.0 3.7840e-06 1.6 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 1 0 0 0 0 0 VecSet 19 1.0 2.4540e-06 1.3 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecAXPY 10 1.0 2.1130e-06 1.2 1.20e+02 1.0 0.0e+00 0.0e+00 0.0e+00 0 2 0 0 0 0 4 0 0 0 227 VecAYPX 9 1.0 1.3480e-06 1.5 5.40e+01 1.0 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 2 0 0 0 160 VecMAXPY 17 1.0 3.6180e-06 1.3 9.60e+02 1.0 0.0e+00 0.0e+00 0.0e+00 0 13 0 0 0 1 34 0 0 0 1061 VecScatterBegin 18 1.0 2.5301e-05 1.3 0.00e+00 0.0 1.8e+02 3.0e+01 0.0e+00 1 0 38 39 0 4 0 89 84 0 0 VecScatterEnd 18 1.0 7.8795e-05 2.6 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 1 0 0 0 0 9 0 0 0 0 0 VecNormalize 9 1.0 2.5470e-05 1.3 1.62e+02 1.0 0.0e+00 0.0e+00 9.0e+00 1 2 0 0 9 4 6 0 0 28 25 SFPack 18 1.0 3.6420e-06 1.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 1 0 0 0 0 0 SFUnpack 18 1.0 1.5920e-06 1.3 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 KSPSetUp 2 1.0 2.6500e-07 1.7 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 KSPSolve 1 1.0 2.8180e-04 1.0 2.78e+03 1.0 1.7e+02 3.0e+01 2.7e+01 7 38 36 37 26 44 96 84 79 84 39 KSPGMRESOrthog 8 1.0 5.5511e-05 2.6 8.28e+02 1.0 0.0e+00 0.0e+00 8.0e+00 1 11 0 0 8 5 29 0 0 25 60 PCSetUp 2 1.0 4.3870e-06 1.2 2.10e+01 1.2 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 1 1 0 0 0 18 PCSetUpOnBlocks 1 1.0 3.4260e-06 1.4 2.10e+01 1.2 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 1 0 0 0 23 PCApply 9 1.0 2.2834e-05 1.5 2.34e+02 1.2 0.0e+00 0.0e+00 0.0e+00 1 3 0 0 0 3 8 0 0 0 38 ------------------------------------------------------------------------------------------------------------------------ Memory usage is given in bytes: Object Type Creations Destructions Memory Descendants' Mem. Reports information only for process 0. --- Event Stage 0: Main Stage Viewer 1 1 856 0. --- Event Stage 1: Original Solve Matrix 5 1 3376 0. Vector 47 26 47584 0. Index Set 5 2 1844 0. Star Forest Graph 3 0 0 0. Krylov Solver 2 0 0 0. Preconditioner 2 0 0 0. Viewer 2 1 856 0. Distributed Mesh 1 0 0 0. Discrete System 1 0 0 0. Weak Form 1 0 0 0. --- Event Stage 2: Second Solve Matrix 0 4 14732 0. Vector 18 39 70976 0. Index Set 0 3 2808 0. Star Forest Graph 0 3 3408 0. Krylov Solver 0 2 20534 0. Preconditioner 0 2 1968 0. Viewer 1 1 856 0. Distributed Mesh 0 1 5080 0. Discrete System 0 1 976 0. Weak Form 0 1 632 0. ======================================================================================================================== Average time to get PetscTime(): 2.95e-08 Average time for MPI_Barrier(): 1.2316e-06 Average time for zero size MPI_Send(): 6.7525e-07 #PETSc Option Table entries: -ksp_converged_reason -ksp_monitor_true_residual -ksp_view -log_view #End of PETSc Option Table entries Compiled without FORTRAN kernels Compiled with full precision matrices (default) sizeof(short) 2 sizeof(int) 4 sizeof(long) 8 sizeof(void*) 8 sizeof(PetscScalar) 8 sizeof(PetscInt) 4 Configure options: --with-python --prefix=/data/raid1/tmp/petsc/install --with-debugging=no --with-blas-lib=/usr/lib/libblas.so --with-lapack-lib=/usr/lib/liblapack.so --with-openmp=true --with-mpi=true --download-openmpi=yes ----------------------------------------- Libraries compiled on 2022-05-31 10:31:52 on kitesrv Machine characteristics: Linux-4.4.0-116-generic-x86_64-with-Ubuntu-16.04-xenial Using PETSc directory: /data/raid1/tmp/petsc/install Using PETSc arch: ----------------------------------------- Using C compiler: /data/raid1/tmp/petsc/install/bin/mpicc -fPIC -Wall -Wwrite-strings -Wno-unknown-pragmas -Wno-lto-type-mismatch -fstack-protector -fvisibility=hidden -g -O -fopenmp Using Fortran compiler: /data/raid1/tmp/petsc/install/bin/mpif90 -fPIC -Wall -ffree-line-length-0 -Wno-lto-type-mismatch -Wno-unused-dummy-argument -g -O -fopenmp ----------------------------------------- Using include paths: -I/data/raid1/tmp/petsc/install/include ----------------------------------------- Using C linker: /data/raid1/tmp/petsc/install/bin/mpicc Using Fortran linker: /data/raid1/tmp/petsc/install/bin/mpif90 Using libraries: -Wl,-rpath,/data/raid1/tmp/petsc/install/lib -L/data/raid1/tmp/petsc/install/lib -lpetsc -Wl,-rpath,/data/raid1/tmp/petsc/install/lib -L/data/raid1/tmp/petsc/install/lib -Wl,-rpath,/usr/lib/gcc/x86_64-linux-gnu/5 -L/usr/lib/gcc/x86_64-linux-gnu/5 /usr/lib/liblapack.so /usr/lib/libblas.so -lm -lstdc++ -ldl -lmpi_usempif08 -lmpi_usempi_ignore_tkr -lmpi_mpifh -lmpi -lgfortran -lm -lgfortran -lm -lgcc_s -lquadmath -lpthread -lstdc++ -ldl ----------------------------------------- 0.18user 0.03system 0:00.12elapsed 177%CPU (0avgtext+0avgdata 20060maxresident)k 0inputs+0outputs (0major+9903minor)pagefaults 0swaps user at kitesrv:/data/raid1/tmp/petsc/src/ksp/ksp/tutorials$ -------------- next part -------------- user at kitesrv:/data/raid1/tmp/petsc/src/ksp/ksp/tutorials$ LD_LIBRARY_PATH=/data/raid1/tmp/petsc/install/lib time /data/raid1/tmp/petsc/install/bin/mpirun -n 2 ./ex5 -ksp_view -ksp_monitor_true_residual -ksp_converged_reason -log_view 0 KSP preconditioned resid norm 2.540548908415e+02 true resid norm 5.690404203569e+02 ||r(i)||/||b|| 1.000000000000e+00 1 KSP preconditioned resid norm 5.524657937376e+01 true resid norm 1.536964982563e+02 ||r(i)||/||b|| 2.700976815669e-01 2 KSP preconditioned resid norm 1.904775180107e+01 true resid norm 6.861661839730e+01 ||r(i)||/||b|| 1.205830305592e-01 3 KSP preconditioned resid norm 4.708471233594e+00 true resid norm 2.115510975962e+01 ||r(i)||/||b|| 3.717681381290e-02 4 KSP preconditioned resid norm 1.055333034486e+00 true resid norm 4.779687437415e+00 ||r(i)||/||b|| 8.399556984752e-03 5 KSP preconditioned resid norm 5.287930275880e-02 true resid norm 2.395448983884e-01 ||r(i)||/||b|| 4.209628873783e-04 6 KSP preconditioned resid norm 6.852115363810e-03 true resid norm 2.760282135279e-02 ||r(i)||/||b|| 4.850766371829e-05 7 KSP preconditioned resid norm 8.937861499755e-04 true resid norm 3.024963693385e-03 ||r(i)||/||b|| 5.315903027570e-06 Linear solve converged due to CONVERGED_RTOL iterations 7 KSP Object: 2 MPI processes type: gmres restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement happy breakdown tolerance 1e-30 maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using PRECONDITIONED norm type for convergence test PC Object: 2 MPI processes type: bjacobi number of blocks = 2 Local solver information for first block is in the following KSP and PC objects on rank 0: Use -ksp_view ::ascii_info_detail to display information for all blocks KSP Object: (sub_) 1 MPI processes type: preonly maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using NONE norm type for convergence test PC Object: (sub_) 1 MPI processes type: ilu out-of-place factorization 0 levels of fill tolerance for zero pivot 2.22045e-14 matrix ordering: natural factor fill ratio given 1., needed 1. Factored matrix follows: Mat Object: 1 MPI processes type: seqaij rows=6, cols=6 package used to perform factorization: petsc total: nonzeros=18, allocated nonzeros=18 not using I-node routines linear system matrix = precond matrix: Mat Object: (sub_) 1 MPI processes type: seqaij rows=6, cols=6 total: nonzeros=18, allocated nonzeros=30 total number of mallocs used during MatSetValues calls=0 not using I-node routines linear system matrix = precond matrix: Mat Object: 2 MPI processes type: mpiaij rows=12, cols=12 total: nonzeros=46, allocated nonzeros=120 total number of mallocs used during MatSetValues calls=0 not using I-node (on process 0) routines Norm of error 0.00121238, Iterations 7 0 KSP preconditioned resid norm 2.502924928760e+02 true resid norm 1.031693268370e+03 ||r(i)||/||b|| 1.000000000000e+00 1 KSP preconditioned resid norm 4.655858214018e+01 true resid norm 2.232934339942e+02 ||r(i)||/||b|| 2.164339352015e-01 2 KSP preconditioned resid norm 7.917304062742e+00 true resid norm 4.856470565410e+01 ||r(i)||/||b|| 4.707281431702e-02 3 KSP preconditioned resid norm 9.318670651891e-01 true resid norm 6.270700755466e+00 ||r(i)||/||b|| 6.078066948496e-03 4 KSP preconditioned resid norm 1.408588814076e-01 true resid norm 8.958183659370e-01 ||r(i)||/||b|| 8.682991286279e-04 5 KSP preconditioned resid norm 4.306995949139e-03 true resid norm 2.763097772714e-02 ||r(i)||/||b|| 2.678216343390e-05 6 KSP preconditioned resid norm 3.435999542630e-04 true resid norm 2.406928714402e-03 ||r(i)||/||b|| 2.332988678122e-06 Linear solve converged due to CONVERGED_RTOL iterations 6 KSP Object: 2 MPI processes type: gmres restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement happy breakdown tolerance 1e-30 maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using PRECONDITIONED norm type for convergence test PC Object: 2 MPI processes type: bjacobi number of blocks = 2 Local solver information for first block is in the following KSP and PC objects on rank 0: Use -ksp_view ::ascii_info_detail to display information for all blocks KSP Object: (sub_) 1 MPI processes type: preonly maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using NONE norm type for convergence test PC Object: (sub_) 1 MPI processes type: ilu out-of-place factorization 0 levels of fill tolerance for zero pivot 2.22045e-14 matrix ordering: natural factor fill ratio given 1., needed 1. Factored matrix follows: Mat Object: 1 MPI processes type: seqaij rows=6, cols=6 package used to perform factorization: petsc total: nonzeros=18, allocated nonzeros=18 not using I-node routines linear system matrix = precond matrix: Mat Object: (sub_) 1 MPI processes type: seqaij rows=6, cols=6 total: nonzeros=18, allocated nonzeros=30 total number of mallocs used during MatSetValues calls=0 not using I-node routines linear system matrix = precond matrix: Mat Object: 2 MPI processes type: mpiaij rows=12, cols=12 total: nonzeros=46, allocated nonzeros=120 total number of mallocs used during MatSetValues calls=0 not using I-node (on process 0) routines Norm of error 0.000322889, Iterations 6 **************************************** *********************************************************************************************************************** *** WIDEN YOUR WINDOW TO 160 CHARACTERS. Use 'enscript -r -fCourier9' to print this document *** **************************************************************************************************************************************************************** ------------------------------------------------------------------ PETSc Performance Summary: ------------------------------------------------------------------- ./ex5 on a named kitesrv with 2 processors, by user Tue May 31 13:42:24 2022 Using 3 OpenMP threads Using Petsc Release Version 3.17.1, unknown Max Max/Min Avg Total Time (sec): 3.459e-03 1.000 3.459e-03 Objects: 7.700e+01 1.000 7.700e+01 Flops: 4.400e+03 1.002 4.396e+03 8.792e+03 Flops/sec: 1.272e+06 1.002 1.271e+06 2.542e+06 MPI Msg Count: 3.600e+01 1.000 3.600e+01 7.200e+01 MPI Msg Len (bytes): 1.104e+03 1.000 3.067e+01 2.208e+03 MPI Reductions: 8.700e+01 1.000 Flop counting convention: 1 flop = 1 real number operation of type (multiply/divide/add/subtract) e.g., VecAXPY() for real vectors of length N --> 2N flops and VecAXPY() for complex vectors of length N --> 8N flops Summary of Stages: ----- Time ------ ----- Flop ------ --- Messages --- -- Message Lengths -- -- Reductions -- Avg %Total Avg %Total Count %Total Avg %Total Count %Total 0: Main Stage: 1.2309e-04 3.6% 0.0000e+00 0.0% 0.000e+00 0.0% 0.000e+00 0.0% 2.000e+00 2.3% 1: Original Solve: 2.7587e-03 79.8% 4.7880e+03 54.5% 3.800e+01 52.8% 2.821e+01 48.6% 4.100e+01 47.1% 2: Second Solve: 5.7332e-04 16.6% 4.0040e+03 45.5% 3.400e+01 47.2% 3.341e+01 51.4% 2.600e+01 29.9% ------------------------------------------------------------------------------------------------------------------------ See the 'Profiling' chapter of the users' manual for details on interpreting output. Phase summary info: Count: number of times phase was executed Time and Flop: Max - maximum over all processors Ratio - ratio of maximum to minimum over all processors Mess: number of messages sent AvgLen: average message length (bytes) Reduct: number of global reductions Global: entire computation Stage: stages of a computation. Set stages with PetscLogStagePush() and PetscLogStagePop(). %T - percent time in this phase %F - percent flop in this phase %M - percent messages in this phase %L - percent message lengths in this phase %R - percent reductions in this phase Total Mflop/s: 10e-6 * (sum of flop over all processors)/(max time over all processors) ------------------------------------------------------------------------------------------------------------------------ Event Count Time (sec) Flop --- Global --- --- Stage ---- Total Max Ratio Max Ratio Max Ratio Mess AvgLen Reduct %T %F %M %L %R %T %F %M %L %R Mflop/s ------------------------------------------------------------------------------------------------------------------------ --- Event Stage 0: Main Stage --- Event Stage 1: Original Solve BuildTwoSided 3 1.0 2.1347e-05 1.0 0.00e+00 0.0 2.0e+00 4.0e+00 3.0e+00 1 0 3 0 3 1 0 5 1 7 0 BuildTwoSidedF 2 1.0 1.9346e-05 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 2.0e+00 1 0 0 0 2 1 0 0 0 5 0 MatMult 16 1.0 1.5458e-04 1.8 6.40e+02 1.0 3.6e+01 3.0e+01 1.0e+00 4 15 50 48 1 4 27 95 99 2 8 MatSolve 8 1.0 2.2670e-06 1.5 2.40e+02 1.0 0.0e+00 0.0e+00 0.0e+00 0 5 0 0 0 0 10 0 0 0 212 MatLUFactorNum 1 1.0 2.4620e-06 1.3 3.20e+01 1.1 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 1 0 0 0 24 MatILUFactorSym 1 1.0 1.1469e-05 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatAssemblyBegin 1 1.0 1.7848e-05 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 1.0e+00 0 0 0 0 1 1 0 0 0 2 0 MatAssemblyEnd 1 1.0 1.5707e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 5.0e+00 5 0 0 0 6 6 0 0 0 12 0 MatGetRowIJ 1 1.0 3.0100e-07 2.4 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatGetOrdering 1 1.0 8.4270e-06 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatView 3 3.0 9.6613e-05 3.2 0.00e+00 0.0 0.0e+00 0.0e+00 1.0e+00 2 0 0 0 1 2 0 0 0 2 0 VecMDot 7 1.0 3.1623e-05 2.2 3.08e+02 1.0 0.0e+00 0.0e+00 7.0e+00 1 7 0 0 8 1 13 0 0 17 19 VecNorm 18 1.0 2.5209e-04 1.0 2.16e+02 1.0 0.0e+00 0.0e+00 1.8e+01 7 5 0 0 21 9 9 0 0 44 2 VecScale 8 1.0 2.9390e-06 1.0 4.80e+01 1.0 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 2 0 0 0 33 VecCopy 9 1.0 2.5980e-06 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecSet 17 1.0 3.0360e-06 1.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecAXPY 9 1.0 5.6680e-06 1.2 1.08e+02 1.0 0.0e+00 0.0e+00 0.0e+00 0 2 0 0 0 0 5 0 0 0 38 VecAYPX 8 1.0 1.1690e-06 1.1 4.80e+01 1.0 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 2 0 0 0 82 VecMAXPY 15 1.0 2.9050e-06 1.0 7.56e+02 1.0 0.0e+00 0.0e+00 0.0e+00 0 17 0 0 0 0 32 0 0 0 520 VecAssemblyBegin 1 1.0 1.5341e-05 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 1.0e+00 0 0 0 0 1 1 0 0 0 2 0 VecAssemblyEnd 1 1.0 9.4300e-07 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecScatterBegin 16 1.0 5.1321e-05 1.0 0.00e+00 0.0 3.6e+01 3.0e+01 1.0e+00 1 0 50 48 1 2 0 95 99 2 0 VecScatterEnd 16 1.0 9.3558e-05 3.8 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 2 0 0 0 0 2 0 0 0 0 0 VecNormalize 8 1.0 2.0754e-05 1.0 1.44e+02 1.0 0.0e+00 0.0e+00 8.0e+00 1 3 0 0 9 1 6 0 0 20 14 SFSetGraph 1 1.0 1.0930e-06 1.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 SFSetUp 1 1.0 2.0803e-05 1.0 0.00e+00 0.0 4.0e+00 1.0e+01 1.0e+00 1 0 6 2 1 1 0 11 4 2 0 SFPack 16 1.0 2.1140e-06 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 SFUnpack 16 1.0 1.5950e-06 1.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 KSPSetUp 2 1.0 2.2458e-05 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 2.0e+00 1 0 0 0 2 1 0 0 0 5 0 KSPSolve 1 1.0 9.9155e-04 1.0 2.30e+03 1.0 3.0e+01 3.2e+01 2.4e+01 29 52 42 43 28 36 96 79 90 59 5 KSPGMRESOrthog 7 1.0 3.8334e-05 1.9 6.44e+02 1.0 0.0e+00 0.0e+00 7.0e+00 1 15 0 0 8 1 27 0 0 17 34 PCSetUp 2 1.0 1.1958e-04 1.0 3.20e+01 1.1 0.0e+00 0.0e+00 0.0e+00 3 1 0 0 0 4 1 0 0 0 1 PCSetUpOnBlocks 1 1.0 5.4691e-05 1.0 3.20e+01 1.1 0.0e+00 0.0e+00 0.0e+00 2 1 0 0 0 2 1 0 0 0 1 PCApply 8 1.0 5.9135e-05 1.5 2.40e+02 1.0 0.0e+00 0.0e+00 0.0e+00 1 5 0 0 0 2 10 0 0 0 8 --- Event Stage 2: Second Solve BuildTwoSided 1 1.0 4.5420e-06 1.3 0.00e+00 0.0 2.0e+00 4.0e+00 1.0e+00 0 0 3 0 1 1 0 6 1 4 0 BuildTwoSidedF 1 1.0 5.9960e-06 1.2 0.00e+00 0.0 4.0e+00 5.8e+01 1.0e+00 0 0 6 11 1 1 0 12 20 4 0 MatMult 14 1.0 9.5330e-05 1.8 5.60e+02 1.0 2.8e+01 3.2e+01 0.0e+00 2 13 39 41 0 13 28 82 79 0 12 MatSolve 7 1.0 1.4380e-06 1.3 2.10e+02 1.0 0.0e+00 0.0e+00 0.0e+00 0 5 0 0 0 0 10 0 0 0 292 MatLUFactorNum 1 1.0 1.4460e-06 1.0 3.20e+01 1.1 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 1 0 0 0 41 MatAssemblyBegin 1 1.0 1.4872e-05 1.1 0.00e+00 0.0 4.0e+00 5.8e+01 1.0e+00 0 0 6 11 1 2 0 12 20 4 0 MatAssemblyEnd 1 1.0 1.9540e-05 1.0 7.00e+00 1.0 0.0e+00 0.0e+00 2.0e+00 1 0 0 0 2 3 0 0 0 8 1 MatZeroEntries 1 1.0 2.3710e-06 1.4 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatView 3 3.0 7.2415e-05 3.9 0.00e+00 0.0 0.0e+00 0.0e+00 1.0e+00 1 0 0 0 1 8 0 0 0 4 0 VecMDot 6 1.0 1.1563e-05 1.3 2.31e+02 1.0 0.0e+00 0.0e+00 6.0e+00 0 5 0 0 7 2 12 0 0 23 40 VecNorm 16 1.0 3.3041e-05 2.4 1.92e+02 1.0 0.0e+00 0.0e+00 1.6e+01 1 4 0 0 18 4 10 0 0 62 12 VecScale 7 1.0 1.4170e-06 1.1 4.20e+01 1.0 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 2 0 0 0 59 VecCopy 8 1.0 2.3300e-06 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecSet 15 1.0 1.9460e-06 1.3 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecAXPY 8 1.0 1.4530e-06 1.4 9.60e+01 1.0 0.0e+00 0.0e+00 0.0e+00 0 2 0 0 0 0 5 0 0 0 132 VecAYPX 7 1.0 1.0070e-06 1.2 4.20e+01 1.0 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 2 0 0 0 83 VecMAXPY 13 1.0 2.4410e-06 1.2 5.76e+02 1.0 0.0e+00 0.0e+00 0.0e+00 0 13 0 0 0 0 29 0 0 0 472 VecScatterBegin 14 1.0 1.5003e-05 1.1 0.00e+00 0.0 2.8e+01 3.2e+01 0.0e+00 0 0 39 41 0 2 0 82 79 0 0 VecScatterEnd 14 1.0 7.5876e-05 2.5 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 2 0 0 0 0 9 0 0 0 0 0 VecNormalize 7 1.0 1.0780e-05 1.2 1.26e+02 1.0 0.0e+00 0.0e+00 7.0e+00 0 3 0 0 8 2 6 0 0 27 23 SFPack 14 1.0 1.7980e-06 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 SFUnpack 14 1.0 1.3100e-06 1.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 KSPSetUp 2 1.0 2.2500e-07 1.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 KSPSolve 1 1.0 2.0139e-04 1.0 1.88e+03 1.0 2.6e+01 3.2e+01 2.1e+01 6 43 36 38 24 34 94 76 73 81 19 KSPGMRESOrthog 6 1.0 1.4939e-05 1.2 4.83e+02 1.0 0.0e+00 0.0e+00 6.0e+00 0 11 0 0 7 2 24 0 0 23 65 PCSetUp 2 1.0 5.7350e-06 1.2 3.20e+01 1.1 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 1 1 0 0 0 10 PCSetUpOnBlocks 1 1.0 3.6650e-06 1.1 3.20e+01 1.1 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 1 1 0 0 0 16 PCApply 7 1.0 1.4785e-05 1.2 2.10e+02 1.0 0.0e+00 0.0e+00 0.0e+00 0 5 0 0 0 2 10 0 0 0 28 ------------------------------------------------------------------------------------------------------------------------ Memory usage is given in bytes: Object Type Creations Destructions Memory Descendants' Mem. Reports information only for process 0. --- Event Stage 0: Main Stage Viewer 1 1 856 0. --- Event Stage 1: Original Solve Matrix 5 1 3376 0. Vector 39 18 32928 0. Index Set 5 2 1832 0. Star Forest Graph 3 0 0 0. Krylov Solver 2 0 0 0. Preconditioner 2 0 0 0. Viewer 2 1 856 0. Distributed Mesh 1 0 0 0. Discrete System 1 0 0 0. Weak Form 1 0 0 0. --- Event Stage 2: Second Solve Matrix 0 4 14744 0. Vector 14 35 63624 0. Index Set 0 3 2808 0. Star Forest Graph 0 3 3408 0. Krylov Solver 0 2 20534 0. Preconditioner 0 2 1968 0. Viewer 1 1 856 0. Distributed Mesh 0 1 5080 0. Discrete System 0 1 976 0. Weak Form 0 1 632 0. ======================================================================================================================== Average time to get PetscTime(): 2.31e-08 Average time for MPI_Barrier(): 3.43e-07 Average time for zero size MPI_Send(): 5.015e-07 #PETSc Option Table entries: -ksp_converged_reason -ksp_monitor_true_residual -ksp_view -log_view #End of PETSc Option Table entries Compiled without FORTRAN kernels Compiled with full precision matrices (default) sizeof(short) 2 sizeof(int) 4 sizeof(long) 8 sizeof(void*) 8 sizeof(PetscScalar) 8 sizeof(PetscInt) 4 Configure options: --with-python --prefix=/data/raid1/tmp/petsc/install --with-debugging=no --with-blas-lib=/usr/lib/libblas.so --with-lapack-lib=/usr/lib/liblapack.so --with-openmp=true --with-mpi=true --download-openmpi=yes ----------------------------------------- Libraries compiled on 2022-05-31 10:31:52 on kitesrv Machine characteristics: Linux-4.4.0-116-generic-x86_64-with-Ubuntu-16.04-xenial Using PETSc directory: /data/raid1/tmp/petsc/install Using PETSc arch: ----------------------------------------- Using C compiler: /data/raid1/tmp/petsc/install/bin/mpicc -fPIC -Wall -Wwrite-strings -Wno-unknown-pragmas -Wno-lto-type-mismatch -fstack-protector -fvisibility=hidden -g -O -fopenmp Using Fortran compiler: /data/raid1/tmp/petsc/install/bin/mpif90 -fPIC -Wall -ffree-line-length-0 -Wno-lto-type-mismatch -Wno-unused-dummy-argument -g -O -fopenmp ----------------------------------------- Using include paths: -I/data/raid1/tmp/petsc/install/include ----------------------------------------- Using C linker: /data/raid1/tmp/petsc/install/bin/mpicc Using Fortran linker: /data/raid1/tmp/petsc/install/bin/mpif90 Using libraries: -Wl,-rpath,/data/raid1/tmp/petsc/install/lib -L/data/raid1/tmp/petsc/install/lib -lpetsc -Wl,-rpath,/data/raid1/tmp/petsc/install/lib -L/data/raid1/tmp/petsc/install/lib -Wl,-rpath,/usr/lib/gcc/x86_64-linux-gnu/5 -L/usr/lib/gcc/x86_64-linux-gnu/5 /usr/lib/liblapack.so /usr/lib/libblas.so -lm -lstdc++ -ldl -lmpi_usempif08 -lmpi_usempi_ignore_tkr -lmpi_mpifh -lmpi -lgfortran -lm -lgfortran -lm -lgcc_s -lquadmath -lpthread -lstdc++ -ldl ----------------------------------------- 0.07user 0.01system 0:00.09elapsed 92%CPU (0avgtext+0avgdata 19980maxresident)k 0inputs+0outputs (0major+6673minor)pagefaults 0swaps user at kitesrv:/data/raid1/tmp/petsc/src/ksp/ksp/tutorials$ -------------- next part -------------- [lida at head1 tutorials]$ time mpirun -n 3 ./ex5 -ksp_view -ksp_monitor_true_residual -ksp_converged_reason -log_view 2>/dev/null 0 KSP preconditioned resid norm 4.020939481591e+02 true resid norm 9.763918270858e+02 ||r(i)||/||b|| 1.000000000000e+00 1 KSP preconditioned resid norm 1.222668532006e+02 true resid norm 2.875957243877e+02 ||r(i)||/||b|| 2.945494999135e-01 2 KSP preconditioned resid norm 6.915159267765e+01 true resid norm 1.565583438606e+02 ||r(i)||/||b|| 1.603437672434e-01 3 KSP preconditioned resid norm 1.304338629042e+01 true resid norm 4.745383154681e+01 ||r(i)||/||b|| 4.860121749323e-02 4 KSP preconditioned resid norm 4.706344725891e+00 true resid norm 1.682392035308e+01 ||r(i)||/||b|| 1.723070583589e-02 5 KSP preconditioned resid norm 5.039609363554e-01 true resid norm 2.504548411461e+00 ||r(i)||/||b|| 2.565105874489e-03 6 KSP preconditioned resid norm 1.055361110378e-01 true resid norm 5.260792846119e-01 ||r(i)||/||b|| 5.387993529013e-04 7 KSP preconditioned resid norm 1.234936672024e-02 true resid norm 3.467184429455e-02 ||r(i)||/||b|| 3.551017463761e-05 8 KSP preconditioned resid norm 6.180717049285e-03 true resid norm 1.824456980445e-02 ||r(i)||/||b|| 1.868570516296e-05 9 KSP preconditioned resid norm 1.482144308741e-04 true resid norm 6.740391026576e-04 ||r(i)||/||b|| 6.903366906188e-07 Linear solve converged due to CONVERGED_RTOL iterations 9 KSP Object: 3 MPI processes type: gmres restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement happy breakdown tolerance 1e-30 maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using PRECONDITIONED norm type for convergence test PC Object: 3 MPI processes type: bjacobi number of blocks = 3 Local solver information for first block is in the following KSP and PC objects on rank 0: Use -ksp_view ::ascii_info_detail to display information for all blocks KSP Object: (sub_) 1 MPI processes type: preonly maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using NONE norm type for convergence test PC Object: (sub_) 1 MPI processes type: ilu out-of-place factorization 0 levels of fill tolerance for zero pivot 2.22045e-14 matrix ordering: natural factor fill ratio given 1., needed 1. Factored matrix follows: Mat Object: 1 MPI processes type: seqaij rows=6, cols=6 package used to perform factorization: petsc total: nonzeros=16, allocated nonzeros=16 not using I-node routines linear system matrix = precond matrix: Mat Object: (sub_) 1 MPI processes type: seqaij rows=6, cols=6 total: nonzeros=16, allocated nonzeros=30 total number of mallocs used during MatSetValues calls=0 not using I-node routines linear system matrix = precond matrix: Mat Object: 3 MPI processes type: mpiaij rows=18, cols=18 total: nonzeros=72, allocated nonzeros=180 total number of mallocs used during MatSetValues calls=0 not using I-node (on process 0) routines Norm of error 0.000208575, Iterations 9 0 KSP preconditioned resid norm 4.608512576274e+02 true resid norm 2.016631101615e+03 ||r(i)||/||b|| 1.000000000000e+00 1 KSP preconditioned resid norm 9.412072370874e+01 true resid norm 4.038990960334e+02 ||r(i)||/||b|| 2.002840756101e-01 2 KSP preconditioned resid norm 2.282248495386e+01 true resid norm 9.733980188154e+01 ||r(i)||/||b|| 4.826852159703e-02 3 KSP preconditioned resid norm 1.365448582262e+00 true resid norm 7.954929613540e+00 ||r(i)||/||b|| 3.944662763145e-03 4 KSP preconditioned resid norm 2.252869372987e-01 true resid norm 1.285361707036e+00 ||r(i)||/||b|| 6.373806820723e-04 5 KSP preconditioned resid norm 1.586897237676e-02 true resid norm 1.098248593144e-01 ||r(i)||/||b|| 5.445956834962e-05 6 KSP preconditioned resid norm 1.905899805612e-03 true resid norm 1.346481495832e-02 ||r(i)||/||b|| 6.676885498560e-06 Linear solve converged due to CONVERGED_RTOL iterations 6 KSP Object: 3 MPI processes type: gmres restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement happy breakdown tolerance 1e-30 maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using PRECONDITIONED norm type for convergence test PC Object: 3 MPI processes type: bjacobi number of blocks = 3 Local solver information for first block is in the following KSP and PC objects on rank 0: Use -ksp_view ::ascii_info_detail to display information for all blocks KSP Object: (sub_) 1 MPI processes type: preonly maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using NONE norm type for convergence test PC Object: (sub_) 1 MPI processes type: ilu out-of-place factorization 0 levels of fill tolerance for zero pivot 2.22045e-14 matrix ordering: natural factor fill ratio given 1., needed 1. Factored matrix follows: Mat Object: 1 MPI processes type: seqaij rows=6, cols=6 package used to perform factorization: petsc total: nonzeros=16, allocated nonzeros=16 not using I-node routines linear system matrix = precond matrix: Mat Object: (sub_) 1 MPI processes type: seqaij rows=6, cols=6 total: nonzeros=16, allocated nonzeros=30 total number of mallocs used during MatSetValues calls=0 not using I-node routines linear system matrix = precond matrix: Mat Object: 3 MPI processes type: mpiaij rows=18, cols=18 total: nonzeros=72, allocated nonzeros=180 total number of mallocs used during MatSetValues calls=0 not using I-node (on process 0) routines Norm of error 0.00172305, Iterations 6 **************************************** *********************************************************************************************************************** *** WIDEN YOUR WINDOW TO 160 CHARACTERS. Use 'enscript -r -fCourier9' to print this document *** **************************************************************************************************************************************************************** ------------------------------------------------------------------ PETSc Performance Summary: ------------------------------------------------------------------- ./ex5 on a named head1.hpc with 3 processors, by lida Tue May 31 12:18:30 2022 Using Petsc Release Version 3.17.1, unknown Max Max/Min Avg Total Time (sec): 1.783e-02 1.000 1.783e-02 Objects: 8.100e+01 1.000 8.100e+01 Flops: 5.590e+03 1.080 5.314e+03 1.594e+04 Flops/sec: 3.136e+05 1.080 2.981e+05 8.942e+05 MPI Msg Count: 7.800e+01 1.857 5.467e+01 1.640e+02 MPI Msg Len (bytes): 3.808e+03 1.827 4.868e+01 7.984e+03 MPI Reductions: 9.300e+01 1.000 Flop counting convention: 1 flop = 1 real number operation of type (multiply/divide/add/subtract) e.g., VecAXPY() for real vectors of length N --> 2N flops and VecAXPY() for complex vectors of length N --> 8N flops Summary of Stages: ----- Time ------ ----- Flop ------ --- Messages --- -- Message Lengths -- -- Reductions -- Avg %Total Avg %Total Count %Total Avg %Total Count %Total 0: Main Stage: 2.8862e-04 1.6% 0.0000e+00 0.0% 0.000e+00 0.0% 0.000e+00 0.0% 2.000e+00 2.2% 1: Original Solve: 1.3756e-02 77.2% 9.9600e+03 62.5% 9.200e+01 56.1% 4.435e+01 51.1% 4.700e+01 50.5% 2: Second Solve: 3.7698e-03 21.1% 5.9820e+03 37.5% 7.200e+01 43.9% 5.422e+01 48.9% 2.600e+01 28.0% ------------------------------------------------------------------------------------------------------------------------ See the 'Profiling' chapter of the users' manual for details on interpreting output. Phase summary info: Count: number of times phase was executed Time and Flop: Max - maximum over all processors Ratio - ratio of maximum to minimum over all processors Mess: number of messages sent AvgLen: average message length (bytes) Reduct: number of global reductions Global: entire computation Stage: stages of a computation. Set stages with PetscLogStagePush() and PetscLogStagePop(). %T - percent time in this phase %F - percent flop in this phase %M - percent messages in this phase %L - percent message lengths in this phase %R - percent reductions in this phase Total Mflop/s: 10e-6 * (sum of flop over all processors)/(max time over all processors) ------------------------------------------------------------------------------------------------------------------------ Event Count Time (sec) Flop --- Global --- --- Stage ---- Total Max Ratio Max Ratio Max Ratio Mess AvgLen Reduct %T %F %M %L %R %T %F %M %L %R Mflop/s ------------------------------------------------------------------------------------------------------------------------ --- Event Stage 0: Main Stage --- Event Stage 1: Original Solve BuildTwoSided 3 1.0 1.1443e-04 1.5 0.00e+00 0.0 4.0e+00 8.0e+00 3.0e+00 1 0 2 0 3 1 0 4 1 6 0 BuildTwoSidedF 2 1.0 7.2485e-05 1.8 0.00e+00 0.0 0.0e+00 0.0e+00 2.0e+00 0 0 0 0 2 0 0 0 0 4 0 MatMult 20 1.0 5.2744e-0311.5 1.00e+03 1.3 8.8e+01 4.6e+01 1.0e+00 12 16 54 51 1 15 25 96100 2 0 MatSolve 10 1.0 9.1130e-06 1.2 2.60e+02 1.0 0.0e+00 0.0e+00 0.0e+00 0 5 0 0 0 0 8 0 0 0 86 MatLUFactorNum 1 1.0 2.2792e-05 2.1 2.10e+01 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 1 0 0 0 3 MatILUFactorSym 1 1.0 3.5026e-05 1.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatAssemblyBegin 1 1.0 7.2907e-05 1.3 0.00e+00 0.0 0.0e+00 0.0e+00 1.0e+00 0 0 0 0 1 0 0 0 0 2 0 MatAssemblyEnd 1 1.0 5.0211e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 5.0e+00 3 0 0 0 5 4 0 0 0 11 0 MatGetRowIJ 1 1.0 4.2003e-07 3.5 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatGetOrdering 1 1.0 2.0961e-05 1.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatView 3 3.0 5.7218e-04 2.5 0.00e+00 0.0 0.0e+00 0.0e+00 1.0e+00 2 0 0 0 1 3 0 0 0 2 0 VecMDot 9 1.0 4.9467e-0372.0 4.95e+02 1.0 0.0e+00 0.0e+00 9.0e+00 10 9 0 0 10 12 15 0 0 19 0 VecNorm 22 1.0 4.1558e-04 2.1 2.64e+02 1.0 0.0e+00 0.0e+00 2.2e+01 2 5 0 0 24 2 8 0 0 47 2 VecScale 10 1.0 1.0288e-05 1.1 6.00e+01 1.0 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 2 0 0 0 17 VecCopy 11 1.0 8.5477e-06 1.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecSet 21 1.0 1.1027e-05 1.4 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecAXPY 11 1.0 7.3556e-06 1.3 1.32e+02 1.0 0.0e+00 0.0e+00 0.0e+00 0 2 0 0 0 0 4 0 0 0 54 VecAYPX 10 1.0 4.7944e-06 1.1 6.00e+01 1.0 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 2 0 0 0 38 VecMAXPY 19 1.0 8.6166e-06 1.2 1.19e+03 1.0 0.0e+00 0.0e+00 0.0e+00 0 22 0 0 0 0 36 0 0 0 414 VecAssemblyBegin 1 1.0 3.5299e-05 1.4 0.00e+00 0.0 0.0e+00 0.0e+00 1.0e+00 0 0 0 0 1 0 0 0 0 2 0 VecAssemblyEnd 1 1.0 1.8599e-06 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecScatterBegin 20 1.0 2.5115e-04 1.1 0.00e+00 0.0 8.8e+01 4.6e+01 1.0e+00 1 0 54 51 1 2 0 96100 2 0 VecScatterEnd 20 1.0 4.9759e-0325.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 10 0 0 0 0 13 0 0 0 0 0 VecNormalize 10 1.0 1.1400e-04 1.2 1.80e+02 1.0 0.0e+00 0.0e+00 1.0e+01 1 3 0 0 11 1 5 0 0 21 5 SFSetGraph 1 1.0 8.3325e-06 1.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 SFSetUp 1 1.0 1.1758e-04 1.2 0.00e+00 0.0 8.0e+00 2.8e+01 1.0e+00 1 0 5 3 1 1 0 9 5 2 0 SFPack 20 1.0 1.2662e-05 2.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 SFUnpack 20 1.0 4.5374e-06 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 KSPSetUp 2 1.0 2.3160e-04 4.7 0.00e+00 0.0 0.0e+00 0.0e+00 2.0e+00 1 0 0 0 2 1 0 0 0 4 0 KSPSolve 1 1.0 7.0816e-03 1.0 3.38e+03 1.1 7.6e+01 4.8e+01 3.0e+01 39 61 46 46 32 51 97 83 89 64 1 KSPGMRESOrthog 9 1.0 4.9614e-0358.1 1.04e+03 1.0 0.0e+00 0.0e+00 9.0e+00 10 19 0 0 10 12 31 0 0 19 1 PCSetUp 2 1.0 3.2492e-04 1.1 2.10e+01 1.0 0.0e+00 0.0e+00 0.0e+00 2 0 0 0 0 2 1 0 0 0 0 PCSetUpOnBlocks 1 1.0 1.5849e-04 1.0 2.10e+01 1.0 0.0e+00 0.0e+00 0.0e+00 1 0 0 0 0 1 1 0 0 0 0 PCApply 10 1.0 7.6554e-05 1.4 2.60e+02 1.0 0.0e+00 0.0e+00 0.0e+00 0 5 0 0 0 0 8 0 0 0 10 --- Event Stage 2: Second Solve BuildTwoSided 1 1.0 5.5462e-05 2.6 0.00e+00 0.0 6.0e+00 8.0e+00 1.0e+00 0 0 4 1 1 1 0 8 1 4 0 BuildTwoSidedF 1 1.0 6.0628e-05 2.3 0.00e+00 0.0 1.2e+01 1.0e+02 1.0e+00 0 0 7 15 1 1 0 17 31 4 0 MatMult 14 1.0 9.3008e-04 6.9 7.00e+02 1.3 5.6e+01 4.8e+01 0.0e+00 2 11 34 34 0 11 29 78 69 0 2 MatSolve 7 1.0 3.2000e-06 1.2 1.82e+02 1.0 0.0e+00 0.0e+00 0.0e+00 0 3 0 0 0 0 9 0 0 0 171 MatLUFactorNum 1 1.0 2.4736e-06 1.2 2.10e+01 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 1 0 0 0 25 MatAssemblyBegin 1 1.0 8.0139e-05 1.9 0.00e+00 0.0 1.2e+01 1.0e+02 1.0e+00 0 0 7 15 1 2 0 17 31 4 0 MatAssemblyEnd 1 1.0 3.1728e-05 1.1 1.80e+01 1.2 0.0e+00 0.0e+00 2.0e+00 0 0 0 0 2 1 1 0 0 8 2 MatZeroEntries 1 1.0 4.5244e-06 1.4 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatView 3 3.0 4.9627e-04 2.1 0.00e+00 0.0 0.0e+00 0.0e+00 1.0e+00 2 0 0 0 1 9 0 0 0 4 0 VecMDot 6 1.0 8.6141e-0420.9 2.31e+02 1.0 0.0e+00 0.0e+00 6.0e+00 2 4 0 0 6 8 12 0 0 23 1 VecNorm 16 1.0 3.6575e-04 3.1 1.92e+02 1.0 0.0e+00 0.0e+00 1.6e+01 2 4 0 0 17 7 10 0 0 62 2 VecScale 7 1.0 4.1863e-06 2.5 4.20e+01 1.0 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 2 0 0 0 30 VecCopy 8 1.0 5.2368e-06 1.4 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecSet 15 1.0 5.1260e-06 2.3 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecAXPY 8 1.0 4.3251e-06 2.1 9.60e+01 1.0 0.0e+00 0.0e+00 0.0e+00 0 2 0 0 0 0 5 0 0 0 67 VecAYPX 7 1.0 1.7947e-06 1.1 4.20e+01 1.0 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 2 0 0 0 70 VecMAXPY 13 1.0 5.2378e-06 1.5 5.76e+02 1.0 0.0e+00 0.0e+00 0.0e+00 0 11 0 0 0 0 29 0 0 0 330 VecScatterBegin 14 1.0 4.4449e-05 1.6 0.00e+00 0.0 5.6e+01 4.8e+01 0.0e+00 0 0 34 34 0 1 0 78 69 0 0 VecScatterEnd 14 1.0 8.6884e-0411.8 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 2 0 0 0 0 9 0 0 0 0 0 VecNormalize 7 1.0 5.4329e-05 1.2 1.26e+02 1.0 0.0e+00 0.0e+00 7.0e+00 0 2 0 0 8 1 6 0 0 27 7 SFPack 14 1.0 5.7872e-06 1.6 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 SFUnpack 14 1.0 2.3032e-06 1.4 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 KSPSetUp 2 1.0 4.6380e-07 1.4 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 KSPSolve 1 1.0 1.3922e-03 1.1 1.99e+03 1.1 5.2e+01 4.8e+01 2.1e+01 7 35 32 31 23 35 94 72 64 81 4 KSPGMRESOrthog 6 1.0 8.6742e-0417.4 4.83e+02 1.0 0.0e+00 0.0e+00 6.0e+00 2 9 0 0 6 9 24 0 0 23 2 PCSetUp 2 1.0 8.6613e-06 1.2 2.10e+01 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 1 0 0 0 7 PCSetUpOnBlocks 1 1.0 6.2576e-06 1.2 2.10e+01 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 1 0 0 0 10 PCApply 7 1.0 3.7187e-05 1.9 1.82e+02 1.0 0.0e+00 0.0e+00 0.0e+00 0 3 0 0 0 1 9 0 0 0 15 ------------------------------------------------------------------------------------------------------------------------ Memory usage is given in bytes: Object Type Creations Destructions Memory Descendants' Mem. Reports information only for process 0. --- Event Stage 0: Main Stage Viewer 1 1 896 0. --- Event Stage 1: Original Solve Matrix 5 1 3616 0. Vector 43 22 42896 0. Index Set 5 2 1952 0. Star Forest Graph 3 0 0 0. Krylov Solver 2 0 0 0. Preconditioner 2 0 0 0. Viewer 2 1 896 0. Distributed Mesh 1 0 0 0. Discrete System 1 0 0 0. Weak Form 1 0 0 0. --- Event Stage 2: Second Solve Matrix 0 4 16328 0. Vector 14 35 67840 0. Index Set 0 3 3000 0. Star Forest Graph 0 3 3688 0. Krylov Solver 0 2 20942 0. Preconditioner 0 2 2064 0. Viewer 1 1 896 0. Distributed Mesh 0 1 5128 0. Discrete System 0 1 1024 0. Weak Form 0 1 664 0. ======================================================================================================================== Average time to get PetscTime(): 3.89293e-08 Average time for MPI_Barrier(): 4.57484e-06 Average time for zero size MPI_Send(): 1.97627e-06 #PETSc Option Table entries: -ksp_converged_reason -ksp_monitor_true_residual -ksp_view -log_view #End of PETSc Option Table entries Compiled without FORTRAN kernels Compiled with 64 bit PetscInt Compiled with full precision matrices (default) sizeof(short) 2 sizeof(int) 4 sizeof(long) 8 sizeof(void*) 8 sizeof(PetscScalar) 8 sizeof(PetscInt) 8 Configure options: --with-python --prefix=/home/lida -with-mpi-dir=/opt/ohpc/pub/mpi/openmpi3-gnu8/3.1.4 LDFLAGS="-L/home/lida/lib64 -L/home/lida/lib -L/home/lida/jdk/lib" CPPFLAGS="-I/home/lida/include -I/home/lida/jdk/include -march=native -O3" CXXFLAGS="-I/home/lida/include -I/home/lida/jdk/include -march=native -O3" CFLAGS="-I/home/lida/include -I/home/lida/jdk/include -march=native -O3" --with-debugging=no --with-64-bit-indices ----------------------------------------- Libraries compiled on 2022-05-25 08:44:48 on head1.hpc Machine characteristics: Linux-3.10.0-1062.el7.x86_64-x86_64-with-centos-7.7.1908-Core Using PETSc directory: /home/lida Using PETSc arch: ----------------------------------------- Using C compiler: /opt/ohpc/pub/mpi/openmpi3-gnu8/3.1.4/bin/mpicc -I/home/lida/include -I/home/lida/jdk/include -march=native -O3 -fPIC -I/home/lida/include -I/home/lida/jdk/include -march=native -O3 Using Fortran compiler: /opt/ohpc/pub/mpi/openmpi3-gnu8/3.1.4/bin/mpif90 -fPIC -Wall -ffree-line-length-0 -Wno-lto-type-mismatch -Wno-unused-dummy-argument -g -O -I/home/lida/include -I/home/lida/jdk/include -march=native -O3 ----------------------------------------- Using include paths: -I/home/lida/include -I/opt/ohpc/pub/mpi/openmpi3-gnu8/3.1.4/include ----------------------------------------- Using C linker: /opt/ohpc/pub/mpi/openmpi3-gnu8/3.1.4/bin/mpicc Using Fortran linker: /opt/ohpc/pub/mpi/openmpi3-gnu8/3.1.4/bin/mpif90 Using libraries: -Wl,-rpath,/home/lida/lib -L/home/lida/lib -lpetsc -Wl,-rpath,/home/lida/lib64 -L/home/lida/lib64 -Wl,-rpath,/home/lida/lib -L/home/lida/lib -Wl,-rpath,/home/lida/jdk/lib -L/home/lida/jdk/lib -Wl,-rpath,/opt/ohpc/pub/mpi/openmpi3-gnu8/3.1.4/lib -L/opt/ohpc/pub/mpi/openmpi3-gnu8/3.1.4/lib -Wl,-rpath,/opt/ohpc/pub/compiler/gcc/8.3.0/lib/gcc/x86_64-pc-linux-gnu/8.3.0 -L/opt/ohpc/pub/compiler/gcc/8.3.0/lib/gcc/x86_64-pc-linux-gnu/8.3.0 -Wl,-rpath,/opt/ohpc/pub/compiler/gcc/8.3.0/lib64 -L/opt/ohpc/pub/compiler/gcc/8.3.0/lib64 -Wl,-rpath,/home/lida/intel/oneapi/mkl/2022.0.2/lib/intel64 -L/home/lida/intel/oneapi/mkl/2022.0.2/lib/intel64 -Wl,-rpath,/opt/software/intel/compilers_and_libraries_2020.2.254/linux/tbb/lib/intel64/gcc4.8 -L/opt/software/intel/compilers_and_libraries_2020.2.254/linux/tbb/lib/intel64/gcc4.8 -Wl,-rpath,/opt/ohpc/pub/compiler/gcc/8.3.0/lib -L/opt/ohpc/pub/compiler/gcc/8.3.0/lib -lopenblas -lm -lstdc++ -ldl -lmpi_usempif08 -lmpi_usempi_ignore_tkr -lmpi_mpifh -lmpi -lgfortran -lm -lgfortran -lm -lgcc_s -lquadmath -lpthread -lquadmath -lstdc++ -ldl ----------------------------------------- real 0m2.268s user 0m1.456s sys 0m0.583s [lida at head1 tutorials]$ -------------- next part -------------- [lida at head1 tutorials]$ time mpirun -n 1 ./ex5 -ksp_view -ksp_monitor_true_residual -ksp_converged_reason -log_view 2>/dev/null 0 KSP preconditioned resid norm 6.889609116885e+00 true resid norm 1.664331697709e+01 ||r(i)||/||b|| 1.000000000000e+00 1 KSP preconditioned resid norm 3.735626684805e-01 true resid norm 1.277849582311e+00 ||r(i)||/||b|| 7.677854024348e-02 2 KSP preconditioned resid norm 1.736201489518e-02 true resid norm 6.598680019925e-02 ||r(i)||/||b|| 3.964762570470e-03 3 KSP preconditioned resid norm 6.503363094580e-04 true resid norm 3.819222199389e-03 ||r(i)||/||b|| 2.294748219147e-04 4 KSP preconditioned resid norm 7.801321330590e-06 true resid norm 3.420262185151e-05 ||r(i)||/||b|| 2.055036378781e-06 Linear solve converged due to CONVERGED_RTOL iterations 4 KSP Object: 1 MPI processes type: gmres restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement happy breakdown tolerance 1e-30 maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using PRECONDITIONED norm type for convergence test PC Object: 1 MPI processes type: icc out-of-place factorization 0 levels of fill tolerance for zero pivot 2.22045e-14 using Manteuffel shift [POSITIVE_DEFINITE] matrix ordering: natural factor fill ratio given 1., needed 1. Factored matrix follows: Mat Object: 1 MPI processes type: seqsbaij rows=6, cols=6 package used to perform factorization: petsc total: nonzeros=13, allocated nonzeros=13 block size is 1 linear system matrix = precond matrix: Mat Object: 1 MPI processes type: seqaij rows=6, cols=6 total: nonzeros=20, allocated nonzeros=30 total number of mallocs used during MatSetValues calls=0 not using I-node routines Norm of error 7.36852e-06, Iterations 4 0 KSP preconditioned resid norm 7.247863732790e+00 true resid norm 3.100000000000e+01 ||r(i)||/||b|| 1.000000000000e+00 1 KSP preconditioned resid norm 1.273569592250e-01 true resid norm 7.519049129476e-01 ||r(i)||/||b|| 2.425499719186e-02 2 KSP preconditioned resid norm 2.160197987659e-03 true resid norm 1.245148592634e-02 ||r(i)||/||b|| 4.016608363334e-04 3 KSP preconditioned resid norm 3.358674359432e-05 true resid norm 2.551377456617e-04 ||r(i)||/||b|| 8.230249860056e-06 Linear solve converged due to CONVERGED_RTOL iterations 3 KSP Object: 1 MPI processes type: gmres restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement happy breakdown tolerance 1e-30 maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using PRECONDITIONED norm type for convergence test PC Object: 1 MPI processes type: icc out-of-place factorization 0 levels of fill tolerance for zero pivot 2.22045e-14 using Manteuffel shift [POSITIVE_DEFINITE] matrix ordering: natural factor fill ratio given 1., needed 1. Factored matrix follows: Mat Object: 1 MPI processes type: seqsbaij rows=6, cols=6 package used to perform factorization: petsc total: nonzeros=13, allocated nonzeros=13 block size is 1 linear system matrix = precond matrix: Mat Object: 1 MPI processes type: seqaij rows=6, cols=6 total: nonzeros=20, allocated nonzeros=30 total number of mallocs used during MatSetValues calls=0 not using I-node routines Norm of error 3.41929e-05, Iterations 3 **************************************** *********************************************************************************************************************** *** WIDEN YOUR WINDOW TO 160 CHARACTERS. Use 'enscript -r -fCourier9' to print this document *** **************************************************************************************************************************************************************** ------------------------------------------------------------------ PETSc Performance Summary: ------------------------------------------------------------------- ./ex5 on a named head1.hpc with 1 processor, by lida Tue May 31 12:18:46 2022 Using Petsc Release Version 3.17.1, unknown Max Max/Min Avg Total Time (sec): 1.397e-02 1.000 1.397e-02 Objects: 5.200e+01 1.000 5.200e+01 Flops: 2.076e+03 1.000 2.076e+03 2.076e+03 Flops/sec: 1.486e+05 1.000 1.486e+05 1.486e+05 MPI Msg Count: 0.000e+00 0.000 0.000e+00 0.000e+00 MPI Msg Len (bytes): 0.000e+00 0.000 0.000e+00 0.000e+00 MPI Reductions: 0.000e+00 0.000 Flop counting convention: 1 flop = 1 real number operation of type (multiply/divide/add/subtract) e.g., VecAXPY() for real vectors of length N --> 2N flops and VecAXPY() for complex vectors of length N --> 8N flops Summary of Stages: ----- Time ------ ----- Flop ------ --- Messages --- -- Message Lengths -- -- Reductions -- Avg %Total Avg %Total Count %Total Avg %Total Count %Total 0: Main Stage: 2.2292e-04 1.6% 0.0000e+00 0.0% 0.000e+00 0.0% 0.000e+00 0.0% 0.000e+00 0.0% 1: Original Solve: 9.2670e-03 66.3% 1.1780e+03 56.7% 0.000e+00 0.0% 0.000e+00 0.0% 0.000e+00 0.0% 2: Second Solve: 4.4709e-03 32.0% 8.9800e+02 43.3% 0.000e+00 0.0% 0.000e+00 0.0% 0.000e+00 0.0% ------------------------------------------------------------------------------------------------------------------------ See the 'Profiling' chapter of the users' manual for details on interpreting output. Phase summary info: Count: number of times phase was executed Time and Flop: Max - maximum over all processors Ratio - ratio of maximum to minimum over all processors Mess: number of messages sent AvgLen: average message length (bytes) Reduct: number of global reductions Global: entire computation Stage: stages of a computation. Set stages with PetscLogStagePush() and PetscLogStagePop(). %T - percent time in this phase %F - percent flop in this phase %M - percent messages in this phase %L - percent message lengths in this phase %R - percent reductions in this phase Total Mflop/s: 10e-6 * (sum of flop over all processors)/(max time over all processors) ------------------------------------------------------------------------------------------------------------------------ Event Count Time (sec) Flop --- Global --- --- Stage ---- Total Max Ratio Max Ratio Max Ratio Mess AvgLen Reduct %T %F %M %L %R %T %F %M %L %R Mflop/s ------------------------------------------------------------------------------------------------------------------------ --- Event Stage 0: Main Stage --- Event Stage 1: Original Solve MatMult 10 1.0 1.5418e-05 1.0 3.40e+02 1.0 0.0e+00 0.0e+00 0.0e+00 0 16 0 0 0 0 29 0 0 0 22 MatSolve 5 1.0 9.0795e-06 1.0 1.70e+02 1.0 0.0e+00 0.0e+00 0.0e+00 0 8 0 0 0 0 14 0 0 0 19 MatCholFctrNum 1 1.0 1.2116e-05 1.0 6.00e+00 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 1 0 0 0 0 MatICCFactorSym 1 1.0 1.7540e-05 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatAssemblyBegin 1 1.0 2.3469e-07 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatAssemblyEnd 1 1.0 2.3169e-05 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatGetRowIJ 1 1.0 1.6764e-07 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatGetOrdering 1 1.0 1.0921e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 1 0 0 0 0 1 0 0 0 0 0 MatView 2 1.0 2.4787e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 18 0 0 0 0 27 0 0 0 0 0 VecMDot 4 1.0 6.2725e-06 1.0 1.10e+02 1.0 0.0e+00 0.0e+00 0.0e+00 0 5 0 0 0 0 9 0 0 0 18 VecNorm 12 1.0 1.2450e-05 1.0 1.32e+02 1.0 0.0e+00 0.0e+00 0.0e+00 0 6 0 0 0 0 11 0 0 0 11 VecScale 5 1.0 8.2748e-06 1.0 3.00e+01 1.0 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 3 0 0 0 4 VecCopy 6 1.0 4.5085e-06 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecSet 6 1.0 6.6161e-06 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecAXPY 6 1.0 4.7898e-06 1.0 7.20e+01 1.0 0.0e+00 0.0e+00 0.0e+00 0 3 0 0 0 0 6 0 0 0 15 VecAYPX 5 1.0 3.7868e-06 1.0 3.00e+01 1.0 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 3 0 0 0 8 VecMAXPY 9 1.0 3.1283e-06 1.0 2.88e+02 1.0 0.0e+00 0.0e+00 0.0e+00 0 14 0 0 0 0 24 0 0 0 92 VecAssemblyBegin 1 1.0 1.3318e-07 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecAssemblyEnd 1 1.0 1.5367e-07 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecNormalize 5 1.0 2.7158e-05 1.0 8.50e+01 1.0 0.0e+00 0.0e+00 0.0e+00 0 4 0 0 0 0 7 0 0 0 3 KSPSetUp 1 1.0 4.6344e-05 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 1 0 0 0 0 0 KSPSolve 1 1.0 2.0673e-03 1.0 1.12e+03 1.0 0.0e+00 0.0e+00 0.0e+00 15 54 0 0 0 22 95 0 0 0 1 KSPGMRESOrthog 4 1.0 1.6174e-05 1.0 2.30e+02 1.0 0.0e+00 0.0e+00 0.0e+00 0 11 0 0 0 0 20 0 0 0 14 PCSetUp 1 1.0 2.1121e-04 1.0 6.00e+00 1.0 0.0e+00 0.0e+00 0.0e+00 2 0 0 0 0 2 1 0 0 0 0 PCApply 5 1.0 1.4246e-05 1.0 1.70e+02 1.0 0.0e+00 0.0e+00 0.0e+00 0 8 0 0 0 0 14 0 0 0 12 --- Event Stage 2: Second Solve MatMult 8 1.0 3.8277e-06 1.0 2.72e+02 1.0 0.0e+00 0.0e+00 0.0e+00 0 13 0 0 0 0 30 0 0 0 71 MatSolve 4 1.0 1.9213e-06 1.0 1.36e+02 1.0 0.0e+00 0.0e+00 0.0e+00 0 7 0 0 0 0 15 0 0 0 71 MatCholFctrNum 1 1.0 3.3919e-06 1.0 6.00e+00 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 1 0 0 0 2 MatAssemblyBegin 1 1.0 1.1362e-07 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatAssemblyEnd 1 1.0 4.7777e-07 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatZeroEntries 1 1.0 4.4834e-06 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatView 2 1.0 2.4329e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 17 0 0 0 0 54 0 0 0 0 0 VecMDot 3 1.0 1.1474e-06 1.0 6.60e+01 1.0 0.0e+00 0.0e+00 0.0e+00 0 3 0 0 0 0 7 0 0 0 58 VecNorm 10 1.0 2.7772e-06 1.0 1.10e+02 1.0 0.0e+00 0.0e+00 0.0e+00 0 5 0 0 0 0 12 0 0 0 40 VecScale 4 1.0 1.5302e-06 1.0 2.40e+01 1.0 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 3 0 0 0 16 VecCopy 5 1.0 3.1414e-06 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecSet 5 1.0 1.7434e-06 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecAXPY 5 1.0 2.1365e-06 1.0 6.00e+01 1.0 0.0e+00 0.0e+00 0.0e+00 0 3 0 0 0 0 7 0 0 0 28 VecAYPX 4 1.0 9.7509e-07 1.0 2.40e+01 1.0 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 3 0 0 0 25 VecMAXPY 7 1.0 2.1346e-06 1.0 1.80e+02 1.0 0.0e+00 0.0e+00 0.0e+00 0 9 0 0 0 0 20 0 0 0 84 VecNormalize 4 1.0 5.5209e-06 1.0 6.80e+01 1.0 0.0e+00 0.0e+00 0.0e+00 0 3 0 0 0 0 8 0 0 0 12 KSPSetUp 1 1.0 4.0047e-07 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 KSPSolve 1 1.0 3.7580e-04 1.0 8.15e+02 1.0 0.0e+00 0.0e+00 0.0e+00 3 39 0 0 0 8 91 0 0 0 2 KSPGMRESOrthog 3 1.0 4.7795e-06 1.0 1.38e+02 1.0 0.0e+00 0.0e+00 0.0e+00 0 7 0 0 0 0 15 0 0 0 29 PCSetUp 1 1.0 6.9309e-06 1.0 6.00e+00 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 1 0 0 0 1 PCApply 4 1.0 3.5232e-06 1.0 1.36e+02 1.0 0.0e+00 0.0e+00 0.0e+00 0 7 0 0 0 0 15 0 0 0 39 ------------------------------------------------------------------------------------------------------------------------ Memory usage is given in bytes: Object Type Creations Destructions Memory Descendants' Mem. Reports information only for process 0. --- Event Stage 0: Main Stage Viewer 1 1 896 0. --- Event Stage 1: Original Solve Matrix 3 1 3672 0. Vector 29 11 20064 0. Krylov Solver 1 0 0 0. Preconditioner 1 0 0 0. Viewer 1 0 0 0. Index Set 3 1 1000 0. Distributed Mesh 1 0 0 0. Star Forest Graph 2 0 0 0. Discrete System 1 0 0 0. Weak Form 1 0 0 0. --- Event Stage 2: Second Solve Matrix 0 2 8224 0. Vector 8 26 47424 0. Krylov Solver 0 1 19198 0. Preconditioner 0 1 1048 0. Index Set 0 2 2000 0. Distributed Mesh 0 1 5128 0. Star Forest Graph 0 2 2352 0. Discrete System 0 1 1024 0. Weak Form 0 1 664 0. ======================================================================================================================== Average time to get PetscTime(): 3.68804e-08 #PETSc Option Table entries: -ksp_converged_reason -ksp_monitor_true_residual -ksp_view -log_view #End of PETSc Option Table entries Compiled without FORTRAN kernels Compiled with 64 bit PetscInt Compiled with full precision matrices (default) sizeof(short) 2 sizeof(int) 4 sizeof(long) 8 sizeof(void*) 8 sizeof(PetscScalar) 8 sizeof(PetscInt) 8 Configure options: --with-python --prefix=/home/lida -with-mpi-dir=/opt/ohpc/pub/mpi/openmpi3-gnu8/3.1.4 LDFLAGS="-L/home/lida/lib64 -L/home/lida/lib -L/home/lida/jdk/lib" CPPFLAGS="-I/home/lida/include -I/home/lida/jdk/include -march=native -O3" CXXFLAGS="-I/home/lida/include -I/home/lida/jdk/include -march=native -O3" CFLAGS="-I/home/lida/include -I/home/lida/jdk/include -march=native -O3" --with-debugging=no --with-64-bit-indices ----------------------------------------- Libraries compiled on 2022-05-25 08:44:48 on head1.hpc Machine characteristics: Linux-3.10.0-1062.el7.x86_64-x86_64-with-centos-7.7.1908-Core Using PETSc directory: /home/lida Using PETSc arch: ----------------------------------------- Using C compiler: /opt/ohpc/pub/mpi/openmpi3-gnu8/3.1.4/bin/mpicc -I/home/lida/include -I/home/lida/jdk/include -march=native -O3 -fPIC -I/home/lida/include -I/home/lida/jdk/include -march=native -O3 Using Fortran compiler: /opt/ohpc/pub/mpi/openmpi3-gnu8/3.1.4/bin/mpif90 -fPIC -Wall -ffree-line-length-0 -Wno-lto-type-mismatch -Wno-unused-dummy-argument -g -O -I/home/lida/include -I/home/lida/jdk/include -march=native -O3 ----------------------------------------- Using include paths: -I/home/lida/include -I/opt/ohpc/pub/mpi/openmpi3-gnu8/3.1.4/include ----------------------------------------- Using C linker: /opt/ohpc/pub/mpi/openmpi3-gnu8/3.1.4/bin/mpicc Using Fortran linker: /opt/ohpc/pub/mpi/openmpi3-gnu8/3.1.4/bin/mpif90 Using libraries: -Wl,-rpath,/home/lida/lib -L/home/lida/lib -lpetsc -Wl,-rpath,/home/lida/lib64 -L/home/lida/lib64 -Wl,-rpath,/home/lida/lib -L/home/lida/lib -Wl,-rpath,/home/lida/jdk/lib -L/home/lida/jdk/lib -Wl,-rpath,/opt/ohpc/pub/mpi/openmpi3-gnu8/3.1.4/lib -L/opt/ohpc/pub/mpi/openmpi3-gnu8/3.1.4/lib -Wl,-rpath,/opt/ohpc/pub/compiler/gcc/8.3.0/lib/gcc/x86_64-pc-linux-gnu/8.3.0 -L/opt/ohpc/pub/compiler/gcc/8.3.0/lib/gcc/x86_64-pc-linux-gnu/8.3.0 -Wl,-rpath,/opt/ohpc/pub/compiler/gcc/8.3.0/lib64 -L/opt/ohpc/pub/compiler/gcc/8.3.0/lib64 -Wl,-rpath,/home/lida/intel/oneapi/mkl/2022.0.2/lib/intel64 -L/home/lida/intel/oneapi/mkl/2022.0.2/lib/intel64 -Wl,-rpath,/opt/software/intel/compilers_and_libraries_2020.2.254/linux/tbb/lib/intel64/gcc4.8 -L/opt/software/intel/compilers_and_libraries_2020.2.254/linux/tbb/lib/intel64/gcc4.8 -Wl,-rpath,/opt/ohpc/pub/compiler/gcc/8.3.0/lib -L/opt/ohpc/pub/compiler/gcc/8.3.0/lib -lopenblas -lm -lstdc++ -ldl -lmpi_usempif08 -lmpi_usempi_ignore_tkr -lmpi_mpifh -lmpi -lgfortran -lm -lgfortran -lm -lgcc_s -lquadmath -lpthread -lquadmath -lstdc++ -ldl ----------------------------------------- real 0m1.711s user 0m0.368s sys 0m0.259s [lida at head1 tutorials]$ From knepley at gmail.com Tue May 31 07:19:45 2022 From: knepley at gmail.com (Matthew Knepley) Date: Tue, 31 May 2022 08:19:45 -0400 Subject: [petsc-users] Sparse linear system solving In-Reply-To: <2e7ebbf1-511a-7055-ff92-131d3bf73f1e@mail.ioffe.ru> References: <026d55af-c978-81da-f571-46519e5e6f8e@mail.ioffe.ru> <2e7ebbf1-511a-7055-ff92-131d3bf73f1e@mail.ioffe.ru> Message-ID: On Tue, May 31, 2022 at 7:39 AM Lidia wrote: > Matt, Mark, thank you much for your answers! > > > Now we have run example # 5 on our computer cluster and on the local > server and also have not seen any performance increase, but by unclear > reason running times on the local server are much better than on the > cluster. > I suspect that you are trying to get speedup without increasing the memory bandwidth: https://petsc.org/main/faq/#what-kind-of-parallel-computers-or-clusters-are-needed-to-use-petsc-or-why-do-i-get-little-speedup Thanks, Matt > Now we will try to run petsc #5 example inside a docker container on our > server and see if the problem is in our environment. I'll write you the > results of this test as soon as we get it. > > The ksp_monitor outs for the 5th test at the current local server > configuration (for 2 and 4 mpi processes) and for the cluster (for 1 and 3 > mpi processes) are attached . > > > And one more question. Potentially we can use 10 nodes and 96 threads at > each node on our cluster. What do you think, which combination of numbers > of mpi processes and openmp threads may be the best for the 5th example? > > Thank you! > > > Best, > Lidiia > > On 31.05.2022 05:42, Mark Adams wrote: > > And if you see "NO" change in performance I suspect the solver/matrix is > all on one processor. > (PETSc does not use threads by default so threads should not change > anything). > > As Matt said, it is best to start with a PETSc example that does something > like what you want (parallel linear solve, see src/ksp/ksp/tutorials for > examples), and then add your code to it. > That way you get the basic infrastructure in place for you, which is > pretty obscure to the uninitiated. > > Mark > > On Mon, May 30, 2022 at 10:18 PM Matthew Knepley > wrote: > >> On Mon, May 30, 2022 at 10:12 PM Lidia wrote: >> >>> Dear colleagues, >>> >>> Is here anyone who have solved big sparse linear matrices using PETSC? >>> >> >> There are lots of publications with this kind of data. Here is one recent >> one: https://arxiv.org/abs/2204.01722 >> >> >>> We have found NO performance improvement while using more and more mpi >>> processes (1-2-3) and open-mp threads (from 1 to 72 threads). Did anyone >>> faced to this problem? Does anyone know any possible reasons of such >>> behaviour? >>> >> >> Solver behavior is dependent on the input matrix. The only >> general-purpose solvers >> are direct, but they do not scale linearly and have high memory >> requirements. >> >> Thus, in order to make progress you will have to be specific about your >> matrices. >> >> >>> We use AMG preconditioner and GMRES solver from KSP package, as our >>> matrix is large (from 100 000 to 1e+6 rows and columns), sparse, >>> non-symmetric and includes both positive and negative values. But >>> performance problems also exist while using CG solvers with symmetric >>> matrices. >>> >> >> There are many PETSc examples, such as example 5 for the Laplacian, that >> exhibit >> good scaling with both AMG and GMG. >> >> >>> Could anyone help us to set appropriate options of the preconditioner >>> and solver? Now we use default parameters, maybe they are not the best, >>> but we do not know a good combination. Or maybe you could suggest any >>> other pairs of preconditioner+solver for such tasks? >>> >>> I can provide more information: the matrices that we solve, c++ script >>> to run solving using petsc and any statistics obtained by our runs. >>> >> >> First, please provide a description of the linear system, and the output >> of >> >> -ksp_view -ksp_monitor_true_residual -ksp_converged_reason -log_view >> >> for each test case. >> >> Thanks, >> >> Matt >> >> >>> Thank you in advance! >>> >>> Best regards, >>> Lidiia Varshavchik, >>> Ioffe Institute, St. Petersburg, Russia >>> >> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which their >> experiments lead. >> -- Norbert Wiener >> >> https://www.cse.buffalo.edu/~knepley/ >> >> > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Tue May 31 07:21:47 2022 From: knepley at gmail.com (Matthew Knepley) Date: Tue, 31 May 2022 08:21:47 -0400 Subject: [petsc-users] Sparse linear system solving In-Reply-To: References: <026d55af-c978-81da-f571-46519e5e6f8e@mail.ioffe.ru> <2e7ebbf1-511a-7055-ff92-131d3bf73f1e@mail.ioffe.ru> Message-ID: I have looked at the local logs. First, you have run problems of size 12 and 24. As a rule of thumb, you need 10,000 variables per process in order to see good speedup. Thanks, Matt On Tue, May 31, 2022 at 8:19 AM Matthew Knepley wrote: > On Tue, May 31, 2022 at 7:39 AM Lidia wrote: > >> Matt, Mark, thank you much for your answers! >> >> >> Now we have run example # 5 on our computer cluster and on the local >> server and also have not seen any performance increase, but by unclear >> reason running times on the local server are much better than on the >> cluster. >> > I suspect that you are trying to get speedup without increasing the memory > bandwidth: > > > https://petsc.org/main/faq/#what-kind-of-parallel-computers-or-clusters-are-needed-to-use-petsc-or-why-do-i-get-little-speedup > > Thanks, > > Matt > >> Now we will try to run petsc #5 example inside a docker container on our >> server and see if the problem is in our environment. I'll write you the >> results of this test as soon as we get it. >> >> The ksp_monitor outs for the 5th test at the current local server >> configuration (for 2 and 4 mpi processes) and for the cluster (for 1 and 3 >> mpi processes) are attached . >> >> >> And one more question. Potentially we can use 10 nodes and 96 threads at >> each node on our cluster. What do you think, which combination of numbers >> of mpi processes and openmp threads may be the best for the 5th example? >> >> Thank you! >> >> >> Best, >> Lidiia >> >> On 31.05.2022 05:42, Mark Adams wrote: >> >> And if you see "NO" change in performance I suspect the solver/matrix is >> all on one processor. >> (PETSc does not use threads by default so threads should not change >> anything). >> >> As Matt said, it is best to start with a PETSc example that does >> something like what you want (parallel linear solve, see >> src/ksp/ksp/tutorials for examples), and then add your code to it. >> That way you get the basic infrastructure in place for you, which is >> pretty obscure to the uninitiated. >> >> Mark >> >> On Mon, May 30, 2022 at 10:18 PM Matthew Knepley >> wrote: >> >>> On Mon, May 30, 2022 at 10:12 PM Lidia >>> wrote: >>> >>>> Dear colleagues, >>>> >>>> Is here anyone who have solved big sparse linear matrices using PETSC? >>>> >>> >>> There are lots of publications with this kind of data. Here is one >>> recent one: https://arxiv.org/abs/2204.01722 >>> >>> >>>> We have found NO performance improvement while using more and more mpi >>>> processes (1-2-3) and open-mp threads (from 1 to 72 threads). Did >>>> anyone >>>> faced to this problem? Does anyone know any possible reasons of such >>>> behaviour? >>>> >>> >>> Solver behavior is dependent on the input matrix. The only >>> general-purpose solvers >>> are direct, but they do not scale linearly and have high memory >>> requirements. >>> >>> Thus, in order to make progress you will have to be specific about your >>> matrices. >>> >>> >>>> We use AMG preconditioner and GMRES solver from KSP package, as our >>>> matrix is large (from 100 000 to 1e+6 rows and columns), sparse, >>>> non-symmetric and includes both positive and negative values. But >>>> performance problems also exist while using CG solvers with symmetric >>>> matrices. >>>> >>> >>> There are many PETSc examples, such as example 5 for the Laplacian, that >>> exhibit >>> good scaling with both AMG and GMG. >>> >>> >>>> Could anyone help us to set appropriate options of the preconditioner >>>> and solver? Now we use default parameters, maybe they are not the best, >>>> but we do not know a good combination. Or maybe you could suggest any >>>> other pairs of preconditioner+solver for such tasks? >>>> >>>> I can provide more information: the matrices that we solve, c++ script >>>> to run solving using petsc and any statistics obtained by our runs. >>>> >>> >>> First, please provide a description of the linear system, and the output >>> of >>> >>> -ksp_view -ksp_monitor_true_residual -ksp_converged_reason -log_view >>> >>> for each test case. >>> >>> Thanks, >>> >>> Matt >>> >>> >>>> Thank you in advance! >>>> >>>> Best regards, >>>> Lidiia Varshavchik, >>>> Ioffe Institute, St. Petersburg, Russia >>>> >>> >>> >>> -- >>> What most experimenters take for granted before they begin their >>> experiments is infinitely more interesting than any results to which their >>> experiments lead. >>> -- Norbert Wiener >>> >>> https://www.cse.buffalo.edu/~knepley/ >>> >>> >> > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > > https://www.cse.buffalo.edu/~knepley/ > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Tue May 31 07:43:21 2022 From: knepley at gmail.com (Matthew Knepley) Date: Tue, 31 May 2022 08:43:21 -0400 Subject: [petsc-users] How to read and write HDF5 in parallel withPETSc In-Reply-To: References: Message-ID: On Tue, May 31, 2022 at 1:02 AM ??? <12132391 at mail.sustech.edu.cn> wrote: > my code is below: > ierr = PetscViewerCreate(PETSC_COMM_WORLD,&h5);CHKERRQ(ierr); > ierr = PetscViewerHDF5Open(PETSC_COMM_WORLD,"explicit.h5", > FILE_MODE_WRITE, &h5);CHKERRQ(ierr); > ierr = PetscObjectSetName((PetscObject) z, > "explicit-vector");CHKERRQ(ierr); > ierr = PetscObjectSetName((PetscObject) tem, > "explicit-necess-data");CHKERRQ(ierr); > ierr = VecView(tem, h5);CHKERRQ(ierr); > ierr = VecView(z, h5);CHKERRQ(ierr); > ierr = PetscViewerDestroy(&h5);CHKERRQ(ierr); > > when I use 1 core run this, it can save the right answer. > > when I use 3 cores run this, it prints that : > [0]PETSC ERROR: No support for this operation for this object type > [0]PETSC ERROR: Cannot use parallel HDF5 viewer since the given HDF5 does > not support parallel I/O (H5_HAVE_PARALLEL is unset) > If you want parallel HDF5, the package has to be compiled for parallelism. One way to do this is to have PETSc build it for you: --download-hdf5 in configure. Thanks, Matt > I don't know how to solve this problem. > > By the way, I use PETSc-3.16.6, HDF5-1.12.1, C for programming. > > > > > > ??? > > ??????/???/???2021? > > ?????????????1088? > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Tue May 31 09:06:01 2022 From: knepley at gmail.com (Matthew Knepley) Date: Tue, 31 May 2022 10:06:01 -0400 Subject: [petsc-users] PetscSF Object on Distributed DMPlex for Halo Data Exchange In-Reply-To: References: Message-ID: On Tue, May 31, 2022 at 10:04 AM Mike Michell wrote: > As a follow-up question on your example, is it possible to call > PetscSFCreateRemoteOffsets() from Fortran? > > My code is written in .F90 and in "petsc/finclude/" there is no petscsf.h > so that the code currently cannot find PetscSFCreateRemoteOffsets(). > I believe if you pass in NULL for remoteOffsets, that function will be called internally. Thanks, Matt > Thanks, > Mike > > 2022? 5? 24? (?) ?? 8:46, Matthew Knepley ?? ??: > >> I will also point out that Toby has created a nice example showing how to >> create an SF for halo exchange between local vectors. >> >> https://gitlab.com/petsc/petsc/-/merge_requests/5267 >> >> Thanks, >> >> Matt >> >> On Sun, May 22, 2022 at 9:47 PM Matthew Knepley >> wrote: >> >>> On Sun, May 22, 2022 at 4:28 PM Mike Michell >>> wrote: >>> >>>> Thanks for the reply. The diagram makes sense and is helpful for >>>> understanding 1D representation. >>>> >>>> However, something is still unclear. From your diagram, the number of >>>> roots per process seems to vary according to run arguments, such as >>>> "-dm_distribute_overlap", because "the number of roots for a DMPlex is the >>>> number of mesh points in the local portion of the mesh (cited from your >>>> answer to my question (1))" will end up change according to that argument. >>>> However, from my mock-up code, number of roots is independent to >>>> -dm_distribute_overlap argument. The summation of "number of roots" through >>>> processes was always equal to number of physical vertex on my mesh, if I >>>> define the section layout on vertex with 1DOF. But in your diagram example, >>>> the summation of "nroots" is larger than the actual number of mesh points, >>>> which is 13. >>>> >>> >>> I do not understand your question. Notice the -dm_distribute_overlap >>> does _not_ change the owned points for any process. It only puts in new >>> leaves, so it also never >>> changes the roots for this way of using the SF. >>> >>> >>>> Also, it is still unclear how to get the size of "roots" from the >>>> PetscSection & PetscSF on distributed DMPlex? >>>> >>> >>> For an SF mapping ghost dofs in a global vector, the number of roots is >>> just the size of the local portion of the vector. >>> >>> >>>> In your diagram, how can you tell your code and make it allocate the >>>> "nroots=7 for P0, nroots=9 for P1, and nroots=7 for P2" arrays before you >>>> call PetscSFBcastBegin/End()? It seems that we need to define arrays having >>>> the size of nroots & nleaves before calling PetscSFBcastBegin/End(). >>>> >>> >>> I just want to note that this usage is different from the canonical >>> usage in Plex. It is fine to do this, but this will not match what I do in >>> the library if you look. >>> In Plex, I distinguish two linear spaces: >>> >>> 1) Global space: This is the vector space for the solvers. Each point >>> is uniquely represented and owned by some process >>> >>> 2) Local space: This is the vector space for assembly. Some points are >>> represented multiple times. >>> >>> I create an SF that maps from the global space (roots) to the local >>> space (leaves), and it is called in DMGlobalToLocal() (and >>> associated functions). This >>> is more natural in FEM. You seem to want an SF that maps between global >>> vectors. This will also work. The roots would be the local dofs, and the >>> leaves >>> would be shared dofs. >>> >>> Does this make sense? >>> >>> Thanks, >>> >>> Matt >>> >>> >>>> Thanks, >>>> Mike >>>> >>>> Here's a diagram of a 1D mesh with overlap and 3 partitions, showing >>>>> what the petscsf data is for each. The number of roots is the number of >>>>> mesh points in the local representation, and the number of leaves is the >>>>> number of mesh points that are duplicates of mesh points on other >>>>> processes. With that in mind, answering your questions >>>>> >>>>> > (1) It seems that the "roots" means the number of vertex not >>>>> considering overlap layer, and "leaves" seems the number of distributed >>>>> vertex for each processor that includes overlap layer. Can you acknowledge >>>>> that this is correct understanding? I have tried to find clearer examples >>>>> from PETSc team's articles relevant to Star Forest, but I am still unclear >>>>> about the exact relation & graphical notation of roots & leaves in SF if >>>>> it's the case of DMPlex solution arrays. >>>>> >>>>> No, the number of roots for a DMPlex is the number of mesh points in >>>>> the local portion of the mesh >>>>> >>>>> > (2) If it is so, there is an issue that I cannot define "root data" >>>>> and "leave data" generally. I am trying to following >>>>> "src/vec/is/sf/tutorials/ex1f.F90", however, in that example, size of roots >>>>> and leaves are predefined as 6. How can I generalize that? Because I can >>>>> get size of leaves using DAG depth(or height), which is equal to number of >>>>> vertices each proc has. But, how can I get the size of my "roots" region >>>>> from SF? Any example about that? This question is connected to how can I >>>>> define "rootdata" for "PetscSFBcastBegin/End()". >>>>> >>>>> Does the diagram help you generalize? >>>>> >>>>> > (3) More importantly, with the attached PetscSection & SF layout, my >>>>> vector is only resolved for the size equal to "number of roots" for each >>>>> proc, but not for the overlapping area(i.e., "leaves"). What I wish to do >>>>> is to exchange (or reduce) the solution data between each proc, in the >>>>> overlapping region. Can I get some advices why my vector does not encompass >>>>> the "leaves" regime? Is there any example doing similar things? >>>>> Going back to my first response: if you use a section to say how many >>>>> pieces of data are associated with each local mesh point, then a PetscSF is >>>>> constructed that requires no more manipulation from you. >>>>> >>>>> >>>>> On Sun, May 22, 2022 at 10:47 AM Mike Michell >>>>> wrote: >>>>> >>>>>> Thank you for the reply. >>>>>> The PetscSection and PetscSF objects are defined as in the attached >>>>>> mock-up code (Q_PetscSF_1.tar). 1-DOF is defined on vertex as my solution >>>>>> is determined on each vertex with 1-DOF from a finite-volume method. >>>>>> >>>>>> As follow up questions: >>>>>> (1) It seems that the "roots" means the number of vertex not >>>>>> considering overlap layer, and "leaves" seems the number of distributed >>>>>> vertex for each processor that includes overlap layer. Can you acknowledge >>>>>> that this is correct understanding? I have tried to find clearer examples >>>>>> from PETSc team's articles relevant to Star Forest, but I am still unclear >>>>>> about the exact relation & graphical notation of roots & leaves in SF if >>>>>> it's the case of DMPlex solution arrays. >>>>>> >>>>>> (2) If it is so, there is an issue that I cannot define "root data" >>>>>> and "leave data" generally. I am trying to following >>>>>> "src/vec/is/sf/tutorials/ex1f.F90", however, in that example, size of roots >>>>>> and leaves are predefined as 6. How can I generalize that? Because I can >>>>>> get size of leaves using DAG depth(or height), which is equal to number of >>>>>> vertices each proc has. But, how can I get the size of my "roots" region >>>>>> from SF? Any example about that? This question is connected to how can I >>>>>> define "rootdata" for "PetscSFBcastBegin/End()". >>>>>> >>>>>> (3) More importantly, with the attached PetscSection & SF layout, my >>>>>> vector is only resolved for the size equal to "number of roots" for each >>>>>> proc, but not for the overlapping area(i.e., "leaves"). What I wish to do >>>>>> is to exchange (or reduce) the solution data between each proc, in the >>>>>> overlapping region. Can I get some advices why my vector does not encompass >>>>>> the "leaves" regime? Is there any example doing similar things? >>>>>> >>>>>> Thanks, >>>>>> Mike >>>>>> >>>>>> >>>>>>> On Fri, May 20, 2022 at 4:45 PM Mike Michell >>>>>>> wrote: >>>>>>> >>>>>>>> Thanks for the reply. >>>>>>>> >>>>>>>> > "What I want to do is to exchange data (probably just >>>>>>>> MPI_Reduce)" which confuses me, because halo exchange is a point-to-point >>>>>>>> exchange and not a reduction. Can you clarify? >>>>>>>> PetscSFReduceBegin/End seems to be the function that do reduction >>>>>>>> for PetscSF object. What I intended to mention was either reduction or >>>>>>>> exchange, not specifically intended "reduction". >>>>>>>> >>>>>>>> As a follow-up question: >>>>>>>> Assuming that the code has its own local solution arrays (not Petsc >>>>>>>> type), and if the plex's DAG indices belong to the halo region are the only >>>>>>>> information that I want to know (not the detailed section description, such >>>>>>>> as degree of freedom on vertex, cells, etc.). I have another PetscSection >>>>>>>> for printing out my solution. >>>>>>>> Also if I can convert that DAG indices into my local cell/vertex >>>>>>>> index, can I just use the PetscSF object created from DMGetPointSF(), >>>>>>>> instead of "creating PetscSection + DMGetSectionSF()"? In other words, can >>>>>>>> I use the PetscSF object declared from DMGetPointSF() for the halo >>>>>>>> communication? >>>>>>>> >>>>>>> >>>>>>> No, because that point SF will index information by point number. >>>>>>> You would need to build a new SF that indexes your dofs. The steps you would >>>>>>> go through are exactly the same as you would if you just told us >>>>>>> what the Section is that indexes your data. >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Matt >>>>>>> >>>>>>> >>>>>>>> Thanks, >>>>>>>> Mike >>>>>>>> >>>>>>>> >>>>>>>> The PetscSF that is created automatically is the "point sf" ( >>>>>>>>> https://petsc.org/main/docs/manualpages/DM/DMGetPointSF/): it >>>>>>>>> says which mesh points (cells, faces, edges and vertices) are duplicates of >>>>>>>>> others. >>>>>>>>> >>>>>>>>> In a finite volume application we typically want to assign degrees >>>>>>>>> of freedom just to cells: some applications may only have one degree of >>>>>>>>> freedom, others may have multiple. >>>>>>>>> >>>>>>>>> You encode where you want degrees of freedom in a PetscSection and >>>>>>>>> set that as the section for the DM in DMSetLocalSection() ( >>>>>>>>> https://petsc.org/release/docs/manualpages/DM/DMSetLocalSection.html >>>>>>>>> ) >>>>>>>>> >>>>>>>>> (A c example of these steps that sets degrees of freedom for >>>>>>>>> *vertices* instead of cells is `src/dm/impls/plex/tutorials/ex7.c`) >>>>>>>>> >>>>>>>>> After that you can call DMGetSectionSF() ( >>>>>>>>> https://petsc.org/main/docs/manualpages/DM/DMGetSectionSF/) to >>>>>>>>> the the PetscSF that you want for halo exchange: the one for your solution >>>>>>>>> variables. >>>>>>>>> >>>>>>>>> After that, the only calls you typically need in a finite volume >>>>>>>>> code is PetscSFBcastBegin() to start a halo exchange and PetscSFBcastEnd() >>>>>>>>> to complete it. >>>>>>>>> >>>>>>>>> You say >>>>>>>>> >>>>>>>>> > What I want to do is to exchange data (probably just MPI_Reduce) >>>>>>>>> >>>>>>>>> which confuses me, because halo exchange is a point-to-point >>>>>>>>> exchange and not a reduction. Can you clarify? >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Fri, May 20, 2022 at 8:35 PM Mike Michell < >>>>>>>>> mi.mike1021 at gmail.com> wrote: >>>>>>>>> >>>>>>>>>> Dear PETSc developer team, >>>>>>>>>> >>>>>>>>>> Hi, I am using DMPlex for a finite-volume code and trying to >>>>>>>>>> understand the usage of PetscSF. What is a typical procedure for doing halo >>>>>>>>>> data exchange at parallel boundary using PetscSF object on DMPlex? Is there >>>>>>>>>> any example that I can refer to usage of PetscSF with distributed DMPlex? >>>>>>>>>> >>>>>>>>>> Assuming to use the attached mock-up code and mesh, if I give >>>>>>>>>> "-dm_distribute_overlap 1 -over_dm_view" to run the code, I can see a >>>>>>>>>> PetscSF object is already created, although I have not called >>>>>>>>>> "PetscSFCreate" in the code. How can I import & use that PetscSF already >>>>>>>>>> created by the code to do the halo data exchange? >>>>>>>>>> >>>>>>>>>> What I want to do is to exchange data (probably just MPI_Reduce) >>>>>>>>>> in a parallel boundary region using PetscSF and its functions. I might need >>>>>>>>>> to have an overlapping layer or not. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Mike >>>>>>>>>> >>>>>>>>> >>>>>>> >>>>>>> -- >>>>>>> What most experimenters take for granted before they begin their >>>>>>> experiments is infinitely more interesting than any results to which their >>>>>>> experiments lead. >>>>>>> -- Norbert Wiener >>>>>>> >>>>>>> https://www.cse.buffalo.edu/~knepley/ >>>>>>> >>>>>>> >>>>>> >>> >>> -- >>> What most experimenters take for granted before they begin their >>> experiments is infinitely more interesting than any results to which their >>> experiments lead. >>> -- Norbert Wiener >>> >>> https://www.cse.buffalo.edu/~knepley/ >>> >>> >> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which their >> experiments lead. >> -- Norbert Wiener >> >> https://www.cse.buffalo.edu/~knepley/ >> >> > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From mi.mike1021 at gmail.com Tue May 31 09:04:07 2022 From: mi.mike1021 at gmail.com (Mike Michell) Date: Tue, 31 May 2022 09:04:07 -0500 Subject: [petsc-users] PetscSF Object on Distributed DMPlex for Halo Data Exchange In-Reply-To: References: Message-ID: As a follow-up question on your example, is it possible to call PetscSFCreateRemoteOffsets() from Fortran? My code is written in .F90 and in "petsc/finclude/" there is no petscsf.h so that the code currently cannot find PetscSFCreateRemoteOffsets(). Thanks, Mike 2022? 5? 24? (?) ?? 8:46, Matthew Knepley ?? ??: > I will also point out that Toby has created a nice example showing how to > create an SF for halo exchange between local vectors. > > https://gitlab.com/petsc/petsc/-/merge_requests/5267 > > Thanks, > > Matt > > On Sun, May 22, 2022 at 9:47 PM Matthew Knepley wrote: > >> On Sun, May 22, 2022 at 4:28 PM Mike Michell >> wrote: >> >>> Thanks for the reply. The diagram makes sense and is helpful for >>> understanding 1D representation. >>> >>> However, something is still unclear. From your diagram, the number of >>> roots per process seems to vary according to run arguments, such as >>> "-dm_distribute_overlap", because "the number of roots for a DMPlex is the >>> number of mesh points in the local portion of the mesh (cited from your >>> answer to my question (1))" will end up change according to that argument. >>> However, from my mock-up code, number of roots is independent to >>> -dm_distribute_overlap argument. The summation of "number of roots" through >>> processes was always equal to number of physical vertex on my mesh, if I >>> define the section layout on vertex with 1DOF. But in your diagram example, >>> the summation of "nroots" is larger than the actual number of mesh points, >>> which is 13. >>> >> >> I do not understand your question. Notice the -dm_distribute_overlap does >> _not_ change the owned points for any process. It only puts in new leaves, >> so it also never >> changes the roots for this way of using the SF. >> >> >>> Also, it is still unclear how to get the size of "roots" from the >>> PetscSection & PetscSF on distributed DMPlex? >>> >> >> For an SF mapping ghost dofs in a global vector, the number of roots is >> just the size of the local portion of the vector. >> >> >>> In your diagram, how can you tell your code and make it allocate the >>> "nroots=7 for P0, nroots=9 for P1, and nroots=7 for P2" arrays before you >>> call PetscSFBcastBegin/End()? It seems that we need to define arrays having >>> the size of nroots & nleaves before calling PetscSFBcastBegin/End(). >>> >> >> I just want to note that this usage is different from the canonical usage >> in Plex. It is fine to do this, but this will not match what I do in the >> library if you look. >> In Plex, I distinguish two linear spaces: >> >> 1) Global space: This is the vector space for the solvers. Each point >> is uniquely represented and owned by some process >> >> 2) Local space: This is the vector space for assembly. Some points are >> represented multiple times. >> >> I create an SF that maps from the global space (roots) to the local space >> (leaves), and it is called in DMGlobalToLocal() (and associated functions). >> This >> is more natural in FEM. You seem to want an SF that maps between global >> vectors. This will also work. The roots would be the local dofs, and the >> leaves >> would be shared dofs. >> >> Does this make sense? >> >> Thanks, >> >> Matt >> >> >>> Thanks, >>> Mike >>> >>> Here's a diagram of a 1D mesh with overlap and 3 partitions, showing >>>> what the petscsf data is for each. The number of roots is the number of >>>> mesh points in the local representation, and the number of leaves is the >>>> number of mesh points that are duplicates of mesh points on other >>>> processes. With that in mind, answering your questions >>>> >>>> > (1) It seems that the "roots" means the number of vertex not >>>> considering overlap layer, and "leaves" seems the number of distributed >>>> vertex for each processor that includes overlap layer. Can you acknowledge >>>> that this is correct understanding? I have tried to find clearer examples >>>> from PETSc team's articles relevant to Star Forest, but I am still unclear >>>> about the exact relation & graphical notation of roots & leaves in SF if >>>> it's the case of DMPlex solution arrays. >>>> >>>> No, the number of roots for a DMPlex is the number of mesh points in >>>> the local portion of the mesh >>>> >>>> > (2) If it is so, there is an issue that I cannot define "root data" >>>> and "leave data" generally. I am trying to following >>>> "src/vec/is/sf/tutorials/ex1f.F90", however, in that example, size of roots >>>> and leaves are predefined as 6. How can I generalize that? Because I can >>>> get size of leaves using DAG depth(or height), which is equal to number of >>>> vertices each proc has. But, how can I get the size of my "roots" region >>>> from SF? Any example about that? This question is connected to how can I >>>> define "rootdata" for "PetscSFBcastBegin/End()". >>>> >>>> Does the diagram help you generalize? >>>> >>>> > (3) More importantly, with the attached PetscSection & SF layout, my >>>> vector is only resolved for the size equal to "number of roots" for each >>>> proc, but not for the overlapping area(i.e., "leaves"). What I wish to do >>>> is to exchange (or reduce) the solution data between each proc, in the >>>> overlapping region. Can I get some advices why my vector does not encompass >>>> the "leaves" regime? Is there any example doing similar things? >>>> Going back to my first response: if you use a section to say how many >>>> pieces of data are associated with each local mesh point, then a PetscSF is >>>> constructed that requires no more manipulation from you. >>>> >>>> >>>> On Sun, May 22, 2022 at 10:47 AM Mike Michell >>>> wrote: >>>> >>>>> Thank you for the reply. >>>>> The PetscSection and PetscSF objects are defined as in the attached >>>>> mock-up code (Q_PetscSF_1.tar). 1-DOF is defined on vertex as my solution >>>>> is determined on each vertex with 1-DOF from a finite-volume method. >>>>> >>>>> As follow up questions: >>>>> (1) It seems that the "roots" means the number of vertex not >>>>> considering overlap layer, and "leaves" seems the number of distributed >>>>> vertex for each processor that includes overlap layer. Can you acknowledge >>>>> that this is correct understanding? I have tried to find clearer examples >>>>> from PETSc team's articles relevant to Star Forest, but I am still unclear >>>>> about the exact relation & graphical notation of roots & leaves in SF if >>>>> it's the case of DMPlex solution arrays. >>>>> >>>>> (2) If it is so, there is an issue that I cannot define "root data" >>>>> and "leave data" generally. I am trying to following >>>>> "src/vec/is/sf/tutorials/ex1f.F90", however, in that example, size of roots >>>>> and leaves are predefined as 6. How can I generalize that? Because I can >>>>> get size of leaves using DAG depth(or height), which is equal to number of >>>>> vertices each proc has. But, how can I get the size of my "roots" region >>>>> from SF? Any example about that? This question is connected to how can I >>>>> define "rootdata" for "PetscSFBcastBegin/End()". >>>>> >>>>> (3) More importantly, with the attached PetscSection & SF layout, my >>>>> vector is only resolved for the size equal to "number of roots" for each >>>>> proc, but not for the overlapping area(i.e., "leaves"). What I wish to do >>>>> is to exchange (or reduce) the solution data between each proc, in the >>>>> overlapping region. Can I get some advices why my vector does not encompass >>>>> the "leaves" regime? Is there any example doing similar things? >>>>> >>>>> Thanks, >>>>> Mike >>>>> >>>>> >>>>>> On Fri, May 20, 2022 at 4:45 PM Mike Michell >>>>>> wrote: >>>>>> >>>>>>> Thanks for the reply. >>>>>>> >>>>>>> > "What I want to do is to exchange data (probably just MPI_Reduce)" >>>>>>> which confuses me, because halo exchange is a point-to-point exchange and >>>>>>> not a reduction. Can you clarify? >>>>>>> PetscSFReduceBegin/End seems to be the function that do reduction >>>>>>> for PetscSF object. What I intended to mention was either reduction or >>>>>>> exchange, not specifically intended "reduction". >>>>>>> >>>>>>> As a follow-up question: >>>>>>> Assuming that the code has its own local solution arrays (not Petsc >>>>>>> type), and if the plex's DAG indices belong to the halo region are the only >>>>>>> information that I want to know (not the detailed section description, such >>>>>>> as degree of freedom on vertex, cells, etc.). I have another PetscSection >>>>>>> for printing out my solution. >>>>>>> Also if I can convert that DAG indices into my local cell/vertex >>>>>>> index, can I just use the PetscSF object created from DMGetPointSF(), >>>>>>> instead of "creating PetscSection + DMGetSectionSF()"? In other words, can >>>>>>> I use the PetscSF object declared from DMGetPointSF() for the halo >>>>>>> communication? >>>>>>> >>>>>> >>>>>> No, because that point SF will index information by point number. You >>>>>> would need to build a new SF that indexes your dofs. The steps you would >>>>>> go through are exactly the same as you would if you just told us what >>>>>> the Section is that indexes your data. >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Matt >>>>>> >>>>>> >>>>>>> Thanks, >>>>>>> Mike >>>>>>> >>>>>>> >>>>>>> The PetscSF that is created automatically is the "point sf" ( >>>>>>>> https://petsc.org/main/docs/manualpages/DM/DMGetPointSF/): it says >>>>>>>> which mesh points (cells, faces, edges and vertices) are duplicates of >>>>>>>> others. >>>>>>>> >>>>>>>> In a finite volume application we typically want to assign degrees >>>>>>>> of freedom just to cells: some applications may only have one degree of >>>>>>>> freedom, others may have multiple. >>>>>>>> >>>>>>>> You encode where you want degrees of freedom in a PetscSection and >>>>>>>> set that as the section for the DM in DMSetLocalSection() ( >>>>>>>> https://petsc.org/release/docs/manualpages/DM/DMSetLocalSection.html >>>>>>>> ) >>>>>>>> >>>>>>>> (A c example of these steps that sets degrees of freedom for >>>>>>>> *vertices* instead of cells is `src/dm/impls/plex/tutorials/ex7.c`) >>>>>>>> >>>>>>>> After that you can call DMGetSectionSF() ( >>>>>>>> https://petsc.org/main/docs/manualpages/DM/DMGetSectionSF/) to the >>>>>>>> the PetscSF that you want for halo exchange: the one for your solution >>>>>>>> variables. >>>>>>>> >>>>>>>> After that, the only calls you typically need in a finite volume >>>>>>>> code is PetscSFBcastBegin() to start a halo exchange and PetscSFBcastEnd() >>>>>>>> to complete it. >>>>>>>> >>>>>>>> You say >>>>>>>> >>>>>>>> > What I want to do is to exchange data (probably just MPI_Reduce) >>>>>>>> >>>>>>>> which confuses me, because halo exchange is a point-to-point >>>>>>>> exchange and not a reduction. Can you clarify? >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Fri, May 20, 2022 at 8:35 PM Mike Michell >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Dear PETSc developer team, >>>>>>>>> >>>>>>>>> Hi, I am using DMPlex for a finite-volume code and trying to >>>>>>>>> understand the usage of PetscSF. What is a typical procedure for doing halo >>>>>>>>> data exchange at parallel boundary using PetscSF object on DMPlex? Is there >>>>>>>>> any example that I can refer to usage of PetscSF with distributed DMPlex? >>>>>>>>> >>>>>>>>> Assuming to use the attached mock-up code and mesh, if I give >>>>>>>>> "-dm_distribute_overlap 1 -over_dm_view" to run the code, I can see a >>>>>>>>> PetscSF object is already created, although I have not called >>>>>>>>> "PetscSFCreate" in the code. How can I import & use that PetscSF already >>>>>>>>> created by the code to do the halo data exchange? >>>>>>>>> >>>>>>>>> What I want to do is to exchange data (probably just MPI_Reduce) >>>>>>>>> in a parallel boundary region using PetscSF and its functions. I might need >>>>>>>>> to have an overlapping layer or not. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Mike >>>>>>>>> >>>>>>>> >>>>>> >>>>>> -- >>>>>> What most experimenters take for granted before they begin their >>>>>> experiments is infinitely more interesting than any results to which their >>>>>> experiments lead. >>>>>> -- Norbert Wiener >>>>>> >>>>>> https://www.cse.buffalo.edu/~knepley/ >>>>>> >>>>>> >>>>> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which their >> experiments lead. >> -- Norbert Wiener >> >> https://www.cse.buffalo.edu/~knepley/ >> >> > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > > https://www.cse.buffalo.edu/~knepley/ > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mi.mike1021 at gmail.com Tue May 31 09:25:50 2022 From: mi.mike1021 at gmail.com (Mike Michell) Date: Tue, 31 May 2022 09:25:50 -0500 Subject: [petsc-users] PetscSF Object on Distributed DMPlex for Halo Data Exchange In-Reply-To: References: Message-ID: Thank you. But, it seems that PetscSFCreateSectionSF() also requires petscsf.h file. Which header file I should include to call PetscSFCreateSectionSF() from Fortran? Thanks, Mike On Tue, May 31, 2022 at 10:04 AM Mike Michell wrote: > >> As a follow-up question on your example, is it possible to call >> PetscSFCreateRemoteOffsets() from Fortran? >> >> My code is written in .F90 and in "petsc/finclude/" there is no petscsf.h >> so that the code currently cannot find PetscSFCreateRemoteOffsets(). >> > > I believe if you pass in NULL for remoteOffsets, that function will be > called internally. > > Thanks, > > Matt > > >> Thanks, >> Mike >> >> >> I will also point out that Toby has created a nice example showing how to >>> create an SF for halo exchange between local vectors. >>> >>> https://gitlab.com/petsc/petsc/-/merge_requests/5267 >>> >>> Thanks, >>> >>> Matt >>> >>> On Sun, May 22, 2022 at 9:47 PM Matthew Knepley >>> wrote: >>> >>>> On Sun, May 22, 2022 at 4:28 PM Mike Michell >>>> wrote: >>>> >>>>> Thanks for the reply. The diagram makes sense and is helpful for >>>>> understanding 1D representation. >>>>> >>>>> However, something is still unclear. From your diagram, the number of >>>>> roots per process seems to vary according to run arguments, such as >>>>> "-dm_distribute_overlap", because "the number of roots for a DMPlex is the >>>>> number of mesh points in the local portion of the mesh (cited from your >>>>> answer to my question (1))" will end up change according to that argument. >>>>> However, from my mock-up code, number of roots is independent to >>>>> -dm_distribute_overlap argument. The summation of "number of roots" through >>>>> processes was always equal to number of physical vertex on my mesh, if I >>>>> define the section layout on vertex with 1DOF. But in your diagram example, >>>>> the summation of "nroots" is larger than the actual number of mesh points, >>>>> which is 13. >>>>> >>>> >>>> I do not understand your question. Notice the -dm_distribute_overlap >>>> does _not_ change the owned points for any process. It only puts in new >>>> leaves, so it also never >>>> changes the roots for this way of using the SF. >>>> >>>> >>>>> Also, it is still unclear how to get the size of "roots" from the >>>>> PetscSection & PetscSF on distributed DMPlex? >>>>> >>>> >>>> For an SF mapping ghost dofs in a global vector, the number of roots is >>>> just the size of the local portion of the vector. >>>> >>>> >>>>> In your diagram, how can you tell your code and make it allocate the >>>>> "nroots=7 for P0, nroots=9 for P1, and nroots=7 for P2" arrays before you >>>>> call PetscSFBcastBegin/End()? It seems that we need to define arrays having >>>>> the size of nroots & nleaves before calling PetscSFBcastBegin/End(). >>>>> >>>> >>>> I just want to note that this usage is different from the canonical >>>> usage in Plex. It is fine to do this, but this will not match what I do in >>>> the library if you look. >>>> In Plex, I distinguish two linear spaces: >>>> >>>> 1) Global space: This is the vector space for the solvers. Each point >>>> is uniquely represented and owned by some process >>>> >>>> 2) Local space: This is the vector space for assembly. Some points >>>> are represented multiple times. >>>> >>>> I create an SF that maps from the global space (roots) to the local >>>> space (leaves), and it is called in DMGlobalToLocal() (and >>>> associated functions). This >>>> is more natural in FEM. You seem to want an SF that maps between global >>>> vectors. This will also work. The roots would be the local dofs, and the >>>> leaves >>>> would be shared dofs. >>>> >>>> Does this make sense? >>>> >>>> Thanks, >>>> >>>> Matt >>>> >>>> >>>>> Thanks, >>>>> Mike >>>>> >>>>> Here's a diagram of a 1D mesh with overlap and 3 partitions, showing >>>>>> what the petscsf data is for each. The number of roots is the number of >>>>>> mesh points in the local representation, and the number of leaves is the >>>>>> number of mesh points that are duplicates of mesh points on other >>>>>> processes. With that in mind, answering your questions >>>>>> >>>>>> > (1) It seems that the "roots" means the number of vertex not >>>>>> considering overlap layer, and "leaves" seems the number of distributed >>>>>> vertex for each processor that includes overlap layer. Can you acknowledge >>>>>> that this is correct understanding? I have tried to find clearer examples >>>>>> from PETSc team's articles relevant to Star Forest, but I am still unclear >>>>>> about the exact relation & graphical notation of roots & leaves in SF if >>>>>> it's the case of DMPlex solution arrays. >>>>>> >>>>>> No, the number of roots for a DMPlex is the number of mesh points in >>>>>> the local portion of the mesh >>>>>> >>>>>> > (2) If it is so, there is an issue that I cannot define "root data" >>>>>> and "leave data" generally. I am trying to following >>>>>> "src/vec/is/sf/tutorials/ex1f.F90", however, in that example, size of roots >>>>>> and leaves are predefined as 6. How can I generalize that? Because I can >>>>>> get size of leaves using DAG depth(or height), which is equal to number of >>>>>> vertices each proc has. But, how can I get the size of my "roots" region >>>>>> from SF? Any example about that? This question is connected to how can I >>>>>> define "rootdata" for "PetscSFBcastBegin/End()". >>>>>> >>>>>> Does the diagram help you generalize? >>>>>> >>>>>> > (3) More importantly, with the attached PetscSection & SF layout, >>>>>> my vector is only resolved for the size equal to "number of roots" for each >>>>>> proc, but not for the overlapping area(i.e., "leaves"). What I wish to do >>>>>> is to exchange (or reduce) the solution data between each proc, in the >>>>>> overlapping region. Can I get some advices why my vector does not encompass >>>>>> the "leaves" regime? Is there any example doing similar things? >>>>>> Going back to my first response: if you use a section to say how many >>>>>> pieces of data are associated with each local mesh point, then a PetscSF is >>>>>> constructed that requires no more manipulation from you. >>>>>> >>>>>> >>>>>> On Sun, May 22, 2022 at 10:47 AM Mike Michell >>>>>> wrote: >>>>>> >>>>>>> Thank you for the reply. >>>>>>> The PetscSection and PetscSF objects are defined as in the attached >>>>>>> mock-up code (Q_PetscSF_1.tar). 1-DOF is defined on vertex as my solution >>>>>>> is determined on each vertex with 1-DOF from a finite-volume method. >>>>>>> >>>>>>> As follow up questions: >>>>>>> (1) It seems that the "roots" means the number of vertex not >>>>>>> considering overlap layer, and "leaves" seems the number of distributed >>>>>>> vertex for each processor that includes overlap layer. Can you acknowledge >>>>>>> that this is correct understanding? I have tried to find clearer examples >>>>>>> from PETSc team's articles relevant to Star Forest, but I am still unclear >>>>>>> about the exact relation & graphical notation of roots & leaves in SF if >>>>>>> it's the case of DMPlex solution arrays. >>>>>>> >>>>>>> (2) If it is so, there is an issue that I cannot define "root data" >>>>>>> and "leave data" generally. I am trying to following >>>>>>> "src/vec/is/sf/tutorials/ex1f.F90", however, in that example, size of roots >>>>>>> and leaves are predefined as 6. How can I generalize that? Because I can >>>>>>> get size of leaves using DAG depth(or height), which is equal to number of >>>>>>> vertices each proc has. But, how can I get the size of my "roots" region >>>>>>> from SF? Any example about that? This question is connected to how can I >>>>>>> define "rootdata" for "PetscSFBcastBegin/End()". >>>>>>> >>>>>>> (3) More importantly, with the attached PetscSection & SF layout, my >>>>>>> vector is only resolved for the size equal to "number of roots" for each >>>>>>> proc, but not for the overlapping area(i.e., "leaves"). What I wish to do >>>>>>> is to exchange (or reduce) the solution data between each proc, in the >>>>>>> overlapping region. Can I get some advices why my vector does not encompass >>>>>>> the "leaves" regime? Is there any example doing similar things? >>>>>>> >>>>>>> Thanks, >>>>>>> Mike >>>>>>> >>>>>>> >>>>>>>> On Fri, May 20, 2022 at 4:45 PM Mike Michell >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Thanks for the reply. >>>>>>>>> >>>>>>>>> > "What I want to do is to exchange data (probably just >>>>>>>>> MPI_Reduce)" which confuses me, because halo exchange is a point-to-point >>>>>>>>> exchange and not a reduction. Can you clarify? >>>>>>>>> PetscSFReduceBegin/End seems to be the function that do reduction >>>>>>>>> for PetscSF object. What I intended to mention was either reduction or >>>>>>>>> exchange, not specifically intended "reduction". >>>>>>>>> >>>>>>>>> As a follow-up question: >>>>>>>>> Assuming that the code has its own local solution arrays (not >>>>>>>>> Petsc type), and if the plex's DAG indices belong to the halo region are >>>>>>>>> the only information that I want to know (not the detailed section >>>>>>>>> description, such as degree of freedom on vertex, cells, etc.). I have >>>>>>>>> another PetscSection for printing out my solution. >>>>>>>>> Also if I can convert that DAG indices into my local cell/vertex >>>>>>>>> index, can I just use the PetscSF object created from DMGetPointSF(), >>>>>>>>> instead of "creating PetscSection + DMGetSectionSF()"? In other words, can >>>>>>>>> I use the PetscSF object declared from DMGetPointSF() for the halo >>>>>>>>> communication? >>>>>>>>> >>>>>>>> >>>>>>>> No, because that point SF will index information by point number. >>>>>>>> You would need to build a new SF that indexes your dofs. The steps you would >>>>>>>> go through are exactly the same as you would if you just told us >>>>>>>> what the Section is that indexes your data. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Matt >>>>>>>> >>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Mike >>>>>>>>> >>>>>>>>> >>>>>>>>> The PetscSF that is created automatically is the "point sf" ( >>>>>>>>>> https://petsc.org/main/docs/manualpages/DM/DMGetPointSF/): it >>>>>>>>>> says which mesh points (cells, faces, edges and vertices) are duplicates of >>>>>>>>>> others. >>>>>>>>>> >>>>>>>>>> In a finite volume application we typically want to assign >>>>>>>>>> degrees of freedom just to cells: some applications may only have one >>>>>>>>>> degree of freedom, others may have multiple. >>>>>>>>>> >>>>>>>>>> You encode where you want degrees of freedom in a PetscSection >>>>>>>>>> and set that as the section for the DM in DMSetLocalSection() ( >>>>>>>>>> https://petsc.org/release/docs/manualpages/DM/DMSetLocalSection.html >>>>>>>>>> ) >>>>>>>>>> >>>>>>>>>> (A c example of these steps that sets degrees of freedom for >>>>>>>>>> *vertices* instead of cells is `src/dm/impls/plex/tutorials/ex7.c`) >>>>>>>>>> >>>>>>>>>> After that you can call DMGetSectionSF() ( >>>>>>>>>> https://petsc.org/main/docs/manualpages/DM/DMGetSectionSF/) to >>>>>>>>>> the the PetscSF that you want for halo exchange: the one for your solution >>>>>>>>>> variables. >>>>>>>>>> >>>>>>>>>> After that, the only calls you typically need in a finite volume >>>>>>>>>> code is PetscSFBcastBegin() to start a halo exchange and PetscSFBcastEnd() >>>>>>>>>> to complete it. >>>>>>>>>> >>>>>>>>>> You say >>>>>>>>>> >>>>>>>>>> > What I want to do is to exchange data (probably just MPI_Reduce) >>>>>>>>>> >>>>>>>>>> which confuses me, because halo exchange is a point-to-point >>>>>>>>>> exchange and not a reduction. Can you clarify? >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Fri, May 20, 2022 at 8:35 PM Mike Michell < >>>>>>>>>> mi.mike1021 at gmail.com> wrote: >>>>>>>>>> >>>>>>>>>>> Dear PETSc developer team, >>>>>>>>>>> >>>>>>>>>>> Hi, I am using DMPlex for a finite-volume code and trying to >>>>>>>>>>> understand the usage of PetscSF. What is a typical procedure for doing halo >>>>>>>>>>> data exchange at parallel boundary using PetscSF object on DMPlex? Is there >>>>>>>>>>> any example that I can refer to usage of PetscSF with distributed DMPlex? >>>>>>>>>>> >>>>>>>>>>> Assuming to use the attached mock-up code and mesh, if I give >>>>>>>>>>> "-dm_distribute_overlap 1 -over_dm_view" to run the code, I can see a >>>>>>>>>>> PetscSF object is already created, although I have not called >>>>>>>>>>> "PetscSFCreate" in the code. How can I import & use that PetscSF already >>>>>>>>>>> created by the code to do the halo data exchange? >>>>>>>>>>> >>>>>>>>>>> What I want to do is to exchange data (probably just MPI_Reduce) >>>>>>>>>>> in a parallel boundary region using PetscSF and its functions. I might need >>>>>>>>>>> to have an overlapping layer or not. >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Mike >>>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> What most experimenters take for granted before they begin their >>>>>>>> experiments is infinitely more interesting than any results to which their >>>>>>>> experiments lead. >>>>>>>> -- Norbert Wiener >>>>>>>> >>>>>>>> https://www.cse.buffalo.edu/~knepley/ >>>>>>>> >>>>>>>> >>>>>>> >>>> >>>> -- >>>> What most experimenters take for granted before they begin their >>>> experiments is infinitely more interesting than any results to which their >>>> experiments lead. >>>> -- Norbert Wiener >>>> >>>> https://www.cse.buffalo.edu/~knepley/ >>>> >>>> >>> >>> >>> -- >>> What most experimenters take for granted before they begin their >>> experiments is infinitely more interesting than any results to which their >>> experiments lead. >>> -- Norbert Wiener >>> >>> https://www.cse.buffalo.edu/~knepley/ >>> >>> >> > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > > https://www.cse.buffalo.edu/~knepley/ > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Ye_Changqing at outlook.com Tue May 31 08:55:40 2022 From: Ye_Changqing at outlook.com (Ye Changqing) Date: Tue, 31 May 2022 13:55:40 +0000 Subject: [petsc-users] Mat created by DMStag cannot access ghost points Message-ID: Dear developers of PETSc, I encountered a problem when using the DMStag module. The program could be executed perfectly in serial, while errors are thrown out in parallel (using mpiexec). Some rows in Mat cannot be accessed in local processes when looping all elements in DMStag. The DM object I used only has one DOF in each element. Hence, I could switch to the DMDA module easily, and the program now is back to normal. Some snippets are below. Initialise a DMStag object: PetscCall(DMStagCreate2d(PETSC_COMM_WORLD, DM_BOUNDARY_NONE, DM_BOUNDARY_NONE, M, N, PETSC_DECIDE, PETSC_DECIDE, 0, 0, 1, DMSTAG_STENCIL_BOX, 1, NULL, NULL, &(s_ctx->dm_P))); Created a Mat: PetscCall(DMCreateMatrix(s_ctx->dm_P, A)); Loop: PetscCall(DMStagGetCorners(s_ctx->dm_V, &startx, &starty, &startz, &nx, &ny, &nz, &extrax, &extray, &extraz)); for (ey = starty; ey < starty + ny; ++ey) for (ex = startx; ex < startx + nx; ++ex) { ... PetscCall(DMStagMatSetValuesStencil(s_ctx->dm_P, *A, 2, &row[0], 2, &col[0], &val_A[0][0], ADD_VALUES)); // The traceback shows the problem is in here. } Best, Changqing -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Tue May 31 09:34:41 2022 From: knepley at gmail.com (Matthew Knepley) Date: Tue, 31 May 2022 10:34:41 -0400 Subject: [petsc-users] PetscSF Object on Distributed DMPlex for Halo Data Exchange In-Reply-To: References: Message-ID: On Tue, May 31, 2022 at 10:26 AM Mike Michell wrote: > Thank you. But, it seems that PetscSFCreateSectionSF() also requires > petscsf.h file. Which header file I should include to call > PetscSFCreateSectionSF() from Fortran? > I will have to write a binding. I will send you the MR when I finish. THanks, Matt > Thanks, > Mike > > > On Tue, May 31, 2022 at 10:04 AM Mike Michell >> wrote: >> >>> As a follow-up question on your example, is it possible to call >>> PetscSFCreateRemoteOffsets() from Fortran? >>> >>> My code is written in .F90 and in "petsc/finclude/" there is no >>> petscsf.h so that the code currently cannot find >>> PetscSFCreateRemoteOffsets(). >>> >> >> I believe if you pass in NULL for remoteOffsets, that function will be >> called internally. >> >> Thanks, >> >> Matt >> >> >>> Thanks, >>> Mike >>> >>> >>> I will also point out that Toby has created a nice example showing how >>>> to create an SF for halo exchange between local vectors. >>>> >>>> https://gitlab.com/petsc/petsc/-/merge_requests/5267 >>>> >>>> Thanks, >>>> >>>> Matt >>>> >>>> On Sun, May 22, 2022 at 9:47 PM Matthew Knepley >>>> wrote: >>>> >>>>> On Sun, May 22, 2022 at 4:28 PM Mike Michell >>>>> wrote: >>>>> >>>>>> Thanks for the reply. The diagram makes sense and is helpful for >>>>>> understanding 1D representation. >>>>>> >>>>>> However, something is still unclear. From your diagram, the number of >>>>>> roots per process seems to vary according to run arguments, such as >>>>>> "-dm_distribute_overlap", because "the number of roots for a DMPlex is the >>>>>> number of mesh points in the local portion of the mesh (cited from your >>>>>> answer to my question (1))" will end up change according to that argument. >>>>>> However, from my mock-up code, number of roots is independent to >>>>>> -dm_distribute_overlap argument. The summation of "number of roots" through >>>>>> processes was always equal to number of physical vertex on my mesh, if I >>>>>> define the section layout on vertex with 1DOF. But in your diagram example, >>>>>> the summation of "nroots" is larger than the actual number of mesh points, >>>>>> which is 13. >>>>>> >>>>> >>>>> I do not understand your question. Notice the -dm_distribute_overlap >>>>> does _not_ change the owned points for any process. It only puts in new >>>>> leaves, so it also never >>>>> changes the roots for this way of using the SF. >>>>> >>>>> >>>>>> Also, it is still unclear how to get the size of "roots" from the >>>>>> PetscSection & PetscSF on distributed DMPlex? >>>>>> >>>>> >>>>> For an SF mapping ghost dofs in a global vector, the number of roots >>>>> is just the size of the local portion of the vector. >>>>> >>>>> >>>>>> In your diagram, how can you tell your code and make it allocate the >>>>>> "nroots=7 for P0, nroots=9 for P1, and nroots=7 for P2" arrays before you >>>>>> call PetscSFBcastBegin/End()? It seems that we need to define arrays having >>>>>> the size of nroots & nleaves before calling PetscSFBcastBegin/End(). >>>>>> >>>>> >>>>> I just want to note that this usage is different from the canonical >>>>> usage in Plex. It is fine to do this, but this will not match what I do in >>>>> the library if you look. >>>>> In Plex, I distinguish two linear spaces: >>>>> >>>>> 1) Global space: This is the vector space for the solvers. Each >>>>> point is uniquely represented and owned by some process >>>>> >>>>> 2) Local space: This is the vector space for assembly. Some points >>>>> are represented multiple times. >>>>> >>>>> I create an SF that maps from the global space (roots) to the local >>>>> space (leaves), and it is called in DMGlobalToLocal() (and >>>>> associated functions). This >>>>> is more natural in FEM. You seem to want an SF that maps between >>>>> global vectors. This will also work. The roots would be the local dofs, and >>>>> the leaves >>>>> would be shared dofs. >>>>> >>>>> Does this make sense? >>>>> >>>>> Thanks, >>>>> >>>>> Matt >>>>> >>>>> >>>>>> Thanks, >>>>>> Mike >>>>>> >>>>>> Here's a diagram of a 1D mesh with overlap and 3 partitions, showing >>>>>>> what the petscsf data is for each. The number of roots is the number of >>>>>>> mesh points in the local representation, and the number of leaves is the >>>>>>> number of mesh points that are duplicates of mesh points on other >>>>>>> processes. With that in mind, answering your questions >>>>>>> >>>>>>> > (1) It seems that the "roots" means the number of vertex not >>>>>>> considering overlap layer, and "leaves" seems the number of distributed >>>>>>> vertex for each processor that includes overlap layer. Can you acknowledge >>>>>>> that this is correct understanding? I have tried to find clearer examples >>>>>>> from PETSc team's articles relevant to Star Forest, but I am still unclear >>>>>>> about the exact relation & graphical notation of roots & leaves in SF if >>>>>>> it's the case of DMPlex solution arrays. >>>>>>> >>>>>>> No, the number of roots for a DMPlex is the number of mesh points in >>>>>>> the local portion of the mesh >>>>>>> >>>>>>> > (2) If it is so, there is an issue that I cannot define "root >>>>>>> data" and "leave data" generally. I am trying to following >>>>>>> "src/vec/is/sf/tutorials/ex1f.F90", however, in that example, size of roots >>>>>>> and leaves are predefined as 6. How can I generalize that? Because I can >>>>>>> get size of leaves using DAG depth(or height), which is equal to number of >>>>>>> vertices each proc has. But, how can I get the size of my "roots" region >>>>>>> from SF? Any example about that? This question is connected to how can I >>>>>>> define "rootdata" for "PetscSFBcastBegin/End()". >>>>>>> >>>>>>> Does the diagram help you generalize? >>>>>>> >>>>>>> > (3) More importantly, with the attached PetscSection & SF layout, >>>>>>> my vector is only resolved for the size equal to "number of roots" for each >>>>>>> proc, but not for the overlapping area(i.e., "leaves"). What I wish to do >>>>>>> is to exchange (or reduce) the solution data between each proc, in the >>>>>>> overlapping region. Can I get some advices why my vector does not encompass >>>>>>> the "leaves" regime? Is there any example doing similar things? >>>>>>> Going back to my first response: if you use a section to say how >>>>>>> many pieces of data are associated with each local mesh point, then a >>>>>>> PetscSF is constructed that requires no more manipulation from you. >>>>>>> >>>>>>> >>>>>>> On Sun, May 22, 2022 at 10:47 AM Mike Michell >>>>>>> wrote: >>>>>>> >>>>>>>> Thank you for the reply. >>>>>>>> The PetscSection and PetscSF objects are defined as in the attached >>>>>>>> mock-up code (Q_PetscSF_1.tar). 1-DOF is defined on vertex as my solution >>>>>>>> is determined on each vertex with 1-DOF from a finite-volume method. >>>>>>>> >>>>>>>> As follow up questions: >>>>>>>> (1) It seems that the "roots" means the number of vertex not >>>>>>>> considering overlap layer, and "leaves" seems the number of distributed >>>>>>>> vertex for each processor that includes overlap layer. Can you acknowledge >>>>>>>> that this is correct understanding? I have tried to find clearer examples >>>>>>>> from PETSc team's articles relevant to Star Forest, but I am still unclear >>>>>>>> about the exact relation & graphical notation of roots & leaves in SF if >>>>>>>> it's the case of DMPlex solution arrays. >>>>>>>> >>>>>>>> (2) If it is so, there is an issue that I cannot define "root data" >>>>>>>> and "leave data" generally. I am trying to following >>>>>>>> "src/vec/is/sf/tutorials/ex1f.F90", however, in that example, size of roots >>>>>>>> and leaves are predefined as 6. How can I generalize that? Because I can >>>>>>>> get size of leaves using DAG depth(or height), which is equal to number of >>>>>>>> vertices each proc has. But, how can I get the size of my "roots" region >>>>>>>> from SF? Any example about that? This question is connected to how can I >>>>>>>> define "rootdata" for "PetscSFBcastBegin/End()". >>>>>>>> >>>>>>>> (3) More importantly, with the attached PetscSection & SF layout, >>>>>>>> my vector is only resolved for the size equal to "number of roots" for each >>>>>>>> proc, but not for the overlapping area(i.e., "leaves"). What I wish to do >>>>>>>> is to exchange (or reduce) the solution data between each proc, in the >>>>>>>> overlapping region. Can I get some advices why my vector does not encompass >>>>>>>> the "leaves" regime? Is there any example doing similar things? >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Mike >>>>>>>> >>>>>>>> >>>>>>>>> On Fri, May 20, 2022 at 4:45 PM Mike Michell < >>>>>>>>> mi.mike1021 at gmail.com> wrote: >>>>>>>>> >>>>>>>>>> Thanks for the reply. >>>>>>>>>> >>>>>>>>>> > "What I want to do is to exchange data (probably just >>>>>>>>>> MPI_Reduce)" which confuses me, because halo exchange is a point-to-point >>>>>>>>>> exchange and not a reduction. Can you clarify? >>>>>>>>>> PetscSFReduceBegin/End seems to be the function that do reduction >>>>>>>>>> for PetscSF object. What I intended to mention was either reduction or >>>>>>>>>> exchange, not specifically intended "reduction". >>>>>>>>>> >>>>>>>>>> As a follow-up question: >>>>>>>>>> Assuming that the code has its own local solution arrays (not >>>>>>>>>> Petsc type), and if the plex's DAG indices belong to the halo region are >>>>>>>>>> the only information that I want to know (not the detailed section >>>>>>>>>> description, such as degree of freedom on vertex, cells, etc.). I have >>>>>>>>>> another PetscSection for printing out my solution. >>>>>>>>>> Also if I can convert that DAG indices into my local cell/vertex >>>>>>>>>> index, can I just use the PetscSF object created from DMGetPointSF(), >>>>>>>>>> instead of "creating PetscSection + DMGetSectionSF()"? In other words, can >>>>>>>>>> I use the PetscSF object declared from DMGetPointSF() for the halo >>>>>>>>>> communication? >>>>>>>>>> >>>>>>>>> >>>>>>>>> No, because that point SF will index information by point number. >>>>>>>>> You would need to build a new SF that indexes your dofs. The steps you would >>>>>>>>> go through are exactly the same as you would if you just told us >>>>>>>>> what the Section is that indexes your data. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> Matt >>>>>>>>> >>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Mike >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> The PetscSF that is created automatically is the "point sf" ( >>>>>>>>>>> https://petsc.org/main/docs/manualpages/DM/DMGetPointSF/): it >>>>>>>>>>> says which mesh points (cells, faces, edges and vertices) are duplicates of >>>>>>>>>>> others. >>>>>>>>>>> >>>>>>>>>>> In a finite volume application we typically want to assign >>>>>>>>>>> degrees of freedom just to cells: some applications may only have one >>>>>>>>>>> degree of freedom, others may have multiple. >>>>>>>>>>> >>>>>>>>>>> You encode where you want degrees of freedom in a PetscSection >>>>>>>>>>> and set that as the section for the DM in DMSetLocalSection() ( >>>>>>>>>>> https://petsc.org/release/docs/manualpages/DM/DMSetLocalSection.html >>>>>>>>>>> ) >>>>>>>>>>> >>>>>>>>>>> (A c example of these steps that sets degrees of freedom for >>>>>>>>>>> *vertices* instead of cells is `src/dm/impls/plex/tutorials/ex7.c`) >>>>>>>>>>> >>>>>>>>>>> After that you can call DMGetSectionSF() ( >>>>>>>>>>> https://petsc.org/main/docs/manualpages/DM/DMGetSectionSF/) to >>>>>>>>>>> the the PetscSF that you want for halo exchange: the one for your solution >>>>>>>>>>> variables. >>>>>>>>>>> >>>>>>>>>>> After that, the only calls you typically need in a finite volume >>>>>>>>>>> code is PetscSFBcastBegin() to start a halo exchange and PetscSFBcastEnd() >>>>>>>>>>> to complete it. >>>>>>>>>>> >>>>>>>>>>> You say >>>>>>>>>>> >>>>>>>>>>> > What I want to do is to exchange data (probably just >>>>>>>>>>> MPI_Reduce) >>>>>>>>>>> >>>>>>>>>>> which confuses me, because halo exchange is a point-to-point >>>>>>>>>>> exchange and not a reduction. Can you clarify? >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Fri, May 20, 2022 at 8:35 PM Mike Michell < >>>>>>>>>>> mi.mike1021 at gmail.com> wrote: >>>>>>>>>>> >>>>>>>>>>>> Dear PETSc developer team, >>>>>>>>>>>> >>>>>>>>>>>> Hi, I am using DMPlex for a finite-volume code and trying to >>>>>>>>>>>> understand the usage of PetscSF. What is a typical procedure for doing halo >>>>>>>>>>>> data exchange at parallel boundary using PetscSF object on DMPlex? Is there >>>>>>>>>>>> any example that I can refer to usage of PetscSF with distributed DMPlex? >>>>>>>>>>>> >>>>>>>>>>>> Assuming to use the attached mock-up code and mesh, if I give >>>>>>>>>>>> "-dm_distribute_overlap 1 -over_dm_view" to run the code, I can see a >>>>>>>>>>>> PetscSF object is already created, although I have not called >>>>>>>>>>>> "PetscSFCreate" in the code. How can I import & use that PetscSF already >>>>>>>>>>>> created by the code to do the halo data exchange? >>>>>>>>>>>> >>>>>>>>>>>> What I want to do is to exchange data (probably just >>>>>>>>>>>> MPI_Reduce) in a parallel boundary region using PetscSF and its functions. >>>>>>>>>>>> I might need to have an overlapping layer or not. >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Mike >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> What most experimenters take for granted before they begin their >>>>>>>>> experiments is infinitely more interesting than any results to which their >>>>>>>>> experiments lead. >>>>>>>>> -- Norbert Wiener >>>>>>>>> >>>>>>>>> https://www.cse.buffalo.edu/~knepley/ >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>> >>>>> -- >>>>> What most experimenters take for granted before they begin their >>>>> experiments is infinitely more interesting than any results to which their >>>>> experiments lead. >>>>> -- Norbert Wiener >>>>> >>>>> https://www.cse.buffalo.edu/~knepley/ >>>>> >>>>> >>>> >>>> >>>> -- >>>> What most experimenters take for granted before they begin their >>>> experiments is infinitely more interesting than any results to which their >>>> experiments lead. >>>> -- Norbert Wiener >>>> >>>> https://www.cse.buffalo.edu/~knepley/ >>>> >>>> >>> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which their >> experiments lead. >> -- Norbert Wiener >> >> https://www.cse.buffalo.edu/~knepley/ >> >> > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Tue May 31 09:38:17 2022 From: knepley at gmail.com (Matthew Knepley) Date: Tue, 31 May 2022 10:38:17 -0400 Subject: [petsc-users] Mat created by DMStag cannot access ghost points In-Reply-To: References: Message-ID: On Tue, May 31, 2022 at 10:28 AM Ye Changqing wrote: > Dear developers of PETSc, > > I encountered a problem when using the DMStag module. The program could be > executed perfectly in serial, while errors are thrown out in parallel > (using mpiexec). Some rows in Mat cannot be accessed in local processes > when looping all elements in DMStag. The DM object I used only has one DOF > in each element. Hence, I could switch to the DMDA module easily, and the > program now is back to normal. > Is it possible to send a simple example? We have parallel examples with DMStag in PETSc, just from this description I cannot see what is going wrong for you. Thanks, Matt > Some snippets are below. > > Initialise a DMStag object: > PetscCall(DMStagCreate2d(PETSC_COMM_WORLD, DM_BOUNDARY_NONE, > DM_BOUNDARY_NONE, M, N, PETSC_DECIDE, PETSC_DECIDE, 0, 0, 1, > DMSTAG_STENCIL_BOX, 1, NULL, NULL, &(s_ctx->dm_P))); > Created a Mat: > PetscCall(DMCreateMatrix(s_ctx->dm_P, A)); > Loop: > PetscCall(DMStagGetCorners(s_ctx->dm_V, &startx, &starty, &startz, &nx, > &ny, &nz, &extrax, &extray, &extraz)); > for (ey = starty; ey < starty + ny; ++ey) > for (ex = startx; ex < startx + nx; ++ex) > { > ... > PetscCall(DMStagMatSetValuesStencil(s_ctx->dm_P, *A, 2, &row[0], 2, > &col[0], &val_A[0][0], ADD_VALUES)); // The traceback shows the problem is > in here. > } > > Best, > Changqing > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From dave.mayhem23 at gmail.com Tue May 31 09:55:05 2022 From: dave.mayhem23 at gmail.com (Dave May) Date: Tue, 31 May 2022 16:55:05 +0200 Subject: [petsc-users] Mat created by DMStag cannot access ghost points In-Reply-To: References: Message-ID: On Tue 31. May 2022 at 16:28, Ye Changqing wrote: > Dear developers of PETSc, > > I encountered a problem when using the DMStag module. The program could be > executed perfectly in serial, while errors are thrown out in parallel > (using mpiexec). Some rows in Mat cannot be accessed in local processes > when looping all elements in DMStag. The DM object I used only has one DOF > in each element. Hence, I could switch to the DMDA module easily, and the > program now is back to normal. > > Some snippets are below. > > Initialise a DMStag object: > PetscCall(DMStagCreate2d(PETSC_COMM_WORLD, DM_BOUNDARY_NONE, > DM_BOUNDARY_NONE, M, N, PETSC_DECIDE, PETSC_DECIDE, 0, 0, 1, > DMSTAG_STENCIL_BOX, 1, NULL, NULL, &(s_ctx->dm_P))); > Created a Mat: > PetscCall(DMCreateMatrix(s_ctx->dm_P, A)); > Loop: > PetscCall(DMStagGetCorners(s_ctx->dm_V, &startx, &starty, &startz, &nx, > &ny, &nz, &extrax, &extray, &extraz)); > for (ey = starty; ey < starty + ny; ++ey) > for (ex = startx; ex < startx + nx; ++ex) > { > ... > PetscCall(DMStagMatSetValuesStencil(s_ctx->dm_P, *A, 2, &row[0], 2, > &col[0], &val_A[0][0], ADD_VALUES)); // The traceback shows the problem is > in here. > } > In addition to the code or MWE, please forward us the complete stack trace / error thrown to stdout. Thanks, Dave > Best, > Changqing > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sami.ben-elhaj-salah at ensma.fr Tue May 31 11:08:17 2022 From: sami.ben-elhaj-salah at ensma.fr (Sami BEN ELHAJ SALAH) Date: Tue, 31 May 2022 18:08:17 +0200 Subject: [petsc-users] Question about DMPlexDistribute & distribute mesh over processes In-Reply-To: References: <7E83D8A1-E911-4B48-98C9-BCFA9429AA11@ensma.fr> <15043588-F026-46C0-8334-E7ABA28F44FF@ensma.fr> Message-ID: <556834F6-DAC5-46B3-9121-283E362F72AA@ensma.fr> Hi Matthew, Two question in this mail: I tested my code with 2 processes. I use gmsh mesh( example: 8 hexahedral elements, 27 nodes, and Global Size of my jacobian matrix = 81 ). Then I used DMView(dm_mesh, PETSC_VIEWER_STDOUT_WORLD) to visualize my DM and see if it is correctly distributed over processes, so I got this: (***) DM Object: DM_0x3_0 2 MPI processes type: plex DM_0x3_0 in 3 dimensions: 0-cells: 27 0 3-cells: 8 0 Labels: material-id: 3 strata with value/size (66 (4), 67 (2), 68 (2)) depth: 2 strata with value/size (0 (27), 1 (8)) celltype: 2 strata with value/size (7 (8), 0 (27)) As you see, I created some material-id labels, so I found them over my 8 cells when using DMView. So it seems good to me. The SNES viewer is shown below: SNES Object: 2 MPI processes type: ksponly SNES has not been set up so information may be incomplete maximum iterations=50, maximum function evaluations=10000 tolerances: relative=1e-08, absolute=1e-50, solution=1e-08 total number of linear solver iterations=0 total number of function evaluations=0 norm schedule ALWAYS SNES Object: 2 MPI processes type: ksponly SNES has not been set up so information may be incomplete maximum iterations=50, maximum function evaluations=10000 tolerances: relative=1e-08, absolute=1e-50, solution=1e-08 total number of linear solver iterations=0 total number of function evaluations=0 norm schedule ALWAYS SNESLineSearch Object: 2 MPI processes type: bt interpolation: cubic alpha=1.000000e-04 SNESLineSearch Object: 2 MPI processes type: bt interpolation: cubic alpha=1.000000e-04 maxstep=1.000000e+08, minlambda=1.000000e-12 tolerances: relative=1.000000e-08, absolute=1.000000e-15, lambda=1.000000e-08 maximum iterations=40 maxstep=1.000000e+08, minlambda=1.000000e-12 tolerances: relative=1.000000e-08, absolute=1.000000e-15, lambda=1.000000e-08 maximum iterations=40 KSP Object: 2 MPI processes type: gmres restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement KSP Object: 2 MPI processes type: gmres restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement happy breakdown tolerance 1e-30 maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using DEFAULT norm type for convergence test PC Object: 2 MPI processes happy breakdown tolerance 1e-30 maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using DEFAULT norm type for convergence test PC Object: 2 MPI processes type: bjacobi PC has not been set up so information may be incomplete number of blocks = -1 Local solve is same for all blocks, in the following KSP and PC objects: linear system matrix = precond matrix: type: bjacobi PC has not been set up so information may be incomplete number of blocks = -1 Local solve is same for all blocks, in the following KSP and PC objects: linear system matrix = precond matrix: Mat Object: 2 MPI processes type: mpiaij Mat Object: 2 MPI processes type: mpiaij rows=81, cols=81, bs=3 rows=81, cols=81, bs=3 total: nonzeros=3087, allocated nonzeros=3087 total number of mallocs used during MatSetValues calls=0 total: nonzeros=3087, allocated nonzeros=3087 total number of mallocs used during MatSetValues calls=0 using I-node (on process 0) routines: found 27 nodes, limit used is 5 not using I-node (on process 0) routines Question 1: Based on the result given by DMView (see (***) ), I didn't understand if my mesh is correctly distributed ? or my code is missing something ? because when I visualize the local and global sizes of my Jacobian matrix, I found PETSc::NonLinearSolver::INIT Size from Jac Matrix: M=81 m =0 //(M: global size, m: local size) this result is given by the proc 1 and PETSc::NonLinearSolver::INIT Size from Jac Matrix: M=81 m =81 // this result is given by the proc 2 Let me give other information: I create my jacobian matrix using: PetscErrorCode err = DMCreateMatrix(dm_mesh, &_matrix); and I use the PetscSection to tell the DM to use this data layout. In my code I wrote this line: DMSetFromOptions(dm_mesh); Then, to run my code I use mpirun -np 2 /home/benelhasa/fox_petsc/build_test/bin/Debug/FoXtroT -snes_test_jacobian_view -snes_converged_reason -snes_monitor -ksp_monitor -ksp_xmonitor -dm_view -petscpartitioner_type parmetis -dm_distribute -dm_refine 0 cub_8C3D8.fxt Question 2: I have another question, just for the comprehension. I understand that DMSetFromOptions(dm_mesh) allows me to use the parameter dm_distribute but I didn't understand how I can use -petscpartitioner_type parmetis argument (see this example that I used to help me (https://gitlab.com/petsc/petsc/-/blob/main/src/dm/impls/plex/tutorials/ex14.c), may be when the DM uses a data layout either by PetscSection or PetscDS, then I can use automatically the -petscpartitioner_type parmetis/simple/scotch ?? (Can you tell me more about this topic please) Ps: My solution is converging when I use 1 process and I don?t have problem. -- Dr. Sami BEN ELHAJ SALAH Ing?nieur de Recherche (CNRS) Institut Pprime - ISAE - ENSMA Mobile: 06.62.51.26.74 Email: sami.ben-elhaj-salah at ensma.fr www.samibenelhajsalah.com > Le 29 mai 2022 ? 18:02, Sami BEN ELHAJ SALAH a ?crit : > > Hi Matthew, > Thank you for this example. It seems exactly what I am looking for. > Thank you again for your help and have a good day. > Sami, > -- > Dr. Sami BEN ELHAJ SALAH > Ing?nieur de Recherche (CNRS) > Institut Pprime - ISAE - ENSMA > Mobile: 06.62.51.26.74 > Email: sami.ben-elhaj-salah at ensma.fr > www.samibenelhajsalah.com > > > >> Le 28 mai 2022 ? 20:20, Matthew Knepley > a ?crit : >> >> On Sat, May 28, 2022 at 2:19 PM Matthew Knepley > wrote: >> On Sat, May 28, 2022 at 1:35 PM Sami BEN ELHAJ SALAH > wrote: >> Hi Matthew, >> >> Thank you for your response. >> >> I don't have that. My DM object is not linked to PetscSection yet. I'll try that. >> Is there any example that manages this case? (DMPlexDistribute & PetscSection) or any guideline will be helpful. >> >> Here is an example where we create a section without DS. >> >> Forgot the link: https://gitlab.com/petsc/petsc/-/blob/main/src/dm/impls/plex/tutorials/ex14.c >> >> THanks, >> >> Matt >> >> Thanks in advance, >> >> Sami, >> >> -- >> Dr. Sami BEN ELHAJ SALAH >> Ing?nieur de Recherche (CNRS) >> Institut Pprime - ISAE - ENSMA >> Mobile: 06.62.51.26.74 >> Email: sami.ben-elhaj-salah at ensma.fr >> www.samibenelhajsalah.com >> >> >> >>> Le 27 mai 2022 ? 20:45, Matthew Knepley > a ?crit : >>> >>> On Fri, May 27, 2022 at 9:42 AM Sami BEN ELHAJ SALAH > wrote: >>> Hello Isaac, >>> >>> Thank you for your reply! >>> >>> Let me confirm that when I use DMCreateMatrix() with the orig_dm, I got my jacobian_matrix. Also, I have succeeded to solve my system and my solution was converged by using one process. >>> Let me give you some other information about my code. Currently, I am using my own discretization system and not the PetscDS object. Considering the nonlinear solver SNES, I use the following routines (like the basic snes usage): >>> - SNESSetFunction(snes,residual_vector,compute_residual ,(void*) _finite_element_formulation) >>> - SNESSetJacobian(snes, jacobian.matrix(), _jacobian_precond_matrix, compute_jacobian, (void*) _finite_element_formulation) >>> - SNESSolve(snes,NULL,x) >>> >>> Regarding your last answer, I will try to reformulate my question as follows: >>> Using a distributed dm instead of the original dm (not distributed dm) and my own discretization system (not the PetscDS object), >>> >>> You do not have to use PetscDS, but the DM does need to a PetscSection in order to compute sizes and sparsity patterns. Do you have that? >>> >>> Thanks, >>> >>> Matt >>> >>> should I add something specific to get a distributed jacobian_matrix over processes? >>> I precise that I just replaced the orig_dm by a distributed mesh with the routine that I wrote in my first mail. So is it enough ? >>> >>> Thank you and have a good day, >>> Sami, >>> >>> >>> -- >>> Dr. Sami BEN ELHAJ SALAH >>> Ing?nieur de Recherche (CNRS) >>> Institut Pprime - ISAE - ENSMA >>> Mobile: 06.62.51.26.74 >>> Email: sami.ben-elhaj-salah at ensma.fr >>> www.samibenelhajsalah.com >>> >>> >>> >>>> Le 25 mai 2022 ? 19:41, Toby Isaac > a ?crit : >>>> >>>> Hi Sami, >>>> >>>> Just to verify: if you call DMCreateMatrix() on orig_dm, do you get a >>>> Jacobian matrix? >>>> >>>> The DMPlex must be told what kind of discretized fields you want a >>>> matrix for and what equations you are discretizing. This is handled >>>> by the PetscDS object. In snes/tutorials/ex59.c, see the code after >>>> DMGetDS() for an example. >>>> >>>> - Toby >>>> >>>> On Wed, May 25, 2022 at 1:17 PM Sami BEN ELHAJ SALAH >>>> > wrote: >>>>> >>>>> Dear PETSc developer team, >>>>> >>>>> I m trying to create ? jacobian_matrix from distributed DM. I have followed the two examples (snes/tests/ex2.c and ex56.c). So I wrote this routine: >>>>> >>>>> PetscDM orig_dm; >>>>> PetscDM dist_dm = NULL; >>>>> PetscPartitioner part; >>>>> DMPlexGetPartitioner(orig_dm, &part); >>>>> PetscPartitionerSetType(part, PETSCPARTITIONERPARMETIS); >>>>> DMPlexDistribute(orig_dm, 0, NULL, &dist_dm); >>>>> >>>>> PetscErrorCode err = DMCreateMatrix(dist_dm, &jacobian_matrix); >>>>> PetscInt M, N, m, n; >>>>> MatGetSize(jacobian_matrix, &M, &N); >>>>> MatGetLocalSize(jacobian_matrix, &m, &n); >>>>> >>>>> Then I run my code with 2 processes and I obtained this result: >>>>> Size from jacobian_matrix: M=0 m =0 (this result is the same in all processes). >>>>> >>>>> I didn't understand if I forgot something in my code to obtain the correct values for the local and global sizes for the jacobean_matrix? (I mean if my code is missing something to obtain a distributed mesh over processes ?) >>>>> >>>>> Thank you in advance for any help! >>>>> Sami, >>>>> >>>>> >>>>> -- >>>>> Dr. Sami BEN ELHAJ SALAH >>>>> Ing?nieur de Recherche (CNRS) >>>>> Institut Pprime - ISAE - ENSMA >>>>> Mobile: 06.62.51.26.74 >>>>> Email: sami.ben-elhaj-salah at ensma.fr >>>>> www.samibenelhajsalah.com >>>>> >>>>> >>>>> >>> >>> >>> >>> -- >>> What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. >>> -- Norbert Wiener >>> >>> https://www.cse.buffalo.edu/~knepley/ >> >> >> >> -- >> What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. >> -- Norbert Wiener >> >> https://www.cse.buffalo.edu/~knepley/ >> >> >> -- >> What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. >> -- Norbert Wiener >> >> https://www.cse.buffalo.edu/~knepley/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Tue May 31 11:20:35 2022 From: knepley at gmail.com (Matthew Knepley) Date: Tue, 31 May 2022 12:20:35 -0400 Subject: [petsc-users] Question about DMPlexDistribute & distribute mesh over processes In-Reply-To: <556834F6-DAC5-46B3-9121-283E362F72AA@ensma.fr> References: <7E83D8A1-E911-4B48-98C9-BCFA9429AA11@ensma.fr> <15043588-F026-46C0-8334-E7ABA28F44FF@ensma.fr> <556834F6-DAC5-46B3-9121-283E362F72AA@ensma.fr> Message-ID: On Tue, May 31, 2022 at 12:08 PM Sami BEN ELHAJ SALAH < sami.ben-elhaj-salah at ensma.fr> wrote: > Hi Matthew, > > *Two question in this mail:* > > I tested my code *with 2 processes*. I use gmsh mesh( example: 8 > hexahedral elements, 27 nodes, and Global Size of my jacobian matrix = 81 > ). Then I used DMView(dm_mesh, PETSC_VIEWER_STDOUT_WORLD) to visualize my > DM and see if it is correctly distributed over processes, so I got this: > > *(***)* > *DM Object: DM_0x3_0 2 MPI processes* > * type: plex* > *DM_0x3_0 in 3 dimensions:* > * 0-cells: 27 0* > * 3-cells: 8 0* > Notice here that your whole mesh is on process 0. You would probably call DMPlexDistribute() to rebalance it. > *Labels:* > * material-id: 3 strata with value/size (66 (4), 67 (2), 68 (2))* > * depth: 2 strata with value/size (0 (27), 1 (8))* > * celltype: 2 strata with value/size (7 (8), 0 (27))* > > As you see, I created some material-id labels, so I found them over my 8 > cells when using DMView. So it seems good to me. The SNES viewer is shown > below: > > SNES Object: 2 MPI processes > type: ksponly > SNES has not been set up so information may be incomplete > maximum iterations=50, maximum function evaluations=10000 > tolerances: relative=1e-08, absolute=1e-50, solution=1e-08 > total number of linear solver iterations=0 > total number of function evaluations=0 > norm schedule ALWAYS > SNES Object: 2 MPI processes > type: ksponly > SNES has not been set up so information may be incomplete > maximum iterations=50, maximum function evaluations=10000 > tolerances: relative=1e-08, absolute=1e-50, solution=1e-08 > total number of linear solver iterations=0 > total number of function evaluations=0 > norm schedule ALWAYS > SNESLineSearch Object: 2 MPI processes > type: bt > interpolation: cubic > alpha=1.000000e-04 > SNESLineSearch Object: 2 MPI processes > type: bt > interpolation: cubic > alpha=1.000000e-04 > maxstep=1.000000e+08, minlambda=1.000000e-12 > tolerances: relative=1.000000e-08, absolute=1.000000e-15, > lambda=1.000000e-08 > maximum iterations=40 > maxstep=1.000000e+08, minlambda=1.000000e-12 > tolerances: relative=1.000000e-08, absolute=1.000000e-15, > lambda=1.000000e-08 > maximum iterations=40 > KSP Object: 2 MPI processes > type: gmres > restart=30, using Classical (unmodified) Gram-Schmidt > Orthogonalization with no iterative refinement > KSP Object: 2 MPI processes > type: gmres > restart=30, using Classical (unmodified) Gram-Schmidt > Orthogonalization with no iterative refinement > happy breakdown tolerance 1e-30 > maximum iterations=10000, initial guess is zero > tolerances: relative=1e-05, absolute=1e-50, divergence=10000. > left preconditioning > using DEFAULT norm type for convergence test > PC Object: 2 MPI processes > happy breakdown tolerance 1e-30 > maximum iterations=10000, initial guess is zero > tolerances: relative=1e-05, absolute=1e-50, divergence=10000. > left preconditioning > using DEFAULT norm type for convergence test > PC Object: 2 MPI processes > type: bjacobi > PC has not been set up so information may be incomplete > number of blocks = -1 > Local solve is same for all blocks, in the following KSP and PC > objects: > linear system matrix = precond matrix: > type: bjacobi > PC has not been set up so information may be incomplete > number of blocks = -1 > Local solve is same for all blocks, in the following KSP and PC > objects: > linear system matrix = precond matrix: > Mat Object: 2 MPI processes > type: mpiaij > Mat Object: 2 MPI processes > type: mpiaij > rows=81, cols=81, bs=3 > rows=81, cols=81, bs=3 > total: nonzeros=3087, allocated nonzeros=3087 > total number of mallocs used during MatSetValues calls=0 > total: nonzeros=3087, allocated nonzeros=3087 > total number of mallocs used during MatSetValues calls=0 > using I-node (on process 0) routines: found 27 nodes, limit used > is 5 > not using I-node (on process 0) routines > > *Question 1:* > *B**ased on the result given by DMView (see **(***)* *), **I didn't > understand if my mesh is correctly distributed **? or my code is missing > something ? because when I visualize the local and global sizes of my > Jacobian matrix, I found* > > *PETSc::NonLinearSolver::INIT Size from Jac Matrix: M=81 m =0 //(M: > global size, m: local size) this result is given by the proc 1* > *and * > *PETSc::NonLinearSolver::INIT Size from Jac Matrix: M=81 m =81 // **this > result is given by the proc 2* > Yes, your mesh in only on process 0. > Let me give other information: > I create my jacobian matrix using: > PetscErrorCode err = DMCreateMatrix(dm_mesh, &_matrix); > *and I use the PetscSection to tell the DM to use this data layout.* > > In my code I wrote this line: > DMSetFromOptions(dm_mesh); > Then, to run my code I use > mpirun -np 2 /home/benelhasa/fox_petsc/build_test/bin/Debug/FoXtroT > -snes_test_jacobian_view -snes_converged_reason -snes_monitor -ksp_monitor > -ksp_xmonitor -dm_view *-petscpartitioner_type parmetis -dm_distribute* > -dm_refine 0 cub_8C3D8.fxt > Something else is wrong in how this code is creating the mesh, because it is not distributing the mesh. Can you just use PetscCall(DMCreate(comm, dm)); PetscCall(DMSetType(*dm, DMPLEX)); PetscCall(DMSetFromOptions(*dm)); as we do in the examples and then -dm_plex_filename > *Question 2:* > *I have another question, just for the comprehension. I understand that > DMSetFromOptions(dm_mesh) allows me to use the parameter dm_distribute but > I didn't understand how I can use -petscpartitioner_type parmetis argument > (see this example that I used to help me (**https://gitlab.com/petsc/petsc/-/blob/main/src/dm/impls/plex/tutorials/ex14.c > **), > may be when the DM uses a data layout either by PetscSection or PetscDS, > then I can use automatically the -petscpartitioner_type > parmetis/simple/scotch ?? (Can you tell me more about this topic please) * > I do not yet understand the question. The partitioner only applies to the topology, not to functions on the mesh. Thanks, Matt > Ps: My solution is converging when I use 1 process and I don?t have > problem. > > -- > Dr. Sami BEN ELHAJ SALAH > Ing?nieur de Recherche (CNRS) > Institut Pprime - ISAE - ENSMA > Mobile: 06.62.51.26.74 > Email: sami.ben-elhaj-salah at ensma.fr > www.samibenelhajsalah.com > > > > > Le 29 mai 2022 ? 18:02, Sami BEN ELHAJ SALAH < > sami.ben-elhaj-salah at ensma.fr> a ?crit : > > Hi Matthew, > Thank you for this example. It seems exactly what I am looking for. > Thank you again for your help and have a good day. > Sami, > -- > Dr. Sami BEN ELHAJ SALAH > Ing?nieur de Recherche (CNRS) > Institut Pprime - ISAE - ENSMA > Mobile: 06.62.51.26.74 > Email: sami.ben-elhaj-salah at ensma.fr > www.samibenelhajsalah.com > > > > > Le 28 mai 2022 ? 20:20, Matthew Knepley a ?crit : > > On Sat, May 28, 2022 at 2:19 PM Matthew Knepley wrote: > >> On Sat, May 28, 2022 at 1:35 PM Sami BEN ELHAJ SALAH < >> sami.ben-elhaj-salah at ensma.fr> wrote: >> >>> Hi Matthew, >>> >>> Thank you for your response. >>> >>> I don't have that. My DM object is not linked to PetscSection yet. I'll >>> try that. >>> Is there any example that manages this case? (DMPlexDistribute & >>> PetscSection) or any guideline will be helpful. >>> >> >> Here is an example where we create a section without DS. >> > > Forgot the link: > https://gitlab.com/petsc/petsc/-/blob/main/src/dm/impls/plex/tutorials/ex14.c > > >> THanks, >> >> Matt >> >> >>> Thanks in advance, >>> >>> Sami, >>> >>> -- >>> Dr. Sami BEN ELHAJ SALAH >>> Ing?nieur de Recherche (CNRS) >>> Institut Pprime - ISAE - ENSMA >>> Mobile: 06.62.51.26.74 >>> Email: sami.ben-elhaj-salah at ensma.fr >>> www.samibenelhajsalah.com >>> >>> >>> >>> >>> Le 27 mai 2022 ? 20:45, Matthew Knepley a ?crit : >>> >>> On Fri, May 27, 2022 at 9:42 AM Sami BEN ELHAJ SALAH < >>> sami.ben-elhaj-salah at ensma.fr> wrote: >>> >>>> Hello Isaac, >>>> >>>> Thank you for your reply! >>>> >>>> Let me confirm that when I use DMCreateMatrix() with the orig_dm, I got >>>> my jacobian_matrix. Also, I have succeeded to solve my system and my >>>> solution was converged* by using one process*. >>>> Let me give you some other information about my code. Currently, I am >>>> using my own discretization system and not the PetscDS object. Considering >>>> the nonlinear solver SNES, I use the following routines (like the basic >>>> snes usage): >>>> - SNESSetFunction(snes,residual_vector,compute_residual ,(void*) >>>> _finite_element_formulation) >>>> - SNESSetJacobian(snes, jacobian.matrix(), _jacobian_precond_matrix, >>>> compute_jacobian, (void*) _finite_element_formulation) >>>> - SNESSolve(snes,NULL,x) >>>> >>>> Regarding your last answer, I will try to reformulate my question as >>>> follows: >>>> Using a distributed dm instead of the original dm (not distributed dm) >>>> and my own discretization system (not the PetscDS object), >>>> >>> >>> You do not have to use PetscDS, but the DM does need to a PetscSection >>> in order to compute sizes and sparsity patterns. Do you have that? >>> >>> Thanks, >>> >>> Matt >>> >>> >>>> should I add something specific to get a distributed jacobian_matrix >>>> over processes? >>>> I precise that I just replaced the orig_dm by a distributed mesh with >>>> the routine that I wrote in my first mail. So is it enough ? >>>> >>>> Thank you and have a good day, >>>> Sami, >>>> >>>> >>>> -- >>>> Dr. Sami BEN ELHAJ SALAH >>>> Ing?nieur de Recherche (CNRS) >>>> Institut Pprime - ISAE - ENSMA >>>> Mobile: 06.62.51.26.74 >>>> Email: sami.ben-elhaj-salah at ensma.fr >>>> www.samibenelhajsalah.com >>>> >>>> >>>> >>>> >>>> Le 25 mai 2022 ? 19:41, Toby Isaac a ?crit : >>>> >>>> Hi Sami, >>>> >>>> Just to verify: if you call DMCreateMatrix() on orig_dm, do you get a >>>> Jacobian matrix? >>>> >>>> The DMPlex must be told what kind of discretized fields you want a >>>> matrix for and what equations you are discretizing. This is handled >>>> by the PetscDS object. In snes/tutorials/ex59.c, see the code after >>>> DMGetDS() for an example. >>>> >>>> - Toby >>>> >>>> On Wed, May 25, 2022 at 1:17 PM Sami BEN ELHAJ SALAH >>>> wrote: >>>> >>>> >>>> Dear PETSc developer team, >>>> >>>> I m trying to create ? jacobian_matrix from distributed DM. I have >>>> followed the two examples (snes/tests/ex2.c and ex56.c). So I wrote this >>>> routine: >>>> >>>> PetscDM orig_dm; >>>> PetscDM dist_dm = NULL; >>>> PetscPartitioner part; >>>> DMPlexGetPartitioner(orig_dm, &part); >>>> PetscPartitionerSetType(part, PETSCPARTITIONERPARMETIS); >>>> DMPlexDistribute(orig_dm, 0, NULL, &dist_dm); >>>> >>>> PetscErrorCode err = DMCreateMatrix(dist_dm, &jacobian_matrix); >>>> PetscInt M, N, m, n; >>>> MatGetSize(jacobian_matrix, &M, &N); >>>> MatGetLocalSize(jacobian_matrix, &m, &n); >>>> >>>> Then I run my code with 2 processes and I obtained this result: >>>> Size from jacobian_matrix: M=0 m =0 (this result is the same in all >>>> processes). >>>> >>>> I didn't understand if I forgot something in my code to obtain the >>>> correct values for the local and global sizes for the jacobean_matrix? (I >>>> mean if my code is missing something to obtain a distributed mesh over >>>> processes ?) >>>> >>>> Thank you in advance for any help! >>>> Sami, >>>> >>>> >>>> -- >>>> Dr. Sami BEN ELHAJ SALAH >>>> Ing?nieur de Recherche (CNRS) >>>> Institut Pprime - ISAE - ENSMA >>>> Mobile: 06.62.51.26.74 >>>> Email: sami.ben-elhaj-salah at ensma.fr >>>> www.samibenelhajsalah.com >>>> >>>> >>>> >>>> >>>> >>> >>> -- >>> What most experimenters take for granted before they begin their >>> experiments is infinitely more interesting than any results to which their >>> experiments lead. >>> -- Norbert Wiener >>> >>> https://www.cse.buffalo.edu/~knepley/ >>> >>> >>> >>> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which their >> experiments lead. >> -- Norbert Wiener >> >> https://www.cse.buffalo.edu/~knepley/ >> >> > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > > https://www.cse.buffalo.edu/~knepley/ > > > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From bantingl at myumanitoba.ca Tue May 31 12:28:33 2022 From: bantingl at myumanitoba.ca (Lucas Banting) Date: Tue, 31 May 2022 17:28:33 +0000 Subject: [petsc-users] Accelerating eigenvalue computation / removing portion of spectrum Message-ID: Hello, I have a general non hermitian eigenvalue problem arising from the 3D helmholtz equation. The form of the helmholtz equaton is: (S - k^2M)v = lambda k^2 M v Where S is the stiffness/curl-curl matrix and M is the mass matrix associated with edge elements used to discretize the problem. The helmholtz equation creates eigenvalues of -1.0, which I believe are eigenvectors that are part of the null space of the curl-curl operator S. For my application, I would like to compute eigenvalues > -1.0, and avoid computation of eigenvalues of -1.0. I am currently using shift invert ST with mumps LU direct solver. By increasing the shift away from lambda=-1.0. I get faster computation of eigenvectors, and the lambda=-1.0 eigenvectors appear to slow down the computation by about a factor of two. Is there a way to avoid these lambda = -1.0 eigenpairs with a GNHEP problem type? Regards, Lucas -------------- next part -------------- An HTML attachment was scrubbed... URL: From jroman at dsic.upv.es Tue May 31 13:11:42 2022 From: jroman at dsic.upv.es (Jose E. Roman) Date: Tue, 31 May 2022 20:11:42 +0200 Subject: [petsc-users] Accelerating eigenvalue computation / removing portion of spectrum In-Reply-To: References: Message-ID: <7E80B1DF-0F06-4EFB-99BA-63471F55165D@dsic.upv.es> If you know how to cheaply compute a basis of the nullspace of S, then you can try passing it to the solver via EPSSetDeflationSpace() https://slepc.upv.es/documentation/current/docs/manualpages/EPS/EPSSetDeflationSpace.html Jose > El 31 may 2022, a las 19:28, Lucas Banting escribi?: > > Hello, > > I have a general non hermitian eigenvalue problem arising from the 3D helmholtz equation. > The form of the helmholtz equaton is: > > (S - k^2M)v = lambda k^2 M v > > Where S is the stiffness/curl-curl matrix and M is the mass matrix associated with edge elements used to discretize the problem. > The helmholtz equation creates eigenvalues of -1.0, which I believe are eigenvectors that are part of the null space of the curl-curl operator S. > > For my application, I would like to compute eigenvalues > -1.0, and avoid computation of eigenvalues of -1.0. > I am currently using shift invert ST with mumps LU direct solver. By increasing the shift away from lambda=-1.0. I get faster computation of eigenvectors, and the lambda=-1.0 eigenvectors appear to slow down the computation by about a factor of two. > Is there a way to avoid these lambda = -1.0 eigenpairs with a GNHEP problem type? > > Regards, > Lucas From sami.ben-elhaj-salah at ensma.fr Tue May 31 13:18:49 2022 From: sami.ben-elhaj-salah at ensma.fr (Sami BEN ELHAJ SALAH) Date: Tue, 31 May 2022 20:18:49 +0200 Subject: [petsc-users] Question about DMPlexDistribute & distribute mesh over processes In-Reply-To: References: <7E83D8A1-E911-4B48-98C9-BCFA9429AA11@ensma.fr> <15043588-F026-46C0-8334-E7ABA28F44FF@ensma.fr> <556834F6-DAC5-46B3-9121-283E362F72AA@ensma.fr> Message-ID: <566D01C9-2463-4533-9F5E-0AAF1856FD02@ensma.fr> Thank you very much ! I have succeed to distribute the mesh by this using this routine. PetscDM dmdist = NULL; PetscPartitioner part; DMPlexGetPartitioner(dm_mesh, &part); PetscPartitionerSetType(part, PETSCPARTITIONERPARMETIS); PetscPartitionerSetFromOptions(part); DMPlexDistribute(dm_mesh, 0, NULL, &dmdist); if (dmdist) { DMDestroy(&dm_mesh); dm_mesh = dmdist; } DMView(dm_mesh, PETSC_VIEWER_STDOUT_WORLD); Th result is given as below: DM Object: Parallel Mesh 4 MPI processes type: plex Parallel Mesh in 3 dimensions: 0-cells: 12 12 12 12 3-cells: 2 2 2 2 Labels: depth: 2 strata with value/size (0 (12), 1 (2)) material-id: 1 strata with value/size (68 (2)) Thank you a lot for your help and have a good day, Sami -- Dr. Sami BEN ELHAJ SALAH Ing?nieur de Recherche (CNRS) Institut Pprime - ISAE - ENSMA Mobile: 06.62.51.26.74 Email: sami.ben-elhaj-salah at ensma.fr www.samibenelhajsalah.com > Le 31 mai 2022 ? 18:20, Matthew Knepley a ?crit : > > On Tue, May 31, 2022 at 12:08 PM Sami BEN ELHAJ SALAH > wrote: > Hi Matthew, > > Two question in this mail: > > I tested my code with 2 processes. I use gmsh mesh( example: 8 hexahedral elements, 27 nodes, and Global Size of my jacobian matrix = 81 ). Then I used DMView(dm_mesh, PETSC_VIEWER_STDOUT_WORLD) to visualize my DM and see if it is correctly distributed over processes, so I got this: > > (***) > DM Object: DM_0x3_0 2 MPI processes > type: plex > DM_0x3_0 in 3 dimensions: > 0-cells: 27 0 > 3-cells: 8 0 > > Notice here that your whole mesh is on process 0. You would probably call DMPlexDistribute() to rebalance it. > > Labels: > material-id: 3 strata with value/size (66 (4), 67 (2), 68 (2)) > depth: 2 strata with value/size (0 (27), 1 (8)) > celltype: 2 strata with value/size (7 (8), 0 (27)) > > As you see, I created some material-id labels, so I found them over my 8 cells when using DMView. So it seems good to me. The SNES viewer is shown below: > > SNES Object: 2 MPI processes > type: ksponly > SNES has not been set up so information may be incomplete > maximum iterations=50, maximum function evaluations=10000 > tolerances: relative=1e-08, absolute=1e-50, solution=1e-08 > total number of linear solver iterations=0 > total number of function evaluations=0 > norm schedule ALWAYS > SNES Object: 2 MPI processes > type: ksponly > SNES has not been set up so information may be incomplete > maximum iterations=50, maximum function evaluations=10000 > tolerances: relative=1e-08, absolute=1e-50, solution=1e-08 > total number of linear solver iterations=0 > total number of function evaluations=0 > norm schedule ALWAYS > SNESLineSearch Object: 2 MPI processes > type: bt > interpolation: cubic > alpha=1.000000e-04 > SNESLineSearch Object: 2 MPI processes > type: bt > interpolation: cubic > alpha=1.000000e-04 > maxstep=1.000000e+08, minlambda=1.000000e-12 > tolerances: relative=1.000000e-08, absolute=1.000000e-15, lambda=1.000000e-08 > maximum iterations=40 > maxstep=1.000000e+08, minlambda=1.000000e-12 > tolerances: relative=1.000000e-08, absolute=1.000000e-15, lambda=1.000000e-08 > maximum iterations=40 > KSP Object: 2 MPI processes > type: gmres > restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement > KSP Object: 2 MPI processes > type: gmres > restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement > happy breakdown tolerance 1e-30 > maximum iterations=10000, initial guess is zero > tolerances: relative=1e-05, absolute=1e-50, divergence=10000. > left preconditioning > using DEFAULT norm type for convergence test > PC Object: 2 MPI processes > happy breakdown tolerance 1e-30 > maximum iterations=10000, initial guess is zero > tolerances: relative=1e-05, absolute=1e-50, divergence=10000. > left preconditioning > using DEFAULT norm type for convergence test > PC Object: 2 MPI processes > type: bjacobi > PC has not been set up so information may be incomplete > number of blocks = -1 > Local solve is same for all blocks, in the following KSP and PC objects: > linear system matrix = precond matrix: > type: bjacobi > PC has not been set up so information may be incomplete > number of blocks = -1 > Local solve is same for all blocks, in the following KSP and PC objects: > linear system matrix = precond matrix: > Mat Object: 2 MPI processes > type: mpiaij > Mat Object: 2 MPI processes > type: mpiaij > rows=81, cols=81, bs=3 > rows=81, cols=81, bs=3 > total: nonzeros=3087, allocated nonzeros=3087 > total number of mallocs used during MatSetValues calls=0 > total: nonzeros=3087, allocated nonzeros=3087 > total number of mallocs used during MatSetValues calls=0 > using I-node (on process 0) routines: found 27 nodes, limit used is 5 > not using I-node (on process 0) routines > > Question 1: > Based on the result given by DMView (see (***) ), I didn't understand if my mesh is correctly distributed ? or my code is missing something ? because when I visualize the local and global sizes of my Jacobian matrix, I found > > PETSc::NonLinearSolver::INIT Size from Jac Matrix: M=81 m =0 //(M: global size, m: local size) this result is given by the proc 1 > and > PETSc::NonLinearSolver::INIT Size from Jac Matrix: M=81 m =81 // this result is given by the proc 2 > > Yes, your mesh in only on process 0. > > Let me give other information: > I create my jacobian matrix using: > PetscErrorCode err = DMCreateMatrix(dm_mesh, &_matrix); > and I use the PetscSection to tell the DM to use this data layout. > > In my code I wrote this line: > DMSetFromOptions(dm_mesh); > Then, to run my code I use > mpirun -np 2 /home/benelhasa/fox_petsc/build_test/bin/Debug/FoXtroT -snes_test_jacobian_view -snes_converged_reason -snes_monitor -ksp_monitor -ksp_xmonitor -dm_view -petscpartitioner_type parmetis -dm_distribute -dm_refine 0 cub_8C3D8.fxt > > Something else is wrong in how this code is creating the mesh, because it is not distributing the mesh. Can you just use > > PetscCall(DMCreate(comm, dm)); > PetscCall(DMSetType(*dm, DMPLEX)); > PetscCall(DMSetFromOptions(*dm)); > > as we do in the examples and then > > -dm_plex_filename > > Question 2: > I have another question, just for the comprehension. I understand that DMSetFromOptions(dm_mesh) allows me to use the parameter dm_distribute but I didn't understand how I can use -petscpartitioner_type parmetis argument (see this example that I used to help me (https://gitlab.com/petsc/petsc/-/blob/main/src/dm/impls/plex/tutorials/ex14.c ), may be when the DM uses a data layout either by PetscSection or PetscDS, then I can use automatically the -petscpartitioner_type parmetis/simple/scotch ?? (Can you tell me more about this topic please) > > I do not yet understand the question. The partitioner only applies to the topology, not to functions on the mesh. > > Thanks, > > Matt > > Ps: My solution is converging when I use 1 process and I don?t have problem. > > -- > Dr. Sami BEN ELHAJ SALAH > Ing?nieur de Recherche (CNRS) > Institut Pprime - ISAE - ENSMA > Mobile: 06.62.51.26.74 > Email: sami.ben-elhaj-salah at ensma.fr > www.samibenelhajsalah.com > > > >> Le 29 mai 2022 ? 18:02, Sami BEN ELHAJ SALAH > a ?crit : >> >> Hi Matthew, >> Thank you for this example. It seems exactly what I am looking for. >> Thank you again for your help and have a good day. >> Sami, >> -- >> Dr. Sami BEN ELHAJ SALAH >> Ing?nieur de Recherche (CNRS) >> Institut Pprime - ISAE - ENSMA >> Mobile: 06.62.51.26.74 >> Email: sami.ben-elhaj-salah at ensma.fr >> www.samibenelhajsalah.com >> >> >> >>> Le 28 mai 2022 ? 20:20, Matthew Knepley > a ?crit : >>> >>> On Sat, May 28, 2022 at 2:19 PM Matthew Knepley > wrote: >>> On Sat, May 28, 2022 at 1:35 PM Sami BEN ELHAJ SALAH > wrote: >>> Hi Matthew, >>> >>> Thank you for your response. >>> >>> I don't have that. My DM object is not linked to PetscSection yet. I'll try that. >>> Is there any example that manages this case? (DMPlexDistribute & PetscSection) or any guideline will be helpful. >>> >>> Here is an example where we create a section without DS. >>> >>> Forgot the link: https://gitlab.com/petsc/petsc/-/blob/main/src/dm/impls/plex/tutorials/ex14.c >>> >>> THanks, >>> >>> Matt >>> >>> Thanks in advance, >>> >>> Sami, >>> >>> -- >>> Dr. Sami BEN ELHAJ SALAH >>> Ing?nieur de Recherche (CNRS) >>> Institut Pprime - ISAE - ENSMA >>> Mobile: 06.62.51.26.74 >>> Email: sami.ben-elhaj-salah at ensma.fr >>> www.samibenelhajsalah.com >>> >>> >>> >>>> Le 27 mai 2022 ? 20:45, Matthew Knepley > a ?crit : >>>> >>>> On Fri, May 27, 2022 at 9:42 AM Sami BEN ELHAJ SALAH > wrote: >>>> Hello Isaac, >>>> >>>> Thank you for your reply! >>>> >>>> Let me confirm that when I use DMCreateMatrix() with the orig_dm, I got my jacobian_matrix. Also, I have succeeded to solve my system and my solution was converged by using one process. >>>> Let me give you some other information about my code. Currently, I am using my own discretization system and not the PetscDS object. Considering the nonlinear solver SNES, I use the following routines (like the basic snes usage): >>>> - SNESSetFunction(snes,residual_vector,compute_residual ,(void*) _finite_element_formulation) >>>> - SNESSetJacobian(snes, jacobian.matrix(), _jacobian_precond_matrix, compute_jacobian, (void*) _finite_element_formulation) >>>> - SNESSolve(snes,NULL,x) >>>> >>>> Regarding your last answer, I will try to reformulate my question as follows: >>>> Using a distributed dm instead of the original dm (not distributed dm) and my own discretization system (not the PetscDS object), >>>> >>>> You do not have to use PetscDS, but the DM does need to a PetscSection in order to compute sizes and sparsity patterns. Do you have that? >>>> >>>> Thanks, >>>> >>>> Matt >>>> >>>> should I add something specific to get a distributed jacobian_matrix over processes? >>>> I precise that I just replaced the orig_dm by a distributed mesh with the routine that I wrote in my first mail. So is it enough ? >>>> >>>> Thank you and have a good day, >>>> Sami, >>>> >>>> >>>> -- >>>> Dr. Sami BEN ELHAJ SALAH >>>> Ing?nieur de Recherche (CNRS) >>>> Institut Pprime - ISAE - ENSMA >>>> Mobile: 06.62.51.26.74 >>>> Email: sami.ben-elhaj-salah at ensma.fr >>>> www.samibenelhajsalah.com >>>> >>>> >>>> >>>>> Le 25 mai 2022 ? 19:41, Toby Isaac > a ?crit : >>>>> >>>>> Hi Sami, >>>>> >>>>> Just to verify: if you call DMCreateMatrix() on orig_dm, do you get a >>>>> Jacobian matrix? >>>>> >>>>> The DMPlex must be told what kind of discretized fields you want a >>>>> matrix for and what equations you are discretizing. This is handled >>>>> by the PetscDS object. In snes/tutorials/ex59.c, see the code after >>>>> DMGetDS() for an example. >>>>> >>>>> - Toby >>>>> >>>>> On Wed, May 25, 2022 at 1:17 PM Sami BEN ELHAJ SALAH >>>>> > wrote: >>>>>> >>>>>> Dear PETSc developer team, >>>>>> >>>>>> I m trying to create ? jacobian_matrix from distributed DM. I have followed the two examples (snes/tests/ex2.c and ex56.c). So I wrote this routine: >>>>>> >>>>>> PetscDM orig_dm; >>>>>> PetscDM dist_dm = NULL; >>>>>> PetscPartitioner part; >>>>>> DMPlexGetPartitioner(orig_dm, &part); >>>>>> PetscPartitionerSetType(part, PETSCPARTITIONERPARMETIS); >>>>>> DMPlexDistribute(orig_dm, 0, NULL, &dist_dm); >>>>>> >>>>>> PetscErrorCode err = DMCreateMatrix(dist_dm, &jacobian_matrix); >>>>>> PetscInt M, N, m, n; >>>>>> MatGetSize(jacobian_matrix, &M, &N); >>>>>> MatGetLocalSize(jacobian_matrix, &m, &n); >>>>>> >>>>>> Then I run my code with 2 processes and I obtained this result: >>>>>> Size from jacobian_matrix: M=0 m =0 (this result is the same in all processes). >>>>>> >>>>>> I didn't understand if I forgot something in my code to obtain the correct values for the local and global sizes for the jacobean_matrix? (I mean if my code is missing something to obtain a distributed mesh over processes ?) >>>>>> >>>>>> Thank you in advance for any help! >>>>>> Sami, >>>>>> >>>>>> >>>>>> -- >>>>>> Dr. Sami BEN ELHAJ SALAH >>>>>> Ing?nieur de Recherche (CNRS) >>>>>> Institut Pprime - ISAE - ENSMA >>>>>> Mobile: 06.62.51.26.74 >>>>>> Email: sami.ben-elhaj-salah at ensma.fr >>>>>> www.samibenelhajsalah.com >>>>>> >>>>>> >>>>>> >>>> >>>> >>>> >>>> -- >>>> What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. >>>> -- Norbert Wiener >>>> >>>> https://www.cse.buffalo.edu/~knepley/ >>> >>> >>> >>> -- >>> What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. >>> -- Norbert Wiener >>> >>> https://www.cse.buffalo.edu/~knepley/ >>> >>> >>> -- >>> What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. >>> -- Norbert Wiener >>> >>> https://www.cse.buffalo.edu/~knepley/ > > > > -- > What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. > -- Norbert Wiener > > https://www.cse.buffalo.edu/~knepley/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From jroman at dsic.upv.es Tue May 31 14:08:59 2022 From: jroman at dsic.upv.es (Jose E. Roman) Date: Tue, 31 May 2022 21:08:59 +0200 Subject: [petsc-users] Accelerating eigenvalue computation / removing portion of spectrum In-Reply-To: References: <7E80B1DF-0F06-4EFB-99BA-63471F55165D@dsic.upv.es> Message-ID: <62743559-BA54-4828-B0D8-B84111C2E1EA@dsic.upv.es> Please respond to the list also. The problem with EPSSetDeflationSpace() is that it internally orthogonalizes the vectors that you pass in, so it is not viable for thousands of vectors. You can try implementing any of the alternative schemes described in https://doi.org/10.1002/nla.307 Another thing you can try is to use a region for filtering, as explained in section 2.6.4 of the users manual. Use a region that excludes -1.0 and you will have more chances to get the wanted eigenvalues faster. But still convergence may be slow. Jose > El 31 may 2022, a las 20:52, Lucas Banting escribi?: > > Thanks for the response Jose, > > There is an analytical solution for these modes actually, however there are thousands of them and they are all sparse. > I assume it is a non-trivial thing for EPSSetDeflationSpace() to take something like a MATAIJ as input? > > Lucas > From: Jose E. Roman > Sent: Tuesday, May 31, 2022 1:11 PM > To: Lucas Banting > Cc: petsc-users at mcs.anl.gov > Subject: Re: [petsc-users] Accelerating eigenvalue computation / removing portion of spectrum > > Caution: This message was sent from outside the University of Manitoba. > > > If you know how to cheaply compute a basis of the nullspace of S, then you can try passing it to the solver via EPSSetDeflationSpace()https://slepc.upv.es/documentation/current/docs/manualpages/EPS/EPSSetDeflationSpace.html > > Jose > > > > El 31 may 2022, a las 19:28, Lucas Banting escribi?: > > > > Hello, > > > > I have a general non hermitian eigenvalue problem arising from the 3D helmholtz equation. > > The form of the helmholtz equaton is: > > > > (S - k^2M)v = lambda k^2 M v > > > > Where S is the stiffness/curl-curl matrix and M is the mass matrix associated with edge elements used to discretize the problem. > > The helmholtz equation creates eigenvalues of -1.0, which I believe are eigenvectors that are part of the null space of the curl-curl operator S. > > > > For my application, I would like to compute eigenvalues > -1.0, and avoid computation of eigenvalues of -1.0. > > I am currently using shift invert ST with mumps LU direct solver. By increasing the shift away from lambda=-1.0. I get faster computation of eigenvectors, and the lambda=-1.0 eigenvectors appear to slow down the computation by about a factor of two. > > Is there a way to avoid these lambda = -1.0 eigenpairs with a GNHEP problem type? > > > > Regards, > > Lucas From bantingl at myumanitoba.ca Tue May 31 15:13:25 2022 From: bantingl at myumanitoba.ca (Lucas Banting) Date: Tue, 31 May 2022 20:13:25 +0000 Subject: [petsc-users] Accelerating eigenvalue computation / removing portion of spectrum In-Reply-To: <62743559-BA54-4828-B0D8-B84111C2E1EA@dsic.upv.es> References: <7E80B1DF-0F06-4EFB-99BA-63471F55165D@dsic.upv.es> <62743559-BA54-4828-B0D8-B84111C2E1EA@dsic.upv.es> Message-ID: Thanks for the sharing the article. For my application, I think using an interval region to exclude the unneeded eigenvalues will still be faster than forming a larger constrained system. Specifying an interval appears to run in a similar amount of time. Lucas ________________________________ From: Jose E. Roman Sent: Tuesday, May 31, 2022 2:08 PM To: Lucas Banting Cc: PETSc Subject: Re: [petsc-users] Accelerating eigenvalue computation / removing portion of spectrum Caution: This message was sent from outside the University of Manitoba. Please respond to the list also. The problem with EPSSetDeflationSpace() is that it internally orthogonalizes the vectors that you pass in, so it is not viable for thousands of vectors. You can try implementing any of the alternative schemes described in https://doi.org/10.1002/nla.307 Another thing you can try is to use a region for filtering, as explained in section 2.6.4 of the users manual. Use a region that excludes -1.0 and you will have more chances to get the wanted eigenvalues faster. But still convergence may be slow. Jose > El 31 may 2022, a las 20:52, Lucas Banting escribi?: > > Thanks for the response Jose, > > There is an analytical solution for these modes actually, however there are thousands of them and they are all sparse. > I assume it is a non-trivial thing for EPSSetDeflationSpace() to take something like a MATAIJ as input? > > Lucas > From: Jose E. Roman > Sent: Tuesday, May 31, 2022 1:11 PM > To: Lucas Banting > Cc: petsc-users at mcs.anl.gov > Subject: Re: [petsc-users] Accelerating eigenvalue computation / removing portion of spectrum > > Caution: This message was sent from outside the University of Manitoba. > > > If you know how to cheaply compute a basis of the nullspace of S, then you can try passing it to the solver via EPSSetDeflationSpace()https://slepc.upv.es/documentation/current/docs/manualpages/EPS/EPSSetDeflationSpace.html > > Jose > > > > El 31 may 2022, a las 19:28, Lucas Banting escribi?: > > > > Hello, > > > > I have a general non hermitian eigenvalue problem arising from the 3D helmholtz equation. > > The form of the helmholtz equaton is: > > > > (S - k^2M)v = lambda k^2 M v > > > > Where S is the stiffness/curl-curl matrix and M is the mass matrix associated with edge elements used to discretize the problem. > > The helmholtz equation creates eigenvalues of -1.0, which I believe are eigenvectors that are part of the null space of the curl-curl operator S. > > > > For my application, I would like to compute eigenvalues > -1.0, and avoid computation of eigenvalues of -1.0. > > I am currently using shift invert ST with mumps LU direct solver. By increasing the shift away from lambda=-1.0. I get faster computation of eigenvectors, and the lambda=-1.0 eigenvectors appear to slow down the computation by about a factor of two. > > Is there a way to avoid these lambda = -1.0 eigenpairs with a GNHEP problem type? > > > > Regards, > > Lucas -------------- next part -------------- An HTML attachment was scrubbed... URL: From jroman at dsic.upv.es Tue May 31 15:28:36 2022 From: jroman at dsic.upv.es (Jose E. Roman) Date: Tue, 31 May 2022 22:28:36 +0200 Subject: [petsc-users] Accelerating eigenvalue computation / removing portion of spectrum In-Reply-To: References: <7E80B1DF-0F06-4EFB-99BA-63471F55165D@dsic.upv.es> <62743559-BA54-4828-B0D8-B84111C2E1EA@dsic.upv.es> Message-ID: Probably MUMPS is taking most of the time... If the matrices are not too large, send them to my personal email and I will have a look. Jose > El 31 may 2022, a las 22:13, Lucas Banting escribi?: > > Thanks for the sharing the article. > For my application, I think using an interval region to exclude the unneeded eigenvalues will still be faster than forming a larger constrained system. Specifying an interval appears to run in a similar amount of time. > > Lucas > From: Jose E. Roman > Sent: Tuesday, May 31, 2022 2:08 PM > To: Lucas Banting > Cc: PETSc > Subject: Re: [petsc-users] Accelerating eigenvalue computation / removing portion of spectrum > > Caution: This message was sent from outside the University of Manitoba. > > > Please respond to the list also. > > The problem with EPSSetDeflationSpace() is that it internally orthogonalizes the vectors that you pass in, so it is not viable for thousands of vectors. > > You can try implementing any of the alternative schemes described in https://doi.org/10.1002/nla.307 > > Another thing you can try is to use a region for filtering, as explained in section 2.6.4 of the users manual. Use a region that excludes -1.0 and you will have more chances to get the wanted eigenvalues faster. But still convergence may be slow. > > Jose > > > > El 31 may 2022, a las 20:52, Lucas Banting escribi?: > > > > Thanks for the response Jose, > > > > There is an analytical solution for these modes actually, however there are thousands of them and they are all sparse. > > I assume it is a non-trivial thing for EPSSetDeflationSpace() to take something like a MATAIJ as input? > > > > Lucas > > From: Jose E. Roman > > Sent: Tuesday, May 31, 2022 1:11 PM > > To: Lucas Banting > > Cc: petsc-users at mcs.anl.gov > > Subject: Re: [petsc-users] Accelerating eigenvalue computation / removing portion of spectrum > > > > Caution: This message was sent from outside the University of Manitoba. > > > > > > If you know how to cheaply compute a basis of the nullspace of S, then you can try passing it to the solver via EPSSetDeflationSpace()https://slepc.upv.es/documentation/current/docs/manualpages/EPS/EPSSetDeflationSpace.html > > > > Jose > > > > > > > El 31 may 2022, a las 19:28, Lucas Banting escribi?: > > > > > > Hello, > > > > > > I have a general non hermitian eigenvalue problem arising from the 3D helmholtz equation. > > > The form of the helmholtz equaton is: > > > > > > (S - k^2M)v = lambda k^2 M v > > > > > > Where S is the stiffness/curl-curl matrix and M is the mass matrix associated with edge elements used to discretize the problem. > > > The helmholtz equation creates eigenvalues of -1.0, which I believe are eigenvectors that are part of the null space of the curl-curl operator S. > > > > > > For my application, I would like to compute eigenvalues > -1.0, and avoid computation of eigenvalues of -1.0. > > > I am currently using shift invert ST with mumps LU direct solver. By increasing the shift away from lambda=-1.0. I get faster computation of eigenvectors, and the lambda=-1.0 eigenvectors appear to slow down the computation by about a factor of two. > > > Is there a way to avoid these lambda = -1.0 eigenpairs with a GNHEP problem type? > > > > > > Regards, > > > Lucas From Ye_Changqing at outlook.com Tue May 31 22:08:52 2022 From: Ye_Changqing at outlook.com (Ye Changqing) Date: Wed, 1 Jun 2022 03:08:52 +0000 Subject: [petsc-users] =?gb2312?b?u9i4tDogIE1hdCBjcmVhdGVkIGJ5IERNU3Rh?= =?gb2312?b?ZyBjYW5ub3QgYWNjZXNzIGdob3N0IHBvaW50cw==?= In-Reply-To: References: Message-ID: Dear all, [BugReport.c] is a sample code, [BugReportParallel.output] is the output when execute BugReport with mpiexec, [BugReportSerial.output] is the output in serial execution. Best, Changqing ________________________________ ???: Dave May ????: 2022?5?31? 22:55 ???: Ye Changqing ??: petsc-users at mcs.anl.gov ??: Re: [petsc-users] Mat created by DMStag cannot access ghost points On Tue 31. May 2022 at 16:28, Ye Changqing > wrote: Dear developers of PETSc, I encountered a problem when using the DMStag module. The program could be executed perfectly in serial, while errors are thrown out in parallel (using mpiexec). Some rows in Mat cannot be accessed in local processes when looping all elements in DMStag. The DM object I used only has one DOF in each element. Hence, I could switch to the DMDA module easily, and the program now is back to normal. Some snippets are below. Initialise a DMStag object: PetscCall(DMStagCreate2d(PETSC_COMM_WORLD, DM_BOUNDARY_NONE, DM_BOUNDARY_NONE, M, N, PETSC_DECIDE, PETSC_DECIDE, 0, 0, 1, DMSTAG_STENCIL_BOX, 1, NULL, NULL, &(s_ctx->dm_P))); Created a Mat: PetscCall(DMCreateMatrix(s_ctx->dm_P, A)); Loop: PetscCall(DMStagGetCorners(s_ctx->dm_V, &startx, &starty, &startz, &nx, &ny, &nz, &extrax, &extray, &extraz)); for (ey = starty; ey < starty + ny; ++ey) for (ex = startx; ex < startx + nx; ++ex) { ... PetscCall(DMStagMatSetValuesStencil(s_ctx->dm_P, *A, 2, &row[0], 2, &col[0], &val_A[0][0], ADD_VALUES)); // The traceback shows the problem is in here. } In addition to the code or MWE, please forward us the complete stack trace / error thrown to stdout. Thanks, Dave Best, Changqing -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: BugReport.c URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: BugReportParallel.output Type: application/octet-stream Size: 4033 bytes Desc: BugReportParallel.output URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: BugReportSerial.output Type: application/octet-stream Size: 31 bytes Desc: BugReportSerial.output URL: