From Eric.Chamberland at giref.ulaval.ca Mon Dec 2 16:14:40 2024 From: Eric.Chamberland at giref.ulaval.ca (Eric Chamberland) Date: Mon, 2 Dec 2024 17:14:40 -0500 Subject: [petsc-users] Using MatCreateComposite with petsc4py Message-ID: Hello, We were exploring the use of the |MatCreateComposite| function with |petsc4py|, but we noticed that it doesn't seem to be available in the Python interface (https://urldefense.us/v3/__https://petsc.org/release/petsc4py/reference/petsc4py.PETSc.Mat.html__;!!G_uCfscf7eWS!fphemsDPNmxHUr-ybYLt_xWUrP575eKuI4xixwypEzd2g7uc6qMilt8etg13o24gU0vloOVrsHDXwqlgoY1T7jrcQUiv83PrXvNSYlNU$ ). We were wondering if this is due to a technical limitation or if there is a better alternative to achieve the same result. Our goal is to create a matrix representing PTAP without explicitly computing the product. We also considered using |MatShell|, but it seems that this approach is not supported in |petsc4py| either. On the other hand, we noticed that |MatCreateTranspose| is implemented, and it aligns somewhat with the kind of operation we're aiming for. Any insights or advice you could provide would be greatly appreciated! Thank you, Eric -- Eric Chamberland, ing., M. Ing Professionnel de recherche GIREF/Universit? Laval (418) 656-2131 poste 41 22 42 -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Mon Dec 2 16:32:36 2024 From: knepley at gmail.com (Matthew Knepley) Date: Mon, 2 Dec 2024 17:32:36 -0500 Subject: [petsc-users] Using MatCreateComposite with petsc4py In-Reply-To: References: Message-ID: On Mon, Dec 2, 2024 at 5:14?PM Eric Chamberland via petsc-users < petsc-users at mcs.anl.gov> wrote: > Hello, > > We were exploring the use of the MatCreateComposite function with petsc4py, > but we noticed that it doesn't seem to be available in the Python interface > (https://urldefense.us/v3/__https://petsc.org/release/petsc4py/reference/petsc4py.PETSc.Mat.html__;!!G_uCfscf7eWS!aHjys_axrlOjrR_fWK2NdV-rCkGLXuEuWPoyrHbLp_01Vvr48evkik5dT077BtK20PuTlFz8mMpKHhBMaM2o$ > > ). > > We were wondering if this is due to a technical limitation or if there is > a better alternative to achieve the same result. > No, we just forgot to put in this interface. It should be done. It does not look hard, cutting and pasting from other functions. Thanks, Matt > Our goal is to create a matrix representing PTAP without explicitly > computing the product. > > We also considered using MatShell, but it seems that this approach is not > supported in petsc4py either. > > On the other hand, we noticed that MatCreateTranspose is implemented, and > it aligns somewhat with the kind of operation we're aiming for. > > Any insights or advice you could provide would be greatly appreciated! > > Thank you, > > Eric > > -- > Eric Chamberland, ing., M. Ing > Professionnel de recherche > GIREF/Universit? Laval > (418) 656-2131 poste 41 22 42 > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://urldefense.us/v3/__https://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!aHjys_axrlOjrR_fWK2NdV-rCkGLXuEuWPoyrHbLp_01Vvr48evkik5dT077BtK20PuTlFz8mMpKHripB8Lo$ -------------- next part -------------- An HTML attachment was scrubbed... URL: From pierre at joliv.et Tue Dec 3 00:05:24 2024 From: pierre at joliv.et (Pierre Jolivet) Date: Tue, 3 Dec 2024 07:05:24 +0100 Subject: [petsc-users] Using MatCreateComposite with petsc4py In-Reply-To: References: Message-ID: <51EBBA48-7967-447F-9200-536E59CD8587@joliv.et> An HTML attachment was scrubbed... URL: From stefano.zampini at gmail.com Tue Dec 3 00:36:49 2024 From: stefano.zampini at gmail.com (Stefano Zampini) Date: Tue, 3 Dec 2024 09:36:49 +0300 Subject: [petsc-users] Using MatCreateComposite with petsc4py In-Reply-To: <51EBBA48-7967-447F-9200-536E59CD8587@joliv.et> References: <51EBBA48-7967-447F-9200-536E59CD8587@joliv.et> Message-ID: Eric the python equivalent of a MATSHELL is MATPYTHON, where you are in control of the implementation of the matrix vector product and other operations, see https://urldefense.us/v3/__https://petsc.org/release/petsc4py/petsc_python_types.html*petsc-python-matrix-type__;Iw!!G_uCfscf7eWS!cBWJhNKIAgapcwHEU8n-DdtvG1b4efLyBq4hz-hD3u12q055vU-37qhdbu7HsyS9_9yUlAmmIPPCBqfdlCsXdy-GASOOnmY$ This should cover your needs, let me know if you need methods that are not wrapped Il giorno mar 3 dic 2024 alle ore 09:15 Pierre Jolivet ha scritto: > > > On 2 Dec 2024, at 11:15?PM, Eric Chamberland via petsc-users < > petsc-users at mcs.anl.gov> wrote: > > ? > > Hello, > > We were exploring the use of the MatCreateComposite function with petsc4py, > but we noticed that it doesn't seem to be available in the Python interface > (https://urldefense.us/v3/__https://petsc.org/release/petsc4py/reference/petsc4py.PETSc.Mat.html__;!!G_uCfscf7eWS!cBWJhNKIAgapcwHEU8n-DdtvG1b4efLyBq4hz-hD3u12q055vU-37qhdbu7HsyS9_9yUlAmmIPPCBqfdlCsXdy-GYsA6br0$ > > ). > > We were wondering if this is due to a technical limitation or if there is > a better alternative to achieve the same result. > > Our goal is to create a matrix representing PTAP without explicitly > computing the product. > > We also considered using MatShell, but it seems that this approach is not > supported in petsc4py either. > > What do you mean? > You can use either a MatShell or a MatPython in petsc4py out of the box. > > Thanks, > Pierre > > On the other hand, we noticed that MatCreateTranspose is implemented, and > it aligns somewhat with the kind of operation we're aiming for. > > Any insights or advice you could provide would be greatly appreciated! > > Thank you, > > Eric > > -- > Eric Chamberland, ing., M. Ing > Professionnel de recherche > GIREF/Universit? Laval > (418) 656-2131 poste 41 22 42 > > -- Stefano -------------- next part -------------- An HTML attachment was scrubbed... URL: From Eric.Chamberland at giref.ulaval.ca Tue Dec 3 15:33:17 2024 From: Eric.Chamberland at giref.ulaval.ca (Eric Chamberland) Date: Tue, 3 Dec 2024 16:33:17 -0500 Subject: [petsc-users] Using MatCreateComposite with petsc4py In-Reply-To: References: <51EBBA48-7967-447F-9200-536E59CD8587@joliv.et> Message-ID: <6b61b2da-af1b-470f-9406-48cd3a0989a7@giref.ulaval.ca> Hi, great!? Thank you all for this information!? Indeed MATPYTHON cover our needs! :) Eric On 2024-12-03 01:36, Stefano Zampini wrote: > Eric > > the python equivalent of a MATSHELL is MATPYTHON, where you are in > control of the implementation of the matrix vector product and other > operations, see > https://urldefense.us/v3/__https://petsc.org/release/petsc4py/petsc_python_types.html*petsc-python-matrix-type__;Iw!!G_uCfscf7eWS!ZD3aymPzmAu2afLqaAuAYv18OwPDU-6XWUzuj2lTZ7oyjCjYPx39HcSblHApdBD0BtxMNmBq8_BssfuDCbR1d2QRvp0ds5fft1-XkhiS$ > > This should cover your needs, let me know if you need methods that are > not wrapped > > Il giorno mar 3 dic 2024 alle ore 09:15 Pierre Jolivet > ha scritto: > > > >> On 2 Dec 2024, at 11:15?PM, Eric Chamberland via petsc-users >> wrote: >> >> ? >> >> Hello, >> >> We were exploring the use of the |MatCreateComposite| function >> with |petsc4py|, but we noticed that it doesn't seem to be >> available in the Python interface >> (https://urldefense.us/v3/__https://petsc.org/release/petsc4py/reference/petsc4py.PETSc.Mat.html__;!!G_uCfscf7eWS!ZD3aymPzmAu2afLqaAuAYv18OwPDU-6XWUzuj2lTZ7oyjCjYPx39HcSblHApdBD0BtxMNmBq8_BssfuDCbR1d2QRvp0ds5fftzVa96TG$ >> ). >> >> We were wondering if this is due to a technical limitation or if >> there is a better alternative to achieve the same result. >> >> Our goal is to create a matrix representing PTAP without >> explicitly computing the product. >> >> We also considered using |MatShell|, but it seems that this >> approach is not supported in |petsc4py| either. >> > What do you mean? > You can use either a MatShell or a MatPython in petsc4py out of > the box. > > Thanks, > Pierre >> >> On the other hand, we noticed that |MatCreateTranspose| is >> implemented, and it aligns somewhat with the kind of operation >> we're aiming for. >> >> Any insights or advice you could provide would be greatly >> appreciated! >> >> Thank you, >> >> Eric >> >> -- >> Eric Chamberland, ing., M. Ing >> Professionnel de recherche >> GIREF/Universit? Laval >> (418) 656-2131 poste 41 22 42 > > > > -- > Stefano -- Eric Chamberland, ing., M. Ing Professionnel de recherche GIREF/Universit? Laval (418) 656-2131 poste 41 22 42 -------------- next part -------------- An HTML attachment was scrubbed... URL: From sebastian.blauth at itwm.fraunhofer.de Wed Dec 4 03:44:13 2024 From: sebastian.blauth at itwm.fraunhofer.de (Blauth, Sebastian) Date: Wed, 4 Dec 2024 09:44:13 +0000 Subject: [petsc-users] Setting up nonlinear preconditioning with petsc4py Message-ID: Hi everyone, I wanted to try nonlinear preconditioning for solving nonlinear systems. The problems arise from PDEs discretized with FEniCS and I have successfully implemented a wrapper for SNES. However, once I want to try nonlinear preconditioning by adding the option ?-npc_snes_type newtonls? (or other solvers such as nrichardson), I get the error code 56. My approach looks something like this from petsc4py import PETSc snes = PETSc.SNES().create() snes.setFunction(self.assemble_function, self.residual_petsc) snes.setJacobian(self.assemble_jacobian, self.A_petsc, self.P_petsc) snes.setFromOptions() snes.solve(None, self.u.vector().vec()) snes.destroy() I don?t think that my user defined functions are problematic. The code runs well with the options (just an example) -snes_type newtonls -snes_rtol 1e-6 -snes_monitor -ksp_type gmres -ksp_monitor_true_residual -pc_type lu -pc_factor_mat_solver_type mumps -mat_mumps_icntl_24 Here, the gmres is just used to verify that the direct solver works as expected. Newton Krylov methods also work well. However, once I use -snes_type nrichardson -npc_snes_type newtonls -npc_snes_max_it 4 (as discussed e.g. in https://climatemodeling.org/~rmills/talks/PSU-ACM-seminar-2024.pdf) I get the error message File "/p/tv/blauths/cashocs/cashocs/nonlinear_solvers/snes.py", line 250, in solve snes.solve(None, self.u.vector().vec()) File "petsc4py/PETSc/SNES.pyx", line 1555, in petsc4py.PETSc.SNES.solve petsc4py.PETSc.Error: error code 56 I would be really grateful if someone could point me to the right direction on how to use nonlinear precondition with petsc4py. Thanks a lot in advance, Sebastian -- Dr. Sebastian Blauth Fraunhofer-Institut f?r Techno- und Wirtschaftsmathematik ITWM Abteilung Transportvorg?nge Fraunhofer-Platz 1, 67663 Kaiserslautern Telefon: +49 631 31600-4968 sebastian.blauth at itwm.fraunhofer.de https://www.itwm.fraunhofer.de -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 7943 bytes Desc: not available URL: From knepley at gmail.com Wed Dec 4 06:47:54 2024 From: knepley at gmail.com (Matthew Knepley) Date: Wed, 4 Dec 2024 07:47:54 -0500 Subject: [petsc-users] Setting up nonlinear preconditioning with petsc4py In-Reply-To: References: Message-ID: On Wed, Dec 4, 2024 at 4:44?AM Blauth, Sebastian < sebastian.blauth at itwm.fraunhofer.de> wrote: > Hi everyone, > > > > I wanted to try nonlinear preconditioning for solving nonlinear systems. > The problems arise from PDEs discretized with FEniCS and I have > successfully implemented a wrapper for SNES. However, once I want to try > nonlinear preconditioning by adding the option ?-npc_snes_type newtonls? > (or other solvers such as nrichardson), I get the error code 56. > > > > My approach looks something like this > > > > from petsc4py import PETSc > > > > snes = PETSc.SNES().create() > > snes.setFunction(self.assemble_function, self.residual_petsc) > > snes.setJacobian(self.assemble_jacobian, self.A_petsc, self.P_petsc) > > snes.setFromOptions() > > > > snes.solve(None, self.u.vector().vec()) > > snes.destroy() > > > > I don?t think that my user defined functions are problematic. The code > runs well with the options (just an example) > > -snes_type newtonls > > -snes_rtol 1e-6 > > -snes_monitor > > -ksp_type gmres > > -ksp_monitor_true_residual > > -pc_type lu > > -pc_factor_mat_solver_type mumps > > -mat_mumps_icntl_24 > > > > Here, the gmres is just used to verify that the direct solver works as > expected. Newton Krylov methods also work well. > > However, once I use > > > > -snes_type nrichardson > > -npc_snes_type newtonls > > -npc_snes_max_it 4 > > > > (as discussed e.g. in > https://urldefense.us/v3/__https://climatemodeling.org/*rmills/talks/PSU-ACM-seminar-2024.pdf__;fg!!G_uCfscf7eWS!ZpQxakVxVcyih0WUbW_MxFNHXR0tUbZAvis8z497q782yqbJraYBBCMst79xJm6nlGdBUqW3I2vIDAtJiUTR$ ) I get > the error message > > > > File "/p/tv/blauths/cashocs/cashocs/nonlinear_solvers/snes.py", line 250, > in solve > > snes.solve(None, self.u.vector().vec()) > > File "petsc4py/PETSc/SNES.pyx", line 1555, in petsc4py.PETSc.SNES.solve > > petsc4py.PETSc.Error: error code 56 > > > > I would be really grateful if someone could point me to the right > direction on how to use nonlinear precondition with petsc4py. > 56 is PETSC_ERR_SUP, which means we hit an unsupported operation. petsc4py preserves the stack trace, so FEniCS should not be throwing it away (along with the error message). Maybe run in the debugger so we can see the error message and stack? Thanks, Matt > Thanks a lot in advance, > > Sebastian > > > > -- > > Dr. Sebastian Blauth > > Fraunhofer-Institut f?r > > Techno- und Wirtschaftsmathematik ITWM > > Abteilung Transportvorg?nge > > Fraunhofer-Platz 1, 67663 Kaiserslautern > > Telefon: +49 631 31600-4968 > > sebastian.blauth at itwm.fraunhofer.de > > https://urldefense.us/v3/__https://www.itwm.fraunhofer.de__;!!G_uCfscf7eWS!ZpQxakVxVcyih0WUbW_MxFNHXR0tUbZAvis8z497q782yqbJraYBBCMst79xJm6nlGdBUqW3I2vIDJ9nKGZK$ > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://urldefense.us/v3/__https://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!ZpQxakVxVcyih0WUbW_MxFNHXR0tUbZAvis8z497q782yqbJraYBBCMst79xJm6nlGdBUqW3I2vIDOFKYWl_$ -------------- next part -------------- An HTML attachment was scrubbed... URL: From stefano.zampini at gmail.com Wed Dec 4 06:55:58 2024 From: stefano.zampini at gmail.com (Stefano Zampini) Date: Wed, 4 Dec 2024 15:55:58 +0300 Subject: [petsc-users] Setting up nonlinear preconditioning with petsc4py In-Reply-To: References: Message-ID: FENICS swallows any useful error information returned from PETSc. You can try using the below code snippet at the beginning of your script right after you loaded dolfin stuff PETSc.Sys.pushErrorHandler('python') On Wed, Dec 4, 2024, 15:48 Matthew Knepley wrote: > On Wed, Dec 4, 2024 at 4:44?AM Blauth, Sebastian < > sebastian.blauth at itwm.fraunhofer.de> wrote: > >> Hi everyone, >> >> >> >> I wanted to try nonlinear preconditioning for solving nonlinear systems. >> The problems arise from PDEs discretized with FEniCS and I have >> successfully implemented a wrapper for SNES. However, once I want to try >> nonlinear preconditioning by adding the option ?-npc_snes_type newtonls? >> (or other solvers such as nrichardson), I get the error code 56. >> >> >> >> My approach looks something like this >> >> >> >> from petsc4py import PETSc >> >> >> >> snes = PETSc.SNES().create() >> >> snes.setFunction(self.assemble_function, self.residual_petsc) >> >> snes.setJacobian(self.assemble_jacobian, self.A_petsc, self.P_petsc) >> >> snes.setFromOptions() >> >> >> >> snes.solve(None, self.u.vector().vec()) >> >> snes.destroy() >> >> >> >> I don?t think that my user defined functions are problematic. The code >> runs well with the options (just an example) >> >> -snes_type newtonls >> >> -snes_rtol 1e-6 >> >> -snes_monitor >> >> -ksp_type gmres >> >> -ksp_monitor_true_residual >> >> -pc_type lu >> >> -pc_factor_mat_solver_type mumps >> >> -mat_mumps_icntl_24 >> >> >> >> Here, the gmres is just used to verify that the direct solver works as >> expected. Newton Krylov methods also work well. >> >> However, once I use >> >> >> >> -snes_type nrichardson >> >> -npc_snes_type newtonls >> >> -npc_snes_max_it 4 >> >> >> >> (as discussed e.g. in >> https://urldefense.us/v3/__https://climatemodeling.org/*rmills/talks/PSU-ACM-seminar-2024.pdf__;fg!!G_uCfscf7eWS!ddLXEuCg9AG1oNXdkiB6pY7sY3ZOpEaDWNyhYKwKS2P8nbA0H1AFyYPQBQSJrJcMQ45JRCpHPnnWfFvp-POAMMAmyadHT5M$ >> ) >> I get the error message >> >> >> >> File "/p/tv/blauths/cashocs/cashocs/nonlinear_solvers/snes.py", line 250, >> in solve >> >> snes.solve(None, self.u.vector().vec()) >> >> File "petsc4py/PETSc/SNES.pyx", line 1555, in petsc4py.PETSc.SNES.solve >> >> petsc4py.PETSc.Error: error code 56 >> >> >> >> I would be really grateful if someone could point me to the right >> direction on how to use nonlinear precondition with petsc4py. >> > > 56 is PETSC_ERR_SUP, which means we hit an unsupported operation. petsc4py > preserves the stack trace, so FEniCS should not be throwing it away (along > with the error message). Maybe run in the debugger so we can see the error > message and stack? > > Thanks, > > Matt > > >> Thanks a lot in advance, >> >> Sebastian >> >> >> >> -- >> >> Dr. Sebastian Blauth >> >> Fraunhofer-Institut f?r >> >> Techno- und Wirtschaftsmathematik ITWM >> >> Abteilung Transportvorg?nge >> >> Fraunhofer-Platz 1, 67663 Kaiserslautern >> >> >> Telefon: +49 631 31600-4968 >> >> sebastian.blauth at itwm.fraunhofer.de >> >> https://urldefense.us/v3/__https://www.itwm.fraunhofer.de__;!!G_uCfscf7eWS!ddLXEuCg9AG1oNXdkiB6pY7sY3ZOpEaDWNyhYKwKS2P8nbA0H1AFyYPQBQSJrJcMQ45JRCpHPnnWfFvp-POAMMAmdaZkhYY$ >> >> >> >> > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > > https://urldefense.us/v3/__https://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!ddLXEuCg9AG1oNXdkiB6pY7sY3ZOpEaDWNyhYKwKS2P8nbA0H1AFyYPQBQSJrJcMQ45JRCpHPnnWfFvp-POAMMAmc_3ZJg0$ > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sebastian.blauth at itwm.fraunhofer.de Wed Dec 4 07:09:42 2024 From: sebastian.blauth at itwm.fraunhofer.de (Blauth, Sebastian) Date: Wed, 4 Dec 2024 13:09:42 +0000 Subject: [petsc-users] Setting up nonlinear preconditioning with petsc4py In-Reply-To: References: Message-ID: Thanks a lot for the hint. With this I could find the error myself: I had not initialized the residual vector correctly, which has to be duplicated, hence the error. Assembling this once with FEniCS fixed the error and I can now use nonlinear preconditioning. Thanks a lot! Best, Sebastian -- Dr. Sebastian Blauth Fraunhofer-Institut f?r Techno- und Wirtschaftsmathematik ITWM Abteilung Transportvorg?nge Fraunhofer-Platz 1, 67663 Kaiserslautern Telefon: +49 631 31600-4968 sebastian.blauth at itwm.fraunhofer.de https://www.itwm.fraunhofer.de From: Stefano Zampini Sent: Wednesday, December 4, 2024 1:56 PM To: Matthew Knepley Cc: Blauth, Sebastian ; PETSc users list Subject: Re: [petsc-users] Setting up nonlinear preconditioning with petsc4py FENICS swallows any useful error information returned from PETSc. You can try using the below code snippet at the beginning of your script right after you loaded dolfin stuff PETSc.Sys.pushErrorHandler('python') On Wed, Dec 4, 2024, 15:48 Matthew Knepley > wrote: On Wed, Dec 4, 2024 at 4:44?AM Blauth, Sebastian > wrote: Hi everyone, I wanted to try nonlinear preconditioning for solving nonlinear systems. The problems arise from PDEs discretized with FEniCS and I have successfully implemented a wrapper for SNES. However, once I want to try nonlinear preconditioning by adding the option ?-npc_snes_type newtonls? (or other solvers such as nrichardson), I get the error code 56. My approach looks something like this from petsc4py import PETSc snes = PETSc.SNES().create() snes.setFunction(self.assemble_function, self.residual_petsc) snes.setJacobian(self.assemble_jacobian, self.A_petsc, self.P_petsc) snes.setFromOptions() snes.solve(None, self.u.vector().vec()) snes.destroy() I don?t think that my user defined functions are problematic. The code runs well with the options (just an example) -snes_type newtonls -snes_rtol 1e-6 -snes_monitor -ksp_type gmres -ksp_monitor_true_residual -pc_type lu -pc_factor_mat_solver_type mumps -mat_mumps_icntl_24 Here, the gmres is just used to verify that the direct solver works as expected. Newton Krylov methods also work well. However, once I use -snes_type nrichardson -npc_snes_type newtonls -npc_snes_max_it 4 (as discussed e.g. in https://climatemodeling.org/~rmills/talks/PSU-ACM-seminar-2024.pdf ) I get the error message File "/p/tv/blauths/cashocs/cashocs/nonlinear_solvers/snes.py", line 250, in solve snes.solve(None, self.u.vector().vec()) File "petsc4py/PETSc/SNES.pyx", line 1555, in petsc4py.PETSc.SNES.solve petsc4py.PETSc.Error: error code 56 I would be really grateful if someone could point me to the right direction on how to use nonlinear precondition with petsc4py. 56 is PETSC_ERR_SUP, which means we hit an unsupported operation. petsc4py preserves the stack trace, so FEniCS should not be throwing it away (along with the error message). Maybe run in the debugger so we can see the error message and stack? Thanks, Matt Thanks a lot in advance, Sebastian -- Dr. Sebastian Blauth Fraunhofer-Institut f?r Techno- und Wirtschaftsmathematik ITWM Abteilung Transportvorg?nge Fraunhofer-Platz 1, 67663 Kaiserslautern Telefon: +49 631 31600-4968 sebastian.blauth at itwm.fraunhofer.de https://www.itwm.fraunhofer.de -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 7943 bytes Desc: not available URL: From knepley at gmail.com Wed Dec 4 08:51:21 2024 From: knepley at gmail.com (Matthew Knepley) Date: Wed, 4 Dec 2024 09:51:21 -0500 Subject: [petsc-users] Setting up nonlinear preconditioning with petsc4py In-Reply-To: References: Message-ID: On Wed, Dec 4, 2024 at 8:09?AM Blauth, Sebastian < sebastian.blauth at itwm.fraunhofer.de> wrote: > Thanks a lot for the hint. With this I could find the error myself: I had > not initialized the residual vector correctly, which has to be duplicated, > hence the error. Assembling this once with FEniCS fixed the error and I can > now use nonlinear preconditioning. Thanks a lot! > I would be very interested to hear how it works if you have the time to mail us back. Thanks Matt > > > Best, > > Sebastian > > > > -- > > Dr. Sebastian Blauth > > Fraunhofer-Institut f?r > > Techno- und Wirtschaftsmathematik ITWM > > Abteilung Transportvorg?nge > > Fraunhofer-Platz 1, 67663 Kaiserslautern > > Telefon: +49 631 31600-4968 > > sebastian.blauth at itwm.fraunhofer.de > > https://urldefense.us/v3/__https://www.itwm.fraunhofer.de__;!!G_uCfscf7eWS!apTDPPZWCcPl0SpwhG_O2c_qShB9EsoiAl8XyoHxoNowvDTMyo8-w17OoKH1i5zqmYbvLI9l0EMpe17p3IXT$ > > > > *From:* Stefano Zampini > *Sent:* Wednesday, December 4, 2024 1:56 PM > *To:* Matthew Knepley > *Cc:* Blauth, Sebastian ; PETSc > users list > *Subject:* Re: [petsc-users] Setting up nonlinear preconditioning with > petsc4py > > > > FENICS swallows any useful error information returned from PETSc. You can > try using the below code snippet at the beginning of your script right > after you loaded dolfin stuff > > > > > > PETSc.Sys.pushErrorHandler('python') > > > > On Wed, Dec 4, 2024, 15:48 Matthew Knepley wrote: > > On Wed, Dec 4, 2024 at 4:44?AM Blauth, Sebastian < > sebastian.blauth at itwm.fraunhofer.de> wrote: > > Hi everyone, > > > > I wanted to try nonlinear preconditioning for solving nonlinear systems. > The problems arise from PDEs discretized with FEniCS and I have > successfully implemented a wrapper for SNES. However, once I want to try > nonlinear preconditioning by adding the option ?-npc_snes_type newtonls? > (or other solvers such as nrichardson), I get the error code 56. > > > > My approach looks something like this > > > > from petsc4py import PETSc > > > > snes = PETSc.SNES().create() > > snes.setFunction(self.assemble_function, self.residual_petsc) > > snes.setJacobian(self.assemble_jacobian, self.A_petsc, self.P_petsc) > > snes.setFromOptions() > > > > snes.solve(None, self.u.vector().vec()) > > snes.destroy() > > > > I don?t think that my user defined functions are problematic. The code > runs well with the options (just an example) > > -snes_type newtonls > > -snes_rtol 1e-6 > > -snes_monitor > > -ksp_type gmres > > -ksp_monitor_true_residual > > -pc_type lu > > -pc_factor_mat_solver_type mumps > > -mat_mumps_icntl_24 > > > > Here, the gmres is just used to verify that the direct solver works as > expected. Newton Krylov methods also work well. > > However, once I use > > > > -snes_type nrichardson > > -npc_snes_type newtonls > > -npc_snes_max_it 4 > > > > (as discussed e.g. in > https://urldefense.us/v3/__https://climatemodeling.org/*rmills/talks/PSU-ACM-seminar-2024.pdf__;fg!!G_uCfscf7eWS!apTDPPZWCcPl0SpwhG_O2c_qShB9EsoiAl8XyoHxoNowvDTMyo8-w17OoKH1i5zqmYbvLI9l0EMpe83o8hLf$ > ) > I get the error message > > > > File "/p/tv/blauths/cashocs/cashocs/nonlinear_solvers/snes.py", line 250, > in solve > > snes.solve(None, self.u.vector().vec()) > > File "petsc4py/PETSc/SNES.pyx", line 1555, in petsc4py.PETSc.SNES.solve > > petsc4py.PETSc.Error: error code 56 > > > > I would be really grateful if someone could point me to the right > direction on how to use nonlinear precondition with petsc4py. > > > > 56 is PETSC_ERR_SUP, which means we hit an unsupported operation. petsc4py > preserves the stack trace, so FEniCS should not be throwing it away (along > with the error message). Maybe run in the debugger so we can see the error > message and stack? > > > > Thanks, > > > > Matt > > > > Thanks a lot in advance, > > Sebastian > > > > -- > > Dr. Sebastian Blauth > > Fraunhofer-Institut f?r > > Techno- und Wirtschaftsmathematik ITWM > > Abteilung Transportvorg?nge > > Fraunhofer-Platz 1, 67663 Kaiserslautern > > > Telefon: +49 631 31600-4968 > > sebastian.blauth at itwm.fraunhofer.de > > https://urldefense.us/v3/__https://www.itwm.fraunhofer.de__;!!G_uCfscf7eWS!apTDPPZWCcPl0SpwhG_O2c_qShB9EsoiAl8XyoHxoNowvDTMyo8-w17OoKH1i5zqmYbvLI9l0EMpe17p3IXT$ > > > > > > > > -- > > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > > > > https://urldefense.us/v3/__https://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!apTDPPZWCcPl0SpwhG_O2c_qShB9EsoiAl8XyoHxoNowvDTMyo8-w17OoKH1i5zqmYbvLI9l0EMpezolAj75$ > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://urldefense.us/v3/__https://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!apTDPPZWCcPl0SpwhG_O2c_qShB9EsoiAl8XyoHxoNowvDTMyo8-w17OoKH1i5zqmYbvLI9l0EMpezolAj75$ -------------- next part -------------- An HTML attachment was scrubbed... URL: From sebastian.blauth at itwm.fraunhofer.de Thu Dec 5 01:11:01 2024 From: sebastian.blauth at itwm.fraunhofer.de (Blauth, Sebastian) Date: Thu, 5 Dec 2024 07:11:01 +0000 Subject: [petsc-users] Setting up nonlinear preconditioning with petsc4py In-Reply-To: References: Message-ID: Sure, I can even share the relevant parts of the code as this is part of my project cashocs. The related PR can be found here: https://github.com/sblauth/cashocs/pull/530 The relevant part of the SNES solver setup with FEniCS is here: https://github.com/sblauth/cashocs/blob/main/cashocs/nonlinear_solvers/snes.py#L235 There, I realized that FEniCS by default generates empty matrices and vectors which will have the correct size and sparsity pattern only after they have been assembled at least once. But I noticed that if I did only assemble them during the call to snes.solve, then, e.g., using MUMPS as solver caused issues. If I preassemble the matrices, everything works fine. And the same goes for the residual, which is what caused the issues with the nonlinear preconditioning. Best, Sebastian -- Dr. Sebastian Blauth Fraunhofer-Institut f?r Techno- und Wirtschaftsmathematik ITWM Abteilung Transportvorg?nge Fraunhofer-Platz 1, 67663 Kaiserslautern Telefon: +49 631 31600-4968 sebastian.blauth at itwm.fraunhofer.de https://www.itwm.fraunhofer.de From: Matthew Knepley Sent: Wednesday, December 4, 2024 3:51 PM To: Blauth, Sebastian Cc: Stefano Zampini ; PETSc users list Subject: Re: [petsc-users] Setting up nonlinear preconditioning with petsc4py On Wed, Dec 4, 2024 at 8:09?AM Blauth, Sebastian > wrote: Thanks a lot for the hint. With this I could find the error myself: I had not initialized the residual vector correctly, which has to be duplicated, hence the error. Assembling this once with FEniCS fixed the error and I can now use nonlinear preconditioning. Thanks a lot! I would be very interested to hear how it works if you have the time to mail us back. Thanks Matt Best, Sebastian -- Dr. Sebastian Blauth Fraunhofer-Institut f?r Techno- und Wirtschaftsmathematik ITWM Abteilung Transportvorg?nge Fraunhofer-Platz 1, 67663 Kaiserslautern Telefon: +49 631 31600-4968 sebastian.blauth at itwm.fraunhofer.de https://www.itwm.fraunhofer.de From: Stefano Zampini > Sent: Wednesday, December 4, 2024 1:56 PM To: Matthew Knepley > Cc: Blauth, Sebastian >; PETSc users list > Subject: Re: [petsc-users] Setting up nonlinear preconditioning with petsc4py FENICS swallows any useful error information returned from PETSc. You can try using the below code snippet at the beginning of your script right after you loaded dolfin stuff PETSc.Sys.pushErrorHandler('python') On Wed, Dec 4, 2024, 15:48 Matthew Knepley > wrote: On Wed, Dec 4, 2024 at 4:44?AM Blauth, Sebastian > wrote: Hi everyone, I wanted to try nonlinear preconditioning for solving nonlinear systems. The problems arise from PDEs discretized with FEniCS and I have successfully implemented a wrapper for SNES. However, once I want to try nonlinear preconditioning by adding the option ?-npc_snes_type newtonls? (or other solvers such as nrichardson), I get the error code 56. My approach looks something like this from petsc4py import PETSc snes = PETSc.SNES().create() snes.setFunction(self.assemble_function, self.residual_petsc) snes.setJacobian(self.assemble_jacobian, self.A_petsc, self.P_petsc) snes.setFromOptions() snes.solve(None, self.u.vector().vec()) snes.destroy() I don?t think that my user defined functions are problematic. The code runs well with the options (just an example) -snes_type newtonls -snes_rtol 1e-6 -snes_monitor -ksp_type gmres -ksp_monitor_true_residual -pc_type lu -pc_factor_mat_solver_type mumps -mat_mumps_icntl_24 Here, the gmres is just used to verify that the direct solver works as expected. Newton Krylov methods also work well. However, once I use -snes_type nrichardson -npc_snes_type newtonls -npc_snes_max_it 4 (as discussed e.g. in https://climatemodeling.org/~rmills/talks/PSU-ACM-seminar-2024.pdf ) I get the error message File "/p/tv/blauths/cashocs/cashocs/nonlinear_solvers/snes.py", line 250, in solve snes.solve(None, self.u.vector().vec()) File "petsc4py/PETSc/SNES.pyx", line 1555, in petsc4py.PETSc.SNES.solve petsc4py.PETSc.Error: error code 56 I would be really grateful if someone could point me to the right direction on how to use nonlinear precondition with petsc4py. 56 is PETSC_ERR_SUP, which means we hit an unsupported operation. petsc4py preserves the stack trace, so FEniCS should not be throwing it away (along with the error message). Maybe run in the debugger so we can see the error message and stack? Thanks, Matt Thanks a lot in advance, Sebastian -- Dr. Sebastian Blauth Fraunhofer-Institut f?r Techno- und Wirtschaftsmathematik ITWM Abteilung Transportvorg?nge Fraunhofer-Platz 1, 67663 Kaiserslautern Telefon: +49 631 31600-4968 sebastian.blauth at itwm.fraunhofer.de https://www.itwm.fraunhofer.de -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 7943 bytes Desc: not available URL: From qiyuelu1 at gmail.com Sat Dec 7 20:47:56 2024 From: qiyuelu1 at gmail.com (Qiyue Lu) Date: Sat, 7 Dec 2024 20:47:56 -0600 Subject: [petsc-users] VecScatter() causing processes hanging Message-ID: Hello, I am trying to fetch 12 entries from a distributed vector global_vec and put them into a local vector on each process. IS is_source, is_dest; ISCreateGeneral(PETSC_COMM_SELF, 12, idx_global, PETSC_COPY_VALUES, & is_source); ISCreateGeneral(PETSC_COMM_SELF, 12, idx_local, PETSC_COPY_VALUES, &is_dest ); Vec local_vec; VecCreate(PETSC_COMM_SELF, &local_vec); VecSetSizes(local_vec, PETSC_DECIDE, 12); VecSetType(local_vec, VECSEQ); VecScatter scat; VecScatterCreate(global_vec, is_source, local_vec, is_dest, &scat); I create the local vector as sequential. However, the last two lines which create a scatter object, will cause more than half processes to hang and no error pops out. Does the scatter have to be VECMPI to VECMPI and cannot VECMPI to VECSEQ? Thanks, Qiyue Lu -------------- next part -------------- An HTML attachment was scrubbed... URL: From mfadams at lbl.gov Sun Dec 8 08:41:33 2024 From: mfadams at lbl.gov (Mark Adams) Date: Sun, 8 Dec 2024 09:41:33 -0500 Subject: [petsc-users] VecScatter() causing processes hanging In-Reply-To: References: Message-ID: You want to look at example codes and docs ( https://urldefense.us/v3/__https://petsc.org/release/manualpages/PetscSF/VecScatterCreate/__;!!G_uCfscf7eWS!ZfPM2-JGiNd8Yo2xIkHZlQtcjvcJFjXXbMbQPIZ3C5d-RVC-yfvFjoiEm5yaWLcKxfLTPhKkdLX9BM2atRAJ6eg$ ) but I am guessing global_vec and local_vec do not have the same communicator. Mark On Sat, Dec 7, 2024 at 9:50?PM Qiyue Lu wrote: > Hello, > I am trying to fetch 12 entries from a distributed vector global_vec and > put them into a local vector on each process. > > IS is_source, is_dest; > ISCreateGeneral(PETSC_COMM_SELF, 12, idx_global, PETSC_COPY_VALUES, & > is_source); > ISCreateGeneral(PETSC_COMM_SELF, 12, idx_local, PETSC_COPY_VALUES, & > is_dest); > Vec local_vec; > VecCreate(PETSC_COMM_SELF, &local_vec); > VecSetSizes(local_vec, PETSC_DECIDE, 12); > VecSetType(local_vec, VECSEQ); > VecScatter scat; > VecScatterCreate(global_vec, is_source, local_vec, is_dest, &scat); > > > I create the local vector as sequential. However, the last two lines which > create a scatter object, will cause more than half processes to hang and no > error pops out. > > Does the scatter have to be VECMPI to VECMPI and cannot VECMPI to VECSEQ? > > Thanks, > Qiyue Lu > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at petsc.dev Sun Dec 8 11:12:35 2024 From: bsmith at petsc.dev (Barry Smith) Date: Sun, 8 Dec 2024 12:12:35 -0500 Subject: [petsc-users] VecScatter() causing processes hanging In-Reply-To: References: Message-ID: <6AEBF2D9-ECAA-4078-9A78-D39CF6A5621D@petsc.dev> You can scatter from a global vector to a local vector; numerous PETSc examples demonstrate this. So hanging here is surprising. Please display the entire code so we can see the context of the VecScatterCreate() usage. Perhaps not all the MPI process that are in the global_vec communicator are involved in the call to VecScatterCreate(). To determine where the hang occurs you can run with -start_in_debugger use c for continue in each debugger window and then after a long time of hanging do control d in the hanging windows and then type bt to see where the code is hanging. Barry > On Dec 7, 2024, at 9:47?PM, Qiyue Lu wrote: > > Hello, > I am trying to fetch 12 entries from a distributed vector global_vec and put them into a local vector on each process. > > IS is_source, is_dest; > ISCreateGeneral(PETSC_COMM_SELF, 12, idx_global, PETSC_COPY_VALUES, &is_source); > ISCreateGeneral(PETSC_COMM_SELF, 12, idx_local, PETSC_COPY_VALUES, &is_dest); > Vec local_vec; > VecCreate(PETSC_COMM_SELF, &local_vec); > VecSetSizes(local_vec, PETSC_DECIDE, 12); > VecSetType(local_vec, VECSEQ); > VecScatter scat; > VecScatterCreate(global_vec, is_source, local_vec, is_dest, &scat); > > > I create the local vector as sequential. However, the last two lines which create a scatter object, will cause more than half processes to hang and no error pops out. > > Does the scatter have to be VECMPI to VECMPI and cannot VECMPI to VECSEQ? > > Thanks, > Qiyue Lu -------------- next part -------------- An HTML attachment was scrubbed... URL: From qiyuelu1 at gmail.com Sun Dec 8 15:52:40 2024 From: qiyuelu1 at gmail.com (Qiyue Lu) Date: Sun, 8 Dec 2024 15:52:40 -0600 Subject: [petsc-users] VecScatter() causing processes hanging In-Reply-To: <6AEBF2D9-ECAA-4078-9A78-D39CF6A5621D@petsc.dev> References: <6AEBF2D9-ECAA-4078-9A78-D39CF6A5621D@petsc.dev> Message-ID: Thank you all for the reply. Here is the code. Since I need to fetch data from the local_vec, so I set the type as VECSEQ, so I can use VecGetValues(). std::vector> vc(numNodesSurface, std::vector(numDimsProb)); // 2-D matrix storing velocity u, v. PetscScalar *ptr_vc = NULL; // memory storing the fetched values ptr_vc = new double[numNodesSurface*numDimsProb]; // numNodesSurfac = 6, numDimsProb = 2, are number of nodes per element and problem dimension PetscInt idx_global[numNodesSurface*numDimsProb]; // for creating global Index Set PetscInt idx_local[numNodesSurface*numDimsProb]; // for creating local Index Set for (int i = 0; i < numNodesSurface; i++){ for (int j = 0; j < numDimsProb; j++){ idx_local[i*numDimsProb + j] = i*numDimsProb + j; // local index idx_global[i*numDimsProb + j] = node_el[i]*numDOFs + j; // global index } } // global_vec is distributed, and uses MPI_COMM_WORLD as a communicator which includes all processes. // I want to fetch 12 (numNodesSurface*numDimsProb) values from global_vec and put them in a local vector, // Therefore, I set up this local_vec as VECSEQ and using PETSC_COMM_SELF. IS is_source, is_dest; ISCreateGeneral(PETSC_COMM_SELF, numNodesSurface*numDimsProb, idx_global, PETSC_COPY_VALUES, &is_source); ISCreateGeneral(PETSC_COMM_SELF, numNodesSurface*numDimsProb, idx_local, PETSC_COPY_VALUES, &is_dest); Vec local_vec; VecCreate(PETSC_COMM_SELF, &local_vec); VecSetSizes(local_vec, PETSC_DECIDE, numNodesSurface*numDimsProb); VecSetType(local_vec, VECSEQ); VecScatter scat; VecScatterCreate(global_vec, is_source, local_vec, is_dest, &scat); // Got Stuck here VecScatterBegin(scat, global_vec, local_vec, INSERT_VALUES, SCATTER_FORWARD); VecScatterEnd(scat, global_vec, local_vec, INSERT_VALUES, SCATTER_FORWARD); VecGetValues(local_vec, numNodesSurface*numDimsProb, idx_local, ptr_vc); for (int i = 0; i < numNodesSurface; i++){ for (int j = 0; j < numDimsProb; j++){ vc[i][j] = ptr_vc[i*numDimsProb + j]; // From 1-D to 2-D } } ISDestroy(&is_source); ISDestroy(&is_dest); VecDestroy(&local_vec); VecScatterDestroy(&scat); On Sun, Dec 8, 2024 at 11:12?AM Barry Smith wrote: > > You can scatter from a global vector to a local vector; numerous PETSc > examples demonstrate this. So hanging here is surprising. Please display > the entire code so we can see the context of the VecScatterCreate() usage. > Perhaps not all the MPI process that are in the global_vec communicator are > involved in the call to VecScatterCreate(). > > To determine where the hang occurs you can run with -start_in_debugger > use c for continue in each debugger window and then after a long time of > hanging do control d in the hanging windows and then type bt to see where > the code is hanging. > > Barry > > > On Dec 7, 2024, at 9:47?PM, Qiyue Lu wrote: > > Hello, > I am trying to fetch 12 entries from a distributed vector global_vec and > put them into a local vector on each process. > > IS is_source, is_dest; > ISCreateGeneral(PETSC_COMM_SELF, 12, idx_global, PETSC_COPY_VALUES, & > is_source); > ISCreateGeneral(PETSC_COMM_SELF, 12, idx_local, PETSC_COPY_VALUES, & > is_dest); > Vec local_vec; > VecCreate(PETSC_COMM_SELF, &local_vec); > VecSetSizes(local_vec, PETSC_DECIDE, 12); > VecSetType(local_vec, VECSEQ); > VecScatter scat; > VecScatterCreate(global_vec, is_source, local_vec, is_dest, &scat); > > > I create the local vector as sequential. However, the last two lines which > create a scatter object, will cause more than half processes to hang and no > error pops out. > > Does the scatter have to be VECMPI to VECMPI and cannot VECMPI to VECSEQ? > > Thanks, > Qiyue Lu > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at petsc.dev Sun Dec 8 16:00:18 2024 From: bsmith at petsc.dev (Barry Smith) Date: Sun, 8 Dec 2024 17:00:18 -0500 Subject: [petsc-users] VecScatter() causing processes hanging In-Reply-To: References: <6AEBF2D9-ECAA-4078-9A78-D39CF6A5621D@petsc.dev> Message-ID: <243002AC-7B71-4634-B77D-0FAD1E85BEB6@petsc.dev> Where does global_vec come from, and are you sure that all MPI processes that share global_vec enter this code region? > On Dec 8, 2024, at 4:52?PM, Qiyue Lu wrote: > > Thank you all for the reply. Here is the code. Since I need to fetch data from the local_vec, so I set the type as VECSEQ, so I can use VecGetValues(). > std::vector> vc(numNodesSurface, std::vector(numDimsProb)); // 2-D matrix storing velocity u, v. > PetscScalar *ptr_vc = NULL; // memory storing the fetched values > ptr_vc = new double[numNodesSurface*numDimsProb]; // numNodesSurfac = 6, numDimsProb = 2, are number of nodes per element and problem dimension > PetscInt idx_global[numNodesSurface*numDimsProb]; // for creating global Index Set > PetscInt idx_local[numNodesSurface*numDimsProb]; // for creating local Index Set > for (int i = 0; i < numNodesSurface; i++){ > for (int j = 0; j < numDimsProb; j++){ > idx_local[i*numDimsProb + j] = i*numDimsProb + j; // local index > idx_global[i*numDimsProb + j] = node_el[i]*numDOFs + j; // global index > } > } > // global_vec is distributed, and uses MPI_COMM_WORLD as a communicator which includes all processes. > // I want to fetch 12 (numNodesSurface*numDimsProb) values from global_vec and put them in a local vector, > // Therefore, I set up this local_vec as VECSEQ and using PETSC_COMM_SELF. > IS is_source, is_dest; > ISCreateGeneral(PETSC_COMM_SELF, numNodesSurface*numDimsProb, idx_global, PETSC_COPY_VALUES, &is_source); > ISCreateGeneral(PETSC_COMM_SELF, numNodesSurface*numDimsProb, idx_local, PETSC_COPY_VALUES, &is_dest); > Vec local_vec; > VecCreate(PETSC_COMM_SELF, &local_vec); > VecSetSizes(local_vec, PETSC_DECIDE, numNodesSurface*numDimsProb); > VecSetType(local_vec, VECSEQ); > VecScatter scat; > VecScatterCreate(global_vec, is_source, local_vec, is_dest, &scat); // Got Stuck here > VecScatterBegin(scat, global_vec, local_vec, INSERT_VALUES, SCATTER_FORWARD); > VecScatterEnd(scat, global_vec, local_vec, INSERT_VALUES, SCATTER_FORWARD); > VecGetValues(local_vec, numNodesSurface*numDimsProb, idx_local, ptr_vc); > for (int i = 0; i < numNodesSurface; i++){ > for (int j = 0; j < numDimsProb; j++){ > vc[i][j] = ptr_vc[i*numDimsProb + j]; // From 1-D to 2-D > } > } > ISDestroy(&is_source); > ISDestroy(&is_dest); > VecDestroy(&local_vec); > VecScatterDestroy(&scat); > > On Sun, Dec 8, 2024 at 11:12?AM Barry Smith > wrote: >> >> You can scatter from a global vector to a local vector; numerous PETSc examples demonstrate this. So hanging here is surprising. Please display the entire code so we can see the context of the VecScatterCreate() usage. Perhaps not all the MPI process that are in the global_vec communicator are involved in the call to VecScatterCreate(). >> >> To determine where the hang occurs you can run with -start_in_debugger use c for continue in each debugger window and then after a long time of hanging do control d in the hanging windows and then type bt to see where the code is hanging. >> >> Barry >> >> >>> On Dec 7, 2024, at 9:47?PM, Qiyue Lu > wrote: >>> >>> Hello, >>> I am trying to fetch 12 entries from a distributed vector global_vec and put them into a local vector on each process. >>> >>> IS is_source, is_dest; >>> ISCreateGeneral(PETSC_COMM_SELF, 12, idx_global, PETSC_COPY_VALUES, &is_source); >>> ISCreateGeneral(PETSC_COMM_SELF, 12, idx_local, PETSC_COPY_VALUES, &is_dest); >>> Vec local_vec; >>> VecCreate(PETSC_COMM_SELF, &local_vec); >>> VecSetSizes(local_vec, PETSC_DECIDE, 12); >>> VecSetType(local_vec, VECSEQ); >>> VecScatter scat; >>> VecScatterCreate(global_vec, is_source, local_vec, is_dest, &scat); >>> >>> >>> I create the local vector as sequential. However, the last two lines which create a scatter object, will cause more than half processes to hang and no error pops out. >>> >>> Does the scatter have to be VECMPI to VECMPI and cannot VECMPI to VECSEQ? >>> >>> Thanks, >>> Qiyue Lu >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From qiyuelu1 at gmail.com Sun Dec 8 16:40:53 2024 From: qiyuelu1 at gmail.com (Qiyue Lu) Date: Sun, 8 Dec 2024 16:40:53 -0600 Subject: [petsc-users] VecScatter() causing processes hanging In-Reply-To: <243002AC-7B71-4634-B77D-0FAD1E85BEB6@petsc.dev> References: <6AEBF2D9-ECAA-4078-9A78-D39CF6A5621D@petsc.dev> <243002AC-7B71-4634-B77D-0FAD1E85BEB6@petsc.dev> Message-ID: Yes, it seems the global_vec has some problems. It is passed by as a function parameter. And the original vector is declared as in the code. However, I got some print-out as below, while I am expecting to get evenly-distributed values. process 0 has 5422 entries. process 1 has 0 entries. process 2 has 0 entries. process 3 has 0 entries. process 4 has 0 entries. process 5 has 0 entries. process 6 has 0 entries. process 7 has 0 entries. process 8 has 0 entries. process 9 has 0 entries. process 10 has 0 entries. process 11 has 0 entries. process 12 has 0 entries. Vec sol_vector_rp1; VecCreate(MPI_COMM_WORLD, &sol_vector_rp1); VecSetType(sol_vector_rp1, VECMPI); VecSetSizes(sol_vector_rp1, PETSC_DECIDE, numNodesTotal*numDOFs); VecSet(sol_vector_rp1, 0.0); VecAssemblyBegin(sol_vector_rp1); VecAssemblyEnd(sol_vector_rp1); int *vec_local_size = nullptr; vec_local_size = new int[world_size]; VecGetLocalSize(sol_vector_rp1, vec_local_size); for (int i = 0; i < world_size; i++){ std::cout << "process " << i << " has " << vec_local_size[i] << " entries." << std::endl; } On Sun, Dec 8, 2024 at 4:00?PM Barry Smith wrote: > > Where does global_vec come from, and are you sure that all MPI > processes that share global_vec enter this code region? > > > > On Dec 8, 2024, at 4:52?PM, Qiyue Lu wrote: > > Thank you all for the reply. Here is the code. Since I need to fetch data > from the local_vec, so I set the type as VECSEQ, so I can use > VecGetValues(). > std::vector> vc(numNodesSurface, std::vector >(numDimsProb)); // 2-D matrix storing velocity u, v. > PetscScalar *ptr_vc = NULL; // memory storing the fetched values > ptr_vc = new double[numNodesSurface*numDimsProb]; // numNodesSurfac = 6, > numDimsProb = 2, are number of nodes per element and problem dimension > PetscInt idx_global[numNodesSurface*numDimsProb]; // for creating global > Index Set > PetscInt idx_local[numNodesSurface*numDimsProb]; // for creating local > Index Set > for (int i = 0; i < numNodesSurface; i++){ > for (int j = 0; j < numDimsProb; j++){ > idx_local[i*numDimsProb + j] = i*numDimsProb + j; // local index > idx_global[i*numDimsProb + j] = node_el[i]*numDOFs + j; // global index > } > } > // global_vec is distributed, and uses MPI_COMM_WORLD as a communicator > which includes all processes. > // I want to fetch 12 (numNodesSurface*numDimsProb) values from global_vec > and put them in a local vector, > // Therefore, I set up this local_vec as VECSEQ and using PETSC_COMM_SELF. > IS is_source, is_dest; > ISCreateGeneral(PETSC_COMM_SELF, numNodesSurface*numDimsProb, idx_global, > PETSC_COPY_VALUES, &is_source); > ISCreateGeneral(PETSC_COMM_SELF, numNodesSurface*numDimsProb, idx_local, > PETSC_COPY_VALUES, &is_dest); > Vec local_vec; > VecCreate(PETSC_COMM_SELF, &local_vec); > VecSetSizes(local_vec, PETSC_DECIDE, numNodesSurface*numDimsProb); > VecSetType(local_vec, VECSEQ); > VecScatter scat; > VecScatterCreate(global_vec, is_source, local_vec, is_dest, &scat); // > Got Stuck here > VecScatterBegin(scat, global_vec, local_vec, INSERT_VALUES, > SCATTER_FORWARD); > VecScatterEnd(scat, global_vec, local_vec, INSERT_VALUES, > SCATTER_FORWARD); > VecGetValues(local_vec, numNodesSurface*numDimsProb, idx_local, ptr_vc); > for (int i = 0; i < numNodesSurface; i++){ > for (int j = 0; j < numDimsProb; j++){ > vc[i][j] = ptr_vc[i*numDimsProb + j]; // From 1-D to 2-D > } > } > ISDestroy(&is_source); > ISDestroy(&is_dest); > VecDestroy(&local_vec); > VecScatterDestroy(&scat); > > On Sun, Dec 8, 2024 at 11:12?AM Barry Smith wrote: > >> >> You can scatter from a global vector to a local vector; numerous PETSc >> examples demonstrate this. So hanging here is surprising. Please display >> the entire code so we can see the context of the VecScatterCreate() usage. >> Perhaps not all the MPI process that are in the global_vec communicator are >> involved in the call to VecScatterCreate(). >> >> To determine where the hang occurs you can run with -start_in_debugger >> use c for continue in each debugger window and then after a long time of >> hanging do control d in the hanging windows and then type bt to see where >> the code is hanging. >> >> Barry >> >> >> On Dec 7, 2024, at 9:47?PM, Qiyue Lu wrote: >> >> Hello, >> I am trying to fetch 12 entries from a distributed vector global_vec and >> put them into a local vector on each process. >> >> IS is_source, is_dest; >> ISCreateGeneral(PETSC_COMM_SELF, 12, idx_global, PETSC_COPY_VALUES, & >> is_source); >> ISCreateGeneral(PETSC_COMM_SELF, 12, idx_local, PETSC_COPY_VALUES, & >> is_dest); >> Vec local_vec; >> VecCreate(PETSC_COMM_SELF, &local_vec); >> VecSetSizes(local_vec, PETSC_DECIDE, 12); >> VecSetType(local_vec, VECSEQ); >> VecScatter scat; >> VecScatterCreate(global_vec, is_source, local_vec, is_dest, &scat); >> >> >> I create the local vector as sequential. However, the last two lines >> which create a scatter object, will cause more than half processes to hang >> and no error pops out. >> >> Does the scatter have to be VECMPI to VECMPI and cannot VECMPI to VECSEQ? >> >> Thanks, >> Qiyue Lu >> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From qiyuelu1 at gmail.com Sun Dec 8 16:57:05 2024 From: qiyuelu1 at gmail.com (Qiyue Lu) Date: Sun, 8 Dec 2024 16:57:05 -0600 Subject: [petsc-users] VecScatter() causing processes hanging In-Reply-To: References: <6AEBF2D9-ECAA-4078-9A78-D39CF6A5621D@petsc.dev> <243002AC-7B71-4634-B77D-0FAD1E85BEB6@petsc.dev> Message-ID: Yes, I got the right Vector local size using the code below and I am also sure all processes entered that function by printing out a message, which shows up the number of processes times. But the hanging problem is still there. I need to spend more time and do more tests. int vec_local_size; // vec_local_size = new int[world_size]; VecGetLocalSize(sol_vector_rp1, &vec_local_size); std::cout << "process " << world_rank << " has " << vec_local_size << " entries." << std::endl; On Sun, Dec 8, 2024 at 4:40?PM Qiyue Lu wrote: > Yes, it seems the global_vec has some problems. It is passed by as a > function parameter. And the original vector is declared as in the code. > However, I got some print-out as below, while I am expecting to get > evenly-distributed values. > process 0 has 5422 entries. > process 1 has 0 entries. > process 2 has 0 entries. > process 3 has 0 entries. > process 4 has 0 entries. > process 5 has 0 entries. > process 6 has 0 entries. > process 7 has 0 entries. > process 8 has 0 entries. > process 9 has 0 entries. > process 10 has 0 entries. > process 11 has 0 entries. > process 12 has 0 entries. > Vec sol_vector_rp1; > VecCreate(MPI_COMM_WORLD, &sol_vector_rp1); > VecSetType(sol_vector_rp1, VECMPI); > VecSetSizes(sol_vector_rp1, PETSC_DECIDE, numNodesTotal*numDOFs); > VecSet(sol_vector_rp1, 0.0); > VecAssemblyBegin(sol_vector_rp1); > VecAssemblyEnd(sol_vector_rp1); > int *vec_local_size = nullptr; > vec_local_size = new int[world_size]; > VecGetLocalSize(sol_vector_rp1, vec_local_size); > for (int i = 0; i < world_size; i++){ > std::cout << "process " << i << " has " << vec_local_size[i] << " > entries." << std::endl; > } > > > On Sun, Dec 8, 2024 at 4:00?PM Barry Smith wrote: > >> >> Where does global_vec come from, and are you sure that all MPI >> processes that share global_vec enter this code region? >> >> >> >> On Dec 8, 2024, at 4:52?PM, Qiyue Lu wrote: >> >> Thank you all for the reply. Here is the code. Since I need to fetch data >> from the local_vec, so I set the type as VECSEQ, so I can use >> VecGetValues(). >> std::vector> vc(numNodesSurface, std::vector> >(numDimsProb)); // 2-D matrix storing velocity u, v. >> PetscScalar *ptr_vc = NULL; // memory storing the fetched values >> ptr_vc = new double[numNodesSurface*numDimsProb]; // numNodesSurfac = 6, >> numDimsProb = 2, are number of nodes per element and problem dimension >> PetscInt idx_global[numNodesSurface*numDimsProb]; // for creating global >> Index Set >> PetscInt idx_local[numNodesSurface*numDimsProb]; // for creating local >> Index Set >> for (int i = 0; i < numNodesSurface; i++){ >> for (int j = 0; j < numDimsProb; j++){ >> idx_local[i*numDimsProb + j] = i*numDimsProb + j; // local index >> idx_global[i*numDimsProb + j] = node_el[i]*numDOFs + j; // global index >> } >> } >> // global_vec is distributed, and uses MPI_COMM_WORLD as a communicator >> which includes all processes. >> // I want to fetch 12 (numNodesSurface*numDimsProb) values from >> global_vec and put them in a local vector, >> // Therefore, I set up this local_vec as VECSEQ and using PETSC_COMM_SELF. >> IS is_source, is_dest; >> ISCreateGeneral(PETSC_COMM_SELF, numNodesSurface*numDimsProb, >> idx_global, PETSC_COPY_VALUES, &is_source); >> ISCreateGeneral(PETSC_COMM_SELF, numNodesSurface*numDimsProb, idx_local, >> PETSC_COPY_VALUES, &is_dest); >> Vec local_vec; >> VecCreate(PETSC_COMM_SELF, &local_vec); >> VecSetSizes(local_vec, PETSC_DECIDE, numNodesSurface*numDimsProb); >> VecSetType(local_vec, VECSEQ); >> VecScatter scat; >> VecScatterCreate(global_vec, is_source, local_vec, is_dest, &scat); // >> Got Stuck here >> VecScatterBegin(scat, global_vec, local_vec, INSERT_VALUES, >> SCATTER_FORWARD); >> VecScatterEnd(scat, global_vec, local_vec, INSERT_VALUES, >> SCATTER_FORWARD); >> VecGetValues(local_vec, numNodesSurface*numDimsProb, idx_local, ptr_vc); >> for (int i = 0; i < numNodesSurface; i++){ >> for (int j = 0; j < numDimsProb; j++){ >> vc[i][j] = ptr_vc[i*numDimsProb + j]; // From 1-D to 2-D >> } >> } >> ISDestroy(&is_source); >> ISDestroy(&is_dest); >> VecDestroy(&local_vec); >> VecScatterDestroy(&scat); >> >> On Sun, Dec 8, 2024 at 11:12?AM Barry Smith wrote: >> >>> >>> You can scatter from a global vector to a local vector; numerous >>> PETSc examples demonstrate this. So hanging here is surprising. Please >>> display the entire code so we can see the context of the VecScatterCreate() >>> usage. Perhaps not all the MPI process that are in the global_vec >>> communicator are involved in the call to VecScatterCreate(). >>> >>> To determine where the hang occurs you can run with -start_in_debugger >>> use c for continue in each debugger window and then after a long time of >>> hanging do control d in the hanging windows and then type bt to see where >>> the code is hanging. >>> >>> Barry >>> >>> >>> On Dec 7, 2024, at 9:47?PM, Qiyue Lu wrote: >>> >>> Hello, >>> I am trying to fetch 12 entries from a distributed vector global_vec and >>> put them into a local vector on each process. >>> >>> IS is_source, is_dest; >>> ISCreateGeneral(PETSC_COMM_SELF, 12, idx_global, PETSC_COPY_VALUES, & >>> is_source); >>> ISCreateGeneral(PETSC_COMM_SELF, 12, idx_local, PETSC_COPY_VALUES, & >>> is_dest); >>> Vec local_vec; >>> VecCreate(PETSC_COMM_SELF, &local_vec); >>> VecSetSizes(local_vec, PETSC_DECIDE, 12); >>> VecSetType(local_vec, VECSEQ); >>> VecScatter scat; >>> VecScatterCreate(global_vec, is_source, local_vec, is_dest, &scat); >>> >>> >>> I create the local vector as sequential. However, the last two lines >>> which create a scatter object, will cause more than half processes to hang >>> and no error pops out. >>> >>> Does the scatter have to be VECMPI to VECMPI and cannot VECMPI to VECSEQ? >>> >>> Thanks, >>> Qiyue Lu >>> >>> >>> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From yc17470 at connect.um.edu.mo Mon Dec 9 05:34:22 2024 From: yc17470 at connect.um.edu.mo (Gong Yujie) Date: Mon, 9 Dec 2024 11:34:22 +0000 Subject: [petsc-users] Question about ordering of the gradient in DS Message-ID: Dear PETSc group, I would like to inquire about the storage of the gradient in function PetscDSSetResidual: PetscErrorCode PetscDSSetResidual(PetscDS ds, PetscInt f, void (*f0)(PetscInt dim, PetscInt Nf, PetscInt NfAux, const PetscInt uOff[], const PetscInt uOff_x[], const PetscScalar u[], const PetscScalar u_t[], const PetscScalar u_x[], const PetscInt aOff[], const PetscInt aOff_x[], const PetscScalar a[], const PetscScalar a_t[], const PetscScalar a_x[], PetscReal t, const PetscReal x[], PetscInt numConstants, const PetscScalar constants[], PetscScalar f0[]), void (*f1)(PetscInt, PetscInt, PetscInt, const PetscInt[], const PetscInt[], const PetscScalar[], const PetscScalar[], const PetscScalar[], const PetscInt[], const PetscInt[], const PetscScalar[], const PetscScalar[], const PetscScalar[], PetscReal, const PetscReal[], PetscInt, const PetscScalar[], PetscScalar[])) Assume u = (u1, u2, u3)^T, the gradient of u is [cid:68b0e2f6-2465-4550-8ae1-4631bfe74163] In PetscDSSetResidual, it is stored in an array. Is it ordered as (u1x,u2x, u3x, u1y, u2y, u3y, u1z, u2z,u3z)? Thanks in advance! Best Regards, Yujie -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 12296 bytes Desc: image.png URL: From knepley at gmail.com Mon Dec 9 05:39:31 2024 From: knepley at gmail.com (Matthew Knepley) Date: Mon, 9 Dec 2024 06:39:31 -0500 Subject: [petsc-users] Question about ordering of the gradient in DS In-Reply-To: References: Message-ID: On Mon, Dec 9, 2024 at 6:34?AM Gong Yujie wrote: > Dear PETSc group, > > I would like to inquire about the storage of the gradient in function > PetscDSSetResidual: > > PetscErrorCode PetscDSSetResidual(PetscDS ds, PetscInt f, void > (*f0)(PetscInt dim, PetscInt Nf, PetscInt NfAux, const PetscInt uOff[], > const PetscInt uOff_x[], const PetscScalar u[], const PetscScalar u_t[], > const PetscScalar u_x[], const PetscInt aOff[], const PetscInt aOff_x[], > const PetscScalar a[], const PetscScalar a_t[], const PetscScalar a_x[], > PetscReal t, const PetscReal x[], PetscInt numConstants, const PetscScalar > constants[], PetscScalar f0[]), void (*f1)(PetscInt, PetscInt, PetscInt, > const PetscInt[], const PetscInt[], const PetscScalar[], const > PetscScalar[], const PetscScalar[], const PetscInt[], const PetscInt[], > const PetscScalar[], const PetscScalar[], const PetscScalar[], PetscReal, > const PetscReal[], PetscInt, const PetscScalar[], PetscScalar[])) > > Assume u = (u1, u2, u3)^T, the gradient of u is > > In PetscDSSetResidual, it is stored in an array. Is it ordered as > (u1x,u2x, u3x, u1y, u2y, u3y, u1z, u2z,u3z)? > It is a C array grad[Nc][dE], so that the spatial dimension is fastest ux, uy, ux, vx, vy, vz, wx, wy, wz Thanks, Matt > Thanks in advance! > > Best Regards, > Yujie > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://urldefense.us/v3/__https://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!eb8ID0j7ucY5S-5H6K_Ig6DMc1VVC246S03W0vRWPw0ovJcnweTr4TvjN-SNwxLXHzOGC0c1gXcXEn6q0qfy$ -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 12296 bytes Desc: not available URL: From sblondel at utk.edu Mon Dec 9 13:56:21 2024 From: sblondel at utk.edu (Blondel, Sophie) Date: Mon, 9 Dec 2024 19:56:21 +0000 Subject: [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED Message-ID: Hi, I am trying to understand a strange behavior I'm encountering: when running my application with "-ts_exact_final_time stepover" everything goes well, but when I switch to "matchstep" I get DIVERGED_STEP_REJECTED before the first time step is finished. I tried increasing the maximum number of rejections and it just takes longer to diverge, and if I set the value to "unlimited" it is basically an infinite loop. Is there a way to check why is the step rejected? Could the "matchstep" option change tolerances somewhere that would cause that behavior? Let me know if I should provide more information. Best, Sophie Blondel -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at petsc.dev Mon Dec 9 14:12:55 2024 From: bsmith at petsc.dev (Barry Smith) Date: Mon, 9 Dec 2024 15:12:55 -0500 Subject: [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED In-Reply-To: References: Message-ID: <1E4CF02D-32EB-482B-ACFC-9AE1C7F9102E@petsc.dev> > On Dec 9, 2024, at 2:56?PM, Blondel, Sophie via petsc-users wrote: > > Hi, > > I am trying to understand a strange behavior I'm encountering: when running my application with "-ts_exact_final_time stepover" everything goes well, but when I switch to "matchstep" I get DIVERGED_STEP_REJECTED before the first time step is finished. This is in the very first time-step in TSSolve? Please run with -ts_monitor and send all the output (best for a short time interval and do it twice once with -ts_exact_final_time stepover and once with exact. Barry > I tried increasing the maximum number of rejections and it just takes longer to diverge, and if I set the value to "unlimited" it is basically an infinite loop. > > Is there a way to check why is the step rejected? Could the "matchstep" option change tolerances somewhere that would cause that behavior? > > Let me know if I should provide more information. > > Best, > > Sophie Blondel -------------- next part -------------- An HTML attachment was scrubbed... URL: From lscalia at pa.uc3m.es Mon Dec 9 15:50:01 2024 From: lscalia at pa.uc3m.es (LUCA SCALIA) Date: Mon, 9 Dec 2024 22:50:01 +0100 Subject: [petsc-users] SLEPC/PETSC for large scale FEM generalised eigenvalue problem Message-ID: Dear group, I am using libraries PETSC and SLEPC for a Finite Element code written in c++ that I am developing, oriented to the solution of generalised eigenvalue problems for large structural systems featuring up to 10^5 -10^6 degrees of freedom. I manage to successfully run the code for a generalised eigenvalue problem of structures with around 10^4 degrees of freedom in a reasonable amount of time but the problem arises in the case of systems with a number of degrees of freedom from 10^5 or above. The current test case I'm trying has 120.000 degrees of freedom. The EPS solver gets slower and slower as the iterations pass. After a rough estimate, I would say it may take no less than 5-6 hours with 20 MPI processes ( allowing an unlimited number of iterations) I need to compute the first 10 to 30 (nev) modes of the structure (EPS_SMALLEST_MAGNITUDE). I'm using the default KRYLOVSCHUR iterative solver. The problem is generalised Hermitian I tried to play a little bit with different settings and options of the library, e.g.: 1) ncv parameter. different values of ncv, between 10*nev and 40*nev 2) different solvers for the ksp object ( GMRES and CG ), and tolerance as well preconditioner (PCJACOBI and PCBJACOBI) 3) Normalisation of matrices A and B with the respective Frobenius or Infinite norms. ... but I wasn't able to solve the problem. I kindly ask for advice on which strategies I should resort to in order to improve the speed of SLEPC for such large numbers of degrees of freedom, since I'm quite new to the solution of the generalised eigenvalue problem. I also attach two files with the stiffness and mass matrix of one element to show the magnitude of the entries of the two matrices Best regards, Luca -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- Stiffness matrix: Mat Object: 1 MPI process type: seqaij row 0: (0, 9.6009e+11) (1, 62500.) (2, 0.) (3, 0.) (4, 0.) (5, 0.) (6, -2736.91) (7, 4807.69) (8, 0.) (9, 0.) (10, 0.) (11, 0.) (12, -92933.8) (13, -62500.) (14, 0.) (15, 0.) (16, 0.) (17, 0.) (18, -42361.5) (19, -4807.69) (20, 0.) (21, 0.) (22, 0.) (23, 0.) row 1: (0, 62500.) (1, 9.6009e+11) (2, 0.) (3, 0.) (4, 0.) (5, 0.) (6, -4807.69) (7, -158553.) (8, 0.) (9, 0.) (10, 0.) (11, 0.) (12, -62500.) (13, -114792.) (14, 0.) (15, 0.) (16, 0.) (17, 0.) (18, 4807.69) (19, 67438.2) (20, 0.) (21, 0.) (22, 0.) (23, 0.) row 2: (0, 0.) (1, 0.) (2, 9.6009e+11) (3, 140929.) (4, -99148.5) (5, 0.) (6, 0.) (7, 0.) (8, -20132.2) (9, 140929.) (10, -99148.5) (11, 0.) (12, 0.) (13, 0.) (14, -59593.3) (15, 140929.) (16, -99148.5) (17, 0.) (18, 0.) (19, 0.) (20, 20132.2) (21, 140929.) (22, -99148.5) (23, 0.) row 3: (0, 0.) (1, 0.) (2, 140929.) (3, 9.6009e+11) (4, -130208.) (5, 0.) (6, 0.) (7, 0.) (8, -140929.) (9, 135030.) (10, 10016.) (11, 0.) (12, 0.) (13, 0.) (14, -140929.) (15, 291969.) (16, 130208.) (17, 0.) (18, 0.) (19, 0.) (20, 140929.) (21, 605846.) (22, -10016.) (23, 0.) row 4: (0, 0.) (1, 0.) (2, -99148.5) (3, -130208.) (4, 9.6009e+11) (5, 0.) (6, 0.) (7, 0.) (8, -99148.5) (9, -10016.) (10, 426094.) (11, 0.) (12, 0.) (13, 0.) (14, 99148.5) (15, 130208.) (16, 371060.) (17, 0.) (18, 0.) (19, 0.) (20, 99148.5) (21, 10016.) (22, 343543.) (23, 0.) row 5: (0, 0.) (1, 0.) (2, 0.) (3, 0.) (4, 0.) (5, 9.6009e+11) (6, 0.) (7, 0.) (8, 0.) (9, 0.) (10, 0.) (11, 0.) (12, 0.) (13, 0.) (14, 0.) (15, 0.) (16, 0.) (17, 0.) (18, 0.) (19, 0.) (20, 0.) (21, 0.) (22, 0.) (23, 0.) row 6: (0, -2736.91) (1, -4807.69) (2, 0.) (3, 0.) (4, 0.) (5, 0.) (6, 138032.) (7, -62500.) (8, 0.) (9, 0.) (10, 0.) (11, 0.) (12, -42361.5) (13, 4807.69) (14, 0.) (15, 0.) (16, 0.) (17, 0.) (18, -92933.8) (19, 62500.) (20, 0.) (21, 0.) (22, 0.) (23, 0.) row 7: (0, 4807.69) (1, -158553.) (2, 0.) (3, 0.) (4, 0.) (5, 0.) (6, -62500.) (7, 205907.) (8, 0.) (9, 0.) (10, 0.) (11, 0.) (12, -4807.69) (13, 67438.2) (14, 0.) (15, 0.) (16, 0.) (17, 0.) (18, 62500.) (19, -114792.) (20, 0.) (21, 0.) (22, 0.) (23, 0.) row 8: (0, 0.) (1, 0.) (2, -20132.2) (3, -140929.) (4, -99148.5) (5, 0.) (6, 0.) (7, 0.) (8, 59593.3) (9, -140929.) (10, -99148.5) (11, 0.) (12, 0.) (13, 0.) (14, 20132.2) (15, -140929.) (16, -99148.5) (17, 0.) (18, 0.) (19, 0.) (20, -59593.3) (21, -140929.) (22, -99148.5) (23, 0.) row 9: (0, 0.) (1, 0.) (2, 140929.) (3, 135030.) (4, -10016.) (5, 0.) (6, 0.) (7, 0.) (8, -140929.) (9, 960090.) (10, 130208.) (11, 0.) (12, 0.) (13, 0.) (14, -140929.) (15, 605846.) (16, 10016.) (17, 0.) (18, 0.) (19, 0.) (20, 140929.) (21, 291969.) (22, -130208.) (23, 0.) row 10: (0, 0.) (1, 0.) (2, -99148.5) (3, 10016.) (4, 426094.) (5, 0.) (6, 0.) (7, 0.) (8, -99148.5) (9, 130208.) (10, 852239.) (11, 0.) (12, 0.) (13, 0.) (14, 99148.5) (15, -10016.) (16, 343543.) (17, 0.) (18, 0.) (19, 0.) (20, 99148.5) (21, -130208.) (22, 371060.) (23, 0.) row 11: (0, 0.) (1, 0.) (2, 0.) (3, 0.) (4, 0.) (5, 0.) (6, 0.) (7, 0.) (8, 0.) (9, 0.) (10, 0.) (11, 5.95933) (12, 0.) (13, 0.) (14, 0.) (15, 0.) (16, 0.) (17, 0.) (18, 0.) (19, 0.) (20, 0.) (21, 0.) (22, 0.) (23, 0.) row 12: (0, -92933.8) (1, -62500.) (2, 0.) (3, 0.) (4, 0.) (5, 0.) (6, -42361.5) (7, -4807.69) (8, 0.) (9, 0.) (10, 0.) (11, 0.) (12, 138032.) (13, 62500.) (14, 0.) (15, 0.) (16, 0.) (17, 0.) (18, -2736.91) (19, 4807.69) (20, 0.) (21, 0.) (22, 0.) (23, 0.) row 13: (0, -62500.) (1, -114792.) (2, 0.) (3, 0.) (4, 0.) (5, 0.) (6, 4807.69) (7, 67438.2) (8, 0.) (9, 0.) (10, 0.) (11, 0.) (12, 62500.) (13, 205907.) (14, 0.) (15, 0.) (16, 0.) (17, 0.) (18, -4807.69) (19, -158553.) (20, 0.) (21, 0.) (22, 0.) (23, 0.) row 14: (0, 0.) (1, 0.) (2, -59593.3) (3, -140929.) (4, 99148.5) (5, 0.) (6, 0.) (7, 0.) (8, 20132.2) (9, -140929.) (10, 99148.5) (11, 0.) (12, 0.) (13, 0.) (14, 59593.3) (15, -140929.) (16, 99148.5) (17, 0.) (18, 0.) (19, 0.) (20, -20132.2) (21, -140929.) (22, 99148.5) (23, 0.) row 15: (0, 0.) (1, 0.) (2, 140929.) (3, 291969.) (4, 130208.) (5, 0.) (6, 0.) (7, 0.) (8, -140929.) (9, 605846.) (10, -10016.) (11, 0.) (12, 0.) (13, 0.) (14, -140929.) (15, 960090.) (16, -130208.) (17, 0.) (18, 0.) (19, 0.) (20, 140929.) (21, 135030.) (22, 10016.) (23, 0.) row 16: (0, 0.) (1, 0.) (2, -99148.5) (3, 130208.) (4, 371060.) (5, 0.) (6, 0.) (7, 0.) (8, -99148.5) (9, 10016.) (10, 343543.) (11, 0.) (12, 0.) (13, 0.) (14, 99148.5) (15, -130208.) (16, 852239.) (17, 0.) (18, 0.) (19, 0.) (20, 99148.5) (21, -10016.) (22, 426094.) (23, 0.) row 17: (0, 0.) (1, 0.) (2, 0.) (3, 0.) (4, 0.) (5, 0.) (6, 0.) (7, 0.) (8, 0.) (9, 0.) (10, 0.) (11, 0.) (12, 0.) (13, 0.) (14, 0.) (15, 0.) (16, 0.) (17, 5.95933) (18, 0.) (19, 0.) (20, 0.) (21, 0.) (22, 0.) (23, 0.) row 18: (0, -42361.5) (1, 4807.69) (2, 0.) (3, 0.) (4, 0.) (5, 0.) (6, -92933.8) (7, 62500.) (8, 0.) (9, 0.) (10, 0.) (11, 0.) (12, -2736.91) (13, -4807.69) (14, 0.) (15, 0.) (16, 0.) (17, 0.) (18, 9.6009e+11) (19, -62500.) (20, 0.) (21, 0.) (22, 0.) (23, 0.) row 19: (0, -4807.69) (1, 67438.2) (2, 0.) (3, 0.) (4, 0.) (5, 0.) (6, 62500.) (7, -114792.) (8, 0.) (9, 0.) (10, 0.) (11, 0.) (12, 4807.69) (13, -158553.) (14, 0.) (15, 0.) (16, 0.) (17, 0.) (18, -62500.) (19, 9.6009e+11) (20, 0.) (21, 0.) (22, 0.) (23, 0.) row 20: (0, 0.) (1, 0.) (2, 20132.2) (3, 140929.) (4, 99148.5) (5, 0.) (6, 0.) (7, 0.) (8, -59593.3) (9, 140929.) (10, 99148.5) (11, 0.) (12, 0.) (13, 0.) (14, -20132.2) (15, 140929.) (16, 99148.5) (17, 0.) (18, 0.) (19, 0.) (20, 9.6009e+11) (21, 140929.) (22, 99148.5) (23, 0.) row 21: (0, 0.) (1, 0.) (2, 140929.) (3, 605846.) (4, 10016.) (5, 0.) (6, 0.) (7, 0.) (8, -140929.) (9, 291969.) (10, -130208.) (11, 0.) (12, 0.) (13, 0.) (14, -140929.) (15, 135030.) (16, -10016.) (17, 0.) (18, 0.) (19, 0.) (20, 140929.) (21, 9.6009e+11) (22, 130208.) (23, 0.) row 22: (0, 0.) (1, 0.) (2, -99148.5) (3, -10016.) (4, 343543.) (5, 0.) (6, 0.) (7, 0.) (8, -99148.5) (9, -130208.) (10, 371060.) (11, 0.) (12, 0.) (13, 0.) (14, 99148.5) (15, 10016.) (16, 426094.) (17, 0.) (18, 0.) (19, 0.) (20, 99148.5) (21, 130208.) (22, 9.6009e+11) (23, 0.) row 23: (0, 0.) (1, 0.) (2, 0.) (3, 0.) (4, 0.) (5, 0.) (6, 0.) (7, 0.) (8, 0.) (9, 0.) (10, 0.) (11, 0.) (12, 0.) (13, 0.) (14, 0.) (15, 0.) (16, 0.) (17, 0.) (18, 0.) (19, 0.) (20, 0.) (21, 0.) (22, 0.) (23, 9.6009e+11) -------------- next part -------------- Mass matrix: Mat Object: 1 MPI process type: seqaij row 0: (0, 239.836) row 1: (1, 239.836) row 2: (2, 239.836) row 3: (3, 239.836) row 4: (4, 239.836) row 5: (5, 239.836) row 6: (6, 0.000239836) row 7: (7, 0.000239836) row 8: (8, 0.000239836) row 9: (9, 2.39836e-08) row 10: (10, 2.39836e-08) row 11: (11, 2.39836e-08) row 12: (12, 0.000239836) row 13: (13, 0.000239836) row 14: (14, 0.000239836) row 15: (15, 2.39836e-08) row 16: (16, 2.39836e-08) row 17: (17, 2.39836e-08) row 18: (18, 239.836) row 19: (19, 239.836) row 20: (20, 239.836) row 21: (21, 239.836) row 22: (22, 239.836) row 23: (23, 239.836) From bsmith at petsc.dev Mon Dec 9 15:57:44 2024 From: bsmith at petsc.dev (Barry Smith) Date: Mon, 9 Dec 2024 16:57:44 -0500 Subject: [petsc-users] SLEPC/PETSC for large scale FEM generalised eigenvalue problem In-Reply-To: References: Message-ID: I'll leave it to Jose to provide a complete, comprehensive answer, but given your problem sizes, I think using a direct solver instead of an iterative solver may be the best bet. So configure PETSc without debugging and with MUMPS ./configure --with-debugging=0 --download-mumps --download-scalapack --download-ptscotch and use -ksp_type preconly -pc_type lu -pc_factor_mat_solver_type mumps Good luck > On Dec 9, 2024, at 4:50?PM, LUCA SCALIA wrote: > > Dear group, > > I am using libraries PETSC and SLEPC for a Finite Element code written in c++ that I am developing, oriented to the solution of generalised eigenvalue problems for large structural systems featuring up to 10^5 -10^6 degrees of freedom. > > I manage to successfully run the code for a generalised eigenvalue problem of structures with around 10^4 degrees of freedom in a reasonable amount of time but the problem arises in the case of systems with a number of degrees of freedom from 10^5 or above. > > The current test case I'm trying has 120.000 degrees of freedom. The EPS solver gets slower and slower as the iterations pass. After a rough estimate, I would say it may take no less than 5-6 hours with 20 MPI processes ( allowing an unlimited number of iterations) > > I need to compute the first 10 to 30 (nev) modes of the structure (EPS_SMALLEST_MAGNITUDE). I'm using the default KRYLOVSCHUR iterative solver. The problem is generalised Hermitian > > > I tried to play a little bit with different settings and options of the library, e.g.: > > 1) ncv parameter. different values of ncv, between 10*nev and 40*nev > > 2) different solvers for the ksp object ( GMRES and CG ), and tolerance as well preconditioner (PCJACOBI and PCBJACOBI) > > 3) Normalisation of matrices A and B with the respective Frobenius or Infinite norms. > > ... but I wasn't able to solve the problem. > > I kindly ask for advice on which strategies I should resort to in order to improve the speed of SLEPC for such large numbers of degrees of freedom, since I'm quite new to the solution of the generalised eigenvalue problem. > > I also attach two files with the stiffness and mass matrix of one element to show the magnitude of the entries of the two matrices > > Best regards, > > Luca > From jroman at dsic.upv.es Mon Dec 9 16:18:26 2024 From: jroman at dsic.upv.es (Jose E. Roman) Date: Mon, 9 Dec 2024 22:18:26 +0000 Subject: [petsc-users] SLEPC/PETSC for large scale FEM generalised eigenvalue problem In-Reply-To: References: Message-ID: <74698199-2A8C-4070-8BAA-A1D03BE1B972@dsic.upv.es> Most probably you want EPS_TARGET_MAGNITUDE with shift-and-invert, i.e., -st_type sinvert -eps_target 0 -eps_target_magnitude -st_ksp_type preonly -st_pc_type lu -st_pc_factor_mat_solver_type mumps This will solve a linear system with your A matrix instead of your B matrix. If A is singular, try a nonzero target located on the left of your eigenvalues. Jose > El 9 dic 2024, a las 22:57, Barry Smith escribi?: > > > I'll leave it to Jose to provide a complete, comprehensive answer, but given your problem sizes, I think using a direct solver instead of an iterative solver may be the best bet. So configure PETSc without debugging and with MUMPS > > ./configure --with-debugging=0 --download-mumps --download-scalapack --download-ptscotch > > and use -ksp_type preconly -pc_type lu -pc_factor_mat_solver_type mumps > > Good luck > > >> On Dec 9, 2024, at 4:50?PM, LUCA SCALIA wrote: >> >> Dear group, >> >> I am using libraries PETSC and SLEPC for a Finite Element code written in c++ that I am developing, oriented to the solution of generalised eigenvalue problems for large structural systems featuring up to 10^5 -10^6 degrees of freedom. >> >> I manage to successfully run the code for a generalised eigenvalue problem of structures with around 10^4 degrees of freedom in a reasonable amount of time but the problem arises in the case of systems with a number of degrees of freedom from 10^5 or above. >> >> The current test case I'm trying has 120.000 degrees of freedom. The EPS solver gets slower and slower as the iterations pass. After a rough estimate, I would say it may take no less than 5-6 hours with 20 MPI processes ( allowing an unlimited number of iterations) >> >> I need to compute the first 10 to 30 (nev) modes of the structure (EPS_SMALLEST_MAGNITUDE). I'm using the default KRYLOVSCHUR iterative solver. The problem is generalised Hermitian >> >> >> I tried to play a little bit with different settings and options of the library, e.g.: >> >> 1) ncv parameter. different values of ncv, between 10*nev and 40*nev >> >> 2) different solvers for the ksp object ( GMRES and CG ), and tolerance as well preconditioner (PCJACOBI and PCBJACOBI) >> >> 3) Normalisation of matrices A and B with the respective Frobenius or Infinite norms. >> >> ... but I wasn't able to solve the problem. >> >> I kindly ask for advice on which strategies I should resort to in order to improve the speed of SLEPC for such large numbers of degrees of freedom, since I'm quite new to the solution of the generalised eigenvalue problem. >> >> I also attach two files with the stiffness and mass matrix of one element to show the magnitude of the entries of the two matrices >> >> Best regards, >> >> Luca >> > From sblondel at utk.edu Mon Dec 9 16:29:28 2024 From: sblondel at utk.edu (Blondel, Sophie) Date: Mon, 9 Dec 2024 22:29:28 +0000 Subject: [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED In-Reply-To: <1E4CF02D-32EB-482B-ACFC-9AE1C7F9102E@petsc.dev> References: <1E4CF02D-32EB-482B-ACFC-9AE1C7F9102E@petsc.dev> Message-ID: Hi Barry, I hope you are doing well. Attached are the output. To give a little more context, this is a "new" way of running the code where multiple instances are created and communicate together every few time steps (like coupling the code with itself in memory). Here there are 3 instances that each have a separate TS object, plus one "main" instance that doesn't solve anything but compute rates to exchange between the other instances. Cheers, Sophie ________________________________ From: Barry Smith Sent: Monday, December 9, 2024 15:12 To: Blondel, Sophie Cc: petsc-users at mcs.anl.gov ; xolotl-psi-development at lists.sourceforge.net Subject: Re: [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED On Dec 9, 2024, at 2:56?PM, Blondel, Sophie via petsc-users wrote: Hi, I am trying to understand a strange behavior I'm encountering: when running my application with "-ts_exact_final_time stepover" everything goes well, but when I switch to "matchstep" I get DIVERGED_STEP_REJECTED before the first time step is finished. This is in the very first time-step in TSSolve? Please run with -ts_monitor and send all the output (best for a short time interval and do it twice once with -ts_exact_final_time stepover and once with exact. Barry I tried increasing the maximum number of rejections and it just takes longer to diverge, and if I set the value to "unlimited" it is basically an infinite loop. Is there a way to check why is the step rejected? Could the "matchstep" option change tolerances somewhere that would cause that behavior? Let me know if I should provide more information. Best, Sophie Blondel -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: stepover.txt URL: -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: matchstep.txt URL: From bsmith at petsc.dev Mon Dec 9 19:04:00 2024 From: bsmith at petsc.dev (Barry Smith) Date: Mon, 9 Dec 2024 20:04:00 -0500 Subject: [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED In-Reply-To: References: <1E4CF02D-32EB-482B-ACFC-9AE1C7F9102E@petsc.dev> Message-ID: Sorry, try both with -ts_adapt_monitor > On Dec 9, 2024, at 5:29?PM, Blondel, Sophie wrote: > > Hi Barry, > > I hope you are doing well. > > Attached are the output. To give a little more context, this is a "new" way of running the code where multiple instances are created and communicate together every few time steps (like coupling the code with itself in memory). Here there are 3 instances that each have a separate TS object, plus one "main" instance that doesn't solve anything but compute rates to exchange between the other instances. > > Cheers, > > Sophie > From: Barry Smith > Sent: Monday, December 9, 2024 15:12 > To: Blondel, Sophie > Cc: petsc-users at mcs.anl.gov ; xolotl-psi-development at lists.sourceforge.net > Subject: Re: [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED > > > >> On Dec 9, 2024, at 2:56?PM, Blondel, Sophie via petsc-users wrote: >> >> Hi, >> >> I am trying to understand a strange behavior I'm encountering: when running my application with "-ts_exact_final_time stepover" everything goes well, but when I switch to "matchstep" I get DIVERGED_STEP_REJECTED before the first time step is finished. > > This is in the very first time-step in TSSolve? > > Please run with -ts_monitor and send all the output (best for a short time interval and do it twice once with -ts_exact_final_time stepover and once with exact. > > Barry > > >> I tried increasing the maximum number of rejections and it just takes longer to diverge, and if I set the value to "unlimited" it is basically an infinite loop. >> >> Is there a way to check why is the step rejected? Could the "matchstep" option change tolerances somewhere that would cause that behavior? >> >> Let me know if I should provide more information. >> >> Best, >> >> Sophie Blondel > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sebastian.blauth at itwm.fraunhofer.de Tue Dec 10 02:27:58 2024 From: sebastian.blauth at itwm.fraunhofer.de (Blauth, Sebastian) Date: Tue, 10 Dec 2024 08:27:58 +0000 Subject: [petsc-users] Issue with SNES with petsc4py 3.22 Message-ID: Hi everyone, I am having trouble with using SNES from petsc4py in PETSc 3.22. The problem is similar to https://gitlab.com/petsc/petsc/-/issues/1611 and related to the readonly access of ?x?. Particularly, in my application, based on FEniCS, I want to solve a nonlinear PDE. Therefore, in my callbacks, I want to set the vector of DoFs of my FEniCS function to ?x? with a line like self.u.vector().vec().setArray(x) However, this now raises the error [0] Vector 'Vec_0x84000003_0' (argument #1) was locked for read-only access in unknown_function() at unknown file:0 (line numbers only accurate to function begin) I believe that this is a consequence of https://gitlab.com/petsc/petsc/-/merge_requests/7567#22c226fc29b6fc9dc5781b0 aff4dcc70ac6c3489, but I did not find any information on how to remove the lock from the vector or how I can set use ?x? to update the vector of my function. Using xx = x.getArray(readonly=True) as described in https://gitlab.com/petsc/petsc/-/issues/1611 does not work either, as calling self.u.vector().vec().setArray(xx) then fails with the same error message. Does anyone have an idea how to fix this? Thanks a lot in advance, Sebastian -- Dr. Sebastian Blauth Fraunhofer-Institut f?r Techno- und Wirtschaftsmathematik ITWM Abteilung Transportvorg?nge Fraunhofer-Platz 1, 67663 Kaiserslautern Telefon: +49 631 31600-4968 sebastian.blauth at itwm.fraunhofer.de https://www.itwm.fraunhofer.de -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 7943 bytes Desc: not available URL: From 12431140 at mail.sustech.edu.cn Tue Dec 10 03:31:43 2024 From: 12431140 at mail.sustech.edu.cn (=?utf-8?B?RGF2aWQgSmlhd2VpIExVTyBMSUFORw==?=) Date: Tue, 10 Dec 2024 17:31:43 +0800 Subject: [petsc-users] Parallel computing in SNES Message-ID: Hello PETSc, I want to substitute the existing nonlinear solver codes with SNES. The existing code is Parallel computing. My FormFunction and FormJacobian is attached in file Res_Jac.cpp. The global tangent matrix assembly and residual function assembly are both implemented by invoking user pointers. The following is the message, we can find that SNES just iterate once. I think the problem is input solution Vec y doesn't link to Residual Vec f, because my update is stored in the user variables. If I use  const PetscScalar *yy; VecGetArrayRead(y, &yy);  the value of yy is not identical to Vec y, because there are multi thread. So, my problem is how to assembly residual and tangent in parallel environment? David Jiawei LUO LIANG ??????/??/???/2024 ?????????????1088?   -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: C30690D0 at 0F60110C.7F0A586700000000.jpg Type: image/jpeg Size: 67593 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Res_Jac.cpp Type: application/octet-stream Size: 4488 bytes Desc: not available URL: From matteo.semplice at uninsubria.it Tue Dec 10 06:02:35 2024 From: matteo.semplice at uninsubria.it (Matteo Semplice) Date: Tue, 10 Dec 2024 13:02:35 +0100 Subject: [petsc-users] PCMG with shell restrictions Message-ID: <7071ff6e-52cb-4715-8cbc-bacc6a385363@uninsubria.it> Dear petsc-users, ??? I am trying with a student to modify the MG example 65 to use mat-shells instead of assembled matrices. Our use case is for a method that will use custom shell operators and shell interpolation/restrictions. To start we have modified ex65 and tried to replace the standard restriction/interpolation with a shell matrix that performs the same operations on the DMDA grids. I attach our modifications to ex65. The problem is that the code sometimes completes execution and sometimes errors out like $ ./ex65shell -ksp_monitor -pc_type mg -da_refine 2 -ksp_rtol 1e-1 >> Created DMshell 0x55bbc83c91a0 (0x55bbc8390a80) Calling KSPSolve from main computeRHS on grid 513 computeMatrix on grid 513 Inside Coarsen >> Created DMshell 0x55bbc84c5270 (0x55bbc84b2ff0) Inside Coarsen >> Created DMshell 0x55bbc84e0a30 (0x55bbc84bdb60) >> Create interpolation from 0x55bbc84c5270(0x55bbc84b2ff0) to 0x55bbc83c91a0(0x55bbc8390a80) [0]PETSC ERROR: --------------------- Error Message -------------------------------------------------------------- [0]PETSC ERROR: Invalid pointer [0]PETSC ERROR: Invalid Pointer to PetscObject: Argument 'obj' (parameter # 1) [0]PETSC ERROR: See https://urldefense.us/v3/__https://petsc.org/release/faq/__;!!G_uCfscf7eWS!Y29qRoOD71oxcd-BQFTUohkIToMJNW3puZ3CerpZ6pBnecqhBQ8TBwhSnikLt4BYQTmsBgtKvoCFi-JuLXQAE_Oe_a_d2XOL8KvOmw$ for trouble shooting. [0]PETSC ERROR: Petsc Release Version 3.22.0, unknown [0]PETSC ERROR: ./ex65shell with 1 MPI process(es) and PETSC_ARCH ?on signalkuppe by matteo Tue Dec 10 12:55:12 2024 [0]PETSC ERROR: Configure options: --prefix=/home/matteo/software/petscsaved/3.22-opt/ PETSC_DIR=/home/matteo/software/petsc --PETSC_ARCH=dbg --with-debugging=1 --with-st rict-petscerrorcode --download-hdf5 --download-ml --with-metis --with-parmetis --with-gmsh --with-triangle --with-zlib --with-p4est-dir=~/software/p4est/local/ [0]PETSC ERROR: #1 PetscObjectReference() at /home/matteo/software/petsc/src/sys/objects/inherit.c:620 [0]PETSC ERROR: #2 PCMGSetRScale() at /home/matteo/software/petsc/src/ksp/pc/impls/mg/mgfunc.c:394 [0]PETSC ERROR: #3 PCSetUp_MG() at /home/matteo/software/petsc/src/ksp/pc/impls/mg/mg.c:998 [0]PETSC ERROR: #4 PCSetUp() at /home/matteo/software/petsc/src/ksp/pc/interface/precon.c:1071 [0]PETSC ERROR: #5 KSPSetUp() at /home/matteo/software/petsc/src/ksp/ksp/interface/itfunc.c:415 [0]PETSC ERROR: #6 KSPSolve_Private() at /home/matteo/software/petsc/src/ksp/ksp/interface/itfunc.c:826 [0]PETSC ERROR: #7 KSPSolve() at /home/matteo/software/petsc/src/ksp/ksp/interface/itfunc.c:1075 [0]PETSC ERROR: #8 main() at ../src/ex65shell.c:79 [0]PETSC ERROR: PETSc Option Table entries: [0]PETSC ERROR: -da_refine 2 (source: command line) [0]PETSC ERROR: -ksp_monitor (source: command line) [0]PETSC ERROR: -ksp_rtol 1e-1 (source: command line) [0]PETSC ERROR: -pc_type mg (source: command line) [0]PETSC ERROR: ----------------End of Error Message -------send entire error message to petsc-maint at mcs.anl.gov---------- In all cases valgrind complains like $ valgrind ./ex65shell -ksp_monitor -pc_type mg -da_refine 2 -ksp_rtol 1e-1 -int_view ascii::ascii_info ==2130767== Memcheck, a memory error detector ==2130767== Copyright (C) 2002-2022, and GNU GPL'd, by Julian Seward et al. ==2130767== Using Valgrind-3.19.0 and LibVEX; rerun with -h for copyright info ==2130767== Command: ./ex65shell -ksp_monitor -pc_type mg -da_refine 2 -ksp_rtol 1e-1 -int_view ascii::ascii_info ==2130767== hwloc x86 backend cannot work under Valgrind, disabling. May be reenabled by dumping CPUIDs with hwloc-gather-cpuid and reloading them under Valgrind with HWLOC_CPUID_PATH. >> Created DMshell 0xfbf50f0 (0xfb49480) Calling KSPSolve from main computeRHS on grid 513 computeMatrix on grid 513 Inside Coarsen >> Created DMshell 0xff404a0 (0xff062d0) Inside Coarsen >> Created DMshell 0xff78ca0 (0xff4a530) >> Create interpolation from 0xff404a0(0xff062d0) to 0xfbf50f0(0xfb49480) Mat Object: 1 MPI process ?type: shell ?rows=513, cols=257 ==2130767== Conditional jump or move depends on uninitialised value(s) ==2130767== ???at 0x83FDAAE: PCSetUp_MG (mg.c:998) ==2130767== ???by 0x8620C04: PCSetUp (precon.c:1071) ==2130767== ???by 0x7DA78D5: KSPSetUp (itfunc.c:415) ==2130767== ???by 0x7DB2093: KSPSolve_Private (itfunc.c:826) ==2130767== ???by 0x7DB7A53: KSPSolve (itfunc.c:1075) ==2130767== ???by 0x10D2F7: main (ex65shell.c:79) ==2130767== ==2130767== Conditional jump or move depends on uninitialised value(s) ==2130767== ???at 0x5794061: VecDestroy (vector.c:570) ==2130767== ???by 0x83FDC36: PCSetUp_MG (mg.c:999) ==2130767== ???by 0x8620C04: PCSetUp (precon.c:1071) ==2130767== ???by 0x7DA78D5: KSPSetUp (itfunc.c:415) ==2130767== ???by 0x7DB2093: KSPSolve_Private (itfunc.c:826) ==2130767== ???by 0x7DB7A53: KSPSolve (itfunc.c:1075) ==2130767== ???by 0x10D2F7: main (ex65shell.c:79) ==2130767== We are clearly doing something wrong since PCSetUp_MG (mg.c:998) is an area of code where I wouldn't expect we would enter. Can you advise on the proper way to achieve our goal? Best regards ??? Matteo -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ex65.patch Type: text/x-patch Size: 4486 bytes Desc: not available URL: From knepley at gmail.com Tue Dec 10 07:23:31 2024 From: knepley at gmail.com (Matthew Knepley) Date: Tue, 10 Dec 2024 08:23:31 -0500 Subject: [petsc-users] Parallel computing in SNES In-Reply-To: References: Message-ID: On Tue, Dec 10, 2024 at 4:32?AM David Jiawei LUO LIANG < 12431140 at mail.sustech.edu.cn> wrote: > Hello PETSc, > > I want to substitute the existing nonlinear solver codes with SNES. > > The existing code is Parallel computing. > > My FormFunction and FormJacobian is attached in file Res_Jac.cpp. > > The global tangent matrix assembly and residual function assembly are both > implemented by invoking user pointers. > > The following is the message, we can find that SNES just iterate once. > > > I think the problem is input solution Vec y doesn't link to Residual Vec > f, because my update is stored in the user variables. > If I use > const PetscScalar *yy; > VecGetArrayRead(y, &yy); > the value of yy is not identical to Vec y, because there are multi thread. > > So, my problem is how to assembly residual and tangent in parallel > environment? > I am having trouble understanding your question. 1) Can you run with -snes_monitor -snes_converged_reason -ksp_monitor_true_residual -ksp_converged_reason -snes_view and send the output? 2) You have the line VecCopy(user->gassem_ptr->G, f); which looks like it puts the values of your G into f, so I am not sure what the problem is. THanks, Matt > > David Jiawei LUO LIANG > > ??????/??/???/2024 > > ?????????????1088? > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://urldefense.us/v3/__https://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!awWXPHw1GNXswCx9rANXgow3jupJ1On7I9qTmMvSDr0WuSRayb_cCkSj1qvn6spDBOJX-GVhIvjDx8-R6uqs$ -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: C30690D0 at 0F60110C.7F0A586700000000.jpg Type: image/jpeg Size: 67593 bytes Desc: not available URL: From knepley at gmail.com Tue Dec 10 07:37:13 2024 From: knepley at gmail.com (Matthew Knepley) Date: Tue, 10 Dec 2024 08:37:13 -0500 Subject: [petsc-users] PCMG with shell restrictions In-Reply-To: <7071ff6e-52cb-4715-8cbc-bacc6a385363@uninsubria.it> References: <7071ff6e-52cb-4715-8cbc-bacc6a385363@uninsubria.it> Message-ID: On Tue, Dec 10, 2024 at 7:03?AM Matteo Semplice via petsc-users < petsc-users at mcs.anl.gov> wrote: > Dear petsc-users, > > I am trying with a student to modify the MG example 65 to use > mat-shells instead of assembled matrices. Our use case is for a method that > will use custom shell operators and shell interpolation/restrictions. > > To start we have modified ex65 and tried to replace the standard > restriction/interpolation with a shell matrix that performs the same > operations on the DMDA grids. > > I attach our modifications to ex65. > > I will find time to look at the code. Right now, you might look at this tutorial I setup for GMG with a user-defined operator. https://urldefense.us/v3/__https://gitlab.com/petsc/petsc/-/commits/knepley/tutorial-gmg__;!!G_uCfscf7eWS!adh8Ragc8P_2BSKs-gf4XfTABoIU_2W2Fn447_lRBIjR7zJMd9UsbMXr3idQcRxB_wNg8MS-WuJHGa-DiVe-$ Thanks! Matt > The problem is that the code sometimes completes execution and sometimes > errors out like > > $ ./ex65shell -ksp_monitor -pc_type mg -da_refine 2 -ksp_rtol 1e-1 > >> Created DMshell 0x55bbc83c91a0 (0x55bbc8390a80) > Calling KSPSolve from main > computeRHS on grid 513 > computeMatrix on grid 513 > Inside Coarsen > >> Created DMshell 0x55bbc84c5270 (0x55bbc84b2ff0) > Inside Coarsen > >> Created DMshell 0x55bbc84e0a30 (0x55bbc84bdb60) > >> Create interpolation from 0x55bbc84c5270(0x55bbc84b2ff0) to > 0x55bbc83c91a0(0x55bbc8390a80) > [0]PETSC ERROR: --------------------- Error Message > -------------------------------------------------------------- > [0]PETSC ERROR: Invalid pointer > [0]PETSC ERROR: Invalid Pointer to PetscObject: Argument 'obj' (parameter > # 1) > [0]PETSC ERROR: See https://urldefense.us/v3/__https://petsc.org/release/faq/__;!!G_uCfscf7eWS!adh8Ragc8P_2BSKs-gf4XfTABoIU_2W2Fn447_lRBIjR7zJMd9UsbMXr3idQcRxB_wNg8MS-WuJHGbc9Nz8M$ > > for trouble shooting. > [0]PETSC ERROR: Petsc Release Version 3.22.0, unknown > [0]PETSC ERROR: ./ex65shell with 1 MPI process(es) and PETSC_ARCH on > signalkuppe by matteo Tue Dec 10 12:55:12 2024 > [0]PETSC ERROR: Configure options: > --prefix=/home/matteo/software/petscsaved/3.22-opt/ > PETSC_DIR=/home/matteo/software/petsc --PETSC_ARCH=dbg --with-debugging=1 > --with-st > rict-petscerrorcode --download-hdf5 --download-ml --with-metis > --with-parmetis --with-gmsh --with-triangle --with-zlib > --with-p4est-dir=~/software/p4est/local/ > [0]PETSC ERROR: #1 PetscObjectReference() at > /home/matteo/software/petsc/src/sys/objects/inherit.c:620 > [0]PETSC ERROR: #2 PCMGSetRScale() at > /home/matteo/software/petsc/src/ksp/pc/impls/mg/mgfunc.c:394 > [0]PETSC ERROR: #3 PCSetUp_MG() at > /home/matteo/software/petsc/src/ksp/pc/impls/mg/mg.c:998 > [0]PETSC ERROR: #4 PCSetUp() at > /home/matteo/software/petsc/src/ksp/pc/interface/precon.c:1071 > [0]PETSC ERROR: #5 KSPSetUp() at > /home/matteo/software/petsc/src/ksp/ksp/interface/itfunc.c:415 > [0]PETSC ERROR: #6 KSPSolve_Private() at > /home/matteo/software/petsc/src/ksp/ksp/interface/itfunc.c:826 > [0]PETSC ERROR: #7 KSPSolve() at > /home/matteo/software/petsc/src/ksp/ksp/interface/itfunc.c:1075 > [0]PETSC ERROR: #8 main() at ../src/ex65shell.c:79 > [0]PETSC ERROR: PETSc Option Table entries: > [0]PETSC ERROR: -da_refine 2 (source: command line) > [0]PETSC ERROR: -ksp_monitor (source: command line) > [0]PETSC ERROR: -ksp_rtol 1e-1 (source: command line) > [0]PETSC ERROR: -pc_type mg (source: command line) > [0]PETSC ERROR: ----------------End of Error Message -------send entire > error message to petsc-maint at mcs.anl.gov---------- > > In all cases valgrind complains like > > $ valgrind ./ex65shell -ksp_monitor -pc_type mg -da_refine 2 -ksp_rtol > 1e-1 -int_view ascii::ascii_info > ==2130767== Memcheck, a memory error detector > ==2130767== Copyright (C) 2002-2022, and GNU GPL'd, by Julian Seward et > al. > ==2130767== Using Valgrind-3.19.0 and LibVEX; rerun with -h for copyright > info > ==2130767== Command: ./ex65shell -ksp_monitor -pc_type mg -da_refine 2 > -ksp_rtol 1e-1 -int_view ascii::ascii_info > ==2130767== > hwloc x86 backend cannot work under Valgrind, disabling. > May be reenabled by dumping CPUIDs with hwloc-gather-cpuid > and reloading them under Valgrind with HWLOC_CPUID_PATH. > >> Created DMshell 0xfbf50f0 (0xfb49480) > Calling KSPSolve from main > computeRHS on grid 513 > computeMatrix on grid 513 > Inside Coarsen > >> Created DMshell 0xff404a0 (0xff062d0) > Inside Coarsen > >> Created DMshell 0xff78ca0 (0xff4a530) > >> Create interpolation from 0xff404a0(0xff062d0) to 0xfbf50f0(0xfb49480) > Mat Object: 1 MPI process > type: shell > rows=513, cols=257 > ==2130767== Conditional jump or move depends on uninitialised value(s) > ==2130767== at 0x83FDAAE: PCSetUp_MG (mg.c:998) > ==2130767== by 0x8620C04: PCSetUp (precon.c:1071) > ==2130767== by 0x7DA78D5: KSPSetUp (itfunc.c:415) > ==2130767== by 0x7DB2093: KSPSolve_Private (itfunc.c:826) > ==2130767== by 0x7DB7A53: KSPSolve (itfunc.c:1075) > ==2130767== by 0x10D2F7: main (ex65shell.c:79) > ==2130767== > ==2130767== Conditional jump or move depends on uninitialised value(s) > ==2130767== at 0x5794061: VecDestroy (vector.c:570) > ==2130767== by 0x83FDC36: PCSetUp_MG (mg.c:999) > ==2130767== by 0x8620C04: PCSetUp (precon.c:1071) > ==2130767== by 0x7DA78D5: KSPSetUp (itfunc.c:415) > ==2130767== by 0x7DB2093: KSPSolve_Private (itfunc.c:826) > ==2130767== by 0x7DB7A53: KSPSolve (itfunc.c:1075) > ==2130767== by 0x10D2F7: main (ex65shell.c:79) > ==2130767== > > We are clearly doing something wrong since PCSetUp_MG (mg.c:998) is an > area of code where I wouldn't expect we would enter. > > Can you advise on the proper way to achieve our goal? > > Best regards > > Matteo > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://urldefense.us/v3/__https://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!adh8Ragc8P_2BSKs-gf4XfTABoIU_2W2Fn447_lRBIjR7zJMd9UsbMXr3idQcRxB_wNg8MS-WuJHGdoVXWI2$ -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at petsc.dev Tue Dec 10 09:31:35 2024 From: bsmith at petsc.dev (Barry Smith) Date: Tue, 10 Dec 2024 10:31:35 -0500 Subject: [petsc-users] Issue with SNES with petsc4py 3.22 In-Reply-To: References: Message-ID: <44778586-8CEB-4475-BAEA-9C8243269EF8@petsc.dev> The callbacks for SNES, such as SNESSetFunction(), are not allowed to change the value of the solution x. The updates to the solution are controlled by the nonlinear solver being used by SNES; for example, with Newton's method with line search, the solution is updated by the SNES Newton solver after a successful linear search is computed. Could you please fully explain the context in which you wish to change the value of x during the SNES solve directly and what callback the issue is occurring in? Barry > On Dec 10, 2024, at 3:27?AM, Blauth, Sebastian wrote: > > Hi everyone, > > I am having trouble with using SNES from petsc4py in PETSc 3.22. The problem is similar to https://urldefense.us/v3/__https://gitlab.com/petsc/petsc/-/issues/1611__;!!G_uCfscf7eWS!e_sGSUu4uYIocD18SOI4JfyPT1NUuUlkmunWUuuC6WaK00lEvX4Blk795PTQ5WYfjVVE3jQP6g9RcRiDAzJ6Xd4$ and related to the readonly access of ?x?. Particularly, in my application, based on FEniCS, I want to solve a nonlinear PDE. Therefore, in my callbacks, I want to set the vector of DoFs of my FEniCS function to ?x? with a line like > > self.u.vector().vec().setArray(x) > > However, this now raises the error > > [0] Vector 'Vec_0x84000003_0' (argument #1) was locked for read-only access in unknown_function() at unknown file:0 (line numbers only accurate to function begin) > > I believe that this is a consequence of https://urldefense.us/v3/__https://gitlab.com/petsc/petsc/-/merge_requests/7567*22c226fc29b6fc9dc5781b0aff4dcc70ac6c3489__;Iw!!G_uCfscf7eWS!e_sGSUu4uYIocD18SOI4JfyPT1NUuUlkmunWUuuC6WaK00lEvX4Blk795PTQ5WYfjVVE3jQP6g9RcRiDGTemu34$ , but I did not find any information on how to remove the lock from the vector or how I can set use ?x? to update the vector of my function. Using > > xx = x.getArray(readonly=True) > > as described in https://urldefense.us/v3/__https://gitlab.com/petsc/petsc/-/issues/1611__;!!G_uCfscf7eWS!e_sGSUu4uYIocD18SOI4JfyPT1NUuUlkmunWUuuC6WaK00lEvX4Blk795PTQ5WYfjVVE3jQP6g9RcRiDAzJ6Xd4$ does not work either, as calling self.u.vector().vec().setArray(xx) then fails with the same error message. > > Does anyone have an idea how to fix this? > Thanks a lot in advance, > Sebastian > > > > -- > Dr. Sebastian Blauth > Fraunhofer-Institut f?r > Techno- und Wirtschaftsmathematik ITWM > Abteilung Transportvorg?nge > Fraunhofer-Platz 1, 67663 Kaiserslautern > Telefon: +49 631 31600-4968 > sebastian.blauth at itwm.fraunhofer.de > https://urldefense.us/v3/__https://www.itwm.fraunhofer.de__;!!G_uCfscf7eWS!e_sGSUu4uYIocD18SOI4JfyPT1NUuUlkmunWUuuC6WaK00lEvX4Blk795PTQ5WYfjVVE3jQP6g9RcRiDineKawk$ -------------- next part -------------- An HTML attachment was scrubbed... URL: From sebastian.blauth at itwm.fraunhofer.de Tue Dec 10 09:35:55 2024 From: sebastian.blauth at itwm.fraunhofer.de (Blauth, Sebastian) Date: Tue, 10 Dec 2024 15:35:55 +0000 Subject: [petsc-users] Issue with SNES with petsc4py 3.22 In-Reply-To: <44778586-8CEB-4475-BAEA-9C8243269EF8@petsc.dev> References: <44778586-8CEB-4475-BAEA-9C8243269EF8@petsc.dev> Message-ID: Thanks fort he clarification. And it makes sense that x is read-only. I solved my issue by actually creating a new, empty vector which is used in snes.solve and can update my FEniCS vector from this, see: https://github.com/sblauth/cashocs/pull/533 What I did not get at first is that PETSc.VEC.setArray seems to create a shallow copy, whereas PETSc.VEC.aypx(0.0, ?) seems to create something like a deep copy. Using aypx this is not an issue anymore. Best, Sebastian -- Dr. Sebastian Blauth Fraunhofer-Institut f?r Techno- und Wirtschaftsmathematik ITWM Abteilung Transportvorg?nge Fraunhofer-Platz 1, 67663 Kaiserslautern Telefon: +49 631 31600-4968 sebastian.blauth at itwm.fraunhofer.de https://www.itwm.fraunhofer.de From: Barry Smith Sent: Tuesday, December 10, 2024 4:32 PM To: Blauth, Sebastian Cc: petsc-users at mcs.anl.gov Subject: Re: [petsc-users] Issue with SNES with petsc4py 3.22 The callbacks for SNES, such as SNESSetFunction(), are not allowed to change the value of the solution x. The updates to the solution are controlled by the nonlinear solver being used by SNES; for example, with Newton's method with line search, the solution is updated by the SNES Newton solver after a successful linear search is computed. Could you please fully explain the context in which you wish to change the value of x during the SNES solve directly and what callback the issue is occurring in? Barry On Dec 10, 2024, at 3:27?AM, Blauth, Sebastian > wrote: Hi everyone, I am having trouble with using SNES from petsc4py in PETSc 3.22. The problem is similar to https://gitlab.com/petsc/petsc/-/issues/1611 and related to the readonly access of ?x?. Particularly, in my application, based on FEniCS, I want to solve a nonlinear PDE. Therefore, in my callbacks, I want to set the vector of DoFs of my FEniCS function to ?x? with a line like self.u.vector().vec().setArray(x) However, this now raises the error [0] Vector 'Vec_0x84000003_0' (argument #1) was locked for read-only access in unknown_function() at unknown file:0 (line numbers only accurate to function begin) I believe that this is a consequence of https://gitlab.com/petsc/petsc/-/merge_requests/7567#22c226fc29b6fc9dc5781b0aff4dcc70ac6c3489, but I did not find any information on how to remove the lock from the vector or how I can set use ?x? to update the vector of my function. Using xx = x.getArray(readonly=True) as described in https://gitlab.com/petsc/petsc/-/issues/1611 does not work either, as calling self.u.vector().vec().setArray(xx) then fails with the same error message. Does anyone have an idea how to fix this? Thanks a lot in advance, Sebastian -- Dr. Sebastian Blauth Fraunhofer-Institut f?r Techno- und Wirtschaftsmathematik ITWM Abteilung Transportvorg?nge Fraunhofer-Platz 1, 67663 Kaiserslautern Telefon: +49 631 31600-4968 sebastian.blauth at itwm.fraunhofer.de https://www.itwm.fraunhofer.de -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 7943 bytes Desc: not available URL: From bsmith at petsc.dev Tue Dec 10 09:38:08 2024 From: bsmith at petsc.dev (Barry Smith) Date: Tue, 10 Dec 2024 10:38:08 -0500 Subject: [petsc-users] PCMG with shell restrictions In-Reply-To: <7071ff6e-52cb-4715-8cbc-bacc6a385363@uninsubria.it> References: <7071ff6e-52cb-4715-8cbc-bacc6a385363@uninsubria.it> Message-ID: <97C19201-BA91-4661-9E03-0CD1817AA663@petsc.dev> It appears you are completely ignoring the vec argument? Take a look at, for example, DMCreateInterpolation_DA() you will see you need to provide an appropriate vec in the same way you need (and do) provide an appropriate mat. Barry > On Dec 10, 2024, at 7:02?AM, Matteo Semplice via petsc-users wrote: > > Dear petsc-users, > > I am trying with a student to modify the MG example 65 to use mat-shells instead of assembled matrices. Our use case is for a method that will use custom shell operators and shell interpolation/restrictions. > > To start we have modified ex65 and tried to replace the standard restriction/interpolation with a shell matrix that performs the same operations on the DMDA grids. > > I attach our modifications to ex65. > > The problem is that the code sometimes completes execution and sometimes errors out like > > $ ./ex65shell -ksp_monitor -pc_type mg -da_refine 2 -ksp_rtol 1e-1 > >> Created DMshell 0x55bbc83c91a0 (0x55bbc8390a80) > Calling KSPSolve from main > computeRHS on grid 513 > computeMatrix on grid 513 > Inside Coarsen > >> Created DMshell 0x55bbc84c5270 (0x55bbc84b2ff0) > Inside Coarsen > >> Created DMshell 0x55bbc84e0a30 (0x55bbc84bdb60) > >> Create interpolation from 0x55bbc84c5270(0x55bbc84b2ff0) to 0x55bbc83c91a0(0x55bbc8390a80) > [0]PETSC ERROR: --------------------- Error Message -------------------------------------------------------------- > [0]PETSC ERROR: Invalid pointer > [0]PETSC ERROR: Invalid Pointer to PetscObject: Argument 'obj' (parameter # 1) > [0]PETSC ERROR: See https://urldefense.us/v3/__https://petsc.org/release/faq/__;!!G_uCfscf7eWS!ZLNRcAWCboFilZVAcU1gjUddxWbwjb25jRNK4CCZCz9YuJ22SpOf-bCIgcXbHcwhuzs4UiqEsjcnvKnre3OGa-c$ for trouble shooting. > [0]PETSC ERROR: Petsc Release Version 3.22.0, unknown > [0]PETSC ERROR: ./ex65shell with 1 MPI process(es) and PETSC_ARCH on signalkuppe by matteo Tue Dec 10 12:55:12 2024 > [0]PETSC ERROR: Configure options: --prefix=/home/matteo/software/petscsaved/3.22-opt/ PETSC_DIR=/home/matteo/software/petsc --PETSC_ARCH=dbg --with-debugging=1 --with-st > rict-petscerrorcode --download-hdf5 --download-ml --with-metis --with-parmetis --with-gmsh --with-triangle --with-zlib --with-p4est-dir=~/software/p4est/local/ > [0]PETSC ERROR: #1 PetscObjectReference() at /home/matteo/software/petsc/src/sys/objects/inherit.c:620 > [0]PETSC ERROR: #2 PCMGSetRScale() at /home/matteo/software/petsc/src/ksp/pc/impls/mg/mgfunc.c:394 > [0]PETSC ERROR: #3 PCSetUp_MG() at /home/matteo/software/petsc/src/ksp/pc/impls/mg/mg.c:998 > [0]PETSC ERROR: #4 PCSetUp() at /home/matteo/software/petsc/src/ksp/pc/interface/precon.c:1071 > [0]PETSC ERROR: #5 KSPSetUp() at /home/matteo/software/petsc/src/ksp/ksp/interface/itfunc.c:415 > [0]PETSC ERROR: #6 KSPSolve_Private() at /home/matteo/software/petsc/src/ksp/ksp/interface/itfunc.c:826 > [0]PETSC ERROR: #7 KSPSolve() at /home/matteo/software/petsc/src/ksp/ksp/interface/itfunc.c:1075 > [0]PETSC ERROR: #8 main() at ../src/ex65shell.c:79 > [0]PETSC ERROR: PETSc Option Table entries: > [0]PETSC ERROR: -da_refine 2 (source: command line) > [0]PETSC ERROR: -ksp_monitor (source: command line) > [0]PETSC ERROR: -ksp_rtol 1e-1 (source: command line) > [0]PETSC ERROR: -pc_type mg (source: command line) > [0]PETSC ERROR: ----------------End of Error Message -------send entire error message to petsc-maint at mcs.anl.gov ---------- > > In all cases valgrind complains like > > $ valgrind ./ex65shell -ksp_monitor -pc_type mg -da_refine 2 -ksp_rtol 1e-1 -int_view ascii::ascii_info > ==2130767== Memcheck, a memory error detector > ==2130767== Copyright (C) 2002-2022, and GNU GPL'd, by Julian Seward et al. > ==2130767== Using Valgrind-3.19.0 and LibVEX; rerun with -h for copyright info > ==2130767== Command: ./ex65shell -ksp_monitor -pc_type mg -da_refine 2 -ksp_rtol 1e-1 -int_view ascii::ascii_info > ==2130767== > hwloc x86 backend cannot work under Valgrind, disabling. > May be reenabled by dumping CPUIDs with hwloc-gather-cpuid > and reloading them under Valgrind with HWLOC_CPUID_PATH. > >> Created DMshell 0xfbf50f0 (0xfb49480) > Calling KSPSolve from main > computeRHS on grid 513 > computeMatrix on grid 513 > Inside Coarsen > >> Created DMshell 0xff404a0 (0xff062d0) > Inside Coarsen > >> Created DMshell 0xff78ca0 (0xff4a530) > >> Create interpolation from 0xff404a0(0xff062d0) to 0xfbf50f0(0xfb49480) > Mat Object: 1 MPI process > type: shell > rows=513, cols=257 > ==2130767== Conditional jump or move depends on uninitialised value(s) > ==2130767== at 0x83FDAAE: PCSetUp_MG (mg.c:998) > ==2130767== by 0x8620C04: PCSetUp (precon.c:1071) > ==2130767== by 0x7DA78D5: KSPSetUp (itfunc.c:415) > ==2130767== by 0x7DB2093: KSPSolve_Private (itfunc.c:826) > ==2130767== by 0x7DB7A53: KSPSolve (itfunc.c:1075) > ==2130767== by 0x10D2F7: main (ex65shell.c:79) > ==2130767== > ==2130767== Conditional jump or move depends on uninitialised value(s) > ==2130767== at 0x5794061: VecDestroy (vector.c:570) > ==2130767== by 0x83FDC36: PCSetUp_MG (mg.c:999) > ==2130767== by 0x8620C04: PCSetUp (precon.c:1071) > ==2130767== by 0x7DA78D5: KSPSetUp (itfunc.c:415) > ==2130767== by 0x7DB2093: KSPSolve_Private (itfunc.c:826) > ==2130767== by 0x7DB7A53: KSPSolve (itfunc.c:1075) > ==2130767== by 0x10D2F7: main (ex65shell.c:79) > ==2130767== > > We are clearly doing something wrong since PCSetUp_MG (mg.c:998) is an area of code where I wouldn't expect we would enter. > > Can you advise on the proper way to achieve our goal? > > Best regards > > Matteo > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sblondel at utk.edu Tue Dec 10 10:08:13 2024 From: sblondel at utk.edu (Blondel, Sophie) Date: Tue, 10 Dec 2024 16:08:13 +0000 Subject: [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED In-Reply-To: References: <1E4CF02D-32EB-482B-ACFC-9AE1C7F9102E@petsc.dev> Message-ID: Good morning Barry, Attached are the updated files, there is more useful information in them. Cheers, Sophie ________________________________ From: Blondel, Sophie via Xolotl-psi-development Sent: Monday, December 9, 2024 17:29 To: Barry Smith Cc: petsc-users at mcs.anl.gov ; xolotl-psi-development at lists.sourceforge.net Subject: Re: [Xolotl-psi-development] [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED Hi Barry, I hope you are doing well. Attached are the output. To give a little more context, this is a "new" way of running the code where multiple instances are created and communicate together every few time steps (like coupling the code with itself in memory). Here there are 3 instances that each have a separate TS object, plus one "main" instance that doesn't solve anything but compute rates to exchange between the other instances. Cheers, Sophie ________________________________ From: Barry Smith Sent: Monday, December 9, 2024 15:12 To: Blondel, Sophie Cc: petsc-users at mcs.anl.gov ; xolotl-psi-development at lists.sourceforge.net Subject: Re: [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED On Dec 9, 2024, at 2:56?PM, Blondel, Sophie via petsc-users wrote: Hi, I am trying to understand a strange behavior I'm encountering: when running my application with "-ts_exact_final_time stepover" everything goes well, but when I switch to "matchstep" I get DIVERGED_STEP_REJECTED before the first time step is finished. This is in the very first time-step in TSSolve? Please run with -ts_monitor and send all the output (best for a short time interval and do it twice once with -ts_exact_final_time stepover and once with exact. Barry I tried increasing the maximum number of rejections and it just takes longer to diverge, and if I set the value to "unlimited" it is basically an infinite loop. Is there a way to check why is the step rejected? Could the "matchstep" option change tolerances somewhere that would cause that behavior? Let me know if I should provide more information. Best, Sophie Blondel -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: matchstep.txt URL: -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: stepover.txt URL: From sblondel at utk.edu Tue Dec 10 10:10:49 2024 From: sblondel at utk.edu (Blondel, Sophie) Date: Tue, 10 Dec 2024 16:10:49 +0000 Subject: [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED In-Reply-To: References: <1E4CF02D-32EB-482B-ACFC-9AE1C7F9102E@petsc.dev> Message-ID: Now I see that the dt is set to 0 s in that case instead of 1.0e-12 s. Sophie ________________________________ From: Blondel, Sophie Sent: Tuesday, December 10, 2024 11:08 To: Barry Smith ; Blondel, Sophie Cc: petsc-users at mcs.anl.gov ; xolotl-psi-development at lists.sourceforge.net Subject: Re: [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED Good morning Barry, Attached are the updated files, there is more useful information in them. Cheers, Sophie ________________________________ From: Blondel, Sophie via Xolotl-psi-development Sent: Monday, December 9, 2024 17:29 To: Barry Smith Cc: petsc-users at mcs.anl.gov ; xolotl-psi-development at lists.sourceforge.net Subject: Re: [Xolotl-psi-development] [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED Hi Barry, I hope you are doing well. Attached are the output. To give a little more context, this is a "new" way of running the code where multiple instances are created and communicate together every few time steps (like coupling the code with itself in memory). Here there are 3 instances that each have a separate TS object, plus one "main" instance that doesn't solve anything but compute rates to exchange between the other instances. Cheers, Sophie ________________________________ From: Barry Smith Sent: Monday, December 9, 2024 15:12 To: Blondel, Sophie Cc: petsc-users at mcs.anl.gov ; xolotl-psi-development at lists.sourceforge.net Subject: Re: [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED On Dec 9, 2024, at 2:56?PM, Blondel, Sophie via petsc-users wrote: Hi, I am trying to understand a strange behavior I'm encountering: when running my application with "-ts_exact_final_time stepover" everything goes well, but when I switch to "matchstep" I get DIVERGED_STEP_REJECTED before the first time step is finished. This is in the very first time-step in TSSolve? Please run with -ts_monitor and send all the output (best for a short time interval and do it twice once with -ts_exact_final_time stepover and once with exact. Barry I tried increasing the maximum number of rejections and it just takes longer to diverge, and if I set the value to "unlimited" it is basically an infinite loop. Is there a way to check why is the step rejected? Could the "matchstep" option change tolerances somewhere that would cause that behavior? Let me know if I should provide more information. Best, Sophie Blondel -------------- next part -------------- An HTML attachment was scrubbed... URL: From matteo.semplice at uninsubria.it Tue Dec 10 10:20:16 2024 From: matteo.semplice at uninsubria.it (Matteo Semplice) Date: Tue, 10 Dec 2024 17:20:16 +0100 Subject: [petsc-users] PCMG with shell restrictions In-Reply-To: <97C19201-BA91-4661-9E03-0CD1817AA663@petsc.dev> References: <7071ff6e-52cb-4715-8cbc-bacc6a385363@uninsubria.it> <97C19201-BA91-4661-9E03-0CD1817AA663@petsc.dev> Message-ID: <46997481-abe3-4d76-888b-95db02d3617a@uninsubria.it> Thanks, Barry. Just to be sure, one should refer to the help of https://urldefense.us/v3/__https://petsc.org/release/manualpages/DM/DMCreateInterpolationScale/__;!!G_uCfscf7eWS!c1PFRpf9jqVzaF3p98Od2T92U1LOgy4Q3oqZCcYrv2QsIO885A3iP7Lo3XpDveKwS7CMeHxzVseroufdL1mP8nbei0E_jEnpPsLtNQ$ and the vec should be set to the point-wise inverse of (INTERPOLATION)*(vector of ones)? ? Matteo On 10/12/2024 16:38, Barry Smith wrote: > > ? ?It appears you are completely ignoring the vec argument? Take a > look at, for example,?DMCreateInterpolation_DA() you will see you need > to provide an appropriate vec in the same way you need (and do) > provide an appropriate mat. > > ? ?Barry > > >> On Dec 10, 2024, at 7:02?AM, Matteo Semplice via petsc-users >> wrote: >> >> Dear petsc-users, >> >> ??? I am trying with a student to modify the MG example 65 to use >> mat-shells instead of assembled matrices. Our use case is for a >> method that will use custom shell operators and shell >> interpolation/restrictions. >> >> To start we have modified ex65 and tried to replace the standard >> restriction/interpolation with a shell matrix that performs the same >> operations on the DMDA grids. >> >> I attach our modifications to ex65. >> >> The problem is that the code sometimes completes execution and >> sometimes errors out like >> >> $ ./ex65shell -ksp_monitor -pc_type mg -da_refine 2 -ksp_rtol 1e-1 >> >> Created DMshell 0x55bbc83c91a0 (0x55bbc8390a80) >> Calling KSPSolve from main >> computeRHS on grid 513 >> computeMatrix on grid 513 >> Inside Coarsen >> >> Created DMshell 0x55bbc84c5270 (0x55bbc84b2ff0) >> Inside Coarsen >> >> Created DMshell 0x55bbc84e0a30 (0x55bbc84bdb60) >> >> Create interpolation from 0x55bbc84c5270(0x55bbc84b2ff0) to >> 0x55bbc83c91a0(0x55bbc8390a80) >> [0]PETSC ERROR: --------------------- Error Message >> -------------------------------------------------------------- >> [0]PETSC ERROR: Invalid pointer >> [0]PETSC ERROR: Invalid Pointer to PetscObject: Argument 'obj' >> (parameter # 1) >> [0]PETSC ERROR: See https://urldefense.us/v3/__https://petsc.org/release/faq/__;!!G_uCfscf7eWS!c1PFRpf9jqVzaF3p98Od2T92U1LOgy4Q3oqZCcYrv2QsIO885A3iP7Lo3XpDveKwS7CMeHxzVseroufdL1mP8nbei0E_jEmoPycXyw$ for trouble shooting. >> [0]PETSC ERROR: Petsc Release Version 3.22.0, unknown >> [0]PETSC ERROR: ./ex65shell with 1 MPI process(es) and PETSC_ARCH ?on >> signalkuppe by matteo Tue Dec 10 12:55:12 2024 >> [0]PETSC ERROR: Configure options: >> --prefix=/home/matteo/software/petscsaved/3.22-opt/ >> PETSC_DIR=/home/matteo/software/petsc --PETSC_ARCH=dbg >> --with-debugging=1 --with-st >> rict-petscerrorcode --download-hdf5 --download-ml --with-metis >> --with-parmetis --with-gmsh --with-triangle --with-zlib >> --with-p4est-dir=~/software/p4est/local/ >> [0]PETSC ERROR: #1 PetscObjectReference() at >> /home/matteo/software/petsc/src/sys/objects/inherit.c:620 >> [0]PETSC ERROR: #2 PCMGSetRScale() at >> /home/matteo/software/petsc/src/ksp/pc/impls/mg/mgfunc.c:394 >> [0]PETSC ERROR: #3 PCSetUp_MG() at >> /home/matteo/software/petsc/src/ksp/pc/impls/mg/mg.c:998 >> [0]PETSC ERROR: #4 PCSetUp() at >> /home/matteo/software/petsc/src/ksp/pc/interface/precon.c:1071 >> [0]PETSC ERROR: #5 KSPSetUp() at >> /home/matteo/software/petsc/src/ksp/ksp/interface/itfunc.c:415 >> [0]PETSC ERROR: #6 KSPSolve_Private() at >> /home/matteo/software/petsc/src/ksp/ksp/interface/itfunc.c:826 >> [0]PETSC ERROR: #7 KSPSolve() at >> /home/matteo/software/petsc/src/ksp/ksp/interface/itfunc.c:1075 >> [0]PETSC ERROR: #8 main() at ../src/ex65shell.c:79 >> [0]PETSC ERROR: PETSc Option Table entries: >> [0]PETSC ERROR: -da_refine 2 (source: command line) >> [0]PETSC ERROR: -ksp_monitor (source: command line) >> [0]PETSC ERROR: -ksp_rtol 1e-1 (source: command line) >> [0]PETSC ERROR: -pc_type mg (source: command line) >> [0]PETSC ERROR: ----------------End of Error Message -------send >> entire error message to petsc-maint at mcs.anl.gov---------- >> >> In all cases valgrind complains like >> >> $ valgrind ./ex65shell -ksp_monitor -pc_type mg -da_refine 2 >> -ksp_rtol 1e-1 -int_view ascii::ascii_info >> ==2130767== Memcheck, a memory error detector >> ==2130767== Copyright (C) 2002-2022, and GNU GPL'd, by Julian Seward >> et al. >> ==2130767== Using Valgrind-3.19.0 and LibVEX; rerun with -h for >> copyright info >> ==2130767== Command: ./ex65shell -ksp_monitor -pc_type mg -da_refine >> 2 -ksp_rtol 1e-1 -int_view ascii::ascii_info >> ==2130767== >> hwloc x86 backend cannot work under Valgrind, disabling. >> May be reenabled by dumping CPUIDs with hwloc-gather-cpuid >> and reloading them under Valgrind with HWLOC_CPUID_PATH. >> >> Created DMshell 0xfbf50f0 (0xfb49480) >> Calling KSPSolve from main >> computeRHS on grid 513 >> computeMatrix on grid 513 >> Inside Coarsen >> >> Created DMshell 0xff404a0 (0xff062d0) >> Inside Coarsen >> >> Created DMshell 0xff78ca0 (0xff4a530) >> >> Create interpolation from 0xff404a0(0xff062d0) to >> 0xfbf50f0(0xfb49480) >> Mat Object: 1 MPI process >> ?type: shell >> ?rows=513, cols=257 >> ==2130767== Conditional jump or move depends on uninitialised value(s) >> ==2130767== ???at 0x83FDAAE: PCSetUp_MG (mg.c:998) >> ==2130767== ???by 0x8620C04: PCSetUp (precon.c:1071) >> ==2130767== ???by 0x7DA78D5: KSPSetUp (itfunc.c:415) >> ==2130767== ???by 0x7DB2093: KSPSolve_Private (itfunc.c:826) >> ==2130767== ???by 0x7DB7A53: KSPSolve (itfunc.c:1075) >> ==2130767== ???by 0x10D2F7: main (ex65shell.c:79) >> ==2130767== >> ==2130767== Conditional jump or move depends on uninitialised value(s) >> ==2130767== ???at 0x5794061: VecDestroy (vector.c:570) >> ==2130767== ???by 0x83FDC36: PCSetUp_MG (mg.c:999) >> ==2130767== ???by 0x8620C04: PCSetUp (precon.c:1071) >> ==2130767== ???by 0x7DA78D5: KSPSetUp (itfunc.c:415) >> ==2130767== ???by 0x7DB2093: KSPSolve_Private (itfunc.c:826) >> ==2130767== ???by 0x7DB7A53: KSPSolve (itfunc.c:1075) >> ==2130767== ???by 0x10D2F7: main (ex65shell.c:79) >> ==2130767== >> >> We are clearly doing something wrong since PCSetUp_MG (mg.c:998)? is >> an area of code where I wouldn't expect we would enter. >> >> Can you advise on the proper way to achieve our goal? >> >> Best regards >> >> ??? Matteo >> >> > -- Prof. Matteo Semplice Universit? degli Studi dell?Insubria Dipartimento di Scienza e Alta Tecnologia ? DiSAT Professore Associato Via Valleggio, 11 ? 22100 Como (CO) ? Italia tel.: +39 031 2386316 -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at petsc.dev Tue Dec 10 10:27:22 2024 From: bsmith at petsc.dev (Barry Smith) Date: Tue, 10 Dec 2024 11:27:22 -0500 Subject: [petsc-users] PCMG with shell restrictions In-Reply-To: <46997481-abe3-4d76-888b-95db02d3617a@uninsubria.it> References: <7071ff6e-52cb-4715-8cbc-bacc6a385363@uninsubria.it> <97C19201-BA91-4661-9E03-0CD1817AA663@petsc.dev> <46997481-abe3-4d76-888b-95db02d3617a@uninsubria.it> Message-ID: <6310733B-AECC-40AF-B63F-95ADBC9A8DA3@petsc.dev> Beats me :-) Just set it to whatever the DMDA sets it too Barry > On Dec 10, 2024, at 11:20?AM, Matteo Semplice wrote: > > Thanks, Barry. > > Just to be sure, one should refer to the help of https://urldefense.us/v3/__https://petsc.org/release/manualpages/DM/DMCreateInterpolationScale/__;!!G_uCfscf7eWS!adeZHH1r8zFo3A5H1wpZDveIiI1ryL_lOJwl2MsOC0uuANOMda4SvkEHBjoh9sZgxTrk1G2RbGRejVvgyba9OYQ$ and the vec should be set to the point-wise inverse of (INTERPOLATION)*(vector of ones) ? > > Matteo > > On 10/12/2024 16:38, Barry Smith wrote: >> >> It appears you are completely ignoring the vec argument? Take a look at, for example, DMCreateInterpolation_DA() you will see you need to provide an appropriate vec in the same way you need (and do) provide an appropriate mat. >> >> Barry >> >> >>> On Dec 10, 2024, at 7:02?AM, Matteo Semplice via petsc-users wrote: >>> >>> Dear petsc-users, >>> >>> I am trying with a student to modify the MG example 65 to use mat-shells instead of assembled matrices. Our use case is for a method that will use custom shell operators and shell interpolation/restrictions. >>> >>> To start we have modified ex65 and tried to replace the standard restriction/interpolation with a shell matrix that performs the same operations on the DMDA grids. >>> >>> I attach our modifications to ex65. >>> >>> The problem is that the code sometimes completes execution and sometimes errors out like >>> >>> $ ./ex65shell -ksp_monitor -pc_type mg -da_refine 2 -ksp_rtol 1e-1 >>> >> Created DMshell 0x55bbc83c91a0 (0x55bbc8390a80) >>> Calling KSPSolve from main >>> computeRHS on grid 513 >>> computeMatrix on grid 513 >>> Inside Coarsen >>> >> Created DMshell 0x55bbc84c5270 (0x55bbc84b2ff0) >>> Inside Coarsen >>> >> Created DMshell 0x55bbc84e0a30 (0x55bbc84bdb60) >>> >> Create interpolation from 0x55bbc84c5270(0x55bbc84b2ff0) to 0x55bbc83c91a0(0x55bbc8390a80) >>> [0]PETSC ERROR: --------------------- Error Message -------------------------------------------------------------- >>> [0]PETSC ERROR: Invalid pointer >>> [0]PETSC ERROR: Invalid Pointer to PetscObject: Argument 'obj' (parameter # 1) >>> [0]PETSC ERROR: See https://urldefense.us/v3/__https://petsc.org/release/faq/__;!!G_uCfscf7eWS!adeZHH1r8zFo3A5H1wpZDveIiI1ryL_lOJwl2MsOC0uuANOMda4SvkEHBjoh9sZgxTrk1G2RbGRejVvgZJWXKXM$ for trouble shooting. >>> [0]PETSC ERROR: Petsc Release Version 3.22.0, unknown >>> [0]PETSC ERROR: ./ex65shell with 1 MPI process(es) and PETSC_ARCH on signalkuppe by matteo Tue Dec 10 12:55:12 2024 >>> [0]PETSC ERROR: Configure options: --prefix=/home/matteo/software/petscsaved/3.22-opt/ PETSC_DIR=/home/matteo/software/petsc --PETSC_ARCH=dbg --with-debugging=1 --with-st >>> rict-petscerrorcode --download-hdf5 --download-ml --with-metis --with-parmetis --with-gmsh --with-triangle --with-zlib --with-p4est-dir=~/software/p4est/local/ >>> [0]PETSC ERROR: #1 PetscObjectReference() at /home/matteo/software/petsc/src/sys/objects/inherit.c:620 >>> [0]PETSC ERROR: #2 PCMGSetRScale() at /home/matteo/software/petsc/src/ksp/pc/impls/mg/mgfunc.c:394 >>> [0]PETSC ERROR: #3 PCSetUp_MG() at /home/matteo/software/petsc/src/ksp/pc/impls/mg/mg.c:998 >>> [0]PETSC ERROR: #4 PCSetUp() at /home/matteo/software/petsc/src/ksp/pc/interface/precon.c:1071 >>> [0]PETSC ERROR: #5 KSPSetUp() at /home/matteo/software/petsc/src/ksp/ksp/interface/itfunc.c:415 >>> [0]PETSC ERROR: #6 KSPSolve_Private() at /home/matteo/software/petsc/src/ksp/ksp/interface/itfunc.c:826 >>> [0]PETSC ERROR: #7 KSPSolve() at /home/matteo/software/petsc/src/ksp/ksp/interface/itfunc.c:1075 >>> [0]PETSC ERROR: #8 main() at ../src/ex65shell.c:79 >>> [0]PETSC ERROR: PETSc Option Table entries: >>> [0]PETSC ERROR: -da_refine 2 (source: command line) >>> [0]PETSC ERROR: -ksp_monitor (source: command line) >>> [0]PETSC ERROR: -ksp_rtol 1e-1 (source: command line) >>> [0]PETSC ERROR: -pc_type mg (source: command line) >>> [0]PETSC ERROR: ----------------End of Error Message -------send entire error message to petsc-maint at mcs.anl.gov ---------- >>> >>> In all cases valgrind complains like >>> >>> $ valgrind ./ex65shell -ksp_monitor -pc_type mg -da_refine 2 -ksp_rtol 1e-1 -int_view ascii::ascii_info >>> ==2130767== Memcheck, a memory error detector >>> ==2130767== Copyright (C) 2002-2022, and GNU GPL'd, by Julian Seward et al. >>> ==2130767== Using Valgrind-3.19.0 and LibVEX; rerun with -h for copyright info >>> ==2130767== Command: ./ex65shell -ksp_monitor -pc_type mg -da_refine 2 -ksp_rtol 1e-1 -int_view ascii::ascii_info >>> ==2130767== >>> hwloc x86 backend cannot work under Valgrind, disabling. >>> May be reenabled by dumping CPUIDs with hwloc-gather-cpuid >>> and reloading them under Valgrind with HWLOC_CPUID_PATH. >>> >> Created DMshell 0xfbf50f0 (0xfb49480) >>> Calling KSPSolve from main >>> computeRHS on grid 513 >>> computeMatrix on grid 513 >>> Inside Coarsen >>> >> Created DMshell 0xff404a0 (0xff062d0) >>> Inside Coarsen >>> >> Created DMshell 0xff78ca0 (0xff4a530) >>> >> Create interpolation from 0xff404a0(0xff062d0) to 0xfbf50f0(0xfb49480) >>> Mat Object: 1 MPI process >>> type: shell >>> rows=513, cols=257 >>> ==2130767== Conditional jump or move depends on uninitialised value(s) >>> ==2130767== at 0x83FDAAE: PCSetUp_MG (mg.c:998) >>> ==2130767== by 0x8620C04: PCSetUp (precon.c:1071) >>> ==2130767== by 0x7DA78D5: KSPSetUp (itfunc.c:415) >>> ==2130767== by 0x7DB2093: KSPSolve_Private (itfunc.c:826) >>> ==2130767== by 0x7DB7A53: KSPSolve (itfunc.c:1075) >>> ==2130767== by 0x10D2F7: main (ex65shell.c:79) >>> ==2130767== >>> ==2130767== Conditional jump or move depends on uninitialised value(s) >>> ==2130767== at 0x5794061: VecDestroy (vector.c:570) >>> ==2130767== by 0x83FDC36: PCSetUp_MG (mg.c:999) >>> ==2130767== by 0x8620C04: PCSetUp (precon.c:1071) >>> ==2130767== by 0x7DA78D5: KSPSetUp (itfunc.c:415) >>> ==2130767== by 0x7DB2093: KSPSolve_Private (itfunc.c:826) >>> ==2130767== by 0x7DB7A53: KSPSolve (itfunc.c:1075) >>> ==2130767== by 0x10D2F7: main (ex65shell.c:79) >>> ==2130767== >>> >>> We are clearly doing something wrong since PCSetUp_MG (mg.c:998) is an area of code where I wouldn't expect we would enter. >>> >>> Can you advise on the proper way to achieve our goal? >>> >>> Best regards >>> >>> Matteo >>> >>> >> > -- > Prof. Matteo Semplice > Universit? degli Studi dell?Insubria > Dipartimento di Scienza e Alta Tecnologia ? DiSAT > Professore Associato > Via Valleggio, 11 ? 22100 Como (CO) ? Italia > tel.: +39 031 2386316 -------------- next part -------------- An HTML attachment was scrubbed... URL: From matteo.semplice at uninsubria.it Thu Dec 12 05:22:02 2024 From: matteo.semplice at uninsubria.it (Matteo Semplice) Date: Thu, 12 Dec 2024 12:22:02 +0100 Subject: [petsc-users] PCMG with shell restrictions In-Reply-To: <6310733B-AECC-40AF-B63F-95ADBC9A8DA3@petsc.dev> References: <7071ff6e-52cb-4715-8cbc-bacc6a385363@uninsubria.it> <97C19201-BA91-4661-9E03-0CD1817AA663@petsc.dev> <46997481-abe3-4d76-888b-95db02d3617a@uninsubria.it> <6310733B-AECC-40AF-B63F-95ADBC9A8DA3@petsc.dev> Message-ID: <56026c96-179f-4deb-aacd-ad8bc37dedf2@uninsubria.it> Hi. Indeed creating a vec like in DMCreateInterpolationScale does the job. Thank you! Matteo & Samuele On 10/12/2024 17:27, Barry Smith wrote: > > ? ?Beats me :-) Just set it to whatever the DMDA sets it too > > ? Barry > > >> On Dec 10, 2024, at 11:20?AM, Matteo Semplice >> wrote: >> >> Thanks, Barry. >> >> Just to be sure, one should refer to the help of >> https://urldefense.us/v3/__https://petsc.org/release/manualpages/DM/DMCreateInterpolationScale/__;!!G_uCfscf7eWS!ZIJKHA2S0bAQI-Aea29FNqC4ZzzTaTqYr_2n6i3DkrdUoub4yRrIlmdsYpESo1wSqyGMK4FAk1NBt_WyJvQANYWMxdSgl5M7s1pT4w$ >> and the vec should be set to the point-wise inverse of >> (INTERPOLATION)*(vector of ones)? ? >> >> Matteo >> >> On 10/12/2024 16:38, Barry Smith wrote: >>> >>> ? ?It appears you are completely ignoring the vec argument? Take a >>> look at, for example,?DMCreateInterpolation_DA() you will see you >>> need to provide an appropriate vec in the same way you need (and do) >>> provide an appropriate mat. >>> >>> ? ?Barry >>> >>> >>>> On Dec 10, 2024, at 7:02?AM, Matteo Semplice via petsc-users >>>> wrote: >>>> >>>> Dear petsc-users, >>>> >>>> ??? I am trying with a student to modify the MG example 65 to use >>>> mat-shells instead of assembled matrices. Our use case is for a >>>> method that will use custom shell operators and shell >>>> interpolation/restrictions. >>>> >>>> To start we have modified ex65 and tried to replace the standard >>>> restriction/interpolation with a shell matrix that performs the >>>> same operations on the DMDA grids. >>>> >>>> I attach our modifications to ex65. >>>> >>>> The problem is that the code sometimes completes execution and >>>> sometimes errors out like >>>> >>>> $ ./ex65shell -ksp_monitor -pc_type mg -da_refine 2 -ksp_rtol 1e-1 >>>> >> Created DMshell 0x55bbc83c91a0 (0x55bbc8390a80) >>>> Calling KSPSolve from main >>>> computeRHS on grid 513 >>>> computeMatrix on grid 513 >>>> Inside Coarsen >>>> >> Created DMshell 0x55bbc84c5270 (0x55bbc84b2ff0) >>>> Inside Coarsen >>>> >> Created DMshell 0x55bbc84e0a30 (0x55bbc84bdb60) >>>> >> Create interpolation from 0x55bbc84c5270(0x55bbc84b2ff0) to >>>> 0x55bbc83c91a0(0x55bbc8390a80) >>>> [0]PETSC ERROR: --------------------- Error Message >>>> -------------------------------------------------------------- >>>> [0]PETSC ERROR: Invalid pointer >>>> [0]PETSC ERROR: Invalid Pointer to PetscObject: Argument 'obj' >>>> (parameter # 1) >>>> [0]PETSC ERROR: See https://urldefense.us/v3/__https://petsc.org/release/faq/__;!!G_uCfscf7eWS!ZIJKHA2S0bAQI-Aea29FNqC4ZzzTaTqYr_2n6i3DkrdUoub4yRrIlmdsYpESo1wSqyGMK4FAk1NBt_WyJvQANYWMxdSgl5MFmiZ_RA$ for trouble >>>> shooting. >>>> [0]PETSC ERROR: Petsc Release Version 3.22.0, unknown >>>> [0]PETSC ERROR: ./ex65shell with 1 MPI process(es) and PETSC_ARCH >>>> ?on signalkuppe by matteo Tue Dec 10 12:55:12 2024 >>>> [0]PETSC ERROR: Configure options: >>>> --prefix=/home/matteo/software/petscsaved/3.22-opt/ >>>> PETSC_DIR=/home/matteo/software/petsc --PETSC_ARCH=dbg >>>> --with-debugging=1 --with-st >>>> rict-petscerrorcode --download-hdf5 --download-ml --with-metis >>>> --with-parmetis --with-gmsh --with-triangle --with-zlib >>>> --with-p4est-dir=~/software/p4est/local/ >>>> [0]PETSC ERROR: #1 PetscObjectReference() at >>>> /home/matteo/software/petsc/src/sys/objects/inherit.c:620 >>>> [0]PETSC ERROR: #2 PCMGSetRScale() at >>>> /home/matteo/software/petsc/src/ksp/pc/impls/mg/mgfunc.c:394 >>>> [0]PETSC ERROR: #3 PCSetUp_MG() at >>>> /home/matteo/software/petsc/src/ksp/pc/impls/mg/mg.c:998 >>>> [0]PETSC ERROR: #4 PCSetUp() at >>>> /home/matteo/software/petsc/src/ksp/pc/interface/precon.c:1071 >>>> [0]PETSC ERROR: #5 KSPSetUp() at >>>> /home/matteo/software/petsc/src/ksp/ksp/interface/itfunc.c:415 >>>> [0]PETSC ERROR: #6 KSPSolve_Private() at >>>> /home/matteo/software/petsc/src/ksp/ksp/interface/itfunc.c:826 >>>> [0]PETSC ERROR: #7 KSPSolve() at >>>> /home/matteo/software/petsc/src/ksp/ksp/interface/itfunc.c:1075 >>>> [0]PETSC ERROR: #8 main() at ../src/ex65shell.c:79 >>>> [0]PETSC ERROR: PETSc Option Table entries: >>>> [0]PETSC ERROR: -da_refine 2 (source: command line) >>>> [0]PETSC ERROR: -ksp_monitor (source: command line) >>>> [0]PETSC ERROR: -ksp_rtol 1e-1 (source: command line) >>>> [0]PETSC ERROR: -pc_type mg (source: command line) >>>> [0]PETSC ERROR: ----------------End of Error Message -------send >>>> entire error message to petsc-maint at mcs.anl.gov---------- >>>> >>>> In all cases valgrind complains like >>>> >>>> $ valgrind ./ex65shell -ksp_monitor -pc_type mg -da_refine 2 >>>> -ksp_rtol 1e-1 -int_view ascii::ascii_info >>>> ==2130767== Memcheck, a memory error detector >>>> ==2130767== Copyright (C) 2002-2022, and GNU GPL'd, by Julian >>>> Seward et al. >>>> ==2130767== Using Valgrind-3.19.0 and LibVEX; rerun with -h for >>>> copyright info >>>> ==2130767== Command: ./ex65shell -ksp_monitor -pc_type mg >>>> -da_refine 2 -ksp_rtol 1e-1 -int_view ascii::ascii_info >>>> ==2130767== >>>> hwloc x86 backend cannot work under Valgrind, disabling. >>>> May be reenabled by dumping CPUIDs with hwloc-gather-cpuid >>>> and reloading them under Valgrind with HWLOC_CPUID_PATH. >>>> >> Created DMshell 0xfbf50f0 (0xfb49480) >>>> Calling KSPSolve from main >>>> computeRHS on grid 513 >>>> computeMatrix on grid 513 >>>> Inside Coarsen >>>> >> Created DMshell 0xff404a0 (0xff062d0) >>>> Inside Coarsen >>>> >> Created DMshell 0xff78ca0 (0xff4a530) >>>> >> Create interpolation from 0xff404a0(0xff062d0) to >>>> 0xfbf50f0(0xfb49480) >>>> Mat Object: 1 MPI process >>>> ?type: shell >>>> ?rows=513, cols=257 >>>> ==2130767== Conditional jump or move depends on uninitialised value(s) >>>> ==2130767== ???at 0x83FDAAE: PCSetUp_MG (mg.c:998) >>>> ==2130767== ???by 0x8620C04: PCSetUp (precon.c:1071) >>>> ==2130767== ???by 0x7DA78D5: KSPSetUp (itfunc.c:415) >>>> ==2130767== ???by 0x7DB2093: KSPSolve_Private (itfunc.c:826) >>>> ==2130767== ???by 0x7DB7A53: KSPSolve (itfunc.c:1075) >>>> ==2130767== ???by 0x10D2F7: main (ex65shell.c:79) >>>> ==2130767== >>>> ==2130767== Conditional jump or move depends on uninitialised value(s) >>>> ==2130767== ???at 0x5794061: VecDestroy (vector.c:570) >>>> ==2130767== ???by 0x83FDC36: PCSetUp_MG (mg.c:999) >>>> ==2130767== ???by 0x8620C04: PCSetUp (precon.c:1071) >>>> ==2130767== ???by 0x7DA78D5: KSPSetUp (itfunc.c:415) >>>> ==2130767== ???by 0x7DB2093: KSPSolve_Private (itfunc.c:826) >>>> ==2130767== ???by 0x7DB7A53: KSPSolve (itfunc.c:1075) >>>> ==2130767== ???by 0x10D2F7: main (ex65shell.c:79) >>>> ==2130767== >>>> >>>> We are clearly doing something wrong since PCSetUp_MG (mg.c:998)? >>>> is an area of code where I wouldn't expect we would enter. >>>> >>>> Can you advise on the proper way to achieve our goal? >>>> >>>> Best regards >>>> >>>> ??? Matteo >>>> >>>> >>> >> -- >> Prof. Matteo Semplice >> Universit? degli Studi dell?Insubria >> Dipartimento di Scienza e Alta Tecnologia ? DiSAT >> Professore Associato >> Via Valleggio, 11 ? 22100 Como (CO) ? Italia >> tel.: +39 031 2386316 > -- Prof. Matteo Semplice Universit? degli Studi dell?Insubria Dipartimento di Scienza e Alta Tecnologia ? DiSAT Professore Associato Via Valleggio, 11 ? 22100 Como (CO) ? Italia tel.: +39 031 2386316 -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at petsc.dev Fri Dec 13 13:21:58 2024 From: bsmith at petsc.dev (Barry Smith) Date: Fri, 13 Dec 2024 14:21:58 -0500 Subject: [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED In-Reply-To: References: <1E4CF02D-32EB-482B-ACFC-9AE1C7F9102E@petsc.dev> Message-ID: Hm, what is the final time you are stepping towards in this run? There is something wrong with the adapt code since it seems to start with a dt of 0 but then tries "adapting" several times, but it could be the monitor function does not correctly format numbers smaller than 1.e-12 and it is just using truly small dt. Jed, Hong, Emil? Barry > On Dec 10, 2024, at 11:08?AM, Blondel, Sophie wrote: > > Good morning Barry, > > Attached are the updated files, there is more useful information in them. > > Cheers, > > Sophie > From: Blondel, Sophie via Xolotl-psi-development > > Sent: Monday, December 9, 2024 17:29 > To: Barry Smith > > Cc: petsc-users at mcs.anl.gov >; xolotl-psi-development at lists.sourceforge.net > > Subject: Re: [Xolotl-psi-development] [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED > > Hi Barry, > > I hope you are doing well. > > Attached are the output. To give a little more context, this is a "new" way of running the code where multiple instances are created and communicate together every few time steps (like coupling the code with itself in memory). Here there are 3 instances that each have a separate TS object, plus one "main" instance that doesn't solve anything but compute rates to exchange between the other instances. > > Cheers, > > Sophie > From: Barry Smith > > Sent: Monday, December 9, 2024 15:12 > To: Blondel, Sophie > > Cc: petsc-users at mcs.anl.gov >; xolotl-psi-development at lists.sourceforge.net > > Subject: Re: [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED > > > >> On Dec 9, 2024, at 2:56?PM, Blondel, Sophie via petsc-users > wrote: >> >> Hi, >> >> I am trying to understand a strange behavior I'm encountering: when running my application with "-ts_exact_final_time stepover" everything goes well, but when I switch to "matchstep" I get DIVERGED_STEP_REJECTED before the first time step is finished. > > This is in the very first time-step in TSSolve? > > Please run with -ts_monitor and send all the output (best for a short time interval and do it twice once with -ts_exact_final_time stepover and once with exact. > > Barry > > >> I tried increasing the maximum number of rejections and it just takes longer to diverge, and if I set the value to "unlimited" it is basically an infinite loop. >> >> Is there a way to check why is the step rejected? Could the "matchstep" option change tolerances somewhere that would cause that behavior? >> >> Let me know if I should provide more information. >> >> Best, >> >> Sophie Blondel > ?? -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: matchstep.txt URL: -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: stepover.txt URL: -------------- next part -------------- An HTML attachment was scrubbed... URL: From sblondel at utk.edu Fri Dec 13 13:34:42 2024 From: sblondel at utk.edu (Blondel, Sophie) Date: Fri, 13 Dec 2024 19:34:42 +0000 Subject: [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED In-Reply-To: References: <1E4CF02D-32EB-482B-ACFC-9AE1C7F9102E@petsc.dev> Message-ID: Hi everyone, The first max time it is trying to reach is 1.0e-12 s, and the initial dt is set to 1.0e-12 s from the commandline options. I believe it's not a formatting issue and that the dt is actually set somewhere to 0 s because that's why the step is rejected. Best, Sophie ________________________________ From: Barry Smith Sent: Friday, December 13, 2024 14:21 To: Blondel, Sophie ; Jed Brown ; Zhang, Hong ; Emil Constantinescu Cc: petsc-users at mcs.anl.gov ; xolotl-psi-development at lists.sourceforge.net Subject: Re: [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED Hm, what is the final time you are stepping towards in this run? There is something wrong with the adapt code since it seems to start with a dt of 0 but then tries "adapting" several times, but it could be the monitor function does not correctly format numbers smaller than 1.e-12 and it is just using truly small dt. Jed, Hong, Emil? Barry On Dec 10, 2024, at 11:08?AM, Blondel, Sophie wrote: Good morning Barry, Attached are the updated files, there is more useful information in them. Cheers, Sophie ________________________________ From: Blondel, Sophie via Xolotl-psi-development > Sent: Monday, December 9, 2024 17:29 To: Barry Smith > Cc: petsc-users at mcs.anl.gov >; xolotl-psi-development at lists.sourceforge.net > Subject: Re: [Xolotl-psi-development] [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED Hi Barry, I hope you are doing well. Attached are the output. To give a little more context, this is a "new" way of running the code where multiple instances are created and communicate together every few time steps (like coupling the code with itself in memory). Here there are 3 instances that each have a separate TS object, plus one "main" instance that doesn't solve anything but compute rates to exchange between the other instances. Cheers, Sophie ________________________________ From: Barry Smith > Sent: Monday, December 9, 2024 15:12 To: Blondel, Sophie > Cc: petsc-users at mcs.anl.gov >; xolotl-psi-development at lists.sourceforge.net > Subject: Re: [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED On Dec 9, 2024, at 2:56?PM, Blondel, Sophie via petsc-users > wrote: Hi, I am trying to understand a strange behavior I'm encountering: when running my application with "-ts_exact_final_time stepover" everything goes well, but when I switch to "matchstep" I get DIVERGED_STEP_REJECTED before the first time step is finished. This is in the very first time-step in TSSolve? Please run with -ts_monitor and send all the output (best for a short time interval and do it twice once with -ts_exact_final_time stepover and once with exact. Barry I tried increasing the maximum number of rejections and it just takes longer to diverge, and if I set the value to "unlimited" it is basically an infinite loop. Is there a way to check why is the step rejected? Could the "matchstep" option change tolerances somewhere that would cause that behavior? Let me know if I should provide more information. Best, Sophie Blondel -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at petsc.dev Fri Dec 13 13:56:15 2024 From: bsmith at petsc.dev (Barry Smith) Date: Fri, 13 Dec 2024 14:56:15 -0500 Subject: [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED In-Reply-To: References: <1E4CF02D-32EB-482B-ACFC-9AE1C7F9102E@petsc.dev> Message-ID: <61974628-9044-45AB-9F9A-3E426D021C93@petsc.dev> There is a bit of complicated logic to determine the "adjusted" timestep in TSAdaptChoose() when if (*accept && ts->exact_final_time == TS_EXACTFINALTIME_MATCHSTEP) { Is it possible that hmax = tmax - t; is exactly zero, and the logic below does not correctly handle that case? 0 TS dt 0. time 0. 0 TS dt 0. time 0. 0 TS dt 0. time 0. 0 TS dt 0. time 0. TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 Sophie, Any idea why SNES reason DIVERGED_FNORM_NAN? Could you run with -snes_error_if_not_converged? > On Dec 13, 2024, at 2:34?PM, Blondel, Sophie wrote: > > Hi everyone, > > The first max time it is trying to reach is 1.0e-12 s, and the initial dt is set to 1.0e-12 s from the commandline options. I believe it's not a formatting issue and that the dt is actually set somewhere to 0 s because that's why the step is rejected. > > Best, > > Sophie > From: Barry Smith > > Sent: Friday, December 13, 2024 14:21 > To: Blondel, Sophie >; Jed Brown >; Zhang, Hong >; Emil Constantinescu > > Cc: petsc-users at mcs.anl.gov >; xolotl-psi-development at lists.sourceforge.net > > Subject: Re: [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED > > > Hm, what is the final time you are stepping towards in this run? > > There is something wrong with the adapt code since it seems to start with a dt of 0 but then tries "adapting" several times, but it could be the > monitor function does not correctly format numbers smaller than 1.e-12 and it is just using truly small dt. > > Jed, Hong, Emil? > > Barry > > >> On Dec 10, 2024, at 11:08?AM, Blondel, Sophie > wrote: >> >> Good morning Barry, >> >> Attached are the updated files, there is more useful information in them. >> >> Cheers, >> >> Sophie >> >> From: Blondel, Sophie via Xolotl-psi-development > >> Sent: Monday, December 9, 2024 17:29 >> To: Barry Smith > >> Cc: petsc-users at mcs.anl.gov >; xolotl-psi-development at lists.sourceforge.net > >> Subject: Re: [Xolotl-psi-development] [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED >> >> Hi Barry, >> >> I hope you are doing well. >> >> Attached are the output. To give a little more context, this is a "new" way of running the code where multiple instances are created and communicate together every few time steps (like coupling the code with itself in memory). Here there are 3 instances that each have a separate TS object, plus one "main" instance that doesn't solve anything but compute rates to exchange between the other instances. >> >> Cheers, >> >> Sophie >> >> From: Barry Smith > >> Sent: Monday, December 9, 2024 15:12 >> To: Blondel, Sophie > >> Cc: petsc-users at mcs.anl.gov >; xolotl-psi-development at lists.sourceforge.net > >> Subject: Re: [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED >> >> >> >>> On Dec 9, 2024, at 2:56?PM, Blondel, Sophie via petsc-users > wrote: >>> >>> Hi, >>> >>> I am trying to understand a strange behavior I'm encountering: when running my application with "-ts_exact_final_time stepover" everything goes well, but when I switch to "matchstep" I get DIVERGED_STEP_REJECTED before the first time step is finished. >> >> This is in the very first time-step in TSSolve? >> >> Please run with -ts_monitor and send all the output (best for a short time interval and do it twice once with -ts_exact_final_time stepover and once with exact. >> >> Barry >> >> >>> I tried increasing the maximum number of rejections and it just takes longer to diverge, and if I set the value to "unlimited" it is basically an infinite loop. >>> >>> Is there a way to check why is the step rejected? Could the "matchstep" option change tolerances somewhere that would cause that behavior? >>> >>> Let me know if I should provide more information. >>> >>> Best, >>> >>> Sophie Blondel -------------- next part -------------- An HTML attachment was scrubbed... URL: From sblondel at utk.edu Fri Dec 13 15:40:57 2024 From: sblondel at utk.edu (Blondel, Sophie) Date: Fri, 13 Dec 2024 21:40:57 +0000 Subject: [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED In-Reply-To: <61974628-9044-45AB-9F9A-3E426D021C93@petsc.dev> References: <1E4CF02D-32EB-482B-ACFC-9AE1C7F9102E@petsc.dev> <61974628-9044-45AB-9F9A-3E426D021C93@petsc.dev> Message-ID: Barry, The short output is "SNESSolve has not converged due to Nan or Inf norm", the full one is attached. Cheers, Sophie ________________________________ From: Barry Smith Sent: Friday, December 13, 2024 14:56 To: Blondel, Sophie Cc: Jed Brown ; Zhang, Hong ; Emil Constantinescu ; petsc-users at mcs.anl.gov ; xolotl-psi-development at lists.sourceforge.net Subject: Re: [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED There is a bit of complicated logic to determine the "adjusted" timestep in TSAdaptChoose() when if (*accept && ts->exact_final_time == TS_EXACTFINALTIME_MATCHSTEP) { Is it possible that hmax = tmax - t; is exactly zero, and the logic below does not correctly handle that case? 0 TS dt 0. time 0. 0 TS dt 0. time 0. 0 TS dt 0. time 0. 0 TS dt 0. time 0. TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 Sophie, Any idea why SNES reason DIVERGED_FNORM_NAN? Could you run with -snes_error_if_not_converged? On Dec 13, 2024, at 2:34?PM, Blondel, Sophie wrote: Hi everyone, The first max time it is trying to reach is 1.0e-12 s, and the initial dt is set to 1.0e-12 s from the commandline options. I believe it's not a formatting issue and that the dt is actually set somewhere to 0 s because that's why the step is rejected. Best, Sophie ________________________________ From: Barry Smith > Sent: Friday, December 13, 2024 14:21 To: Blondel, Sophie >; Jed Brown >; Zhang, Hong >; Emil Constantinescu > Cc: petsc-users at mcs.anl.gov >; xolotl-psi-development at lists.sourceforge.net > Subject: Re: [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED Hm, what is the final time you are stepping towards in this run? There is something wrong with the adapt code since it seems to start with a dt of 0 but then tries "adapting" several times, but it could be the monitor function does not correctly format numbers smaller than 1.e-12 and it is just using truly small dt. Jed, Hong, Emil? Barry On Dec 10, 2024, at 11:08?AM, Blondel, Sophie > wrote: Good morning Barry, Attached are the updated files, there is more useful information in them. Cheers, Sophie ________________________________ From: Blondel, Sophie via Xolotl-psi-development > Sent: Monday, December 9, 2024 17:29 To: Barry Smith > Cc: petsc-users at mcs.anl.gov >; xolotl-psi-development at lists.sourceforge.net > Subject: Re: [Xolotl-psi-development] [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED Hi Barry, I hope you are doing well. Attached are the output. To give a little more context, this is a "new" way of running the code where multiple instances are created and communicate together every few time steps (like coupling the code with itself in memory). Here there are 3 instances that each have a separate TS object, plus one "main" instance that doesn't solve anything but compute rates to exchange between the other instances. Cheers, Sophie ________________________________ From: Barry Smith > Sent: Monday, December 9, 2024 15:12 To: Blondel, Sophie > Cc: petsc-users at mcs.anl.gov >; xolotl-psi-development at lists.sourceforge.net > Subject: Re: [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED On Dec 9, 2024, at 2:56?PM, Blondel, Sophie via petsc-users > wrote: Hi, I am trying to understand a strange behavior I'm encountering: when running my application with "-ts_exact_final_time stepover" everything goes well, but when I switch to "matchstep" I get DIVERGED_STEP_REJECTED before the first time step is finished. This is in the very first time-step in TSSolve? Please run with -ts_monitor and send all the output (best for a short time interval and do it twice once with -ts_exact_final_time stepover and once with exact. Barry I tried increasing the maximum number of rejections and it just takes longer to diverge, and if I set the value to "unlimited" it is basically an infinite loop. Is there a way to check why is the step rejected? Could the "matchstep" option change tolerances somewhere that would cause that behavior? Let me know if I should provide more information. Best, Sophie Blondel -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: matchstep_reason.txt URL: From bsmith at petsc.dev Fri Dec 13 21:27:54 2024 From: bsmith at petsc.dev (Barry Smith) Date: Fri, 13 Dec 2024 22:27:54 -0500 Subject: [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED In-Reply-To: References: <1E4CF02D-32EB-482B-ACFC-9AE1C7F9102E@petsc.dev> <61974628-9044-45AB-9F9A-3E426D021C93@petsc.dev> Message-ID: <40A610F0-CE1F-471C-B369-A4E2708A88DC@petsc.dev> Ok, the SNES reason DIVERGED_FNORM_NAN is likely due to the TS using a dt of zero, and hence, the evaluation of the TS causing a divide by zero. So the TS adapt needs to be understood better in this situation. My guess is still TS_EXACTFINALTIME_MATCHSTEP is buggy when one actually does get an exact match. Please try using a different initial timestep like 0.5e-12 s on the command line. Barry > On Dec 13, 2024, at 4:40?PM, Blondel, Sophie wrote: > > Barry, > > The short output is "SNESSolve has not converged due to Nan or Inf norm", the full one is attached. > > Cheers, > > Sophie > From: Barry Smith > > Sent: Friday, December 13, 2024 14:56 > To: Blondel, Sophie > > Cc: Jed Brown >; Zhang, Hong >; Emil Constantinescu >; petsc-users at mcs.anl.gov >; xolotl-psi-development at lists.sourceforge.net > > Subject: Re: [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED > > > There is a bit of complicated logic to determine the "adjusted" timestep in TSAdaptChoose() when if (*accept && ts->exact_final_time == TS_EXACTFINALTIME_MATCHSTEP) { > > Is it possible that hmax = tmax - t; is exactly zero, and the logic below does not correctly handle that case? > > 0 TS dt 0. time 0. > 0 TS dt 0. time 0. > 0 TS dt 0. time 0. > 0 TS dt 0. time 0. > TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 > TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 > TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 > TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 > TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 > TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 > TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 > TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 > TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 > TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 > TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 > > Sophie, > > Any idea why SNES reason DIVERGED_FNORM_NAN? Could you run with -snes_error_if_not_converged? > >> On Dec 13, 2024, at 2:34?PM, Blondel, Sophie > wrote: >> >> Hi everyone, >> >> The first max time it is trying to reach is 1.0e-12 s, and the initial dt is set to 1.0e-12 s from the commandline options. I believe it's not a formatting issue and that the dt is actually set somewhere to 0 s because that's why the step is rejected. >> >> Best, >> >> Sophie >> From: Barry Smith > >> Sent: Friday, December 13, 2024 14:21 >> To: Blondel, Sophie >; Jed Brown >; Zhang, Hong >; Emil Constantinescu > >> Cc: petsc-users at mcs.anl.gov >; xolotl-psi-development at lists.sourceforge.net > >> Subject: Re: [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED >> >> >> Hm, what is the final time you are stepping towards in this run? >> >> There is something wrong with the adapt code since it seems to start with a dt of 0 but then tries "adapting" several times, but it could be the >> monitor function does not correctly format numbers smaller than 1.e-12 and it is just using truly small dt. >> >> Jed, Hong, Emil? >> >> Barry >> >> >>> On Dec 10, 2024, at 11:08?AM, Blondel, Sophie > wrote: >>> >>> Good morning Barry, >>> >>> Attached are the updated files, there is more useful information in them. >>> >>> Cheers, >>> >>> Sophie >>> >>> From: Blondel, Sophie via Xolotl-psi-development > >>> Sent: Monday, December 9, 2024 17:29 >>> To: Barry Smith > >>> Cc: petsc-users at mcs.anl.gov >; xolotl-psi-development at lists.sourceforge.net > >>> Subject: Re: [Xolotl-psi-development] [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED >>> >>> Hi Barry, >>> >>> I hope you are doing well. >>> >>> Attached are the output. To give a little more context, this is a "new" way of running the code where multiple instances are created and communicate together every few time steps (like coupling the code with itself in memory). Here there are 3 instances that each have a separate TS object, plus one "main" instance that doesn't solve anything but compute rates to exchange between the other instances. >>> >>> Cheers, >>> >>> Sophie >>> >>> From: Barry Smith > >>> Sent: Monday, December 9, 2024 15:12 >>> To: Blondel, Sophie > >>> Cc: petsc-users at mcs.anl.gov >; xolotl-psi-development at lists.sourceforge.net > >>> Subject: Re: [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED >>> >>> >>> >>>> On Dec 9, 2024, at 2:56?PM, Blondel, Sophie via petsc-users > wrote: >>>> >>>> Hi, >>>> >>>> I am trying to understand a strange behavior I'm encountering: when running my application with "-ts_exact_final_time stepover" everything goes well, but when I switch to "matchstep" I get DIVERGED_STEP_REJECTED before the first time step is finished. >>> >>> This is in the very first time-step in TSSolve? >>> >>> Please run with -ts_monitor and send all the output (best for a short time interval and do it twice once with -ts_exact_final_time stepover and once with exact. >>> >>> Barry >>> >>> >>>> I tried increasing the maximum number of rejections and it just takes longer to diverge, and if I set the value to "unlimited" it is basically an infinite loop. >>>> >>>> Is there a way to check why is the step rejected? Could the "matchstep" option change tolerances somewhere that would cause that behavior? >>>> >>>> Let me know if I should provide more information. >>>> >>>> Best, >>>> >>>> Sophie Blondel > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sblondel at utk.edu Mon Dec 16 08:53:17 2024 From: sblondel at utk.edu (Blondel, Sophie) Date: Mon, 16 Dec 2024 14:53:17 +0000 Subject: [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED In-Reply-To: <40A610F0-CE1F-471C-B369-A4E2708A88DC@petsc.dev> References: <1E4CF02D-32EB-482B-ACFC-9AE1C7F9102E@petsc.dev> <61974628-9044-45AB-9F9A-3E426D021C93@petsc.dev> <40A610F0-CE1F-471C-B369-A4E2708A88DC@petsc.dev> Message-ID: Good morning Barry, Changing the initial timestep on the commandline didn't change anything. It's like there is something happening afterwards that sets the initial dt to 0 s that doesn't happen with TS_EXACTFINALTIME_STEPOVER. Cheers, Sophie ________________________________ From: Barry Smith Sent: Friday, December 13, 2024 22:27 To: Blondel, Sophie Cc: Jed Brown ; Zhang, Hong ; Emil Constantinescu ; petsc-users at mcs.anl.gov ; xolotl-psi-development at lists.sourceforge.net Subject: Re: [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED Ok, the SNES reason DIVERGED_FNORM_NAN is likely due to the TS using a dt of zero, and hence, the evaluation of the TS causing a divide by zero. So the TS adapt needs to be understood better in this situation. My guess is still TS_EXACTFINALTIME_MATCHSTEP is buggy when one actually does get an exact match. Please try using a different initial timestep like 0.5e-12 s on the command line. Barry On Dec 13, 2024, at 4:40?PM, Blondel, Sophie wrote: Barry, The short output is "SNESSolve has not converged due to Nan or Inf norm", the full one is attached. Cheers, Sophie ________________________________ From: Barry Smith > Sent: Friday, December 13, 2024 14:56 To: Blondel, Sophie > Cc: Jed Brown >; Zhang, Hong >; Emil Constantinescu >; petsc-users at mcs.anl.gov >; xolotl-psi-development at lists.sourceforge.net > Subject: Re: [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED There is a bit of complicated logic to determine the "adjusted" timestep in TSAdaptChoose() when if (*accept && ts->exact_final_time == TS_EXACTFINALTIME_MATCHSTEP) { Is it possible that hmax = tmax - t; is exactly zero, and the logic below does not correctly handle that case? 0 TS dt 0. time 0. 0 TS dt 0. time 0. 0 TS dt 0. time 0. 0 TS dt 0. time 0. TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 Sophie, Any idea why SNES reason DIVERGED_FNORM_NAN? Could you run with -snes_error_if_not_converged? On Dec 13, 2024, at 2:34?PM, Blondel, Sophie > wrote: Hi everyone, The first max time it is trying to reach is 1.0e-12 s, and the initial dt is set to 1.0e-12 s from the commandline options. I believe it's not a formatting issue and that the dt is actually set somewhere to 0 s because that's why the step is rejected. Best, Sophie ________________________________ From: Barry Smith > Sent: Friday, December 13, 2024 14:21 To: Blondel, Sophie >; Jed Brown >; Zhang, Hong >; Emil Constantinescu > Cc: petsc-users at mcs.anl.gov >; xolotl-psi-development at lists.sourceforge.net > Subject: Re: [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED Hm, what is the final time you are stepping towards in this run? There is something wrong with the adapt code since it seems to start with a dt of 0 but then tries "adapting" several times, but it could be the monitor function does not correctly format numbers smaller than 1.e-12 and it is just using truly small dt. Jed, Hong, Emil? Barry On Dec 10, 2024, at 11:08?AM, Blondel, Sophie > wrote: Good morning Barry, Attached are the updated files, there is more useful information in them. Cheers, Sophie ________________________________ From: Blondel, Sophie via Xolotl-psi-development > Sent: Monday, December 9, 2024 17:29 To: Barry Smith > Cc: petsc-users at mcs.anl.gov >; xolotl-psi-development at lists.sourceforge.net > Subject: Re: [Xolotl-psi-development] [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED Hi Barry, I hope you are doing well. Attached are the output. To give a little more context, this is a "new" way of running the code where multiple instances are created and communicate together every few time steps (like coupling the code with itself in memory). Here there are 3 instances that each have a separate TS object, plus one "main" instance that doesn't solve anything but compute rates to exchange between the other instances. Cheers, Sophie ________________________________ From: Barry Smith > Sent: Monday, December 9, 2024 15:12 To: Blondel, Sophie > Cc: petsc-users at mcs.anl.gov >; xolotl-psi-development at lists.sourceforge.net > Subject: Re: [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED On Dec 9, 2024, at 2:56?PM, Blondel, Sophie via petsc-users > wrote: Hi, I am trying to understand a strange behavior I'm encountering: when running my application with "-ts_exact_final_time stepover" everything goes well, but when I switch to "matchstep" I get DIVERGED_STEP_REJECTED before the first time step is finished. This is in the very first time-step in TSSolve? Please run with -ts_monitor and send all the output (best for a short time interval and do it twice once with -ts_exact_final_time stepover and once with exact. Barry I tried increasing the maximum number of rejections and it just takes longer to diverge, and if I set the value to "unlimited" it is basically an infinite loop. Is there a way to check why is the step rejected? Could the "matchstep" option change tolerances somewhere that would cause that behavior? Let me know if I should provide more information. Best, Sophie Blondel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jed at jedbrown.org Mon Dec 16 16:44:48 2024 From: jed at jedbrown.org (Jed Brown) Date: Mon, 16 Dec 2024 15:44:48 -0700 Subject: [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED In-Reply-To: References: <1E4CF02D-32EB-482B-ACFC-9AE1C7F9102E@petsc.dev> <61974628-9044-45AB-9F9A-3E426D021C93@petsc.dev> <40A610F0-CE1F-471C-B369-A4E2708A88DC@petsc.dev> Message-ID: <87ed27dz8f.fsf@jedbrown.org> Can you make a PETSc example exhibit this behavior, or otherwise share a reproduce? "Blondel, Sophie" writes: > Good morning Barry, > > Changing the initial timestep on the commandline didn't change anything. It's like there is something happening afterwards that sets the initial dt to 0 s that doesn't happen with TS_EXACTFINALTIME_STEPOVER. > > Cheers, > > Sophie > ________________________________ > From: Barry Smith > Sent: Friday, December 13, 2024 22:27 > To: Blondel, Sophie > Cc: Jed Brown ; Zhang, Hong ; Emil Constantinescu ; petsc-users at mcs.anl.gov ; xolotl-psi-development at lists.sourceforge.net > Subject: Re: [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED > > > Ok, the SNES reason DIVERGED_FNORM_NAN is likely due to the TS using a dt of zero, and hence, the evaluation of the TS causing a divide by zero. > > So the TS adapt needs to be understood better in this situation. My guess is still TS_EXACTFINALTIME_MATCHSTEP is buggy when one actually does get an exact match. > > Please try using a different initial timestep like 0.5e-12 s on the command line. > > Barry > > > On Dec 13, 2024, at 4:40?PM, Blondel, Sophie wrote: > > Barry, > > The short output is "SNESSolve has not converged due to Nan or Inf norm", the full one is attached. > > Cheers, > > Sophie > ________________________________ > From: Barry Smith > > Sent: Friday, December 13, 2024 14:56 > To: Blondel, Sophie > > Cc: Jed Brown >; Zhang, Hong >; Emil Constantinescu >; petsc-users at mcs.anl.gov >; xolotl-psi-development at lists.sourceforge.net > > Subject: Re: [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED > > > There is a bit of complicated logic to determine the "adjusted" timestep in TSAdaptChoose() when if (*accept && ts->exact_final_time == TS_EXACTFINALTIME_MATCHSTEP) { > > Is it possible that hmax = tmax - t; is exactly zero, and the logic below does not correctly handle that case? > > 0 TS dt 0. time 0. > 0 TS dt 0. time 0. > 0 TS dt 0. time 0. > 0 TS dt 0. time 0. > TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 > TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 > TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 > TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 > TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 > TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 > TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 > TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 > TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 > TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 > TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 > > Sophie, > > Any idea why SNES reason DIVERGED_FNORM_NAN? Could you run with -snes_error_if_not_converged? > > On Dec 13, 2024, at 2:34?PM, Blondel, Sophie > wrote: > > Hi everyone, > > The first max time it is trying to reach is 1.0e-12 s, and the initial dt is set to 1.0e-12 s from the commandline options. I believe it's not a formatting issue and that the dt is actually set somewhere to 0 s because that's why the step is rejected. > > Best, > > Sophie > ________________________________ > From: Barry Smith > > Sent: Friday, December 13, 2024 14:21 > To: Blondel, Sophie >; Jed Brown >; Zhang, Hong >; Emil Constantinescu > > Cc: petsc-users at mcs.anl.gov >; xolotl-psi-development at lists.sourceforge.net > > Subject: Re: [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED > > > Hm, what is the final time you are stepping towards in this run? > > There is something wrong with the adapt code since it seems to start with a dt of 0 but then tries "adapting" several times, but it could be the > monitor function does not correctly format numbers smaller than 1.e-12 and it is just using truly small dt. > > Jed, Hong, Emil? > > Barry > > > On Dec 10, 2024, at 11:08?AM, Blondel, Sophie > wrote: > > Good morning Barry, > > Attached are the updated files, there is more useful information in them. > > Cheers, > > Sophie > ________________________________ > > From: Blondel, Sophie via Xolotl-psi-development > > Sent: Monday, December 9, 2024 17:29 > To: Barry Smith > > Cc: petsc-users at mcs.anl.gov >; xolotl-psi-development at lists.sourceforge.net > > Subject: Re: [Xolotl-psi-development] [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED > > Hi Barry, > > I hope you are doing well. > > Attached are the output. To give a little more context, this is a "new" way of running the code where multiple instances are created and communicate together every few time steps (like coupling the code with itself in memory). Here there are 3 instances that each have a separate TS object, plus one "main" instance that doesn't solve anything but compute rates to exchange between the other instances. > > Cheers, > > Sophie > ________________________________ > > From: Barry Smith > > Sent: Monday, December 9, 2024 15:12 > To: Blondel, Sophie > > Cc: petsc-users at mcs.anl.gov >; xolotl-psi-development at lists.sourceforge.net > > Subject: Re: [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED > > > > On Dec 9, 2024, at 2:56?PM, Blondel, Sophie via petsc-users > wrote: > > Hi, > > I am trying to understand a strange behavior I'm encountering: when running my application with "-ts_exact_final_time stepover" everything goes well, but when I switch to "matchstep" I get DIVERGED_STEP_REJECTED before the first time step is finished. > > This is in the very first time-step in TSSolve? > > Please run with -ts_monitor and send all the output (best for a short time interval and do it twice once with -ts_exact_final_time stepover and once with exact. > > Barry > > > I tried increasing the maximum number of rejections and it just takes longer to diverge, and if I set the value to "unlimited" it is basically an infinite loop. > > Is there a way to check why is the step rejected? Could the "matchstep" option change tolerances somewhere that would cause that behavior? > > Let me know if I should provide more information. > > Best, > > Sophie Blondel > > From bsmith at petsc.dev Tue Dec 17 17:02:12 2024 From: bsmith at petsc.dev (Barry Smith) Date: Tue, 17 Dec 2024 18:02:12 -0500 Subject: [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED In-Reply-To: References: <1E4CF02D-32EB-482B-ACFC-9AE1C7F9102E@petsc.dev> <61974628-9044-45AB-9F9A-3E426D021C93@petsc.dev> <40A610F0-CE1F-471C-B369-A4E2708A88DC@petsc.dev> Message-ID: <108B55CC-77BA-49BA-A5B5-A8DAC40DA9A5@petsc.dev> Any chance you are using an ancient version of PETSc (i.e. not the latest) that may have a bug that was fixed? Barry > On Dec 16, 2024, at 9:53?AM, Blondel, Sophie wrote: > > Good morning Barry, > > Changing the initial timestep on the commandline didn't change anything. It's like there is something happening afterwards that sets the initial dt to 0 s that doesn't happen with TS_EXACTFINALTIME_STEPOVER. > > Cheers, > > Sophie > From: Barry Smith > > Sent: Friday, December 13, 2024 22:27 > To: Blondel, Sophie > > Cc: Jed Brown >; Zhang, Hong >; Emil Constantinescu >; petsc-users at mcs.anl.gov >; xolotl-psi-development at lists.sourceforge.net > > Subject: Re: [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED > > > Ok, the SNES reason DIVERGED_FNORM_NAN is likely due to the TS using a dt of zero, and hence, the evaluation of the TS causing a divide by zero. > > So the TS adapt needs to be understood better in this situation. My guess is still TS_EXACTFINALTIME_MATCHSTEP is buggy when one actually does get an exact match. > > Please try using a different initial timestep like 0.5e-12 s on the command line. > > Barry > > >> On Dec 13, 2024, at 4:40?PM, Blondel, Sophie > wrote: >> >> Barry, >> >> The short output is "SNESSolve has not converged due to Nan or Inf norm", the full one is attached. >> >> Cheers, >> >> Sophie >> From: Barry Smith > >> Sent: Friday, December 13, 2024 14:56 >> To: Blondel, Sophie > >> Cc: Jed Brown >; Zhang, Hong >; Emil Constantinescu >; petsc-users at mcs.anl.gov >; xolotl-psi-development at lists.sourceforge.net > >> Subject: Re: [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED >> >> >> There is a bit of complicated logic to determine the "adjusted" timestep in TSAdaptChoose() when if (*accept && ts->exact_final_time == TS_EXACTFINALTIME_MATCHSTEP) { >> >> Is it possible that hmax = tmax - t; is exactly zero, and the logic below does not correctly handle that case? >> >> 0 TS dt 0. time 0. >> 0 TS dt 0. time 0. >> 0 TS dt 0. time 0. >> 0 TS dt 0. time 0. >> TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 >> TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 >> TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 >> TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 >> TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 >> TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 >> TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 >> TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 >> TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 >> TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 >> TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 >> >> Sophie, >> >> Any idea why SNES reason DIVERGED_FNORM_NAN? Could you run with -snes_error_if_not_converged? >> >>> On Dec 13, 2024, at 2:34?PM, Blondel, Sophie > wrote: >>> >>> Hi everyone, >>> >>> The first max time it is trying to reach is 1.0e-12 s, and the initial dt is set to 1.0e-12 s from the commandline options. I believe it's not a formatting issue and that the dt is actually set somewhere to 0 s because that's why the step is rejected. >>> >>> Best, >>> >>> Sophie >>> From: Barry Smith > >>> Sent: Friday, December 13, 2024 14:21 >>> To: Blondel, Sophie >; Jed Brown >; Zhang, Hong >; Emil Constantinescu > >>> Cc: petsc-users at mcs.anl.gov >; xolotl-psi-development at lists.sourceforge.net > >>> Subject: Re: [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED >>> >>> >>> Hm, what is the final time you are stepping towards in this run? >>> >>> There is something wrong with the adapt code since it seems to start with a dt of 0 but then tries "adapting" several times, but it could be the >>> monitor function does not correctly format numbers smaller than 1.e-12 and it is just using truly small dt. >>> >>> Jed, Hong, Emil? >>> >>> Barry >>> >>> >>>> On Dec 10, 2024, at 11:08?AM, Blondel, Sophie > wrote: >>>> >>>> Good morning Barry, >>>> >>>> Attached are the updated files, there is more useful information in them. >>>> >>>> Cheers, >>>> >>>> Sophie >>>> >>>> From: Blondel, Sophie via Xolotl-psi-development > >>>> Sent: Monday, December 9, 2024 17:29 >>>> To: Barry Smith > >>>> Cc: petsc-users at mcs.anl.gov >; xolotl-psi-development at lists.sourceforge.net > >>>> Subject: Re: [Xolotl-psi-development] [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED >>>> >>>> Hi Barry, >>>> >>>> I hope you are doing well. >>>> >>>> Attached are the output. To give a little more context, this is a "new" way of running the code where multiple instances are created and communicate together every few time steps (like coupling the code with itself in memory). Here there are 3 instances that each have a separate TS object, plus one "main" instance that doesn't solve anything but compute rates to exchange between the other instances. >>>> >>>> Cheers, >>>> >>>> Sophie >>>> >>>> From: Barry Smith > >>>> Sent: Monday, December 9, 2024 15:12 >>>> To: Blondel, Sophie > >>>> Cc: petsc-users at mcs.anl.gov >; xolotl-psi-development at lists.sourceforge.net > >>>> Subject: Re: [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED >>>> >>>> >>>> >>>>> On Dec 9, 2024, at 2:56?PM, Blondel, Sophie via petsc-users > wrote: >>>>> >>>>> Hi, >>>>> >>>>> I am trying to understand a strange behavior I'm encountering: when running my application with "-ts_exact_final_time stepover" everything goes well, but when I switch to "matchstep" I get DIVERGED_STEP_REJECTED before the first time step is finished. >>>> >>>> This is in the very first time-step in TSSolve? >>>> >>>> Please run with -ts_monitor and send all the output (best for a short time interval and do it twice once with -ts_exact_final_time stepover and once with exact. >>>> >>>> Barry >>>> >>>> >>>>> I tried increasing the maximum number of rejections and it just takes longer to diverge, and if I set the value to "unlimited" it is basically an infinite loop. >>>>> >>>>> Is there a way to check why is the step rejected? Could the "matchstep" option change tolerances somewhere that would cause that behavior? >>>>> >>>>> Let me know if I should provide more information. >>>>> >>>>> Best, >>>>> >>>>> Sophie Blondel >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at petsc.dev Tue Dec 17 17:04:42 2024 From: bsmith at petsc.dev (Barry Smith) Date: Tue, 17 Dec 2024 18:04:42 -0500 Subject: [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED In-Reply-To: References: <1E4CF02D-32EB-482B-ACFC-9AE1C7F9102E@petsc.dev> <61974628-9044-45AB-9F9A-3E426D021C93@petsc.dev> <40A610F0-CE1F-471C-B369-A4E2708A88DC@petsc.dev> Message-ID: <901E2EA5-97E0-4C36-8E2D-CB28E6AAAA23@petsc.dev> Could you please send all the options you are using and any that are set in the code, like -ts_type etc > On Dec 16, 2024, at 9:53?AM, Blondel, Sophie wrote: > > Good morning Barry, > > Changing the initial timestep on the commandline didn't change anything. It's like there is something happening afterwards that sets the initial dt to 0 s that doesn't happen with TS_EXACTFINALTIME_STEPOVER. > > Cheers, > > Sophie > From: Barry Smith > > Sent: Friday, December 13, 2024 22:27 > To: Blondel, Sophie > > Cc: Jed Brown >; Zhang, Hong >; Emil Constantinescu >; petsc-users at mcs.anl.gov >; xolotl-psi-development at lists.sourceforge.net > > Subject: Re: [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED > > > Ok, the SNES reason DIVERGED_FNORM_NAN is likely due to the TS using a dt of zero, and hence, the evaluation of the TS causing a divide by zero. > > So the TS adapt needs to be understood better in this situation. My guess is still TS_EXACTFINALTIME_MATCHSTEP is buggy when one actually does get an exact match. > > Please try using a different initial timestep like 0.5e-12 s on the command line. > > Barry > > >> On Dec 13, 2024, at 4:40?PM, Blondel, Sophie > wrote: >> >> Barry, >> >> The short output is "SNESSolve has not converged due to Nan or Inf norm", the full one is attached. >> >> Cheers, >> >> Sophie >> From: Barry Smith > >> Sent: Friday, December 13, 2024 14:56 >> To: Blondel, Sophie > >> Cc: Jed Brown >; Zhang, Hong >; Emil Constantinescu >; petsc-users at mcs.anl.gov >; xolotl-psi-development at lists.sourceforge.net > >> Subject: Re: [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED >> >> >> There is a bit of complicated logic to determine the "adjusted" timestep in TSAdaptChoose() when if (*accept && ts->exact_final_time == TS_EXACTFINALTIME_MATCHSTEP) { >> >> Is it possible that hmax = tmax - t; is exactly zero, and the logic below does not correctly handle that case? >> >> 0 TS dt 0. time 0. >> 0 TS dt 0. time 0. >> 0 TS dt 0. time 0. >> 0 TS dt 0. time 0. >> TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 >> TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 >> TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 >> TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 >> TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 >> TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 >> TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 >> TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 >> TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 >> TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 >> TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 >> >> Sophie, >> >> Any idea why SNES reason DIVERGED_FNORM_NAN? Could you run with -snes_error_if_not_converged? >> >>> On Dec 13, 2024, at 2:34?PM, Blondel, Sophie > wrote: >>> >>> Hi everyone, >>> >>> The first max time it is trying to reach is 1.0e-12 s, and the initial dt is set to 1.0e-12 s from the commandline options. I believe it's not a formatting issue and that the dt is actually set somewhere to 0 s because that's why the step is rejected. >>> >>> Best, >>> >>> Sophie >>> From: Barry Smith > >>> Sent: Friday, December 13, 2024 14:21 >>> To: Blondel, Sophie >; Jed Brown >; Zhang, Hong >; Emil Constantinescu > >>> Cc: petsc-users at mcs.anl.gov >; xolotl-psi-development at lists.sourceforge.net > >>> Subject: Re: [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED >>> >>> >>> Hm, what is the final time you are stepping towards in this run? >>> >>> There is something wrong with the adapt code since it seems to start with a dt of 0 but then tries "adapting" several times, but it could be the >>> monitor function does not correctly format numbers smaller than 1.e-12 and it is just using truly small dt. >>> >>> Jed, Hong, Emil? >>> >>> Barry >>> >>> >>>> On Dec 10, 2024, at 11:08?AM, Blondel, Sophie > wrote: >>>> >>>> Good morning Barry, >>>> >>>> Attached are the updated files, there is more useful information in them. >>>> >>>> Cheers, >>>> >>>> Sophie >>>> >>>> From: Blondel, Sophie via Xolotl-psi-development > >>>> Sent: Monday, December 9, 2024 17:29 >>>> To: Barry Smith > >>>> Cc: petsc-users at mcs.anl.gov >; xolotl-psi-development at lists.sourceforge.net > >>>> Subject: Re: [Xolotl-psi-development] [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED >>>> >>>> Hi Barry, >>>> >>>> I hope you are doing well. >>>> >>>> Attached are the output. To give a little more context, this is a "new" way of running the code where multiple instances are created and communicate together every few time steps (like coupling the code with itself in memory). Here there are 3 instances that each have a separate TS object, plus one "main" instance that doesn't solve anything but compute rates to exchange between the other instances. >>>> >>>> Cheers, >>>> >>>> Sophie >>>> >>>> From: Barry Smith > >>>> Sent: Monday, December 9, 2024 15:12 >>>> To: Blondel, Sophie > >>>> Cc: petsc-users at mcs.anl.gov >; xolotl-psi-development at lists.sourceforge.net > >>>> Subject: Re: [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED >>>> >>>> >>>> >>>>> On Dec 9, 2024, at 2:56?PM, Blondel, Sophie via petsc-users > wrote: >>>>> >>>>> Hi, >>>>> >>>>> I am trying to understand a strange behavior I'm encountering: when running my application with "-ts_exact_final_time stepover" everything goes well, but when I switch to "matchstep" I get DIVERGED_STEP_REJECTED before the first time step is finished. >>>> >>>> This is in the very first time-step in TSSolve? >>>> >>>> Please run with -ts_monitor and send all the output (best for a short time interval and do it twice once with -ts_exact_final_time stepover and once with exact. >>>> >>>> Barry >>>> >>>> >>>>> I tried increasing the maximum number of rejections and it just takes longer to diverge, and if I set the value to "unlimited" it is basically an infinite loop. >>>>> >>>>> Is there a way to check why is the step rejected? Could the "matchstep" option change tolerances somewhere that would cause that behavior? >>>>> >>>>> Let me know if I should provide more information. >>>>> >>>>> Best, >>>>> >>>>> Sophie Blondel >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at petsc.dev Tue Dec 17 17:33:33 2024 From: bsmith at petsc.dev (Barry Smith) Date: Tue, 17 Dec 2024 18:33:33 -0500 Subject: [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED In-Reply-To: References: <1E4CF02D-32EB-482B-ACFC-9AE1C7F9102E@petsc.dev> <61974628-9044-45AB-9F9A-3E426D021C93@petsc.dev> Message-ID: This output is odd and seems wrong > 0 TS dt 0. time 0. This is printing the initial timestep and time before it does any timestepping. We expect dt to be 1e-12 as your other case does print 0 TS dt 1e-12 time 0. What the exact final time flag is set to shouldn't affect the timestep this early in the computation. You could try in the debugger to trace the variable ts->time_step to see when it is being set from 1.e-12 to 0. > On Dec 13, 2024, at 4:40?PM, Blondel, Sophie wrote: > > Barry, > > The short output is "SNESSolve has not converged due to Nan or Inf norm", the full one is attached. > > Cheers, > > Sophie > From: Barry Smith > > Sent: Friday, December 13, 2024 14:56 > To: Blondel, Sophie > > Cc: Jed Brown >; Zhang, Hong >; Emil Constantinescu >; petsc-users at mcs.anl.gov >; xolotl-psi-development at lists.sourceforge.net > > Subject: Re: [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED > > > There is a bit of complicated logic to determine the "adjusted" timestep in TSAdaptChoose() when if (*accept && ts->exact_final_time == TS_EXACTFINALTIME_MATCHSTEP) { > > Is it possible that hmax = tmax - t; is exactly zero, and the logic below does not correctly handle that case? > > 0 TS dt 0. time 0. > 0 TS dt 0. time 0. > 0 TS dt 0. time 0. > 0 TS dt 0. time 0. > TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 > TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 > TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 > TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 > TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 > TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 > TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 > TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 > TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 > TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 > TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 > > Sophie, > > Any idea why SNES reason DIVERGED_FNORM_NAN? Could you run with -snes_error_if_not_converged? > >> On Dec 13, 2024, at 2:34?PM, Blondel, Sophie > wrote: >> >> Hi everyone, >> >> The first max time it is trying to reach is 1.0e-12 s, and the initial dt is set to 1.0e-12 s from the commandline options. I believe it's not a formatting issue and that the dt is actually set somewhere to 0 s because that's why the step is rejected. >> >> Best, >> >> Sophie >> From: Barry Smith > >> Sent: Friday, December 13, 2024 14:21 >> To: Blondel, Sophie >; Jed Brown >; Zhang, Hong >; Emil Constantinescu > >> Cc: petsc-users at mcs.anl.gov >; xolotl-psi-development at lists.sourceforge.net > >> Subject: Re: [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED >> >> >> Hm, what is the final time you are stepping towards in this run? >> >> There is something wrong with the adapt code since it seems to start with a dt of 0 but then tries "adapting" several times, but it could be the >> monitor function does not correctly format numbers smaller than 1.e-12 and it is just using truly small dt. >> >> Jed, Hong, Emil? >> >> Barry >> >> >>> On Dec 10, 2024, at 11:08?AM, Blondel, Sophie > wrote: >>> >>> Good morning Barry, >>> >>> Attached are the updated files, there is more useful information in them. >>> >>> Cheers, >>> >>> Sophie >>> >>> From: Blondel, Sophie via Xolotl-psi-development > >>> Sent: Monday, December 9, 2024 17:29 >>> To: Barry Smith > >>> Cc: petsc-users at mcs.anl.gov >; xolotl-psi-development at lists.sourceforge.net > >>> Subject: Re: [Xolotl-psi-development] [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED >>> >>> Hi Barry, >>> >>> I hope you are doing well. >>> >>> Attached are the output. To give a little more context, this is a "new" way of running the code where multiple instances are created and communicate together every few time steps (like coupling the code with itself in memory). Here there are 3 instances that each have a separate TS object, plus one "main" instance that doesn't solve anything but compute rates to exchange between the other instances. >>> >>> Cheers, >>> >>> Sophie >>> >>> From: Barry Smith > >>> Sent: Monday, December 9, 2024 15:12 >>> To: Blondel, Sophie > >>> Cc: petsc-users at mcs.anl.gov >; xolotl-psi-development at lists.sourceforge.net > >>> Subject: Re: [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED >>> >>> >>> >>>> On Dec 9, 2024, at 2:56?PM, Blondel, Sophie via petsc-users > wrote: >>>> >>>> Hi, >>>> >>>> I am trying to understand a strange behavior I'm encountering: when running my application with "-ts_exact_final_time stepover" everything goes well, but when I switch to "matchstep" I get DIVERGED_STEP_REJECTED before the first time step is finished. >>> >>> This is in the very first time-step in TSSolve? >>> >>> Please run with -ts_monitor and send all the output (best for a short time interval and do it twice once with -ts_exact_final_time stepover and once with exact. >>> >>> Barry >>> >>> >>>> I tried increasing the maximum number of rejections and it just takes longer to diverge, and if I set the value to "unlimited" it is basically an infinite loop. >>>> >>>> Is there a way to check why is the step rejected? Could the "matchstep" option change tolerances somewhere that would cause that behavior? >>>> >>>> Let me know if I should provide more information. >>>> >>>> Best, >>>> >>>> Sophie Blondel > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at petsc.dev Tue Dec 17 17:33:33 2024 From: bsmith at petsc.dev (Barry Smith) Date: Tue, 17 Dec 2024 18:33:33 -0500 Subject: [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED In-Reply-To: References: <1E4CF02D-32EB-482B-ACFC-9AE1C7F9102E@petsc.dev> <61974628-9044-45AB-9F9A-3E426D021C93@petsc.dev> Message-ID: This output is odd and seems wrong > 0 TS dt 0. time 0. This is printing the initial timestep and time before it does any timestepping. We expect dt to be 1e-12 as your other case does print 0 TS dt 1e-12 time 0. What the exact final time flag is set to shouldn't affect the timestep this early in the computation. You could try in the debugger to trace the variable ts->time_step to see when it is being set from 1.e-12 to 0. > On Dec 13, 2024, at 4:40?PM, Blondel, Sophie wrote: > > Barry, > > The short output is "SNESSolve has not converged due to Nan or Inf norm", the full one is attached. > > Cheers, > > Sophie > From: Barry Smith > > Sent: Friday, December 13, 2024 14:56 > To: Blondel, Sophie > > Cc: Jed Brown >; Zhang, Hong >; Emil Constantinescu >; petsc-users at mcs.anl.gov >; xolotl-psi-development at lists.sourceforge.net > > Subject: Re: [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED > > > There is a bit of complicated logic to determine the "adjusted" timestep in TSAdaptChoose() when if (*accept && ts->exact_final_time == TS_EXACTFINALTIME_MATCHSTEP) { > > Is it possible that hmax = tmax - t; is exactly zero, and the logic below does not correctly handle that case? > > 0 TS dt 0. time 0. > 0 TS dt 0. time 0. > 0 TS dt 0. time 0. > 0 TS dt 0. time 0. > TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 > TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 > TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 > TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 > TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 > TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 > TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 > TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 > TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 > TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 > TSAdapt basic step 0 stage rejected (SNES reason DIVERGED_FNORM_NAN) t=0 + 0.000e+00 retrying with dt=0.000e+00 > > Sophie, > > Any idea why SNES reason DIVERGED_FNORM_NAN? Could you run with -snes_error_if_not_converged? > >> On Dec 13, 2024, at 2:34?PM, Blondel, Sophie > wrote: >> >> Hi everyone, >> >> The first max time it is trying to reach is 1.0e-12 s, and the initial dt is set to 1.0e-12 s from the commandline options. I believe it's not a formatting issue and that the dt is actually set somewhere to 0 s because that's why the step is rejected. >> >> Best, >> >> Sophie >> From: Barry Smith > >> Sent: Friday, December 13, 2024 14:21 >> To: Blondel, Sophie >; Jed Brown >; Zhang, Hong >; Emil Constantinescu > >> Cc: petsc-users at mcs.anl.gov >; xolotl-psi-development at lists.sourceforge.net > >> Subject: Re: [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED >> >> >> Hm, what is the final time you are stepping towards in this run? >> >> There is something wrong with the adapt code since it seems to start with a dt of 0 but then tries "adapting" several times, but it could be the >> monitor function does not correctly format numbers smaller than 1.e-12 and it is just using truly small dt. >> >> Jed, Hong, Emil? >> >> Barry >> >> >>> On Dec 10, 2024, at 11:08?AM, Blondel, Sophie > wrote: >>> >>> Good morning Barry, >>> >>> Attached are the updated files, there is more useful information in them. >>> >>> Cheers, >>> >>> Sophie >>> >>> From: Blondel, Sophie via Xolotl-psi-development > >>> Sent: Monday, December 9, 2024 17:29 >>> To: Barry Smith > >>> Cc: petsc-users at mcs.anl.gov >; xolotl-psi-development at lists.sourceforge.net > >>> Subject: Re: [Xolotl-psi-development] [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED >>> >>> Hi Barry, >>> >>> I hope you are doing well. >>> >>> Attached are the output. To give a little more context, this is a "new" way of running the code where multiple instances are created and communicate together every few time steps (like coupling the code with itself in memory). Here there are 3 instances that each have a separate TS object, plus one "main" instance that doesn't solve anything but compute rates to exchange between the other instances. >>> >>> Cheers, >>> >>> Sophie >>> >>> From: Barry Smith > >>> Sent: Monday, December 9, 2024 15:12 >>> To: Blondel, Sophie > >>> Cc: petsc-users at mcs.anl.gov >; xolotl-psi-development at lists.sourceforge.net > >>> Subject: Re: [petsc-users] "-ts_exact_final_time matchstep" leads to DIVERGED_STEP_REJECTED >>> >>> >>> >>>> On Dec 9, 2024, at 2:56?PM, Blondel, Sophie via petsc-users > wrote: >>>> >>>> Hi, >>>> >>>> I am trying to understand a strange behavior I'm encountering: when running my application with "-ts_exact_final_time stepover" everything goes well, but when I switch to "matchstep" I get DIVERGED_STEP_REJECTED before the first time step is finished. >>> >>> This is in the very first time-step in TSSolve? >>> >>> Please run with -ts_monitor and send all the output (best for a short time interval and do it twice once with -ts_exact_final_time stepover and once with exact. >>> >>> Barry >>> >>> >>>> I tried increasing the maximum number of rejections and it just takes longer to diverge, and if I set the value to "unlimited" it is basically an infinite loop. >>>> >>>> Is there a way to check why is the step rejected? Could the "matchstep" option change tolerances somewhere that would cause that behavior? >>>> >>>> Let me know if I should provide more information. >>>> >>>> Best, >>>> >>>> Sophie Blondel > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lscalia at pa.uc3m.es Thu Dec 19 10:08:03 2024 From: lscalia at pa.uc3m.es (LUCA SCALIA) Date: Thu, 19 Dec 2024 17:08:03 +0100 Subject: [petsc-users] SLEPC/PETSC for large scale FEM generalised eigenvalue problem In-Reply-To: <74698199-2A8C-4070-8BAA-A1D03BE1B972@dsic.upv.es> References: <74698199-2A8C-4070-8BAA-A1D03BE1B972@dsic.upv.es> Message-ID: Thank you both for your precious advice, I managed to solve the problem in that way Luca On Mon, 9 Dec 2024 at 23:18, Jose E. Roman wrote: > Most probably you want EPS_TARGET_MAGNITUDE with shift-and-invert, i.e., > > -st_type sinvert -eps_target 0 -eps_target_magnitude -st_ksp_type preonly > -st_pc_type lu -st_pc_factor_mat_solver_type mumps > > This will solve a linear system with your A matrix instead of your B > matrix. If A is singular, try a nonzero target located on the left of your > eigenvalues. > > Jose > > > El 9 dic 2024, a las 22:57, Barry Smith escribi?: > > > > > > I'll leave it to Jose to provide a complete, comprehensive answer, but > given your problem sizes, I think using a direct solver instead of an > iterative solver may be the best bet. So configure PETSc without debugging > and with MUMPS > > > > ./configure --with-debugging=0 --download-mumps --download-scalapack > --download-ptscotch > > > > and use -ksp_type preconly -pc_type lu -pc_factor_mat_solver_type mumps > > > > Good luck > > > > > >> On Dec 9, 2024, at 4:50?PM, LUCA SCALIA wrote: > >> > >> Dear group, > >> > >> I am using libraries PETSC and SLEPC for a Finite Element code written > in c++ that I am developing, oriented to the solution of generalised > eigenvalue problems for large structural systems featuring up to 10^5 -10^6 > degrees of freedom. > >> > >> I manage to successfully run the code for a generalised eigenvalue > problem of structures with around 10^4 degrees of freedom in a reasonable > amount of time but the problem arises in the case of systems with a number > of degrees of freedom from 10^5 or above. > >> > >> The current test case I'm trying has 120.000 degrees of freedom. The > EPS solver gets slower and slower as the iterations pass. After a rough > estimate, I would say it may take no less than 5-6 hours with 20 MPI > processes ( allowing an unlimited number of iterations) > >> > >> I need to compute the first 10 to 30 (nev) modes of the structure > (EPS_SMALLEST_MAGNITUDE). I'm using the default KRYLOVSCHUR iterative > solver. The problem is generalised Hermitian > >> > >> > >> I tried to play a little bit with different settings and options of the > library, e.g.: > >> > >> 1) ncv parameter. different values of ncv, between 10*nev and 40*nev > >> > >> 2) different solvers for the ksp object ( GMRES and CG ), and tolerance > as well preconditioner (PCJACOBI and PCBJACOBI) > >> > >> 3) Normalisation of matrices A and B with the respective Frobenius or > Infinite norms. > >> > >> ... but I wasn't able to solve the problem. > >> > >> I kindly ask for advice on which strategies I should resort to in order > to improve the speed of SLEPC for such large numbers of degrees of freedom, > since I'm quite new to the solution of the generalised eigenvalue problem. > >> > >> I also attach two files with the stiffness and mass matrix of one > element to show the magnitude of the entries of the two matrices > >> > >> Best regards, > >> > >> Luca > >> > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From aldo.bonfiglioli at unibas.it Thu Dec 19 11:39:02 2024 From: aldo.bonfiglioli at unibas.it (Aldo Bonfiglioli) Date: Thu, 19 Dec 2024 18:39:02 +0100 Subject: [petsc-users] Advice on setting Dirichlet bcs in DMPlex Message-ID: Dear all, I am using TS to solve the PDE u_t + c * u_c = eps * u_xx in 1D i.e. smthg. similar to https://urldefense.us/v3/__https://petsc.org/release/src/ts/tutorials/ex4.c.html__;!!G_uCfscf7eWS!eeErx_FNg50N1eRlD9eyfoEp8I0llTQmVFpPWoz5_LQhL8K6uOId5zbgle0ghE71NJmon8T_vwwfN3LnEbf4cTUXtlkWJRflikU$ except that I am using DMPlex, instead of a structured grid. Since the problem is linear, I tried smthg. like: > ???PetscCall(RHSMatrixHeat(ts,0.0,u,A,A,&appctx)); > ???PetscCall(TSSetRHSFunction(ts,NULL,TSComputeRHSFunctionLinear,&appctx)); > > ???PetscCall(TSSetRHSJacobian(ts,A,A,TSComputeRHSJacobianConstant,&appctx)); > My problem is that the A matrix (as well as the global vector) is missing the rows/cols corresponding to the endpoints where Dirichlet boundary conditions are prescribed; > A global vector is missing both the shared dofs which are not owned by > this process, as well as /constrained/ dofs. These constraints > represent essential (Dirichlet) boundary conditions. They are dofs > that have a given fixed value, so they are present in local vectors > for assembly purposes, but absent from global vectors since they are > never solved for during algebraic solves. therefore, A*u = f(u) except in the two vertices adjacent to the endpoints, i.e. first and last entries of the global vector. The code is attached; when the flag -use_jacobian is set, f(u) is computed as A*u, if it is not set, f(u) is computed in FormRHSFunction. The latter approach works, the former does not. Regards, Aldo -- Dr. Aldo Bonfiglioli Associate professor of Fluid Machines Scuola di Ingegneria Universita' della Basilicata V.le dell'Ateneo lucano, 10 85100 Potenza ITALY tel:+39.0971.205203 fax:+39.0971.205215 web:https://urldefense.us/v3/__http://docenti.unibas.it/site/home/docente.html?m=002423__;!!G_uCfscf7eWS!eeErx_FNg50N1eRlD9eyfoEp8I0llTQmVFpPWoz5_LQhL8K6uOId5zbgle0ghE71NJmon8T_vwwfN3LnEbf4cTUXtlkWzhmq6Yg$ -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ex15f90.F90 Type: text/x-fortran Size: 28898 bytes Desc: not available URL: -------------- next part -------------- -dm_plex_dim 1 -Peclet 50 -dm_plex_shape box -dm_plex_box_faces 30 -dm_plex_simplex true -dm_plex_interpolate -dm_plex_check_all -dm_petscsection_view -ts_max_steps 5000 -options_left From knepley at gmail.com Thu Dec 19 12:13:16 2024 From: knepley at gmail.com (Matthew Knepley) Date: Thu, 19 Dec 2024 13:13:16 -0500 Subject: [petsc-users] Advice on setting Dirichlet bcs in DMPlex In-Reply-To: References: Message-ID: On Thu, Dec 19, 2024 at 12:39?PM Aldo Bonfiglioli < aldo.bonfiglioli at unibas.it> wrote: > Dear all, > > I am using TS to solve the PDE > > u_t + c * u_c = eps * u_xx in 1D i.e. smthg. similar to > https://urldefense.us/v3/__https://petsc.org/release/src/ts/tutorials/ex4.c.html__;!!G_uCfscf7eWS!bO7mL8fLGJ1HQeUpeYEkzSCNp7tF-J8HcWxslYKIr_sOi2FHpOJIG9c7fuoLoLEyWD7TSfN4tfhCD1mcIe-Z$ > > > except that I am using DMPlex, instead of a structured grid. > > Since the problem is linear, I tried smthg. like: > > PetscCall(RHSMatrixHeat(ts,0.0,u,A,A,&appctx)); > PetscCall(TSSetRHSFunction(ts,NULL,TSComputeRHSFunctionLinear,&appctx)); > > > PetscCall(TSSetRHSJacobian(ts,A,A,TSComputeRHSJacobianConstant,&appctx)); > > My problem is that the A matrix (as well as the global vector) is missing > the rows/cols corresponding to the endpoints where Dirichlet boundary > conditions are prescribed; > > A global vector is missing both the shared dofs which are not owned by > this process, as well as *constrained* dofs. These constraints represent > essential (Dirichlet) boundary conditions. They are dofs that have a given > fixed value, so they are present in local vectors for assembly purposes, > but absent from global vectors since they are never solved for during > algebraic solves. > > therefore, A*u = f(u) except in the two vertices adjacent to the > endpoints, i.e. first and last entries of the global vector. > > The code is attached; when the flag -use_jacobian is set, f(u) is computed > as A*u, if it is not set, f(u) is computed in FormRHSFunction. > > The latter approach works, the former does not. > > I think I understand. When you declare Dirichlet conditions using Plex, it eliminates them from the solver. _However_, the action must be computing using the local space, which contains the constraints. With FEM, this is natural. What you cannot do is use an input vector in the global space, since it is missing the boundary values. Things you could possibly do: 1) Apply your operator in a matrix-free way, taking input in the full space, and giving output in the constrained space. You can assemble the unconstrained operator if assembly is faster, and then map the output to the constrained space. 2) Ignore the constraint support in Plex and manage the constraints yourself using MatZeroRowsColumns() and constants in the rhs for the boundary values. This is what DMDA tends to do. 3) Solve only for the update, since the boundary values do not change in the solve. This is what all the SNES/TS/TAO support for linear solver using Plex does. We assemble using the local space (unconstrained), and output in the global (constrained) space. This is why FormRHSFunction works. Does that make sense? Thanks, Matt > Regards, > > Aldo > > -- > Dr. Aldo Bonfiglioli > Associate professor of Fluid Machines > Scuola di Ingegneria > Universita' della Basilicata > V.le dell'Ateneo lucano, 10 85100 Potenza ITALY > tel:+39.0971.205203 fax:+39.0971.205215 > web: https://urldefense.us/v3/__http://docenti.unibas.it/site/home/docente.html?m=002423__;!!G_uCfscf7eWS!bO7mL8fLGJ1HQeUpeYEkzSCNp7tF-J8HcWxslYKIr_sOi2FHpOJIG9c7fuoLoLEyWD7TSfN4tfhCD1nxZp-t$ > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://urldefense.us/v3/__https://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!bO7mL8fLGJ1HQeUpeYEkzSCNp7tF-J8HcWxslYKIr_sOi2FHpOJIG9c7fuoLoLEyWD7TSfN4tfhCD4BXqfL8$ -------------- next part -------------- An HTML attachment was scrubbed... URL: From hj2000 at mail.ustc.edu.cn Fri Dec 27 01:56:59 2024 From: hj2000 at mail.ustc.edu.cn (hj2000 at mail.ustc.edu.cn) Date: Fri, 27 Dec 2024 15:56:59 +0800 (GMT+08:00) Subject: [petsc-users] PETSc using problem Message-ID: Dear Developer: I am a PETSc user and I have the following error when calculating 1 million grids using PETSc in my windows computer. It came up after calculating it for a while, and before it could calculate many time steps.I wanted to know what caused it. The following is the error message: [1]PETSC ERROR: --------------------- Error Message -------------------------------------------------------------- [1]PETSC ERROR: MatSeqAIJSetPreallocation_SeqAIJ [1]PETSC ERROR: See https://urldefense.us/v3/__https://petsc.org/release/faq/__;!!G_uCfscf7eWS!Yvk0IyBSNujv074PP5zehMZgi-UMJ2Gdw8YLIpjym2HIIXPdlZgEriAuhW47O-G879gHiHUPPkFysUrShi1LkTl9ZYck$ for trouble shooting. [1]PETSC ERROR: Petsc Release Version 3.18.6, unknown [1]PETSC ERROR: Unknown Name on a arch-mswin-c-opt named DESKTOP-B75P775 by Administrator Wed Dec 25 15:56:22 2024 [1]PETSC ERROR: Configure options --with-cc="win32fe cl" --with-cxx="win32fe cl" --with-fc=0 --with-debugging=0 --with-shared-libraries=0 --with-x=0 --with-quad-precision=0 --with-blaslapack-include=/cygdrive/c/PROGRA~2/Intel/oneAPI/mkl/latest/include --with-blaslapack-lib="-L/cygdrive/c/PROGRA~2/Intel/oneAPI/mkl/latest/lib/intel64 -L/cygdrive/c/PROGRA~2/Intel/oneAPI/compiler/latest/windows/compiler/lib/intel64_win mkl_core.lib mkl_intel_thread.lib mkl_intel_lp64.lib libiomp5md.lib" --with-mpi-include=/cygdrive/c/PROGRA~2/Intel/oneAPI/mpi/latest/include --with-mpi-lib="-L/cygdrive/c/PROGRA~2/Intel/oneAPI/mpi/latest/lib/release impi.lib" --with-mpiexec=/cygdrive/c/PROGRA~2/Intel/oneAPI/mpi/latest/bin/mpiexec --with-metis-include=/cygdrive/d/PETSc/petsc-v3.18.6-pkgconfig//packages/pkg-metis/install/include --with-metis-lib="-L/cygdrive/d/PETSc/petsc-v3.18.6-pkgconfig//packages/pkg-metis/install/lib metis.lib" --with-parmetis-include=/cygdrive/d/PETSc/petsc-v3.18.6-pkgconfig//packages/pkg-parmetis/install/include --with-parmetis-lib="-L/cygdrive/d/PETSc/petsc-v3.18.6-pkgconfig//packages/pkg-parmetis/install/lib parmetis.lib" --with-hdf5-include=/cygdrive/d/PETSc/petsc-v3.18.6-pkgconfig//packages/pkg-hdf5/install/include --with-hdf5-lib="-L/cygdrive/d/PETSc/petsc-v3.18.6-pkgconfig//packages/pkg-hdf5/install/lib libhdf5.lib" --ignore-cygwin-link [5]PETSC ERROR: --------------------- Error Message -------------------------------------------------------------- [5]PETSC ERROR: MatSeqAIJSetPreallocation_SeqAIJ [5]PETSC ERROR: See https://urldefense.us/v3/__https://petsc.org/release/faq/__;!!G_uCfscf7eWS!Yvk0IyBSNujv074PP5zehMZgi-UMJ2Gdw8YLIpjym2HIIXPdlZgEriAuhW47O-G879gHiHUPPkFysUrShi1LkTl9ZYck$ for trouble shooting. [5]PETSC ERROR: Petsc Release Version 3.18.6, unknown [5]PETSC ERROR: Unknown Name on a arch-mswin-c-opt named DESKTOP-B75P775 by Administrator Wed Dec 25 15:56:22 2024 [5]PETSC ERROR: Configure options --with-cc="win32fe cl" --with-cxx="win32fe cl" --with-fc=0 --with-debugging=0 --with-shared-libraries=0 --with-x=0 --with-quad-precision=0 --with-blaslapack-include=/cygdrive/c/PROGRA~2/Intel/oneAPI/mkl/latest/include --with-blaslapack-lib="-L/cygdrive/c/PROGRA~2/Intel/oneAPI/mkl/latest/lib/intel64 -L/cygdrive/c/PROGRA~2/Intel/oneAPI/compiler/latest/windows/compiler/lib/intel64_win mkl_core.lib mkl_intel_thread.lib mkl_intel_lp64.lib libiomp5md.lib" --with-mpi-include=/cygdrive/c/PROGRA~2/Intel/oneAPI/mpi/latest/include --with-mpi-lib="-L/cygdrive/c/PROGRA~2/Intel/oneAPI/mpi/latest/lib/release impi.lib" --with-mpiexec=/cygdrive/c/PROGRA~2/Intel/oneAPI/mpi/latest/bin/mpiexec --with-metis-include=/cygdrive/d/PETSc/petsc-v3.18.6-pkgconfig//packages/pkg-metis/install/include --with-metis-lib="-L/cygdrive/d/PETSc/petsc-v3.18.6-pkgconfig//packages/pkg-metis/install/lib metis.lib" --with-parmetis-include=/cygdrive/d/PETSc/petsc-v3.18.6-pkgconfig//packages/pkg-parmetis/install/include --with-parmetis-lib="-L/cygdrive/d/PETSc/petsc-v3.18.6-pkgconfig//packages/pkg-parmetis/install/lib parmetis.lib" --with-hdf5-include=/cygdrive/d/PETSc/petsc-v3.18.6-pkgconfig//packages/pkg-hdf5/install/include --with-hdf5-lib="-L/cygdrive/d/PETSc/petsc-v3.18.6-pkgconfig//packages/pkg-hdf5/install/lib libhdf5.lib" --ignore-cygwin-link [4]PETSC ERROR: --------------------- Error Message -------------------------------------------------------------- [4]PETSC ERROR: MatSeqAIJSetPreallocation_SeqAIJ [4]PETSC ERROR: See https://urldefense.us/v3/__https://petsc.org/release/faq/__;!!G_uCfscf7eWS!Yvk0IyBSNujv074PP5zehMZgi-UMJ2Gdw8YLIpjym2HIIXPdlZgEriAuhW47O-G879gHiHUPPkFysUrShi1LkTl9ZYck$ for trouble shooting. [4]PETSC ERROR: Petsc Release Version 3.18.6, unknown [4]PETSC ERROR: Unknown Name on a arch-mswin-c-opt named DESKTOP-B75P775 by Administrator Wed Dec 25 15:56:22 2024 [4]PETSC ERROR: Configure options --with-cc="win32fe cl" --with-cxx="win32fe cl" --with-fc=0 --with-debugging=0 --with-shared-libraries=0 --with-x=0 --with-quad-precision=0 --with-blaslapack-include=/cygdrive/c/PROGRA~2/Intel/oneAPI/mkl/latest/include --with-blaslapack-lib="-L/cygdrive/c/PROGRA~2/Intel/oneAPI/mkl/latest/lib/intel64 -L/cygdrive/c/PROGRA~2/Intel/oneAPI/compiler/latest/windows/compiler/lib/intel64_win mkl_core.lib mkl_intel_thread.lib mkl_intel_lp64.lib libiomp5md.lib" --with-mpi-include=/cygdrive/c/PROGRA~2/Intel/oneAPI/mpi/latest/include --with-mpi-lib="-L/cygdrive/c/PROGRA~2/Intel/oneAPI/mpi/latest/lib/release impi.lib" --with-mpiexec=/cygdrive/c/PROGRA~2/Intel/oneAPI/mpi/latest/bin/mpiexec --with-metis-include=/cygdrive/d/PETSc/petsc-v3.18.6-pkgconfig//packages/pkg-metis/install/include --with-metis-lib="-L/cygdrive/d/PETSc/petsc-v3.18.6-pkgconfig//packages/pkg-metis/install/lib metis.lib" --with-parmetis-include=/cygdrive/d/PETSc/petsc-v3.18.6-pkgconfig//packages/pkg-parmetis/install/include --with-parmetis-lib="-L/cygdrive/d/PETSc/petsc-v3.18.6-pkgconfig//packages/pkg-parmetis/install/lib parmetis.lib" --with-hdf5-include=/cygdrive/d/PETSc/petsc-v3.18.6-pkgconfig//packages/pkg-hdf5/install/include --with-hdf5-lib="-L/cygdrive/d/PETSc/petsc-v3.18.6-pkgconfig//packages/pkg-hdf5/install/lib libhdf5.lib" --ignore-cygwin-link [1]PETSC ERROR: #1 PetscMallocAlign() at D:\PETSc\petsc-v3.18.6-pkgconfig\src\sys\memory\mal.c:68 [1]PETSC ERROR: #2 PetscMallocA() at D:\PETSc\petsc-v3.18.6-pkgconfig\src\sys\memory\mal.c:410 [1]PETSC ERROR: #3 MatSeqAIJSetPreallocation_SeqAIJ() at D:\PETSc\petsc-v3.18.6-pkgconfig\src\mat\impls\aij\seq\aij.c:4008 [1]PETSC ERROR: #4 MatCreateSubMatrix_SeqAIJ() at D:\PETSc\petsc-v3.18.6-pkgconfig\src\mat\impls\aij\seq\aij.c:2588 [1]PETSC ERROR: #5 MatCreateSubMatrix_MPIAIJ_SameRowColDist() at D:\PETSc\petsc-v3.18.6-pkgconfig\src\mat\impls\aij\mpi\mpiaij.c:3283 [1]PETSC ERROR: #6 MatCreateSubMatrix_MPIAIJ() at D:\PETSc\petsc-v3.18.6-pkgconfig\src\mat\impls\aij\mpi\mpiaij.c:3390 [1]PETSC ERROR: #7 MatCreateSubMatrix() at D:\PETSc\petsc-v3.18.6-pkgconfig\src\mat\interface\matrix.c:8382 [1]PETSC ERROR: #8 SNESSolve_VINEWTONRSLS() at D:\PETSc\petsc-v3.18.6-pkgconfig\src\snes\impls\vi\rs\virs.c:392 [1]PETSC ERROR: [5]PETSC ERROR: #1 PetscMallocAlign() at D:\PETSc\petsc-v3.18.6-pkgconfig\src\sys\memory\mal.c:68 [5]PETSC ERROR: #2 PetscMallocA() at D:\PETSc\petsc-v3.18.6-pkgconfig\src\sys\memory\mal.c:410 [5]PETSC ERROR: #3 MatSeqAIJSetPreallocation_SeqAIJ() at D:\PETSc\petsc-v3.18.6-pkgconfig\src\mat\impls\aij\seq\aij.c:4008 [5]PETSC ERROR: #4 MatCreateSubMatrix_SeqAIJ() at D:\PETSc\petsc-v3.18.6-pkgconfig\src\mat\impls\aij\seq\aij.c:2588 [5]PETSC ERROR: #5 MatCreateSubMatrix_MPIAIJ_SameRowColDist() at D:\PETSc\petsc-v3.18.6-pkgconfig\src\mat\impls\aij\mpi\mpiaij.c:3283 [5]PETSC ERROR: #6 MatCreateSubMatrix_MPIAIJ() at D:\PETSc\petsc-v3.18.6-pkgconfig\src\mat\impls\aij\mpi\mpiaij.c:3390 [5]PETSC ERROR: #7 MatCreateSubMatrix() at D:\PETSc\petsc-v3.18.6-pkgconfig\src\mat\interface\matrix.c:8382 [5]PETSC ERROR: #8 SNESSolve_VINEWTONRSLS() at D:\PETSc\petsc-v3.18.6-pkgconfig\src\snes\impls\vi\rs\virs.c:392 [5]PETSC ERROR: [4]PETSC ERROR: #1 PetscMallocAlign() at D:\PETSc\petsc-v3.18.6-pkgconfig\src\sys\memory\mal.c:68 [4]PETSC ERROR: #2 PetscMallocA() at D:\PETSc\petsc-v3.18.6-pkgconfig\src\sys\memory\mal.c:410 [4]PETSC ERROR: #3 MatSeqAIJSetPreallocation_SeqAIJ() at D:\PETSc\petsc-v3.18.6-pkgconfig\src\mat\impls\aij\seq\aij.c:4008 [4]PETSC ERROR: #4 MatCreateSubMatrix_SeqAIJ() at D:\PETSc\petsc-v3.18.6-pkgconfig\src\mat\impls\aij\seq\aij.c:2588 [4]PETSC ERROR: #5 MatCreateSubMatrix_MPIAIJ_SameRowColDist() at D:\PETSc\petsc-v3.18.6-pkgconfig\src\mat\impls\aij\mpi\mpiaij.c:3283 [4]PETSC ERROR: #6 MatCreateSubMatrix_MPIAIJ() at D:\PETSc\petsc-v3.18.6-pkgconfig\src\mat\impls\aij\mpi\mpiaij.c:3390 #9 SNESSolve() at D:\PETSc\petsc-v3.18.6-pkgconfig\src\snes\interface\snes.c:4692 [4]PETSC ERROR: #7 MatCreateSubMatrix() at D:\PETSc\petsc-v3.18.6-pkgconfig\src\mat\interface\matrix.c:8382 [4]PETSC ERROR: #8 SNESSolve_VINEWTONRSLS() at D:\PETSc\petsc-v3.18.6-pkgconfig\src\snes\impls\vi\rs\virs.c:392 [4]PETSC ERROR: #9 SNESSolve() at D:\PETSc\petsc-v3.18.6-pkgconfig\src\snes\interface\snes.c:4692 #9 SNESSolve() at D:\PETSc\petsc-v3.18.6-pkgconfig\src\snes\interface\snes.c:4692 -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Fri Dec 27 09:23:36 2024 From: knepley at gmail.com (Matthew Knepley) Date: Fri, 27 Dec 2024 10:23:36 -0500 Subject: [petsc-users] PETSc using problem In-Reply-To: References: Message-ID: On Fri, Dec 27, 2024 at 2:57?AM jun hu via petsc-users < petsc-users at mcs.anl.gov> wrote: > Dear Developer: > > I am a PETSc user and I have the following error when calculating 1 > million grids using PETSc in my windows computer. It came up after > calculating it for a while, > This is an out-of-memory error. Perhaps you are holding onto memory? You can run a small run using -malloc_debug and it will show unfreed memory at the end. Thanks, Matt > and before it could calculate many time steps.I wanted to know what > caused it. The following is the error message: > > [1]PETSC ERROR: --------------------- Error Message > -------------------------------------------------------------- > > [1]PETSC ERROR: MatSeqAIJSetPreallocation_SeqAIJ > > [1]PETSC ERROR: See https://urldefense.us/v3/__https://petsc.org/release/faq/__;!!G_uCfscf7eWS!eAD-KairPbYZWQ-aKVerz_lz1iMg9eDKuiDquFFftoE6orMSZ_p0mceQ-3CXYflKBORJAbSjb6b5vV6LwXMP$ > > for trouble shooting. > > [1]PETSC ERROR: Petsc Release Version 3.18.6, unknown > > [1]PETSC ERROR: Unknown Name on a arch-mswin-c-opt named DESKTOP-B75P775 > by Administrator Wed Dec 25 15:56:22 2024 > > [1]PETSC ERROR: Configure options --with-cc="win32fe cl" > --with-cxx="win32fe cl" --with-fc=0 --with-debugging=0 > --with-shared-libraries=0 --with-x=0 --with-quad-precision=0 > --with-blaslapack-include=/cygdrive/c/PROGRA~2/Intel/oneAPI/mkl/latest/include > --with-blaslapack-lib="-L/cygdrive/c/PROGRA~2/Intel/oneAPI/mkl/latest/lib/intel64 > -L/cygdrive/c/PROGRA~2/Intel/oneAPI/compiler/latest/windows/compiler/lib/intel64_win > mkl_core.lib mkl_intel_thread.lib > > mkl_intel_lp64.lib libiomp5md.lib" > --with-mpi-include=/cygdrive/c/PROGRA~2/Intel/oneAPI/mpi/latest/include > --with-mpi-lib="-L/cygdrive/c/PROGRA~2/Intel/oneAPI/mpi/latest/lib/release > impi.lib" > --with-mpiexec=/cygdrive/c/PROGRA~2/Intel/oneAPI/mpi/latest/bin/mpiexec > --with-metis-include=/cygdrive/d/PETSc/petsc-v3.18.6-pkgconfig//packages/pkg-metis/install/include > --with-metis-lib="-L/cygdrive/d/PETSc/petsc-v3.18.6-pkgconfig//packages/pkg-metis/install/lib > metis.lib" > --with-parmetis-include=/cygdrive/d/PETSc/petsc-v3.18.6-pkgconfig//packages/pkg-parmetis/install/include > --with-parmetis-lib="-L/cygdrive/d/PETSc/petsc-v3.18.6-pkgconfig//packages/pkg-parmetis/install/lib > parmetis.lib" > --with-hdf5-include=/cygdrive/d/PETSc/petsc-v3.18.6-pkgconfig//packages/pkg-hdf5/install/include > --with-hdf5-lib="-L/cygdrive/d/PETSc/petsc-v3.18.6-pkgconfig//packages/pkg-hdf5/install/lib > libhdf5.lib" --ignore-cygwin-link > > [5]PETSC ERROR: --------------------- Error Message > -------------------------------------------------------------- > > [5]PETSC ERROR: MatSeqAIJSetPreallocation_SeqAIJ > > [5]PETSC ERROR: See https://urldefense.us/v3/__https://petsc.org/release/faq/__;!!G_uCfscf7eWS!eAD-KairPbYZWQ-aKVerz_lz1iMg9eDKuiDquFFftoE6orMSZ_p0mceQ-3CXYflKBORJAbSjb6b5vV6LwXMP$ > > for trouble shooting. > > [5]PETSC ERROR: Petsc Release Version 3.18.6, unknown > > [5]PETSC ERROR: Unknown Name on a arch-mswin-c-opt named DESKTOP-B75P775 > by Administrator Wed Dec 25 15:56:22 2024 > > [5]PETSC ERROR: Configure options --with-cc="win32fe cl" > --with-cxx="win32fe cl" --with-fc=0 --with-debugging=0 > --with-shared-libraries=0 --with-x=0 --with-quad-precision=0 > --with-blaslapack-include=/cygdrive/c/PROGRA~2/Intel/oneAPI/mkl/latest/include > --with-blaslapack-lib="-L/cygdrive/c/PROGRA~2/Intel/oneAPI/mkl/latest/lib/intel64 > -L/cygdrive/c/PROGRA~2/Intel/oneAPI/compiler/latest/windows/compiler/lib/intel64_win > mkl_core.lib mkl_intel_thread.lib > > mkl_intel_lp64.lib libiomp5md.lib" > --with-mpi-include=/cygdrive/c/PROGRA~2/Intel/oneAPI/mpi/latest/include > --with-mpi-lib="-L/cygdrive/c/PROGRA~2/Intel/oneAPI/mpi/latest/lib/release > impi.lib" > --with-mpiexec=/cygdrive/c/PROGRA~2/Intel/oneAPI/mpi/latest/bin/mpiexec > --with-metis-include=/cygdrive/d/PETSc/petsc-v3.18.6-pkgconfig//packages/pkg-metis/install/include > --with-metis-lib="-L/cygdrive/d/PETSc/petsc-v3.18.6-pkgconfig//packages/pkg-metis/install/lib > metis.lib" > --with-parmetis-include=/cygdrive/d/PETSc/petsc-v3.18.6-pkgconfig//packages/pkg-parmetis/install/include > --with-parmetis-lib="-L/cygdrive/d/PETSc/petsc-v3.18.6-pkgconfig//packages/pkg-parmetis/install/lib > parmetis.lib" > --with-hdf5-include=/cygdrive/d/PETSc/petsc-v3.18.6-pkgconfig//packages/pkg-hdf5/install/include > --with-hdf5-lib="-L/cygdrive/d/PETSc/petsc-v3.18.6-pkgconfig//packages/pkg-hdf5/install/lib > libhdf5.lib" --ignore-cygwin-link > > [4]PETSC ERROR: --------------------- Error Message > -------------------------------------------------------------- > > [4]PETSC ERROR: MatSeqAIJSetPreallocation_SeqAIJ > > [4]PETSC ERROR: See https://urldefense.us/v3/__https://petsc.org/release/faq/__;!!G_uCfscf7eWS!eAD-KairPbYZWQ-aKVerz_lz1iMg9eDKuiDquFFftoE6orMSZ_p0mceQ-3CXYflKBORJAbSjb6b5vV6LwXMP$ > > for trouble shooting. > > [4]PETSC ERROR: Petsc Release Version 3.18.6, unknown > > [4]PETSC ERROR: Unknown Name on a arch-mswin-c-opt named DESKTOP-B75P775 > by Administrator Wed Dec 25 15:56:22 2024 > > [4]PETSC ERROR: Configure options --with-cc="win32fe cl" > --with-cxx="win32fe cl" --with-fc=0 --with-debugging=0 > --with-shared-libraries=0 --with-x=0 --with-quad-precision=0 > --with-blaslapack-include=/cygdrive/c/PROGRA~2/Intel/oneAPI/mkl/latest/include > --with-blaslapack-lib="-L/cygdrive/c/PROGRA~2/Intel/oneAPI/mkl/latest/lib/intel64 > -L/cygdrive/c/PROGRA~2/Intel/oneAPI/compiler/latest/windows/compiler/lib/intel64_win > mkl_core.lib mkl_intel_thread.lib > > mkl_intel_lp64.lib libiomp5md.lib" > --with-mpi-include=/cygdrive/c/PROGRA~2/Intel/oneAPI/mpi/latest/include > --with-mpi-lib="-L/cygdrive/c/PROGRA~2/Intel/oneAPI/mpi/latest/lib/release > impi.lib" > --with-mpiexec=/cygdrive/c/PROGRA~2/Intel/oneAPI/mpi/latest/bin/mpiexec > --with-metis-include=/cygdrive/d/PETSc/petsc-v3.18.6-pkgconfig//packages/pkg-metis/install/include > --with-metis-lib="-L/cygdrive/d/PETSc/petsc-v3.18.6-pkgconfig//packages/pkg-metis/install/lib > metis.lib" > --with-parmetis-include=/cygdrive/d/PETSc/petsc-v3.18.6-pkgconfig//packages/pkg-parmetis/install/include > --with-parmetis-lib="-L/cygdrive/d/PETSc/petsc-v3.18.6-pkgconfig//packages/pkg-parmetis/install/lib > parmetis.lib" > --with-hdf5-include=/cygdrive/d/PETSc/petsc-v3.18.6-pkgconfig//packages/pkg-hdf5/install/include > --with-hdf5-lib="-L/cygdrive/d/PETSc/petsc-v3.18.6-pkgconfig//packages/pkg-hdf5/install/lib > libhdf5.lib" --ignore-cygwin-link > > [1]PETSC ERROR: #1 PetscMallocAlign() at > D:\PETSc\petsc-v3.18.6-pkgconfig\src\sys\memory\mal.c:68 > > [1]PETSC ERROR: #2 PetscMallocA() at > D:\PETSc\petsc-v3.18.6-pkgconfig\src\sys\memory\mal.c:410 > > [1]PETSC ERROR: #3 MatSeqAIJSetPreallocation_SeqAIJ() at > D:\PETSc\petsc-v3.18.6-pkgconfig\src\mat\impls\aij\seq\aij.c:4008 > > [1]PETSC ERROR: #4 MatCreateSubMatrix_SeqAIJ() at > D:\PETSc\petsc-v3.18.6-pkgconfig\src\mat\impls\aij\seq\aij.c:2588 > > [1]PETSC ERROR: #5 MatCreateSubMatrix_MPIAIJ_SameRowColDist() at > D:\PETSc\petsc-v3.18.6-pkgconfig\src\mat\impls\aij\mpi\mpiaij.c:3283 > > [1]PETSC ERROR: #6 MatCreateSubMatrix_MPIAIJ() at > D:\PETSc\petsc-v3.18.6-pkgconfig\src\mat\impls\aij\mpi\mpiaij.c:3390 > > [1]PETSC ERROR: #7 MatCreateSubMatrix() at > D:\PETSc\petsc-v3.18.6-pkgconfig\src\mat\interface\matrix.c:8382 > > [1]PETSC ERROR: #8 SNESSolve_VINEWTONRSLS() at > D:\PETSc\petsc-v3.18.6-pkgconfig\src\snes\impls\vi\rs\virs.c:392 > > [1]PETSC ERROR: [5]PETSC ERROR: #1 PetscMallocAlign() at > D:\PETSc\petsc-v3.18.6-pkgconfig\src\sys\memory\mal.c:68 > > [5]PETSC ERROR: #2 PetscMallocA() at > D:\PETSc\petsc-v3.18.6-pkgconfig\src\sys\memory\mal.c:410 > > [5]PETSC ERROR: #3 MatSeqAIJSetPreallocation_SeqAIJ() at > D:\PETSc\petsc-v3.18.6-pkgconfig\src\mat\impls\aij\seq\aij.c:4008 > > [5]PETSC ERROR: #4 MatCreateSubMatrix_SeqAIJ() at > D:\PETSc\petsc-v3.18.6-pkgconfig\src\mat\impls\aij\seq\aij.c:2588 > > [5]PETSC ERROR: #5 MatCreateSubMatrix_MPIAIJ_SameRowColDist() at > D:\PETSc\petsc-v3.18.6-pkgconfig\src\mat\impls\aij\mpi\mpiaij.c:3283 > > [5]PETSC ERROR: #6 MatCreateSubMatrix_MPIAIJ() at > D:\PETSc\petsc-v3.18.6-pkgconfig\src\mat\impls\aij\mpi\mpiaij.c:3390 > > [5]PETSC ERROR: #7 MatCreateSubMatrix() at > D:\PETSc\petsc-v3.18.6-pkgconfig\src\mat\interface\matrix.c:8382 > > [5]PETSC ERROR: #8 SNESSolve_VINEWTONRSLS() at > D:\PETSc\petsc-v3.18.6-pkgconfig\src\snes\impls\vi\rs\virs.c:392 > > [5]PETSC ERROR: [4]PETSC ERROR: #1 PetscMallocAlign() at > D:\PETSc\petsc-v3.18.6-pkgconfig\src\sys\memory\mal.c:68 > > [4]PETSC ERROR: #2 PetscMallocA() at > D:\PETSc\petsc-v3.18.6-pkgconfig\src\sys\memory\mal.c:410 > > [4]PETSC ERROR: #3 MatSeqAIJSetPreallocation_SeqAIJ() at > D:\PETSc\petsc-v3.18.6-pkgconfig\src\mat\impls\aij\seq\aij.c:4008 > > [4]PETSC ERROR: #4 MatCreateSubMatrix_SeqAIJ() at > D:\PETSc\petsc-v3.18.6-pkgconfig\src\mat\impls\aij\seq\aij.c:2588 > > [4]PETSC ERROR: #5 MatCreateSubMatrix_MPIAIJ_SameRowColDist() at > D:\PETSc\petsc-v3.18.6-pkgconfig\src\mat\impls\aij\mpi\mpiaij.c:3283 > > [4]PETSC ERROR: #6 MatCreateSubMatrix_MPIAIJ() at > D:\PETSc\petsc-v3.18.6-pkgconfig\src\mat\impls\aij\mpi\mpiaij.c:3390 > > #9 SNESSolve() at > D:\PETSc\petsc-v3.18.6-pkgconfig\src\snes\interface\snes.c:4692 > > [4]PETSC ERROR: #7 MatCreateSubMatrix() at > D:\PETSc\petsc-v3.18.6-pkgconfig\src\mat\interface\matrix.c:8382 > > [4]PETSC ERROR: #8 SNESSolve_VINEWTONRSLS() at > D:\PETSc\petsc-v3.18.6-pkgconfig\src\snes\impls\vi\rs\virs.c:392 > > [4]PETSC ERROR: #9 SNESSolve() at > D:\PETSc\petsc-v3.18.6-pkgconfig\src\snes\interface\snes.c:4692 > > #9 SNESSolve() at > D:\PETSc\petsc-v3.18.6-pkgconfig\src\snes\interface\snes.c:4692 > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://urldefense.us/v3/__https://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!eAD-KairPbYZWQ-aKVerz_lz1iMg9eDKuiDquFFftoE6orMSZ_p0mceQ-3CXYflKBORJAbSjb6b5vUp9gcgS$ -------------- next part -------------- An HTML attachment was scrubbed... URL: From balay.anl at fastmail.org Fri Dec 27 10:22:47 2024 From: balay.anl at fastmail.org (Satish Balay) Date: Fri, 27 Dec 2024 10:22:47 -0600 (CST) Subject: [petsc-users] PETSc using problem In-Reply-To: References: Message-ID: <3d1c082b-ae63-768e-46de-3dc8f24dc0bd@fastmail.org> Perhaps --with-64-bit-indices=1 would help this use case? if using latest petsc version - the following build might work: --with-cc=win32fe_cl --with-cxx=win32fe_cl --with-64-bit-indices=1 --download-metis --download-parmetis .... Satish On Fri, 27 Dec 2024, Matthew Knepley wrote: > On Fri, Dec 27, 2024 at 2:57?AM jun hu via petsc-users < > petsc-users at mcs.anl.gov> wrote: > > > Dear Developer: > > > > I am a PETSc user and I have the following error when calculating 1 > > million grids using PETSc in my windows computer. It came up after > > calculating it for a while, > > > This is an out-of-memory error. Perhaps you are holding onto memory? You > can run a small run using -malloc_debug and > it will show unfreed memory at the end. > > Thanks, > > Matt > > > > and before it could calculate many time steps.I wanted to know what > > caused it. The following is the error message: > > > > [1]PETSC ERROR: --------------------- Error Message > > -------------------------------------------------------------- > > > > [1]PETSC ERROR: MatSeqAIJSetPreallocation_SeqAIJ > > > > [1]PETSC ERROR: See https://urldefense.us/v3/__https://petsc.org/release/faq/__;!!G_uCfscf7eWS!eAD-KairPbYZWQ-aKVerz_lz1iMg9eDKuiDquFFftoE6orMSZ_p0mceQ-3CXYflKBORJAbSjb6b5vV6LwXMP$ > > > > for trouble shooting. > > > > [1]PETSC ERROR: Petsc Release Version 3.18.6, unknown > > > > [1]PETSC ERROR: Unknown Name on a arch-mswin-c-opt named DESKTOP-B75P775 > > by Administrator Wed Dec 25 15:56:22 2024 > > > > [1]PETSC ERROR: Configure options --with-cc="win32fe cl" > > --with-cxx="win32fe cl" --with-fc=0 --with-debugging=0 > > --with-shared-libraries=0 --with-x=0 --with-quad-precision=0 > > --with-blaslapack-include=/cygdrive/c/PROGRA~2/Intel/oneAPI/mkl/latest/include > > --with-blaslapack-lib="-L/cygdrive/c/PROGRA~2/Intel/oneAPI/mkl/latest/lib/intel64 > > -L/cygdrive/c/PROGRA~2/Intel/oneAPI/compiler/latest/windows/compiler/lib/intel64_win > > mkl_core.lib mkl_intel_thread.lib > > > > mkl_intel_lp64.lib libiomp5md.lib" > > --with-mpi-include=/cygdrive/c/PROGRA~2/Intel/oneAPI/mpi/latest/include > > --with-mpi-lib="-L/cygdrive/c/PROGRA~2/Intel/oneAPI/mpi/latest/lib/release > > impi.lib" > > --with-mpiexec=/cygdrive/c/PROGRA~2/Intel/oneAPI/mpi/latest/bin/mpiexec > > --with-metis-include=/cygdrive/d/PETSc/petsc-v3.18.6-pkgconfig//packages/pkg-metis/install/include > > --with-metis-lib="-L/cygdrive/d/PETSc/petsc-v3.18.6-pkgconfig//packages/pkg-metis/install/lib > > metis.lib" > > --with-parmetis-include=/cygdrive/d/PETSc/petsc-v3.18.6-pkgconfig//packages/pkg-parmetis/install/include > > --with-parmetis-lib="-L/cygdrive/d/PETSc/petsc-v3.18.6-pkgconfig//packages/pkg-parmetis/install/lib > > parmetis.lib" > > --with-hdf5-include=/cygdrive/d/PETSc/petsc-v3.18.6-pkgconfig//packages/pkg-hdf5/install/include > > --with-hdf5-lib="-L/cygdrive/d/PETSc/petsc-v3.18.6-pkgconfig//packages/pkg-hdf5/install/lib > > libhdf5.lib" --ignore-cygwin-link > > > > [5]PETSC ERROR: --------------------- Error Message > > -------------------------------------------------------------- > > > > [5]PETSC ERROR: MatSeqAIJSetPreallocation_SeqAIJ > > > > [5]PETSC ERROR: See https://urldefense.us/v3/__https://petsc.org/release/faq/__;!!G_uCfscf7eWS!eAD-KairPbYZWQ-aKVerz_lz1iMg9eDKuiDquFFftoE6orMSZ_p0mceQ-3CXYflKBORJAbSjb6b5vV6LwXMP$ > > > > for trouble shooting. > > > > [5]PETSC ERROR: Petsc Release Version 3.18.6, unknown > > > > [5]PETSC ERROR: Unknown Name on a arch-mswin-c-opt named DESKTOP-B75P775 > > by Administrator Wed Dec 25 15:56:22 2024 > > > > [5]PETSC ERROR: Configure options --with-cc="win32fe cl" > > --with-cxx="win32fe cl" --with-fc=0 --with-debugging=0 > > --with-shared-libraries=0 --with-x=0 --with-quad-precision=0 > > --with-blaslapack-include=/cygdrive/c/PROGRA~2/Intel/oneAPI/mkl/latest/include > > --with-blaslapack-lib="-L/cygdrive/c/PROGRA~2/Intel/oneAPI/mkl/latest/lib/intel64 > > -L/cygdrive/c/PROGRA~2/Intel/oneAPI/compiler/latest/windows/compiler/lib/intel64_win > > mkl_core.lib mkl_intel_thread.lib > > > > mkl_intel_lp64.lib libiomp5md.lib" > > --with-mpi-include=/cygdrive/c/PROGRA~2/Intel/oneAPI/mpi/latest/include > > --with-mpi-lib="-L/cygdrive/c/PROGRA~2/Intel/oneAPI/mpi/latest/lib/release > > impi.lib" > > --with-mpiexec=/cygdrive/c/PROGRA~2/Intel/oneAPI/mpi/latest/bin/mpiexec > > --with-metis-include=/cygdrive/d/PETSc/petsc-v3.18.6-pkgconfig//packages/pkg-metis/install/include > > --with-metis-lib="-L/cygdrive/d/PETSc/petsc-v3.18.6-pkgconfig//packages/pkg-metis/install/lib > > metis.lib" > > --with-parmetis-include=/cygdrive/d/PETSc/petsc-v3.18.6-pkgconfig//packages/pkg-parmetis/install/include > > --with-parmetis-lib="-L/cygdrive/d/PETSc/petsc-v3.18.6-pkgconfig//packages/pkg-parmetis/install/lib > > parmetis.lib" > > --with-hdf5-include=/cygdrive/d/PETSc/petsc-v3.18.6-pkgconfig//packages/pkg-hdf5/install/include > > --with-hdf5-lib="-L/cygdrive/d/PETSc/petsc-v3.18.6-pkgconfig//packages/pkg-hdf5/install/lib > > libhdf5.lib" --ignore-cygwin-link > > > > [4]PETSC ERROR: --------------------- Error Message > > -------------------------------------------------------------- > > > > [4]PETSC ERROR: MatSeqAIJSetPreallocation_SeqAIJ > > > > [4]PETSC ERROR: See https://urldefense.us/v3/__https://petsc.org/release/faq/__;!!G_uCfscf7eWS!eAD-KairPbYZWQ-aKVerz_lz1iMg9eDKuiDquFFftoE6orMSZ_p0mceQ-3CXYflKBORJAbSjb6b5vV6LwXMP$ > > > > for trouble shooting. > > > > [4]PETSC ERROR: Petsc Release Version 3.18.6, unknown > > > > [4]PETSC ERROR: Unknown Name on a arch-mswin-c-opt named DESKTOP-B75P775 > > by Administrator Wed Dec 25 15:56:22 2024 > > > > [4]PETSC ERROR: Configure options --with-cc="win32fe cl" > > --with-cxx="win32fe cl" --with-fc=0 --with-debugging=0 > > --with-shared-libraries=0 --with-x=0 --with-quad-precision=0 > > --with-blaslapack-include=/cygdrive/c/PROGRA~2/Intel/oneAPI/mkl/latest/include > > --with-blaslapack-lib="-L/cygdrive/c/PROGRA~2/Intel/oneAPI/mkl/latest/lib/intel64 > > -L/cygdrive/c/PROGRA~2/Intel/oneAPI/compiler/latest/windows/compiler/lib/intel64_win > > mkl_core.lib mkl_intel_thread.lib > > > > mkl_intel_lp64.lib libiomp5md.lib" > > --with-mpi-include=/cygdrive/c/PROGRA~2/Intel/oneAPI/mpi/latest/include > > --with-mpi-lib="-L/cygdrive/c/PROGRA~2/Intel/oneAPI/mpi/latest/lib/release > > impi.lib" > > --with-mpiexec=/cygdrive/c/PROGRA~2/Intel/oneAPI/mpi/latest/bin/mpiexec > > --with-metis-include=/cygdrive/d/PETSc/petsc-v3.18.6-pkgconfig//packages/pkg-metis/install/include > > --with-metis-lib="-L/cygdrive/d/PETSc/petsc-v3.18.6-pkgconfig//packages/pkg-metis/install/lib > > metis.lib" > > --with-parmetis-include=/cygdrive/d/PETSc/petsc-v3.18.6-pkgconfig//packages/pkg-parmetis/install/include > > --with-parmetis-lib="-L/cygdrive/d/PETSc/petsc-v3.18.6-pkgconfig//packages/pkg-parmetis/install/lib > > parmetis.lib" > > --with-hdf5-include=/cygdrive/d/PETSc/petsc-v3.18.6-pkgconfig//packages/pkg-hdf5/install/include > > --with-hdf5-lib="-L/cygdrive/d/PETSc/petsc-v3.18.6-pkgconfig//packages/pkg-hdf5/install/lib > > libhdf5.lib" --ignore-cygwin-link > > > > [1]PETSC ERROR: #1 PetscMallocAlign() at > > D:\PETSc\petsc-v3.18.6-pkgconfig\src\sys\memory\mal.c:68 > > > > [1]PETSC ERROR: #2 PetscMallocA() at > > D:\PETSc\petsc-v3.18.6-pkgconfig\src\sys\memory\mal.c:410 > > > > [1]PETSC ERROR: #3 MatSeqAIJSetPreallocation_SeqAIJ() at > > D:\PETSc\petsc-v3.18.6-pkgconfig\src\mat\impls\aij\seq\aij.c:4008 > > > > [1]PETSC ERROR: #4 MatCreateSubMatrix_SeqAIJ() at > > D:\PETSc\petsc-v3.18.6-pkgconfig\src\mat\impls\aij\seq\aij.c:2588 > > > > [1]PETSC ERROR: #5 MatCreateSubMatrix_MPIAIJ_SameRowColDist() at > > D:\PETSc\petsc-v3.18.6-pkgconfig\src\mat\impls\aij\mpi\mpiaij.c:3283 > > > > [1]PETSC ERROR: #6 MatCreateSubMatrix_MPIAIJ() at > > D:\PETSc\petsc-v3.18.6-pkgconfig\src\mat\impls\aij\mpi\mpiaij.c:3390 > > > > [1]PETSC ERROR: #7 MatCreateSubMatrix() at > > D:\PETSc\petsc-v3.18.6-pkgconfig\src\mat\interface\matrix.c:8382 > > > > [1]PETSC ERROR: #8 SNESSolve_VINEWTONRSLS() at > > D:\PETSc\petsc-v3.18.6-pkgconfig\src\snes\impls\vi\rs\virs.c:392 > > > > [1]PETSC ERROR: [5]PETSC ERROR: #1 PetscMallocAlign() at > > D:\PETSc\petsc-v3.18.6-pkgconfig\src\sys\memory\mal.c:68 > > > > [5]PETSC ERROR: #2 PetscMallocA() at > > D:\PETSc\petsc-v3.18.6-pkgconfig\src\sys\memory\mal.c:410 > > > > [5]PETSC ERROR: #3 MatSeqAIJSetPreallocation_SeqAIJ() at > > D:\PETSc\petsc-v3.18.6-pkgconfig\src\mat\impls\aij\seq\aij.c:4008 > > > > [5]PETSC ERROR: #4 MatCreateSubMatrix_SeqAIJ() at > > D:\PETSc\petsc-v3.18.6-pkgconfig\src\mat\impls\aij\seq\aij.c:2588 > > > > [5]PETSC ERROR: #5 MatCreateSubMatrix_MPIAIJ_SameRowColDist() at > > D:\PETSc\petsc-v3.18.6-pkgconfig\src\mat\impls\aij\mpi\mpiaij.c:3283 > > > > [5]PETSC ERROR: #6 MatCreateSubMatrix_MPIAIJ() at > > D:\PETSc\petsc-v3.18.6-pkgconfig\src\mat\impls\aij\mpi\mpiaij.c:3390 > > > > [5]PETSC ERROR: #7 MatCreateSubMatrix() at > > D:\PETSc\petsc-v3.18.6-pkgconfig\src\mat\interface\matrix.c:8382 > > > > [5]PETSC ERROR: #8 SNESSolve_VINEWTONRSLS() at > > D:\PETSc\petsc-v3.18.6-pkgconfig\src\snes\impls\vi\rs\virs.c:392 > > > > [5]PETSC ERROR: [4]PETSC ERROR: #1 PetscMallocAlign() at > > D:\PETSc\petsc-v3.18.6-pkgconfig\src\sys\memory\mal.c:68 > > > > [4]PETSC ERROR: #2 PetscMallocA() at > > D:\PETSc\petsc-v3.18.6-pkgconfig\src\sys\memory\mal.c:410 > > > > [4]PETSC ERROR: #3 MatSeqAIJSetPreallocation_SeqAIJ() at > > D:\PETSc\petsc-v3.18.6-pkgconfig\src\mat\impls\aij\seq\aij.c:4008 > > > > [4]PETSC ERROR: #4 MatCreateSubMatrix_SeqAIJ() at > > D:\PETSc\petsc-v3.18.6-pkgconfig\src\mat\impls\aij\seq\aij.c:2588 > > > > [4]PETSC ERROR: #5 MatCreateSubMatrix_MPIAIJ_SameRowColDist() at > > D:\PETSc\petsc-v3.18.6-pkgconfig\src\mat\impls\aij\mpi\mpiaij.c:3283 > > > > [4]PETSC ERROR: #6 MatCreateSubMatrix_MPIAIJ() at > > D:\PETSc\petsc-v3.18.6-pkgconfig\src\mat\impls\aij\mpi\mpiaij.c:3390 > > > > #9 SNESSolve() at > > D:\PETSc\petsc-v3.18.6-pkgconfig\src\snes\interface\snes.c:4692 > > > > [4]PETSC ERROR: #7 MatCreateSubMatrix() at > > D:\PETSc\petsc-v3.18.6-pkgconfig\src\mat\interface\matrix.c:8382 > > > > [4]PETSC ERROR: #8 SNESSolve_VINEWTONRSLS() at > > D:\PETSc\petsc-v3.18.6-pkgconfig\src\snes\impls\vi\rs\virs.c:392 > > > > [4]PETSC ERROR: #9 SNESSolve() at > > D:\PETSc\petsc-v3.18.6-pkgconfig\src\snes\interface\snes.c:4692 > > > > #9 SNESSolve() at > > D:\PETSc\petsc-v3.18.6-pkgconfig\src\snes\interface\snes.c:4692 > > > > > > > > From brbfernandes at utexas.edu Sat Dec 28 23:16:51 2024 From: brbfernandes at utexas.edu (Bruno Ramon Batista Fernande) Date: Sat, 28 Dec 2024 23:16:51 -0600 Subject: [petsc-users] PETSC configuration on linux with intel MKL Message-ID: Hi, I am trying to use the oneAPI MKL 2025 into my PETSC configuration on Linux. However, I get the following error as I enter the names of the mkl libraries: ********************************************************************************************* ERROR in COMMAND LINE ARGUMENT to ./configure --------------------------------------------------------------------------------------------- The option /opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_lapack95_lp64.a,/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_gf_lp64.a, should probably be /opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl-lapack95-lp64.a,/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl-gf-lp64.a, ********************************************************************************************* As you can see, the configure suggest to use libraries with hyphen in their names but the original libraries have underscores in it. Any way to bypass this issue without creating links or creating copies of files? Best regards, Bruno -------------- next part -------------- An HTML attachment was scrubbed... URL: From balay.anl at fastmail.org Sun Dec 29 08:15:38 2024 From: balay.anl at fastmail.org (Satish Balay) Date: Sun, 29 Dec 2024 08:15:38 -0600 (CST) Subject: [petsc-users] PETSC configuration on linux with intel MKL In-Reply-To: References: Message-ID: <3411c7e8-6060-2ce7-564e-5d46ac4ad972@fastmail.org> Hm - what was the configure command [with options] used here? Can you send corresponding configure.log? Satish On Sat, 28 Dec 2024, Bruno Ramon Batista Fernande wrote: > Hi, > I am trying to use the oneAPI MKL 2025 into my PETSC configuration on > Linux. However, I get the following error as I enter the names of the mkl > libraries: > ********************************************************************************************* > ERROR in COMMAND LINE ARGUMENT to ./configure > --------------------------------------------------------------------------------------------- > The option > > /opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_lapack95_lp64.a,/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_gf_lp64.a, > should probably be > > /opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl-lapack95-lp64.a,/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl-gf-lp64.a, > ********************************************************************************************* > As you can see, the configure suggest to use libraries with hyphen in their > names but the original libraries have underscores in it. Any way to bypass > this issue without creating links or creating copies of files? > Best regards, > Bruno > From brbfernandes at utexas.edu Sun Dec 29 19:24:21 2024 From: brbfernandes at utexas.edu (Bruno Ramon Batista Fernande) Date: Sun, 29 Dec 2024 19:24:21 -0600 Subject: [petsc-users] PETSC configuration on linux with intel MKL In-Reply-To: <3411c7e8-6060-2ce7-564e-5d46ac4ad972@fastmail.org> References: <3411c7e8-6060-2ce7-564e-5d46ac4ad972@fastmail.org> Message-ID: Hi Satish, Thank you for your prompt response. It seems like it doesn't reach the point of writing the configure.log for this issue. The command used is below. I am using mpich and oneAPI both installed from apt from Ubuntu. ./configure --with-cc=mpicc --with-cxx=mpicxx --with-fc=mpif90 --with-debugging=0 --download-hypre --with-blaslapack-libs=["/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_blas95_lp64.a", "/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_lapack95_lp64.a","/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_gf_lp64.a", "/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_sequential.a","/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_core.a"] --with-scalapack-libs=["/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_scalapack_lp64.a","/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_blacs_lp64.a"] --with-scalapack-include=/opt/intel/oneapi/mkl/2025.0/include Best regards, Bruno On Sun, Dec 29, 2024 at 8:15?AM Satish Balay wrote: > Hm - what was the configure command [with options] used here? Can you send > corresponding configure.log? > > Satish > > On Sat, 28 Dec 2024, Bruno Ramon Batista Fernande wrote: > > > Hi, > > I am trying to use the oneAPI MKL 2025 into my PETSC configuration on > > Linux. However, I get the following error as I enter the names of the mkl > > libraries: > > > ********************************************************************************************* > > ERROR in COMMAND LINE ARGUMENT to ./configure > > > --------------------------------------------------------------------------------------------- > > The option > > > > > /opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_lapack95_lp64.a,/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_gf_lp64.a, > > should probably be > > > > > /opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl-lapack95-lp64.a,/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl-gf-lp64.a, > > > ********************************************************************************************* > > As you can see, the configure suggest to use libraries with hyphen in > their > > names but the original libraries have underscores in it. Any way to > bypass > > this issue without creating links or creating copies of files? > > Best regards, > > Bruno > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From pierre at joliv.et Sun Dec 29 13:44:58 2024 From: pierre at joliv.et (Pierre Jolivet) Date: Sun, 29 Dec 2024 20:44:58 +0100 Subject: [petsc-users] PETSC configuration on linux with intel MKL In-Reply-To: References: <3411c7e8-6060-2ce7-564e-5d46ac4ad972@fastmail.org> Message-ID: <5D2D5D13-9B79-49F2-9F16-2919B05ADEEC@joliv.et> > On 30 Dec 2024, at 2:24?AM, Bruno Ramon Batista Fernande wrote: > > Hi Satish, > Thank you for your prompt response. > It seems like it doesn't reach the point of writing the configure.log for this issue. > The command used is below. I am using mpich and oneAPI both installed from apt from Ubuntu. > > ./configure --with-cc=mpicc --with-cxx=mpicxx --with-fc=mpif90 --with-debugging=0 --download-hypre --with-blaslapack-libs=["/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_blas95_lp64.a", "/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_lapack95_lp64.a","/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_gf_lp64.a", "/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_sequential.a","/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_core.a"] --with-scalapack-libs=["/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_scalapack_lp64.a","/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_blacs_lp64.a"] --with-scalapack-include=/opt/intel/oneapi/mkl/2025.0/include It?s not -libs=, it?s -lib=, and there is no need for quotation marks. Try again, and if it still does not work, please send configure.log, there is no reason for it not being generated. Thanks, Pierre > Best regards, > Bruno > > On Sun, Dec 29, 2024 at 8:15?AM Satish Balay > wrote: >> Hm - what was the configure command [with options] used here? Can you send corresponding configure.log? >> >> Satish >> >> On Sat, 28 Dec 2024, Bruno Ramon Batista Fernande wrote: >> >> > Hi, >> > I am trying to use the oneAPI MKL 2025 into my PETSC configuration on >> > Linux. However, I get the following error as I enter the names of the mkl >> > libraries: >> > ********************************************************************************************* >> > ERROR in COMMAND LINE ARGUMENT to ./configure >> > --------------------------------------------------------------------------------------------- >> > The option >> > >> > /opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_lapack95_lp64.a,/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_gf_lp64.a, >> > should probably be >> > >> > /opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl-lapack95-lp64.a,/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl-gf-lp64.a, >> > ********************************************************************************************* >> > As you can see, the configure suggest to use libraries with hyphen in their >> > names but the original libraries have underscores in it. Any way to bypass >> > this issue without creating links or creating copies of files? >> > Best regards, >> > Bruno >> > >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From brbfernandes at utexas.edu Sun Dec 29 14:09:05 2024 From: brbfernandes at utexas.edu (Bruno Ramon Batista Fernande) Date: Sun, 29 Dec 2024 14:09:05 -0600 Subject: [petsc-users] PETSC configuration on linux with intel MKL In-Reply-To: <5D2D5D13-9B79-49F2-9F16-2919B05ADEEC@joliv.et> References: <3411c7e8-6060-2ce7-564e-5d46ac4ad972@fastmail.org> <5D2D5D13-9B79-49F2-9F16-2919B05ADEEC@joliv.et> Message-ID: Hi Pierre, I tried it and got the same error message. Still, no configure.log was generated. It seems like it doesn't even start the configuration. Best regards, Bruno On Sun, Dec 29, 2024 at 1:45?PM Pierre Jolivet wrote: > > > On 30 Dec 2024, at 2:24?AM, Bruno Ramon Batista Fernande < > brbfernandes at utexas.edu> wrote: > > Hi Satish, > Thank you for your prompt response. > It seems like it doesn't reach the point of writing the configure.log for > this issue. > The command used is below. I am using mpich and oneAPI both installed from > apt from Ubuntu. > > ./configure --with-cc=mpicc --with-cxx=mpicxx --with-fc=mpif90 > --with-debugging=0 --download-hypre > --with-blaslapack-libs=["/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_blas95_lp64.a", > "/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_lapack95_lp64.a","/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_gf_lp64.a", > "/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_sequential.a","/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_core.a"] > --with-scalapack-libs=["/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_scalapack_lp64.a","/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_blacs_lp64.a"] > --with-scalapack-include=/opt/intel/oneapi/mkl/2025.0/include > > > It?s not -libs=, it?s -lib=, and there is no need for quotation marks. > Try again, and if it still does not work, please send configure.log, there > is no reason for it not being generated. > > Thanks, > Pierre > > Best regards, > Bruno > > On Sun, Dec 29, 2024 at 8:15?AM Satish Balay > wrote: > >> Hm - what was the configure command [with options] used here? Can you >> send corresponding configure.log? >> >> Satish >> >> On Sat, 28 Dec 2024, Bruno Ramon Batista Fernande wrote: >> >> > Hi, >> > I am trying to use the oneAPI MKL 2025 into my PETSC configuration on >> > Linux. However, I get the following error as I enter the names of the >> mkl >> > libraries: >> > >> ********************************************************************************************* >> > ERROR in COMMAND LINE ARGUMENT to ./configure >> > >> --------------------------------------------------------------------------------------------- >> > The option >> > >> > >> /opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_lapack95_lp64.a,/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_gf_lp64.a, >> > should probably be >> > >> > >> /opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl-lapack95-lp64.a,/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl-gf-lp64.a, >> > >> ********************************************************************************************* >> > As you can see, the configure suggest to use libraries with hyphen in >> their >> > names but the original libraries have underscores in it. Any way to >> bypass >> > this issue without creating links or creating copies of files? >> > Best regards, >> > Bruno >> > >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From pierre at joliv.et Sun Dec 29 14:15:25 2024 From: pierre at joliv.et (Pierre Jolivet) Date: Sun, 29 Dec 2024 21:15:25 +0100 Subject: [petsc-users] PETSC configuration on linux with intel MKL In-Reply-To: References: <3411c7e8-6060-2ce7-564e-5d46ac4ad972@fastmail.org> <5D2D5D13-9B79-49F2-9F16-2919B05ADEEC@joliv.et> Message-ID: I just copied you configure line, you also have a whitespace in the [ ], best to remove that: libmkl_blas95_lp64.a", "/opt/intel/ -> libmkl_blas95_lp64.a,/opt/intel/ and so on. Thanks, Pierre > On 29 Dec 2024, at 9:09?PM, Bruno Ramon Batista Fernande wrote: > > Hi Pierre, > I tried it and got the same error message. Still, no configure.log was generated. It seems like it doesn't even start the configuration. > Best regards, > Bruno > > On Sun, Dec 29, 2024 at 1:45?PM Pierre Jolivet > wrote: >> >> >>> On 30 Dec 2024, at 2:24?AM, Bruno Ramon Batista Fernande > wrote: >>> >>> Hi Satish, >>> Thank you for your prompt response. >>> It seems like it doesn't reach the point of writing the configure.log for this issue. >>> The command used is below. I am using mpich and oneAPI both installed from apt from Ubuntu. >>> >>> ./configure --with-cc=mpicc --with-cxx=mpicxx --with-fc=mpif90 --with-debugging=0 --download-hypre --with-blaslapack-libs=["/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_blas95_lp64.a", "/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_lapack95_lp64.a","/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_gf_lp64.a", "/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_sequential.a","/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_core.a"] --with-scalapack-libs=["/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_scalapack_lp64.a","/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_blacs_lp64.a"] --with-scalapack-include=/opt/intel/oneapi/mkl/2025.0/include >> >> It?s not -libs=, it?s -lib=, and there is no need for quotation marks. >> Try again, and if it still does not work, please send configure.log, there is no reason for it not being generated. >> >> Thanks, >> Pierre >> >>> Best regards, >>> Bruno >>> >>> On Sun, Dec 29, 2024 at 8:15?AM Satish Balay > wrote: >>>> Hm - what was the configure command [with options] used here? Can you send corresponding configure.log? >>>> >>>> Satish >>>> >>>> On Sat, 28 Dec 2024, Bruno Ramon Batista Fernande wrote: >>>> >>>> > Hi, >>>> > I am trying to use the oneAPI MKL 2025 into my PETSC configuration on >>>> > Linux. However, I get the following error as I enter the names of the mkl >>>> > libraries: >>>> > ********************************************************************************************* >>>> > ERROR in COMMAND LINE ARGUMENT to ./configure >>>> > --------------------------------------------------------------------------------------------- >>>> > The option >>>> > >>>> > /opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_lapack95_lp64.a,/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_gf_lp64.a, >>>> > should probably be >>>> > >>>> > /opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl-lapack95-lp64.a,/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl-gf-lp64.a, >>>> > ********************************************************************************************* >>>> > As you can see, the configure suggest to use libraries with hyphen in their >>>> > names but the original libraries have underscores in it. Any way to bypass >>>> > this issue without creating links or creating copies of files? >>>> > Best regards, >>>> > Bruno >>>> > >>>> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From balay.anl at fastmail.org Sun Dec 29 16:06:54 2024 From: balay.anl at fastmail.org (Satish Balay) Date: Sun, 29 Dec 2024 16:06:54 -0600 (CST) Subject: [petsc-users] PETSC configuration on linux with intel MKL In-Reply-To: References: <3411c7e8-6060-2ce7-564e-5d46ac4ad972@fastmail.org> <5D2D5D13-9B79-49F2-9F16-2919B05ADEEC@joliv.et> Message-ID: This syntax is useful for python - for ex.: config/examples*.py Best to use shell friendly notation - when not using python scripts. i.e.: --with-blaslapack-lib="/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_blas95_lp64.a /opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_lapack95_lp64.a /opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_gf_lp64.a /opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_sequential.a /opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_core.a" or --with-blaslapack-lib="-L/opt/intel/oneapi/mkl/2025.0/lib/intel64/ -lmkl_lapack95_lp64 -lmklblas95_lp64 -lmkl_gf_lp64 -lmkl_sequential -lmkl_lapack95_lp64" Also "--with-blaslapack-dir=/opt/intel/oneapi/mkl/2025.0/lib/intel64" should automatically find and give something like: BlasLapack: Intel MKL Version: 20220000 Includes: -I/nfs/gce/software/custom/linux-ubuntu22.04-x86_64/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/intel-oneapi-mkl-2022.0.2-lzrncoj/mkl/2022.0.2/include Libraries: -Wl,-rpath,/nfs/gce/software/custom/linux-ubuntu22.04-x86_64/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/intel-oneapi-mkl-2022.0.2-lzrncoj/mkl/2022.0.2/lib/intel64 -L/nfs/gce/software/custom/linux-ubuntu22.04-x86_64/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/intel-oneapi-mkl-2022.0.2-lzrncoj/mkl/2022.0.2/lib/intel64 -lmkl_intel_lp64 -lmkl_core -lmkl_intel_thread -liomp5 -ldl -lpthread Satish On Sun, 29 Dec 2024, Pierre Jolivet wrote: > I just copied you configure line, you also have a whitespace in the [ ], best to remove that: libmkl_blas95_lp64.a", "/opt/intel/ -> libmkl_blas95_lp64.a,/opt/intel/ and so on. > > Thanks, > Pierre > > > On 29 Dec 2024, at 9:09?PM, Bruno Ramon Batista Fernande wrote: > > > > Hi Pierre, > > I tried it and got the same error message. Still, no configure.log was generated. It seems like it doesn't even start the configuration. > > Best regards, > > Bruno > > > > On Sun, Dec 29, 2024 at 1:45?PM Pierre Jolivet > wrote: > >> > >> > >>> On 30 Dec 2024, at 2:24?AM, Bruno Ramon Batista Fernande > wrote: > >>> > >>> Hi Satish, > >>> Thank you for your prompt response. > >>> It seems like it doesn't reach the point of writing the configure.log for this issue. > >>> The command used is below. I am using mpich and oneAPI both installed from apt from Ubuntu. > >>> > >>> ./configure --with-cc=mpicc --with-cxx=mpicxx --with-fc=mpif90 --with-debugging=0 --download-hypre --with-blaslapack-libs=["/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_blas95_lp64.a", "/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_lapack95_lp64.a","/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_gf_lp64.a", "/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_sequential.a","/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_core.a"] --with-scalapack-libs=["/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_scalapack_lp64.a","/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_blacs_lp64.a"] --with-scalapack-include=/opt/intel/oneapi/mkl/2025.0/include > >> > >> It?s not -libs=, it?s -lib=, and there is no need for quotation marks. > >> Try again, and if it still does not work, please send configure.log, there is no reason for it not being generated. > >> > >> Thanks, > >> Pierre > >> > >>> Best regards, > >>> Bruno > >>> > >>> On Sun, Dec 29, 2024 at 8:15?AM Satish Balay > wrote: > >>>> Hm - what was the configure command [with options] used here? Can you send corresponding configure.log? > >>>> > >>>> Satish > >>>> > >>>> On Sat, 28 Dec 2024, Bruno Ramon Batista Fernande wrote: > >>>> > >>>> > Hi, > >>>> > I am trying to use the oneAPI MKL 2025 into my PETSC configuration on > >>>> > Linux. However, I get the following error as I enter the names of the mkl > >>>> > libraries: > >>>> > ********************************************************************************************* > >>>> > ERROR in COMMAND LINE ARGUMENT to ./configure > >>>> > --------------------------------------------------------------------------------------------- > >>>> > The option > >>>> > > >>>> > /opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_lapack95_lp64.a,/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_gf_lp64.a, > >>>> > should probably be > >>>> > > >>>> > /opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl-lapack95-lp64.a,/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl-gf-lp64.a, > >>>> > ********************************************************************************************* > >>>> > As you can see, the configure suggest to use libraries with hyphen in their > >>>> > names but the original libraries have underscores in it. Any way to bypass > >>>> > this issue without creating links or creating copies of files? > >>>> > Best regards, > >>>> > Bruno > >>>> > > >>>> > >> > > From brbfernandes at utexas.edu Sun Dec 29 16:50:46 2024 From: brbfernandes at utexas.edu (Bruno Ramon Batista Fernande) Date: Sun, 29 Dec 2024 16:50:46 -0600 Subject: [petsc-users] PETSC configuration on linux with intel MKL In-Reply-To: References: <3411c7e8-6060-2ce7-564e-5d46ac4ad972@fastmail.org> <5D2D5D13-9B79-49F2-9F16-2919B05ADEEC@joliv.et> Message-ID: Hi Satish and Pierre, It worked with the first shell friendly syntax recommended by Satish. I got that other syntax from a Windows configuration we have. Thank you so much for the help. Best regards, Bruno On Sun, Dec 29, 2024 at 4:07?PM Satish Balay wrote: > This syntax is useful for python - for ex.: config/examples*.py > > Best to use shell friendly notation - when not using python scripts. i.e.: > > --with-blaslapack-lib="/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_blas95_lp64.a > /opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_lapack95_lp64.a > /opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_gf_lp64.a > /opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_sequential.a > /opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_core.a" > > or > > --with-blaslapack-lib="-L/opt/intel/oneapi/mkl/2025.0/lib/intel64/ > -lmkl_lapack95_lp64 -lmklblas95_lp64 -lmkl_gf_lp64 -lmkl_sequential > -lmkl_lapack95_lp64" > > Also "--with-blaslapack-dir=/opt/intel/oneapi/mkl/2025.0/lib/intel64" > should automatically find and give something like: > > BlasLapack: > Intel MKL Version: 20220000 > Includes: > -I/nfs/gce/software/custom/linux-ubuntu22.04-x86_64/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/intel-oneapi-mkl-2022.0.2-lzrncoj/mkl/2022.0.2/include > Libraries: > -Wl,-rpath,/nfs/gce/software/custom/linux-ubuntu22.04-x86_64/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/intel-oneapi-mkl-2022.0.2-lzrncoj/mkl/2022.0.2/lib/intel64 > -L/nfs/gce/software/custom/linux-ubuntu22.04-x86_64/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/intel-oneapi-mkl-2022.0.2-lzrncoj/mkl/2022.0.2/lib/intel64 > -lmkl_intel_lp64 -lmkl_core -lmkl_intel_thread -liomp5 -ldl -lpthread > > Satish > > On Sun, 29 Dec 2024, Pierre Jolivet wrote: > > > I just copied you configure line, you also have a whitespace in the [ ], > best to remove that: libmkl_blas95_lp64.a", "/opt/intel/ -> > libmkl_blas95_lp64.a,/opt/intel/ and so on. > > > > Thanks, > > Pierre > > > > > On 29 Dec 2024, at 9:09?PM, Bruno Ramon Batista Fernande < > brbfernandes at utexas.edu> wrote: > > > > > > Hi Pierre, > > > I tried it and got the same error message. Still, no configure.log was > generated. It seems like it doesn't even start the configuration. > > > Best regards, > > > Bruno > > > > > > On Sun, Dec 29, 2024 at 1:45?PM Pierre Jolivet > wrote: > > >> > > >> > > >>> On 30 Dec 2024, at 2:24?AM, Bruno Ramon Batista Fernande < > brbfernandes at utexas.edu > wrote: > > >>> > > >>> Hi Satish, > > >>> Thank you for your prompt response. > > >>> It seems like it doesn't reach the point of writing the > configure.log for this issue. > > >>> The command used is below. I am using mpich and oneAPI both > installed from apt from Ubuntu. > > >>> > > >>> ./configure --with-cc=mpicc --with-cxx=mpicxx --with-fc=mpif90 > --with-debugging=0 --download-hypre > --with-blaslapack-libs=["/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_blas95_lp64.a", > "/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_lapack95_lp64.a","/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_gf_lp64.a", > "/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_sequential.a","/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_core.a"] > --with-scalapack-libs=["/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_scalapack_lp64.a","/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_blacs_lp64.a"] > --with-scalapack-include=/opt/intel/oneapi/mkl/2025.0/include > > >> > > >> It?s not -libs=, it?s -lib=, and there is no need for quotation marks. > > >> Try again, and if it still does not work, please send configure.log, > there is no reason for it not being generated. > > >> > > >> Thanks, > > >> Pierre > > >> > > >>> Best regards, > > >>> Bruno > > >>> > > >>> On Sun, Dec 29, 2024 at 8:15?AM Satish Balay > wrote: > > >>>> Hm - what was the configure command [with options] used here? Can > you send corresponding configure.log? > > >>>> > > >>>> Satish > > >>>> > > >>>> On Sat, 28 Dec 2024, Bruno Ramon Batista Fernande wrote: > > >>>> > > >>>> > Hi, > > >>>> > I am trying to use the oneAPI MKL 2025 into my PETSC > configuration on > > >>>> > Linux. However, I get the following error as I enter the names of > the mkl > > >>>> > libraries: > > >>>> > > ********************************************************************************************* > > >>>> > ERROR in COMMAND LINE ARGUMENT to > ./configure > > >>>> > > --------------------------------------------------------------------------------------------- > > >>>> > The option > > >>>> > > > >>>> > > /opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_lapack95_lp64.a,/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_gf_lp64.a, > > >>>> > should probably be > > >>>> > > > >>>> > > /opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl-lapack95-lp64.a,/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl-gf-lp64.a, > > >>>> > > ********************************************************************************************* > > >>>> > As you can see, the configure suggest to use libraries with > hyphen in their > > >>>> > names but the original libraries have underscores in it. Any way > to bypass > > >>>> > this issue without creating links or creating copies of files? > > >>>> > Best regards, > > >>>> > Bruno > > >>>> > > > >>>> > > >> > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From balay.anl at fastmail.org Tue Dec 31 11:06:55 2024 From: balay.anl at fastmail.org (Satish Balay) Date: Tue, 31 Dec 2024 11:06:55 -0600 (CST) Subject: [petsc-users] PETSC configuration on linux with intel MKL In-Reply-To: References: <3411c7e8-6060-2ce7-564e-5d46ac4ad972@fastmail.org> <5D2D5D13-9B79-49F2-9F16-2919B05ADEEC@joliv.et> Message-ID: Glad its working! Thanks for the update. Satish On Sun, 29 Dec 2024, Bruno Ramon Batista Fernande wrote: > Hi Satish and Pierre, > It worked with the first shell friendly syntax recommended by Satish. > I got that other syntax from a Windows configuration we have. > Thank you so much for the help. > Best regards, > Bruno > > On Sun, Dec 29, 2024 at 4:07?PM Satish Balay wrote: > > > This syntax is useful for python - for ex.: config/examples*.py > > > > Best to use shell friendly notation - when not using python scripts. i.e.: > > > > --with-blaslapack-lib="/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_blas95_lp64.a > > /opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_lapack95_lp64.a > > /opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_gf_lp64.a > > /opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_sequential.a > > /opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_core.a" > > > > or > > > > --with-blaslapack-lib="-L/opt/intel/oneapi/mkl/2025.0/lib/intel64/ > > -lmkl_lapack95_lp64 -lmklblas95_lp64 -lmkl_gf_lp64 -lmkl_sequential > > -lmkl_lapack95_lp64" > > > > Also "--with-blaslapack-dir=/opt/intel/oneapi/mkl/2025.0/lib/intel64" > > should automatically find and give something like: > > > > BlasLapack: > > Intel MKL Version: 20220000 > > Includes: > > -I/nfs/gce/software/custom/linux-ubuntu22.04-x86_64/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/intel-oneapi-mkl-2022.0.2-lzrncoj/mkl/2022.0.2/include > > Libraries: > > -Wl,-rpath,/nfs/gce/software/custom/linux-ubuntu22.04-x86_64/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/intel-oneapi-mkl-2022.0.2-lzrncoj/mkl/2022.0.2/lib/intel64 > > -L/nfs/gce/software/custom/linux-ubuntu22.04-x86_64/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.2.0/intel-oneapi-mkl-2022.0.2-lzrncoj/mkl/2022.0.2/lib/intel64 > > -lmkl_intel_lp64 -lmkl_core -lmkl_intel_thread -liomp5 -ldl -lpthread > > > > Satish > > > > On Sun, 29 Dec 2024, Pierre Jolivet wrote: > > > > > I just copied you configure line, you also have a whitespace in the [ ], > > best to remove that: libmkl_blas95_lp64.a", "/opt/intel/ -> > > libmkl_blas95_lp64.a,/opt/intel/ and so on. > > > > > > Thanks, > > > Pierre > > > > > > > On 29 Dec 2024, at 9:09?PM, Bruno Ramon Batista Fernande < > > brbfernandes at utexas.edu> wrote: > > > > > > > > Hi Pierre, > > > > I tried it and got the same error message. Still, no configure.log was > > generated. It seems like it doesn't even start the configuration. > > > > Best regards, > > > > Bruno > > > > > > > > On Sun, Dec 29, 2024 at 1:45?PM Pierre Jolivet > > wrote: > > > >> > > > >> > > > >>> On 30 Dec 2024, at 2:24?AM, Bruno Ramon Batista Fernande < > > brbfernandes at utexas.edu > wrote: > > > >>> > > > >>> Hi Satish, > > > >>> Thank you for your prompt response. > > > >>> It seems like it doesn't reach the point of writing the > > configure.log for this issue. > > > >>> The command used is below. I am using mpich and oneAPI both > > installed from apt from Ubuntu. > > > >>> > > > >>> ./configure --with-cc=mpicc --with-cxx=mpicxx --with-fc=mpif90 > > --with-debugging=0 --download-hypre > > --with-blaslapack-libs=["/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_blas95_lp64.a", > > "/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_lapack95_lp64.a","/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_gf_lp64.a", > > "/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_sequential.a","/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_core.a"] > > --with-scalapack-libs=["/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_scalapack_lp64.a","/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_blacs_lp64.a"] > > --with-scalapack-include=/opt/intel/oneapi/mkl/2025.0/include > > > >> > > > >> It?s not -libs=, it?s -lib=, and there is no need for quotation marks. > > > >> Try again, and if it still does not work, please send configure.log, > > there is no reason for it not being generated. > > > >> > > > >> Thanks, > > > >> Pierre > > > >> > > > >>> Best regards, > > > >>> Bruno > > > >>> > > > >>> On Sun, Dec 29, 2024 at 8:15?AM Satish Balay > > wrote: > > > >>>> Hm - what was the configure command [with options] used here? Can > > you send corresponding configure.log? > > > >>>> > > > >>>> Satish > > > >>>> > > > >>>> On Sat, 28 Dec 2024, Bruno Ramon Batista Fernande wrote: > > > >>>> > > > >>>> > Hi, > > > >>>> > I am trying to use the oneAPI MKL 2025 into my PETSC > > configuration on > > > >>>> > Linux. However, I get the following error as I enter the names of > > the mkl > > > >>>> > libraries: > > > >>>> > > > ********************************************************************************************* > > > >>>> > ERROR in COMMAND LINE ARGUMENT to > > ./configure > > > >>>> > > > --------------------------------------------------------------------------------------------- > > > >>>> > The option > > > >>>> > > > > >>>> > > > /opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_lapack95_lp64.a,/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl_gf_lp64.a, > > > >>>> > should probably be > > > >>>> > > > > >>>> > > > /opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl-lapack95-lp64.a,/opt/intel/oneapi/mkl/2025.0/lib/intel64/libmkl-gf-lp64.a, > > > >>>> > > > ********************************************************************************************* > > > >>>> > As you can see, the configure suggest to use libraries with > > hyphen in their > > > >>>> > names but the original libraries have underscores in it. Any way > > to bypass > > > >>>> > this issue without creating links or creating copies of files? > > > >>>> > Best regards, > > > >>>> > Bruno > > > >>>> > > > > >>>> > > > >> > > > > > > > > >