From hzhang at mcs.anl.gov Thu Sep 1 09:35:49 2011 From: hzhang at mcs.anl.gov (Hong Zhang) Date: Thu, 1 Sep 2011 09:35:49 -0500 Subject: [petsc-users] The default nonzero pattern for SuperLU_dist. In-Reply-To: <36F5FE78-2395-4E99-95A7-25C5477EDDFC@lbl.gov> References: <2EF00818-6652-4E41-86FC-475AEF952AEB@lbl.gov> <02A1F7C5-433E-448C-9D2F-3FE26B84B9DA@mcs.anl.gov> <36F5FE78-2395-4E99-95A7-25C5477EDDFC@lbl.gov> Message-ID: Rebecca: > Thanks for your kind reply. So if I set up the following: > > ? ? ? ?ierr = DAGetMatrix(DMMGGetDA(dmmg), MATAIJ, &jacobian);CHKERRQ(ierr); > ? ? ? ?KSP ? ? ksp; > ? ? ? ?ierr = SNESGetKSP(DMMGGetSNES(dmmg),&ksp);CHKERRQ(ierr); > ? ? ? ?ierr = KSPSetOperators(ksp,jacobian,jacobian,SAME_NONZERO_PATTERN);CHKERRQ(ierr); > > ? ? ? ?ierr = DMMGSolve(dmmg);CHKERRQ(ierr); > > This will specify the nonzero pattern as the same? Yes. >How could I check if I have set this up right? This is a user input. If you know the non-zero pattern will not change, then this flag tells petsc library to skip symbolic factorization and reuse previous matrix factor structure. I would suggest use DIFFERENT_NONZERO_PATTERN first. After the solver works well and you know the matrix structure should not change, then use SAME_NONZERO_PATTERN for optimization. Hong > On Aug 31, 2011, at 2:42 PM, Barry Smith wrote: > >> >> On Aug 31, 2011, at 4:40 PM, Xuefei (Rebecca) Yuan wrote: >> >>> Hello all, >>> >>> Is the default nonzero pattern for SuperLU_dist is SAME_NONZERO_PATTERN? >> >> ? There is no default nonzero pattern. You always must set it when you call KSPSetOperators() or in your SNES FormFunction. The PETSc SUPERLU_dist interface allows either pattern and should work correctly. >> >> >> ? ?Barry >> >>> >>> Thanks! >>> >>> Best, >>> >>> Rebecca >>> >> > > From Andrew.Barker at Colorado.EDU Thu Sep 1 09:36:23 2011 From: Andrew.Barker at Colorado.EDU (Andrew T Barker) Date: Thu, 1 Sep 2011 08:36:23 -0600 (MDT) Subject: [petsc-users] Matrix is missing diagonal entry In-Reply-To: References: Message-ID: <20110901083623.AKJ70453@batman.int.colorado.edu> This has happened to me if I forget to call MatAssemblyBegin/End. The entry will still print out in the MatView(), but it won't compute. Might not be your problem, but worth a shot. ---- Original message ---- >Date: Wed, 31 Aug 2011 18:15:14 -0600 >From: petsc-users-bounces at mcs.anl.gov (on behalf of Rongliang Chen ) >Subject: [petsc-users] Matrix is missing diagonal entry >To: PETSc Users mail list > > Hello, > > I got the error message "Matrix is missing diagonal > entry 23548!" when I tried to run my code. I print > out the 23548th row and find that the 23548th > element of this row is one and the others are zero. > Can anyone tell me what's maybe the problem and how > to fix this? Thanks. > > Best Regards, > Rongliang > > /bin/bash: SHELL: readonly variable > /bin/bash: PATH: readonly variable > [255]PETSC ERROR: --------------------- Error > Message ------------------------------------ > [255]PETSC ERROR: Object is in wrong state! > [255]PETSC ERROR: Matrix is missing diagonal entry > 23548! > [255]PETSC ERROR: > ------------------------------------------------------------------------ > [255]PETSC ERROR: Petsc Release Version 3.1.0, Patch > 8, Thu Mar 17 13:37:48 CDT 2011 > [255]PETSC ERROR: See docs/changes/index.html for > recent updates. > [255]PETSC ERROR: See docs/faq.html for hints about > trouble shooting. > [255]PETSC ERROR: See docs/index.html for manual > pages. > [255]PETSC ERROR: > ------------------------------------------------------------------------ > [255]PETSC ERROR: > /home/rchen/soft/fixedmesh/Cannula/./joab on a > bgl-ibm-g named R02M1N0 by Unknown Wed Aug 31 > 17:36:32 2011 > [255]PETSC ERROR: Libraries linked from > /home/rchen/soft/petsc-3.1-p8-nodebug/bgl-ibm-goto-O3_440d/lib > > [255]PETSC ERROR: > ------------------------------------------------------------------------ > [255]PETSC ERROR: MatILUFactorSymbolic_SeqAIJ_ilu0() > line 1627 in src/mat/impls/aij/seq/aijfact.c > [255]PETSC ERROR: MatILUFactorSymbolic_SeqAIJ() line > 1731 in src/mat/impls/aij/seq/aijfact.c > [255]PETSC ERROR: MatILUFactorSymbolic() line 5464 > in src/mat/interface/matrix.c > [255]PETSC ERROR: PCSetUp_ILU() line 204 in > src/ksp/pc/impls/factor/ilu/ilu.c > [255]PETSC ERROR: PCSetUp() line 795 in > src/ksp/pc/interface/precon.c > [255]PETSC ERROR: KSPSetUp() line 237 in > src/ksp/ksp/interface/itfunc.c > [255]PETSC ERROR: PCSetUpOnBlocks_ASM() line 335 in > src/ksp/pc/impls/asm/asm.c > [255]PETSC ERROR: PCSetUpOnBlocks() line 828 in > src/ksp/pc/interface/precon.c > [255]PETSC ERROR: KSPSetUpOnBlocks() line 159 in > src/ksp/ksp/interface/itfunc.c > [255]PETSC ERROR: KSPSolve() line 354 in > src/ksp/ksp/interface/itfunc.c > [255]PETSC ERROR: SNES_KSPSolve() line 2944 in > src/snes/interface/snes.c > [255]PETSC ERROR: SNESSolve() line 2255 in > src/snes/interface/snes.c > [255]PETSC ERROR: TimeStep() line 1462 in > /home/rlchen/rlchen/soft/fixedmesh/code_changed/joab.c > [255]PETSC ERROR: SolveSteadyState() line 1362 in > /home/rlchen/rlchen/soft/fixedmesh/code_changed/joab.c > [255]PETSC ERROR: ComputefixedBoundary() line 505 in > /home/rlchen/rlchen/soft/fixedmesh/code_changed/joab.c > [255]PETSC ERROR: JoabCtxSolve() line 165 in > /home/rlchen/rlchen/soft/fixedmesh/code_changed/joab.c > [255]PETSC ERROR: main() line 107 in > /home/rlchen/rlchen/soft/fixedmesh/code_changed/joab.c > ..................... From likunt at andrew.cmu.edu Thu Sep 1 10:24:27 2011 From: likunt at andrew.cmu.edu (Likun Tan) Date: Thu, 1 Sep 2011 11:24:27 -0400 Subject: [petsc-users] about MatTranspose In-Reply-To: References: <3033ca5d22302bbf5272f571c73a092f.squirrel@webmail.andrew.cmu.edu> <45C4BE1F-928F-4E34-8AAB-7D26AA34995B@mcs.anl.gov> <92ee3133d6e2801fc1e3f727947cf3e6.squirrel@webmail.andrew.cmu.edu> <4bf0fcffeb9f2c0d9b6206327df6e8c1.squirrel@webmail.andrew.cmu.edu> <62da0198e4d4ef47b647666e720c2dc9.squirrel@webmail.andrew.cmu.edu> <0c9c2be4b200e3bfddcb930ae8e02b58.squirrel@webmail.andrew.cmu.edu> Message-ID: <0b35b32b3d16b7d6a02fd185b7a163ad.squirrel@webmail.andrew.cmu.edu> Dear all, I used MatCreate() to create a non-square matrix B, and set values to it in parallel. Now i want to get the tranpose of it to solve Ax=B', where A is SeqDense. I define a SeqDense C=B' and use MatTranspose() to get it, then apply MatMatSolve(fact,C,x), but i get the following error: [0]PETSC ERROR: --------------------- Error Message ------------------------------------ [0]PETSC ERROR: Nonconforming object sizes! [0]PETSC ERROR: Mat A,Mat B: local dim 27 14! [0]PETSC ERROR: ------------------------------------------------------------------------ Thank you in advance, Likun From jedbrown at mcs.anl.gov Thu Sep 1 10:29:44 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Thu, 1 Sep 2011 10:29:44 -0500 Subject: [petsc-users] about MatTranspose In-Reply-To: <0b35b32b3d16b7d6a02fd185b7a163ad.squirrel@webmail.andrew.cmu.edu> References: <3033ca5d22302bbf5272f571c73a092f.squirrel@webmail.andrew.cmu.edu> <45C4BE1F-928F-4E34-8AAB-7D26AA34995B@mcs.anl.gov> <92ee3133d6e2801fc1e3f727947cf3e6.squirrel@webmail.andrew.cmu.edu> <4bf0fcffeb9f2c0d9b6206327df6e8c1.squirrel@webmail.andrew.cmu.edu> <62da0198e4d4ef47b647666e720c2dc9.squirrel@webmail.andrew.cmu.edu> <0c9c2be4b200e3bfddcb930ae8e02b58.squirrel@webmail.andrew.cmu.edu> <0b35b32b3d16b7d6a02fd185b7a163ad.squirrel@webmail.andrew.cmu.edu> Message-ID: On Thu, Sep 1, 2011 at 10:24, Likun Tan wrote: > I used MatCreate() to create a non-square matrix B, and set values to it > in parallel. Now i want to get the tranpose of it to solve Ax=B', where A > is SeqDense. > > I define a SeqDense C=B' and use MatTranspose() to get it, then apply > MatMatSolve(fact,C,x), but i get the following error: > > [0]PETSC ERROR: --------------------- Error Message > ------------------------------------ > [0]PETSC ERROR: Nonconforming object sizes! > [0]PETSC ERROR: Mat A,Mat B: local dim 27 14! > When you send error messages, please send the whole stack trace. It looks like C=B' is not being partitioned correctly. Did you take my earlier suggestions about how to partition this problem? -------------- next part -------------- An HTML attachment was scrubbed... URL: From likunt at andrew.cmu.edu Thu Sep 1 10:46:36 2011 From: likunt at andrew.cmu.edu (Likun Tan) Date: Thu, 1 Sep 2011 11:46:36 -0400 Subject: [petsc-users] about MatTranspose In-Reply-To: References: <3033ca5d22302bbf5272f571c73a092f.squirrel@webmail.andrew.cmu.edu> <45C4BE1F-928F-4E34-8AAB-7D26AA34995B@mcs.anl.gov> <92ee3133d6e2801fc1e3f727947cf3e6.squirrel@webmail.andrew.cmu.edu> <4bf0fcffeb9f2c0d9b6206327df6e8c1.squirrel@webmail.andrew.cmu.edu> <62da0198e4d4ef47b647666e720c2dc9.squirrel@webmail.andrew.cmu.edu> <0c9c2be4b200e3bfddcb930ae8e02b58.squirrel@webmail.andrew.cmu.edu> <0b35b32b3d16b7d6a02fd185b7a163ad.squirrel@webmail.andrew.cmu.edu> Message-ID: Thank you for your reply~ I tried to use PetscSplitOwnership(PETSC_COMM_WORLD,&n,&N); MatCreateSeqDense(PETSC_COMM_SELF,M,n,PETSC_NULL,&B); But it seems not working. This command is make B store in N/n processors with n columns in each processor? I try to calculate the N/n parts in parallel, but when i apply MatGetOwnershipRange(), i found the matrix will be automatically partitioned by row. That's why i go back to use MatCreate() to create a 200*27 matrix and try to get the tranpose of it.I used MatCreateSeqDense(PETSC_COMM_SELF, M, N, PETSC_NULL, &C) to create C, there is no partition of C, i guess this is the reason of the error. How should i fix it? Thanks, Likun On Thu, September 1, 2011 11:29 am, Jed Brown wrote: > On Thu, Sep 1, 2011 at 10:24, Likun Tan wrote: > > >> I used MatCreate() to create a non-square matrix B, and set values to >> it in parallel. Now i want to get the tranpose of it to solve Ax=B', >> where A is SeqDense. >> >> I define a SeqDense C=B' and use MatTranspose() to get it, then apply >> MatMatSolve(fact,C,x), but i get the following error: >> >> >> [0]PETSC ERROR: --------------------- Error Message >> ------------------------------------ >> [0]PETSC ERROR: Nonconforming object sizes! >> [0]PETSC ERROR: Mat A,Mat B: local dim 27 14! >> >> > > When you send error messages, please send the whole stack trace. > > > It looks like C=B' is not being partitioned correctly. > Did you take my earlier suggestions about how to partition this problem? > > From jedbrown at mcs.anl.gov Thu Sep 1 10:59:46 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Thu, 1 Sep 2011 10:59:46 -0500 Subject: [petsc-users] about MatTranspose In-Reply-To: References: <3033ca5d22302bbf5272f571c73a092f.squirrel@webmail.andrew.cmu.edu> <45C4BE1F-928F-4E34-8AAB-7D26AA34995B@mcs.anl.gov> <92ee3133d6e2801fc1e3f727947cf3e6.squirrel@webmail.andrew.cmu.edu> <4bf0fcffeb9f2c0d9b6206327df6e8c1.squirrel@webmail.andrew.cmu.edu> <62da0198e4d4ef47b647666e720c2dc9.squirrel@webmail.andrew.cmu.edu> <0c9c2be4b200e3bfddcb930ae8e02b58.squirrel@webmail.andrew.cmu.edu> <0b35b32b3d16b7d6a02fd185b7a163ad.squirrel@webmail.andrew.cmu.edu> Message-ID: On Thu, Sep 1, 2011 at 10:46, Likun Tan wrote: > I tried to use > PetscSplitOwnership(PETSC_COMM_WORLD,&n,&N); > MatCreateSeqDense(PETSC_COMM_SELF,M,n,PETSC_NULL,&B); > > But it seems not working. This command is make B store in N/n processors > with n columns in each processor? > B (in the code) is a different matrix on each process. You should fill the columns so that each process has part of B (the logical thing you are working with). > I try to calculate the N/n parts in > parallel, but when i apply MatGetOwnershipRange(), i found the matrix will > be automatically partitioned by row. > MatGetOwnershipRange() just gives you the rows from 0 to 26. You need to sum the column partition that was computed earlier. The most explicit way to do that is PetscInt cstart; MPI_Scan(&n,&cstart,1,MPIU_INT,MPI_SUM,PETSC_COMM_WORLD); cstart -= n; Now each process should compute columns of the logical B in the range [cstart,cstart+n). The B object in your code will indexed [0,n) on each process, but these columns correspond to [cstart,cstart+n). > That's why i go back to use MatCreate() to create a 200*27 matrix and try > to get the tranpose of it.I used > MatCreateSeqDense(PETSC_COMM_SELF, M, N, PETSC_NULL, &C) > to create C, there is no partition of C, i guess this is the reason of the > error. > Don't go down this route, it doesn't do what you want. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rongliang.chan at gmail.com Thu Sep 1 11:38:53 2011 From: rongliang.chan at gmail.com (Rongliang Chen) Date: Thu, 1 Sep 2011 10:38:53 -0600 Subject: [petsc-users] Matrix is missing diagonal entry Message-ID: Hi Jed and Andrew, Thanks for your reply. It's a little strange that this problem appears for some meshes and number of processors. When I run the code with 512 processors and the same mesh there is no such problem. And for some meshes, the code works well with any number of processors. Best, Rongliang > Message: 3 > Date: Wed, 31 Aug 2011 21:34:00 -0500 > From: Jed Brown > Subject: Re: [petsc-users] Matrix is missing diagonal entry > To: PETSc users list > Message-ID: > > > Content-Type: text/plain; charset="utf-8" > > Maybe a mistake implementing boundary conditions? > > ------------------------------ > > > ------------------------------ > > Message: 5 > Date: Thu, 1 Sep 2011 08:36:23 -0600 (MDT) > From: Andrew T Barker > Subject: Re: [petsc-users] Matrix is missing diagonal entry > To: "PETSc users list" > Message-ID: <20110901083623.AKJ70453 at batman.int.colorado.edu> > Content-Type: text/plain; charset=us-ascii > > This has happened to me if I forget to call MatAssemblyBegin/End. The > entry will still print out in the MatView(), but it won't compute. > > Might not be your problem, but worth a shot. > > ---- Original message ---- > >Date: Wed, 31 Aug 2011 18:15:14 -0600 > >From: petsc-users-bounces at mcs.anl.gov (on behalf of Rongliang Chen < > rongliang.chan at gmail.com>) > >Subject: [petsc-users] Matrix is missing diagonal entry > >To: PETSc Users mail list > > > > Hello, > > > > I got the error message "Matrix is missing diagonal > > entry 23548!" when I tried to run my code. I print > > out the 23548th row and find that the 23548th > > element of this row is one and the others are zero. > > Can anyone tell me what's maybe the problem and how > > to fix this? Thanks. > > > > Best Regards, > > Rongliang > > > > /bin/bash: SHELL: readonly variable > > /bin/bash: PATH: readonly variable > > [255]PETSC ERROR: --------------------- Error > > Message ------------------------------------ > > [255]PETSC ERROR: Object is in wrong state! > > [255]PETSC ERROR: Matrix is missing diagonal entry > > 23548! > > [255]PETSC ERROR: > > > ------------------------------------------------------------------------ > > [255]PETSC ERROR: Petsc Release Version 3.1.0, Patch > > 8, Thu Mar 17 13:37:48 CDT 2011 > > [255]PETSC ERROR: See docs/changes/index.html for > > recent updates. > > [255]PETSC ERROR: See docs/faq.html for hints about > > trouble shooting. > > [255]PETSC ERROR: See docs/index.html for manual > > pages. > > [255]PETSC ERROR: > > > ------------------------------------------------------------------------ > > [255]PETSC ERROR: > > /home/rchen/soft/fixedmesh/Cannula/./joab on a > > bgl-ibm-g named R02M1N0 by Unknown Wed Aug 31 > > 17:36:32 2011 > > [255]PETSC ERROR: Libraries linked from > > /home/rchen/soft/petsc-3.1-p8-nodebug/bgl-ibm-goto-O3_440d/lib > > > > [255]PETSC ERROR: > > > ------------------------------------------------------------------------ > > [255]PETSC ERROR: MatILUFactorSymbolic_SeqAIJ_ilu0() > > line 1627 in src/mat/impls/aij/seq/aijfact.c > > [255]PETSC ERROR: MatILUFactorSymbolic_SeqAIJ() line > > 1731 in src/mat/impls/aij/seq/aijfact.c > > [255]PETSC ERROR: MatILUFactorSymbolic() line 5464 > > in src/mat/interface/matrix.c > > [255]PETSC ERROR: PCSetUp_ILU() line 204 in > > src/ksp/pc/impls/factor/ilu/ilu.c > > [255]PETSC ERROR: PCSetUp() line 795 in > > src/ksp/pc/interface/precon.c > > [255]PETSC ERROR: KSPSetUp() line 237 in > > src/ksp/ksp/interface/itfunc.c > > [255]PETSC ERROR: PCSetUpOnBlocks_ASM() line 335 in > > src/ksp/pc/impls/asm/asm.c > > [255]PETSC ERROR: PCSetUpOnBlocks() line 828 in > > src/ksp/pc/interface/precon.c > > [255]PETSC ERROR: KSPSetUpOnBlocks() line 159 in > > src/ksp/ksp/interface/itfunc.c > > [255]PETSC ERROR: KSPSolve() line 354 in > > src/ksp/ksp/interface/itfunc.c > > [255]PETSC ERROR: SNES_KSPSolve() line 2944 in > > src/snes/interface/snes.c > > [255]PETSC ERROR: SNESSolve() line 2255 in > > src/snes/interface/snes.c > > [255]PETSC ERROR: TimeStep() line 1462 in > > /home/rlchen/rlchen/soft/fixedmesh/code_changed/joab.c > > [255]PETSC ERROR: SolveSteadyState() line 1362 in > > /home/rlchen/rlchen/soft/fixedmesh/code_changed/joab.c > > [255]PETSC ERROR: ComputefixedBoundary() line 505 in > > /home/rlchen/rlchen/soft/fixedmesh/code_changed/joab.c > > [255]PETSC ERROR: JoabCtxSolve() line 165 in > > /home/rlchen/rlchen/soft/fixedmesh/code_changed/joab.c > > [255]PETSC ERROR: main() line 107 in > > /home/rlchen/rlchen/soft/fixedmesh/code_changed/joab.c > > ..................... > > > ------------------------------ > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From balay at mcs.anl.gov Thu Sep 1 11:41:02 2011 From: balay at mcs.anl.gov (Satish Balay) Date: Thu, 1 Sep 2011 11:41:02 -0500 (CDT) Subject: [petsc-users] Matrix is missing diagonal entry In-Reply-To: References: Message-ID: Could be random memory corruption. perhaps you can valgrind the code on a small problem [on smaller set of nodes] - with --download-mpich build of petsc. Satish On Thu, 1 Sep 2011, Rongliang Chen wrote: > Hi Jed and Andrew, > > Thanks for your reply. > > It's a little strange that this problem appears for some meshes and number > of processors. When I run the code with 512 processors and the same mesh > there is no such problem. And for some meshes, the code works well with any > number of processors. > > Best, > Rongliang > > > > > Message: 3 > > Date: Wed, 31 Aug 2011 21:34:00 -0500 > > From: Jed Brown > > Subject: Re: [petsc-users] Matrix is missing diagonal entry > > To: PETSc users list > > Message-ID: > > > > > > Content-Type: text/plain; charset="utf-8" > > > > Maybe a mistake implementing boundary conditions? > > > > ------------------------------ > > > > > > ------------------------------ > > > > Message: 5 > > Date: Thu, 1 Sep 2011 08:36:23 -0600 (MDT) > > From: Andrew T Barker > > Subject: Re: [petsc-users] Matrix is missing diagonal entry > > To: "PETSc users list" > > Message-ID: <20110901083623.AKJ70453 at batman.int.colorado.edu> > > Content-Type: text/plain; charset=us-ascii > > > > This has happened to me if I forget to call MatAssemblyBegin/End. The > > entry will still print out in the MatView(), but it won't compute. > > > > Might not be your problem, but worth a shot. > > > > ---- Original message ---- > > >Date: Wed, 31 Aug 2011 18:15:14 -0600 > > >From: petsc-users-bounces at mcs.anl.gov (on behalf of Rongliang Chen < > > rongliang.chan at gmail.com>) > > >Subject: [petsc-users] Matrix is missing diagonal entry > > >To: PETSc Users mail list > > > > > > Hello, > > > > > > I got the error message "Matrix is missing diagonal > > > entry 23548!" when I tried to run my code. I print > > > out the 23548th row and find that the 23548th > > > element of this row is one and the others are zero. > > > Can anyone tell me what's maybe the problem and how > > > to fix this? Thanks. > > > > > > Best Regards, > > > Rongliang > > > > > > /bin/bash: SHELL: readonly variable > > > /bin/bash: PATH: readonly variable > > > [255]PETSC ERROR: --------------------- Error > > > Message ------------------------------------ > > > [255]PETSC ERROR: Object is in wrong state! > > > [255]PETSC ERROR: Matrix is missing diagonal entry > > > 23548! > > > [255]PETSC ERROR: > > > > > ------------------------------------------------------------------------ > > > [255]PETSC ERROR: Petsc Release Version 3.1.0, Patch > > > 8, Thu Mar 17 13:37:48 CDT 2011 > > > [255]PETSC ERROR: See docs/changes/index.html for > > > recent updates. > > > [255]PETSC ERROR: See docs/faq.html for hints about > > > trouble shooting. > > > [255]PETSC ERROR: See docs/index.html for manual > > > pages. > > > [255]PETSC ERROR: > > > > > ------------------------------------------------------------------------ > > > [255]PETSC ERROR: > > > /home/rchen/soft/fixedmesh/Cannula/./joab on a > > > bgl-ibm-g named R02M1N0 by Unknown Wed Aug 31 > > > 17:36:32 2011 > > > [255]PETSC ERROR: Libraries linked from > > > /home/rchen/soft/petsc-3.1-p8-nodebug/bgl-ibm-goto-O3_440d/lib > > > > > > [255]PETSC ERROR: > > > > > ------------------------------------------------------------------------ > > > [255]PETSC ERROR: MatILUFactorSymbolic_SeqAIJ_ilu0() > > > line 1627 in src/mat/impls/aij/seq/aijfact.c > > > [255]PETSC ERROR: MatILUFactorSymbolic_SeqAIJ() line > > > 1731 in src/mat/impls/aij/seq/aijfact.c > > > [255]PETSC ERROR: MatILUFactorSymbolic() line 5464 > > > in src/mat/interface/matrix.c > > > [255]PETSC ERROR: PCSetUp_ILU() line 204 in > > > src/ksp/pc/impls/factor/ilu/ilu.c > > > [255]PETSC ERROR: PCSetUp() line 795 in > > > src/ksp/pc/interface/precon.c > > > [255]PETSC ERROR: KSPSetUp() line 237 in > > > src/ksp/ksp/interface/itfunc.c > > > [255]PETSC ERROR: PCSetUpOnBlocks_ASM() line 335 in > > > src/ksp/pc/impls/asm/asm.c > > > [255]PETSC ERROR: PCSetUpOnBlocks() line 828 in > > > src/ksp/pc/interface/precon.c > > > [255]PETSC ERROR: KSPSetUpOnBlocks() line 159 in > > > src/ksp/ksp/interface/itfunc.c > > > [255]PETSC ERROR: KSPSolve() line 354 in > > > src/ksp/ksp/interface/itfunc.c > > > [255]PETSC ERROR: SNES_KSPSolve() line 2944 in > > > src/snes/interface/snes.c > > > [255]PETSC ERROR: SNESSolve() line 2255 in > > > src/snes/interface/snes.c > > > [255]PETSC ERROR: TimeStep() line 1462 in > > > /home/rlchen/rlchen/soft/fixedmesh/code_changed/joab.c > > > [255]PETSC ERROR: SolveSteadyState() line 1362 in > > > /home/rlchen/rlchen/soft/fixedmesh/code_changed/joab.c > > > [255]PETSC ERROR: ComputefixedBoundary() line 505 in > > > /home/rlchen/rlchen/soft/fixedmesh/code_changed/joab.c > > > [255]PETSC ERROR: JoabCtxSolve() line 165 in > > > /home/rlchen/rlchen/soft/fixedmesh/code_changed/joab.c > > > [255]PETSC ERROR: main() line 107 in > > > /home/rlchen/rlchen/soft/fixedmesh/code_changed/joab.c > > > ..................... > > > > > > ------------------------------ > > > > > From xyuan at lbl.gov Thu Sep 1 11:53:49 2011 From: xyuan at lbl.gov (Xuefei (Rebecca) Yuan) Date: Thu, 1 Sep 2011 09:53:49 -0700 Subject: [petsc-users] The default nonzero pattern for SuperLU_dist. In-Reply-To: References: <2EF00818-6652-4E41-86FC-475AEF952AEB@lbl.gov> <02A1F7C5-433E-448C-9D2F-3FE26B84B9DA@mcs.anl.gov> <36F5FE78-2395-4E99-95A7-25C5477EDDFC@lbl.gov> Message-ID: <0875405B-97E5-4038-9575-8DB90AF923DD@lbl.gov> Dear Hong, Yes, I am using this SAME_NONZERO_PATTERN to optimize the code,hehe! Thanks very much! Best, Rebecca On Sep 1, 2011, at 7:35 AM, Hong Zhang wrote: > Rebecca: > >> Thanks for your kind reply. So if I set up the following: >> >> ierr = DAGetMatrix(DMMGGetDA(dmmg), MATAIJ, &jacobian);CHKERRQ(ierr); >> KSP ksp; >> ierr = SNESGetKSP(DMMGGetSNES(dmmg),&ksp);CHKERRQ(ierr); >> ierr = KSPSetOperators(ksp,jacobian,jacobian,SAME_NONZERO_PATTERN);CHKERRQ(ierr); >> >> ierr = DMMGSolve(dmmg);CHKERRQ(ierr); >> >> This will specify the nonzero pattern as the same? > Yes. > >> How could I check if I have set this up right? > > This is a user input. If you know the non-zero pattern will not change, then > this flag tells petsc library to skip symbolic factorization and reuse previous > matrix factor structure. I would suggest use DIFFERENT_NONZERO_PATTERN > first. After the solver works well and you know the matrix structure > should not change, > then use SAME_NONZERO_PATTERN for optimization. > > Hong > > >> On Aug 31, 2011, at 2:42 PM, Barry Smith wrote: >> >>> >>> On Aug 31, 2011, at 4:40 PM, Xuefei (Rebecca) Yuan wrote: >>> >>>> Hello all, >>>> >>>> Is the default nonzero pattern for SuperLU_dist is SAME_NONZERO_PATTERN? >>> >>> There is no default nonzero pattern. You always must set it when you call KSPSetOperators() or in your SNES FormFunction. The PETSc SUPERLU_dist interface allows either pattern and should work correctly. >>> >>> >>> Barry >>> >>>> >>>> Thanks! >>>> >>>> Best, >>>> >>>> Rebecca >>>> >>> >> >> From likunt at andrew.cmu.edu Thu Sep 1 12:45:45 2011 From: likunt at andrew.cmu.edu (Likun Tan) Date: Thu, 1 Sep 2011 13:45:45 -0400 Subject: [petsc-users] about MatTranspose Message-ID: Thank you for your patience. I still have some confusions. When computing B before MPI_Scan(), could i compute the values in parallel? After using MPI_Scan(), does that mean the columns of B will be gathered in one processor? Here are main steps i took based on your suggestion, PetscSplitOwnership(PETSC_COMM_WORLD, &n, &N); MatCreateSeqDense(PETSC_COMM_SELF, M, n, PETSC_NULL, &B); MatCreateSeqDense(PETSC_COMM_SELF, M, M, PETSC_NULL, &A); MatCreateSeqDense(PETSC_COMM_SELF, M, N, PETSC_NULL, &x); for(j=0; j On Thu, Sep 1, 2011 at 10:46, Likun Tan wrote: > > >> I tried to use >> PetscSplitOwnership(PETSC_COMM_WORLD,&n,&N); >> MatCreateSeqDense(PETSC_COMM_SELF,M,n,PETSC_NULL,&B); >> >> >> But it seems not working. This command is make B store in N/n >> processors with n columns in each processor? >> > > B (in the code) is a different matrix on each process. You should fill > the columns so that each process has part of B (the logical thing you are > working with). > > >> I try to calculate the N/n parts in >> parallel, but when i apply MatGetOwnershipRange(), i found the matrix >> will be automatically partitioned by row. >> > > MatGetOwnershipRange() just gives you the rows from 0 to 26. You need to > sum the column partition that was computed earlier. The most explicit way > to do that is > > PetscInt cstart; > MPI_Scan(&n,&cstart,1,MPIU_INT,MPI_SUM,PETSC_COMM_WORLD); > cstart -= n; > > Now each process should compute columns of the logical B in the range > [cstart,cstart+n). The B object in your code will indexed [0,n) on each > process, but these columns correspond to [cstart,cstart+n). > > >> That's why i go back to use MatCreate() to create a 200*27 matrix and >> try to get the tranpose of it.I used MatCreateSeqDense(PETSC_COMM_SELF, >> M, N, PETSC_NULL, &C) >> to create C, there is no partition of C, i guess this is the reason of >> the error. >> > > Don't go down this route, it doesn't do what you want. > > From jedbrown at mcs.anl.gov Thu Sep 1 12:54:27 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Thu, 1 Sep 2011 12:54:27 -0500 Subject: [petsc-users] about MatTranspose In-Reply-To: References: Message-ID: On Thu, Sep 1, 2011 at 12:45, Likun Tan wrote: > I still have some confusions. When computing B before MPI_Scan(), could i > compute the values in parallel? After using MPI_Scan(), does that mean the > columns of B will be gathered in one processor? > No, the scan is just computing the start column for a given process. > > Here are main steps i took based on your suggestion, > > PetscSplitOwnership(PETSC_COMM_WORLD, &n, &N); > Add these two lines here: MPI_Scan(&n, &cstart, 1, MPIU_INT, MPI_SUM, PETSC_COMM_WORLD); cstart -= n; > MatCreateSeqDense(PETSC_COMM_SELF, M, n, PETSC_NULL, &B); > MatCreateSeqDense(PETSC_COMM_SELF, M, M, PETSC_NULL, &A); > Good > MatCreateSeqDense(PETSC_COMM_SELF, M, N, PETSC_NULL, &x); > Replace with MatDuplicate(B,MAT_DO_NOT_COPY_VALUES,&x); > for(j=0; j change the loop bounds here: for(j=0; j { > for(i=0; i { > Good, now compute value as the value that goes in (i,cstart+j). MatSetValues(B, 1, &i, 1, &j, &value, INSERT_VALUES); > } > } > MatAssemblyBegin(...); > MatAssemblyEnd(...) > This part is correct. > MPI_Scan(&n, &cstart, 1, MPIU_INT, MPI_SUM, PETSC_COMM_WORLD); > cstart -= n; > We did this already. > > MatConvert(...); > MatCholeskyFactor(...); > MatMatSolve(...); > Yes. You can gather the matrix x onto all processes if you need the whole result everywhere, but for performance reasons if you scale further, you should avoid it if possible. -------------- next part -------------- An HTML attachment was scrubbed... URL: From gjost at tacc.utexas.edu Thu Sep 1 13:03:56 2011 From: gjost at tacc.utexas.edu (Gabriele Jost) Date: Thu, 1 Sep 2011 18:03:56 +0000 Subject: [petsc-users] Mumps Error In-Reply-To: <8D782424-55CE-4C7A-9E6C-38A782D770B3@berkeley.edu> References: , <8D782424-55CE-4C7A-9E6C-38A782D770B3@berkeley.edu> Message-ID: Hello! I am getting an error with a different number, but the message is the same: 59]PETSC ERROR: ------------------------------------------------------------------------ [59]PETSC ERROR: MatFactorNumeric_MUMPS() line 517 in src/mat/impls/aij/mpi/mumps/mumps.c [10]PETSC ERROR: --------------------- Error Message ------------------------------------ [10]PETSC ERROR: Error in external library! [10]PETSC ERROR: Error reported by MUMPS in numerical factorization phase: INFO(1)=-1, INFO(2)=39 I tried rerunning with the flags below: -log_summary -ksp__monitor -ksp_type preonly -ksp_view -ksp_max_it 5000 -pc_type lu -pc_factor_mat_solver_package mumps -mat_mumps_icntl_14 100 but still get the same error. Did you find out anything new in the meantime? Thanks and greetings, Gabriele ________________________________________ From: petsc-users-bounces at mcs.anl.gov [petsc-users-bounces at mcs.anl.gov] on behalf of Max Rudolph [rudolph at berkeley.edu] Sent: Wednesday, August 31, 2011 3:34 PM To: petsc-users at mcs.anl.gov Subject: Re: [petsc-users] Mumps Error I'm not sure if you figured out a solution yet, but I think that you might want to run with -mat_mumps_icntl_14 100 Max >> Dear Users, >> >> I'm having trouble figuring out why the MUMPS solver is failing on a specific range of one of my parameters. When using the PETSc direct solver on a single processor I have no issues. There error is: >> >> [0]PETSC ERROR: --------------------- Error Message ------------------------------------ >> [0]PETSC ERROR: Error in external library! >> [0]PETSC ERROR: Error reported by MUMPS in numerical factorization phase: INFO(1)=-9, INFO(2)=13 >> ! >> [0]PETSC ERROR: ------------------------------------------------------------------------ >> [0]PETSC ERROR: Petsc Release Version 3.1.0, Patch 8, Thu Mar 17 13:37:48 CDT 2011 >> [0]PETSC ERROR: See docs/changes/index.html for recent updates. >> [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. >> [0]PETSC ERROR: See docs/index.html for manual pages. >> [0]PETSC ERROR: ------------------------------------------------------------------------ >> [0]PETSC ERROR: ./cntor on a complex-c named hpc-1-14.local by abyrd Wed Aug 31 10:53:42 2011 >> [0]PETSC ERROR: Libraries linked from /panfs/storage.local/scs/home/abyrd/petsc-3.1-p8/complex-cpp-mumps/lib >> [0]PETSC ERROR: Configure run at Mon Jul 11 15:28:42 2011 >> [0]PETSC ERROR: Configure options PETSC_ARCH=complex-cpp-mumps --with-cc=mpicc --with-fc=mpif90 --with-blas-lapack-dir=/usr/lib64 --with-shared --with-clanguage=c++ --with-scalar-type=complex --download-mumps=1 --download-blacs=1 --download-scalapack=1 --download-parmetis=1 --with-cxx=mpicxx >> [0]PETSC ERROR: ------------------------------------------------------------------------ >> [0]PETSC ERROR: MatFactorNumeric_MUMPS() line 517 in src/mat/impls/aij/mpi/mumps/mumps.c >> [0]PETSC ERROR: MatLUFactorNumeric() line 2587 in src/mat/interface/matrix.c >> [0]PETSC ERROR: PCSetUp_LU() line 158 in src/ksp/pc/impls/factor/lu/lu.c >> [0]PETSC ERROR: PCSetUp() line 795 in src/ksp/pc/interface/precon.c >> [0]PETSC ERROR: KSPSetUp() line 237 in src/ksp/ksp/interface/itfunc.c >> [0]PETSC ERROR: InvertHamiltonian() line 102 in WDinvert.h >> >> >> I suspect it has something to do with the preconditioning or setup of the matrix I am trying to invert. The matrix becomes singular at energy = 0 eV, and is nearly singular for values close to that, but the code is failing on energies relatively far from that point. The affected energy interval is [-0.03095, 0.03095]. >> >> Is anyone able to point in the right direction to figure out what I'm not setting up properly? >> >> Respectfully, >> Adam Byrd >> > > > > > _______________________________________________ > petsc-users mailing list > petsc-users at mcs.anl.gov > https://lists.mcs.anl.gov/mailman/listinfo/petsc-users From hzhang at mcs.anl.gov Thu Sep 1 13:26:37 2011 From: hzhang at mcs.anl.gov (Hong Zhang) Date: Thu, 1 Sep 2011 13:26:37 -0500 Subject: [petsc-users] Mumps Error In-Reply-To: References: <8D782424-55CE-4C7A-9E6C-38A782D770B3@berkeley.edu> Message-ID: Gabriele: This time, you get INFO(1)=-1, "An error occurred on processor INFO(2)=39", according to mumps manual. Please ask mumps developer about INFO(1)=-1. My guess is that your matrix might be numerical singular, causing numerical factorization fail. Hong > > I am getting an error with a different number, but the message is the same: > > 59]PETSC ERROR: ------------------------------------------------------------------------ > [59]PETSC ERROR: MatFactorNumeric_MUMPS() line 517 in src/mat/impls/aij/mpi/mumps/mumps.c > [10]PETSC ERROR: --------------------- Error Message ------------------------------------ > [10]PETSC ERROR: Error in external library! > [10]PETSC ERROR: Error reported by MUMPS in numerical factorization phase: INFO(1)=-1, INFO(2)=39 > > I tried rerunning with the flags below: > ?-log_summary -ksp__monitor -ksp_type preonly -ksp_view -ksp_max_it 5000 -pc_type lu -pc_factor_mat_solver_package mumps -mat_mumps_icntl_14 100 > but still get the same error. Did you find out anything new in the meantime? > > Thanks and greetings, > Gabriele > > > ________________________________________ > From: petsc-users-bounces at mcs.anl.gov [petsc-users-bounces at mcs.anl.gov] on behalf of Max Rudolph [rudolph at berkeley.edu] > Sent: Wednesday, August 31, 2011 3:34 PM > To: petsc-users at mcs.anl.gov > Subject: Re: [petsc-users] Mumps Error > > I'm not sure if you figured out a solution yet, but I think that you might want to run with > > -mat_mumps_icntl_14 100 > > Max > > >>> Dear Users, >>> >>> I'm having trouble figuring out why the MUMPS solver is failing on a specific range of one of my parameters. When using the PETSc direct solver on a single processor I have no issues. There error is: >>> >>> [0]PETSC ERROR: --------------------- Error Message ------------------------------------ >>> [0]PETSC ERROR: Error in external library! >>> [0]PETSC ERROR: Error reported by MUMPS in numerical factorization phase: INFO(1)=-9, INFO(2)=13 >>> ! >>> [0]PETSC ERROR: ------------------------------------------------------------------------ >>> [0]PETSC ERROR: Petsc Release Version 3.1.0, Patch 8, Thu Mar 17 13:37:48 CDT 2011 >>> [0]PETSC ERROR: See docs/changes/index.html for recent updates. >>> [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. >>> [0]PETSC ERROR: See docs/index.html for manual pages. >>> [0]PETSC ERROR: ------------------------------------------------------------------------ >>> [0]PETSC ERROR: ./cntor on a complex-c named hpc-1-14.local by abyrd Wed Aug 31 10:53:42 2011 >>> [0]PETSC ERROR: Libraries linked from /panfs/storage.local/scs/home/abyrd/petsc-3.1-p8/complex-cpp-mumps/lib >>> [0]PETSC ERROR: Configure run at Mon Jul 11 15:28:42 2011 >>> [0]PETSC ERROR: Configure options PETSC_ARCH=complex-cpp-mumps --with-cc=mpicc --with-fc=mpif90 --with-blas-lapack-dir=/usr/lib64 --with-shared --with-clanguage=c++ --with-scalar-type=complex --download-mumps=1 --download-blacs=1 --download-scalapack=1 --download-parmetis=1 --with-cxx=mpicxx >>> [0]PETSC ERROR: ------------------------------------------------------------------------ >>> [0]PETSC ERROR: MatFactorNumeric_MUMPS() line 517 in src/mat/impls/aij/mpi/mumps/mumps.c >>> [0]PETSC ERROR: MatLUFactorNumeric() line 2587 in src/mat/interface/matrix.c >>> [0]PETSC ERROR: PCSetUp_LU() line 158 in src/ksp/pc/impls/factor/lu/lu.c >>> [0]PETSC ERROR: PCSetUp() line 795 in src/ksp/pc/interface/precon.c >>> [0]PETSC ERROR: KSPSetUp() line 237 in src/ksp/ksp/interface/itfunc.c >>> [0]PETSC ERROR: InvertHamiltonian() line 102 in WDinvert.h >>> >>> >>> I suspect it has something to do with the preconditioning or setup of the matrix I am trying to invert. The matrix becomes singular at energy = 0 eV, and is nearly singular for values close to that, but the code is failing on energies relatively far from that point. The affected energy interval is [-0.03095, 0.03095]. >>> >>> Is anyone able to point in the right direction to figure out what I'm not setting up properly? >>> >>> Respectfully, >>> Adam Byrd >>> >> >> >> >> >> _______________________________________________ >> petsc-users mailing list >> petsc-users at mcs.anl.gov >> https://lists.mcs.anl.gov/mailman/listinfo/petsc-users > > From likunt at andrew.cmu.edu Thu Sep 1 13:33:29 2011 From: likunt at andrew.cmu.edu (Likun Tan) Date: Thu, 1 Sep 2011 14:33:29 -0400 Subject: [petsc-users] about MatTranspose Message-ID: Thank you very much. With the command: for(j=0; j On Thu, Sep 1, 2011 at 12:45, Likun Tan wrote: > > >> I still have some confusions. When computing B before MPI_Scan(), could >> i compute the values in parallel? After using MPI_Scan(), does that mean >> the columns of B will be gathered in one processor? >> > > No, the scan is just computing the start column for a given process. > > > >> >> Here are main steps i took based on your suggestion, >> >> >> PetscSplitOwnership(PETSC_COMM_WORLD, &n, &N); >> >> > > Add these two lines here: > > > MPI_Scan(&n, &cstart, 1, MPIU_INT, MPI_SUM, PETSC_COMM_WORLD); > cstart -= n; > > >> MatCreateSeqDense(PETSC_COMM_SELF, M, n, PETSC_NULL, &B); >> MatCreateSeqDense(PETSC_COMM_SELF, M, M, PETSC_NULL, &A); >> >> > > Good > > > >> MatCreateSeqDense(PETSC_COMM_SELF, M, N, PETSC_NULL, &x); >> >> > > Replace with > > > MatDuplicate(B,MAT_DO_NOT_COPY_VALUES,&x); > > > >> for(j=0; j> > > change the loop bounds here: > > for(j=0; j > >> { >> for(i=0; i> >> > > Good, now compute value as the value that goes in (i,cstart+j). > > > MatSetValues(B, 1, &i, 1, &j, &value, INSERT_VALUES); > >> } >> } >> MatAssemblyBegin(...); >> MatAssemblyEnd(...) >> >> > > This part is correct. > > > >> MPI_Scan(&n, &cstart, 1, MPIU_INT, MPI_SUM, PETSC_COMM_WORLD); >> cstart -= n; >> > > We did this already. > > > >> >> MatConvert(...); >> MatCholeskyFactor(...); >> MatMatSolve(...); >> >> > > Yes. > > > > You can gather the matrix x onto all processes if you need the whole > result everywhere, but for performance reasons if you scale further, you > should avoid it if possible. > From zhenglun.wei at gmail.com Thu Sep 1 16:47:21 2011 From: zhenglun.wei at gmail.com (Alan Wei) Date: Thu, 1 Sep 2011 16:47:21 -0500 Subject: [petsc-users] Neumann BC in src/ksp/ksp/example/tutorial/ex29.c Message-ID: Dear All, I hope you're having a nice day. I'm still working on this Poisson solver example, and I still have a little confusion on the boundary condition. I wonder if I have a 17*17 grid. (index i = 0~16 and j = 0~16, solving for variable 'u') 1) If a Dirichlet BC applied on the left boundary, i.e. u= 1, is that all values with u[i= 0][] = 1 or with u[i= -1][] = 1 (where, u[i=-1][] are ghost grids left next to grid of u[i= 0][]). 2) If a Neumann BC applied on the right boundary, i.e. du/dx = 0, is that (u[i=16][]-u[i=15][])/dx= or (u[i=17][]-u[i=16][])/dx = 0 (where u[i =17][] are ghost grids right next to grid of u[i = 16][]). Thank you so much. best, Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Fri Sep 2 07:14:24 2011 From: knepley at gmail.com (Matthew Knepley) Date: Fri, 2 Sep 2011 12:14:24 +0000 Subject: [petsc-users] Neumann BC in src/ksp/ksp/example/tutorial/ex29.c In-Reply-To: References: Message-ID: On Thu, Sep 1, 2011 at 9:47 PM, Alan Wei wrote: > Dear All, > I hope you're having a nice day. > I'm still working on this Poisson solver example, and I still have a > little confusion on the boundary condition. I wonder if I have a 17*17 grid. > (index i = 0~16 and j = 0~16, solving for variable 'u') > 1) If a Dirichlet BC applied on the left boundary, i.e. u= 1, is that all > values with u[i= 0][] = 1 or with u[i= -1][] = 1 (where, u[i=-1][] are > ghost grids left next to grid of u[i= 0][]). > Unless you define the boundary as PERIODIC or GHOSTED, the -1 does not exist. I set Dirichlet conditions at 0 and M-1. > 2) If a Neumann BC applied on the right boundary, i.e. du/dx = 0, is that > (u[i=16][]-u[i=15][])/dx= or (u[i=17][]-u[i=16][])/dx = 0 (where u[i =17][] > are ghost grids right next to grid of u[i = 16][]). > Thank you so much. > Same answer. Matt > best, > Alan > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From zhenglun.wei at gmail.com Fri Sep 2 09:20:56 2011 From: zhenglun.wei at gmail.com (Alan Wei) Date: Fri, 2 Sep 2011 09:20:56 -0500 Subject: [petsc-users] Neumann BC in src/ksp/ksp/example/tutorial/ex29.c In-Reply-To: References: Message-ID: Thanks, Matt. I thought there is a ghosted grid around Dirichlet and Neumann BC's. ^_^ best, Alan On Fri, Sep 2, 2011 at 7:14 AM, Matthew Knepley wrote: > On Thu, Sep 1, 2011 at 9:47 PM, Alan Wei wrote: > >> Dear All, >> I hope you're having a nice day. >> I'm still working on this Poisson solver example, and I still have a >> little confusion on the boundary condition. I wonder if I have a 17*17 grid. >> (index i = 0~16 and j = 0~16, solving for variable 'u') >> 1) If a Dirichlet BC applied on the left boundary, i.e. u= 1, is that all >> values with u[i= 0][] = 1 or with u[i= -1][] = 1 (where, u[i=-1][] are >> ghost grids left next to grid of u[i= 0][]). >> > > Unless you define the boundary as PERIODIC or GHOSTED, the -1 does not > exist. I set Dirichlet conditions at 0 and M-1. > > >> 2) If a Neumann BC applied on the right boundary, i.e. du/dx = 0, is that >> (u[i=16][]-u[i=15][])/dx= or (u[i=17][]-u[i=16][])/dx = 0 (where u[i =17][] >> are ghost grids right next to grid of u[i = 16][]). >> Thank you so much. >> > > Same answer. > > Matt > > >> best, >> Alan >> >> > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jedbrown at mcs.anl.gov Fri Sep 2 10:20:02 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Fri, 2 Sep 2011 10:20:02 -0500 Subject: [petsc-users] Neumann BC in src/ksp/ksp/example/tutorial/ex29.c In-Reply-To: References: Message-ID: On Fri, Sep 2, 2011 at 09:20, Alan Wei wrote: > I thought there is a ghosted grid around Dirichlet and Neumann BC's. Only if you ask for it with DMDA_BOUNDARY_GHOSTED (not available in petsc-3.1). This example does not. -------------- next part -------------- An HTML attachment was scrubbed... URL: From zhenglun.wei at gmail.com Fri Sep 2 11:21:23 2011 From: zhenglun.wei at gmail.com (Alan Wei) Date: Fri, 2 Sep 2011 11:21:23 -0500 Subject: [petsc-users] Neumann BC in src/ksp/ksp/example/tutorial/ex29.c In-Reply-To: References: Message-ID: Thanks, Matt. Let me make sure, so Dirichlet BC on the left boundary --> u[i= 0][] = constant Neumann BC on the right boundary --> u[i=16][]-u[i=15][])/dx = constant is that correct? best, Alan On Fri, Sep 2, 2011 at 10:20 AM, Jed Brown wrote: > On Fri, Sep 2, 2011 at 09:20, Alan Wei wrote: > >> I thought there is a ghosted grid around Dirichlet and Neumann BC's. > > > Only if you ask for it with DMDA_BOUNDARY_GHOSTED (not available in > petsc-3.1). This example does not. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Fri Sep 2 11:24:24 2011 From: knepley at gmail.com (Matthew Knepley) Date: Fri, 2 Sep 2011 16:24:24 +0000 Subject: [petsc-users] Neumann BC in src/ksp/ksp/example/tutorial/ex29.c In-Reply-To: References: Message-ID: On Fri, Sep 2, 2011 at 4:21 PM, Alan Wei wrote: > Thanks, Matt. Let me make sure, so > Dirichlet BC on the left boundary --> u[i= 0][] = constant > Yes, and usually f[i=0][] = constant - u[i=0][] > Neumann BC on the right boundary --> u[i=16][]-u[i=15][])/dx = constant > is that correct? > Yes. Matt > best, > Alan > > > On Fri, Sep 2, 2011 at 10:20 AM, Jed Brown wrote: > >> On Fri, Sep 2, 2011 at 09:20, Alan Wei wrote: >> >>> I thought there is a ghosted grid around Dirichlet and Neumann BC's. >> >> >> Only if you ask for it with DMDA_BOUNDARY_GHOSTED (not available in >> petsc-3.1). This example does not. >> > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From jedbrown at mcs.anl.gov Fri Sep 2 18:33:38 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Fri, 2 Sep 2011 18:33:38 -0500 Subject: [petsc-users] about MatTranspose In-Reply-To: References: Message-ID: Remember that the code has a different B on every process. You can use a viewer on COMM_SELF, but you should send it to different files so the output isn't all mixed together. On Sep 1, 2011 1:33 PM, "Likun Tan" wrote: > Thank you very much. > > With the command: > for(j=0; j { > for(i=0; i { > MatSetValues(B, 1, &i, 1, &j, &Value, INSERT_VALUES); > } > } > This only defines the columns from 0 to n-1? How exactly MPI_Scan() > getting used here? > > And when i try to print out the result with > > PetscViewer viewer; > PetscViewerSetFormat(viewer, PETSC_VIEWER_ASCII_MATLAB); > MatView(B, viewer); > > I can only see the columns from 0 to n-1. Maybe there is still problem in > computing B. > > Many thanks, > Likun > > On Thu, September 1, 2011 1:54 pm, Jed Brown wrote: >> On Thu, Sep 1, 2011 at 12:45, Likun Tan wrote: >> >> >>> I still have some confusions. When computing B before MPI_Scan(), could >>> i compute the values in parallel? After using MPI_Scan(), does that mean >>> the columns of B will be gathered in one processor? >>> >> >> No, the scan is just computing the start column for a given process. >> >> >> >>> >>> Here are main steps i took based on your suggestion, >>> >>> >>> PetscSplitOwnership(PETSC_COMM_WORLD, &n, &N); >>> >>> >> >> Add these two lines here: >> >> >> MPI_Scan(&n, &cstart, 1, MPIU_INT, MPI_SUM, PETSC_COMM_WORLD); >> cstart -= n; >> >> >>> MatCreateSeqDense(PETSC_COMM_SELF, M, n, PETSC_NULL, &B); >>> MatCreateSeqDense(PETSC_COMM_SELF, M, M, PETSC_NULL, &A); >>> >>> >> >> Good >> >> >> >>> MatCreateSeqDense(PETSC_COMM_SELF, M, N, PETSC_NULL, &x); >>> >>> >> >> Replace with >> >> >> MatDuplicate(B,MAT_DO_NOT_COPY_VALUES,&x); >> >> >> >>> for(j=0; j>> >> >> change the loop bounds here: >> >> for(j=0; j> >> >>> { >>> for(i=0; i>> >>> >> >> Good, now compute value as the value that goes in (i,cstart+j). >> >> >> MatSetValues(B, 1, &i, 1, &j, &value, INSERT_VALUES); >> >>> } >>> } >>> MatAssemblyBegin(...); >>> MatAssemblyEnd(...) >>> >>> >> >> This part is correct. >> >> >> >>> MPI_Scan(&n, &cstart, 1, MPIU_INT, MPI_SUM, PETSC_COMM_WORLD); >>> cstart -= n; >>> >> >> We did this already. >> >> >> >>> >>> MatConvert(...); >>> MatCholeskyFactor(...); >>> MatMatSolve(...); >>> >>> >> >> Yes. >> >> >> >> You can gather the matrix x onto all processes if you need the whole >> result everywhere, but for performance reasons if you scale further, you >> should avoid it if possible. >> > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From likunt at andrew.cmu.edu Fri Sep 2 18:45:55 2011 From: likunt at andrew.cmu.edu (Likun Tan) Date: Fri, 2 Sep 2011 19:45:55 -0400 Subject: [petsc-users] about MatTranspose In-Reply-To: References: Message-ID: I see, thank you. On Fri, September 2, 2011 7:33 pm, Jed Brown wrote: > Remember that the code has a different B on every process. You can use a > viewer on COMM_SELF, but you should send it to different files so the > output isn't all mixed together. On Sep 1, 2011 1:33 PM, "Likun Tan" > wrote: > >> Thank you very much. >> >> >> With the command: >> for(j=0; j> for(i=0; i> MatSetValues(B, 1, &i, 1, &j, &Value, INSERT_VALUES); >> } >> } >> This only defines the columns from 0 to n-1? How exactly MPI_Scan() >> getting used here? >> >> And when i try to print out the result with >> >> >> PetscViewer viewer; >> PetscViewerSetFormat(viewer, PETSC_VIEWER_ASCII_MATLAB); >> MatView(B, viewer); >> >> >> I can only see the columns from 0 to n-1. Maybe there is still problem >> in computing B. >> >> Many thanks, >> Likun >> >> >> On Thu, September 1, 2011 1:54 pm, Jed Brown wrote: >> >>> On Thu, Sep 1, 2011 at 12:45, Likun Tan >>> wrote: >>> >>> >>> >>>> I still have some confusions. When computing B before MPI_Scan(), >>>> could i compute the values in parallel? After using MPI_Scan(), does >>>> that mean the columns of B will be gathered in one processor? >>>> >>> >>> No, the scan is just computing the start column for a given process. >>> >>> >>> >>> >>>> >>>> Here are main steps i took based on your suggestion, >>>> >>>> >>>> >>>> PetscSplitOwnership(PETSC_COMM_WORLD, &n, &N); >>>> >>>> >>>> >>> >>> Add these two lines here: >>> >>> >>> >>> MPI_Scan(&n, &cstart, 1, MPIU_INT, MPI_SUM, PETSC_COMM_WORLD); >>> cstart -= n; >>> >>> >>>> MatCreateSeqDense(PETSC_COMM_SELF, M, n, PETSC_NULL, &B); >>>> MatCreateSeqDense(PETSC_COMM_SELF, M, M, PETSC_NULL, &A); >>>> >>>> >>>> >>> >>> Good >>> >>> >>> >>> >>>> MatCreateSeqDense(PETSC_COMM_SELF, M, N, PETSC_NULL, &x); >>>> >>>> >>>> >>> >>> Replace with >>> >>> >>> >>> MatDuplicate(B,MAT_DO_NOT_COPY_VALUES,&x); >>> >>> >>> >>> >>>> for(j=0; j>>> >>> >>> change the loop bounds here: >>> >>> for(j=0; j>> >>> >>>> { >>>> for(i=0; i>>> >>>> >>> >>> Good, now compute value as the value that goes in (i,cstart+j). >>> >>> >>> >>> MatSetValues(B, 1, &i, 1, &j, &value, INSERT_VALUES); >>> >>> >>>> } >>>> } >>>> MatAssemblyBegin(...); >>>> MatAssemblyEnd(...) >>>> >>>> >>>> >>> >>> This part is correct. >>> >>> >>> >>> >>>> MPI_Scan(&n, &cstart, 1, MPIU_INT, MPI_SUM, PETSC_COMM_WORLD); >>>> cstart -= n; >>>> >>> >>> We did this already. >>> >>> >>> >>> >>>> >>>> MatConvert(...); >>>> MatCholeskyFactor(...); >>>> MatMatSolve(...); >>>> >>>> >>>> >>> >>> Yes. >>> >>> >>> >>> >>> You can gather the matrix x onto all processes if you need the whole >>> result everywhere, but for performance reasons if you scale further, >>> you should avoid it if possible. >>> >> >> >> >> > From bsmith at mcs.anl.gov Fri Sep 2 20:18:26 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Fri, 2 Sep 2011 20:18:26 -0500 Subject: [petsc-users] PETSc 3.2 release candidate References: Message-ID: <73A52DBA-548D-456D-A613-E8EC1EE23558@mcs.anl.gov> You can find the PETSc 3.2 release candidate via Mecurial at: > > ssh://petsc at petsc.cs.iit.edu//hg/petsc/releases/petsc-3.2 or http://petsc.cs.iit.edu/petsc/releases/petsc-3.2 > ssh://petsc at petsc.cs.iit.edu//hg/petsc/releases/BuildSystem-3.2 or http://petsc.cs.iit.edu/petsc/releases/BuildSystem-3.2 If you find any problems please let us know. We'll be making the tarball in a couple of days. Thanks Barry If you fix anything, please make the fix in petsc-3.2 not petsc-dev > > From balay at mcs.anl.gov Fri Sep 2 21:01:27 2011 From: balay at mcs.anl.gov (Satish Balay) Date: Fri, 2 Sep 2011 21:01:27 -0500 (CDT) Subject: [petsc-users] PETSc 3.2 release candidate In-Reply-To: <73A52DBA-548D-456D-A613-E8EC1EE23558@mcs.anl.gov> References: <73A52DBA-548D-456D-A613-E8EC1EE23558@mcs.anl.gov> Message-ID: Also one can use the following tarball. It will correspond to petsc-3.2 sources for the next few days. And i've set it up to be updated every 3 hours [from the sources in mercurial]. http://ftp.mcs.anl.gov/pub/petsc/petsc-dev.tar.gz Satish On Fri, 2 Sep 2011, Barry Smith wrote: > > You can find the PETSc 3.2 release candidate via Mecurial at: > > > > ssh://petsc at petsc.cs.iit.edu//hg/petsc/releases/petsc-3.2 or http://petsc.cs.iit.edu/petsc/releases/petsc-3.2 > > ssh://petsc at petsc.cs.iit.edu//hg/petsc/releases/BuildSystem-3.2 or http://petsc.cs.iit.edu/petsc/releases/BuildSystem-3.2 > > If you find any problems please let us know. We'll be making the tarball in a couple of days. > > Thanks > > Barry > > If you fix anything, please make the fix in petsc-3.2 not petsc-dev > > > > > > > From dominik at itis.ethz.ch Sat Sep 3 12:48:29 2011 From: dominik at itis.ethz.ch (Dominik Szczerba) Date: Sat, 3 Sep 2011 19:48:29 +0200 Subject: [petsc-users] ERROR: sum weight for constraint 0 is zero Message-ID: I am occasionally and not reproducibly getting this error: ERROR: sum weight for constraint 0 is zero It is emitted by parmetis. My code passed valgrind reasonably well. Well, the input mesh could be invalid in some way or another of course (even when I do not think so), but then the error should be reproducible, while it is not. Does anyone have an idea what can cause it and how to prevent it? Many hanks for any hints, Dominik -------------- next part -------------- An HTML attachment was scrubbed... URL: From stali at geology.wisc.edu Sat Sep 3 13:04:15 2011 From: stali at geology.wisc.edu (Tabrez Ali) Date: Sat, 03 Sep 2011 13:04:15 -0500 Subject: [petsc-users] PETSc 3.2 release candidate In-Reply-To: References: <73A52DBA-548D-456D-A613-E8EC1EE23558@mcs.anl.gov> Message-ID: <4E626C1F.9030207@geology.wisc.edu> Hello What is the equivalent of $(PETSC_INCLUDE) in petsc-3.2? I am using Fortran and my files end in .f90. My working Makefile for petsc-3.1 (with gfortran) looks like include ${PETSC_DIR}/conf/variables include ${PETSC_DIR}/conf/rules all: mpif90 -c -Wall -x f95-cpp-input $(PETSC_INCLUDE) a.f90 b.f90 c.f90 $(FLINKER) a.o b.o c.o $(PETSC_FORTRAN_LIB) $(PETSC_LIB) -o exec $(RM) *.o *.mod With 3.2 I get the error stali at i5:~/src$ make all mpif90 -c -Wall -x f95-cpp-input a.f90 b.f90 c.f90 b.f90:8:0: fatal error: finclude/petscsys.h: No such file or directory Thanks in advance Tabrez On 09/02/2011 09:01 PM, Satish Balay wrote: > Also one can use the following tarball. It will correspond to > petsc-3.2 sources for the next few days. And i've set it up to be > updated every 3 hours [from the sources in mercurial]. > > http://ftp.mcs.anl.gov/pub/petsc/petsc-dev.tar.gz > > Satish > > On Fri, 2 Sep 2011, Barry Smith wrote: > >> You can find the PETSc 3.2 release candidate via Mecurial at: >>> ssh://petsc at petsc.cs.iit.edu//hg/petsc/releases/petsc-3.2 or http://petsc.cs.iit.edu/petsc/releases/petsc-3.2 >>> ssh://petsc at petsc.cs.iit.edu//hg/petsc/releases/BuildSystem-3.2 or http://petsc.cs.iit.edu/petsc/releases/BuildSystem-3.2 >> If you find any problems please let us know. We'll be making the tarball in a couple of days. >> >> Thanks >> >> Barry >> >> If you fix anything, please make the fix in petsc-3.2 not petsc-dev >> >> >>> From balay at mcs.anl.gov Sat Sep 3 13:03:49 2011 From: balay at mcs.anl.gov (Satish Balay) Date: Sat, 3 Sep 2011 13:03:49 -0500 (CDT) Subject: [petsc-users] PETSc 3.2 release candidate In-Reply-To: <4E626C1F.9030207@geology.wisc.edu> References: <73A52DBA-548D-456D-A613-E8EC1EE23558@mcs.anl.gov> <4E626C1F.9030207@geology.wisc.edu> Message-ID: You can try FCPPFLAGS instead of PETSC_INCLUDE Also - suggest renaming all sources [that call PESc] to .F90 - and simplify makefile. [as PETSc targets compile .F90 files properly.] Satish On Sat, 3 Sep 2011, Tabrez Ali wrote: > Hello > > What is the equivalent of $(PETSC_INCLUDE) in petsc-3.2? I am using Fortran > and my files end in .f90. My working Makefile for petsc-3.1 (with gfortran) > looks like > > include ${PETSC_DIR}/conf/variables > include ${PETSC_DIR}/conf/rules > all: > mpif90 -c -Wall -x f95-cpp-input $(PETSC_INCLUDE) a.f90 b.f90 c.f90 > $(FLINKER) a.o b.o c.o $(PETSC_FORTRAN_LIB) $(PETSC_LIB) -o exec > $(RM) *.o *.mod > > With 3.2 I get the error > > stali at i5:~/src$ make all > mpif90 -c -Wall -x f95-cpp-input a.f90 b.f90 c.f90 > b.f90:8:0: fatal error: finclude/petscsys.h: No such file or directory > > Thanks in advance > > Tabrez > > On 09/02/2011 09:01 PM, Satish Balay wrote: > > Also one can use the following tarball. It will correspond to > > petsc-3.2 sources for the next few days. And i've set it up to be > > updated every 3 hours [from the sources in mercurial]. > > > > http://ftp.mcs.anl.gov/pub/petsc/petsc-dev.tar.gz > > > > Satish > > > > On Fri, 2 Sep 2011, Barry Smith wrote: > > > > > You can find the PETSc 3.2 release candidate via Mecurial at: > > > > ssh://petsc at petsc.cs.iit.edu//hg/petsc/releases/petsc-3.2 > > > > or http://petsc.cs.iit.edu/petsc/releases/petsc-3.2 > > > > ssh://petsc at petsc.cs.iit.edu//hg/petsc/releases/BuildSystem-3.2 > > > > or http://petsc.cs.iit.edu/petsc/releases/BuildSystem-3.2 > > > If you find any problems please let us know. We'll be making the > > > tarball in a couple of days. > > > > > > Thanks > > > > > > Barry > > > > > > If you fix anything, please make the fix in petsc-3.2 not petsc-dev > > > > > > > > > > > > From knepley at gmail.com Sat Sep 3 13:04:04 2011 From: knepley at gmail.com (Matthew Knepley) Date: Sat, 3 Sep 2011 18:04:04 +0000 Subject: [petsc-users] PETSc 3.2 release candidate In-Reply-To: <4E626C1F.9030207@geology.wisc.edu> References: <73A52DBA-548D-456D-A613-E8EC1EE23558@mcs.anl.gov> <4E626C1F.9030207@geology.wisc.edu> Message-ID: On Sat, Sep 3, 2011 at 6:04 PM, Tabrez Ali wrote: > Hello > > What is the equivalent of $(PETSC_INCLUDE) in petsc-3.2? I am using Fortran > and my files end in .f90. My working Makefile for petsc-3.1 (with gfortran) > looks like > > include ${PETSC_DIR}/conf/variables > include ${PETSC_DIR}/conf/rules > all: > mpif90 -c -Wall -x f95-cpp-input $(PETSC_INCLUDE) a.f90 b.f90 c.f90 > $(FLINKER) a.o b.o c.o $(PETSC_FORTRAN_LIB) $(PETSC_LIB) -o exec > $(RM) *.o *.mod > > With 3.2 I get the error > > stali at i5:~/src$ make all > mpif90 -c -Wall -x f95-cpp-input a.f90 b.f90 c.f90 > b.f90:8:0: fatal error: finclude/petscsys.h: No such file or directory > In order to get around some portability problems we split PETSC_INCLUDE into ${PETSC_CC_INCLUDES} and ${PETSC_FC_INCLUDES} I believe you want the later. Thanks, Matt > Thanks in advance > > Tabrez > > On 09/02/2011 09:01 PM, Satish Balay wrote: > >> Also one can use the following tarball. It will correspond to >> petsc-3.2 sources for the next few days. And i've set it up to be >> updated every 3 hours [from the sources in mercurial]. >> >> http://ftp.mcs.anl.gov/pub/**petsc/petsc-dev.tar.gz >> >> Satish >> >> On Fri, 2 Sep 2011, Barry Smith wrote: >> >> You can find the PETSc 3.2 release candidate via Mecurial at: >>> >>>> ssh://petsc at petsc.cs.iit.edu//**hg/petsc/releases/petsc-3.2 or >>>> http://petsc.cs.iit.edu/petsc/**releases/petsc-3.2 >>>> ssh://petsc at petsc.cs.iit.edu//**hg/petsc/releases/BuildSystem-**3.2 or >>>> http://petsc.cs.iit.edu/petsc/**releases/BuildSystem-3.2 >>>> >>> If you find any problems please let us know. We'll be making the >>> tarball in a couple of days. >>> >>> Thanks >>> >>> Barry >>> >>> If you fix anything, please make the fix in petsc-3.2 not petsc-dev >>> >>> >>> >>>> > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From stali at geology.wisc.edu Sat Sep 3 13:08:18 2011 From: stali at geology.wisc.edu (Tabrez Ali) Date: Sat, 03 Sep 2011 13:08:18 -0500 Subject: [petsc-users] PETSc 3.2 release candidate In-Reply-To: References: <73A52DBA-548D-456D-A613-E8EC1EE23558@mcs.anl.gov> <4E626C1F.9030207@geology.wisc.edu> Message-ID: <4E626D12.3050705@geology.wisc.edu> Thanks! FCPPFLAGS did it. On 09/03/2011 01:03 PM, Satish Balay wrote: > You can try FCPPFLAGS instead of PETSC_INCLUDE > > Also - suggest renaming all sources [that call PESc] to .F90 - and > simplify makefile. [as PETSc targets compile .F90 files properly.] > > Satish > > On Sat, 3 Sep 2011, Tabrez Ali wrote: > >> Hello >> >> What is the equivalent of $(PETSC_INCLUDE) in petsc-3.2? I am using Fortran >> and my files end in .f90. My working Makefile for petsc-3.1 (with gfortran) >> looks like >> >> include ${PETSC_DIR}/conf/variables >> include ${PETSC_DIR}/conf/rules >> all: >> mpif90 -c -Wall -x f95-cpp-input $(PETSC_INCLUDE) a.f90 b.f90 c.f90 >> $(FLINKER) a.o b.o c.o $(PETSC_FORTRAN_LIB) $(PETSC_LIB) -o exec >> $(RM) *.o *.mod >> >> With 3.2 I get the error >> >> stali at i5:~/src$ make all >> mpif90 -c -Wall -x f95-cpp-input a.f90 b.f90 c.f90 >> b.f90:8:0: fatal error: finclude/petscsys.h: No such file or directory >> >> Thanks in advance >> >> Tabrez >> >> On 09/02/2011 09:01 PM, Satish Balay wrote: >>> Also one can use the following tarball. It will correspond to >>> petsc-3.2 sources for the next few days. And i've set it up to be >>> updated every 3 hours [from the sources in mercurial]. >>> >>> http://ftp.mcs.anl.gov/pub/petsc/petsc-dev.tar.gz >>> >>> Satish >>> >>> On Fri, 2 Sep 2011, Barry Smith wrote: >>> >>>> You can find the PETSc 3.2 release candidate via Mecurial at: >>>>> ssh://petsc at petsc.cs.iit.edu//hg/petsc/releases/petsc-3.2 >>>>> or http://petsc.cs.iit.edu/petsc/releases/petsc-3.2 >>>>> ssh://petsc at petsc.cs.iit.edu//hg/petsc/releases/BuildSystem-3.2 >>>>> or http://petsc.cs.iit.edu/petsc/releases/BuildSystem-3.2 >>>> If you find any problems please let us know. We'll be making the >>>> tarball in a couple of days. >>>> >>>> Thanks >>>> >>>> Barry >>>> >>>> If you fix anything, please make the fix in petsc-3.2 not petsc-dev >>>> >>>> >> From ram at ibrae.ac.ru Sun Sep 4 10:08:18 2011 From: ram at ibrae.ac.ru (=?UTF-8?B?0JDQu9C10LrRgdC10Lkg0KDRj9C30LDQvdC+0LI=?=) Date: Sun, 4 Sep 2011 19:08:18 +0400 Subject: [petsc-users] src/ksp/ksp/examples/tutorials/ex22.c Message-ID: Hello! Help me please to solve my tiny problem with dmmg. In the example src/ksp/ksp/examples/tutorials/ex22.c, which is discussed on the 111th page of users manual, we use function (line 36): DACreate3d(PETSC_COMM_WORLD,DA_NONPERIODIC,DA_STENCIL_STAR,-3,-3,-3,PETSC_DECIDE,PETSC_DECIDE,PETSC_DECIDE,1,1,0,0,0,&da); to "*create the DA that stores information about the coarsest grid you wish to use*". We can see that negative dimension values (-M, -N, -P) are used "*to indicate that it may be set to a different value from the command line with -da_grid_x -da_grid_y -da_grid_z

". * The question is: when I use -da_grid_ M,N,P options as the size of the coarsest matrix, what's the size of the finest (most accurate, original problem) matrix? I've tried to clear this up, using VecVew on the residual vector and figured out, that when i'm using M,N,P as input parameters (so define the coarsest grid as field of M*N*P dots), I get the residual vector of size (4*M-3)*(4*N-3)*(4*P-3). Is my result correct? If yes, could you please explain why it happens in this way. Sorry if my question is incorrect. I could mix up everything at all. Thanks in advance! -- Best regards, Alexey Ryazanov ______________________________________ Nuclear Safety Institute of Russian Academy of Sciences -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Sun Sep 4 10:50:39 2011 From: knepley at gmail.com (Matthew Knepley) Date: Sun, 4 Sep 2011 10:50:39 -0500 Subject: [petsc-users] src/ksp/ksp/examples/tutorials/ex22.c In-Reply-To: References: Message-ID: On Sun, Sep 4, 2011 at 10:08 AM, ??????? ??????? wrote: > Hello! > Help me please to solve my tiny problem with dmmg. > > In the example src/ksp/ksp/examples/tutorials/ex22.c, which is discussed on > the 111th page of users manual, we use function > (line 36): > DACreate3d(PETSC_COMM_WORLD,DA_NONPERIODIC,DA_STENCIL_STAR,-3,-3,-3,PETSC_DECIDE,PETSC_DECIDE,PETSC_DECIDE,1,1,0,0,0,&da); > to "*create the DA that stores information about the coarsest grid you > wish to use*". > We can see that negative dimension values (-M, -N, -P) are used "*to > indicate that it may be set to a different value from the command line with > -da_grid_x -da_grid_y -da_grid_z

". * > The question is: when I use -da_grid_ M,N,P options as the size of the > coarsest matrix, what's the size of the finest (most accurate, original > problem) matrix? > > I've tried to clear this up, using VecVew on the residual vector and > figured out, that when i'm using M,N,P as input parameters (so define the > coarsest grid as field of M*N*P dots), I get the residual vector of size > (4*M-3)*(4*N-3)*(4*P-3). > Is my result correct? If yes, could you please explain why it happens in > this way. > When refinement happens, it adds a vertex between any two vertices, starting with the mesh you provide. Matt > Sorry if my question is incorrect. I could mix up everything at all. > Thanks in advance! > > -- > Best regards, > Alexey Ryazanov > ______________________________________ > Nuclear Safety Institute of Russian Academy of Sciences > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From amryazanov at gmail.com Sun Sep 4 11:55:24 2011 From: amryazanov at gmail.com (Alexey Ryazanov) Date: Sun, 4 Sep 2011 20:55:24 +0400 Subject: [petsc-users] src/ksp/ksp/examples/tutorials/ex22.c In-Reply-To: References: Message-ID: Thank You! 2011/9/4 Matthew Knepley > On Sun, Sep 4, 2011 at 10:08 AM, ??????? ??????? wrote: > >> Hello! >> Help me please to solve my tiny problem with dmmg. >> >> In the example src/ksp/ksp/examples/tutorials/ex22.c, which is discussed >> on the 111th page of users manual, we use function >> (line 36): >> DACreate3d(PETSC_COMM_WORLD,DA_NONPERIODIC,DA_STENCIL_STAR,-3,-3,-3,PETSC_DECIDE,PETSC_DECIDE,PETSC_DECIDE,1,1,0,0,0,&da); >> to "*create the DA that stores information about the coarsest grid you >> wish to use*". >> We can see that negative dimension values (-M, -N, -P) are used "*to >> indicate that it may be set to a different value from the command line with >> -da_grid_x -da_grid_y -da_grid_z

". * >> The question is: when I use -da_grid_ M,N,P options as the size of the >> coarsest matrix, what's the size of the finest (most accurate, original >> problem) matrix? >> >> I've tried to clear this up, using VecVew on the residual vector and >> figured out, that when i'm using M,N,P as input parameters (so define the >> coarsest grid as field of M*N*P dots), I get the residual vector of size >> (4*M-3)*(4*N-3)*(4*P-3). >> Is my result correct? If yes, could you please explain why it happens in >> this way. >> > > When refinement happens, it adds a vertex between any two vertices, > starting with the mesh you provide. > > Matt > > >> Sorry if my question is incorrect. I could mix up everything at all. >> Thanks in advance! >> >> -- >> Best regards, >> Alexey Ryazanov >> ______________________________________ >> Nuclear Safety Institute of Russian Academy of Sciences >> >> >> > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > -------------- next part -------------- An HTML attachment was scrubbed... URL: From amrit_pou at hotmail.com Sun Sep 4 21:05:55 2011 From: amrit_pou at hotmail.com (amrit poudel) Date: Mon, 5 Sep 2011 02:05:55 +0000 Subject: [petsc-users] Write complex matrix/vector from Matlab Message-ID: Hello everyone, Does anyone here know how one can write a complex or real, matrix or vector (matrix could be sparse or full) in MATLAB to a binary file such that it can be read correctly by PETSc? I tried using PetscBinaryWrite.m that came with PETSc ( /bin/matlab) to write a simple real (or complex), full (or sparse) matrix in binary file named 'matrix.dat' and tried reading it in PETSc, but it absolutely does not work :( I did PetscBinaryWrite('matrix.dat', C) to write a matrix C in an empty file named 'matrix.dat' in MATLAB. Then I tried reading 'matrix.dat' file in PETSc using the following command : PetscViewer binviewer ; Mat C ; char fileC[PETSC_MAX_PATH_LEN] ; ierr = PetscOptionsGetString(PETSC_NULL, "-fC", fileC, PETSC_MAX_PATH_LEN-1, PETSC_NULL) ; CHKERRQ(ierr) ; ierr = PetscViewerBinaryOpen(PETSC_COMM_WORLD, fileC, FILE_MODE_READ, &binviewer) ; CHKERRQ(ierr) ; ierr = MatLoad(binviewer, MATMPIAIJ, &C) ; CHKERRQ(ierr) ; I must add that I compiled PETSc with complex numbers. I plan on assembling my matrix (very large square sparse matrix, size greater than 1e+8) in MATLAB, which does not take too long to assemble in MATLAB at all. Then I will write the matrix to a binary file and read that in PETSc and solve a very large linear system of equations in parallel in PETSc using MPI. Hence, I really need to be able to write the matrix in MATLAB to a binary file such that PETSc can correctly read the matrix. I am really surprised to find out that PETSc cannot read/write a matrix or vector, full or sparse, real or complex in MATRIX MARKET FORMAT. I have gone though the manual, googled around quite a bit, but I just have not found anything useful. I will truly appreciate any help or a pointer in this regard. Thank you. Amrit -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Sun Sep 4 21:14:24 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Sun, 4 Sep 2011 21:14:24 -0500 Subject: [petsc-users] Write complex matrix/vector from Matlab In-Reply-To: References: Message-ID: <1B77A54D-D1E5-4235-A4BE-9A3ECAAE2675@mcs.anl.gov> Amrit, We have just recently made sure that PetscBinaryWrite.m and PetscBinaryRead.m fully support complex numbers; you can access the release candidate for PETSc 3.2 for this support One can use the following tarball. It will correspond to petsc-3.2 sources for the next few days. And i've set it up to be updated every 3 hours [from the sources in mercurial]. http://ftp.mcs.anl.gov/pub/petsc/petsc-dev.tar.gz or > > You can find the PETSc 3.2 release candidate via Mecurial at: >> >> ssh://petsc at petsc.cs.iit.edu//hg/petsc/releases/petsc-3.2 or http://petsc.cs.iit.edu/petsc/releases/petsc-3.2 >> ssh://petsc at petsc.cs.iit.edu//hg/petsc/releases/BuildSystem-3.2 or http://petsc.cs.iit.edu/petsc/releases/BuildSystem-3.2 Should you have problems reading the complex matrices/vectors saved from MATLAB please send all information to petsc-maint at mcs.anl.gov We do not support the Matrix Market sparse format because it is not scalable for large matrices (i.e. it is too slow) ; reading ASCII files in parallel is not viable. Barry On Sep 4, 2011, at 9:05 PM, amrit poudel wrote: > > Hello everyone, > > Does anyone here know how one can write a complex or real, matrix or vector (matrix could be sparse or full) in MATLAB to a binary file such that it can be read correctly by PETSc? > I tried using PetscBinaryWrite.m that came with PETSc ( /bin/matlab) to write a simple real (or complex), full (or sparse) matrix in binary file named 'matrix.dat' and tried reading it in PETSc, but it absolutely does not work :( > > I did PetscBinaryWrite('matrix.dat', C) to write a matrix C in an empty file named 'matrix.dat' in MATLAB. Then I tried reading 'matrix.dat' file in PETSc using the following command : > > PetscViewer binviewer ; > Mat C ; > char fileC[PETSC_MAX_PATH_LEN] ; > > ierr = PetscOptionsGetString(PETSC_NULL, "-fC", fileC, PETSC_MAX_PATH_LEN-1, PETSC_NULL) ; CHKERRQ(ierr) ; > > ierr = PetscViewerBinaryOpen(PETSC_COMM_WORLD, fileC, FILE_MODE_READ, &binviewer) ; CHKERRQ(ierr) ; > > ierr = MatLoad(binviewer, MATMPIAIJ, &C) ; CHKERRQ(ierr) ; > > > I must add that I compiled PETSc with complex numbers. > > I plan on assembling my matrix (very large square sparse matrix, size greater than 1e+8) in MATLAB, which does not take too long to assemble in MATLAB at all. Then I will write the matrix to a binary file and read that in PETSc and solve a very large linear system of equations in parallel in PETSc using MPI. Hence, I really need to be able to write the matrix in MATLAB to a binary file such that PETSc can correctly read the matrix. > > I am really surprised to find out that PETSc cannot read/write a matrix or vector, full or sparse, real or complex in MATRIX MARKET FORMAT. I have gone though the manual, googled around quite a bit, but I just have not found anything useful. > > I will truly appreciate any help or a pointer in this regard. > > Thank you. > > Amrit > From amrit_pou at hotmail.com Sun Sep 4 21:52:18 2011 From: amrit_pou at hotmail.com (amrit poudel) Date: Mon, 5 Sep 2011 02:52:18 +0000 Subject: [petsc-users] Write complex matrix/vector from Matlab In-Reply-To: <1B77A54D-D1E5-4235-A4BE-9A3ECAAE2675@mcs.anl.gov> References: , <1B77A54D-D1E5-4235-A4BE-9A3ECAAE2675@mcs.anl.gov> Message-ID: Hello Barry, Thanks for your quick response. The one that I have compiled was PETSc-3.1-p8. Does this mean I will need to re-compile the most recent 3.2 version to get this working? I was surprised due to the fact that PetscBinaryWrite.m did not even work for REAL matrix. The error I got while reading the file in PETSc was : [0]PETSC ERROR: Read from file failed! [0]PETSC ERROR: Read past end of file! You mentioned in your reply that "reading ASCII files in parallel is not viable. " Does any part of my code show that I am trying to do this? I am asking this just to make sure that I am doing the right thing and not making any such mistakes. Did you mean that I should avoid doing something like this : ierr = MatLoad(binviewer, MATMPIAIJ, &C) ; CHKERRQ(ierr) ; Is this really not storing a sparse matrix(if C is sparse) in parallel across different processors( if I were running this on a multi-processor computer or a cluster)? My ultimate goal is the following, which I am copying here from my previous email : I plan on assembling my matrix (very large square sparse matrix, size greater than 1e+8) in MATLAB, which does not take too long to assemble in MATLAB at all. Then I will write the matrix to a binary file and read that in PETSc and solve a very large linear system of equations in parallel in PETSc using MPI. All I want to make sure is be able to read the matrix in PETSc and then have it distribute over different processors and then solve the linear system. Sorry for repeating my goals here again, but I was just trying to get any feedback whether this is really the way to work with PETSc. Thank you for your help. -Amrit > From: bsmith at mcs.anl.gov > Date: Sun, 4 Sep 2011 21:14:24 -0500 > To: petsc-users at mcs.anl.gov > Subject: Re: [petsc-users] Write complex matrix/vector from Matlab > > > Amrit, > > We have just recently made sure that PetscBinaryWrite.m and PetscBinaryRead.m fully support complex numbers; you can access the release candidate for PETSc 3.2 for this support > > One can use the following tarball. It will correspond to > petsc-3.2 sources for the next few days. And i've set it up to be > updated every 3 hours [from the sources in mercurial]. > > http://ftp.mcs.anl.gov/pub/petsc/petsc-dev.tar.gz > > or > > > > > You can find the PETSc 3.2 release candidate via Mecurial at: > >> > >> ssh://petsc at petsc.cs.iit.edu//hg/petsc/releases/petsc-3.2 or http://petsc.cs.iit.edu/petsc/releases/petsc-3.2 > >> ssh://petsc at petsc.cs.iit.edu//hg/petsc/releases/BuildSystem-3.2 or http://petsc.cs.iit.edu/petsc/releases/BuildSystem-3.2 > > Should you have problems reading the complex matrices/vectors saved from MATLAB please send all information to petsc-maint at mcs.anl.gov > > We do not support the Matrix Market sparse format because it is not scalable for large matrices (i.e. it is too slow) ; reading ASCII files in parallel is not viable. > > > Barry > > On Sep 4, 2011, at 9:05 PM, amrit poudel wrote: > > > > > Hello everyone, > > > > Does anyone here know how one can write a complex or real, matrix or vector (matrix could be sparse or full) in MATLAB to a binary file such that it can be read correctly by PETSc? > > I tried using PetscBinaryWrite.m that came with PETSc ( /bin/matlab) to write a simple real (or complex), full (or sparse) matrix in binary file named 'matrix.dat' and tried reading it in PETSc, but it absolutely does not work :( > > > > I did PetscBinaryWrite('matrix.dat', C) to write a matrix C in an empty file named 'matrix.dat' in MATLAB. Then I tried reading 'matrix.dat' file in PETSc using the following command : > > > > PetscViewer binviewer ; > > Mat C ; > > char fileC[PETSC_MAX_PATH_LEN] ; > > > > ierr = PetscOptionsGetString(PETSC_NULL, "-fC", fileC, PETSC_MAX_PATH_LEN-1, PETSC_NULL) ; CHKERRQ(ierr) ; > > > > ierr = PetscViewerBinaryOpen(PETSC_COMM_WORLD, fileC, FILE_MODE_READ, &binviewer) ; CHKERRQ(ierr) ; > > > > ierr = MatLoad(binviewer, MATMPIAIJ, &C) ; CHKERRQ(ierr) ; > > > > > > I must add that I compiled PETSc with complex numbers. > > > > I plan on assembling my matrix (very large square sparse matrix, size greater than 1e+8) in MATLAB, which does not take too long to assemble in MATLAB at all. Then I will write the matrix to a binary file and read that in PETSc and solve a very large linear system of equations in parallel in PETSc using MPI. Hence, I really need to be able to write the matrix in MATLAB to a binary file such that PETSc can correctly read the matrix. > > > > I am really surprised to find out that PETSc cannot read/write a matrix or vector, full or sparse, real or complex in MATRIX MARKET FORMAT. I have gone though the manual, googled around quite a bit, but I just have not found anything useful. > > > > I will truly appreciate any help or a pointer in this regard. > > > > Thank you. > > > > Amrit > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Sun Sep 4 21:59:27 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Sun, 4 Sep 2011 21:59:27 -0500 Subject: [petsc-users] Write complex matrix/vector from Matlab In-Reply-To: References: , <1B77A54D-D1E5-4235-A4BE-9A3ECAAE2675@mcs.anl.gov> Message-ID: <6DF61307-4C2B-4B7B-954F-6EA90917489D@mcs.anl.gov> On Sep 4, 2011, at 9:52 PM, amrit poudel wrote: > > > Hello Barry, > Thanks for your quick response. The one that I have compiled was PETSc-3.1-p8. Does this mean I will need to re-compile the most recent 3.2 version to get this working? I was surprised due to the fact that PetscBinaryWrite.m did not even work for REAL matrix. The error I got while reading the file in PETSc was : > [0]PETSC ERROR: Read from file failed! > [0]PETSC ERROR: Read past end of file! > Please send your MATLAB code that generates the matrix to petsc-maint at mcs.anl.gov saving real matrices and reading them in PETSc should not be an issue. This should all work with petsc-3.1 > > You mentioned in your reply that "reading ASCII files in parallel is not viable. " > Does any part of my code show that I am trying to do this? I am asking this just to make sure that I am doing the right thing and not making any such mistakes. > Did you mean that I should avoid doing something like this : > ierr = MatLoad(binviewer, MATMPIAIJ, &C) ; CHKERRQ(ierr) ; No, this is correct; this is reading a binary file. It is only reading ASCII sparse matrices in parallel that is a problem. > > Is this really not storing a sparse matrix(if C is sparse) in parallel across different processors( if I were running this on a multi-processor computer or a cluster)? > My ultimate goal is the following, which I am copying here from my previous email : > I plan on assembling my matrix (very large square sparse matrix, size greater than 1e+8) in MATLAB, which does not take too long to assemble in MATLAB at all. Then I will write the matrix to a binary file and read that in PETSc and solve a very large linear system of equations in parallel in PETSc using MPI. > > All I want to make sure is be able to read the matrix in PETSc and then have it distribute over different processors and then solve the linear system. Yes, this is all supported and you should be able to do it. Barry > > Sorry for repeating my goals here again, but I was just trying to get any feedback whether this is really the way to work with PETSc. > > Thank you for your help. > > -Amrit > > > > > From: bsmith at mcs.anl.gov > > Date: Sun, 4 Sep 2011 21:14:24 -0500 > > To: petsc-users at mcs.anl.gov > > Subject: Re: [petsc-users] Write complex matrix/vector from Matlab > > > > > > Amrit, > > > > We have just recently made sure that PetscBinaryWrite.m and PetscBinaryRead.m fully support complex numbers; you can access the release candidate for PETSc 3.2 for this support > > > > One can use the following tarball. It will correspond to > > petsc-3.2 sources for the next few days. And i've set it up to be > > updated every 3 hours [from the sources in mercurial]. > > > > http://ftp.mcs.anl.gov/pub/petsc/petsc-dev.tar.gz > > > > or > > > > > > > > You can find the PETSc 3.2 release candidate via Mecurial at: > > >> > > >> ssh://petsc at petsc.cs.iit.edu//hg/petsc/releases/petsc-3.2 or http://petsc.cs.iit.edu/petsc/releases/petsc-3.2 > > >> ssh://petsc at petsc.cs.iit.edu//hg/petsc/releases/BuildSystem-3.2 or http://petsc.cs.iit.edu/petsc/releases/BuildSystem-3.2 > > > > Should you have problems reading the complex matrices/vectors saved from MATLAB please send all information to petsc-maint at mcs.anl.gov > > > > We do not support the Matrix Market sparse format because it is not scalable for large matrices (i.e. it is too slow) ; reading ASCII files in parallel is not viable. > > > > > > Barry > > > > On Sep 4, 2011, at 9:05 PM, amrit poudel wrote: > > > > > > > > Hello everyone, > > > > > > Does anyone here know how one can write a complex or real, matrix or vector (matrix could be sparse or full) in MATLAB to a binary file such that it can be read correctly by PETSc? > > > I tried using PetscBinaryWrite.m that came with PETSc ( /bin/matlab) to write a simple real (or complex), full (or sparse) matrix in binary file named 'matrix.dat' and tried reading it in PETSc, but it absolutely does not work :( > > > > > > I did PetscBinaryWrite('matrix.dat', C) to write a matrix C in an empty file named 'matrix.dat' in MATLAB. Then I tried reading 'matrix.dat' file in PETSc using the following command : > > > > > > PetscViewer binviewer ; > > > Mat C ; > > > char fileC[PETSC_MAX_PATH_LEN] ; > > > > > > ierr = PetscOptionsGetString(PETSC_NULL, "-fC", fileC, PETSC_MAX_PATH_LEN-1, PETSC_NULL) ; CHKERRQ(ierr) ; > > > > > > ierr = PetscViewerBinaryOpen(PETSC_COMM_WORLD, fileC, FILE_MODE_READ, &binviewer) ; CHKERRQ(ierr) ; > > > > > > ierr = MatLoad(binviewer, MATMPIAIJ, &C) ; CHKERRQ(ierr) ; > > > > > > > > > I must add that I compiled PETSc with complex numbers. > > > > > > I plan on assembling my matrix (very large square sparse matrix, size greater than 1e+8) in MATLAB, which does not take too long to assemble in MATLAB at all. Then I will write the matrix to a binary file and read that in PETSc and solve a very large linear system of equations in parallel in PETSc using MPI. Hence, I really need to be able to write the matrix in MATLAB to a binary file such that PETSc can correctly read the matrix. > > > > > > I am really surprised to find out that PETSc cannot read/write a matrix or vector, full or sparse, real or complex in MATRIX MARKET FORMAT. I have gone though the manual, googled around quite a bit, but I just have not found anything useful. > > > > > > I will truly appreciate any help or a pointer in this regard. > > > > > > Thank you. > > > > > > Amrit > > > > > From kristen.eisenberg at yahoo.com Mon Sep 5 16:08:16 2011 From: kristen.eisenberg at yahoo.com (Kristen Eisenberg) Date: Mon, 5 Sep 2011 14:08:16 -0700 (PDT) Subject: [petsc-users] (no subject) Message-ID: <1315256896.65089.YahooMailNeo@web122307.mail.ne1.yahoo.com> [petsc-users] SLEPc generalized eigenvalue problem question? I create 2 matrices using: MatCreateSeqDense(PETSC_COMM_SELF, n, n, Ka, &A); MatCreateSeqDense(PETSC_COMM_SELF, n, n, Kb, &B); These matrices are 99% zeros ( 16,016,004 entries and 18660 non-zeros). They are symmetric and real. Their tri-diagonal elements are non-zero plus a few other entries. I tried to use ex7 for the generalized eigenvalue problem: ./ex7.exe -f1 k.dat -f2 m.dat -eps_gen_hermitian -eps_smallest_real > x.out 2>&1 without specifying an EPS and get: Generalized eigenproblem stored in file. Reading REAL matrices from binary files... Number of iterations of the method: 500 Number of linear iterations of the method: 4009 Solution method: krylovschur Number of requested eigenvalues: 1 Stopping condition: tol=1e-07, maxit=500 Number of converged approximate eigenpairs: 0 Is krylovschur inappropriate for this problem or have I set up the problem incorrectly by using MatCreateSeqDense(...) to create the matrix input files in PETSc binary form? Kristen Eisenberg Billige Fl?ge Marketing GmbH Emanuelstr. 3, 10317 Berlin Deutschland Telefon: +49 (33) 5310967 Email: utebachmeier at gmail.com Site: http://flug.airego.de - Billige Fl?ge vergleichen -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Mon Sep 5 16:16:23 2011 From: knepley at gmail.com (Matthew Knepley) Date: Mon, 5 Sep 2011 16:16:23 -0500 Subject: [petsc-users] (no subject) In-Reply-To: <1315256896.65089.YahooMailNeo@web122307.mail.ne1.yahoo.com> References: <1315256896.65089.YahooMailNeo@web122307.mail.ne1.yahoo.com> Message-ID: On Mon, Sep 5, 2011 at 4:08 PM, Kristen Eisenberg < kristen.eisenberg at yahoo.com> wrote: > [petsc-users] SLEPc generalized eigenvalue problem question? > > I create 2 matrices using: > > MatCreateSeqDense(PETSC_COMM_SELF, n, n, Ka, &A); > MatCreateSeqDense(PETSC_COMM_SELF, n, n, Kb, &B); > These matrices are 99% zeros ( 16,016,004 entries and 18660 non-zeros). > They are symmetric and real. Their tri-diagonal elements are non-zero plus > a few other entries. > You would get better performance and use less memory if you made these matrices of type MATAIJ. Matt > I tried to use ex7 for the generalized eigenvalue problem: > > ./ex7.exe -f1 k.dat -f2 m.dat -eps_gen_hermitian -eps_smallest_real > x.out > 2>&1 > > without specifying an EPS and get: > > Generalized eigenproblem stored in file. > > Reading REAL matrices from binary files... > Number of iterations of the method: 500 > Number of linear iterations of the method: 4009 > Solution method: krylovschur > > Number of requested eigenvalues: 1 > Stopping condition: tol=1e-07, maxit=500 > Number of converged approximate eigenpairs: 0 > > Is krylovschur inappropriate for this problem or have I set up the problem > incorrectly by using MatCreateSeqDense(...) to create the matrix input > files in PETSc binary form? > > Kristen Eisenberg > Billige Fl?ge Marketing GmbH > Emanuelstr. 3, > 10317 Berlin > Deutschland > Telefon: +49 (33) 5310967 > Email: utebachmeier at gmail.com > Site: http://flug.airego.de - Billige Fl?ge vergleichen > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From elam at cs.toronto.edu Tue Sep 6 05:52:31 2011 From: elam at cs.toronto.edu (elam at cs.toronto.edu) Date: Tue, 6 Sep 2011 06:52:31 -0400 Subject: [petsc-users] Matricize a vector Message-ID: <3feafef1b99f48dfb9465aa45740ff26.squirrel@webmail.cs.toronto.edu> Hello, Is it possible to reshape a vector into a matrix in PETSc? I need to do this to compute a matrix vector multiplication (Ax) where the matrix A can be decomposed into Kronecker tensor format. For example, A = A1 kron A2 kron A3, where A1, A2, A3 are smaller sparse matrices. Essentially, I need to matricize the vector x and perform a series of matrix matrix multiplications to compute Ax. In Matlab the function to perform the matricization is reshape. Is there a similar function in PETSc? I could implement the (tensor product) matrix vector multiplication by creating temporary matrices to store each of the intermediate matrix matrix multiplication results. But this seems inefficient. I was wondering if there is a more efficient way to implement this in PETSc? Can I simply reset the size of a matrix (as long as the number of elements remain the same) to reshape it? Any suggestions would be appreciated. Thank you for your help. Best regards, Elizabeth From knepley at gmail.com Tue Sep 6 06:02:48 2011 From: knepley at gmail.com (Matthew Knepley) Date: Tue, 6 Sep 2011 06:02:48 -0500 Subject: [petsc-users] Matricize a vector In-Reply-To: <3feafef1b99f48dfb9465aa45740ff26.squirrel@webmail.cs.toronto.edu> References: <3feafef1b99f48dfb9465aa45740ff26.squirrel@webmail.cs.toronto.edu> Message-ID: On Tue, Sep 6, 2011 at 5:52 AM, wrote: > Hello, > > Is it possible to reshape a vector into a matrix in PETSc? I need to do > this to compute a matrix vector multiplication (Ax) where the matrix A can > be decomposed into Kronecker tensor format. For example, A = A1 kron A2 > kron A3, where A1, A2, A3 are smaller sparse matrices. > > Essentially, I need to matricize the vector x and perform a series of > matrix matrix multiplications to compute Ax. In Matlab the function to > perform the matricization is reshape. Is there a similar function in > PETSc? > 1) This function only makes sense for dense matrices. That is not a huge deal. > I could implement the (tensor product) matrix vector multiplication by > creating temporary matrices to store each of the intermediate matrix > matrix multiplication results. But this seems inefficient. I was > wondering if there is a more efficient way to implement this in PETSc? > Can I simply reset the size of a matrix (as long as the number of elements > remain the same) to reshape it? Any suggestions would be appreciated. > 2) However, it only makes sense in a very limited way in parallel. From your description above, I suspect what you need are calls to VecScatter() to select the part of the vector you need, do a local multiplication which can easily wrap your vector as a dense matrix, and VecScatter back. This is essentially how SpMV is implemented. Thus, I am saying that you will have to implement a MatMult() for your new KroneckerProduct matrix class. You can set MATSHELL and set the mult operation. Thanks, Matt > Thank you for your help. > > Best regards, > > Elizabeth > > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From jedbrown at mcs.anl.gov Tue Sep 6 06:05:18 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Tue, 6 Sep 2011 13:05:18 +0200 Subject: [petsc-users] Matricize a vector In-Reply-To: <3feafef1b99f48dfb9465aa45740ff26.squirrel@webmail.cs.toronto.edu> References: <3feafef1b99f48dfb9465aa45740ff26.squirrel@webmail.cs.toronto.edu> Message-ID: On Tue, Sep 6, 2011 at 12:52, wrote: > Is it possible to reshape a vector into a matrix in PETSc? I need to do > this to compute a matrix vector multiplication (Ax) where the matrix A can > be decomposed into Kronecker tensor format. For example, A = A1 kron A2 > kron A3, where A1, A2, A3 are smaller sparse matrices. > > Essentially, I need to matricize the vector x and perform a series of > matrix matrix multiplications to compute Ax. In Matlab the function to > perform the matricization is reshape. Is there a similar function in > PETSc? > No, but there are two functions you can use to apply this operator. You would likely use both strategies, one for the first and one for the last part of the tensor product. 1. MatCreateMAIJ() creates a matrix that applies the same scalar operation to every component of a "block" in a normal vector. With an MAIJ matrix, MatMult() is semantically like reshaping in row-major order, doing a matrix-matrix multiply, and putting the result back in a flat vector. 2. Use VecGetArray() followed by MatMPIDenseSetPreallocation() to make a matrix in column-major order out of the same memory. Then you can MatMatMult() and make the result go back into a similar matrix. -------------- next part -------------- An HTML attachment was scrubbed... URL: From elam at cs.toronto.edu Tue Sep 6 08:07:06 2011 From: elam at cs.toronto.edu (elam at cs.toronto.edu) Date: Tue, 6 Sep 2011 09:07:06 -0400 Subject: [petsc-users] Matricize a vector In-Reply-To: References: <3feafef1b99f48dfb9465aa45740ff26.squirrel@webmail.cs.toronto.edu> Message-ID: Thank you, the two functions described below are what I need. Are there more examples available on how to use the MAIJ matrix? The MatCreateMAIJ page is very brief. Also, are there example codes for computing a kronecker product in PETSc? > On Tue, Sep 6, 2011 at 12:52, wrote: > >> Is it possible to reshape a vector into a matrix in PETSc? I need to do >> this to compute a matrix vector multiplication (Ax) where the matrix A >> can >> be decomposed into Kronecker tensor format. For example, A = A1 kron A2 >> kron A3, where A1, A2, A3 are smaller sparse matrices. >> >> Essentially, I need to matricize the vector x and perform a series of >> matrix matrix multiplications to compute Ax. In Matlab the function to >> perform the matricization is reshape. Is there a similar function in >> PETSc? >> > > No, but there are two functions you can use to apply this operator. You > would likely use both strategies, one for the first and one for the last > part of the tensor product. > > 1. MatCreateMAIJ() creates a matrix that applies the same scalar operation > to every component of a "block" in a normal vector. With an MAIJ matrix, > MatMult() is semantically like reshaping in row-major order, doing a > matrix-matrix multiply, and putting the result back in a flat vector. > > 2. Use VecGetArray() followed by MatMPIDenseSetPreallocation() to make a > matrix in column-major order out of the same memory. Then you can > MatMatMult() and make the result go back into a similar matrix. > From jedbrown at mcs.anl.gov Tue Sep 6 08:21:28 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Tue, 6 Sep 2011 15:21:28 +0200 Subject: [petsc-users] Matricize a vector In-Reply-To: References: <3feafef1b99f48dfb9465aa45740ff26.squirrel@webmail.cs.toronto.edu> Message-ID: On Tue, Sep 6, 2011 at 15:07, wrote: > Thank you, the two functions described below are what I need. Are there > more examples available on how to use the MAIJ matrix? The MatCreateMAIJ > page is very brief. Also, are there example codes for computing a > kronecker product in PETSc? > Not that I know of. MAIJ is mostly used internally, primarily for multigrid interpolants for systems of equations. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mmnasr at gmail.com Tue Sep 6 19:13:21 2011 From: mmnasr at gmail.com (Mohamad M. Nasr-Azadani) Date: Tue, 6 Sep 2011 17:13:21 -0700 Subject: [petsc-users] Problem with running HDF5 example Message-ID: Hi guys, I was trying to run the example on HDF5, http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-3.1/src/dm/da/examples/tutorials/ex10.c.html But I could not. This is what I get: *************** >>make PETSC_DIR=/home/mmnasr/Softwares/petsc-3.1-p7 PETSC_ARCH=linux-gnu-c-opt ex10 /home/mmnasr/Softwares/petsc-3.1-p7/linux-gnu-c-opt/bin/mpicc -o ex10.o -c -Wall -Wwrite-strings -Wno-strict-aliasing -O -I/home/mmnasr/Softwares/petsc-3.1-p7/linux-gnu-c-opt/include -I/home/mmnasr/Softwares/petsc-3.1-p7/include -I/home/mmnasr/Softwares/petsc-3.1-p7/linux-gnu-c-opt/include -D__INSDIR__=src/dm/da/examples/tutorials/ ex10.c /home/mmnasr/Softwares/petsc-3.1-p7/linux-gnu-c-opt/bin/mpicc -Wall -Wwrite-strings -Wno-strict-aliasing -O -o ex10 ex10.o -Wl,-rpath,/home/mmnasr/Softwares/petsc-3.1-p7/linux-gnu-c-opt/lib -L/home/mmnasr/Softwares/petsc-3.1-p7/linux-gnu-c-opt/lib -lpetsc -Wl,-rpath,/home/mmnasr/Softwares/petsc-3.1-p7/linux-gnu-c-opt/lib -L/home/mmnasr/Softwares/petsc-3.1-p7/linux-gnu-c-opt/lib -lml -lmpichcxx -lstdc++ -lHYPRE -lmpichcxx -lstdc++ -lflapack -lfblas -lnsl -lrt -L/home/mmnasr/Softwares/petsc-3.1-p7/linux-gnu-c-opt/lib -L/usr/lib/gcc/x86_64-linux-gnu/4.4.5 -L/usr/lib/x86_64-linux-gnu -ldl -lmpich -lpthread -lrt -lgcc_s -lmpichf90 -lgfortran -lm -lm -lmpichcxx -lstdc++ -lmpichcxx -lstdc++ -ldl -lmpich -lpthread -lrt -lgcc_s -ldl ex10.o: In function `main': *ex10.c:(.text+0x3b7): undefined reference to `PetscViewerHDF5Open'* collect2: ld returned 1 exit status make: [ex10] Error 1 (ignored) /bin/rm -f -f ex10.o ************ It seems that it does not find the compiled library and/or header files for it. Any ideas? Thanks, Mohamad -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Tue Sep 6 20:16:25 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Tue, 6 Sep 2011 20:16:25 -0500 Subject: [petsc-users] Problem with running HDF5 example In-Reply-To: References: Message-ID: Did you ./configure PETSc with the option --download-hdf5 or the option --with-hdf5-dir=directoryname ? PETSc must be configured with hdf5 in a similar way to that it needs to be configured with hypre to use hypre Barry On Sep 6, 2011, at 7:13 PM, Mohamad M. Nasr-Azadani wrote: > Hi guys, > > I was trying to run the example on HDF5, > http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-3.1/src/dm/da/examples/tutorials/ex10.c.html > > But I could not. > This is what I get: > *************** > >>make PETSC_DIR=/home/mmnasr/Softwares/petsc-3.1-p7 PETSC_ARCH=linux-gnu-c-opt ex10 > > /home/mmnasr/Softwares/petsc-3.1-p7/linux-gnu-c-opt/bin/mpicc -o ex10.o -c -Wall -Wwrite-strings -Wno-strict-aliasing -O -I/home/mmnasr/Softwares/petsc-3.1-p7/linux-gnu-c-opt/include -I/home/mmnasr/Softwares/petsc-3.1-p7/include -I/home/mmnasr/Softwares/petsc-3.1-p7/linux-gnu-c-opt/include -D__INSDIR__=src/dm/da/examples/tutorials/ ex10.c > /home/mmnasr/Softwares/petsc-3.1-p7/linux-gnu-c-opt/bin/mpicc -Wall -Wwrite-strings -Wno-strict-aliasing -O -o ex10 ex10.o -Wl,-rpath,/home/mmnasr/Softwares/petsc-3.1-p7/linux-gnu-c-opt/lib -L/home/mmnasr/Softwares/petsc-3.1-p7/linux-gnu-c-opt/lib -lpetsc -Wl,-rpath,/home/mmnasr/Softwares/petsc-3.1-p7/linux-gnu-c-opt/lib -L/home/mmnasr/Softwares/petsc-3.1-p7/linux-gnu-c-opt/lib -lml -lmpichcxx -lstdc++ -lHYPRE -lmpichcxx -lstdc++ -lflapack -lfblas -lnsl -lrt -L/home/mmnasr/Softwares/petsc-3.1-p7/linux-gnu-c-opt/lib -L/usr/lib/gcc/x86_64-linux-gnu/4.4.5 -L/usr/lib/x86_64-linux-gnu -ldl -lmpich -lpthread -lrt -lgcc_s -lmpichf90 -lgfortran -lm -lm -lmpichcxx -lstdc++ -lmpichcxx -lstdc++ -ldl -lmpich -lpthread -lrt -lgcc_s -ldl > ex10.o: In function `main': > ex10.c:(.text+0x3b7): undefined reference to `PetscViewerHDF5Open' > collect2: ld returned 1 exit status > make: [ex10] Error 1 (ignored) > /bin/rm -f -f ex10.o > > ************ > It seems that it does not find the compiled library and/or header files for it. Any ideas? > > Thanks, > Mohamad > From vishy at stat.purdue.edu Tue Sep 6 20:51:09 2011 From: vishy at stat.purdue.edu (S V N Vishwanathan) Date: Tue, 06 Sep 2011 21:51:09 -0400 Subject: [petsc-users] Problems when saving a large matrix Message-ID: <87wrdlt90y.wl%vishy@stat.purdue.edu> Dear all I have a large matrix in a text file which I am trying to convert and save in PETSc binary format. However I run into some strange MPI errors when I try to save the matrix. I have created a minimal example file to reproduce my problem. I ran it with #PBS -l pmem=16GB #PBS -l select=8:mpiprocs=1:ncpus=1 #PBS -l place=excl:scatter and it crashed when writing the file to disk. Note that if I use smaller values of m in the code then it works reliably. My PETSc installation is compiled with --with-64-bit-indices and I took care to ensure that all arguments to PETSc functions are PetscInt wherever appropriate. Please find code, output, and configure.log attached. Any pointers on what is going wrong are deeply appreciated. thanks vishy -------------- next part -------------- A non-text attachment was scrubbed... Name: test.cpp Type: application/octet-stream Size: 3405 bytes Desc: not available URL: -------------- next part -------------- -------------- next part -------------- A non-text attachment was scrubbed... Name: test.out Type: application/octet-stream Size: 469 bytes Desc: not available URL: -------------- next part -------------- -------------- next part -------------- A non-text attachment was scrubbed... Name: configure.log.bz2 Type: application/octet-stream Size: 123364 bytes Desc: not available URL: -------------- next part -------------- From bsmith at mcs.anl.gov Tue Sep 6 20:57:18 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Tue, 6 Sep 2011 20:57:18 -0500 Subject: [petsc-users] Problems when saving a large matrix In-Reply-To: <87wrdlt90y.wl%vishy@stat.purdue.edu> References: <87wrdlt90y.wl%vishy@stat.purdue.edu> Message-ID: <6903F831-3DDA-4C6C-BE38-64763B59180C@mcs.anl.gov> You will need to switch to the 3.2 release candidate: http://ftp.mcs.anl.gov/pub/petsc/petsc-dev.tar.gz > or via Mecurial at: >> >> ssh://petsc at petsc.cs.iit.edu//hg/petsc/releases/petsc-3.2 or http://petsc.cs.iit.edu/petsc/releases/petsc-3.2 >> ssh://petsc at petsc.cs.iit.edu//hg/petsc/releases/BuildSystem-3.2 or http://petsc.cs.iit.edu/petsc/releases/BuildSystem-3.2 there are bugs in all the MPI implementations for very long buffers that 3.2 works around to allow any size matrix. Barry On Sep 6, 2011, at 8:51 PM, S V N Vishwanathan wrote: > Dear all > > I have a large matrix in a text file which I am trying to convert and > save in PETSc binary format. However I run into some strange MPI errors > when I try to save the matrix. > > I have created a minimal example file to reproduce my problem. I ran it > with > > #PBS -l pmem=16GB > #PBS -l select=8:mpiprocs=1:ncpus=1 > #PBS -l place=excl:scatter > > and it crashed when writing the file to disk. Note that if I use smaller > values of m in the code then it works reliably. > > My PETSc installation is compiled with --with-64-bit-indices and I took > care to ensure that all arguments to PETSc functions are PetscInt > wherever appropriate. Please find code, output, and configure.log > attached. > > Any pointers on what is going wrong are deeply appreciated. > > thanks > > vishy > > > > From mmnasr at gmail.com Tue Sep 6 21:33:44 2011 From: mmnasr at gmail.com (Mohamad M. Nasr-Azadani) Date: Tue, 6 Sep 2011 19:33:44 -0700 Subject: [petsc-users] Problem with running HDF5 example In-Reply-To: References: Message-ID: Thanks Barry. That fixed the problem, i.e. adding --download-hdf5 -M On Tue, Sep 6, 2011 at 6:16 PM, Barry Smith wrote: > > Did you ./configure PETSc with the option --download-hdf5 or the option > --with-hdf5-dir=directoryname ? > > PETSc must be configured with hdf5 in a similar way to that it needs to > be configured with hypre to use hypre > > Barry > > On Sep 6, 2011, at 7:13 PM, Mohamad M. Nasr-Azadani wrote: > > > Hi guys, > > > > I was trying to run the example on HDF5, > > > http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-3.1/src/dm/da/examples/tutorials/ex10.c.html > > > > But I could not. > > This is what I get: > > *************** > > >>make PETSC_DIR=/home/mmnasr/Softwares/petsc-3.1-p7 > PETSC_ARCH=linux-gnu-c-opt ex10 > > > > /home/mmnasr/Softwares/petsc-3.1-p7/linux-gnu-c-opt/bin/mpicc -o ex10.o > -c -Wall -Wwrite-strings -Wno-strict-aliasing -O > -I/home/mmnasr/Softwares/petsc-3.1-p7/linux-gnu-c-opt/include > -I/home/mmnasr/Softwares/petsc-3.1-p7/include > -I/home/mmnasr/Softwares/petsc-3.1-p7/linux-gnu-c-opt/include > -D__INSDIR__=src/dm/da/examples/tutorials/ ex10.c > > /home/mmnasr/Softwares/petsc-3.1-p7/linux-gnu-c-opt/bin/mpicc -Wall > -Wwrite-strings -Wno-strict-aliasing -O -o ex10 ex10.o > -Wl,-rpath,/home/mmnasr/Softwares/petsc-3.1-p7/linux-gnu-c-opt/lib > -L/home/mmnasr/Softwares/petsc-3.1-p7/linux-gnu-c-opt/lib -lpetsc > -Wl,-rpath,/home/mmnasr/Softwares/petsc-3.1-p7/linux-gnu-c-opt/lib > -L/home/mmnasr/Softwares/petsc-3.1-p7/linux-gnu-c-opt/lib -lml -lmpichcxx > -lstdc++ -lHYPRE -lmpichcxx -lstdc++ -lflapack -lfblas -lnsl -lrt > -L/home/mmnasr/Softwares/petsc-3.1-p7/linux-gnu-c-opt/lib > -L/usr/lib/gcc/x86_64-linux-gnu/4.4.5 -L/usr/lib/x86_64-linux-gnu -ldl > -lmpich -lpthread -lrt -lgcc_s -lmpichf90 -lgfortran -lm -lm -lmpichcxx > -lstdc++ -lmpichcxx -lstdc++ -ldl -lmpich -lpthread -lrt -lgcc_s -ldl > > ex10.o: In function `main': > > ex10.c:(.text+0x3b7): undefined reference to `PetscViewerHDF5Open' > > collect2: ld returned 1 exit status > > make: [ex10] Error 1 (ignored) > > /bin/rm -f -f ex10.o > > > > ************ > > It seems that it does not find the compiled library and/or header files > for it. Any ideas? > > > > Thanks, > > Mohamad > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From aeronova.mailing at gmail.com Tue Sep 6 22:17:58 2011 From: aeronova.mailing at gmail.com (Kyunghoon Lee) Date: Wed, 7 Sep 2011 11:17:58 +0800 Subject: [petsc-users] compilation with mumps on Mac OS 10.6.8 Message-ID: Hello all, I need to compile petsc with mumps. First I tried ./configure --prefix=/Users/aeronova/Development/local/lib64/petsc/petsc-3.1-p8 --download-c-blas-lapack=1 ?download-parmetis=1 --download-scalapack=1 --download-mumps=1 but I got the following message: Fortran error! mpif.h could not be located at: [] So I included "-download-mpich=1," then I got the following: Should request f-blas-lapack, not --download-c-blas-lapack=yes since you have a fortran compiler? After that, I tried several options with/without --download-f-blas-lapack or --download-mpich=1, but none of them worked out. I'd appreciate it if someone can help me with this compilation problem with mumps. Regards, K. Lee. p.s. I do not need to compile petsc for FORTRAN. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Tue Sep 6 22:21:52 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Tue, 6 Sep 2011 22:21:52 -0500 Subject: [petsc-users] compilation with mumps on Mac OS 10.6.8 In-Reply-To: References: Message-ID: <53E590B7-A59C-4DA2-95FE-A54C1A9531AA@mcs.anl.gov> > p.s. > I do not need to compile petsc for FORTRAN. Sadly you do because the packages MUMPS uses are all Fortran. So you will need a real Fortran compiler (perhaps gfortran) and you will also need --download-blacs and you cannot use --download-c-blas-lapack with scalapack. Note you can use SuperLU_Dist instead of MUMPS without a Fortran compiler. Barry On Sep 6, 2011, at 10:17 PM, Kyunghoon Lee wrote: > Hello all, > > I need to compile petsc with mumps. First I tried > > ./configure --prefix=/Users/aeronova/Development/local/lib64/petsc/petsc-3.1-p8 --download-c-blas-lapack=1 ?download-parmetis=1 --download-scalapack=1 --download-mumps=1 > > but I got the following message: > > Fortran error! mpif.h could not be located at: [] > > So I included "-download-mpich=1," then I got the following: > > Should request f-blas-lapack, not --download-c-blas-lapack=yes since you have a fortran compiler? > > After that, I tried several options with/without --download-f-blas-lapack or --download-mpich=1, but none of them worked out. I'd appreciate it if someone can help me with this compilation problem with mumps. > > Regards, > K. Lee. > > p.s. > I do not need to compile petsc for FORTRAN. > From vishy at stat.purdue.edu Tue Sep 6 22:22:56 2011 From: vishy at stat.purdue.edu (S V N Vishwanathan) Date: Tue, 06 Sep 2011 23:22:56 -0400 Subject: [petsc-users] Still no luck Message-ID: <87sjo9t4rz.wl%vishy@stat.purdue.edu> I updated my code to compile with the latest development branch and ran it. Now I get the following error: Writing data in binary format to /scratch/lustreA/v/vishy/temp/Binary/biclass/test.txt [0]PETSC ERROR: MatView_MPIAIJ_Binary() line 1369 in src/mat/impls/aij/mpi/mpiaij.c [0]PETSC ERROR: MatView_MPIAIJ_ASCIIorDraworSocket() line 1479 in src/mat/impls/aij/mpi/mpiaij.c [0]PETSC ERROR: MatView_MPIAIJ() line 1575 in src/mat/impls/aij/mpi/mpiaij.c [0]PETSC ERROR: MatView() line 755 in src/mat/interface/matrix.c [0]PETSC ERROR: main() line 93 in test.cpp application called MPI_Abort(MPI_COMM_WORLD, 1) - process 0 [1]PETSC ERROR: MatView_MPIAIJ_Binary() line 1376 in src/mat/impls/aij/mpi/mpiaij.c [1]PETSC ERROR: MatView_MPIAIJ_ASCIIorDraworSocket() line 1479 in src/mat/impls/aij/mpi/mpiaij.c [1]PETSC ERROR: MatView_MPIAIJ() line 1575 in src/mat/impls/aij/mpi/mpiaij.c [1]PETSC ERROR: MatView() line 755 in src/mat/interface/matrix.c [1]PETSC ERROR: main() line 93 in test.cpp application called MPI_Abort(MPI_COMM_WORLD, 1) - process 1 APPLICATION TERMINATED WITH THE EXIT STRING: Hangup (signal 1) vishy From aeronova.mailing at gmail.com Tue Sep 6 22:41:38 2011 From: aeronova.mailing at gmail.com (Kyunghoon Lee) Date: Wed, 7 Sep 2011 11:41:38 +0800 Subject: [petsc-users] compilation with mumps on Mac OS 10.6.8 In-Reply-To: <53E590B7-A59C-4DA2-95FE-A54C1A9531AA@mcs.anl.gov> References: <53E590B7-A59C-4DA2-95FE-A54C1A9531AA@mcs.anl.gov> Message-ID: Hi Barry, Thanks for the reply. I did install g95 through macports. (I could not find gfortran.) I wonder g95 can do the job for me. Re SuperLU_Dist, all I know is I need mumps to deal with complex numbers. K. Lee. On Wed, Sep 7, 2011 at 11:21 AM, Barry Smith wrote: > > > p.s. > > I do not need to compile petsc for FORTRAN. > > Sadly you do because the packages MUMPS uses are all Fortran. > > So you will need a real Fortran compiler (perhaps gfortran) and you will > also need --download-blacs and you cannot use --download-c-blas-lapack with > scalapack. > > Note you can use SuperLU_Dist instead of MUMPS without a Fortran > compiler. > > Barry > > On Sep 6, 2011, at 10:17 PM, Kyunghoon Lee wrote: > > > Hello all, > > > > I need to compile petsc with mumps. First I tried > > > > ./configure > --prefix=/Users/aeronova/Development/local/lib64/petsc/petsc-3.1-p8 > --download-c-blas-lapack=1 ?download-parmetis=1 --download-scalapack=1 > --download-mumps=1 > > > > but I got the following message: > > > > Fortran error! mpif.h could not be located at: [] > > > > So I included "-download-mpich=1," then I got the following: > > > > Should request f-blas-lapack, not --download-c-blas-lapack=yes since you > have a fortran compiler? > > > > After that, I tried several options with/without --download-f-blas-lapack > or --download-mpich=1, but none of them worked out. I'd appreciate it if > someone can help me with this compilation problem with mumps. > > > > Regards, > > K. Lee. > > > > p.s. > > I do not need to compile petsc for FORTRAN. > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Tue Sep 6 22:46:35 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Tue, 6 Sep 2011 22:46:35 -0500 Subject: [petsc-users] Still no luck In-Reply-To: <87sjo9t4rz.wl%vishy@stat.purdue.edu> References: <87sjo9t4rz.wl%vishy@stat.purdue.edu> Message-ID: On Sep 6, 2011, at 10:22 PM, S V N Vishwanathan wrote: > I updated my code to compile with the latest development branch and ran > it. Now I get the following error: > I think I see the problem. We fixed MatLoad() to handle the very large matrices with 64 bit indices but not the MatView(). I have updated both the 3.2 release candidate and petsc-dev to handle this case. But you need to use Mecurial to get the updates and cannot use the tarball. If you have used Mecurial then all you need do is; hg pull; hg update; cd src/mat/impls/aij/mpi; make to update the libraries Please let me know if this does not resolve the problem. barry > Writing data in binary format to /scratch/lustreA/v/vishy/temp/Binary/biclass/test.txt > [0]PETSC ERROR: MatView_MPIAIJ_Binary() line 1369 in src/mat/impls/aij/mpi/mpiaij.c > [0]PETSC ERROR: MatView_MPIAIJ_ASCIIorDraworSocket() line 1479 in src/mat/impls/aij/mpi/mpiaij.c > [0]PETSC ERROR: MatView_MPIAIJ() line 1575 in src/mat/impls/aij/mpi/mpiaij.c > [0]PETSC ERROR: MatView() line 755 in src/mat/interface/matrix.c > [0]PETSC ERROR: main() line 93 in test.cpp > application called MPI_Abort(MPI_COMM_WORLD, 1) - process 0 > [1]PETSC ERROR: MatView_MPIAIJ_Binary() line 1376 in src/mat/impls/aij/mpi/mpiaij.c > [1]PETSC ERROR: MatView_MPIAIJ_ASCIIorDraworSocket() line 1479 in src/mat/impls/aij/mpi/mpiaij.c > [1]PETSC ERROR: MatView_MPIAIJ() line 1575 in src/mat/impls/aij/mpi/mpiaij.c > [1]PETSC ERROR: MatView() line 755 in src/mat/interface/matrix.c > [1]PETSC ERROR: main() line 93 in test.cpp > application called MPI_Abort(MPI_COMM_WORLD, 1) - process 1 > APPLICATION TERMINATED WITH THE EXIT STRING: Hangup (signal 1) > > > vishy From bsmith at mcs.anl.gov Tue Sep 6 22:49:10 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Tue, 6 Sep 2011 22:49:10 -0500 Subject: [petsc-users] compilation with mumps on Mac OS 10.6.8 In-Reply-To: References: <53E590B7-A59C-4DA2-95FE-A54C1A9531AA@mcs.anl.gov> Message-ID: <719F1C8A-A89A-425E-A347-A0C0EC77A2B4@mcs.anl.gov> On Sep 6, 2011, at 10:41 PM, Kyunghoon Lee wrote: > Hi Barry, > > Thanks for the reply. I did install g95 through macports. (I could not find gfortran.) I wonder g95 can do the job for me. > g95 might work, give it a try. BTW: on Macs you don't need to provide blas/lapack with --download; Apple provides them on the system. > Re SuperLU_Dist, all I know is I need mumps to deal with complex numbers. SuperLU_Dist can also be used with complex numbers (at least with PETSc-dev/petsc-3.2) > > K. Lee. > > On Wed, Sep 7, 2011 at 11:21 AM, Barry Smith wrote: > > > p.s. > > I do not need to compile petsc for FORTRAN. > > Sadly you do because the packages MUMPS uses are all Fortran. > > So you will need a real Fortran compiler (perhaps gfortran) and you will also need --download-blacs and you cannot use --download-c-blas-lapack with scalapack. > > Note you can use SuperLU_Dist instead of MUMPS without a Fortran compiler. > > Barry > > On Sep 6, 2011, at 10:17 PM, Kyunghoon Lee wrote: > > > Hello all, > > > > I need to compile petsc with mumps. First I tried > > > > ./configure --prefix=/Users/aeronova/Development/local/lib64/petsc/petsc-3.1-p8 --download-c-blas-lapack=1 ?download-parmetis=1 --download-scalapack=1 --download-mumps=1 > > > > but I got the following message: > > > > Fortran error! mpif.h could not be located at: [] > > > > So I included "-download-mpich=1," then I got the following: > > > > Should request f-blas-lapack, not --download-c-blas-lapack=yes since you have a fortran compiler? > > > > After that, I tried several options with/without --download-f-blas-lapack or --download-mpich=1, but none of them worked out. I'd appreciate it if someone can help me with this compilation problem with mumps. > > > > Regards, > > K. Lee. > > > > p.s. > > I do not need to compile petsc for FORTRAN. > > > > From aeronova.mailing at gmail.com Tue Sep 6 23:04:38 2011 From: aeronova.mailing at gmail.com (Kyunghoon Lee) Date: Wed, 7 Sep 2011 12:04:38 +0800 Subject: [petsc-users] compilation with mumps on Mac OS 10.6.8 In-Reply-To: <719F1C8A-A89A-425E-A347-A0C0EC77A2B4@mcs.anl.gov> References: <53E590B7-A59C-4DA2-95FE-A54C1A9531AA@mcs.anl.gov> <719F1C8A-A89A-425E-A347-A0C0EC77A2B4@mcs.anl.gov> Message-ID: Thanks for the help. Now I finished the installation successfully. :) K. Lee. On Wed, Sep 7, 2011 at 11:49 AM, Barry Smith wrote: > > On Sep 6, 2011, at 10:41 PM, Kyunghoon Lee wrote: > > > Hi Barry, > > > > Thanks for the reply. I did install g95 through macports. (I could not > find gfortran.) I wonder g95 can do the job for me. > > > g95 might work, give it a try. BTW: on Macs you don't need to provide > blas/lapack with --download; Apple provides them on the system. > > > > > Re SuperLU_Dist, all I know is I need mumps to deal with complex numbers. > > SuperLU_Dist can also be used with complex numbers (at least with > PETSc-dev/petsc-3.2) > > > > > K. Lee. > > > > On Wed, Sep 7, 2011 at 11:21 AM, Barry Smith wrote: > > > > > p.s. > > > I do not need to compile petsc for FORTRAN. > > > > Sadly you do because the packages MUMPS uses are all Fortran. > > > > So you will need a real Fortran compiler (perhaps gfortran) and you > will also need --download-blacs and you cannot use --download-c-blas-lapack > with scalapack. > > > > Note you can use SuperLU_Dist instead of MUMPS without a Fortran > compiler. > > > > Barry > > > > On Sep 6, 2011, at 10:17 PM, Kyunghoon Lee wrote: > > > > > Hello all, > > > > > > I need to compile petsc with mumps. First I tried > > > > > > ./configure > --prefix=/Users/aeronova/Development/local/lib64/petsc/petsc-3.1-p8 > --download-c-blas-lapack=1 ?download-parmetis=1 --download-scalapack=1 > --download-mumps=1 > > > > > > but I got the following message: > > > > > > Fortran error! mpif.h could not be located at: [] > > > > > > So I included "-download-mpich=1," then I got the following: > > > > > > Should request f-blas-lapack, not --download-c-blas-lapack=yes since > you have a fortran compiler? > > > > > > After that, I tried several options with/without > --download-f-blas-lapack or --download-mpich=1, but none of them worked out. > I'd appreciate it if someone can help me with this compilation problem with > mumps. > > > > > > Regards, > > > K. Lee. > > > > > > p.s. > > > I do not need to compile petsc for FORTRAN. > > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From stirukkovalur at rnet-tech.com Tue Sep 6 22:05:26 2011 From: stirukkovalur at rnet-tech.com (Sravya Tirukkovalur) Date: Tue, 6 Sep 2011 23:05:26 -0400 Subject: [petsc-users] Build Petsc with single precision Message-ID: Hi, I am trying to build petsc with single precision. The options I am using are: ./configure options --download-f-blas-lapack=1 --with-x=0 --with-debugging=1 --CFLAGS="-O3 -g -ggdb" --FFLAGS="-O3 -g" --with-hdf5=1 --download-hdf5=1 --with-batch=0 --known-mpi-shared-libraries=no --with-cuda=1 --with-cudac="nvcc -m64" --with-cusp=1 --with-thrust=1 PETSC_ARCH=SOMETHING Configuration step is being completed successfully. And while doing $make all, it compiles a few source files and prints an error message which says cannot build petsc with this options, contact the mailing list. Did anyone face a similar problem while building petsc with single precision or has anyone successfully built it? Thanks -- ~Sravya -------------- next part -------------- An HTML attachment was scrubbed... URL: From balay at mcs.anl.gov Tue Sep 6 23:11:18 2011 From: balay at mcs.anl.gov (Satish Balay) Date: Tue, 6 Sep 2011 23:11:18 -0500 (CDT) Subject: [petsc-users] Build Petsc with single precision In-Reply-To: References: Message-ID: On Tue, 6 Sep 2011, Sravya Tirukkovalur wrote: > Hi, > > I am trying to build petsc with single precision. The options I am using > are: > > ./configure options --download-f-blas-lapack=1 --with-x=0 --with-debugging=1 > --CFLAGS="-O3 -g -ggdb" --FFLAGS="-O3 -g" --with-hdf5=1 --download-hdf5=1 > --with-batch=0 --known-mpi-shared-libraries=no --with-cuda=1 > --with-cudac="nvcc -m64" --with-cusp=1 --with-thrust=1 PETSC_ARCH=SOMETHING > > Configuration step is being completed successfully. And while doing $make > all, it compiles a few source files and prints an error message which says > cannot build petsc with this options, contact the mailing list. Perhaps it said petsc-maint - not petsc-users. Please send relavent logs to petsc-maint Satish > > Did anyone face a similar problem while building petsc with single precision > or has anyone successfully built it? From balay at mcs.anl.gov Tue Sep 6 23:16:49 2011 From: balay at mcs.anl.gov (Satish Balay) Date: Tue, 6 Sep 2011 23:16:49 -0500 (CDT) Subject: [petsc-users] Still no luck In-Reply-To: References: <87sjo9t4rz.wl%vishy@stat.purdue.edu> Message-ID: On Tue, 6 Sep 2011, Barry Smith wrote: > > On Sep 6, 2011, at 10:22 PM, S V N Vishwanathan wrote: > > > I updated my code to compile with the latest development branch and ran > > it. Now I get the following error: > > > I think I see the problem. We fixed MatLoad() to handle the very large matrices with 64 bit indices but not the MatView(). > > I have updated both the 3.2 release candidate and petsc-dev to handle this case. But you need to use Mecurial to get the updates and cannot use the tarball. If tarball is preferable - one can use http://ftp.mcs.anl.gov/pub/petsc/petsc-dev.tar.gz. It should get updated every 3 hours with petsc32 tree. [for atleast the next couple of days..] Satish > > If you have used Mecurial then all you need do is; hg pull; hg update; cd src/mat/impls/aij/mpi; make to update the libraries > > Please let me know if this does not resolve the problem. > > barry > > > > > > Writing data in binary format to /scratch/lustreA/v/vishy/temp/Binary/biclass/test.txt > > [0]PETSC ERROR: MatView_MPIAIJ_Binary() line 1369 in src/mat/impls/aij/mpi/mpiaij.c > > [0]PETSC ERROR: MatView_MPIAIJ_ASCIIorDraworSocket() line 1479 in src/mat/impls/aij/mpi/mpiaij.c > > [0]PETSC ERROR: MatView_MPIAIJ() line 1575 in src/mat/impls/aij/mpi/mpiaij.c > > [0]PETSC ERROR: MatView() line 755 in src/mat/interface/matrix.c > > [0]PETSC ERROR: main() line 93 in test.cpp > > application called MPI_Abort(MPI_COMM_WORLD, 1) - process 0 > > [1]PETSC ERROR: MatView_MPIAIJ_Binary() line 1376 in src/mat/impls/aij/mpi/mpiaij.c > > [1]PETSC ERROR: MatView_MPIAIJ_ASCIIorDraworSocket() line 1479 in src/mat/impls/aij/mpi/mpiaij.c > > [1]PETSC ERROR: MatView_MPIAIJ() line 1575 in src/mat/impls/aij/mpi/mpiaij.c > > [1]PETSC ERROR: MatView() line 755 in src/mat/interface/matrix.c > > [1]PETSC ERROR: main() line 93 in test.cpp > > application called MPI_Abort(MPI_COMM_WORLD, 1) - process 1 > > APPLICATION TERMINATED WITH THE EXIT STRING: Hangup (signal 1) > > > > > > vishy > > From ckontzialis at lycos.com Wed Sep 7 01:59:33 2011 From: ckontzialis at lycos.com (Kostas Kontzialis) Date: Wed, 07 Sep 2011 09:59:33 +0300 Subject: [petsc-users] Coloring of a parallel matrix Message-ID: <4E671655.5050100@lycos.com> Dear all, I'm using a matrix free method for the solution of the Navier-Stokes equaiotns on unstructured meshes. I want to numerically evaluate the elements of the residual's Jacobian matrix. I know where the non zero elements of the matrix are, but I read on the manual, that function MatGetColoring works only on sequential matrices. What should I do? Kostas From jedbrown at mcs.anl.gov Wed Sep 7 03:00:15 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Wed, 7 Sep 2011 10:00:15 +0200 Subject: [petsc-users] Coloring of a parallel matrix In-Reply-To: <4E671655.5050100@lycos.com> References: <4E671655.5050100@lycos.com> Message-ID: On Wed, Sep 7, 2011 at 08:59, Kostas Kontzialis wrote: > I'm using a matrix free method for the solution of the Navier-Stokes > equaiotns on unstructured meshes. I want to numerically evaluate the > elements of the residual's Jacobian matrix. I know where the non zero > elements of the matrix are, but I read on the manual, that function > MatGetColoring works only on sequential matrices. What should I do? It actually works for parallel matrices, but the underlying algorithm is serial. You can use it for tests and small problems, but it does not scale. PETSc does not currently have a parallel graph coloring implementation, so you would have to compute a coloring yourself if you intend to use this for larger problem sizes. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Wed Sep 7 07:29:04 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Wed, 7 Sep 2011 07:29:04 -0500 Subject: [petsc-users] Coloring of a parallel matrix In-Reply-To: References: <4E671655.5050100@lycos.com> Message-ID: On Sep 7, 2011, at 3:00 AM, Jed Brown wrote: > On Wed, Sep 7, 2011 at 08:59, Kostas Kontzialis wrote: > I'm using a matrix free method for the solution of the Navier-Stokes equaiotns on unstructured meshes. I want to numerically evaluate the elements of the residual's Jacobian matrix. I know where the non zero elements of the matrix are, but I read on the manual, that function MatGetColoring works only on sequential matrices. What should I do? > > It actually works for parallel matrices, but the underlying algorithm is serial. You can use it for tests and small problems, but it does not scale. Even though it does not scale for super large problems the algorithms are pretty fast so you can likely run parallel problems with matrices up to a few million (sparse) rows and it will likely run satisfactory. Barry > > PETSc does not currently have a parallel graph coloring implementation, so you would have to compute a coloring yourself if you intend to use this for larger problem sizes. From m.lysaght at qub.ac.uk Wed Sep 7 05:38:31 2011 From: m.lysaght at qub.ac.uk (Micheal Lysaght) Date: Wed, 07 Sep 2011 11:38:31 +0100 Subject: [petsc-users] petsc with slepc Message-ID: <4E6749A7.6050604@qub.ac.uk> Dear PETSc team, I am currently using PETSc with SLEPc to diagonalize a sparse Hermitian matrix. I have had success with doing so, so long as I create the PETSc matrix object using MatCreateMPISBAIJ with PETSC_DECIDE used for both the number of rows and columns. Upon assembling the matrix object, I then call the relevant SLEPc EPS routines (where the default Krylov Schur method is invoked) Doing this, everything works fine and the correct eigenvalues are returned. However, if I try to do the same thing with the difference that I decide the number of rows and columns assigned to each MPI process during the matrix creation, SLEPc returns an error message with regard to the dimensions being used. I have checked that the matrix object has the correct form and values by viewing the matrix, so it is certainly assembled in the correct way using PETSc. My question is, can SLEPc handle matrices that are constructed according to the users choice of the numbers of rows and columns assigned to each MPI process. I realize that this would need to be accounted for in the vector that is being multiplied into, but how do I get access to this vector in SLEPc? Or do I even need to? Any help or tips would be very much appreciated. Best regards, Michael Lysaght Visiting Research Fellow, CTAMOP, Queen's University Belfast From jroman at dsic.upv.es Wed Sep 7 09:40:41 2011 From: jroman at dsic.upv.es (Jose E. Roman) Date: Wed, 7 Sep 2011 16:40:41 +0200 Subject: [petsc-users] petsc with slepc In-Reply-To: <4E6749A7.6050604@qub.ac.uk> References: <4E6749A7.6050604@qub.ac.uk> Message-ID: On 07/09/2011, Micheal Lysaght wrote: > > Dear PETSc team, > > I am currently using PETSc with SLEPc to diagonalize a sparse Hermitian matrix. I have had success with doing so, so long as I create the PETSc matrix object using > > MatCreateMPISBAIJ > > with PETSC_DECIDE used for both the number of rows and columns. > > Upon assembling the matrix object, I then call the relevant SLEPc EPS routines (where the default Krylov Schur method is invoked) > > Doing this, everything works fine and the correct eigenvalues are returned. > > However, if I try to do the same thing with the difference that I decide the number of rows and columns assigned to each MPI process during the matrix creation, SLEPc returns an error message with regard to the dimensions being used. > > I have checked that the matrix object has the correct form and values by viewing the matrix, so it is certainly assembled in the correct way using PETSc. > > My question is, can SLEPc handle matrices that are constructed according to the users choice of the numbers of rows and columns assigned to each MPI process. I realize that this would need to be accounted for in the vector that is being multiplied into, but how do I get access to this vector in SLEPc? Or do I even need to? > > Any help or tips would be very much appreciated. > > Best regards, > > Michael Lysaght > Visiting Research Fellow, > CTAMOP, > Queen's University Belfast Send the complete error message. Also, if you can reproduce the problem in a simple example, send it to slepc-maint at grycap.upv.es and we will give it a try. Jose From ckontzialis at lycos.com Wed Sep 7 11:41:16 2011 From: ckontzialis at lycos.com (Kostas Kontzialis) Date: Wed, 07 Sep 2011 19:41:16 +0300 Subject: [petsc-users] petsc-users Digest, Vol 33, Issue 13 In-Reply-To: References: Message-ID: <4E679EAC.1060906@lycos.com> On 09/07/2011 04:53 PM, petsc-users-request at mcs.anl.gov wrote: > Send petsc-users mailing list submissions to > petsc-users at mcs.anl.gov > > To subscribe or unsubscribe via the World Wide Web, visit > https://lists.mcs.anl.gov/mailman/listinfo/petsc-users > or, via email, send a message with subject or body 'help' to > petsc-users-request at mcs.anl.gov > > You can reach the person managing the list at > petsc-users-owner at mcs.anl.gov > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of petsc-users digest..." > > > Today's Topics: > > 1. Re: compilation with mumps on Mac OS 10.6.8 (Kyunghoon Lee) > 2. Build Petsc with single precision (Sravya Tirukkovalur) > 3. Re: Build Petsc with single precision (Satish Balay) > 4. Re: Still no luck (Satish Balay) > 5. Coloring of a parallel matrix (Kostas Kontzialis) > 6. Re: Coloring of a parallel matrix (Jed Brown) > 7. Re: Coloring of a parallel matrix (Barry Smith) > 8. petsc with slepc (Micheal Lysaght) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Wed, 7 Sep 2011 12:04:38 +0800 > From: Kyunghoon Lee > Subject: Re: [petsc-users] compilation with mumps on Mac OS 10.6.8 > To: PETSc users list > Message-ID: > > Content-Type: text/plain; charset="windows-1252" > > Thanks for the help. Now I finished the installation successfully. :) > > K. Lee. > > On Wed, Sep 7, 2011 at 11:49 AM, Barry Smith wrote: > >> On Sep 6, 2011, at 10:41 PM, Kyunghoon Lee wrote: >> >>> Hi Barry, >>> >>> Thanks for the reply. I did install g95 through macports. (I could not >> find gfortran.) I wonder g95 can do the job for me. >> g95 might work, give it a try. BTW: on Macs you don't need to provide >> blas/lapack with --download; Apple provides them on the system. >> >> >> >>> Re SuperLU_Dist, all I know is I need mumps to deal with complex numbers. >> SuperLU_Dist can also be used with complex numbers (at least with >> PETSc-dev/petsc-3.2) >> >>> K. Lee. >>> >>> On Wed, Sep 7, 2011 at 11:21 AM, Barry Smith wrote: >>> >>>> p.s. >>>> I do not need to compile petsc for FORTRAN. >>> Sadly you do because the packages MUMPS uses are all Fortran. >>> >>> So you will need a real Fortran compiler (perhaps gfortran) and you >> will also need --download-blacs and you cannot use --download-c-blas-lapack >> with scalapack. >>> Note you can use SuperLU_Dist instead of MUMPS without a Fortran >> compiler. >>> Barry >>> >>> On Sep 6, 2011, at 10:17 PM, Kyunghoon Lee wrote: >>> >>>> Hello all, >>>> >>>> I need to compile petsc with mumps. First I tried >>>> >>>> ./configure >> --prefix=/Users/aeronova/Development/local/lib64/petsc/petsc-3.1-p8 >> --download-c-blas-lapack=1 ?download-parmetis=1 --download-scalapack=1 >> --download-mumps=1 >>>> but I got the following message: >>>> >>>> Fortran error! mpif.h could not be located at: [] >>>> >>>> So I included "-download-mpich=1," then I got the following: >>>> >>>> Should request f-blas-lapack, not --download-c-blas-lapack=yes since >> you have a fortran compiler? >>>> After that, I tried several options with/without >> --download-f-blas-lapack or --download-mpich=1, but none of them worked out. >> I'd appreciate it if someone can help me with this compilation problem with >> mumps. >>>> Regards, >>>> K. Lee. >>>> >>>> p.s. >>>> I do not need to compile petsc for FORTRAN. >>>> >>> >> > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > > ------------------------------ > > Message: 2 > Date: Tue, 6 Sep 2011 23:05:26 -0400 > From: Sravya Tirukkovalur > Subject: [petsc-users] Build Petsc with single precision > To: petsc-users at mcs.anl.gov > Message-ID: > > Content-Type: text/plain; charset="iso-8859-1" > > Hi, > > I am trying to build petsc with single precision. The options I am using > are: > > ./configure options --download-f-blas-lapack=1 --with-x=0 --with-debugging=1 > --CFLAGS="-O3 -g -ggdb" --FFLAGS="-O3 -g" --with-hdf5=1 --download-hdf5=1 > --with-batch=0 --known-mpi-shared-libraries=no --with-cuda=1 > --with-cudac="nvcc -m64" --with-cusp=1 --with-thrust=1 PETSC_ARCH=SOMETHING > > Configuration step is being completed successfully. And while doing $make > all, it compiles a few source files and prints an error message which says > cannot build petsc with this options, contact the mailing list. > > Did anyone face a similar problem while building petsc with single precision > or has anyone successfully built it? > > Thanks > Barry, I have the following code for implementing the setting up snes and the numerical evaluation of the jacobian: ierr = jacobian_matrix_nz_pattern(sys); CHKERRQ(ierr); ierr = MatCreateMPIBAIJ(sys.comm, sys.num->nbq[sys.con->n], sys.ldof, sys.ldof, sys.gdof, sys.gdof, PETSC_NULL, sys.idxm, PETSC_NULL, sys.idxn, &sys.P); CHKERRQ(ierr); ierr = SNESCreate(sys.comm, &sys.snes); CHKERRQ(ierr); ierr = SNESSetFunction(sys.snes, sys.gres[0], base_residual_implicit, &sys); CHKERRQ(ierr); ierr = MatCreateSNESMF(sys.snes, &sys.J); CHKERRQ(ierr); ierr = MatMFFDSetFromOptions(sys.J); CHKERRQ(ierr); ierr = jacobian_diff_numerical(sys, &sys.P); CHKERRQ(ierr); ISColoring iscoloring; MatFDColoring fdcoloring; MatGetColoring(sys.P, MATCOLORING_SL, &iscoloring); MatFDColoringCreate(sys.P, iscoloring, &fdcoloring); ISColoringDestroy(iscoloring); MatFDColoringSetFromOptions(fdcoloring); ierr = SNESSetJacobian(sys.snes, sys.J, sys.P, SNESDefaultComputeJacobianColor, fdcoloring); CHKERRQ(ierr); As you can see I'm using a matrix free algorithm, However, when I run the code with with the -snes_mf_operator option I get on return the following error: Timestep 0: dt = 0.008, T = 0, Res[rho] = 0, Res[rhou] = 0, Res[rhov] = 0, Res[E] = 0, CFL = 0.177549 /*********************Stage 1 of SSPIRK (3,4)******************/ 0 SNES Function norm 2.755099585674e+01 [4]PETSC ERROR: --------------------- Error Message ------------------------------------ [4]PETSC ERROR: Object is in wrong state! [4]PETSC ERROR: Must call MatFDColoringSetFunction()! I haven't use the MatFDColoringSetFunction. But when I try to put it I read on the help page: Notes: This function is usually used automatically by SNES or TS (when one uses SNESSetJacobian () with the argument SNESDefaultComputeJacobianColor () or TSSetRHSJacobian () with the argument TSDefaultComputeJacobianColor ()) and only needs to be used by someone computing a matrix via coloring directly by calling MatFDColoringApply () and furthermore I cannot figure out what the arguments are for the (*f) function. Any help? Kostas -------------- next part -------------- An HTML attachment was scrubbed... URL: From amrit_pou at hotmail.com Wed Sep 7 12:05:16 2011 From: amrit_pou at hotmail.com (amrit poudel) Date: Wed, 7 Sep 2011 17:05:16 +0000 Subject: [petsc-users] petsc-users Digest, Vol 33, Issue 13 In-Reply-To: <4E679EAC.1060906@lycos.com> References: , <4E679EAC.1060906@lycos.com> Message-ID: Hello PETSc users, I am trying to compile PETSC with MUMPS, but I am not sure how I compile MUMPS libraries in MAC OSX 10.6.8 . I know that I need BLACS and SCALAPACK libraries, but I am having trouble understanding how to compile these libraries in MAC OSX 10.6.8. I saw that some users here were able to do this in MAC OSX 10.6.8. Would any one of you who have successfully compiled these libraries in MAC OSX 10.6.8 mind sharing their MAKE FILES ? I will truly appreciate your help. I am sorry if it is inappropriate to ask for a make file. It is just that I am not understanding where to start from. NOTE : I already have PETSc compiled and installed and is running fine (in Intel core 2 duo MAC OSX 10.6.8), but I am in need of an efficient direct solver (that can run in a multiprocessor computer, or in a cluster with distributed or shared memory), and surprisingly, PETSc does not come with any default direct solver package, unlike an iterative solver. I was also wondering whether UMFPACK can be used for solving an extremely large sparse system of linear equations in a multiprocessor computer or in a cluster with distributed or shared memory. SuperLU_Dist or SuperLU_MT installation seems more confounding than any other packages, so I am trying to stay away from them. Thanks for any help, and sorry if my request for make file is inappropriate. -Amrit Date: Wed, 7 Sep 2011 19:41:16 +0300 From: ckontzialis at lycos.com To: petsc-users at mcs.anl.gov Subject: Re: [petsc-users] petsc-users Digest, Vol 33, Issue 13 On 09/07/2011 04:53 PM, petsc-users-request at mcs.anl.gov wrote: Send petsc-users mailing list submissions to petsc-users at mcs.anl.gov To subscribe or unsubscribe via the World Wide Web, visit https://lists.mcs.anl.gov/mailman/listinfo/petsc-users or, via email, send a message with subject or body 'help' to petsc-users-request at mcs.anl.gov You can reach the person managing the list at petsc-users-owner at mcs.anl.gov When replying, please edit your Subject line so it is more specific than "Re: Contents of petsc-users digest..." Today's Topics: 1. Re: compilation with mumps on Mac OS 10.6.8 (Kyunghoon Lee) 2. Build Petsc with single precision (Sravya Tirukkovalur) 3. Re: Build Petsc with single precision (Satish Balay) 4. Re: Still no luck (Satish Balay) 5. Coloring of a parallel matrix (Kostas Kontzialis) 6. Re: Coloring of a parallel matrix (Jed Brown) 7. Re: Coloring of a parallel matrix (Barry Smith) 8. petsc with slepc (Micheal Lysaght) ---------------------------------------------------------------------- Message: 1 Date: Wed, 7 Sep 2011 12:04:38 +0800 From: Kyunghoon Lee Subject: Re: [petsc-users] compilation with mumps on Mac OS 10.6.8 To: PETSc users list Message-ID: Content-Type: text/plain; charset="windows-1252" Thanks for the help. Now I finished the installation successfully. :) K. Lee. On Wed, Sep 7, 2011 at 11:49 AM, Barry Smith wrote: On Sep 6, 2011, at 10:41 PM, Kyunghoon Lee wrote: Hi Barry, Thanks for the reply. I did install g95 through macports. (I could not find gfortran.) I wonder g95 can do the job for me. g95 might work, give it a try. BTW: on Macs you don't need to provide blas/lapack with --download; Apple provides them on the system. Re SuperLU_Dist, all I know is I need mumps to deal with complex numbers. SuperLU_Dist can also be used with complex numbers (at least with PETSc-dev/petsc-3.2) K. Lee. On Wed, Sep 7, 2011 at 11:21 AM, Barry Smith wrote: p.s. I do not need to compile petsc for FORTRAN. Sadly you do because the packages MUMPS uses are all Fortran. So you will need a real Fortran compiler (perhaps gfortran) and you will also need --download-blacs and you cannot use --download-c-blas-lapack with scalapack. Note you can use SuperLU_Dist instead of MUMPS without a Fortran compiler. Barry On Sep 6, 2011, at 10:17 PM, Kyunghoon Lee wrote: Hello all, I need to compile petsc with mumps. First I tried ./configure --prefix=/Users/aeronova/Development/local/lib64/petsc/petsc-3.1-p8 --download-c-blas-lapack=1 ?download-parmetis=1 --download-scalapack=1 --download-mumps=1 but I got the following message: Fortran error! mpif.h could not be located at: [] So I included "-download-mpich=1," then I got the following: Should request f-blas-lapack, not --download-c-blas-lapack=yes since you have a fortran compiler? After that, I tried several options with/without --download-f-blas-lapack or --download-mpich=1, but none of them worked out. I'd appreciate it if someone can help me with this compilation problem with mumps. Regards, K. Lee. p.s. I do not need to compile petsc for FORTRAN. -------------- next part -------------- An HTML attachment was scrubbed... URL: ------------------------------ Message: 2 Date: Tue, 6 Sep 2011 23:05:26 -0400 From: Sravya Tirukkovalur Subject: [petsc-users] Build Petsc with single precision To: petsc-users at mcs.anl.gov Message-ID: Content-Type: text/plain; charset="iso-8859-1" Hi, I am trying to build petsc with single precision. The options I am using are: ./configure options --download-f-blas-lapack=1 --with-x=0 --with-debugging=1 --CFLAGS="-O3 -g -ggdb" --FFLAGS="-O3 -g" --with-hdf5=1 --download-hdf5=1 --with-batch=0 --known-mpi-shared-libraries=no --with-cuda=1 --with-cudac="nvcc -m64" --with-cusp=1 --with-thrust=1 PETSC_ARCH=SOMETHING Configuration step is being completed successfully. And while doing $make all, it compiles a few source files and prints an error message which says cannot build petsc with this options, contact the mailing list. Did anyone face a similar problem while building petsc with single precision or has anyone successfully built it? Thanks Barry, I have the following code for implementing the setting up snes and the numerical evaluation of the jacobian: ierr = jacobian_matrix_nz_pattern(sys); CHKERRQ(ierr); ierr = MatCreateMPIBAIJ(sys.comm, sys.num->nbq[sys.con->n], sys.ldof, sys.ldof, sys.gdof, sys.gdof, PETSC_NULL, sys.idxm, PETSC_NULL, sys.idxn, &sys.P); CHKERRQ(ierr); ierr = SNESCreate(sys.comm, &sys.snes); CHKERRQ(ierr); ierr = SNESSetFunction(sys.snes, sys.gres[0], base_residual_implicit, &sys); CHKERRQ(ierr); ierr = MatCreateSNESMF(sys.snes, &sys.J); CHKERRQ(ierr); ierr = MatMFFDSetFromOptions(sys.J); CHKERRQ(ierr); ierr = jacobian_diff_numerical(sys, &sys.P); CHKERRQ(ierr); ISColoring iscoloring; MatFDColoring fdcoloring; MatGetColoring(sys.P, MATCOLORING_SL, &iscoloring); MatFDColoringCreate(sys.P, iscoloring, &fdcoloring); ISColoringDestroy(iscoloring); MatFDColoringSetFromOptions(fdcoloring); ierr = SNESSetJacobian(sys.snes, sys.J, sys.P, SNESDefaultComputeJacobianColor, fdcoloring); CHKERRQ(ierr); As you can see I'm using a matrix free algorithm, However, when I run the code with with the -snes_mf_operator option I get on return the following error: Timestep 0: dt = 0.008, T = 0, Res[rho] = 0, Res[rhou] = 0, Res[rhov] = 0, Res[E] = 0, CFL = 0.177549 /*********************Stage 1 of SSPIRK (3,4)******************/ 0 SNES Function norm 2.755099585674e+01 [4]PETSC ERROR: --------------------- Error Message ------------------------------------ [4]PETSC ERROR: Object is in wrong state! [4]PETSC ERROR: Must call MatFDColoringSetFunction()! I haven't use the MatFDColoringSetFunction. But when I try to put it I read on the help page: Notes: This function is usually used automatically by SNES or TS (when one uses SNESSetJacobian() with the argument SNESDefaultComputeJacobianColor() or TSSetRHSJacobian() with the argument TSDefaultComputeJacobianColor()) and only needs to be used by someone computing a matrix via coloring directly by calling MatFDColoringApply() and furthermore I cannot figure out what the arguments are for the (*f) function. Any help? Kostas -------------- next part -------------- An HTML attachment was scrubbed... URL: From sean at mcs.anl.gov Wed Sep 7 12:15:40 2011 From: sean at mcs.anl.gov (Sean Farley) Date: Wed, 7 Sep 2011 12:15:40 -0500 Subject: [petsc-users] petsc-users Digest, Vol 33, Issue 13 In-Reply-To: References: <4E679EAC.1060906@lycos.com> Message-ID: > > I am trying to compile PETSC with MUMPS, but I am not sure how I compile > MUMPS libraries in MAC OSX 10.6.8 . I know that I need BLACS and SCALAPACK > libraries, but I am having trouble understanding how to compile these > libraries in MAC OSX 10.6.8. I saw that some users here were able to do > this in MAC OSX 10.6.8. Would any one of you who have successfully compiled > these libraries in MAC OSX 10.6.8 mind sharing their MAKE FILES ? I will > truly appreciate your help. I am sorry if it is inappropriate to ask for a > make file. It is just that I am not understanding where to start from. > If you only want PETSc to use MUMPS, then try the --download-blacs --download-scalapack --download-mumps options. Sean -------------- next part -------------- An HTML attachment was scrubbed... URL: From amrit_pou at hotmail.com Wed Sep 7 12:23:53 2011 From: amrit_pou at hotmail.com (amrit poudel) Date: Wed, 7 Sep 2011 17:23:53 +0000 Subject: [petsc-users] petsc-users Digest, Vol 33, Issue 13 In-Reply-To: References: , <4E679EAC.1060906@lycos.com>, , Message-ID: If I do this, will I get all the libraries compiled properly? I was thinking of independently compiling these external packages and then use them during run time. From: sean at mcs.anl.gov Date: Wed, 7 Sep 2011 12:15:40 -0500 To: petsc-users at mcs.anl.gov Subject: Re: [petsc-users] petsc-users Digest, Vol 33, Issue 13 I am trying to compile PETSC with MUMPS, but I am not sure how I compile MUMPS libraries in MAC OSX 10.6.8 . I know that I need BLACS and SCALAPACK libraries, but I am having trouble understanding how to compile these libraries in MAC OSX 10.6.8. I saw that some users here were able to do this in MAC OSX 10.6.8. Would any one of you who have successfully compiled these libraries in MAC OSX 10.6.8 mind sharing their MAKE FILES ? I will truly appreciate your help. I am sorry if it is inappropriate to ask for a make file. It is just that I am not understanding where to start from. If you only want PETSc to use MUMPS, then try the --download-blacs --download-scalapack --download-mumps options. Sean -------------- next part -------------- An HTML attachment was scrubbed... URL: From sean at mcs.anl.gov Wed Sep 7 12:35:51 2011 From: sean at mcs.anl.gov (Sean Farley) Date: Wed, 7 Sep 2011 12:35:51 -0500 Subject: [petsc-users] petsc-users Digest, Vol 33, Issue 13 In-Reply-To: References: <4E679EAC.1060906@lycos.com> Message-ID: > > If I do this, will I get all the libraries compiled properly? > Yes. All the libraries will be compiled properly on the mac and put int $PETSC_DIR/$PETSC_ARCH/lib. More importantly, PETSc will be compiled with MUMPS support easiest with this method. > I was thinking of independently compiling these external packages and then > use them during run time. > Do you need to use MUMPS without PETSc? You could look at my extension to MacPorts: https://bitbucket.org/seanfarley/scienceports But it will require you to: 1) Install (and use) MacPorts 2) Install the MacPorts gfortran A big caveat is that it is not as flexible as having different PETSC_ARCH's. This is due to MacPorts insistence on having everything installed into /opt/local but if you only want one PETSC_ARCH then it might be handy. You would then specify to PETSc's configure: ./configure --with-blacs-dir=/opt/local --with-scalapack-dir=/opt/local --with-mumps-dir=/opt/local Use this method at your own risk. Sean -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Wed Sep 7 12:54:40 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Wed, 7 Sep 2011 12:54:40 -0500 Subject: [petsc-users] petsc-users Digest, Vol 33, Issue 13 In-Reply-To: <4E679EAC.1060906@lycos.com> References: <4E679EAC.1060906@lycos.com> Message-ID: <649A4729-4CA5-4821-936F-C59A6AEBD74A@mcs.anl.gov> On Sep 7, 2011, at 11:41 AM, Kostas Kontzialis wrote: > > Barry, > > I have the following code for implementing the setting up snes and the numerical evaluation of the jacobian: > > > ierr = jacobian_matrix_nz_pattern(sys); > CHKERRQ(ierr); > > ierr = MatCreateMPIBAIJ(sys.comm, sys.num->nbq[sys.con->n], > sys.ldof, sys.ldof, sys.gdof, sys.gdof, PETSC_NULL, > sys.idxm, PETSC_NULL, sys.idxn, &sys.P); > CHKERRQ(ierr); > > ierr = SNESCreate(sys.comm, &sys.snes); > CHKERRQ(ierr); > > ierr = SNESSetFunction(sys.snes, sys.gres[0], base_residual_implicit, &sys); > CHKERRQ(ierr); > > ierr = MatCreateSNESMF(sys.snes, &sys.J); > CHKERRQ(ierr); > > ierr = MatMFFDSetFromOptions(sys.J); > CHKERRQ(ierr); > > > ierr = jacobian_diff_numerical(sys, &sys.P); > CHKERRQ(ierr); > > ISColoring iscoloring; > > MatFDColoring fdcoloring; > > MatGetColoring(sys.P, MATCOLORING_SL, &iscoloring); > > MatFDColoringCreate(sys.P, iscoloring, &fdcoloring); > ISColoringDestroy(iscoloring); > MatFDColoringSetFromOptions(fdcoloring); > > ierr = SNESSetJacobian(sys.snes, sys.J, sys.P, > SNESDefaultComputeJacobianColor, fdcoloring); > CHKERRQ(ierr); > > > As you can see I'm using a matrix free algorithm, However, when I run the code with with the -snes_mf_operator option I get on return the following error: > > Timestep 0: dt = 0.008, T = 0, Res[rho] = 0, Res[rhou] = 0, Res[rhov] = 0, Res[E] = 0, CFL = 0.177549 > > /*********************Stage 1 of SSPIRK (3,4)******************/ > > 0 SNES Function norm 2.755099585674e+01 > [4]PETSC ERROR: --------------------- Error Message ------------------------------------ > [4]PETSC ERROR: Object is in wrong state! > [4]PETSC ERROR: Must call MatFDColoringSetFunction()! > > I haven't use the MatFDColoringSetFunction. But when I try to put it I read on the help page: > > Notes: This function is usually used automatically by SNES or TS (when one uses SNESSetJacobian() with the argument SNESDefaultComputeJacobianColor() or TSSetRHSJacobian() with the argument TSDefaultComputeJacobianColor()) and only needs to be used by someone computing a matrix via coloring directly by calling MatFDColoringApply() > Since you are creating the MatFDColoring object yourself you are obligated to provide it with the function it needs to compute columns of the Jacobian. Thus you must call MatFDColoringSetFunction() the users manual is MatFDColoringSetFunction - Sets the function to use for computing the Jacobian. Logically Collective on MatFDColoring Input Parameters: + coloring - the coloring context . f - the function - fctx - the optional user-defined function context Calling sequence of (*f) function: For SNES: PetscErrorCode (*f)(SNES,Vec,Vec,void*) For TS: PetscErrorCode (*f)(TS,PetscReal,Vec,Vec,void*) If not using SNES or TS: PetscErrorCode (*f)(void *dummy,Vec,Vec,void*) and dummy is ignored Level: advanced Notes: This function is usually used automatically by SNES or TS (when one uses SNESSetJacobian() with the argument SNESDefaultComputeJacobianColor() or TSSetRHSJacobian() with the argument TSDefaultComputeJacobianColor()) and only needs to be used by someone computing a matrix via coloring directly by calling MatFDColoringApply() Fortran Notes: In Fortran you must call MatFDColoringSetFunction() for a coloring object to be used without SNES or TS or within the SNES solvers and MatFDColoringSetFunctionTS() if it is to be used within the TS solvers. so you pass it the same function you pass to SNESSetFunction() that is in your case base_residual_implicit, If that does not work send a bug report to petsc-maint at mcs.anl.gov Barry > and furthermore I cannot figure out what the arguments are for the (*f) function. > > Any help? > > Kostas > From amrit_pou at hotmail.com Wed Sep 7 13:16:19 2011 From: amrit_pou at hotmail.com (amrit poudel) Date: Wed, 7 Sep 2011 18:16:19 +0000 Subject: [petsc-users] petsc-users Digest, Vol 33, Issue 13 In-Reply-To: References: , <4E679EAC.1060906@lycos.com>, , , , Message-ID: I do not plan on using MUMPS without PETSc, but I would like to learn how I can compile these libraries independent of PETSc so that later when I try to configure PETSc in some other computer I know how to take care of MUMPS libraries. Coming to think of it, it sounds like I should not be doing this, because the other computer (which will be a cluster since this is where I plan on doing my number crunching ultimately) will have different OS and I will be out of luck again. By the way, I already have Macports installed and I have gcc44, but I do not have gfortran (I checked inside /opt/local/lib), which I can install . I am not quite understanding why we need gfortran from Macports, because I already have them in /usr/local/lib . Moreover, when we ./configure PETSc with MPI we need to use wrapper compilers mpicc and mpif77 anyways. I am sorry if this wrong, bu this is what my understanding was, and this is how I compiled PETSc on my computer. If I configure PETSc the way you mentioned (that is, with download mumps etc), will I then be able to run the executable of my program (the one that we get out by typing make myprogram) on any computer anywhere regardless of whether it has MAC OSX or not? Do I need to carry all the libraries with the program if I do so? Sorry for digressing with elementary questions. My original plan was to install PETSc on my laptop ( MAC OSX) and any external libraries or packages on the laptop, then compile my C program on my laptop, and then send the executable to cluster or more powerful computer around the country/national labs for number crunching. I am not implying that compiling MUMPS separately from PETSc will allow me to do this, but if your method of configuring PETSc with download mumps option will allow me to meet these needs, then I'd definitely do it that way. My ultimate goal is to be able to run my program on powerful computers and cluster. Thanks. -Amrit From: sean at mcs.anl.gov Date: Wed, 7 Sep 2011 12:35:51 -0500 To: petsc-users at mcs.anl.gov Subject: Re: [petsc-users] petsc-users Digest, Vol 33, Issue 13 If I do this, will I get all the libraries compiled properly? Yes. All the libraries will be compiled properly on the mac and put int $PETSC_DIR/$PETSC_ARCH/lib. More importantly, PETSc will be compiled with MUMPS support easiest with this method. I was thinking of independently compiling these external packages and then use them during run time. Do you need to use MUMPS without PETSc? You could look at my extension to MacPorts: https://bitbucket.org/seanfarley/scienceports But it will require you to: 1) Install (and use) MacPorts 2) Install the MacPorts gfortran A big caveat is that it is not as flexible as having different PETSC_ARCH's. This is due to MacPorts insistence on having everything installed into /opt/local but if you only want one PETSC_ARCH then it might be handy. You would then specify to PETSc's configure: ./configure --with-blacs-dir=/opt/local --with-scalapack-dir=/opt/local --with-mumps-dir=/opt/local Use this method at your own risk. Sean -------------- next part -------------- An HTML attachment was scrubbed... URL: From sean at mcs.anl.gov Wed Sep 7 13:20:15 2011 From: sean at mcs.anl.gov (Sean Farley) Date: Wed, 7 Sep 2011 18:20:15 +0000 Subject: [petsc-users] petsc-users Digest, Vol 33, Issue 13 In-Reply-To: References: <4E679EAC.1060906@lycos.com> Message-ID: > > I do not plan on using MUMPS without PETSc, but I would like to learn how I > can compile these libraries independent of PETSc so that later when I try to > configure PETSc in some other computer I know how to take care of MUMPS > libraries. Coming to think of it, it sounds like I should not be doing this, > because the other computer (which will be a cluster since this is where I > plan on doing my number crunching ultimately) will have different OS and I > will be out of luck again. > Then just use --download-mumps. BuildSystem will use the cluster provided MPI (or the mpi you have on your system) and build the external libraries correctly. Sean -------------- next part -------------- An HTML attachment was scrubbed... URL: From sean at mcs.anl.gov Wed Sep 7 13:24:03 2011 From: sean at mcs.anl.gov (Sean Farley) Date: Wed, 7 Sep 2011 18:24:03 +0000 Subject: [petsc-users] petsc-users Digest, Vol 33, Issue 13 In-Reply-To: References: <4E679EAC.1060906@lycos.com> Message-ID: > > If I configure PETSc the way you mentioned (that is, with download mumps > etc), will I then be able to run the executable of my program (the one that > we get out by typing make myprogram) on any computer anywhere regardless of > whether it has MAC OSX or not? Do I need to carry all the libraries with the > program if I do so? Sorry for digressing with elementary questions. > No, absolutely not. There is no way to move an executable to any other computer and have it run regardless of the OS. My original plan was to install PETSc on my laptop ( MAC OSX) and any > external libraries or packages on the laptop, then compile my C program on > my laptop, and then send the executable to cluster or more powerful computer > around the country/national labs for number crunching. I am not implying > that compiling MUMPS separately from PETSc will allow me to do this, but if > your method of configuring PETSc with download mumps option will allow me to > meet these needs, then I'd definitely do it that way. My ultimate goal is to > be able to run my program on powerful computers and cluster. > No. Please, please rethink your deployment method. Sean -------------- next part -------------- An HTML attachment was scrubbed... URL: From amrit_pou at hotmail.com Wed Sep 7 13:29:04 2011 From: amrit_pou at hotmail.com (amrit poudel) Date: Wed, 7 Sep 2011 18:29:04 +0000 Subject: [petsc-users] petsc-users Digest, Vol 33, Issue 13 In-Reply-To: References: , <4E679EAC.1060906@lycos.com>, , , , , , Message-ID: Okay, I will do that, but do I need to carry all MUMPS and PETSc libraries when I try to run the executables in a cluster or some other computer which will not have PETSc or MUMPS installed? I do not plan on building PETSc and MUMPS again in some computer in the national lab. I am planning on sending the executable only from my laptop to the cluster. Is this possible? Thanks for your help ! -Amrit From: sean at mcs.anl.gov Date: Wed, 7 Sep 2011 18:20:15 +0000 To: petsc-users at mcs.anl.gov Subject: Re: [petsc-users] petsc-users Digest, Vol 33, Issue 13 I do not plan on using MUMPS without PETSc, but I would like to learn how I can compile these libraries independent of PETSc so that later when I try to configure PETSc in some other computer I know how to take care of MUMPS libraries. Coming to think of it, it sounds like I should not be doing this, because the other computer (which will be a cluster since this is where I plan on doing my number crunching ultimately) will have different OS and I will be out of luck again. Then just use --download-mumps. BuildSystem will use the cluster provided MPI (or the mpi you have on your system) and build the external libraries correctly. Sean -------------- next part -------------- An HTML attachment was scrubbed... URL: From amrit_pou at hotmail.com Wed Sep 7 13:30:47 2011 From: amrit_pou at hotmail.com (amrit poudel) Date: Wed, 7 Sep 2011 18:30:47 +0000 Subject: [petsc-users] petsc-users Digest, Vol 33, Issue 13 In-Reply-To: References: , <4E679EAC.1060906@lycos.com>, , , , , , Message-ID: This means that I will need to carry my libraries along with the program making sure that OS is also the same. Is that correct? From: sean at mcs.anl.gov Date: Wed, 7 Sep 2011 18:24:03 +0000 To: petsc-users at mcs.anl.gov Subject: Re: [petsc-users] petsc-users Digest, Vol 33, Issue 13 If I configure PETSc the way you mentioned (that is, with download mumps etc), will I then be able to run the executable of my program (the one that we get out by typing make myprogram) on any computer anywhere regardless of whether it has MAC OSX or not? Do I need to carry all the libraries with the program if I do so? Sorry for digressing with elementary questions. No, absolutely not. There is no way to move an executable to any other computer and have it run regardless of the OS. My original plan was to install PETSc on my laptop ( MAC OSX) and any external libraries or packages on the laptop, then compile my C program on my laptop, and then send the executable to cluster or more powerful computer around the country/national labs for number crunching. I am not implying that compiling MUMPS separately from PETSc will allow me to do this, but if your method of configuring PETSc with download mumps option will allow me to meet these needs, then I'd definitely do it that way. My ultimate goal is to be able to run my program on powerful computers and cluster. No. Please, please rethink your deployment method. Sean -------------- next part -------------- An HTML attachment was scrubbed... URL: From balay at mcs.anl.gov Wed Sep 7 13:33:49 2011 From: balay at mcs.anl.gov (Satish Balay) Date: Wed, 7 Sep 2011 13:33:49 -0500 (CDT) Subject: [petsc-users] petsc-users Digest, Vol 33, Issue 13 In-Reply-To: References: , <4E679EAC.1060906@lycos.com>, , , , , , Message-ID: There is some fundamantal misunderstanding about computing in your statements. Perhaps you should do attempt some of the multi-machine things you'd like to do with simple [mpi] code - without PETSc - to get a better understanding about portability of applications. Satish On Wed, 7 Sep 2011, amrit poudel wrote: > > > > > This means that I will need to carry my libraries along with the program making sure that OS is also the same. Is that correct? > > > From: sean at mcs.anl.gov > Date: Wed, 7 Sep 2011 18:24:03 +0000 > To: petsc-users at mcs.anl.gov > Subject: Re: [petsc-users] petsc-users Digest, Vol 33, Issue 13 > > If I configure PETSc the way you mentioned (that is, with download mumps etc), will I then be able to run the executable of my program (the one that we get out by typing make myprogram) on any computer anywhere regardless of whether it has MAC OSX or not? Do I need to carry all the libraries with the program if I do so? Sorry for digressing with elementary questions. > > > > No, absolutely not. There is no way to move an executable to any other computer and have it run regardless of the OS. > > > My original plan was to install PETSc on my laptop ( MAC OSX) and any external libraries or packages on the laptop, then compile my C program on my laptop, and then send the executable to cluster or more powerful computer around the country/national labs for number crunching. I am not implying that compiling MUMPS separately from PETSc will allow me to do this, but if your method of configuring PETSc with download mumps option will allow me to meet these needs, then I'd definitely do it that way. My ultimate goal is to be able to run my program on powerful computers and cluster. > > > No. Please, please rethink your deployment method. > Sean From sean at mcs.anl.gov Wed Sep 7 13:35:20 2011 From: sean at mcs.anl.gov (Sean Farley) Date: Wed, 7 Sep 2011 18:35:20 +0000 Subject: [petsc-users] petsc-users Digest, Vol 33, Issue 13 In-Reply-To: References: <4E679EAC.1060906@lycos.com> Message-ID: > > Okay, I will do that, but do I need to carry all MUMPS and PETSc > libraries when I try to run the executables in a cluster or some other > computer which will not have PETSc or MUMPS installed? I do not plan on > building PETSc and MUMPS again in some computer in the national lab. I am > planning on sending the executable only from my laptop to the cluster. Is > this possible? > If that is possible, then you should update the wiki page: http://en.wikipedia.org/wiki/Execution_(computing) Sean -------------- next part -------------- An HTML attachment was scrubbed... URL: From amrit_pou at hotmail.com Wed Sep 7 13:49:30 2011 From: amrit_pou at hotmail.com (amrit poudel) Date: Wed, 7 Sep 2011 18:49:30 +0000 Subject: [petsc-users] petsc-users Digest, Vol 33, Issue 13 In-Reply-To: References: , <4E679EAC.1060906@lycos.com>, , , , , , , , Message-ID: I see. From all this discussion what I've understood is that if I need to run my program on say Scietific Linux cluster, then I need to build PETSc (along with MUMPS or any other external packages) on a computer with Scientific Linux OS, then I will be able to run my program in that cluster. Please correct me if I am wrong. Thanks for clearing up my confusion ! From: sean at mcs.anl.gov Date: Wed, 7 Sep 2011 18:35:20 +0000 To: petsc-users at mcs.anl.gov Subject: Re: [petsc-users] petsc-users Digest, Vol 33, Issue 13 Okay, I will do that, but do I need to carry all MUMPS and PETSc libraries when I try to run the executables in a cluster or some other computer which will not have PETSc or MUMPS installed? I do not plan on building PETSc and MUMPS again in some computer in the national lab. I am planning on sending the executable only from my laptop to the cluster. Is this possible? If that is possible, then you should update the wiki page: http://en.wikipedia.org/wiki/Execution_(computing) Sean -------------- next part -------------- An HTML attachment was scrubbed... URL: From sean at mcs.anl.gov Wed Sep 7 13:54:26 2011 From: sean at mcs.anl.gov (Sean Farley) Date: Wed, 7 Sep 2011 18:54:26 +0000 Subject: [petsc-users] petsc-users Digest, Vol 33, Issue 13 In-Reply-To: References: <4E679EAC.1060906@lycos.com> Message-ID: > > I see. From all this discussion what I've understood is that if I need to > run my program on say Scietific Linux cluster, then I need to build PETSc > (along with MUMPS or any other external packages) on a computer with > Scientific Linux OS, then I will be able to run my program in that cluster. > Please correct me if I am wrong. > You will need to build PETSc+MUMPS using the cluster specific MPI (usually this is built on top of some infiniband, myranet, fancy network) *on the actual cluster*. Sean -------------- next part -------------- An HTML attachment was scrubbed... URL: From balay at mcs.anl.gov Wed Sep 7 13:56:41 2011 From: balay at mcs.anl.gov (Satish Balay) Date: Wed, 7 Sep 2011 13:56:41 -0500 (CDT) Subject: [petsc-users] petsc-users Digest, Vol 33, Issue 13 In-Reply-To: References: , <4E679EAC.1060906@lycos.com>, , , , , , , , Message-ID: yes - fundamentally source code is more portable. [binary is generally not portable - even on the same OS - due to versions of OS or dependent libraries or other issues] Whatever machine you need to run your code on - you'll have to move your code [and supporting library code] to that machine and build/run there. Sometimes you might find some libraries prebuilt on the target machine. Compute clusters generally will have MPI prebuilt [by the admin of the cluster] - and one should build all mpi related packages,applcations with that recommended/installed MPI etc.. PETSc [and all externalpackages it installs] are provided to you at the source code level - so you build and use them on each target machine you need them on. Satish On Wed, 7 Sep 2011, amrit poudel wrote: > > I see. From all this discussion what I've understood is that if I need to run my program on say Scietific Linux cluster, then I need to build PETSc (along with MUMPS or any other external packages) on a computer with Scientific Linux OS, then I will be able to run my program in that cluster. Please correct me if I am wrong. > > Thanks for clearing up my confusion ! > > > > > From: sean at mcs.anl.gov > Date: Wed, 7 Sep 2011 18:35:20 +0000 > To: petsc-users at mcs.anl.gov > Subject: Re: [petsc-users] petsc-users Digest, Vol 33, Issue 13 > > Okay, I will do that, but do I need to carry all MUMPS and PETSc libraries when I try to run the executables in a cluster or some other computer which will not have PETSc or MUMPS installed? I do not plan on building PETSc and MUMPS again in some computer in the national lab. I am planning on sending the executable only from my laptop to the cluster. Is this possible? > > > If that is possible, then you should update the wiki page: > http://en.wikipedia.org/wiki/Execution_(computing) > > > Sean From amrit_pou at hotmail.com Wed Sep 7 14:26:17 2011 From: amrit_pou at hotmail.com (amrit poudel) Date: Wed, 7 Sep 2011 19:26:17 +0000 Subject: [petsc-users] petsc-users Digest, Vol 33, Issue 13 In-Reply-To: References: , , <4E679EAC.1060906@lycos.com>, , , , , , , , , , , , , , , , , , Message-ID: Thank you Sean and Satish for clarifying ! One last question, just to clear up my confusion : Sean mentioned that I will need to install PETSc on the actual cluster that I want to run my program, which, as I understand, means that I need to do ./configure make make install on that actual cluster I plan on running my code. I was originally thinking that if I do ./configure make make install on some Scientific Linux computer(with appropriate MPI), then I will get all the libraries (.a files) that I need. Then, if I have to run my program, say program.c, that calls PETSc libraries on a Scientific Linux cluster, all I need to do is carry all the libraries (and anything that my program is linked with) together with my program to the cluster and run my program there. Is this really incorrect ? Thanks for bearing with my elementary questions. I appreciate all your feedback. -Amrit > Date: Wed, 7 Sep 2011 13:56:41 -0500 > From: balay at mcs.anl.gov > To: petsc-users at mcs.anl.gov > Subject: Re: [petsc-users] petsc-users Digest, Vol 33, Issue 13 > > yes - fundamentally source code is more portable. [binary is generally > not portable - even on the same OS - due to versions of OS or > dependent libraries or other issues] > > Whatever machine you need to run your code on - you'll have to move > your code [and supporting library code] to that machine and build/run > there. > > Sometimes you might find some libraries prebuilt on the target > machine. Compute clusters generally will have MPI prebuilt [by the > admin of the cluster] - and one should build all mpi related > packages,applcations with that recommended/installed MPI etc.. > > PETSc [and all externalpackages it installs] are provided to you at > the source code level - so you build and use them on each target > machine you need them on. > > Satish > > On Wed, 7 Sep 2011, amrit poudel wrote: > > > > > I see. From all this discussion what I've understood is that if I need to run my program on say Scietific Linux cluster, then I need to build PETSc (along with MUMPS or any other external packages) on a computer with Scientific Linux OS, then I will be able to run my program in that cluster. Please correct me if I am wrong. > > > > Thanks for clearing up my confusion ! > > > > > > > > > > From: sean at mcs.anl.gov > > Date: Wed, 7 Sep 2011 18:35:20 +0000 > > To: petsc-users at mcs.anl.gov > > Subject: Re: [petsc-users] petsc-users Digest, Vol 33, Issue 13 > > > > Okay, I will do that, but do I need to carry all MUMPS and PETSc libraries when I try to run the executables in a cluster or some other computer which will not have PETSc or MUMPS installed? I do not plan on building PETSc and MUMPS again in some computer in the national lab. I am planning on sending the executable only from my laptop to the cluster. Is this possible? > > > > > > If that is possible, then you should update the wiki page: > > http://en.wikipedia.org/wiki/Execution_(computing) > > > > > > Sean > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sean at mcs.anl.gov Wed Sep 7 14:31:29 2011 From: sean at mcs.anl.gov (Sean Farley) Date: Wed, 7 Sep 2011 19:31:29 +0000 Subject: [petsc-users] petsc-users Digest, Vol 33, Issue 13 In-Reply-To: References: <4E679EAC.1060906@lycos.com> Message-ID: > > Sean mentioned that I will need to install PETSc on the actual cluster that > I want to run my program, which, as I understand, means that I need to do > ./configure make make install on that actual cluster I plan on running my > code. I was originally thinking that if I do ./configure make make install > on some Scientific Linux computer(with appropriate MPI), then I will get all > the libraries (.a files) that I need. Then, if I have to run my program, say > program.c, that calls PETSc libraries on a Scientific Linux cluster, all I > need to do is carry all the libraries (and anything that my program is > linked with) together with my program to the cluster and run my program > there. Is this really incorrect ? > That is 100% incorrect. If you don't believe me, you are welcome to try it on your own. You really have to give up on this "carry all the libraries around" idea. Sean -------------- next part -------------- An HTML attachment was scrubbed... URL: From amrit_pou at hotmail.com Wed Sep 7 16:44:39 2011 From: amrit_pou at hotmail.com (amrit poudel) Date: Wed, 7 Sep 2011 21:44:39 +0000 Subject: [petsc-users] petsc-users Digest, Vol 33, Issue 13 In-Reply-To: References: , <4E679EAC.1060906@lycos.com>, , , , , , , , , , , , Message-ID: HI Sean,It is really not the question of "not believing you", it is just that I do not have any formal training in programming/parallel programming and hence my misconceptions. I am simply trying to clear up my doubts by asking questions to experts. Please bear with me and accept my apology if you find any of my questions ridiculously incorrect. I found the following question in FAQ section of http://www.mcs.anl.gov/petsc/petsc-2/documentation/faq.html#already-installed . I know that this is related to shared libraries, but it sounds as if "moving libraries around" may be an option. Again, I assure you that I 100% believe you. I am simply trying to understand what this question is referring to. What do I do if I want to move my executable to a different machine? You would also need to have access to the shared libraries on this new machine. The other alternative is to build the exeutable without shared libraries by first deleting the shared libraries, and then creating the executable. Thanks for all your help ! From: sean at mcs.anl.gov Date: Wed, 7 Sep 2011 19:31:29 +0000 To: petsc-users at mcs.anl.gov Subject: Re: [petsc-users] petsc-users Digest, Vol 33, Issue 13 Sean mentioned that I will need to install PETSc on the actual cluster that I want to run my program, which, as I understand, means that I need to do ./configure make make install on that actual cluster I plan on running my code. I was originally thinking that if I do ./configure make make install on some Scientific Linux computer(with appropriate MPI), then I will get all the libraries (.a files) that I need. Then, if I have to run my program, say program.c, that calls PETSc libraries on a Scientific Linux cluster, all I need to do is carry all the libraries (and anything that my program is linked with) together with my program to the cluster and run my program there. Is this really incorrect ? That is 100% incorrect. If you don't believe me, you are welcome to try it on your own. You really have to give up on this "carry all the libraries around" idea. Sean -------------- next part -------------- An HTML attachment was scrubbed... URL: From balay at mcs.anl.gov Wed Sep 7 17:02:37 2011 From: balay at mcs.anl.gov (Satish Balay) Date: Wed, 7 Sep 2011 17:02:37 -0500 (CDT) Subject: [petsc-users] petsc-users Digest, Vol 33, Issue 13 In-Reply-To: References: , <4E679EAC.1060906@lycos.com>, , , , , , , , , , , , Message-ID: Yes- moving binaries [between machines of the same OS] can work. But practically other issues mentioned dominate [like mpi] that can cause it not to work - hence its best to work at source level of portability. The Primary folks that have interest in this mode are comercial software developers - who can put sufficient effort to address its shortcommings. We don't support such usage with PETSc. you are on your own with it. Satish On Wed, 7 Sep 2011, amrit poudel wrote: > > HI Sean,It is really not the question of "not believing you", it is just that I do not have any formal training in programming/parallel programming and hence my misconceptions. I am simply trying to clear up my doubts by asking questions to experts. Please bear with me and accept my apology if you find any of my questions ridiculously incorrect. > > I found the following question in FAQ section of http://www.mcs.anl.gov/petsc/petsc-2/documentation/faq.html#already-installed . I know that this is related to shared libraries, but it sounds as if "moving libraries around" may be an option. Again, I assure you that I 100% believe you. I am simply trying to understand what this question is referring to. > > > What > do > I > do if I want to move > my executable to a different machine? > You would also need to have access to the shared > libraries on this new machine. The other alternative is to > build the > exeutable without shared libraries by first deleting the > shared > libraries, and then creating the executable. > > > > Thanks for all your help ! > > > > > From: sean at mcs.anl.gov > Date: Wed, 7 Sep 2011 19:31:29 +0000 > To: petsc-users at mcs.anl.gov > Subject: Re: [petsc-users] petsc-users Digest, Vol 33, Issue 13 > > Sean mentioned that I will need to install PETSc on the actual cluster that I want to run my program, which, as I understand, means that I need to do ./configure make make install on that actual cluster I plan on running my code. I was originally thinking that if I do ./configure make make install on some Scientific Linux computer(with appropriate MPI), then I will get all the libraries (.a files) that I need. Then, if I have to run my program, say program.c, that calls PETSc libraries on a Scientific Linux cluster, all I need to do is carry all the libraries (and anything that my program is linked with) together with my program to the cluster and run my program there. Is this really incorrect ? > > > That is 100% incorrect. If you don't believe me, you are welcome to try it on your own. You really have to give up on this "carry all the libraries around" idea. > > > Sean From amrit_pou at hotmail.com Wed Sep 7 18:50:46 2011 From: amrit_pou at hotmail.com (amrit poudel) Date: Wed, 7 Sep 2011 23:50:46 +0000 Subject: [petsc-users] PETSc config error In-Reply-To: References: <4E671655.5050100@lycos.com>, , Message-ID: I get the following error (down below) when I try to configure PETSc with this command (on a scientific Linux OS) : ./config/configure.py --prefix=/home/amrit/PETSc --with-scalar-type=complex --with-cc=/home/amrit/MPI/bin/mpicc --with-fc=/home/amrit/MPI/bin/mpif77 =============================================================================== Configuring PETSc to compile on your system =============================================================================== TESTING: checkCCompiler from config.setCompilers(config/BuildSystem/config/setCo******************************************************************************* UNABLE to EXECUTE BINARIES for config/configure.py ------------------------------------------------------------------------------- Cannot run executables created with C. If this machine uses a batch system to submit jobs you will need to configure using/configure.py with the additional option --with-batch. Otherwise there is problem with the compilers. Can you compile and run code with your C/C++ (and maybe Fortran) compilers? ******************************************************************************* I am sure that fortran and C/C++ compilers are okay on this computer. Any help will be appreciated. Thanks. -------------- next part -------------- An HTML attachment was scrubbed... URL: From amrit_pou at hotmail.com Wed Sep 7 19:01:04 2011 From: amrit_pou at hotmail.com (amrit poudel) Date: Thu, 8 Sep 2011 00:01:04 +0000 Subject: [petsc-users] PETSc config error In-Reply-To: References: <4E671655.5050100@lycos.com>, , , , , Message-ID: I tried configuring it with --with-batch , but I still cannot get past the error (see below) ./config/configure.py --prefix=/home/amrit/PETSc --with-scalar-type=complex --with-batch --with-cc=/home/amrit/MPI/bin/mpicc --with-fc=/home/amrit/MPI/bin/mpif77 =============================================================================== Configuring PETSc to compile on your system =============================================================================== TESTING: configureMPIEXEC from config.packages.MPI(config/BuildSystem/config/pac******************************************************************************* UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log for details): ------------------------------------------------------------------------------- Must give a default value for known-mpi-shared since executables cannot be run ******************************************************************************* Thanks for any help. From: amrit_pou at hotmail.com To: petsc-users at mcs.anl.gov Date: Wed, 7 Sep 2011 23:50:46 +0000 Subject: [petsc-users] PETSc config error I get the following error (down below) when I try to configure PETSc with this command (on a scientific Linux OS) : ./config/configure.py --prefix=/home/amrit/PETSc --with-scalar-type=complex --with-cc=/home/amrit/MPI/bin/mpicc --with-fc=/home/amrit/MPI/bin/mpif77 =============================================================================== Configuring PETSc to compile on your system =============================================================================== TESTING: checkCCompiler from config.setCompilers(config/BuildSystem/config/setCo******************************************************************************* UNABLE to EXECUTE BINARIES for config/configure.py ------------------------------------------------------------------------------- Cannot run executables created with C. If this machine uses a batch system to submit jobs you will need to configure using/configure.py with the additional option --with-batch. Otherwise there is problem with the compilers. Can you compile and run code with your C/C++ (and maybe Fortran) compilers? ******************************************************************************* I am sure that fortran and C/C++ compilers are okay on this computer. Any help will be appreciated. Thanks. -------------- next part -------------- An HTML attachment was scrubbed... URL: From sean at mcs.anl.gov Wed Sep 7 19:52:07 2011 From: sean at mcs.anl.gov (Sean Farley) Date: Thu, 8 Sep 2011 00:52:07 +0000 Subject: [petsc-users] PETSc config error In-Reply-To: References: <4E671655.5050100@lycos.com> Message-ID: > > I tried configuring it with --with-batch , but I still cannot get past > the error (see below) > > ./config/configure.py --prefix=/home/amrit/PETSc > --with-scalar-type=complex --with-batch --with-cc=/home/amrit/MPI/bin/mpicc > --with-fc=/home/amrit/MPI/bin/mpif77 > > > =============================================================================== > Configuring PETSc to compile on your > system > > =============================================================================== > TESTING: configureMPIEXEC from > config.packages.MPI(config/BuildSystem/config/pac******************************************************************************* > UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log for > details): > > ------------------------------------------------------------------------------- > Must give a default value for known-mpi-shared since executables cannot be > run > > ******************************************************************************* > > Thanks for any help. > Amrit, This message belongs on petsc-maint, not petsc-users. Please resend this to petsc-maint with configure.log. Sean -------------- next part -------------- An HTML attachment was scrubbed... URL: From jedbrown at mcs.anl.gov Wed Sep 7 19:57:27 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Thu, 8 Sep 2011 02:57:27 +0200 Subject: [petsc-users] PETSc config error In-Reply-To: References: <4E671655.5050100@lycos.com> Message-ID: On Thu, Sep 8, 2011 at 02:01, amrit poudel wrote: > > ./config/configure.py --prefix=/home/amrit/PETSc > --with-scalar-type=complex --with-batch --with-cc=/home/amrit/MPI/bin/mpicc > --with-fc=/home/amrit/MPI/bin/mpif77 > > > =============================================================================== > Configuring PETSc to compile on your > system > > =============================================================================== > TESTING: configureMPIEXEC from > config.packages.MPI(config/BuildSystem/config/pac******************************************************************************* > UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log for > details): > > ------------------------------------------------------------------------------- > Must give a default value for known-mpi-shared since executables cannot be > run You need to determine whether your MPI implementation uses static or shared libraries, then pass --known-mpi-shared=1 if it shared and --known-mpi-shared=0 otherwise. You can determine whether it is shared by examining the libraries. mpicc -show ls /home/amrit/MPI/lib # look for *.a for shared libs and *.so or *.dylib for shared libs. -------------- next part -------------- An HTML attachment was scrubbed... URL: From amrit_pou at hotmail.com Wed Sep 7 20:13:29 2011 From: amrit_pou at hotmail.com (amrit poudel) Date: Thu, 8 Sep 2011 01:13:29 +0000 Subject: [petsc-users] PETSc config error In-Reply-To: References: <4E671655.5050100@lycos.com>, , , , , Message-ID: This is what I have in /lib, and by looking at libmpi.so, I believe I have a shared library. Would you recommend building a static library for open mpi or is shared library okay? libmca_common_sm.la libmpi_f90.so.0 libopen-rte.la libmca_common_sm.so libmpi_f90.so.0.0.1 libopen-rte.so libmca_common_sm.so.1 libmpi.la libopen-rte.so.0 libmca_common_sm.so.1.0.0 libmpi.so libopen-rte.so.0.0.0 libmpi_cxx.la libmpi.so.0 libotf.a libmpi_cxx.so libmpi.so.0.0.2 libotf.la libmpi_cxx.so.0 libopenmpi_malloc.la libvt.a libmpi_cxx.so.0.0.1 libopenmpi_malloc.so libvt.fmpi.a libmpi_f77.la libopenmpi_malloc.so.0 libvt.mpi.a libmpi_f77.so libopenmpi_malloc.so.0.0.0 libvt.omp.a libmpi_f77.so.0 libopen-pal.la libvt.ompi.a libmpi_f77.so.0.0.1 libopen-pal.so mpi.mod libmpi_f90.la libopen-pal.so.0 openmpi libmpi_f90.so libopen-pal.so.0.0.0 Date: Thu, 8 Sep 2011 02:57:27 +0200 From: jedbrown at mcs.anl.gov To: petsc-users at mcs.anl.gov Subject: Re: [petsc-users] PETSc config error On Thu, Sep 8, 2011 at 02:01, amrit poudel wrote: ./config/configure.py --prefix=/home/amrit/PETSc --with-scalar-type=complex --with-batch --with-cc=/home/amrit/MPI/bin/mpicc --with-fc=/home/amrit/MPI/bin/mpif77 =============================================================================== Configuring PETSc to compile on your system =============================================================================== TESTING: configureMPIEXEC from config.packages.MPI(config/BuildSystem/config/pac******************************************************************************* UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log for details): ------------------------------------------------------------------------------- Must give a default value for known-mpi-shared since executables cannot be run You need to determine whether your MPI implementation uses static or shared libraries, then pass --known-mpi-shared=1 if it shared and --known-mpi-shared=0 otherwise. You can determine whether it is shared by examining the libraries. mpicc -showls /home/amrit/MPI/lib # look for *.a for shared libs and *.so or *.dylib for shared libs. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jedbrown at mcs.anl.gov Wed Sep 7 20:14:35 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Thu, 8 Sep 2011 03:14:35 +0200 Subject: [petsc-users] PETSc config error In-Reply-To: References: <4E671655.5050100@lycos.com> Message-ID: On Thu, Sep 8, 2011 at 03:13, amrit poudel wrote: > This is what I have in /lib, and by looking at libmpi.so, I believe I have > a shared library. Would you recommend building a static library for open > mpi or is shared library okay? Shared is usually fine. Just add --known-mpi-shared=1 to the configure line you ran before. -------------- next part -------------- An HTML attachment was scrubbed... URL: From amrit_pou at hotmail.com Wed Sep 7 21:06:41 2011 From: amrit_pou at hotmail.com (amrit poudel) Date: Thu, 8 Sep 2011 02:06:41 +0000 Subject: [petsc-users] PETSc config error In-Reply-To: References: <4E671655.5050100@lycos.com>, , , , , , , Message-ID: Thanks Jade. That worked out fine. But I am running into another problem now. The error is shown below. I am not sure why it says no such file or directory. -bash-3.2$ ./config/configure.py --prefix=/home/amrit/PETSc --with-scalar-type=complex --with-batch --with-cc=/home/amrit/MPI/bin/mpicc --with-fc=/home/amrit/MPI/bin/mpif77 --known-mpi-shared=1 --download-f-blas-lapack=1 =============================================================================== Configuring PETSc to compile on your system =============================================================================== =============================================================================== Compiling FBLASLAPACK; this may take several minutes =============================================================================== ================================================================================= Since your compute nodes require use of a batch system or mpiexec you must: 1) Submit ./conftest to 1 processor of your batch system or system you are cross-compiling for; this will generate the file reconfigure.py 2) Run ./reconfigure.py (to complete the configure process). ================================================================================= -bash-3.2$ ./conftest ./conftest: error while loading shared libraries: libmpi.so.0: cannot open shared object file: No such file or directory Date: Thu, 8 Sep 2011 03:14:35 +0200 From: jedbrown at mcs.anl.gov To: petsc-users at mcs.anl.gov Subject: Re: [petsc-users] PETSc config error On Thu, Sep 8, 2011 at 03:13, amrit poudel wrote: This is what I have in /lib, and by looking at libmpi.so, I believe I have a shared library. Would you recommend building a static library for open mpi or is shared library okay? Shared is usually fine. Just add --known-mpi-shared=1 to the configure line you ran before. -------------- next part -------------- An HTML attachment was scrubbed... URL: From amrit_pou at hotmail.com Wed Sep 7 21:17:31 2011 From: amrit_pou at hotmail.com (amrit poudel) Date: Thu, 8 Sep 2011 02:17:31 +0000 Subject: [petsc-users] PETSc Error when writing a vector object to a binary file In-Reply-To: References: <4E671655.5050100@lycos.com>, , , , , , , , , , , , , , , Message-ID: I am using the following sequence of commands to write a Vec b to a binary file and then eventually read it in MATLAB, but I get an error when I try writing this vector to a file in PETSc. ierr = PetscViewerBinaryOpen(PETSC_COMM_WORLD, filesol, FILE_MODE_WRITE, &binviewersol) ; CHKERRQ(ierr) ; ierr = PetscViewerBinaryGetDescriptor(binviewersol, &fd); CHKERRQ(ierr) ; ierr = PetscBinaryWrite(fd, b, 5, PETSC_SCALAR, PETSC_FALSE); CHKERRQ(ierr) ; Here is the error : [1]PETSC ERROR: Write to file failed! [1]PETSC ERROR: Error writing to file.! NOTE : Petsc is configured with complex option and Vec b has complex entries. Length of Vec b is 5 . -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Wed Sep 7 21:51:44 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Wed, 7 Sep 2011 21:51:44 -0500 Subject: [petsc-users] PETSc Error when writing a vector object to a binary file In-Reply-To: References: <4E671655.5050100@lycos.com>, , , , , , , , , , , , , , , Message-ID: On Sep 7, 2011, at 9:17 PM, amrit poudel wrote: > I am using the following sequence of commands to write a Vec b to a binary file and then eventually read it in MATLAB, but I get an error when I try writing this vector to a file in PETSc. > > ierr = PetscViewerBinaryOpen(PETSC_COMM_WORLD, filesol, FILE_MODE_WRITE, &binviewersol) ; CHKERRQ(ierr) ; > ierr = PetscViewerBinaryGetDescriptor(binviewersol, &fd); CHKERRQ(ierr) ; > ierr = PetscBinaryWrite(fd, b, 5, PETSC_SCALAR, PETSC_FALSE); CHKERRQ(ierr) ; This is not how you would write a Vec to a binary file. PetscBinaryWrite() is for writing raw bytes to a binary file (a low level operation). Use VecView(b, binviewersol); to write a vector. Barry > > > > Here is the error : > > > [1]PETSC ERROR: Write to file failed! > [1]PETSC ERROR: Error writing to file.! > > > > NOTE : Petsc is configured with complex option and Vec b has complex entries. Length of Vec b is 5 . From bsmith at mcs.anl.gov Wed Sep 7 21:55:03 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Wed, 7 Sep 2011 21:55:03 -0500 Subject: [petsc-users] PETSc config error In-Reply-To: References: <4E671655.5050100@lycos.com>, , , , , , , Message-ID: <51B9E65B-6477-42DB-86C2-B8E7A53BF8B7@mcs.anl.gov> These installation questions should be sent to petsc-maint at mcs.anl.gov not to petsc-users. It appears there is something wrong/incomplete with the mpicc that your are using; it is not properly connecting the executable to the MPI shared libraries that it is needs which it should do automatically. Where did you get this MPI? Do /home/amrit/MPI/bin/mpicc -show and send us the results. Please send all future correspondence on this issue to petsc-maint at mcs.anl.gov Barry On Sep 7, 2011, at 9:06 PM, amrit poudel wrote: > Thanks Jade. That worked out fine. But I am running into another problem now. The error is shown below. I am not sure why it says no such file or directory. > > -bash-3.2$ ./config/configure.py --prefix=/home/amrit/PETSc --with-scalar-type=complex --with-batch --with-cc=/home/amrit/MPI/bin/mpicc --with-fc=/home/amrit/MPI/bin/mpif77 --known-mpi-shared=1 --download-f-blas-lapack=1 > > =============================================================================== > Configuring PETSc to compile on your system > =============================================================================== > =============================================================================== > Compiling FBLASLAPACK; this may take several minutes =============================================================================== ================================================================================= > Since your compute nodes require use of a batch system or mpiexec you must: > 1) Submit ./conftest to 1 processor of your batch system or system you are > cross-compiling for; this will generate the file reconfigure.py > 2) Run ./reconfigure.py (to complete the configure process). > ================================================================================= > > -bash-3.2$ ./conftest > ./conftest: error while loading shared libraries: libmpi.so.0: cannot open shared object file: No such file or directory > > > > Date: Thu, 8 Sep 2011 03:14:35 +0200 > From: jedbrown at mcs.anl.gov > To: petsc-users at mcs.anl.gov > Subject: Re: [petsc-users] PETSc config error > > On Thu, Sep 8, 2011 at 03:13, amrit poudel wrote: > This is what I have in /lib, and by looking at libmpi.so, I believe I have a shared library. Would you recommend building a static library for open mpi or is shared library okay? > > Shared is usually fine. Just add --known-mpi-shared=1 to the configure line you ran before. From abhyshr at hawk.iit.edu Wed Sep 7 21:43:17 2011 From: abhyshr at hawk.iit.edu (Shrirang Abhyankar) Date: Wed, 7 Sep 2011 21:43:17 -0500 Subject: [petsc-users] PETSc Error when writing a vector object to a binary file In-Reply-To: References: <4E671655.5050100@lycos.com> Message-ID: Use VecView(), http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/Vec/VecView.html, to write the vector to a binary file and then use PetscBinaryRead() in Matlab ierr = PetscViewerBinaryOpen(PETSC_COMM_WORLD, filesol, FILE_MODE_WRITE, &binviewersol) ; CHKERRQ(ierr) ; ierr = VecView(b,binviewersol);CHKERRQ(ierr); In Matlab, b = PetscBinaryRead(binviewersol,'complex', true); See the function PetscBinaryRead() in $PETSC_DIR/bin/matlab On Wed, Sep 7, 2011 at 9:17 PM, amrit poudel wrote: > I am using the following sequence of commands to write a Vec b to a binary > file and then eventually read it in MATLAB, but I get an error when I try > writing this vector to a file in PETSc. > > ierr = PetscViewerBinaryOpen(PETSC_COMM_WORLD, filesol, FILE_MODE_WRITE, > &binviewersol) ; CHKERRQ(ierr) ; > ierr = PetscViewerBinaryGetDescriptor(binviewersol, &fd); CHKERRQ(ierr) ; > ierr = PetscBinaryWrite(fd, b, 5, PETSC_SCALAR, PETSC_FALSE); > CHKERRQ(ierr) ; > > > > Here is the error : > > > [1]PETSC ERROR: Write to file failed! > [1]PETSC ERROR: Error writing to file.! > > > > NOTE : Petsc is configured with complex option and Vec b has complex > entries. Length of Vec b is 5 . > -------------- next part -------------- An HTML attachment was scrubbed... URL: From amrit_pou at hotmail.com Wed Sep 7 22:15:10 2011 From: amrit_pou at hotmail.com (amrit poudel) Date: Thu, 8 Sep 2011 03:15:10 +0000 Subject: [petsc-users] PETSc Error when writing a vector object to a binary file In-Reply-To: References: <4E671655.5050100@lycos.com>, , , , , , , , , , Message-ID: Thanks. That worked out okay! Date: Wed, 7 Sep 2011 21:43:17 -0500 From: abhyshr at hawk.iit.edu To: petsc-users at mcs.anl.gov Subject: Re: [petsc-users] PETSc Error when writing a vector object to a binary file Use VecView(), http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/Vec/VecView.html, to write the vector to a binary file and then use PetscBinaryRead() in Matlab ierr = PetscViewerBinaryOpen(PETSC_COMM_WORLD, filesol, FILE_MODE_WRITE, &binviewersol) ; CHKERRQ(ierr) ; ierr = VecView(b,binviewersol);CHKERRQ(ierr); In Matlab, b = PetscBinaryRead(binviewersol,'complex', true); See the function PetscBinaryRead() in $PETSC_DIR/bin/matlab On Wed, Sep 7, 2011 at 9:17 PM, amrit poudel wrote: I am using the following sequence of commands to write a Vec b to a binary file and then eventually read it in MATLAB, but I get an error when I try writing this vector to a file in PETSc. ierr = PetscViewerBinaryOpen(PETSC_COMM_WORLD, filesol, FILE_MODE_WRITE, &binviewersol) ; CHKERRQ(ierr) ; ierr = PetscViewerBinaryGetDescriptor(binviewersol, &fd); CHKERRQ(ierr) ; ierr = PetscBinaryWrite(fd, b, 5, PETSC_SCALAR, PETSC_FALSE); CHKERRQ(ierr) ; Here is the error : [1]PETSC ERROR: Write to file failed! [1]PETSC ERROR: Error writing to file.! NOTE : Petsc is configured with complex option and Vec b has complex entries. Length of Vec b is 5 . -------------- next part -------------- An HTML attachment was scrubbed... URL: From rongtian at ncic.ac.cn Thu Sep 8 02:02:35 2011 From: rongtian at ncic.ac.cn (Tian(ICT)) Date: Thu, 8 Sep 2011 15:02:35 +0800 Subject: [petsc-users] nonzero prescribed boundary condition In-Reply-To: References: <5EB9541F-90AE-4373-ACCF-108106E59AD4@mcs.anl.gov><040E28514B5D407BA8F7617141426051@rti458laptop><8664863E669F4B7F8A045B2FEC5E22CC@rti458laptop><8DC60B1B-A0BB-4571-90CE-C0CD9E05D446@mcs.anl.gov><0B5B1110339742BC8990AB61531039A8@rti458laptop> Message-ID: Hi, Again about prescribed boundary condition issue. >>>> So my quick follow up question is How a displacement-controled load is >>>> done corrently in Petsc 2.3.3? >>> >>> To do it in 2.3.3 simply have for those components of F() the formula >>> F_i = x_i - givenvalue_i and in your Jacobian just use MatZeroRows() for >>> those rows >>> in a nonlinear solving, I followed the above suggestion: in FormFunction, set F_i=x_i-given_value (nonzero) and in FormJacobian, MatZeroRows for the corresponding rows. However, the convergence is must slowed down as the exact satisfaction of xi=givenvalue has to be satisfied in an interative manner. Acturally convergence is not just slow but almost not practical (normally > 50 interations). Am I am missing something? Rong ----- Original Message ----- From: "Tian(ICT)" To: "PETSc users list" Sent: Wednesday, May 11, 2011 2:16 PM Subject: Re: [petsc-users] nonzero prescribed boundary condition > With additional -snes_type test: > > "-pc_type > lu -ksp_monitor_true_residual -snes_monitor -ksp_monitor -snes_type test" > > atol=1e-050, rtol=1e-008, stol=1e-008, maxit=50, maxf=10000 > Testing hand-coded Jacobian, if the ratio is > O(1.e-8), the hand-coded Jacobian is probably correct. > Run with -snes_test_display to show difference > of hand-coded and finite difference Jacobian. > Norm of matrix ratio 9.05702e-008 difference 3.94971 > Norm of matrix ratio 3.6015e-007 difference 15.7059 > Norm of matrix ratio 3.73941e-007 difference 16.3073 > [0]PETSC ERROR: SNESSolve() line 1871 in > src/snes/interface/C:\cygwin\home\PETSC-~1.3-P\src\snes\INTERF~1\snes.c > [0]PETSC ERROR: User provided function() line 572 in > unknowndirectory/c:\cygwin\home\rabbit\pgfem - csrbf\domain.cpp > [0]PETSC ERROR: User provided function() line 40 in > unknowndirectory/c:\cygwin\home\rabbit\pgfem - csrbf\main.cpp > > > -snes_mf_operator does not work: > > "-pc_type > lu -ksp_monitor_true_residual -snes_monitor -ksp_monitor -snes_ms_operator" > > [0]PETSC ERROR: --------------------- Error > Message ------------------------------------ > [0]PETSC ERROR: Object is in wrong state! > [0]PETSC ERROR: Not for unassembled matrix! > [0]PETSC > ERROR: ------------------------------------------------------------------------ > [0]PETSC ERROR: Petsc Release Version 2.3.3, Patch 15, Tue Sep 23 10:02:49 > CDT 2008 HG revision: 31306062cd1a6f6a2496fccb4878f485c9b91760 > [0]PETSC ERROR: See docs/changes/index.html for recent updates. > [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. > [0]PETSC ERROR: See docs/index.html for manual pages. > [0]PETSC > ERROR: ------------------------------------------------------------------------ > [0]PETSC ERROR: pgfem on a cygwin-gc named RTI458-LAPTOP by rabbit Wed May > 11 14:16:08 2011 > [0]PETSC ERROR: Libraries linked from /home/petsc-2.3.3-p15/lib/cygwin-gcc > [0]PETSC ERROR: Configure run at Wed Dec 1 17:02:20 2010 > [0]PETSC ERROR: Configure options --with-cc="win32fe > cl" --with-cxx="win32fe > cl" --download-c-blas-lapack=1 --with-clanguage=cxx --with-fc=0 --useThreads=0 > --with-shared=0 > [0]PETSC > ERROR: ------------------------------------------------------------------------ > [0]PETSC ERROR: MatDuplicate() line 3218 in > src/mat/interface/C:\cygwin\home\PETSC-~1.3-P\src\mat\INTERF~1\matrix.c > atol=1e-050, rtol=1e-008, stol=1e-008, maxit=50, maxf=10000 > 0 SNES Function norm 7.071067811865e-002 > 0 KSP Residual norm 9.965778978387e-002 > [0]PETSC ERROR: --------------------- Error > Message ------------------------------------ > [0]PETSC ERROR: Null argument, when expecting valid pointer! > [0]PETSC ERROR: Null Object: Parameter # 1! > [0]PETSC > ERROR: ------------------------------------------------------------------------ > [0]PETSC ERROR: Petsc Release Version 2.3.3, Patch 15, Tue Sep 23 10:02:49 > CDT 2008 HG revision: 31306062cd1a6f6a2496fccb4878f485c9b91760 > [0]PETSC ERROR: See docs/changes/index.html for recent updates. > [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. > [0]PETSC ERROR: See docs/index.html for manual pages. > [0]PETSC > ERROR: ------------------------------------------------------------------------ > [0]PETSC ERROR: pgfem on a cygwin-gc named RTI458-LAPTOP by rabbit Wed May > 11 14:16:08 2011 > [0]PETSC ERROR: Libraries linked from /home/petsc-2.3.3-p15/lib/cygwin-gcc > [0]PETSC ERROR: Configure run at Wed Dec 1 17:02:20 2010 > [0]PETSC ERROR: Configure options --with-cc="win32fe > cl" --with-cxx="win32fe > cl" --download-c-blas-lapack=1 --with-clanguage=cxx --with-fc=0 --useThreads=0 > --with-shared=0 > [0]PETSC > ERROR: ------------------------------------------------------------------------ > [0]PETSC ERROR: PetscObjectGetComm() line 34 in > src/sys/objects/C:\cygwin\home\PETSC-~1.3-P\src\sys\objects\gcomm.c > [0]PETSC ERROR: VecNormBegin() line 495 in > src/vec/vec/utils/C:\cygwin\home\PETSC-~1.3-P\src\vec\vec\utils\comb.c > [0]PETSC ERROR: MatMFFDCompute_WP() line 73 in > src/mat/impls/mffd/C:\cygwin\home\PETSC-~1.3-P\src\mat\impls\mffd\wp.c > [0]PETSC ERROR: MatMult_MFFD() line 294 in > src/mat/impls/mffd/C:\cygwin\home\PETSC-~1.3-P\src\mat\impls\mffd\mffd.c > [0]PETSC ERROR: MatMult() line 1632 in > src/mat/interface/C:\cygwin\home\PETSC-~1.3-P\src\mat\INTERF~1\matrix.c > [0]PETSC ERROR: KSPDefaultBuildResidual() line 605 in > src/ksp/ksp/interface/C:\cygwin\home\PETSC-~1.3-P\src\ksp\ksp\INTERF~1\iterativ.c > [0]PETSC ERROR: KSPBuildResidual() line 1587 in > src/ksp/ksp/interface/C:\cygwin\home\PETSC-~1.3-P\src\ksp\ksp\INTERF~1\itfunc.c > [0]PETSC ERROR: KSPMonitorTrueResidualNorm() line 258 in > src/ksp/ksp/interface/C:\cygwin\home\PETSC-~1.3-P\src\ksp\ksp\INTERF~1\iterativ.c > [0]PETSC ERROR: GMREScycle() line 155 in > src/ksp/ksp/impls/gmres/C:\cygwin\home\PETSC-~1.3-P\src\ksp\ksp\impls\gmres\gmres.c > [0]PETSC ERROR: KSPSolve_GMRES() line 241 in > src/ksp/ksp/impls/gmres/C:\cygwin\home\PETSC-~1.3-P\src\ksp\ksp\impls\gmres\gmres.c > [0]PETSC ERROR: KSPSolve() line 379 in > src/ksp/ksp/interface/C:\cygwin\home\PETSC-~1.3-P\src\ksp\ksp\INTERF~1\itfunc.c > [0]PETSC ERROR: SNES_KSPSolve() line 2578 in > src/snes/interface/C:\cygwin\home\PETSC-~1.3-P\src\snes\INTERF~1\snes.c > [0]PETSC ERROR: SNESSolve_LS() line 184 in > src/snes/impls/ls/C:\cygwin\home\PETSC-~1.3-P\src\snes\impls\ls\ls.c > [0]PETSC ERROR: SNESSolve() line 1871 in > src/snes/interface/C:\cygwin\home\PETSC-~1.3-P\src\snes\INTERF~1\snes.c > [0]PETSC ERROR: User provided function() line 574 in > unknowndirectory/c:\cygwin\home\rabbit\pgfem - csrbf\domain.cpp > [0]PETSC ERROR: User provided function() line 40 in > unknowndirectory/c:\cygwin\home\rabbit\pgfem - csrbf\main.cpp > > > Rong > > ----- Original Message ----- > From: "Barry Smith" > To: "PETSc users list" > Sent: Wednesday, May 11, 2011 11:05 AM > Subject: Re: [petsc-users] nonzero prescribed boundary condition > > > > Ok, the linear solver looks like it is working ok. The likely problem is > that Jacobian does not match the function evaluation. > > Run the same thing but with the additional option -snes_mf_operator > > Then run with -snes_type test (instead of -snes_mf_operator). > > Barry > > On May 10, 2011, at 8:14 PM, Tian(ICT) wrote: > >> Dear Barry, here is the output using -pc_type >> lu -ksp_monitor_true_residual -snes_monitor -ksp_monitor >> the attached is the same and for clear reference. Thanks again for helps. >> >> atol=1e-050, rtol=1e-008, stol=1e-008, maxit=50, maxf=10000 >> 0 SNES Function norm 7.071067811865e-002 >> 0 KSP Residual norm 9.965778978387e-002 >> 0 KSP preconditioned resid norm 9.965778978387e-002 true resid norm >> 7.071067811865e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 6.823187455811e-017 >> 1 KSP preconditioned resid norm 6.823187455811e-017 true resid norm >> 8.847298885656e-011 ||Ae||/||Ax|| 1.251197007446e-009 >> 1 SNES Function norm 6.401926523423e-002 >> 0 KSP Residual norm 8.969200212486e-002 >> 0 KSP preconditioned resid norm 8.969200212486e-002 true resid norm >> 6.401926523423e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 1.106757475780e-016 >> 1 KSP preconditioned resid norm 1.106757475780e-016 true resid norm >> 6.211830067439e-011 ||Ae||/||Ax|| 9.703063671087e-010 >> 2 SNES Function norm 5.849992149767e-002 >> 0 KSP Residual norm 8.072279488157e-002 >> 0 KSP preconditioned resid norm 8.072279488157e-002 true resid norm >> 5.849992149767e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 1.268750073799e-017 >> 1 KSP preconditioned resid norm 1.268750073799e-017 true resid norm >> 3.802431036387e-011 ||Ae||/||Ax|| 6.499890835816e-010 >> 3 SNES Function norm 5.376618503592e-002 >> 0 KSP Residual norm 7.265050969883e-002 >> 0 KSP preconditioned resid norm 7.265050969883e-002 true resid norm >> 5.376618503592e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 2.677655733356e-017 >> 1 KSP preconditioned resid norm 2.677655733356e-017 true resid norm >> 8.120397788686e-011 ||Ae||/||Ax|| 1.510316899602e-009 >> 4 SNES Function norm 4.956894354459e-002 >> 0 KSP Residual norm 6.538545411661e-002 >> 0 KSP preconditioned resid norm 6.538545411661e-002 true resid norm >> 4.956894354459e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 9.557004153175e-017 >> 1 KSP preconditioned resid norm 9.557004153175e-017 true resid norm >> 2.944250802029e-011 ||Ae||/||Ax|| 5.939708598754e-010 >> 5 SNES Function norm 4.575418613137e-002 >> 0 KSP Residual norm 5.884690496914e-002 >> 0 KSP preconditioned resid norm 5.884690496914e-002 true resid norm >> 4.575418613137e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 5.470969262115e-017 >> 1 KSP preconditioned resid norm 5.470969262115e-017 true resid norm >> 3.659003166095e-011 ||Ae||/||Ax|| 7.997089393284e-010 >> 6 SNES Function norm 4.223022245585e-002 >> 0 KSP Residual norm 5.296221144636e-002 >> 0 KSP preconditioned resid norm 5.296221144636e-002 true resid norm >> 4.223022245585e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 8.255198782390e-017 >> 1 KSP preconditioned resid norm 8.255198782390e-017 true resid norm >> 1.955545658933e-011 ||Ae||/||Ax|| 4.630678090739e-010 >> 7 SNES Function norm 3.894430065910e-002 >> 0 KSP Residual norm 4.766598785088e-002 >> 0 KSP preconditioned resid norm 4.766598785088e-002 true resid norm >> 3.894430065910e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 3.322615478395e-017 >> 1 KSP preconditioned resid norm 3.322615478395e-017 true resid norm >> 3.485328148673e-011 ||Ae||/||Ax|| 8.949520442496e-010 >> 8 SNES Function norm 3.586683371135e-002 >> 0 KSP Residual norm 4.289938708067e-002 >> 0 KSP preconditioned resid norm 4.289938708067e-002 true resid norm >> 3.586683371135e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 6.181358328498e-017 >> 1 KSP preconditioned resid norm 6.181358328498e-017 true resid norm >> 3.246902818086e-011 ||Ae||/||Ax|| 9.052660862724e-010 >> 9 SNES Function norm 3.298130202025e-002 >> 0 KSP Residual norm 3.860944676473e-002 >> 0 KSP preconditioned resid norm 3.860944676473e-002 true resid norm >> 3.298130202025e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 4.635174776374e-017 >> 1 KSP preconditioned resid norm 4.635174776374e-017 true resid norm >> 1.497516842272e-011 ||Ae||/||Ax|| 4.540502498513e-010 >> 10 SNES Function norm 3.027806208930e-002 >> 0 KSP Residual norm 3.474850078591e-002 >> 0 KSP preconditioned resid norm 3.474850078591e-002 true resid norm >> 3.027806208930e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 2.389914053685e-017 >> 1 KSP preconditioned resid norm 2.389914053685e-017 true resid norm >> 6.007440888596e-011 ||Ae||/||Ax|| 1.984090286517e-009 >> 11 SNES Function norm 2.749422924729e-002 >> 0 KSP Residual norm 3.081350823297e-002 >> 0 KSP preconditioned resid norm 3.081350823297e-002 true resid norm >> 2.749422924729e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 2.640567497647e-017 >> 1 KSP preconditioned resid norm 2.640567497647e-017 true resid norm >> 1.281638295853e-011 ||Ae||/||Ax|| 4.661481085089e-010 >> 12 SNES Function norm 2.437488247885e-002 >> 0 KSP Residual norm 2.633007441879e-002 >> 0 KSP preconditioned resid norm 2.633007441879e-002 true resid norm >> 2.437488247885e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 2.772331460094e-017 >> 1 KSP preconditioned resid norm 2.772331460094e-017 true resid norm >> 1.918212496143e-011 ||Ae||/||Ax|| 7.869627670236e-010 >> 13 SNES Function norm 2.079664278637e-002 >> 0 KSP Residual norm 2.104738289397e-002 >> 0 KSP preconditioned resid norm 2.104738289397e-002 true resid norm >> 2.079664278637e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 1.650632708670e-017 >> 1 KSP preconditioned resid norm 1.650632708670e-017 true resid norm >> 2.316371967362e-011 ||Ae||/||Ax|| 1.113820144509e-009 >> 14 SNES Function norm 1.657344626858e-002 >> 0 KSP Residual norm 1.454141853505e-002 >> 0 KSP preconditioned resid norm 1.454141853505e-002 true resid norm >> 1.657344626858e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 1.129401160070e-017 >> 1 KSP preconditioned resid norm 1.129401160070e-017 true resid norm >> 7.885499327559e-012 ||Ae||/||Ax|| 4.757911661686e-010 >> 15 SNES Function norm 1.484243752612e-002 >> 0 KSP Residual norm 5.241948491751e-009 >> 0 KSP preconditioned resid norm 5.241948491751e-009 true resid norm >> 1.484243752612e-002 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 2.729506849025e-024 >> 1 KSP preconditioned resid norm 2.729506849025e-024 true resid norm >> 6.386677851085e-018 ||Ae||/||Ax|| 4.302984492839e-016 >> 16 SNES Function norm 2.828002157497e-008 >> 0 KSP Residual norm 6.042518362322e-015 >> 0 KSP preconditioned resid norm 6.042518362322e-015 true resid norm >> 2.828002157497e-008 ||Ae||/||Ax|| 1.000000000000e+000 >> 1 KSP Residual norm 6.272441346127e-030 >> 1 KSP preconditioned resid norm 6.272441346127e-030 true resid norm >> 1.112857698032e-023 ||Ae||/||Ax|| 3.935137372797e-016 >> 17 SNES Function norm 2.960967020289e-008 >> STEP 0 (Newton iterations: 17) >> >> diverged reason: -6 >> >> >> ----- Original Message ----- From: "Barry Smith" >> To: "PETSc users list" >> Sent: Wednesday, May 11, 2011 2:54 AM >> Subject: Re: [petsc-users] nonzero prescribed boundary condition >> >> >> >> Use -pc_type lu -ksp_monitor_true_residual -snes_monitor -ksp_monitor >> and >> send the outputs >> >> >> Barry >> >> On May 9, 2011, at 10:43 PM, Tian(ICT) wrote: >> >>> by the way, the increment size is like that >>> for a 100 lengh model, the increment is set to 0.05, >>> the engineering strain is around 5%% per load step. >>> This is already too small increment size for a large deformation >>> analysis. >>> a 0.5 increment size leads to both linear search and trust region >>> failed. >>> linear search failed for 0.05 while trust region converges with 17 >>> Newton >>> iterations each load step. >>> Rong >>> >>> ----- Original Message ----- From: "Tian(ICT)" >>> To: "PETSc users list" >>> Sent: Tuesday, May 10, 2011 11:37 AM >>> Subject: Re: [petsc-users] nonzero prescribed boundary condition >>> >>> >>>> First, thanks again, the issue was gone. >>>> >>>> I just followed up with some test results. >>>> I have tested SNES using one finite element for a geometric large >>>> deformation problem. >>>> Those are just the very early test results so they may be not telling >>>> what happened exactly. >>>> For the displacement controlled load, I found that convergence is much >>>> slower than that of force loading. >>>> Even worse, linear search is so sensitive to the displacement increment >>>> and diverged no matter what the increment size was used (too small >>>> incremnt also led to diverged soloution (-6 reason), trust region works >>>> well in the sense of not sensitive to the displacement increment, but >>>> during each load step, it requires around ten to several tens of Newton >>>> interations whereas for the force loading case and the almost same >>>> amount of deformation, this is normally 3. This is against my >>>> expectation. Any hint? >>>> >>>> Rong >>>> >>>> ----- Original Message ----- From: "Barry Smith" >>>> To: "PETSc users list" >>>> Sent: Tuesday, May 10, 2011 10:22 AM >>>> Subject: Re: [petsc-users] nonzero prescribed boundary condition >>>> >>>> >>>> >>>> On May 9, 2011, at 9:15 PM, Tian(ICT) wrote: >>>> >>>>> Dear Barry, Thanks a lot for quick answering. >>>>> I checked the development documents and found the new version of >>>>> MatZeroRows() does support the nonzero prescribed boundary conditions. >>>>> >>>>> I followed up with more details. >>>>> I am using Petasc 2.3.3. to solve a nonlinear problem, e.g. using SNES >>>>> solvers. >>>>> I used a displacement-controlled load (as this type of loading works >>>>> well for all cases). >>>>> This is the reason the nonzero prescribed boundary came up. >>>>> >>>>> In FormJacobian, I modified Jacobian and residual to satisfy the >>>>> nonzero prescribed boundary. >>>>> In FormFunction, I modified the solution to the known solution(this >>>>> should not be necessary as the modified Jacobian and rhs should give >>>>> the prescribed solution also) >>>> >>>> You should not do it this way. See below. >>>>> >>>>> Now I found another issue, no matter if I prescried the solution or >>>>> not >>>>> in FormFunction, >>>>> SNES solver always call FormFunction and never call FormJacobian. >>>> >>>> The only reason it would not call FormJacobian is if decided that the >>>> residual norm was small enough before any Newton steps; for example if >>>> the FormFunction() computed exactly the zero function initially. When >>>> you run with -snes_monitor -ksp_monitor what does it print for residual >>>> norms. >>>> >>>>> Of course the solver finally diverged or converged to a zero solution. >>>>> >>>>> So my quick follow up question is How a displacement-controled load is >>>>> done corrently in Petsc 2.3.3? >>>> >>>> To do it in 2.3.3 simply have for those components of F() the formula >>>> F_i = x_i - givenvalue_i and in your Jacobian just use MatZeroRows() >>>> for >>>> those rows >>>> >>>> We strongly urge you to upgrade to the latest PETSc before doing >>>> anything further. >>>> >>>> >>>> Barry >>>> >>>>> >>>>> Rong >>>>> >>>>> ----- Original Message ----- From: "Barry Smith" >>>>> To: "PETSc users list" >>>>> Sent: Tuesday, May 10, 2011 9:31 AM >>>>> Subject: Re: [petsc-users] nonzero prescribed boundary condition >>>>> >>>>> >>>>> >>>>> In petsc-dev >>>>> http://www.mcs.anl.gov/petsc/petsc-as/developers/index.html we have >>>>> modified the calling sequence for MatZeroRows() so that it can >>>>> automatically adjust the appropriate right hand side values for the >>>>> zeroed rows to support zero or non-zero prescribed boundary conditions >>>>> easily. >>>>> >>>>> Barry >>>>> >>>>> On May 9, 2011, at 8:18 PM, Tian(ICT) wrote: >>>>> >>>>>> Dear all, >>>>>> >>>>>> I got this question long ago and searched the prior posting but did >>>>>> not find the solution. >>>>>> The question is about nonzero prescribed boundary condition. >>>>>> My understanding is that MatZeroRows() works only for zero prescribed >>>>>> value, not non-zero value. >>>>>> For the non-zero values, we have to remove the rows associated with >>>>>> the boundary, but this >>>>>> will lead to a zero dignal and accordingly the rows in r.h.s should >>>>>> also be removed. >>>>>> My question is that does MatZeroRows() also works for nonzero >>>>>> prescribed boundary and if so how to do it simply? >>>>>> >>>>>> Rong >>>>> >>>>> >>>> >>>> >>> >> >> > > > From jedbrown at mcs.anl.gov Thu Sep 8 04:09:56 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Thu, 8 Sep 2011 11:09:56 +0200 Subject: [petsc-users] Fwd: nonzero prescribed boundary condition In-Reply-To: References: <5EB9541F-90AE-4373-ACCF-108106E59AD4@mcs.anl.gov> <040E28514B5D407BA8F7617141426051@rti458laptop> <8664863E669F4B7F8A045B2FEC5E22CC@rti458laptop> <8DC60B1B-A0BB-4571-90CE-C0CD9E05D446@mcs.anl.gov> <0B5B1110339742BC8990AB61531039A8@rti458laptop> Message-ID: On Thu, Sep 8, 2011 at 09:02, Tian(ICT) wrote: > in a nonlinear solving, I followed the above suggestion: in FormFunction, > set F_i=x_i-given_value (nonzero) and in FormJacobian, MatZeroRows for the > corresponding rows. > However, the convergence is must slowed down as the exact satisfaction of > xi=givenvalue > has to be satisfied in an interative manner. Acturally convergence is not > just slow but almost > not practical (normally > 50 interations). > What are you using for the "diag" argument to MatZeroRows? If you place 1 on the diagonal, those equations should be satisfied after one application of the preconditioner. Why do you think that 50 iterations is more than the solver algorithm you are using should need? Note that for some problems, it is desirable to preserve symmetry. The implementation of boundary conditions described here does not preserve symmetry. To preserve symmetry, the column should also be zeroed. In residual evaluation, this is implemented by using the correct Dirichlet value instead of the current approximation. It is much better to zero the column by modifying the arguments to MatSetValues instead of "applying" boundary conditions after assembly. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rongtian at ncic.ac.cn Thu Sep 8 07:58:40 2011 From: rongtian at ncic.ac.cn (Tian(ICT)) Date: Thu, 8 Sep 2011 20:58:40 +0800 Subject: [petsc-users] Fwd: nonzero prescribed boundary condition In-Reply-To: References: <5EB9541F-90AE-4373-ACCF-108106E59AD4@mcs.anl.gov><040E28514B5D407BA8F7617141426051@rti458laptop><8664863E669F4B7F8A045B2FEC5E22CC@rti458laptop><8DC60B1B-A0BB-4571-90CE-C0CD9E05D446@mcs.anl.gov><0B5B1110339742BC8990AB61531039A8@rti458laptop> Message-ID: <8A1482FA5FCE46CDBBBB7BA3593EF18B@rti458laptop> By 50 iterations, I meant my implementation of this prescribed boundary condition was not working, not converging. Yes, I put 1 on the diagonal. Can I just enforce the solution x within each Newton iteration to the prescribed values to satisfy the prescribed boundary condition, not MatZeroRowsing Jacobian? ----- Original Message ----- From: Jed Brown To: PETSc users list Sent: Thursday, September 08, 2011 5:09 PM Subject: [petsc-users] Fwd: nonzero prescribed boundary condition On Thu, Sep 8, 2011 at 09:02, Tian(ICT) wrote: in a nonlinear solving, I followed the above suggestion: in FormFunction, set F_i=x_i-given_value (nonzero) and in FormJacobian, MatZeroRows for the corresponding rows. However, the convergence is must slowed down as the exact satisfaction of xi=givenvalue has to be satisfied in an interative manner. Acturally convergence is not just slow but almost not practical (normally > 50 interations). What are you using for the "diag" argument to MatZeroRows? If you place 1 on the diagonal, those equations should be satisfied after one application of the preconditioner. Why do you think that 50 iterations is more than the solver algorithm you are using should need? Note that for some problems, it is desirable to preserve symmetry. The implementation of boundary conditions described here does not preserve symmetry. To preserve symmetry, the column should also be zeroed. In residual evaluation, this is implemented by using the correct Dirichlet value instead of the current approximation. It is much better to zero the column by modifying the arguments to MatSetValues instead of "applying" boundary conditions after assembly. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jedbrown at mcs.anl.gov Thu Sep 8 08:09:30 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Thu, 8 Sep 2011 15:09:30 +0200 Subject: [petsc-users] Fwd: nonzero prescribed boundary condition In-Reply-To: <8A1482FA5FCE46CDBBBB7BA3593EF18B@rti458laptop> References: <5EB9541F-90AE-4373-ACCF-108106E59AD4@mcs.anl.gov> <040E28514B5D407BA8F7617141426051@rti458laptop> <8664863E669F4B7F8A045B2FEC5E22CC@rti458laptop> <8DC60B1B-A0BB-4571-90CE-C0CD9E05D446@mcs.anl.gov> <0B5B1110339742BC8990AB61531039A8@rti458laptop> <8A1482FA5FCE46CDBBBB7BA3593EF18B@rti458laptop> Message-ID: On Thu, Sep 8, 2011 at 14:58, Tian(ICT) wrote: > By 50 iterations, I meant my implementation of this prescribed boundary > condition was not working, > not converging. Yes, I put 1 on the diagonal.\ > Is the system singular? What equations are you solving? Do boundary conditions give you a constant null space? Incompressible flow is this way. Run with -ksp_monitor_true_residual -pc_type lu -snes_view and send the output. > Can I just enforce the solution x within each Newton iteration to the > prescribed values to satisfy the prescribed boundary condition, not > MatZeroRowsing Jacobian? > It's important for the linear system to be nonsingular. -------------- next part -------------- An HTML attachment was scrubbed... URL: From amrit_pou at hotmail.com Thu Sep 8 12:57:54 2011 From: amrit_pou at hotmail.com (amrit poudel) Date: Thu, 8 Sep 2011 17:57:54 +0000 Subject: [petsc-users] Fwd: nonzero prescribed boundary condition In-Reply-To: References: , <5EB9541F-90AE-4373-ACCF-108106E59AD4@mcs.anl.gov>, <040E28514B5D407BA8F7617141426051@rti458laptop>, , , <8664863E669F4B7F8A045B2FEC5E22CC@rti458laptop>, <8DC60B1B-A0BB-4571-90CE-C0CD9E05D446@mcs.anl.gov>, <0B5B1110339742BC8990AB61531039A8@rti458laptop>, , , , , <8A1482FA5FCE46CDBBBB7BA3593EF18B@rti458laptop>, Message-ID: After running my simulation multiple times on a multiprocessor computer I've just verified that using iterative solver (default gmres) in PETSc to solve a linear system of equations ( Cx=b) with more than 2 processors setting ALWAYS lead to erroneous result. Running identical code with identical setting except for the number of processors ( set this to 2) ALWAYS gives me correct result . I am really not sure what is the point behind including iterative solvers if they result into erroneous result on a multiprocessor computer. The result I get from multiprocessor computer is a complete garbage, so I am really not talking about small percentage of error here. Also, if somebody could enlighten why the iterative solvers are error prone on multiprocessors that will be highly appreciated. I am very hopeful that there is a way around to this problem, because PETSc is such a powerful and useful library that I really do not want to give up on this and start something else from scratch. Would you think that a DIRECT SOLVER would circumvent this problem? My problem is that I have a very large system of equations and the size of a sparse coefficient matrix is huge ( > 1e+8). I assemble this matrix in MATLAB, write to a binary file, and read it in PETSc. So I really need to be able to solve this system of equations in a cluster of computers (which inherently has multiprocessors and distributed memory setting). Does this mean I am completely out of luck with PETSc's iterative solver package and the only hope for me is the direct solver? I do have MUMPS downloaded and compiled with PETSc, so I will give that a try and see what results I obtain, but I am really surprised that iterative solvers are no good in a large multiprocessor settings. Any insights, suggestions/advice will be highly appreciated. Thanks. PS (I can attach my entire code, plots that compare the results obtained by solving Cx=b in 2 processors vs 12 or 6 processors if any body wants to take a look at it. I get garbage if I run iterative solver on 12 processors) -------------- next part -------------- An HTML attachment was scrubbed... URL: From amrit_pou at hotmail.com Thu Sep 8 12:59:29 2011 From: amrit_pou at hotmail.com (amrit poudel) Date: Thu, 8 Sep 2011 17:59:29 +0000 Subject: [petsc-users] iterative solver in PETSc on a multiprocessor computer In-Reply-To: References: , , <5EB9541F-90AE-4373-ACCF-108106E59AD4@mcs.anl.gov>, , <040E28514B5D407BA8F7617141426051@rti458laptop>, , , , , , <8664863E669F4B7F8A045B2FEC5E22CC@rti458laptop>, , <8DC60B1B-A0BB-4571-90CE-C0CD9E05D446@mcs.anl.gov>, , <0B5B1110339742BC8990AB61531039A8@rti458laptop>, , , , , , , , , , <8A1482FA5FCE46CDBBBB7BA3593EF18B@rti458laptop>, , , Message-ID: After running my simulation multiple times on a multiprocessor computer I've just verified that using iterative solver (default gmres) in PETSc to solve a linear system of equations ( Cx=b) with more than 2 processors setting ALWAYS lead to erroneous result. Running identical code with identical setting except for the number of processors ( set this to 2) ALWAYS gives me correct result . I am really not sure what is the point behind including iterative solvers if they result into erroneous result on a multiprocessor computer. The result I get from multiprocessor computer is a complete garbage, so I am really not talking about small percentage of error here. Also, if somebody could enlighten why the iterative solvers are error prone on multiprocessors that will be highly appreciated. I am very hopeful that there is a way around to this problem, because PETSc is such a powerful and useful library that I really do not want to give up on this and start something else from scratch. Would you think that a DIRECT SOLVER would circumvent this problem? My problem is that I have a very large system of equations and the size of a sparse coefficient matrix is huge ( > 1e+8). I assemble this matrix in MATLAB, write to a binary file, and read it in PETSc. So I really need to be able to solve this system of equations in a cluster of computers (which inherently has multiprocessors and distributed memory setting). Does this mean I am completely out of luck with PETSc's iterative solver package and the only hope for me is the direct solver? I do have MUMPS downloaded and compiled with PETSc, so I will give that a try and see what results I obtain, but I am really surprised that iterative solvers are no good in a large multiprocessor settings. Any insights, suggestions/advice will be highly appreciated. Thanks. PS (I can attach my entire code, plots that compare the results obtained by solving Cx=b in 2 processors vs 12 or 6 processors if any body wants to take a look at it. I get garbage if I run iterative solver on 12 processors) -------------- next part -------------- An HTML attachment was scrubbed... URL: From jedbrown at mcs.anl.gov Thu Sep 8 13:07:04 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Thu, 8 Sep 2011 20:07:04 +0200 Subject: [petsc-users] Fwd: nonzero prescribed boundary condition In-Reply-To: References: <5EB9541F-90AE-4373-ACCF-108106E59AD4@mcs.anl.gov> <040E28514B5D407BA8F7617141426051@rti458laptop> <8664863E669F4B7F8A045B2FEC5E22CC@rti458laptop> <8DC60B1B-A0BB-4571-90CE-C0CD9E05D446@mcs.anl.gov> <0B5B1110339742BC8990AB61531039A8@rti458laptop> <8A1482FA5FCE46CDBBBB7BA3593EF18B@rti458laptop> Message-ID: On Thu, Sep 8, 2011 at 19:57, amrit poudel wrote: > After running my simulation multiple times on a multiprocessor computer > I've just verified that using iterative solver (default gmres) in PETSc to > solve a linear system of equations ( Cx=b) with more than 2 processors > setting ALWAYS lead to erroneous result. Running identical code with > identical setting except for the number of processors ( set this to 2) > ALWAYS gives me correct result . > You have to explain what "erroneous result" means here. > > I am really not sure what is the point behind including iterative solvers > if they result into erroneous result on a multiprocessor computer. The > result I get from multiprocessor computer is a complete garbage, so I am > really not talking about small percentage of error here. Also, if somebody > could enlighten why the iterative solvers are error prone on multiprocessors > that will be highly appreciated. > Well let's not jump to conclusions. Iterative solvers can fail, as can direct solvers, but it's more common that (a) the problem you are building is not actually the same or the results are being misinterpreted or (b) the default methods are not working well and needs some algorithmic adjustments for your problem. Please explain what kind of problem you are solving, how you are going about it, and what symptoms you have observed. -------------- next part -------------- An HTML attachment was scrubbed... URL: From amrit_pou at hotmail.com Thu Sep 8 13:42:23 2011 From: amrit_pou at hotmail.com (amrit poudel) Date: Thu, 8 Sep 2011 18:42:23 +0000 Subject: [petsc-users] Iterative solver in Petsc on a multiprocessor computer In-Reply-To: References: , <5EB9541F-90AE-4373-ACCF-108106E59AD4@mcs.anl.gov>, <040E28514B5D407BA8F7617141426051@rti458laptop>, , , <8664863E669F4B7F8A045B2FEC5E22CC@rti458laptop>, <8DC60B1B-A0BB-4571-90CE-C0CD9E05D446@mcs.anl.gov>, <0B5B1110339742BC8990AB61531039A8@rti458laptop>, , , , , <8A1482FA5FCE46CDBBBB7BA3593EF18B@rti458laptop>, , , Message-ID: Please find the attached plots where I've compared results obtained from 2 processors (2.pdf) numerical simulation in PETSc with analytical result and they agree fine, but for 6 processors (6.pdf) simulation, the numerical result is a complete garbage. My major confusion is this : why does the default GMRES *works* for 2 processors and not for 6 processors? GMRES algorithm is definitely not the problem here because otherwise it would not give me correct result when I run my simulation on 2 processors. Please note that I *do not* change anything in my code other than the number of processors with this command at run time : mpiexec -n 2 test3 Also, I make sure that I write the output to a separate file. It is very hard for me to believe when you say : "(a) the problem you are building is not actually the same or the results are being misinterpreted or " It is the same. Like I said above, I am only changing the number of processors at run time. "(b) the default methods are not working well and needs some algorithmic adjustments for your problem." if this were an issue, then why does it work for two processors? The problem I am simulating is a vector wave equation in two dimensions (or in other words, Maxwell's equations). The coefficient matrix is assembled using finite difference frequency domain method on a Yee grid. I will appreciate any help in the regard. Thanks. -Amrit Date: Thu, 8 Sep 2011 20:07:04 +0200 From: jedbrown at mcs.anl.gov To: petsc-users at mcs.anl.gov Subject: Re: [petsc-users] Fwd: nonzero prescribed boundary condition On Thu, Sep 8, 2011 at 19:57, amrit poudel wrote: After running my simulation multiple times on a multiprocessor computer I've just verified that using iterative solver (default gmres) in PETSc to solve a linear system of equations ( Cx=b) with more than 2 processors setting ALWAYS lead to erroneous result. Running identical code with identical setting except for the number of processors ( set this to 2) ALWAYS gives me correct result . You have to explain what "erroneous result" means here. I am really not sure what is the point behind including iterative solvers if they result into erroneous result on a multiprocessor computer. The result I get from multiprocessor computer is a complete garbage, so I am really not talking about small percentage of error here. Also, if somebody could enlighten why the iterative solvers are error prone on multiprocessors that will be highly appreciated. Well let's not jump to conclusions. Iterative solvers can fail, as can direct solvers, but it's more common that (a) the problem you are building is not actually the same or the results are being misinterpreted or (b) the default methods are not working well and needs some algorithmic adjustments for your problem. Please explain what kind of problem you are solving, how you are going about it, and what symptoms you have observed. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 2.pdf Type: application/save-as Size: 4861 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 6.pdf Type: application/save-as Size: 4638 bytes Desc: not available URL: From bsmith at mcs.anl.gov Thu Sep 8 14:02:09 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Thu, 8 Sep 2011 14:02:09 -0500 Subject: [petsc-users] Fwd: nonzero prescribed boundary condition In-Reply-To: References: , <5EB9541F-90AE-4373-ACCF-108106E59AD4@mcs.anl.gov>, <040E28514B5D407BA8F7617141426051@rti458laptop>, , , <8664863E669F4B7F8A045B2FEC5E22CC@rti458laptop>, <8DC60B1B-A0BB-4571-90CE-C0CD9E05D446@mcs.anl.gov>, <0B5B1110339742BC8990AB61531039A8@rti458laptop>, , , , , <8A1482FA5FCE46CDBBBB7BA3593EF18B@rti458laptop>, Message-ID: Are you running with -ksp_converged_reason and -ksp_monitor_true_residual to see if the iterative method is actually converging and how rapidly. Also if you impose a tight tolerance on the iterative solver say with -ksp_rtol 1.e-12 how much do the "solutions" differ, it should get smaller the smaller you make this tolerance (in double precision you cannot expect to use a tolerance less than 1.e-3 or 1.e-14) If your "big" matrix is very complicated and comes from no well understood simulation it may be that it is very ill-conditioned, in that case you either need to understand the underlying equations real well to develop an appropriate preconditioner or just use parallel direct solvers (which get slow for very large problems but can handle more ill-conditioning.n To use the MUMPS parallel direct solver you can configure PETSc with --download-mumps --download-scalapack --download-blacs and run the program with -pc_type lu -pc_factor_mat_solver_package mumps Barry On Sep 8, 2011, at 12:57 PM, amrit poudel wrote: > After running my simulation multiple times on a multiprocessor computer I've just verified that using iterative solver (default gmres) in PETSc to solve a linear system of equations ( Cx=b) with more than 2 processors setting ALWAYS lead to erroneous result. Running identical code with identical setting except for the number of processors ( set this to 2) ALWAYS gives me correct result . > > I am really not sure what is the point behind including iterative solvers if they result into erroneous result on a multiprocessor computer. The result I get from multiprocessor computer is a complete garbage, so I am really not talking about small percentage of error here. Also, if somebody could enlighten why the iterative solvers are error prone on multiprocessors that will be highly appreciated. > > I am very hopeful that there is a way around to this problem, because PETSc is such a powerful and useful library that I really do not want to give up on this and start something else from scratch. > > > Would you think that a DIRECT SOLVER would circumvent this problem? My problem is that I have a very large system of equations and the size of a sparse coefficient matrix is huge ( > 1e+8). I assemble this matrix in MATLAB, write to a binary file, and read it in PETSc. So I really need to be able to solve this system of equations in a cluster of computers (which inherently has multiprocessors and distributed memory setting). Does this mean I am completely out of luck with PETSc's iterative solver package and the only hope for me is the direct solver? I do have MUMPS downloaded and compiled with PETSc, so I will give that a try and see what results I obtain, but I am really surprised that iterative solvers are no good in a large multiprocessor settings. > > Any insights, suggestions/advice will be highly appreciated. > > Thanks. > > PS (I can attach my entire code, plots that compare the results obtained by solving Cx=b in 2 processors vs 12 or 6 processors if any body wants to take a look at it. I get garbage if I run iterative solver on 12 processors) From abhyshr at mcs.anl.gov Thu Sep 8 14:08:36 2011 From: abhyshr at mcs.anl.gov (Shri) Date: Thu, 8 Sep 2011 14:08:36 -0500 (CDT) Subject: [petsc-users] Iterative solver in Petsc on a multiprocessor computer In-Reply-To: Message-ID: <942867418.286645.1315508916223.JavaMail.root@zimbra.anl.gov> Amrit, These plots are just reiterating the problem you've stated previously but provide no clue as to what's the issue here. I suggest the following : Try running your code with a direct solver, mpiexec -n 6 test3 -ksp_type preonly -pc_factor_mat_solver_package mumps, and see if you are getting the expected results on 6 processors. If you do get the expected results using a direct solver then, as Jed points out, the iterative solver options need to be tuned as Jed points. Note that while a direct solver may be able to solve your problem it is not scalable hence iterative solvers are prefered. If the results are not correct then perhaps something is amiss with your code. The default 'parallel' preconditioner in PETSc is the block-jacobi preconditioner with an ILU(0) factorization which may not be a suitable preconditioner on more number of processors. Try mpiexec -n 6 test3 -ksp_type gmres -pc_type bjacobi -sub_pc_type lu This does an LU factorization on each block instead of ILU(0) Perhaps, you should also attach the code, and the command line options. Shri ----- Original Message ----- Please find the attached plots where I've compared results obtained from 2 processors (2.pdf) numerical simulation in PETSc with analytical result and they agree fine, but for 6 processors (6.pdf) simulation, the numerical result is a complete garbage. My major confusion is this : why does the default GMRES *works* for 2 processors and not for 6 processors? GMRES algorithm is definitely not the problem here because otherwise it would not give me correct result when I run my simulation on 2 processors. Please note that I *do not* change anything in my code other than the number of processors with this command at run time : mpiexec -n 2 test3 Also, I make sure that I write the output to a separate file. It is very hard for me to believe when you say : "(a) the problem you are building is not actually the same or the results are being misinterpreted or " It is the same. Like I said above, I am only changing the number of processors at run time. "(b) the default methods are not working well and needs some algorithmic adjustments for your problem." if this were an issue, then why does it work for two processors? The problem I am simulating is a vector wave equation in two dimensions (or in other words, Maxwell's equations). The coefficient matrix is assembled using finite difference frequency domain method on a Yee grid. I will appreciate any help in the regard. Thanks. -Amrit Date: Thu, 8 Sep 2011 20:07:04 +0200 From: jedbrown at mcs.anl.gov To: petsc-users at mcs.anl.gov Subject: Re: [petsc-users] Fwd: nonzero prescribed boundary condition On Thu, Sep 8, 2011 at 19:57, amrit poudel < amrit_pou at hotmail.com > wrote: After running my simulation multiple times on a multiprocessor computer I've just verified that using iterative solver (default gmres) in PETSc to solve a linear system of equations ( Cx=b) with more than 2 processors setting ALWAYS lead to erroneous result. Running identical code with identical setting except for the number of processors ( set this to 2) ALWAYS gives me correct result . You have to explain what "erroneous result" means here. I am really not sure what is the point behind including iterative solvers if they result into erroneous result on a multiprocessor computer. The result I get from multiprocessor computer is a complete garbage, so I am really not talking about small percentage of error here. Also, if somebody could enlighten why the iterative solvers are error prone on multiprocessors that will be highly appreciated. Well let's not jump to conclusions. Iterative solvers can fail, as can direct solvers, but it's more common that (a) the problem you are building is not actually the same or the results are being misinterpreted or (b) the default methods are not working well and needs some algorithmic adjustments for your problem. Please explain what kind of problem you are solving, how you are going about it, and what symptoms you have observed. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mccomic at mcs.anl.gov Thu Sep 8 15:06:24 2011 From: mccomic at mcs.anl.gov (Mike McCourt) Date: Thu, 8 Sep 2011 15:06:24 -0500 (CDT) Subject: [petsc-users] iterative solver in PETSc on a multiprocessor computer In-Reply-To: Message-ID: <962237593.287045.1315512384951.JavaMail.root@zimbra.anl.gov> When you say that the runs are "identical setting except for the number of processors" does that include running with 3 processors? If the code works on 1 and 3 but not 2, that would be very weird. -Mike ----- Original Message ----- From: "amrit poudel" To: petsc-users at mcs.anl.gov Sent: Thursday, September 8, 2011 12:59:29 PM Subject: [petsc-users] iterative solver in PETSc on a multiprocessor computer After running my simulation multiple times on a multiprocessor computer I've just verified that using iterative solver (default gmres) in PETSc to solve a linear system of equations ( Cx=b) with more than 2 processors setting ALWAYS lead to erroneous result. Running identical code with identical setting except for the number of processors ( set this to 2) ALWAYS gives me correct result . I am really not sure what is the point behind including iterative solvers if they result into erroneous result on a multiprocessor computer. The result I get from multiprocessor computer is a complete garbage, so I am really not talking about small percentage of error here. Also, if somebody could enlighten why the iterative solvers are error prone on multiprocessors that will be highly appreciated. I am very hopeful that there is a way around to this problem, because PETSc is such a powerful and useful library that I really do not want to give up on this and start something else from scratch. Would you think that a DIRECT SOLVER would circumvent this problem? My problem is that I have a very large system of equations and the size of a sparse coefficient matrix is huge ( > 1e+8). I assemble this matrix in MATLAB, write to a binary file, and read it in PETSc. So I really need to be able to solve this system of equations in a cluster of computers (which inherently has multiprocessors and distributed memory setting). Does this mean I am completely out of luck with PETSc's iterative solver package and the only hope for me is the direct solver? I do have MUMPS downloaded and compiled with PETSc, so I will give that a try and see what results I obtain, but I am really surprised that iterative solvers are no good in a large multiprocessor settings. Any insights, suggestions/advice will be highly appreciated. Thanks. PS (I can attach my entire code, plots that compare the results obtained by solving Cx=b in 2 processors vs 12 or 6 processors if any body wants to take a look at it. I get garbage if I run iterative solver on 12 processors) From amrit_pou at hotmail.com Thu Sep 8 15:38:52 2011 From: amrit_pou at hotmail.com (amrit poudel) Date: Thu, 8 Sep 2011 20:38:52 +0000 Subject: [petsc-users] iterative solver in PETSc on a multiprocessor computer In-Reply-To: <962237593.287045.1315512384951.JavaMail.root@zimbra.anl.gov> References: , <962237593.287045.1315512384951.JavaMail.root@zimbra.anl.gov> Message-ID: It works for 2 and 3 processors, and I just found out that it does not work for 1 processor either :(. It also does not work for any number of processors greater than 6. Please check my previous email for further detail. I have attached my source code and everything there. Thanks for your help. -Amrit > Date: Thu, 8 Sep 2011 15:06:24 -0500 > From: mccomic at mcs.anl.gov > To: petsc-users at mcs.anl.gov > Subject: Re: [petsc-users] iterative solver in PETSc on a multiprocessor computer > > When you say that the runs are "identical setting except for the number of processors" does that include running with 3 processors? If the code works on 1 and 3 but not 2, that would be very weird. > > -Mike > > ----- Original Message ----- > From: "amrit poudel" > To: petsc-users at mcs.anl.gov > Sent: Thursday, September 8, 2011 12:59:29 PM > Subject: [petsc-users] iterative solver in PETSc on a multiprocessor computer > > > > > > After running my simulation multiple times on a multiprocessor computer I've just verified that using iterative solver (default gmres) in PETSc to solve a linear system of equations ( Cx=b) with more than 2 processors setting ALWAYS lead to erroneous result. Running identical code with identical setting except for the number of processors ( set this to 2) ALWAYS gives me correct result . > > > I am really not sure what is the point behind including iterative solvers if they result into erroneous result on a multiprocessor computer. The result I get from multiprocessor computer is a complete garbage, so I am really not talking about small percentage of error here. Also, if somebody could enlighten why the iterative solvers are error prone on multiprocessors that will be highly appreciated. > > I am very hopeful that there is a way around to this problem, because PETSc is such a powerful and useful library that I really do not want to give up on this and start something else from scratch. > > > Would you think that a DIRECT SOLVER would circumvent this problem? My problem is that I have a very large system of equations and the size of a sparse coefficient matrix is huge ( > 1e+8). I assemble this matrix in MATLAB, write to a binary file, and read it in PETSc. So I really need to be able to solve this system of equations in a cluster of computers (which inherently has multiprocessors and distributed memory setting). Does this mean I am completely out of luck with PETSc's iterative solver package and the only hope for me is the direct solver? I do have MUMPS downloaded and compiled with PETSc, so I will give that a try and see what results I obtain, but I am really surprised that iterative solvers are no good in a large multiprocessor settings. > > Any insights, suggestions/advice will be highly appreciated. > > Thanks. > > PS (I can attach my entire code, plots that compare the results obtained by solving Cx=b in 2 processors vs 12 or 6 processors if any body wants to take a look at it. I get garbage if I run iterative solver on 12 processors) -------------- next part -------------- An HTML attachment was scrubbed... URL: From jedbrown at mcs.anl.gov Thu Sep 8 15:44:09 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Thu, 8 Sep 2011 22:44:09 +0200 Subject: [petsc-users] iterative solver in PETSc on a multiprocessor computer In-Reply-To: References: <962237593.287045.1315512384951.JavaMail.root@zimbra.anl.gov> Message-ID: On Thu, Sep 8, 2011 at 22:38, amrit poudel wrote: > It works for 2 and 3 processors, and I just found out that it does not work > for 1 processor either :(. It also does not work for any number of > processors greater than 6. Please check my previous email for further > detail. I have attached my source code and everything there. Your attachment was too large so it was scrubbed by the list. You can send it to petsc-maint at mcs.anl.gov. (You can also compress it first.) Please send instructions for how to run your code, what you expect to see, and what you actually see. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jbacks at mcmillan-mcgee.com Thu Sep 8 18:07:57 2011 From: jbacks at mcmillan-mcgee.com (Jonathan Backs) Date: Thu, 8 Sep 2011 17:07:57 -0600 Subject: [petsc-users] Conditional Constraints In-Reply-To: <909877427.287766.1315518146646.JavaMail.root@zimbra.anl.gov> References: <909877427.287766.1315518146646.JavaMail.root@zimbra.anl.gov> Message-ID: Shri, Thanks very much for your response. In fact, -snes_converged_reason revealed that the solution was not converging, as you suggested: "Nonlinear solve did not converge due to DIVERGED_LINE_SEARCH." I had not realized this earlier since the voltage distribution in the solution appeared to be correct, even though the SNES_VI constraints were not satisfied. Given that and your explanation, I am sure something is wrong with my choices of f. I do not want Ve to be constant, however; I would like it to either decrease in response to Imag > Imag_max or increase in response to Imag < Imag_max, providing the condition Vmag <= Vmag_max remains true. Should I then have something like f(Ve_ijk) = Imag - Imag_max? One more thing from your response: > >If a block (with coordinates i, j, k) is not occupied by an electrode, C_e is set to zero in the expression for Vb_ijk and all function and Jacobian entries for Ve, >Vmag, and Imag are either not set or set to zero. > Does this mean f(Ve_ijk) = f(Vmag_ijk) = f(Imag_ijk) = 0? If so, then Ve_ijk, Vmag_ijk, and Imag_ijk would retain their initial values for all time steps. In my present scheme, f(Ve_ijk) = f(Vmag_ijk) = 0 and f(Imag_ijk) is non-zero for electrode blocks; all three are zero for non-electrode blocks. Depending on the choice of initial conditions, I can make f(Vmag_ijk) non-zero as well (until Ve changes to match Ve_max). Thanks again, Jon On 2011-09-08, at 3:42 PM, Shri wrote: > Jon, > > > for Ve_ijk, f = 0.0 + 0.0*i, and all corresponding Jacobian entries are zero as well. Ve_ijk is set to the maximum voltage for each electrode when the initial solution >is - set. Since the magnitude of Ve is constrained rather than Ve itself, I believe this is correct, but could this be why Ve does not change as it should when the >problem is - being solved? > > This f(Ve_ijk) seems very odd to me since normally you would always have f(some/all variables) = 0 as the function. If f=0.0+0.0*i for all steps, then you don't need a seperate variable Ve_ijk, you can treat it as a constant since Ve_ijk will not change its value during the solutions. > > Moroever, the Jacobian rows for df(Ve_ijk)/d(all variables) would be all zeros which implies a singular matrix. I am not sure how the solution converged for this case. Were you using -pc_type lu with some shift or some external direct solver such as superlu? What do you get for -snes_monitor -snes_converged_reason? > > > > Ve_ijk is set to the maximum voltage for each electrode when the initial solution is set. Since the magnitude of Ve is constrained rather than Ve itself. > > for Vmag_ijk, f = abs(Ve_ijk) - abs(Ve_ijk_max) > > Since Ve_ijk = Ve_ijk_max, f for Vmag_ijk = 0.0 and hence Vmag_ijk also will also retain its initial value throughout. > > >If a block (with coordinates i, j, k) is not occupied by an electrode, C_e is set to zero in the expression for Vb_ijk and all function and Jacobian entries for Ve, >Vmag, and Imag are either not set or set to zero. > Does this mean f(Ve_ijk) = f(Vmag_ijk) = f(Imag_ijk) = 0? If so, then Ve_ijk, Vmag_ijk, and Imag_ijk would retain their initial values for all time steps. > > > When I run my program with -snes_vi_type rs, the solution converges quickly but Vmag voltages remain the same as Ve_max for every electrode. As a result, Imag > currents are far larger than abs(iE_max). > Seems to me like the problem is not converging, or giving the incorrect solution, since the solution of a variational inequality problem has to satisfy Imag < iE_max, which in your case is not happening. > > Thanks for the detailed explanation. Would you mind if I (or you) forward this to petsc-users list so that others could share their thoughts too. > > Shri > > > > Hi Shri, > > Thanks again for your patience with my being away this past month. I have spent some more time with this problem and I have added Jacobian entries for 'cross terms' such as d(f(Vb)) / d(Ve). However, this did not appear to change the solution arrived at by PETSc. > > I will give you some examples of the f functions and corresponding Jacobian entries as you suggested, with the hope you will be able to see where I may be going wrong. I will use Vb = V_block, Ve = V_electrode, Vmag = magnitude(V_electrode), Imag = magnitude(I_electrode) to name the four degrees of freedom. > > In my 3D finite differencing grid, there are two types of blocks: blocks occupied by electrodes, and blocks occupied by a resistive medium. The block type of the current block and its neighbouring blocks determines the form of f. This is complicated by the fact that individual electrodes generally occupy three vertically-contiguous blocks, and I want to limit the electric current flowing from the whole electrode rather than the current through each block. Note that Ve, Vmag, and Imag are thus computed or recorded only for the top block of each electrode. > > If a block (with coordinates i, j, k in the x, y, and z directions, respectively) is occupied by an electrode, > - for Vb_ijk, f = C_posX * Vb_posX + C_posY * Vb_posY + C_posZ * Vb_posZ > - (C_posX + C_posY + C_posZ + C_negX + C_negY + C_negZ - C_e) * Vb_ijk > + C_negX * Vb_negX + C_negY * Vb_negY + C_negZ * Vb_negZ > - C_e * Ve, > where all C's are calculated constants that do not depend on the other degrees of freedom, but do depend on the type of the neighbouring block in their assigned directions. Then, > the Jacobian entries for Vb_ijk are > df/dVb_negZ = C_negZ (col.i = i, col.j = j, col.k = k - 1) > df/dVb_negY = C_negY (col.i = i, col.j = j - 1, col.k = k) > df/dVb_negX = C_negX (col.i = i - 1, col.j = j, col.k = k) > df/dVb_ijk = - (C_posX + C_posY + C_posZ + C_negX + C_negY + C_negZ - C_e) (col.i = i, col.j = j, col.k = k) > df/dVb_posX = C_posX (col.i = i + 1, col.j = j, col.k = k) > df/dVb_posY = C_posY (col.i = i, col.j = j + 1, col.k = k) > df/dVb_posZ = C_posZ (col.i = i, col.j = j, col.k = k + 1) > For each of these, row.i = i, row.j = j, row.k = k, and row.c = DOF_VB (the DOF index for Vb). Also, col.c = DOF_VB. > The only cross term for the Vb_ijk function is > df/dVe_ijk = -C_e (row.i = i, row.j = j, row.k = k, row.c = DOF_VB, col.i = i, col.j = j, col.k = k col.c = DOF_VE) > > - for Ve_ijk, f = 0.0 + 0.0*i, and all corresponding Jacobian entries are zero as well. Ve_ijk is set to the maximum voltage for each electrode when the initial solution is set. Since the magnitude of Ve is constrained rather than Ve itself, I believe this is correct, but could this be why Ve does not change as it should when the problem is being solved? > > - for Vmag_ijk, f = abs(Ve_ijk) - abs(Ve_ijk_max). The idea was to provide some 'pressure' towards Ve = Ve_max. The corresponding Jacobian entries are > df/dVmag_ijk = 1.0 + 0.0*i (row.i = i, row.j = j, row.k = k, row.c = DOF_VMAG, col.i = i, col.j = j, col.k = k, col.c = DOF_VMAG) > df/dVe_ijk = 0.5 * conj(Ve_ijk) / abs(Ve_ijk) (row.i = i, row.j = j, row.k = k, row.c = DOF_VMAG, col.i = i, col.j = j, col.k = k, col.c = DOF_VE) > (However, since d(abs(Ve_ijk))/dVe_ijk does not have satisfied Cauchy-Reimann equations for the entire complex plane, this derivative does not technically exist.) > > - for Imag_ijk, f = abs(iE) - iE_max. Again, the idea was to provide some 'pressure' towards iE = iE_max. iE is calculated by adding iE_block currents for each block occupied by an electrode. iE_block for block i,j,k is found from > iE_block_ijk = aE_ijk * (Ve - Vb_ijk) > where aE_ijk is a constant that does not depend on any degrees of freedom. There is one Ve value per electrode. The Jacobian entries are then > df/dImag_ijk = 1.0 + 0.0*i (row.i = i, row.j = j, row.k = k, row.c = DOF_IMAG, col.i = i, col.j = j, col.k = k, col.c = DOF_IMAG) > df/dVe = 0.5 * aE_sum * conj(iE) / abs(iE) (row.i = i, row.j = j, row.k = k, row.c = DOF_IMAG, col.i = i, col.j = j, col.k = k, col.c = DOF_VE) > df/dVb_ijk = - 0.5 * aE_ijk * conj(iE) / abs(iE) (row.i = i, row.j = j, row.k = k, row.c = DOF_IMAG, col.i = i, col.j = j, col.k = k, col.c = DOF_VB) > df/dVb_ij(k-1) = - 0.5 * aE_ij(k-1) * conj(iE) / abs(iE) (row.i = i, row.j = j, row.k = k, row.c = DOF_IMAG, col.i = i, col.j = j, col.k = k-1, col.c = DOF_VB) > df/dVb_ij(k-2) = - 0.5 * aE_ij(k-2) * conj(iE) / abs(iE) (row.i = i, row.j = j, row.k = k, row.c = DOF_IMAG, col.i = i, col.j = j, col.k = k-2, col.c = DOF_VB) > > If a block (with coordinates i, j, k) is not occupied by an electrode, C_e is set to zero in the expression for Vb_ijk and all function and Jacobian entries for Ve, Vmag, and Imag are either not set or set to zero. > > My SNES_VI constraints are set as follows for every block: > xl_ijk_DOF_V = -SNES_VI_INF > xu_ijk_DOF_V = SNES_VI_INF > xl_ijk_DOF_VE = -SNES_VI_INF > xu_ijk_DOF_VE = SNES_VI_INF > xl_ijk_DOF_VMAG = 0.0 > xu_ijk_DOF_VMAG = abs(Ve_max) > xl_ijk_DOF_IMAG = 0.0 > xu_ijk_DOF_IMAG = abs(iE_max) > > > When I run my program with -snes_vi_type rs, the solution converges quickly but Vmag voltages remain the same as Ve_max for every electrode. As a result, Imag currents are far larger than abs(iE_max). Are you able to see from here what the problem might be? Again, my hunches are that the complex derivatives do not work in my Jacobian because they do not exist, or that my choices of f for Ve, Vmag, and Imag are not correct. > > Please let me know if you would like clarification on any portion of my description., and thank you again for all your help with this. > > Jon > > On 2011-07-29, at 12:13 PM, Shri wrote: > > Jon, > I am trying to compile petsc with complex data type right now and will try to test the VI solver with complex variables within the next few hours. > > > My understanding is that comparison operators (such as < and >) are > not defined for complex numbers, so would it be correct to say that > anyone using complex scalars within PETSc will encounter this problem > with SNESVI? In other words, SNESVI inequality constraints can only be > applied to real variables or real-valued functions of complex > variables? > > For complex scalars, we only compare the real part of the complex variable and bound,i.e., > real(xl) <= real(x) <= real(x_u). For your application, this real part comparison is sufficient > since the magnitude is saved in the real part. > > > The residuals are set as follows, and the Jacobian entries are set > correspondingly: > f[V_block] = {function of adjacent V_blocks, V_electrode} > f[V_electrode] = 0.0 + 0.0i > f[magnitude(V_electrode)] = 0.0 + 0.0i > f[magnitude(I_electrode)] = 0.0 + 0.0i > > Can you give an example of f for all dofs maybe just for one grid point with the following dof names > Vb - V_block > Ve - V_electrode > Vmag - magnitude(V_electrode) > Imag - magnitude(I_electrode) > > Since I set all the residuals for both magnitude variables to zero, > and each entry in the Jacobian matrix is a partial derivative of a > residual, I believe the Jacobian entries for both magnitudes would all > be zero as well. However, your point made me realize I have no > Jacobian entry for the partial derivative of f[V_block] w.r.t. > V_electrode. Is that needed? > > Since f[V_block] is a function of V_electrode, you would need to set the partial derivatives of f[V_block] w.r.t V_electrode > for correct jacobian evaluation and for having a good newton convergence. > > How would one enter it with > MatSetValuesStencil? > ex28.c, > http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-dev/src/ksp/ksp/examples/tutorials/ex28.c.html, > is a petsc example which shows how to use MatSetValuesStencil with multiple degrees of freedom. > See the routine ComputeMatrix in the example. > > My working understanding was that each degree of > freedom conceptually has its own residual and Jacobian, even though > all residuals are contained in one Vec and all Jacobians contained in > one Mat. > > > Please bare with us as we are trying to figure out whether the issue is with our VI implementation or something in > your code is amiss. I appreciate your patience on this matter. > > Btw, one quick question, are all the problem formulations in your field done using complex variables or by decomposing the complex term into real/imaginary or magnitude/angle? My background is in electrical power grid and we mostly use either real/imaginary part or magnitude/angle as the variables. > > > Shri > > > ----- Original Message ----- > Thanks for your reply, Shri. Responses below: > > On 2011-07-28, at 1:03 PM, Shri wrote: > > > ----- Original Message ----- > Hi Barry, Shri, > > I reworked my implementation to include only the time-independent > electrical problem, and to make use of SNESVISetVariableBounds(). > However, it does not work quite as I expected: Constraints on > variables that are functions of other variables have no effect on > the > variables by which the functions are defined. > > My constraints are set on the four degrees of freedom as follows: > -SNES_VI_INF <= V_block <= SNES_VI_INF > -SNES_VI_INF <= V_electrode <= SNES_VI_INF > 0.0 + 0.0i <= magnitude(V_electrode) <= magnitude(V_electrode_max) > 0.0 + 0.0i <= magnitude(I_electrode) <= magnitude(I_electrode_max) > > Note that while V_block and V_electrode and complex quantities,the > magnitude terms are actually real numbers. What function did you use > to compute the magnitude? abs()? sqrt of product of complex number > with its conjugate? > > I used the "0.0 + 0.0i" there to emphasize that I built PETSc with > complex scalars, so all the scalars are stored as complex even if the > imaginary components are always zero. I have been using std::abs() > from the C++ standard library complex template class > (std::complex). I believe the values of type 'double' returned > by std::abs() are implicitly cast to the real part of > std::complex or PetscScalar, though I usually try to cast > explicitly. > > This seems like an application where the variables are of type > complex but the constraints need to be set for functions of the > variables which are actually real numbers. I am not sure whether > this can be supported. Maybe Barry or others can shed more > light on this. > > My understanding is that comparison operators (such as < and >) are > not defined for complex numbers, so would it be correct to say that > anyone using complex scalars within PETSc will encounter this problem > with SNESVI? In other words, SNESVI inequality constraints can only be > applied to real variables or real-valued functions of complex > variables? > > > The residuals are set as follows, and the Jacobian entries are set > correspondingly: > f[V_block] = {function of adjacent V_blocks, V_electrode} > f[V_electrode] = 0.0 + 0.0i > f[magnitude(V_electrode)] = 0.0 + 0.0i > f[magnitude(I_electrode)] = 0.0 + 0.0i > > What equations do you use for the partial derivatives of the > magnitude functions w.r.t. the actual complex variables (V_block and > V_electrode) in the jacobian evaluation? > > Since I set all the residuals for both magnitude variables to zero, > and each entry in the Jacobian matrix is a partial derivative of a > residual, I believe the Jacobian entries for both magnitudes would all > be zero as well. However, your point made me realize I have no > Jacobian entry for the partial derivative of f[V_block] w.r.t. > V_electrode. Is that needed? How would one enter it with > MatSetValuesStencil? My working understanding was that each degree of > freedom conceptually has its own residual and Jacobian, even though > all residuals are contained in one Vec and all Jacobians contained in > one Mat. > > > The first two potentials, V_block and V_electrode, are independent, > while magnitude(V_electrode) is a function of V_electrode only and > the > electrode current magnitude(I_electrode) is a function of V_block > and > V_electrode. Note that V_electrode acts as a source term in the > finite > difference formulation for V_block. While all of these variables > are > complex, the latter two always have zero imaginary parts. I suppose > I > am assuming that PETSc knows to compare only the real parts if the > imaginary parts are zero. (Is this a bad assumption?) > > When solving with PETSc (using -snes_vi_type rs) > > Did you also use -snes_type vi? > > I have been using SNESSetType(snes, SNESVI) in my code, and the use of > -snes_type vi in the command line does not seem to change any > behaviour. > > > V_electrode stays at > its maximum, as set in the initial guess, and the V_block > distribution > falls properly from that. However, measurements of > magnitude(I_electrode) are way above maximum and do not move > downwards > with successive SNES iterations. I can also set the constraint on > magnitude(V_electrode) to less than maximum and it does not affect > the > value of V_electrode. How can I tell PETSc to change V_electrode > when > the magnitude(V_electrode) or magnitude(I_electrode) constraints > are > not met? > > Send the code for your application with instructions on how to run > it and we'll try to figure out what needs to be > done. > > I really appreciate this generous offer of your time, and I will > certainly take you up on it if we cannot resolve this otherwise. > Unfortunately, I will be travelling for the next few weeks and it may > take me some time to isolate the PETSc-dependent parts of my > application. Please bear with me as I will try to keep in touch while > I am away. > > As I mentioned above, I think the issue may lie in the conceptual > incompatibility of complex variables and inequality constraints, but I > would appreciate your thoughts on that. > > Thanks again, > > Jon > > > > Shri > > > Thanks again for your help. > > Jon > > > On 2011-07-25, at 8:58 PM, Barry Smith wrote: > > > On Jul 25, 2011, at 5:50 PM, Jonathan Backs wrote: > > Hi Shri, > > Thanks for your message and all the helpful tips. If the > TSVISetVariableBounds() functions are available now, I would like > to try them as well. Is the interface essentially the same as > with > SNESVISetVariableBounds()? I will get back to you and Barry when > I > have had a chance to modify my application. > > For my problem, I believe it makes sense to have bounds on one of > the variables as well as one function of the variables. The two > relevant degrees of freedom are the block voltage (one for each > finite difference block) and the electrode voltage (one for each > electrode, which may be present in multiple blocks). The > electrode > voltage should keep a constant phase while the magnitude is > constrained between zero and some maximum. The block voltages > near > the electrodes depend on the electrode voltages as well as the > neighbouring block voltages. The electrode current for a given > electrode is a function of its electrode voltage and several > nearby > block voltages, and should be constrained in magnitude between > zero > and some maximum (and the phase unconstrained). Would I need to > use > SNESVISetComputeVariableBounds since the electrode current is a > function of the other variables? Would any other provisions need > to > be made for the block voltages, since they depend on the > electrode > voltages? > > > You cannot directly make a bound on some function of other > variables. Instead you need to introduce another variable that is > defined to be equal to that function and put the bound on that > new > variable. > > Barry > > Thank you again, > > Jon > > On 2011-07-25, at 11:58 AM, Shri wrote: > > > ----- Original Message ----- > On Jul 22, 2011, at 4:16 PM, Jonathan Backs wrote: > > Barry, > > Thank you so much for your response. Lucky, indeed! I look > forward > to trying out these new features. > > I neglected to mention in my original post that my electrical > problem is part of a DAE, which includes a time-dependent > heating > problem. Can SNESVI constraints be used in conjunction with > TSSetIFunction() and TSSetIJacobian() as well? (Of course, I > only > need the constraints for the time-independent electrical > portion.) > > We have not yet put that in but Shri is starting to look at > that > just > now. Basically we would have a TSVISetVariableBounds() and > handle > everything from there. I suggest you start with a simple time > electrical portion with constraints to explore and we'll go > from > there. Shri is actually a electrical networks guy and so can > speak > your language. > > > I've added TSVISetVariableBounds() for setting the bounds on the > variables using the TS object directly. > A few things that i want to mention here about using the > variational inequality nonlinear solver (SNESVI). > i) Use the runtime option -snes_type vi or explicitly set > SNESSetType(snes,SNESVI). > ii) There are two tested algorithms currently available, (a) > semismooth (-snes_vi_type ss) and (b) active set or reduced > space > (-snes_vi_type rs). > iii) Take a look at example,ex61.c, in > src/snes/examples/tutorials > which is a time-stepping nonlinear problem with constraints on > the > variables. This example does not use the TS object directly but > rather a time-loop is explicitly written. Another example,ex8.c, > in src/snes/examples/tests/ is a minimum surface area problem > which uses SNESVI. > > By the way, does your problem have bounds on the variables or > bounds on some function of the variables? > > Shri > > > > > Barry > > > > > Thank you again, > > Jon > > On 2011-07-22, at 2:46 PM, Barry Smith wrote: > > > Jon, > > You may be in luck. In PETSc-dev > http://www.mcs.anl.gov/petsc/petsc-as/developers/index.html > we > have now implemented variational inequality non-linear > solvers > with box constraints. > > That is one has a set of variables u and algebraic equations > F(u) = 0 (say of size n each) but in addition one has > constraints lu <= u <= uu where some or all of lu may be > negative infinity (no constraints) and some or all of uu may > be > infinity (no constraints). There is also a constraint on the > sign of F() for those equations associated with active > constraints. If your constraints are not in this form > sometimes > you can introduce new additional variables to get it in this > form. Read up a little on variational inequalities on the > web. > > To use this you provide the usual SNES function and Jacobian > but > you also provide SNESVISetVariableBounds() there is a manual > page for this function and for for SNESVI at > http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-dev/docs/index.html > (the link is broken right now but Satish is fixing it). Plus > several examples in src/snes/examples/tutorials (in > petsc-dev). > > This is all new code so we would be interested in your > feedback. > > > Barry > > > > > > On Jul 22, 2011, at 3:33 PM, Jonathan Backs wrote: > > Hi, > > I am trying to add a constraint feature to my first PETSc > application, which uses the finite difference method to > calculate > the potential distribution produced by a collection of > electrodes > in a resistive medium. I would like to make this simulation > more > realistic by imposing a maximum electric current and a > maximum > potential difference that can be supplied to each electrode > by > the > power supply. If the medium between the electrodes is very > conductive, the current maximum would be exceeded by the > maximum > potential difference, so the potential difference should be > decreased from maximum until it produces the maximum > current. > On > the other hand, the potential difference between the > electrodes > should remain at maximum as long as the current remains > below > maximum (say, for a less conductive medium). > > I added an extra degree of freedom (the electrode voltages) > to > my > DMDA, and I developed a set of conditional expressions that > describe the above constraints, but one problem is that the > logic > relies on if-then-else decisions that are made when forming > the > function/residual and the Jacobian. Once these decisions are > made, > of course, the conditions are not checked again until the > next > function or Jacobian evaluation. The non-linear solver then > tends > to oscillate between extreme solutions to the opposing > conditions > with each iteration, and never converges towards a > reasonable > solution. > > Is there a better strategy for solving such problems? Does > PETSc > offer mechanisms to aid in their solution? I would very much > appreciate any hints. > > Thank you for your time, > > Jon > > > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From abhyshr at mcs.anl.gov Thu Sep 8 21:34:55 2011 From: abhyshr at mcs.anl.gov (Shri) Date: Thu, 8 Sep 2011 21:34:55 -0500 (CDT) Subject: [petsc-users] Conditional Constraints In-Reply-To: Message-ID: <794251637.288354.1315535695188.JavaMail.root@zimbra.anl.gov> Jon, > I do not want Ve to be constant, however; I would like it to either decrease in response to Imag > Imag_max or increase in response to Imag < Imag_max, providing > the condition Vmag <= Vmag_max remains true. > Should I then have something like f(Ve_ijk) = Imag - Imag_max? Note here that (a) f(Ve_ijk) is a complex function while Imag and Imag_max are real. So the above equation does not hold. (b) In your previous email, you had bounds on Imag, 0 < Imag < Imag_max. Now you want Ve_ijk to increase if Imag > Imag_max and decrease if Imag < Imag_max. Are you trying to find the value of Ve at which the current is maximum, i.e. Imag = Imag_max, by doing this? ----- Original Message ----- Shri, Thanks very much for your response. In fact, -snes_converged_reason revealed that the solution was not converging, as you suggested: "Nonlinear solve did not converge due to DIVERGED_LINE_SEARCH." I had not realized this earlier since the voltage distribution in the solution appeared to be correct, even though the SNES_VI constraints were not satisfied. Given that and your explanation, I am sure something is wrong with my choices of f. I do not want Ve to be constant, however; I would like it to either decrease in response to Imag > Imag_max or increase in response to Imag < Imag_max, providing the condition Vmag <= Vmag_max remains true. Should I then have something like f(Ve_ijk) = Imag - Imag_max? Note that you have One more thing from your response: >If a block (with coordinates i, j, k) is not occupied by an electrode , C_e is set to zero in the expression for Vb_ijk and all function and Jacobian entries for Ve, >Vmag, and Imag are either not set or set to zero. Does this mean f(Ve_ijk) = f(Vmag_ijk) = f(Imag_ijk) = 0? If so, then Ve_ijk, Vmag_ijk, and Imag_ijk would retain their initial values for all time steps. In my present scheme, f(Ve_ijk) = f(Vmag_ijk) = 0 and f(Imag_ijk) is non-zero for electrode blocks; all three are zero for non-electrode blocks. Depending on the choice of initial conditions, I can make f(Vmag_ijk) non-zero as well (until Ve changes to match Ve_max). Thanks again, Jon On 2011-09-08, at 3:42 PM, Shri wrote: Jon, > for Ve_ijk, f = 0.0 + 0.0*i, and all corresponding Jacobian entries are zero as well. Ve_ijk is set to the maximum voltage for each electrode when the initial solution >is - set. Since the magnitude of Ve is constrained rather than Ve itself, I believe this is correct, but could this be why Ve does not change as it should when the >problem is - being solved? This f(Ve_ijk) seems very odd to me since normally you would always have f(some/all variables) = 0 as the function. If f=0.0+0.0*i for all steps, then you don't need a seperate variable Ve_ijk, you can treat it as a constant since Ve_ijk will not change its value during the solutions. Moroever, the Jacobian rows for df(Ve_ijk)/d(all variables) would be all zeros which implies a singular matrix. I am not sure how the solution converged for this case. Were you using -pc_type lu with some shift or some external direct solver such as superlu? What do you get for -snes_monitor -snes_converged_reason? > Ve_ijk is set to the maximum voltage for each electrode when the initial solution is set. Since the magnitude of Ve is constrained rather than Ve itself. > for Vmag_ijk, f = abs(Ve_ijk) - abs(Ve_ijk_max) Since Ve_ijk = Ve_ijk_max, f for Vmag_ijk = 0.0 and hence Vmag_ijk also will also retain its initial value throughout. >If a block (with coordinates i, j, k) is not occupied by an electrode , C_e is set to zero in the expression for Vb_ijk and all function and Jacobian entries for Ve, >Vmag, and Imag are either not set or set to zero. Does this mean f(Ve_ijk) = f(Vmag_ijk) = f(Imag_ijk) = 0? If so, then Ve_ijk, Vmag_ijk, and Imag_ijk would retain their initial values for all time steps. > When I run my program with -snes_vi_type rs, the solution converges quickly but Vmag voltages remain the same as Ve_max for every electrode. As a result, Imag > currents are far larger than abs(iE_max). Seems to me like the problem is not converging, or giving the incorrect solution, since the solution of a variational inequality problem has to satisfy Imag < iE_max, which in your case is not happening. Thanks for the detailed explanation. Would you mind if I (or you) forward this to petsc-users list so that others could share their thoughts too. Shri ----- Original Message ----- Hi Shri, Thanks again for your patience with my being away this past month. I have spent some more time with this problem and I have added Jacobian entries for 'cross terms' such as d(f(Vb)) / d(Ve). However, this did not appear to change the solution arrived at by PETSc. I will give you some examples of the f functions and corresponding Jacobian entries as you suggested, with the hope you will be able to see where I may be going wrong. I will use Vb = V_block, Ve = V_electrode, Vmag = magnitude(V_electrode), Imag = magnitude(I_electrode) to name the four degrees of freedom. In my 3D finite differencing grid, there are two types of blocks: blocks occupied by electrodes, and blocks occupied by a resistive medium. The block type of the current block and its neighbouring blocks determines the form of f. This is complicated by the fact that individual electrodes generally occupy three vertically-contiguous blocks, and I want to limit the electric current flowing from the whole electrode rather than the current through each block. Note that Ve, Vmag, and Imag are thus computed or recorded only for the top block of each electrode. If a block (with coordinates i, j, k in the x, y, and z directions, respectively) is occupied by an electrode , - for Vb_ijk, f = C_posX * Vb_posX + C_posY * Vb_posY + C_posZ * Vb_posZ - (C_posX + C_posY + C_posZ + C_negX + C_negY + C_negZ - C_e) * Vb_ijk + C_negX * Vb_negX + C_negY * Vb_negY + C_negZ * Vb_negZ - C_e * Ve, where all C's are calculated constants that do not depend on the other degrees of freedom, but do depend on the type of the neighbouring block in their assigned directions. Then, the Jacobian entries for Vb_ijk are df/dVb_negZ = C_negZ (col.i = i, col.j = j, col.k = k - 1) df/dVb_negY = C_negY (col.i = i, col.j = j - 1, col.k = k) df/dVb_negX = C_negX (col.i = i - 1, col.j = j, col.k = k) df/dVb_ijk = - (C_posX + C_posY + C_posZ + C_negX + C_negY + C_negZ - C_e) (col.i = i, col.j = j, col.k = k) df/dVb_posX = C_posX (col.i = i + 1, col.j = j, col.k = k) df/dVb_posY = C_posY (col.i = i, col.j = j + 1, col.k = k) df/dVb_posZ = C_posZ (col.i = i, col.j = j, col.k = k + 1) For each of these, row.i = i, row.j = j, row.k = k, and row.c = DOF_VB (the DOF index for Vb). Also, col.c = DOF_VB. The only cross term for the Vb_ijk function is df/dVe_ijk = -C_e (row.i = i, row.j = j, row.k = k, row.c = DOF_VB, col.i = i, col.j = j, col.k = k col.c = DOF_VE) - for Ve_ijk, f = 0.0 + 0.0*i, and all corresponding Jacobian entries are zero as well. Ve_ijk is set to the maximum voltage for each electrode when the initial solution is set. Since the magnitude of Ve is constrained rather than Ve itself, I believe this is correct, but could this be why Ve does not change as it should when the problem is being solved? - for Vmag_ijk, f = abs(Ve_ijk) - abs(Ve_ijk_max). The idea was to provide some 'pressure' towards Ve = Ve_max. The corresponding Jacobian entries are df/dVmag_ijk = 1.0 + 0.0*i (row.i = i, row.j = j, row.k = k, row.c = DOF_VMAG, col.i = i, col.j = j, col.k = k, col.c = DOF_VMAG) df/dVe_ijk = 0.5 * conj(Ve_ijk) / abs(Ve_ijk) (row.i = i, row.j = j, row.k = k, row.c = DOF_VMAG, col.i = i, col.j = j, col.k = k, col.c = DOF_VE) (However, since d(abs(Ve_ijk))/dVe_ijk does not have satisfied Cauchy-Reimann equations for the entire complex plane, this derivative does not technically exist.) - for Imag_ijk, f = abs(iE) - iE_max. Again, the idea was to provide some 'pressure' towards iE = iE_max. iE is calculated by adding iE_block currents for each block occupied by an electrode. iE_block for block i,j,k is found from iE_block_ijk = aE_ijk * (Ve - Vb_ijk) where aE_ijk is a constant that does not depend on any degrees of freedom. There is one Ve value per electrode. The Jacobian entries are then df/dImag_ijk = 1.0 + 0.0*i (row.i = i, row.j = j, row.k = k, row.c = DOF_IMAG, col.i = i, col.j = j, col.k = k, col.c = DOF_IMAG) df/dVe = 0.5 * aE_sum * conj(iE) / abs(iE) (row.i = i, row.j = j, row.k = k, row.c = DOF_IMAG, col.i = i, col.j = j, col.k = k, col.c = DOF_VE) df/dVb_ijk = - 0.5 * aE_ijk * conj(iE) / abs(iE) (row.i = i, row.j = j, row.k = k, row.c = DOF_IMAG, col.i = i, col.j = j, col.k = k, col.c = DOF_VB) df/dVb_ij(k-1) = - 0.5 * aE_ij(k-1) * conj(iE) / abs(iE) (row.i = i, row.j = j, row.k = k, row.c = DOF_IMAG, col.i = i, col.j = j, col.k = k-1, col.c = DOF_VB) df/dVb_ij(k-2) = - 0.5 * aE_ij(k-2) * conj(iE) / abs(iE) (row.i = i, row.j = j, row.k = k, row.c = DOF_IMAG, col.i = i, col.j = j, col.k = k-2, col.c = DOF_VB) If a block (with coordinates i, j, k) is not occupied by an electrode , C_e is set to zero in the expression for Vb_ijk and all function and Jacobian entries for Ve, Vmag, and Imag are either not set or set to zero. My SNES_VI constraints are set as follows for every block: xl_ijk_DOF_V = -SNES_VI_INF xu_ijk_DOF_V = SNES_VI_INF xl_ijk_DOF_VE = -SNES_VI_INF xu_ijk_DOF_VE = SNES_VI_INF xl_ijk_DOF_VMAG = 0.0 xu_ijk_DOF_VMAG = abs(Ve_max) xl_ijk_DOF_IMAG = 0.0 xu_ijk_DOF_IMAG = abs(iE_max) When I run my program with -snes_vi_type rs, the solution converges quickly but Vmag voltages remain the same as Ve_max for every electrode. As a result, Imag currents are far larger than abs(iE_max). Are you able to see from here what the problem might be? Again, my hunches are that the complex derivatives do not work in my Jacobian because they do not exist, or that my choices of f for Ve, Vmag, and Imag are not correct. Please let me know if you would like clarification on any portion of my description., and thank you again for all your help with this. Jon On 2011-07-29, at 12:13 PM, Shri wrote: Jon, I am trying to compile petsc with complex data type right now and will try to test the VI solver with complex variables within the next few hours. My understanding is that comparison operators (such as < and >) are not defined for complex numbers, so would it be correct to say that anyone using complex scalars within PETSc will encounter this problem with SNESVI? In other words, SNESVI inequality constraints can only be applied to real variables or real-valued functions of complex variables? For complex scalars, we only compare the real part of the complex variable and bound,i.e., real(xl) <= real(x) <= real(x_u). For your application, this real part comparison is sufficient since the magnitude is saved in the real part. The residuals are set as follows, and the Jacobian entries are set correspondingly: f[V_block] = {function of adjacent V_blocks, V_electrode} f[V_electrode] = 0.0 + 0.0i f[magnitude(V_electrode)] = 0.0 + 0.0i f[magnitude(I_electrode)] = 0.0 + 0.0i Can you give an example of f for all dofs maybe just for one grid point with the following dof names Vb - V_block Ve - V_electrode Vmag - magnitude(V_electrode) Imag - magnitude(I_electrode) Since I set all the residuals for both magnitude variables to zero, and each entry in the Jacobian matrix is a partial derivative of a residual, I believe the Jacobian entries for both magnitudes would all be zero as well. However, your point made me realize I have no Jacobian entry for the partial derivative of f[V_block] w.r.t. V_electrode. Is that needed? Since f[V_block] is a function of V_electrode, you would need to set the partial derivatives of f[V_block] w.r.t V_electrode for correct jacobian evaluation and for having a good newton convergence. How would one enter it with MatSetValuesStencil? ex28.c, http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-dev/src/ksp/ksp/examples/tutorials/ex28.c.html , is a petsc example which shows how to use MatSetValuesStencil with multiple degrees of freedom. See the routine ComputeMatrix in the example. My working understanding was that each degree of freedom conceptually has its own residual and Jacobian, even though all residuals are contained in one Vec and all Jacobians contained in one Mat. Please bare with us as we are trying to figure out whether the issue is with our VI implementation or something in your code is amiss. I appreciate your patience on this matter. Btw, one quick question, are all the problem formulations in your field done using complex variables or by decomposing the complex term into real/imaginary or magnitude/angle? My background is in electrical power grid and we mostly use either real/imaginary part or magnitude/angle as the variables. Shri ----- Original Message ----- Thanks for your reply, Shri. Responses below: On 2011-07-28, at 1:03 PM, Shri wrote: ----- Original Message ----- Hi Barry, Shri, I reworked my implementation to include only the time-independent electrical problem, and to make use of SNESVISetVariableBounds(). However, it does not work quite as I expected: Constraints on variables that are functions of other variables have no effect on the variables by which the functions are defined. My constraints are set on the four degrees of freedom as follows: -SNES_VI_INF <= V_block <= SNES_VI_INF -SNES_VI_INF <= V_electrode <= SNES_VI_INF 0.0 + 0.0i <= magnitude(V_electrode) <= magnitude(V_electrode_max) 0.0 + 0.0i <= magnitude(I_electrode) <= magnitude(I_electrode_max) Note that while V_block and V_electrode and complex quantities,the magnitude terms are actually real numbers. What function did you use to compute the magnitude? abs()? sqrt of product of complex number with its conjugate? I used the "0.0 + 0.0i" there to emphasize that I built PETSc with complex scalars, so all the scalars are stored as complex even if the imaginary components are always zero. I have been using std::abs() from the C++ standard library complex template class (std::complex). I believe the values of type 'double' returned by std::abs() are implicitly cast to the real part of std::complex or PetscScalar, though I usually try to cast explicitly. This seems like an application where the variables are of type complex but the constraints need to be set for functions of the variables which are actually real numbers. I am not sure whether this can be supported. Maybe Barry or others can shed more light on this. My understanding is that comparison operators (such as < and >) are not defined for complex numbers, so would it be correct to say that anyone using complex scalars within PETSc will encounter this problem with SNESVI? In other words, SNESVI inequality constraints can only be applied to real variables or real-valued functions of complex variables? The residuals are set as follows, and the Jacobian entries are set correspondingly: f[V_block] = {function of adjacent V_blocks, V_electrode} f[V_electrode] = 0.0 + 0.0i f[magnitude(V_electrode)] = 0.0 + 0.0i f[magnitude(I_electrode)] = 0.0 + 0.0i What equations do you use for the partial derivatives of the magnitude functions w.r.t. the actual complex variables (V_block and V_electrode) in the jacobian evaluation? Since I set all the residuals for both magnitude variables to zero, and each entry in the Jacobian matrix is a partial derivative of a residual, I believe the Jacobian entries for both magnitudes would all be zero as well. However, your point made me realize I have no Jacobian entry for the partial derivative of f[V_block] w.r.t. V_electrode. Is that needed? How would one enter it with MatSetValuesStencil? My working understanding was that each degree of freedom conceptually has its own residual and Jacobian, even though all residuals are contained in one Vec and all Jacobians contained in one Mat. The first two potentials, V_block and V_electrode, are independent, while magnitude(V_electrode) is a function of V_electrode only and the electrode current magnitude(I_electrode) is a function of V_block and V_electrode. Note that V_electrode acts as a source term in the finite difference formulation for V_block. While all of these variables are complex, the latter two always have zero imaginary parts. I suppose I am assuming that PETSc knows to compare only the real parts if the imaginary parts are zero. (Is this a bad assumption?) When solving with PETSc (using -snes_vi_type rs) Did you also use -snes_type vi? I have been using SNESSetType(snes, SNESVI) in my code, and the use of -snes_type vi in the command line does not seem to change any behaviour. V_electrode stays at its maximum, as set in the initial guess, and the V_block distribution falls properly from that. However, measurements of magnitude(I_electrode) are way above maximum and do not move downwards with successive SNES iterations. I can also set the constraint on magnitude(V_electrode) to less than maximum and it does not affect the value of V_electrode. How can I tell PETSc to change V_electrode when the magnitude(V_electrode) or magnitude(I_electrode) constraints are not met? Send the code for your application with instructions on how to run it and we'll try to figure out what needs to be done. I really appreciate this generous offer of your time, and I will certainly take you up on it if we cannot resolve this otherwise. Unfortunately, I will be travelling for the next few weeks and it may take me some time to isolate the PETSc-dependent parts of my application. Please bear with me as I will try to keep in touch while I am away. As I mentioned above, I think the issue may lie in the conceptual incompatibility of complex variables and inequality constraints, but I would appreciate your thoughts on that. Thanks again, Jon Shri Thanks again for your help. Jon On 2011-07-25, at 8:58 PM, Barry Smith wrote: On Jul 25, 2011, at 5:50 PM, Jonathan Backs wrote: Hi Shri, Thanks for your message and all the helpful tips. If the TSVISetVariableBounds() functions are available now, I would like to try them as well. Is the interface essentially the same as with SNESVISetVariableBounds()? I will get back to you and Barry when I have had a chance to modify my application. For my problem, I believe it makes sense to have bounds on one of the variables as well as one function of the variables. The two relevant degrees of freedom are the block voltage (one for each finite difference block) and the electrode voltage (one for each electrode, which may be present in multiple blocks). The electrode voltage should keep a constant phase while the magnitude is constrained between zero and some maximum. The block voltages near the electrodes depend on the electrode voltages as well as the neighbouring block voltages. The electrode current for a given electrode is a function of its electrode voltage and several nearby block voltages, and should be constrained in magnitude between zero and some maximum (and the phase unconstrained). Would I need to use SNESVISetComputeVariableBounds since the electrode current is a function of the other variables? Would any other provisions need to be made for the block voltages, since they depend on the electrode voltages? You cannot directly make a bound on some function of other variables. Instead you need to introduce another variable that is defined to be equal to that function and put the bound on that new variable. Barry Thank you again, Jon On 2011-07-25, at 11:58 AM, Shri wrote: ----- Original Message ----- On Jul 22, 2011, at 4:16 PM, Jonathan Backs wrote: Barry, Thank you so much for your response. Lucky, indeed! I look forward to trying out these new features. I neglected to mention in my original post that my electrical problem is part of a DAE, which includes a time-dependent heating problem. Can SNESVI constraints be used in conjunction with TSSetIFunction() and TSSetIJacobian() as well? (Of course, I only need the constraints for the time-independent electrical portion.) We have not yet put that in but Shri is starting to look at that just now. Basically we would have a TSVISetVariableBounds() and handle everything from there. I suggest you start with a simple time electrical portion with constraints to explore and we'll go from there. Shri is actually a electrical networks guy and so can speak your language. I've added TSVISetVariableBounds() for setting the bounds on the variables using the TS object directly. A few things that i want to mention here about using the variational inequality nonlinear solver (SNESVI). i) Use the runtime option -snes_type vi or explicitly set SNESSetType(snes,SNESVI). ii) There are two tested algorithms currently available, (a) semismooth (-snes_vi_type ss) and (b) active set or reduced space (-snes_vi_type rs). iii) Take a look at example,ex61.c, in src/snes/examples/tutorials which is a time-stepping nonlinear problem with constraints on the variables. This example does not use the TS object directly but rather a time-loop is explicitly written. Another example,ex8.c, in src/snes/examples/tests/ is a minimum surface area problem which uses SNESVI. By the way, does your problem have bounds on the variables or bounds on some function of the variables? Shri Barry Thank you again, Jon On 2011-07-22, at 2:46 PM, Barry Smith wrote: Jon, You may be in luck. In PETSc-dev http://www.mcs.anl.gov/petsc/petsc-as/developers/index.html we have now implemented variational inequality non-linear solvers with box constraints. That is one has a set of variables u and algebraic equations F(u) = 0 (say of size n each) but in addition one has constraints lu <= u <= uu where some or all of lu may be negative infinity (no constraints) and some or all of uu may be infinity (no constraints). There is also a constraint on the sign of F() for those equations associated with active constraints. If your constraints are not in this form sometimes you can introduce new additional variables to get it in this form. Read up a little on variational inequalities on the web. To use this you provide the usual SNES function and Jacobian but you also provide SNESVISetVariableBounds() there is a manual page for this function and for for SNESVI at http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-dev/docs/index.html (the link is broken right now but Satish is fixing it). Plus several examples in src/snes/examples/tutorials (in petsc-dev). This is all new code so we would be interested in your feedback. Barry On Jul 22, 2011, at 3:33 PM, Jonathan Backs wrote: Hi, I am trying to add a constraint feature to my first PETSc application, which uses the finite difference method to calculate the potential distribution produced by a collection of electrodes in a resistive medium. I would like to make this simulation more realistic by imposing a maximum electric current and a maximum potential difference that can be supplied to each electrode by the power supply. If the medium between the electrodes is very conductive, the current maximum would be exceeded by the maximum potential difference, so the potential difference should be decreased from maximum until it produces the maximum current. On the other hand, the potential difference between the electrodes should remain at maximum as long as the current remains below maximum (say, for a less conductive medium). I added an extra degree of freedom (the electrode voltages) to my DMDA, and I developed a set of conditional expressions that describe the above constraints, but one problem is that the logic relies on if-then-else decisions that are made when forming the function/residual and the Jacobian. Once these decisions are made, of course, the conditions are not checked again until the next function or Jacobian evaluation. The non-linear solver then tends to oscillate between extreme solutions to the opposing conditions with each iteration, and never converges towards a reasonable solution. Is there a better strategy for solving such problems? Does PETSc offer mechanisms to aid in their solution? I would very much appreciate any hints. Thank you for your time, Jon -------------- next part -------------- An HTML attachment was scrubbed... URL: From jbacks at mcmillan-mcgee.com Fri Sep 9 09:32:54 2011 From: jbacks at mcmillan-mcgee.com (Jonathan Backs) Date: Fri, 9 Sep 2011 08:32:54 -0600 Subject: [petsc-users] Conditional Constraints In-Reply-To: <794251637.288354.1315535695188.JavaMail.root@zimbra.anl.gov> References: <794251637.288354.1315535695188.JavaMail.root@zimbra.anl.gov> Message-ID: <1DDCA07C-1833-4D45-B3F7-2D9B2A45AE1C@mcmillan-mcgee.com> Shri, > Note here that (a) f(Ve_ijk) is a complex function while Imag and Imag_max are real. So the above equation does not hold. Would f(Ve_ijk) = Imag - Imag_max = 0 simply be equivalent to saying, solve the real system of equations Imag - Imag_max = 0 and 0 = 0 (the imaginary part)? Regardless, I am not sure what a correct choice of f(Ve_ijk) would be. > (b) In your previous email, you had bounds on Imag, 0 < Imag < Imag_max. Now you want Ve_ijk to increase if Imag > Imag_max and decrease if Imag < Imag_max. Are you trying to find the value of Ve at which the current is maximum, i.e. Imag = Imag_max, by doing this? Yes, that is essentially what I am trying to find. Ve determines Vb, and the difference between the two determines Imag. Vmag (magnitude of Ve) and Imag have the restrictions 0 < Vmag < Vmag_max and 0 < Imag < Imag_max. I want to find either Ve for Imag = Imag_max or Imag for Vmag = Vmag_max. In my test scenario, Vmag_max happens to be quite high so Imag_max will always be reached first, but in general either situation is possible. Thank you again, Jon On 2011-09-08, at 8:34 PM, Shri wrote: > Jon, > > > I do not want Ve to be constant, however; I would like it to either decrease in response to Imag > Imag_max or increase in response to Imag < Imag_max, providing > the condition Vmag <= Vmag_max remains true. > > Should I then have something like f(Ve_ijk) = Imag - Imag_max? > > Note here that (a) f(Ve_ijk) is a complex function while Imag and Imag_max are real. So the above equation does not hold. > (b) In your previous email, you had bounds on Imag, 0 < Imag < Imag_max. Now you want Ve_ijk to increase if Imag > Imag_max and decrease if Imag < Imag_max. Are you trying to find the value of Ve at which the current is maximum, i.e. Imag = Imag_max, by doing this? > > Shri, > > Thanks very much for your response. In fact, -snes_converged_reason revealed that the solution was not converging, as you suggested: "Nonlinear solve did not converge due to DIVERGED_LINE_SEARCH." I had not realized this earlier since the voltage distribution in the solution appeared to be correct, even though the SNES_VI constraints were not satisfied. > > Given that and your explanation, I am sure something is wrong with my choices of f. I do not want Ve to be constant, however; I would like it to either decrease in response to Imag > Imag_max or increase in response to Imag < Imag_max, providing the condition Vmag <= Vmag_max remains true. Should I then have something like f(Ve_ijk) = Imag - Imag_max? > > Note that you have > > > One more thing from your response: > >If a block (with coordinates i, j, k) is not occupied by an electrode, C_e is set to zero in the expression for Vb_ijk and all function and Jacobian entries for Ve, >Vmag, and Imag are either not set or set to zero. > > Does this mean f(Ve_ijk) = f(Vmag_ijk) = f(Imag_ijk) = 0? If so, then Ve_ijk, Vmag_ijk, and Imag_ijk would retain their initial values for all time steps. > > In my present scheme, f(Ve_ijk) = f(Vmag_ijk) = 0 and f(Imag_ijk) is non-zero for electrode blocks; all three are zero for non-electrode blocks. Depending on the choice of initial conditions, I can make f(Vmag_ijk) non-zero as well (until Ve changes to match Ve_max). > > Thanks again, > > Jon > > On 2011-09-08, at 3:42 PM, Shri wrote: > > Jon, > > > for Ve_ijk, f = 0.0 + 0.0*i, and all corresponding Jacobian entries are zero as well. Ve_ijk is set to the maximum voltage for each electrode when the initial solution >is - set. Since the magnitude of Ve is constrained rather than Ve itself, I believe this is correct, but could this be why Ve does not change as it should when the >problem is - being solved? > > This f(Ve_ijk) seems very odd to me since normally you would always have f(some/all variables) = 0 as the function. If f=0.0+0.0*i for all steps, then you don't need a seperate variable Ve_ijk, you can treat it as a constant since Ve_ijk will not change its value during the solutions. > > Moroever, the Jacobian rows for df(Ve_ijk)/d(all variables) would be all zeros which implies a singular matrix. I am not sure how the solution converged for this case. Were you using -pc_type lu with some shift or some external direct solver such as superlu? What do you get for -snes_monitor -snes_converged_reason? > > > > Ve_ijk is set to the maximum voltage for each electrode when the initial solution is set. Since the magnitude of Ve is constrained rather than Ve itself. > > for Vmag_ijk, f = abs(Ve_ijk) - abs(Ve_ijk_max) > > Since Ve_ijk = Ve_ijk_max, f for Vmag_ijk = 0.0 and hence Vmag_ijk also will also retain its initial value throughout. > > >If a block (with coordinates i, j, k) is not occupied by an electrode, C_e is set to zero in the expression for Vb_ijk and all function and Jacobian entries for Ve, >Vmag, and Imag are either not set or set to zero. > Does this mean f(Ve_ijk) = f(Vmag_ijk) = f(Imag_ijk) = 0? If so, then Ve_ijk, Vmag_ijk, and Imag_ijk would retain their initial values for all time steps. > > > When I run my program with -snes_vi_type rs, the solution converges quickly but Vmag voltages remain the same as Ve_max for every electrode. As a result, Imag > currents are far larger than abs(iE_max). > Seems to me like the problem is not converging, or giving the incorrect solution, since the solution of a variational inequality problem has to satisfy Imag < iE_max, which in your case is not happening. > > Thanks for the detailed explanation. Would you mind if I (or you) forward this to petsc-users list so that others could share their thoughts too. > > Shri > > > > Hi Shri, > > Thanks again for your patience with my being away this past month. I have spent some more time with this problem and I have added Jacobian entries for 'cross terms' such as d(f(Vb)) / d(Ve). However, this did not appear to change the solution arrived at by PETSc. > > I will give you some examples of the f functions and corresponding Jacobian entries as you suggested, with the hope you will be able to see where I may be going wrong. I will use Vb = V_block, Ve = V_electrode, Vmag = magnitude(V_electrode), Imag = magnitude(I_electrode) to name the four degrees of freedom. > > In my 3D finite differencing grid, there are two types of blocks: blocks occupied by electrodes, and blocks occupied by a resistive medium. The block type of the current block and its neighbouring blocks determines the form of f. This is complicated by the fact that individual electrodes generally occupy three vertically-contiguous blocks, and I want to limit the electric current flowing from the whole electrode rather than the current through each block. Note that Ve, Vmag, and Imag are thus computed or recorded only for the top block of each electrode. > > If a block (with coordinates i, j, k in the x, y, and z directions, respectively) is occupied by an electrode, > - for Vb_ijk, f = C_posX * Vb_posX + C_posY * Vb_posY + C_posZ * Vb_posZ > - (C_posX + C_posY + C_posZ + C_negX + C_negY + C_negZ - C_e) * Vb_ijk > + C_negX * Vb_negX + C_negY * Vb_negY + C_negZ * Vb_negZ > - C_e * Ve, > where all C's are calculated constants that do not depend on the other degrees of freedom, but do depend on the type of the neighbouring block in their assigned directions. Then, > the Jacobian entries for Vb_ijk are > df/dVb_negZ = C_negZ (col.i = i, col.j = j, col.k = k - 1) > df/dVb_negY = C_negY (col.i = i, col.j = j - 1, col.k = k) > df/dVb_negX = C_negX (col.i = i - 1, col.j = j, col.k = k) > df/dVb_ijk = - (C_posX + C_posY + C_posZ + C_negX + C_negY + C_negZ - C_e) (col.i = i, col.j = j, col.k = k) > df/dVb_posX = C_posX (col.i = i + 1, col.j = j, col.k = k) > df/dVb_posY = C_posY (col.i = i, col.j = j + 1, col.k = k) > df/dVb_posZ = C_posZ (col.i = i, col.j = j, col.k = k + 1) > For each of these, row.i = i, row.j = j, row.k = k, and row.c = DOF_VB (the DOF index for Vb). Also, col.c = DOF_VB. > The only cross term for the Vb_ijk function is > df/dVe_ijk = -C_e (row.i = i, row.j = j, row.k = k, row.c = DOF_VB, col.i = i, col.j = j, col.k = k col.c = DOF_VE) > > - for Ve_ijk, f = 0.0 + 0.0*i, and all corresponding Jacobian entries are zero as well. Ve_ijk is set to the maximum voltage for each electrode when the initial solution is set. Since the magnitude of Ve is constrained rather than Ve itself, I believe this is correct, but could this be why Ve does not change as it should when the problem is being solved? > > - for Vmag_ijk, f = abs(Ve_ijk) - abs(Ve_ijk_max). The idea was to provide some 'pressure' towards Ve = Ve_max. The corresponding Jacobian entries are > df/dVmag_ijk = 1.0 + 0.0*i (row.i = i, row.j = j, row.k = k, row.c = DOF_VMAG, col.i = i, col.j = j, col.k = k, col.c = DOF_VMAG) > df/dVe_ijk = 0.5 * conj(Ve_ijk) / abs(Ve_ijk) (row.i = i, row.j = j, row.k = k, row.c = DOF_VMAG, col.i = i, col.j = j, col.k = k, col.c = DOF_VE) > (However, since d(abs(Ve_ijk))/dVe_ijk does not have satisfied Cauchy-Reimann equations for the entire complex plane, this derivative does not technically exist.) > > - for Imag_ijk, f = abs(iE) - iE_max. Again, the idea was to provide some 'pressure' towards iE = iE_max. iE is calculated by adding iE_block currents for each block occupied by an electrode. iE_block for block i,j,k is found from > iE_block_ijk = aE_ijk * (Ve - Vb_ijk) > where aE_ijk is a constant that does not depend on any degrees of freedom. There is one Ve value per electrode. The Jacobian entries are then > df/dImag_ijk = 1.0 + 0.0*i (row.i = i, row.j = j, row.k = k, row.c = DOF_IMAG, col.i = i, col.j = j, col.k = k, col.c = DOF_IMAG) > df/dVe = 0.5 * aE_sum * conj(iE) / abs(iE) (row.i = i, row.j = j, row.k = k, row.c = DOF_IMAG, col.i = i, col.j = j, col.k = k, col.c = DOF_VE) > df/dVb_ijk = - 0.5 * aE_ijk * conj(iE) / abs(iE) (row.i = i, row.j = j, row.k = k, row.c = DOF_IMAG, col.i = i, col.j = j, col.k = k, col.c = DOF_VB) > df/dVb_ij(k-1) = - 0.5 * aE_ij(k-1) * conj(iE) / abs(iE) (row.i = i, row.j = j, row.k = k, row.c = DOF_IMAG, col.i = i, col.j = j, col.k = k-1, col.c = DOF_VB) > df/dVb_ij(k-2) = - 0.5 * aE_ij(k-2) * conj(iE) / abs(iE) (row.i = i, row.j = j, row.k = k, row.c = DOF_IMAG, col.i = i, col.j = j, col.k = k-2, col.c = DOF_VB) > > If a block (with coordinates i, j, k) is not occupied by an electrode, C_e is set to zero in the expression for Vb_ijk and all function and Jacobian entries for Ve, Vmag, and Imag are either not set or set to zero. > > My SNES_VI constraints are set as follows for every block: > xl_ijk_DOF_V = -SNES_VI_INF > xu_ijk_DOF_V = SNES_VI_INF > xl_ijk_DOF_VE = -SNES_VI_INF > xu_ijk_DOF_VE = SNES_VI_INF > xl_ijk_DOF_VMAG = 0.0 > xu_ijk_DOF_VMAG = abs(Ve_max) > xl_ijk_DOF_IMAG = 0.0 > xu_ijk_DOF_IMAG = abs(iE_max) > > > When I run my program with -snes_vi_type rs, the solution converges quickly but Vmag voltages remain the same as Ve_max for every electrode. As a result, Imag currents are far larger than abs(iE_max). Are you able to see from here what the problem might be? Again, my hunches are that the complex derivatives do not work in my Jacobian because they do not exist, or that my choices of f for Ve, Vmag, and Imag are not correct. > > Please let me know if you would like clarification on any portion of my description., and thank you again for all your help with this. > > Jon -------------- next part -------------- An HTML attachment was scrubbed... URL: From dominik at itis.ethz.ch Fri Sep 9 10:06:49 2011 From: dominik at itis.ethz.ch (Dominik Szczerba) Date: Fri, 9 Sep 2011 17:06:49 +0200 Subject: [petsc-users] arbitrary number of linear systems in an application Message-ID: Hi, I have an application where I do not know how many linears systems I will solve. Solution of the first will determine coefficient of the second etc., e.g., in case of a pressure correction method. I do not see a problem of having a std::vector of KSP objects, matrices and vectors (at least not yet...) but I am not sure how to handle KSPSetConvergenceTest(this->ksp, DefaultConvergenceTest, (void*) &ctxConvergence, DestroyDefaultConvergenceContext); where the functions are defined as #undef __FUNCT__ #define __FUNCT__ "DefaultConvergenceTest" PetscErrorCode DefaultConvergenceTest(KSP ksp, int n, PetscReal rnorm, KSPConvergedReason *reason, void *void_ctx) { // ... } and #undef __FUNCT__ #define __FUNCT__ "destroyConvergenceContext" PetscErrorCode DestroyDefaultConvergenceContext(void *void_ctx) // ... } I would like something like a user defining how many linear system he wants and I want these functions to be pre-filled for them with a default generic content, which the user could, however, overwrite to possibly have different criteria for different solvers. I am unsure how to achieve this elegantly. I am thinking about a class, whereby such a function would be virtual static member. Then I could create a vector of instances. But will the __FUNCT__ mechanism accept such functions? Any better ideas? Many thanks for any feedback. Regards, Dominik From abhyshr at mcs.anl.gov Fri Sep 9 11:54:44 2011 From: abhyshr at mcs.anl.gov (Shri) Date: Fri, 9 Sep 2011 11:54:44 -0500 (CDT) Subject: [petsc-users] Conditional Constraints In-Reply-To: <1DDCA07C-1833-4D45-B3F7-2D9B2A45AE1C@mcmillan-mcgee.com> Message-ID: <381892016.290363.1315587284725.JavaMail.root@zimbra.anl.gov> > Would f(Ve_ijk) = Imag - Imag_max = 0 simply be equivalent to saying, solve the real system of equations Imag - Imag_max = 0 and 0 = 0 (the imaginary part)? Yes. > Regardless, I am not sure what a correct choice of f(Ve_ijk) would be Maybe, having the equation for current block iE_block_ijk = aE_ijk * (Ve - Vb_ijk) for f(Ve_ijk)? where iE_block_ijk = Imag_max + j*0 ? Perhaps, you can have a different and better expression for f(Ve_ijk) based on some related physics. ----- Original Message ----- Shri, Note here that (a) f(Ve_ijk) is a complex function while Imag and Imag_max are real. So the above equation does not hold. Would f(Ve_ijk) = Imag - Imag_max = 0 simply be equivalent to saying, solve the real system of equations Imag - Imag_max = 0 and 0 = 0 (the imaginary part)? Regardless, I am not sure what a correct choice of f(Ve_ijk) would be. (b) In your previous email, you had bounds on Imag, 0 < Imag < Imag_max. Now you want Ve_ijk to increase if Imag > Imag_max and decrease if Imag < Imag_max. Are you trying to find the value of Ve at which the current is maximum, i.e. Imag = Imag_max, by doing this? Yes, that is essentially what I am trying to find. Ve determines Vb, and the difference between the two determines Imag. Vmag (magnitude of Ve) and Imag have the restrictions 0 < Vmag < Vmag_max and 0 < Imag < Imag_max. I want to find either Ve for Imag = Imag_max or Imag for Vmag = Vmag_max. In my test scenario, Vmag_max happens to be quite high so Imag_max will always be reached first, but in general either situation is possible. Thank you again, Jon On 2011-09-08, at 8:34 PM, Shri wrote: Jon, > I do not want Ve to be constant, however; I would like it to either decrease in response to Imag > Imag_max or increase in response to Imag < Imag_max, providing > the condition Vmag <= Vmag_max remains true. > Should I then have something like f(Ve_ijk) = Imag - Imag_max? Note here that (a) f(Ve_ijk) is a complex function while Imag and Imag_max are real. So the above equation does not hold. (b) In your previous email, you had bounds on Imag, 0 < Imag < Imag_max. Now you want Ve_ijk to increase if Imag > Imag_max and decrease if Imag < Imag_max. Are you trying to find the value of Ve at which the current is maximum, i.e. Imag = Imag_max, by doing this? ----- Original Message ----- Shri, Thanks very much for your response. In fact, -snes_converged_reason revealed that the solution was not converging, as you suggested: "Nonlinear solve did not converge due to DIVERGED_LINE_SEARCH." I had not realized this earlier since the voltage distribution in the solution appeared to be correct, even though the SNES_VI constraints were not satisfied. Given that and your explanation, I am sure something is wrong with my choices of f. I do not want Ve to be constant, however; I would like it to either decrease in response to Imag > Imag_max or increase in response to Imag < Imag_max, providing the condition Vmag <= Vmag_max remains true. Should I then have something like f(Ve_ijk) = Imag - Imag_max? Note that you have One more thing from your response: >If a block (with coordinates i, j, k) is not occupied by an electrode , C_e is set to zero in the expression for Vb_ijk and all function and Jacobian entries for Ve, >Vmag, and Imag are either not set or set to zero. Does this mean f(Ve_ijk) = f(Vmag_ijk) = f(Imag_ijk) = 0? If so, then Ve_ijk, Vmag_ijk, and Imag_ijk would retain their initial values for all time steps. In my present scheme, f(Ve_ijk) = f(Vmag_ijk) = 0 and f(Imag_ijk) is non-zero for electrode blocks; all three are zero for non-electrode blocks. Depending on the choice of initial conditions, I can make f(Vmag_ijk) non-zero as well (until Ve changes to match Ve_max). Thanks again, Jon On 2011-09-08, at 3:42 PM, Shri wrote: Jon, > for Ve_ijk, f = 0.0 + 0.0*i, and all corresponding Jacobian entries are zero as well. Ve_ijk is set to the maximum voltage for each electrode when the initial solution >is - set. Since the magnitude of Ve is constrained rather than Ve itself, I believe this is correct, but could this be why Ve does not change as it should when the >problem is - being solved? This f(Ve_ijk) seems very odd to me since normally you would always have f(some/all variables) = 0 as the function. If f=0.0+0.0*i for all steps, then you don't need a seperate variable Ve_ijk, you can treat it as a constant since Ve_ijk will not change its value during the solutions. Moroever, the Jacobian rows for df(Ve_ijk)/d(all variables) would be all zeros which implies a singular matrix. I am not sure how the solution converged for this case. Were you using -pc_type lu with some shift or some external direct solver such as superlu? What do you get for -snes_monitor -snes_converged_reason? > Ve_ijk is set to the maximum voltage for each electrode when the initial solution is set. Since the magnitude of Ve is constrained rather than Ve itself. > for Vmag_ijk, f = abs(Ve_ijk) - abs(Ve_ijk_max) Since Ve_ijk = Ve_ijk_max, f for Vmag_ijk = 0.0 and hence Vmag_ijk also will also retain its initial value throughout. >If a block (with coordinates i, j, k) is not occupied by an electrode , C_e is set to zero in the expression for Vb_ijk and all function and Jacobian entries for Ve, >Vmag, and Imag are either not set or set to zero. Does this mean f(Ve_ijk) = f(Vmag_ijk) = f(Imag_ijk) = 0? If so, then Ve_ijk, Vmag_ijk, and Imag_ijk would retain their initial values for all time steps. > When I run my program with -snes_vi_type rs, the solution converges quickly but Vmag voltages remain the same as Ve_max for every electrode. As a result, Imag > currents are far larger than abs(iE_max). Seems to me like the problem is not converging, or giving the incorrect solution, since the solution of a variational inequality problem has to satisfy Imag < iE_max, which in your case is not happening. Thanks for the detailed explanation. Would you mind if I (or you) forward this to petsc-users list so that others could share their thoughts too. Shri ----- Original Message ----- Hi Shri, Thanks again for your patience with my being away this past month. I have spent some more time with this problem and I have added Jacobian entries for 'cross terms' such as d(f(Vb)) / d(Ve). However, this did not appear to change the solution arrived at by PETSc. I will give you some examples of the f functions and corresponding Jacobian entries as you suggested, with the hope you will be able to see where I may be going wrong. I will use Vb = V_block, Ve = V_electrode, Vmag = magnitude(V_electrode), Imag = magnitude(I_electrode) to name the four degrees of freedom. In my 3D finite differencing grid, there are two types of blocks: blocks occupied by electrodes, and blocks occupied by a resistive medium. The block type of the current block and its neighbouring blocks determines the form of f. This is complicated by the fact that individual electrodes generally occupy three vertically-contiguous blocks, and I want to limit the electric current flowing from the whole electrode rather than the current through each block. Note that Ve, Vmag, and Imag are thus computed or recorded only for the top block of each electrode. If a block (with coordinates i, j, k in the x, y, and z directions, respectively) is occupied by an electrode , - for Vb_ijk, f = C_posX * Vb_posX + C_posY * Vb_posY + C_posZ * Vb_posZ - (C_posX + C_posY + C_posZ + C_negX + C_negY + C_negZ - C_e) * Vb_ijk + C_negX * Vb_negX + C_negY * Vb_negY + C_negZ * Vb_negZ - C_e * Ve, where all C's are calculated constants that do not depend on the other degrees of freedom, but do depend on the type of the neighbouring block in their assigned directions. Then, the Jacobian entries for Vb_ijk are df/dVb_negZ = C_negZ (col.i = i, col.j = j, col.k = k - 1) df/dVb_negY = C_negY (col.i = i, col.j = j - 1, col.k = k) df/dVb_negX = C_negX (col.i = i - 1, col.j = j, col.k = k) df/dVb_ijk = - (C_posX + C_posY + C_posZ + C_negX + C_negY + C_negZ - C_e) (col.i = i, col.j = j, col.k = k) df/dVb_posX = C_posX (col.i = i + 1, col.j = j, col.k = k) df/dVb_posY = C_posY (col.i = i, col.j = j + 1, col.k = k) df/dVb_posZ = C_posZ (col.i = i, col.j = j, col.k = k + 1) For each of these, row.i = i, row.j = j, row.k = k, and row.c = DOF_VB (the DOF index for Vb). Also, col.c = DOF_VB. The only cross term for the Vb_ijk function is df/dVe_ijk = -C_e (row.i = i, row.j = j, row.k = k, row.c = DOF_VB, col.i = i, col.j = j, col.k = k col.c = DOF_VE) - for Ve_ijk, f = 0.0 + 0.0*i, and all corresponding Jacobian entries are zero as well. Ve_ijk is set to the maximum voltage for each electrode when the initial solution is set. Since the magnitude of Ve is constrained rather than Ve itself, I believe this is correct, but could this be why Ve does not change as it should when the problem is being solved? - for Vmag_ijk, f = abs(Ve_ijk) - abs(Ve_ijk_max). The idea was to provide some 'pressure' towards Ve = Ve_max. The corresponding Jacobian entries are df/dVmag_ijk = 1.0 + 0.0*i (row.i = i, row.j = j, row.k = k, row.c = DOF_VMAG, col.i = i, col.j = j, col.k = k, col.c = DOF_VMAG) df/dVe_ijk = 0.5 * conj(Ve_ijk) / abs(Ve_ijk) (row.i = i, row.j = j, row.k = k, row.c = DOF_VMAG, col.i = i, col.j = j, col.k = k, col.c = DOF_VE) (However, since d(abs(Ve_ijk))/dVe_ijk does not have satisfied Cauchy-Reimann equations for the entire complex plane, this derivative does not technically exist.) - for Imag_ijk, f = abs(iE) - iE_max. Again, the idea was to provide some 'pressure' towards iE = iE_max. iE is calculated by adding iE_block currents for each block occupied by an electrode. iE_block for block i,j,k is found from iE_block_ijk = aE_ijk * (Ve - Vb_ijk) where aE_ijk is a constant that does not depend on any degrees of freedom. There is one Ve value per electrode. The Jacobian entries are then df/dImag_ijk = 1.0 + 0.0*i (row.i = i, row.j = j, row.k = k, row.c = DOF_IMAG, col.i = i, col.j = j, col.k = k, col.c = DOF_IMAG) df/dVe = 0.5 * aE_sum * conj(iE) / abs(iE) (row.i = i, row.j = j, row.k = k, row.c = DOF_IMAG, col.i = i, col.j = j, col.k = k, col.c = DOF_VE) df/dVb_ijk = - 0.5 * aE_ijk * conj(iE) / abs(iE) (row.i = i, row.j = j, row.k = k, row.c = DOF_IMAG, col.i = i, col.j = j, col.k = k, col.c = DOF_VB) df/dVb_ij(k-1) = - 0.5 * aE_ij(k-1) * conj(iE) / abs(iE) (row.i = i, row.j = j, row.k = k, row.c = DOF_IMAG, col.i = i, col.j = j, col.k = k-1, col.c = DOF_VB) df/dVb_ij(k-2) = - 0.5 * aE_ij(k-2) * conj(iE) / abs(iE) (row.i = i, row.j = j, row.k = k, row.c = DOF_IMAG, col.i = i, col.j = j, col.k = k-2, col.c = DOF_VB) If a block (with coordinates i, j, k) is not occupied by an electrode , C_e is set to zero in the expression for Vb_ijk and all function and Jacobian entries for Ve, Vmag, and Imag are either not set or set to zero. My SNES_VI constraints are set as follows for every block: xl_ijk_DOF_V = -SNES_VI_INF xu_ijk_DOF_V = SNES_VI_INF xl_ijk_DOF_VE = -SNES_VI_INF xu_ijk_DOF_VE = SNES_VI_INF xl_ijk_DOF_VMAG = 0.0 xu_ijk_DOF_VMAG = abs(Ve_max) xl_ijk_DOF_IMAG = 0.0 xu_ijk_DOF_IMAG = abs(iE_max) When I run my program with -snes_vi_type rs, the solution converges quickly but Vmag voltages remain the same as Ve_max for every electrode. As a result, Imag currents are far larger than abs(iE_max). Are you able to see from here what the problem might be? Again, my hunches are that the complex derivatives do not work in my Jacobian because they do not exist, or that my choices of f for Ve, Vmag, and Imag are not correct. Please let me know if you would like clarification on any portion of my description., and thank you again for all your help with this. Jon -------------- next part -------------- An HTML attachment was scrubbed... URL: From amrit_pou at hotmail.com Fri Sep 9 21:46:08 2011 From: amrit_pou at hotmail.com (amrit poudel) Date: Sat, 10 Sep 2011 02:46:08 +0000 Subject: [petsc-users] Fwd: nonzero prescribed boundary condition In-Reply-To: References: , , <5EB9541F-90AE-4373-ACCF-108106E59AD4@mcs.anl.gov>, , <040E28514B5D407BA8F7617141426051@rti458laptop>, , , , , , <8664863E669F4B7F8A045B2FEC5E22CC@rti458laptop>, , <8DC60B1B-A0BB-4571-90CE-C0CD9E05D446@mcs.anl.gov>, , <0B5B1110339742BC8990AB61531039A8@rti458laptop>, , , , , , , , , , <8A1482FA5FCE46CDBBBB7BA3593EF18B@rti458laptop>, , , , Message-ID: If I run a program with the following runtime option (notice that I have not included -pc_type lu ), will the program use mumps to solve the linear system of equations directly, or does it run with default gmres iterative solver? Why do we have to specify a preconditioner for a direct solver ? -pc_factor_mat_solver_package mumps Thanks for your help.. > From: bsmith at mcs.anl.gov > Date: Thu, 8 Sep 2011 14:02:09 -0500 > To: petsc-users at mcs.anl.gov > Subject: Re: [petsc-users] Fwd: nonzero prescribed boundary condition > > > Are you running with -ksp_converged_reason and -ksp_monitor_true_residual to see if the iterative method is actually converging and how rapidly. Also if you impose a tight tolerance on the iterative solver say with -ksp_rtol 1.e-12 how much do the "solutions" differ, it should get smaller the smaller you make this tolerance (in double precision you cannot expect to use a tolerance less than 1.e-3 or 1.e-14) > > If your "big" matrix is very complicated and comes from no well understood simulation it may be that it is very ill-conditioned, in that case you either need to understand the underlying equations real well to develop an appropriate preconditioner or just use parallel direct solvers (which get slow for very large problems but can handle more ill-conditioning.n To use the MUMPS parallel direct solver you can configure PETSc with > --download-mumps --download-scalapack --download-blacs and run the program with -pc_type lu -pc_factor_mat_solver_package mumps > > Barry > > > > On Sep 8, 2011, at 12:57 PM, amrit poudel wrote: > > > After running my simulation multiple times on a multiprocessor computer I've just verified that using iterative solver (default gmres) in PETSc to solve a linear system of equations ( Cx=b) with more than 2 processors setting ALWAYS lead to erroneous result. Running identical code with identical setting except for the number of processors ( set this to 2) ALWAYS gives me correct result . > > > > I am really not sure what is the point behind including iterative solvers if they result into erroneous result on a multiprocessor computer. The result I get from multiprocessor computer is a complete garbage, so I am really not talking about small percentage of error here. Also, if somebody could enlighten why the iterative solvers are error prone on multiprocessors that will be highly appreciated. > > > > I am very hopeful that there is a way around to this problem, because PETSc is such a powerful and useful library that I really do not want to give up on this and start something else from scratch. > > > > > > Would you think that a DIRECT SOLVER would circumvent this problem? My problem is that I have a very large system of equations and the size of a sparse coefficient matrix is huge ( > 1e+8). I assemble this matrix in MATLAB, write to a binary file, and read it in PETSc. So I really need to be able to solve this system of equations in a cluster of computers (which inherently has multiprocessors and distributed memory setting). Does this mean I am completely out of luck with PETSc's iterative solver package and the only hope for me is the direct solver? I do have MUMPS downloaded and compiled with PETSc, so I will give that a try and see what results I obtain, but I am really surprised that iterative solvers are no good in a large multiprocessor settings. > > > > Any insights, suggestions/advice will be highly appreciated. > > > > Thanks. > > > > PS (I can attach my entire code, plots that compare the results obtained by solving Cx=b in 2 processors vs 12 or 6 processors if any body wants to take a look at it. I get garbage if I run iterative solver on 12 processors) > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hzhang at mcs.anl.gov Fri Sep 9 22:13:46 2011 From: hzhang at mcs.anl.gov (Hong Zhang) Date: Fri, 9 Sep 2011 22:13:46 -0500 Subject: [petsc-users] Fwd: nonzero prescribed boundary condition In-Reply-To: References: <5EB9541F-90AE-4373-ACCF-108106E59AD4@mcs.anl.gov> <040E28514B5D407BA8F7617141426051@rti458laptop> <8664863E669F4B7F8A045B2FEC5E22CC@rti458laptop> <8DC60B1B-A0BB-4571-90CE-C0CD9E05D446@mcs.anl.gov> <0B5B1110339742BC8990AB61531039A8@rti458laptop> <8A1482FA5FCE46CDBBBB7BA3593EF18B@rti458laptop> Message-ID: amrit : > > If I run a program with the following runtime option (notice that I have not > included -pc_type lu ), will the program use mumps to solve the linear > system of equations directly, or does it? run with default gmres iterative > solver? Why do we have to specify a preconditioner for a direct solver ? > > ?-pc_factor_mat_solver_package mumps You must include '-pc_type lu' to use mumps. There are many preconditioners for solving linear system, lu is one of pcs, and mumps is one of packages implement lu. Hong > > >> From: bsmith at mcs.anl.gov >> Date: Thu, 8 Sep 2011 14:02:09 -0500 >> To: petsc-users at mcs.anl.gov >> Subject: Re: [petsc-users] Fwd: nonzero prescribed boundary condition >> >> >> Are you running with -ksp_converged_reason and -ksp_monitor_true_residual >> to see if the iterative method is actually converging and how rapidly. Also >> if you impose a tight tolerance on the iterative solver say with -ksp_rtol >> 1.e-12 how much do the "solutions" differ, it should get smaller the smaller >> you make this tolerance (in double precision you cannot expect to use a >> tolerance less than 1.e-3 or 1.e-14) >> >> If your "big" matrix is very complicated and comes from no well understood >> simulation it may be that it is very ill-conditioned, in that case you >> either need to understand the underlying equations real well to develop an >> appropriate preconditioner or just use parallel direct solvers (which get >> slow for very large problems but can handle more ill-conditioning.n To use >> the MUMPS parallel direct solver you can configure PETSc with >> --download-mumps --download-scalapack --download-blacs and run the program >> with -pc_type lu -pc_factor_mat_solver_package mumps >> >> Barry >> >> >> >> On Sep 8, 2011, at 12:57 PM, amrit poudel wrote: >> >> > After running my simulation multiple times on a multiprocessor computer >> > I've just verified that using iterative solver (default gmres) in PETSc to >> > solve a linear system of equations ( Cx=b) with more than 2 processors >> > setting ALWAYS lead to erroneous result. Running identical code with >> > identical setting except for the number of processors ( set this to 2) >> > ALWAYS gives me correct result . >> > >> > I am really not sure what is the point behind including iterative >> > solvers if they result into erroneous result on a multiprocessor computer. >> > The result I get from multiprocessor computer is a complete garbage, so I am >> > really not talking about small percentage of error here. Also, if somebody >> > could enlighten why the iterative solvers are error prone on multiprocessors >> > that will be highly appreciated. >> > >> > I am very hopeful that there is a way around to this problem, because >> > PETSc is such a powerful and useful library that I really do not want to >> > give up on this and start something else from scratch. >> > >> > >> > Would you think that a DIRECT SOLVER would circumvent this problem? My >> > problem is that I have a very large system of equations and the size of a >> > sparse coefficient matrix is huge ( > 1e+8). I assemble this matrix in >> > MATLAB, write to a binary file, and read it in PETSc. So I really need to be >> > able to solve this system of equations in a cluster of computers (which >> > inherently has multiprocessors and distributed memory setting). Does this >> > mean I am completely out of luck with PETSc's iterative solver package and >> > the only hope for me is the direct solver? I do have MUMPS downloaded and >> > compiled with PETSc, so I will give that a try and see what results I >> > obtain, but I am really surprised that iterative solvers are no good in a >> > large multiprocessor settings. >> > >> > Any insights, suggestions/advice will be highly appreciated. >> > >> > Thanks. >> > >> > PS (I can attach my entire code, plots that compare the results obtained >> > by solving Cx=b in 2 processors vs 12 or 6 processors if any body wants to >> > take a look at it. I get garbage if I run iterative solver on 12 processors) >> > From balay at mcs.anl.gov Fri Sep 9 22:31:57 2011 From: balay at mcs.anl.gov (Satish Balay) Date: Fri, 9 Sep 2011 22:31:57 -0500 (CDT) Subject: [petsc-users] Fwd: nonzero prescribed boundary condition In-Reply-To: References: <5EB9541F-90AE-4373-ACCF-108106E59AD4@mcs.anl.gov> <040E28514B5D407BA8F7617141426051@rti458laptop> <8664863E669F4B7F8A045B2FEC5E22CC@rti458laptop> <8DC60B1B-A0BB-4571-90CE-C0CD9E05D446@mcs.anl.gov> <0B5B1110339742BC8990AB61531039A8@rti458laptop> <8A1482FA5FCE46CDBBBB7BA3593EF18B@rti458laptop> Message-ID: On Fri, 9 Sep 2011, Hong Zhang wrote: > amrit : > > > > If I run a program with the following runtime option (notice that I have not > > included -pc_type lu ), will the program use mumps to solve the linear > > system of equations directly, or does it? run with default gmres iterative > > solver? Why do we have to specify a preconditioner for a direct solver ? > > > > ?-pc_factor_mat_solver_package mumps > > You must include '-pc_type lu' to use mumps. > There are many preconditioners for solving linear system, lu is one of pcs, > and mumps is one of packages implement lu. PETSc solver is organized as 2 parts: KSP, PC With this organization - direct solvers are categorized under PC. Sure - you don't need gmres with a direct solver - so you would use: --ksp_type preonly -pc_type lu The reason this organization makes sense for us is - we easily support stuff like: -ksp_type gmres -pc_type bjacobi -sub_pc lu [-sub_ksp preonly] And as Hong mentioned - mumps is one of the direct solvers we provide interface to - other prominant one is superlu_dist. Both are accessiable with '-pc_type lu -pc_factor_mat_solver_package [mumps or superlu_dist]' etc... Satish From amrit_pou at hotmail.com Fri Sep 9 22:32:14 2011 From: amrit_pou at hotmail.com (amrit poudel) Date: Sat, 10 Sep 2011 03:32:14 +0000 Subject: [petsc-users] Petsc direct solver In-Reply-To: References: , <5EB9541F-90AE-4373-ACCF-108106E59AD4@mcs.anl.gov>, <040E28514B5D407BA8F7617141426051@rti458laptop>, , , <8664863E669F4B7F8A045B2FEC5E22CC@rti458laptop>, <8DC60B1B-A0BB-4571-90CE-C0CD9E05D446@mcs.anl.gov>, <0B5B1110339742BC8990AB61531039A8@rti458laptop>, , , , , <8A1482FA5FCE46CDBBBB7BA3593EF18B@rti458laptop>, , , , , Message-ID: Thanks for the hint. However, my system is pretty large (but sparse) and using -pc_type lu preconditioner throws memory related error, and according to it, the memory requirement is astronomical :( [1]PETSC ERROR: Error reported by MUMPS in numerical factorization phase: Cannot allocate required memory 2022169721 megabytes That amount of memory requirement just does not seem right, and I think it is lu that is creating the problem here, though I could be wrong. If I use -pc_type none, and then do -pc_factor_mat_solver_package mumps, will it use direct solver(mumps) or some default iterative solver ? Sorry for my elementary question. I was under the impression that by setting -pc_type to some preconditioner I was forcing it use iterative solver instead of using mumps. Now that you mentioned I can set -pc_type to any of the available preconditioners and still use the specified direct solver, that cleared up my confusion. Thanks for your help. > Date: Fri, 9 Sep 2011 22:13:46 -0500 > From: hzhang at mcs.anl.gov > To: petsc-users at mcs.anl.gov > Subject: Re: [petsc-users] Fwd: nonzero prescribed boundary condition > > amrit : > > > > If I run a program with the following runtime option (notice that I have not > > included -pc_type lu ), will the program use mumps to solve the linear > > system of equations directly, or does it run with default gmres iterative > > solver? Why do we have to specify a preconditioner for a direct solver ? > > > > -pc_factor_mat_solver_package mumps > > You must include '-pc_type lu' to use mumps. > There are many preconditioners for solving linear system, lu is one of pcs, > and mumps is one of packages implement lu. > > Hong > > > > > > >> From: bsmith at mcs.anl.gov > >> Date: Thu, 8 Sep 2011 14:02:09 -0500 > >> To: petsc-users at mcs.anl.gov > >> Subject: Re: [petsc-users] Fwd: nonzero prescribed boundary condition > >> > >> > >> Are you running with -ksp_converged_reason and -ksp_monitor_true_residual > >> to see if the iterative method is actually converging and how rapidly. Also > >> if you impose a tight tolerance on the iterative solver say with -ksp_rtol > >> 1.e-12 how much do the "solutions" differ, it should get smaller the smaller > >> you make this tolerance (in double precision you cannot expect to use a > >> tolerance less than 1.e-3 or 1.e-14) > >> > >> If your "big" matrix is very complicated and comes from no well understood > >> simulation it may be that it is very ill-conditioned, in that case you > >> either need to understand the underlying equations real well to develop an > >> appropriate preconditioner or just use parallel direct solvers (which get > >> slow for very large problems but can handle more ill-conditioning.n To use > >> the MUMPS parallel direct solver you can configure PETSc with > >> --download-mumps --download-scalapack --download-blacs and run the program > >> with -pc_type lu -pc_factor_mat_solver_package mumps > >> > >> Barry > >> > >> > >> > >> On Sep 8, 2011, at 12:57 PM, amrit poudel wrote: > >> > >> > After running my simulation multiple times on a multiprocessor computer > >> > I've just verified that using iterative solver (default gmres) in PETSc to > >> > solve a linear system of equations ( Cx=b) with more than 2 processors > >> > setting ALWAYS lead to erroneous result. Running identical code with > >> > identical setting except for the number of processors ( set this to 2) > >> > ALWAYS gives me correct result . > >> > > >> > I am really not sure what is the point behind including iterative > >> > solvers if they result into erroneous result on a multiprocessor computer. > >> > The result I get from multiprocessor computer is a complete garbage, so I am > >> > really not talking about small percentage of error here. Also, if somebody > >> > could enlighten why the iterative solvers are error prone on multiprocessors > >> > that will be highly appreciated. > >> > > >> > I am very hopeful that there is a way around to this problem, because > >> > PETSc is such a powerful and useful library that I really do not want to > >> > give up on this and start something else from scratch. > >> > > >> > > >> > Would you think that a DIRECT SOLVER would circumvent this problem? My > >> > problem is that I have a very large system of equations and the size of a > >> > sparse coefficient matrix is huge ( > 1e+8). I assemble this matrix in > >> > MATLAB, write to a binary file, and read it in PETSc. So I really need to be > >> > able to solve this system of equations in a cluster of computers (which > >> > inherently has multiprocessors and distributed memory setting). Does this > >> > mean I am completely out of luck with PETSc's iterative solver package and > >> > the only hope for me is the direct solver? I do have MUMPS downloaded and > >> > compiled with PETSc, so I will give that a try and see what results I > >> > obtain, but I am really surprised that iterative solvers are no good in a > >> > large multiprocessor settings. > >> > > >> > Any insights, suggestions/advice will be highly appreciated. > >> > > >> > Thanks. > >> > > >> > PS (I can attach my entire code, plots that compare the results obtained > >> > by solving Cx=b in 2 processors vs 12 or 6 processors if any body wants to > >> > take a look at it. I get garbage if I run iterative solver on 12 processors) > >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From balay at mcs.anl.gov Fri Sep 9 22:40:16 2011 From: balay at mcs.anl.gov (Satish Balay) Date: Fri, 9 Sep 2011 22:40:16 -0500 (CDT) Subject: [petsc-users] Petsc direct solver In-Reply-To: References: , <5EB9541F-90AE-4373-ACCF-108106E59AD4@mcs.anl.gov>, <040E28514B5D407BA8F7617141426051@rti458laptop>, , , <8664863E669F4B7F8A045B2FEC5E22CC@rti458laptop>, <8DC60B1B-A0BB-4571-90CE-C0CD9E05D446@mcs.anl.gov>, <0B5B1110339742BC8990AB61531039A8@rti458laptop>, , , , , <8A1482FA5FCE46CDBBBB7BA3593EF18B@rti458laptop>, , , , , Message-ID: You are misinterpreting Hong's statement. Satish On Sat, 10 Sep 2011, amrit poudel wrote: > Now that you mentioned I can set -pc_type to any of the available > preconditioners and still use the specified direct solver, that > cleared up my confusion. > > You must include '-pc_type lu' to use mumps. There are many > > preconditioners for solving linear system, lu is one of pcs, and > > mumps is one of packages implement lu. From amrit_pou at hotmail.com Fri Sep 9 22:43:18 2011 From: amrit_pou at hotmail.com (amrit poudel) Date: Sat, 10 Sep 2011 03:43:18 +0000 Subject: [petsc-users] Petsc direct solver In-Reply-To: References: , <5EB9541F-90AE-4373-ACCF-108106E59AD4@mcs.anl.gov>, <040E28514B5D407BA8F7617141426051@rti458laptop>, , , <8664863E669F4B7F8A045B2FEC5E22CC@rti458laptop>, <8DC60B1B-A0BB-4571-90CE-C0CD9E05D446@mcs.anl.gov>, <0B5B1110339742BC8990AB61531039A8@rti458laptop>, , , , , <8A1482FA5FCE46CDBBBB7BA3593EF18B@rti458laptop>, , , , , , Message-ID: Thanks Satish for the clarification. If I use the following run time option why does the mumps get ignored as per the warning shown down below? -ksp_type preonly -pc_type none -pc_factor_mat_solver_package mumps As I understand it, this should read, no Krylov subspace method, no preconditioner, use direct solver. WARNING! There are options you set that were not used! WARNING! could be spelling mistake, etc! Option left: name:-pc_factor_mat_solver_package value: mumps > Date: Fri, 9 Sep 2011 22:31:57 -0500 > From: balay at mcs.anl.gov > To: petsc-users at mcs.anl.gov > Subject: Re: [petsc-users] Fwd: nonzero prescribed boundary condition > > On Fri, 9 Sep 2011, Hong Zhang wrote: > > > amrit : > > > > > > If I run a program with the following runtime option (notice that I have not > > > included -pc_type lu ), will the program use mumps to solve the linear > > > system of equations directly, or does it run with default gmres iterative > > > solver? Why do we have to specify a preconditioner for a direct solver ? > > > > > > -pc_factor_mat_solver_package mumps > > > > You must include '-pc_type lu' to use mumps. > > There are many preconditioners for solving linear system, lu is one of pcs, > > and mumps is one of packages implement lu. > > PETSc solver is organized as 2 parts: KSP, PC > > With this organization - direct solvers are categorized under PC. > Sure - you don't need gmres with a direct solver - so you would use: > > --ksp_type preonly -pc_type lu > > The reason this organization makes sense for us is - we easily support > stuff like: > > -ksp_type gmres -pc_type bjacobi -sub_pc lu [-sub_ksp preonly] > > And as Hong mentioned - mumps is one of the direct solvers we provide > interface to - other prominant one is superlu_dist. Both are > accessiable with > > '-pc_type lu -pc_factor_mat_solver_package [mumps or superlu_dist]' etc... > > Satish -------------- next part -------------- An HTML attachment was scrubbed... URL: From amrit_pou at hotmail.com Fri Sep 9 22:46:21 2011 From: amrit_pou at hotmail.com (amrit poudel) Date: Sat, 10 Sep 2011 03:46:21 +0000 Subject: [petsc-users] Petsc direct solver In-Reply-To: References: , , <5EB9541F-90AE-4373-ACCF-108106E59AD4@mcs.anl.gov>, , <040E28514B5D407BA8F7617141426051@rti458laptop>, , , , , , <8664863E669F4B7F8A045B2FEC5E22CC@rti458laptop>, , <8DC60B1B-A0BB-4571-90CE-C0CD9E05D446@mcs.anl.gov>, , <0B5B1110339742BC8990AB61531039A8@rti458laptop>, , , , , , , , , , <8A1482FA5FCE46CDBBBB7BA3593EF18B@rti458laptop>, , , , , , , , , , , , , Message-ID: All right, got it. I must specify -pc_type to lu if I want to use mumps and there is no way around it. This basically means I will need astronomical amount of memory :( as noted in my previous email. From: amrit_pou at hotmail.com To: petsc-users at mcs.anl.gov Date: Sat, 10 Sep 2011 03:43:18 +0000 Subject: [petsc-users] Petsc direct solver Thanks Satish for the clarification. If I use the following run time option why does the mumps get ignored as per the warning shown down below? -ksp_type preonly -pc_type none -pc_factor_mat_solver_package mumps As I understand it, this should read, no Krylov subspace method, no preconditioner, use direct solver. WARNING! There are options you set that were not used! WARNING! could be spelling mistake, etc! Option left: name:-pc_factor_mat_solver_package value: mumps > Date: Fri, 9 Sep 2011 22:31:57 -0500 > From: balay at mcs.anl.gov > To: petsc-users at mcs.anl.gov > Subject: Re: [petsc-users] Fwd: nonzero prescribed boundary condition > > On Fri, 9 Sep 2011, Hong Zhang wrote: > > > amrit : > > > > > > If I run a program with the following runtime option (notice that I have not > > > included -pc_type lu ), will the program use mumps to solve the linear > > > system of equations directly, or does it run with default gmres iterative > > > solver? Why do we have to specify a preconditioner for a direct solver ? > > > > > > -pc_factor_mat_solver_package mumps > > > > You must include '-pc_type lu' to use mumps. > > There are many preconditioners for solving linear system, lu is one of pcs, > > and mumps is one of packages implement lu. > > PETSc solver is organized as 2 parts: KSP, PC > > With this organization - direct solvers are categorized under PC. > Sure - you don't need gmres with a direct solver - so you would use: > > --ksp_type preonly -pc_type lu > > The reason this organization makes sense for us is - we easily support > stuff like: > > -ksp_type gmres -pc_type bjacobi -sub_pc lu [-sub_ksp preonly] > > And as Hong mentioned - mumps is one of the direct solvers we provide > interface to - other prominant one is superlu_dist. Both are > accessiable with > > '-pc_type lu -pc_factor_mat_solver_package [mumps or superlu_dist]' etc... > > Satish -------------- next part -------------- An HTML attachment was scrubbed... URL: From balay at mcs.anl.gov Fri Sep 9 22:49:50 2011 From: balay at mcs.anl.gov (Satish Balay) Date: Fri, 9 Sep 2011 22:49:50 -0500 (CDT) Subject: [petsc-users] Petsc direct solver In-Reply-To: References: , <5EB9541F-90AE-4373-ACCF-108106E59AD4@mcs.anl.gov>, <040E28514B5D407BA8F7617141426051@rti458laptop>, , , <8664863E669F4B7F8A045B2FEC5E22CC@rti458laptop>, <8DC60B1B-A0BB-4571-90CE-C0CD9E05D446@mcs.anl.gov>, <0B5B1110339742BC8990AB61531039A8@rti458laptop>, , , , , <8A1482FA5FCE46CDBBBB7BA3593EF18B@rti458laptop>, , , , , , Message-ID: On Sat, 10 Sep 2011, amrit poudel wrote: > > Thanks Satish for the clarification. > > If I use the following run time option why does the mumps get ignored as per the warning shown down below? > > -ksp_type preonly -pc_type none -pc_factor_mat_solver_package mumps As mentioned to you twice already - mumps is a component of '-pc_type lu' so you need: -ksp_type preonly -pc_type lu -pc_factor_mat_solver_package mumps > As I understand it, this should read, no Krylov subspace method, no preconditioner, use direct solver. '-pc_type lu' *IS* the direct solver in PETSc. There is no separate -directsolver category in PETSc. Satish From balay at mcs.anl.gov Fri Sep 9 23:04:12 2011 From: balay at mcs.anl.gov (Satish Balay) Date: Fri, 9 Sep 2011 23:04:12 -0500 (CDT) Subject: [petsc-users] Petsc direct solver In-Reply-To: References: , , <5EB9541F-90AE-4373-ACCF-108106E59AD4@mcs.anl.gov>, , <040E28514B5D407BA8F7617141426051@rti458laptop>, , , , , , <8664863E669F4B7F8A045B2FEC5E22CC@rti458laptop>, , <8DC60B1B-A0BB-4571-90CE-C0CD9E05D446@mcs.anl.gov>, , <0B5B1110339742BC8990AB61531039A8@rti458laptop>, , , , , , , , , , <8A1482FA5FCE46CDBBBB7BA3593EF18B@rti458laptop>, , , , , , , , , , , , , Message-ID: On Sat, 10 Sep 2011, amrit poudel wrote: > All right, got it. I must specify -pc_type to lu if I want to use > mumps and there is no way around it. This basically means I will > need astronomical amount of memory :( as noted in my previous email. Yes - direct solvers can need lot of memory. > [1]PETSC ERROR: Error reported by MUMPS in numerical factorization phase: Cannot allocate required memory > 2022169721 megabytes As the message says - MUMPS needs that much memory to solve your problem. Your attempt to work arround 'lu' and to call mumps directly - and save memory is based on the false notion that 'lu' in PETSc is something different [from direct solver] - and comsumes its own memory. [which is incorrect] LU in PETSc is an iterface to direct solver implemneations. [who do their own factiorizations - and need memory for that work] Satish From mmnasr at gmail.com Sat Sep 10 05:36:37 2011 From: mmnasr at gmail.com (Mohamad M. Nasr-Azadani) Date: Sat, 10 Sep 2011 03:36:37 -0700 Subject: [petsc-users] DAGetMatrix with extra nonzeros: memory preallocation Message-ID: Hi guys, I am setting up a matrix (parallel) using DA's in 3D. DA is built upon a width=1, STAR_STENCIL type. The nonzero structure of the resulting matrix is hepta-diagonal for the most part, however, since I am imposing boundary conditions using interpolation methods, for some boundary nodes, it is possible that some extra nonzeros are added into new locations. This obviously reduces the matrix setup and assembly efficiency since the new nonzeros are not preallocated. Is there any way that I could do it myself, i.e. preallocate the new nonzeros before matrix setup to get the best performance? Thanks and have a great weekend, Mohamad -------------- next part -------------- An HTML attachment was scrubbed... URL: From stali at geology.wisc.edu Sat Sep 10 06:09:24 2011 From: stali at geology.wisc.edu (Tabrez Ali) Date: Sat, 10 Sep 2011 06:09:24 -0500 Subject: [petsc-users] PETSc 3.2 gmres error Message-ID: <4E6B4564.1070002@geology.wisc.edu> Hello I am running an application using PETSc 3.2 RC on a poor mans cluster at my home (for testing only) which has two nodes running different versions of Debian (they also have different versions of gcc/gfortran) but have the same MPICH2 1.4 and PETSc 3.2 installed on them. Also they do not share the same file system but I make sure that input file/executable paths are exactly same on both machines. After compiling the code separately on the two nodes I launch the parallel program from node 1 using mpiexec -f hosts -n 2 .... (hydra process manager). With PETSc 3.1 the application runs fine, both with CG and GMRES and correct output is generated on both nodes. With PETSc 3.2 the application runs fine with CG. But whenever I use GMRES in 3.2 I get an error (listed below) during KSPSolve. Now admittedly my setup is inconsistent but is there some other obvious reason why the error occurs only with GMRES in 3.2. Thanks in advance. Tabrez [0]PETSC ERROR: --------------------- Error Message ------------------------------------ [0]PETSC ERROR: Invalid argument! [0]PETSC ERROR: Scalar value must be same on all processes, argument # 3! [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: Petsc Development HG revision: a8941623c0b6225ff3688949b01271e9ae85a545 HG Date: Fri Sep 09 19:37:41 2011 -0500 [0]PETSC ERROR: See docs/changes/index.html for recent updates. [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. [0]PETSC ERROR: See docs/index.html for manual pages. [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: ./defmod on a arch-linu named i5 by stali Sat Sep 10 05:32:52 2011 [0]PETSC ERROR: Libraries linked from /opt/petsc-3.2/lib [0]PETSC ERROR: Configure run at Sat Sep 10 04:34:28 2011 [0]PETSC ERROR: Configure options --prefix=/opt/petsc-3.2 --with-mpi-dir=/opt/mpich2-gcc --with-parmetis=1 --download-parmetis=1 --with-shared-libraries [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: VecMAXPY() line 1190 in src/vec/vec/interface/rvector.c [0]PETSC ERROR: BuildGmresSoln() line 345 in src/ksp/ksp/impls/gmres/gmres.c [0]PETSC ERROR: GMREScycle() line 206 in src/ksp/ksp/impls/gmres/gmres.c [0]PETSC ERROR: KSPSolve_GMRES() line 231 in src/ksp/ksp/impls/gmres/gmres.c [0]PETSC ERROR: KSPSolve() line 423 in src/ksp/ksp/interface/itfunc.c From jedbrown at mcs.anl.gov Sat Sep 10 06:38:02 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Sat, 10 Sep 2011 13:38:02 +0200 Subject: [petsc-users] PETSc 3.2 gmres error In-Reply-To: <4E6B4564.1070002@geology.wisc.edu> References: <4E6B4564.1070002@geology.wisc.edu> Message-ID: On Sat, Sep 10, 2011 at 13:09, Tabrez Ali wrote: > Hello > > I am running an application using PETSc 3.2 RC on a poor mans cluster at my > home (for testing only) which has two nodes running different versions of > Debian (they also have different versions of gcc/gfortran) but have the same > MPICH2 1.4 and PETSc 3.2 installed on them. > > Also they do not share the same file system but I make sure that input > file/executable paths are exactly same on both machines. After compiling the > code separately on the two nodes I launch the parallel program from node 1 > using mpiexec -f hosts -n 2 .... (hydra process manager). > > With PETSc 3.1 the application runs fine, both with CG and GMRES and > correct output is generated on both nodes. > > With PETSc 3.2 the application runs fine with CG. > > But whenever I use GMRES in 3.2 I get an error (listed below) during > KSPSolve. Can you reproduce this with any of the examples? For example cd src/ksp/ksp/examples/tutorials make ex2 mpiexec -f hosts -n 2 ./ex2 -ksp_type gmres or, to use your matrix, run (any version that works, including 3.1) with -ksp_view_binary and then cd src/ksp/ksp/examples/tutorials make ex10 mpiexec -f hosts -n 2 ./ex10 -f binaryoutput -ksp_type gmres If these work, there might be memory corruption somewhere in your code causing this. You can also run with -start_in_debugger and check what is in the "alpha" array on each process. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jedbrown at mcs.anl.gov Sat Sep 10 07:14:54 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Sat, 10 Sep 2011 14:14:54 +0200 Subject: [petsc-users] arbitrary number of linear systems in an application In-Reply-To: References: Message-ID: On Fri, Sep 9, 2011 at 17:06, Dominik Szczerba wrote: > Hi, > > I have an application where I do not know how many linears systems I > will solve. Solution of the first will determine coefficient of the > second etc., e.g., in case of a pressure correction method. > I do not see a problem of having a std::vector of KSP objects, > matrices and vectors (at least not yet...) but I am not sure how to > handle > > KSPSetConvergenceTest(this->ksp, DefaultConvergenceTest, (void*) > &ctxConvergence, DestroyDefaultConvergenceContext); > > where the functions are defined as > > > It sounds like you are wanting to define something like class MyConvergenceTest { virtual PetscErrorCode test(KSP ksp,PetscInt it,PetscReal rnorm,KSPConvergedReason *reason) { // ... *reason = ...; } > #undef __FUNCT__ > #define __FUNCT__ "DefaultConvergenceTest" > PetscErrorCode DefaultConvergenceTest(KSP ksp, int n, PetscReal rnorm, > KSPConvergedReason *reason, void *void_ctx) > { > // ... > MyConvergenceTest *p = (MyConvergenceClass*)void_ctx; return p->(ksp,n,rnorm,reason); > > #undef __FUNCT__ > #define __FUNCT__ "destroyConvergenceContext" > PetscErrorCode DestroyDefaultConvergenceContext(void *void_ctx) > // ... MyConvergenceTest *p = (MyConvergenceClass*)void_ctx; delete p; > } > > I would like something like a user defining how many linear system he > wants and I want these functions to be pre-filled for them with a > default generic content, which the user could, however, overwrite to > possibly have different criteria for different solvers. > > I am unsure how to achieve this elegantly. I am thinking about a > class, whereby such a function would be virtual static member. Then I > could create a vector of instances. You can manage creation of these however you want. The Destroy callback can be used to call the destructor on your object, so you don't need to manage that yourself. > But will the __FUNCT__ mechanism > accept such functions? __FUNCT__ has nothing to do with this. It's just a way to get stack traces despite only C99 having standardized a way to get the name of the current function (__func__). -------------- next part -------------- An HTML attachment was scrubbed... URL: From jedbrown at mcs.anl.gov Sat Sep 10 07:35:59 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Sat, 10 Sep 2011 14:35:59 +0200 Subject: [petsc-users] DAGetMatrix with extra nonzeros: memory preallocation In-Reply-To: References: Message-ID: On Sat, Sep 10, 2011 at 12:36, Mohamad M. Nasr-Azadani wrote: > I am setting up a matrix (parallel) using DA's in 3D. > DA is built upon a width=1, STAR_STENCIL type. > The nonzero structure of the resulting matrix is hepta-diagonal for the > most part, however, since I am imposing boundary conditions using > interpolation methods, for some boundary nodes, it is possible that some > extra nonzeros are added into new locations. > This obviously reduces the matrix setup and assembly efficiency since the > new nonzeros are not preallocated. > Is there any way that I could do it myself, i.e. preallocate the new > nonzeros before matrix setup to get the best performance? > Unfortunately there is not a convenient way to express this such that DMGetMatrix() will preallocate for those entries. You can preallocate your own matrix without calling DMGetMatrix(). It's not very hard for a stencil width of 1 with STAR, but it is still some code to write. -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Sat Sep 10 08:29:40 2011 From: knepley at gmail.com (Matthew Knepley) Date: Sat, 10 Sep 2011 08:29:40 -0500 Subject: [petsc-users] PETSc 3.2 gmres error In-Reply-To: References: <4E6B4564.1070002@geology.wisc.edu> Message-ID: On Sat, Sep 10, 2011 at 6:38 AM, Jed Brown wrote: > On Sat, Sep 10, 2011 at 13:09, Tabrez Ali wrote: > >> Hello >> >> I am running an application using PETSc 3.2 RC on a poor mans cluster at >> my home (for testing only) which has two nodes running different versions of >> Debian (they also have different versions of gcc/gfortran) but have the same >> MPICH2 1.4 and PETSc 3.2 installed on them. >> >> Also they do not share the same file system but I make sure that input >> file/executable paths are exactly same on both machines. After compiling the >> code separately on the two nodes I launch the parallel program from node 1 >> using mpiexec -f hosts -n 2 .... (hydra process manager). >> >> With PETSc 3.1 the application runs fine, both with CG and GMRES and >> correct output is generated on both nodes. >> >> With PETSc 3.2 the application runs fine with CG. >> >> But whenever I use GMRES in 3.2 I get an error (listed below) during >> KSPSolve. > > > Can you reproduce this with any of the examples? For example > > cd src/ksp/ksp/examples/tutorials > make ex2 > mpiexec -f hosts -n 2 ./ex2 -ksp_type gmres > > or, to use your matrix, run (any version that works, including 3.1) with > -ksp_view_binary and then > > cd src/ksp/ksp/examples/tutorials > make ex10 > mpiexec -f hosts -n 2 ./ex10 -f binaryoutput -ksp_type gmres > > If these work, there might be memory corruption somewhere in your code > causing this. > > > You can also run with -start_in_debugger and check what is in the "alpha" > array on each process. > This can happen if a NaN is produced as well. You can easily check by launching the debugger and printing the value. Matt -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From balay at mcs.anl.gov Sat Sep 10 08:41:35 2011 From: balay at mcs.anl.gov (Satish Balay) Date: Sat, 10 Sep 2011 08:41:35 -0500 (CDT) Subject: [petsc-users] PETSc 3.2 gmres error In-Reply-To: <4E6B4564.1070002@geology.wisc.edu> References: <4E6B4564.1070002@geology.wisc.edu> Message-ID: On Sat, 10 Sep 2011, Tabrez Ali wrote: > Hello > > I am running an application using PETSc 3.2 RC on a poor mans cluster at my > home (for testing only) which has two nodes running different versions of > Debian (they also have different versions of gcc/gfortran) but have the same > MPICH2 1.4 and PETSc 3.2 installed on them. BTW: - if this is for development/testing - why not just use 1 node [and run mpiexec -n 2] jobs on it? This is what we all do. We primarily use different nodes only for performance measurements on a cluster. [in which case - then nodes have identical os - and the same binary is usually used]. Yeah hetrogeneous usage should generally work - but we hardly test for that. Satish From stali at geology.wisc.edu Sat Sep 10 10:12:31 2011 From: stali at geology.wisc.edu (Tabrez Ali) Date: Sat, 10 Sep 2011 10:12:31 -0500 Subject: [petsc-users] PETSc 3.2 gmres error In-Reply-To: References: <4E6B4564.1070002@geology.wisc.edu> Message-ID: <4E6B7E5F.4070709@geology.wisc.edu> Jed Thanks for your reply. Yes I should have tested with a PETSc example. I do get the same error (see below) with ex2 in src/ksp/ksp/examples/tutorials with GMRES. Again CG works fine though. So I guess my code is fine and something is wrong with my setup. Satish Yes everything runs fine as 2 jobs on a single machine and that is how I usually test. I was just experimenting here. Tabrez stali at i5:~/petsc-dev/src/ksp/ksp/examples/tutorials$ mpiexec -f hosts -n 2 ./ex2 -ksp_type cg -ksp_monitor 0 KSP Residual norm 3.562148313266e+00 1 KSP Residual norm 1.215355568718e+00 2 KSP Residual norm 5.908378943191e-01 3 KSP Residual norm 2.388447476613e-01 4 KSP Residual norm 5.291449320146e-02 5 KSP Residual norm 1.227766600895e-02 6 KSP Residual norm 2.190918491891e-03 7 KSP Residual norm 3.758527933277e-04 Norm of error 0.000432115 iterations 7 stali at i5:~/petsc-dev/src/ksp/ksp/examples/tutorials$ mpiexec -f hosts -n 2 ./ex2 -ksp_type gmres -ksp_monitor 0 KSP Residual norm 3.562148313266e+00 1 KSP Residual norm 1.215348368658e+00 2 KSP Residual norm 5.599263969157e-01 3 KSP Residual norm 2.185276631601e-01 4 KSP Residual norm 5.060212909332e-02 5 KSP Residual norm 1.172638597604e-02 6 KSP Residual norm 2.158149739691e-03 7 KSP Residual norm 3.696833900173e-04 [0]PETSC ERROR: --------------------- Error Message ------------------------------------ [0]PETSC ERROR: Invalid argument! [0]PETSC ERROR: Scalar value must be same on all processes, argument # 3! [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: Petsc Development HG revision: a8941623c0b6225ff3688949b01271e9ae85a545 HG Date: Fri Sep 09 19:37:41 2011 -0500 [0]PETSC ERROR: See docs/changes/index.html for recent updates. [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. [0]PETSC ERROR: See docs/index.html for manual pages. [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: ./ex2 on a arch-linu named i5 by stali Sat Sep 10 10:27:41 2011 [0]PETSC ERROR: Libraries linked from /opt/petsc-3.2/lib [0]PETSC ERROR: Configure run at Sat Sep 10 10:02:38 2011 [0]PETSC ERROR: Configure options --prefix=/opt/petsc-3.2 --with-mpi-dir=/opt/mpich2-gcc --with-parmetis=1 --download-parmetis=1 --with-shared-libraries [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: VecMAXPY() line 1190 in src/vec/vec/interface/rvector.c [0]PETSC ERROR: BuildGmresSoln() line 345 in src/ksp/ksp/impls/gmres/gmres.c [0]PETSC ERROR: GMREScycle() line 206 in src/ksp/ksp/impls/gmres/gmres.c [0]PETSC ERROR: KSPSolve_GMRES() line 231 in src/ksp/ksp/impls/gmres/gmres.c [0]PETSC ERROR: KSPSolve() line 423 in src/ksp/ksp/interface/itfunc.c [0]PETSC ERROR: main() line 199 in src/ksp/ksp/examples/tutorials/ex2.c application called MPI_Abort(MPI_COMM_WORLD, 62) - process 0 [cli_0]: aborting job: application called MPI_Abort(MPI_COMM_WORLD, 62) - process 0 On 09/10/2011 06:38 AM, Jed Brown wrote: > On Sat, Sep 10, 2011 at 13:09, Tabrez Ali > wrote: > > Hello > > I am running an application using PETSc 3.2 RC on a poor mans > cluster at my home (for testing only) which has two nodes running > different versions of Debian (they also have different versions of > gcc/gfortran) but have the same MPICH2 1.4 and PETSc 3.2 installed > on them. > > Also they do not share the same file system but I make sure that > input file/executable paths are exactly same on both machines. > After compiling the code separately on the two nodes I launch the > parallel program from node 1 using mpiexec -f hosts -n 2 .... > (hydra process manager). > > With PETSc 3.1 the application runs fine, both with CG and GMRES > and correct output is generated on both nodes. > > With PETSc 3.2 the application runs fine with CG. > > But whenever I use GMRES in 3.2 I get an error (listed below) > during KSPSolve. > > > Can you reproduce this with any of the examples? For example > > cd src/ksp/ksp/examples/tutorials > make ex2 > mpiexec -f hosts -n 2 ./ex2 -ksp_type gmres > > or, to use your matrix, run (any version that works, including 3.1) > with -ksp_view_binary and then > > cd src/ksp/ksp/examples/tutorials > make ex10 > mpiexec -f hosts -n 2 ./ex10 -f binaryoutput -ksp_type gmres > > If these work, there might be memory corruption somewhere in your code > causing this. > > > You can also run with -start_in_debugger and check what is in the > "alpha" array on each process. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From balay at mcs.anl.gov Sat Sep 10 10:50:09 2011 From: balay at mcs.anl.gov (Satish Balay) Date: Sat, 10 Sep 2011 10:50:09 -0500 (CDT) Subject: [petsc-users] PETSc 3.2 gmres error In-Reply-To: <4E6B7E5F.4070709@geology.wisc.edu> References: <4E6B4564.1070002@geology.wisc.edu> <4E6B7E5F.4070709@geology.wisc.edu> Message-ID: You can disable these checks if you build PETSc with --with-debugging=0 The slight difference in numerical values computed by the 2 different compiler/glibc versions [or slight difference in fpu hardware in the 2 machines] could be the reason for this test to fail. Satish On Sat, 10 Sep 2011, Tabrez Ali wrote: > Jed > > Thanks for your reply. Yes I should have tested with a PETSc example. I do get > the same error (see below) with ex2 in src/ksp/ksp/examples/tutorials with > GMRES. Again CG works fine though. > > So I guess my code is fine and something is wrong with my setup. > > Satish > > Yes everything runs fine as 2 jobs on a single machine and that is how I > usually test. I was just experimenting here. > > Tabrez > > stali at i5:~/petsc-dev/src/ksp/ksp/examples/tutorials$ mpiexec -f hosts -n 2 > ./ex2 -ksp_type cg -ksp_monitor > 0 KSP Residual norm 3.562148313266e+00 > 1 KSP Residual norm 1.215355568718e+00 > 2 KSP Residual norm 5.908378943191e-01 > 3 KSP Residual norm 2.388447476613e-01 > 4 KSP Residual norm 5.291449320146e-02 > 5 KSP Residual norm 1.227766600895e-02 > 6 KSP Residual norm 2.190918491891e-03 > 7 KSP Residual norm 3.758527933277e-04 > Norm of error 0.000432115 iterations 7 > > stali at i5:~/petsc-dev/src/ksp/ksp/examples/tutorials$ mpiexec -f hosts -n 2 > ./ex2 -ksp_type gmres -ksp_monitor > 0 KSP Residual norm 3.562148313266e+00 > 1 KSP Residual norm 1.215348368658e+00 > 2 KSP Residual norm 5.599263969157e-01 > 3 KSP Residual norm 2.185276631601e-01 > 4 KSP Residual norm 5.060212909332e-02 > 5 KSP Residual norm 1.172638597604e-02 > 6 KSP Residual norm 2.158149739691e-03 > 7 KSP Residual norm 3.696833900173e-04 > [0]PETSC ERROR: --------------------- Error Message > ------------------------------------ > [0]PETSC ERROR: Invalid argument! > [0]PETSC ERROR: Scalar value must be same on all processes, argument # 3! > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: Petsc Development HG revision: > a8941623c0b6225ff3688949b01271e9ae85a545 HG Date: Fri Sep 09 19:37:41 2011 > -0500 > [0]PETSC ERROR: See docs/changes/index.html for recent updates. > [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. > [0]PETSC ERROR: See docs/index.html for manual pages. > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: ./ex2 on a arch-linu named i5 by stali Sat Sep 10 10:27:41 > 2011 > [0]PETSC ERROR: Libraries linked from /opt/petsc-3.2/lib > [0]PETSC ERROR: Configure run at Sat Sep 10 10:02:38 2011 > [0]PETSC ERROR: Configure options --prefix=/opt/petsc-3.2 > --with-mpi-dir=/opt/mpich2-gcc --with-parmetis=1 --download-parmetis=1 > --with-shared-libraries > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: VecMAXPY() line 1190 in src/vec/vec/interface/rvector.c > [0]PETSC ERROR: BuildGmresSoln() line 345 in src/ksp/ksp/impls/gmres/gmres.c > [0]PETSC ERROR: GMREScycle() line 206 in src/ksp/ksp/impls/gmres/gmres.c > [0]PETSC ERROR: KSPSolve_GMRES() line 231 in src/ksp/ksp/impls/gmres/gmres.c > [0]PETSC ERROR: KSPSolve() line 423 in src/ksp/ksp/interface/itfunc.c > [0]PETSC ERROR: main() line 199 in src/ksp/ksp/examples/tutorials/ex2.c > application called MPI_Abort(MPI_COMM_WORLD, 62) - process 0 > [cli_0]: aborting job: > application called MPI_Abort(MPI_COMM_WORLD, 62) - process 0 > > > On 09/10/2011 06:38 AM, Jed Brown wrote: > > On Sat, Sep 10, 2011 at 13:09, Tabrez Ali > > wrote: > > > > Hello > > > > I am running an application using PETSc 3.2 RC on a poor mans > > cluster at my home (for testing only) which has two nodes running > > different versions of Debian (they also have different versions of > > gcc/gfortran) but have the same MPICH2 1.4 and PETSc 3.2 installed > > on them. > > > > Also they do not share the same file system but I make sure that > > input file/executable paths are exactly same on both machines. > > After compiling the code separately on the two nodes I launch the > > parallel program from node 1 using mpiexec -f hosts -n 2 .... > > (hydra process manager). > > > > With PETSc 3.1 the application runs fine, both with CG and GMRES > > and correct output is generated on both nodes. > > > > With PETSc 3.2 the application runs fine with CG. > > > > But whenever I use GMRES in 3.2 I get an error (listed below) > > during KSPSolve. > > > > > > Can you reproduce this with any of the examples? For example > > > > cd src/ksp/ksp/examples/tutorials > > make ex2 > > mpiexec -f hosts -n 2 ./ex2 -ksp_type gmres > > > > or, to use your matrix, run (any version that works, including 3.1) with > > -ksp_view_binary and then > > > > cd src/ksp/ksp/examples/tutorials > > make ex10 > > mpiexec -f hosts -n 2 ./ex10 -f binaryoutput -ksp_type gmres > > > > If these work, there might be memory corruption somewhere in your code > > causing this. > > > > > > You can also run with -start_in_debugger and check what is in the "alpha" > > array on each process. > > > > From stali at geology.wisc.edu Sat Sep 10 12:13:47 2011 From: stali at geology.wisc.edu (Tabrez Ali) Date: Sat, 10 Sep 2011 12:13:47 -0500 Subject: [petsc-users] PETSc 3.2 gmres error In-Reply-To: References: <4E6B4564.1070002@geology.wisc.edu> <4E6B7E5F.4070709@geology.wisc.edu> Message-ID: <4E6B9ACB.7070503@geology.wisc.edu> Thanks! Everything works fine now. Tabrez On 09/10/2011 10:50 AM, Satish Balay wrote: > You can disable these checks if you build PETSc with --with-debugging=0 > > The slight difference in numerical values computed by the 2 different > compiler/glibc versions [or slight difference in fpu hardware in the 2 > machines] could be the reason for this test to fail. > > Satish > > On Sat, 10 Sep 2011, Tabrez Ali wrote: > >> Jed >> >> Thanks for your reply. Yes I should have tested with a PETSc example. I do get >> the same error (see below) with ex2 in src/ksp/ksp/examples/tutorials with >> GMRES. Again CG works fine though. >> >> So I guess my code is fine and something is wrong with my setup. >> >> Satish >> >> Yes everything runs fine as 2 jobs on a single machine and that is how I >> usually test. I was just experimenting here. >> >> Tabrez >> >> stali at i5:~/petsc-dev/src/ksp/ksp/examples/tutorials$ mpiexec -f hosts -n 2 >> ./ex2 -ksp_type cg -ksp_monitor >> 0 KSP Residual norm 3.562148313266e+00 >> 1 KSP Residual norm 1.215355568718e+00 >> 2 KSP Residual norm 5.908378943191e-01 >> 3 KSP Residual norm 2.388447476613e-01 >> 4 KSP Residual norm 5.291449320146e-02 >> 5 KSP Residual norm 1.227766600895e-02 >> 6 KSP Residual norm 2.190918491891e-03 >> 7 KSP Residual norm 3.758527933277e-04 >> Norm of error 0.000432115 iterations 7 >> >> stali at i5:~/petsc-dev/src/ksp/ksp/examples/tutorials$ mpiexec -f hosts -n 2 >> ./ex2 -ksp_type gmres -ksp_monitor >> 0 KSP Residual norm 3.562148313266e+00 >> 1 KSP Residual norm 1.215348368658e+00 >> 2 KSP Residual norm 5.599263969157e-01 >> 3 KSP Residual norm 2.185276631601e-01 >> 4 KSP Residual norm 5.060212909332e-02 >> 5 KSP Residual norm 1.172638597604e-02 >> 6 KSP Residual norm 2.158149739691e-03 >> 7 KSP Residual norm 3.696833900173e-04 >> [0]PETSC ERROR: --------------------- Error Message >> ------------------------------------ >> [0]PETSC ERROR: Invalid argument! >> [0]PETSC ERROR: Scalar value must be same on all processes, argument # 3! >> [0]PETSC ERROR: >> ------------------------------------------------------------------------ >> [0]PETSC ERROR: Petsc Development HG revision: >> a8941623c0b6225ff3688949b01271e9ae85a545 HG Date: Fri Sep 09 19:37:41 2011 >> -0500 >> [0]PETSC ERROR: See docs/changes/index.html for recent updates. >> [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. >> [0]PETSC ERROR: See docs/index.html for manual pages. >> [0]PETSC ERROR: >> ------------------------------------------------------------------------ >> [0]PETSC ERROR: ./ex2 on a arch-linu named i5 by stali Sat Sep 10 10:27:41 >> 2011 >> [0]PETSC ERROR: Libraries linked from /opt/petsc-3.2/lib >> [0]PETSC ERROR: Configure run at Sat Sep 10 10:02:38 2011 >> [0]PETSC ERROR: Configure options --prefix=/opt/petsc-3.2 >> --with-mpi-dir=/opt/mpich2-gcc --with-parmetis=1 --download-parmetis=1 >> --with-shared-libraries >> [0]PETSC ERROR: >> ------------------------------------------------------------------------ >> [0]PETSC ERROR: VecMAXPY() line 1190 in src/vec/vec/interface/rvector.c >> [0]PETSC ERROR: BuildGmresSoln() line 345 in src/ksp/ksp/impls/gmres/gmres.c >> [0]PETSC ERROR: GMREScycle() line 206 in src/ksp/ksp/impls/gmres/gmres.c >> [0]PETSC ERROR: KSPSolve_GMRES() line 231 in src/ksp/ksp/impls/gmres/gmres.c >> [0]PETSC ERROR: KSPSolve() line 423 in src/ksp/ksp/interface/itfunc.c >> [0]PETSC ERROR: main() line 199 in src/ksp/ksp/examples/tutorials/ex2.c >> application called MPI_Abort(MPI_COMM_WORLD, 62) - process 0 >> [cli_0]: aborting job: >> application called MPI_Abort(MPI_COMM_WORLD, 62) - process 0 >> >> >> On 09/10/2011 06:38 AM, Jed Brown wrote: >>> On Sat, Sep 10, 2011 at 13:09, Tabrez Ali>> > wrote: >>> >>> Hello >>> >>> I am running an application using PETSc 3.2 RC on a poor mans >>> cluster at my home (for testing only) which has two nodes running >>> different versions of Debian (they also have different versions of >>> gcc/gfortran) but have the same MPICH2 1.4 and PETSc 3.2 installed >>> on them. >>> >>> Also they do not share the same file system but I make sure that >>> input file/executable paths are exactly same on both machines. >>> After compiling the code separately on the two nodes I launch the >>> parallel program from node 1 using mpiexec -f hosts -n 2 .... >>> (hydra process manager). >>> >>> With PETSc 3.1 the application runs fine, both with CG and GMRES >>> and correct output is generated on both nodes. >>> >>> With PETSc 3.2 the application runs fine with CG. >>> >>> But whenever I use GMRES in 3.2 I get an error (listed below) >>> during KSPSolve. >>> >>> >>> Can you reproduce this with any of the examples? For example >>> >>> cd src/ksp/ksp/examples/tutorials >>> make ex2 >>> mpiexec -f hosts -n 2 ./ex2 -ksp_type gmres >>> >>> or, to use your matrix, run (any version that works, including 3.1) with >>> -ksp_view_binary and then >>> >>> cd src/ksp/ksp/examples/tutorials >>> make ex10 >>> mpiexec -f hosts -n 2 ./ex10 -f binaryoutput -ksp_type gmres >>> >>> If these work, there might be memory corruption somewhere in your code >>> causing this. >>> >>> >>> You can also run with -start_in_debugger and check what is in the "alpha" >>> array on each process. >>> >> From mmnasr at gmail.com Sat Sep 10 16:03:02 2011 From: mmnasr at gmail.com (Mohamad M. Nasr-Azadani) Date: Sat, 10 Sep 2011 14:03:02 -0700 Subject: [petsc-users] DAGetMatrix with extra nonzeros: memory preallocation In-Reply-To: References: Message-ID: Thanks Jed. Is there any example that could guide me on that? Best, Mohamad On Sat, Sep 10, 2011 at 5:35 AM, Jed Brown wrote: > On Sat, Sep 10, 2011 at 12:36, Mohamad M. Nasr-Azadani wrote: > >> I am setting up a matrix (parallel) using DA's in 3D. >> DA is built upon a width=1, STAR_STENCIL type. >> The nonzero structure of the resulting matrix is hepta-diagonal for the >> most part, however, since I am imposing boundary conditions using >> interpolation methods, for some boundary nodes, it is possible that some >> extra nonzeros are added into new locations. >> This obviously reduces the matrix setup and assembly efficiency since the >> new nonzeros are not preallocated. >> Is there any way that I could do it myself, i.e. preallocate the new >> nonzeros before matrix setup to get the best performance? >> > > Unfortunately there is not a convenient way to express this such that > DMGetMatrix() will preallocate for those entries. You can preallocate your > own matrix without calling DMGetMatrix(). It's not very hard for a stencil > width of 1 with STAR, but it is still some code to write. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jedbrown at mcs.anl.gov Sat Sep 10 16:15:12 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Sat, 10 Sep 2011 23:15:12 +0200 Subject: [petsc-users] DAGetMatrix with extra nonzeros: memory preallocation In-Reply-To: References: Message-ID: On Sat, Sep 10, 2011 at 23:03, Mohamad M. Nasr-Azadani wrote: > Is there any example that could guide me on that? Unfortunately there is not. You can read the man page for MatMPIAIJSetPreallocation() and look at the implementations in DMDA that use this API (e.g. DMGetMatrix_DA_2d_MPIAIJ in src/dm/impls/da/fdda.c -- your implementation should be somewhat simpler because you only need one stencil in the interior). It would be good to have an example of preallocating for something nontrivial, so if you write one, we can add it (and help debug if you get stuck). -------------- next part -------------- An HTML attachment was scrubbed... URL: From mmnasr at gmail.com Sun Sep 11 05:44:20 2011 From: mmnasr at gmail.com (Mohamad M. Nasr-Azadani) Date: Sun, 11 Sep 2011 03:44:20 -0700 Subject: [petsc-users] DAGetMatrix with extra nonzeros: memory preallocation In-Reply-To: References: Message-ID: Thanks Jed, I will give it a shot. I think I will need your help though, it sounds a tedious task to digg into low-level PETSc codes. :-) Best, Mohamad On Sat, Sep 10, 2011 at 2:15 PM, Jed Brown wrote: > On Sat, Sep 10, 2011 at 23:03, Mohamad M. Nasr-Azadani wrote: > >> Is there any example that could guide me on that? > > > Unfortunately there is not. You can read the man page for > MatMPIAIJSetPreallocation() and look at the implementations in DMDA that use > this API (e.g. DMGetMatrix_DA_2d_MPIAIJ in src/dm/impls/da/fdda.c -- your > implementation should be somewhat simpler because you only need one stencil > in the interior). > > It would be good to have an example of preallocating for something > nontrivial, so if you write one, we can add it (and help debug if you get > stuck). > -------------- next part -------------- An HTML attachment was scrubbed... URL: From amesga1 at tigers.lsu.edu Mon Sep 12 10:27:47 2011 From: amesga1 at tigers.lsu.edu (Ataollah Mesgarnejad) Date: Mon, 12 Sep 2011 10:27:47 -0500 Subject: [petsc-users] convergence with PCLU Message-ID: <41CD08D3-C212-4220-9BE7-C909F4349B34@tigers.lsu.edu> Dear all, I was running an small problem (~9000 unknowns) and I decided to try the direct solver with PCLU. What is puzzling me is that when it converges if I check the residual with KSPGetResidualNorm it gives me zero but when I calculate the residual myself ( ||r||:=||Ax-b||) it gives me ||r||~8.8e-6? -I know my matrix is ill-conditioned (I'm solving a very bad 4th order problem) and I get a poor convergence with gmres.- Is there a remedy for this or I should forget using direct solvers all together. Thanks for your time, Ata From hzhang at mcs.anl.gov Mon Sep 12 10:41:30 2011 From: hzhang at mcs.anl.gov (Hong Zhang) Date: Mon, 12 Sep 2011 10:41:30 -0500 Subject: [petsc-users] convergence with PCLU In-Reply-To: <41CD08D3-C212-4220-9BE7-C909F4349B34@tigers.lsu.edu> References: <41CD08D3-C212-4220-9BE7-C909F4349B34@tigers.lsu.edu> Message-ID: Ata: > > I was running an small problem (~9000 unknowns) and I decided to try the direct solver with PCLU. What is puzzling me is that when it converges if I check the residual with KSPGetResidualNorm it gives me zero KSPGetResidualNorm() gives "approximate preconditioned residual norm". Run your code with option '-ksp_monitor_true_residual' for true residual. I never get zero for preconditioned residual with LU. Also use option '-ksp_converged_reason' to check. but when I calculate the residual myself ( ||r||:=||Ax-b||) it gives me ||r||~8.8e-6? -I know my matrix is ill-conditioned (I'm solving a very bad 4th order problem) and I get a poor convergence with gmres.- Is there a remedy for this or I should forget using direct solvers all together. Direct solver should be used for ill-conditioned problem. It it does not work for your problem, then something might be wrong with the matrix/model. Hong From bsmith at mcs.anl.gov Mon Sep 12 19:47:49 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Mon, 12 Sep 2011 19:47:49 -0500 Subject: [petsc-users] Release of petsc-3.2 Message-ID: We are pleased to announce the release of PETSc version 3.2 at http://www.mcs.anl.gov/petsc The major changes and updates can be found at http://www.mcs.anl.gov/petsc/petsc-as/documentation/changes/32.html; these include * Improvements to the ODE solver interface and many new ODE integrators including IMEX methods * The addition of variational inequality solvers * A binding for MATLAB * vector and matrix operations that run on Nvidia GPUs * vector and matrix operations that run on multicore We recommend upgrading to PETSc 3.2 immediately. As always please report problems to petsc-maint at mcs.anl.gov Barry From zhenglun.wei at gmail.com Mon Sep 12 22:59:12 2011 From: zhenglun.wei at gmail.com (Alan Wei) Date: Mon, 12 Sep 2011 22:59:12 -0500 Subject: [petsc-users] Quick Question on /src/ksp/ksp/example/ex29.c Message-ID: Dear guys, I hope you're having a nice day. I'm still working on this poisson solver, and find a small problem. I found from the original code that DMMGGetx(dmmg) can get the solution vector after calculation. However, I want to have them as a format of 2-dimensional array (since I solve for a 2D Poisson Equation), like coors[j][i]. How can I do that? thanks in advance, Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: From jedbrown at mcs.anl.gov Tue Sep 13 01:10:45 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Tue, 13 Sep 2011 08:10:45 +0200 Subject: [petsc-users] Quick Question on /src/ksp/ksp/example/ex29.c In-Reply-To: References: Message-ID: On Tue, Sep 13, 2011 at 05:59, Alan Wei wrote: > I'm still working on this poisson solver, and find a small problem. > I found from the original code that DMMGGetx(dmmg) can get the > solution vector after calculation. However, I want to have them as a format > of 2-dimensional array (since I solve for a 2D Poisson Equation), like > coors[j][i]. How can I do that? > http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/DM/DMDAVecGetArray.html See also the section of the users manual regarding structured grids. If you upgrade to petsc-3.2, you can also do multigrid without DMMG. See src/ksp/ksp/examples/tutorials/ex45.c for an example. This is now the preferred interface: we would like to eventually remove DMMG because is more complicated to use and does not compose well with other solvers, e.g. TS. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mstefano at milan.westerngeco.slb.com Tue Sep 13 03:03:07 2011 From: mstefano at milan.westerngeco.slb.com (Michele De Stefano) Date: Tue, 13 Sep 2011 10:03:07 +0200 Subject: [petsc-users] [petsc-maint #86567] Question on KSPs In-Reply-To: References: <4E6E16EE.3030905@milan.westerngeco.slb.com> Message-ID: <4E6F0E3B.5040007@milan.westerngeco.slb.com> An HTML attachment was scrubbed... URL: From marco.cisternino at polito.it Tue Sep 13 05:56:35 2011 From: marco.cisternino at polito.it (Marco Cisternino) Date: Tue, 13 Sep 2011 12:56:35 +0200 Subject: [petsc-users] Neumann BC for interface elliptic problem Message-ID: <4E6F36E3.8070708@polito.it> Hi to everybody, I'm trying to solve this equation nabla(k*nabla(u))=f, with neumann homogeneous boundary conditions, where k is piecewise constant and f is a zero mean source (compatibility is respected). To do this I solve an extended by transmission conditions linear system, obtained with FD second order scheme. Everything works fine with dirichlet bc and with neumann bc too, if the interface, where k jumps, cut the computational domain. But if the interface is in the middle of the computational domain, something wrong happens where the processes overlap, with an overall loss of symmetry: the source, the interface and the bc are symmetric. I use gmres with asm. These are the lines to create the nullspace has_cnst=PETSC_TRUE call MatNullSpaceCreate(MPI_CART_COMM,has_cnst,0,PETSC_NULL_OBJECT,nsppoi,ierr) call KSPSetNullSpace(ksppoi,nsppoi,ierr) Ask me everything you need to better understand the problem. Could you help me? Thanks. Marco -- Marco Cisternino PhD Student Politecnico di Torino Email:marco.cisternino at polito.it From petsc-maint at mcs.anl.gov Tue Sep 13 06:35:11 2011 From: petsc-maint at mcs.anl.gov (Matthew Knepley) Date: Tue, 13 Sep 2011 06:35:11 -0500 Subject: [petsc-users] [petsc-maint #86567] Question on KSPs In-Reply-To: <4E6F0E3B.5040007@milan.westerngeco.slb.com> References: <4E6E16EE.3030905@milan.westerngeco.slb.com> <4E6F0E3B.5040007@milan.westerngeco.slb.com> Message-ID: On Tue, Sep 13, 2011 at 3:03 AM, Michele De Stefano < mstefano at milan.westerngeco.slb.com> wrote: > ** > Jed and others, > > we would like to explore more in detail the possibility to have a > completely matrix-free preconditioner. > May you explain which are the "few methods" that we should implement and > that you was talking of ? > > Second question: are there algorithms for creating matrix-free > preconditioners ? > If yes, may you suggest at least one, please ? > There is really no conceptual difference between Krylov methods and "preconditioners" in that they are both approximate linear solvers. The distinction was made mostly because Krylov solvers only need the action of the matrix (or its transpose) and thus are "matrix-free". Therefore, "matrix-free preconditioner" basically means Krylov method (where I include Chebychev, which is a KSP in PETSc). So what many scalable solvers do is combine KSPs in a nice way, like a multigrid iteration. There may be things you can do for your particular equations, like analytic simplifications, but these would not be in PETSc since they are not generic. Thanks, Matt > Thank you in advance. > Best regards, > > Michele > > Jed Brown wrote: > > On Mon, Sep 12, 2011 at 16:28, Michele De Stefano < > mstefano at milan.westerngeco.slb.com> wrote: > >> KSPSetOperators is able to accept a shell matrix for the "Amat" and the >> manual >> says that the "Pmat" is usually the same as the "Amat". Does this really >> work >> also when "Amat" is a shell matrix ? >> >> I mean, in the multi-process case, the default preconditioning method is >> block >> Jacobi ... but does this work also when "Pmat" is a shell matrix ? >> > > Only if a few methods are implemented for the shell matrix. This is usually > not practical. It would be more common to assemble some approximation Pmat > (e.g. in AIJ format) and use that for preconditioning. To do everything > matrix-free, you would normally have to put some effort into writing your > own custom preconditioner. > > > -- > Michele De Stefano > Senior Geophysicist - REMS > Integrated EM Center of Excellence > > WesternGeco GeoSolutions, > Via Clericetti 42/A, > 20133, Milan - Italy > > +39 02 266279232 (direct) > +39 02 266279279 (fax) > > mstefano at slb.com > > Schlumberger Private > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Tue Sep 13 06:39:31 2011 From: knepley at gmail.com (Matthew Knepley) Date: Tue, 13 Sep 2011 06:39:31 -0500 Subject: [petsc-users] Neumann BC for interface elliptic problem In-Reply-To: <4E6F36E3.8070708@polito.it> References: <4E6F36E3.8070708@polito.it> Message-ID: On Tue, Sep 13, 2011 at 5:56 AM, Marco Cisternino < marco.cisternino at polito.it> wrote: > Hi to everybody, > I'm trying to solve this equation > nabla(k*nabla(u))=f, > with neumann homogeneous boundary conditions, where k is piecewise > constant and f is a zero mean source (compatibility is respected). > To do this I solve an extended by transmission conditions linear system, > obtained with FD second order scheme. > Everything works fine with dirichlet bc and with neumann bc too, if the > interface, where k jumps, cut the computational domain. > But if the interface is in the middle of the computational domain, > something wrong happens where the processes overlap, with an overall loss of > symmetry: the source, the interface and the bc are symmetric. > I use gmres with asm. These are the lines to create the nullspace > > has_cnst=PETSC_TRUE > call MatNullSpaceCreate(MPI_CART_**COMM,has_cnst,0,PETSC_NULL_** > OBJECT,nsppoi,ierr) > call KSPSetNullSpace(ksppoi,nsppoi,**ierr) > > Ask me everything you need to better understand the problem. > Could you help me? > If you have matrices, or submatrices, which must be symmetric, you can check this using http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/Mat/MatIsSymmetric.html Thanks, Matt > Thanks. > > Marco > > -- > Marco Cisternino > PhD Student > Politecnico di Torino > Email:marco.cisternino at polito.**it > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From caner at candan.fr Tue Sep 13 05:58:28 2011 From: caner at candan.fr (Caner Candan) Date: Tue, 13 Sep 2011 12:58:28 +0200 Subject: [petsc-users] Non-official C++ binding for PETSc Message-ID: Hi PETSc users, Let's introduce a non-official C++ version of PETSc developed during my MSc's degree, this work does not include all the components of the library, however you can already find some operators like KSPSolver, MultiplyMatVec and Dot. The framework uses a functors-based design requiring all components to define an operator function. There are also a Matrix and Vector data structure classes. The sources are available as free software under the terms of the GNU GPL license at https://github.com/canercandan/petsc-cxx Have a look also into the "test" folder in order to see how it works. (https://github.com/canercandan/petsc-cxx/tree/master/test) To build the project, kindly read the README file at https://github.com/canercandan/petsc-cxx/blob/master/README BR, Caner Candan From john.fettig at gmail.com Tue Sep 13 08:55:37 2011 From: john.fettig at gmail.com (John Fettig) Date: Tue, 13 Sep 2011 09:55:37 -0400 Subject: [petsc-users] Building PETSc with cmake Message-ID: What are the requirements for building PETSc with cmake? I have cmake version 2.8.1 installed, and configure finds it, but it does nothing with it. Is it because my python version is too old? This is on CentOS 5, I get these messages about cmake in the configure.log (with 3.2-p0): Checking for program /usr/local/bin/cmake...found Defined make macro "CMAKE" to "/usr/local/bin/cmake" ... Skipping cmakegen due to old python version: (2, 4, 3, 'final', 0) Skipping cmakeboot due to old python version: (2, 4, 3, 'final', 0) At the end of the configure, I'm not offered a way to build with cmake. I presume that cmake is the only way to do a parallel build? John From knepley at gmail.com Tue Sep 13 08:59:27 2011 From: knepley at gmail.com (Matthew Knepley) Date: Tue, 13 Sep 2011 08:59:27 -0500 Subject: [petsc-users] Building PETSc with cmake In-Reply-To: References: Message-ID: On Tue, Sep 13, 2011 at 8:55 AM, John Fettig wrote: > What are the requirements for building PETSc with cmake? I have cmake > version 2.8.1 installed, and configure finds it, but it does nothing > with it. Is it because my python version is too old? This is on > CentOS 5, I get these messages about cmake in the configure.log (with > 3.2-p0): > > Checking for program /usr/local/bin/cmake...found > Defined make macro "CMAKE" to "/usr/local/bin/cmake" > > ... > > Skipping cmakegen due to old python version: (2, 4, 3, 'final', 0) > Skipping cmakeboot due to old python version: (2, 4, 3, 'final', 0) > > At the end of the configure, I'm not offered a way to build with > cmake. I presume that cmake is the only way to do a parallel build? Yes, and it is because your Python version is too old. I believe it needs 2.6. I would install 2.7 if you are upgrading. Matt > > John -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From balay at mcs.anl.gov Tue Sep 13 09:01:00 2011 From: balay at mcs.anl.gov (Satish Balay) Date: Tue, 13 Sep 2011 09:01:00 -0500 (CDT) Subject: [petsc-users] Building PETSc with cmake In-Reply-To: References: Message-ID: On Tue, 13 Sep 2011, John Fettig wrote: > What are the requirements for building PETSc with cmake? I have cmake > version 2.8.1 installed, and configure finds it, but it does nothing > with it. Is it because my python version is too old? This is on > CentOS 5, I get these messages about cmake in the configure.log (with > 3.2-p0): > > Checking for program /usr/local/bin/cmake...found > Defined make macro "CMAKE" to "/usr/local/bin/cmake" > > ... > > Skipping cmakegen due to old python version: (2, 4, 3, 'final', 0) > Skipping cmakeboot due to old python version: (2, 4, 3, 'final', 0) yes - cmake part of configure requires python-2.5. Perhaps we should look at this code and see if it can be made to work with 2.3+ [current configure requirement] If configure determines that cmake build won't work [either due to the python version issue - or some other configue issue - for eg: windows compilers - it won't offer this build option at tthe end. > At the end of the configure, I'm not offered a way to build with > cmake. I presume that cmake is the only way to do a parallel build? Yes - currently cmake build is the only way to do parallel builds. . Also builder.py [if it works for your config] could be faster than the 'make all-legacy' build. Satish From john.fettig at gmail.com Tue Sep 13 09:07:01 2011 From: john.fettig at gmail.com (John Fettig) Date: Tue, 13 Sep 2011 10:07:01 -0400 Subject: [petsc-users] Building PETSc with cmake In-Reply-To: References: Message-ID: On Tue, Sep 13, 2011 at 9:59 AM, Matthew Knepley wrote: > On Tue, Sep 13, 2011 at 8:55 AM, John Fettig wrote: >> >> What are the requirements for building PETSc with cmake? ?I have cmake >> version 2.8.1 installed, and configure finds it, but it does nothing >> with it. ?Is it because my python version is too old? ?This is on >> CentOS 5, I get these messages about cmake in the configure.log (with >> 3.2-p0): >> >> Checking for program /usr/local/bin/cmake...found >> ? ? ? ? ? ?Defined make macro "CMAKE" to "/usr/local/bin/cmake" >> >> ... >> >> ? ? ?Skipping cmakegen due to old python version: (2, 4, 3, 'final', 0) >> ? ? ?Skipping cmakeboot due to old python version: (2, 4, 3, 'final', 0) >> >> At the end of the configure, I'm not offered a way to build with >> cmake. ?I presume that cmake is the only way to do a parallel build? > > Yes, and it is because your Python version is too old. I believe it needs > 2.6. I would > install 2.7 if you are upgrading. Is there any way to tell configure which python to use? I have python 2.6 installed, it's just not what runs when you type `python`. I don't think I can change the default python in CentOS 5 without breaking a lot of other things. John From balay at mcs.anl.gov Tue Sep 13 09:09:16 2011 From: balay at mcs.anl.gov (Satish Balay) Date: Tue, 13 Sep 2011 09:09:16 -0500 (CDT) Subject: [petsc-users] Building PETSc with cmake In-Reply-To: References: Message-ID: On Tue, 13 Sep 2011, John Fettig wrote: > On Tue, Sep 13, 2011 at 9:59 AM, Matthew Knepley wrote: > > On Tue, Sep 13, 2011 at 8:55 AM, John Fettig wrote: > >> > >> What are the requirements for building PETSc with cmake? ?I have cmake > >> version 2.8.1 installed, and configure finds it, but it does nothing > >> with it. ?Is it because my python version is too old? ?This is on > >> CentOS 5, I get these messages about cmake in the configure.log (with > >> 3.2-p0): > >> > >> Checking for program /usr/local/bin/cmake...found > >> ? ? ? ? ? ?Defined make macro "CMAKE" to "/usr/local/bin/cmake" > >> > >> ... > >> > >> ? ? ?Skipping cmakegen due to old python version: (2, 4, 3, 'final', 0) > >> ? ? ?Skipping cmakeboot due to old python version: (2, 4, 3, 'final', 0) > >> > >> At the end of the configure, I'm not offered a way to build with > >> cmake. ?I presume that cmake is the only way to do a parallel build? > > > > Yes, and it is because your Python version is too old. I believe it needs > > 2.6. I would > > install 2.7 if you are upgrading. > > Is there any way to tell configure which python to use? I have python > 2.6 installed, it's just not what runs when you type `python`. I > don't think I can change the default python in CentOS 5 without > breaking a lot of other things. /path/to/python2.6 configure [configure_options] Satish From zhenglun.wei at gmail.com Tue Sep 13 09:26:29 2011 From: zhenglun.wei at gmail.com (Alan Wei) Date: Tue, 13 Sep 2011 09:26:29 -0500 Subject: [petsc-users] Quick Question on /src/ksp/ksp/example/ex29.c In-Reply-To: References: Message-ID: Thanks, Jed. Let me finish this part first and I will upgrade to version 3.2 to try new things. best, Alan On Tue, Sep 13, 2011 at 1:10 AM, Jed Brown wrote: > On Tue, Sep 13, 2011 at 05:59, Alan Wei wrote: > >> I'm still working on this poisson solver, and find a small problem. >> I found from the original code that DMMGGetx(dmmg) can get the >> solution vector after calculation. However, I want to have them as a format >> of 2-dimensional array (since I solve for a 2D Poisson Equation), like >> coors[j][i]. How can I do that? >> > > > http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/DM/DMDAVecGetArray.html > > See also the section of the users manual regarding structured grids. > > If you upgrade to petsc-3.2, you can also do multigrid without DMMG. See > src/ksp/ksp/examples/tutorials/ex45.c for an example. This is now the > preferred interface: we would like to eventually remove DMMG because is more > complicated to use and does not compose well with other solvers, e.g. TS. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From john.fettig at gmail.com Tue Sep 13 09:30:55 2011 From: john.fettig at gmail.com (John Fettig) Date: Tue, 13 Sep 2011 10:30:55 -0400 Subject: [petsc-users] Building PETSc with cmake In-Reply-To: References: Message-ID: On Tue, Sep 13, 2011 at 10:09 AM, Satish Balay wrote: > On Tue, 13 Sep 2011, John Fettig wrote: >> Is there any way to tell configure which python to use? ?I have python >> 2.6 installed, it's just not what runs when you type `python`. ?I >> don't think I can change the default python in CentOS 5 without >> breaking a lot of other things. > > /path/to/python2.6 configure ?[configure_options] This gets configure to try cmake, but it still doesn't use it. It doesn't set "PETSC_BUILD_USING_CMAKE" to "1" as I see the nightly builds doing, and again doesn't offer for me to use cmake at the end of configure. Is this only available in petsc-dev? Thanks, John Invoking: ['/usr/local/bin/cmake', '/home/jfe/local/centos/petsc-3.2-p0', '-DCMAKE_C_COMPILER:FILEPATH=icc', '-DCMAKE_C_FLAGS:STRING= -fPIC -wd1572 -Qoption,cpp,--extended_float_type -g -debug inline_debug_info', '-DCMAKE_Fortran_COMPILER:FILEPATH=ifort', '-DCMAKE_Fortran_FLAGS:STRING= -fPIC -g -debug inline_debug_info'] sh: ['/usr/local/bin/cmake', '/home/jfe/local/centos/petsc-3.2-p0', '-DCMAKE_C_COMPILER:FILEPATH=icc', '-DCMAKE_C_FLAGS:STRING= -fPIC -wd1572 -Qoption,cpp,--extended_float_type -g -debug inline_debug_info', '-DCMAKE_Fortran_COMPILER:FILEPATH=ifort', '-DCMAKE_Fortran_FLAGS:STRING= -fPIC -g -debug inline_debug_info'] Executing: ['/usr/local/bin/cmake', '/home/jfe/local/centos/petsc-3.2-p0', '-DCMAKE_C_COMPILER:FILEPATH=icc', '-DCMAKE_C_FLAGS:STRING= -fPIC -wd1572 -Qoption,cpp,--extended_float_type -g -debug inline_debug_info', '-DCMAKE_Fortran_COMPILER:FILEPATH=ifort', '-DCMAKE_Fortran_FLAGS:STRING= -fPIC -g -debug inline_debug_info'] sh: -- The C compiler identification is Intel -- Check for working C compiler: /opt/intel/Compiler/11.1/072/bin/intel64/icc -- Check for working C compiler: /opt/intel/Compiler/11.1/072/bin/intel64/icc -- works -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- The Fortran compiler identification is Intel -- Check for working Fortran compiler: /opt/intel/Compiler/11.1/072/bin/intel64/ifort -- Check for working Fortran compiler: /opt/intel/Compiler/11.1/072/bin/intel64/ifort -- works -- Detecting Fortran compiler ABI info -- Detecting Fortran compiler ABI info - done -- Checking whether /opt/intel/Compiler/11.1/072/bin/intel64/ifort supports Fortran 90 -- Checking whether /opt/intel/Compiler/11.1/072/bin/intel64/ifort supports Fortran 90 -- yes -- Configuring done -- Generating done -- Build files have been written to: /home/jfe/local/centos/petsc-3.2-p0/intel-debug ================================================================================ ================================================================================ From balay at mcs.anl.gov Tue Sep 13 09:48:28 2011 From: balay at mcs.anl.gov (Satish Balay) Date: Tue, 13 Sep 2011 09:48:28 -0500 (CDT) Subject: [petsc-users] Building PETSc with cmake In-Reply-To: References: Message-ID: It should work in petsc-32. Can you send configure.log to petsc-maint? Satish On Tue, 13 Sep 2011, John Fettig wrote: > On Tue, Sep 13, 2011 at 10:09 AM, Satish Balay wrote: > > On Tue, 13 Sep 2011, John Fettig wrote: > >> Is there any way to tell configure which python to use? ?I have python > >> 2.6 installed, it's just not what runs when you type `python`. ?I > >> don't think I can change the default python in CentOS 5 without > >> breaking a lot of other things. > > > > /path/to/python2.6 configure ?[configure_options] > > > This gets configure to try cmake, but it still doesn't use it. It > doesn't set "PETSC_BUILD_USING_CMAKE" to "1" as I see the nightly > builds doing, and again doesn't offer for me to use cmake at the end > of configure. Is this only available in petsc-dev? > > Thanks, > John > > Invoking: ['/usr/local/bin/cmake', > '/home/jfe/local/centos/petsc-3.2-p0', > '-DCMAKE_C_COMPILER:FILEPATH=icc', '-DCMAKE_C_FLAGS:STRING= -fPIC > -wd1572 -Qoption,cpp,--extended_float_type -g -debug > inline_debug_info', '-DCMAKE_Fortran_COMPILER:FILEPATH=ifort', > '-DCMAKE_Fortran_FLAGS:STRING= -fPIC -g -debug inline_debug_info'] > sh: ['/usr/local/bin/cmake', '/home/jfe/local/centos/petsc-3.2-p0', > '-DCMAKE_C_COMPILER:FILEPATH=icc', '-DCMAKE_C_FLAGS:STRING= -fPIC > -wd1572 -Qoption,cpp,--extended_float_type -g -debug > inline_debug_info', '-DCMAKE_Fortran_COMPILER:FILEPATH=ifort', > '-DCMAKE_Fortran_FLAGS:STRING= -fPIC -g -debug inline_debug_info'] > Executing: ['/usr/local/bin/cmake', > '/home/jfe/local/centos/petsc-3.2-p0', > '-DCMAKE_C_COMPILER:FILEPATH=icc', '-DCMAKE_C_FLAGS:STRING= -fPIC > -wd1572 -Qoption,cpp,--extended_float_type -g -debug > inline_debug_info', '-DCMAKE_Fortran_COMPILER:FILEPATH=ifort', > '-DCMAKE_Fortran_FLAGS:STRING= -fPIC -g -debug inline_debug_info'] > sh: -- The C compiler identification is Intel > -- Check for working C compiler: /opt/intel/Compiler/11.1/072/bin/intel64/icc > -- Check for working C compiler: > /opt/intel/Compiler/11.1/072/bin/intel64/icc -- works > -- Detecting C compiler ABI info > -- Detecting C compiler ABI info - done > -- The Fortran compiler identification is Intel > -- Check for working Fortran compiler: > /opt/intel/Compiler/11.1/072/bin/intel64/ifort > -- Check for working Fortran compiler: > /opt/intel/Compiler/11.1/072/bin/intel64/ifort -- works > -- Detecting Fortran compiler ABI info > -- Detecting Fortran compiler ABI info - done > -- Checking whether /opt/intel/Compiler/11.1/072/bin/intel64/ifort > supports Fortran 90 > -- Checking whether /opt/intel/Compiler/11.1/072/bin/intel64/ifort > supports Fortran 90 -- yes > -- Configuring done > -- Generating done > -- Build files have been written to: > /home/jfe/local/centos/petsc-3.2-p0/intel-debug > > ================================================================================ > ================================================================================ > From marco.cisternino at polito.it Tue Sep 13 10:58:55 2011 From: marco.cisternino at polito.it (Marco Cisternino) Date: Tue, 13 Sep 2011 17:58:55 +0200 Subject: [petsc-users] Neumann BC for interface elliptic problem In-Reply-To: References: Message-ID: <4E6F7DBF.5090602@polito.it> Thanks, Matthew, for the quick reply. My matrices are not symmetric. The differential problem is symmetric, but matrices are not. And not only for the presence of transmission conditions at the interface but also because I explicitly put the discretization of the forward or backward second order Neumann boundary conditions as rows in my matrix. Then even if I try to solve a problem without interfaces my matrix is not symmetric. And even in this case I got problems along the processes boundaries and about the loss of symmetry of the solution. But the matrix structure is exactly the same when I solve the problem with the interface cutting the domain (always with the same implementation of Neumann boundaries conditions) with no problem. I can eliminate the bc rows nesting them in the discretization of the laplacian, but the interface will always give me an asymmetric matrix. But most of all I can't understand why for one kind of interface everything works and for the other not. Thanks again. Marco From bsmith at mcs.anl.gov Tue Sep 13 12:42:01 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Tue, 13 Sep 2011 12:42:01 -0500 Subject: [petsc-users] Neumann BC for interface elliptic problem In-Reply-To: <4E6F7DBF.5090602@polito.it> References: <4E6F7DBF.5090602@polito.it> Message-ID: On Sep 13, 2011, at 10:58 AM, Marco Cisternino wrote: > Thanks, Matthew, for the quick reply. > My matrices are not symmetric. The differential problem is symmetric, but matrices are not. > And not only for the presence of transmission conditions at the interface but also because I explicitly put the discretization of the forward or backward second order Neumann boundary conditions as rows in my matrix. Then even if I try to solve a problem without interfaces my matrix is not symmetric. And even in this case I got problems along the processes boundaries and about the loss of symmetry of the solution. > But the matrix structure is exactly the same when I solve the problem with the interface cutting the domain (always with the same implementation of Neumann boundaries conditions) with no problem. > I can eliminate the bc rows nesting them in the discretization of the laplacian, but the interface will always give me an asymmetric matrix. > But most of all I can't understand why for one kind of interface everything works What is "everything" that works? Do you mean the iterative solver converges in one case but not the other? Do you mean it always converges but the answer is wrong in one case but not the other? Barry > and for the other not. > Thanks again. > > Marco > From jack.poulson at gmail.com Tue Sep 13 13:39:03 2011 From: jack.poulson at gmail.com (Jack Poulson) Date: Tue, 13 Sep 2011 13:39:03 -0500 Subject: [petsc-users] Non-official C++ binding for PETSc In-Reply-To: References: Message-ID: Hi Caner, Is the templating more than aesthetic? It would seem to me from, for instance, https://github.com/canercandan/petsc-cxx/blob/master/src/petsc_cxx/Vector.h that only a single datatype is supported for your Vec and Matrix wrappers (PetscScalar). So, for example, in this test https://github.com/canercandan/petsc-cxx/blob/master/test/t-create_vector_cxx.cpp could the Vector's x and y instead be built on top of int rather than Scalar? If not, why bother with the template parameter? It would seem that the main advantage of these wrappers is the syntactic sugar for printing. Jack On Tue, Sep 13, 2011 at 5:58 AM, Caner Candan wrote: > Hi PETSc users, > > Let's introduce a non-official C++ version of PETSc developed during > my MSc's degree, this work does not include all the components of the > library, however you can already find some operators like KSPSolver, > MultiplyMatVec and Dot. The framework uses a functors-based design > requiring all components to define an operator function. There are > also a Matrix and Vector data structure classes. > > The sources are available as free software under the terms of the GNU > GPL license at https://github.com/canercandan/petsc-cxx > > Have a look also into the "test" folder in order to see how it works. > (https://github.com/canercandan/petsc-cxx/tree/master/test) > > To build the project, kindly read the README file at > https://github.com/canercandan/petsc-cxx/blob/master/README > > BR, > Caner Candan > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jedbrown at mcs.anl.gov Tue Sep 13 14:52:00 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Tue, 13 Sep 2011 21:52:00 +0200 Subject: [petsc-users] [petsc-maint #86567] Question on KSPs In-Reply-To: <4E6F0E3B.5040007@milan.westerngeco.slb.com> References: <4E6E16EE.3030905@milan.westerngeco.slb.com> <4E6F0E3B.5040007@milan.westerngeco.slb.com> Message-ID: On Tue, Sep 13, 2011 at 10:03, Michele De Stefano < mstefano at milan.westerngeco.slb.com> wrote: > we would like to explore more in detail the possibility to have a > completely matrix-free preconditioner. > May you explain which are the "few methods" that we should implement and > that you was talking of ? > Matrix-free multigrid, approximate the problem with a simpler model for which a fast solver (e.g. FMM or FFT-based) is available, the nested iteration that Matt mentioned, low rank + identity approximations of the inverse (e.g. built as a byproduct of an outer iteration like BFGS). None of these are truly generic. I absolutely do not recommend trying to build matrix-free preconditioners to avoid writing code to assemble a matrix. Indeed, I think it's well worth having the code to assemble a matrix even if you usually run without it because it enables you to try many algorithms without extra effort. Having the matrix available can inform the design of matrix-free methods. The performance asymptotics of matrix-free methods are better for some problems. In many cases, it makes sense to apply the Jacobian using some matrix-free method and assemble a less expensive matrix for preconditioning. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rongliang.chan at gmail.com Tue Sep 13 16:26:04 2011 From: rongliang.chan at gmail.com (Rongliang Chen) Date: Tue, 13 Sep 2011 15:26:04 -0600 Subject: [petsc-users] Problem with PETSc-3.2 configuration Message-ID: Hi, I got the following error message when I try to configure PETSc-3.2 with MUMPS. What's maybe the problem? Best, Rongliang ******************************************************************************* UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log for details): ------------------------------------------------------------------------------- Error running make on MUMPS: Could not execute "cd /home/ronglian/soft/petsc-3.2-p1/externalpackages/MUMPS_4.10.0 && make alllib": make ARITH=s mumps_lib make[1]: Entering directory `/home/ronglian/soft/petsc-3.2-p1/externalpackages/MUMPS_4.10.0' if [ "./PORD/lib/" != "" ] ; then \ cd ./PORD/lib/; \ make CC="mpicc" CFLAGS="-Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g3 " AR="/usr/bin/ar cr " RANLIB="/usr/bin/ranlib" OUTC=-o LIBEXT=.a; \ fi; make[2]: Entering directory `/home/ronglian/soft/petsc-3.2-p1/externalpackages/MUMPS_4.10.0/PORD/lib' mpicc -I../include -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g3 -c graph.c -o graph.o mpicc -I../include -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g3 -c gbipart.c -o gbipart.o mpicc -I../include -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g3 -c gbisect.c -o gbisect.o mpicc -I../include -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g3 -c ddcreate.c -o ddcreate.o mpicc -I../include -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g3 -c ddbisect.c -o ddbisect.o mpicc -I../include -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g3 -c nestdiss.c -o nestdiss.o mpicc -I../include -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g3 -c multisector.c -o multisector.o mpicc -I../include -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g3 -c gelim.c -o gelim.o mpicc -I../include -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g3 -c bucket.c -o bucket.o mpicc -I../include -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g3 -c tree.c -o tree.o mpicc -I../include -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g3 -c symbfac.c -o symbfac.o mpicc -I../include -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g3 -c interface.c -o interface.o mpicc -I../include -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g3 -c sort.c -o sort.o mpicc -I../include -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g3 -c minpriority.c -o minpriority.o /usr/bin/ar cr libpord.a graph.o gbipart.o gbisect.o ddcreate.o ddbisect.o nestdiss.o multisector.o gelim.o bucket.o tree.o symbfac.o interface.o sort.o minpriority.o /usr/bin/ranlib libpord.a make[2]: Leaving directory `/home/ronglian/soft/petsc-3.2-p1/externalpackages/MUMPS_4.10.0/PORD/lib' if [ "./PORD/lib/" != "" ] ; then \ cp ./PORD/lib//libpord.a lib/libpord.a; \ fi; (cd src ; make s) make[2]: Entering directory `/home/ronglian/soft/petsc-3.2-p1/externalpackages/MUMPS_4.10.0/src' make ARITH=s mumps_lib make[3]: Entering directory `/home/ronglian/soft/petsc-3.2-p1/externalpackages/MUMPS_4.10.0/src' mpif90 -Wno-unused-variable -g -Dmetis -Dparmetis -Dpord -I. -I../include -c mumps_part9.F -o mumps_part9.o mpicc -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g3 -I../include -DAdd_ -I/home/ronglian/soft/petsc-3.2-p1/Janus-debug/include -I../PORD/include/ -Dmetis -Dparmetis -Dpord -c mumps_common.c -o mumps_common.o mpif90 -Wno-unused-variable -g -Dmetis -Dparmetis -Dpord -I. -I../include -c mumps_ooc_common.F -o mumps_ooc_common.o mpicc -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g3 -I../include -DAdd_ -I/home/ronglian/soft/petsc-3.2-p1/Janus-debug/include -I../PORD/include/ -Dmetis -Dparmetis -Dpord -c mumps_orderings.c -o mumps_orderings.o mpicc -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g3 -I../include -DAdd_ -I/home/ronglian/soft/petsc-3.2-p1/Janus-debug/include -I../PORD/include/ -Dmetis -Dparmetis -Dpord -c mumps_size.c -o mumps_size.o mpicc -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g3 -I../include -DAdd_ -I/home/ronglian/soft/petsc-3.2-p1/Janus-debug/include -I../PORD/include/ -Dmetis -Dparmetis -Dpord -c mumps_io.c -o mumps_io.o mpicc -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g3 -I../include -DAdd_ -I/home/ronglian/soft/petsc-3.2-p1/Janus-debug/include -I../PORD/include/ -Dmetis -Dparmetis -Dpord -c mumps_io_basic.c -o mumps_io_basic.o mpicc -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g3 -I../include -DAdd_ -I/home/ronglian/soft/petsc-3.2-p1/Janus-debug/include -I../PORD/include/ -Dmetis -Dparmetis -Dpord -c mumps_io_thread.c -o mumps_io_thread.o mpicc -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g3 -I../include -DAdd_ -I/home/ronglian/soft/petsc-3.2-p1/Janus-debug/include -I../PORD/include/ -Dmetis -Dparmetis -Dpord -c mumps_io_err.c -o mumps_io_err.o mpif90 -Wno-unused-variable -g -Dmetis -Dparmetis -Dpord -I. -I../include -c mumps_static_mapping.F -o mumps_static_mapping.o make[3]: Leaving directory `/home/ronglian/soft/petsc-3.2-p1/externalpackages/MUMPS_4.10.0/src' make[2]: Leaving directory `/home/ronglian/soft/petsc-3.2-p1/externalpackages/MUMPS_4.10.0/src' make[1]: Leaving directory `/home/ronglian/soft/petsc-3.2-p1/externalpackages/MUMPS_4.10.0' mumps_io_basic.c: In function ?mumps_init_file_name?: mumps_io_basic.c:564: warning: assignment discards qualifiers from pointer target type mumps_static_mapping.F: In function ?mumps_369?: mumps_static_mapping.F:333: internal compiler error: in modified_type_die, at dwarf2out.c:8495 Please submit a full bug report, with preprocessed source if appropriate. See for instructions. make[3]: *** [mumps_static_mapping.o] Error 1 make[2]: *** [s] Error 2 make[1]: *** [mumps_lib] Error 2 make: *** [s] Error 2 ******************************************************************************* -------------- next part -------------- An HTML attachment was scrubbed... URL: From jedbrown at mcs.anl.gov Tue Sep 13 16:29:56 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Tue, 13 Sep 2011 23:29:56 +0200 Subject: [petsc-users] Problem with PETSc-3.2 configuration In-Reply-To: References: Message-ID: On Tue, Sep 13, 2011 at 23:26, Rongliang Chen wrote: > mumps_static_mapping.F: In function ?mumps_369?: > mumps_static_mapping.F:333: internal compiler error: in modified_type_die, > at dwarf2out.c:8495 > Please submit a full bug report, > with preprocessed source if appropriate. > See for instructions. > Um, internal compiler error. File a bug report with your vendor and/or upgrade your compilers? -------------- next part -------------- An HTML attachment was scrubbed... URL: From balay at mcs.anl.gov Tue Sep 13 16:43:04 2011 From: balay at mcs.anl.gov (Satish Balay) Date: Tue, 13 Sep 2011 16:43:04 -0500 (CDT) Subject: [petsc-users] Problem with PETSc-3.2 configuration In-Reply-To: References: Message-ID: On Tue, 13 Sep 2011, Jed Brown wrote: > On Tue, Sep 13, 2011 at 23:26, Rongliang Chen wrote: > > > mumps_static_mapping.F: In function ?mumps_369?: > > mumps_static_mapping.F:333: internal compiler error: in modified_type_die, > > at dwarf2out.c:8495 > > Please submit a full bug report, > > with preprocessed source if appropriate. > > See for instructions. > > > > Um, internal compiler error. File a bug report with your vendor and/or > upgrade your compilers? What version of compilers are you using? mpicc -show mpif90 -show mpicc --version mpif90 --version Satish From rongliang.chan at gmail.com Tue Sep 13 17:34:40 2011 From: rongliang.chan at gmail.com (Rongliang Chen) Date: Tue, 13 Sep 2011 16:34:40 -0600 Subject: [petsc-users] Problem with PETSc-3.2 configuration Message-ID: Hi Jed and Satish, Thank you for your reply. Following is the information of the compilers. mpicc -show gcc -I/curc/tools/free/redhat_5_x86_64/openmpi-1.4.3_ib/include -pthread -L/curc/tools/free/redhat_5_x86_64/openmpi-1.4.3_ib/lib -lmpi -lopen-rte -lopen-pal -ldl -Wl,--export-dynamic -lnsl -lutil -lm -ldl mpif90 -show gfortran -I/curc/tools/free/redhat_5_x86_64/openmpi-1.4.3_ib/include -pthread -I/curc/tools/free/redhat_5_x86_64/openmpi-1.4.3_ib/lib -L/curc/tools/free/redhat_5_x86_64/openmpi-1.4.3_ib/lib -lmpi_f90 -lmpi_f77 -lmpi -lopen-rte -lopen-pal -ldl -Wl,--export-dynamic -lnsl -lutil -lm -ldl mpicc --version gcc (GCC) 4.1.2 20080704 (Red Hat 4.1.2-50) Copyright (C) 2006 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. mpif90 --version GNU Fortran (GCC) 4.1.2 20080704 (Red Hat 4.1.2-50) Copyright (C) 2007 Free Software Foundation, Inc. GNU Fortran comes with NO WARRANTY, to the extent permitted by law. You may redistribute copies of GNU Fortran under the terms of the GNU General Public License. For more information about these matters, see the file named COPYING Best, Rongliang ------------------------------ > > Message: 5 > Date: Tue, 13 Sep 2011 23:29:56 +0200 > From: Jed Brown > Subject: Re: [petsc-users] Problem with PETSc-3.2 configuration > To: PETSc users list > Message-ID: > > > Content-Type: text/plain; charset="utf-8" > > On Tue, Sep 13, 2011 at 23:26, Rongliang Chen >wrote: > > > mumps_static_mapping.F: In function ?mumps_369?: > > mumps_static_mapping.F:333: internal compiler error: in > modified_type_die, > > at dwarf2out.c:8495 > > Please submit a full bug report, > > with preprocessed source if appropriate. > > See for instructions. > > > > Um, internal compiler error. File a bug report with your vendor and/or > upgrade your compilers? > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: < > http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20110913/eced13ed/attachment-0001.htm > > > > ------------------------------ > > Message: 6 > Date: Tue, 13 Sep 2011 16:43:04 -0500 (CDT) > From: Satish Balay > Subject: Re: [petsc-users] Problem with PETSc-3.2 configuration > To: PETSc users list > Message-ID: > Content-Type: text/plain; charset="iso-8859-7" > > On Tue, 13 Sep 2011, Jed Brown wrote: > > > On Tue, Sep 13, 2011 at 23:26, Rongliang Chen >wrote: > > > > > mumps_static_mapping.F: In function ?mumps_369?: > > > mumps_static_mapping.F:333: internal compiler error: in > modified_type_die, > > > at dwarf2out.c:8495 > > > Please submit a full bug report, > > > with preprocessed source if appropriate. > > > See for instructions. > > > > > > > Um, internal compiler error. File a bug report with your vendor and/or > > upgrade your compilers? > > What version of compilers are you using? > > mpicc -show > mpif90 -show > mpicc --version > mpif90 --version > > Satish > > ------------------------------ > > _______________________________________________ > petsc-users mailing list > petsc-users at mcs.anl.gov > https://lists.mcs.anl.gov/mailman/listinfo/petsc-users > > > End of petsc-users Digest, Vol 33, Issue 36 > ******************************************* > -------------- next part -------------- An HTML attachment was scrubbed... URL: From balay at mcs.anl.gov Tue Sep 13 18:13:31 2011 From: balay at mcs.anl.gov (Satish Balay) Date: Tue, 13 Sep 2011 18:13:31 -0500 (CDT) Subject: [petsc-users] Problem with PETSc-3.2 configuration In-Reply-To: References: Message-ID: Ok - this is on a RHEL5 box. And I'm able to reproduce the internal compiler error. > mumps_static_mapping.F:333: internal compiler error: in modified_type_die, at dwarf2out.c:8495 The error is in debug symbol handler in the compiler. Looks like you can workarround this by using optized build. i.e --with-debugging=0 or debug build with: [default --with-debugging=1 ] FOPTFLAGS='-g -O' Satish On Tue, 13 Sep 2011, Rongliang Chen wrote: > Hi Jed and Satish, > > Thank you for your reply. > > Following is the information of the compilers. > mpicc -show > > gcc -I/curc/tools/free/redhat_5_x86_64/openmpi-1.4.3_ib/include -pthread > -L/curc/tools/free/redhat_5_x86_64/openmpi-1.4.3_ib/lib -lmpi -lopen-rte > -lopen-pal -ldl -Wl,--export-dynamic -lnsl -lutil -lm -ldl > > mpif90 -show > > gfortran -I/curc/tools/free/redhat_5_x86_64/openmpi-1.4.3_ib/include > -pthread -I/curc/tools/free/redhat_5_x86_64/openmpi-1.4.3_ib/lib > -L/curc/tools/free/redhat_5_x86_64/openmpi-1.4.3_ib/lib -lmpi_f90 -lmpi_f77 > -lmpi -lopen-rte -lopen-pal -ldl -Wl,--export-dynamic -lnsl -lutil -lm -ldl > > mpicc --version > > gcc (GCC) 4.1.2 20080704 (Red Hat 4.1.2-50) > Copyright (C) 2006 Free Software Foundation, Inc. > This is free software; see the source for copying conditions. There is NO > warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. > > mpif90 --version > > GNU Fortran (GCC) 4.1.2 20080704 (Red Hat 4.1.2-50) > Copyright (C) 2007 Free Software Foundation, Inc. > > GNU Fortran comes with NO WARRANTY, to the extent permitted by law. > You may redistribute copies of GNU Fortran > under the terms of the GNU General Public License. > For more information about these matters, see the file named COPYING > > > Best, > Rongliang > > > ------------------------------ > > > > Message: 5 > > Date: Tue, 13 Sep 2011 23:29:56 +0200 > > From: Jed Brown > > Subject: Re: [petsc-users] Problem with PETSc-3.2 configuration > > To: PETSc users list > > Message-ID: > > > > > > Content-Type: text/plain; charset="utf-8" > > > > On Tue, Sep 13, 2011 at 23:26, Rongliang Chen > >wrote: > > > > > mumps_static_mapping.F: In function ?mumps_369?: > > > mumps_static_mapping.F:333: internal compiler error: in > > modified_type_die, > > > at dwarf2out.c:8495 > > > Please submit a full bug report, > > > with preprocessed source if appropriate. > > > See for instructions. > > > > > > > Um, internal compiler error. File a bug report with your vendor and/or > > upgrade your compilers? > > -------------- next part -------------- > > An HTML attachment was scrubbed... > > URL: < > > http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20110913/eced13ed/attachment-0001.htm > > > > > > > ------------------------------ > > > > Message: 6 > > Date: Tue, 13 Sep 2011 16:43:04 -0500 (CDT) > > From: Satish Balay > > Subject: Re: [petsc-users] Problem with PETSc-3.2 configuration > > To: PETSc users list > > Message-ID: > > Content-Type: text/plain; charset="iso-8859-7" > > > > On Tue, 13 Sep 2011, Jed Brown wrote: > > > > > On Tue, Sep 13, 2011 at 23:26, Rongliang Chen > >wrote: > > > > > > > mumps_static_mapping.F: In function ?mumps_369?: > > > > mumps_static_mapping.F:333: internal compiler error: in > > modified_type_die, > > > > at dwarf2out.c:8495 > > > > Please submit a full bug report, > > > > with preprocessed source if appropriate. > > > > See for instructions. > > > > > > > > > > Um, internal compiler error. File a bug report with your vendor and/or > > > upgrade your compilers? > > > > What version of compilers are you using? > > > > mpicc -show > > mpif90 -show > > mpicc --version > > mpif90 --version > > > > Satish > > > > ------------------------------ > > > > _______________________________________________ > > petsc-users mailing list > > petsc-users at mcs.anl.gov > > https://lists.mcs.anl.gov/mailman/listinfo/petsc-users > > > > > > End of petsc-users Digest, Vol 33, Issue 36 > > ******************************************* > > > From bsmith at mcs.anl.gov Tue Sep 13 19:48:46 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Tue, 13 Sep 2011 19:48:46 -0500 Subject: [petsc-users] Problem with PETSc-3.2 configuration In-Reply-To: References: Message-ID: Also that gfortran compiler is terribly old, you really should install the 4.6 version of the Gnu compiler suite instead of living with that old stuff. Barry On Sep 13, 2011, at 6:13 PM, Satish Balay wrote: > Ok - this is on a RHEL5 box. And I'm able to reproduce the internal compiler error. > >> mumps_static_mapping.F:333: internal compiler error: in modified_type_die, at dwarf2out.c:8495 > > The error is in debug symbol handler in the compiler. Looks like you > can workarround this by using optized build. i.e > > --with-debugging=0 > > or debug build with: > > [default --with-debugging=1 ] FOPTFLAGS='-g -O' > > Satish > > On Tue, 13 Sep 2011, Rongliang Chen wrote: > >> Hi Jed and Satish, >> >> Thank you for your reply. >> >> Following is the information of the compilers. >> mpicc -show >> >> gcc -I/curc/tools/free/redhat_5_x86_64/openmpi-1.4.3_ib/include -pthread >> -L/curc/tools/free/redhat_5_x86_64/openmpi-1.4.3_ib/lib -lmpi -lopen-rte >> -lopen-pal -ldl -Wl,--export-dynamic -lnsl -lutil -lm -ldl >> >> mpif90 -show >> >> gfortran -I/curc/tools/free/redhat_5_x86_64/openmpi-1.4.3_ib/include >> -pthread -I/curc/tools/free/redhat_5_x86_64/openmpi-1.4.3_ib/lib >> -L/curc/tools/free/redhat_5_x86_64/openmpi-1.4.3_ib/lib -lmpi_f90 -lmpi_f77 >> -lmpi -lopen-rte -lopen-pal -ldl -Wl,--export-dynamic -lnsl -lutil -lm -ldl >> >> mpicc --version >> >> gcc (GCC) 4.1.2 20080704 (Red Hat 4.1.2-50) >> Copyright (C) 2006 Free Software Foundation, Inc. >> This is free software; see the source for copying conditions. There is NO >> warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. >> >> mpif90 --version >> >> GNU Fortran (GCC) 4.1.2 20080704 (Red Hat 4.1.2-50) >> Copyright (C) 2007 Free Software Foundation, Inc. >> >> GNU Fortran comes with NO WARRANTY, to the extent permitted by law. >> You may redistribute copies of GNU Fortran >> under the terms of the GNU General Public License. >> For more information about these matters, see the file named COPYING >> >> >> Best, >> Rongliang >> >> >> ------------------------------ >>> >>> Message: 5 >>> Date: Tue, 13 Sep 2011 23:29:56 +0200 >>> From: Jed Brown >>> Subject: Re: [petsc-users] Problem with PETSc-3.2 configuration >>> To: PETSc users list >>> Message-ID: >>> >>> >>> Content-Type: text/plain; charset="utf-8" >>> >>> On Tue, Sep 13, 2011 at 23:26, Rongliang Chen >>> wrote: >>> >>>> mumps_static_mapping.F: In function ?mumps_369?: >>>> mumps_static_mapping.F:333: internal compiler error: in >>> modified_type_die, >>>> at dwarf2out.c:8495 >>>> Please submit a full bug report, >>>> with preprocessed source if appropriate. >>>> See for instructions. >>>> >>> >>> Um, internal compiler error. File a bug report with your vendor and/or >>> upgrade your compilers? >>> -------------- next part -------------- >>> An HTML attachment was scrubbed... >>> URL: < >>> http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20110913/eced13ed/attachment-0001.htm >>>> >>> >>> ------------------------------ >>> >>> Message: 6 >>> Date: Tue, 13 Sep 2011 16:43:04 -0500 (CDT) >>> From: Satish Balay >>> Subject: Re: [petsc-users] Problem with PETSc-3.2 configuration >>> To: PETSc users list >>> Message-ID: >>> Content-Type: text/plain; charset="iso-8859-7" >>> >>> On Tue, 13 Sep 2011, Jed Brown wrote: >>> >>>> On Tue, Sep 13, 2011 at 23:26, Rongliang Chen >>> wrote: >>>> >>>>> mumps_static_mapping.F: In function ?mumps_369?: >>>>> mumps_static_mapping.F:333: internal compiler error: in >>> modified_type_die, >>>>> at dwarf2out.c:8495 >>>>> Please submit a full bug report, >>>>> with preprocessed source if appropriate. >>>>> See for instructions. >>>>> >>>> >>>> Um, internal compiler error. File a bug report with your vendor and/or >>>> upgrade your compilers? >>> >>> What version of compilers are you using? >>> >>> mpicc -show >>> mpif90 -show >>> mpicc --version >>> mpif90 --version >>> >>> Satish >>> >>> ------------------------------ >>> >>> _______________________________________________ >>> petsc-users mailing list >>> petsc-users at mcs.anl.gov >>> https://lists.mcs.anl.gov/mailman/listinfo/petsc-users >>> >>> >>> End of petsc-users Digest, Vol 33, Issue 36 >>> ******************************************* >>> >> > From marco.cisternino at polito.it Wed Sep 14 05:20:21 2011 From: marco.cisternino at polito.it (Marco Cisternino) Date: Wed, 14 Sep 2011 12:20:21 +0200 Subject: [petsc-users] Neumann BC for interface elliptic problem In-Reply-To: References: Message-ID: <4E707FE5.10907@polito.it> "Everything" means everything: the solver converges with rtol=10^(-12) to the right solution. For tests with problems the solver converges but to a deformed solution. I'm trying some other test, I would like to tell you exactly which is the difference between working and not working cases. thanks. Marco -- Marco Cisternino PhD Student Politecnico di Torino Email:marco.cisternino at polito.it From bsmith at mcs.anl.gov Wed Sep 14 07:28:00 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Wed, 14 Sep 2011 07:28:00 -0500 Subject: [petsc-users] Neumann BC for interface elliptic problem In-Reply-To: <4E707FE5.10907@polito.it> References: <4E707FE5.10907@polito.it> Message-ID: On Sep 14, 2011, at 5:20 AM, Marco Cisternino wrote: > "Everything" means everything: the solver converges with rtol=10^(-12) to the right solution. > For tests with problems the solver converges but to a deformed solution. If the KSP solver converges (but to a deformed solution) this means the linear system must not be defined to be the same as before, for example the matrix is wrong. Barry > I'm trying some other test, I would like to tell you exactly which is the difference between working and not working cases. > thanks. > > Marco > > -- > Marco Cisternino > PhD Student > Politecnico di Torino > Email:marco.cisternino at polito.it > From rongliang.chan at gmail.com Wed Sep 14 12:42:28 2011 From: rongliang.chan at gmail.com (Rongliang Chen) Date: Wed, 14 Sep 2011 11:42:28 -0600 Subject: [petsc-users] Problem with PETSc-3.2 configuration Message-ID: Hi Satish and Barry, Thanks for your reply. It works by using optimized build. This is a supercomputer and I do not have the permission to update the compliers. Best, Rongliang > Message: 2 > Date: Tue, 13 Sep 2011 18:13:31 -0500 (CDT) > From: Satish Balay > Subject: Re: [petsc-users] Problem with PETSc-3.2 configuration > To: PETSc users list > Message-ID: > Content-Type: TEXT/PLAIN; charset=US-ASCII > > Ok - this is on a RHEL5 box. And I'm able to reproduce the internal > compiler error. > > > mumps_static_mapping.F:333: internal compiler error: in > modified_type_die, at dwarf2out.c:8495 > > The error is in debug symbol handler in the compiler. Looks like you > can workarround this by using optized build. i.e > > --with-debugging=0 > > or debug build with: > > [default --with-debugging=1 ] FOPTFLAGS='-g -O' > > Satish > > > ------------------------------ > > Message: 3 > Date: Tue, 13 Sep 2011 19:48:46 -0500 > From: Barry Smith > Subject: Re: [petsc-users] Problem with PETSc-3.2 configuration > To: PETSc users list > Message-ID: > Content-Type: text/plain; charset=us-ascii > > > Also that gfortran compiler is terribly old, you really should install the > 4.6 version of the Gnu compiler suite instead of living with that old stuff. > > Barry > > On Sep 13, 2011, at 6:13 PM, Satish Balay wrote: > > > Ok - this is on a RHEL5 box. And I'm able to reproduce the internal > compiler error. > > > >> mumps_static_mapping.F:333: internal compiler error: in > modified_type_die, at dwarf2out.c:8495 > > > > The error is in debug symbol handler in the compiler. Looks like you > > can workarround this by using optized build. i.e > > > > --with-debugging=0 > > > > or debug build with: > > > > [default --with-debugging=1 ] FOPTFLAGS='-g -O' > > > > Satish > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From balay at mcs.anl.gov Wed Sep 14 12:52:38 2011 From: balay at mcs.anl.gov (Satish Balay) Date: Wed, 14 Sep 2011 12:52:38 -0500 (CDT) Subject: [petsc-users] Problem with PETSc-3.2 configuration In-Reply-To: References: Message-ID: Glad its working now. Wrt newer compilers - usually sysadmins [for clusters] install separate copies of newer compilers [and leave the default system compiler untouched]. They also install another copy of MPI with this new set of compilers. This workarround is ok for getting going with PETSc - but gfortran [prior to 4.3] has some f90 deficiencies - so some of the petsc f90 interface doesn't work because of this. So its good to use newer set of compilers. [However if using PETSc only from c/c++ - you can ignore this issue] Satish On Wed, 14 Sep 2011, Rongliang Chen wrote: > Hi Satish and Barry, > > Thanks for your reply. > > It works by using optimized build. > > This is a supercomputer and I do not have the permission to update the > compliers. > > Best, > Rongliang > > > > Message: 2 > > Date: Tue, 13 Sep 2011 18:13:31 -0500 (CDT) > > From: Satish Balay > > Subject: Re: [petsc-users] Problem with PETSc-3.2 configuration > > To: PETSc users list > > Message-ID: > > Content-Type: TEXT/PLAIN; charset=US-ASCII > > > > Ok - this is on a RHEL5 box. And I'm able to reproduce the internal > > compiler error. > > > > > mumps_static_mapping.F:333: internal compiler error: in > > modified_type_die, at dwarf2out.c:8495 > > > > The error is in debug symbol handler in the compiler. Looks like you > > can workarround this by using optized build. i.e > > > > --with-debugging=0 > > > > or debug build with: > > > > [default --with-debugging=1 ] FOPTFLAGS='-g -O' > > > > Satish > > > > > > ------------------------------ > > > > Message: 3 > > Date: Tue, 13 Sep 2011 19:48:46 -0500 > > From: Barry Smith > > Subject: Re: [petsc-users] Problem with PETSc-3.2 configuration > > To: PETSc users list > > Message-ID: > > Content-Type: text/plain; charset=us-ascii > > > > > > Also that gfortran compiler is terribly old, you really should install the > > 4.6 version of the Gnu compiler suite instead of living with that old stuff. > > > > Barry > > > > On Sep 13, 2011, at 6:13 PM, Satish Balay wrote: > > > > > Ok - this is on a RHEL5 box. And I'm able to reproduce the internal > > compiler error. > > > > > >> mumps_static_mapping.F:333: internal compiler error: in > > modified_type_die, at dwarf2out.c:8495 > > > > > > The error is in debug symbol handler in the compiler. Looks like you > > > can workarround this by using optized build. i.e > > > > > > --with-debugging=0 > > > > > > or debug build with: > > > > > > [default --with-debugging=1 ] FOPTFLAGS='-g -O' > > > > > > Satish > > > > > > From milan.v.mitrovic at gmail.com Wed Sep 14 14:16:52 2011 From: milan.v.mitrovic at gmail.com (Milan Mitrovic) Date: Wed, 14 Sep 2011 21:16:52 +0200 Subject: [petsc-users] Quick question Message-ID: How do I use MatGetRow properly from fortran? If I use -snes_mf_operator can I somehow get the computed values eg. with MatGetRow? thanks! milan From jedbrown at mcs.anl.gov Wed Sep 14 14:22:56 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Wed, 14 Sep 2011 21:22:56 +0200 Subject: [petsc-users] Quick question In-Reply-To: References: Message-ID: On Wed, Sep 14, 2011 at 21:16, Milan Mitrovic wrote: > How do I use MatGetRow properly from fortran? > Fortran calling sequence is in the man page: http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-dev/docs/manualpages/Mat/MatGetRow.html > If I use -snes_mf_operator can I somehow get the computed values eg. > with MatGetRow? > -snes_mf_operator does not compute entries. You can build a finite difference Jacobian with coloring or build a dense finite difference Jacobian. http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-dev/docs/manualpages/MatFD/MatFDColoringApply.html http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-dev/docs/manualpages/Mat/MatComputeExplicitOperator.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From agrayver at gfz-potsdam.de Thu Sep 15 02:49:59 2011 From: agrayver at gfz-potsdam.de (Alexander Grayver) Date: Thu, 15 Sep 2011 09:49:59 +0200 Subject: [petsc-users] Problems with last PETSC-3.2 release on Windows Message-ID: <4E71AE27.3030600@gfz-potsdam.de> Hello, I hope you are having nice day! I have been using petsc-3.1-p7 under windows for 6 months and now decided to get new version. I used my old configure line which was successful for petsc-3.1: ./config/configure.py --PETSC_DIR=/cygdrive/d/dev/petsc-3.2-p1 --with-petsc-arch=cygwin-cxx-debug-fortran --with-fortran-interfaces=1 --with-cc='win32fe cl' --with-fc='win32fe ifort' --with-cxx='win32fe cl' --with-windows-graphics=0 --download-f-blas-lapack=1 --with-precision=double --with-scalar-type=complex --with-clanguage=cxx --with-mpi-include="/cygdrive/d/Dev/HPC_2008_SDK/Include" --with-mpi-lib=["/cygdrive/d/Dev/HPC_2008_SDK/Lib/amd64/msmpi.lib","/cygdrive/d/Dev/HPC_2008_SDK/Lib/amd64/msmpifec.lib"] --useThreads=0 But unfortunately it doesn't work properly anymore, even though my development environment has not been changed. The error is: ******************************************************************************* UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log for details): ------------------------------------------------------------------------------- Cannot determine Fortran module include flag Please find configure.log attached. Regards, Alexander -------------- next part -------------- A non-text attachment was scrubbed... Name: configure.zip Type: application/x-zip-compressed Size: 65094 bytes Desc: not available URL: From knepley at gmail.com Thu Sep 15 07:54:55 2011 From: knepley at gmail.com (Matthew Knepley) Date: Thu, 15 Sep 2011 07:54:55 -0500 Subject: [petsc-users] Problems with last PETSC-3.2 release on Windows In-Reply-To: <4E71AE27.3030600@gfz-potsdam.de> References: <4E71AE27.3030600@gfz-potsdam.de> Message-ID: On Thu, Sep 15, 2011 at 2:49 AM, Alexander Grayver wrote: > Hello, > > I hope you are having nice day! > > I have been using petsc-3.1-p7 under windows for 6 months and now decided > to get new version. I used my old configure line which was successful for > petsc-3.1: > > ./config/configure.py --PETSC_DIR=/cygdrive/d/dev/**petsc-3.2-p1 > --with-petsc-arch=cygwin-cxx-**debug-fortran --with-fortran-interfaces=1 > --with-cc='win32fe cl' --with-fc='win32fe ifort' --with-cxx='win32fe cl' > --with-windows-graphics=0 --download-f-blas-lapack=1 --with-precision=double > --with-scalar-type=complex --with-clanguage=cxx > --with-mpi-include="/cygdrive/**d/Dev/HPC_2008_SDK/Include" > --with-mpi-lib=["/cygdrive/d/**Dev/HPC_2008_SDK/Lib/amd64/** > msmpi.lib","/cygdrive/d/Dev/**HPC_2008_SDK/Lib/amd64/**msmpifec.lib"] > --useThreads=0 > > But unfortunately it doesn't work properly anymore, even though my > development environment has not been changed. The error is: > > **************************************************************** > ******************* > UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log for > details): > ------------------------------**------------------------------** > ------------------- > Cannot determine Fortran module include flag > > Please find configure.log attached. > The problem here is that your Fortran compiler reports that is supports F90, but cannot locate modules using an include path. This will make parts of the PETSc build fail. Do you know what flag it uses? I try -I, -p, and -M. Thanks, Matt > Regards, > Alexander -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From agrayver at gfz-potsdam.de Thu Sep 15 08:01:54 2011 From: agrayver at gfz-potsdam.de (Alexander Grayver) Date: Thu, 15 Sep 2011 15:01:54 +0200 Subject: [petsc-users] Problems with last PETSC-3.2 release on Windows In-Reply-To: References: <4E71AE27.3030600@gfz-potsdam.de> Message-ID: <4E71F742.70202@gfz-potsdam.de> Matthew, I found this check in configure.log I obtained for petsc-3.1-p7: ================================================================================ TEST checkFortranModuleInclude from config.compilers(/cygdrive/d/Dev/petsc-3.1-p7/config/BuildSystem/config/compilers.py:1084) TESTING: checkFortranModuleInclude from config.compilers(config/BuildSystem/config/compilers.py:1084) Figures out what flag is used to specify the include path for Fortran modules Pushing language FC sh: /cygdrive/d/Dev/petsc-3.1-p7/bin/win32fe/win32fe ifort -c -o conftest.o -MT -Z7 -fpp conftest.F Executing: /cygdrive/d/Dev/petsc-3.1-p7/bin/win32fe/win32fe ifort -c -o conftest.o -MT -Z7 -fpp conftest.F sh: Pushing language FC sh: /cygdrive/d/Dev/petsc-3.1-p7/bin/win32fe/win32fe ifort -c -o conftest.o -Iconfdir -MT -Z7 -fpp conftest.F Executing: /cygdrive/d/Dev/petsc-3.1-p7/bin/win32fe/win32fe ifort -c -o conftest.o -Iconfdir -MT -Z7 -fpp conftest.F sh: Pushing language FC Popping language FC sh: /cygdrive/d/Dev/petsc-3.1-p7/bin/win32fe/win32fe ifort -o conftest.exe -Iconfdir -MT -Z7 -fpp conftest.o configtest.o Executing: /cygdrive/d/Dev/petsc-3.1-p7/bin/win32fe/win32fe ifort -o conftest.exe -Iconfdir -MT -Z7 -fpp conftest.o configtest.o sh: LINK : D:\Dev\PETSC-~2.1-P\conftest.exe not found or not built by the last incremental link; performing full link compilers: Fortran module include flag -I found Popping language FC ================================================================================ The compiler I used is the same I use for petsc-3.2-p1. The difference is in the paths where you try to build this tests. As far as I understand for petsc-3.1-p7 you used local directory and for petsc-3.2-p1 temp directory is used. Could it be the reason? Regards, Alexander On 15.09.2011 14:54, Matthew Knepley wrote: > On Thu, Sep 15, 2011 at 2:49 AM, Alexander Grayver > > wrote: > > Hello, > > I hope you are having nice day! > > I have been using petsc-3.1-p7 under windows for 6 months and now > decided to get new version. I used my old configure line which was > successful for petsc-3.1: > > ./config/configure.py --PETSC_DIR=/cygdrive/d/dev/petsc-3.2-p1 > --with-petsc-arch=cygwin-cxx-debug-fortran > --with-fortran-interfaces=1 --with-cc='win32fe cl' > --with-fc='win32fe ifort' --with-cxx='win32fe cl' > --with-windows-graphics=0 --download-f-blas-lapack=1 > --with-precision=double --with-scalar-type=complex > --with-clanguage=cxx > --with-mpi-include="/cygdrive/d/Dev/HPC_2008_SDK/Include" > --with-mpi-lib=["/cygdrive/d/Dev/HPC_2008_SDK/Lib/amd64/msmpi.lib","/cygdrive/d/Dev/HPC_2008_SDK/Lib/amd64/msmpifec.lib"] > --useThreads=0 > > But unfortunately it doesn't work properly anymore, even though my > development environment has not been changed. The error is: > > ******************************************************************************* > UNABLE to CONFIGURE with GIVEN OPTIONS (see > configure.log for details): > ------------------------------------------------------------------------------- > Cannot determine Fortran module include flag > > Please find configure.log attached. > > > The problem here is that your Fortran compiler reports that is > supports F90, but cannot > locate modules using an include path. This will make parts of the > PETSc build fail. Do you > know what flag it uses? I try -I, -p, and -M. > > Thanks, > > Matt > > Regards, > Alexander > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which > their experiments lead. > -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Thu Sep 15 08:43:17 2011 From: knepley at gmail.com (Matthew Knepley) Date: Thu, 15 Sep 2011 08:43:17 -0500 Subject: [petsc-users] Problems with last PETSC-3.2 release on Windows In-Reply-To: <4E71F742.70202@gfz-potsdam.de> References: <4E71AE27.3030600@gfz-potsdam.de> <4E71F742.70202@gfz-potsdam.de> Message-ID: On Thu, Sep 15, 2011 at 8:01 AM, Alexander Grayver wrote: > ** > Matthew, > > I found this check in configure.log I obtained for petsc-3.1-p7: > > > ================================================================================ > TEST checkFortranModuleInclude from > config.compilers(/cygdrive/d/Dev/petsc-3.1-p7/config/BuildSystem/config/compilers.py:1084) > TESTING: checkFortranModuleInclude from > config.compilers(config/BuildSystem/config/compilers.py:1084) > Figures out what flag is used to specify the include path for Fortran > modules > Pushing language FC > sh: /cygdrive/d/Dev/petsc-3.1-p7/bin/win32fe/win32fe ifort -c -o > conftest.o -MT -Z7 -fpp conftest.F > Executing: /cygdrive/d/Dev/petsc-3.1-p7/bin/win32fe/win32fe ifort -c -o > conftest.o -MT -Z7 -fpp conftest.F > sh: > Pushing language FC > sh: /cygdrive/d/Dev/petsc-3.1-p7/bin/win32fe/win32fe ifort -c -o conftest.o > -Iconfdir -MT -Z7 -fpp conftest.F > Executing: /cygdrive/d/Dev/petsc-3.1-p7/bin/win32fe/win32fe ifort -c -o > conftest.o -Iconfdir -MT -Z7 -fpp conftest.F > sh: > Pushing language FC > Popping language FC > sh: /cygdrive/d/Dev/petsc-3.1-p7/bin/win32fe/win32fe ifort -o > conftest.exe -Iconfdir -MT -Z7 -fpp conftest.o configtest.o > Executing: /cygdrive/d/Dev/petsc-3.1-p7/bin/win32fe/win32fe ifort -o > conftest.exe -Iconfdir -MT -Z7 -fpp conftest.o configtest.o > sh: LINK : D:\Dev\PETSC-~2.1-P\conftest.exe not found or not built by the > last incremental link; performing full link > > compilers: Fortran module include flag -I found > Popping language FC > > ================================================================================ > > The compiler I used is the same I use for petsc-3.2-p1. > The difference is in the paths where you try to build this tests. As far as > I understand for petsc-3.1-p7 you used local directory and for petsc-3.2-p1 > temp directory is used. > Could it be the reason? > That would be broken behavior from ifort, but I guess it is possible. I will talk with Satish about how this should really be fixed. For right now, go to config/BuildSystem/config/compilers.py to the checkFortranModuleInclude() function and comment out the last two lines, which are killing the configure. Your build will break on the Fortran modules (I assume), but everything else should work. Thanks, Matt > Regards, > Alexander > > > On 15.09.2011 14:54, Matthew Knepley wrote: > > On Thu, Sep 15, 2011 at 2:49 AM, Alexander Grayver < > agrayver at gfz-potsdam.de> wrote: > >> Hello, >> >> I hope you are having nice day! >> >> I have been using petsc-3.1-p7 under windows for 6 months and now decided >> to get new version. I used my old configure line which was successful for >> petsc-3.1: >> >> ./config/configure.py --PETSC_DIR=/cygdrive/d/dev/petsc-3.2-p1 >> --with-petsc-arch=cygwin-cxx-debug-fortran --with-fortran-interfaces=1 >> --with-cc='win32fe cl' --with-fc='win32fe ifort' --with-cxx='win32fe cl' >> --with-windows-graphics=0 --download-f-blas-lapack=1 --with-precision=double >> --with-scalar-type=complex --with-clanguage=cxx >> --with-mpi-include="/cygdrive/d/Dev/HPC_2008_SDK/Include" >> --with-mpi-lib=["/cygdrive/d/Dev/HPC_2008_SDK/Lib/amd64/msmpi.lib","/cygdrive/d/Dev/HPC_2008_SDK/Lib/amd64/msmpifec.lib"] >> --useThreads=0 >> >> But unfortunately it doesn't work properly anymore, even though my >> development environment has not been changed. The error is: >> >> >> ******************************************************************************* >> UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log for >> details): >> >> ------------------------------------------------------------------------------- >> Cannot determine Fortran module include flag >> >> Please find configure.log attached. >> > > The problem here is that your Fortran compiler reports that is supports > F90, but cannot > locate modules using an include path. This will make parts of the PETSc > build fail. Do you > know what flag it uses? I try -I, -p, and -M. > > Thanks, > > Matt > > >> Regards, >> Alexander > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From agrayver at gfz-potsdam.de Thu Sep 15 08:43:49 2011 From: agrayver at gfz-potsdam.de (Alexander Grayver) Date: Thu, 15 Sep 2011 15:43:49 +0200 Subject: [petsc-users] Problems with last PETSC-3.2 release on Windows In-Reply-To: References: <4E71AE27.3030600@gfz-potsdam.de> <4E71F742.70202@gfz-potsdam.de> Message-ID: <4E720115.9090600@gfz-potsdam.de> Thanks Matt, I will that, but I use PETSC from Fortran so I need Fortran functionality. Regards, Alexander On 15.09.2011 15:43, Matthew Knepley wrote: > On Thu, Sep 15, 2011 at 8:01 AM, Alexander Grayver > > wrote: > > Matthew, > > I found this check in configure.log I obtained for petsc-3.1-p7: > > ================================================================================ > TEST checkFortranModuleInclude from > config.compilers(/cygdrive/d/Dev/petsc-3.1-p7/config/BuildSystem/config/compilers.py:1084) > TESTING: checkFortranModuleInclude from > config.compilers(config/BuildSystem/config/compilers.py:1084) > Figures out what flag is used to specify the include path for > Fortran modules > Pushing language FC > sh: /cygdrive/d/Dev/petsc-3.1-p7/bin/win32fe/win32fe ifort -c -o > conftest.o -MT -Z7 -fpp conftest.F > Executing: /cygdrive/d/Dev/petsc-3.1-p7/bin/win32fe/win32fe ifort > -c -o conftest.o -MT -Z7 -fpp conftest.F > sh: > Pushing language FC > sh: /cygdrive/d/Dev/petsc-3.1-p7/bin/win32fe/win32fe ifort -c -o > conftest.o -Iconfdir -MT -Z7 -fpp conftest.F > Executing: /cygdrive/d/Dev/petsc-3.1-p7/bin/win32fe/win32fe ifort > -c -o conftest.o -Iconfdir -MT -Z7 -fpp conftest.F > sh: > Pushing language FC > Popping language FC > sh: /cygdrive/d/Dev/petsc-3.1-p7/bin/win32fe/win32fe ifort -o > conftest.exe -Iconfdir -MT -Z7 -fpp conftest.o configtest.o > Executing: /cygdrive/d/Dev/petsc-3.1-p7/bin/win32fe/win32fe ifort > -o conftest.exe -Iconfdir -MT -Z7 -fpp conftest.o configtest.o > sh: LINK : D:\Dev\PETSC-~2.1-P\conftest.exe not found or not built > by the last incremental link; performing full link > > compilers: Fortran module include flag -I found > Popping language FC > ================================================================================ > > The compiler I used is the same I use for petsc-3.2-p1. > The difference is in the paths where you try to build this tests. > As far as I understand for petsc-3.1-p7 you used local directory > and for petsc-3.2-p1 temp directory is used. > Could it be the reason? > > > That would be broken behavior from ifort, but I guess it is possible. > I will talk with Satish about how > this should really be fixed. For right now, go to > > config/BuildSystem/config/compilers.py > > to the checkFortranModuleInclude() function and comment out the last > two lines, which are killing the configure. > Your build will break on the Fortran modules (I assume), but > everything else should work. > > Thanks, > > Matt > > Regards, > Alexander > > > On 15.09.2011 14:54, Matthew Knepley wrote: >> On Thu, Sep 15, 2011 at 2:49 AM, Alexander Grayver >> > wrote: >> >> Hello, >> >> I hope you are having nice day! >> >> I have been using petsc-3.1-p7 under windows for 6 months and >> now decided to get new version. I used my old configure line >> which was successful for petsc-3.1: >> >> ./config/configure.py >> --PETSC_DIR=/cygdrive/d/dev/petsc-3.2-p1 >> --with-petsc-arch=cygwin-cxx-debug-fortran >> --with-fortran-interfaces=1 --with-cc='win32fe cl' >> --with-fc='win32fe ifort' --with-cxx='win32fe cl' >> --with-windows-graphics=0 --download-f-blas-lapack=1 >> --with-precision=double --with-scalar-type=complex >> --with-clanguage=cxx >> --with-mpi-include="/cygdrive/d/Dev/HPC_2008_SDK/Include" >> --with-mpi-lib=["/cygdrive/d/Dev/HPC_2008_SDK/Lib/amd64/msmpi.lib","/cygdrive/d/Dev/HPC_2008_SDK/Lib/amd64/msmpifec.lib"] >> --useThreads=0 >> >> But unfortunately it doesn't work properly anymore, even >> though my development environment has not been changed. The >> error is: >> >> ******************************************************************************* >> UNABLE to CONFIGURE with GIVEN OPTIONS (see >> configure.log for details): >> ------------------------------------------------------------------------------- >> Cannot determine Fortran module include flag >> >> Please find configure.log attached. >> >> >> The problem here is that your Fortran compiler reports that is >> supports F90, but cannot >> locate modules using an include path. This will make parts of the >> PETSc build fail. Do you >> know what flag it uses? I try -I, -p, and -M. >> >> Thanks, >> >> Matt >> >> Regards, >> Alexander >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to >> which their experiments lead. >> -- Norbert Wiener > > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which > their experiments lead. > -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Thu Sep 15 08:49:54 2011 From: knepley at gmail.com (Matthew Knepley) Date: Thu, 15 Sep 2011 08:49:54 -0500 Subject: [petsc-users] Problems with last PETSC-3.2 release on Windows In-Reply-To: <4E720115.9090600@gfz-potsdam.de> References: <4E71AE27.3030600@gfz-potsdam.de> <4E71F742.70202@gfz-potsdam.de> <4E720115.9090600@gfz-potsdam.de> Message-ID: On Thu, Sep 15, 2011 at 8:43 AM, Alexander Grayver wrote: > ** > Thanks Matt, > > I will that, but I use PETSC from Fortran so I need Fortran functionality. > Sure. Do you use the F90 modules? If so, send us the build problem. If there is not build problem, then ifort supports only relative paths for module includes, which is crazy and will break sometimes. Thanks, Matt > Regards, > Alexander > > > On 15.09.2011 15:43, Matthew Knepley wrote: > > On Thu, Sep 15, 2011 at 8:01 AM, Alexander Grayver < > agrayver at gfz-potsdam.de> wrote: > >> Matthew, >> >> I found this check in configure.log I obtained for petsc-3.1-p7: >> >> >> ================================================================================ >> TEST checkFortranModuleInclude from >> config.compilers(/cygdrive/d/Dev/petsc-3.1-p7/config/BuildSystem/config/compilers.py:1084) >> TESTING: checkFortranModuleInclude from >> config.compilers(config/BuildSystem/config/compilers.py:1084) >> Figures out what flag is used to specify the include path for Fortran >> modules >> Pushing language FC >> sh: /cygdrive/d/Dev/petsc-3.1-p7/bin/win32fe/win32fe ifort -c -o >> conftest.o -MT -Z7 -fpp conftest.F >> Executing: /cygdrive/d/Dev/petsc-3.1-p7/bin/win32fe/win32fe ifort -c -o >> conftest.o -MT -Z7 -fpp conftest.F >> sh: >> Pushing language FC >> sh: /cygdrive/d/Dev/petsc-3.1-p7/bin/win32fe/win32fe ifort -c -o >> conftest.o -Iconfdir -MT -Z7 -fpp conftest.F >> Executing: /cygdrive/d/Dev/petsc-3.1-p7/bin/win32fe/win32fe ifort -c -o >> conftest.o -Iconfdir -MT -Z7 -fpp conftest.F >> sh: >> Pushing language FC >> Popping language FC >> sh: /cygdrive/d/Dev/petsc-3.1-p7/bin/win32fe/win32fe ifort -o >> conftest.exe -Iconfdir -MT -Z7 -fpp conftest.o configtest.o >> Executing: /cygdrive/d/Dev/petsc-3.1-p7/bin/win32fe/win32fe ifort -o >> conftest.exe -Iconfdir -MT -Z7 -fpp conftest.o configtest.o >> sh: LINK : D:\Dev\PETSC-~2.1-P\conftest.exe not found or not built by the >> last incremental link; performing full link >> >> compilers: Fortran module include flag -I found >> Popping language FC >> >> ================================================================================ >> >> The compiler I used is the same I use for petsc-3.2-p1. >> The difference is in the paths where you try to build this tests. As far >> as I understand for petsc-3.1-p7 you used local directory and for >> petsc-3.2-p1 temp directory is used. >> Could it be the reason? >> > > That would be broken behavior from ifort, but I guess it is possible. I > will talk with Satish about how > this should really be fixed. For right now, go to > > config/BuildSystem/config/compilers.py > > to the checkFortranModuleInclude() function and comment out the last two > lines, which are killing the configure. > Your build will break on the Fortran modules (I assume), but everything > else should work. > > Thanks, > > Matt > > >> Regards, >> Alexander >> >> >> On 15.09.2011 14:54, Matthew Knepley wrote: >> >> On Thu, Sep 15, 2011 at 2:49 AM, Alexander Grayver < >> agrayver at gfz-potsdam.de> wrote: >> >>> Hello, >>> >>> I hope you are having nice day! >>> >>> I have been using petsc-3.1-p7 under windows for 6 months and now decided >>> to get new version. I used my old configure line which was successful for >>> petsc-3.1: >>> >>> ./config/configure.py --PETSC_DIR=/cygdrive/d/dev/petsc-3.2-p1 >>> --with-petsc-arch=cygwin-cxx-debug-fortran --with-fortran-interfaces=1 >>> --with-cc='win32fe cl' --with-fc='win32fe ifort' --with-cxx='win32fe cl' >>> --with-windows-graphics=0 --download-f-blas-lapack=1 --with-precision=double >>> --with-scalar-type=complex --with-clanguage=cxx >>> --with-mpi-include="/cygdrive/d/Dev/HPC_2008_SDK/Include" >>> --with-mpi-lib=["/cygdrive/d/Dev/HPC_2008_SDK/Lib/amd64/msmpi.lib","/cygdrive/d/Dev/HPC_2008_SDK/Lib/amd64/msmpifec.lib"] >>> --useThreads=0 >>> >>> But unfortunately it doesn't work properly anymore, even though my >>> development environment has not been changed. The error is: >>> >>> >>> ******************************************************************************* >>> UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log for >>> details): >>> >>> ------------------------------------------------------------------------------- >>> Cannot determine Fortran module include flag >>> >>> Please find configure.log attached. >>> >> >> The problem here is that your Fortran compiler reports that is supports >> F90, but cannot >> locate modules using an include path. This will make parts of the PETSc >> build fail. Do you >> know what flag it uses? I try -I, -p, and -M. >> >> Thanks, >> >> Matt >> >> >>> Regards, >>> Alexander >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which their >> experiments lead. >> -- Norbert Wiener >> >> >> > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From balay at mcs.anl.gov Thu Sep 15 09:48:17 2011 From: balay at mcs.anl.gov (Satish Balay) Date: Thu, 15 Sep 2011 09:48:17 -0500 (CDT) Subject: [petsc-users] Problems with last PETSC-3.2 release on Windows In-Reply-To: References: <4E71AE27.3030600@gfz-potsdam.de> <4E71F742.70202@gfz-potsdam.de> <4E720115.9090600@gfz-potsdam.de> Message-ID: I just ran petsc-32 with ifort - and it ran fine. So I'm not sure whats hapenning in this case.. Is this error reproduceable [if configure is rerun with the same options?] Satish Intel(R) Visual Fortran Compiler for applications running on Intel(R) 64, Version 10.1 Build 20080212 Package ID: w_fc_p_10.1.019 >>>>>>>>>>> ================================================================================ TEST checkFortranModuleInclude from config.compilers(/home/balay/petsc-3.2/config/BuildSystem/config/compilers.py:1155) TESTING: checkFortranModuleInclude from config.compilers(config/BuildSystem/config/compilers.py:1155) Figures out what flag is used to specify the include path for Fortran modules Pushing language FC sh: /home/balay/petsc-3.2/bin/win32fe/win32fe ifort -c -o /tmp/petsc-zRoAJW/config.compilers/conftest.o -I/tmp/petsc-zRoAJW/config.compilers -I/tmp/petsc-zRoAJW/config.setCompilers -MT -Z7 -fpp /tmp/petsc-zRoAJW/config.compilers/conftest.F Executing: /home/balay/petsc-3.2/bin/win32fe/win32fe ifort -c -o /tmp/petsc-zRoAJW/config.compilers/conftest.o -I/tmp/petsc-zRoAJW/config.compilers -I/tmp/petsc-zRoAJW/config.setCompilers -MT -Z7 -fpp /tmp/petsc-zRoAJW/config.compilers/conftest.F sh: Successful compile: Source: module configtest integer testint parameter (testint = 42) end module configtest Pushing language FC sh: /home/balay/petsc-3.2/bin/win32fe/win32fe ifort -c -o /tmp/petsc-zRoAJW/config.compilers/conftest.o -I/tmp/petsc-zRoAJW/config.compilers -I/tmp/petsc-zRoAJW/config.setCompilers -I/tmp/petsc-zRoAJW/config.compilers/confdir -MT -Z7 -fpp /tmp/petsc-zRoAJW/config.compilers/conftest.F Executing: /home/balay/petsc-3.2/bin/win32fe/win32fe ifort -c -o /tmp/petsc-zRoAJW/config.compilers/conftest.o -I/tmp/petsc-zRoAJW/config.compilers -I/tmp/petsc-zRoAJW/config.setCompilers -I/tmp/petsc-zRoAJW/config.compilers/confdir -MT -Z7 -fpp /tmp/petsc-zRoAJW/config.compilers/conftest.F sh: Successful compile: Source: program main use configtest write(*,*) testint end Pushing language FC Popping language FC sh: /home/balay/petsc-3.2/bin/win32fe/win32fe ifort -o /tmp/petsc-zRoAJW/config.compilers/conftest.exe -I/tmp/petsc-zRoAJW/config.compilers/confdir -MT -Z7 -fpp /tmp/petsc-zRoAJW/config.compilers/conftest.o /tmp/petsc-zRoAJW/config.compilers/configtest.o Executing: /home/balay/petsc-3.2/bin/win32fe/win32fe ifort -o /tmp/petsc-zRoAJW/config.compilers/conftest.exe -I/tmp/petsc-zRoAJW/config.compilers/confdir -MT -Z7 -fpp /tmp/petsc-zRoAJW/config.compilers/conftest.o /tmp/petsc-zRoAJW/config.compilers/configtest.o sh: LINK : C:\cygwin\tmp\PETSC-~3\CONFIG~1.COM\conftest.exe not found or not built by the last incremental link; performing full link^M compilers: Fortran module include flag -I found Popping language FC <<<<<<<<<<<<<<<<<<<<<<< On Thu, 15 Sep 2011, Matthew Knepley wrote: > On Thu, Sep 15, 2011 at 8:43 AM, Alexander Grayver > wrote: > > > ** > > Thanks Matt, > > > > I will that, but I use PETSC from Fortran so I need Fortran functionality. > > > > Sure. Do you use the F90 modules? If so, send us the build problem. If there > is not build problem, > then ifort supports only relative paths for module includes, which is crazy > and will break sometimes. > > Thanks, > > Matt > > > > Regards, > > Alexander > > > > > > On 15.09.2011 15:43, Matthew Knepley wrote: > > > > On Thu, Sep 15, 2011 at 8:01 AM, Alexander Grayver < > > agrayver at gfz-potsdam.de> wrote: > > > >> Matthew, > >> > >> I found this check in configure.log I obtained for petsc-3.1-p7: > >> > >> > >> ================================================================================ > >> TEST checkFortranModuleInclude from > >> config.compilers(/cygdrive/d/Dev/petsc-3.1-p7/config/BuildSystem/config/compilers.py:1084) > >> TESTING: checkFortranModuleInclude from > >> config.compilers(config/BuildSystem/config/compilers.py:1084) > >> Figures out what flag is used to specify the include path for Fortran > >> modules > >> Pushing language FC > >> sh: /cygdrive/d/Dev/petsc-3.1-p7/bin/win32fe/win32fe ifort -c -o > >> conftest.o -MT -Z7 -fpp conftest.F > >> Executing: /cygdrive/d/Dev/petsc-3.1-p7/bin/win32fe/win32fe ifort -c -o > >> conftest.o -MT -Z7 -fpp conftest.F > >> sh: > >> Pushing language FC > >> sh: /cygdrive/d/Dev/petsc-3.1-p7/bin/win32fe/win32fe ifort -c -o > >> conftest.o -Iconfdir -MT -Z7 -fpp conftest.F > >> Executing: /cygdrive/d/Dev/petsc-3.1-p7/bin/win32fe/win32fe ifort -c -o > >> conftest.o -Iconfdir -MT -Z7 -fpp conftest.F > >> sh: > >> Pushing language FC > >> Popping language FC > >> sh: /cygdrive/d/Dev/petsc-3.1-p7/bin/win32fe/win32fe ifort -o > >> conftest.exe -Iconfdir -MT -Z7 -fpp conftest.o configtest.o > >> Executing: /cygdrive/d/Dev/petsc-3.1-p7/bin/win32fe/win32fe ifort -o > >> conftest.exe -Iconfdir -MT -Z7 -fpp conftest.o configtest.o > >> sh: LINK : D:\Dev\PETSC-~2.1-P\conftest.exe not found or not built by the > >> last incremental link; performing full link > >> > >> compilers: Fortran module include flag -I found > >> Popping language FC > >> > >> ================================================================================ > >> > >> The compiler I used is the same I use for petsc-3.2-p1. > >> The difference is in the paths where you try to build this tests. As far > >> as I understand for petsc-3.1-p7 you used local directory and for > >> petsc-3.2-p1 temp directory is used. > >> Could it be the reason? > >> > > > > That would be broken behavior from ifort, but I guess it is possible. I > > will talk with Satish about how > > this should really be fixed. For right now, go to > > > > config/BuildSystem/config/compilers.py > > > > to the checkFortranModuleInclude() function and comment out the last two > > lines, which are killing the configure. > > Your build will break on the Fortran modules (I assume), but everything > > else should work. > > > > Thanks, > > > > Matt > > > > > >> Regards, > >> Alexander > >> > >> > >> On 15.09.2011 14:54, Matthew Knepley wrote: > >> > >> On Thu, Sep 15, 2011 at 2:49 AM, Alexander Grayver < > >> agrayver at gfz-potsdam.de> wrote: > >> > >>> Hello, > >>> > >>> I hope you are having nice day! > >>> > >>> I have been using petsc-3.1-p7 under windows for 6 months and now decided > >>> to get new version. I used my old configure line which was successful for > >>> petsc-3.1: > >>> > >>> ./config/configure.py --PETSC_DIR=/cygdrive/d/dev/petsc-3.2-p1 > >>> --with-petsc-arch=cygwin-cxx-debug-fortran --with-fortran-interfaces=1 > >>> --with-cc='win32fe cl' --with-fc='win32fe ifort' --with-cxx='win32fe cl' > >>> --with-windows-graphics=0 --download-f-blas-lapack=1 --with-precision=double > >>> --with-scalar-type=complex --with-clanguage=cxx > >>> --with-mpi-include="/cygdrive/d/Dev/HPC_2008_SDK/Include" > >>> --with-mpi-lib=["/cygdrive/d/Dev/HPC_2008_SDK/Lib/amd64/msmpi.lib","/cygdrive/d/Dev/HPC_2008_SDK/Lib/amd64/msmpifec.lib"] > >>> --useThreads=0 > >>> > >>> But unfortunately it doesn't work properly anymore, even though my > >>> development environment has not been changed. The error is: > >>> > >>> > >>> ******************************************************************************* > >>> UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log for > >>> details): > >>> > >>> ------------------------------------------------------------------------------- > >>> Cannot determine Fortran module include flag > >>> > >>> Please find configure.log attached. > >>> > >> > >> The problem here is that your Fortran compiler reports that is supports > >> F90, but cannot > >> locate modules using an include path. This will make parts of the PETSc > >> build fail. Do you > >> know what flag it uses? I try -I, -p, and -M. > >> > >> Thanks, > >> > >> Matt > >> > >> > >>> Regards, > >>> Alexander > >> > >> -- > >> What most experimenters take for granted before they begin their > >> experiments is infinitely more interesting than any results to which their > >> experiments lead. > >> -- Norbert Wiener > >> > >> > >> > > > > > > -- > > What most experimenters take for granted before they begin their > > experiments is infinitely more interesting than any results to which their > > experiments lead. > > -- Norbert Wiener > > > > > > > > > From agrayver at gfz-potsdam.de Thu Sep 15 12:11:19 2011 From: agrayver at gfz-potsdam.de (Alexander Grayver) Date: Thu, 15 Sep 2011 19:11:19 +0200 Subject: [petsc-users] Problems with last PETSC-3.2 release on Windows In-Reply-To: References: <4E71AE27.3030600@gfz-potsdam.de> <4E71F742.70202@gfz-potsdam.de> <4E720115.9090600@gfz-potsdam.de> Message-ID: <4E7231B7.1020909@gfz-potsdam.de> Satish, It is reproducible. So that is the nasty error, I don't know where to look for solution. Anyway, I commented these lines in compiler.py as Matt suggested: #if not found: # raise RuntimeError('Cannot determine Fortran module include flag') And configure fails further down, but I suspect that this new error is also somehow linked to the relative paths (seen log attached). Can you confirmthis? Regards, Alexander On 15.09.2011 16:48, Satish Balay wrote: > I just ran petsc-32 with ifort - and it ran fine. So I'm not sure > whats hapenning in this case.. > > Is this error reproduceable [if configure is rerun with the same options?] > > Satish > > > Intel(R) Visual Fortran Compiler for applications running on Intel(R) 64, Version 10.1 Build 20080212 Package ID: w_fc_p_10.1.019 > > ================================================================================ > TEST checkFortranModuleInclude from config.compilers(/home/balay/petsc-3.2/config/BuildSystem/config/compilers.py:1155) > TESTING: checkFortranModuleInclude from config.compilers(config/BuildSystem/config/compilers.py:1155) > Figures out what flag is used to specify the include path for Fortran modules > Pushing language FC > sh: /home/balay/petsc-3.2/bin/win32fe/win32fe ifort -c -o /tmp/petsc-zRoAJW/config.compilers/conftest.o -I/tmp/petsc-zRoAJW/config.compilers -I/tmp/petsc-zRoAJW/config.setCompilers -MT -Z7 -fpp /tmp/petsc-zRoAJW/config.compilers/conftest.F > Executing: /home/balay/petsc-3.2/bin/win32fe/win32fe ifort -c -o /tmp/petsc-zRoAJW/config.compilers/conftest.o -I/tmp/petsc-zRoAJW/config.compilers -I/tmp/petsc-zRoAJW/config.setCompilers -MT -Z7 -fpp /tmp/petsc-zRoAJW/config.compilers/conftest.F > sh: > Successful compile: > Source: > module configtest > integer testint > parameter (testint = 42) > end module configtest > Pushing language FC > sh: /home/balay/petsc-3.2/bin/win32fe/win32fe ifort -c -o /tmp/petsc-zRoAJW/config.compilers/conftest.o -I/tmp/petsc-zRoAJW/config.compilers -I/tmp/petsc-zRoAJW/config.setCompilers -I/tmp/petsc-zRoAJW/config.compilers/confdir -MT -Z7 -fpp /tmp/petsc-zRoAJW/config.compilers/conftest.F > Executing: /home/balay/petsc-3.2/bin/win32fe/win32fe ifort -c -o /tmp/petsc-zRoAJW/config.compilers/conftest.o -I/tmp/petsc-zRoAJW/config.compilers -I/tmp/petsc-zRoAJW/config.setCompilers -I/tmp/petsc-zRoAJW/config.compilers/confdir -MT -Z7 -fpp /tmp/petsc-zRoAJW/config.compilers/conftest.F > sh: > Successful compile: > Source: > program main > use configtest > > write(*,*) testint > > end > Pushing language FC > Popping language FC > sh: /home/balay/petsc-3.2/bin/win32fe/win32fe ifort -o /tmp/petsc-zRoAJW/config.compilers/conftest.exe -I/tmp/petsc-zRoAJW/config.compilers/confdir -MT -Z7 -fpp /tmp/petsc-zRoAJW/config.compilers/conftest.o /tmp/petsc-zRoAJW/config.compilers/configtest.o > Executing: /home/balay/petsc-3.2/bin/win32fe/win32fe ifort -o /tmp/petsc-zRoAJW/config.compilers/conftest.exe -I/tmp/petsc-zRoAJW/config.compilers/confdir -MT -Z7 -fpp /tmp/petsc-zRoAJW/config.compilers/conftest.o /tmp/petsc-zRoAJW/config.compilers/configtest.o > sh: LINK : C:\cygwin\tmp\PETSC-~3\CONFIG~1.COM\conftest.exe not found or not built by the last incremental link; performing full link^M > > compilers: Fortran module include flag -I found > Popping language FC > <<<<<<<<<<<<<<<<<<<<<<< > > > > > > > On Thu, 15 Sep 2011, Matthew Knepley wrote: > >> On Thu, Sep 15, 2011 at 8:43 AM, Alexander Grayver >> wrote: >> >>> ** >>> Thanks Matt, >>> >>> I will that, but I use PETSC from Fortran so I need Fortran functionality. >>> >> Sure. Do you use the F90 modules? If so, send us the build problem. If there >> is not build problem, >> then ifort supports only relative paths for module includes, which is crazy >> and will break sometimes. >> >> Thanks, >> >> Matt >> >> >>> Regards, >>> Alexander >>> >>> >>> On 15.09.2011 15:43, Matthew Knepley wrote: >>> >>> On Thu, Sep 15, 2011 at 8:01 AM, Alexander Grayver< >>> agrayver at gfz-potsdam.de> wrote: >>> >>>> Matthew, >>>> >>>> I found this check in configure.log I obtained for petsc-3.1-p7: >>>> >>>> >>>> ================================================================================ >>>> TEST checkFortranModuleInclude from >>>> config.compilers(/cygdrive/d/Dev/petsc-3.1-p7/config/BuildSystem/config/compilers.py:1084) >>>> TESTING: checkFortranModuleInclude from >>>> config.compilers(config/BuildSystem/config/compilers.py:1084) >>>> Figures out what flag is used to specify the include path for Fortran >>>> modules >>>> Pushing language FC >>>> sh: /cygdrive/d/Dev/petsc-3.1-p7/bin/win32fe/win32fe ifort -c -o >>>> conftest.o -MT -Z7 -fpp conftest.F >>>> Executing: /cygdrive/d/Dev/petsc-3.1-p7/bin/win32fe/win32fe ifort -c -o >>>> conftest.o -MT -Z7 -fpp conftest.F >>>> sh: >>>> Pushing language FC >>>> sh: /cygdrive/d/Dev/petsc-3.1-p7/bin/win32fe/win32fe ifort -c -o >>>> conftest.o -Iconfdir -MT -Z7 -fpp conftest.F >>>> Executing: /cygdrive/d/Dev/petsc-3.1-p7/bin/win32fe/win32fe ifort -c -o >>>> conftest.o -Iconfdir -MT -Z7 -fpp conftest.F >>>> sh: >>>> Pushing language FC >>>> Popping language FC >>>> sh: /cygdrive/d/Dev/petsc-3.1-p7/bin/win32fe/win32fe ifort -o >>>> conftest.exe -Iconfdir -MT -Z7 -fpp conftest.o configtest.o >>>> Executing: /cygdrive/d/Dev/petsc-3.1-p7/bin/win32fe/win32fe ifort -o >>>> conftest.exe -Iconfdir -MT -Z7 -fpp conftest.o configtest.o >>>> sh: LINK : D:\Dev\PETSC-~2.1-P\conftest.exe not found or not built by the >>>> last incremental link; performing full link >>>> >>>> compilers: Fortran module include flag -I found >>>> Popping language FC >>>> >>>> ================================================================================ >>>> >>>> The compiler I used is the same I use for petsc-3.2-p1. >>>> The difference is in the paths where you try to build this tests. As far >>>> as I understand for petsc-3.1-p7 you used local directory and for >>>> petsc-3.2-p1 temp directory is used. >>>> Could it be the reason? >>>> >>> That would be broken behavior from ifort, but I guess it is possible. I >>> will talk with Satish about how >>> this should really be fixed. For right now, go to >>> >>> config/BuildSystem/config/compilers.py >>> >>> to the checkFortranModuleInclude() function and comment out the last two >>> lines, which are killing the configure. >>> Your build will break on the Fortran modules (I assume), but everything >>> else should work. >>> >>> Thanks, >>> >>> Matt >>> >>> >>>> Regards, >>>> Alexander >>>> >>>> >>>> On 15.09.2011 14:54, Matthew Knepley wrote: >>>> >>>> On Thu, Sep 15, 2011 at 2:49 AM, Alexander Grayver< >>>> agrayver at gfz-potsdam.de> wrote: >>>> >>>>> Hello, >>>>> >>>>> I hope you are having nice day! >>>>> >>>>> I have been using petsc-3.1-p7 under windows for 6 months and now decided >>>>> to get new version. I used my old configure line which was successful for >>>>> petsc-3.1: >>>>> >>>>> ./config/configure.py --PETSC_DIR=/cygdrive/d/dev/petsc-3.2-p1 >>>>> --with-petsc-arch=cygwin-cxx-debug-fortran --with-fortran-interfaces=1 >>>>> --with-cc='win32fe cl' --with-fc='win32fe ifort' --with-cxx='win32fe cl' >>>>> --with-windows-graphics=0 --download-f-blas-lapack=1 --with-precision=double >>>>> --with-scalar-type=complex --with-clanguage=cxx >>>>> --with-mpi-include="/cygdrive/d/Dev/HPC_2008_SDK/Include" >>>>> --with-mpi-lib=["/cygdrive/d/Dev/HPC_2008_SDK/Lib/amd64/msmpi.lib","/cygdrive/d/Dev/HPC_2008_SDK/Lib/amd64/msmpifec.lib"] >>>>> --useThreads=0 >>>>> >>>>> But unfortunately it doesn't work properly anymore, even though my >>>>> development environment has not been changed. The error is: >>>>> >>>>> >>>>> ******************************************************************************* >>>>> UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log for >>>>> details): >>>>> >>>>> ------------------------------------------------------------------------------- >>>>> Cannot determine Fortran module include flag >>>>> >>>>> Please find configure.log attached. >>>>> >>>> The problem here is that your Fortran compiler reports that is supports >>>> F90, but cannot >>>> locate modules using an include path. This will make parts of the PETSc >>>> build fail. Do you >>>> know what flag it uses? I try -I, -p, and -M. >>>> >>>> Thanks, >>>> >>>> Matt >>>> >>>> >>>>> Regards, >>>>> Alexander >>>> -- >>>> What most experimenters take for granted before they begin their >>>> experiments is infinitely more interesting than any results to which their >>>> experiments lead. >>>> -- Norbert Wiener >>>> >>>> >>>> >>> >>> -- >>> What most experimenters take for granted before they begin their >>> experiments is infinitely more interesting than any results to which their >>> experiments lead. >>> -- Norbert Wiener >>> >>> >>> >> >> -------------- next part -------------- A non-text attachment was scrubbed... Name: configure.zip Type: application/x-zip-compressed Size: 125439 bytes Desc: not available URL: From balay at mcs.anl.gov Thu Sep 15 12:19:31 2011 From: balay at mcs.anl.gov (Satish Balay) Date: Thu, 15 Sep 2011 12:19:31 -0500 (CDT) Subject: [petsc-users] Problems with last PETSC-3.2 release on Windows In-Reply-To: <4E7231B7.1020909@gfz-potsdam.de> References: <4E71AE27.3030600@gfz-potsdam.de> <4E71F742.70202@gfz-potsdam.de> <4E720115.9090600@gfz-potsdam.de> <4E7231B7.1020909@gfz-potsdam.de> Message-ID: Looks like its related to the the commenting out of the lines [and the failed test before that]. After the commented lines - try adding: self.setCompilers.fortranModuleIncludeFlag = '-I' Satish On Thu, 15 Sep 2011, Alexander Grayver wrote: > Satish, > > It is reproducible. > So that is the nasty error, I don't know where to look for solution. > Anyway, I commented these lines in compiler.py as Matt suggested: > #if not found: > # raise RuntimeError('Cannot determine Fortran module include flag') > > And configure fails further down, but I suspect that this new error is also > somehow linked to the relative paths (seen log attached). Can you confirmthis? > > Regards, > Alexander > > On 15.09.2011 16:48, Satish Balay wrote: > > I just ran petsc-32 with ifort - and it ran fine. So I'm not sure > > whats hapenning in this case.. > > > > Is this error reproduceable [if configure is rerun with the same options?] > > > > Satish > > > > > > Intel(R) Visual Fortran Compiler for applications running on Intel(R) 64, > > Version 10.1 Build 20080212 Package ID: w_fc_p_10.1.019 > > > > ================================================================================ > > TEST checkFortranModuleInclude from > > config.compilers(/home/balay/petsc-3.2/config/BuildSystem/config/compilers.py:1155) > > TESTING: checkFortranModuleInclude from > > config.compilers(config/BuildSystem/config/compilers.py:1155) > > Figures out what flag is used to specify the include path for Fortran > > modules > > Pushing language FC > > sh: /home/balay/petsc-3.2/bin/win32fe/win32fe ifort -c -o > > /tmp/petsc-zRoAJW/config.compilers/conftest.o > > -I/tmp/petsc-zRoAJW/config.compilers -I/tmp/petsc-zRoAJW/config.setCompilers > > -MT -Z7 -fpp /tmp/petsc-zRoAJW/config.compilers/conftest.F > > Executing: /home/balay/petsc-3.2/bin/win32fe/win32fe ifort -c -o > > /tmp/petsc-zRoAJW/config.compilers/conftest.o > > -I/tmp/petsc-zRoAJW/config.compilers -I/tmp/petsc-zRoAJW/config.setCompilers > > -MT -Z7 -fpp /tmp/petsc-zRoAJW/config.compilers/conftest.F > > sh: > > Successful compile: > > Source: > > module configtest > > integer testint > > parameter (testint = 42) > > end module configtest > > Pushing language FC > > sh: /home/balay/petsc-3.2/bin/win32fe/win32fe ifort -c -o > > /tmp/petsc-zRoAJW/config.compilers/conftest.o > > -I/tmp/petsc-zRoAJW/config.compilers -I/tmp/petsc-zRoAJW/config.setCompilers > > -I/tmp/petsc-zRoAJW/config.compilers/confdir -MT -Z7 -fpp > > /tmp/petsc-zRoAJW/config.compilers/conftest.F > > Executing: /home/balay/petsc-3.2/bin/win32fe/win32fe ifort -c -o > > /tmp/petsc-zRoAJW/config.compilers/conftest.o > > -I/tmp/petsc-zRoAJW/config.compilers -I/tmp/petsc-zRoAJW/config.setCompilers > > -I/tmp/petsc-zRoAJW/config.compilers/confdir -MT -Z7 -fpp > > /tmp/petsc-zRoAJW/config.compilers/conftest.F > > sh: > > Successful compile: > > Source: > > program main > > use configtest > > > > write(*,*) testint > > > > end > > Pushing language FC > > Popping language FC > > sh: /home/balay/petsc-3.2/bin/win32fe/win32fe ifort -o > > /tmp/petsc-zRoAJW/config.compilers/conftest.exe > > -I/tmp/petsc-zRoAJW/config.compilers/confdir -MT -Z7 -fpp > > /tmp/petsc-zRoAJW/config.compilers/conftest.o > > /tmp/petsc-zRoAJW/config.compilers/configtest.o > > Executing: /home/balay/petsc-3.2/bin/win32fe/win32fe ifort -o > > /tmp/petsc-zRoAJW/config.compilers/conftest.exe > > -I/tmp/petsc-zRoAJW/config.compilers/confdir -MT -Z7 -fpp > > /tmp/petsc-zRoAJW/config.compilers/conftest.o > > /tmp/petsc-zRoAJW/config.compilers/configtest.o > > sh: LINK : C:\cygwin\tmp\PETSC-~3\CONFIG~1.COM\conftest.exe not found or not > > built by the last incremental link; performing full link^M > > > > compilers: Fortran module include flag -I found > > Popping language FC > > <<<<<<<<<<<<<<<<<<<<<<< > > > > > > > > > > > > > > On Thu, 15 Sep 2011, Matthew Knepley wrote: > > > > > On Thu, Sep 15, 2011 at 8:43 AM, Alexander Grayver > > > wrote: > > > > > > > ** > > > > Thanks Matt, > > > > > > > > I will that, but I use PETSC from Fortran so I need Fortran > > > > functionality. > > > > > > > Sure. Do you use the F90 modules? If so, send us the build problem. If > > > there > > > is not build problem, > > > then ifort supports only relative paths for module includes, which is > > > crazy > > > and will break sometimes. > > > > > > Thanks, > > > > > > Matt > > > > > > > > > > Regards, > > > > Alexander > > > > > > > > > > > > On 15.09.2011 15:43, Matthew Knepley wrote: > > > > > > > > On Thu, Sep 15, 2011 at 8:01 AM, Alexander Grayver< > > > > agrayver at gfz-potsdam.de> wrote: > > > > > > > > > Matthew, > > > > > > > > > > I found this check in configure.log I obtained for petsc-3.1-p7: > > > > > > > > > > > > > > > ================================================================================ > > > > > TEST checkFortranModuleInclude from > > > > > config.compilers(/cygdrive/d/Dev/petsc-3.1-p7/config/BuildSystem/config/compilers.py:1084) > > > > > TESTING: checkFortranModuleInclude from > > > > > config.compilers(config/BuildSystem/config/compilers.py:1084) > > > > > Figures out what flag is used to specify the include path for > > > > > Fortran > > > > > modules > > > > > Pushing language FC > > > > > sh: /cygdrive/d/Dev/petsc-3.1-p7/bin/win32fe/win32fe ifort -c -o > > > > > conftest.o -MT -Z7 -fpp conftest.F > > > > > Executing: /cygdrive/d/Dev/petsc-3.1-p7/bin/win32fe/win32fe ifort -c > > > > > -o > > > > > conftest.o -MT -Z7 -fpp conftest.F > > > > > sh: > > > > > Pushing language FC > > > > > sh: /cygdrive/d/Dev/petsc-3.1-p7/bin/win32fe/win32fe ifort -c -o > > > > > conftest.o -Iconfdir -MT -Z7 -fpp conftest.F > > > > > Executing: /cygdrive/d/Dev/petsc-3.1-p7/bin/win32fe/win32fe ifort -c > > > > > -o > > > > > conftest.o -Iconfdir -MT -Z7 -fpp conftest.F > > > > > sh: > > > > > Pushing language FC > > > > > Popping language FC > > > > > sh: /cygdrive/d/Dev/petsc-3.1-p7/bin/win32fe/win32fe ifort -o > > > > > conftest.exe -Iconfdir -MT -Z7 -fpp conftest.o configtest.o > > > > > Executing: /cygdrive/d/Dev/petsc-3.1-p7/bin/win32fe/win32fe ifort -o > > > > > conftest.exe -Iconfdir -MT -Z7 -fpp conftest.o configtest.o > > > > > sh: LINK : D:\Dev\PETSC-~2.1-P\conftest.exe not found or not built by > > > > > the > > > > > last incremental link; performing full link > > > > > > > > > > compilers: Fortran module include flag -I found > > > > > Popping language FC > > > > > > > > > > ================================================================================ > > > > > > > > > > The compiler I used is the same I use for petsc-3.2-p1. > > > > > The difference is in the paths where you try to build this tests. As > > > > > far > > > > > as I understand for petsc-3.1-p7 you used local directory and for > > > > > petsc-3.2-p1 temp directory is used. > > > > > Could it be the reason? > > > > > > > > > That would be broken behavior from ifort, but I guess it is possible. > > > > I > > > > will talk with Satish about how > > > > this should really be fixed. For right now, go to > > > > > > > > config/BuildSystem/config/compilers.py > > > > > > > > to the checkFortranModuleInclude() function and comment out the last > > > > two > > > > lines, which are killing the configure. > > > > Your build will break on the Fortran modules (I assume), but everything > > > > else should work. > > > > > > > > Thanks, > > > > > > > > Matt > > > > > > > > > > > > > Regards, > > > > > Alexander > > > > > > > > > > > > > > > On 15.09.2011 14:54, Matthew Knepley wrote: > > > > > > > > > > On Thu, Sep 15, 2011 at 2:49 AM, Alexander Grayver< > > > > > agrayver at gfz-potsdam.de> wrote: > > > > > > > > > > > Hello, > > > > > > > > > > > > I hope you are having nice day! > > > > > > > > > > > > I have been using petsc-3.1-p7 under windows for 6 months and now > > > > > > decided > > > > > > to get new version. I used my old configure line which was > > > > > > successful for > > > > > > petsc-3.1: > > > > > > > > > > > > ./config/configure.py --PETSC_DIR=/cygdrive/d/dev/petsc-3.2-p1 > > > > > > --with-petsc-arch=cygwin-cxx-debug-fortran > > > > > > --with-fortran-interfaces=1 > > > > > > --with-cc='win32fe cl' --with-fc='win32fe ifort' > > > > > > --with-cxx='win32fe cl' > > > > > > --with-windows-graphics=0 --download-f-blas-lapack=1 > > > > > > --with-precision=double > > > > > > --with-scalar-type=complex --with-clanguage=cxx > > > > > > --with-mpi-include="/cygdrive/d/Dev/HPC_2008_SDK/Include" > > > > > > > > > > > > --with-mpi-lib=["/cygdrive/d/Dev/HPC_2008_SDK/Lib/amd64/msmpi.lib","/cygdrive/d/Dev/HPC_2008_SDK/Lib/amd64/msmpifec.lib"] > > > > > > --useThreads=0 > > > > > > > > > > > > But unfortunately it doesn't work properly anymore, even though my > > > > > > development environment has not been changed. The error is: > > > > > > > > > > > > > > > > > > ******************************************************************************* > > > > > > UNABLE to CONFIGURE with GIVEN OPTIONS (see > > > > > > configure.log for > > > > > > details): > > > > > > > > > > > > > > > > > > ------------------------------------------------------------------------------- > > > > > > Cannot determine Fortran module include flag > > > > > > > > > > > > Please find configure.log attached. > > > > > > > > > > > The problem here is that your Fortran compiler reports that is > > > > > supports > > > > > F90, but cannot > > > > > locate modules using an include path. This will make parts of the > > > > > PETSc > > > > > build fail. Do you > > > > > know what flag it uses? I try -I, -p, and -M. > > > > > > > > > > Thanks, > > > > > > > > > > Matt > > > > > > > > > > > > > > > > Regards, > > > > > > Alexander > > > > > -- > > > > > What most experimenters take for granted before they begin their > > > > > experiments is infinitely more interesting than any results to which > > > > > their > > > > > experiments lead. > > > > > -- Norbert Wiener > > > > > > > > > > > > > > > > > > > > > > > -- > > > > What most experimenters take for granted before they begin their > > > > experiments is infinitely more interesting than any results to which > > > > their > > > > experiments lead. > > > > -- Norbert Wiener > > > > > > > > > > > > > > > > > > > > From agrayver at gfz-potsdam.de Fri Sep 16 05:33:58 2011 From: agrayver at gfz-potsdam.de (Alexander Grayver) Date: Fri, 16 Sep 2011 12:33:58 +0200 Subject: [petsc-users] Problems with last PETSC-3.2 release on Windows In-Reply-To: References: <4E71AE27.3030600@gfz-potsdam.de> <4E71F742.70202@gfz-potsdam.de> <4E720115.9090600@gfz-potsdam.de> <4E7231B7.1020909@gfz-potsdam.de> Message-ID: <4E732616.1070506@gfz-potsdam.de> Satish, Thanks a lot! The configure and make stages work well now. Regards, Alexander On 15.09.2011 19:19, Satish Balay wrote: > Looks like its related to the the commenting out of the lines [and the > failed test before that]. > > After the commented lines - try adding: > self.setCompilers.fortranModuleIncludeFlag = '-I' > > Satish > > On Thu, 15 Sep 2011, Alexander Grayver wrote: > >> Satish, >> >> It is reproducible. >> So that is the nasty error, I don't know where to look for solution. >> Anyway, I commented these lines in compiler.py as Matt suggested: >> #if not found: >> # raise RuntimeError('Cannot determine Fortran module include flag') >> >> And configure fails further down, but I suspect that this new error is also >> somehow linked to the relative paths (seen log attached). Can you confirmthis? >> >> Regards, >> Alexander >> >> On 15.09.2011 16:48, Satish Balay wrote: >>> I just ran petsc-32 with ifort - and it ran fine. So I'm not sure >>> whats hapenning in this case.. >>> >>> Is this error reproduceable [if configure is rerun with the same options?] >>> >>> Satish >>> >>> >>> Intel(R) Visual Fortran Compiler for applications running on Intel(R) 64, >>> Version 10.1 Build 20080212 Package ID: w_fc_p_10.1.019 >>> >>> ================================================================================ >>> TEST checkFortranModuleInclude from >>> config.compilers(/home/balay/petsc-3.2/config/BuildSystem/config/compilers.py:1155) >>> TESTING: checkFortranModuleInclude from >>> config.compilers(config/BuildSystem/config/compilers.py:1155) >>> Figures out what flag is used to specify the include path for Fortran >>> modules >>> Pushing language FC >>> sh: /home/balay/petsc-3.2/bin/win32fe/win32fe ifort -c -o >>> /tmp/petsc-zRoAJW/config.compilers/conftest.o >>> -I/tmp/petsc-zRoAJW/config.compilers -I/tmp/petsc-zRoAJW/config.setCompilers >>> -MT -Z7 -fpp /tmp/petsc-zRoAJW/config.compilers/conftest.F >>> Executing: /home/balay/petsc-3.2/bin/win32fe/win32fe ifort -c -o >>> /tmp/petsc-zRoAJW/config.compilers/conftest.o >>> -I/tmp/petsc-zRoAJW/config.compilers -I/tmp/petsc-zRoAJW/config.setCompilers >>> -MT -Z7 -fpp /tmp/petsc-zRoAJW/config.compilers/conftest.F >>> sh: >>> Successful compile: >>> Source: >>> module configtest >>> integer testint >>> parameter (testint = 42) >>> end module configtest >>> Pushing language FC >>> sh: /home/balay/petsc-3.2/bin/win32fe/win32fe ifort -c -o >>> /tmp/petsc-zRoAJW/config.compilers/conftest.o >>> -I/tmp/petsc-zRoAJW/config.compilers -I/tmp/petsc-zRoAJW/config.setCompilers >>> -I/tmp/petsc-zRoAJW/config.compilers/confdir -MT -Z7 -fpp >>> /tmp/petsc-zRoAJW/config.compilers/conftest.F >>> Executing: /home/balay/petsc-3.2/bin/win32fe/win32fe ifort -c -o >>> /tmp/petsc-zRoAJW/config.compilers/conftest.o >>> -I/tmp/petsc-zRoAJW/config.compilers -I/tmp/petsc-zRoAJW/config.setCompilers >>> -I/tmp/petsc-zRoAJW/config.compilers/confdir -MT -Z7 -fpp >>> /tmp/petsc-zRoAJW/config.compilers/conftest.F >>> sh: >>> Successful compile: >>> Source: >>> program main >>> use configtest >>> >>> write(*,*) testint >>> >>> end >>> Pushing language FC >>> Popping language FC >>> sh: /home/balay/petsc-3.2/bin/win32fe/win32fe ifort -o >>> /tmp/petsc-zRoAJW/config.compilers/conftest.exe >>> -I/tmp/petsc-zRoAJW/config.compilers/confdir -MT -Z7 -fpp >>> /tmp/petsc-zRoAJW/config.compilers/conftest.o >>> /tmp/petsc-zRoAJW/config.compilers/configtest.o >>> Executing: /home/balay/petsc-3.2/bin/win32fe/win32fe ifort -o >>> /tmp/petsc-zRoAJW/config.compilers/conftest.exe >>> -I/tmp/petsc-zRoAJW/config.compilers/confdir -MT -Z7 -fpp >>> /tmp/petsc-zRoAJW/config.compilers/conftest.o >>> /tmp/petsc-zRoAJW/config.compilers/configtest.o >>> sh: LINK : C:\cygwin\tmp\PETSC-~3\CONFIG~1.COM\conftest.exe not found or not >>> built by the last incremental link; performing full link^M >>> >>> compilers: Fortran module include flag -I found >>> Popping language FC >>> <<<<<<<<<<<<<<<<<<<<<<< >>> >>> >>> >>> >>> >>> >>> On Thu, 15 Sep 2011, Matthew Knepley wrote: >>> >>>> On Thu, Sep 15, 2011 at 8:43 AM, Alexander Grayver >>>> wrote: >>>> >>>>> ** >>>>> Thanks Matt, >>>>> >>>>> I will that, but I use PETSC from Fortran so I need Fortran >>>>> functionality. >>>>> >>>> Sure. Do you use the F90 modules? If so, send us the build problem. If >>>> there >>>> is not build problem, >>>> then ifort supports only relative paths for module includes, which is >>>> crazy >>>> and will break sometimes. >>>> >>>> Thanks, >>>> >>>> Matt >>>> >>>> >>>>> Regards, >>>>> Alexander >>>>> >>>>> >>>>> On 15.09.2011 15:43, Matthew Knepley wrote: >>>>> >>>>> On Thu, Sep 15, 2011 at 8:01 AM, Alexander Grayver< >>>>> agrayver at gfz-potsdam.de> wrote: >>>>> >>>>>> Matthew, >>>>>> >>>>>> I found this check in configure.log I obtained for petsc-3.1-p7: >>>>>> >>>>>> >>>>>> ================================================================================ >>>>>> TEST checkFortranModuleInclude from >>>>>> config.compilers(/cygdrive/d/Dev/petsc-3.1-p7/config/BuildSystem/config/compilers.py:1084) >>>>>> TESTING: checkFortranModuleInclude from >>>>>> config.compilers(config/BuildSystem/config/compilers.py:1084) >>>>>> Figures out what flag is used to specify the include path for >>>>>> Fortran >>>>>> modules >>>>>> Pushing language FC >>>>>> sh: /cygdrive/d/Dev/petsc-3.1-p7/bin/win32fe/win32fe ifort -c -o >>>>>> conftest.o -MT -Z7 -fpp conftest.F >>>>>> Executing: /cygdrive/d/Dev/petsc-3.1-p7/bin/win32fe/win32fe ifort -c >>>>>> -o >>>>>> conftest.o -MT -Z7 -fpp conftest.F >>>>>> sh: >>>>>> Pushing language FC >>>>>> sh: /cygdrive/d/Dev/petsc-3.1-p7/bin/win32fe/win32fe ifort -c -o >>>>>> conftest.o -Iconfdir -MT -Z7 -fpp conftest.F >>>>>> Executing: /cygdrive/d/Dev/petsc-3.1-p7/bin/win32fe/win32fe ifort -c >>>>>> -o >>>>>> conftest.o -Iconfdir -MT -Z7 -fpp conftest.F >>>>>> sh: >>>>>> Pushing language FC >>>>>> Popping language FC >>>>>> sh: /cygdrive/d/Dev/petsc-3.1-p7/bin/win32fe/win32fe ifort -o >>>>>> conftest.exe -Iconfdir -MT -Z7 -fpp conftest.o configtest.o >>>>>> Executing: /cygdrive/d/Dev/petsc-3.1-p7/bin/win32fe/win32fe ifort -o >>>>>> conftest.exe -Iconfdir -MT -Z7 -fpp conftest.o configtest.o >>>>>> sh: LINK : D:\Dev\PETSC-~2.1-P\conftest.exe not found or not built by >>>>>> the >>>>>> last incremental link; performing full link >>>>>> >>>>>> compilers: Fortran module include flag -I found >>>>>> Popping language FC >>>>>> >>>>>> ================================================================================ >>>>>> >>>>>> The compiler I used is the same I use for petsc-3.2-p1. >>>>>> The difference is in the paths where you try to build this tests. As >>>>>> far >>>>>> as I understand for petsc-3.1-p7 you used local directory and for >>>>>> petsc-3.2-p1 temp directory is used. >>>>>> Could it be the reason? >>>>>> >>>>> That would be broken behavior from ifort, but I guess it is possible. >>>>> I >>>>> will talk with Satish about how >>>>> this should really be fixed. For right now, go to >>>>> >>>>> config/BuildSystem/config/compilers.py >>>>> >>>>> to the checkFortranModuleInclude() function and comment out the last >>>>> two >>>>> lines, which are killing the configure. >>>>> Your build will break on the Fortran modules (I assume), but everything >>>>> else should work. >>>>> >>>>> Thanks, >>>>> >>>>> Matt >>>>> >>>>> >>>>>> Regards, >>>>>> Alexander >>>>>> >>>>>> >>>>>> On 15.09.2011 14:54, Matthew Knepley wrote: >>>>>> >>>>>> On Thu, Sep 15, 2011 at 2:49 AM, Alexander Grayver< >>>>>> agrayver at gfz-potsdam.de> wrote: >>>>>> >>>>>>> Hello, >>>>>>> >>>>>>> I hope you are having nice day! >>>>>>> >>>>>>> I have been using petsc-3.1-p7 under windows for 6 months and now >>>>>>> decided >>>>>>> to get new version. I used my old configure line which was >>>>>>> successful for >>>>>>> petsc-3.1: >>>>>>> >>>>>>> ./config/configure.py --PETSC_DIR=/cygdrive/d/dev/petsc-3.2-p1 >>>>>>> --with-petsc-arch=cygwin-cxx-debug-fortran >>>>>>> --with-fortran-interfaces=1 >>>>>>> --with-cc='win32fe cl' --with-fc='win32fe ifort' >>>>>>> --with-cxx='win32fe cl' >>>>>>> --with-windows-graphics=0 --download-f-blas-lapack=1 >>>>>>> --with-precision=double >>>>>>> --with-scalar-type=complex --with-clanguage=cxx >>>>>>> --with-mpi-include="/cygdrive/d/Dev/HPC_2008_SDK/Include" >>>>>>> >>>>>>> --with-mpi-lib=["/cygdrive/d/Dev/HPC_2008_SDK/Lib/amd64/msmpi.lib","/cygdrive/d/Dev/HPC_2008_SDK/Lib/amd64/msmpifec.lib"] >>>>>>> --useThreads=0 >>>>>>> >>>>>>> But unfortunately it doesn't work properly anymore, even though my >>>>>>> development environment has not been changed. The error is: >>>>>>> >>>>>>> >>>>>>> ******************************************************************************* >>>>>>> UNABLE to CONFIGURE with GIVEN OPTIONS (see >>>>>>> configure.log for >>>>>>> details): >>>>>>> >>>>>>> >>>>>>> ------------------------------------------------------------------------------- >>>>>>> Cannot determine Fortran module include flag >>>>>>> >>>>>>> Please find configure.log attached. >>>>>>> >>>>>> The problem here is that your Fortran compiler reports that is >>>>>> supports >>>>>> F90, but cannot >>>>>> locate modules using an include path. This will make parts of the >>>>>> PETSc >>>>>> build fail. Do you >>>>>> know what flag it uses? I try -I, -p, and -M. >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Matt >>>>>> >>>>>> >>>>>>> Regards, >>>>>>> Alexander >>>>>> -- >>>>>> What most experimenters take for granted before they begin their >>>>>> experiments is infinitely more interesting than any results to which >>>>>> their >>>>>> experiments lead. >>>>>> -- Norbert Wiener >>>>>> >>>>>> >>>>>> >>>>> -- >>>>> What most experimenters take for granted before they begin their >>>>> experiments is infinitely more interesting than any results to which >>>>> their >>>>> experiments lead. >>>>> -- Norbert Wiener >>>>> >>>>> >>>>> >>>> >> From agrayver at gfz-potsdam.de Fri Sep 16 05:38:53 2011 From: agrayver at gfz-potsdam.de (Alexander Grayver) Date: Fri, 16 Sep 2011 12:38:53 +0200 Subject: [petsc-users] [petsc-3.2] MatView on dense matrix Message-ID: <4E73273D.4060601@gfz-potsdam.de> Hello, With petsc-3.1 I used this code to output MPI dense matrix: PetscViewer :: dviewer call PetscViewerBinaryOpen(MPI_COMM_WORLD,'A.dat',FILE_MODE_WRITE,dviewer,ierr); CHKERRQ(ierr) call PetscViewerSetFormat(dviewer,PETSC_VIEWER_NATIVE,ierr); CHKERRQ(ierr) call MatView(A,dviewer,ierr); CHKERRQ(ierr) call PetscViewerDestroy(dviewer,ierr); CHKERRQ(ierr) With petsc-3.2 this code fails with error: [7]PETSC ERROR: --------------------- Error Message ------------------------------------ [7]PETSC ERROR: No support for this operation for this object type! [7]PETSC ERROR: To store a parallel dense matrix you must first call PetscViewerSetFormat(viewer,PETSC_VIEWER_NATIVE)! [7]PETSC ERROR: ------------------------------------------------------------------------ [7]PETSC ERROR: Petsc Release Version 3.2.0, Patch 1, Mon Sep 12 16:01:51 CDT 2011 [7]PETSC ERROR: See docs/changes/index.html for recent updates. [7]PETSC ERROR: See docs/faq.html for hints about trouble shooting. [7]PETSC ERROR: See docs/index.html for manual pages. [7]PETSC ERROR: ------------------------------------------------------------------------ [7]PETSC ERROR: /home/run/test on a openmpi-i named node205 by user Fri Sep 16 09:17:13 2011 [7]PETSC ERROR: Libraries linked from /home/lib/petsc-3.2-p1/openmpi-intel-complex-debug-f/lib [7]PETSC ERROR: Configure run at Wed Sep 14 14:32:55 2011 [7]PETSC ERROR: Configure options --with-petsc-arch=openmpi-intel-complex-debug-f --with-fortran-interfaces=1 --with-mpi-dir=/opt/mpi/intel/openmpi-1.4.2 --with-scalar-type=complex --with-blas-lapack-dir=/opt/intel/Compiler/11.1/072/mkl/lib/em64t --with-precision=double --with-x=0 [7]PETSC ERROR: ------------------------------------------------------------------------ [7]PETSC ERROR: MatView_MPIDense_Binary() line 667 in /home/lib/petsc-3.2-p1/src/mat/impls/dense/mpi/mpidense.c [7]PETSC ERROR: MatView_MPIDense() line 789 in /home/lib/petsc-3.2-p1/src/mat/impls/dense/mpi/mpidense.c [7]PETSC ERROR: MatView() line 757 in /home/lib/petsc-3.2-p1/src/mat/interface/matrix.c MatView_MPIDense() line 789 in /home/lib/petsc-3.2-p1/src/mat/impls/dense/mpi/mpidense.c See full error attached. Is there something new in petsc-3.2 concerning matrix output? Regards, Alexander -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: error.txt URL: From gianmail at gmail.com Fri Sep 16 07:17:21 2011 From: gianmail at gmail.com (Gianluca Meneghello) Date: Fri, 16 Sep 2011 14:17:21 +0200 Subject: [petsc-users] viewers and nohup (unrelated questions) Message-ID: Hi, I would like to ask you 2 unrelated questions. #_1 The first one concerns PETSc viewer. In my code I'm writing outputs with PETSC_VIEWER_ASCII_VTK, more precisely PetscViewer viewer; PetscViewerASCIIOpen(PETSC_COMM_SELF,filename.c_str(),&viewer); PetscViewerSetFormat(viewer,PETSC_VIEWER_ASCII_VTK); DMView(dau,viewer); VecView(foou,viewer); VecView(foop,viewer); PetscViewerDestroy(&viewer); where foou and foop (velocity and pressure respectively) are vectors extracted from the vector containing the full solution of my problem. The VTK so obtained nicely contains a vector field ? from foou ? and a scalar field ? foop ? and can be read by Paraview. Nonetheless, I guess writing (and reading from Paraview) ASCII files is not the best choice. Are there better possibilities in PETSc? Is there a PETSC_VIEWER_BINARY_VTK or equivalent? #_2 The second is about the nohup command. If I run my code with "nohup ./main &" and then exit the console, the program dies as if there was no nohup command in front of it. The same is true for some examples I've tested from the PETSc library. I guess this is related to the machine I'm using more than to the library but... has anyone of you already experienced a similar issue? Any advice is welcome and please let me know if you need any information that can help. Thanks Gianluca -- "[Je pense que] l'homme est un monde qui vaut des fois les mondes et que les plus ardentes ambitions sont celles qui ont eu l'orgueil de l'Anonymat" -- Non omnibus, sed mihi et tibi Amedeo Modigliani From balay at mcs.anl.gov Fri Sep 16 07:33:18 2011 From: balay at mcs.anl.gov (Satish Balay) Date: Fri, 16 Sep 2011 07:33:18 -0500 (CDT) Subject: [petsc-users] Problems with last PETSC-3.2 release on Windows In-Reply-To: <4E732616.1070506@gfz-potsdam.de> References: <4E71AE27.3030600@gfz-potsdam.de> <4E71F742.70202@gfz-potsdam.de> <4E720115.9090600@gfz-potsdam.de> <4E7231B7.1020909@gfz-potsdam.de> <4E732616.1070506@gfz-potsdam.de> Message-ID: Could you tell us - which version of ifort you have? Its strange that this test works for me [with ifort 10.1] - but not for you. Satish On Fri, 16 Sep 2011, Alexander Grayver wrote: > Satish, > > Thanks a lot! The configure and make stages work well now. > > Regards, > Alexander > > On 15.09.2011 19:19, Satish Balay wrote: > > Looks like its related to the the commenting out of the lines [and the > > failed test before that]. > > > > After the commented lines - try adding: > > self.setCompilers.fortranModuleIncludeFlag = '-I' > > > > Satish > > > > On Thu, 15 Sep 2011, Alexander Grayver wrote: > > > > > Satish, > > > > > > It is reproducible. > > > So that is the nasty error, I don't know where to look for solution. > > > Anyway, I commented these lines in compiler.py as Matt suggested: > > > #if not found: > > > # raise RuntimeError('Cannot determine Fortran module include flag') > > > > > > And configure fails further down, but I suspect that this new error is > > > also > > > somehow linked to the relative paths (seen log attached). Can you > > > confirmthis? > > > > > > Regards, > > > Alexander > > > > > > On 15.09.2011 16:48, Satish Balay wrote: > > > > I just ran petsc-32 with ifort - and it ran fine. So I'm not sure > > > > whats hapenning in this case.. > > > > > > > > Is this error reproduceable [if configure is rerun with the same > > > > options?] > > > > > > > > Satish > > > > > > > > > > > > Intel(R) Visual Fortran Compiler for applications running on Intel(R) > > > > 64, > > > > Version 10.1 Build 20080212 Package ID: w_fc_p_10.1.019 > > > > > > > > ================================================================================ > > > > TEST checkFortranModuleInclude from > > > > config.compilers(/home/balay/petsc-3.2/config/BuildSystem/config/compilers.py:1155) > > > > TESTING: checkFortranModuleInclude from > > > > config.compilers(config/BuildSystem/config/compilers.py:1155) > > > > Figures out what flag is used to specify the include path for > > > > Fortran > > > > modules > > > > Pushing language FC > > > > sh: /home/balay/petsc-3.2/bin/win32fe/win32fe ifort -c -o > > > > /tmp/petsc-zRoAJW/config.compilers/conftest.o > > > > -I/tmp/petsc-zRoAJW/config.compilers > > > > -I/tmp/petsc-zRoAJW/config.setCompilers > > > > -MT -Z7 -fpp /tmp/petsc-zRoAJW/config.compilers/conftest.F > > > > Executing: /home/balay/petsc-3.2/bin/win32fe/win32fe ifort -c -o > > > > /tmp/petsc-zRoAJW/config.compilers/conftest.o > > > > -I/tmp/petsc-zRoAJW/config.compilers > > > > -I/tmp/petsc-zRoAJW/config.setCompilers > > > > -MT -Z7 -fpp /tmp/petsc-zRoAJW/config.compilers/conftest.F > > > > sh: > > > > Successful compile: > > > > Source: > > > > module configtest > > > > integer testint > > > > parameter (testint = 42) > > > > end module configtest > > > > Pushing language FC > > > > sh: /home/balay/petsc-3.2/bin/win32fe/win32fe ifort -c -o > > > > /tmp/petsc-zRoAJW/config.compilers/conftest.o > > > > -I/tmp/petsc-zRoAJW/config.compilers > > > > -I/tmp/petsc-zRoAJW/config.setCompilers > > > > -I/tmp/petsc-zRoAJW/config.compilers/confdir -MT -Z7 -fpp > > > > /tmp/petsc-zRoAJW/config.compilers/conftest.F > > > > Executing: /home/balay/petsc-3.2/bin/win32fe/win32fe ifort -c -o > > > > /tmp/petsc-zRoAJW/config.compilers/conftest.o > > > > -I/tmp/petsc-zRoAJW/config.compilers > > > > -I/tmp/petsc-zRoAJW/config.setCompilers > > > > -I/tmp/petsc-zRoAJW/config.compilers/confdir -MT -Z7 -fpp > > > > /tmp/petsc-zRoAJW/config.compilers/conftest.F > > > > sh: > > > > Successful compile: > > > > Source: > > > > program main > > > > use configtest > > > > > > > > write(*,*) testint > > > > > > > > end > > > > Pushing language FC > > > > Popping language FC > > > > sh: /home/balay/petsc-3.2/bin/win32fe/win32fe ifort -o > > > > /tmp/petsc-zRoAJW/config.compilers/conftest.exe > > > > -I/tmp/petsc-zRoAJW/config.compilers/confdir -MT -Z7 -fpp > > > > /tmp/petsc-zRoAJW/config.compilers/conftest.o > > > > /tmp/petsc-zRoAJW/config.compilers/configtest.o > > > > Executing: /home/balay/petsc-3.2/bin/win32fe/win32fe ifort -o > > > > /tmp/petsc-zRoAJW/config.compilers/conftest.exe > > > > -I/tmp/petsc-zRoAJW/config.compilers/confdir -MT -Z7 -fpp > > > > /tmp/petsc-zRoAJW/config.compilers/conftest.o > > > > /tmp/petsc-zRoAJW/config.compilers/configtest.o > > > > sh: LINK : C:\cygwin\tmp\PETSC-~3\CONFIG~1.COM\conftest.exe not found or > > > > not > > > > built by the last incremental link; performing full link^M > > > > > > > > compilers: Fortran module include flag -I found > > > > Popping language FC > > > > <<<<<<<<<<<<<<<<<<<<<<< > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, 15 Sep 2011, Matthew Knepley wrote: > > > > > > > > > On Thu, Sep 15, 2011 at 8:43 AM, Alexander Grayver > > > > > wrote: > > > > > > > > > > > ** > > > > > > Thanks Matt, > > > > > > > > > > > > I will that, but I use PETSC from Fortran so I need Fortran > > > > > > functionality. > > > > > > > > > > > Sure. Do you use the F90 modules? If so, send us the build problem. If > > > > > there > > > > > is not build problem, > > > > > then ifort supports only relative paths for module includes, which is > > > > > crazy > > > > > and will break sometimes. > > > > > > > > > > Thanks, > > > > > > > > > > Matt > > > > > > > > > > > > > > > > Regards, > > > > > > Alexander > > > > > > > > > > > > > > > > > > On 15.09.2011 15:43, Matthew Knepley wrote: > > > > > > > > > > > > On Thu, Sep 15, 2011 at 8:01 AM, Alexander Grayver< > > > > > > agrayver at gfz-potsdam.de> wrote: > > > > > > > > > > > > > Matthew, > > > > > > > > > > > > > > I found this check in configure.log I obtained for petsc-3.1-p7: > > > > > > > > > > > > > > > > > > > > > ================================================================================ > > > > > > > TEST checkFortranModuleInclude from > > > > > > > config.compilers(/cygdrive/d/Dev/petsc-3.1-p7/config/BuildSystem/config/compilers.py:1084) > > > > > > > TESTING: checkFortranModuleInclude from > > > > > > > config.compilers(config/BuildSystem/config/compilers.py:1084) > > > > > > > Figures out what flag is used to specify the include path for > > > > > > > Fortran > > > > > > > modules > > > > > > > Pushing language FC > > > > > > > sh: /cygdrive/d/Dev/petsc-3.1-p7/bin/win32fe/win32fe ifort -c -o > > > > > > > conftest.o -MT -Z7 -fpp conftest.F > > > > > > > Executing: /cygdrive/d/Dev/petsc-3.1-p7/bin/win32fe/win32fe ifort > > > > > > > -c > > > > > > > -o > > > > > > > conftest.o -MT -Z7 -fpp conftest.F > > > > > > > sh: > > > > > > > Pushing language FC > > > > > > > sh: /cygdrive/d/Dev/petsc-3.1-p7/bin/win32fe/win32fe ifort -c -o > > > > > > > conftest.o -Iconfdir -MT -Z7 -fpp conftest.F > > > > > > > Executing: /cygdrive/d/Dev/petsc-3.1-p7/bin/win32fe/win32fe ifort > > > > > > > -c > > > > > > > -o > > > > > > > conftest.o -Iconfdir -MT -Z7 -fpp conftest.F > > > > > > > sh: > > > > > > > Pushing language FC > > > > > > > Popping language FC > > > > > > > sh: /cygdrive/d/Dev/petsc-3.1-p7/bin/win32fe/win32fe ifort -o > > > > > > > conftest.exe -Iconfdir -MT -Z7 -fpp conftest.o configtest.o > > > > > > > Executing: /cygdrive/d/Dev/petsc-3.1-p7/bin/win32fe/win32fe ifort > > > > > > > -o > > > > > > > conftest.exe -Iconfdir -MT -Z7 -fpp conftest.o configtest.o > > > > > > > sh: LINK : D:\Dev\PETSC-~2.1-P\conftest.exe not found or not built > > > > > > > by > > > > > > > the > > > > > > > last incremental link; performing full link > > > > > > > > > > > > > > compilers: Fortran module include flag -I found > > > > > > > Popping language FC > > > > > > > > > > > > > > ================================================================================ > > > > > > > > > > > > > > The compiler I used is the same I use for petsc-3.2-p1. > > > > > > > The difference is in the paths where you try to build this tests. > > > > > > > As > > > > > > > far > > > > > > > as I understand for petsc-3.1-p7 you used local directory and for > > > > > > > petsc-3.2-p1 temp directory is used. > > > > > > > Could it be the reason? > > > > > > > > > > > > > That would be broken behavior from ifort, but I guess it is > > > > > > possible. > > > > > > I > > > > > > will talk with Satish about how > > > > > > this should really be fixed. For right now, go to > > > > > > > > > > > > config/BuildSystem/config/compilers.py > > > > > > > > > > > > to the checkFortranModuleInclude() function and comment out the > > > > > > last > > > > > > two > > > > > > lines, which are killing the configure. > > > > > > Your build will break on the Fortran modules (I assume), but > > > > > > everything > > > > > > else should work. > > > > > > > > > > > > Thanks, > > > > > > > > > > > > Matt > > > > > > > > > > > > > > > > > > > Regards, > > > > > > > Alexander > > > > > > > > > > > > > > > > > > > > > On 15.09.2011 14:54, Matthew Knepley wrote: > > > > > > > > > > > > > > On Thu, Sep 15, 2011 at 2:49 AM, Alexander Grayver< > > > > > > > agrayver at gfz-potsdam.de> wrote: > > > > > > > > > > > > > > > Hello, > > > > > > > > > > > > > > > > I hope you are having nice day! > > > > > > > > > > > > > > > > I have been using petsc-3.1-p7 under windows for 6 months and > > > > > > > > now > > > > > > > > decided > > > > > > > > to get new version. I used my old configure line which was > > > > > > > > successful for > > > > > > > > petsc-3.1: > > > > > > > > > > > > > > > > ./config/configure.py --PETSC_DIR=/cygdrive/d/dev/petsc-3.2-p1 > > > > > > > > --with-petsc-arch=cygwin-cxx-debug-fortran > > > > > > > > --with-fortran-interfaces=1 > > > > > > > > --with-cc='win32fe cl' --with-fc='win32fe ifort' > > > > > > > > --with-cxx='win32fe cl' > > > > > > > > --with-windows-graphics=0 --download-f-blas-lapack=1 > > > > > > > > --with-precision=double > > > > > > > > --with-scalar-type=complex --with-clanguage=cxx > > > > > > > > --with-mpi-include="/cygdrive/d/Dev/HPC_2008_SDK/Include" > > > > > > > > > > > > > > > > > > > > > > > > --with-mpi-lib=["/cygdrive/d/Dev/HPC_2008_SDK/Lib/amd64/msmpi.lib","/cygdrive/d/Dev/HPC_2008_SDK/Lib/amd64/msmpifec.lib"] > > > > > > > > --useThreads=0 > > > > > > > > > > > > > > > > But unfortunately it doesn't work properly anymore, even though > > > > > > > > my > > > > > > > > development environment has not been changed. The error is: > > > > > > > > > > > > > > > > > > > > > > > > ******************************************************************************* > > > > > > > > UNABLE to CONFIGURE with GIVEN OPTIONS (see > > > > > > > > configure.log for > > > > > > > > details): > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > ------------------------------------------------------------------------------- > > > > > > > > Cannot determine Fortran module include flag > > > > > > > > > > > > > > > > Please find configure.log attached. > > > > > > > > > > > > > > > The problem here is that your Fortran compiler reports that is > > > > > > > supports > > > > > > > F90, but cannot > > > > > > > locate modules using an include path. This will make parts of the > > > > > > > PETSc > > > > > > > build fail. Do you > > > > > > > know what flag it uses? I try -I, -p, and -M. > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > Matt > > > > > > > > > > > > > > > > > > > > > > Regards, > > > > > > > > Alexander > > > > > > > -- > > > > > > > What most experimenters take for granted before they begin their > > > > > > > experiments is infinitely more interesting than any results to > > > > > > > which > > > > > > > their > > > > > > > experiments lead. > > > > > > > -- Norbert Wiener > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > What most experimenters take for granted before they begin their > > > > > > experiments is infinitely more interesting than any results to which > > > > > > their > > > > > > experiments lead. > > > > > > -- Norbert Wiener > > > > > > > > > > > > > > > > > > > > > > > > > > > > From agrayver at gfz-potsdam.de Fri Sep 16 07:40:07 2011 From: agrayver at gfz-potsdam.de (Alexander Grayver) Date: Fri, 16 Sep 2011 14:40:07 +0200 Subject: [petsc-users] Problems with last PETSC-3.2 release on Windows In-Reply-To: References: <4E71AE27.3030600@gfz-potsdam.de> <4E71F742.70202@gfz-potsdam.de> <4E720115.9090600@gfz-potsdam.de> <4E7231B7.1020909@gfz-potsdam.de> <4E732616.1070506@gfz-potsdam.de> Message-ID: <4E7343A7.6060500@gfz-potsdam.de> Satish, A also have 10.1: alex at group /cygdrive/d/dev/petsc-3.2-p1 # ifort Intel(R) Visual Fortran Compiler for applications running on Intel(R) 64, Version 10.1 Build 20080801 Package ID: w_fc_p_10.1.025 Copyright (C) 1985-2008 Intel Corporation. All rights reserved. Regards, Alexander On 16.09.2011 14:33, Satish Balay wrote: > Could you tell us - which version of ifort you have? Its strange > that this test works for me [with ifort 10.1] - but not for you. > > Satish > > On Fri, 16 Sep 2011, Alexander Grayver wrote: > >> Satish, >> >> Thanks a lot! The configure and make stages work well now. >> >> Regards, >> Alexander >> >> On 15.09.2011 19:19, Satish Balay wrote: >>> Looks like its related to the the commenting out of the lines [and the >>> failed test before that]. >>> >>> After the commented lines - try adding: >>> self.setCompilers.fortranModuleIncludeFlag = '-I' >>> >>> Satish >>> >>> On Thu, 15 Sep 2011, Alexander Grayver wrote: >>> >>>> Satish, >>>> >>>> It is reproducible. >>>> So that is the nasty error, I don't know where to look for solution. >>>> Anyway, I commented these lines in compiler.py as Matt suggested: >>>> #if not found: >>>> # raise RuntimeError('Cannot determine Fortran module include flag') >>>> >>>> And configure fails further down, but I suspect that this new error is >>>> also >>>> somehow linked to the relative paths (seen log attached). Can you >>>> confirmthis? >>>> >>>> Regards, >>>> Alexander >>>> >>>> On 15.09.2011 16:48, Satish Balay wrote: >>>>> I just ran petsc-32 with ifort - and it ran fine. So I'm not sure >>>>> whats hapenning in this case.. >>>>> >>>>> Is this error reproduceable [if configure is rerun with the same >>>>> options?] >>>>> >>>>> Satish >>>>> >>>>> >>>>> Intel(R) Visual Fortran Compiler for applications running on Intel(R) >>>>> 64, >>>>> Version 10.1 Build 20080212 Package ID: w_fc_p_10.1.019 >>>>> >>>>> ================================================================================ >>>>> TEST checkFortranModuleInclude from >>>>> config.compilers(/home/balay/petsc-3.2/config/BuildSystem/config/compilers.py:1155) >>>>> TESTING: checkFortranModuleInclude from >>>>> config.compilers(config/BuildSystem/config/compilers.py:1155) >>>>> Figures out what flag is used to specify the include path for >>>>> Fortran >>>>> modules >>>>> Pushing language FC >>>>> sh: /home/balay/petsc-3.2/bin/win32fe/win32fe ifort -c -o >>>>> /tmp/petsc-zRoAJW/config.compilers/conftest.o >>>>> -I/tmp/petsc-zRoAJW/config.compilers >>>>> -I/tmp/petsc-zRoAJW/config.setCompilers >>>>> -MT -Z7 -fpp /tmp/petsc-zRoAJW/config.compilers/conftest.F >>>>> Executing: /home/balay/petsc-3.2/bin/win32fe/win32fe ifort -c -o >>>>> /tmp/petsc-zRoAJW/config.compilers/conftest.o >>>>> -I/tmp/petsc-zRoAJW/config.compilers >>>>> -I/tmp/petsc-zRoAJW/config.setCompilers >>>>> -MT -Z7 -fpp /tmp/petsc-zRoAJW/config.compilers/conftest.F >>>>> sh: >>>>> Successful compile: >>>>> Source: >>>>> module configtest >>>>> integer testint >>>>> parameter (testint = 42) >>>>> end module configtest >>>>> Pushing language FC >>>>> sh: /home/balay/petsc-3.2/bin/win32fe/win32fe ifort -c -o >>>>> /tmp/petsc-zRoAJW/config.compilers/conftest.o >>>>> -I/tmp/petsc-zRoAJW/config.compilers >>>>> -I/tmp/petsc-zRoAJW/config.setCompilers >>>>> -I/tmp/petsc-zRoAJW/config.compilers/confdir -MT -Z7 -fpp >>>>> /tmp/petsc-zRoAJW/config.compilers/conftest.F >>>>> Executing: /home/balay/petsc-3.2/bin/win32fe/win32fe ifort -c -o >>>>> /tmp/petsc-zRoAJW/config.compilers/conftest.o >>>>> -I/tmp/petsc-zRoAJW/config.compilers >>>>> -I/tmp/petsc-zRoAJW/config.setCompilers >>>>> -I/tmp/petsc-zRoAJW/config.compilers/confdir -MT -Z7 -fpp >>>>> /tmp/petsc-zRoAJW/config.compilers/conftest.F >>>>> sh: >>>>> Successful compile: >>>>> Source: >>>>> program main >>>>> use configtest >>>>> >>>>> write(*,*) testint >>>>> >>>>> end >>>>> Pushing language FC >>>>> Popping language FC >>>>> sh: /home/balay/petsc-3.2/bin/win32fe/win32fe ifort -o >>>>> /tmp/petsc-zRoAJW/config.compilers/conftest.exe >>>>> -I/tmp/petsc-zRoAJW/config.compilers/confdir -MT -Z7 -fpp >>>>> /tmp/petsc-zRoAJW/config.compilers/conftest.o >>>>> /tmp/petsc-zRoAJW/config.compilers/configtest.o >>>>> Executing: /home/balay/petsc-3.2/bin/win32fe/win32fe ifort -o >>>>> /tmp/petsc-zRoAJW/config.compilers/conftest.exe >>>>> -I/tmp/petsc-zRoAJW/config.compilers/confdir -MT -Z7 -fpp >>>>> /tmp/petsc-zRoAJW/config.compilers/conftest.o >>>>> /tmp/petsc-zRoAJW/config.compilers/configtest.o >>>>> sh: LINK : C:\cygwin\tmp\PETSC-~3\CONFIG~1.COM\conftest.exe not found or >>>>> not >>>>> built by the last incremental link; performing full link^M >>>>> >>>>> compilers: Fortran module include flag -I found >>>>> Popping language FC >>>>> <<<<<<<<<<<<<<<<<<<<<<< >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> On Thu, 15 Sep 2011, Matthew Knepley wrote: >>>>> >>>>>> On Thu, Sep 15, 2011 at 8:43 AM, Alexander Grayver >>>>>> wrote: >>>>>> >>>>>>> ** >>>>>>> Thanks Matt, >>>>>>> >>>>>>> I will that, but I use PETSC from Fortran so I need Fortran >>>>>>> functionality. >>>>>>> >>>>>> Sure. Do you use the F90 modules? If so, send us the build problem. If >>>>>> there >>>>>> is not build problem, >>>>>> then ifort supports only relative paths for module includes, which is >>>>>> crazy >>>>>> and will break sometimes. >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Matt >>>>>> >>>>>> >>>>>>> Regards, >>>>>>> Alexander >>>>>>> >>>>>>> >>>>>>> On 15.09.2011 15:43, Matthew Knepley wrote: >>>>>>> >>>>>>> On Thu, Sep 15, 2011 at 8:01 AM, Alexander Grayver< >>>>>>> agrayver at gfz-potsdam.de> wrote: >>>>>>> >>>>>>>> Matthew, >>>>>>>> >>>>>>>> I found this check in configure.log I obtained for petsc-3.1-p7: >>>>>>>> >>>>>>>> >>>>>>>> ================================================================================ >>>>>>>> TEST checkFortranModuleInclude from >>>>>>>> config.compilers(/cygdrive/d/Dev/petsc-3.1-p7/config/BuildSystem/config/compilers.py:1084) >>>>>>>> TESTING: checkFortranModuleInclude from >>>>>>>> config.compilers(config/BuildSystem/config/compilers.py:1084) >>>>>>>> Figures out what flag is used to specify the include path for >>>>>>>> Fortran >>>>>>>> modules >>>>>>>> Pushing language FC >>>>>>>> sh: /cygdrive/d/Dev/petsc-3.1-p7/bin/win32fe/win32fe ifort -c -o >>>>>>>> conftest.o -MT -Z7 -fpp conftest.F >>>>>>>> Executing: /cygdrive/d/Dev/petsc-3.1-p7/bin/win32fe/win32fe ifort >>>>>>>> -c >>>>>>>> -o >>>>>>>> conftest.o -MT -Z7 -fpp conftest.F >>>>>>>> sh: >>>>>>>> Pushing language FC >>>>>>>> sh: /cygdrive/d/Dev/petsc-3.1-p7/bin/win32fe/win32fe ifort -c -o >>>>>>>> conftest.o -Iconfdir -MT -Z7 -fpp conftest.F >>>>>>>> Executing: /cygdrive/d/Dev/petsc-3.1-p7/bin/win32fe/win32fe ifort >>>>>>>> -c >>>>>>>> -o >>>>>>>> conftest.o -Iconfdir -MT -Z7 -fpp conftest.F >>>>>>>> sh: >>>>>>>> Pushing language FC >>>>>>>> Popping language FC >>>>>>>> sh: /cygdrive/d/Dev/petsc-3.1-p7/bin/win32fe/win32fe ifort -o >>>>>>>> conftest.exe -Iconfdir -MT -Z7 -fpp conftest.o configtest.o >>>>>>>> Executing: /cygdrive/d/Dev/petsc-3.1-p7/bin/win32fe/win32fe ifort >>>>>>>> -o >>>>>>>> conftest.exe -Iconfdir -MT -Z7 -fpp conftest.o configtest.o >>>>>>>> sh: LINK : D:\Dev\PETSC-~2.1-P\conftest.exe not found or not built >>>>>>>> by >>>>>>>> the >>>>>>>> last incremental link; performing full link >>>>>>>> >>>>>>>> compilers: Fortran module include flag -I found >>>>>>>> Popping language FC >>>>>>>> >>>>>>>> ================================================================================ >>>>>>>> >>>>>>>> The compiler I used is the same I use for petsc-3.2-p1. >>>>>>>> The difference is in the paths where you try to build this tests. >>>>>>>> As >>>>>>>> far >>>>>>>> as I understand for petsc-3.1-p7 you used local directory and for >>>>>>>> petsc-3.2-p1 temp directory is used. >>>>>>>> Could it be the reason? >>>>>>>> >>>>>>> That would be broken behavior from ifort, but I guess it is >>>>>>> possible. >>>>>>> I >>>>>>> will talk with Satish about how >>>>>>> this should really be fixed. For right now, go to >>>>>>> >>>>>>> config/BuildSystem/config/compilers.py >>>>>>> >>>>>>> to the checkFortranModuleInclude() function and comment out the >>>>>>> last >>>>>>> two >>>>>>> lines, which are killing the configure. >>>>>>> Your build will break on the Fortran modules (I assume), but >>>>>>> everything >>>>>>> else should work. >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Matt >>>>>>> >>>>>>> >>>>>>>> Regards, >>>>>>>> Alexander >>>>>>>> >>>>>>>> >>>>>>>> On 15.09.2011 14:54, Matthew Knepley wrote: >>>>>>>> >>>>>>>> On Thu, Sep 15, 2011 at 2:49 AM, Alexander Grayver< >>>>>>>> agrayver at gfz-potsdam.de> wrote: >>>>>>>> >>>>>>>>> Hello, >>>>>>>>> >>>>>>>>> I hope you are having nice day! >>>>>>>>> >>>>>>>>> I have been using petsc-3.1-p7 under windows for 6 months and >>>>>>>>> now >>>>>>>>> decided >>>>>>>>> to get new version. I used my old configure line which was >>>>>>>>> successful for >>>>>>>>> petsc-3.1: >>>>>>>>> >>>>>>>>> ./config/configure.py --PETSC_DIR=/cygdrive/d/dev/petsc-3.2-p1 >>>>>>>>> --with-petsc-arch=cygwin-cxx-debug-fortran >>>>>>>>> --with-fortran-interfaces=1 >>>>>>>>> --with-cc='win32fe cl' --with-fc='win32fe ifort' >>>>>>>>> --with-cxx='win32fe cl' >>>>>>>>> --with-windows-graphics=0 --download-f-blas-lapack=1 >>>>>>>>> --with-precision=double >>>>>>>>> --with-scalar-type=complex --with-clanguage=cxx >>>>>>>>> --with-mpi-include="/cygdrive/d/Dev/HPC_2008_SDK/Include" >>>>>>>>> >>>>>>>>> >>>>>>>>> --with-mpi-lib=["/cygdrive/d/Dev/HPC_2008_SDK/Lib/amd64/msmpi.lib","/cygdrive/d/Dev/HPC_2008_SDK/Lib/amd64/msmpifec.lib"] >>>>>>>>> --useThreads=0 >>>>>>>>> >>>>>>>>> But unfortunately it doesn't work properly anymore, even though >>>>>>>>> my >>>>>>>>> development environment has not been changed. The error is: >>>>>>>>> >>>>>>>>> >>>>>>>>> ******************************************************************************* >>>>>>>>> UNABLE to CONFIGURE with GIVEN OPTIONS (see >>>>>>>>> configure.log for >>>>>>>>> details): >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> ------------------------------------------------------------------------------- >>>>>>>>> Cannot determine Fortran module include flag >>>>>>>>> >>>>>>>>> Please find configure.log attached. >>>>>>>>> >>>>>>>> The problem here is that your Fortran compiler reports that is >>>>>>>> supports >>>>>>>> F90, but cannot >>>>>>>> locate modules using an include path. This will make parts of the >>>>>>>> PETSc >>>>>>>> build fail. Do you >>>>>>>> know what flag it uses? I try -I, -p, and -M. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Matt >>>>>>>> >>>>>>>> >>>>>>>>> Regards, >>>>>>>>> Alexander >>>>>>>> -- >>>>>>>> What most experimenters take for granted before they begin their >>>>>>>> experiments is infinitely more interesting than any results to >>>>>>>> which >>>>>>>> their >>>>>>>> experiments lead. >>>>>>>> -- Norbert Wiener >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> -- >>>>>>> What most experimenters take for granted before they begin their >>>>>>> experiments is infinitely more interesting than any results to which >>>>>>> their >>>>>>> experiments lead. >>>>>>> -- Norbert Wiener >>>>>>> >>>>>>> >>>>>>> >> From knepley at gmail.com Fri Sep 16 08:04:11 2011 From: knepley at gmail.com (Matthew Knepley) Date: Fri, 16 Sep 2011 08:04:11 -0500 Subject: [petsc-users] [petsc-3.2] MatView on dense matrix In-Reply-To: <4E73273D.4060601@gfz-potsdam.de> References: <4E73273D.4060601@gfz-potsdam.de> Message-ID: On Fri, Sep 16, 2011 at 5:38 AM, Alexander Grayver wrote: > Hello, > > With petsc-3.1 I used this code to output MPI dense matrix: > > PetscViewer :: dviewer > call PetscViewerBinaryOpen(MPI_**COMM_WORLD,'A.dat',FILE_MODE_**WRITE,dviewer,ierr); > CHKERRQ(ierr) > call PetscViewerSetFormat(dviewer,**PETSC_VIEWER_NATIVE,ierr); > CHKERRQ(ierr) > call MatView(A,dviewer,ierr); CHKERRQ(ierr) > call PetscViewerDestroy(dviewer,**ierr); CHKERRQ(ierr) > > With petsc-3.2 this code fails with error: > This error just says that the viewer did not return a format of PETSC_VIEWER_NATIVE. Either you accidentally set it in the wrong viewer, or not on every process, or there is memory corruption. I recommend using the debugger to see what format is returned. Thanks, Matt > [7]PETSC ERROR: --------------------- Error Message > ------------------------------**------ > [7]PETSC ERROR: No support for this operation for this object type! > [7]PETSC ERROR: To store a parallel dense matrix you must first call > PetscViewerSetFormat(viewer,**PETSC_VIEWER_NATIVE)! > [7]PETSC ERROR: ------------------------------** > ------------------------------**------------ > [7]PETSC ERROR: Petsc Release Version 3.2.0, Patch 1, Mon Sep 12 16:01:51 > CDT 2011 > [7]PETSC ERROR: See docs/changes/index.html for recent updates. > [7]PETSC ERROR: See docs/faq.html for hints about trouble shooting. > [7]PETSC ERROR: See docs/index.html for manual pages. > [7]PETSC ERROR: ------------------------------** > ------------------------------**------------ > [7]PETSC ERROR: /home/run/test on a openmpi-i named node205 by user Fri Sep > 16 09:17:13 2011 > [7]PETSC ERROR: Libraries linked from /home/lib/petsc-3.2-p1/** > openmpi-intel-complex-debug-f/**lib > [7]PETSC ERROR: Configure run at Wed Sep 14 14:32:55 2011 > [7]PETSC ERROR: Configure options --with-petsc-arch=openmpi-**intel-complex-debug-f > --with-fortran-interfaces=1 --with-mpi-dir=/opt/mpi/intel/**openmpi-1.4.2 > --with-scalar-type=complex --with-blas-lapack-dir=/opt/** > intel/Compiler/11.1/072/mkl/**lib/em64t --with-precision=double --with-x=0 > [7]PETSC ERROR: ------------------------------** > ------------------------------**------------ > [7]PETSC ERROR: MatView_MPIDense_Binary() line 667 in > /home/lib/petsc-3.2-p1/src/**mat/impls/dense/mpi/mpidense.c > [7]PETSC ERROR: MatView_MPIDense() line 789 in /home/lib/petsc-3.2-p1/src/ > **mat/impls/dense/mpi/mpidense.c > [7]PETSC ERROR: MatView() line 757 in /home/lib/petsc-3.2-p1/src/** > mat/interface/matrix.c > MatView_MPIDense() line 789 in /home/lib/petsc-3.2-p1/src/** > mat/impls/dense/mpi/mpidense.c > > See full error attached. Is there something new in petsc-3.2 concerning > matrix output? > > Regards, > Alexander > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Fri Sep 16 08:09:32 2011 From: knepley at gmail.com (Matthew Knepley) Date: Fri, 16 Sep 2011 08:09:32 -0500 Subject: [petsc-users] viewers and nohup (unrelated questions) In-Reply-To: References: Message-ID: On Fri, Sep 16, 2011 at 7:17 AM, Gianluca Meneghello wrote: > Hi, > > I would like to ask you 2 unrelated questions. > > #_1 The first one concerns PETSc viewer. In my code I'm writing > outputs with PETSC_VIEWER_ASCII_VTK, more precisely > > PetscViewer viewer; > PetscViewerASCIIOpen(PETSC_COMM_SELF,filename.c_str(),&viewer); > PetscViewerSetFormat(viewer,PETSC_VIEWER_ASCII_VTK); > DMView(dau,viewer); > VecView(foou,viewer); > VecView(foop,viewer); > PetscViewerDestroy(&viewer); > > where foou and foop (velocity and pressure respectively) are vectors > extracted from the vector containing the full solution of my problem. > The VTK so obtained nicely contains a vector field ? from foou ? and a > scalar field ? foop ? and can be read by Paraview. > > Nonetheless, I guess writing (and reading from Paraview) ASCII files > is not the best choice. Are there better possibilities in PETSc? Is > there a PETSC_VIEWER_BINARY_VTK or equivalent? > We are working on it but it is not yet ready. > #_2 The second is about the nohup command. If I run my code with > "nohup ./main &" and then exit the console, the program dies as > if there was no nohup command in front of it. The same is true for > some examples I've tested from the PETSc library. I guess this is > related to the machine I'm using more than to the library but... has > anyone of you already experienced a similar issue? Any advice is > welcome and please let me know if you need any information that can > help. > Dos your program die from a HUP signal or another one? Thanks, Matt > Thanks > > Gianluca > > -- > "[Je pense que] l'homme est un monde qui vaut des fois les mondes et > que les plus ardentes ambitions sont celles qui ont eu l'orgueil de > l'Anonymat" -- Non omnibus, sed mihi et tibi > Amedeo Modigliani > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From gianmail at gmail.com Fri Sep 16 09:07:33 2011 From: gianmail at gmail.com (Gianluca Meneghello) Date: Fri, 16 Sep 2011 16:07:33 +0200 Subject: [petsc-users] viewers and nohup (unrelated questions) In-Reply-To: References: Message-ID: Thanks for the binary viewer, I will wait for it! Concerning the nohup, t dies from a HUP signal. This is the error I obtain in nohup.out: [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: Caught signal number 1 Hang up: Some other process (or the batch system) has told this process to end [0]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger [0]PETSC ERROR: or see http://www.mcs.anl.gov/petsc/petsc-as/documentation/faq.html#valgrind[0]PETSC ERROR: or try http://valgrind.org on GNU/linux and Apple Mac OS X to find memory corruption errors [0]PETSC ERROR: configure using --with-debugging=yes, recompile, link, and run [0]PETSC ERROR: to get more information on the crash. [0]PETSC ERROR: --------------------- Error Message ------------------------------------ [0]PETSC ERROR: Signal received! [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: Petsc Release Version 3.2.0, Patch 1, unknown [0]PETSC ERROR: See docs/changes/index.html for recent updates. [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. [0]PETSC ERROR: See docs/index.html for manual pages. [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: ./main on a optnompi_ named tarzan.polytechnique.fr by gianluca Fri Sep 16 16:03:05 2011 [0]PETSC ERROR: Libraries linked from /moogwai/usr/gianluca/soft/petsc-3.2/optnompi_complex/lib [0]PETSC ERROR: Configure run at Thu Sep 15 14:21:03 2011 [0]PETSC ERROR: Configure options --download-umfpack --with-clanguage=cxx --with-debugging=0 --with-fortran-kernels=generic --with-mpi=0 --with-scalar-type=complex --with-umfpack=1 COPTFLAGS=-O3 CXXOPTFLAGS=-O3 FOPTFLAGS=-O3 PETSC_ARCH=optnompi_complex [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: User provided function() line 0 in unknown directory unknown file Any hint is welcome Thanks Gianluca On 16 September 2011 15:09, Matthew Knepley wrote: > On Fri, Sep 16, 2011 at 7:17 AM, Gianluca Meneghello > wrote: >> >> Hi, >> >> I would like to ask you 2 unrelated questions. >> >> #_1 The first one concerns PETSc viewer. In my code I'm writing >> outputs with PETSC_VIEWER_ASCII_VTK, more precisely >> >> ? ?PetscViewer viewer; >> ? ?PetscViewerASCIIOpen(PETSC_COMM_SELF,filename.c_str(),&viewer); >> ? ?PetscViewerSetFormat(viewer,PETSC_VIEWER_ASCII_VTK); >> ? ?DMView(dau,viewer); >> ? ?VecView(foou,viewer); >> ? ?VecView(foop,viewer); >> ? ?PetscViewerDestroy(&viewer); >> >> where foou and foop (velocity and pressure respectively) are vectors >> extracted from the vector containing the full solution of my problem. >> The VTK so obtained nicely contains a vector field ? from foou ? and a >> scalar field ? foop ? and can be read by Paraview. >> >> Nonetheless, I guess writing (and reading from Paraview) ASCII files >> is not the best choice. Are there better possibilities in PETSc? Is >> there a PETSC_VIEWER_BINARY_VTK or equivalent? > > We are working on it but it is not yet ready. > >> >> #_2 The second is about the nohup command. If I run my code with >> "nohup ./main &" and then exit the console, the program dies as >> if there was no nohup command in front of it. The same is true for >> some examples I've tested from the PETSc library. I guess this is >> related to the machine I'm using more than to the library but... has >> anyone of you already experienced a similar issue? Any advice is >> welcome and please let me know if you need any information that can >> help. > > Dos your program die from a HUP signal or another one? > ? Thanks, > ? ? ?Matt > >> >> Thanks >> >> Gianluca >> >> -- >> "[Je pense que] l'homme est un monde qui vaut des fois les mondes et >> que les plus ardentes ambitions sont celles qui ont eu l'orgueil de >> l'Anonymat" -- Non omnibus, sed mihi et tibi >> Amedeo Modigliani > > > > -- > What most experimenters take for granted before they begin their experiments > is infinitely more interesting than any results to which their experiments > lead. > -- Norbert Wiener > -- "[Je pense que] l'homme est un monde qui vaut des fois les mondes et que les plus ardentes ambitions sont celles qui ont eu l'orgueil de l'Anonymat" -- Non omnibus, sed mihi et tibi Amedeo Modigliani From john.fettig at gmail.com Fri Sep 16 10:58:05 2011 From: john.fettig at gmail.com (John Fettig) Date: Fri, 16 Sep 2011 11:58:05 -0400 Subject: [petsc-users] PCSORSetSymmetric Message-ID: I don't really understand what is the purpose of the "local" options in PCSOR. In serial for AIJ for example, SOR_FORWARD_SWEEP and SOR_LOCAL_FORWARD_SWEEP appear to follow the same path in the code. In parallel for MPIAIJ, SOR_FORWARD_SWEEP isn't even handled by MATSOR_MPIAIJ(). It actually results in a segfault if you try it (this looks like a bug), for example in ksp/ksp/examples/tutorials: "mpirun -np 2 ./ex2 -pc_type sor -pc_sor_forward" So in parallel, the only valid options are the "local" ones, which makes sense since you haven't provided a true parallel SOR. What is the purpose of the "local" options then, if there really isn't a "global" SOR? Thanks, John From gianmail at gmail.com Fri Sep 16 11:09:56 2011 From: gianmail at gmail.com (Gianluca Meneghello) Date: Fri, 16 Sep 2011 18:09:56 +0200 Subject: [petsc-users] viewers and nohup (unrelated questions) In-Reply-To: References: Message-ID: If it can be useful, nohup is not respected only when the job is submitted using an ssh connection (and the ssh connection is closed). When submitting from the terminal on the machine everything works fine. PETSc has been compiled --with-mpi=0 Gianluca On 16 September 2011 16:07, Gianluca Meneghello wrote: > Thanks for the binary viewer, I will wait for it! > > Concerning the nohup, t dies from a HUP signal. This is the error I > obtain in nohup.out: > > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: Caught signal number 1 Hang up: Some other process (or > the batch system) has told this process to end > [0]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger > [0]PETSC ERROR: or see > http://www.mcs.anl.gov/petsc/petsc-as/documentation/faq.html#valgrind[0]PETSC > ERROR: or try http://valgrind.org on GNU/linux and Apple Mac OS X to > find memory corruption errors > [0]PETSC ERROR: configure using --with-debugging=yes, recompile, link, and run > [0]PETSC ERROR: to get more information on the crash. > [0]PETSC ERROR: --------------------- Error Message > ------------------------------------ > [0]PETSC ERROR: Signal received! > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: Petsc Release Version 3.2.0, Patch 1, unknown > [0]PETSC ERROR: See docs/changes/index.html for recent updates. > [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. > [0]PETSC ERROR: See docs/index.html for manual pages. > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: ./main on a optnompi_ named tarzan.polytechnique.fr by > gianluca Fri Sep 16 16:03:05 2011 > [0]PETSC ERROR: Libraries linked from > /moogwai/usr/gianluca/soft/petsc-3.2/optnompi_complex/lib > [0]PETSC ERROR: Configure run at Thu Sep 15 14:21:03 2011 > [0]PETSC ERROR: Configure options --download-umfpack > --with-clanguage=cxx --with-debugging=0 --with-fortran-kernels=generic > --with-mpi=0 --with-scalar-type=complex --with-umfpack=1 COPTFLAGS=-O3 > CXXOPTFLAGS=-O3 FOPTFLAGS=-O3 PETSC_ARCH=optnompi_complex > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: User provided function() line 0 in unknown directory > unknown file > > > Any hint is welcome > > Thanks > > Gianluca > > On 16 September 2011 15:09, Matthew Knepley wrote: >> On Fri, Sep 16, 2011 at 7:17 AM, Gianluca Meneghello >> wrote: >>> >>> Hi, >>> >>> I would like to ask you 2 unrelated questions. >>> >>> #_1 The first one concerns PETSc viewer. In my code I'm writing >>> outputs with PETSC_VIEWER_ASCII_VTK, more precisely >>> >>> ? ?PetscViewer viewer; >>> ? ?PetscViewerASCIIOpen(PETSC_COMM_SELF,filename.c_str(),&viewer); >>> ? ?PetscViewerSetFormat(viewer,PETSC_VIEWER_ASCII_VTK); >>> ? ?DMView(dau,viewer); >>> ? ?VecView(foou,viewer); >>> ? ?VecView(foop,viewer); >>> ? ?PetscViewerDestroy(&viewer); >>> >>> where foou and foop (velocity and pressure respectively) are vectors >>> extracted from the vector containing the full solution of my problem. >>> The VTK so obtained nicely contains a vector field ? from foou ? and a >>> scalar field ? foop ? and can be read by Paraview. >>> >>> Nonetheless, I guess writing (and reading from Paraview) ASCII files >>> is not the best choice. Are there better possibilities in PETSc? Is >>> there a PETSC_VIEWER_BINARY_VTK or equivalent? >> >> We are working on it but it is not yet ready. >> >>> >>> #_2 The second is about the nohup command. If I run my code with >>> "nohup ./main &" and then exit the console, the program dies as >>> if there was no nohup command in front of it. The same is true for >>> some examples I've tested from the PETSc library. I guess this is >>> related to the machine I'm using more than to the library but... has >>> anyone of you already experienced a similar issue? Any advice is >>> welcome and please let me know if you need any information that can >>> help. >> >> Dos your program die from a HUP signal or another one? >> ? Thanks, >> ? ? ?Matt >> >>> >>> Thanks >>> >>> Gianluca >>> >>> -- >>> "[Je pense que] l'homme est un monde qui vaut des fois les mondes et >>> que les plus ardentes ambitions sont celles qui ont eu l'orgueil de >>> l'Anonymat" -- Non omnibus, sed mihi et tibi >>> Amedeo Modigliani >> >> >> >> -- >> What most experimenters take for granted before they begin their experiments >> is infinitely more interesting than any results to which their experiments >> lead. >> -- Norbert Wiener >> > > > > -- > "[Je pense que] l'homme est un monde qui vaut des fois les mondes et > que les plus ardentes ambitions sont celles qui ont eu l'orgueil de > l'Anonymat" -- Non omnibus, sed mihi et tibi > Amedeo Modigliani > -- "[Je pense que] l'homme est un monde qui vaut des fois les mondes et que les plus ardentes ambitions sont celles qui ont eu l'orgueil de l'Anonymat" -- Non omnibus, sed mihi et tibi Amedeo Modigliani From balay at mcs.anl.gov Fri Sep 16 11:13:46 2011 From: balay at mcs.anl.gov (Satish Balay) Date: Fri, 16 Sep 2011 11:13:46 -0500 (CDT) Subject: [petsc-users] petsc-3.2-p2.tar.gz now available Message-ID: Dear PETSc users, Patch-2 update to petsc-3.2 is now available for download. http://www.mcs.anl.gov/petsc/petsc-as/download/index.html Satish From jedbrown at mcs.anl.gov Fri Sep 16 11:31:11 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Fri, 16 Sep 2011 18:31:11 +0200 Subject: [petsc-users] viewers and nohup (unrelated questions) In-Reply-To: References: Message-ID: On Fri, Sep 16, 2011 at 18:09, Gianluca Meneghello wrote: > If it can be useful, nohup is not respected only when the job is > submitted using an ssh connection (and the ssh connection is closed). > Sounds like a problem with nohup/ssh. Isn't the whole point of nohup to prevent sending HUP to the application? Is the shell the same (interactive and login) via the terminal and via ssh? Is there a reason you don't want to run the job in a screen or tmux session? (I have always found that preferable to using nohup.) -------------- next part -------------- An HTML attachment was scrubbed... URL: From gianmail at gmail.com Fri Sep 16 11:51:54 2011 From: gianmail at gmail.com (Gianluca Meneghello) Date: Fri, 16 Sep 2011 18:51:54 +0200 Subject: [petsc-users] viewers and nohup (unrelated questions) In-Reply-To: References: Message-ID: No particular reason except the fact that I don't know them! I will have a look at how they work. Thanks for the suggestion. gianluca On 16 September 2011 18:31, Jed Brown wrote: > On Fri, Sep 16, 2011 at 18:09, Gianluca Meneghello > wrote: >> >> If it can be useful, nohup is not respected only when the job is >> submitted using an ssh connection (and the ssh connection is closed). > > Sounds like a problem with nohup/ssh. Isn't the whole point of nohup to > prevent sending HUP to the application? Is the shell the same (interactive > and login) via the terminal and via ssh? > Is there a reason you don't want to run the job in a screen or tmux session? > (I have always found that preferable to using nohup.) -- "[Je pense que] l'homme est un monde qui vaut des fois les mondes et que les plus ardentes ambitions sont celles qui ont eu l'orgueil de l'Anonymat" -- Non omnibus, sed mihi et tibi Amedeo Modigliani From agrayver at gfz-potsdam.de Fri Sep 16 12:15:16 2011 From: agrayver at gfz-potsdam.de (Alexander Grayver) Date: Fri, 16 Sep 2011 19:15:16 +0200 Subject: [petsc-users] [petsc-3.2] MatView on dense matrix In-Reply-To: References: <4E73273D.4060601@gfz-potsdam.de> Message-ID: <4E738424.6020909@gfz-potsdam.de> Matt, Can you explain what do you by seeing what format is returned? I will check in debugger of course, but still I find it strange. I have two versions of petsc (3.1-p7 and 3.2-p1) configured with the same options, my code and runs are absolutely identical and I get this error only with petsc-3.2. Regards, Alexander On 16.09.2011 15:04, Matthew Knepley wrote: > On Fri, Sep 16, 2011 at 5:38 AM, Alexander Grayver > > wrote: > > Hello, > > With petsc-3.1 I used this code to output MPI dense matrix: > > PetscViewer :: dviewer > call > PetscViewerBinaryOpen(MPI_COMM_WORLD,'A.dat',FILE_MODE_WRITE,dviewer,ierr); > CHKERRQ(ierr) > call PetscViewerSetFormat(dviewer,PETSC_VIEWER_NATIVE,ierr); > CHKERRQ(ierr) > call MatView(A,dviewer,ierr); CHKERRQ(ierr) > call PetscViewerDestroy(dviewer,ierr); CHKERRQ(ierr) > > With petsc-3.2 this code fails with error: > > > This error just says that the viewer did not return a format of > PETSC_VIEWER_NATIVE. Either you > accidentally set it in the wrong viewer, or not on every process, or > there is memory corruption. I recommend > using the debugger to see what format is returned. > > Thanks, > > Matt > > [7]PETSC ERROR: --------------------- Error Message > ------------------------------------ > [7]PETSC ERROR: No support for this operation for this object type! > [7]PETSC ERROR: To store a parallel dense matrix you must first > call PetscViewerSetFormat(viewer,PETSC_VIEWER_NATIVE)! > [7]PETSC ERROR: > ------------------------------------------------------------------------ > [7]PETSC ERROR: Petsc Release Version 3.2.0, Patch 1, Mon Sep 12 > 16:01:51 CDT 2011 > [7]PETSC ERROR: See docs/changes/index.html for recent updates. > [7]PETSC ERROR: See docs/faq.html for hints about trouble shooting. > [7]PETSC ERROR: See docs/index.html for manual pages. > [7]PETSC ERROR: > ------------------------------------------------------------------------ > [7]PETSC ERROR: /home/run/test on a openmpi-i named node205 by > user Fri Sep 16 09:17:13 2011 > [7]PETSC ERROR: Libraries linked from > /home/lib/petsc-3.2-p1/openmpi-intel-complex-debug-f/lib > [7]PETSC ERROR: Configure run at Wed Sep 14 14:32:55 2011 > [7]PETSC ERROR: Configure options > --with-petsc-arch=openmpi-intel-complex-debug-f > --with-fortran-interfaces=1 > --with-mpi-dir=/opt/mpi/intel/openmpi-1.4.2 > --with-scalar-type=complex > --with-blas-lapack-dir=/opt/intel/Compiler/11.1/072/mkl/lib/em64t > --with-precision=double --with-x=0 > [7]PETSC ERROR: > ------------------------------------------------------------------------ > [7]PETSC ERROR: MatView_MPIDense_Binary() line 667 in > /home/lib/petsc-3.2-p1/src/mat/impls/dense/mpi/mpidense.c > [7]PETSC ERROR: MatView_MPIDense() line 789 in > /home/lib/petsc-3.2-p1/src/mat/impls/dense/mpi/mpidense.c > [7]PETSC ERROR: MatView() line 757 in > /home/lib/petsc-3.2-p1/src/mat/interface/matrix.c > MatView_MPIDense() line 789 in > /home/lib/petsc-3.2-p1/src/mat/impls/dense/mpi/mpidense.c > > See full error attached. Is there something new in petsc-3.2 > concerning matrix output? > > Regards, > Alexander > > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which > their experiments lead. > -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Fri Sep 16 12:24:10 2011 From: knepley at gmail.com (Matthew Knepley) Date: Fri, 16 Sep 2011 12:24:10 -0500 Subject: [petsc-users] [petsc-3.2] MatView on dense matrix In-Reply-To: <4E738424.6020909@gfz-potsdam.de> References: <4E73273D.4060601@gfz-potsdam.de> <4E738424.6020909@gfz-potsdam.de> Message-ID: On Fri, Sep 16, 2011 at 12:15 PM, Alexander Grayver wrote: > ** > Matt, > > Can you explain what do you by seeing what format is returned? > You can step into the MatView function, into MatView_MPIDense_Binary(), where it checks the format, and see what format it thought it had. It just feels like a memory overwrite since it should work. Matt > I will check in debugger of course, but still I find it strange. > I have two versions of petsc (3.1-p7 and 3.2-p1) configured with the same > options, my code and runs are absolutely identical and I get this error only > with petsc-3.2. > > Regards, > Alexander > > > On 16.09.2011 15:04, Matthew Knepley wrote: > > On Fri, Sep 16, 2011 at 5:38 AM, Alexander Grayver < > agrayver at gfz-potsdam.de> wrote: > >> Hello, >> >> With petsc-3.1 I used this code to output MPI dense matrix: >> >> PetscViewer :: dviewer >> call >> PetscViewerBinaryOpen(MPI_COMM_WORLD,'A.dat',FILE_MODE_WRITE,dviewer,ierr); >> CHKERRQ(ierr) >> call PetscViewerSetFormat(dviewer,PETSC_VIEWER_NATIVE,ierr); CHKERRQ(ierr) >> call MatView(A,dviewer,ierr); CHKERRQ(ierr) >> call PetscViewerDestroy(dviewer,ierr); CHKERRQ(ierr) >> >> With petsc-3.2 this code fails with error: >> > > This error just says that the viewer did not return a format of > PETSC_VIEWER_NATIVE. Either you > accidentally set it in the wrong viewer, or not on every process, or there > is memory corruption. I recommend > using the debugger to see what format is returned. > > Thanks, > > Matt > > >> [7]PETSC ERROR: --------------------- Error Message >> ------------------------------------ >> [7]PETSC ERROR: No support for this operation for this object type! >> [7]PETSC ERROR: To store a parallel dense matrix you must first call >> PetscViewerSetFormat(viewer,PETSC_VIEWER_NATIVE)! >> [7]PETSC ERROR: >> ------------------------------------------------------------------------ >> [7]PETSC ERROR: Petsc Release Version 3.2.0, Patch 1, Mon Sep 12 16:01:51 >> CDT 2011 >> [7]PETSC ERROR: See docs/changes/index.html for recent updates. >> [7]PETSC ERROR: See docs/faq.html for hints about trouble shooting. >> [7]PETSC ERROR: See docs/index.html for manual pages. >> [7]PETSC ERROR: >> ------------------------------------------------------------------------ >> [7]PETSC ERROR: /home/run/test on a openmpi-i named node205 by user Fri >> Sep 16 09:17:13 2011 >> [7]PETSC ERROR: Libraries linked from >> /home/lib/petsc-3.2-p1/openmpi-intel-complex-debug-f/lib >> [7]PETSC ERROR: Configure run at Wed Sep 14 14:32:55 2011 >> [7]PETSC ERROR: Configure options >> --with-petsc-arch=openmpi-intel-complex-debug-f --with-fortran-interfaces=1 >> --with-mpi-dir=/opt/mpi/intel/openmpi-1.4.2 --with-scalar-type=complex >> --with-blas-lapack-dir=/opt/intel/Compiler/11.1/072/mkl/lib/em64t >> --with-precision=double --with-x=0 >> [7]PETSC ERROR: >> ------------------------------------------------------------------------ >> [7]PETSC ERROR: MatView_MPIDense_Binary() line 667 in >> /home/lib/petsc-3.2-p1/src/mat/impls/dense/mpi/mpidense.c >> [7]PETSC ERROR: MatView_MPIDense() line 789 in >> /home/lib/petsc-3.2-p1/src/mat/impls/dense/mpi/mpidense.c >> [7]PETSC ERROR: MatView() line 757 in >> /home/lib/petsc-3.2-p1/src/mat/interface/matrix.c >> MatView_MPIDense() line 789 in >> /home/lib/petsc-3.2-p1/src/mat/impls/dense/mpi/mpidense.c >> >> See full error attached. Is there something new in petsc-3.2 concerning >> matrix output? >> >> Regards, >> Alexander >> > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Fri Sep 16 12:59:24 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Fri, 16 Sep 2011 12:59:24 -0500 Subject: [petsc-users] [petsc-3.2] MatView on dense matrix In-Reply-To: <4E73273D.4060601@gfz-potsdam.de> References: <4E73273D.4060601@gfz-potsdam.de> Message-ID: The problem is from the C values of these parameters have gotten out of sync with the Fortran values. Satish, Could you please fix the finclude/petscviewer.h values to match the C in petsc-3.2 and send a patch? Thanks Barry On Sep 16, 2011, at 5:38 AM, Alexander Grayver wrote: > Hello, > > With petsc-3.1 I used this code to output MPI dense matrix: > > PetscViewer :: dviewer > call PetscViewerBinaryOpen(MPI_COMM_WORLD,'A.dat',FILE_MODE_WRITE,dviewer,ierr); CHKERRQ(ierr) > call PetscViewerSetFormat(dviewer,PETSC_VIEWER_NATIVE,ierr); CHKERRQ(ierr) > call MatView(A,dviewer,ierr); CHKERRQ(ierr) > call PetscViewerDestroy(dviewer,ierr); CHKERRQ(ierr) > > With petsc-3.2 this code fails with error: > > [7]PETSC ERROR: --------------------- Error Message ------------------------------------ > [7]PETSC ERROR: No support for this operation for this object type! > [7]PETSC ERROR: To store a parallel dense matrix you must first call PetscViewerSetFormat(viewer,PETSC_VIEWER_NATIVE)! > [7]PETSC ERROR: ------------------------------------------------------------------------ > [7]PETSC ERROR: Petsc Release Version 3.2.0, Patch 1, Mon Sep 12 16:01:51 CDT 2011 > [7]PETSC ERROR: See docs/changes/index.html for recent updates. > [7]PETSC ERROR: See docs/faq.html for hints about trouble shooting. > [7]PETSC ERROR: See docs/index.html for manual pages. > [7]PETSC ERROR: ------------------------------------------------------------------------ > [7]PETSC ERROR: /home/run/test on a openmpi-i named node205 by user Fri Sep 16 09:17:13 2011 > [7]PETSC ERROR: Libraries linked from /home/lib/petsc-3.2-p1/openmpi-intel-complex-debug-f/lib > [7]PETSC ERROR: Configure run at Wed Sep 14 14:32:55 2011 > [7]PETSC ERROR: Configure options --with-petsc-arch=openmpi-intel-complex-debug-f --with-fortran-interfaces=1 --with-mpi-dir=/opt/mpi/intel/openmpi-1.4.2 --with-scalar-type=complex --with-blas-lapack-dir=/opt/intel/Compiler/11.1/072/mkl/lib/em64t --with-precision=double --with-x=0 > [7]PETSC ERROR: ------------------------------------------------------------------------ > [7]PETSC ERROR: MatView_MPIDense_Binary() line 667 in /home/lib/petsc-3.2-p1/src/mat/impls/dense/mpi/mpidense.c > [7]PETSC ERROR: MatView_MPIDense() line 789 in /home/lib/petsc-3.2-p1/src/mat/impls/dense/mpi/mpidense.c > [7]PETSC ERROR: MatView() line 757 in /home/lib/petsc-3.2-p1/src/mat/interface/matrix.c > MatView_MPIDense() line 789 in /home/lib/petsc-3.2-p1/src/mat/impls/dense/mpi/mpidense.c > > See full error attached. Is there something new in petsc-3.2 concerning matrix output? > > Regards, > Alexander > From bsmith at mcs.anl.gov Fri Sep 16 21:23:29 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Fri, 16 Sep 2011 21:23:29 -0500 Subject: [petsc-users] PCSORSetSymmetric In-Reply-To: References: Message-ID: <8204B643-4D61-454D-BB00-B2616817ED3D@mcs.anl.gov> On Sep 16, 2011, at 10:58 AM, John Fettig wrote: > I don't really understand what is the purpose of the "local" options > in PCSOR. In serial for AIJ for example, SOR_FORWARD_SWEEP and > SOR_LOCAL_FORWARD_SWEEP appear to follow the same path in the code. Yes, the local means each process does its own independent smoothing, when there is only one process, local and nonlocal are identical > > In parallel for MPIAIJ, SOR_FORWARD_SWEEP isn't even handled by > MATSOR_MPIAIJ(). It actually results in a segfault if you try it > (this looks like a bug), Yes, this is a bug, thanks for reporting it, it should have printed a nice error message about lack of support. > for example in ksp/ksp/examples/tutorials: > "mpirun -np 2 ./ex2 -pc_type sor -pc_sor_forward" So in parallel, the > only valid options are the "local" ones, which makes sense since you > haven't provided a true parallel SOR. > > What is the purpose of the "local" options then, if there really isn't > a "global" SOR? There could be a global SOR (for example using a coloring) we just haven't implemented them but maybe someday someone who is interested will. Barry > > Thanks, > John From 5202222ghj at 163.com Sat Sep 17 16:34:26 2011 From: 5202222ghj at 163.com (yuxuan) Date: Sun, 18 Sep 2011 05:34:26 +0800 (CST) Subject: [petsc-users] about Finite Difference Jacobian Approximations Message-ID: <1a13142.4488.132794fd187.Coremail.5202222ghj@163.com> Hi all, I am solving PDEs on a unstructured mesh by finite volume method. At beginning I was using matrix-free method. However, I found that PETSc provides some tools to help approximate the Jacobian matrices ef?ciently via ?nite differences. These tools are intended for use in certain situations where one is unable to compute Jacobian matrices ana- lytically, and matrix-free methods do not work well without a preconditioner, due to very poor conditioning. in the Petsc manual I started to use the Finite difference Jacobian approximations. I customized my MyApproxJacobianStructure(), by the considering the neighbors of my unstructured mesh. I am using the tins. So if i and j are neighbors, I will call MatSetValue(*jac,i,j,zeros,INSERT_VALUES); And finally, I got the structure of my Jacobain Matrix by MatFDColoringView().see the attachment. My question is whether my structure of the Jacobian matrix is good? And how to design preconditioned based on the structure of Jacobian matrix? Thanks! Cos -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Picture 41.png Type: image/png Size: 47512 bytes Desc: not available URL: From gdiso at ustc.edu Sun Sep 18 07:04:48 2011 From: gdiso at ustc.edu (Gong Ding) Date: Sun, 18 Sep 2011 20:04:48 +0800 (CST) Subject: [petsc-users] Does float128 helps ksp solver convergence Message-ID: <22443916.231151316347488701.JavaMail.coremail@mail.ustc.edu> Hi, I always meed some linear system with condition number near 1e10, the ksp solver (BCGSL + ASM + ILU(1))with double presision seems painful to convergence. Will float 128 helps in this situation? And how about the performance degenerate for float 128 under i.e. XEON 5620 CPU compared with double? Thanks Gong Ding From knepley at gmail.com Sun Sep 18 09:10:10 2011 From: knepley at gmail.com (Matthew Knepley) Date: Sun, 18 Sep 2011 09:10:10 -0500 Subject: [petsc-users] Does float128 helps ksp solver convergence In-Reply-To: <22443916.231151316347488701.JavaMail.coremail@mail.ustc.edu> References: <22443916.231151316347488701.JavaMail.coremail@mail.ustc.edu> Message-ID: 2011/9/18 Gong Ding > Hi, > I always meed some linear system with condition number near 1e10, the ksp > solver (BCGSL + ASM + ILU(1))with double presision > seems painful to convergence. > Higher precision is unlikely to affect the convergence rate. It might help with loss of orthogonality. Matt > Will float 128 helps in this situation? > And how about the performance degenerate for float 128 under i.e. XEON 5620 > CPU compared with double? > > Thanks > Gong Ding > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From gdiso at ustc.edu Sun Sep 18 10:07:20 2011 From: gdiso at ustc.edu (Gong Ding) Date: Sun, 18 Sep 2011 23:07:20 +0800 (CST) Subject: [petsc-users] Does float128 helps ksp solver convergence In-Reply-To: References: <22443916.231151316347488701.JavaMail.coremail@mail.ustc.edu> Message-ID: <29833061.231461316358440034.JavaMail.coremail@mail.ustc.edu> 2011/9/18 Gong Ding Hi, I always meed some linear system with condition number near 1e10, the ksp solver (BCGSL + ASM + ILU(1))with double presision seems painful to convergence. Higher precision is unlikely to affect the convergence rate. It might help with loss of orthogonality. Do you mean GMRES-like method will bebenefited from float 128? Matt Will float 128 helps in this situation? And how about the performance degenerate for float 128 under i.e. XEON 5620 CPU compared with double? Thanks Gong Ding -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From jedbrown at mcs.anl.gov Sun Sep 18 10:24:10 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Sun, 18 Sep 2011 17:24:10 +0200 Subject: [petsc-users] Does float128 helps ksp solver convergence In-Reply-To: <29833061.231461316358440034.JavaMail.coremail@mail.ustc.edu> References: <22443916.231151316347488701.JavaMail.coremail@mail.ustc.edu> <29833061.231461316358440034.JavaMail.coremail@mail.ustc.edu> Message-ID: On Sun, Sep 18, 2011 at 17:07, Gong Ding wrote: > Do you mean GMRES-like method will be benefited from float 128? > It could be, especially if you use a finite difference Jacobian. If you suspect precision issues, you can try more robust orthogonalization -ksp_gmres_modifiedgramschmidt or -ksp_type gcr, or you can just try using float128. -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Sun Sep 18 10:48:20 2011 From: knepley at gmail.com (Matthew Knepley) Date: Sun, 18 Sep 2011 10:48:20 -0500 Subject: [petsc-users] Does float128 helps ksp solver convergence In-Reply-To: <29833061.231461316358440034.JavaMail.coremail@mail.ustc.edu> References: <22443916.231151316347488701.JavaMail.coremail@mail.ustc.edu> <29833061.231461316358440034.JavaMail.coremail@mail.ustc.edu> Message-ID: On Sun, Sep 18, 2011 at 10:07 AM, Gong Ding wrote: > > > 2011/9/18 Gong Ding > >> Hi, >> I always meed some linear system with condition number near 1e10, the ksp >> solver (BCGSL + ASM + ILU(1))with double presision >> seems painful to convergence. >> > > Higher precision is unlikely to affect the convergence rate. It might help > with loss of orthogonality. > > Do you mean GMRES-like method will be benefited from float 128? > > My guess is no, but it is possible. Matt > Matt > > >> Will float 128 helps in this situation? >> And how about the performance degenerate for float 128 under i.e. XEON >> 5620 CPU compared with double? >> >> Thanks >> Gong Ding >> >> > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From gdiso at ustc.edu Sun Sep 18 11:09:42 2011 From: gdiso at ustc.edu (Gong Ding) Date: Mon, 19 Sep 2011 00:09:42 +0800 (CST) Subject: [petsc-users] Does float128 helps ksp solver convergence In-Reply-To: References: <22443916.231151316347488701.JavaMail.coremail@mail.ustc.edu> <29833061.231461316358440034.JavaMail.coremail@mail.ustc.edu> Message-ID: <17215513.231541316362182546.JavaMail.coremail@mail.ustc.edu> Thanks, the synopsys TCAD product announced they use 128 bit float for numerically stable reason. I will try some more robust methods first. float128 seems not compatible with most external direct solvers. On Sun, Sep 18, 2011 at 17:07, Gong Ding wrote: Do you mean GMRES-like method will bebenefited from float 128? It could be, especially if you use a finite difference Jacobian. If you suspect precision issues, you can try more robust orthogonalization -ksp_gmres_modifiedgramschmidt or -ksp_type gcr, or you can just try using float128. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Sun Sep 18 11:46:03 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Sun, 18 Sep 2011 11:46:03 -0500 Subject: [petsc-users] Does float128 helps ksp solver convergence In-Reply-To: <17215513.231541316362182546.JavaMail.coremail@mail.ustc.edu> References: <22443916.231151316347488701.JavaMail.coremail@mail.ustc.edu> <29833061.231461316358440034.JavaMail.coremail@mail.ustc.edu> <17215513.231541316362182546.JavaMail.coremail@mail.ustc.edu> Message-ID: <944CF49E-DA21-4045-B80F-E7743DEF8F2D@mcs.anl.gov> On Sep 18, 2011, at 11:09 AM, Gong Ding wrote: > Thanks, the synopsys TCAD product announced they use 128 bit float for numerically stable reason. > I will try some more robust methods first. > float128 seems not compatible with most external direct solvers. Semiconductor simulations are notoriously ill-conditioned; personally I would always run them in quad precision. But note that the extra precision does not make the iterative solvers any faster they will still be very slow without very good preconditioners (which are also notoriously difficult for semiconductors.) I recommend contacting the SuperLU_Dist and MUMPS people and asking them to implement a quad precision version; I suspect it might not be difficult for SuperLU_Dist but since MUMPS lives on top of the rickety infrastructure of ScaLAPACK it is probably very difficult for them. So contact Sherrie Li or see about modifiying SuperLU_Dist yourself. Barry > > On Sun, Sep 18, 2011 at 17:07, Gong Ding wrote: > Do you mean GMRES-like method will be benefited from float 128? > > It could be, especially if you use a finite difference Jacobian. If you suspect precision issues, you can try more robust orthogonalization -ksp_gmres_modifiedgramschmidt or -ksp_type gcr, or you can just try using float128. > > From pwu at mymail.mines.edu Sun Sep 18 11:51:51 2011 From: pwu at mymail.mines.edu (Panruo Wu) Date: Sun, 18 Sep 2011 10:51:51 -0600 Subject: [petsc-users] a quick question about index out of range in MatSetValues Message-ID: Dear list, I read in the MatSetValues documentation and found that Negative indices may be passed in idxm and idxn, these rows and columns are > simply ignored I wonder if it's still the case when passing row or column indices larger than matrix dimensions. If it's true it's very handy to assemble matrix of homogeneous dirichlet boundary condition. Thanks, Robb -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Sun Sep 18 18:50:48 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Sun, 18 Sep 2011 18:50:48 -0500 Subject: [petsc-users] a quick question about index out of range in MatSetValues In-Reply-To: References: Message-ID: On Sep 18, 2011, at 11:51 AM, Panruo Wu wrote: > Dear list, > > I read in the MatSetValues documentation and found that > > Negative indices may be passed in idxm and idxn, these rows and columns are simply ignored > > I wonder if it's still the case when passing row or column indices larger than matrix dimensions. > If it's true it's very handy to assemble matrix of homogeneous dirichlet boundary condition. No, you need to label those nodes with negative indices to get them ignored; larger than N will generate an error. Note if you provide your own local to global mapping to the matrix you can map the "extra" large values to negative values and thus have them ignored. But I recommend since labeling all nodes you want ignored with negative indices. Barry > > Thanks, > Robb From agrayver at gfz-potsdam.de Mon Sep 19 01:53:45 2011 From: agrayver at gfz-potsdam.de (Alexander Grayver) Date: Mon, 19 Sep 2011 08:53:45 +0200 Subject: [petsc-users] [petsc-3.2] MatView on dense matrix In-Reply-To: References: <4E73273D.4060601@gfz-potsdam.de> Message-ID: <4E76E6F9.30606@gfz-potsdam.de> Hello, Is that possible to get fixed version? Regards, Alexander On 16.09.2011 19:59, Barry Smith wrote: > The problem is from the C values of these parameters have gotten out of sync with the Fortran values. > > Satish, > > Could you please fix the finclude/petscviewer.h values to match the C in petsc-3.2 and send a patch? > > Thanks > > Barry > > > On Sep 16, 2011, at 5:38 AM, Alexander Grayver wrote: > >> Hello, >> >> With petsc-3.1 I used this code to output MPI dense matrix: >> >> PetscViewer :: dviewer >> call PetscViewerBinaryOpen(MPI_COMM_WORLD,'A.dat',FILE_MODE_WRITE,dviewer,ierr); CHKERRQ(ierr) >> call PetscViewerSetFormat(dviewer,PETSC_VIEWER_NATIVE,ierr); CHKERRQ(ierr) >> call MatView(A,dviewer,ierr); CHKERRQ(ierr) >> call PetscViewerDestroy(dviewer,ierr); CHKERRQ(ierr) >> >> With petsc-3.2 this code fails with error: >> >> [7]PETSC ERROR: --------------------- Error Message ------------------------------------ >> [7]PETSC ERROR: No support for this operation for this object type! >> [7]PETSC ERROR: To store a parallel dense matrix you must first call PetscViewerSetFormat(viewer,PETSC_VIEWER_NATIVE)! >> [7]PETSC ERROR: ------------------------------------------------------------------------ >> [7]PETSC ERROR: Petsc Release Version 3.2.0, Patch 1, Mon Sep 12 16:01:51 CDT 2011 >> [7]PETSC ERROR: See docs/changes/index.html for recent updates. >> [7]PETSC ERROR: See docs/faq.html for hints about trouble shooting. >> [7]PETSC ERROR: See docs/index.html for manual pages. >> [7]PETSC ERROR: ------------------------------------------------------------------------ >> [7]PETSC ERROR: /home/run/test on a openmpi-i named node205 by user Fri Sep 16 09:17:13 2011 >> [7]PETSC ERROR: Libraries linked from /home/lib/petsc-3.2-p1/openmpi-intel-complex-debug-f/lib >> [7]PETSC ERROR: Configure run at Wed Sep 14 14:32:55 2011 >> [7]PETSC ERROR: Configure options --with-petsc-arch=openmpi-intel-complex-debug-f --with-fortran-interfaces=1 --with-mpi-dir=/opt/mpi/intel/openmpi-1.4.2 --with-scalar-type=complex --with-blas-lapack-dir=/opt/intel/Compiler/11.1/072/mkl/lib/em64t --with-precision=double --with-x=0 >> [7]PETSC ERROR: ------------------------------------------------------------------------ >> [7]PETSC ERROR: MatView_MPIDense_Binary() line 667 in /home/lib/petsc-3.2-p1/src/mat/impls/dense/mpi/mpidense.c >> [7]PETSC ERROR: MatView_MPIDense() line 789 in /home/lib/petsc-3.2-p1/src/mat/impls/dense/mpi/mpidense.c >> [7]PETSC ERROR: MatView() line 757 in /home/lib/petsc-3.2-p1/src/mat/interface/matrix.c >> MatView_MPIDense() line 789 in /home/lib/petsc-3.2-p1/src/mat/impls/dense/mpi/mpidense.c >> >> See full error attached. Is there something new in petsc-3.2 concerning matrix output? >> >> Regards, >> Alexander >> From delalf at scorec.rpi.edu Mon Sep 19 05:26:30 2011 From: delalf at scorec.rpi.edu (fabien delalondre) Date: Mon, 19 Sep 2011 12:26:30 +0200 Subject: [petsc-users] Reusing preconditionner values Message-ID: Hi, To maybe try to speed up my resolution, I would like to not compute the values of the preconditionner at every time step, I was wondering if I could do the following: 1) At a certain point in time, I ll use KSPGetPC to get the preconditionner 2) I would then use KSPSetPC to set the values of the preconditionner, assuming this will prevent the calculation of the preconditionner values. Is it a correct assumption or should I use a parameter to enforce not re-calculating the precontionner values ? Note that I am using a GMRES solver with a block jacobi preconditionner (superlu). Thank you Fabien -- Fabien Delalondre, PhD. Senior Research Associate, Scientific Computation Research Center (SCOREC). Rensselaer Polytechnic Institute (RPI), Troy, NY. Email: delalf at scorec.rpi.edu, Phone: (518)-276-8045 -------------- next part -------------- An HTML attachment was scrubbed... URL: From jedbrown at mcs.anl.gov Mon Sep 19 06:08:35 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Mon, 19 Sep 2011 13:08:35 +0200 Subject: [petsc-users] Reusing preconditionner values In-Reply-To: References: Message-ID: On Mon, Sep 19, 2011 at 12:26, fabien delalondre wrote: > To maybe try to speed up my resolution, I would like to not compute the > values of the preconditionner at every time step, I was wondering if I could > do the following: > 1) At a certain point in time, I ll use KSPGetPC to > get the preconditionner > 2) I would then use KSPSetPC to > set the values of the preconditionner, assuming this will prevent the > calculation of the preconditionner values. Is it a correct assumption or > should I use a parameter to enforce not re-calculating the precontionner > values ? Note that I am using a GMRES solver with a block jacobi > preconditionner (superlu). > If you call KSP directly, you should pass SAME_PRECONDITIONER to KSPSetOperators(). This will use the new matrix, but not recompute the preconditioner. It will work with any preconditioner. If you use SNES or TS, you they also have options for lagging the preconditioner and/or Jacobian. -------------- next part -------------- An HTML attachment was scrubbed... URL: From delalf at scorec.rpi.edu Mon Sep 19 10:47:53 2011 From: delalf at scorec.rpi.edu (fabien delalondre) Date: Mon, 19 Sep 2011 17:47:53 +0200 Subject: [petsc-users] Latest release and block jacobi preconditioner Message-ID: Dear All, I was wondering whether the latest release includes all block jacobi capabilities supported in the development version. If that's not the case, can you please let me know what has not been included as part of this release. Thank you, Fabien -- Fabien Delalondre, PhD. Senior Research Associate, Scientific Computation Research Center (SCOREC). Rensselaer Polytechnic Institute (RPI), Troy, NY. Email: delalf at scorec.rpi.edu, Phone: (518)-276-8045 -------------- next part -------------- An HTML attachment was scrubbed... URL: From pwu at mymail.mines.edu Mon Sep 19 10:48:56 2011 From: pwu at mymail.mines.edu (Panruo Wu) Date: Mon, 19 Sep 2011 09:48:56 -0600 Subject: [petsc-users] a quick question about index out of range in MatSetValues In-Reply-To: References: Message-ID: Got it, Thank you. Robb On Sun, Sep 18, 2011 at 5:50 PM, Barry Smith wrote: > > On Sep 18, 2011, at 11:51 AM, Panruo Wu wrote: > > > Dear list, > > > > I read in the MatSetValues documentation and found that > > > > Negative indices may be passed in idxm and idxn, these rows and columns > are simply ignored > > > > I wonder if it's still the case when passing row or column indices larger > than matrix dimensions. > > If it's true it's very handy to assemble matrix of homogeneous dirichlet > boundary condition. > > No, you need to label those nodes with negative indices to get them > ignored; larger than N will generate an error. > > Note if you provide your own local to global mapping to the matrix you can > map the "extra" large values to negative values and thus have them ignored. > But I recommend since labeling all nodes you want ignored with negative > indices. > > Barry > > > > > Thanks, > > Robb > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Mon Sep 19 11:11:54 2011 From: knepley at gmail.com (Matthew Knepley) Date: Mon, 19 Sep 2011 11:11:54 -0500 Subject: [petsc-users] Latest release and block jacobi preconditioner In-Reply-To: References: Message-ID: On Mon, Sep 19, 2011 at 10:47 AM, fabien delalondre wrote: > Dear All, > > I was wondering whether the latest release includes all block jacobi > capabilities supported in the development version. If that's not the case, > can you please let me know what has not been included as part of this > release. > The release is exactly the dev version on the day of release. Matt > Thank you, > > Fabien > > -- > Fabien Delalondre, PhD. > Senior Research Associate, Scientific Computation Research Center (SCOREC). > Rensselaer Polytechnic Institute (RPI), Troy, NY. > Email: delalf at scorec.rpi.edu, Phone: (518)-276-8045 > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From zhenglun.wei at gmail.com Mon Sep 19 18:25:48 2011 From: zhenglun.wei at gmail.com (Alan Wei) Date: Mon, 19 Sep 2011 18:25:48 -0500 Subject: [petsc-users] Convergence Criteria? Message-ID: Dear folks, I hope you guys are having a nice day. I'm reading the /src/ksp/ksp/examples/tutorials/ex29.c.html and wonder how to set up the convergence criteria? Currently I use it as a poisson solver to solve a Poisson Equation with three direction Neumann BC's and one direction Diriechlet BC's. It seems very bad on it. thanks in advance, Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: From 5202222ghj at 163.com Mon Sep 19 19:14:28 2011 From: 5202222ghj at 163.com (yuxuan) Date: Tue, 20 Sep 2011 08:14:28 +0800 (CST) Subject: [petsc-users] Jacobian of TS problem Message-ID: <4f1c6c0.b124.132842f0bda.Coremail.5202222ghj@163.com> Hi all, I am running the example: http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-3.1/src/ts/examples/tutorials/ex7.c I found that there is no user-provided jacobian data structure function for Finite Difference Jacobian Approximation. And also how can I have look at the values of the Jacobian Matrix. The Matview can show the value, however how I can call it within the solving processes. Thanks Xuan -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Mon Sep 19 20:41:25 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Mon, 19 Sep 2011 20:41:25 -0500 Subject: [petsc-users] Jacobian of TS problem In-Reply-To: <4f1c6c0.b124.132842f0bda.Coremail.5202222ghj@163.com> References: <4f1c6c0.b124.132842f0bda.Coremail.5202222ghj@163.com> Message-ID: <3BEA937B-6226-4C94-B25E-3385443F82E7@mcs.anl.gov> Switch to petsc-3.2 it has a much improved set of ODE integraters The matrix generation (and its non-zero pattern) are handled in DMGetMatrix(). To get access to the values of Jacobian at each Newton step provide your own custom monitor with SNESMonitorSet() then inside your custom monitor call SNESGetJacobian() and call MatView() on the retrieved matrix. Barry On Sep 19, 2011, at 7:14 PM, yuxuan wrote: > Hi all, > > I am running the example: > http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-3.1/src/ts/examples/tutorials/ex7.c > > I found that there is no user-provided jacobian data structure function for Finite Difference Jacobian Approximation. And also how can I have look at the values of the Jacobian Matrix. The Matview can show the value, however how I can call it within the solving processes. > > Thanks > > Xuan > > > From bsmith at mcs.anl.gov Mon Sep 19 20:43:45 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Mon, 19 Sep 2011 20:43:45 -0500 Subject: [petsc-users] Convergence Criteria? In-Reply-To: References: Message-ID: <1C05DB25-FA8B-49A9-8B33-D606C3F7126C@mcs.anl.gov> On Sep 19, 2011, at 6:25 PM, Alan Wei wrote: > Dear folks, > I hope you guys are having a nice day. > I'm reading the /src/ksp/ksp/examples/tutorials/ex29.c.html and wonder how to set up the convergence criteria? -ksp_rtol 1.e-10 for example Run with -ksp_monitor_true_residual -ksp_converged_reason > Currently I use it as a poisson solver to solve a Poisson Equation with three direction Neumann BC's and one direction Diriechlet BC's. It seems very bad on it. Hmm, multigrid should likely converge well. Are you sure you've set the BC's correctly? Barry > > thanks in advance, > Alan From zhenglun.wei at gmail.com Mon Sep 19 21:28:51 2011 From: zhenglun.wei at gmail.com (Alan Wei) Date: Mon, 19 Sep 2011 21:28:51 -0500 Subject: [petsc-users] Convergence Criteria? In-Reply-To: <1C05DB25-FA8B-49A9-8B33-D606C3F7126C@mcs.anl.gov> References: <1C05DB25-FA8B-49A9-8B33-D606C3F7126C@mcs.anl.gov> Message-ID: Thanks, Dr. Smith. I guess my BC's settings are wrong. I will check and contact you if I have further questions. Thanks again. best, Alan On Mon, Sep 19, 2011 at 8:43 PM, Barry Smith wrote: > > On Sep 19, 2011, at 6:25 PM, Alan Wei wrote: > > > Dear folks, > > I hope you guys are having a nice day. > > I'm reading the /src/ksp/ksp/examples/tutorials/ex29.c.html and > wonder how to set up the convergence criteria? > > -ksp_rtol 1.e-10 for example > > Run with -ksp_monitor_true_residual -ksp_converged_reason > > > > Currently I use it as a poisson solver to solve a Poisson Equation with > three direction Neumann BC's and one direction Diriechlet BC's. It seems > very bad on it. > > Hmm, multigrid should likely converge well. Are you sure you've set the > BC's correctly? > > Barry > > > > > thanks in advance, > > Alan > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From balay at mcs.anl.gov Mon Sep 19 22:16:04 2011 From: balay at mcs.anl.gov (Satish Balay) Date: Mon, 19 Sep 2011 22:16:04 -0500 (CDT) Subject: [petsc-users] [petsc-3.2] MatView on dense matrix In-Reply-To: <4E76E6F9.30606@gfz-potsdam.de> References: <4E73273D.4060601@gfz-potsdam.de> <4E76E6F9.30606@gfz-potsdam.de> Message-ID: Can you try the attached patch - and see if it fixes the problem? cd petsc-3.2-p2 patch -Np1 < viewer-f.patch Satish On Mon, 19 Sep 2011, Alexander Grayver wrote: > Hello, > > Is that possible to get fixed version? > > Regards, > Alexander > > On 16.09.2011 19:59, Barry Smith wrote: > > The problem is from the C values of these parameters have gotten out of > > sync with the Fortran values. > > > > Satish, > > > > Could you please fix the finclude/petscviewer.h values to match the C > > in petsc-3.2 and send a patch? > > > > Thanks > > > > Barry > > > > > > On Sep 16, 2011, at 5:38 AM, Alexander Grayver wrote: > > > > > Hello, > > > > > > With petsc-3.1 I used this code to output MPI dense matrix: > > > > > > PetscViewer :: dviewer > > > call > > > PetscViewerBinaryOpen(MPI_COMM_WORLD,'A.dat',FILE_MODE_WRITE,dviewer,ierr); > > > CHKERRQ(ierr) > > > call PetscViewerSetFormat(dviewer,PETSC_VIEWER_NATIVE,ierr); CHKERRQ(ierr) > > > call MatView(A,dviewer,ierr); CHKERRQ(ierr) > > > call PetscViewerDestroy(dviewer,ierr); CHKERRQ(ierr) > > > > > > With petsc-3.2 this code fails with error: > > > > > > [7]PETSC ERROR: --------------------- Error Message > > > ------------------------------------ > > > [7]PETSC ERROR: No support for this operation for this object type! > > > [7]PETSC ERROR: To store a parallel dense matrix you must first call > > > PetscViewerSetFormat(viewer,PETSC_VIEWER_NATIVE)! > > > [7]PETSC ERROR: > > > ------------------------------------------------------------------------ > > > [7]PETSC ERROR: Petsc Release Version 3.2.0, Patch 1, Mon Sep 12 16:01:51 > > > CDT 2011 > > > [7]PETSC ERROR: See docs/changes/index.html for recent updates. > > > [7]PETSC ERROR: See docs/faq.html for hints about trouble shooting. > > > [7]PETSC ERROR: See docs/index.html for manual pages. > > > [7]PETSC ERROR: > > > ------------------------------------------------------------------------ > > > [7]PETSC ERROR: /home/run/test on a openmpi-i named node205 by user Fri > > > Sep 16 09:17:13 2011 > > > [7]PETSC ERROR: Libraries linked from > > > /home/lib/petsc-3.2-p1/openmpi-intel-complex-debug-f/lib > > > [7]PETSC ERROR: Configure run at Wed Sep 14 14:32:55 2011 > > > [7]PETSC ERROR: Configure options > > > --with-petsc-arch=openmpi-intel-complex-debug-f > > > --with-fortran-interfaces=1 --with-mpi-dir=/opt/mpi/intel/openmpi-1.4.2 > > > --with-scalar-type=complex > > > --with-blas-lapack-dir=/opt/intel/Compiler/11.1/072/mkl/lib/em64t > > > --with-precision=double --with-x=0 > > > [7]PETSC ERROR: > > > ------------------------------------------------------------------------ > > > [7]PETSC ERROR: MatView_MPIDense_Binary() line 667 in > > > /home/lib/petsc-3.2-p1/src/mat/impls/dense/mpi/mpidense.c > > > [7]PETSC ERROR: MatView_MPIDense() line 789 in > > > /home/lib/petsc-3.2-p1/src/mat/impls/dense/mpi/mpidense.c > > > [7]PETSC ERROR: MatView() line 757 in > > > /home/lib/petsc-3.2-p1/src/mat/interface/matrix.c > > > MatView_MPIDense() line 789 in > > > /home/lib/petsc-3.2-p1/src/mat/impls/dense/mpi/mpidense.c > > > > > > See full error attached. Is there something new in petsc-3.2 concerning > > > matrix output? > > > > > > Regards, > > > Alexander > > > > > -------------- next part -------------- diff -r 143491d19702 include/finclude/petscviewer.h --- a/include/finclude/petscviewer.h Mon Sep 19 15:07:17 2011 -0500 +++ b/include/finclude/petscviewer.h Mon Sep 19 22:14:41 2011 -0500 @@ -30,7 +30,7 @@ PetscEnum PETSC_VIEWER_ASCII_MATHEMATICA PetscEnum PETSC_VIEWER_ASCII_IMPL PetscEnum PETSC_VIEWER_ASCII_INFO - PetscEnum PETSC_VIEWER_ASCII_INFO_DETAIL + PetscEnum PETSC_VIEWER_ASCII_INFO_DETAIL PetscEnum PETSC_VIEWER_ASCII_COMMON PetscEnum PETSC_VIEWER_ASCII_SYMMODU PetscEnum PETSC_VIEWER_ASCII_INDEX @@ -40,10 +40,14 @@ PetscEnum PETSC_VIEWER_ASCII_VTK_CELL PetscEnum PETSC_VIEWER_ASCII_VTK_COORDS PetscEnum PETSC_VIEWER_ASCII_PCICE - PetscEnum PETSC_VIEWER_ASCII_PYLITH - PetscEnum PETSC_VIEWER_ASCII_PYLITH_LOCAL PetscEnum PETSC_VIEWER_ASCII_PYTHON PetscEnum PETSC_VIEWER_ASCII_FACTOR_INFO + PetscEnum PETSC_VIEWER_DRAW_BASIC + PetscEnum PETSC_VIEWER_DRAW_LG + PetscEnum PETSC_VIEWER_DRAW_CONTOUR + PetscEnum PETSC_VIEWER_DRAW_PORTS + PetscEnum PETSC_VIEWER_NATIVE + PetscEnum PETSC_VIEWER_NOFORMAT parameter (PETSC_VIEWER_DEFAULT = 0) parameter (PETSC_VIEWER_ASCII_MATLAB = 1) @@ -60,24 +64,14 @@ parameter (PETSC_VIEWER_ASCII_VTK_CELL = 12) parameter (PETSC_VIEWER_ASCII_VTK_COORDS = 13) parameter (PETSC_VIEWER_ASCII_PCICE = 14) - parameter (PETSC_VIEWER_ASCII_PYLITH = 15) - parameter (PETSC_VIEWER_ASCII_PYLITH_LOCAL = 16) - parameter (PETSC_VIEWER_ASCII_PYTHON = 17) - parameter (PETSC_VIEWER_ASCII_FACTOR_INFO = 18) - - PetscEnum PETSC_VIEWER_DRAW_BASIC - PetscEnum PETSC_VIEWER_DRAW_LG - PetscEnum PETSC_VIEWER_DRAW_CONTOUR - PetscEnum PETSC_VIEWER_DRAW_PORTS - - parameter (PETSC_VIEWER_DRAW_BASIC = 19) - parameter (PETSC_VIEWER_DRAW_LG = 20) - parameter (PETSC_VIEWER_DRAW_CONTOUR = 21) - parameter (PETSC_VIEWER_DRAW_PORTS = 22) - PetscEnum PETSC_VIEWER_NATIVE - parameter (PETSC_VIEWER_NATIVE = 23) - PetscEnum PETSC_VIEWER_NOFORMAT - parameter (PETSC_VIEWER_NOFORMAT = 24) + parameter (PETSC_VIEWER_ASCII_PYTHON = 15) + parameter (PETSC_VIEWER_ASCII_FACTOR_INFO = 16) + parameter (PETSC_VIEWER_DRAW_BASIC = 17) + parameter (PETSC_VIEWER_DRAW_LG = 18) + parameter (PETSC_VIEWER_DRAW_CONTOUR = 19) + parameter (PETSC_VIEWER_DRAW_PORTS = 20) + parameter (PETSC_VIEWER_NATIVE = 21) + parameter (PETSC_VIEWER_NOFORMAT = 22) ! ! End of Fortran include file for the PetscViewer package in PETSc diff -r 143491d19702 include/finclude/petscviewerdef.h --- a/include/finclude/petscviewerdef.h Mon Sep 19 15:07:17 2011 -0500 +++ b/include/finclude/petscviewerdef.h Mon Sep 19 22:14:41 2011 -0500 @@ -18,9 +18,11 @@ #define PETSCVIEWERBINARY 'binary' #define PETSCVIEWERSTRING 'string' #define PETSCVIEWERDRAW 'draw' +#define PETSCVIEWERVU 'vu' +#define PETSCVIEWERMATHEMATICA 'mathematica' +#define PETSCVIEWERNETCDF 'netcdf' +#define PETSCVIEWERHDF5 'hdf5' +#define PETSCVIEWERMATLAB 'matlab' #define PETSCVIEWERAMS 'ams' -#define PETSCVIEWERHDF5 'hdf5' -#define PETSCVIEWERNETCDF 'netcdf' -#define PETSCVIEWERMATLAB 'matlab' #endif From agrayver at gfz-potsdam.de Tue Sep 20 02:44:11 2011 From: agrayver at gfz-potsdam.de (Alexander Grayver) Date: Tue, 20 Sep 2011 09:44:11 +0200 Subject: [petsc-users] [petsc-3.2] MatView on dense matrix In-Reply-To: References: <4E73273D.4060601@gfz-potsdam.de> <4E76E6F9.30606@gfz-potsdam.de> Message-ID: <4E78444B.4080202@gfz-potsdam.de> Hi Satish, The problem seems to be fixed now. Thanks! Regards, Alexander On 20.09.2011 05:16, Satish Balay wrote: > Can you try the attached patch - and see if it fixes the problem? > > cd petsc-3.2-p2 > patch -Np1< viewer-f.patch > > Satish > > On Mon, 19 Sep 2011, Alexander Grayver wrote: > >> Hello, >> >> Is that possible to get fixed version? >> >> Regards, >> Alexander >> >> On 16.09.2011 19:59, Barry Smith wrote: >>> The problem is from the C values of these parameters have gotten out of >>> sync with the Fortran values. >>> >>> Satish, >>> >>> Could you please fix the finclude/petscviewer.h values to match the C >>> in petsc-3.2 and send a patch? >>> >>> Thanks >>> >>> Barry >>> >>> >>> On Sep 16, 2011, at 5:38 AM, Alexander Grayver wrote: >>> >>>> Hello, >>>> >>>> With petsc-3.1 I used this code to output MPI dense matrix: >>>> >>>> PetscViewer :: dviewer >>>> call >>>> PetscViewerBinaryOpen(MPI_COMM_WORLD,'A.dat',FILE_MODE_WRITE,dviewer,ierr); >>>> CHKERRQ(ierr) >>>> call PetscViewerSetFormat(dviewer,PETSC_VIEWER_NATIVE,ierr); CHKERRQ(ierr) >>>> call MatView(A,dviewer,ierr); CHKERRQ(ierr) >>>> call PetscViewerDestroy(dviewer,ierr); CHKERRQ(ierr) >>>> >>>> With petsc-3.2 this code fails with error: >>>> >>>> [7]PETSC ERROR: --------------------- Error Message >>>> ------------------------------------ >>>> [7]PETSC ERROR: No support for this operation for this object type! >>>> [7]PETSC ERROR: To store a parallel dense matrix you must first call >>>> PetscViewerSetFormat(viewer,PETSC_VIEWER_NATIVE)! >>>> [7]PETSC ERROR: >>>> ------------------------------------------------------------------------ >>>> [7]PETSC ERROR: Petsc Release Version 3.2.0, Patch 1, Mon Sep 12 16:01:51 >>>> CDT 2011 >>>> [7]PETSC ERROR: See docs/changes/index.html for recent updates. >>>> [7]PETSC ERROR: See docs/faq.html for hints about trouble shooting. >>>> [7]PETSC ERROR: See docs/index.html for manual pages. >>>> [7]PETSC ERROR: >>>> ------------------------------------------------------------------------ >>>> [7]PETSC ERROR: /home/run/test on a openmpi-i named node205 by user Fri >>>> Sep 16 09:17:13 2011 >>>> [7]PETSC ERROR: Libraries linked from >>>> /home/lib/petsc-3.2-p1/openmpi-intel-complex-debug-f/lib >>>> [7]PETSC ERROR: Configure run at Wed Sep 14 14:32:55 2011 >>>> [7]PETSC ERROR: Configure options >>>> --with-petsc-arch=openmpi-intel-complex-debug-f >>>> --with-fortran-interfaces=1 --with-mpi-dir=/opt/mpi/intel/openmpi-1.4.2 >>>> --with-scalar-type=complex >>>> --with-blas-lapack-dir=/opt/intel/Compiler/11.1/072/mkl/lib/em64t >>>> --with-precision=double --with-x=0 >>>> [7]PETSC ERROR: >>>> ------------------------------------------------------------------------ >>>> [7]PETSC ERROR: MatView_MPIDense_Binary() line 667 in >>>> /home/lib/petsc-3.2-p1/src/mat/impls/dense/mpi/mpidense.c >>>> [7]PETSC ERROR: MatView_MPIDense() line 789 in >>>> /home/lib/petsc-3.2-p1/src/mat/impls/dense/mpi/mpidense.c >>>> [7]PETSC ERROR: MatView() line 757 in >>>> /home/lib/petsc-3.2-p1/src/mat/interface/matrix.c >>>> MatView_MPIDense() line 789 in >>>> /home/lib/petsc-3.2-p1/src/mat/impls/dense/mpi/mpidense.c >>>> >>>> See full error attached. Is there something new in petsc-3.2 concerning >>>> matrix output? >>>> >>>> Regards, >>>> Alexander >>>> >> From gdiso at ustc.edu Tue Sep 20 03:08:05 2011 From: gdiso at ustc.edu (Gong Ding) Date: Tue, 20 Sep 2011 16:08:05 +0800 (CST) Subject: [petsc-users] floating region help Message-ID: <588771.234921316506085787.JavaMail.coremail@mail.ustc.edu> Hi, I am testing the floating region problem. The model problem can be described as a 1D poisson equation. The potential of left end is fixed as 1V, the right end is fixed as 2V. In the left/right part, the conductance is low, i.e. 1 In the middle part, the conductance is much higher, i.e. 1e6 *-1-*-1-*-1-*-1-*-1-*-1e6-*-1e6-*-1e6-*-1-*-1-*-1-*-1-* As a result, the potential in the middle region is weakly controled by the left and right endpoint. Since the conductance of left part and right part is low, the middle region is floating. Is it possible to use null space method here? I guess the null space is the vector that the location of corresponding point in the floating region is 1.0, otherwise 0.0. And here we solve Ax=b with null vector n. The solution xn has no component parallel to n. Assuming the real solution should be xn+a*n, Then A(xn+a*n)=b, and a can be solved from a*n = b-Axn. Am I right? Or there are some better method to process the floating region? Gong Ding -------------- next part -------------- An HTML attachment was scrubbed... URL: From jedbrown at mcs.anl.gov Tue Sep 20 03:35:00 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Tue, 20 Sep 2011 10:35:00 +0200 Subject: [petsc-users] floating region help In-Reply-To: <588771.234921316506085787.JavaMail.coremail@mail.ustc.edu> References: <588771.234921316506085787.JavaMail.coremail@mail.ustc.edu> Message-ID: 2011/9/20 Gong Ding > Since the conductance of left part and right part is low, the middle region > is floating. > Not really floating, just sensitive to parameters. It has a unique value, of physical interest, but if you change the conductivity or thickness of the thin region on either side, then it's value changes significantly. This is an ill-conditioned problem, but it's ill-conditioned for physically meaningful reasons. It's not singular and the smallest singular value has great significance, so a null space method is not appropriate. You could reformulate for a slightly different problem which would have a constant null space, but not for this problem. -------------- next part -------------- An HTML attachment was scrubbed... URL: From gdiso at ustc.edu Tue Sep 20 04:17:17 2011 From: gdiso at ustc.edu (Gong Ding) Date: Tue, 20 Sep 2011 17:17:17 +0800 (CST) Subject: [petsc-users] floating region help In-Reply-To: References: <588771.234921316506085787.JavaMail.coremail@mail.ustc.edu> Message-ID: <1525625.235221316510237990.JavaMail.coremail@mail.ustc.edu> 2011/9/20 Gong Ding Since the conductance of left part and right part is low, the middle region is floating. Not really floating, just sensitive to parameters. It has a unique value, of physical interest, but if you change the conductivity or thickness of the thin region on either side, then it's value changes significantly. This is an ill-conditioned problem, but it's ill-conditioned for physically meaningful reasons. It's not singular and the smallest singular value has great significance, so a null space method is not appropriate. You could reformulate for a slightly different problem which would have a constant null space, but not for this problem. Thanks. Any recommends for solving this type of problems? It's our bottleneck now. Direct solver works, but not efficient for large 3D problem. Krylov space based solver seems not rubost enough. A more exact ILU preconditioner may help? -------------- next part -------------- An HTML attachment was scrubbed... URL: From jedbrown at mcs.anl.gov Tue Sep 20 04:27:45 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Tue, 20 Sep 2011 11:27:45 +0200 Subject: [petsc-users] floating region help In-Reply-To: <1525625.235221316510237990.JavaMail.coremail@mail.ustc.edu> References: <588771.234921316506085787.JavaMail.coremail@mail.ustc.edu> <1525625.235221316510237990.JavaMail.coremail@mail.ustc.edu> Message-ID: On Tue, Sep 20, 2011 at 11:17, Gong Ding wrote: > Thanks. Any recommends for solving this type of problems? > It's our bottleneck now. > Direct solver works, but not efficient for large 3D problem. > Krylov space based solver seems not rubost enough. > A more exact ILU preconditioner may help? > You can use domain decomposition with the jump aligned to subdomain boundaries. There are a variety of schemes with convergence rate independent of the jump, from both the overlapping Schwarz and Neumann-Neumann families. Can you describe the 3D problem in more detail? -------------- next part -------------- An HTML attachment was scrubbed... URL: From agrayver at gfz-potsdam.de Tue Sep 20 04:40:35 2011 From: agrayver at gfz-potsdam.de (Alexander Grayver) Date: Tue, 20 Sep 2011 11:40:35 +0200 Subject: [petsc-users] [petsc-3.2] LSQR new convergence test Message-ID: <4E785F93.8020106@gfz-potsdam.de> Hello, In comparison with petsc-3.1-p7 in the latest petsc LSQR solver behaves differently. In petsc31 I had: 0 KSP Residual norm 9.386670021557e-17 ... 95 KSP Residual norm 9.341367317075e-18 Linear solve converged due to CONVERGED_RTOL iterations 95 KSP Object: type: lsqr maximum iterations=200, initial guess is zero tolerances: relative=0.1, absolute=1e-50, divergence=10000 left preconditioning using PRECONDITIONED norm type for convergence test PC Object: type: none linear system matrix = precond matrix: Matrix Object: type=shell, rows=19584, cols=19584 relative residual norm = 0.9951737192872134E-01 Now, with petsc32 I have: 0 KSP Residual norm 9.386670021557e-17 1 KSP Residual norm 8.258308650175e-17 Linear solve converged due to CONVERGED_RTOL_NORMAL iterations 1 KSP Object: 12 MPI processes type: lsqr maximum iterations=200, initial guess is zero tolerances: relative=0.1, absolute=1e-50, divergence=10000 left preconditioning using UNPRECONDITIONED norm type for convergence test PC Object: 12 MPI processes type: none linear system matrix = precond matrix: Matrix Object: 12 MPI processes type: shell rows=19584, cols=19584 relative residual norm = 0.8797910900468064E+00 So I found new convergence test here which sets CONVERGED_RTOL_NORMAL: http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/KSP/KSPLSQRDefaultConverged.html The question is, how to interpret this new test and make it works properly for me? Thanks in advance. Regards, Alexander From matijakecman at gmail.com Tue Sep 20 04:45:37 2011 From: matijakecman at gmail.com (Matija Kecman) Date: Tue, 20 Sep 2011 10:45:37 +0100 Subject: [petsc-users] Question about ksp ex3.c Message-ID: Hi all, I have been playing around with ksp/examples/tutorials/ex3.c, a brief summary of this example follows: The Laplace equation in two-dimensions on a unit square [0,1]x[0,1] is solved using a structured mesh of linear quadrilateral finite elements, m elements in each direction. Dirichlet boundary conditions are specified: u_hat = y on the boundary, which means the exact solution is u_star = y. The global system matrix dimension is specified and PETSc decides how to split the matrix amongst processors. The DOFs are numbered 'naturally' from the origin, left to right, bottom to top. Boundary conditions are applied by zeroing the rows of the matrix corresponding to the boundary DOFs, inserting 1's on the diagonal entries of these rows and inserting the boundary conditions into the global RHS vector entries corresponding to these DOFs. I ran the following uniprocess tests (to run the direct solve tests I had to comment the line 'ierr = KSPSetInitialGuessNonzero(ksp,PETSC_TRUE);CHKERRQ(ierr);' but otherwise I made no changes to the example code): ---Tests with 5x5 element mesh (36 DOFs)--- $ mpirun -np 1 ./ex3 -ksp_type gmres Norm of error 1.73622e-06 Iterations 3 $ mpirun -np 1 ./ex3 -ksp_type bcgs Norm of error 1.5157e-07 Iterations 2 $ mpirun -np 1 ./ex3 -ksp_type cg Norm of error 1.88557e-06 Iterations 3 $ mpirun -np 1 ./ex3 -ksp_type gmres -pc_type none Norm of error < 1.e-12 Iterations 7 $ mpirun -np 1 ./ex3 -ksp_type bcgs -pc_type none Norm of error 0.00109538 Iterations 6 $ mpirun -np 1 ./ex3 -ksp_type cg -pc_type none Norm of error < 1.e-12 Iterations 7 $ mpirun -np 1 ./ex3 -ksp_type preonly -pc_type lu Norm of error < 1.e-12 Iterations 1 ---Tests with 100x100 element mesh (10201 DOFs)--- $ mpirun -np 1 ./ex3 -ksp_type gmres -m 100 Norm of error 2.90652e-05 Iterations 32 $ mpirun -np 1 ./ex3 -ksp_type bcgs -m 100 Norm of error 3.92733e-05 Iterations 24 $ mpirun -np 1 ./ex3 -ksp_type cg -m 100 Norm of error 2.4332e-05 Iterations 32 $ mpirun -np 1 ./ex3 -ksp_type gmres -pc_type none -m 100 Norm of error 0.570146 Iterations 0 $ mpirun -np 1 ./ex3 -ksp_type bcgs -pc_type none -m 100 Norm of error 0.570146 Iterations 0 $ mpirun -np 1 ./ex3 -ksp_type cg -pc_type none -m 100 Norm of error 0.570146 Iterations 0 $ mpirun -np 1 ./ex3 -ksp_type preonly -pc_type lu -m 100 Norm of error < 1.e-12 Iterations 1 I had two questions: 1. I was wondering if someone could explain why the system cannot be solved using for a mesh of 100x100 elements without a using preconditioner? The way the boundary conditions are applied causes the system matrix to lose symmetry but GMRes and BiCGStab should work for non-symmetric systems. 2. I'm also unsure about why the CG method seems to be able to solve this non-symmetric system in some cases, are there some cases where this is possible? Thanks, Matija -------------- next part -------------- An HTML attachment was scrubbed... URL: From jedbrown at mcs.anl.gov Tue Sep 20 05:44:35 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Tue, 20 Sep 2011 12:44:35 +0200 Subject: [petsc-users] Question about ksp ex3.c In-Reply-To: References: Message-ID: On Tue, Sep 20, 2011 at 11:45, Matija Kecman wrote: > $ mpirun -np 1 ./ex3 -ksp_type gmres -pc_type none -m 100 > Norm of error 0.570146 Iterations 0 > This uses a nonzero initial guess so the initial residual norm is compared to the right hand side. $ ./ex3 -ksp_type gmres -ksp_monitor -m 100 -pc_type none -ksp_converged_reason -info |grep Converged [0] KSPDefaultConverged(): user has provided nonzero initial guess, computing 2-norm of preconditioned RHS [0] KSPDefaultConverged(): Linear solver has converged. Residual norm 1.113646413065e-04 is less than relative tolerance 1.000000000000e-05 times initial right hand side norm 1.291007358616e+01 at iteration 0 You can use the true residual, it just costs something so it's not enabled by default: $ ./ex3 -ksp_type gmres -ksp_monitor -m 100 -pc_type none -ksp_converged_reason -ksp_converged_use_initial_residual_norm [many iterations] Linear solve converged due to CONVERGED_RTOL iterations 1393 Norm of error 0.000664957 Iterations 1393 -------------- next part -------------- An HTML attachment was scrubbed... URL: From gdiso at ustc.edu Tue Sep 20 08:26:12 2011 From: gdiso at ustc.edu (Gong Ding) Date: Tue, 20 Sep 2011 21:26:12 +0800 (CST) Subject: [petsc-users] floating region help In-Reply-To: References: <588771.234921316506085787.JavaMail.coremail@mail.ustc.edu> <1525625.235221316510237990.JavaMail.coremail@mail.ustc.edu> Message-ID: <6672045.235591316525172753.JavaMail.coremail@mail.ustc.edu> On Tue, Sep 20, 2011 at 11:17, Gong Ding wrote: Thanks. Any recommends for solving this type of problems? It's our bottleneck now. Direct solver works, but not efficient for large 3D problem. Krylov space based solver seems not rubost enough. A more exact ILU preconditioner may help? You can use domain decomposition with the jump aligned to subdomain boundaries. There are a variety of schemes with convergence rate independent of the jump, from both the overlapping Schwarz and Neumann-Neumann families. Can you describe the 3D problem in more detail? The problem arising from a CMOS inverter, or any CMOS gate. For the inverter, there are PMOS and NMOS connected together. When the PMOS and NMOS are both closed -- means nearly no current pass though the inverter. the connection metal will see both huge resistance at VCC and GND end. VCC connection metal GND ____ PGate _____________________ NGate _____ | ____________ | | ____________ | __|________________|__ __|________________|__ P | | PSource NDrain | | N ----- ----- ----- ----- NSub PSub In fact, I had already add some leakage current to the inverter -- can not be more due to accurate limit. The conductance of metal still 1e6~1e10 times larger than "effective" conductance in the NMOS and PMOS. Can anyone give some detailed suggestion? Thanks. -------------- next part -------------- An HTML attachment was scrubbed... URL: From balay at mcs.anl.gov Tue Sep 20 08:27:36 2011 From: balay at mcs.anl.gov (Satish Balay) Date: Tue, 20 Sep 2011 08:27:36 -0500 (CDT) Subject: [petsc-users] [petsc-3.2] MatView on dense matrix In-Reply-To: <4E78444B.4080202@gfz-potsdam.de> References: <4E73273D.4060601@gfz-potsdam.de> <4E76E6F9.30606@gfz-potsdam.de> <4E78444B.4080202@gfz-potsdam.de> Message-ID: Great! Will have this fix in the next patch update to petsc-3.2 Satish On Tue, 20 Sep 2011, Alexander Grayver wrote: > Hi Satish, > > The problem seems to be fixed now. Thanks! > > Regards, > Alexander > > On 20.09.2011 05:16, Satish Balay wrote: > > Can you try the attached patch - and see if it fixes the problem? > > > > cd petsc-3.2-p2 > > patch -Np1< viewer-f.patch > > > > Satish > > > > On Mon, 19 Sep 2011, Alexander Grayver wrote: > > > > > Hello, > > > > > > Is that possible to get fixed version? > > > > > > Regards, > > > Alexander > > > > > > On 16.09.2011 19:59, Barry Smith wrote: > > > > The problem is from the C values of these parameters have gotten > > > > out of > > > > sync with the Fortran values. > > > > > > > > Satish, > > > > > > > > Could you please fix the finclude/petscviewer.h values to match > > > > the C > > > > in petsc-3.2 and send a patch? > > > > > > > > Thanks > > > > > > > > Barry > > > > > > > > > > > > On Sep 16, 2011, at 5:38 AM, Alexander Grayver wrote: > > > > > > > > > Hello, > > > > > > > > > > With petsc-3.1 I used this code to output MPI dense matrix: > > > > > > > > > > PetscViewer :: dviewer > > > > > call > > > > > PetscViewerBinaryOpen(MPI_COMM_WORLD,'A.dat',FILE_MODE_WRITE,dviewer,ierr); > > > > > CHKERRQ(ierr) > > > > > call PetscViewerSetFormat(dviewer,PETSC_VIEWER_NATIVE,ierr); > > > > > CHKERRQ(ierr) > > > > > call MatView(A,dviewer,ierr); CHKERRQ(ierr) > > > > > call PetscViewerDestroy(dviewer,ierr); CHKERRQ(ierr) > > > > > > > > > > With petsc-3.2 this code fails with error: > > > > > > > > > > [7]PETSC ERROR: --------------------- Error Message > > > > > ------------------------------------ > > > > > [7]PETSC ERROR: No support for this operation for this object type! > > > > > [7]PETSC ERROR: To store a parallel dense matrix you must first call > > > > > PetscViewerSetFormat(viewer,PETSC_VIEWER_NATIVE)! > > > > > [7]PETSC ERROR: > > > > > > > > > > ------------------------------------------------------------------------ > > > > > [7]PETSC ERROR: Petsc Release Version 3.2.0, Patch 1, Mon Sep 12 > > > > > 16:01:51 > > > > > CDT 2011 > > > > > [7]PETSC ERROR: See docs/changes/index.html for recent updates. > > > > > [7]PETSC ERROR: See docs/faq.html for hints about trouble shooting. > > > > > [7]PETSC ERROR: See docs/index.html for manual pages. > > > > > [7]PETSC ERROR: > > > > > > > > > > ------------------------------------------------------------------------ > > > > > [7]PETSC ERROR: /home/run/test on a openmpi-i named node205 by user > > > > > Fri > > > > > Sep 16 09:17:13 2011 > > > > > [7]PETSC ERROR: Libraries linked from > > > > > /home/lib/petsc-3.2-p1/openmpi-intel-complex-debug-f/lib > > > > > [7]PETSC ERROR: Configure run at Wed Sep 14 14:32:55 2011 > > > > > [7]PETSC ERROR: Configure options > > > > > --with-petsc-arch=openmpi-intel-complex-debug-f > > > > > --with-fortran-interfaces=1 > > > > > --with-mpi-dir=/opt/mpi/intel/openmpi-1.4.2 > > > > > --with-scalar-type=complex > > > > > --with-blas-lapack-dir=/opt/intel/Compiler/11.1/072/mkl/lib/em64t > > > > > --with-precision=double --with-x=0 > > > > > [7]PETSC ERROR: > > > > > > > > > > ------------------------------------------------------------------------ > > > > > [7]PETSC ERROR: MatView_MPIDense_Binary() line 667 in > > > > > /home/lib/petsc-3.2-p1/src/mat/impls/dense/mpi/mpidense.c > > > > > [7]PETSC ERROR: MatView_MPIDense() line 789 in > > > > > /home/lib/petsc-3.2-p1/src/mat/impls/dense/mpi/mpidense.c > > > > > [7]PETSC ERROR: MatView() line 757 in > > > > > /home/lib/petsc-3.2-p1/src/mat/interface/matrix.c > > > > > MatView_MPIDense() line 789 in > > > > > /home/lib/petsc-3.2-p1/src/mat/impls/dense/mpi/mpidense.c > > > > > > > > > > See full error attached. Is there something new in petsc-3.2 > > > > > concerning > > > > > matrix output? > > > > > > > > > > Regards, > > > > > Alexander > > > > > > > > > > From hzhang at mcs.anl.gov Tue Sep 20 09:35:03 2011 From: hzhang at mcs.anl.gov (Hong Zhang) Date: Tue, 20 Sep 2011 09:35:03 -0500 Subject: [petsc-users] [petsc-3.2] LSQR new convergence test In-Reply-To: <4E785F93.8020106@gfz-potsdam.de> References: <4E785F93.8020106@gfz-potsdam.de> Message-ID: Alexander: I'm not sure these two runs use the same tolerance. The output of petsc-3.2 gives more details about the solver used. Try a smaller -ksp_rtol, e.g., '-ksp_rtol 1.e-12', which likely allows more KSP iterations and produce better final result. You may also use preconditioner to improve convergence. Hong > In petsc31 I had: > ?0 KSP Residual norm 9.386670021557e-17 > ... > ?95 KSP Residual norm 9.341367317075e-18 > Linear solve converged due to CONVERGED_RTOL iterations 95 > ?tolerances: ?relative=0.1, absolute=1e-50, divergence=10000 > ?left preconditioning > ?using PRECONDITIONED norm type for convergence test > PC Object: > ?type: none > ?linear system matrix = precond matrix: > ?Matrix Object: > ? ?type=shell, rows=19584, cols=19584 > > relative residual norm = ? ?0.9951737192872134E-01 > > Now, with petsc32 I have: > > ?0 KSP Residual norm 9.386670021557e-17 > ?1 KSP Residual norm 8.258308650175e-17 > Linear solve converged due to CONVERGED_RTOL_NORMAL iterations 1 > KSP Object: 12 MPI processes > ?type: lsqr > ?maximum iterations=200, initial guess is zero > ?tolerances: ?relative=0.1, absolute=1e-50, divergence=10000 > ?left preconditioning > ?using UNPRECONDITIONED norm type for convergence test > PC Object: 12 MPI processes > ?type: none > ?linear system matrix = precond matrix: > ?Matrix Object: ? 12 MPI processes > ? ?type: shell > ? ?rows=19584, cols=19584 > relative residual norm = ? ?0.8797910900468064E+00 > > So I found new convergence test here which sets CONVERGED_RTOL_NORMAL: > http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/KSP/KSPLSQRDefaultConverged.html > > The question is, how to interpret this new test and make it works properly > for me? > Thanks in advance. > > Regards, > Alexander > From jedbrown at mcs.anl.gov Tue Sep 20 14:36:49 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Tue, 20 Sep 2011 21:36:49 +0200 Subject: [petsc-users] floating region help In-Reply-To: <6672045.235591316525172753.JavaMail.coremail@mail.ustc.edu> References: <588771.234921316506085787.JavaMail.coremail@mail.ustc.edu> <1525625.235221316510237990.JavaMail.coremail@mail.ustc.edu> <6672045.235591316525172753.JavaMail.coremail@mail.ustc.edu> Message-ID: On Tue, Sep 20, 2011 at 15:26, Gong Ding wrote: > The problem arising from a CMOS inverter, or any CMOS gate. > For the inverter, there are PMOS and NMOS connected together. > When the PMOS and NMOS are both closed -- means nearly no current pass > though the inverter. > the connection metal will see both huge resistance at VCC and GND end. > > VCC connection metal GND > ____ PGate _____________________ NGate _____ > | ____________ | | ____________ | > __|________________|__ __|________________|__ > P | | PSource NDrain | | N > ----- ----- ----- ----- > NSub PSub > > In fact, I had already add some leakage current to the inverter -- can not > be more due to accurate limit. > The conductance of metal still 1e6~1e10 times larger than "effective" > conductance in the NMOS and PMOS. > Is this a PDE formulation or a discrete model that happens to have similar characteristics. Semiconductor problems produce notoriously difficult matrices for iterative methods and my understanding is that nearly all practitioners end up using direct methods. Perhaps there is still some exploitable structure buried in the problem, but people don't seem to have been very successful and I don't know enough about it speculate. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Tue Sep 20 18:10:05 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Tue, 20 Sep 2011 18:10:05 -0500 Subject: [petsc-users] [petsc-3.2] LSQR new convergence test In-Reply-To: <4E785F93.8020106@gfz-potsdam.de> References: <4E785F93.8020106@gfz-potsdam.de> Message-ID: <5C5AE85F-F5D5-4BE2-95F6-DE1D09A150DF@mcs.anl.gov> It has its own monitor that provides additional information -ksp_monitor_lsqr You can also remove the new convergence test and get back the old one with code like void *cctx; KSPDefaultConvergedCreate(&cctx); KSPSetConvergenceTest(ksp,KSPDefaultConverged,cctx); after the KSPType is set to LSQR. So if you are happy with the old test. Do both versions give the same first two norms? > 0 KSP Residual norm 9.386670021557e-17 > 1 KSP Residual norm 8.258308650175e-17 Barry On Sep 20, 2011, at 4:40 AM, Alexander Grayver wrote: > Hello, > > In comparison with petsc-3.1-p7 in the latest petsc LSQR solver behaves differently. > > In petsc31 I had: > > 0 KSP Residual norm 9.386670021557e-17 > ... > 95 KSP Residual norm 9.341367317075e-18 > Linear solve converged due to CONVERGED_RTOL iterations 95 > KSP Object: > type: lsqr > maximum iterations=200, initial guess is zero > tolerances: relative=0.1, absolute=1e-50, divergence=10000 > left preconditioning > using PRECONDITIONED norm type for convergence test > PC Object: > type: none > linear system matrix = precond matrix: > Matrix Object: > type=shell, rows=19584, cols=19584 > > relative residual norm = 0.9951737192872134E-01 > > Now, with petsc32 I have: > > 0 KSP Residual norm 9.386670021557e-17 > 1 KSP Residual norm 8.258308650175e-17 > Linear solve converged due to CONVERGED_RTOL_NORMAL iterations 1 > KSP Object: 12 MPI processes > type: lsqr > maximum iterations=200, initial guess is zero > tolerances: relative=0.1, absolute=1e-50, divergence=10000 > left preconditioning > using UNPRECONDITIONED norm type for convergence test > PC Object: 12 MPI processes > type: none > linear system matrix = precond matrix: > Matrix Object: 12 MPI processes > type: shell > rows=19584, cols=19584 > relative residual norm = 0.8797910900468064E+00 > > So I found new convergence test here which sets CONVERGED_RTOL_NORMAL: > http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/KSP/KSPLSQRDefaultConverged.html > > The question is, how to interpret this new test and make it works properly for me? > Thanks in advance. > > Regards, > Alexander From bsmith at mcs.anl.gov Tue Sep 20 18:13:52 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Tue, 20 Sep 2011 18:13:52 -0500 Subject: [petsc-users] floating region help In-Reply-To: <1525625.235221316510237990.JavaMail.coremail@mail.ustc.edu> References: <588771.234921316506085787.JavaMail.coremail@mail.ustc.edu> <1525625.235221316510237990.JavaMail.coremail@mail.ustc.edu> Message-ID: <12951132-DE22-4F20-A7F1-6305B1A4EAB3@mcs.anl.gov> On Sep 20, 2011, at 4:17 AM, Gong Ding wrote: > > 2011/9/20 Gong Ding > Since the conductance of left part and right part is low, the middle region is floating. > > Not really floating, just sensitive to parameters. It has a unique value, of physical interest, but if you change the conductivity or thickness of the thin region on either side, then it's value changes significantly. This is an ill-conditioned problem, but it's ill-conditioned for physically meaningful reasons. It's not singular and the smallest singular value has great significance, so a null space method is not appropriate. You could reformulate for a slightly different problem which would have a constant null space, but not for this problem. > > Thanks. Any recommends for solving this type of problems? > It's our bottleneck now. > Direct solver works, but not efficient for large 3D problem. Are you running SuperLU_Dist or MUMPS in parallel? How large a problem do you want to do? I think those direct solvers should be fine in the few million unknown range (slow yes, but likely the best you will find). Barry > Krylov space based solver seems not rubost enough. > A more exact ILU preconditioner may help? > > From rongliang.chan at gmail.com Tue Sep 20 19:02:29 2011 From: rongliang.chan at gmail.com (Rongliang Chen) Date: Tue, 20 Sep 2011 18:02:29 -0600 Subject: [petsc-users] Matrix Factorization fill Message-ID: Hi, I have a question about the Sparse Matrix Factorization fill. Following the manual, I use the option "-info" and get the following result: [0] MatLUFactorSymbolic_SeqAIJ(): Reallocs 0 Fill ratio:given 5 needed 5.0746e-05 [0] MatLUFactorSymbolic_SeqAIJ(): Run with -pc_factor_fill 5.0746e-05 or use [0] MatLUFactorSymbolic_SeqAIJ(): PCFactorSetFill(pc,5.0746e-05); What's the numbers mean in "Reallocs 0 Fill ratio:given 5 needed 5.0746e-05"? Does the " Run with -pc_factor_fill 5.0746e-05 or use" mean that I should run my code with option " -pc_factor_fill 5.0746e-05"? But when I run my code with option "-snes_view", I get "factor fill ratio given 5, needed 3.28701". So what should I give to the option "-pc_factor_fill", 3.28701 or 5.0746e-05? Thanks. Best, Rongliang -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Tue Sep 20 19:06:41 2011 From: knepley at gmail.com (Matthew Knepley) Date: Tue, 20 Sep 2011 19:06:41 -0500 Subject: [petsc-users] Matrix Factorization fill In-Reply-To: References: Message-ID: On Tue, Sep 20, 2011 at 7:02 PM, Rongliang Chen wrote: > Hi, > > I have a question about the Sparse Matrix Factorization fill. > Following the manual, I use the option "-info" and get the following > result: > > [0] MatLUFactorSymbolic_SeqAIJ(): Reallocs 0 Fill ratio:given 5 needed > 5.0746e-05 > [0] MatLUFactorSymbolic_SeqAIJ(): Run with -pc_factor_fill 5.0746e-05 or > use > [0] MatLUFactorSymbolic_SeqAIJ(): PCFactorSetFill(pc,5.0746e-05); > > What's the numbers mean in "Reallocs 0 Fill ratio:given 5 needed > 5.0746e-05"? > Does the " Run with -pc_factor_fill 5.0746e-05 or use" mean that I should > run my code with option " -pc_factor_fill 5.0746e-05"? > "Reallocs 0": no additional malloc()s were needed "Fill rato: give 5 needed 5.0746e-05" We guessed that the factor would need 5 times the matrix memory. It actually needed a lot less. If you run -pc_factor_fill 5.0746e-05, you will get the same performance and use a lot less memory. Matt > But when I run my code with option "-snes_view", I get "factor fill ratio > given 5, needed 3.28701". > So what should I give to the option "-pc_factor_fill", 3.28701 or > 5.0746e-05? Thanks. > > Best, > Rongliang > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Tue Sep 20 20:07:44 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Tue, 20 Sep 2011 20:07:44 -0500 Subject: [petsc-users] Matrix Factorization fill In-Reply-To: References: Message-ID: <48785310-F532-48E8-8220-88392A5047FA@mcs.anl.gov> On Sep 20, 2011, at 7:02 PM, Rongliang Chen wrote: > Hi, > > I have a question about the Sparse Matrix Factorization fill. > Following the manual, I use the option "-info" and get the following result: > > [0] MatLUFactorSymbolic_SeqAIJ(): Reallocs 0 Fill ratio:given 5 needed 5.0746e-05 > [0] MatLUFactorSymbolic_SeqAIJ(): Run with -pc_factor_fill 5.0746e-05 or use > [0] MatLUFactorSymbolic_SeqAIJ(): PCFactorSetFill(pc,5.0746e-05); > > What's the numbers mean in "Reallocs 0 Fill ratio:given 5 needed 5.0746e-05"? > Does the " Run with -pc_factor_fill 5.0746e-05 or use" mean that I should run my code with option " -pc_factor_fill 5.0746e-05"? This is printed incorrectly due to a change in our data structures, we will fix it. > > But when I run my code with option "-snes_view", I get "factor fill ratio given 5, needed 3.28701". > So what should I give to the option "-pc_factor_fill", 3.28701 or 5.0746e-05? Thanks. The 3.28701 Barrty > > Best, > Rongliang From abhyshr at mcs.anl.gov Tue Sep 20 22:21:53 2011 From: abhyshr at mcs.anl.gov (Shri) Date: Tue, 20 Sep 2011 22:21:53 -0500 (CDT) Subject: [petsc-users] Matrix Factorization fill In-Reply-To: <48785310-F532-48E8-8220-88392A5047FA@mcs.anl.gov> Message-ID: <1333380760.24263.1316575313230.JavaMail.root@zimbra.anl.gov> I can't reproduce this with snes/examples/tutorials/ex3.c,ex5.c, or ex19.c. Are you running a petsc example or your own application code? Shri ----- Original Message ----- > On Sep 20, 2011, at 7:02 PM, Rongliang Chen wrote: > > > Hi, > > > > I have a question about the Sparse Matrix Factorization fill. > > Following the manual, I use the option "-info" and get the following > > result: > > > > [0] MatLUFactorSymbolic_SeqAIJ(): Reallocs 0 Fill ratio:given 5 > > needed 5.0746e-05 > > [0] MatLUFactorSymbolic_SeqAIJ(): Run with -pc_factor_fill > > 5.0746e-05 or use > > [0] MatLUFactorSymbolic_SeqAIJ(): PCFactorSetFill(pc,5.0746e-05); > > > > What's the numbers mean in "Reallocs 0 Fill ratio:given 5 needed > > 5.0746e-05"? > > Does the " Run with -pc_factor_fill 5.0746e-05 or use" mean that I > > should run my code with option " -pc_factor_fill 5.0746e-05"? > > This is printed incorrectly due to a change in our data structures, we > will fix it. > > > > > But when I run my code with option "-snes_view", I get "factor fill > > ratio given 5, needed 3.28701". > > So what should I give to the option "-pc_factor_fill", 3.28701 or > > 5.0746e-05? Thanks. > > The 3.28701 > > Barrty > > > > > Best, > > Rongliang From zhenglun.wei at gmail.com Tue Sep 20 22:47:33 2011 From: zhenglun.wei at gmail.com (Alan Wei) Date: Tue, 20 Sep 2011 22:47:33 -0500 Subject: [petsc-users] Convergence Criteria? In-Reply-To: <1C05DB25-FA8B-49A9-8B33-D606C3F7126C@mcs.anl.gov> References: <1C05DB25-FA8B-49A9-8B33-D606C3F7126C@mcs.anl.gov> Message-ID: Dear Dr. Smith, I figure out this problem. Actually, I made my own RHS, but I did not multiply them by the volume (dx*dy). However, I met another problem. All values calculated from this example are exactly opposite to values from my own code. I wonder if the RHS I input by ComputeRHS are the really RHS or -1.*RHS? thanks in advance, Alan On Mon, Sep 19, 2011 at 8:43 PM, Barry Smith wrote: > > On Sep 19, 2011, at 6:25 PM, Alan Wei wrote: > > > Dear folks, > > I hope you guys are having a nice day. > > I'm reading the /src/ksp/ksp/examples/tutorials/ex29.c.html and > wonder how to set up the convergence criteria? > > -ksp_rtol 1.e-10 for example > > Run with -ksp_monitor_true_residual -ksp_converged_reason > > > > Currently I use it as a poisson solver to solve a Poisson Equation with > three direction Neumann BC's and one direction Diriechlet BC's. It seems > very bad on it. > > Hmm, multigrid should likely converge well. Are you sure you've set the > BC's correctly? > > Barry > > > > > thanks in advance, > > Alan > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gdiso at ustc.edu Tue Sep 20 22:51:56 2011 From: gdiso at ustc.edu (Gong Ding) Date: Wed, 21 Sep 2011 11:51:56 +0800 (CST) Subject: [petsc-users] floating region help In-Reply-To: References: <588771.234921316506085787.JavaMail.coremail@mail.ustc.edu> <1525625.235221316510237990.JavaMail.coremail@mail.ustc.edu> <6672045.235591316525172753.JavaMail.coremail@mail.ustc.edu> Message-ID: <4817529.236491316577116602.JavaMail.coremail@mail.ustc.edu> On Tue, Sep 20, 2011 at 15:26, Gong Ding wrote: The problem arising from a CMOS inverter, or any CMOS gate. For the inverter, there are PMOS and NMOS connected together. When the PMOS and NMOS are both closed -- means nearly no current pass though the inverter. the connection metal will see both huge resistance at VCC and GND end. VCC connection metal GND ____ PGate _____________________ NGate _____ | ____________ | | ____________ | __|________________|__ __|________________|__ P | | PSource NDrain | | N ----- ----- ----- ----- NSub PSub In fact, I had already add some leakage current to the inverter -- can not be more due to accurate limit. The conductance of metal still 1e6~1e10 times larger than "effective" conductance in the NMOS and PMOS. Is this a PDE formulation or a discrete model that happens to have similar characteristics. Semiconductor problems produce notoriously difficult matrices for iterative methods and my understanding is that nearly all practitioners end up using direct methods. Perhaps there is still some exploitable structure buried in the problem, but people don't seem to have been very successful and I don't know enough about it speculate. The background of this problem is the single event upset simulation of CMOS unit circuit, i.e. SRAM, latch, flip-flop. They all very big, ~ 1M mesh nodes. Direct method is too slow and memory consuming. Some stupid commercial code needs more than one week to simulate one particle event. People usually need 100-1000 events to determine the circuit behavior! Fortunately?the problem is in time domain. the transport equation of carrier has a d/dt term which helps stable. The most difficult part comes from metal connection. Just use above inverter, when both NMOS and PMOS are closed, the metal connection region has only displacement current from semiconductor region as eps*dE/dt, while metal region the current can be written as sigma*E. The current conservation here is the boundary condition at the metal-semiconductor interface. (Since E is \frac{\partical phi}{\partical n}, here phi is the independent variable) Obviously, sigma >> eps/T, the governing equation of phi has a large jump parameter at the metal/semiconductorinterface. The eigen value analysis confirmed that the number of samllest eigen value equals to the number of floating metal region. And theeigen vector is exactly the floating region. I hope some deflation preconditioner or Neumann-Neumannpreconditioner helps. Any suggestion? -------------- next part -------------- An HTML attachment was scrubbed... URL: From agrayver at gfz-potsdam.de Wed Sep 21 02:39:26 2011 From: agrayver at gfz-potsdam.de (Alexander Grayver) Date: Wed, 21 Sep 2011 09:39:26 +0200 Subject: [petsc-users] [petsc-3.2] LSQR new convergence test In-Reply-To: <5C5AE85F-F5D5-4BE2-95F6-DE1D09A150DF@mcs.anl.gov> References: <4E785F93.8020106@gfz-potsdam.de> <5C5AE85F-F5D5-4BE2-95F6-DE1D09A150DF@mcs.anl.gov> Message-ID: <4E7994AE.1060200@gfz-potsdam.de> Hong, Barry, Thanks! Sorry for not to be clear, but Barry right, my question was how to get back to DefaultConvergenceTest in case of LSQR solver. Barry, yes, both versions give the same two norms. Can you clarify please how to implement that in Fortran: void *cctx; KSPDefaultConvergedCreate(&cctx); KSPSetConvergenceTest(ksp,KSPDefaultConverged,cctx); I'm a little bit confused about cctx parameter. Regards, Alexander On 21.09.2011 01:10, Barry Smith wrote: > It has its own monitor that provides additional information -ksp_monitor_lsqr > > You can also remove the new convergence test and get back the old one with code like > > void *cctx; > KSPDefaultConvergedCreate(&cctx); > KSPSetConvergenceTest(ksp,KSPDefaultConverged,cctx); > > after the KSPType is set to LSQR. So if you are happy with the old test. > > > Do both versions give the same first two norms? > >> 0 KSP Residual norm 9.386670021557e-17 >> 1 KSP Residual norm 8.258308650175e-17 > Barry > > > On Sep 20, 2011, at 4:40 AM, Alexander Grayver wrote: > >> Hello, >> >> In comparison with petsc-3.1-p7 in the latest petsc LSQR solver behaves differently. >> >> In petsc31 I had: >> >> 0 KSP Residual norm 9.386670021557e-17 >> ... >> 95 KSP Residual norm 9.341367317075e-18 >> Linear solve converged due to CONVERGED_RTOL iterations 95 >> KSP Object: >> type: lsqr >> maximum iterations=200, initial guess is zero >> tolerances: relative=0.1, absolute=1e-50, divergence=10000 >> left preconditioning >> using PRECONDITIONED norm type for convergence test >> PC Object: >> type: none >> linear system matrix = precond matrix: >> Matrix Object: >> type=shell, rows=19584, cols=19584 >> >> relative residual norm = 0.9951737192872134E-01 >> >> Now, with petsc32 I have: >> >> 0 KSP Residual norm 9.386670021557e-17 >> 1 KSP Residual norm 8.258308650175e-17 >> Linear solve converged due to CONVERGED_RTOL_NORMAL iterations 1 >> KSP Object: 12 MPI processes >> type: lsqr >> maximum iterations=200, initial guess is zero >> tolerances: relative=0.1, absolute=1e-50, divergence=10000 >> left preconditioning >> using UNPRECONDITIONED norm type for convergence test >> PC Object: 12 MPI processes >> type: none >> linear system matrix = precond matrix: >> Matrix Object: 12 MPI processes >> type: shell >> rows=19584, cols=19584 >> relative residual norm = 0.8797910900468064E+00 >> >> So I found new convergence test here which sets CONVERGED_RTOL_NORMAL: >> http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/KSP/KSPLSQRDefaultConverged.html >> >> The question is, how to interpret this new test and make it works properly for me? >> Thanks in advance. >> >> Regards, >> Alexander From bsmith at mcs.anl.gov Wed Sep 21 08:22:54 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Wed, 21 Sep 2011 08:22:54 -0500 Subject: [petsc-users] Convergence Criteria? In-Reply-To: References: <1C05DB25-FA8B-49A9-8B33-D606C3F7126C@mcs.anl.gov> Message-ID: On Sep 20, 2011, at 10:47 PM, Alan Wei wrote: > Dear Dr. Smith, > I figure out this problem. Actually, I made my own RHS, but I did not multiply them by the volume (dx*dy). > However, I met another problem. All values calculated from this example are exactly opposite to values from my own code. I wonder if the RHS I input by ComputeRHS are the really RHS or -1.*RHS? We do not change the sign of the right hand side. Barry > > thanks in advance, > Alan > > On Mon, Sep 19, 2011 at 8:43 PM, Barry Smith wrote: > > On Sep 19, 2011, at 6:25 PM, Alan Wei wrote: > > > Dear folks, > > I hope you guys are having a nice day. > > I'm reading the /src/ksp/ksp/examples/tutorials/ex29.c.html and wonder how to set up the convergence criteria? > > -ksp_rtol 1.e-10 for example > > Run with -ksp_monitor_true_residual -ksp_converged_reason > > > > Currently I use it as a poisson solver to solve a Poisson Equation with three direction Neumann BC's and one direction Diriechlet BC's. It seems very bad on it. > > Hmm, multigrid should likely converge well. Are you sure you've set the BC's correctly? > > Barry > > > > > thanks in advance, > > Alan > > From bsmith at mcs.anl.gov Wed Sep 21 08:36:56 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Wed, 21 Sep 2011 08:36:56 -0500 Subject: [petsc-users] [petsc-3.2] LSQR new convergence test In-Reply-To: <4E7994AE.1060200@gfz-potsdam.de> References: <4E785F93.8020106@gfz-potsdam.de> <5C5AE85F-F5D5-4BE2-95F6-DE1D09A150DF@mcs.anl.gov> <4E7994AE.1060200@gfz-potsdam.de> Message-ID: On Sep 21, 2011, at 2:39 AM, Alexander Grayver wrote: > Hong, Barry, > > Thanks! Sorry for not to be clear, but Barry right, my question was how to get back to DefaultConvergenceTest in case of LSQR solver. > > Barry, yes, both versions give the same two norms. > > Can you clarify please how to implement that in Fortran: > void *cctx; > KSPDefaultConvergedCreate(&cctx); > KSPSetConvergenceTest(ksp,KSPDefaultConverged,cctx); It should be almost the same PetscFortranAddr cctx call KSPDefaultConvergedCreate(cctx,ierr) call KSPSetConvergenceTest(ksp,KSPDefaultConverged,cctx,PETSC_NULL_FUNCTION,ierr) Barry > > I'm a little bit confused about cctx parameter. > > Regards, > Alexander > > > On 21.09.2011 01:10, Barry Smith wrote: >> It has its own monitor that provides additional information -ksp_monitor_lsqr >> >> You can also remove the new convergence test and get back the old one with code like >> >> void *cctx; >> KSPDefaultConvergedCreate(&cctx); >> KSPSetConvergenceTest(ksp,KSPDefaultConverged,cctx); >> >> after the KSPType is set to LSQR. So if you are happy with the old test. >> >> >> Do both versions give the same first two norms? >> >>> 0 KSP Residual norm 9.386670021557e-17 >>> 1 KSP Residual norm 8.258308650175e-17 >> Barry >> >> >> On Sep 20, 2011, at 4:40 AM, Alexander Grayver wrote: >> >>> Hello, >>> >>> In comparison with petsc-3.1-p7 in the latest petsc LSQR solver behaves differently. >>> >>> In petsc31 I had: >>> >>> 0 KSP Residual norm 9.386670021557e-17 >>> ... >>> 95 KSP Residual norm 9.341367317075e-18 >>> Linear solve converged due to CONVERGED_RTOL iterations 95 >>> KSP Object: >>> type: lsqr >>> maximum iterations=200, initial guess is zero >>> tolerances: relative=0.1, absolute=1e-50, divergence=10000 >>> left preconditioning >>> using PRECONDITIONED norm type for convergence test >>> PC Object: >>> type: none >>> linear system matrix = precond matrix: >>> Matrix Object: >>> type=shell, rows=19584, cols=19584 >>> >>> relative residual norm = 0.9951737192872134E-01 >>> >>> Now, with petsc32 I have: >>> >>> 0 KSP Residual norm 9.386670021557e-17 >>> 1 KSP Residual norm 8.258308650175e-17 >>> Linear solve converged due to CONVERGED_RTOL_NORMAL iterations 1 >>> KSP Object: 12 MPI processes >>> type: lsqr >>> maximum iterations=200, initial guess is zero >>> tolerances: relative=0.1, absolute=1e-50, divergence=10000 >>> left preconditioning >>> using UNPRECONDITIONED norm type for convergence test >>> PC Object: 12 MPI processes >>> type: none >>> linear system matrix = precond matrix: >>> Matrix Object: 12 MPI processes >>> type: shell >>> rows=19584, cols=19584 >>> relative residual norm = 0.8797910900468064E+00 >>> >>> So I found new convergence test here which sets CONVERGED_RTOL_NORMAL: >>> http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/KSP/KSPLSQRDefaultConverged.html >>> >>> The question is, how to interpret this new test and make it works properly for me? >>> Thanks in advance. >>> >>> Regards, >>> Alexander > From rongliang.chan at gmail.com Wed Sep 21 10:08:21 2011 From: rongliang.chan at gmail.com (Rongliang Chen) Date: Wed, 21 Sep 2011 09:08:21 -0600 Subject: [petsc-users] Matrix Factorization fill Message-ID: Hello All, Thank you for your reply. This result is obtained by running my own application code. Best, Rongliang ------------------------------ > > Message: 5 > Date: Tue, 20 Sep 2011 19:06:41 -0500 > From: Matthew Knepley > Subject: Re: [petsc-users] Matrix Factorization fill > To: PETSc users list > Message-ID: > > > Content-Type: text/plain; charset="iso-8859-1" > > On Tue, Sep 20, 2011 at 7:02 PM, Rongliang Chen >wrote: > > > Hi, > > > > I have a question about the Sparse Matrix Factorization fill. > > Following the manual, I use the option "-info" and get the following > > result: > > > > [0] MatLUFactorSymbolic_SeqAIJ(): Reallocs 0 Fill ratio:given 5 needed > > 5.0746e-05 > > [0] MatLUFactorSymbolic_SeqAIJ(): Run with -pc_factor_fill 5.0746e-05 or > > use > > [0] MatLUFactorSymbolic_SeqAIJ(): PCFactorSetFill(pc,5.0746e-05); > > > > What's the numbers mean in "Reallocs 0 Fill ratio:given 5 needed > > 5.0746e-05"? > > Does the " Run with -pc_factor_fill 5.0746e-05 or use" mean that I should > > run my code with option " -pc_factor_fill 5.0746e-05"? > > > > "Reallocs 0": no additional malloc()s were needed > > "Fill rato: give 5 needed 5.0746e-05" We guessed that the factor would need > 5 times the matrix memory. It actually needed a lot less. > > If you run -pc_factor_fill 5.0746e-05, you will get the same performance > and > use a lot less memory. > > Matt > > > > But when I run my code with option "-snes_view", I get "factor fill ratio > > given 5, needed 3.28701". > > So what should I give to the option "-pc_factor_fill", 3.28701 or > > 5.0746e-05? Thanks. > > > > Best, > > Rongliang > > > > > > -- > What most experimenters take for granted before they begin their > experiments > is infinitely more interesting than any results to which their experiments > lead. > -- Norbert Wiener > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: < > http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20110920/1f878139/attachment-0001.htm > > > > ------------------------------ > > Message: 6 > Date: Tue, 20 Sep 2011 20:07:44 -0500 > From: Barry Smith > Subject: Re: [petsc-users] Matrix Factorization fill > To: PETSc users list > Message-ID: <48785310-F532-48E8-8220-88392A5047FA at mcs.anl.gov> > Content-Type: text/plain; charset=us-ascii > > > On Sep 20, 2011, at 7:02 PM, Rongliang Chen wrote: > > > Hi, > > > > I have a question about the Sparse Matrix Factorization fill. > > Following the manual, I use the option "-info" and get the following > result: > > > > [0] MatLUFactorSymbolic_SeqAIJ(): Reallocs 0 Fill ratio:given 5 needed > 5.0746e-05 > > [0] MatLUFactorSymbolic_SeqAIJ(): Run with -pc_factor_fill 5.0746e-05 or > use > > [0] MatLUFactorSymbolic_SeqAIJ(): PCFactorSetFill(pc,5.0746e-05); > > > > What's the numbers mean in "Reallocs 0 Fill ratio:given 5 needed > 5.0746e-05"? > > Does the " Run with -pc_factor_fill 5.0746e-05 or use" mean that I should > run my code with option " -pc_factor_fill 5.0746e-05"? > > This is printed incorrectly due to a change in our data structures, we > will fix it. > > > > > But when I run my code with option "-snes_view", I get "factor fill ratio > given 5, needed 3.28701". > > So what should I give to the option "-pc_factor_fill", 3.28701 or > 5.0746e-05? Thanks. > > The 3.28701 > > Barrty > > > > > Best, > > Rongliang > > > > ------------------------------ > > Message: 7 > Date: Tue, 20 Sep 2011 22:21:53 -0500 (CDT) > From: Shri > Subject: Re: [petsc-users] Matrix Factorization fill > To: PETSc users list > Message-ID: > <1333380760.24263.1316575313230.JavaMail.root at zimbra.anl.gov> > Content-Type: text/plain; charset=utf-8 > > I can't reproduce this with snes/examples/tutorials/ex3.c,ex5.c, or ex19.c. > Are you running a petsc example or your own application code? > > Shri > ----- Original Message ----- > > On Sep 20, 2011, at 7:02 PM, Rongliang Chen wrote: > > > > > Hi, > > > > > > I have a question about the Sparse Matrix Factorization fill. > > > Following the manual, I use the option "-info" and get the following > > > result: > > > > > > [0] MatLUFactorSymbolic_SeqAIJ(): Reallocs 0 Fill ratio:given 5 > > > needed 5.0746e-05 > > > [0] MatLUFactorSymbolic_SeqAIJ(): Run with -pc_factor_fill > > > 5.0746e-05 or use > > > [0] MatLUFactorSymbolic_SeqAIJ(): PCFactorSetFill(pc,5.0746e-05); > > > > > > What's the numbers mean in "Reallocs 0 Fill ratio:given 5 needed > > > 5.0746e-05"? > > > Does the " Run with -pc_factor_fill 5.0746e-05 or use" mean that I > > > should run my code with option " -pc_factor_fill 5.0746e-05"? > > > > This is printed incorrectly due to a change in our data structures, we > > will fix it. > > > > > > > > But when I run my code with option "-snes_view", I get "factor fill > > > ratio given 5, needed 3.28701". > > > So what should I give to the option "-pc_factor_fill", 3.28701 or > > > 5.0746e-05? Thanks. > > > > The 3.28701 > > > > Barrty > > > > > > > > Best, > > > Rongliang > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From agrayver at gfz-potsdam.de Wed Sep 21 10:17:35 2011 From: agrayver at gfz-potsdam.de (Alexander Grayver) Date: Wed, 21 Sep 2011 17:17:35 +0200 Subject: [petsc-users] [petsc-3.2] LSQR new convergence test In-Reply-To: References: <4E785F93.8020106@gfz-potsdam.de> <5C5AE85F-F5D5-4BE2-95F6-DE1D09A150DF@mcs.anl.gov> <4E7994AE.1060200@gfz-potsdam.de> Message-ID: <4E7A000F.4030009@gfz-potsdam.de> Thanks Barry, I added your code and solver works well now, but when I call KSPDestroy afterward it crashes: [8]PETSC ERROR: --------------------- Error Message ------------------------------------ [8]PETSC ERROR: Invalid argument! [8]PETSC ERROR: Wrong type of object: Parameter # 1! [8]PETSC ERROR: ------------------------------------------------------------------------ [8]PETSC ERROR: Petsc Release Version 3.2.0, Patch 2, Fri Sep 16 10:10:45 CDT 2011 [8]PETSC ERROR: See docs/changes/index.html for recent updates. [8]PETSC ERROR: See docs/faq.html for hints about trouble shooting. [8]PETSC ERROR: See docs/index.html for manual pages. [8]PETSC ERROR: ------------------------------------------------------------------------ [8]PETSC ERROR: /home/test on a openmpi-i named node226 by user Wed Sep 21 17:01:23 2011 [8]PETSC ERROR: Libraries linked from /home/lib/petsc-3.2-p2/openmpi-intel-complex-debug-f/lib [8]PETSC ERROR: Configure run at Tue Sep 20 09:28:29 2011 [8]PETSC ERROR: Configure options --with-petsc-arch=openmpi-intel-complex-debug-f --with-fortran-interfaces=1 --with-mpi-dir=/opt/mpi/intel/openmpi-1.4.2 --with-scalar-type=complex --with-blas-lapack-dir=/opt/intel/Compiler/11.1/072/mkl/lib/em64t --with-precision=double --with-x=0 [8]PETSC ERROR: ------------------------------------------------------------------------ [8]PETSC ERROR: VecDestroy() line 566 in /home/lib/petsc-3.2-p2/src/vec/vec/interface/vector.c [8]PETSC ERROR: KSPDefaultConvergedDestroy() line 669 in /home/lib/petsc-3.2-p2/src/ksp/ksp/interface/iterativ.c [8]PETSC ERROR: KSPDestroy() line 758 in /home/lib/petsc-3.2-p2/src/ksp/ksp/interface/itfunc.c Is it my fault? Regards, Alexander On 21.09.2011 15:36, Barry Smith wrote: > On Sep 21, 2011, at 2:39 AM, Alexander Grayver wrote: > >> Hong, Barry, >> >> Thanks! Sorry for not to be clear, but Barry right, my question was how to get back to DefaultConvergenceTest in case of LSQR solver. >> >> Barry, yes, both versions give the same two norms. >> >> Can you clarify please how to implement that in Fortran: >> void *cctx; >> KSPDefaultConvergedCreate(&cctx); >> KSPSetConvergenceTest(ksp,KSPDefaultConverged,cctx); > It should be almost the same > > PetscFortranAddr cctx > > call KSPDefaultConvergedCreate(cctx,ierr) > call KSPSetConvergenceTest(ksp,KSPDefaultConverged,cctx,PETSC_NULL_FUNCTION,ierr) > > Barry > > >> I'm a little bit confused about cctx parameter. >> >> Regards, >> Alexander >> >> >> On 21.09.2011 01:10, Barry Smith wrote: >>> It has its own monitor that provides additional information -ksp_monitor_lsqr >>> >>> You can also remove the new convergence test and get back the old one with code like >>> >>> void *cctx; >>> KSPDefaultConvergedCreate(&cctx); >>> KSPSetConvergenceTest(ksp,KSPDefaultConverged,cctx); >>> >>> after the KSPType is set to LSQR. So if you are happy with the old test. >>> >>> >>> Do both versions give the same first two norms? >>> >>>> 0 KSP Residual norm 9.386670021557e-17 >>>> 1 KSP Residual norm 8.258308650175e-17 >>> Barry >>> >>> >>> On Sep 20, 2011, at 4:40 AM, Alexander Grayver wrote: >>> >>>> Hello, >>>> >>>> In comparison with petsc-3.1-p7 in the latest petsc LSQR solver behaves differently. >>>> >>>> In petsc31 I had: >>>> >>>> 0 KSP Residual norm 9.386670021557e-17 >>>> ... >>>> 95 KSP Residual norm 9.341367317075e-18 >>>> Linear solve converged due to CONVERGED_RTOL iterations 95 >>>> KSP Object: >>>> type: lsqr >>>> maximum iterations=200, initial guess is zero >>>> tolerances: relative=0.1, absolute=1e-50, divergence=10000 >>>> left preconditioning >>>> using PRECONDITIONED norm type for convergence test >>>> PC Object: >>>> type: none >>>> linear system matrix = precond matrix: >>>> Matrix Object: >>>> type=shell, rows=19584, cols=19584 >>>> >>>> relative residual norm = 0.9951737192872134E-01 >>>> >>>> Now, with petsc32 I have: >>>> >>>> 0 KSP Residual norm 9.386670021557e-17 >>>> 1 KSP Residual norm 8.258308650175e-17 >>>> Linear solve converged due to CONVERGED_RTOL_NORMAL iterations 1 >>>> KSP Object: 12 MPI processes >>>> type: lsqr >>>> maximum iterations=200, initial guess is zero >>>> tolerances: relative=0.1, absolute=1e-50, divergence=10000 >>>> left preconditioning >>>> using UNPRECONDITIONED norm type for convergence test >>>> PC Object: 12 MPI processes >>>> type: none >>>> linear system matrix = precond matrix: >>>> Matrix Object: 12 MPI processes >>>> type: shell >>>> rows=19584, cols=19584 >>>> relative residual norm = 0.8797910900468064E+00 >>>> >>>> So I found new convergence test here which sets CONVERGED_RTOL_NORMAL: >>>> http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/KSP/KSPLSQRDefaultConverged.html >>>> >>>> The question is, how to interpret this new test and make it works properly for me? >>>> Thanks in advance. >>>> >>>> Regards, >>>> Alexander From hzhang at mcs.anl.gov Wed Sep 21 10:32:40 2011 From: hzhang at mcs.anl.gov (Hong Zhang) Date: Wed, 21 Sep 2011 10:32:40 -0500 Subject: [petsc-users] Matrix Factorization fill In-Reply-To: References: Message-ID: Rongliang: Which petsc version do you use? Please switch to the latest release petsc-3.2. Hong > Hello All, > > Thank you for your reply. > This result is obtained by running my own application code. > > Best, > Rongliang > >> ------------------------------ >> >> Message: 5 >> Date: Tue, 20 Sep 2011 19:06:41 -0500 >> From: Matthew Knepley >> Subject: Re: [petsc-users] Matrix Factorization fill >> To: PETSc users list >> Message-ID: >> >> ? >> Content-Type: text/plain; charset="iso-8859-1" >> >> On Tue, Sep 20, 2011 at 7:02 PM, Rongliang Chen >> wrote: >> >> > Hi, >> > >> > I have a question about the Sparse Matrix Factorization fill. >> > Following the manual, I use the option "-info" and get the following >> > result: >> > >> > [0] MatLUFactorSymbolic_SeqAIJ(): Reallocs 0 Fill ratio:given 5 needed >> > 5.0746e-05 >> > [0] MatLUFactorSymbolic_SeqAIJ(): Run with -pc_factor_fill 5.0746e-05 or >> > use >> > [0] MatLUFactorSymbolic_SeqAIJ(): PCFactorSetFill(pc,5.0746e-05); >> > >> > What's the numbers mean in "Reallocs 0 Fill ratio:given 5 needed >> > 5.0746e-05"? >> > Does the " Run with -pc_factor_fill 5.0746e-05 or use" mean that I >> > should >> > run my code with option " -pc_factor_fill 5.0746e-05"? >> > >> >> "Reallocs 0": no additional malloc()s were needed >> >> "Fill rato: give 5 needed 5.0746e-05" We guessed that the factor would >> need >> 5 times the matrix memory. It actually needed a lot less. >> >> If you run -pc_factor_fill 5.0746e-05, you will get the same performance >> and >> use a lot less memory. >> >> ? ?Matt >> >> >> > But when I run my code with option "-snes_view", I get "factor fill >> > ratio >> > given 5, needed 3.28701". >> > So what should I give to the option "-pc_factor_fill", 3.28701 or >> > 5.0746e-05? Thanks. >> > >> > Best, >> > Rongliang >> > >> >> >> >> -- >> What most experimenters take for granted before they begin their >> experiments >> is infinitely more interesting than any results to which their experiments >> lead. >> -- Norbert Wiener >> -------------- next part -------------- >> An HTML attachment was scrubbed... >> URL: >> >> >> ------------------------------ >> >> Message: 6 >> Date: Tue, 20 Sep 2011 20:07:44 -0500 >> From: Barry Smith >> Subject: Re: [petsc-users] Matrix Factorization fill >> To: PETSc users list >> Message-ID: <48785310-F532-48E8-8220-88392A5047FA at mcs.anl.gov> >> Content-Type: text/plain; charset=us-ascii >> >> >> On Sep 20, 2011, at 7:02 PM, Rongliang Chen wrote: >> >> > Hi, >> > >> > I have a question about the Sparse Matrix Factorization fill. >> > Following the manual, I use the option "-info" and get the following >> > result: >> > >> > [0] MatLUFactorSymbolic_SeqAIJ(): Reallocs 0 Fill ratio:given 5 needed >> > 5.0746e-05 >> > [0] MatLUFactorSymbolic_SeqAIJ(): Run with -pc_factor_fill 5.0746e-05 or >> > use >> > [0] MatLUFactorSymbolic_SeqAIJ(): PCFactorSetFill(pc,5.0746e-05); >> > >> > What's the numbers mean in "Reallocs 0 Fill ratio:given 5 needed >> > 5.0746e-05"? >> > Does the " Run with -pc_factor_fill 5.0746e-05 or use" mean that I >> > should run my code with option " -pc_factor_fill 5.0746e-05"? >> >> ? This is printed incorrectly due to a change in our data structures, we >> will fix it. >> >> > >> > But when I run my code with option "-snes_view", I get "factor fill >> > ratio given 5, needed 3.28701". >> > So what should I give to the option "-pc_factor_fill", 3.28701 or >> > 5.0746e-05? Thanks. >> >> ? The 3.28701 >> >> Barrty >> >> > >> > Best, >> > Rongliang >> >> >> >> ------------------------------ >> >> Message: 7 >> Date: Tue, 20 Sep 2011 22:21:53 -0500 (CDT) >> From: Shri >> Subject: Re: [petsc-users] Matrix Factorization fill >> To: PETSc users list >> Message-ID: >> ? ? ? ?<1333380760.24263.1316575313230.JavaMail.root at zimbra.anl.gov> >> Content-Type: text/plain; charset=utf-8 >> >> I can't reproduce this with snes/examples/tutorials/ex3.c,ex5.c, or >> ex19.c. Are you running a petsc example or your own application code? >> >> Shri >> ----- Original Message ----- >> > On Sep 20, 2011, at 7:02 PM, Rongliang Chen wrote: >> > >> > > Hi, >> > > >> > > I have a question about the Sparse Matrix Factorization fill. >> > > Following the manual, I use the option "-info" and get the following >> > > result: >> > > >> > > [0] MatLUFactorSymbolic_SeqAIJ(): Reallocs 0 Fill ratio:given 5 >> > > needed 5.0746e-05 >> > > [0] MatLUFactorSymbolic_SeqAIJ(): Run with -pc_factor_fill >> > > 5.0746e-05 or use >> > > [0] MatLUFactorSymbolic_SeqAIJ(): PCFactorSetFill(pc,5.0746e-05); >> > > >> > > What's the numbers mean in "Reallocs 0 Fill ratio:given 5 needed >> > > 5.0746e-05"? >> > > Does the " Run with -pc_factor_fill 5.0746e-05 or use" mean that I >> > > should run my code with option " -pc_factor_fill 5.0746e-05"? >> > >> > This is printed incorrectly due to a change in our data structures, we >> > will fix it. >> > >> > > >> > > But when I run my code with option "-snes_view", I get "factor fill >> > > ratio given 5, needed 3.28701". >> > > So what should I give to the option "-pc_factor_fill", 3.28701 or >> > > 5.0746e-05? Thanks. >> > >> > The 3.28701 >> > >> > Barrty >> > >> > > >> > > Best, >> > > Rongliang >> > From agrayver at gfz-potsdam.de Wed Sep 21 11:00:33 2011 From: agrayver at gfz-potsdam.de (Alexander Grayver) Date: Wed, 21 Sep 2011 18:00:33 +0200 Subject: [petsc-users] [petsc-3.2] LSQR new convergence test In-Reply-To: <4E7A000F.4030009@gfz-potsdam.de> References: <4E785F93.8020106@gfz-potsdam.de> <5C5AE85F-F5D5-4BE2-95F6-DE1D09A150DF@mcs.anl.gov> <4E7994AE.1060200@gfz-potsdam.de> <4E7A000F.4030009@gfz-potsdam.de> Message-ID: <4E7A0A21.70606@gfz-potsdam.de> Barry, I think the error is related to this PetscFortranAddr cctx definition. Speaking in terms of C++ notation, it seems that PETSc takes &cctx as KSPDefaultConvergedCtx, however real KSPDefaultConvergedCtx is *cctx. Not sure if I'm clear now, but it's easy to see if you trace code like that: PetscFortranAddr cctx KSP ksp call KSPCreate(MPI_COMM_WORLD,ksp,ierr) call KSPSetType(ksp,KSPLSQR,ierr) call KSPDefaultConvergedCreate(cctx,ierr) call KSPSetConvergenceTest(ksp,KSPDefaultConverged,cctx,PETSC_NULL_FUNCTION,ierr) call KSPDestroy(ksp,ierr) I guess the error should be reproducible from the code above. Regards, Alexander On 21.09.2011 17:17, Alexander Grayver wrote: > Thanks Barry, I added your code and solver works well now, but when I > call KSPDestroy afterward it crashes: > > [8]PETSC ERROR: --------------------- Error Message > ------------------------------------ > [8]PETSC ERROR: Invalid argument! > [8]PETSC ERROR: Wrong type of object: Parameter # 1! > [8]PETSC ERROR: > ------------------------------------------------------------------------ > [8]PETSC ERROR: Petsc Release Version 3.2.0, Patch 2, Fri Sep 16 > 10:10:45 CDT 2011 > [8]PETSC ERROR: See docs/changes/index.html for recent updates. > [8]PETSC ERROR: See docs/faq.html for hints about trouble shooting. > [8]PETSC ERROR: See docs/index.html for manual pages. > [8]PETSC ERROR: > ------------------------------------------------------------------------ > [8]PETSC ERROR: /home/test on a openmpi-i named node226 by user Wed > Sep 21 17:01:23 2011 > [8]PETSC ERROR: Libraries linked from > /home/lib/petsc-3.2-p2/openmpi-intel-complex-debug-f/lib > [8]PETSC ERROR: Configure run at Tue Sep 20 09:28:29 2011 > [8]PETSC ERROR: Configure options > --with-petsc-arch=openmpi-intel-complex-debug-f > --with-fortran-interfaces=1 > --with-mpi-dir=/opt/mpi/intel/openmpi-1.4.2 --with-scalar-type=complex > --with-blas-lapack-dir=/opt/intel/Compiler/11.1/072/mkl/lib/em64t > --with-precision=double --with-x=0 > [8]PETSC ERROR: > ------------------------------------------------------------------------ > [8]PETSC ERROR: VecDestroy() line 566 in > /home/lib/petsc-3.2-p2/src/vec/vec/interface/vector.c > [8]PETSC ERROR: KSPDefaultConvergedDestroy() line 669 in > /home/lib/petsc-3.2-p2/src/ksp/ksp/interface/iterativ.c > [8]PETSC ERROR: KSPDestroy() line 758 in > /home/lib/petsc-3.2-p2/src/ksp/ksp/interface/itfunc.c > > Is it my fault? > > Regards, > Alexander > > On 21.09.2011 15:36, Barry Smith wrote: >> On Sep 21, 2011, at 2:39 AM, Alexander Grayver wrote: >> >>> Hong, Barry, >>> >>> Thanks! Sorry for not to be clear, but Barry right, my question was >>> how to get back to DefaultConvergenceTest in case of LSQR solver. >>> >>> Barry, yes, both versions give the same two norms. >>> >>> Can you clarify please how to implement that in Fortran: >>> void *cctx; >>> KSPDefaultConvergedCreate(&cctx); >>> KSPSetConvergenceTest(ksp,KSPDefaultConverged,cctx); >> It should be almost the same >> >> PetscFortranAddr cctx >> >> call KSPDefaultConvergedCreate(cctx,ierr) >> call >> KSPSetConvergenceTest(ksp,KSPDefaultConverged,cctx,PETSC_NULL_FUNCTION,ierr) >> >> Barry >> >> >>> I'm a little bit confused about cctx parameter. >>> >>> Regards, >>> Alexander >>> >>> >>> On 21.09.2011 01:10, Barry Smith wrote: >>>> It has its own monitor that provides additional information >>>> -ksp_monitor_lsqr >>>> >>>> You can also remove the new convergence test and get back the old >>>> one with code like >>>> >>>> void *cctx; >>>> KSPDefaultConvergedCreate(&cctx); >>>> KSPSetConvergenceTest(ksp,KSPDefaultConverged,cctx); >>>> >>>> after the KSPType is set to LSQR. So if you are happy with the old >>>> test. >>>> >>>> >>>> Do both versions give the same first two norms? >>>> >>>>> 0 KSP Residual norm 9.386670021557e-17 >>>>> 1 KSP Residual norm 8.258308650175e-17 >>>> Barry >>>> >>>> >>>> On Sep 20, 2011, at 4:40 AM, Alexander Grayver wrote: >>>> >>>>> Hello, >>>>> >>>>> In comparison with petsc-3.1-p7 in the latest petsc LSQR solver >>>>> behaves differently. >>>>> >>>>> In petsc31 I had: >>>>> >>>>> 0 KSP Residual norm 9.386670021557e-17 >>>>> ... >>>>> 95 KSP Residual norm 9.341367317075e-18 >>>>> Linear solve converged due to CONVERGED_RTOL iterations 95 >>>>> KSP Object: >>>>> type: lsqr >>>>> maximum iterations=200, initial guess is zero >>>>> tolerances: relative=0.1, absolute=1e-50, divergence=10000 >>>>> left preconditioning >>>>> using PRECONDITIONED norm type for convergence test >>>>> PC Object: >>>>> type: none >>>>> linear system matrix = precond matrix: >>>>> Matrix Object: >>>>> type=shell, rows=19584, cols=19584 >>>>> >>>>> relative residual norm = 0.9951737192872134E-01 >>>>> >>>>> Now, with petsc32 I have: >>>>> >>>>> 0 KSP Residual norm 9.386670021557e-17 >>>>> 1 KSP Residual norm 8.258308650175e-17 >>>>> Linear solve converged due to CONVERGED_RTOL_NORMAL iterations 1 >>>>> KSP Object: 12 MPI processes >>>>> type: lsqr >>>>> maximum iterations=200, initial guess is zero >>>>> tolerances: relative=0.1, absolute=1e-50, divergence=10000 >>>>> left preconditioning >>>>> using UNPRECONDITIONED norm type for convergence test >>>>> PC Object: 12 MPI processes >>>>> type: none >>>>> linear system matrix = precond matrix: >>>>> Matrix Object: 12 MPI processes >>>>> type: shell >>>>> rows=19584, cols=19584 >>>>> relative residual norm = 0.8797910900468064E+00 >>>>> >>>>> So I found new convergence test here which sets >>>>> CONVERGED_RTOL_NORMAL: >>>>> http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/KSP/KSPLSQRDefaultConverged.html >>>>> >>>>> >>>>> The question is, how to interpret this new test and make it works >>>>> properly for me? >>>>> Thanks in advance. >>>>> >>>>> Regards, >>>>> Alexander > From zhenglun.wei at gmail.com Wed Sep 21 11:16:20 2011 From: zhenglun.wei at gmail.com (Alan Wei) Date: Wed, 21 Sep 2011 11:16:20 -0500 Subject: [petsc-users] Convergence Criteria? In-Reply-To: References: <1C05DB25-FA8B-49A9-8B33-D606C3F7126C@mcs.anl.gov> Message-ID: However, why signs for v[] in ComputeMatrix, which contains the values of a row of the matrix. They all have a negative signs. Therefore, I got confused which equation does this program solve for: 1) u[j][i] = (u[j+1][i] + u[j-1][i] + u[j][i+1] + u[j][i-1] - rhs * dx*dy)/4 or 2) 4u[j][i] - u[j+1][i] - u[j-1][i] - u[j][i+1] - u[j][i-1] + rhs * dx*dy = 0 thanks, Alan On Wed, Sep 21, 2011 at 8:22 AM, Barry Smith wrote: > > On Sep 20, 2011, at 10:47 PM, Alan Wei wrote: > > > Dear Dr. Smith, > > I figure out this problem. Actually, I made my own RHS, but I did not > multiply them by the volume (dx*dy). > > However, I met another problem. All values calculated from this > example are exactly opposite to values from my own code. I wonder if the RHS > I input by ComputeRHS are the really RHS or -1.*RHS? > > We do not change the sign of the right hand side. > > Barry > > > > > thanks in advance, > > Alan > > > > On Mon, Sep 19, 2011 at 8:43 PM, Barry Smith wrote: > > > > On Sep 19, 2011, at 6:25 PM, Alan Wei wrote: > > > > > Dear folks, > > > I hope you guys are having a nice day. > > > I'm reading the /src/ksp/ksp/examples/tutorials/ex29.c.html and > wonder how to set up the convergence criteria? > > > > -ksp_rtol 1.e-10 for example > > > > Run with -ksp_monitor_true_residual -ksp_converged_reason > > > > > > > Currently I use it as a poisson solver to solve a Poisson Equation with > three direction Neumann BC's and one direction Diriechlet BC's. It seems > very bad on it. > > > > Hmm, multigrid should likely converge well. Are you sure you've set > the BC's correctly? > > > > Barry > > > > > > > > thanks in advance, > > > Alan > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Wed Sep 21 11:19:04 2011 From: knepley at gmail.com (Matthew Knepley) Date: Wed, 21 Sep 2011 16:19:04 +0000 Subject: [petsc-users] Convergence Criteria? In-Reply-To: References: <1C05DB25-FA8B-49A9-8B33-D606C3F7126C@mcs.anl.gov> Message-ID: On Wed, Sep 21, 2011 at 4:16 PM, Alan Wei wrote: > However, why signs for v[] in ComputeMatrix, which contains the values of > a row of the matrix. They all have a negative signs. Therefore, I got > confused which equation does this program solve for: > 1) u[j][i] = (u[j+1][i] + u[j-1][i] + u[j][i+1] + u[j][i-1] - rhs * > dx*dy)/4 > or > 2) 4u[j][i] - u[j+1][i] - u[j-1][i] - u[j][i+1] - u[j][i-1] + rhs * dx*dy > = 0 > The Laplacian is a negative definite operator, so we usually solver -\Delta u = f since that is a positive definite problem. Thanks, Matt > thanks, > Alan > > > On Wed, Sep 21, 2011 at 8:22 AM, Barry Smith wrote: > >> >> On Sep 20, 2011, at 10:47 PM, Alan Wei wrote: >> >> > Dear Dr. Smith, >> > I figure out this problem. Actually, I made my own RHS, but I did >> not multiply them by the volume (dx*dy). >> > However, I met another problem. All values calculated from this >> example are exactly opposite to values from my own code. I wonder if the RHS >> I input by ComputeRHS are the really RHS or -1.*RHS? >> >> We do not change the sign of the right hand side. >> >> Barry >> >> > >> > thanks in advance, >> > Alan >> > >> > On Mon, Sep 19, 2011 at 8:43 PM, Barry Smith >> wrote: >> > >> > On Sep 19, 2011, at 6:25 PM, Alan Wei wrote: >> > >> > > Dear folks, >> > > I hope you guys are having a nice day. >> > > I'm reading the /src/ksp/ksp/examples/tutorials/ex29.c.html and >> wonder how to set up the convergence criteria? >> > >> > -ksp_rtol 1.e-10 for example >> > >> > Run with -ksp_monitor_true_residual -ksp_converged_reason >> > >> > >> > > Currently I use it as a poisson solver to solve a Poisson Equation >> with three direction Neumann BC's and one direction Diriechlet BC's. It >> seems very bad on it. >> > >> > Hmm, multigrid should likely converge well. Are you sure you've set >> the BC's correctly? >> > >> > Barry >> > >> > > >> > > thanks in advance, >> > > Alan >> > >> > >> >> > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From zhenglun.wei at gmail.com Wed Sep 21 11:28:31 2011 From: zhenglun.wei at gmail.com (Alan Wei) Date: Wed, 21 Sep 2011 11:28:31 -0500 Subject: [petsc-users] Convergence Criteria? In-Reply-To: References: <1C05DB25-FA8B-49A9-8B33-D606C3F7126C@mcs.anl.gov> Message-ID: Oh, Thanks, Matt, I got a little bit confused, since in the code, it described: div \rho grad u = f, 0 < x,y < 1, But you said, the solver solves -\Delta u = f (Eq.1), which means: for example, to solve a equation like Delta p = 1, I should put rhs = -1 = f in (Eq.1) in the code, therefore -\Delta u = -1, which, then, will give me a good result for Delta p = 1, is that true? thanks in advance, Alan On Wed, Sep 21, 2011 at 11:19 AM, Matthew Knepley wrote: > On Wed, Sep 21, 2011 at 4:16 PM, Alan Wei wrote: > >> However, why signs for v[] in ComputeMatrix, which contains the values of >> a row of the matrix. They all have a negative signs. Therefore, I got >> confused which equation does this program solve for: >> 1) u[j][i] = (u[j+1][i] + u[j-1][i] + u[j][i+1] + u[j][i-1] - rhs * >> dx*dy)/4 >> or >> 2) 4u[j][i] - u[j+1][i] - u[j-1][i] - u[j][i+1] - u[j][i-1] + rhs * dx*dy >> = 0 >> > > The Laplacian is a negative definite operator, so we usually solver -\Delta > u = f since that > is a positive definite problem. > > Thanks, > > Matt > > >> thanks, >> Alan >> >> >> On Wed, Sep 21, 2011 at 8:22 AM, Barry Smith wrote: >> >>> >>> On Sep 20, 2011, at 10:47 PM, Alan Wei wrote: >>> >>> > Dear Dr. Smith, >>> > I figure out this problem. Actually, I made my own RHS, but I did >>> not multiply them by the volume (dx*dy). >>> > However, I met another problem. All values calculated from this >>> example are exactly opposite to values from my own code. I wonder if the RHS >>> I input by ComputeRHS are the really RHS or -1.*RHS? >>> >>> We do not change the sign of the right hand side. >>> >>> Barry >>> >>> > >>> > thanks in advance, >>> > Alan >>> > >>> > On Mon, Sep 19, 2011 at 8:43 PM, Barry Smith >>> wrote: >>> > >>> > On Sep 19, 2011, at 6:25 PM, Alan Wei wrote: >>> > >>> > > Dear folks, >>> > > I hope you guys are having a nice day. >>> > > I'm reading the /src/ksp/ksp/examples/tutorials/ex29.c.html and >>> wonder how to set up the convergence criteria? >>> > >>> > -ksp_rtol 1.e-10 for example >>> > >>> > Run with -ksp_monitor_true_residual -ksp_converged_reason >>> > >>> > >>> > > Currently I use it as a poisson solver to solve a Poisson Equation >>> with three direction Neumann BC's and one direction Diriechlet BC's. It >>> seems very bad on it. >>> > >>> > Hmm, multigrid should likely converge well. Are you sure you've set >>> the BC's correctly? >>> > >>> > Barry >>> > >>> > > >>> > > thanks in advance, >>> > > Alan >>> > >>> > >>> >>> >> > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rongliang.chan at gmail.com Wed Sep 21 11:49:09 2011 From: rongliang.chan at gmail.com (Rongliang Chen) Date: Wed, 21 Sep 2011 10:49:09 -0600 Subject: [petsc-users] petsc-users Digest, Vol 33, Issue 54 In-Reply-To: References: Message-ID: Hi Hong, I am using the petsc-3.1-p8 and I am trying to switch to the petsc-3.2. Is it very important to switch to the latest version? Some of my friends are still using the petsc-3.0. Can the latest version improve the program's performance? Best, Rongliang ------------------------------ > > Message: 2 > Date: Wed, 21 Sep 2011 10:32:40 -0500 > From: Hong Zhang > Subject: Re: [petsc-users] Matrix Factorization fill > To: PETSc users list > Message-ID: > > > Content-Type: text/plain; charset=ISO-8859-1 > > Rongliang: > Which petsc version do you use? Please switch to the latest release > petsc-3.2. > > Hong > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hzhang at mcs.anl.gov Wed Sep 21 11:57:53 2011 From: hzhang at mcs.anl.gov (Hong Zhang) Date: Wed, 21 Sep 2011 11:57:53 -0500 Subject: [petsc-users] petsc-users Digest, Vol 33, Issue 54 In-Reply-To: References: Message-ID: Rongliang : > > I am using the petsc-3.1-p8 and I am trying to switch to the petsc-3.2. > Is it very important to switch to the latest version? Yes, very important. 3.1 is more than than two year old. petsc-3.2 provides many bugfixes, performance improvement and new solvers. Your bug report was fixed in petsc-3.2. Some of my friends are > still using the petsc-3.0. They are too lazy :-( See http://www.mcs.anl.gov/petsc/petsc-as/documentation/changes/32.html when updating your code to the latest petsc. > Can the latest version improve the program's performance? Yes, absolutely. Hong > > Best, > Rongliang > > ------------------------------ >> >> Message: 2 >> Date: Wed, 21 Sep 2011 10:32:40 -0500 >> From: Hong Zhang >> Subject: Re: [petsc-users] Matrix Factorization fill >> To: PETSc users list >> Message-ID: >> >> ? >> Content-Type: text/plain; charset=ISO-8859-1 >> >> Rongliang: >> Which petsc version do you use? Please switch to the latest release >> petsc-3.2. >> >> Hong >> > From bsmith at mcs.anl.gov Wed Sep 21 12:03:15 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Wed, 21 Sep 2011 12:03:15 -0500 Subject: [petsc-users] Convergence Criteria? In-Reply-To: References: <1C05DB25-FA8B-49A9-8B33-D606C3F7126C@mcs.anl.gov> Message-ID: Alan, It is very possible that the comment in the example code is wrong and has an incorrect sign. If you tell us what example this is we'll check it and fix the comment it if is wrong. Barry On Sep 21, 2011, at 11:28 AM, Alan Wei wrote: > Oh, Thanks, Matt, > I got a little bit confused, since in the code, it described: > div \rho grad u = f, 0 < x,y < 1, > But you said, the solver solves -\Delta u = f (Eq.1), which means: > for example, to solve a equation like Delta p = 1, I should put rhs = -1 = f in (Eq.1) in the code, therefore -\Delta u = -1, which, then, will give me a good result for Delta p = 1, is that true? > > thanks in advance, > Alan > > On Wed, Sep 21, 2011 at 11:19 AM, Matthew Knepley wrote: > On Wed, Sep 21, 2011 at 4:16 PM, Alan Wei wrote: > However, why signs for v[] in ComputeMatrix, which contains the values of a row of the matrix. They all have a negative signs. Therefore, I got confused which equation does this program solve for: > 1) u[j][i] = (u[j+1][i] + u[j-1][i] + u[j][i+1] + u[j][i-1] - rhs * dx*dy)/4 > or > 2) 4u[j][i] - u[j+1][i] - u[j-1][i] - u[j][i+1] - u[j][i-1] + rhs * dx*dy = 0 > > The Laplacian is a negative definite operator, so we usually solver -\Delta u = f since that > is a positive definite problem. > > Thanks, > > Matt > > thanks, > Alan > > > On Wed, Sep 21, 2011 at 8:22 AM, Barry Smith wrote: > > On Sep 20, 2011, at 10:47 PM, Alan Wei wrote: > > > Dear Dr. Smith, > > I figure out this problem. Actually, I made my own RHS, but I did not multiply them by the volume (dx*dy). > > However, I met another problem. All values calculated from this example are exactly opposite to values from my own code. I wonder if the RHS I input by ComputeRHS are the really RHS or -1.*RHS? > > We do not change the sign of the right hand side. > > Barry > > > > > thanks in advance, > > Alan > > > > On Mon, Sep 19, 2011 at 8:43 PM, Barry Smith wrote: > > > > On Sep 19, 2011, at 6:25 PM, Alan Wei wrote: > > > > > Dear folks, > > > I hope you guys are having a nice day. > > > I'm reading the /src/ksp/ksp/examples/tutorials/ex29.c.html and wonder how to set up the convergence criteria? > > > > -ksp_rtol 1.e-10 for example > > > > Run with -ksp_monitor_true_residual -ksp_converged_reason > > > > > > > Currently I use it as a poisson solver to solve a Poisson Equation with three direction Neumann BC's and one direction Diriechlet BC's. It seems very bad on it. > > > > Hmm, multigrid should likely converge well. Are you sure you've set the BC's correctly? > > > > Barry > > > > > > > > thanks in advance, > > > Alan > > > > > > > > > > -- > What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. > -- Norbert Wiener > From zhenglun.wei at gmail.com Wed Sep 21 13:19:58 2011 From: zhenglun.wei at gmail.com (Alan Wei) Date: Wed, 21 Sep 2011 13:19:58 -0500 Subject: [petsc-users] Convergence Criteria? In-Reply-To: References: <1C05DB25-FA8B-49A9-8B33-D606C3F7126C@mcs.anl.gov> Message-ID: Thanks, Dr. Smith, I guess it is wrong since I validate with my own code to solve a Poisson equation: Delta p = 1.0. The result from PETSc is exactly the negative value of what I get. The file is in /src/ksp/ksp/example/tutorial/ex29.c best, Alan On Wed, Sep 21, 2011 at 12:03 PM, Barry Smith wrote: > > Alan, > > It is very possible that the comment in the example code is wrong and has > an incorrect sign. If you tell us what example this is we'll check it and > fix the comment it if is wrong. > > Barry > > On Sep 21, 2011, at 11:28 AM, Alan Wei wrote: > > > Oh, Thanks, Matt, > > I got a little bit confused, since in the code, it described: > > div \rho grad u = f, 0 < x,y < 1, > > But you said, the solver solves -\Delta u = f (Eq.1), which means: > > for example, to solve a equation like Delta p = 1, I should put rhs = > -1 = f in (Eq.1) in the code, therefore -\Delta u = -1, which, then, will > give me a good result for Delta p = 1, is that true? > > > > thanks in advance, > > Alan > > > > On Wed, Sep 21, 2011 at 11:19 AM, Matthew Knepley > wrote: > > On Wed, Sep 21, 2011 at 4:16 PM, Alan Wei > wrote: > > However, why signs for v[] in ComputeMatrix, which contains the values of > a row of the matrix. They all have a negative signs. Therefore, I got > confused which equation does this program solve for: > > 1) u[j][i] = (u[j+1][i] + u[j-1][i] + u[j][i+1] + u[j][i-1] - rhs * > dx*dy)/4 > > or > > 2) 4u[j][i] - u[j+1][i] - u[j-1][i] - u[j][i+1] - u[j][i-1] + rhs * > dx*dy = 0 > > > > The Laplacian is a negative definite operator, so we usually solver > -\Delta u = f since that > > is a positive definite problem. > > > > Thanks, > > > > Matt > > > > thanks, > > Alan > > > > > > On Wed, Sep 21, 2011 at 8:22 AM, Barry Smith wrote: > > > > On Sep 20, 2011, at 10:47 PM, Alan Wei wrote: > > > > > Dear Dr. Smith, > > > I figure out this problem. Actually, I made my own RHS, but I did > not multiply them by the volume (dx*dy). > > > However, I met another problem. All values calculated from this > example are exactly opposite to values from my own code. I wonder if the RHS > I input by ComputeRHS are the really RHS or -1.*RHS? > > > > We do not change the sign of the right hand side. > > > > Barry > > > > > > > > thanks in advance, > > > Alan > > > > > > On Mon, Sep 19, 2011 at 8:43 PM, Barry Smith > wrote: > > > > > > On Sep 19, 2011, at 6:25 PM, Alan Wei wrote: > > > > > > > Dear folks, > > > > I hope you guys are having a nice day. > > > > I'm reading the /src/ksp/ksp/examples/tutorials/ex29.c.html and > wonder how to set up the convergence criteria? > > > > > > -ksp_rtol 1.e-10 for example > > > > > > Run with -ksp_monitor_true_residual -ksp_converged_reason > > > > > > > > > > Currently I use it as a poisson solver to solve a Poisson Equation > with three direction Neumann BC's and one direction Diriechlet BC's. It > seems very bad on it. > > > > > > Hmm, multigrid should likely converge well. Are you sure you've set > the BC's correctly? > > > > > > Barry > > > > > > > > > > > thanks in advance, > > > > Alan > > > > > > > > > > > > > > > > > > -- > > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > > -- Norbert Wiener > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Wed Sep 21 13:26:11 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Wed, 21 Sep 2011 13:26:11 -0500 Subject: [petsc-users] Convergence Criteria? In-Reply-To: References: <1C05DB25-FA8B-49A9-8B33-D606C3F7126C@mcs.anl.gov> Message-ID: <1587D981-EBB2-4EB9-A324-E93D62321A16@mcs.anl.gov> The comment will be fixed in the next patch Thanks for letting us know about the error Barry On Sep 21, 2011, at 1:19 PM, Alan Wei wrote: > Thanks, Dr. Smith, > I guess it is wrong since I validate with my own code to solve a Poisson equation: Delta p = 1.0. The result from PETSc is exactly the negative value of what I get. The file is in /src/ksp/ksp/example/tutorial/ex29.c > > best, > Alan > > On Wed, Sep 21, 2011 at 12:03 PM, Barry Smith wrote: > > Alan, > > It is very possible that the comment in the example code is wrong and has an incorrect sign. If you tell us what example this is we'll check it and fix the comment it if is wrong. > > Barry > > On Sep 21, 2011, at 11:28 AM, Alan Wei wrote: > > > Oh, Thanks, Matt, > > I got a little bit confused, since in the code, it described: > > div \rho grad u = f, 0 < x,y < 1, > > But you said, the solver solves -\Delta u = f (Eq.1), which means: > > for example, to solve a equation like Delta p = 1, I should put rhs = -1 = f in (Eq.1) in the code, therefore -\Delta u = -1, which, then, will give me a good result for Delta p = 1, is that true? > > > > thanks in advance, > > Alan > > > > On Wed, Sep 21, 2011 at 11:19 AM, Matthew Knepley wrote: > > On Wed, Sep 21, 2011 at 4:16 PM, Alan Wei wrote: > > However, why signs for v[] in ComputeMatrix, which contains the values of a row of the matrix. They all have a negative signs. Therefore, I got confused which equation does this program solve for: > > 1) u[j][i] = (u[j+1][i] + u[j-1][i] + u[j][i+1] + u[j][i-1] - rhs * dx*dy)/4 > > or > > 2) 4u[j][i] - u[j+1][i] - u[j-1][i] - u[j][i+1] - u[j][i-1] + rhs * dx*dy = 0 > > > > The Laplacian is a negative definite operator, so we usually solver -\Delta u = f since that > > is a positive definite problem. > > > > Thanks, > > > > Matt > > > > thanks, > > Alan > > > > > > On Wed, Sep 21, 2011 at 8:22 AM, Barry Smith wrote: > > > > On Sep 20, 2011, at 10:47 PM, Alan Wei wrote: > > > > > Dear Dr. Smith, > > > I figure out this problem. Actually, I made my own RHS, but I did not multiply them by the volume (dx*dy). > > > However, I met another problem. All values calculated from this example are exactly opposite to values from my own code. I wonder if the RHS I input by ComputeRHS are the really RHS or -1.*RHS? > > > > We do not change the sign of the right hand side. > > > > Barry > > > > > > > > thanks in advance, > > > Alan > > > > > > On Mon, Sep 19, 2011 at 8:43 PM, Barry Smith wrote: > > > > > > On Sep 19, 2011, at 6:25 PM, Alan Wei wrote: > > > > > > > Dear folks, > > > > I hope you guys are having a nice day. > > > > I'm reading the /src/ksp/ksp/examples/tutorials/ex29.c.html and wonder how to set up the convergence criteria? > > > > > > -ksp_rtol 1.e-10 for example > > > > > > Run with -ksp_monitor_true_residual -ksp_converged_reason > > > > > > > > > > Currently I use it as a poisson solver to solve a Poisson Equation with three direction Neumann BC's and one direction Diriechlet BC's. It seems very bad on it. > > > > > > Hmm, multigrid should likely converge well. Are you sure you've set the BC's correctly? > > > > > > Barry > > > > > > > > > > > thanks in advance, > > > > Alan > > > > > > > > > > > > > > > > > > -- > > What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. > > -- Norbert Wiener > > > > From zhenglun.wei at gmail.com Wed Sep 21 13:28:54 2011 From: zhenglun.wei at gmail.com (Alan Wei) Date: Wed, 21 Sep 2011 13:28:54 -0500 Subject: [petsc-users] Convergence Criteria? In-Reply-To: <1587D981-EBB2-4EB9-A324-E93D62321A16@mcs.anl.gov> References: <1C05DB25-FA8B-49A9-8B33-D606C3F7126C@mcs.anl.gov> <1587D981-EBB2-4EB9-A324-E93D62321A16@mcs.anl.gov> Message-ID: No problem, thank you guys to help me figure out the issue. ^_^ Alan On Wed, Sep 21, 2011 at 1:26 PM, Barry Smith wrote: > > The comment will be fixed in the next patch > > Thanks for letting us know about the error > > Barry > > On Sep 21, 2011, at 1:19 PM, Alan Wei wrote: > > > Thanks, Dr. Smith, > > I guess it is wrong since I validate with my own code to solve a > Poisson equation: Delta p = 1.0. The result from PETSc is exactly the > negative value of what I get. The file is in > /src/ksp/ksp/example/tutorial/ex29.c > > > > best, > > Alan > > > > On Wed, Sep 21, 2011 at 12:03 PM, Barry Smith > wrote: > > > > Alan, > > > > It is very possible that the comment in the example code is wrong and > has an incorrect sign. If you tell us what example this is we'll check it > and fix the comment it if is wrong. > > > > Barry > > > > On Sep 21, 2011, at 11:28 AM, Alan Wei wrote: > > > > > Oh, Thanks, Matt, > > > I got a little bit confused, since in the code, it described: > > > div \rho grad u = f, 0 < x,y < 1, > > > But you said, the solver solves -\Delta u = f (Eq.1), which means: > > > for example, to solve a equation like Delta p = 1, I should put rhs > = -1 = f in (Eq.1) in the code, therefore -\Delta u = -1, which, then, will > give me a good result for Delta p = 1, is that true? > > > > > > thanks in advance, > > > Alan > > > > > > On Wed, Sep 21, 2011 at 11:19 AM, Matthew Knepley > wrote: > > > On Wed, Sep 21, 2011 at 4:16 PM, Alan Wei > wrote: > > > However, why signs for v[] in ComputeMatrix, which contains the values > of a row of the matrix. They all have a negative signs. Therefore, I got > confused which equation does this program solve for: > > > 1) u[j][i] = (u[j+1][i] + u[j-1][i] + u[j][i+1] + u[j][i-1] - rhs * > dx*dy)/4 > > > or > > > 2) 4u[j][i] - u[j+1][i] - u[j-1][i] - u[j][i+1] - u[j][i-1] + rhs * > dx*dy = 0 > > > > > > The Laplacian is a negative definite operator, so we usually solver > -\Delta u = f since that > > > is a positive definite problem. > > > > > > Thanks, > > > > > > Matt > > > > > > thanks, > > > Alan > > > > > > > > > On Wed, Sep 21, 2011 at 8:22 AM, Barry Smith > wrote: > > > > > > On Sep 20, 2011, at 10:47 PM, Alan Wei wrote: > > > > > > > Dear Dr. Smith, > > > > I figure out this problem. Actually, I made my own RHS, but I did > not multiply them by the volume (dx*dy). > > > > However, I met another problem. All values calculated from this > example are exactly opposite to values from my own code. I wonder if the RHS > I input by ComputeRHS are the really RHS or -1.*RHS? > > > > > > We do not change the sign of the right hand side. > > > > > > Barry > > > > > > > > > > > thanks in advance, > > > > Alan > > > > > > > > On Mon, Sep 19, 2011 at 8:43 PM, Barry Smith > wrote: > > > > > > > > On Sep 19, 2011, at 6:25 PM, Alan Wei wrote: > > > > > > > > > Dear folks, > > > > > I hope you guys are having a nice day. > > > > > I'm reading the /src/ksp/ksp/examples/tutorials/ex29.c.html and > wonder how to set up the convergence criteria? > > > > > > > > -ksp_rtol 1.e-10 for example > > > > > > > > Run with -ksp_monitor_true_residual -ksp_converged_reason > > > > > > > > > > > > > Currently I use it as a poisson solver to solve a Poisson Equation > with three direction Neumann BC's and one direction Diriechlet BC's. It > seems very bad on it. > > > > > > > > Hmm, multigrid should likely converge well. Are you sure you've > set the BC's correctly? > > > > > > > > Barry > > > > > > > > > > > > > > thanks in advance, > > > > > Alan > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > > > -- Norbert Wiener > > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Wed Sep 21 14:58:13 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Wed, 21 Sep 2011 14:58:13 -0500 Subject: [petsc-users] [petsc-3.2] LSQR new convergence test In-Reply-To: <4E7A0A21.70606@gfz-potsdam.de> References: <4E785F93.8020106@gfz-potsdam.de> <5C5AE85F-F5D5-4BE2-95F6-DE1D09A150DF@mcs.anl.gov> <4E7994AE.1060200@gfz-potsdam.de> <4E7A000F.4030009@gfz-potsdam.de> <4E7A0A21.70606@gfz-potsdam.de> Message-ID: <21319D2F-148A-4F80-AD37-E05C0383C91F@mcs.anl.gov> Alexander, Yes, a problem with trying to handle C pointers on the Fortran side. It will be fixed in the next patch release. You can drop the attached file into src/ksp/ksp/interface/ftn-custom run make in that directory and then relink your program and it should work. Barry [see attached file: zitfuncf90.c] On Sep 21, 2011, at 11:00 AM, Alexander Grayver wrote: > Barry, > > I think the error is related to this > > PetscFortranAddr cctx > > definition. Speaking in terms of C++ notation, it seems that PETSc takes &cctx as KSPDefaultConvergedCtx, however real KSPDefaultConvergedCtx is *cctx. > Not sure if I'm clear now, but it's easy to see if you trace code like that: > > PetscFortranAddr cctx > KSP ksp > > call KSPCreate(MPI_COMM_WORLD,ksp,ierr) > call KSPSetType(ksp,KSPLSQR,ierr) > call KSPDefaultConvergedCreate(cctx,ierr) > call KSPSetConvergenceTest(ksp,KSPDefaultConverged,cctx,PETSC_NULL_FUNCTION,ierr) > call KSPDestroy(ksp,ierr) > > I guess the error should be reproducible from the code above. > > Regards, > Alexander > > On 21.09.2011 17:17, Alexander Grayver wrote: >> Thanks Barry, I added your code and solver works well now, but when I call KSPDestroy afterward it crashes: >> >> [8]PETSC ERROR: --------------------- Error Message ------------------------------------ >> [8]PETSC ERROR: Invalid argument! >> [8]PETSC ERROR: Wrong type of object: Parameter # 1! >> [8]PETSC ERROR: ------------------------------------------------------------------------ >> [8]PETSC ERROR: Petsc Release Version 3.2.0, Patch 2, Fri Sep 16 10:10:45 CDT 2011 >> [8]PETSC ERROR: See docs/changes/index.html for recent updates. >> [8]PETSC ERROR: See docs/faq.html for hints about trouble shooting. >> [8]PETSC ERROR: See docs/index.html for manual pages. >> [8]PETSC ERROR: ------------------------------------------------------------------------ >> [8]PETSC ERROR: /home/test on a openmpi-i named node226 by user Wed Sep 21 17:01:23 2011 >> [8]PETSC ERROR: Libraries linked from /home/lib/petsc-3.2-p2/openmpi-intel-complex-debug-f/lib >> [8]PETSC ERROR: Configure run at Tue Sep 20 09:28:29 2011 >> [8]PETSC ERROR: Configure options --with-petsc-arch=openmpi-intel-complex-debug-f --with-fortran-interfaces=1 --with-mpi-dir=/opt/mpi/intel/openmpi-1.4.2 --with-scalar-type=complex --with-blas-lapack-dir=/opt/intel/Compiler/11.1/072/mkl/lib/em64t --with-precision=double --with-x=0 >> [8]PETSC ERROR: ------------------------------------------------------------------------ >> [8]PETSC ERROR: VecDestroy() line 566 in /home/lib/petsc-3.2-p2/src/vec/vec/interface/vector.c >> [8]PETSC ERROR: KSPDefaultConvergedDestroy() line 669 in /home/lib/petsc-3.2-p2/src/ksp/ksp/interface/iterativ.c >> [8]PETSC ERROR: KSPDestroy() line 758 in /home/lib/petsc-3.2-p2/src/ksp/ksp/interface/itfunc.c >> >> Is it my fault? >> >> Regards, >> Alexander >> >> On 21.09.2011 15:36, Barry Smith wrote: >>> On Sep 21, 2011, at 2:39 AM, Alexander Grayver wrote: >>> >>>> Hong, Barry, >>>> >>>> Thanks! Sorry for not to be clear, but Barry right, my question was how to get back to DefaultConvergenceTest in case of LSQR solver. >>>> >>>> Barry, yes, both versions give the same two norms. >>>> >>>> Can you clarify please how to implement that in Fortran: >>>> void *cctx; >>>> KSPDefaultConvergedCreate(&cctx); >>>> KSPSetConvergenceTest(ksp,KSPDefaultConverged,cctx); >>> It should be almost the same >>> >>> PetscFortranAddr cctx >>> >>> call KSPDefaultConvergedCreate(cctx,ierr) >>> call KSPSetConvergenceTest(ksp,KSPDefaultConverged,cctx,PETSC_NULL_FUNCTION,ierr) >>> >>> Barry >>> >>> >>>> I'm a little bit confused about cctx parameter. >>>> >>>> Regards, >>>> Alexander >>>> >>>> >>>> On 21.09.2011 01:10, Barry Smith wrote: >>>>> It has its own monitor that provides additional information -ksp_monitor_lsqr >>>>> >>>>> You can also remove the new convergence test and get back the old one with code like >>>>> >>>>> void *cctx; >>>>> KSPDefaultConvergedCreate(&cctx); >>>>> KSPSetConvergenceTest(ksp,KSPDefaultConverged,cctx); >>>>> >>>>> after the KSPType is set to LSQR. So if you are happy with the old test. >>>>> >>>>> >>>>> Do both versions give the same first two norms? >>>>> >>>>>> 0 KSP Residual norm 9.386670021557e-17 >>>>>> 1 KSP Residual norm 8.258308650175e-17 >>>>> Barry >>>>> >>>>> >>>>> On Sep 20, 2011, at 4:40 AM, Alexander Grayver wrote: >>>>> >>>>>> Hello, >>>>>> >>>>>> In comparison with petsc-3.1-p7 in the latest petsc LSQR solver behaves differently. >>>>>> >>>>>> In petsc31 I had: >>>>>> >>>>>> 0 KSP Residual norm 9.386670021557e-17 >>>>>> ... >>>>>> 95 KSP Residual norm 9.341367317075e-18 >>>>>> Linear solve converged due to CONVERGED_RTOL iterations 95 >>>>>> KSP Object: >>>>>> type: lsqr >>>>>> maximum iterations=200, initial guess is zero >>>>>> tolerances: relative=0.1, absolute=1e-50, divergence=10000 >>>>>> left preconditioning >>>>>> using PRECONDITIONED norm type for convergence test >>>>>> PC Object: >>>>>> type: none >>>>>> linear system matrix = precond matrix: >>>>>> Matrix Object: >>>>>> type=shell, rows=19584, cols=19584 >>>>>> >>>>>> relative residual norm = 0.9951737192872134E-01 >>>>>> >>>>>> Now, with petsc32 I have: >>>>>> >>>>>> 0 KSP Residual norm 9.386670021557e-17 >>>>>> 1 KSP Residual norm 8.258308650175e-17 >>>>>> Linear solve converged due to CONVERGED_RTOL_NORMAL iterations 1 >>>>>> KSP Object: 12 MPI processes >>>>>> type: lsqr >>>>>> maximum iterations=200, initial guess is zero >>>>>> tolerances: relative=0.1, absolute=1e-50, divergence=10000 >>>>>> left preconditioning >>>>>> using UNPRECONDITIONED norm type for convergence test >>>>>> PC Object: 12 MPI processes >>>>>> type: none >>>>>> linear system matrix = precond matrix: >>>>>> Matrix Object: 12 MPI processes >>>>>> type: shell >>>>>> rows=19584, cols=19584 >>>>>> relative residual norm = 0.8797910900468064E+00 >>>>>> >>>>>> So I found new convergence test here which sets CONVERGED_RTOL_NORMAL: >>>>>> http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/KSP/KSPLSQRDefaultConverged.html >>>>>> >>>>>> The question is, how to interpret this new test and make it works properly for me? >>>>>> Thanks in advance. >>>>>> >>>>>> Regards, >>>>>> Alexander >> > -------------- next part -------------- A non-text attachment was scrubbed... Name: zitfuncf90.c Type: application/octet-stream Size: 603 bytes Desc: not available URL: From agrayver at gfz-potsdam.de Thu Sep 22 04:01:50 2011 From: agrayver at gfz-potsdam.de (Alexander Grayver) Date: Thu, 22 Sep 2011 11:01:50 +0200 Subject: [petsc-users] [petsc-3.2] LSQR new convergence test In-Reply-To: <21319D2F-148A-4F80-AD37-E05C0383C91F@mcs.anl.gov> References: <4E785F93.8020106@gfz-potsdam.de> <5C5AE85F-F5D5-4BE2-95F6-DE1D09A150DF@mcs.anl.gov> <4E7994AE.1060200@gfz-potsdam.de> <4E7A000F.4030009@gfz-potsdam.de> <4E7A0A21.70606@gfz-potsdam.de> <21319D2F-148A-4F80-AD37-E05C0383C91F@mcs.anl.gov> Message-ID: <4E7AF97E.7040901@gfz-potsdam.de> Barry, Unfortunately, the problem is still there. Even though I put zitfuncf90.c into src/ksp/ksp/interface/ftn-custom, modified makefile there and made configure/make from scratch. When are you going to release new patch? Regards, Alexander On 21.09.2011 21:58, Barry Smith wrote: > Alexander, > > Yes, a problem with trying to handle C pointers on the Fortran side. It will be fixed in the next patch release. You can drop the attached file into src/ksp/ksp/interface/ftn-custom run make in that directory and then relink your program and it should work. > > Barry > [see attached file: zitfuncf90.c] > On Sep 21, 2011, at 11:00 AM, Alexander Grayver wrote: > >> Barry, >> >> I think the error is related to this >> >> PetscFortranAddr cctx >> >> definition. Speaking in terms of C++ notation, it seems that PETSc takes&cctx as KSPDefaultConvergedCtx, however real KSPDefaultConvergedCtx is *cctx. >> Not sure if I'm clear now, but it's easy to see if you trace code like that: >> >> PetscFortranAddr cctx >> KSP ksp >> >> call KSPCreate(MPI_COMM_WORLD,ksp,ierr) >> call KSPSetType(ksp,KSPLSQR,ierr) >> call KSPDefaultConvergedCreate(cctx,ierr) >> call KSPSetConvergenceTest(ksp,KSPDefaultConverged,cctx,PETSC_NULL_FUNCTION,ierr) >> call KSPDestroy(ksp,ierr) >> >> I guess the error should be reproducible from the code above. >> >> Regards, >> Alexander >> >> On 21.09.2011 17:17, Alexander Grayver wrote: >>> Thanks Barry, I added your code and solver works well now, but when I call KSPDestroy afterward it crashes: >>> >>> [8]PETSC ERROR: --------------------- Error Message ------------------------------------ >>> [8]PETSC ERROR: Invalid argument! >>> [8]PETSC ERROR: Wrong type of object: Parameter # 1! >>> [8]PETSC ERROR: ------------------------------------------------------------------------ >>> [8]PETSC ERROR: Petsc Release Version 3.2.0, Patch 2, Fri Sep 16 10:10:45 CDT 2011 >>> [8]PETSC ERROR: See docs/changes/index.html for recent updates. >>> [8]PETSC ERROR: See docs/faq.html for hints about trouble shooting. >>> [8]PETSC ERROR: See docs/index.html for manual pages. >>> [8]PETSC ERROR: ------------------------------------------------------------------------ >>> [8]PETSC ERROR: /home/test on a openmpi-i named node226 by user Wed Sep 21 17:01:23 2011 >>> [8]PETSC ERROR: Libraries linked from /home/lib/petsc-3.2-p2/openmpi-intel-complex-debug-f/lib >>> [8]PETSC ERROR: Configure run at Tue Sep 20 09:28:29 2011 >>> [8]PETSC ERROR: Configure options --with-petsc-arch=openmpi-intel-complex-debug-f --with-fortran-interfaces=1 --with-mpi-dir=/opt/mpi/intel/openmpi-1.4.2 --with-scalar-type=complex --with-blas-lapack-dir=/opt/intel/Compiler/11.1/072/mkl/lib/em64t --with-precision=double --with-x=0 >>> [8]PETSC ERROR: ------------------------------------------------------------------------ >>> [8]PETSC ERROR: VecDestroy() line 566 in /home/lib/petsc-3.2-p2/src/vec/vec/interface/vector.c >>> [8]PETSC ERROR: KSPDefaultConvergedDestroy() line 669 in /home/lib/petsc-3.2-p2/src/ksp/ksp/interface/iterativ.c >>> [8]PETSC ERROR: KSPDestroy() line 758 in /home/lib/petsc-3.2-p2/src/ksp/ksp/interface/itfunc.c >>> >>> Is it my fault? >>> >>> Regards, >>> Alexander >>> >>> On 21.09.2011 15:36, Barry Smith wrote: >>>> On Sep 21, 2011, at 2:39 AM, Alexander Grayver wrote: >>>> >>>>> Hong, Barry, >>>>> >>>>> Thanks! Sorry for not to be clear, but Barry right, my question was how to get back to DefaultConvergenceTest in case of LSQR solver. >>>>> >>>>> Barry, yes, both versions give the same two norms. >>>>> >>>>> Can you clarify please how to implement that in Fortran: >>>>> void *cctx; >>>>> KSPDefaultConvergedCreate(&cctx); >>>>> KSPSetConvergenceTest(ksp,KSPDefaultConverged,cctx); >>>> It should be almost the same >>>> >>>> PetscFortranAddr cctx >>>> >>>> call KSPDefaultConvergedCreate(cctx,ierr) >>>> call KSPSetConvergenceTest(ksp,KSPDefaultConverged,cctx,PETSC_NULL_FUNCTION,ierr) >>>> >>>> Barry >>>> >>>> >>>>> I'm a little bit confused about cctx parameter. >>>>> >>>>> Regards, >>>>> Alexander >>>>> >>>>> >>>>> On 21.09.2011 01:10, Barry Smith wrote: >>>>>> It has its own monitor that provides additional information -ksp_monitor_lsqr >>>>>> >>>>>> You can also remove the new convergence test and get back the old one with code like >>>>>> >>>>>> void *cctx; >>>>>> KSPDefaultConvergedCreate(&cctx); >>>>>> KSPSetConvergenceTest(ksp,KSPDefaultConverged,cctx); >>>>>> >>>>>> after the KSPType is set to LSQR. So if you are happy with the old test. >>>>>> >>>>>> >>>>>> Do both versions give the same first two norms? >>>>>> >>>>>>> 0 KSP Residual norm 9.386670021557e-17 >>>>>>> 1 KSP Residual norm 8.258308650175e-17 >>>>>> Barry >>>>>> >>>>>> >>>>>> On Sep 20, 2011, at 4:40 AM, Alexander Grayver wrote: >>>>>> >>>>>>> Hello, >>>>>>> >>>>>>> In comparison with petsc-3.1-p7 in the latest petsc LSQR solver behaves differently. >>>>>>> >>>>>>> In petsc31 I had: >>>>>>> >>>>>>> 0 KSP Residual norm 9.386670021557e-17 >>>>>>> ... >>>>>>> 95 KSP Residual norm 9.341367317075e-18 >>>>>>> Linear solve converged due to CONVERGED_RTOL iterations 95 >>>>>>> KSP Object: >>>>>>> type: lsqr >>>>>>> maximum iterations=200, initial guess is zero >>>>>>> tolerances: relative=0.1, absolute=1e-50, divergence=10000 >>>>>>> left preconditioning >>>>>>> using PRECONDITIONED norm type for convergence test >>>>>>> PC Object: >>>>>>> type: none >>>>>>> linear system matrix = precond matrix: >>>>>>> Matrix Object: >>>>>>> type=shell, rows=19584, cols=19584 >>>>>>> >>>>>>> relative residual norm = 0.9951737192872134E-01 >>>>>>> >>>>>>> Now, with petsc32 I have: >>>>>>> >>>>>>> 0 KSP Residual norm 9.386670021557e-17 >>>>>>> 1 KSP Residual norm 8.258308650175e-17 >>>>>>> Linear solve converged due to CONVERGED_RTOL_NORMAL iterations 1 >>>>>>> KSP Object: 12 MPI processes >>>>>>> type: lsqr >>>>>>> maximum iterations=200, initial guess is zero >>>>>>> tolerances: relative=0.1, absolute=1e-50, divergence=10000 >>>>>>> left preconditioning >>>>>>> using UNPRECONDITIONED norm type for convergence test >>>>>>> PC Object: 12 MPI processes >>>>>>> type: none >>>>>>> linear system matrix = precond matrix: >>>>>>> Matrix Object: 12 MPI processes >>>>>>> type: shell >>>>>>> rows=19584, cols=19584 >>>>>>> relative residual norm = 0.8797910900468064E+00 >>>>>>> >>>>>>> So I found new convergence test here which sets CONVERGED_RTOL_NORMAL: >>>>>>> http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/KSP/KSPLSQRDefaultConverged.html >>>>>>> >>>>>>> The question is, how to interpret this new test and make it works properly for me? >>>>>>> Thanks in advance. >>>>>>> >>>>>>> Regards, >>>>>>> Alexander From jedbrown at mcs.anl.gov Thu Sep 22 05:04:49 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Thu, 22 Sep 2011 12:04:49 +0200 Subject: [petsc-users] floating region help In-Reply-To: <4817529.236491316577116602.JavaMail.coremail@mail.ustc.edu> References: <588771.234921316506085787.JavaMail.coremail@mail.ustc.edu> <1525625.235221316510237990.JavaMail.coremail@mail.ustc.edu> <6672045.235591316525172753.JavaMail.coremail@mail.ustc.edu> <4817529.236491316577116602.JavaMail.coremail@mail.ustc.edu> Message-ID: On Wed, Sep 21, 2011 at 05:51, Gong Ding wrote: > The background of this problem is the single event upset simulation of CMOS > unit circuit, i.e. SRAM, latch, flip-flop. > They all very big, ~ 1M mesh nodes. Direct method is too slow and memory > consuming. > Some stupid commercial code needs more than one week to simulate one > particle event. > People usually need 100-1000 events to determine the circuit behavior! > > Fortunately?the problem is in time domain. the transport equation of > carrier has a d/dt term which helps stable. > The most difficult part comes from metal connection. Just use above > inverter, when both NMOS and PMOS are closed, > the metal connection region has only displacement current from > semiconductor region as eps*dE/dt, while metal region > the current can be written as sigma*E. The current conservation here is the > boundary condition at the metal-semiconductor interface. > (Since E is \frac{\partical phi}{\partical n}, here phi is the independent > variable) > How much does the time step help? Given the description, I'd expect not much unless it's unreasonably short. > > Obviously, sigma >> eps/T, the governing equation of phi has a large jump > parameter at the metal/semiconductor interface. > The eigen value analysis confirmed that the number of samllest eigen value > equals to the number of floating metal region. > And the eigen vector is exactly the floating region. > Hmm, with a single low-energy mode, could you just start the Krylov method with a nonzero initial guess and make the initial guess exactly this mode. That would ensure that the Krylov space captures it exactly, and further iterations should all come from picking up the rest of the modes. Alternatively, you could formulate a modified system for which this mode was removed analytically. Just tossing out ideas here. -------------- next part -------------- An HTML attachment was scrubbed... URL: From balay at mcs.anl.gov Thu Sep 22 07:54:02 2011 From: balay at mcs.anl.gov (Satish Balay) Date: Thu, 22 Sep 2011 07:54:02 -0500 (CDT) Subject: [petsc-users] [petsc-3.2] LSQR new convergence test In-Reply-To: <4E7AF97E.7040901@gfz-potsdam.de> References: <4E785F93.8020106@gfz-potsdam.de> <5C5AE85F-F5D5-4BE2-95F6-DE1D09A150DF@mcs.anl.gov> <4E7994AE.1060200@gfz-potsdam.de> <4E7A000F.4030009@gfz-potsdam.de> <4E7A0A21.70606@gfz-potsdam.de> <21319D2F-148A-4F80-AD37-E05C0383C91F@mcs.anl.gov> <4E7AF97E.7040901@gfz-potsdam.de> Message-ID: Loks like Barry sent you the wrong file. zitfuncf90.c belongs to src/ksp/ksp/interface/f90-custom [and is unchanged] Try the attached patch cd petsc-3.2-p2 patch -Np1 < ksp-conv.patch Satish On Thu, 22 Sep 2011, Alexander Grayver wrote: > Barry, > > Unfortunately, the problem is still there. Even though I put zitfuncf90.c into > src/ksp/ksp/interface/ftn-custom, modified makefile there and made > configure/make from scratch. > When are you going to release new patch? > > Regards, > Alexander > > On 21.09.2011 21:58, Barry Smith wrote: > > Alexander, > > > > Yes, a problem with trying to handle C pointers on the Fortran side. It > > will be fixed in the next patch release. You can drop the attached file into > > src/ksp/ksp/interface/ftn-custom run make in that directory and then relink > > your program and it should work. > > > > Barry > > [see attached file: zitfuncf90.c] > > On Sep 21, 2011, at 11:00 AM, Alexander Grayver wrote: > > > > > Barry, > > > > > > I think the error is related to this > > > > > > PetscFortranAddr cctx > > > > > > definition. Speaking in terms of C++ notation, it seems that PETSc > > > takes&cctx as KSPDefaultConvergedCtx, however real KSPDefaultConvergedCtx > > > is *cctx. > > > Not sure if I'm clear now, but it's easy to see if you trace code like > > > that: > > > > > > PetscFortranAddr cctx > > > KSP ksp > > > > > > call KSPCreate(MPI_COMM_WORLD,ksp,ierr) > > > call KSPSetType(ksp,KSPLSQR,ierr) > > > call KSPDefaultConvergedCreate(cctx,ierr) > > > call > > > KSPSetConvergenceTest(ksp,KSPDefaultConverged,cctx,PETSC_NULL_FUNCTION,ierr) > > > call KSPDestroy(ksp,ierr) > > > > > > I guess the error should be reproducible from the code above. > > > > > > Regards, > > > Alexander > > > > > > On 21.09.2011 17:17, Alexander Grayver wrote: > > > > Thanks Barry, I added your code and solver works well now, but when I > > > > call KSPDestroy afterward it crashes: > > > > > > > > [8]PETSC ERROR: --------------------- Error Message > > > > ------------------------------------ > > > > [8]PETSC ERROR: Invalid argument! > > > > [8]PETSC ERROR: Wrong type of object: Parameter # 1! > > > > [8]PETSC ERROR: > > > > ------------------------------------------------------------------------ > > > > [8]PETSC ERROR: Petsc Release Version 3.2.0, Patch 2, Fri Sep 16 > > > > 10:10:45 CDT 2011 > > > > [8]PETSC ERROR: See docs/changes/index.html for recent updates. > > > > [8]PETSC ERROR: See docs/faq.html for hints about trouble shooting. > > > > [8]PETSC ERROR: See docs/index.html for manual pages. > > > > [8]PETSC ERROR: > > > > ------------------------------------------------------------------------ > > > > [8]PETSC ERROR: /home/test on a openmpi-i named node226 by user Wed Sep > > > > 21 17:01:23 2011 > > > > [8]PETSC ERROR: Libraries linked from > > > > /home/lib/petsc-3.2-p2/openmpi-intel-complex-debug-f/lib > > > > [8]PETSC ERROR: Configure run at Tue Sep 20 09:28:29 2011 > > > > [8]PETSC ERROR: Configure options > > > > --with-petsc-arch=openmpi-intel-complex-debug-f > > > > --with-fortran-interfaces=1 --with-mpi-dir=/opt/mpi/intel/openmpi-1.4.2 > > > > --with-scalar-type=complex > > > > --with-blas-lapack-dir=/opt/intel/Compiler/11.1/072/mkl/lib/em64t > > > > --with-precision=double --with-x=0 > > > > [8]PETSC ERROR: > > > > ------------------------------------------------------------------------ > > > > [8]PETSC ERROR: VecDestroy() line 566 in > > > > /home/lib/petsc-3.2-p2/src/vec/vec/interface/vector.c > > > > [8]PETSC ERROR: KSPDefaultConvergedDestroy() line 669 in > > > > /home/lib/petsc-3.2-p2/src/ksp/ksp/interface/iterativ.c > > > > [8]PETSC ERROR: KSPDestroy() line 758 in > > > > /home/lib/petsc-3.2-p2/src/ksp/ksp/interface/itfunc.c > > > > > > > > Is it my fault? > > > > > > > > Regards, > > > > Alexander > > > > > > > > On 21.09.2011 15:36, Barry Smith wrote: > > > > > On Sep 21, 2011, at 2:39 AM, Alexander Grayver wrote: > > > > > > > > > > > Hong, Barry, > > > > > > > > > > > > Thanks! Sorry for not to be clear, but Barry right, my question was > > > > > > how to get back to DefaultConvergenceTest in case of LSQR solver. > > > > > > > > > > > > Barry, yes, both versions give the same two norms. > > > > > > > > > > > > Can you clarify please how to implement that in Fortran: > > > > > > void *cctx; > > > > > > KSPDefaultConvergedCreate(&cctx); > > > > > > KSPSetConvergenceTest(ksp,KSPDefaultConverged,cctx); > > > > > It should be almost the same > > > > > > > > > > PetscFortranAddr cctx > > > > > > > > > > call KSPDefaultConvergedCreate(cctx,ierr) > > > > > call > > > > > KSPSetConvergenceTest(ksp,KSPDefaultConverged,cctx,PETSC_NULL_FUNCTION,ierr) > > > > > > > > > > Barry > > > > > > > > > > > > > > > > I'm a little bit confused about cctx parameter. > > > > > > > > > > > > Regards, > > > > > > Alexander > > > > > > > > > > > > > > > > > > On 21.09.2011 01:10, Barry Smith wrote: > > > > > > > It has its own monitor that provides additional information > > > > > > > -ksp_monitor_lsqr > > > > > > > > > > > > > > You can also remove the new convergence test and get back the old > > > > > > > one with code like > > > > > > > > > > > > > > void *cctx; > > > > > > > KSPDefaultConvergedCreate(&cctx); > > > > > > > KSPSetConvergenceTest(ksp,KSPDefaultConverged,cctx); > > > > > > > > > > > > > > after the KSPType is set to LSQR. So if you are happy with the old > > > > > > > test. > > > > > > > > > > > > > > > > > > > > > Do both versions give the same first two norms? > > > > > > > > > > > > > > > 0 KSP Residual norm 9.386670021557e-17 > > > > > > > > 1 KSP Residual norm 8.258308650175e-17 > > > > > > > Barry > > > > > > > > > > > > > > > > > > > > > On Sep 20, 2011, at 4:40 AM, Alexander Grayver wrote: > > > > > > > > > > > > > > > Hello, > > > > > > > > > > > > > > > > In comparison with petsc-3.1-p7 in the latest petsc LSQR solver > > > > > > > > behaves differently. > > > > > > > > > > > > > > > > In petsc31 I had: > > > > > > > > > > > > > > > > 0 KSP Residual norm 9.386670021557e-17 > > > > > > > > ... > > > > > > > > 95 KSP Residual norm 9.341367317075e-18 > > > > > > > > Linear solve converged due to CONVERGED_RTOL iterations 95 > > > > > > > > KSP Object: > > > > > > > > type: lsqr > > > > > > > > maximum iterations=200, initial guess is zero > > > > > > > > tolerances: relative=0.1, absolute=1e-50, divergence=10000 > > > > > > > > left preconditioning > > > > > > > > using PRECONDITIONED norm type for convergence test > > > > > > > > PC Object: > > > > > > > > type: none > > > > > > > > linear system matrix = precond matrix: > > > > > > > > Matrix Object: > > > > > > > > type=shell, rows=19584, cols=19584 > > > > > > > > > > > > > > > > relative residual norm = 0.9951737192872134E-01 > > > > > > > > > > > > > > > > Now, with petsc32 I have: > > > > > > > > > > > > > > > > 0 KSP Residual norm 9.386670021557e-17 > > > > > > > > 1 KSP Residual norm 8.258308650175e-17 > > > > > > > > Linear solve converged due to CONVERGED_RTOL_NORMAL iterations 1 > > > > > > > > KSP Object: 12 MPI processes > > > > > > > > type: lsqr > > > > > > > > maximum iterations=200, initial guess is zero > > > > > > > > tolerances: relative=0.1, absolute=1e-50, divergence=10000 > > > > > > > > left preconditioning > > > > > > > > using UNPRECONDITIONED norm type for convergence test > > > > > > > > PC Object: 12 MPI processes > > > > > > > > type: none > > > > > > > > linear system matrix = precond matrix: > > > > > > > > Matrix Object: 12 MPI processes > > > > > > > > type: shell > > > > > > > > rows=19584, cols=19584 > > > > > > > > relative residual norm = 0.8797910900468064E+00 > > > > > > > > > > > > > > > > So I found new convergence test here which sets > > > > > > > > CONVERGED_RTOL_NORMAL: > > > > > > > > http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/KSP/KSPLSQRDefaultConverged.html > > > > > > > > > > > > > > > > The question is, how to interpret this new test and make it > > > > > > > > works properly for me? > > > > > > > > Thanks in advance. > > > > > > > > > > > > > > > > Regards, > > > > > > > > Alexander > > -------------- next part -------------- tree 80c3b1f3210d parent 98f24f0f10f1 author Barry Smith 1316634856 18000 committer Barry Smith 1316634856 18000 revision 20311 branch default fix for using KSPSetConvergenceTest() with context from KSPDefaultConvergedCreate() in Fortran diff --git a/src/ksp/ksp/interface/ftn-custom/zitfuncf.c b/src/ksp/ksp/interface/ftn-custom/zitfuncf.c --- a/src/ksp/ksp/interface/ftn-custom/zitfuncf.c +++ b/src/ksp/ksp/interface/ftn-custom/zitfuncf.c @@ -8,6 +8,7 @@ #define kspdefaultconverged_ KSPDEFAULTCONVERGED #define kspdefaultconvergedcreate_ KSPDEFAULTCONVERGEDCREATE +#define kspdefaultconvergeddestroy_ KSPDEFAULTCONVERGEDDESTROY #define kspskipconverged_ KSPSKIPCONVERGED #define kspgmresmonitorkrylov_ KSPGMRESMONITORKRYLOV #define kspmonitordefault_ KSPMONITORDEFAULT @@ -24,6 +25,7 @@ #define kspgetresidualhistory_ kspgetresidualhistory #define kspdefaultconverged_ kspdefaultconverged #define kspdefaultconvergedcreate_ kspdefaultconvergedcreate +#define kspdefaultconvergeddestroy_ kspdefaultconvergeddestroy #define kspskipconverged_ kspskipconverged #define kspmonitorsingularvalue_ kspmonitorsingularvalue #define kspgmresmonitorkrylov_ kspgmresmonitorkrylov @@ -163,7 +165,7 @@ } void PETSC_STDCALL kspsetconvergencetest_(KSP *ksp, - void (PETSC_STDCALL *converge)(KSP*,PetscInt*,PetscReal*,KSPConvergedReason*,void*,PetscErrorCode*),void *cctx, + void (PETSC_STDCALL *converge)(KSP*,PetscInt*,PetscReal*,KSPConvergedReason*,void*,PetscErrorCode*),void **cctx, void (PETSC_STDCALL *destroy)(void*,PetscErrorCode*),PetscErrorCode *ierr) { CHKFORTRANNULLOBJECT(cctx); @@ -171,7 +173,7 @@ PetscObjectAllocateFortranPointers(*ksp,6); if ((PetscVoidFunction)converge == (PetscVoidFunction)kspdefaultconverged_) { - *ierr = KSPSetConvergenceTest(*ksp,KSPDefaultConverged,cctx,KSPDefaultConvergedDestroy); + *ierr = KSPSetConvergenceTest(*ksp,KSPDefaultConverged,*cctx,KSPDefaultConvergedDestroy); } else if ((PetscVoidFunction)converge == (PetscVoidFunction)kspskipconverged_) { *ierr = KSPSetConvergenceTest(*ksp,KSPSkipConverged,0,0); } else { @@ -191,6 +193,11 @@ *ierr = KSPDefaultConvergedCreate((void**)ctx); } +void PETSC_STDCALL kspdefaultconvergeddestroy_(PetscFortranAddr *ctx,PetscErrorCode *ierr) +{ + *ierr = KSPDefaultConvergedDestroy(*(void**)ctx); +} + void PETSC_STDCALL kspgetresidualhistory_(KSP *ksp,PetscInt *na,PetscErrorCode *ierr) { *ierr = KSPGetResidualHistory(*ksp,PETSC_NULL,na); From bsmith at mcs.anl.gov Thu Sep 22 08:26:56 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Thu, 22 Sep 2011 08:26:56 -0500 Subject: [petsc-users] [petsc-3.2] LSQR new convergence test In-Reply-To: References: <4E785F93.8020106@gfz-potsdam.de> <5C5AE85F-F5D5-4BE2-95F6-DE1D09A150DF@mcs.anl.gov> <4E7994AE.1060200@gfz-potsdam.de> <4E7A000F.4030009@gfz-potsdam.de> <4E7A0A21.70606@gfz-potsdam.de> <21319D2F-148A-4F80-AD37-E05C0383C91F@mcs.anl.gov> <4E7AF97E.7040901@gfz-potsdam.de> Message-ID: On Sep 22, 2011, at 7:54 AM, Satish Balay wrote: > Loks like Barry sent you the wrong file. zitfuncf90.c belongs to > src/ksp/ksp/interface/f90-custom [and is unchanged] Yes, sorry that was my fault. Barry > > Try the attached patch > > cd petsc-3.2-p2 > patch -Np1 < ksp-conv.patch > > Satish > > On Thu, 22 Sep 2011, Alexander Grayver wrote: > >> Barry, >> >> Unfortunately, the problem is still there. Even though I put zitfuncf90.c into >> src/ksp/ksp/interface/ftn-custom, modified makefile there and made >> configure/make from scratch. >> When are you going to release new patch? >> >> Regards, >> Alexander >> >> On 21.09.2011 21:58, Barry Smith wrote: >>> Alexander, >>> >>> Yes, a problem with trying to handle C pointers on the Fortran side. It >>> will be fixed in the next patch release. You can drop the attached file into >>> src/ksp/ksp/interface/ftn-custom run make in that directory and then relink >>> your program and it should work. >>> >>> Barry >>> [see attached file: zitfuncf90.c] >>> On Sep 21, 2011, at 11:00 AM, Alexander Grayver wrote: >>> >>>> Barry, >>>> >>>> I think the error is related to this >>>> >>>> PetscFortranAddr cctx >>>> >>>> definition. Speaking in terms of C++ notation, it seems that PETSc >>>> takes&cctx as KSPDefaultConvergedCtx, however real KSPDefaultConvergedCtx >>>> is *cctx. >>>> Not sure if I'm clear now, but it's easy to see if you trace code like >>>> that: >>>> >>>> PetscFortranAddr cctx >>>> KSP ksp >>>> >>>> call KSPCreate(MPI_COMM_WORLD,ksp,ierr) >>>> call KSPSetType(ksp,KSPLSQR,ierr) >>>> call KSPDefaultConvergedCreate(cctx,ierr) >>>> call >>>> KSPSetConvergenceTest(ksp,KSPDefaultConverged,cctx,PETSC_NULL_FUNCTION,ierr) >>>> call KSPDestroy(ksp,ierr) >>>> >>>> I guess the error should be reproducible from the code above. >>>> >>>> Regards, >>>> Alexander >>>> >>>> On 21.09.2011 17:17, Alexander Grayver wrote: >>>>> Thanks Barry, I added your code and solver works well now, but when I >>>>> call KSPDestroy afterward it crashes: >>>>> >>>>> [8]PETSC ERROR: --------------------- Error Message >>>>> ------------------------------------ >>>>> [8]PETSC ERROR: Invalid argument! >>>>> [8]PETSC ERROR: Wrong type of object: Parameter # 1! >>>>> [8]PETSC ERROR: >>>>> ------------------------------------------------------------------------ >>>>> [8]PETSC ERROR: Petsc Release Version 3.2.0, Patch 2, Fri Sep 16 >>>>> 10:10:45 CDT 2011 >>>>> [8]PETSC ERROR: See docs/changes/index.html for recent updates. >>>>> [8]PETSC ERROR: See docs/faq.html for hints about trouble shooting. >>>>> [8]PETSC ERROR: See docs/index.html for manual pages. >>>>> [8]PETSC ERROR: >>>>> ------------------------------------------------------------------------ >>>>> [8]PETSC ERROR: /home/test on a openmpi-i named node226 by user Wed Sep >>>>> 21 17:01:23 2011 >>>>> [8]PETSC ERROR: Libraries linked from >>>>> /home/lib/petsc-3.2-p2/openmpi-intel-complex-debug-f/lib >>>>> [8]PETSC ERROR: Configure run at Tue Sep 20 09:28:29 2011 >>>>> [8]PETSC ERROR: Configure options >>>>> --with-petsc-arch=openmpi-intel-complex-debug-f >>>>> --with-fortran-interfaces=1 --with-mpi-dir=/opt/mpi/intel/openmpi-1.4.2 >>>>> --with-scalar-type=complex >>>>> --with-blas-lapack-dir=/opt/intel/Compiler/11.1/072/mkl/lib/em64t >>>>> --with-precision=double --with-x=0 >>>>> [8]PETSC ERROR: >>>>> ------------------------------------------------------------------------ >>>>> [8]PETSC ERROR: VecDestroy() line 566 in >>>>> /home/lib/petsc-3.2-p2/src/vec/vec/interface/vector.c >>>>> [8]PETSC ERROR: KSPDefaultConvergedDestroy() line 669 in >>>>> /home/lib/petsc-3.2-p2/src/ksp/ksp/interface/iterativ.c >>>>> [8]PETSC ERROR: KSPDestroy() line 758 in >>>>> /home/lib/petsc-3.2-p2/src/ksp/ksp/interface/itfunc.c >>>>> >>>>> Is it my fault? >>>>> >>>>> Regards, >>>>> Alexander >>>>> >>>>> On 21.09.2011 15:36, Barry Smith wrote: >>>>>> On Sep 21, 2011, at 2:39 AM, Alexander Grayver wrote: >>>>>> >>>>>>> Hong, Barry, >>>>>>> >>>>>>> Thanks! Sorry for not to be clear, but Barry right, my question was >>>>>>> how to get back to DefaultConvergenceTest in case of LSQR solver. >>>>>>> >>>>>>> Barry, yes, both versions give the same two norms. >>>>>>> >>>>>>> Can you clarify please how to implement that in Fortran: >>>>>>> void *cctx; >>>>>>> KSPDefaultConvergedCreate(&cctx); >>>>>>> KSPSetConvergenceTest(ksp,KSPDefaultConverged,cctx); >>>>>> It should be almost the same >>>>>> >>>>>> PetscFortranAddr cctx >>>>>> >>>>>> call KSPDefaultConvergedCreate(cctx,ierr) >>>>>> call >>>>>> KSPSetConvergenceTest(ksp,KSPDefaultConverged,cctx,PETSC_NULL_FUNCTION,ierr) >>>>>> >>>>>> Barry >>>>>> >>>>>> >>>>>>> I'm a little bit confused about cctx parameter. >>>>>>> >>>>>>> Regards, >>>>>>> Alexander >>>>>>> >>>>>>> >>>>>>> On 21.09.2011 01:10, Barry Smith wrote: >>>>>>>> It has its own monitor that provides additional information >>>>>>>> -ksp_monitor_lsqr >>>>>>>> >>>>>>>> You can also remove the new convergence test and get back the old >>>>>>>> one with code like >>>>>>>> >>>>>>>> void *cctx; >>>>>>>> KSPDefaultConvergedCreate(&cctx); >>>>>>>> KSPSetConvergenceTest(ksp,KSPDefaultConverged,cctx); >>>>>>>> >>>>>>>> after the KSPType is set to LSQR. So if you are happy with the old >>>>>>>> test. >>>>>>>> >>>>>>>> >>>>>>>> Do both versions give the same first two norms? >>>>>>>> >>>>>>>>> 0 KSP Residual norm 9.386670021557e-17 >>>>>>>>> 1 KSP Residual norm 8.258308650175e-17 >>>>>>>> Barry >>>>>>>> >>>>>>>> >>>>>>>> On Sep 20, 2011, at 4:40 AM, Alexander Grayver wrote: >>>>>>>> >>>>>>>>> Hello, >>>>>>>>> >>>>>>>>> In comparison with petsc-3.1-p7 in the latest petsc LSQR solver >>>>>>>>> behaves differently. >>>>>>>>> >>>>>>>>> In petsc31 I had: >>>>>>>>> >>>>>>>>> 0 KSP Residual norm 9.386670021557e-17 >>>>>>>>> ... >>>>>>>>> 95 KSP Residual norm 9.341367317075e-18 >>>>>>>>> Linear solve converged due to CONVERGED_RTOL iterations 95 >>>>>>>>> KSP Object: >>>>>>>>> type: lsqr >>>>>>>>> maximum iterations=200, initial guess is zero >>>>>>>>> tolerances: relative=0.1, absolute=1e-50, divergence=10000 >>>>>>>>> left preconditioning >>>>>>>>> using PRECONDITIONED norm type for convergence test >>>>>>>>> PC Object: >>>>>>>>> type: none >>>>>>>>> linear system matrix = precond matrix: >>>>>>>>> Matrix Object: >>>>>>>>> type=shell, rows=19584, cols=19584 >>>>>>>>> >>>>>>>>> relative residual norm = 0.9951737192872134E-01 >>>>>>>>> >>>>>>>>> Now, with petsc32 I have: >>>>>>>>> >>>>>>>>> 0 KSP Residual norm 9.386670021557e-17 >>>>>>>>> 1 KSP Residual norm 8.258308650175e-17 >>>>>>>>> Linear solve converged due to CONVERGED_RTOL_NORMAL iterations 1 >>>>>>>>> KSP Object: 12 MPI processes >>>>>>>>> type: lsqr >>>>>>>>> maximum iterations=200, initial guess is zero >>>>>>>>> tolerances: relative=0.1, absolute=1e-50, divergence=10000 >>>>>>>>> left preconditioning >>>>>>>>> using UNPRECONDITIONED norm type for convergence test >>>>>>>>> PC Object: 12 MPI processes >>>>>>>>> type: none >>>>>>>>> linear system matrix = precond matrix: >>>>>>>>> Matrix Object: 12 MPI processes >>>>>>>>> type: shell >>>>>>>>> rows=19584, cols=19584 >>>>>>>>> relative residual norm = 0.8797910900468064E+00 >>>>>>>>> >>>>>>>>> So I found new convergence test here which sets >>>>>>>>> CONVERGED_RTOL_NORMAL: >>>>>>>>> http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/KSP/KSPLSQRDefaultConverged.html >>>>>>>>> >>>>>>>>> The question is, how to interpret this new test and make it >>>>>>>>> works properly for me? >>>>>>>>> Thanks in advance. >>>>>>>>> >>>>>>>>> Regards, >>>>>>>>> Alexander >> >> > From agrayver at gfz-potsdam.de Thu Sep 22 08:56:52 2011 From: agrayver at gfz-potsdam.de (Alexander Grayver) Date: Thu, 22 Sep 2011 15:56:52 +0200 Subject: [petsc-users] [petsc-3.2] LSQR new convergence test In-Reply-To: References: <4E785F93.8020106@gfz-potsdam.de> <5C5AE85F-F5D5-4BE2-95F6-DE1D09A150DF@mcs.anl.gov> <4E7994AE.1060200@gfz-potsdam.de> <4E7A000F.4030009@gfz-potsdam.de> <4E7A0A21.70606@gfz-potsdam.de> <21319D2F-148A-4F80-AD37-E05C0383C91F@mcs.anl.gov> <4E7AF97E.7040901@gfz-potsdam.de> Message-ID: <4E7B3EA4.6000803@gfz-potsdam.de> Thank you guys, It runs well now. Finally, my program works with petsc-3.2, at least under linux. Regards, Alexander On 22.09.2011 14:54, Satish Balay wrote: > Loks like Barry sent you the wrong file. zitfuncf90.c belongs to > src/ksp/ksp/interface/f90-custom [and is unchanged] > > Try the attached patch > > cd petsc-3.2-p2 > patch -Np1< ksp-conv.patch > > Satish > > On Thu, 22 Sep 2011, Alexander Grayver wrote: > >> Barry, >> >> Unfortunately, the problem is still there. Even though I put zitfuncf90.c into >> src/ksp/ksp/interface/ftn-custom, modified makefile there and made >> configure/make from scratch. >> When are you going to release new patch? >> >> Regards, >> Alexander >> >> On 21.09.2011 21:58, Barry Smith wrote: >>> Alexander, >>> >>> Yes, a problem with trying to handle C pointers on the Fortran side. It >>> will be fixed in the next patch release. You can drop the attached file into >>> src/ksp/ksp/interface/ftn-custom run make in that directory and then relink >>> your program and it should work. >>> >>> Barry >>> [see attached file: zitfuncf90.c] >>> On Sep 21, 2011, at 11:00 AM, Alexander Grayver wrote: >>> >>>> Barry, >>>> >>>> I think the error is related to this >>>> >>>> PetscFortranAddr cctx >>>> >>>> definition. Speaking in terms of C++ notation, it seems that PETSc >>>> takes&cctx as KSPDefaultConvergedCtx, however real KSPDefaultConvergedCtx >>>> is *cctx. >>>> Not sure if I'm clear now, but it's easy to see if you trace code like >>>> that: >>>> >>>> PetscFortranAddr cctx >>>> KSP ksp >>>> >>>> call KSPCreate(MPI_COMM_WORLD,ksp,ierr) >>>> call KSPSetType(ksp,KSPLSQR,ierr) >>>> call KSPDefaultConvergedCreate(cctx,ierr) >>>> call >>>> KSPSetConvergenceTest(ksp,KSPDefaultConverged,cctx,PETSC_NULL_FUNCTION,ierr) >>>> call KSPDestroy(ksp,ierr) >>>> >>>> I guess the error should be reproducible from the code above. >>>> >>>> Regards, >>>> Alexander >>>> >>>> On 21.09.2011 17:17, Alexander Grayver wrote: >>>>> Thanks Barry, I added your code and solver works well now, but when I >>>>> call KSPDestroy afterward it crashes: >>>>> >>>>> [8]PETSC ERROR: --------------------- Error Message >>>>> ------------------------------------ >>>>> [8]PETSC ERROR: Invalid argument! >>>>> [8]PETSC ERROR: Wrong type of object: Parameter # 1! >>>>> [8]PETSC ERROR: >>>>> ------------------------------------------------------------------------ >>>>> [8]PETSC ERROR: Petsc Release Version 3.2.0, Patch 2, Fri Sep 16 >>>>> 10:10:45 CDT 2011 >>>>> [8]PETSC ERROR: See docs/changes/index.html for recent updates. >>>>> [8]PETSC ERROR: See docs/faq.html for hints about trouble shooting. >>>>> [8]PETSC ERROR: See docs/index.html for manual pages. >>>>> [8]PETSC ERROR: >>>>> ------------------------------------------------------------------------ >>>>> [8]PETSC ERROR: /home/test on a openmpi-i named node226 by user Wed Sep >>>>> 21 17:01:23 2011 >>>>> [8]PETSC ERROR: Libraries linked from >>>>> /home/lib/petsc-3.2-p2/openmpi-intel-complex-debug-f/lib >>>>> [8]PETSC ERROR: Configure run at Tue Sep 20 09:28:29 2011 >>>>> [8]PETSC ERROR: Configure options >>>>> --with-petsc-arch=openmpi-intel-complex-debug-f >>>>> --with-fortran-interfaces=1 --with-mpi-dir=/opt/mpi/intel/openmpi-1.4.2 >>>>> --with-scalar-type=complex >>>>> --with-blas-lapack-dir=/opt/intel/Compiler/11.1/072/mkl/lib/em64t >>>>> --with-precision=double --with-x=0 >>>>> [8]PETSC ERROR: >>>>> ------------------------------------------------------------------------ >>>>> [8]PETSC ERROR: VecDestroy() line 566 in >>>>> /home/lib/petsc-3.2-p2/src/vec/vec/interface/vector.c >>>>> [8]PETSC ERROR: KSPDefaultConvergedDestroy() line 669 in >>>>> /home/lib/petsc-3.2-p2/src/ksp/ksp/interface/iterativ.c >>>>> [8]PETSC ERROR: KSPDestroy() line 758 in >>>>> /home/lib/petsc-3.2-p2/src/ksp/ksp/interface/itfunc.c >>>>> >>>>> Is it my fault? >>>>> >>>>> Regards, >>>>> Alexander >>>>> >>>>> On 21.09.2011 15:36, Barry Smith wrote: >>>>>> On Sep 21, 2011, at 2:39 AM, Alexander Grayver wrote: >>>>>> >>>>>>> Hong, Barry, >>>>>>> >>>>>>> Thanks! Sorry for not to be clear, but Barry right, my question was >>>>>>> how to get back to DefaultConvergenceTest in case of LSQR solver. >>>>>>> >>>>>>> Barry, yes, both versions give the same two norms. >>>>>>> >>>>>>> Can you clarify please how to implement that in Fortran: >>>>>>> void *cctx; >>>>>>> KSPDefaultConvergedCreate(&cctx); >>>>>>> KSPSetConvergenceTest(ksp,KSPDefaultConverged,cctx); >>>>>> It should be almost the same >>>>>> >>>>>> PetscFortranAddr cctx >>>>>> >>>>>> call KSPDefaultConvergedCreate(cctx,ierr) >>>>>> call >>>>>> KSPSetConvergenceTest(ksp,KSPDefaultConverged,cctx,PETSC_NULL_FUNCTION,ierr) >>>>>> >>>>>> Barry >>>>>> >>>>>> >>>>>>> I'm a little bit confused about cctx parameter. >>>>>>> >>>>>>> Regards, >>>>>>> Alexander >>>>>>> >>>>>>> >>>>>>> On 21.09.2011 01:10, Barry Smith wrote: >>>>>>>> It has its own monitor that provides additional information >>>>>>>> -ksp_monitor_lsqr >>>>>>>> >>>>>>>> You can also remove the new convergence test and get back the old >>>>>>>> one with code like >>>>>>>> >>>>>>>> void *cctx; >>>>>>>> KSPDefaultConvergedCreate(&cctx); >>>>>>>> KSPSetConvergenceTest(ksp,KSPDefaultConverged,cctx); >>>>>>>> >>>>>>>> after the KSPType is set to LSQR. So if you are happy with the old >>>>>>>> test. >>>>>>>> >>>>>>>> >>>>>>>> Do both versions give the same first two norms? >>>>>>>> >>>>>>>>> 0 KSP Residual norm 9.386670021557e-17 >>>>>>>>> 1 KSP Residual norm 8.258308650175e-17 >>>>>>>> Barry >>>>>>>> >>>>>>>> >>>>>>>> On Sep 20, 2011, at 4:40 AM, Alexander Grayver wrote: >>>>>>>> >>>>>>>>> Hello, >>>>>>>>> >>>>>>>>> In comparison with petsc-3.1-p7 in the latest petsc LSQR solver >>>>>>>>> behaves differently. >>>>>>>>> >>>>>>>>> In petsc31 I had: >>>>>>>>> >>>>>>>>> 0 KSP Residual norm 9.386670021557e-17 >>>>>>>>> ... >>>>>>>>> 95 KSP Residual norm 9.341367317075e-18 >>>>>>>>> Linear solve converged due to CONVERGED_RTOL iterations 95 >>>>>>>>> KSP Object: >>>>>>>>> type: lsqr >>>>>>>>> maximum iterations=200, initial guess is zero >>>>>>>>> tolerances: relative=0.1, absolute=1e-50, divergence=10000 >>>>>>>>> left preconditioning >>>>>>>>> using PRECONDITIONED norm type for convergence test >>>>>>>>> PC Object: >>>>>>>>> type: none >>>>>>>>> linear system matrix = precond matrix: >>>>>>>>> Matrix Object: >>>>>>>>> type=shell, rows=19584, cols=19584 >>>>>>>>> >>>>>>>>> relative residual norm = 0.9951737192872134E-01 >>>>>>>>> >>>>>>>>> Now, with petsc32 I have: >>>>>>>>> >>>>>>>>> 0 KSP Residual norm 9.386670021557e-17 >>>>>>>>> 1 KSP Residual norm 8.258308650175e-17 >>>>>>>>> Linear solve converged due to CONVERGED_RTOL_NORMAL iterations 1 >>>>>>>>> KSP Object: 12 MPI processes >>>>>>>>> type: lsqr >>>>>>>>> maximum iterations=200, initial guess is zero >>>>>>>>> tolerances: relative=0.1, absolute=1e-50, divergence=10000 >>>>>>>>> left preconditioning >>>>>>>>> using UNPRECONDITIONED norm type for convergence test >>>>>>>>> PC Object: 12 MPI processes >>>>>>>>> type: none >>>>>>>>> linear system matrix = precond matrix: >>>>>>>>> Matrix Object: 12 MPI processes >>>>>>>>> type: shell >>>>>>>>> rows=19584, cols=19584 >>>>>>>>> relative residual norm = 0.8797910900468064E+00 >>>>>>>>> >>>>>>>>> So I found new convergence test here which sets >>>>>>>>> CONVERGED_RTOL_NORMAL: >>>>>>>>> http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/KSP/KSPLSQRDefaultConverged.html >>>>>>>>> >>>>>>>>> The question is, how to interpret this new test and make it >>>>>>>>> works properly for me? >>>>>>>>> Thanks in advance. >>>>>>>>> >>>>>>>>> Regards, >>>>>>>>> Alexander >> From milan.v.mitrovic at gmail.com Thu Sep 22 09:54:37 2011 From: milan.v.mitrovic at gmail.com (Milan Mitrovic) Date: Thu, 22 Sep 2011 16:54:37 +0200 Subject: [petsc-users] snessetjacobian question Message-ID: I have a problem with the snes solver... I initialize it with: call SNESCreate(PETSC_COMM_WORLD, snes, info) call SNESSetType(snes, SNESLS, info) call SNESGetKSP(snes, ksp, info) call KSPGetPC(ksp, pc, info) call PCSetType(pc, PCNONE, info) call KSPSetFromOptions(ksp, info) call SNESSetFunction(snes, r, theta_step, ctx, info) call SNESSetJacobian(snes, J, J, jacobian, PETSC_NULL, info) And then in the jacobian function I create one of the matrices: call MatCreateMPIAIJ(PETSC_COMM_WORLD, Np, Np, Ng, Ng, & nmax+1, PETSC_NULL_INTEGER, nmax+1, PETSC_NULL_INTEGER, Jac, info) call MatSetLocalToGlobalMapping(Jac,lgm,info) then I set the values and: call MatAssemblyBegin(Jac, MAT_FINAL_ASSEMBLY, info) call MatAssemblyEnd(Jac, MAT_FINAL_ASSEMBLY, info) and finally I do: P = Jac This works fine... the solver works, I get my results, but in the end when I try to call SNESDestroy(snes, info) I get: [0]PETSC ERROR: --------------------- Error Message ------------------------------------ [0]PETSC ERROR: Invalid argument! [0]PETSC ERROR: Wrong type of object: Parameter # 1! [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: Petsc Release Version 3.1.0, Patch 8, Thu Mar 17 13:37:48 CDT 2011 [0]PETSC ERROR: See docs/changes/index.html for recent updates. [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. [0]PETSC ERROR: See docs/index.html for manual pages. [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: ./ppm_pf on a linux-gnu named cbl1 by milanm Thu Sep 22 16:41:35 2011 [0]PETSC ERROR: Libraries linked from /home/milanm/ppm/petsc-3.1-p8/linux-gnu-c-debug/lib [0]PETSC ERROR: Configure run at Mon Sep 12 13:38:51 2011 [0]PETSC ERROR: Configure options [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: MatDestroy() line 870 in src/mat/interface/matrix.c [0]PETSC ERROR: PCDestroy() line 88 in src/ksp/pc/interface/precon.c [0]PETSC ERROR: KSPDestroy() line 695 in src/ksp/ksp/interface/itfunc.c [0]PETSC ERROR: SNESDestroy() line 1413 in src/snes/interface/snes.c What is going on here? Is there something I'm doing wrong? From jedbrown at mcs.anl.gov Thu Sep 22 10:01:59 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Thu, 22 Sep 2011 17:01:59 +0200 Subject: [petsc-users] snessetjacobian question In-Reply-To: References: Message-ID: On Thu, Sep 22, 2011 at 16:54, Milan Mitrovic wrote: > call SNESCreate(PETSC_COMM_WORLD, snes, info) > call SNESSetType(snes, SNESLS, info) > call SNESGetKSP(snes, ksp, info) > call KSPGetPC(ksp, pc, info) > call PCSetType(pc, PCNONE, info) > call KSPSetFromOptions(ksp, info) > call SNESSetFunction(snes, r, theta_step, ctx, info) > call SNESSetJacobian(snes, J, J, jacobian, PETSC_NULL, info) > Where did you get this J from? > > And then in the jacobian function I create one of the matrices: > > > call MatCreateMPIAIJ(PETSC_COMM_WORLD, Np, Np, Ng, Ng, & > nmax+1, PETSC_NULL_INTEGER, nmax+1, PETSC_NULL_INTEGER, Jac, info) > You should create the matrix first and just do assembly in this function. -------------- next part -------------- An HTML attachment was scrubbed... URL: From milan.v.mitrovic at gmail.com Thu Sep 22 10:05:20 2011 From: milan.v.mitrovic at gmail.com (Milan Mitrovic) Date: Thu, 22 Sep 2011 17:05:20 +0200 Subject: [petsc-users] snessetjacobian question In-Reply-To: References: Message-ID: The J is not initialized when I call SNESSetJacobian... I will try that, but I dont know nmax at that point... can I set it later? From jedbrown at mcs.anl.gov Thu Sep 22 10:08:58 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Thu, 22 Sep 2011 17:08:58 +0200 Subject: [petsc-users] snessetjacobian question In-Reply-To: References: Message-ID: On Thu, Sep 22, 2011 at 17:05, Milan Mitrovic wrote: > The J is not initialized when I call SNESSetJacobian... > > I will try that, but I dont know nmax at that point... can I set it later? > Yes, create the matrix and set the sizes outside, but skip preallocation. Then, inside the jacobian routine (only on the first time unless it changes), call MatMPIAIJSetPreallocation() and MatSeqAIJSetPreallocation(). You can also arrange to do this in a setup stage before calling SNESSolve(), but there is no problem calling it from your Jacobian evaluation. -------------- next part -------------- An HTML attachment was scrubbed... URL: From milan.v.mitrovic at gmail.com Thu Sep 22 10:14:51 2011 From: milan.v.mitrovic at gmail.com (Milan Mitrovic) Date: Thu, 22 Sep 2011 17:14:51 +0200 Subject: [petsc-users] snessetjacobian question In-Reply-To: References: Message-ID: I use snes for an implicit time stepping scheme... I call it in a loop for every time step... the jacobian is recomputed at every step, and the preallocation can change from time to time... what is the proper way to do it then? destroy the matrix, then recreate it? what should I do with the preconditioner matrix? (petsc complains if I dont do P = J) From jedbrown at mcs.anl.gov Thu Sep 22 10:17:52 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Thu, 22 Sep 2011 17:17:52 +0200 Subject: [petsc-users] snessetjacobian question In-Reply-To: References: Message-ID: On Thu, Sep 22, 2011 at 17:14, Milan Mitrovic wrote: > I use snes for an implicit time stepping scheme... I call it in a loop > for every time step... > > the jacobian is recomputed at every step, and the preallocation can > change from time to time... what is the proper way to do it then? > destroy the matrix, then recreate it? > Don't destroy the matrix, just call the preallocation routines when the nonzero pattern changes. You can actually destroy the matrices from C, but I don't think there is a pure Fortran way to do it (it needs another pointer indirection) and you should never need to anyway. > what should I do with the > preconditioner matrix? (petsc complains if I dont do P = J) > What do you mean "complains"? You can certainly use a different preconditioning matrix. -------------- next part -------------- An HTML attachment was scrubbed... URL: From milan.v.mitrovic at gmail.com Thu Sep 22 10:21:55 2011 From: milan.v.mitrovic at gmail.com (Milan Mitrovic) Date: Thu, 22 Sep 2011 17:21:55 +0200 Subject: [petsc-users] snessetjacobian question In-Reply-To: References: Message-ID: > Don't destroy the matrix, just call the preallocation routines when the > nonzero pattern changes. > You can actually destroy the matrices from C, but I don't think there is a > pure Fortran way to do it (it needs another pointer indirection) and you > should never need to anyway. How should I create a matrix without supplying the preallocation? just set everything to zero and then call the prealloc routines later? > What do you mean "complains"? You can certainly use a different > preconditioning matrix. I dont want to use a different matrix... I want to use the same... I just dont want to get the error that I pasted in the first message... and I have no idea why I get it in the first place... From milan.v.mitrovic at gmail.com Thu Sep 22 10:27:22 2011 From: milan.v.mitrovic at gmail.com (Milan Mitrovic) Date: Thu, 22 Sep 2011 17:27:22 +0200 Subject: [petsc-users] snessetjacobian question In-Reply-To: References: Message-ID: also... can I change the size of the matrix as well? This can also change during the simulation? should I destroy and create a new matrix or just resize it, and how? From jedbrown at mcs.anl.gov Thu Sep 22 10:36:31 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Thu, 22 Sep 2011 17:36:31 +0200 Subject: [petsc-users] snessetjacobian question In-Reply-To: References: Message-ID: On Thu, Sep 22, 2011 at 17:21, Milan Mitrovic wrote: > > Don't destroy the matrix, just call the preallocation routines when the > > nonzero pattern changes. > > You can actually destroy the matrices from C, but I don't think there is > a > > pure Fortran way to do it (it needs another pointer indirection) and you > > should never need to anyway. > > How should I create a matrix without supplying the preallocation? just > set everything to zero and then call the prealloc routines later? > Just create the matrix and set the sizes, but don't preallocate. The wrapper you are calling is just a convenience function: PetscErrorCode MatCreateMPIAIJ(MPI_Comm comm,PetscInt m,PetscInt n,PetscInt M,PetscInt N,PetscInt d_nz,const PetscInt d_nnz[],PetscInt o_nz,const PetscInt o_nnz[],Mat *A) { PetscErrorCode ierr; PetscMPIInt size; PetscFunctionBegin; ierr = MatCreate(comm,A);CHKERRQ(ierr); ierr = MatSetSizes(*A,m,n,M,N);CHKERRQ(ierr); ierr = MPI_Comm_size(comm,&size);CHKERRQ(ierr); if (size > 1) { ierr = MatSetType(*A,MATMPIAIJ);CHKERRQ(ierr); ierr = MatMPIAIJSetPreallocation(*A,d_nz,d_nnz,o_nz,o_nnz);CHKERRQ(ierr); } else { ierr = MatSetType(*A,MATSEQAIJ);CHKERRQ(ierr); ierr = MatSeqAIJSetPreallocation(*A,d_nz,d_nnz);CHKERRQ(ierr); } PetscFunctionReturn(0); } > > > What do you mean "complains"? You can certainly use a different > > preconditioning matrix. > > I dont want to use a different matrix... I want to use the same... I > just dont want to get the error that I pasted in the first message... > and I have no idea why I get it in the first place... > A hanging reference was created so the same object was destroyed multiple times. I'm kinda surprised there wasn't a check earlier to verify that a valid matrix was present. In any case, you shouldn't create a new matrix inside the Jacobian evaluation function, at least not when calling from Fortran. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jedbrown at mcs.anl.gov Thu Sep 22 10:40:49 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Thu, 22 Sep 2011 17:40:49 +0200 Subject: [petsc-users] snessetjacobian question In-Reply-To: References: Message-ID: On Thu, Sep 22, 2011 at 17:27, Milan Mitrovic wrote: > also... can I change the size of the matrix as well? This can also > change during the simulation? should I destroy and create a new matrix > or just resize it, and how? > You cannot change the size inside of SNES, but you can change it between time steps. For that, you should use petsc-3.2 or -dev, call SNESReset() when the size needs to change, create the new vectors and matrices of the new size, and set them with SNESSetFunction() and SNESSetJacobian(). Bear in mind that this is new functionality and is not as well tested. An alternative is to destroy the old SNES and create a new one. -------------- next part -------------- An HTML attachment was scrubbed... URL: From milanm at student.ethz.ch Thu Sep 22 15:10:21 2011 From: milanm at student.ethz.ch (Milan Mitrovic) Date: Thu, 22 Sep 2011 22:10:21 +0200 Subject: [petsc-users] snessetjacobian question In-Reply-To: References: Message-ID: ok... one more error... concerning this line call MatSetValuesLocal(Jac,1,(/i-1/),Particles%nvlist(i)+1,ind(:),val(:),INSERT_VALUES,info) when I try to deallocate val and ind I get a glibc error saying memory is corrupt and that it is probably a double dealloc... testing val and ind with isalloc gives true right before deallocation... commenting out the above line of code, and the call to snes solve, removes the error... any ideas? From goller at intales.com Fri Sep 23 04:14:14 2011 From: goller at intales.com (Barbara Goller) Date: Fri, 23 Sep 2011 11:14:14 +0200 Subject: [petsc-users] Computing determinant using MUMPS Message-ID: <4E7C4DE6.2050000@intales.com> Dear all! I am using version 3.2 of PETSC which uses MUMPS 4.10. When solving a system of linear equations with the option -mat_mumps_icntl_33 1 (= compute determinant) it is output: Option left: name:-mat_mumps_icntl_33 value: 1. Is this option not supported by PETSC? Thanks for your help, Barbara From hzhang at mcs.anl.gov Fri Sep 23 08:35:13 2011 From: hzhang at mcs.anl.gov (Hong Zhang) Date: Fri, 23 Sep 2011 08:35:13 -0500 Subject: [petsc-users] Computing determinant using MUMPS In-Reply-To: <4E7C4DE6.2050000@intales.com> References: <4E7C4DE6.2050000@intales.com> Message-ID: Barbara : > > I am using version 3.2 of PETSC which uses MUMPS 4.10. When solving a system > of linear equations with the option -mat_mumps_icntl_33 1 (= compute > determinant) it is output: Option left: name:-mat_mumps_icntl_33 value: 1. > Is this option not supported by PETSC? Sorry, icntl_33 is not in the petsc-mumps interface. I'll patch petsc-3.2, test it and get back to you soon. Hong From jwiens at sfu.ca Fri Sep 23 00:18:58 2011 From: jwiens at sfu.ca (Jeffrey Wiens) Date: Thu, 22 Sep 2011 22:18:58 -0700 (PDT) Subject: [petsc-users] Petsc and HDF5: example src/dm/da/examples/tutorials/ex9 doesn't work Message-ID: <2016764060.639841.1316755138504.JavaMail.root@jaguar10.sfu.ca> Hello, I am a new Petsc user and am having problems with saving HDF5 files on Petsc 3.1. I am running the following commands: PetscViewer viewer; ierr = PetscViewerHDF5Open(PETSC_COMM_WORLD,file_name,FILE_MODE_WRITE,&viewer); AssertWithException(ierr); VecView(x,viewer); PetscViewerDestroy(viewer); When running these commands using multiple cores, Petsc only saves a portion of the vector. However, if I switch to using one core, the entire vector is saved. I can recreate the same problem using example 9 in petsc/3.1-p7/src/dm/da/examples/tutorials where the following occurs: > ~/sfw/petsc/3.1-p7/src/dm/da/examples/tutorials$ mpiexec -n 2 ./ex9 Vectors are not equal > ~/sfw/petsc/3.1-p7/src/dm/da/examples/tutorials$ mpiexec -n 1 ./ex9 Vectors are equal I have manually installed HDF5 from the source. Does my HDF5 install need a special configuration for this to work? Thanks in advance. Jeff From baagaard at usgs.gov Fri Sep 23 09:44:35 2011 From: baagaard at usgs.gov (Brad Aagaard) Date: Fri, 23 Sep 2011 07:44:35 -0700 Subject: [petsc-users] Petsc and HDF5: example src/dm/da/examples/tutorials/ex9 doesn't work In-Reply-To: <2016764060.639841.1316755138504.JavaMail.root@jaguar10.sfu.ca> References: <2016764060.639841.1316755138504.JavaMail.root@jaguar10.sfu.ca> Message-ID: <4E7C9B53.80005@usgs.gov> Jeff, The behavior you describe will happen if parallel I/O is not enabled when building HDF5. Make sure you configure with --enable-parallel. Brad On 09/22/2011 10:18 PM, Jeffrey Wiens wrote: > Hello, > > I am a new Petsc user and am having problems with saving HDF5 files on Petsc 3.1. I am running the following commands: > > PetscViewer viewer; > ierr = PetscViewerHDF5Open(PETSC_COMM_WORLD,file_name,FILE_MODE_WRITE,&viewer); AssertWithException(ierr); > VecView(x,viewer); > PetscViewerDestroy(viewer); > > When running these commands using multiple cores, Petsc only saves a portion of the vector. However, if I switch to using one core, the > entire vector is saved. I can recreate the same problem using example 9 in petsc/3.1-p7/src/dm/da/examples/tutorials where the following occurs: > >> ~/sfw/petsc/3.1-p7/src/dm/da/examples/tutorials$ mpiexec -n 2 ./ex9 > Vectors are not equal >> ~/sfw/petsc/3.1-p7/src/dm/da/examples/tutorials$ mpiexec -n 1 ./ex9 > Vectors are equal > > I have manually installed HDF5 from the source. Does my HDF5 install need a special configuration for this to work? > > Thanks in advance. > Jeff > From hzhang at mcs.anl.gov Fri Sep 23 09:45:07 2011 From: hzhang at mcs.anl.gov (Hong Zhang) Date: Fri, 23 Sep 2011 09:45:07 -0500 Subject: [petsc-users] Computing determinant using MUMPS In-Reply-To: <4E7C4DE6.2050000@intales.com> References: <4E7C4DE6.2050000@intales.com> Message-ID: Barbara : I added mumps options ICNTL(30 - 33) to petsc-3.2. With the latest petsc-3.2, you can get determinant with '-mat_mumps_icntl_33 1' the result can be found using '-ksp_view'. For example, petsc-3.2/src/ksp/ksp/examples/tutorials>./ex2 -pc_type lu -pc_factor_mat_solver_package mumps -mat_mumps_icntl_33 1 -ksp_view ... (RINFOG(12) RINFOG(13))*2^INFOG(34) (determinant): (0.900768,0)*(2^99) Let us know if you need additional support - I guess you might want MatGetMumpsRINFOG()? Hong > > I am using version 3.2 of PETSC which uses MUMPS 4.10. When solving a system > of linear equations with the option -mat_mumps_icntl_33 1 (= compute > determinant) it is output: Option left: name:-mat_mumps_icntl_33 value: 1. > Is this option not supported by PETSC? > > Thanks for your help, > Barbara > From khalid_eee at yahoo.com Fri Sep 23 13:35:20 2011 From: khalid_eee at yahoo.com (khalid ashraf) Date: Fri, 23 Sep 2011 11:35:20 -0700 (PDT) Subject: [petsc-users] scaling a parallel matrix Message-ID: <1316802920.48907.YahooMailNeo@web112619.mail.gq1.yahoo.com> Hi, I have a vector defined on a 3D DA of size ~1000X1000X30. I run the program on ~4000 processors.? After doing the calculations, print the vector in vtk format. I need to print about ~10 times at different time steps. The printing takes about 3 hours. Is there a way to reduce the time required to print parallel vector printing. I am trying to average over some blocks of grid points and represent the data with a smaller sized vector defined on a smaller 3D DA. One thing I am not clear is that how to get the same layout for the large and smaller DA, so that all the calculations are local when doing the averaging. Could you please provide any guidance ? ? ?? Thanks. Khalid -------------- next part -------------- An HTML attachment was scrubbed... URL: From jedbrown at mcs.anl.gov Fri Sep 23 13:48:32 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Fri, 23 Sep 2011 13:48:32 -0500 Subject: [petsc-users] scaling a parallel matrix In-Reply-To: <1316802920.48907.YahooMailNeo@web112619.mail.gq1.yahoo.com> References: <1316802920.48907.YahooMailNeo@web112619.mail.gq1.yahoo.com> Message-ID: On Fri, Sep 23, 2011 at 13:35, khalid ashraf wrote: > Hi, I have a vector defined on a 3D DA of size ~1000X1000X30. I run the > program on ~4000 processors. > After doing the calculations, print the vector in vtk format. I need to > print about ~10 times at different time steps. > The printing takes about 3 hours. Is there a way to reduce the time > required to print parallel vector printing. > The VTK ASCII format is a horrible parallel format. One reliable option if you like the VTK tool chain is to write a "parallel" VTK XML format with binary appended data. You can also write an HDF5 file and either provide your own reader for whatever visualization software you are using or write an XDMF file to inform the reader of what to find where. > > I am trying to average over some blocks of grid points and represent the > data with a smaller sized vector defined on a smaller 3D DA. One thing I am > not clear is that how to get the same layout for the large and smaller DA, > so that all the calculations are local when doing the averaging. Could you > please provide any guidance ? > See the lx,ly,lz parameters to DMDACreate3d(). http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-dev/docs/manualpages/DM/DMDACreate3d.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Fri Sep 23 14:40:14 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Fri, 23 Sep 2011 14:40:14 -0500 Subject: [petsc-users] scaling a parallel matrix In-Reply-To: References: <1316802920.48907.YahooMailNeo@web112619.mail.gq1.yahoo.com> Message-ID: On Sep 23, 2011, at 1:48 PM, Jed Brown wrote: > On Fri, Sep 23, 2011 at 13:35, khalid ashraf wrote: > Hi, I have a vector defined on a 3D DA of size ~1000X1000X30. I run the program on ~4000 processors. > After doing the calculations, print the vector in vtk format. I need to print about ~10 times at different time steps. > The printing takes about 3 hours. Is there a way to reduce the time required to print parallel vector printing. > > The VTK ASCII format is a horrible parallel format. One reliable option if you like the VTK tool chain is to write a "parallel" VTK XML format with binary appended data. You can also write an HDF5 file and either provide your own reader for whatever visualization software you are using or write an XDMF file to inform the reader of what to find where. > > > I am trying to average over some blocks of grid points and represent the data with a smaller sized vector defined on a smaller 3D DA. One thing I am not clear is that how to get the same layout for the large and smaller DA, so that all the calculations are local when doing the averaging. Could you please provide any guidance ? > > See the lx,ly,lz parameters to DMDACreate3d(). > > http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-dev/docs/manualpages/DM/DMDACreate3d.html Or you can generate the finer DMDA by refining the coarse grid with DMRefine() or generate the coarser mesh by DMCoarsening the finer. They will automatically have the correct layout. Note you can use DMGetInterpolation() to get an operator that can be used to restrict from the fine grid to the coarse then scale by the vector argument returned by DMGetInterpolation(). Using the MatRestrict() with that operator performs a local piecewise linear averaging. Barry From zhenglun.wei at gmail.com Fri Sep 23 16:33:09 2011 From: zhenglun.wei at gmail.com (Alan Wei) Date: Fri, 23 Sep 2011 16:33:09 -0500 Subject: [petsc-users] Problem with petsc-dev update Message-ID: Dear folks, I just update petsc-dev following PETSc can now be configured in the usual way, specified on the Installation page . To update your copy of petsc-dev - cd petsc-dev - hg pull -u - cd config/BuildSystem - hg pull -u Once updated, you will usually want to rebuild completely - cd petsc-dev - ./$PETSC_ARCH/conf/reconfigure-$PETSC_ARCH.py - make however, after I have done that, I test a example which can be run just before I did the update and failed witha error message: /home/zlwei/soft/mercurial/petsc-dev/arch-linux2-c-debug/lib/libpetsc.a(snesregi.o): In function `SNESRegisterAll': /home/zlwei/soft/mercurial/petsc-dev/src/snes/interface/snesregi.c:63: undefined reference to `SNESCreate_FAS' BTW, the configure.log and make.log are attached. thanks in advance, Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: make.log Type: application/octet-stream Size: 46279 bytes Desc: not available URL: From knepley at gmail.com Fri Sep 23 16:39:57 2011 From: knepley at gmail.com (Matthew Knepley) Date: Fri, 23 Sep 2011 14:39:57 -0700 Subject: [petsc-users] Problem with petsc-dev update In-Reply-To: References: Message-ID: On Fri, Sep 23, 2011 at 2:33 PM, Alan Wei wrote: > Dear folks, > I just update petsc-dev following > > PETSc can now be configured in the usual way, specified on the Installation > page. > To update your copy of petsc-dev > > - cd petsc-dev > - hg pull -u > - cd config/BuildSystem > - hg pull -u > > Once updated, you will usually want to rebuild completely > > - cd petsc-dev > - ./$PETSC_ARCH/conf/reconfigure-$PETSC_ARCH.py > - make > > however, after I have done that, I test a example which can be run just > before I did the update and failed witha error message: > > /home/zlwei/soft/mercurial/petsc-dev/arch-linux2-c-debug/lib/libpetsc.a(snesregi.o): > In function `SNESRegisterAll': > /home/zlwei/soft/mercurial/petsc-dev/src/snes/interface/snesregi.c:63: > undefined reference to `SNESCreate_FAS' > Something is wrong with your repo. Pull petsc-dev again and look at snesregi.c Matt > BTW, the configure.log and make.log are attached. > > thanks in advance, > Alan > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From zhenglun.wei at gmail.com Fri Sep 23 16:45:52 2011 From: zhenglun.wei at gmail.com (Alan Wei) Date: Fri, 23 Sep 2011 16:45:52 -0500 Subject: [petsc-users] Problem with petsc-dev update In-Reply-To: References: Message-ID: Thanks, Matt, *Something is wrong with your repo. Pull petsc-dev again and look at snesregi.c* So, I did: cd petsc-dev hg pull -u and reconfigure and make. still have the same problem. >_< Alan On Fri, Sep 23, 2011 at 4:39 PM, Matthew Knepley wrote: > On Fri, Sep 23, 2011 at 2:33 PM, Alan Wei wrote: > >> Dear folks, >> I just update petsc-dev following >> >> PETSc can now be configured in the usual way, specified on the Installation >> page. >> To update your copy of petsc-dev >> >> - cd petsc-dev >> - hg pull -u >> - cd config/BuildSystem >> - hg pull -u >> >> Once updated, you will usually want to rebuild completely >> >> - cd petsc-dev >> - ./$PETSC_ARCH/conf/reconfigure-$PETSC_ARCH.py >> - make >> >> however, after I have done that, I test a example which can be run just >> before I did the update and failed witha error message: >> >> /home/zlwei/soft/mercurial/petsc-dev/arch-linux2-c-debug/lib/libpetsc.a(snesregi.o): >> In function `SNESRegisterAll': >> /home/zlwei/soft/mercurial/petsc-dev/src/snes/interface/snesregi.c:63: >> undefined reference to `SNESCreate_FAS' >> > > Something is wrong with your repo. Pull petsc-dev again and look at > snesregi.c > > Matt > > >> BTW, the configure.log and make.log are attached. >> >> thanks in advance, >> Alan >> > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Fri Sep 23 17:12:00 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Fri, 23 Sep 2011 17:12:00 -0500 Subject: [petsc-users] [petsc-maint #88045] Problem with petsc-dev update In-Reply-To: References: Message-ID: <30AB7C76-7C0C-4B79-9FAB-25E00244B82F@mcs.anl.gov> Please do not send petsc-dev build problems to petsc-users. You can use the petsc-dev mailing list or the petsc-maint mailing list for these types of problems. petsc-users is for asking questions about the usage of the current release of PETSc. Now regarding this problem. Some non-C89 standard code snuck into the repository (where are filters Satish????). I have fixed it. Do another hg pull and run make in src/snes/impls/fas and it should resolve it. You do not have to rerun configure or make on the entire tree. Barry On Sep 23, 2011, at 4:57 PM, Alan Wei wrote: > Dear folks, > I just update petsc-dev following > > PETSc can now be configured in the usual way, specified on the Installation > page . > To update your copy of petsc-dev > > - cd petsc-dev > - hg pull -u > - cd config/BuildSystem > - hg pull -u > > Once updated, you will usually want to rebuild completely > > - cd petsc-dev > - ./$PETSC_ARCH/conf/reconfigure-$PETSC_ARCH.py > - make > > however, after I have done that, I test a example which can be run just > before I did the update and failed witha error message: > > /home/zlwei/soft/mercurial/petsc-dev/arch-linux2-c-debug/lib/libpetsc.a(snesregi.o): > In function `SNESRegisterAll': > /home/zlwei/soft/mercurial/petsc-dev/src/snes/interface/snesregi.c:63: > undefined reference to `SNESCreate_FAS' > > BTW, the configure.log and make.log are attached. > > thanks in advance, > Alan > > From gdiso at ustc.edu Sat Sep 24 00:03:22 2011 From: gdiso at ustc.edu (Gong Ding) Date: Sat, 24 Sep 2011 13:03:22 +0800 (CST) Subject: [petsc-users] floating region help In-Reply-To: References: <588771.234921316506085787.JavaMail.coremail@mail.ustc.edu> <1525625.235221316510237990.JavaMail.coremail@mail.ustc.edu> <6672045.235591316525172753.JavaMail.coremail@mail.ustc.edu> <4817529.236491316577116602.JavaMail.coremail@mail.ustc.edu> Message-ID: <32138523.241991316840602233.JavaMail.coremail@mail.ustc.edu> On Wed, Sep 21, 2011 at 05:51, Gong Ding wrote: The background of this problem is the single event upset simulation of CMOS unit circuit, i.e. SRAM, latch, flip-flop. They all very big, ~ 1M mesh nodes. Direct method is too slow and memory consuming. Some stupid commercial code needs more than one week to simulate one particle event. People usually need 100-1000 events to determine the circuit behavior! Fortunately?the problem is in time domain. the transport equation of carrier has a d/dt term which helps stable. The most difficult part comes from metal connection. Just use above inverter, when both NMOS and PMOS are closed, the metal connection region has only displacement current from semiconductor region as eps*dE/dt, while metal region the current can be written as sigma*E. The current conservation here is the boundary condition at the metal-semiconductor interface. (Since E is \frac{\partical phi}{\partical n}, here phi is the independent variable) How much does the time step help? Given the description, I'd expect not much unless it's unreasonably short. For a PNPN structure, the condition number in DC simulation is about 1e20. A transient simulation with 1 us time step can reduce the condition number to 1e6. As a result, the floating problem of semiconductor region can simply be conqured with transient mode My problem is a poisson's equation with highly heterogeneous conductance. Obviously, sigma >> eps/T, the governing equation of phi has a large jump parameter at the metal/semiconductorinterface. The eigen value analysis confirmed that the number of samllest eigen value equals to the number of floating metal region. And theeigen vector is exactly the floating region. Hmm, with a single low-energy mode, could you just start the Krylov method with a nonzero initial guess and make the initial guess exactly this mode. That would ensure that the Krylov space captures it exactly, and further iterations should all come from picking up the rest of the modes. Alternatively, you could formulate a modified system for which this mode was removed analytically. For some paper I see that the balancing neumann neumann preconditioner also helps. Can you give some suggestion? -------------- next part -------------- An HTML attachment was scrubbed... URL: From jedbrown at mcs.anl.gov Sat Sep 24 09:22:29 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Sat, 24 Sep 2011 09:22:29 -0500 Subject: [petsc-users] floating region help In-Reply-To: <32138523.241991316840602233.JavaMail.coremail@mail.ustc.edu> References: <588771.234921316506085787.JavaMail.coremail@mail.ustc.edu> <1525625.235221316510237990.JavaMail.coremail@mail.ustc.edu> <6672045.235591316525172753.JavaMail.coremail@mail.ustc.edu> <4817529.236491316577116602.JavaMail.coremail@mail.ustc.edu> <32138523.241991316840602233.JavaMail.coremail@mail.ustc.edu> Message-ID: On Sat, Sep 24, 2011 at 00:03, Gong Ding wrote: > For a PNPN structure, the condition number in DC simulation is about 1e20. > A transient simulation with 1 us time step can reduce the condition number > to 1e6. > As a result, the floating problem of semiconductor region can simply be > conqured with transient mode > My problem is a poisson's equation with highly heterogeneous conductance. > It is "just" Poisson with discontinuous coefficients? What discretization are you using? Can the jumps reasonably be aligned with element boundaries, with subdomain boundaries, or with coarse levels? (This affects which methods to use.) It is likely that you can construct low-energy interpolants and get good multigrid performance. As a starting point, have you tried something like mpiexec -n 4 ./your_app -ksp_monitor_true_residual -ksp_type fgmres -pc_type ml -pc_ml_EnergyMinimization 2 -pc_ml_maxnlevels 3 -mg_levels_pc_type asm -mg_levels_sub_pc_type lu -mg_levels_ksp_type gmres (Pick a problem where the subdomains aren't very big because this uses direct subdomain solves.) If this is reliable, it is likely possible to relax many aspects to reduce the run time. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jedbrown at mcs.anl.gov Sat Sep 24 14:04:35 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Sat, 24 Sep 2011 14:04:35 -0500 Subject: [petsc-users] snessetjacobian question In-Reply-To: References: Message-ID: On Thu, Sep 22, 2011 at 15:10, Milan Mitrovic wrote: > ok... one more error... concerning this line > call > MatSetValuesLocal(Jac,1,(/i-1/),Particles%nvlist(i)+1,ind(:),val(:),INSERT_VALUES,info) > What is the passing convention for (/i-1/)? Does this work correctly if you just create explicit arrays for these pieces? Did you try valgrind? -------------- next part -------------- An HTML attachment was scrubbed... URL: From khalid_eee at yahoo.com Sun Sep 25 16:24:53 2011 From: khalid_eee at yahoo.com (khalid ashraf) Date: Sun, 25 Sep 2011 14:24:53 -0700 (PDT) Subject: [petsc-users] scaling a parallel matrix In-Reply-To: References: <1316802920.48907.YahooMailNeo@web112619.mail.gq1.yahoo.com> Message-ID: <1316985893.81884.YahooMailNeo@web112620.mail.gq1.yahoo.com> Thanks Barry and Jed. on the NERSC computers, the installed petsc version of 3.1 is giving the error 'undefined reference to `PetscViewerHDF5Open'? I included petscsys.h? I also tried to run the two example programs? http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-3.1/src/dm/da/examples/tutorials/ex10.c.html and ex9.c illustrating the HDF5 output. They also give the same error. Could you please tell me if I am missing something or the nersc installation is missing something.? ________________________________ From: Barry Smith To: PETSc users list Cc: khalid ashraf Sent: Friday, September 23, 2011 12:40 PM Subject: Re: [petsc-users] scaling a parallel matrix On Sep 23, 2011, at 1:48 PM, Jed Brown wrote: > On Fri, Sep 23, 2011 at 13:35, khalid ashraf wrote: > Hi, I have a vector defined on a 3D DA of size ~1000X1000X30. I run the program on ~4000 processors. > After doing the calculations, print the vector in vtk format. I need to print about ~10 times at different time steps. > The printing takes about 3 hours. Is there a way to reduce the time required to print parallel vector printing. > > The VTK ASCII format is a horrible parallel format. One reliable option if you like the VTK tool chain is to write a "parallel" VTK XML format with binary appended data. You can also write an HDF5 file and either provide your own reader for whatever visualization software you are using or write an XDMF file to inform the reader of what to find where. >? > > I am trying to average over some blocks of grid points and represent the data with a smaller sized vector defined on a smaller 3D DA. One thing I am not clear is that how to get the same layout for the large and smaller DA, so that all the calculations are local when doing the averaging. Could you please provide any guidance ? > > See the lx,ly,lz parameters to DMDACreate3d(). > > http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-dev/docs/manualpages/DM/DMDACreate3d.html ? Or you can generate the finer DMDA by refining the coarse grid with DMRefine() or generate the coarser mesh by DMCoarsening the finer. They will automatically have the correct layout. Note you can use DMGetInterpolation() to get an operator that can be used to restrict from the fine grid to the coarse then scale by the vector argument returned by DMGetInterpolation(). Using the MatRestrict() with that operator performs a local piecewise linear averaging. ? Barry -------------- next part -------------- An HTML attachment was scrubbed... URL: From jedbrown at mcs.anl.gov Sun Sep 25 16:28:23 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Sun, 25 Sep 2011 16:28:23 -0500 Subject: [petsc-users] scaling a parallel matrix In-Reply-To: <1316985893.81884.YahooMailNeo@web112620.mail.gq1.yahoo.com> References: <1316802920.48907.YahooMailNeo@web112619.mail.gq1.yahoo.com> <1316985893.81884.YahooMailNeo@web112620.mail.gq1.yahoo.com> Message-ID: On Sun, Sep 25, 2011 at 16:24, khalid ashraf wrote: > on the NERSC computers, the installed petsc version of 3.1 is giving the > error > 'undefined reference to `PetscViewerHDF5Open' > I included petscsys.h > I also tried to run the two example programs > > http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-3.1/src/dm/da/examples/tutorials/ex10.c.htmland ex9.c > > illustrating the HDF5 output. They also give the same error. > Did you configure --with-hdf5? -------------- next part -------------- An HTML attachment was scrubbed... URL: From mike.hui.zhang at hotmail.com Mon Sep 26 11:30:52 2011 From: mike.hui.zhang at hotmail.com (Hui Zhang) Date: Mon, 26 Sep 2011 18:30:52 +0200 Subject: [petsc-users] Sub-indexing or SparseMult? Message-ID: Hi all, I just come into this question: if one wants a sub-vector with contiguous indices in the whole vector, surely, one can use getsubvec, but also one can setup a sparse boolean matrix for this aim; so which method is better, considering the whole vector is local or parallel? Thanks very much for any suggestions! Best wishes, Hui From knepley at gmail.com Mon Sep 26 11:33:47 2011 From: knepley at gmail.com (Matthew Knepley) Date: Mon, 26 Sep 2011 16:33:47 +0000 Subject: [petsc-users] Sub-indexing or SparseMult? In-Reply-To: References: Message-ID: On Mon, Sep 26, 2011 at 4:30 PM, Hui Zhang wrote: > Hi all, > > I just come into this question: if one wants a sub-vector with contiguous > indices > in the whole vector, surely, one can use getsubvec, but also one can setup > a > sparse boolean matrix for this aim; so which method is better, considering > the whole vector is local or parallel? > http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/Vec/VecScatterCreate.html Matt > Thanks very much for any suggestions! > > Best wishes, > Hui -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From jedbrown at mcs.anl.gov Mon Sep 26 11:40:52 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Mon, 26 Sep 2011 11:40:52 -0500 Subject: [petsc-users] Sub-indexing or SparseMult? In-Reply-To: References: Message-ID: VecGetSubVector has lower overhead and might be implemented without a copy. On Sep 26, 2011 11:32 AM, "Hui Zhang" wrote: > Hi all, > > I just come into this question: if one wants a sub-vector with contiguous indices > in the whole vector, surely, one can use getsubvec, but also one can setup a > sparse boolean matrix for this aim; so which method is better, considering > the whole vector is local or parallel? > > Thanks very much for any suggestions! > > Best wishes, > Hui -------------- next part -------------- An HTML attachment was scrubbed... URL: From mike.hui.zhang at hotmail.com Mon Sep 26 12:56:42 2011 From: mike.hui.zhang at hotmail.com (Hui Zhang) Date: Mon, 26 Sep 2011 19:56:42 +0200 Subject: [petsc-users] Sub-indexing or SparseMult? In-Reply-To: References: Message-ID: thanks a lot! On Sep 26, 2011, at 6:40 PM, Jed Brown wrote: > VecGetSubVector has lower overhead and might be implemented without a copy. > > On Sep 26, 2011 11:32 AM, "Hui Zhang" wrote: > > Hi all, > > > > I just come into this question: if one wants a sub-vector with contiguous indices > > in the whole vector, surely, one can use getsubvec, but also one can setup a > > sparse boolean matrix for this aim; so which method is better, considering > > the whole vector is local or parallel? > > > > Thanks very much for any suggestions! > > > > Best wishes, > > Hui -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesselu at stanford.edu Mon Sep 26 17:01:40 2011 From: jesselu at stanford.edu (Jesse Lu) Date: Mon, 26 Sep 2011 15:01:40 -0700 Subject: [petsc-users] Interfacing with TAO using complex scalars Message-ID: Hi, Does anyone know a way interfacing PETSc with TAO using complex scalars? If not, can you recommend alternatives for solving nonlinear optimization problems involving complex numbers? Thanks! Jesse -------------- next part -------------- An HTML attachment was scrubbed... URL: From recrusader at gmail.com Mon Sep 26 17:12:21 2011 From: recrusader at gmail.com (recrusader) Date: Mon, 26 Sep 2011 17:12:21 -0500 Subject: [petsc-users] Interfacing with TAO using complex scalars In-Reply-To: References: Message-ID: The problem is that TAO can not provide the support for complex scalars. On Mon, Sep 26, 2011 at 5:01 PM, Jesse Lu wrote: > Hi, > > Does anyone know a way interfacing PETSc with TAO using complex scalars? If > not, can you recommend alternatives for solving nonlinear optimization > problems involving complex numbers? Thanks! > > Jesse > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ckontzialis at lycos.com Mon Sep 26 23:04:33 2011 From: ckontzialis at lycos.com (Konstantinos Kontzialis) Date: Tue, 27 Sep 2011 07:04:33 +0300 Subject: [petsc-users] Problem with mpiexec Message-ID: <4E814B51.2060901@lycos.com> Dear all, I use Fedora 15 and I needed to make a new install of the system. However, when I installed petsc mpiexec did not work. Fedora 15 uses now mpich2-1.4.1 which uses hydra as the default process manager. But nothing is working. Any suggestions about what should I check and what to do? Kostas From sean at mcs.anl.gov Mon Sep 26 23:07:26 2011 From: sean at mcs.anl.gov (Sean Farley) Date: Mon, 26 Sep 2011 23:07:26 -0500 Subject: [petsc-users] Problem with mpiexec In-Reply-To: <4E814B51.2060901@lycos.com> References: <4E814B51.2060901@lycos.com> Message-ID: > > I use Fedora 15 and I needed to make a new install of the system. However, > when I installed petsc mpiexec did not work. Fedora 15 uses now mpich2-1.4.1 > which uses hydra as the default process manager. But nothing is working. Any > suggestions about what should I check and what to do? I use mpich2-1.4.1 on all my systems (mac and linux) and have no problem with hydra. What went wrong? A configure error? A make error? Or a runtime error? I think regardless of which error it is, you should send your configure.log (and make.log if it got that far) to petsc-maint. Sean -------------- next part -------------- An HTML attachment was scrubbed... URL: From jedbrown at mcs.anl.gov Mon Sep 26 23:08:07 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Mon, 26 Sep 2011 23:08:07 -0500 Subject: [petsc-users] Problem with mpiexec In-Reply-To: <4E814B51.2060901@lycos.com> References: <4E814B51.2060901@lycos.com> Message-ID: On Mon, Sep 26, 2011 at 23:04, Konstantinos Kontzialis < ckontzialis at lycos.com> wrote: > Dear all, > > I use Fedora 15 and I needed to make a new install of the system. However, > when I installed petsc mpiexec did not work. Fedora 15 uses now mpich2-1.4.1 > which uses hydra as the default process manager. But nothing is working. As always, what are the complete symptoms? What are you trying to do and what happens? > Any suggestions about what should I check and what to do? > The most common problem is that you have more than one mpiexec and that a different MPI is used for compilation. -------------- next part -------------- An HTML attachment was scrubbed... URL: From juan.gonzalez at accelogic.com Tue Sep 27 01:26:58 2011 From: juan.gonzalez at accelogic.com (Juan Gonzalez, Ph.D.) Date: Tue, 27 Sep 2011 02:26:58 -0400 Subject: [petsc-users] Typical number of iterations for CG and GMRES: Compiling knowledge base -- please contribute Message-ID: <00ad01cc7cde$778bdb20$66a39160$@accelogic.com> Friends, I am trying to compile a knowledge base on what could be typical expectations for the number of iterations that one could see in real-life well-conditioned CG, GMRES, and general Krylov problems. I understand that the number of iterations will vary substantially depending on problem size, matrix structure, condition numbers, quality of preconditioners, etc. It would be of great value to have an idea of what to expect and under which circumstances, driven by previous experience. If you send me a quick note with your experience on number of iterations as matrix size and other circumstances vary, I promise I will compile all the info that I receive from the group and send a summary report to the whole group. (please try to include problem type and the things that you notice in your experience that affect the number of iterations the most). The main interest is on instances that are/were useful for real life problems - for example, please do not include studies where you extended artificially the number of iterations to analyze the algorithm or learn things about the convergence. Thanks in advance for your contribution. Best regards, Juan Juan Gonzalez, Ph.D. President & Chief Scientist Accelogic LLC 13680 NW 14th St, Suite 5 Sunrise, FL 33323 Ph: +1.954.838.1100 Fx: +1.954.208.0018 www.accelogic.com Description: Description: accelogic "A pessimist sees the difficulty in every opportunity; an optimist sees the opportunity in every difficulty" - Sir Winston Churchill CONFIDENTIALITY: This email (including any attachments) may contain confidential, proprietary and privileged information, and unauthorized disclosure or use is prohibited. If you received this email in error, please notify the sender and delete this email from your system. Thank you. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: image/jpeg Size: 2416 bytes Desc: not available URL: From ckontzialis at lycos.com Tue Sep 27 01:44:08 2011 From: ckontzialis at lycos.com (Konstantinos Kontzialis) Date: Tue, 27 Sep 2011 09:44:08 +0300 Subject: [petsc-users] Problem with mpiexec Message-ID: <4E8170B8.5040505@lycos.com> Dear all, When I do the following: make PETSC_DIR=/home/kontzialis/petsc-3.1-p8 PETSC_ARCH=linux-gnu-c-debug test I get the following: Running test examples to verify correct installation and I wait for hours. I tried to run an mpi application of my own which uses petsc as follows: mpiexec -n 8 ./hoac cylisupersonic -llf_flux -explicit -explicit_type 1 and I get the following error: ssh: Could not resolve hostname PlusSodaL: Name or service not known Any recommendations? Best regards Kostas From knepley at gmail.com Tue Sep 27 06:43:14 2011 From: knepley at gmail.com (Matthew Knepley) Date: Tue, 27 Sep 2011 11:43:14 +0000 Subject: [petsc-users] Problem with mpiexec In-Reply-To: <4E8170B8.5040505@lycos.com> References: <4E8170B8.5040505@lycos.com> Message-ID: On Tue, Sep 27, 2011 at 6:44 AM, Konstantinos Kontzialis < ckontzialis at lycos.com> wrote: > Dear all, > > When I do the following: > > make PETSC_DIR=/home/kontzialis/**petsc-3.1-p8 > PETSC_ARCH=linux-gnu-c-debug test > > I get the following: > > Running test examples to verify correct installation > > and I wait for hours. > > I tried to run an mpi application of my own which uses petsc as follows: > > mpiexec -n 8 ./hoac cylisupersonic -llf_flux -explicit -explicit_type 1 > > and I get the following error: > > ssh: Could not resolve hostname PlusSodaL: Name or service not known > You have your hostname set to something that your DNS will not resolve. No MPI program will run on this box, not just PETSc. Either set your hostname to something that resolves (using 'hostname'), or put 'PlusSodaL' in /etc/resolve.conf Matt > Any recommendations? > > Best regards > > Kostas > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From balay at mcs.anl.gov Tue Sep 27 07:29:22 2011 From: balay at mcs.anl.gov (Satish Balay) Date: Tue, 27 Sep 2011 07:29:22 -0500 (CDT) Subject: [petsc-users] Problem with mpiexec In-Reply-To: References: <4E8170B8.5040505@lycos.com> Message-ID: On Tue, 27 Sep 2011, Matthew Knepley wrote: > On Tue, Sep 27, 2011 at 6:44 AM, Konstantinos Kontzialis < > ckontzialis at lycos.com> wrote: > > > Dear all, > > > > When I do the following: > > > > make PETSC_DIR=/home/kontzialis/**petsc-3.1-p8 > > PETSC_ARCH=linux-gnu-c-debug test > > > > I get the following: > > > > Running test examples to verify correct installation > > > > and I wait for hours. > > > > I tried to run an mpi application of my own which uses petsc as follows: > > > > mpiexec -n 8 ./hoac cylisupersonic -llf_flux -explicit -explicit_type 1 > > > > and I get the following error: > > > > ssh: Could not resolve hostname PlusSodaL: Name or service not known > > > > You have your hostname set to something that your DNS will not resolve. No > MPI program will run on this box, not just PETSc. > > Either set your hostname to something that resolves (using 'hostname'), or > put 'PlusSodaL' in /etc/resolve.conf Actually you should add the following to /etc/hosts >>> 127.0.0.1 PlusSodaL <<< And make sure 'ssh PlusSodaL ls' works. Its strange that hydra is attempting to use 'ssh' for this run. What do you have for env |grep HYDRA Alternatively you can try: mpiexec -launcher fork -n 8 ./hoac cylisupersonic -llf_flux -explicit -explicit_type 1 More hydra related info at: http://wiki.mcs.anl.gov/mpich2/index.php/Using_the_Hydra_Process_Manager Satish From gshy2014 at gmail.com Tue Sep 27 21:44:54 2011 From: gshy2014 at gmail.com (Shiyuan) Date: Tue, 27 Sep 2011 21:44:54 -0500 Subject: [petsc-users] Set nnz for Mat & MatSetValues Message-ID: Hi, 1. I want to allow setting the MatType by option -mat_type. It seems that I can rewrite ierr=MatCreateSeqAIJ(comm,m,n,nz,nnz,A); CHKERRQ(ierr); to ierr=MatCreate(comm,A);CHKERRQ(ierr); ierr=MatSetSizes(*A,PETSC_DECIDE,PETSC_DECIDE,m,n);CHKERRQ(ierr); ierr=MatSetFromOptions(*A);CHKERRQ(ierr); So that the MatType can be set using -mat_type options. But then how can the nz or nnz be specified? 2. If we set mat_type to cusp and use MatSetValues, when do the values goto device memory? Is it assembly first on host memory and then copy it once to device memory together after MatAssemblyEnd() or is it done in a different way? Does it also contain a flag to indicate whether the data value is current? Thanks. Shiyuan -------------- next part -------------- An HTML attachment was scrubbed... URL: From jedbrown at mcs.anl.gov Tue Sep 27 21:48:46 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Tue, 27 Sep 2011 21:48:46 -0500 Subject: [petsc-users] Set nnz for Mat & MatSetValues In-Reply-To: References: Message-ID: On Tue, Sep 27, 2011 at 21:44, Shiyuan wrote: > 1. > I want to allow setting the MatType by option -mat_type. It seems that I > can rewrite > ierr=MatCreateSeqAIJ(comm,m,n,nz,nnz,A); CHKERRQ(ierr); > to > ierr=MatCreate(comm,A);CHKERRQ(ierr); > ierr=MatSetSizes(*A,PETSC_DECIDE,PETSC_DECIDE,m,n);CHKERRQ(ierr); > ierr=MatSetFromOptions(*A);CHKERRQ(ierr); > So that the MatType can be set using -mat_type options. > But then how can the nz or nnz be specified? > MatSeqAIJSetPreallocation() and others (you can call all of them). > > 2. If we set mat_type to cusp and use MatSetValues, when do the values goto > device memory? Is it assembly first on host memory and then copy it once to > device memory together after MatAssemblyEnd() or is it done in a different > way? Does it also contain a flag to indicate whether the data value is > current? > It's updated when needed and a flag is held for which copies are current so there are no unnecessary copies. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mmnasr at gmail.com Tue Sep 27 22:39:15 2011 From: mmnasr at gmail.com (Mohamad M. Nasr-Azadani) Date: Tue, 27 Sep 2011 20:39:15 -0700 Subject: [petsc-users] Memory usage of KSP (or any PETSc object) Message-ID: Hi, I was trying to find the exact (or more or less exact) memory consumption of any PETSc object. I was wondering how accurate is this method: *PetscMemoryGetMaximumUsage(&mem1);* *.* *(creation of any PETSc object).* *.* * * *PetscMemoryGetMaximumUsage(&mem2);* *Memory = mem2 - mem1; * * * *Thanks in advance, * *Mohamad* * * -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Tue Sep 27 22:45:00 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Tue, 27 Sep 2011 22:45:00 -0500 Subject: [petsc-users] Memory usage of KSP (or any PETSc object) In-Reply-To: References: Message-ID: You must first call PetscMemorySetGetMaximumUsage() at the beginning of your program. This will work, but you need to keep in mind that just creating the object generally doesn't take much space, rather it takes more and more space as you add things to it. For example getmaxusage() MatCreate() ... MatSetValues() MatAssemblyBegin() MatAssemblyEnd() getmaxusage() will tell you how much space the filled matrix takes. Barry On Sep 27, 2011, at 10:39 PM, Mohamad M. Nasr-Azadani wrote: > Hi, > > I was trying to find the exact (or more or less exact) memory consumption of any PETSc object. > I was wondering how accurate is this method: > > PetscMemoryGetMaximumUsage(&mem1); > . > > (creation of any PETSc object). > . > > > > PetscMemoryGetMaximumUsage(&mem2); > > Memory = mem2 - mem1; > > > Thanks in advance, > Mohamad > > > > > From mmnasr at gmail.com Tue Sep 27 23:29:30 2011 From: mmnasr at gmail.com (Mohamad M. Nasr-Azadani) Date: Tue, 27 Sep 2011 21:29:30 -0700 Subject: [petsc-users] Memory usage of KSP (or any PETSc object) In-Reply-To: References: Message-ID: Thanks Barry for your quick response. I am a bit puzzled by how this function works. I was trying it on this simple example: PetscMemoryGetMaximumUsage(&mem1); PetscMalloc(1*sizeof(double), &aa); PetscMemoryGetMaximumUsage(&mem2); The difference I get for (mem2-mem1) is zero. Why is that? Best, Mohamad On Tue, Sep 27, 2011 at 8:45 PM, Barry Smith wrote: > > You must first call PetscMemorySetGetMaximumUsage() at the beginning of > your program. > > This will work, but you need to keep in mind that just creating the object > generally doesn't take much space, rather it takes more and more space as > you add things to it. For example > > getmaxusage() > MatCreate() > > ... MatSetValues() > > MatAssemblyBegin() > MatAssemblyEnd() > getmaxusage() > > will tell you how much space the filled matrix takes. > > Barry > > On Sep 27, 2011, at 10:39 PM, Mohamad M. Nasr-Azadani wrote: > > > Hi, > > > > I was trying to find the exact (or more or less exact) memory consumption > of any PETSc object. > > I was wondering how accurate is this method: > > > > PetscMemoryGetMaximumUsage(&mem1); > > . > > > > (creation of any PETSc object). > > . > > > > > > > > PetscMemoryGetMaximumUsage(&mem2); > > > > Memory = mem2 - mem1; > > > > > > Thanks in advance, > > Mohamad > > > > > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ckontzialis at lycos.com Wed Sep 28 02:22:43 2011 From: ckontzialis at lycos.com (Konstantinos Kontzialis) Date: Wed, 28 Sep 2011 10:22:43 +0300 Subject: [petsc-users] Problem with mpiexec In-Reply-To: References: Message-ID: <4E82CB43.1080400@lycos.com> On 09/27/2011 08:00 PM, petsc-users-request at mcs.anl.gov wrote: > Re: Problem with mpiexec Dear all, Problem fixed! Kostas From jedbrown at mcs.anl.gov Wed Sep 28 05:26:58 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Wed, 28 Sep 2011 05:26:58 -0500 Subject: [petsc-users] Memory usage of KSP (or any PETSc object) In-Reply-To: References: Message-ID: On Tue, Sep 27, 2011 at 23:29, Mohamad M. Nasr-Azadani wrote: > I am a bit puzzled by how this function works. > I was trying it on this simple example: > > PetscMemoryGetMaximumUsage(&mem1); > > PetscMalloc(1*sizeof(double), &aa); > > PetscMemoryGetMaximumUsage(&mem2); > > > The difference I get for (mem2-mem1) is zero. > The maximum usage is actually only updated when objects are destroyed. You can call PetscMemoryGetCurrentUsage() to see how much is used by everything used by the application (not just PETSc). When using tracing malloc (default in debug mode or when run with -malloc). You can use PetscMallocGetCurrentUsage() and PetscMallocGetMaximumUsage() at any time and it should be a current measure of what PETSc has allocated (not just after destroying objects), but does not measure anything allocated directly by the application (e.g. by calling malloc directly, C++ new, Fortran arrays). -------------- next part -------------- An HTML attachment was scrubbed... URL: From gianmail at gmail.com Wed Sep 28 08:02:23 2011 From: gianmail at gmail.com (Gianluca Meneghello) Date: Wed, 28 Sep 2011 15:02:23 +0200 Subject: [petsc-users] TS for incompressible NS equations Message-ID: Hi, I would have a question concerning the TS solver in PETSc. I need to solve the linearized, incompressible Navier Stokes equation in time. I have reformulated the problem as B du/dt = J u where J is the Jacobian of the Navier Stokes equations and u is a vector containing {u,v,,w,p}. B is a diagonal matrix which is 0 in the lines corresponding to the pressure equation (enforcing the divergence free condition) and Dirichlet BC's and 1 otherwise. Backward Euler would then solve it as u(t+1) = (B-dt J)^-1 B u(t) as in page 115 of the user guide (is there a typing error there? Should it be " (B - dt A) u(n+1) = B u(n) " ?) Using PETSc 3.2, I pass J to the TS object as TSSetRHSFunction(ts,PETSC_NULL,TSComputeRHSFunctionLinear,PETSC_NULL);CHKERRQ(ierr); TSSetRHSJacobian(ts,A,A,TSComputeRHSJacobianConstant,PETSC_NULL);CHKERRQ(ierr); but I don't understand how I can set B (or whether I am using a wrong approach). Thanks in advance for your help Gianluca -- "[Je pense que] l'homme est un monde qui vaut des fois les mondes et que les plus ardentes ambitions sont celles qui ont eu l'orgueil de l'Anonymat" -- Non omnibus, sed mihi et tibi Amedeo Modigliani From jedbrown at mcs.anl.gov Wed Sep 28 08:25:26 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Wed, 28 Sep 2011 08:25:26 -0500 Subject: [petsc-users] TS for incompressible NS equations In-Reply-To: References: Message-ID: On Wed, Sep 28, 2011 at 08:02, Gianluca Meneghello wrote: > I need to solve the linearized, incompressible Navier Stokes equation > in time. I have reformulated the problem as > > B du/dt = J u > > where J is the Jacobian of the Navier Stokes equations and u is a > vector containing {u,v,,w,p}. > B is a diagonal matrix which is 0 in the lines corresponding to the > pressure equation (enforcing the divergence free condition) and > Dirichlet BC's and 1 otherwise. > > Backward Euler would then solve it as > > u(t+1) = (B-dt J)^-1 B u(t) > > as in page 115 of the user guide (is there a typing error there? > Should it be " (B - dt A) u(n+1) = B u(n) " ?) > Yes, thank you. I should rewrite this section because it's confusing in its current form. It would be better to look at the later sections 6.1.2 Solving Differential Algebraic Equations and 6.1.3 on IMEX methods. > > Using PETSc 3.2, I pass J to the TS object as > > > TSSetRHSFunction(ts,PETSC_NULL,TSComputeRHSFunctionLinear,PETSC_NULL);CHKERRQ(ierr); > > TSSetRHSJacobian(ts,A,A,TSComputeRHSJacobianConstant,PETSC_NULL);CHKERRQ(ierr); > This sets a constant Jacobian that never changes. I don't know if that's what you mean by linearized Navier-Stokes. > > but I don't understand how I can set B (or whether I am using a wrong > approach). > If you have a B that is not the identity, then you should use TSSetIFunction() and/or TSSetIJacobian(). Note that incompressible flow in velocity-pressure form is an index 2 DAE, so not all coupled time integration methods perform well. There is very new code in petsc-dev that you may want to try for this problem. You have to enable these by configuring --with-rosw and then you can run with -ts_type rosw -ts_rosw_type ra34pw2. (This would use a four stage third order Rosenbrock-W scheme for stiff problems that performed well in the Navier-Stokes test problems of Rang & Angermann 2005, -ts_rosw_type ra3pw is a three-stage third order method that is also worth trying.) The need to configure --with-rosw will go away in a week or two, once that code stabilizes. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jbacks at mcmillan-mcgee.com Wed Sep 28 10:00:06 2011 From: jbacks at mcmillan-mcgee.com (Jonathan Backs) Date: Wed, 28 Sep 2011 09:00:06 -0600 Subject: [petsc-users] Conditional Constraints In-Reply-To: <1232662701.10005.1316110026510.JavaMail.root@zimbra.anl.gov> References: <1232662701.10005.1316110026510.JavaMail.root@zimbra.anl.gov> Message-ID: <096246E5-10D7-44C7-A78E-F2AB0A26A860@mcmillan-mcgee.com> Hi Shri, I have continued to have little success with my SNESVI formulation of the current constraint. However, working outside of PETSc, I believe I have found an explicit formulation that works well: 1) Set Vmag = Vmag_max and use PETSc to solve for the electric potential distribution (V_ijk is the only degree of freedom). 2) For each electrode, let Vmag_new = Imag_max * Vmag_old / Imag_old. 3) Repeat until Imag is sufficiently close to Imag_max, or until Vmag = Vmag_max with Imag < Imag_max. This process converges in about 5 iterations, which is great, but I would like to incorporate the thermal problem again (including TS) now that this electrical problem is solved. How could I implement this constraint inside the PETSc solution loop? Is there any advantage to using SNESVI when my update step is explicit? Or is there a feature I could use to add a Vmag_new computation after each SNES step, and have this taken into account for the SNES convergence criteria? Thanks again for your continued patience and interest in helping me with this problem. Jon On 2011-09-15, at 12:07 PM, Shri wrote: > Jon, > > > Additionally, the potential distribution does not look correct as it did before (even without SNES reporting convergence)---instead, the magnitude of the distribution > looks like the real part of the correct distribution, if that makes sense. > Hmm, in the SNESVI formulation using complex numbers, we have the check real(xlower) <= real(x) <= real(xupper). i am not sure whether this is the cause for such a behavior. I've had conversation with the other PETSc developers to use absolute values for comparison instead of real values, i.e. abs(xl) <= abs(x) <= abs(xupper). By doing this, you can essentially have a function for Ve and its bounds being 0 and Vmag_max. However, the other developers think that absolute values being nonlinear in nature might produce some odd behavior in the solution process, so i've havent' support for it yet. > > >This asymmetry may be due to the fact that I now only record the Vmag and Imag variables for the top block of each electrode, whereas before I recorded Ve for all >three blocks. > Yes, do you want the magnitude and angle for all the three blocks to be same? If so, then probably you'll have to add another constraint. > > >If a block (with coordinates i, j, k) is not occupied by an electrode, C_e is set to zero in the expression for Vb_ijk and all function and Jacobian entries for Vmag >and Imag are either not set or set to zero. > Do you already know the Vmag and Imag values are these blocks? Are they zero/non-zero? I suggest the following for these blocks > > xl_ijk_DOF_VMAG = Vmag0_ijk > xu_ijk_DOF_VMAG = Vmag0_ijk > xl_ijk_DOF_IMAG = Imag0_ijk > xu_ijk_DOF_IMAG = Imag0_ijk > > where Vmag0_ijk and Imag0_ijk is the initial voltage and current magnitudes at these blocks. Basically, the idea here is to keep these magnitudes constant and not have the functions for these variables be a part of the solution process. > > Shri > > > > > > Hi Shri, > > Thanks for following up with me. I have made some progress, although my problem is not yet solved. > > Inspired by your recent suggestions, I realized Ve was redundant as a degree of freedom and did not need to be computed separately from Vmag. I do not want the phase of Ve to be modified by the solution process anyways, so whenever I need the complex value of Ve, I can simply compose it from the Vmag and the constant phase I have recorded for a given electrode. My new function and Jacobian formulations are below if you are interested in seeing them. > > After I made this change, I began to see the desired behaviour where Vmag decreases when Imag is too high. However, it seems to converge on some magnitude that is maybe 200% too high, and the SNES reports DIVERGED_LINE_SEARCH. Additionally, the potential distribution does not look correct as it did before (even without SNES reporting convergence)---instead, the magnitude of the distribution looks like the real part of the correct distribution, if that makes sense. > > A further problem relates to the fact that each electrode is three blocks tall. Before I eliminated Ve, each of the three blocks had a very similar voltage and phase. Now, the top block has a different magnitude and phase than the two beneath it. This asymmetry may be due to the fact that I now only record the Vmag and Imag variables for the top block of each electrode, whereas before I recorded Ve for all three blocks. As you can find below, however, I have included Jacobian entries for the lower two block voltages Vb of each electrode, so I cannot see why this strategy would not work. > > Given your experience with PETSc and Newton's method, do you have any ideas about where these problems might originate? > > Thank you again, > > Jon > > > If a block (with coordinates i, j, k in the x, y, and z directions, respectively) is occupied by an electrode, > - for Vb_ijk, f = C_posX * Vb_posX + C_posY * Vb_posY + C_posZ * Vb_posZ > - (C_posX + C_posY + C_posZ + C_negX + C_negY + C_negZ + C_e) * Vb_ijk > + C_negX * Vb_negX + C_negY * Vb_negY + C_negZ * Vb_negZ > - C_e * polar(Vmag, arg(Ve)), > where all C's are calculated constants that do not depend on the other degrees of freedom, but do depend on the type of the neighbouring block in their assigned directions. Then, > the Jacobian entries for Vb_ijk are > df/dVb_negZ = C_negZ (col.i = i, col.j = j, col.k = k - 1) > df/dVb_negY = C_negY (col.i = i, col.j = j - 1, col.k = k) > df/dVb_negX = C_negX (col.i = i - 1, col.j = j, col.k = k) > df/dVb_ijk = - (C_posX + C_posY + C_posZ + C_negX + C_negY + C_negZ + C_e) (col.i = i, col.j = j, col.k = k) > df/dVb_posX = C_posX (col.i = i + 1, col.j = j, col.k = k) > df/dVb_posY = C_posY (col.i = i, col.j = j + 1, col.k = k) > df/dVb_posZ = C_posZ (col.i = i, col.j = j, col.k = k + 1) > For each of these, row.i = i, row.j = j, row.k = k, and row.c = DOF_VB (the DOF index for Vb). Also, col.c = DOF_VB. > The only cross term for the Vb_ijk function is > df/dVmag = polar(-C_e, arg(Ve)) (row.i = i, row.j = j, row.k = k, row.c = DOF_VB, col.i = i, col.j = j, col.k = k col.c = DOF_VMAG) > > - for Vmag_ijk, f = abs(Ve_ijk) - abs(Ve_ijk_max). The idea was to provide some 'pressure' towards Ve = Ve_max. The corresponding Jacobian entries are > df/dVmag_ijk = 1.0 + 0.0*i (row.i = i, row.j = j, row.k = k, row.c = DOF_VMAG, col.i = i, col.j = j, col.k = k, col.c = DOF_VMAG) > > - for Imag_ijk, f = abs(iE) - iE_max. Again, the idea was to provide some 'pressure' towards iE = iE_max. iE is calculated by adding iE_block currents for each block occupied by an electrode. iE_block for block i,j,k is found from > iE_block_ijk = aE_ijk * (Ve - Vb_ijk) > where aE_ijk is a constant that does not depend on any degrees of freedom. There is one Ve value per electrode. The Jacobian entries are then > df/dImag_ijk = 1.0 + 0.0*i (row.i = i, row.j = j, row.k = k, row.c = DOF_IMAG, col.i = i, col.j = j, col.k = k, col.c = DOF_IMAG) > df/dVmag = aE_sum * (real(iE) * cos(arg(Ve)) + imag(iE) * sin(arg(Ve)))/ abs(iE) (row.i = i, row.j = j, row.k = k, row.c = DOF_IMAG, col.i = i, col.j = j, col.k = k, col.c = DOF_VMAG) > df/dVb_ijk = - 0.5 * aE_ijk * conj(iE) / abs(iE) (row.i = i, row.j = j, row.k = k, row.c = DOF_IMAG, col.i = i, col.j = j, col.k = k, col.c = DOF_VB) > df/dVb_ij(k-1) = - 0.5 * aE_ij(k-1) * conj(iE) / abs(iE) (row.i = i, row.j = j, row.k = k, row.c = DOF_IMAG, col.i = i, col.j = j, col.k = k-1, col.c = DOF_VB) > df/dVb_ij(k-2) = - 0.5 * aE_ij(k-2) * conj(iE) / abs(iE) (row.i = i, row.j = j, row.k = k, row.c = DOF_IMAG, col.i = i, col.j = j, col.k = k-2, col.c = DOF_VB) > (These last three are the only derivatives left that do not technically exist.) > > If a block (with coordinates i, j, k) is not occupied by an electrode, C_e is set to zero in the expression for Vb_ijk and all function and Jacobian entries for Vmag and Imag are either not set or set to zero. > > My SNES_VI constraints are set as follows for every block: > xl_ijk_DOF_V = -SNES_VI_INF > xu_ijk_DOF_V = SNES_VI_INF > xl_ijk_DOF_VMAG = 0.0 > xu_ijk_DOF_VMAG = abs(Ve_max) > xl_ijk_DOF_IMAG = 0.0 > xu_ijk_DOF_IMAG = abs(iE_max) > > > On 2011-09-13, at 12:12 PM, Shri wrote: > > Jon, > Any progress with your work? Send us an email once you've sorted out your formulation and we would be glad to help you out with its PETSc implementation. > > Shri > > Shri, > > Note here that (a) f(Ve_ijk) is a complex function while Imag and Imag_max are real. So the above equation does not hold. > > Would f(Ve_ijk) = Imag - Imag_max = 0 simply be equivalent to saying, solve the real system of equations Imag - Imag_max = 0 and 0 = 0 (the imaginary part)? Regardless, I am not sure what a correct choice of f(Ve_ijk) would be. > > > (b) In your previous email, you had bounds on Imag, 0 < Imag < Imag_max. Now you want Ve_ijk to increase if Imag > Imag_max and decrease if Imag < Imag_max. Are you trying to find the value of Ve at which the current is maximum, i.e. Imag = Imag_max, by doing this? > > Yes, that is essentially what I am trying to find. Ve determines Vb, and the difference between the two determines Imag. Vmag (magnitude of Ve) and Imag have the restrictions 0 < Vmag < Vmag_max and 0 < Imag < Imag_max. I want to find either Ve for Imag = Imag_max or Imag for Vmag = Vmag_max. In my test scenario, Vmag_max happens to be quite high so Imag_max will always be reached first, but in general either situation is possible. > > Thank you again, > > Jon > > > On 2011-09-08, at 8:34 PM, Shri wrote: > > Jon, > > > I do not want Ve to be constant, however; I would like it to either decrease in response to Imag > Imag_max or increase in response to Imag < Imag_max, providing > the condition Vmag <= Vmag_max remains true. > > Should I then have something like f(Ve_ijk) = Imag - Imag_max? > > Note here that (a) f(Ve_ijk) is a complex function while Imag and Imag_max are real. So the above equation does not hold. > (b) In your previous email, you had bounds on Imag, 0 < Imag < Imag_max. Now you want Ve_ijk to increase if Imag > Imag_max and decrease if Imag < Imag_max. Are you trying to find the value of Ve at which the current is maximum, i.e. Imag = Imag_max, by doing this? > > Shri, > > Thanks very much for your response. In fact, -snes_converged_reason revealed that the solution was not converging, as you suggested: "Nonlinear solve did not converge due to DIVERGED_LINE_SEARCH." I had not realized this earlier since the voltage distribution in the solution appeared to be correct, even though the SNES_VI constraints were not satisfied. > > Given that and your explanation, I am sure something is wrong with my choices of f. I do not want Ve to be constant, however; I would like it to either decrease in response to Imag > Imag_max or increase in response to Imag < Imag_max, providing the condition Vmag <= Vmag_max remains true. Should I then have something like f(Ve_ijk) = Imag - Imag_max? > > Note that you have > > > One more thing from your response: > >If a block (with coordinates i, j, k) is not occupied by an electrode, C_e is set to zero in the expression for Vb_ijk and all function and Jacobian entries for Ve, >Vmag, and Imag are either not set or set to zero. > > Does this mean f(Ve_ijk) = f(Vmag_ijk) = f(Imag_ijk) = 0? If so, then Ve_ijk, Vmag_ijk, and Imag_ijk would retain their initial values for all time steps. > > In my present scheme, f(Ve_ijk) = f(Vmag_ijk) = 0 and f(Imag_ijk) is non-zero for electrode blocks; all three are zero for non-electrode blocks. Depending on the choice of initial conditions, I can make f(Vmag_ijk) non-zero as well (until Ve changes to match Ve_max). > > Thanks again, > > Jon > > On 2011-09-08, at 3:42 PM, Shri wrote: > > Jon, > > > for Ve_ijk, f = 0.0 + 0.0*i, and all corresponding Jacobian entries are zero as well. Ve_ijk is set to the maximum voltage for each electrode when the initial solution >is - set. Since the magnitude of Ve is constrained rather than Ve itself, I believe this is correct, but could this be why Ve does not change as it should when the >problem is - being solved? > > This f(Ve_ijk) seems very odd to me since normally you would always have f(some/all variables) = 0 as the function. If f=0.0+0.0*i for all steps, then you don't need a seperate variable Ve_ijk, you can treat it as a constant since Ve_ijk will not change its value during the solutions. > > Moroever, the Jacobian rows for df(Ve_ijk)/d(all variables) would be all zeros which implies a singular matrix. I am not sure how the solution converged for this case. Were you using -pc_type lu with some shift or some external direct solver such as superlu? What do you get for -snes_monitor -snes_converged_reason? > > > > Ve_ijk is set to the maximum voltage for each electrode when the initial solution is set. Since the magnitude of Ve is constrained rather than Ve itself. > > for Vmag_ijk, f = abs(Ve_ijk) - abs(Ve_ijk_max) > > Since Ve_ijk = Ve_ijk_max, f for Vmag_ijk = 0.0 and hence Vmag_ijk also will also retain its initial value throughout. > > >If a block (with coordinates i, j, k) is not occupied by an electrode, C_e is set to zero in the expression for Vb_ijk and all function and Jacobian entries for Ve, >Vmag, and Imag are either not set or set to zero. > Does this mean f(Ve_ijk) = f(Vmag_ijk) = f(Imag_ijk) = 0? If so, then Ve_ijk, Vmag_ijk, and Imag_ijk would retain their initial values for all time steps. > > > When I run my program with -snes_vi_type rs, the solution converges quickly but Vmag voltages remain the same as Ve_max for every electrode. As a result, Imag > currents are far larger than abs(iE_max). > Seems to me like the problem is not converging, or giving the incorrect solution, since the solution of a variational inequality problem has to satisfy Imag < iE_max, which in your case is not happening. > > Thanks for the detailed explanation. Would you mind if I (or you) forward this to petsc-users list so that others could share their thoughts too. > > Shri > > > > Hi Shri, > > Thanks again for your patience with my being away this past month. I have spent some more time with this problem and I have added Jacobian entries for 'cross terms' such as d(f(Vb)) / d(Ve). However, this did not appear to change the solution arrived at by PETSc. > > I will give you some examples of the f functions and corresponding Jacobian entries as you suggested, with the hope you will be able to see where I may be going wrong. I will use Vb = V_block, Ve = V_electrode, Vmag = magnitude(V_electrode), Imag = magnitude(I_electrode) to name the four degrees of freedom. > > In my 3D finite differencing grid, there are two types of blocks: blocks occupied by electrodes, and blocks occupied by a resistive medium. The block type of the current block and its neighbouring blocks determines the form of f. This is complicated by the fact that individual electrodes generally occupy three vertically-contiguous blocks, and I want to limit the electric current flowing from the whole electrode rather than the current through each block. Note that Ve, Vmag, and Imag are thus computed or recorded only for the top block of each electrode. > > If a block (with coordinates i, j, k in the x, y, and z directions, respectively) is occupied by an electrode, > - for Vb_ijk, f = C_posX * Vb_posX + C_posY * Vb_posY + C_posZ * Vb_posZ > - (C_posX + C_posY + C_posZ + C_negX + C_negY + C_negZ - C_e) * Vb_ijk > + C_negX * Vb_negX + C_negY * Vb_negY + C_negZ * Vb_negZ > - C_e * Ve, > where all C's are calculated constants that do not depend on the other degrees of freedom, but do depend on the type of the neighbouring block in their assigned directions. Then, > the Jacobian entries for Vb_ijk are > df/dVb_negZ = C_negZ (col.i = i, col.j = j, col.k = k - 1) > df/dVb_negY = C_negY (col.i = i, col.j = j - 1, col.k = k) > df/dVb_negX = C_negX (col.i = i - 1, col.j = j, col.k = k) > df/dVb_ijk = - (C_posX + C_posY + C_posZ + C_negX + C_negY + C_negZ - C_e) (col.i = i, col.j = j, col.k = k) > df/dVb_posX = C_posX (col.i = i + 1, col.j = j, col.k = k) > df/dVb_posY = C_posY (col.i = i, col.j = j + 1, col.k = k) > df/dVb_posZ = C_posZ (col.i = i, col.j = j, col.k = k + 1) > For each of these, row.i = i, row.j = j, row.k = k, and row.c = DOF_VB (the DOF index for Vb). Also, col.c = DOF_VB. > The only cross term for the Vb_ijk function is > df/dVe_ijk = -C_e (row.i = i, row.j = j, row.k = k, row.c = DOF_VB, col.i = i, col.j = j, col.k = k col.c = DOF_VE) > > - for Ve_ijk, f = 0.0 + 0.0*i, and all corresponding Jacobian entries are zero as well. Ve_ijk is set to the maximum voltage for each electrode when the initial solution is set. Since the magnitude of Ve is constrained rather than Ve itself, I believe this is correct, but could this be why Ve does not change as it should when the problem is being solved? > > - for Vmag_ijk, f = abs(Ve_ijk) - abs(Ve_ijk_max). The idea was to provide some 'pressure' towards Ve = Ve_max. The corresponding Jacobian entries are > df/dVmag_ijk = 1.0 + 0.0*i (row.i = i, row.j = j, row.k = k, row.c = DOF_VMAG, col.i = i, col.j = j, col.k = k, col.c = DOF_VMAG) > df/dVe_ijk = 0.5 * conj(Ve_ijk) / abs(Ve_ijk) (row.i = i, row.j = j, row.k = k, row.c = DOF_VMAG, col.i = i, col.j = j, col.k = k, col.c = DOF_VE) > (However, since d(abs(Ve_ijk))/dVe_ijk does not have satisfied Cauchy-Reimann equations for the entire complex plane, this derivative does not technically exist.) > > - for Imag_ijk, f = abs(iE) - iE_max. Again, the idea was to provide some 'pressure' towards iE = iE_max. iE is calculated by adding iE_block currents for each block occupied by an electrode. iE_block for block i,j,k is found from > iE_block_ijk = aE_ijk * (Ve - Vb_ijk) > where aE_ijk is a constant that does not depend on any degrees of freedom. There is one Ve value per electrode. The Jacobian entries are then > df/dImag_ijk = 1.0 + 0.0*i (row.i = i, row.j = j, row.k = k, row.c = DOF_IMAG, col.i = i, col.j = j, col.k = k, col.c = DOF_IMAG) > df/dVe = 0.5 * aE_sum * conj(iE) / abs(iE) (row.i = i, row.j = j, row.k = k, row.c = DOF_IMAG, col.i = i, col.j = j, col.k = k, col.c = DOF_VE) > df/dVb_ijk = - 0.5 * aE_ijk * conj(iE) / abs(iE) (row.i = i, row.j = j, row.k = k, row.c = DOF_IMAG, col.i = i, col.j = j, col.k = k, col.c = DOF_VB) > df/dVb_ij(k-1) = - 0.5 * aE_ij(k-1) * conj(iE) / abs(iE) (row.i = i, row.j = j, row.k = k, row.c = DOF_IMAG, col.i = i, col.j = j, col.k = k-1, col.c = DOF_VB) > df/dVb_ij(k-2) = - 0.5 * aE_ij(k-2) * conj(iE) / abs(iE) (row.i = i, row.j = j, row.k = k, row.c = DOF_IMAG, col.i = i, col.j = j, col.k = k-2, col.c = DOF_VB) > > If a block (with coordinates i, j, k) is not occupied by an electrode, C_e is set to zero in the expression for Vb_ijk and all function and Jacobian entries for Ve, Vmag, and Imag are either not set or set to zero. > > My SNES_VI constraints are set as follows for every block: > xl_ijk_DOF_V = -SNES_VI_INF > xu_ijk_DOF_V = SNES_VI_INF > xl_ijk_DOF_VE = -SNES_VI_INF > xu_ijk_DOF_VE = SNES_VI_INF > xl_ijk_DOF_VMAG = 0.0 > xu_ijk_DOF_VMAG = abs(Ve_max) > xl_ijk_DOF_IMAG = 0.0 > xu_ijk_DOF_IMAG = abs(iE_max) > > > When I run my program with -snes_vi_type rs, the solution converges quickly but Vmag voltages remain the same as Ve_max for every electrode. As a result, Imag currents are far larger than abs(iE_max). Are you able to see from here what the problem might be? Again, my hunches are that the complex derivatives do not work in my Jacobian because they do not exist, or that my choices of f for Ve, Vmag, and Imag are not correct. > > Please let me know if you would like clarification on any portion of my description., and thank you again for all your help with this. > > Jon > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gianmail at gmail.com Wed Sep 28 10:37:30 2011 From: gianmail at gmail.com (Gianluca Meneghello) Date: Wed, 28 Sep 2011 17:37:30 +0200 Subject: [petsc-users] TS for incompressible NS equations In-Reply-To: References: Message-ID: Jed, thanks for your reply. I am not sure I understand everything... I do have a constant Jacobian that never changes. What I understand is the matrix I should pass to TSSetIJacobian looks like J + a B together with TSComputeIJacobianConstant. Something like TSSetIFunction(ts,PETSC_NULL,TSComputeIFunctionLinear,PETSC_NULL); TSSetIJacobian(ts,A,A,TSComputeIJacobianConstant,PETSC_NULL); where A = J + a B but then I don't understand how I can fix a before deciding the time integration method or even the time step, so I guess I'm wrong. Is there a way I can use TSComputeIJacobianConstant or am I wrong about trying to use it? Thanks Gianluca On 28 September 2011 15:25, Jed Brown wrote: > On Wed, Sep 28, 2011 at 08:02, Gianluca Meneghello > wrote: >> >> I need to solve the linearized, incompressible Navier Stokes equation >> in time. I have reformulated the problem as >> >> B du/dt = J u >> >> where J is the Jacobian of the Navier Stokes equations and u is a >> vector containing {u,v,,w,p}. >> B is a diagonal matrix which is 0 in the lines corresponding to the >> pressure equation (enforcing the divergence free condition) and >> Dirichlet BC's and 1 otherwise. >> >> Backward Euler would then solve it as >> >> ?u(t+1) = (B-dt J)^-1 B u(t) >> >> as in page 115 of the user guide (is there a typing error there? >> Should it be " (B - dt A) u(n+1) = B u(n) " ?) > > Yes, thank you. I should rewrite this section because it's confusing in its > current form. It would be better to look at the later sections 6.1.2 Solving > Differential Algebraic Equations and 6.1.3 on IMEX methods. > >> >> Using PETSc 3.2, I pass J to the TS object as >> >> >> TSSetRHSFunction(ts,PETSC_NULL,TSComputeRHSFunctionLinear,PETSC_NULL);CHKERRQ(ierr); >> >> TSSetRHSJacobian(ts,A,A,TSComputeRHSJacobianConstant,PETSC_NULL);CHKERRQ(ierr); > > This sets a constant Jacobian that never changes. I don't know if that's > what you mean by linearized Navier-Stokes. > >> >> but I don't understand how I can set B (or whether I am using a wrong >> approach). > > If you have a B that is not the identity, then you should use > TSSetIFunction() and/or TSSetIJacobian(). Note that incompressible flow in > velocity-pressure form is an index 2 DAE, so not all coupled time > integration methods perform well. There is very new code in petsc-dev that > you may want to try for this problem. You have to enable these by > configuring --with-rosw and then you can run with -ts_type rosw > -ts_rosw_type ra34pw2. (This would use a four stage third order Rosenbrock-W > scheme for stiff problems that performed well in the Navier-Stokes test > problems of Rang & Angermann 2005, -ts_rosw_type ra3pw is a three-stage > third order method that is also worth trying.) > The need to configure --with-rosw will go away in a week or two, once that > code stabilizes. -- "[Je pense que] l'homme est un monde qui vaut des fois les mondes et que les plus ardentes ambitions sont celles qui ont eu l'orgueil de l'Anonymat" -- Non omnibus, sed mihi et tibi Amedeo Modigliani From mike.hui.zhang at hotmail.com Wed Sep 28 10:54:02 2011 From: mike.hui.zhang at hotmail.com (Hui Zhang) Date: Wed, 28 Sep 2011 17:54:02 +0200 Subject: [petsc-users] VecScatter in 'add' mode? Message-ID: Hi all, When using PetscErrorCode VecScatterCreate(Vec xin,IS ix,Vec yin,IS iy,VecScatter *newctx) I want to add those values of xin to the original values of yin. Is there a way to do this without copy of yin? Thanks! Hui -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Wed Sep 28 10:58:29 2011 From: knepley at gmail.com (Matthew Knepley) Date: Wed, 28 Sep 2011 15:58:29 +0000 Subject: [petsc-users] VecScatter in 'add' mode? In-Reply-To: References: Message-ID: On Wed, Sep 28, 2011 at 3:54 PM, Hui Zhang wrote: > Hi all, > > When using > > PetscErrorCode VecScatterCreate(Vec xin,IS ix,Vec yin,IS iy,VecScatter *newctx) > > I want to add those values of xin to the original values of yin. > > Is there a way to do this without copy of yin? > > I am not sure what you mean here. Do you want xin == yin? Matt > Thanks! > > Hui > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From mike.hui.zhang at hotmail.com Wed Sep 28 11:05:31 2011 From: mike.hui.zhang at hotmail.com (Hui Zhang) Date: Wed, 28 Sep 2011 18:05:31 +0200 Subject: [petsc-users] VecScatter in 'add' mode? In-Reply-To: References: Message-ID: On Sep 28, 2011, at 5:58 PM, Matthew Knepley wrote: > On Wed, Sep 28, 2011 at 3:54 PM, Hui Zhang wrote: > Hi all, > When using > PetscErrorCode VecScatterCreate(Vec xin,IS ix,Vec yin,IS iy,VecScatter *newctx) > I want to add those values of xin to the original values of yin. > Is there a way to do this without copy of yin? > I am not sure what you mean here. Do you want xin == yin? No, I mean part of xin is going to be added to part of yin. > > Matt > > Thanks! > Hui > -- > What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. > -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Wed Sep 28 11:08:40 2011 From: knepley at gmail.com (Matthew Knepley) Date: Wed, 28 Sep 2011 16:08:40 +0000 Subject: [petsc-users] VecScatter in 'add' mode? In-Reply-To: References: Message-ID: On Wed, Sep 28, 2011 at 4:05 PM, Hui Zhang wrote: > > On Sep 28, 2011, at 5:58 PM, Matthew Knepley wrote: > > On Wed, Sep 28, 2011 at 3:54 PM, Hui Zhang wrote: > >> Hi all, >> >> When using >> >> PetscErrorCode VecScatterCreate(Vec xin,IS ix,Vec yin,IS iy,VecScatter *newctx) >> >> I want to add those values of xin to the original values of yin. >> >> Is there a way to do this without copy of yin? >> >> I am not sure what you mean here. Do you want xin == yin? > > > No, I mean part of xin is going to be added to part of yin. > Alright. We do not make a copy of yin. Matt > Matt > > >> Thanks! >> >> Hui >> >> -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From mike.hui.zhang at hotmail.com Wed Sep 28 11:15:10 2011 From: mike.hui.zhang at hotmail.com (Hui Zhang) Date: Wed, 28 Sep 2011 18:15:10 +0200 Subject: [petsc-users] VecScatter in 'add' mode? In-Reply-To: References: Message-ID: On Sep 28, 2011, at 6:08 PM, Matthew Knepley wrote: > On Wed, Sep 28, 2011 at 4:05 PM, Hui Zhang wrote: > > On Sep 28, 2011, at 5:58 PM, Matthew Knepley wrote: > >> On Wed, Sep 28, 2011 at 3:54 PM, Hui Zhang wrote: >> Hi all, >> When using >> PetscErrorCode VecScatterCreate(Vec xin,IS ix,Vec yin,IS iy,VecScatter *newctx) >> I want to add those values of xin to the original values of yin. >> Is there a way to do this without copy of yin? >> I am not sure what you mean here. Do you want xin == yin? > > No, I mean part of xin is going to be added to part of yin. > > Alright. We do not make a copy of yin. My fault, the question is not clear. I mean 'add' as 'sum'. I want the resultant 'yin' to have values y(iy) = y(iy) + x(ix). > > Matt > > >> Matt >> >> Thanks! >> Hui >> -- >> What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. >> -- Norbert Wiener > > > > > -- > What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. > -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Wed Sep 28 11:16:09 2011 From: knepley at gmail.com (Matthew Knepley) Date: Wed, 28 Sep 2011 16:16:09 +0000 Subject: [petsc-users] VecScatter in 'add' mode? In-Reply-To: References: Message-ID: On Wed, Sep 28, 2011 at 4:15 PM, Hui Zhang wrote: > > On Sep 28, 2011, at 6:08 PM, Matthew Knepley wrote: > > On Wed, Sep 28, 2011 at 4:05 PM, Hui Zhang wrote: > >> >> On Sep 28, 2011, at 5:58 PM, Matthew Knepley wrote: >> >> On Wed, Sep 28, 2011 at 3:54 PM, Hui Zhang wrote: >> >>> Hi all, >>> >>> When using >>> >>> PetscErrorCode VecScatterCreate(Vec xin,IS ix,Vec yin,IS iy,VecScatter *newctx) >>> >>> I want to add those values of xin to the original values of yin. >>> >>> Is there a way to do this without copy of yin? >>> >>> I am not sure what you mean here. Do you want xin == yin? >> >> >> No, I mean part of xin is going to be added to part of yin. >> > > Alright. We do not make a copy of yin. > > > My fault, the question is not clear. I mean 'add' as 'sum'. I want the > resultant 'yin' to have values > y(iy) = y(iy) + x(ix). > Use ADD_VALUES in VecScatterBegin/End() Matt > Matt > > > >> Matt >> >> >>> Thanks! >>> >>> Hui >>> >>> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which their >> experiments lead. >> -- Norbert Wiener >> >> >> > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From mike.hui.zhang at hotmail.com Wed Sep 28 11:22:53 2011 From: mike.hui.zhang at hotmail.com (Hui Zhang) Date: Wed, 28 Sep 2011 18:22:53 +0200 Subject: [petsc-users] VecScatter in 'add' mode? In-Reply-To: References: Message-ID: On Sep 28, 2011, at 6:16 PM, Matthew Knepley wrote: > On Wed, Sep 28, 2011 at 4:15 PM, Hui Zhang wrote: > > On Sep 28, 2011, at 6:08 PM, Matthew Knepley wrote: > >> On Wed, Sep 28, 2011 at 4:05 PM, Hui Zhang wrote: >> >> On Sep 28, 2011, at 5:58 PM, Matthew Knepley wrote: >> >>> On Wed, Sep 28, 2011 at 3:54 PM, Hui Zhang wrote: >>> Hi all, >>> When using >>> PetscErrorCode VecScatterCreate(Vec xin,IS ix,Vec yin,IS iy,VecScatter *newctx) >>> I want to add those values of xin to the original values of yin. >>> Is there a way to do this without copy of yin? >>> I am not sure what you mean here. Do you want xin == yin? >> >> No, I mean part of xin is going to be added to part of yin. >> >> Alright. We do not make a copy of yin. > > My fault, the question is not clear. I mean 'add' as 'sum'. I want the resultant 'yin' to have values > y(iy) = y(iy) + x(ix). > > Use ADD_VALUES in VecScatterBegin/End() Yes, thank you! I should read the manual more carefully. > > Matt > >> Matt >> >> >>> Matt >>> >>> Thanks! >>> Hui >>> -- >>> What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. >>> -- Norbert Wiener >> >> >> >> >> -- >> What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. >> -- Norbert Wiener > > > > > -- > What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. > -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From abhyshr at mcs.anl.gov Wed Sep 28 11:24:11 2011 From: abhyshr at mcs.anl.gov (Shri) Date: Wed, 28 Sep 2011 11:24:11 -0500 (CDT) Subject: [petsc-users] Conditional Constraints In-Reply-To: <096246E5-10D7-44C7-A78E-F2AB0A26A860@mcmillan-mcgee.com> Message-ID: <1500242147.46531.1317227051416.JavaMail.root@zimbra.anl.gov> Jon, > How could I implement this constraint inside the PETSc solution loop? You can set bounds on variables directly through TSVISetVariableBounds() http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-dev/docs/manualpages/TS/TSVISetVariableBounds.html Is there any advantage to using SNESVI when my update step is explicit? Or is there a feature I could use to add a Vmag_new computation after each SNES step, and have this taken into account for the SNES convergence criteria? Unfortunately, we do not have some routine like SNESPostIteration(). However, you do this in your nonlinear function evaluation routine by having two variables for SNES iteration numbers i (current iteration) and i_old (previous iteration) i) Get the current SNES iteration number i ii) If i > i_old then update Vmag_new and i_old. ----- Original Message ----- Hi Shri, I have continued to have little success with my SNESVI formulation of the current constraint. However, working outside of PETSc, I believe I have found an explicit formulation that works well: 1) Set Vmag = Vmag_max and use PETSc to solve for the electric potential distribution (V_ijk is the only degree of freedom). 2) For each electrode, let Vmag_new = Imag_max * Vmag_old / Imag_old. 3) Repeat until Imag is sufficiently close to Imag_max, or until Vmag = Vmag_max with Imag < Imag_max. This process converges in about 5 iterations, which is great, but I would like to incorporate the thermal problem again (including TS) now that this electrical problem is solved. How could I implement this constraint inside the PETSc solution loop? Is there any advantage to using SNESVI when my update step is explicit? Or is there a feature I could use to add a Vmag_new computation after each SNES step, and have this taken into account for the SNES convergence criteria? Thanks again for your continued patience and interest in helping me with this problem. Jon On 2011-09-15, at 12:07 PM, Shri wrote: Jon, > Additionally, the potential distribution does not look correct as it did before (even without SNES reporting convergence)---instead, the magnitude of the distribution > looks like the real part of the correct distribution, if that makes sense. Hmm, in the SNESVI formulation using complex numbers, we have the check real(xlower) <= real(x) <= real(xupper). i am not sure whether this is the cause for such a behavior. I've had conversation with the other PETSc developers to use absolute values for comparison instead of real values, i.e. abs(xl) <= abs(x) <= abs(xupper). By doing this, you can essentially have a function for Ve and its bounds being 0 and Vmag_max. However, the other developers think that absolute values being nonlinear in nature might produce some odd behavior in the solution process, so i've havent' support for it yet. >This asymmetry may be due to the fact that I now only record the Vmag and Imag variables for the top block of each electrode, whereas before I recorded Ve for all >three blocks. Yes, do you want the magnitude and angle for all the three blocks to be same? If so, then probably you'll have to add another constraint. >If a block (with coordinates i, j, k) is not occupied by an electrode , C_e is set to zero in the expression for Vb_ijk and all function and Jacobian entries for Vmag >and Imag are either not set or set to zero. Do you already know the Vmag and Imag values are these blocks? Are they zero/non-zero? I suggest the following for these blocks xl_ijk_DOF_VMAG = Vmag0_ijk xu_ijk_DOF_VMAG = Vmag0_ijk xl_ijk_DOF_IMAG = Imag0_ijk xu_ijk_DOF_IMAG = Imag0_ijk where Vmag0_ijk and Imag0_ijk is the initial voltage and current magnitudes at these blocks. Basically, the idea here is to keep these magnitudes constant and not have the functions for these variables be a part of the solution process. Shri ----- Original Message ----- Hi Shri, Thanks for following up with me. I have made some progress, although my problem is not yet solved. Inspired by your recent suggestions, I realized Ve was redundant as a degree of freedom and did not need to be computed separately from Vmag. I do not want the phase of Ve to be modified by the solution process anyways, so whenever I need the complex value of Ve, I can simply compose it from the Vmag and the constant phase I have recorded for a given electrode. My new function and Jacobian formulations are below if you are interested in seeing them. After I made this change, I began to see the desired behaviour where Vmag decreases when Imag is too high. However, it seems to converge on some magnitude that is maybe 200% too high, and the SNES reports DIVERGED_LINE_SEARCH. Additionally, the potential distribution does not look correct as it did before (even without SNES reporting convergence)---instead, the magnitude of the distribution looks like the real part of the correct distribution, if that makes sense. A further problem relates to the fact that each electrode is three blocks tall. Before I eliminated Ve, each of the three blocks had a very similar voltage and phase. Now, the top block has a different magnitude and phase than the two beneath it. This asymmetry may be due to the fact that I now only record the Vmag and Imag variables for the top block of each electrode, whereas before I recorded Ve for all three blocks. As you can find below, however, I have included Jacobian entries for the lower two block voltages Vb of each electrode, so I cannot see why this strategy would not work. Given your experience with PETSc and Newton's method, do you have any ideas about where these problems might originate? Thank you again, Jon If a block (with coordinates i, j, k in the x, y, and z directions, respectively) is occupied by an electrode , - for Vb_ijk, f = C_posX * Vb_posX + C_posY * Vb_posY + C_posZ * Vb_posZ - (C_posX + C_posY + C_posZ + C_negX + C_negY + C_negZ + C_e) * Vb_ijk + C_negX * Vb_negX + C_negY * Vb_negY + C_negZ * Vb_negZ - C_e * polar(Vmag, arg(Ve)), where all C's are calculated constants that do not depend on the other degrees of freedom, but do depend on the type of the neighbouring block in their assigned directions. Then, the Jacobian entries for Vb_ijk are df/dVb_negZ = C_negZ (col.i = i, col.j = j, col.k = k - 1) df/dVb_negY = C_negY (col.i = i, col.j = j - 1, col.k = k) df/dVb_negX = C_negX (col.i = i - 1, col.j = j, col.k = k) df/dVb_ijk = - (C_posX + C_posY + C_posZ + C_negX + C_negY + C_negZ + C_e) (col.i = i, col.j = j, col.k = k) df/dVb_posX = C_posX (col.i = i + 1, col.j = j, col.k = k) df/dVb_posY = C_posY (col.i = i, col.j = j + 1, col.k = k) df/dVb_posZ = C_posZ (col.i = i, col.j = j, col.k = k + 1) For each of these, row.i = i, row.j = j, row.k = k, and row.c = DOF_VB (the DOF index for Vb). Also, col.c = DOF_VB. The only cross term for the Vb_ijk function is df/dVmag = polar(-C_e, arg(Ve)) (row.i = i, row.j = j, row.k = k, row.c = DOF_VB, col.i = i, col.j = j, col.k = k col.c = DOF_VMAG) - for Vmag_ijk, f = abs(Ve_ijk) - abs(Ve_ijk_max). The idea was to provide some 'pressure' towards Ve = Ve_max. The corresponding Jacobian entries are df/dVmag_ijk = 1.0 + 0.0*i (row.i = i, row.j = j, row.k = k, row.c = DOF_VMAG, col.i = i, col.j = j, col.k = k, col.c = DOF_VMAG) - for Imag_ijk, f = abs(iE) - iE_max. Again, the idea was to provide some 'pressure' towards iE = iE_max. iE is calculated by adding iE_block currents for each block occupied by an electrode. iE_block for block i,j,k is found from iE_block_ijk = aE_ijk * (Ve - Vb_ijk) where aE_ijk is a constant that does not depend on any degrees of freedom. There is one Ve value per electrode. The Jacobian entries are then df/dImag_ijk = 1.0 + 0.0*i (row.i = i, row.j = j, row.k = k, row.c = DOF_IMAG, col.i = i, col.j = j, col.k = k, col.c = DOF_IMAG) df/dVmag = aE_sum * (real(iE) * cos(arg(Ve)) + imag(iE) * sin(arg(Ve)))/ abs(iE) (row.i = i, row.j = j, row.k = k, row.c = DOF_IMAG, col.i = i, col.j = j, col.k = k, col.c = DOF_VMAG) df/dVb_ijk = - 0.5 * aE_ijk * conj(iE) / abs(iE) (row.i = i, row.j = j, row.k = k, row.c = DOF_IMAG, col.i = i, col.j = j, col.k = k, col.c = DOF_VB) df/dVb_ij(k-1) = - 0.5 * aE_ij(k-1) * conj(iE) / abs(iE) (row.i = i, row.j = j, row.k = k, row.c = DOF_IMAG, col.i = i, col.j = j, col.k = k-1, col.c = DOF_VB) df/dVb_ij(k-2) = - 0.5 * aE_ij(k-2) * conj(iE) / abs(iE) (row.i = i, row.j = j, row.k = k, row.c = DOF_IMAG, col.i = i, col.j = j, col.k = k-2, col.c = DOF_VB) (These last three are the only derivatives left that do not technically exist.) If a block (with coordinates i, j, k) is not occupied by an electrode , C_e is set to zero in the expression for Vb_ijk and all function and Jacobian entries for Vmag and Imag are either not set or set to zero. My SNES_VI constraints are set as follows for every block: xl_ijk_DOF_V = -SNES_VI_INF xu_ijk_DOF_V = SNES_VI_INF xl_ijk_DOF_VMAG = 0.0 xu_ijk_DOF_VMAG = abs(Ve_max) xl_ijk_DOF_IMAG = 0.0 xu_ijk_DOF_IMAG = abs(iE_max) On 2011-09-13, at 12:12 PM, Shri wrote: Jon, Any progress with your work? Send us an email once you've sorted out your formulation and we would be glad to help you out with its PETSc implementation. Shri ----- Original Message ----- Shri, Note here that (a) f(Ve_ijk) is a complex function while Imag and Imag_max are real. So the above equation does not hold. Would f(Ve_ijk) = Imag - Imag_max = 0 simply be equivalent to saying, solve the real system of equations Imag - Imag_max = 0 and 0 = 0 (the imaginary part)? Regardless, I am not sure what a correct choice of f(Ve_ijk) would be. (b) In your previous email, you had bounds on Imag, 0 < Imag < Imag_max. Now you want Ve_ijk to increase if Imag > Imag_max and decrease if Imag < Imag_max. Are you trying to find the value of Ve at which the current is maximum, i.e. Imag = Imag_max, by doing this? Yes, that is essentially what I am trying to find. Ve determines Vb, and the difference between the two determines Imag. Vmag (magnitude of Ve) and Imag have the restrictions 0 < Vmag < Vmag_max and 0 < Imag < Imag_max. I want to find either Ve for Imag = Imag_max or Imag for Vmag = Vmag_max. In my test scenario, Vmag_max happens to be quite high so Imag_max will always be reached first, but in general either situation is possible. Thank you again, Jon On 2011-09-08, at 8:34 PM, Shri wrote: Jon, > I do not want Ve to be constant, however; I would like it to either decrease in response to Imag > Imag_max or increase in response to Imag < Imag_max, providing > the condition Vmag <= Vmag_max remains true. > Should I then have something like f(Ve_ijk) = Imag - Imag_max? Note here that (a) f(Ve_ijk) is a complex function while Imag and Imag_max are real. So the above equation does not hold. (b) In your previous email, you had bounds on Imag, 0 < Imag < Imag_max. Now you want Ve_ijk to increase if Imag > Imag_max and decrease if Imag < Imag_max. Are you trying to find the value of Ve at which the current is maximum, i.e. Imag = Imag_max, by doing this? ----- Original Message ----- Shri, Thanks very much for your response. In fact, -snes_converged_reason revealed that the solution was not converging, as you suggested: "Nonlinear solve did not converge due to DIVERGED_LINE_SEARCH." I had not realized this earlier since the voltage distribution in the solution appeared to be correct, even though the SNES_VI constraints were not satisfied. Given that and your explanation, I am sure something is wrong with my choices of f. I do not want Ve to be constant, however; I would like it to either decrease in response to Imag > Imag_max or increase in response to Imag < Imag_max, providing the condition Vmag <= Vmag_max remains true. Should I then have something like f(Ve_ijk) = Imag - Imag_max? Note that you have One more thing from your response: >If a block (with coordinates i, j, k) is not occupied by an electrode , C_e is set to zero in the expression for Vb_ijk and all function and Jacobian entries for Ve, >Vmag, and Imag are either not set or set to zero. Does this mean f(Ve_ijk) = f(Vmag_ijk) = f(Imag_ijk) = 0? If so, then Ve_ijk, Vmag_ijk, and Imag_ijk would retain their initial values for all time steps. In my present scheme, f(Ve_ijk) = f(Vmag_ijk) = 0 and f(Imag_ijk) is non-zero for electrode blocks; all three are zero for non-electrode blocks. Depending on the choice of initial conditions, I can make f(Vmag_ijk) non-zero as well (until Ve changes to match Ve_max). Thanks again, Jon On 2011-09-08, at 3:42 PM, Shri wrote: Jon, > for Ve_ijk, f = 0.0 + 0.0*i, and all corresponding Jacobian entries are zero as well. Ve_ijk is set to the maximum voltage for each electrode when the initial solution >is - set. Since the magnitude of Ve is constrained rather than Ve itself, I believe this is correct, but could this be why Ve does not change as it should when the >problem is - being solved? This f(Ve_ijk) seems very odd to me since normally you would always have f(some/all variables) = 0 as the function. If f=0.0+0.0*i for all steps, then you don't need a seperate variable Ve_ijk, you can treat it as a constant since Ve_ijk will not change its value during the solutions. Moroever, the Jacobian rows for df(Ve_ijk)/d(all variables) would be all zeros which implies a singular matrix. I am not sure how the solution converged for this case. Were you using -pc_type lu with some shift or some external direct solver such as superlu? What do you get for -snes_monitor -snes_converged_reason? > Ve_ijk is set to the maximum voltage for each electrode when the initial solution is set. Since the magnitude of Ve is constrained rather than Ve itself. > for Vmag_ijk, f = abs(Ve_ijk) - abs(Ve_ijk_max) Since Ve_ijk = Ve_ijk_max, f for Vmag_ijk = 0.0 and hence Vmag_ijk also will also retain its initial value throughout. >If a block (with coordinates i, j, k) is not occupied by an electrode , C_e is set to zero in the expression for Vb_ijk and all function and Jacobian entries for Ve, >Vmag, and Imag are either not set or set to zero. Does this mean f(Ve_ijk) = f(Vmag_ijk) = f(Imag_ijk) = 0? If so, then Ve_ijk, Vmag_ijk, and Imag_ijk would retain their initial values for all time steps. > When I run my program with -snes_vi_type rs, the solution converges quickly but Vmag voltages remain the same as Ve_max for every electrode. As a result, Imag > currents are far larger than abs(iE_max). Seems to me like the problem is not converging, or giving the incorrect solution, since the solution of a variational inequality problem has to satisfy Imag < iE_max, which in your case is not happening. Thanks for the detailed explanation. Would you mind if I (or you) forward this to petsc-users list so that others could share their thoughts too. Shri ----- Original Message ----- Hi Shri, Thanks again for your patience with my being away this past month. I have spent some more time with this problem and I have added Jacobian entries for 'cross terms' such as d(f(Vb)) / d(Ve). However, this did not appear to change the solution arrived at by PETSc. I will give you some examples of the f functions and corresponding Jacobian entries as you suggested, with the hope you will be able to see where I may be going wrong. I will use Vb = V_block, Ve = V_electrode, Vmag = magnitude(V_electrode), Imag = magnitude(I_electrode) to name the four degrees of freedom. In my 3D finite differencing grid, there are two types of blocks: blocks occupied by electrodes, and blocks occupied by a resistive medium. The block type of the current block and its neighbouring blocks determines the form of f. This is complicated by the fact that individual electrodes generally occupy three vertically-contiguous blocks, and I want to limit the electric current flowing from the whole electrode rather than the current through each block. Note that Ve, Vmag, and Imag are thus computed or recorded only for the top block of each electrode. If a block (with coordinates i, j, k in the x, y, and z directions, respectively) is occupied by an electrode , - for Vb_ijk, f = C_posX * Vb_posX + C_posY * Vb_posY + C_posZ * Vb_posZ - (C_posX + C_posY + C_posZ + C_negX + C_negY + C_negZ - C_e) * Vb_ijk + C_negX * Vb_negX + C_negY * Vb_negY + C_negZ * Vb_negZ - C_e * Ve, where all C's are calculated constants that do not depend on the other degrees of freedom, but do depend on the type of the neighbouring block in their assigned directions. Then, the Jacobian entries for Vb_ijk are df/dVb_negZ = C_negZ (col.i = i, col.j = j, col.k = k - 1) df/dVb_negY = C_negY (col.i = i, col.j = j - 1, col.k = k) df/dVb_negX = C_negX (col.i = i - 1, col.j = j, col.k = k) df/dVb_ijk = - (C_posX + C_posY + C_posZ + C_negX + C_negY + C_negZ - C_e) (col.i = i, col.j = j, col.k = k) df/dVb_posX = C_posX (col.i = i + 1, col.j = j, col.k = k) df/dVb_posY = C_posY (col.i = i, col.j = j + 1, col.k = k) df/dVb_posZ = C_posZ (col.i = i, col.j = j, col.k = k + 1) For each of these, row.i = i, row.j = j, row.k = k, and row.c = DOF_VB (the DOF index for Vb). Also, col.c = DOF_VB. The only cross term for the Vb_ijk function is df/dVe_ijk = -C_e (row.i = i, row.j = j, row.k = k, row.c = DOF_VB, col.i = i, col.j = j, col.k = k col.c = DOF_VE) - for Ve_ijk, f = 0.0 + 0.0*i, and all corresponding Jacobian entries are zero as well. Ve_ijk is set to the maximum voltage for each electrode when the initial solution is set. Since the magnitude of Ve is constrained rather than Ve itself, I believe this is correct, but could this be why Ve does not change as it should when the problem is being solved? - for Vmag_ijk, f = abs(Ve_ijk) - abs(Ve_ijk_max). The idea was to provide some 'pressure' towards Ve = Ve_max. The corresponding Jacobian entries are df/dVmag_ijk = 1.0 + 0.0*i (row.i = i, row.j = j, row.k = k, row.c = DOF_VMAG, col.i = i, col.j = j, col.k = k, col.c = DOF_VMAG) df/dVe_ijk = 0.5 * conj(Ve_ijk) / abs(Ve_ijk) (row.i = i, row.j = j, row.k = k, row.c = DOF_VMAG, col.i = i, col.j = j, col.k = k, col.c = DOF_VE) (However, since d(abs(Ve_ijk))/dVe_ijk does not have satisfied Cauchy-Reimann equations for the entire complex plane, this derivative does not technically exist.) - for Imag_ijk, f = abs(iE) - iE_max. Again, the idea was to provide some 'pressure' towards iE = iE_max. iE is calculated by adding iE_block currents for each block occupied by an electrode. iE_block for block i,j,k is found from iE_block_ijk = aE_ijk * (Ve - Vb_ijk) where aE_ijk is a constant that does not depend on any degrees of freedom. There is one Ve value per electrode. The Jacobian entries are then df/dImag_ijk = 1.0 + 0.0*i (row.i = i, row.j = j, row.k = k, row.c = DOF_IMAG, col.i = i, col.j = j, col.k = k, col.c = DOF_IMAG) df/dVe = 0.5 * aE_sum * conj(iE) / abs(iE) (row.i = i, row.j = j, row.k = k, row.c = DOF_IMAG, col.i = i, col.j = j, col.k = k, col.c = DOF_VE) df/dVb_ijk = - 0.5 * aE_ijk * conj(iE) / abs(iE) (row.i = i, row.j = j, row.k = k, row.c = DOF_IMAG, col.i = i, col.j = j, col.k = k, col.c = DOF_VB) df/dVb_ij(k-1) = - 0.5 * aE_ij(k-1) * conj(iE) / abs(iE) (row.i = i, row.j = j, row.k = k, row.c = DOF_IMAG, col.i = i, col.j = j, col.k = k-1, col.c = DOF_VB) df/dVb_ij(k-2) = - 0.5 * aE_ij(k-2) * conj(iE) / abs(iE) (row.i = i, row.j = j, row.k = k, row.c = DOF_IMAG, col.i = i, col.j = j, col.k = k-2, col.c = DOF_VB) If a block (with coordinates i, j, k) is not occupied by an electrode , C_e is set to zero in the expression for Vb_ijk and all function and Jacobian entries for Ve, Vmag, and Imag are either not set or set to zero. My SNES_VI constraints are set as follows for every block: xl_ijk_DOF_V = -SNES_VI_INF xu_ijk_DOF_V = SNES_VI_INF xl_ijk_DOF_VE = -SNES_VI_INF xu_ijk_DOF_VE = SNES_VI_INF xl_ijk_DOF_VMAG = 0.0 xu_ijk_DOF_VMAG = abs(Ve_max) xl_ijk_DOF_IMAG = 0.0 xu_ijk_DOF_IMAG = abs(iE_max) When I run my program with -snes_vi_type rs, the solution converges quickly but Vmag voltages remain the same as Ve_max for every electrode. As a result, Imag currents are far larger than abs(iE_max). Are you able to see from here what the problem might be? Again, my hunches are that the complex derivatives do not work in my Jacobian because they do not exist, or that my choices of f for Ve, Vmag, and Imag are not correct. Please let me know if you would like clarification on any portion of my description., and thank you again for all your help with this. Jon -------------- next part -------------- An HTML attachment was scrubbed... URL: From jedbrown at mcs.anl.gov Wed Sep 28 14:14:40 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Wed, 28 Sep 2011 14:14:40 -0500 Subject: [petsc-users] TS for incompressible NS equations In-Reply-To: References: Message-ID: On Wed, Sep 28, 2011 at 10:37, Gianluca Meneghello wrote: > I do have a constant Jacobian that never changes. What I understand is > the matrix I should pass to TSSetIJacobian looks like > > J + a B > > together with TSComputeIJacobianConstant. Something like > > > TSSetIFunction(ts,PETSC_NULL,TSComputeIFunctionLinear,PETSC_NULL); > TSSetIJacobian(ts,A,A,TSComputeIJacobianConstant,PETSC_NULL); > I may have made a mistake by introducing TSComputeIJacobianConstant() at all since it only works for constant time steps. > > where A = J + a B > > but then I don't understand how I can fix a before deciding the time > integration method or even the time step, so I guess I'm wrong. > You can cache J and/or B in your application context and add them together with the variable shift. PETSc does not store the matrices J and B because they are not part of the solution process, just an artifact of how some people choose to write the system. Their existence is entirely unnecessary, and I would encourage you to benchmark before concluding that it's worth storing them separately. It may be more cost-effective to just recompute A on the fly whenever it is requested. You can choose to do this only when 'a' changes, which might be infrequent or never depending on how the time integration method and adaptive controller. And do you really mean to solve transient linearized Navier-Stokes (constant "wind")? I don't see any way this makes sense physically. For nonlinear Navier-Stokes, the matrix J (which contains everything that is potentially expensive to build) is changing on each time step. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mmnasr at gmail.com Wed Sep 28 20:44:37 2011 From: mmnasr at gmail.com (Mohamad M. Nasr-Azadani) Date: Wed, 28 Sep 2011 18:44:37 -0700 Subject: [petsc-users] Memory usage of KSP (or any PETSc object) In-Reply-To: References: Message-ID: Thanks Jed, To get this right, I thought I should get a zero value for the code below: * PetscLogDouble mem1, mem2;* * PetscMemoryGetCurrentUsage(&mem1);* * PetscMemoryGetCurrentUsage(&mem2);* * printf("mem1:%f mem2:%f memory:%d\n", mem1, mem2, (int)(mem2 - mem1));* * What I get is not zero! * * "mem1:3047424.000000 mem2:3076096.000000 memory:28672" I ran the code with/without the runtime flag -malloc. Also, PETSc is complied without debugging. I am confused. Apparently, even when I allocate some memory using PetscMalloc() or malloc() between the two PetscMemoryGetCurrentUsage(&mem1); /* some memory allocation using malloc or PescMalloc() PetscMemoryGetCurrentUsage(&mem2); I still get the same number. Thanks for your help, Mohamad * On Wed, Sep 28, 2011 at 3:26 AM, Jed Brown wrote: > On Tue, Sep 27, 2011 at 23:29, Mohamad M. Nasr-Azadani wrote: > >> I am a bit puzzled by how this function works. >> I was trying it on this simple example: >> >> PetscMemoryGetMaximumUsage(&mem1); >> >> PetscMalloc(1*sizeof(double), &aa); >> >> PetscMemoryGetMaximumUsage(&mem2); >> >> >> The difference I get for (mem2-mem1) is zero. >> > > The maximum usage is actually only updated when objects are destroyed. You > can call PetscMemoryGetCurrentUsage() to see how much is used by everything > used by the application (not just PETSc). When using tracing malloc (default > in debug mode or when run with -malloc). You can use > PetscMallocGetCurrentUsage() and PetscMallocGetMaximumUsage() at any time > and it should be a current measure of what PETSc has allocated (not just > after destroying objects), but does not measure anything allocated directly > by the application (e.g. by calling malloc directly, C++ new, Fortran > arrays). > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jedbrown at mcs.anl.gov Wed Sep 28 21:10:02 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Wed, 28 Sep 2011 21:10:02 -0500 Subject: [petsc-users] Memory usage of KSP (or any PETSc object) In-Reply-To: References: Message-ID: On Wed, Sep 28, 2011 at 20:44, Mohamad M. Nasr-Azadani wrote: > Thanks Jed, > > To get this right, I thought I should get a zero value for the code below: > > * PetscLogDouble mem1, mem2;* > > * PetscMemoryGetCurrentUsage(&mem1);* > > * PetscMemoryGetCurrentUsage(&mem2);* > > * printf("mem1:%f mem2:%f memory:%d\n", mem1, mem2, (int)(mem2 - mem1));* > > * > > > What I get is not zero! > * > > * > > "mem1:3047424.000000 mem2:3076096.000000 memory:28672" > * > > So it's important to understand what this function means. It reports the resident set size for the process. This is not the amount of memory malloced (malloc doesn't allocate memory, it just promises to find it if you eventually "fault" it) nor the amount of memory your process is using (pages could be swapped out, thus not resident; read-only segments can be shared with other processes and are not counted). Calling malloc and faulting the memory you get back does not necessarily change the resident size for your application because the malloc implementation might find it already mapped and not need to allocate. This is not a PETSc issue at all, it's just the way operating systems work. I realize that your profiling would be easier if operating systems had the semantics that you seem to be expecting, but that would cause far more problems, so they don't work that way. You have to live with this or ask a different question of the system. > * > > > > I ran the code with/without the runtime flag -malloc. Also, PETSc is complied without debugging. > I am confused. Apparently, even when I allocate some memory using PetscMalloc() or malloc() between the two > > PetscMemoryGetCurrentUsage(&mem1); > > /* some memory allocation using malloc or PescMalloc() > > PetscMemoryGetCurrentUsage(&mem2); > > > I still get the same number. > * > > Use PetscMallocGetCurrentUsage() in this case. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mmnasr at gmail.com Wed Sep 28 21:33:29 2011 From: mmnasr at gmail.com (Mohamad M. Nasr-Azadani) Date: Wed, 28 Sep 2011 19:33:29 -0700 Subject: [petsc-users] Memory usage of KSP (or any PETSc object) In-Reply-To: References: Message-ID: Thanks Jed for your detailed answer. That is more clear now. So, the most accurate way to find the size of a PETSc object would be to use *PetscMallocGetCurrentUsage()* * * before and after the block that creation and/or setting values happens? The reason that I am asking this is I am interested in finding out the size of KSP solver with BoomerAMG preconditioner. I assume the hypre package does not use PetscMalloc() and therefore the returned memory size could be not the most accurate? Thanks again, Mohamad * * * * * * On Wed, Sep 28, 2011 at 7:10 PM, Jed Brown wrote: > On Wed, Sep 28, 2011 at 20:44, Mohamad M. Nasr-Azadani wrote: > >> Thanks Jed, >> >> To get this right, I thought I should get a zero value for the code >> below: >> >> * PetscLogDouble mem1, mem2;* >> >> * PetscMemoryGetCurrentUsage(&mem1);* >> >> * PetscMemoryGetCurrentUsage(&mem2);* >> >> * printf("mem1:%f mem2:%f memory:%d\n", mem1, mem2, (int)(mem2 - mem1));* >> >> * >> >> >> What I get is not zero! >> * >> >> * >> >> >> >> "mem1:3047424.000000 mem2:3076096.000000 memory:28672" >> * >> >> > So it's important to understand what this function means. It reports the > resident set size for the process. This is not the amount of memory malloced > (malloc doesn't allocate memory, it just promises to find it if you > eventually "fault" it) nor the amount of memory your process is using (pages > could be swapped out, thus not resident; read-only segments can be shared > with other processes and are not counted). Calling malloc and faulting the > memory you get back does not necessarily change the resident size for your > application because the malloc implementation might find it already mapped > and not need to allocate. > > This is not a PETSc issue at all, it's just the way operating systems work. > I realize that your profiling would be easier if operating systems had the > semantics that you seem to be expecting, but that would cause far more > problems, so they don't work that way. You have to live with this or ask a > different question of the system. > > >> * >> >> >> >> >> I ran the code with/without the runtime flag -malloc. Also, PETSc is complied without debugging. >> I am confused. Apparently, even when I allocate some memory using PetscMalloc() or malloc() between the two >> >> >> >> PetscMemoryGetCurrentUsage(&mem1); >> >> /* some memory allocation using malloc or PescMalloc() >> >> PetscMemoryGetCurrentUsage(&mem2); >> >> >> I still get the same number. >> * >> >> > Use PetscMallocGetCurrentUsage() in this case. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gshy2014 at gmail.com Wed Sep 28 22:46:51 2011 From: gshy2014 at gmail.com (Shiyuan) Date: Wed, 28 Sep 2011 22:46:51 -0500 Subject: [petsc-users] direct access of seqcusp Vec Message-ID: Hi, How can we acess the data of a seqcusp Vec directly? Is it stored as consecutive 1d array in the device momery? Thanks. Shiyuan -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Wed Sep 28 23:22:02 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Wed, 28 Sep 2011 23:22:02 -0500 Subject: [petsc-users] direct access of seqcusp Vec In-Reply-To: References: Message-ID: <1CFA4EF7-E716-4942-BE0F-6AB2E4EF78E5@mcs.anl.gov> VecCUSPGetArrayReadWrite() gives you access to the CUSP array that lives on the device memory. You can then do things to it using the CUSP header file definitions Barry On Sep 28, 2011, at 10:46 PM, Shiyuan wrote: > Hi, > How can we acess the data of a seqcusp Vec directly? Is it stored as consecutive 1d array in the device momery? > Thanks. > > Shiyuan From balay at mcs.anl.gov Wed Sep 28 23:44:39 2011 From: balay at mcs.anl.gov (Satish Balay) Date: Wed, 28 Sep 2011 23:44:39 -0500 (CDT) Subject: [petsc-users] Memory usage of KSP (or any PETSc object) In-Reply-To: References: Message-ID: On Wed, 28 Sep 2011, Mohamad M. Nasr-Azadani wrote: > Thanks Jed for your detailed answer. That is more clear now. > So, the most accurate way to find the size of a PETSc object would be to > use > > *PetscMallocGetCurrentUsage()* > * > * > before and after the block that creation and/or setting values happens? The > reason that I am asking this is I am interested in finding out the size of > KSP solver with BoomerAMG preconditioner. I assume the hypre package does > not use PetscMalloc() and therefore the returned memory size could be not > the most accurate? Yes - the above call provides info regarding memory allocated by PetscMalloc() - so memory allocated in externalpackages isn't captured by it. Satish From behzad.baghapour at gmail.com Thu Sep 29 06:24:22 2011 From: behzad.baghapour at gmail.com (behzad baghapour) Date: Thu, 29 Sep 2011 14:54:22 +0330 Subject: [petsc-users] Matrix Reordering Message-ID: Dear all, How can I use the result ( rperm and cperm ) of MatGetOrdering( Mat mat, const MatOrderingType type, IS *rperm, IS *cperm ) as an array of integers in order to permute my own Matrix? Thanks a lot, Behzad -- ================================== Behzad Baghapour Ph.D. Candidate, Mechecanical Engineering University of Tehran, Tehran, Iran https://sites.google.com/site/behzadbaghapour Fax: 0098-21-88020741 ================================== -------------- next part -------------- An HTML attachment was scrubbed... URL: From matijakecman at gmail.com Thu Sep 29 06:28:58 2011 From: matijakecman at gmail.com (Matija Kecman) Date: Thu, 29 Sep 2011 12:28:58 +0100 Subject: [petsc-users] Question about ksp ex3.c In-Reply-To: References: Message-ID: Thanks for your response Jed! I've been doing some other investigations using this example. I made some small modifications: 1. Added preallocation as Jed Brown suggested in a previous email (http://lists.mcs.anl.gov/pipermail/petsc-users/2011-June/009054.html). 2. Added a small VTK viewer. 3. Set the initial guess to zero. 4. Changed the entries in the element stiffness matrix to the following: ? Ke[ 0] =? 2./3.; Ke[ 1] = -1./6.; Ke[ 2] = -1./3.; Ke[ 3] = -1./6.; ? Ke[ 4] = -1./6.; Ke[ 5] =? 2./3.; Ke[ 6] = -1./6.; Ke[ 7] = -1./3.; ? Ke[ 8] = -1./3.; Ke[ 9] = -1./6.; Ke[10] =? 2./3.; Ke[11] = -1./6.; ? Ke[12] = -1./6.; Ke[13] = -1./3.; Ke[14] = -1./6.; Ke[15] =? 2./3.; I computed these by evaluating $K^e_{ij} = \int_{\Omega_e} \nabla \psi^e_i \cdot \nabla \psi^e_j \, \mathrm{d}\Omega$ with the shape functions $\psi^e$ corresponding to a bilinear quadratic finite element denoted by $\Omega_e$. This is different to what was originally in the code and I'm not sure where the original code comes from. This isn't important so you can just ignore it if you like, I get the same solution using both matrices. --- I am running on a two compute node clusters which each look as follows: 2 quad-core Intel Xeon 5345 processors, 16GB memory. The node clusters are connected with the following interconnect: Mellanox InfiniScale 2400. I computed my results using a machine file which specifies that (up to) the first 8 processes are computed on node1 and the second group of 8 processes are computed on node2. My timing results are shown in the table below, I'm running each test using Bi-CGStab with no preconditioning (-ksp_type bcgs -pc_type none) on a computational grid of 800 x 800 cells, so 641601 DOFs. I have attached my modified source code (you could look at my changes using diff) and the -log_summary output for each of the tests. # number of processes | time for KSPsolve() | iterations to convergence | norm of error 1 64.008 692 0.00433961 2 36.2767 626 0.00611835 4 35.9989 760 0.00311053 8 30.5215 664 0.00599148 16 14.1164 710 0.00792162 Why is the scaling so poor? I have read the FAQ (http://www.mcs.anl.gov/petsc/petsc-as/documentation/faq.html#computers), am I experiencing the problem described? I think my machine has a bandwidth of 2GB/s per process as suggested. Also, how can you tell if a computation is memory bound by looking at the -log_summary? Many thanks, Matija On Tue, Sep 20, 2011 at 11:44 AM, Jed Brown wrote: > > On Tue, Sep 20, 2011 at 11:45, Matija Kecman wrote: >> >> $ mpirun -np 1 ./ex3 -ksp_type gmres -pc_type none -m 100 >> Norm of error 0.570146 Iterations 0 > > This uses a nonzero initial guess so the initial residual norm is compared to the right hand side. > $ ./ex3 -ksp_type gmres -ksp_monitor -m 100 -pc_type none -ksp_converged_reason -info |grep Converged > [0] KSPDefaultConverged(): user has provided nonzero initial guess, computing 2-norm of preconditioned RHS > [0] KSPDefaultConverged(): Linear solver has converged. Residual norm 1.113646413065e-04 is less than relative tolerance 1.000000000000e-05 times initial right hand side norm 1.291007358616e+01 at iteration 0 > You can use the true residual, it just costs something so it's not enabled by default: > $ ./ex3 -ksp_type gmres -ksp_monitor -m 100 -pc_type none -ksp_converged_reason -ksp_converged_use_initial_residual_norm > [many iterations] > Linear solve converged due to CONVERGED_RTOL iterations 1393 > Norm of error 0.000664957 Iterations 1393 -------------- next part -------------- A non-text attachment was scrubbed... Name: ex3_log_summary_1process Type: application/octet-stream Size: 11235 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ex3_log_summary_2process Type: application/octet-stream Size: 11689 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ex3_log_summary_4process Type: application/octet-stream Size: 11690 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ex3_log_summary_8process Type: application/octet-stream Size: 11690 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ex3_log_summary_16process Type: application/octet-stream Size: 11692 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ex3_modified.c Type: text/x-csrc Size: 13378 bytes Desc: not available URL: From bsmith at mcs.anl.gov Thu Sep 29 07:40:14 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Thu, 29 Sep 2011 07:40:14 -0500 Subject: [petsc-users] Matrix Reordering In-Reply-To: References: Message-ID: <563041AC-B41D-4245-9533-EEFAF98A2891@mcs.anl.gov> On Sep 29, 2011, at 6:24 AM, behzad baghapour wrote: > Dear all, > > How can I use the result ( rperm and cperm ) of > > MatGetOrdering( Mat mat, const MatOrderingType type, IS *rperm, IS *cperm ) > > as an array of integers in order to permute my own Matrix? MatPermute() gives you a new matrix in the new ordering. But note that mostly these routines are used internally and automatically by the -pc_type lu direct solver. You can use ISGetIndices() to get back out of the IS the raw list of integers that you can use in any way you like, Barry > > Thanks a lot, > Behzad > -- > ================================== > Behzad Baghapour > Ph.D. Candidate, Mechecanical Engineering > University of Tehran, Tehran, Iran > https://sites.google.com/site/behzadbaghapour > Fax: 0098-21-88020741 > ================================== > From bsmith at mcs.anl.gov Thu Sep 29 07:42:08 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Thu, 29 Sep 2011 07:42:08 -0500 Subject: [petsc-users] Question about ksp ex3.c In-Reply-To: References: Message-ID: <52CA5812-4BBF-4961-8BA5-891859047A67@mcs.anl.gov> If you are using petsc-3.2 you can also run "make streams" in the $PETSC_DIR and that will run the streams benchmark giving you a pretty good idea of how the memory bandwidth scales with the number of cores. If the streams benchmark does not scale with the cores then no iterative solver will scale with more cores. Barry On Sep 29, 2011, at 6:28 AM, Matija Kecman wrote: > Thanks for your response Jed! I've been doing some other > investigations using this example. I made some small modifications: > > 1. Added preallocation as Jed Brown suggested in a previous email > (http://lists.mcs.anl.gov/pipermail/petsc-users/2011-June/009054.html). > 2. Added a small VTK viewer. > 3. Set the initial guess to zero. > 4. Changed the entries in the element stiffness matrix to the following: > > Ke[ 0] = 2./3.; Ke[ 1] = -1./6.; Ke[ 2] = -1./3.; Ke[ 3] = -1./6.; > Ke[ 4] = -1./6.; Ke[ 5] = 2./3.; Ke[ 6] = -1./6.; Ke[ 7] = -1./3.; > Ke[ 8] = -1./3.; Ke[ 9] = -1./6.; Ke[10] = 2./3.; Ke[11] = -1./6.; > Ke[12] = -1./6.; Ke[13] = -1./3.; Ke[14] = -1./6.; Ke[15] = 2./3.; > > I computed these by evaluating $K^e_{ij} = \int_{\Omega_e} \nabla > \psi^e_i \cdot \nabla \psi^e_j \, \mathrm{d}\Omega$ with the shape > functions $\psi^e$ corresponding to a bilinear quadratic finite > element denoted by $\Omega_e$. This is different to what was > originally in the code and I'm not sure where the original code comes > from. This isn't important so you can just ignore it if you like, I > get the same solution using both matrices. > > --- > > I am running on a two compute node clusters which each look as > follows: 2 quad-core Intel Xeon 5345 processors, 16GB memory. The node > clusters are connected with the following interconnect: Mellanox > InfiniScale 2400. I computed my results using a machine file which > specifies that (up to) the first 8 processes are computed on node1 and > the second group of 8 processes are computed on node2. My timing > results are shown in the table below, I'm running each test using > Bi-CGStab with no preconditioning (-ksp_type bcgs -pc_type none) on a > computational grid of 800 x 800 cells, so 641601 DOFs. I have attached > my modified source code (you could look at my changes using diff) and > the -log_summary output for each of the tests. > > # number of processes | time for KSPsolve() | iterations to > convergence | norm of error > 1 64.008 692 0.00433961 > 2 36.2767 626 0.00611835 > 4 35.9989 760 0.00311053 > 8 30.5215 664 0.00599148 > 16 14.1164 710 0.00792162 > > Why is the scaling so poor? I have read the FAQ > (http://www.mcs.anl.gov/petsc/petsc-as/documentation/faq.html#computers), > am I experiencing the problem described? I think my machine has a > bandwidth of 2GB/s per process as suggested. Also, how can you tell if > a computation is memory bound by looking at the -log_summary? > > Many thanks, > > Matija > > On Tue, Sep 20, 2011 at 11:44 AM, Jed Brown wrote: >> >> On Tue, Sep 20, 2011 at 11:45, Matija Kecman wrote: >>> >>> $ mpirun -np 1 ./ex3 -ksp_type gmres -pc_type none -m 100 >>> Norm of error 0.570146 Iterations 0 >> >> This uses a nonzero initial guess so the initial residual norm is compared to the right hand side. >> $ ./ex3 -ksp_type gmres -ksp_monitor -m 100 -pc_type none -ksp_converged_reason -info |grep Converged >> [0] KSPDefaultConverged(): user has provided nonzero initial guess, computing 2-norm of preconditioned RHS >> [0] KSPDefaultConverged(): Linear solver has converged. Residual norm 1.113646413065e-04 is less than relative tolerance 1.000000000000e-05 times initial right hand side norm 1.291007358616e+01 at iteration 0 >> You can use the true residual, it just costs something so it's not enabled by default: >> $ ./ex3 -ksp_type gmres -ksp_monitor -m 100 -pc_type none -ksp_converged_reason -ksp_converged_use_initial_residual_norm >> [many iterations] >> Linear solve converged due to CONVERGED_RTOL iterations 1393 >> Norm of error 0.000664957 Iterations 1393 > From knepley at gmail.com Thu Sep 29 07:44:53 2011 From: knepley at gmail.com (Matthew Knepley) Date: Thu, 29 Sep 2011 12:44:53 +0000 Subject: [petsc-users] Question about ksp ex3.c In-Reply-To: References: Message-ID: On Thu, Sep 29, 2011 at 11:28 AM, Matija Kecman wrote: > Thanks for your response Jed! I've been doing some other > investigations using this example. I made some small modifications: > > 1. Added preallocation as Jed Brown suggested in a previous email > (http://lists.mcs.anl.gov/pipermail/petsc-users/2011-June/009054.html). > 2. Added a small VTK viewer. > 3. Set the initial guess to zero. > 4. Changed the entries in the element stiffness matrix to the following: > > Ke[ 0] = 2./3.; Ke[ 1] = -1./6.; Ke[ 2] = -1./3.; Ke[ 3] = -1./6.; > Ke[ 4] = -1./6.; Ke[ 5] = 2./3.; Ke[ 6] = -1./6.; Ke[ 7] = -1./3.; > Ke[ 8] = -1./3.; Ke[ 9] = -1./6.; Ke[10] = 2./3.; Ke[11] = -1./6.; > Ke[12] = -1./6.; Ke[13] = -1./3.; Ke[14] = -1./6.; Ke[15] = 2./3.; > > I computed these by evaluating $K^e_{ij} = \int_{\Omega_e} \nabla > \psi^e_i \cdot \nabla \psi^e_j \, \mathrm{d}\Omega$ with the shape > functions $\psi^e$ corresponding to a bilinear quadratic finite > element denoted by $\Omega_e$. This is different to what was > originally in the code and I'm not sure where the original code comes > from. This isn't important so you can just ignore it if you like, I > get the same solution using both matrices. > > --- > > I am running on a two compute node clusters which each look as > follows: 2 quad-core Intel Xeon 5345 processors, 16GB memory. The node > clusters are connected with the following interconnect: Mellanox > InfiniScale 2400. I computed my results using a machine file which > specifies that (up to) the first 8 processes are computed on node1 and > the second group of 8 processes are computed on node2. My timing > results are shown in the table below, I'm running each test using > Bi-CGStab with no preconditioning (-ksp_type bcgs -pc_type none) on a > computational grid of 800 x 800 cells, so 641601 DOFs. I have attached > my modified source code (you could look at my changes using diff) and > the -log_summary output for each of the tests. > The way I read these numbers is that there is bandwidth for about 3 cores on this machine, and non-negligible synchronization penalty: 1 proc 2 proc 4 proc 8 proc VecAXPBYCZ 496 857 1064 1070 VecDot 451 724 1089 736 MatMult 434 638 701 703 The bandwidth tops out between 2 and 4 cores (The 5345 should have 10.6 GB/s but you should runs streams as Barry says to see what is achievable). There is obviously a penalty for VecDot against VecAXPYCZ, which is the sync penalty which also seems to affect MatMult. Maybe Jed can explain that. Matt # number of processes | time for KSPsolve() | iterations to > convergence | norm of error > 1 64.008 692 0.00433961 > 2 36.2767 626 0.00611835 > 4 35.9989 760 0.00311053 > 8 30.5215 664 0.00599148 > 16 14.1164 710 0.00792162 > > Why is the scaling so poor? I have read the FAQ > (http://www.mcs.anl.gov/petsc/petsc-as/documentation/faq.html#computers), > am I experiencing the problem described? I think my machine has a > bandwidth of 2GB/s per process as suggested. Also, how can you tell if > a computation is memory bound by looking at the -log_summary? > > Many thanks, > > Matija > > On Tue, Sep 20, 2011 at 11:44 AM, Jed Brown wrote: > > > > On Tue, Sep 20, 2011 at 11:45, Matija Kecman > wrote: > >> > >> $ mpirun -np 1 ./ex3 -ksp_type gmres -pc_type none -m 100 > >> Norm of error 0.570146 Iterations 0 > > > > This uses a nonzero initial guess so the initial residual norm is > compared to the right hand side. > > $ ./ex3 -ksp_type gmres -ksp_monitor -m 100 -pc_type none > -ksp_converged_reason -info |grep Converged > > [0] KSPDefaultConverged(): user has provided nonzero initial guess, > computing 2-norm of preconditioned RHS > > [0] KSPDefaultConverged(): Linear solver has converged. Residual norm > 1.113646413065e-04 is less than relative tolerance 1.000000000000e-05 times > initial right hand side norm 1.291007358616e+01 at iteration 0 > > You can use the true residual, it just costs something so it's not > enabled by default: > > $ ./ex3 -ksp_type gmres -ksp_monitor -m 100 -pc_type none > -ksp_converged_reason -ksp_converged_use_initial_residual_norm > > [many iterations] > > Linear solve converged due to CONVERGED_RTOL iterations 1393 > > Norm of error 0.000664957 Iterations 1393 > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From matijakecman at gmail.com Thu Sep 29 07:49:00 2011 From: matijakecman at gmail.com (Matija Kecman) Date: Thu, 29 Sep 2011 13:49:00 +0100 Subject: [petsc-users] Question about ksp ex3.c In-Reply-To: <52CA5812-4BBF-4961-8BA5-891859047A67@mcs.anl.gov> References: <52CA5812-4BBF-4961-8BA5-891859047A67@mcs.anl.gov> Message-ID: Thanks Barry, I'm using PETSc 3.1.0 Patch 8, is there something similar I can do that would work for this version? On Thu, Sep 29, 2011 at 1:42 PM, Barry Smith wrote: > > ?If you are using petsc-3.2 you can also run "make streams" in the $PETSC_DIR and that will run the streams benchmark giving you a pretty good idea of how the memory bandwidth scales with the number of cores. If the streams benchmark does not scale with the cores then no iterative solver will scale with more cores. > > ? Barry > > On Sep 29, 2011, at 6:28 AM, Matija Kecman wrote: > >> Thanks for your response Jed! I've been doing some other >> investigations using this example. I made some small modifications: >> >> 1. Added preallocation as Jed Brown suggested in a previous email >> (http://lists.mcs.anl.gov/pipermail/petsc-users/2011-June/009054.html). >> 2. Added a small VTK viewer. >> 3. Set the initial guess to zero. >> 4. Changed the entries in the element stiffness matrix to the following: >> >> ? Ke[ 0] = ?2./3.; Ke[ 1] = -1./6.; Ke[ 2] = -1./3.; Ke[ 3] = -1./6.; >> ? Ke[ 4] = -1./6.; Ke[ 5] = ?2./3.; Ke[ 6] = -1./6.; Ke[ 7] = -1./3.; >> ? Ke[ 8] = -1./3.; Ke[ 9] = -1./6.; Ke[10] = ?2./3.; Ke[11] = -1./6.; >> ? Ke[12] = -1./6.; Ke[13] = -1./3.; Ke[14] = -1./6.; Ke[15] = ?2./3.; >> >> I computed these by evaluating $K^e_{ij} = \int_{\Omega_e} \nabla >> \psi^e_i \cdot \nabla \psi^e_j \, \mathrm{d}\Omega$ with the shape >> functions $\psi^e$ corresponding to a bilinear quadratic finite >> element denoted by $\Omega_e$. This is different to what was >> originally in the code and I'm not sure where the original code comes >> from. This isn't important so you can just ignore it if you like, I >> get the same solution using both matrices. >> >> --- >> >> I am running on a two compute node clusters which each look as >> follows: 2 quad-core Intel Xeon 5345 processors, 16GB memory. The node >> clusters are connected with the following interconnect: Mellanox >> InfiniScale 2400. I computed my results using a machine file which >> specifies that (up to) the first 8 processes are computed on node1 and >> the second group of 8 processes are computed on node2. My timing >> results are shown in the table below, I'm running each test using >> Bi-CGStab with no preconditioning (-ksp_type bcgs -pc_type none) on a >> computational grid of 800 x 800 cells, so 641601 DOFs. I have attached >> my modified source code (you could look at my changes using diff) and >> the -log_summary output for each of the tests. >> >> # number of processes | time for KSPsolve() | iterations to >> convergence | norm of error >> 1 64.008 692 0.00433961 >> 2 36.2767 626 0.00611835 >> 4 35.9989 760 0.00311053 >> 8 30.5215 664 0.00599148 >> 16 14.1164 710 0.00792162 >> >> Why is the scaling so poor? I have read the FAQ >> (http://www.mcs.anl.gov/petsc/petsc-as/documentation/faq.html#computers), >> am I experiencing the problem described? I think my machine has a >> bandwidth of 2GB/s per process as suggested. Also, how can you tell if >> a computation is memory bound by looking at the -log_summary? >> >> Many thanks, >> >> Matija >> >> On Tue, Sep 20, 2011 at 11:44 AM, Jed Brown wrote: >>> >>> On Tue, Sep 20, 2011 at 11:45, Matija Kecman wrote: >>>> >>>> $ mpirun -np 1 ./ex3 -ksp_type gmres -pc_type none -m 100 >>>> Norm of error 0.570146 Iterations 0 >>> >>> This uses a nonzero initial guess so the initial residual norm is compared to the right hand side. >>> $ ./ex3 -ksp_type gmres -ksp_monitor -m 100 -pc_type none -ksp_converged_reason -info |grep Converged >>> [0] KSPDefaultConverged(): user has provided nonzero initial guess, computing 2-norm of preconditioned RHS >>> [0] KSPDefaultConverged(): Linear solver has converged. Residual norm 1.113646413065e-04 is less than relative tolerance 1.000000000000e-05 times initial right hand side norm 1.291007358616e+01 at iteration 0 >>> You can use the true residual, it just costs something so it's not enabled by default: >>> $ ./ex3 -ksp_type gmres -ksp_monitor -m 100 -pc_type none -ksp_converged_reason -ksp_converged_use_initial_residual_norm >>> [many iterations] >>> Linear solve converged due to CONVERGED_RTOL iterations 1393 >>> Norm of error 0.000664957 Iterations 1393 >> > > From jedbrown at mcs.anl.gov Thu Sep 29 07:55:01 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Thu, 29 Sep 2011 07:55:01 -0500 Subject: [petsc-users] Question about ksp ex3.c In-Reply-To: References: <52CA5812-4BBF-4961-8BA5-891859047A67@mcs.anl.gov> Message-ID: On Thu, Sep 29, 2011 at 07:49, Matija Kecman wrote: > Thanks Barry, I'm using PETSc 3.1.0 Patch 8, is there something > similar I can do that would work for this version? > It's a stand-alone micro-benchmark. It has nothing to do with your code. The version in petsc-3.2 has synchronization so that the timing results are clean. You can run the old one with MPI, but the outputs will be all jumbled together. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jedbrown at mcs.anl.gov Thu Sep 29 08:09:52 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Thu, 29 Sep 2011 08:09:52 -0500 Subject: [petsc-users] Question about ksp ex3.c In-Reply-To: References: Message-ID: On Thu, Sep 29, 2011 at 07:44, Matthew Knepley wrote: > The way I read these numbers is that there is bandwidth for about 3 cores > on > this machine, and non-negligible synchronization penalty: > > 1 proc 2 proc 4 proc 8 proc > VecAXPBYCZ 496 857 1064 1070 > VecDot 451 724 1089 736 > MatMult 434 638 701 703 > Matt, thanks for pulling out this summary. The synchronization for the dot product is clearly expensive here. I'm surprised it's so significant on such a small problem, but it is a common problem for scalability. I think that MPI is putting one process per socket when you go from 1 to 2 processes. This gives you pretty good speedup despite the fact that memory traffic for both sockets is routed through the "Blackford" chipset. Intel fixed this in later generations by throwing out the notion of uniform memory access by having independent memory banks for each socket (which AMD had at the time these chips came out). If not odd hardware issues such as having enough memory streams and imbalances, one process per socket can saturate the memory bandwidth, so the speed-up you get from 2 procs (1 per socket) to 4 procs (2 per socket) is minimal. On this architecture, you typically see the STREAM bandwidth go down slightly as you add more than 2 procs per socket, so it's no surprise that it doesn't help here. Note that your naive 1D partition is getting worse as you add processes. The MatMult should scale out somewhat better if you use a 2D decomposition, as is done by any of the examples using a DA (DMDA in 3.2) for grid management. > > The bandwidth tops out between 2 and 4 cores (The 5345 should have 10.6 > GB/s > but you should runs streams as Barry says to see what is achievable). There > is > obviously a penalty for VecDot against VecAXPYCZ, which is the sync penalty > which also seems to affect MatMult. Maybe Jed can explain that. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From matijakecman at gmail.com Thu Sep 29 09:06:58 2011 From: matijakecman at gmail.com (Matija Kecman) Date: Thu, 29 Sep 2011 15:06:58 +0100 Subject: [petsc-users] Question about ksp ex3.c In-Reply-To: References: Message-ID: Thanks for your useful replies. >> The bandwidth tops out between 2 and 4 cores (The 5345 should have 10.6 >> GB/s >> but you should runs streams as Barry says to see what is achievable). I will try to get the STREAM benchmark running > Note that your naive 1D partition is getting worse as you add processes. The > MatMult should scale out somewhat better if you use a 2D decomposition, as > is done by any of the examples using a DA (DMDA in 3.2) for grid management. I am able to recompute the problem using a 2D decomposition and I will look at the log-summary output for those tests also Best, Matija On Thu, Sep 29, 2011 at 2:09 PM, Jed Brown wrote: > On Thu, Sep 29, 2011 at 07:44, Matthew Knepley wrote: >> >> The way I read these numbers is that there is bandwidth for about 3 cores >> on >> this machine, and non-negligible synchronization penalty: >> ? ? ? ? ? ? ? ? ? ? ? ? 1 proc ? 2 proc ? 4 proc ? ?8 proc >> VecAXPBYCZ ? ?496 ? ? ? ?857 ? ? ?1064 ? ? ?1070 >> VecDot ? ? ? ? ? ? ? 451 ? ? ? ?724 ? ? ?1089 ? ? ? ?736 >> MatMult ? ? ? ? ? ? ?434 ? ? ? ?638 ? ? ? ?701 ? ? ? ?703 > > Matt, thanks for pulling out this summary. The synchronization for the dot > product is clearly expensive here. I'm surprised it's so significant on such > a small problem, but it is a common problem for scalability. > I think that MPI is putting one process per socket when you go from 1 to 2 > processes. This gives you pretty good speedup despite the fact that memory > traffic for both sockets is routed through the "Blackford" chipset. Intel > fixed this in later generations by throwing out the notion of uniform memory > access by having independent memory banks for each socket (which AMD had at > the time these chips came out). > If not odd hardware issues such as having enough memory streams and > imbalances, one process per socket can saturate the memory bandwidth, so the > speed-up you get from 2 procs (1 per socket) to 4 procs (2 per socket) is > minimal. On this architecture, you typically see the STREAM bandwidth go > down slightly as you add more than 2 procs per socket, so it's no surprise > that it doesn't help here. > Note that your naive 1D partition is getting worse as you add processes. The > MatMult should scale out somewhat better if you use a 2D decomposition, as > is done by any of the examples using a DA (DMDA in 3.2) for grid management. > >> >> The bandwidth tops out between 2 and 4 cores (The 5345 should have 10.6 >> GB/s >> but you should runs streams as Barry says to see what is achievable). >> There is >> obviously a penalty for VecDot against VecAXPYCZ, which is the sync >> penalty >> which also seems to affect MatMult. Maybe Jed can explain that. > From gshy2014 at gmail.com Thu Sep 29 14:56:09 2011 From: gshy2014 at gmail.com (Shiyuan) Date: Thu, 29 Sep 2011 14:56:09 -0500 Subject: [petsc-users] direct access of seqcusp Vec Message-ID: What about a direct access to seqaijcusp Mat? Is there a function call which return the pointed to the cusp::csr_matrix ? Also, i notice that, the flops count from --log_summary are different if we switch on/off -vec_type seqcusp and -mat_type seqaijcusp? Why is that? what is the option -cusp_synchronize for? Thanks. ------------------------------ Message: 2 Date: Wed, 28 Sep 2011 23:22:02 -0500 From: Barry Smith Subject: Re: [petsc-users] direct access of seqcusp Vec To: PETSc users list Message-ID: <1CFA4EF7-E716-4942-BE0F-6AB2E4EF78E5 at mcs.anl.gov> Content-Type: text/plain; charset=us-ascii VecCUSPGetArrayReadWrite() gives you access to the CUSP array that lives on the device memory. You can then do things to it using the CUSP header file definitions Barry On Sep 28, 2011, at 10:46 PM, Shiyuan wrote: > Hi, > How can we acess the data of a seqcusp Vec directly? Is it stored as consecutive 1d array in the device momery? > Thanks. > > Shiyuan ------------------------------ -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Thu Sep 29 15:22:05 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Thu, 29 Sep 2011 15:22:05 -0500 Subject: [petsc-users] direct access of seqcusp Vec In-Reply-To: References: Message-ID: On Sep 29, 2011, at 2:56 PM, Shiyuan wrote: > What about a direct access to seqaijcusp Mat? Is there a function call which return the pointed to the cusp::csr_matrix ? No. But look at the source code to MatMult_SeqAIJCusp and you will see how it may be accessed. > Also, i notice that, the flops count from --log_summary are different if we switch on/off -vec_type seqcusp and -mat_type seqaijcusp? Are you sure the exact same solver is being used and the exact same number of iterations. Check the -log_summary table for the number of times each operation (vecdot, matmult etc etc) is done for the cusp and non-cusp case, if they are different then you will get different counts. > Why is that? > what is the option -cusp_synchronize for? Nvidia GPU cards can run asynchronizely from the CPU. This means you can start some operation on the GPU that takes a while and the GPU starts work on it but the CPU doesn't wait for the GPU to finish it can do something else. Now for PETSc linear algebra this rarely makes sense because while doing an iterative solver there is really nothing else for the CPU to do but wait. the -cusp_synchronize causes the CPU to wait inside the operation (like VecDot) until the GPU has finished its dot product, this gives the correct timing results. Otherwise the time will only measure the time for the CPU to start the GPU task and not the time the GPU took to do the GPU. When running with -log_summary the the option -cusp_synchronize is automatically enabled so generally you don't need to use that flag yourself. Barry > > Thanks. > ------------------------------ > > Message: 2 > Date: Wed, 28 Sep 2011 23:22:02 -0500 > From: Barry Smith > Subject: Re: [petsc-users] direct access of seqcusp Vec > To: PETSc users list > Message-ID: <1CFA4EF7-E716-4942-BE0F-6AB2E4EF78E5 at mcs.anl.gov> > Content-Type: text/plain; charset=us-ascii > > > VecCUSPGetArrayReadWrite() gives you access to the CUSP array that lives on the device memory. You can then do things to it using the CUSP header file definitions > > Barry > > > > On Sep 28, 2011, at 10:46 PM, Shiyuan wrote: > > > Hi, > > How can we acess the data of a seqcusp Vec directly? Is it stored as consecutive 1d array in the device momery? > > Thanks. > > > > Shiyuan > > > > ------------------------------ From bsmith at mcs.anl.gov Thu Sep 29 16:51:29 2011 From: bsmith at mcs.anl.gov (Barry Smith) Date: Thu, 29 Sep 2011 16:51:29 -0500 Subject: [petsc-users] direct access of seqcusp Vec In-Reply-To: References: Message-ID: <940F7893-16F4-4BD1-8891-F39BAF9F4934@mcs.anl.gov> For VecNorm_MPICUSP() it called VecNorm_SeqCUSP() which logged the flops but then VecNorm_MPICUSP() logged it again. Hence they were double counted. I will push an update to petsc-dev in a few minutes that fixes this. MatMult_SeqAIJ() attempts to be more careful in the count by taking into account that each completely empty row contributes one less flop then expected which the CUSP cannot easily do that since we don't know how the CUSP actually implements the product for empty rows and are too lazy to check. I cannot explain why the VecMDot are slightly different. Since PETSc is all open source and we value improvements by our users you are free to look more closely to see if you can determine why the mdot values are slighly different. Thanks for reporting the bug, Barry On Sep 29, 2011, at 4:36 PM, Shiyuan wrote: > > > Also, i notice that, the flops count from --log_summary are different if we switch on/off -vec_type seqcusp and -mat_type seqaijcusp? > > Are you sure the exact same solver is being used and the exact same number of iterations. Check the -log_summary table for the number of times > each operation (vecdot, matmult etc etc) is done for the cusp and non-cusp case, if they are different then you will get different counts. > > I have two runs of ex19, exactly same options except -da_vec_type seqcusp and -da_mat_type are switch on/off. The count of each operation are the same but the flops count for some operations are different. > > > VecMDot 2024 1.0 4.1337e+00 1.0 2.50e+09 1.0 0.0e+00 0.0e+00 0.0e+00 17 29 0 0 0 19 29 0 0 0 605 > VecNorm 2096 1.0 9.4352e-01 1.0 1.68e+08 1.0 0.0e+00 0.0e+00 0.0e+00 4 2 0 0 0 4 2 0 0 0 178 > MatMult 2092 1.0 1.2145e+01 1.0 3.24e+09 1.0 0.0e+00 0.0e+00 0.0e+00 50 37 0 0 0 56 37 0 0 0 267 > > vs. > VecMDot 2024 1.0 1.0873e+00 1.0 2.54e+09 1.0 0.0e+00 0.0e+00 0.0e+00 17 28 0 0 0 28 28 0 0 0 2340 > VecNorm 2096 1.0 1.2240e-01 1.0 3.35e+08 1.0 0.0e+00 0.0e+00 0.0e+00 2 4 0 0 0 3 4 0 0 0 2740 > MatMult 2092 1.0 1.7572e+00 1.0 3.32e+09 1.0 0.0e+00 0.0e+00 0.0e+00 28 37 0 0 0 46 37 0 0 0 1890 > > #PETSc Option Table entries: > -cusp_synchronize > -da_grid_x 100 > -da_grid_y 100 > -da_mat_type mpiaijcusp > -da_vec_type mpicusp > -dmmg_nlevels 1 > -log_summary > -mat_no_inode > -pc_type none > -preload off > #End of PETSc Option Table entries > > The whole -log_summary is attached. > Thanks. > From dominik at itis.ethz.ch Fri Sep 30 09:34:13 2011 From: dominik at itis.ethz.ch (Dominik Szczerba) Date: Fri, 30 Sep 2011 16:34:13 +0200 Subject: [petsc-users] question about contexts Message-ID: I am currently declaring functions outside my C++ solver class to be used by via MatShellSetOperation like: #undef __FUNCT__ #define __FUNCT__ "StiffnessMatrixMultiplication" PetscErrorCode StiffnessMatrixMultiplication(Mat A, Vec x, Vec y) { // ... ierr = MatShellGetContext(A, (void**) &ctx);CHKERRQ(ierr); Mat foo; foo = ctx->foo; // ... } and I set/get contexts to e.g. pass 'foo' around. Is it possible in Petsc / is it a good idea / to instead declare such functions as static class member functions and then use the class members directly without any contexts shuffled around? Thanks for any hints. Dominik From jedbrown at mcs.anl.gov Fri Sep 30 09:44:01 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Fri, 30 Sep 2011 09:44:01 -0500 Subject: [petsc-users] question about contexts In-Reply-To: References: Message-ID: On Fri, Sep 30, 2011 at 09:34, Dominik Szczerba wrote: > I am currently declaring functions outside my C++ solver class to be > used by via MatShellSetOperation like: > > #undef __FUNCT__ > #define __FUNCT__ "StiffnessMatrixMultiplication" > PetscErrorCode StiffnessMatrixMultiplication(Mat A, Vec x, Vec y) > { > // ... > ierr = MatShellGetContext(A, (void**) &ctx);CHKERRQ(ierr); > Mat foo; > foo = ctx->foo; > // ... > } > > and I set/get contexts to e.g. pass 'foo' around. Is it possible in > Petsc / is it a good idea / to instead declare such functions as > static class member functions and then use the class members directly > without any contexts shuffled around? > You have to go through the context unless your class is a singleton. Or do you still mean casting the ctx to your class type and accessing members through it (explicit "this")? That should work, if you prefer. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mike.hui.zhang at hotmail.com Fri Sep 30 10:51:00 2011 From: mike.hui.zhang at hotmail.com (Hui Zhang) Date: Fri, 30 Sep 2011 17:51:00 +0200 Subject: [petsc-users] MatMult a MatNest and a Vec Message-ID: Hi all, Do you know what type of the resulting vector y by MatMult of a MatNest A and a Vec x, i.e. y = Ax? Because after that I want to modify a SubVec of the result y. If it is a VectNest, I think VecNestGetSubVec should be used but not VecGetSubVector. Thank you for always helps! Hui From dominik at itis.ethz.ch Fri Sep 30 10:54:18 2011 From: dominik at itis.ethz.ch (Dominik Szczerba) Date: Fri, 30 Sep 2011 17:54:18 +0200 Subject: [petsc-users] question about contexts In-Reply-To: References: Message-ID: Yes, I only have one instance of my solver class, if this is what you mean by a singleton. I did not mean to cast my context, I meant to avoid any contexts and just access the members. I so far use the context only to pass KSP, Mat and Vec objects around a few functions. I am asking, because I have not seen anyone doing it, while it appears natural to me. Also, I am doubting if #define __FUNCT__ "StiffnessMatrixMultiplication" will continue to work for static functions. Thanks, Dominik On Fri, Sep 30, 2011 at 4:44 PM, Jed Brown wrote: > On Fri, Sep 30, 2011 at 09:34, Dominik Szczerba > wrote: >> >> I am currently declaring functions outside my C++ solver class to be >> used by via MatShellSetOperation like: >> >> #undef __FUNCT__ >> #define __FUNCT__ "StiffnessMatrixMultiplication" >> PetscErrorCode StiffnessMatrixMultiplication(Mat A, Vec x, Vec y) >> { >> ? ? ? ?// ... >> ? ? ? ?ierr = MatShellGetContext(A, (void**) &ctx);CHKERRQ(ierr); >> ? ? ? ?Mat foo; >> ? ? ? ?foo = ctx->foo; >> ? ? ? ?// ... >> } >> >> and I set/get contexts to e.g. pass 'foo' around. Is it possible in >> Petsc / is it a good idea / to instead declare such functions as >> static class member functions and then use the class members directly >> without any contexts shuffled around? > > You have to go through the context unless your class is a singleton. Or do > you still mean casting the ctx to your class type and accessing members > through it (explicit "this")? That should work, if you prefer. From dominik at itis.ethz.ch Fri Sep 30 11:21:56 2011 From: dominik at itis.ethz.ch (Dominik Szczerba) Date: Fri, 30 Sep 2011 18:21:56 +0200 Subject: [petsc-users] question about contexts In-Reply-To: References: Message-ID: It does not seem to work, unless I just miss some C/C++ idiom. I was wrong, StiffnessMatrixMultiplication can not be static member, else all the other non-static members are inaccessible. So I am trying non-static, but can not pray PCShellSetApply work with such a function, always get some syntax error. Any hints are highly appreciated. Dominik On Fri, Sep 30, 2011 at 5:54 PM, Dominik Szczerba wrote: > Yes, I only have one instance of my solver class, if this is what you > mean by a singleton. > I did not mean to cast my context, I meant to avoid any contexts and > just access the members. I so far use the context only to pass KSP, > Mat and Vec objects around a few functions. I am asking, because I > have not seen anyone doing it, while it appears natural to me. Also, I > am doubting if #define __FUNCT__ "StiffnessMatrixMultiplication" will > continue to work for static functions. > > Thanks, > Dominik > > On Fri, Sep 30, 2011 at 4:44 PM, Jed Brown wrote: >> On Fri, Sep 30, 2011 at 09:34, Dominik Szczerba >> wrote: >>> >>> I am currently declaring functions outside my C++ solver class to be >>> used by via MatShellSetOperation like: >>> >>> #undef __FUNCT__ >>> #define __FUNCT__ "StiffnessMatrixMultiplication" >>> PetscErrorCode StiffnessMatrixMultiplication(Mat A, Vec x, Vec y) >>> { >>> ? ? ? ?// ... >>> ? ? ? ?ierr = MatShellGetContext(A, (void**) &ctx);CHKERRQ(ierr); >>> ? ? ? ?Mat foo; >>> ? ? ? ?foo = ctx->foo; >>> ? ? ? ?// ... >>> } >>> >>> and I set/get contexts to e.g. pass 'foo' around. Is it possible in >>> Petsc / is it a good idea / to instead declare such functions as >>> static class member functions and then use the class members directly >>> without any contexts shuffled around? >> >> You have to go through the context unless your class is a singleton. Or do >> you still mean casting the ctx to your class type and accessing members >> through it (explicit "this")? That should work, if you prefer. > From balay at mcs.anl.gov Fri Sep 30 11:26:07 2011 From: balay at mcs.anl.gov (Satish Balay) Date: Fri, 30 Sep 2011 11:26:07 -0500 (CDT) Subject: [petsc-users] petsc-3.2-p3.tar.gz now available Message-ID: Dear PETSc users, Patch-3 update to petsc-3.2 is now available for download. http://www.mcs.anl.gov/petsc/petsc-as/download/index.html Satish From jedbrown at mcs.anl.gov Fri Sep 30 12:08:38 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Fri, 30 Sep 2011 12:08:38 -0500 Subject: [petsc-users] MatMult a MatNest and a Vec In-Reply-To: References: Message-ID: On Fri, Sep 30, 2011 at 10:51, Hui Zhang wrote: > Do you know what type of the resulting vector y by MatMult of a MatNest A > and a Vec x, i.e. y = Ax? > MatMult does not change the types of vectors. This function works with any types. Note, however, that many Vec operations require all arguments to have the same type, so you can't freely mix native Vec with VecNest. I don't recommend using VecNest unless you have a very specific use case and can explain to me why you should use it. (I do recommend using MatNest, but generally with the MatGetLocalSubMatrix() so you don't depend on it---I always try to avoid depending on implementations.) > Because after that I want to modify a SubVec of the result y. If it is a > VectNest, I think > VecNestGetSubVec should be used but not VecGetSubVector. > This is why VecNest implements VecGetSubVector (see VecGetSubVector_Nest). -------------- next part -------------- An HTML attachment was scrubbed... URL: From dominik at itis.ethz.ch Fri Sep 30 12:09:20 2011 From: dominik at itis.ethz.ch (Dominik Szczerba) Date: Fri, 30 Sep 2011 19:09:20 +0200 Subject: [petsc-users] question about contexts In-Reply-To: References: Message-ID: After educating myself in a long googling session on C++ member function pointers I find that PCShellSetApply would simply only accept C-style static functions (error: argument of type ?PetscErrorCode (FluidSolver::)(_p_PC*, _p_Vec*, _p_Vec*)? does not match ?PetscErrorCode (*)(_p_PC*, _p_Vec*, _p_Vec*)?), so my approach unfortunately is not feasible, what a pity. Please notify me if I am wrong, it is a fundamental issue for my design. Dominik On Fri, Sep 30, 2011 at 6:21 PM, Dominik Szczerba wrote: > It does not seem to work, unless I just miss some C/C++ idiom. > I was wrong, StiffnessMatrixMultiplication can not be static member, > else all the other non-static members are inaccessible. > So I am trying non-static, but can not pray PCShellSetApply work with > such a function, always get some syntax error. > > Any hints are highly appreciated. > > Dominik > > > > On Fri, Sep 30, 2011 at 5:54 PM, Dominik Szczerba wrote: >> Yes, I only have one instance of my solver class, if this is what you >> mean by a singleton. >> I did not mean to cast my context, I meant to avoid any contexts and >> just access the members. I so far use the context only to pass KSP, >> Mat and Vec objects around a few functions. I am asking, because I >> have not seen anyone doing it, while it appears natural to me. Also, I >> am doubting if #define __FUNCT__ "StiffnessMatrixMultiplication" will >> continue to work for static functions. >> >> Thanks, >> Dominik >> >> On Fri, Sep 30, 2011 at 4:44 PM, Jed Brown wrote: >>> On Fri, Sep 30, 2011 at 09:34, Dominik Szczerba >>> wrote: >>>> >>>> I am currently declaring functions outside my C++ solver class to be >>>> used by via MatShellSetOperation like: >>>> >>>> #undef __FUNCT__ >>>> #define __FUNCT__ "StiffnessMatrixMultiplication" >>>> PetscErrorCode StiffnessMatrixMultiplication(Mat A, Vec x, Vec y) >>>> { >>>> ? ? ? ?// ... >>>> ? ? ? ?ierr = MatShellGetContext(A, (void**) &ctx);CHKERRQ(ierr); >>>> ? ? ? ?Mat foo; >>>> ? ? ? ?foo = ctx->foo; >>>> ? ? ? ?// ... >>>> } >>>> >>>> and I set/get contexts to e.g. pass 'foo' around. Is it possible in >>>> Petsc / is it a good idea / to instead declare such functions as >>>> static class member functions and then use the class members directly >>>> without any contexts shuffled around? >>> >>> You have to go through the context unless your class is a singleton. Or do >>> you still mean casting the ctx to your class type and accessing members >>> through it (explicit "this")? That should work, if you prefer. >> > From jedbrown at mcs.anl.gov Fri Sep 30 12:23:54 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Fri, 30 Sep 2011 12:23:54 -0500 Subject: [petsc-users] question about contexts In-Reply-To: References: Message-ID: You should understand what it means to he a static member function of a class. There is no object, you can only access static members which are just namespaced globals. But the method is like a "friend" in the sense that it can access members through an explicit pointer. On Sep 30, 2011 11:22 AM, "Dominik Szczerba" wrote: > It does not seem to work, unless I just miss some C/C++ idiom. > I was wrong, StiffnessMatrixMultiplication can not be static member, > else all the other non-static members are inaccessible. > So I am trying non-static, but can not pray PCShellSetApply work with > such a function, always get some syntax error. > > Any hints are highly appreciated. > > Dominik > > > > On Fri, Sep 30, 2011 at 5:54 PM, Dominik Szczerba wrote: >> Yes, I only have one instance of my solver class, if this is what you >> mean by a singleton. >> I did not mean to cast my context, I meant to avoid any contexts and >> just access the members. I so far use the context only to pass KSP, >> Mat and Vec objects around a few functions. I am asking, because I >> have not seen anyone doing it, while it appears natural to me. Also, I >> am doubting if #define __FUNCT__ "StiffnessMatrixMultiplication" will >> continue to work for static functions. >> >> Thanks, >> Dominik >> >> On Fri, Sep 30, 2011 at 4:44 PM, Jed Brown wrote: >>> On Fri, Sep 30, 2011 at 09:34, Dominik Szczerba >>> wrote: >>>> >>>> I am currently declaring functions outside my C++ solver class to be >>>> used by via MatShellSetOperation like: >>>> >>>> #undef __FUNCT__ >>>> #define __FUNCT__ "StiffnessMatrixMultiplication" >>>> PetscErrorCode StiffnessMatrixMultiplication(Mat A, Vec x, Vec y) >>>> { >>>> // ... >>>> ierr = MatShellGetContext(A, (void**) &ctx);CHKERRQ(ierr); >>>> Mat foo; >>>> foo = ctx->foo; >>>> // ... >>>> } >>>> >>>> and I set/get contexts to e.g. pass 'foo' around. Is it possible in >>>> Petsc / is it a good idea / to instead declare such functions as >>>> static class member functions and then use the class members directly >>>> without any contexts shuffled around? >>> >>> You have to go through the context unless your class is a singleton. Or do >>> you still mean casting the ctx to your class type and accessing members >>> through it (explicit "this")? That should work, if you prefer. >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From dominik at itis.ethz.ch Fri Sep 30 12:28:05 2011 From: dominik at itis.ethz.ch (Dominik Szczerba) Date: Fri, 30 Sep 2011 19:28:05 +0200 Subject: [petsc-users] question about contexts In-Reply-To: References: Message-ID: > But the method is like a "friend" in the sense that it can access members through an explicit pointer. Can you please expand a bit on this one? What is an explicit pointer? Dominik From vijay.m at gmail.com Fri Sep 30 12:33:01 2011 From: vijay.m at gmail.com (Vijay S. Mahadevan) Date: Fri, 30 Sep 2011 12:33:01 -0500 Subject: [petsc-users] question about contexts In-Reply-To: References: Message-ID: You should be able to do this: #undef __FUNCT__ #define __FUNCT__ "StiffnessMatrixMultiplication" PetscErrorCode StiffnessMatrixMultiplication(Mat A, Vec x, Vec y) { MyClass* ctx ; // ... ierr = MatShellGetContext(A, (void**) &ctx);CHKERRQ(ierr); Mat foo; foo = ctx->foo; // non-static public member of MyClass that is of type Mat // ... Mat bar; bar = MyClass::bar; // static member of MyClass that is of type Mat // ... Mat foobar; foobar = MyClass::foobar(); // static member function of MyClass that returns a Mat // ... } And similarly for the PCShellApply, you can get the context MyClass from the PC instance. I don't see what problems you have with this use-case. Look at SampleShellPCApply in http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-dev/src/ksp/ksp/examples/tutorials/ex15.c.html for a similar usage. On Fri, Sep 30, 2011 at 12:28 PM, Dominik Szczerba wrote: >> But the method is like a "friend" in the sense that it can access members through an explicit pointer. > > Can you please expand a bit on this one? What is an explicit pointer? > > Dominik > From jedbrown at mcs.anl.gov Fri Sep 30 13:09:45 2011 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Fri, 30 Sep 2011 13:09:45 -0500 Subject: [petsc-users] question about contexts In-Reply-To: References: Message-ID: On Fri, Sep 30, 2011 at 12:28, Dominik Szczerba wrote: > > But the method is like a "friend" in the sense that it can access members > through an explicit pointer. > > Can you please expand a bit on this one? What is an explicit pointer? Cast the void* to a MyClass*, then access all the members through that pointer. It will look like a normal member function except that there is no implicit "this", you have to explicitly use the MyClass* to access non-static members. -------------- next part -------------- An HTML attachment was scrubbed... URL: