From mfadams at lbl.gov Sun Dec 1 10:32:49 2013 From: mfadams at lbl.gov (Mark Adams) Date: Sun, 1 Dec 2013 11:32:49 -0500 Subject: [petsc-users] fieldsplit: Unhandled case, must have at least two fields, not 1! In-Reply-To: <87haata2io.fsf@jedbrown.org> References: <87d2liblo7.fsf@jedbrown.org> <87haata2io.fsf@jedbrown.org> Message-ID: Perhaps this will help. The code is straight forward but prefixes are used. I've appended the code and a stack trace for the error. ! create KSP call KSPCreate(solver%comm,solver%ksp,ierr) CHKERRQ(ierr) if (solver%prefix.eq.2) then ! turb call KSPSetOptionsPrefix(solver%ksp,'s2_',ierr) if (sml_mype/sml_pe_per_plane .gt. 0) then ! not first, no monitor call PetscOptionsClearValue('-s2_ksp_monitor',ierr) call PetscOptionsClearValue('-s2_ksp_converged_reason',ierr) endif else ! 1 field 00 solver, no prefix call PetscOptionsClearValue('-ksp_monitor',ierr) call PetscOptionsClearValue('-ksp_converged_reason',ierr) end if call KSPSetFromOptions(solver%ksp,ierr) CHKERRQ(ierr) call KSPSetOperators(solver%ksp, solver%Amat, solver%Amat, SAME_NONZERO_PATTERN, ierr ) CHKERRQ(ierr) ! setup solver now that matrix is complete call KSPSetUp( solver%ksp, ierr ) !!! poisson.F90:213 #10 xgc1_3 () at /global/u2/m/madams/epsi/XGC1_3/main.F90:98 (at 0x00000000004b8ed6) #9 setup (grid=..., psn=..., spall=???) at /global/u2/m/madams/epsi/XGC1_3/setup.F90:123 (at 0x00000000004dd3b0) #8 init_poisson (grid=..., psn=..., iflag_dummy=3) at /global/u2/m/madams/epsi/XGC1_3/poisson.F90:36 (at 0x00000000005b06af) #7 init_poisson_solver (grid=..., psn=..., iflag_dummy=3) at /global/u2/m/madams/epsi/XGC1_3/poisson.F90:163 (at 0x00000000005b1f17) #6 init_1field_solver (grid=..., psn=..., solver=...) at /global/u2/m/madams/epsi/XGC1_3/poisson.F90:213 (at 0x00000000005b2862) #5 kspsetup_ (ksp=0x34c7950, __ierr=0x7fffffff5290) at /global/u2/m/madams/petsc_private/src/ksp/ksp/interface/ftn-auto/itfuncf.c:215 (at 0x0000000000a47965) #4 KSPSetUp (ksp=0xb4e4190) at /global/u2/m/madams/petsc_private/src/ksp/ksp/interface/itfunc.c:278 (at 0x00000000015e1c95) #3 PCSetUp (pc=0xb4ea8d0) at /global/u2/m/madams/petsc_private/src/ksp/pc/interface/precon.c:888 (at 0x00000000015a64ec) #2 PCSetUp_FieldSplit (pc=0xb4ea8d0) at /global/u2/m/madams/petsc_private/src/ksp/pc/impls/fieldsplit/fieldsplit.c:458 (at 0x0000000001b97d69) #1 PCFieldSplitSetDefaults (pc=0xb4ea8d0) at /global/u2/m/madams/petsc_private/src/ksp/pc/impls/fieldsplit/fieldsplit.c:440 (at 0x0000000001b979bc) #0 PetscError (comm=-1006632952, line=440, func=0x2ea40c6 "PCFieldSplitSetDefaults", file=0x2ea3a08 "src/ksp/pc/impls/fieldsplit/fieldsplit.c", n=77, p=PETSC_ERROR_INITIAL, mess=0x2ea4090 "Unhandled case, must have at least two fields, not %d") at /global/u2/m/madams/petsc_private/src/sys/error/err.c:353 On Sat, Nov 30, 2013 at 6:33 PM, Jed Brown wrote: > Mark Adams writes: > > > I tried putting what I thought was the problem in ex54 and you are > correct > > it did not cause a problem. > > Okay. > > > I've attached a good and bad "petsc.rc" file from my app and an output > file > > with this error. Perhaps you can see how this broke. > > I don't have your app so I don't know what that code looks like. The > two options files are very different. Presumably you can run on one > process and use a debugger to see what happens in > PCFieldSplitSetDefaults. If you think PETSc is mishandling this, please > add a test to PETSc (perhaps a small addition to an existing test) to > demonstrate the issue. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jedbrown at mcs.anl.gov Sun Dec 1 11:12:33 2013 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Sun, 01 Dec 2013 11:12:33 -0600 Subject: [petsc-users] fieldsplit: Unhandled case, must have at least two fields, not 1! In-Reply-To: References: <87d2liblo7.fsf@jedbrown.org> <87haata2io.fsf@jedbrown.org> Message-ID: <87zjok8pha.fsf@jedbrown.org> Mark Adams writes: > Perhaps this will help. The code is straight forward but prefixes are used. > I've appended the code and a stack trace for the error. > > ! create KSP > call KSPCreate(solver%comm,solver%ksp,ierr) > CHKERRQ(ierr) > if (solver%prefix.eq.2) then ! turb > call KSPSetOptionsPrefix(solver%ksp,'s2_',ierr) > if (sml_mype/sml_pe_per_plane .gt. 0) then ! not first, no monitor > call PetscOptionsClearValue('-s2_ksp_monitor',ierr) > call PetscOptionsClearValue('-s2_ksp_converged_reason',ierr) > endif > else ! 1 field 00 solver, no prefix > call PetscOptionsClearValue('-ksp_monitor',ierr) > call PetscOptionsClearValue('-ksp_converged_reason',ierr) Yuck. The options table is for the (run-time) user. Manually futzing with options, especially clearing options, is a dirty hack that should not be used by choice. > end if > call KSPSetFromOptions(solver%ksp,ierr) > CHKERRQ(ierr) > > call KSPSetOperators(solver%ksp, solver%Amat, solver%Amat, > SAME_NONZERO_PATTERN, ierr ) > CHKERRQ(ierr) > > ! setup solver now that matrix is complete > call KSPSetUp( solver%ksp, ierr ) !!! poisson.F90:213 Where does your code call PCFieldSplitSetIS() or similar? How is the PCFieldSplit supposed to know about the splits? What runtime options are you passing? -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From mfadams at lbl.gov Sun Dec 1 11:30:09 2013 From: mfadams at lbl.gov (Mark Adams) Date: Sun, 1 Dec 2013 12:30:09 -0500 Subject: [petsc-users] fieldsplit: Unhandled case, must have at least two fields, not 1! In-Reply-To: <87zjok8pha.fsf@jedbrown.org> References: <87d2liblo7.fsf@jedbrown.org> <87haata2io.fsf@jedbrown.org> <87zjok8pha.fsf@jedbrown.org> Message-ID: > > > else ! 1 field 00 solver, no prefix > > call PetscOptionsClearValue('-ksp_monitor',ierr) > > call PetscOptionsClearValue('-ksp_converged_reason',ierr) > > Yuck. The options table is for the (run-time) user. Manually futzing > with options, especially clearing options, is a dirty hack that should > not be used by choice. > > What is an alternative? We have several solves on subcommuncators and I don't want to clutter the output with the monitors from all of these solves. This is effective and it is just used for diagnostics. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jedbrown at mcs.anl.gov Sun Dec 1 11:37:16 2013 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Sun, 01 Dec 2013 11:37:16 -0600 Subject: [petsc-users] fieldsplit: Unhandled case, must have at least two fields, not 1! In-Reply-To: References: <87d2liblo7.fsf@jedbrown.org> <87haata2io.fsf@jedbrown.org> <87zjok8pha.fsf@jedbrown.org> Message-ID: <87wqjo8oc3.fsf@jedbrown.org> Mark Adams writes: > What is an alternative? We have several solves on subcommuncators and I > don't want to clutter the output with the monitors from all of these > solves. This is effective and it is just used for diagnostics. Set different prefixes for each solver (or group, if on different subcomms) and only -prefix123_ksp_monitor the ones you want to see. Note that you can monitor to different files -prefix123_ksp_monitor output123.log -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From mfadams at lbl.gov Sun Dec 1 12:23:56 2013 From: mfadams at lbl.gov (Mark Adams) Date: Sun, 1 Dec 2013 13:23:56 -0500 Subject: [petsc-users] fieldsplit: Unhandled case, must have at least two fields, not 1! In-Reply-To: <87zjok8pha.fsf@jedbrown.org> References: <87d2liblo7.fsf@jedbrown.org> <87haata2io.fsf@jedbrown.org> <87zjok8pha.fsf@jedbrown.org> Message-ID: > > > > ! setup solver now that matrix is complete > > call KSPSetUp( solver%ksp, ierr ) !!! poisson.F90:213 > > Where does your code call PCFieldSplitSetIS() or similar? I use DM's. I've appended some of the code and attached the whole file. > How is the > PCFieldSplit supposed to know about the splits? What runtime options > are you passing? > I just use the petsc.rc files that I sent before. We don't usually use command line options. Mark ! lambda DM call VecCreate(comm,x2Vec,ierr) call VecSetSizes(x2Vec,N2loc,nflux,ierr) call VecSetFromOptions(x2Vec,ierr) call DMShellCreate(comm,solver%dalam,ierr) call DMShellSetGlobalVector(solver%dalam,x2Vec,ierr) call DMShellSetMatrix(solver%dalam,solver%Dmat,ierr) call VecCreate(PETSC_COMM_SELF,x2Vecloc,ierr) call VecSetSizes(x2Vecloc,N2loc,N2loc,ierr) call VecSetFromOptions(x2Vecloc,ierr) call DMShellSetLocalVector(solver%dalam,x2Vecloc,ierr) call VecDestroy(x2Vec,ierr) call VecDestroy(x2Vecloc,ierr) ! - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - ! Create field split DM ! - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - call DMCompositeCreate(comm,solver%da,ierr) CHKERRQ(ierr) call DMSetOptionsPrefix(solver%daphi,'phi_',ierr) call DMCompositeAddDM(solver%da,solver%daphi,ierr) CHKERRQ(ierr) call DMSetOptionsPrefix(solver%dalam,'lambda_',ierr) call DMCompositeAddDM(solver%da,solver%dalam,ierr) CHKERRQ(ierr) matArray(1) = solver%Amat matArray(2) = solver%Bmat matArray(3) = solver%Cmat matArray(4) = solver%Dmat call MatCreateNest(comm,itwo,PETSC_NULL_OBJECT,itwo,PETSC_NULL_OBJECT,matArray,solver%KKTmat,ierr) call MatSetFromOptions(solver%KKTmat,ierr) ! Extract global and local vectors from DM; then duplicate for remaining ! vectors that are the same types call MatGetVecs(solver%KKTmat,solver%xVec2,solver%bVec2,ierr) call VecDuplicate(solver%bVec2,solver%rVec2,ierr) ! create SNES call SNESCreate(comm,solver%snes,ierr) CHKERRQ(ierr) call SNESSetDM(solver%snes,solver%da,ierr) CHKERRQ(ierr) -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: poisson.F90 Type: application/octet-stream Size: 90388 bytes Desc: not available URL: From jedbrown at mcs.anl.gov Sun Dec 1 15:40:58 2013 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Sun, 01 Dec 2013 15:40:58 -0600 Subject: [petsc-users] fieldsplit: Unhandled case, must have at least two fields, not 1! In-Reply-To: References: <87d2liblo7.fsf@jedbrown.org> <87haata2io.fsf@jedbrown.org> <87zjok8pha.fsf@jedbrown.org> Message-ID: <87r49w8d1x.fsf@jedbrown.org> Mark Adams writes: >> >> >> > ! setup solver now that matrix is complete >> > call KSPSetUp( solver%ksp, ierr ) !!! poisson.F90:213 >> >> Where does your code call PCFieldSplitSetIS() or similar? > > > I use DM's. I've appended some of the code and attached the whole file. You haven't given me enough to build this so you'll have to open a debugger. Your example should be executing the following at fieldsplit.c:356: if (i == 0) { for (f = 0; f < numFields; ++f) { ierr = PCFieldSplitSetIS(pc, fieldNames[f], fields[f]);CHKERRQ(ierr); ierr = PetscFree(fieldNames[f]);CHKERRQ(ierr); ierr = ISDestroy(&fields[f]);CHKERRQ(ierr); } numFields should be 2. Compare to src/snes/examples/tutorials/ex28.c (adapting runex28_3), ./ex28 -da_grid_x 20 -snes_converged_reason -snes_monitor_short -ksp_monitor_short -problem_type 2 -snes_mf_operator -pack_dm_mat_type nest -pc_type fieldsplit -pc_fieldsplit_dm_splits -pc_fieldsplit_type schur -fieldsplit_u_ksp_type gmres -fieldsplit_k_pc_type jacobi -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From gideon.simpson at gmail.com Sun Dec 1 22:47:41 2013 From: gideon.simpson at gmail.com (Gideon Simpson) Date: Sun, 1 Dec 2013 23:47:41 -0500 Subject: [petsc-users] petsc vector management for monte carlo In-Reply-To: <87mwkla8a9.fsf@jedbrown.org> References: <76D0CDF7-BEBB-4447-B22F-D98A392F5BAE@gmail.com> <87vbz9aati.fsf@jedbrown.org> <87pppha911.fsf@jedbrown.org> <9899E8FF-0DCD-4800-A343-F9E357662BD6@gmail.com> <87mwkla8a9.fsf@jedbrown.org> Message-ID: A yes, you're right, it really just formats things as though it's a .m file to be executed. By the way, I can't seem to find any documentation about how, after doing: PetscViewerBinaryOpen(PETSC_COMM_WORLD,"sim_data.bin", FILE_MODE_WRITE, &output_viewer); VecView(sim_data,output_viewer); PetscViewerDestroy(&output_viewer); how to load the "sim_data.bin" in matlab. -gideon On Nov 30, 2013, at 4:28 PM, Jed Brown wrote: > Gideon Simpson writes: > >> Thanks guys, and I will switch over to binary. Just saying, that's what 2013b is doing. > > I don't think the ASCII_MATLAB format was ever intended to be loaded > with "load". You should be able to source it. From jedbrown at mcs.anl.gov Sun Dec 1 23:08:46 2013 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Sun, 01 Dec 2013 23:08:46 -0600 Subject: [petsc-users] petsc vector management for monte carlo In-Reply-To: References: <76D0CDF7-BEBB-4447-B22F-D98A392F5BAE@gmail.com> <87vbz9aati.fsf@jedbrown.org> <87pppha911.fsf@jedbrown.org> <9899E8FF-0DCD-4800-A343-F9E357662BD6@gmail.com> <87mwkla8a9.fsf@jedbrown.org> Message-ID: <87a9gj96w1.fsf@jedbrown.org> Gideon Simpson writes: > A yes, you're right, it really just formats things as though it's a .m file to be executed. > > By the way, I can't seem to find any documentation about how, after doing: > > PetscViewerBinaryOpen(PETSC_COMM_WORLD,"sim_data.bin", FILE_MODE_WRITE, &output_viewer); > VecView(sim_data,output_viewer); > PetscViewerDestroy(&output_viewer); > > > how to load the "sim_data.bin" in matlab. Add $PETSC_DIR/bin/matlab to your MATLAB path and run x = PetscBinaryRead('sim_data.bin'); And yes, I see that chapter of the user's manual definitely gives the wrong idea. Will fix it. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From lawrence.mitchell at imperial.ac.uk Mon Dec 2 02:26:23 2013 From: lawrence.mitchell at imperial.ac.uk (Lawrence Mitchell) Date: Mon, 2 Dec 2013 08:26:23 +0000 Subject: [petsc-users] Recursive fieldsplit PCs Message-ID: <6A52250F-A0CD-44D2-8B42-70514E4BF98F@imperial.ac.uk> Dear petsc-users, I have a 3x3 block system built as a single MatNest (with 9 Mats in it). I'd like to treat this as a 2x2 system: [ A B C D ] where A is 2x2 and precondition the A block with a schur complement. Since I have a mat nest, the ISes for the three fields are just ISCreateStride(..., mat_i_rows, offset, 1, ...) and these are set on the fieldsplit pc. If I understand the documentation correctly, I think I should now be able to do: -pc_type fieldsplit -fieldsplit_0_fields 0,1 -fieldsplit_1_fields 2 -fieldsplit_0_pc_type field split -fieldsplit_0_pc_fieldsplit_type schur ... However, when doing so, I get an error: "To use Schur complement preconditioner you must have exactly 2 fields". Which suggests to me I have failed to inform PETSc that I want the first two fields to be treated as 1. Note that I am not using a DM to build any of these objects. I build a SNES, pull the KSP out of the SNES and then the PC out of the KSP. I never explicitly call SetFromOptions on the PC. Instead, before the SNES solve I call SNESSetFromOptions. Might this be the problem? Cheers, Lawrence From knepley at gmail.com Mon Dec 2 06:23:31 2013 From: knepley at gmail.com (Matthew Knepley) Date: Mon, 2 Dec 2013 06:23:31 -0600 Subject: [petsc-users] Recursive fieldsplit PCs In-Reply-To: <6A52250F-A0CD-44D2-8B42-70514E4BF98F@imperial.ac.uk> References: <6A52250F-A0CD-44D2-8B42-70514E4BF98F@imperial.ac.uk> Message-ID: On Mon, Dec 2, 2013 at 2:26 AM, Lawrence Mitchell < lawrence.mitchell at imperial.ac.uk> wrote: > Dear petsc-users, > > I have a 3x3 block system built as a single MatNest (with 9 Mats in it). > I'd like to treat this as a 2x2 system: > [ A B > C D ] > > where A is 2x2 and precondition the A block with a schur complement. > Since I have a mat nest, the ISes for the three fields are just > ISCreateStride(..., mat_i_rows, offset, 1, ...) and these are set on the > fieldsplit pc. If I understand the documentation correctly, I think I > should now be able to do: > > -pc_type fieldsplit -fieldsplit_0_fields 0,1 -fieldsplit_1_fields 2 > -fieldsplit_0_pc_type field split -fieldsplit_0_pc_fieldsplit_type schur > > ... > > However, when doing so, I get an error: "To use Schur complement > preconditioner you must have exactly 2 fields". Which suggests to me I > have failed to inform PETSc that I want the first two fields to be treated > as 1. > > Note that I am not using a DM to build any of these objects. I build a > SNES, pull the KSP out of the SNES and then the PC out of the KSP. I never > explicitly call SetFromOptions on the PC. Instead, before the SNES solve I > call SNESSetFromOptions. Might this be the problem? > This is an unfortunately limitation of the implementation right now. This option works if you are on a DA with collocation, or if you use a DM, but not if you just specify the ISes. We should probably write that code. However, the idea is for people to be moving to using DM. Could you tell us why DM did not work for you here? Thanks, Matt > Cheers, > > Lawrence -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From lawrence.mitchell at imperial.ac.uk Mon Dec 2 06:31:55 2013 From: lawrence.mitchell at imperial.ac.uk (Lawrence Mitchell) Date: Mon, 2 Dec 2013 12:31:55 +0000 Subject: [petsc-users] Recursive fieldsplit PCs In-Reply-To: References: <6A52250F-A0CD-44D2-8B42-70514E4BF98F@imperial.ac.uk> Message-ID: <4E8A4CBF-4B9C-4A6D-A6FE-E039BB81DBEA@imperial.ac.uk> On 2 Dec 2013, at 12:23, Matthew Knepley wrote: > On Mon, Dec 2, 2013 at 2:26 AM, Lawrence Mitchell wrote: > Dear petsc-users, > > I have a 3x3 block system built as a single MatNest (with 9 Mats in it). I'd like to treat this as a 2x2 system: > [ A B > C D ] > > where A is 2x2 and precondition the A block with a schur complement. Since I have a mat nest, the ISes for the three fields are just ISCreateStride(..., mat_i_rows, offset, 1, ...) and these are set on the fieldsplit pc. If I understand the documentation correctly, I think I should now be able to do: > > -pc_type fieldsplit -fieldsplit_0_fields 0,1 -fieldsplit_1_fields 2 -fieldsplit_0_pc_type field split -fieldsplit_0_pc_fieldsplit_type schur > > ... > > However, when doing so, I get an error: "To use Schur complement preconditioner you must have exactly 2 fields". Which suggests to me I have failed to inform PETSc that I want the first two fields to be treated as 1. > > Note that I am not using a DM to build any of these objects. I build a SNES, pull the KSP out of the SNES and then the PC out of the KSP. I never explicitly call SetFromOptions on the PC. Instead, before the SNES solve I call SNESSetFromOptions. Might this be the problem? > > This is an unfortunately limitation of the implementation right now. This option works if you are on a DA with collocation, or if you > use a DM, but not if you just specify the ISes. We should probably write that code. However, the idea is for people to be moving > to using DM. Could you tell us why DM did not work for you here? At the moment, our mesh infrastructure is not plumbed in to use petsc data structures. We're attempting to migrate to dmplex at the moment but currently we're building Mats from third-party connectivity information. It's possible that the move to dmplex will happen soon enough that it isn't a big issue, but I don't know the timescales on which we'll be done with that. Cheers, Lawrence From mfadams at lbl.gov Mon Dec 2 07:53:28 2013 From: mfadams at lbl.gov (Mark Adams) Date: Mon, 2 Dec 2013 08:53:28 -0500 Subject: [petsc-users] fieldsplit: Unhandled case, must have at least two fields, not 1! In-Reply-To: <87r49w8d1x.fsf@jedbrown.org> References: <87d2liblo7.fsf@jedbrown.org> <87haata2io.fsf@jedbrown.org> <87zjok8pha.fsf@jedbrown.org> <87r49w8d1x.fsf@jedbrown.org> Message-ID: > > > > if (i == 0) { > for (f = 0; f < numFields; ++f) { > ierr = PCFieldSplitSetIS(pc, fieldNames[f], > fields[f]);CHKERRQ(ierr); > ierr = PetscFree(fieldNames[f]);CHKERRQ(ierr); > ierr = ISDestroy(&fields[f]);CHKERRQ(ierr); > } > > numFields should be 2. Compare to src/snes/examples/tutorials/ex28.c > (adapting runex28_3), > The matrix/solver just has one field. The error comes from giving it field split parameters by mistake. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jedbrown at mcs.anl.gov Mon Dec 2 08:04:32 2013 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Mon, 02 Dec 2013 08:04:32 -0600 Subject: [petsc-users] fieldsplit: Unhandled case, must have at least two fields, not 1! In-Reply-To: References: <87d2liblo7.fsf@jedbrown.org> <87haata2io.fsf@jedbrown.org> <87zjok8pha.fsf@jedbrown.org> <87r49w8d1x.fsf@jedbrown.org> Message-ID: <874n6r8i33.fsf@jedbrown.org> Mark Adams writes: >> >> >> >> if (i == 0) { >> for (f = 0; f < numFields; ++f) { >> ierr = PCFieldSplitSetIS(pc, fieldNames[f], >> fields[f]);CHKERRQ(ierr); >> ierr = PetscFree(fieldNames[f]);CHKERRQ(ierr); >> ierr = ISDestroy(&fields[f]);CHKERRQ(ierr); >> } >> >> numFields should be 2. Compare to src/snes/examples/tutorials/ex28.c >> (adapting runex28_3), >> > > The matrix/solver just has one field. The error comes from giving it field > split parameters by mistake. So this is a different KSP; not the one you want to use fieldsplit on? Please always use different prefixes for different solvers. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From mfadams at lbl.gov Mon Dec 2 08:16:36 2013 From: mfadams at lbl.gov (Mark Adams) Date: Mon, 2 Dec 2013 09:16:36 -0500 Subject: [petsc-users] fieldsplit: Unhandled case, must have at least two fields, not 1! In-Reply-To: <874n6r8i33.fsf@jedbrown.org> References: <87d2liblo7.fsf@jedbrown.org> <87haata2io.fsf@jedbrown.org> <87zjok8pha.fsf@jedbrown.org> <87r49w8d1x.fsf@jedbrown.org> <874n6r8i33.fsf@jedbrown.org> Message-ID: > > > So this is a different KSP; not the one you want to use fieldsplit on? > Yes, > > Please always use different prefixes for different solvers. > We use prefixes. I left them out of the example. So things like '-s2_pc_type fieldsplit' cause an error. The 's2_' solver is a one field (Laplacian) solve. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jedbrown at mcs.anl.gov Mon Dec 2 08:42:44 2013 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Mon, 02 Dec 2013 08:42:44 -0600 Subject: [petsc-users] fieldsplit: Unhandled case, must have at least two fields, not 1! In-Reply-To: References: <87d2liblo7.fsf@jedbrown.org> <87haata2io.fsf@jedbrown.org> <87zjok8pha.fsf@jedbrown.org> <87r49w8d1x.fsf@jedbrown.org> <874n6r8i33.fsf@jedbrown.org> Message-ID: <871u1v8gbf.fsf@jedbrown.org> Mark Adams writes: > We use prefixes. I left them out of the example. So things like > '-s2_pc_type fieldsplit' cause an error. The 's2_' solver is a one field > (Laplacian) solve. So when you say to use fieldsplit for a solve with only one field, you get an error saying that fieldsplit requires at least two fields? 1. Don't do that. 2. Sounds like the error message is correct. 3. Perhaps consider using a more intuitive prefix than "s2_". When someone unfamiliar with your code sees "-s2_pc_type fieldsplit" their first thought will be that "s2_" is a multi-field solve of some sort. If they instead saw "-laplacian_pc_type fieldsplit" or "-pressure_pc_type fieldsplit", they might ask why fieldsplit was being attempted for what seems to be a scalar solve. Let us know if you see a way to improve the message or handling of this case. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From knepley at gmail.com Mon Dec 2 09:20:59 2013 From: knepley at gmail.com (Matthew Knepley) Date: Mon, 2 Dec 2013 09:20:59 -0600 Subject: [petsc-users] Recursive fieldsplit PCs In-Reply-To: <4E8A4CBF-4B9C-4A6D-A6FE-E039BB81DBEA@imperial.ac.uk> References: <6A52250F-A0CD-44D2-8B42-70514E4BF98F@imperial.ac.uk> <4E8A4CBF-4B9C-4A6D-A6FE-E039BB81DBEA@imperial.ac.uk> Message-ID: On Mon, Dec 2, 2013 at 6:31 AM, Lawrence Mitchell < lawrence.mitchell at imperial.ac.uk> wrote: > > On 2 Dec 2013, at 12:23, Matthew Knepley wrote: > > > On Mon, Dec 2, 2013 at 2:26 AM, Lawrence Mitchell < > lawrence.mitchell at imperial.ac.uk> wrote: > > Dear petsc-users, > > > > I have a 3x3 block system built as a single MatNest (with 9 Mats in it). > I'd like to treat this as a 2x2 system: > > [ A B > > C D ] > > > > where A is 2x2 and precondition the A block with a schur complement. > Since I have a mat nest, the ISes for the three fields are just > ISCreateStride(..., mat_i_rows, offset, 1, ...) and these are set on the > fieldsplit pc. If I understand the documentation correctly, I think I > should now be able to do: > > > > -pc_type fieldsplit -fieldsplit_0_fields 0,1 -fieldsplit_1_fields 2 > -fieldsplit_0_pc_type field split -fieldsplit_0_pc_fieldsplit_type schur > > > > ... > > > > However, when doing so, I get an error: "To use Schur complement > preconditioner you must have exactly 2 fields". Which suggests to me I > have failed to inform PETSc that I want the first two fields to be treated > as 1. > > > > Note that I am not using a DM to build any of these objects. I build a > SNES, pull the KSP out of the SNES and then the PC out of the KSP. I never > explicitly call SetFromOptions on the PC. Instead, before the SNES solve I > call SNESSetFromOptions. Might this be the problem? > > > > This is an unfortunately limitation of the implementation right now. > This option works if you are on a DA with collocation, or if you > > use a DM, but not if you just specify the ISes. We should probably write > that code. However, the idea is for people to be moving > > to using DM. Could you tell us why DM did not work for you here? > > At the moment, our mesh infrastructure is not plumbed in to use petsc data > structures. We're attempting to migrate to dmplex at the moment but > currently we're building Mats from third-party connectivity information. > It's possible that the move to dmplex will happen soon enough that it > isn't a big issue, but I don't know the timescales on which we'll be done > with that. > I was feeling bad about not implementing this, but I remembered the problem with the pure IS solution. The obvious solution is to just combine several ISes to create the field for PCFIELDSPLIT. However, once this is done, they lose their identity as separate fields. Thus, it is not possible to untangle for the second level of FieldSplit that you want. The DM version maintains the field identity at the next level so that we can split hierarchically. So, for the above to work, I think you must use a DM. Thanks, Matt > Cheers, > > Lawrence -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From mfadams at lbl.gov Mon Dec 2 09:26:02 2013 From: mfadams at lbl.gov (Mark Adams) Date: Mon, 2 Dec 2013 10:26:02 -0500 Subject: [petsc-users] fieldsplit: Unhandled case, must have at least two fields, not 1! In-Reply-To: <871u1v8gbf.fsf@jedbrown.org> References: <87d2liblo7.fsf@jedbrown.org> <87haata2io.fsf@jedbrown.org> <87zjok8pha.fsf@jedbrown.org> <87r49w8d1x.fsf@jedbrown.org> <874n6r8i33.fsf@jedbrown.org> <871u1v8gbf.fsf@jedbrown.org> Message-ID: > > > So when you say to use fieldsplit for a solve with only one field, you > get an error saying that fieldsplit requires at least two fields? > > Yes > 1. Don't do that. > > My point was that this user got this message and so I suggesting a possible problem: giving fieldsplit parameters to a single field matrix/solver. -------------- next part -------------- An HTML attachment was scrubbed... URL: From lawrence.mitchell at imperial.ac.uk Mon Dec 2 09:47:33 2013 From: lawrence.mitchell at imperial.ac.uk (Lawrence Mitchell) Date: Mon, 2 Dec 2013 15:47:33 +0000 Subject: [petsc-users] Recursive fieldsplit PCs In-Reply-To: References: <6A52250F-A0CD-44D2-8B42-70514E4BF98F@imperial.ac.uk> <4E8A4CBF-4B9C-4A6D-A6FE-E039BB81DBEA@imperial.ac.uk> Message-ID: On 2 Dec 2013, at 15:20, Matthew Knepley wrote: > On Mon, Dec 2, 2013 at 6:31 AM, Lawrence Mitchell wrote: > > On 2 Dec 2013, at 12:23, Matthew Knepley wrote: > > > On Mon, Dec 2, 2013 at 2:26 AM, Lawrence Mitchell wrote: > > Dear petsc-users, > > > > I have a 3x3 block system built as a single MatNest (with 9 Mats in it). I'd like to treat this as a 2x2 system: > > [ A B > > C D ] > > > > where A is 2x2 and precondition the A block with a schur complement. Since I have a mat nest, the ISes for the three fields are just ISCreateStride(..., mat_i_rows, offset, 1, ...) and these are set on the fieldsplit pc. If I understand the documentation correctly, I think I should now be able to do: > > > > -pc_type fieldsplit -fieldsplit_0_fields 0,1 -fieldsplit_1_fields 2 -fieldsplit_0_pc_type field split -fieldsplit_0_pc_fieldsplit_type schur > > > > ... > > > > However, when doing so, I get an error: "To use Schur complement preconditioner you must have exactly 2 fields". Which suggests to me I have failed to inform PETSc that I want the first two fields to be treated as 1. > > > > Note that I am not using a DM to build any of these objects. I build a SNES, pull the KSP out of the SNES and then the PC out of the KSP. I never explicitly call SetFromOptions on the PC. Instead, before the SNES solve I call SNESSetFromOptions. Might this be the problem? > > > > This is an unfortunately limitation of the implementation right now. This option works if you are on a DA with collocation, or if you > > use a DM, but not if you just specify the ISes. We should probably write that code. However, the idea is for people to be moving > > to using DM. Could you tell us why DM did not work for you here? > > At the moment, our mesh infrastructure is not plumbed in to use petsc data structures. We're attempting to migrate to dmplex at the moment but currently we're building Mats from third-party connectivity information. It's possible that the move to dmplex will happen soon enough that it isn't a big issue, but I don't know the timescales on which we'll be done with that. > > I was feeling bad about not implementing this, but I remembered the problem with the pure IS solution. The obvious solution > is to just combine several ISes to create the field for PCFIELDSPLIT. However, once this is done, they lose their identity as > separate fields. Thus, it is not possible to untangle for the second level of FieldSplit that you want. The DM version maintains > the field identity at the next level so that we can split hierarchically. So, for the above to work, I think you must use a DM. OK, I think I see. I wonder if, until I have a DM in place, I can hack this together by building my MatNest recursively? That is build A = [A' B' C' D'] and then build F = [A B C D] would that work, or am I barking up the wrong tree? Cheers, Lawrence From jedbrown at mcs.anl.gov Mon Dec 2 10:03:50 2013 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Mon, 02 Dec 2013 10:03:50 -0600 Subject: [petsc-users] fieldsplit: Unhandled case, must have at least two fields, not 1! In-Reply-To: References: <87d2liblo7.fsf@jedbrown.org> <87haata2io.fsf@jedbrown.org> <87zjok8pha.fsf@jedbrown.org> <87r49w8d1x.fsf@jedbrown.org> <874n6r8i33.fsf@jedbrown.org> <871u1v8gbf.fsf@jedbrown.org> Message-ID: <87siub6xzt.fsf@jedbrown.org> Mark Adams writes: > My point was that this user got this message and so I suggesting a possible > problem: giving fieldsplit parameters to a single field matrix/solver. Okay, I interpreted your original statement as being that this error message has something to do with using monolithic matrix formats (such as AIJ, versus Nest), which it does not, and that you thought the error message was problematic behavior. Your very complicated test case (hundreds of processes, with multiple solves, some intending to use fieldsplit, Schur, MatNest, tinkering with the options database, huge differences between the two options files) can be reduced to src/ksp/ksp/examples/tutorials/ex2.c. $ ./ex2 -pc_type fieldsplit [0]PETSC ERROR: --------------------- Error Message ------------------------------------ [0]PETSC ERROR: Petsc has generated inconsistent data! [0]PETSC ERROR: Unhandled case, must have at least two fields, not 1! [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: Petsc Development GIT revision: ad348534bf2da17dbc29a6a7895b244965ac0e39 GIT Date: 2013-11-13 12:43:46 -0700 [0]PETSC ERROR: See docs/changes/index.html for recent updates. [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. [0]PETSC ERROR: See docs/index.html for manual pages. [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: ./ex2 on a mpich named batura by jed Mon Dec 2 09:55:54 2013 [0]PETSC ERROR: Libraries linked from /home/jed/petsc/mpich/lib [0]PETSC ERROR: Configure run at Wed Nov 13 13:33:49 2013 [0]PETSC ERROR: Configure options --download-blacs --download-chaco --download-ctetgen --download-elemental --download-generator --download-hypre --download-metis --download-ml --download-mumps --download-parmetis --download-scalapack --download-spai --download-spooles --download-sundials --download-superlu --download-superlu_dist --download-triangle --with-blas-lapack=/usr --with-c2html --with-cholmod-dir=/usr --with-cxx-dialect=C++11 --with-exodusii-dir=/usr --with-hdf5-dir=/opt/mpich --with-lgrind --with-mpi-dir=/home/jed/usr/ccache/mpich/ --with-netcdf-dir=/usr --with-openmp --with-pcbddc --with-pthread --with-shared-libraries --with-single-library=0 --with-sowing --with-threadcomm --with-umfpack-dir=/usr --with-x -PETSC_ARCH=mpich COPTFLAGS="-Og -g" CXXOPTFLAGS="-Og -g" FOPTFLAGS="-Og -g" [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: PCFieldSplitSetDefaults() line 441 in src/ksp/pc/impls/fieldsplit/fieldsplit.c [0]PETSC ERROR: PCSetUp_FieldSplit() line 459 in src/ksp/pc/impls/fieldsplit/fieldsplit.c [0]PETSC ERROR: PCSetUp() line 888 in src/ksp/pc/interface/precon.c [0]PETSC ERROR: KSPSetUp() line 278 in src/ksp/ksp/interface/itfunc.c [0]PETSC ERROR: KSPSolve() line 399 in src/ksp/ksp/interface/itfunc.c [0]PETSC ERROR: main() line 198 in /home/jed/petsc/src/ksp/ksp/examples/tutorials/ex2.c application called MPI_Abort(MPI_COMM_WORLD, 77) - process 0 -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From knepley at gmail.com Mon Dec 2 10:26:28 2013 From: knepley at gmail.com (Matthew Knepley) Date: Mon, 2 Dec 2013 10:26:28 -0600 Subject: [petsc-users] Recursive fieldsplit PCs In-Reply-To: References: <6A52250F-A0CD-44D2-8B42-70514E4BF98F@imperial.ac.uk> <4E8A4CBF-4B9C-4A6D-A6FE-E039BB81DBEA@imperial.ac.uk> Message-ID: On Mon, Dec 2, 2013 at 9:47 AM, Lawrence Mitchell < lawrence.mitchell at imperial.ac.uk> wrote: > > On 2 Dec 2013, at 15:20, Matthew Knepley wrote: > > > On Mon, Dec 2, 2013 at 6:31 AM, Lawrence Mitchell < > lawrence.mitchell at imperial.ac.uk> wrote: > > > > On 2 Dec 2013, at 12:23, Matthew Knepley wrote: > > > > > On Mon, Dec 2, 2013 at 2:26 AM, Lawrence Mitchell < > lawrence.mitchell at imperial.ac.uk> wrote: > > > Dear petsc-users, > > > > > > I have a 3x3 block system built as a single MatNest (with 9 Mats in > it). I'd like to treat this as a 2x2 system: > > > [ A B > > > C D ] > > > > > > where A is 2x2 and precondition the A block with a schur complement. > Since I have a mat nest, the ISes for the three fields are just > ISCreateStride(..., mat_i_rows, offset, 1, ...) and these are set on the > fieldsplit pc. If I understand the documentation correctly, I think I > should now be able to do: > > > > > > -pc_type fieldsplit -fieldsplit_0_fields 0,1 -fieldsplit_1_fields 2 > -fieldsplit_0_pc_type field split -fieldsplit_0_pc_fieldsplit_type schur > > > > > > ... > > > > > > However, when doing so, I get an error: "To use Schur complement > preconditioner you must have exactly 2 fields". Which suggests to me I > have failed to inform PETSc that I want the first two fields to be treated > as 1. > > > > > > Note that I am not using a DM to build any of these objects. I build > a SNES, pull the KSP out of the SNES and then the PC out of the KSP. I > never explicitly call SetFromOptions on the PC. Instead, before the SNES > solve I call SNESSetFromOptions. Might this be the problem? > > > > > > This is an unfortunately limitation of the implementation right now. > This option works if you are on a DA with collocation, or if you > > > use a DM, but not if you just specify the ISes. We should probably > write that code. However, the idea is for people to be moving > > > to using DM. Could you tell us why DM did not work for you here? > > > > At the moment, our mesh infrastructure is not plumbed in to use petsc > data structures. We're attempting to migrate to dmplex at the moment but > currently we're building Mats from third-party connectivity information. > It's possible that the move to dmplex will happen soon enough that it > isn't a big issue, but I don't know the timescales on which we'll be done > with that. > > > > I was feeling bad about not implementing this, but I remembered the > problem with the pure IS solution. The obvious solution > > is to just combine several ISes to create the field for PCFIELDSPLIT. > However, once this is done, they lose their identity as > > separate fields. Thus, it is not possible to untangle for the second > level of FieldSplit that you want. The DM version maintains > > the field identity at the next level so that we can split > hierarchically. So, for the above to work, I think you must use a DM. > > OK, I think I see. I wonder if, until I have a DM in place, I can hack > this together by building my MatNest recursively? That is build > > A = [A' B' > C' D'] > > and then build > > F = [A B > C D] > > would that work, or am I barking up the wrong tree? > MatNest is only an optimization. It makes no-copy extraction of submatrices feasible. It is not an interface, and thus does not tell FieldSplit anything. Everything you can do with MatNest you can do without it. Matt > Cheers, > > Lawrence > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From mpovolot at purdue.edu Mon Dec 2 11:47:46 2013 From: mpovolot at purdue.edu (Michael Povolotskyi) Date: Mon, 02 Dec 2013 17:47:46 +0000 Subject: [petsc-users] questions about contour integration solvers in SLEPc Message-ID: <529CC7C2.6020903@purdue.edu> Dear Developers of SLEPc, I have found that SLEPc has now two eigenvalues solver that are based on contour integration, rather than on Krylov space method. They are EPSFEAST and EPSCISS. I'm eager to try these solvers in my code both for Hermitian and non-Hermitian problems. My questions are as follows: 1) Are there any tutorial examples for those solvers? 2) How is the algorithm of EPSCISS is different from EPSFEAST algorithm? Is it similar to one from http://prb.aps.org/abstract/PRB/v86/i7/e075103, where FEAST is extended to the non-Hermitian matrices? 3) Will I be able to configure SLEPC to use FEAST algorithm from MKL? Thank you, Michael. -- Michael Povolotskyi, PhD Research Assistant Professor Network for Computational Nanotechnology 207 S Martin Jischke Drive Purdue University, DLR, room 441-10 West Lafayette, Indiana 47907 phone: +1-765-494-9396 fax: +1-765-496-6026 From jroman at dsic.upv.es Tue Dec 3 02:34:13 2013 From: jroman at dsic.upv.es (Jose E. Roman) Date: Tue, 3 Dec 2013 09:34:13 +0100 Subject: [petsc-users] questions about contour integration solvers in SLEPc In-Reply-To: <529CC7C2.6020903@purdue.edu> References: <529CC7C2.6020903@purdue.edu> Message-ID: <28DB2FE3-542E-40BA-8888-30CF8E84E161@dsic.upv.es> El 02/12/2013, a las 18:47, Michael Povolotskyi escribi?: > Dear Developers of SLEPc, > I have found that SLEPc has now two eigenvalues solver that are based on contour integration, rather than on Krylov space method. > They are EPSFEAST and EPSCISS. > > I'm eager to try these solvers in my code both for Hermitian and non-Hermitian problems. > My questions are as follows: > 1) Are there any tutorial examples for those solvers? No. For EPSFEAST you specify an interval -eps_interval a,b and for EPSCISS you specify an ellipse containing the wanted eigenvalues, with -eps_ciss_center c -eps_ciss_radius r See more options in ciss.c > 2) How is the algorithm of EPSCISS is different from EPSFEAST algorithm? > Is it similar to one from http://prb.aps.org/abstract/PRB/v86/i7/e075103, where FEAST is extended to the non-Hermitian matrices? As far as I know FEAST does not support non-Hermitian problems. CISS does. See references cited at the top of ciss.c We will continue to improve CISS in the next SLEPc versions. If you want help trying it, send us an email to slepc-maint and we will guide you. > 3) Will I be able to configure SLEPC to use FEAST algorithm from MKL? I haven't tried this. Send errors/problems to slepc-maint. Jose > > Thank you, > Michael. From bisheshkh at gmail.com Tue Dec 3 08:29:25 2013 From: bisheshkh at gmail.com (Bishesh Khanal) Date: Tue, 3 Dec 2013 15:29:25 +0100 Subject: [petsc-users] solving stokes-like equation in 3D staggered grid in irregular domain with petscsection In-Reply-To: References: <874n7hb7zj.fsf@jedbrown.org> <87y54s57we.fsf@jedbrown.org> Message-ID: On Thu, Nov 14, 2013 at 1:05 PM, Bishesh Khanal wrote: > > > > On Wed, Nov 13, 2013 at 10:06 PM, Jed Brown wrote: > >> Bishesh Khanal writes: >> >> > Within A, for now, I can consider mu to be constant, although later if >> > possible it can be a variable even a tensor to describe anisotropy. But >> to >> > start with I want put this a constant. >> > The original equations start with mu (grad(u) + grad(u)^T) but then >> > simplifications occur due to div(u) = f2 >> >> Rework that step in case of variable mu. >> > >> > I'm mostly interested in the phenomenon in A with my model, here B is >> the >> > extension of the very irregular domain of A to get a cuboid. Here, in B >> I >> > release the div(u) = f2 constraint and just put a regularisation to >> > penalize large deformation. What is of importance here is to compensate >> the >> > net volume expansion in domain A by corresponding contraction in domain >> B >> > so that the boundaries of the cuboid do not move. It does not >> particularly >> > represent any physics except probably that it gives me a velocity field >> > having a certain divergence field that penalizes big deformations. >> >> Okay, sounds like it's already an artificial equation, so you should be >> able to leave in a normal equation for p, with a big mass matrix on the >> diagonal, >> >> div(mu(grad(u))) - grad(p) = f1 >> div(u) - c(x) p = f2 >> >> c(x) = 0 in domain A and c(x) is large (the inverse of the second Lam? >> parameter) in domain B. >> >> I tried to modify my existing code to incorporate c(x) instead of 0 as you suggested. When c(x) is zero everywhere, the solver converges and gives me the result I expect, as before. However, when c(x) is non-zero in domain B, the solver does not seem to converge. The ouput for both cases are given below. When c(x) was zero everywhere, I had set the constant pressure null space to the system matrix and the schur complement matrix since I am solving the system with pctype fieldsplit and schur. However, when c(x) is non-zero in domain B (I put c(x) = 1 in B for test), we do not have the constant null space for pressure anymore; so for this case I do not set the constant null space. Can you comment on what could be possibly going wrong ? 1. Output for the test case: c(x) = 0 in A, c(x) = 1 in B; mu = 1 everywhere. options given: -pc_type fieldsplit -pc_fieldsplit_type schur -pc_fieldsplit_dm_splits 0 -pc_fieldsplit_0_fields 0,1,2 -pc_fieldsplit_1_fields 3 -fieldsplit_0_pc_type lu -fieldsplit_0_pc_factor_mat_solver_package superlu_dist -fieldsplit_0_ksp_monitor -fieldsplit_0_ksp_converged_reason -fieldsplit_0_ksp_max_it 100 -fieldsplit_1_ksp_monitor -fieldsplit_1_ksp_converged_reason -ksp_converged_reason output: Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.321914527334e+01 1 KSP Residual norm 5.204444045635e-14 Linear solve converged due to CONVERGED_RTOL iterations 1 Residual norms for fieldsplit_1_ solve. 0 KSP Residual norm 2.890735677419e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.993511593752e+00 1 KSP Residual norm 4.050802627628e-15 Linear solve converged due to CONVERGED_RTOL iterations 1 1 KSP Residual norm 2.890735677419e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 6.284336607097e-01 1 KSP Residual norm 2.121905965324e-15 Linear solve converged due to CONVERGED_RTOL iterations 1 2 KSP Residual norm 2.890735677419e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.182615416676e+00 1 KSP Residual norm 6.481437789588e-15 Linear solve converged due to CONVERGED_RTOL iterations 1 3 KSP Residual norm 2.890735677419e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.470958139659e+00 1 KSP Residual norm 9.105094653121e-15 Linear solve converged due to CONVERGED_RTOL iterations 1 4 KSP Residual norm 2.890735677419e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.802783367675e+00 1 KSP Residual norm 1.307376269273e-14 Linear solve converged due to CONVERGED_RTOL iterations 1 5 KSP Residual norm 2.890735677419e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 9.352429007750e-01 1 KSP Residual norm 5.214147133013e-15 Linear solve converged due to CONVERGED_RTOL iterations 1 6 KSP Residual norm 2.890735677419e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 9.088208637857e-01 1 KSP Residual norm 5.202918034148e-15 Linear solve converged due to CONVERGED_RTOL iterations 1 7 KSP Residual norm 2.890735677419e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 8.607808656970e-01 1 KSP Residual norm 5.212231778254e-15 Linear solve converged due to CONVERGED_RTOL iterations 1 8 KSP Residual norm 2.890735677419e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.228224544246e+00 1 KSP Residual norm 7.032688183318e-15 Linear solve converged due to CONVERGED_RTOL iterations 1 9 KSP Residual norm 2.890735677419e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.238989110105e+00 1 KSP Residual norm 7.308937417623e-15 Linear solve converged due to CONVERGED_RTOL iterations 1 10 KSP Residual norm 2.890735677419e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 9.315232322259e-01 1 KSP Residual norm 4.434343770858e-15 Linear solve converged due to CONVERGED_RTOL iterations 1 11 KSP Residual norm 2.890735677419e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.070108662141e+00 1 KSP Residual norm 5.822139925590e-15 Linear solve converged due to CONVERGED_RTOL iterations 1 12 KSP Residual norm 2.890735677419e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.066025697964e+00 1 KSP Residual norm 6.055553792441e-15 Linear solve converged due to CONVERGED_RTOL iterations 1 13 KSP Residual norm 2.890735677419e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 7.981752531989e-01 1 KSP Residual norm 3.750650135599e-15 Linear solve converged due to CONVERGED_RTOL iterations 1 14 KSP Residual norm 2.890735677419e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 7.225821573201e-01 1 KSP Residual norm 3.043640976617e-15 Linear solve converged due to CONVERGED_RTOL iterations 1 15 KSP Residual norm 2.890735677419e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 8.223502060699e-01 1 KSP Residual norm 4.212671738422e-15 Linear solve converged due to CONVERGED_RTOL iterations 1 16 KSP Residual norm 2.890735677419e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 8.322451979979e-01 1 KSP Residual norm 2.999201599777e-15 Linear solve converged due to CONVERGED_RTOL iterations 1 17 KSP Residual norm 2.890735677419e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.052268760248e+00 1 KSP Residual norm 5.911203456590e-15 Linear solve converged due to CONVERGED_RTOL iterations 1 18 KSP Residual norm 2.890735677419e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.082754048106e+00 1 KSP Residual norm 5.992300533204e-15 Linear solve converged due to CONVERGED_RTOL iterations 1 19 KSP Residual norm 2.890735677419e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.061699839350e+00 ^C[mpiexec at edwards] Sending Ctrl-C to processes as requested and it keeps on going like this. The system is converging when I set incompressibility constraint everywhere. Here is a sample output for this case: c(x) = 0 everywhere, mu = 1 everywhere. Options used: -pc_type fieldsplit -pc_fieldsplit_type schur -pc_fieldsplit_dm_splits 0 -pc_fieldsplit_0_fields 0,1,2 -pc_fieldsplit_1_fields 3 -fieldsplit_0_pc_type lu -fieldsplit_0_pc_factor_mat_solver_package superlu_dist -fieldsplit_0_ksp_monitor -fieldsplit_0_ksp_converged_reason -fieldsplit_0_ksp_max_it 100 -fieldsplit_1_ksp_monitor -fieldsplit_1_ksp_converged_reason -ksp_converged_reason Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 0.000000000000e+00 Linear solve converged due to CONVERGED_ATOL iterations 0 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.321914527334e+01 1 KSP Residual norm 5.204444045635e-14 Linear solve converged due to CONVERGED_RTOL iterations 1 Residual norms for fieldsplit_1_ solve. 0 KSP Residual norm 9.964395910568e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.573745221888e+00 1 KSP Residual norm 3.612233230950e-15 Linear solve converged due to CONVERGED_RTOL iterations 1 1 KSP Residual norm 2.667315756754e+12 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.053280683677e+00 1 KSP Residual norm 5.770507184021e-15 Linear solve converged due to CONVERGED_RTOL iterations 1 2 KSP Residual norm 3.176176643790e+11 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.016103066894e+00 1 KSP Residual norm 4.558255925705e-15 Linear solve converged due to CONVERGED_RTOL iterations 1 3 KSP Residual norm 1.046851012581e+11 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.095000885520e+00 1 KSP Residual norm 6.356842242710e-15 Linear solve converged due to CONVERGED_RTOL iterations 1 4 KSP Residual norm 4.078940666596e+10 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.244953433655e+00 1 KSP Residual norm 6.745548989971e-15 Linear solve converged due to CONVERGED_RTOL iterations 1 5 KSP Residual norm 1.774320878731e+10 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.176603450320e+00 1 KSP Residual norm 5.385862671655e-15 Linear solve converged due to CONVERGED_RTOL iterations 1 6 KSP Residual norm 8.586271019948e+09 Linear solve converged due to CONVERGED_RTOL iterations 6 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.162170277427e+01 1 KSP Residual norm 2.912041048579e-14 Linear solve converged due to CONVERGED_RTOL iterations 1 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 9.290138250365e-01 1 KSP Residual norm 2.116243919788e-15 Linear solve converged due to CONVERGED_RTOL iterations 1 Residual norms for fieldsplit_1_ solve. 0 KSP Residual norm 3.986822705891e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.573745244651e+00 1 KSP Residual norm 3.715168656229e-15 Linear solve converged due to CONVERGED_RTOL iterations 1 1 KSP Residual norm 1.067205796344e+11 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.053498210605e+00 1 KSP Residual norm 4.526737475809e-15 Linear solve converged due to CONVERGED_RTOL iterations 1 2 KSP Residual norm 1.270173568462e+10 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.017546903473e+00 1 KSP Residual norm 4.593425818353e-15 Linear solve converged due to CONVERGED_RTOL iterations 1 3 KSP Residual norm 4.160360750454e+09 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.097951946288e+00 1 KSP Residual norm 6.461246234077e-15 Linear solve converged due to CONVERGED_RTOL iterations 1 4 KSP Residual norm 1.609903325354e+09 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.250441943430e+00 1 KSP Residual norm 7.157658262436e-15 Linear solve converged due to CONVERGED_RTOL iterations 1 5 KSP Residual norm 6.919994735907e+08 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.169858508137e+00 1 KSP Residual norm 5.046051842799e-15 Linear solve converged due to CONVERGED_RTOL iterations 1 6 KSP Residual norm 3.217042876248e+08 Linear solve converged due to CONVERGED_RTOL iterations 6 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 4.649924240988e-01 1 KSP Residual norm 1.068254961936e-15 Linear solve converged due to CONVERGED_RTOL iterations 1 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 7.194107998977e-04 1 KSP Residual norm 1.744183339526e-18 Linear solve converged due to CONVERGED_RTOL iterations 1 Residual norms for fieldsplit_1_ solve. 0 KSP Residual norm 4.448127329286e+12 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 7.865130501209e-01 1 KSP Residual norm 5.153218330579e-15 Linear solve converged due to CONVERGED_RTOL iterations 1 1 KSP Residual norm 2.638913823938e+12 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.283369467124e+00 1 KSP Residual norm 7.504488207954e-15 Linear solve converged due to CONVERGED_RTOL iterations 1 2 KSP Residual norm 1.426102610607e+12 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.129662815202e+00 1 KSP Residual norm 5.995004855070e-15 Linear solve converged due to CONVERGED_RTOL iterations 1 3 KSP Residual norm 8.025360882010e+11 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.424860285739e+00 1 KSP Residual norm 8.619123051910e-15 Linear solve converged due to CONVERGED_RTOL iterations 1 4 KSP Residual norm 5.286840519981e+11 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.289665727094e+00 1 KSP Residual norm 7.046395176709e-15 Linear solve converged due to CONVERGED_RTOL iterations 1 5 KSP Residual norm 3.634745308165e+11 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.364708363745e+00 1 KSP Residual norm 7.402799565707e-15 Linear solve converged due to CONVERGED_RTOL iterations 1 6 KSP Residual norm 1.720693722234e+11 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.102971050925e+00 1 KSP Residual norm 6.835965584112e-15 Linear solve converged due to CONVERGED_RTOL iterations 1 7 KSP Residual norm 7.870496175081e+10 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.345118128097e+00 1 KSP Residual norm 7.734345056964e-15 Linear solve converged due to CONVERGED_RTOL iterations 1 8 KSP Residual norm 2.921176674626e+10 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.291212792694e+00 1 KSP Residual norm 7.780332549058e-15 Linear solve converged due to CONVERGED_RTOL iterations 1 9 KSP Residual norm 1.148381817318e+10 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.073046392930e+00 1 KSP Residual norm 6.464513846538e-15 Linear solve converged due to CONVERGED_RTOL iterations 1 10 KSP Residual norm 5.253676669489e+09 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.016140779574e+00 1 KSP Residual norm 5.990033309135e-15 Linear solve converged due to CONVERGED_RTOL iterations 1 11 KSP Residual norm 2.216200526999e+09 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.037961855022e+00 1 KSP Residual norm 6.208761778424e-15 Linear solve converged due to CONVERGED_RTOL iterations 1 12 KSP Residual norm 9.013987504565e+08 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.033050143801e+00 1 KSP Residual norm 5.853852773652e-15 Linear solve converged due to CONVERGED_RTOL iterations 1 13 KSP Residual norm 3.751638125875e+08 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 9.581056747534e-01 1 KSP Residual norm 4.028342105749e-15 Linear solve converged due to CONVERGED_RTOL iterations 1 14 KSP Residual norm 1.372411728307e+08 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.117841059532e+00 1 KSP Residual norm 4.848302320534e-15 Linear solve converged due to CONVERGED_RTOL iterations 1 15 KSP Residual norm 5.353181748969e+07 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 8.645767106479e-01 1 KSP Residual norm 4.186764891168e-15 Linear solve converged due to CONVERGED_RTOL iterations 1 16 KSP Residual norm 2.421357309016e+07 Linear solve converged due to CONVERGED_RTOL iterations 16 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 6.302684114623e-01 1 KSP Residual norm 2.299313600630e-15 Linear solve converged due to CONVERGED_RTOL iterations 1 Linear solve converged due to CONVERGED_RTOL iterations 2 > Thanks, this looks quite reasonable. I'll try to experiment with it. > > >> > I do not know much about FEM. But some of the reasons why I have >> avoided it >> > in this particular problem are: (Please correct me on any of the >> following >> > points if they are wrong) >> > 1. The inputs f1 and f2 are 3D images (in average of size 200^3) that >> come >> > from other image processing pipeline; it's important that I constrain u >> at >> > each voxel for div(u) = f2 in domain A. I am trying to avoid having to >> get >> > the meshing from the 3D image(with very detailed structures), then go >> back >> > to the image from the obtained u again because I have to use the >> obtained u >> > to warp the image, transport other parameters again with u in the image >> > space and again obtain new f1 and f2 images. Then iterate this few >> times. >> >> Okay, there's nothing wrong with that. >> > Thanks for the confirmation. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jedbrown at mcs.anl.gov Tue Dec 3 08:49:52 2013 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Tue, 03 Dec 2013 08:49:52 -0600 Subject: [petsc-users] solving stokes-like equation in 3D staggered grid in irregular domain with petscsection In-Reply-To: References: <874n7hb7zj.fsf@jedbrown.org> <87y54s57we.fsf@jedbrown.org> Message-ID: <87zjoi0z1r.fsf@jedbrown.org> Bishesh Khanal writes: > I tried to modify my existing code to incorporate c(x) instead of 0 as you > suggested. When c(x) is zero everywhere, the solver converges and gives me > the result I expect, as before. However, when c(x) is non-zero in domain B, > the solver does not seem to converge. The ouput for both cases are given > below. When c(x) was zero everywhere, I had set the constant pressure null > space to the system matrix and the schur complement matrix since I am > solving the system with pctype fieldsplit and schur. However, when c(x) is > non-zero in domain B (I put c(x) = 1 in B for test), we do not have the > constant null space for pressure anymore; so for this case I do not set the > constant null space. Can you comment on what could be possibly going wrong > ? > 1. Output for the test case: c(x) = 0 in A, c(x) = 1 in B; mu = 1 > everywhere. > options given: > -pc_type fieldsplit -pc_fieldsplit_type schur -pc_fieldsplit_dm_splits 0 > -pc_fieldsplit_0_fields 0,1,2 -pc_fieldsplit_1_fields 3 > -fieldsplit_0_pc_type lu -fieldsplit_0_pc_factor_mat_solver_package > superlu_dist -fieldsplit_0_ksp_monitor -fieldsplit_0_ksp_converged_reason > -fieldsplit_0_ksp_max_it 100 -fieldsplit_1_ksp_monitor > -fieldsplit_1_ksp_converged_reason -ksp_converged_reason > > output: > > Residual norms for fieldsplit_0_ solve. > 0 KSP Residual norm 2.321914527334e+01 > 1 KSP Residual norm 5.204444045635e-14 > Linear solve converged due to CONVERGED_RTOL iterations 1 > Residual norms for fieldsplit_1_ solve. > 0 KSP Residual norm 2.890735677419e+14 > Residual norms for fieldsplit_0_ solve. -fieldsplit_1_ksp_converged_reason to see why that subsolve does not converge. Also check ksp_monitor_true_residual to get some indication of whether the preconditioner is singular. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From bisheshkh at gmail.com Tue Dec 3 10:07:02 2013 From: bisheshkh at gmail.com (Bishesh Khanal) Date: Tue, 3 Dec 2013 17:07:02 +0100 Subject: [petsc-users] solving stokes-like equation in 3D staggered grid in irregular domain with petscsection In-Reply-To: <87zjoi0z1r.fsf@jedbrown.org> References: <874n7hb7zj.fsf@jedbrown.org> <87y54s57we.fsf@jedbrown.org> <87zjoi0z1r.fsf@jedbrown.org> Message-ID: On Tue, Dec 3, 2013 at 3:49 PM, Jed Brown wrote: > Bishesh Khanal writes: > > I tried to modify my existing code to incorporate c(x) instead of 0 as > you > > suggested. When c(x) is zero everywhere, the solver converges and gives > me > > the result I expect, as before. However, when c(x) is non-zero in domain > B, > > the solver does not seem to converge. The ouput for both cases are given > > below. When c(x) was zero everywhere, I had set the constant pressure > null > > space to the system matrix and the schur complement matrix since I am > > solving the system with pctype fieldsplit and schur. However, when c(x) > is > > non-zero in domain B (I put c(x) = 1 in B for test), we do not have the > > constant null space for pressure anymore; so for this case I do not set > the > > constant null space. Can you comment on what could be possibly going > wrong > > ? > > 1. Output for the test case: c(x) = 0 in A, c(x) = 1 in B; mu = 1 > > everywhere. > > options given: > > -pc_type fieldsplit -pc_fieldsplit_type schur -pc_fieldsplit_dm_splits 0 > > -pc_fieldsplit_0_fields 0,1,2 -pc_fieldsplit_1_fields 3 > > -fieldsplit_0_pc_type lu -fieldsplit_0_pc_factor_mat_solver_package > > superlu_dist -fieldsplit_0_ksp_monitor -fieldsplit_0_ksp_converged_reason > > -fieldsplit_0_ksp_max_it 100 -fieldsplit_1_ksp_monitor > > -fieldsplit_1_ksp_converged_reason -ksp_converged_reason > > > > output: > > > > Residual norms for fieldsplit_0_ solve. > > 0 KSP Residual norm 2.321914527334e+01 > > 1 KSP Residual norm 5.204444045635e-14 > > Linear solve converged due to CONVERGED_RTOL iterations 1 > > Residual norms for fieldsplit_1_ solve. > > 0 KSP Residual norm 2.890735677419e+14 > > Residual norms for fieldsplit_0_ solve. > > -fieldsplit_1_ksp_converged_reason to see why that subsolve does not > converge. Also check ksp_monitor_true_residual to get some indication > of whether the preconditioner is singular. > With options ksp_converged_reason and ksp_monitor_true_residual set only for the fieldsplit_1 , I get the result as below. -ksp_view reports that the preconditioner for the schur complement is built from A11. preconditioned resid norm below looks to be the same and in the order of 10e+14 for every iteration. Does this imply the precondtioner is singular? Does it mean that the problem is possibly in assembling the A11 block? It is the diagonal of this block corresponding to the domain B cells that changes when I set c(x) = 1. Residual norms for fieldsplit_1_ solve. 0 KSP preconditioned resid norm 2.890735677419e+14 true resid norm 1.265820125363e+01 ||r(i)||/||b|| 1.000000000000e+00 1 KSP preconditioned resid norm 2.890735677419e+14 true resid norm 1.117075725063e+01 ||r(i)||/||b|| 8.824916768826e-01 2 KSP preconditioned resid norm 2.890735677419e+14 true resid norm 1.109990265188e+01 ||r(i)||/||b|| 8.768941518210e-01 3 KSP preconditioned resid norm 2.890735677419e+14 true resid norm 1.109895437193e+01 ||r(i)||/||b|| 8.768192375476e-01 4 KSP preconditioned resid norm 2.890735677419e+14 true resid norm 1.109892185160e+01 ||r(i)||/||b|| 8.768166684358e-01 5 KSP preconditioned resid norm 2.890735677419e+14 true resid norm 1.109889770615e+01 ||r(i)||/||b|| 8.768147609417e-01 6 KSP preconditioned resid norm 2.890735677419e+14 true resid norm 1.109886495655e+01 ||r(i)||/||b|| 8.768121737173e-01 7 KSP preconditioned resid norm 2.890735677419e+14 true resid norm 1.109886096749e+01 ||r(i)||/||b|| 8.768118585813e-01 8 KSP preconditioned resid norm 2.890735677419e+14 true resid norm 1.109885236160e+01 ||r(i)||/||b|| 8.768111787141e-01 9 KSP preconditioned resid norm 2.890735677419e+14 true resid norm 1.109882829958e+01 ||r(i)||/||b|| 8.768092778106e-01 10 KSP preconditioned resid norm 2.890735677419e+14 true resid norm 1.109882703198e+01 ||r(i)||/||b|| 8.768091776699e-01 11 KSP preconditioned resid norm 2.890735677419e+14 true resid norm 1.109882171007e+01 ||r(i)||/||b|| 8.768087572386e-01 12 KSP preconditioned resid norm 2.890735677419e+14 true resid norm 1.109882129003e+01 ||r(i)||/||b|| 8.768087240548e-01 13 KSP preconditioned resid norm 2.890735677419e+14 true resid norm 1.109880708177e+01 ||r(i)||/||b|| 8.768076016004e-01 14 KSP preconditioned resid norm 2.890735677419e+14 true resid norm 1.109884240385e+01 ||r(i)||/||b|| 8.768103920501e-01 15 KSP preconditioned resid norm 2.890735677419e+14 true resid norm 1.110342530238e+01 ||r(i)||/||b|| 8.771724417953e-01 16 KSP preconditioned resid norm 2.890735677419e+14 true resid norm 1.668559859400e+01 ||r(i)||/||b|| 1.318165058342e+00 17 KSP preconditioned resid norm 2.890735677419e+14 true resid norm 2.379573238198e+02 ||r(i)||/||b|| 1.879866807707e+01 18 KSP preconditioned resid norm 2.890735677419e+14 true resid norm 7.475251038198e+02 ||r(i)||/||b|| 5.905460727331e+01 19 KSP preconditioned resid norm 2.890735677419e+14 true resid norm 7.610489296073e+02 ||r(i)||/||b|| 6.012299175517e+01 20 KSP preconditioned resid norm 2.890735677419e+14 true resid norm 1.162871674016e+03 ||r(i)||/||b|| 9.186705525659e+01 21 KSP preconditioned resid norm 2.890735677419e+14 true resid norm 2.438552126509e+03 ||r(i)||/||b|| 1.926460227364e+02 22 KSP preconditioned resid norm 2.890735677419e+14 true resid norm 2.112154724517e+03 ||r(i)||/||b|| 1.668605738047e+02 23 KSP preconditioned resid norm 2.890735677419e+14 true resid norm 1.867895524623e+03 ||r(i)||/||b|| 1.475640564719e+02 24 KSP preconditioned resid norm 2.890735677419e+14 true resid norm 2.760528221023e+03 ||r(i)||/||b|| 2.180821876435e+02 25 KSP preconditioned resid norm 2.890735677419e+14 true resid norm 2.585072583244e+03 ||r(i)||/||b|| 2.042211631374e+02 and it continues. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jedbrown at mcs.anl.gov Tue Dec 3 10:13:54 2013 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Tue, 03 Dec 2013 10:13:54 -0600 Subject: [petsc-users] solving stokes-like equation in 3D staggered grid in irregular domain with petscsection In-Reply-To: References: <874n7hb7zj.fsf@jedbrown.org> <87y54s57we.fsf@jedbrown.org> <87zjoi0z1r.fsf@jedbrown.org> Message-ID: <87siu929q5.fsf@jedbrown.org> Bishesh Khanal writes: > With options ksp_converged_reason and ksp_monitor_true_residual set only > for the fieldsplit_1 , I get the result as below. -ksp_view reports that > the preconditioner for the schur complement is built from A11. > preconditioned resid norm below looks to be the same and in the order of > 10e+14 for every iteration. Does this imply the precondtioner is singular? > Does it mean that the problem is possibly in assembling the A11 block? I would suspect that, yes. The preconditioner is amplifying the residual, but then it is not converging either, which indicates that the original system may also be singular, with an inconsistent right hand side. > It is the diagonal of this block corresponding to the domain B cells > that changes when I set c(x) = 1. Residual norms for fieldsplit_1_ > solve. 0 KSP preconditioned resid norm 2.890735677419e+14 true resid > norm 1.265820125363e+01 ||r(i)||/||b|| 1.000000000000e+00 1 KSP > preconditioned resid norm 2.890735677419e+14 true resid norm > 1.117075725063e+01 ||r(i)||/||b|| 8.824916768826e-01 2 KSP > preconditioned resid norm 2.890735677419e+14 true resid norm > 1.109990265188e+01 ||r(i)||/||b|| 8.768941518210e-01 3 KSP > preconditioned resid norm 2.890735677419e+14 true resid norm > 1.109895437193e+01 ||r(i)||/||b|| 8.768192375476e-01 4 KSP > preconditioned resid norm 2.890735677419e+14 true resid norm > 1.109892185160e+01 ||r(i)||/||b|| 8.768166684358e-01 5 KSP > preconditioned resid norm 2.890735677419e+14 true resid norm > 1.109889770615e+01 ||r(i)||/||b|| 8.768147609417e-01 6 KSP > preconditioned resid norm 2.890735677419e+14 true resid norm > 1.109886495655e+01 ||r(i)||/||b|| 8.768121737173e-01 7 KSP > preconditioned resid norm 2.890735677419e+14 true resid norm > 1.109886096749e+01 ||r(i)||/||b|| 8.768118585813e-01 8 KSP > preconditioned resid norm 2.890735677419e+14 true resid norm > 1.109885236160e+01 ||r(i)||/||b|| 8.768111787141e-01 9 KSP > preconditioned resid norm 2.890735677419e+14 true resid norm > 1.109882829958e+01 ||r(i)||/||b|| 8.768092778106e-01 10 KSP > preconditioned resid norm 2.890735677419e+14 true resid norm > 1.109882703198e+01 ||r(i)||/||b|| 8.768091776699e-01 11 KSP > preconditioned resid norm 2.890735677419e+14 true resid norm > 1.109882171007e+01 ||r(i)||/||b|| 8.768087572386e-01 12 KSP > preconditioned resid norm 2.890735677419e+14 true resid norm > 1.109882129003e+01 ||r(i)||/||b|| 8.768087240548e-01 13 KSP > preconditioned resid norm 2.890735677419e+14 true resid norm > 1.109880708177e+01 ||r(i)||/||b|| 8.768076016004e-01 14 KSP > preconditioned resid norm 2.890735677419e+14 true resid norm > 1.109884240385e+01 ||r(i)||/||b|| 8.768103920501e-01 15 KSP > preconditioned resid norm 2.890735677419e+14 true resid norm > 1.110342530238e+01 ||r(i)||/||b|| 8.771724417953e-01 16 KSP > preconditioned resid norm 2.890735677419e+14 true resid norm > 1.668559859400e+01 ||r(i)||/||b|| 1.318165058342e+00 17 KSP > preconditioned resid norm 2.890735677419e+14 true resid norm > 2.379573238198e+02 ||r(i)||/||b|| 1.879866807707e+01 18 KSP > preconditioned resid norm 2.890735677419e+14 true resid norm > 7.475251038198e+02 ||r(i)||/||b|| 5.905460727331e+01 19 KSP > preconditioned resid norm 2.890735677419e+14 true resid norm > 7.610489296073e+02 ||r(i)||/||b|| 6.012299175517e+01 20 KSP > preconditioned resid norm 2.890735677419e+14 true resid norm > 1.162871674016e+03 ||r(i)||/||b|| 9.186705525659e+01 21 KSP > preconditioned resid norm 2.890735677419e+14 true resid norm > 2.438552126509e+03 ||r(i)||/||b|| 1.926460227364e+02 22 KSP > preconditioned resid norm 2.890735677419e+14 true resid norm > 2.112154724517e+03 ||r(i)||/||b|| 1.668605738047e+02 23 KSP > preconditioned resid norm 2.890735677419e+14 true resid norm > 1.867895524623e+03 ||r(i)||/||b|| 1.475640564719e+02 24 KSP > preconditioned resid norm 2.890735677419e+14 true resid norm > 2.760528221023e+03 ||r(i)||/||b|| 2.180821876435e+02 25 KSP > preconditioned resid norm 2.890735677419e+14 true resid norm > 2.585072583244e+03 ||r(i)||/||b|| 2.042211631374e+02 and it continues. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From katyghantous at gmail.com Tue Dec 3 11:05:44 2013 From: katyghantous at gmail.com (Katy Ghantous) Date: Tue, 3 Dec 2013 12:05:44 -0500 Subject: [petsc-users] Stiff ODEs Message-ID: Hi, I have just started with Petsc and have very elementary equations. I am trying to solve a set of stiff ODEs explicitly. They are very stiff and was hoping to use TSRK with variable time steps, but it is throwing a segmentation fault otherwise non existent with Euler or TSSSP. 1. should i look into the TSRK error or is it something with the new version of Petsc? 2. i read somewhere (but cant find the source again) that TSSSP is not recommended for stiff equations (even with the rk solvers set as the type). Is that correct? if so, what would be most optimized to solve stiff odes explicitly. 3. If it is not correct and TSSSP is useful for stiff equations, how can i run it with variable time steps, since with TSStep it is taking equal time steps which is the time step that i am specifying as an initial time step TSSetInitialTimeStep. Thank you, -------------- next part -------------- An HTML attachment was scrubbed... URL: From jedbrown at mcs.anl.gov Tue Dec 3 11:13:46 2013 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Tue, 03 Dec 2013 11:13:46 -0600 Subject: [petsc-users] Stiff ODEs In-Reply-To: References: Message-ID: <87mwkh26yd.fsf@jedbrown.org> Katy Ghantous writes: > Hi, > I have just started with Petsc and have very elementary equations. > I am trying to solve a set of stiff ODEs explicitly. If your ODEs are stiff, why are you trying to use an explicit method? (A common definition of "stiff" is that explicit methods do not work well.) > They are very stiff and was hoping to use TSRK with variable time > steps, but it is throwing a segmentation fault Always show the full error message. Please also use 'master' from petsc-dev, because TSRK was overhauled recently to be consistent with the other integrators (and adding a bunch of new functionality). https://bitbucket.org/petsc/petsc/wiki/ > otherwise non existent with Euler or TSSSP. 1. should i look into the > TSRK error or is it something with the new version of Petsc? 2. i > read somewhere (but cant find the source again) that TSSSP is not > recommended for stiff equations (even with the rk solvers set as the > type). Yes, TSSSP is for non-stiff problems that require "strong stability" (like controlling total variation or positivity in transport-dominated applications). > Is that correct? if so, what would be most optimized to solve stiff > odes explicitly. Stiff ODEs should be solved *implicitly*. > 3. If it is not correct and TSSSP is useful for stiff equations, how > can i run it with variable time steps, since with TSStep it is taking > equal time steps which is the time step that i am specifying as an > initial time step TSSetInitialTimeStep. TSSSP does not have an adaptive controller. A user can set a post-check function that adapts the time step based on a custom stability function (e.g., CFL criteria). TSROSW and TSARKIMEX are IMEX methods that can handle stiff problems. They have adaptive error estimators and the like. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From gideon.simpson at gmail.com Tue Dec 3 13:16:51 2013 From: gideon.simpson at gmail.com (Gideon Simpson) Date: Tue, 3 Dec 2013 14:16:51 -0500 Subject: [petsc-users] error configuring petsc+mpich Message-ID: I'm trying to get a petsc installation going on a Linux machine, and I'm getting the following error when building with --download-mpich=yes ******************************************************************************* UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log for details): ------------------------------------------------------------------------------- Error running configure on MPICH: Could not execute "cd /home/simpson/software/petsc-3.4.3/externalpackages/mpich-3.0.4-106 && ./configure --prefix=/home/simpson/software/petsc-3.4.3/arch-linux2-c-debug CC="gcc" CFLAGS=" -Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g3 -fno-inline -O0 " CXX="g++" CXXFLAGS=" -Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g " FC="gfortran" FCFLAGS=" -Wall -Wno-unused-variable -Wno-unused-dummy-argument -g " F77="gfortran" FFLAGS=" -Wall -Wno-unused-variable -Wno-unused-dummy-argument -g " --with-device=ch3:sock --without-mpe --with-pm=hydra --enable-g=meminit --enable-fast": Configuring MPICH version 3.0.4 with '--prefix=/home/simpson/software/petsc-3.4.3/arch-linux2-c-debug' 'CC=gcc' 'CFLAGS= -Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g3 -fno-inline -O0 ' 'CXX=g++' 'CXXFLAGS= -Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g ' 'FC=gfortran' 'FCFLAGS= -Wall -Wno-unused-variable -Wno-unused-dummy-argument -g ' 'F77=gfortran' 'FFLAGS= -Wall -Wno-unused-variable -Wno-unused-dummy-argument -g ' '--with-device=ch3:sock' '--without-mpe' '--with-pm=hydra' '--enable-g=meminit' '--enable-fast' Running on system: Linux pegasus 3.2.0-41-generic #66-Ubuntu SMP Thu Apr 25 03:27:11 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux configure: error: cannot find install-sh, install.sh, or shtool in confdb "."/confdb ******************************************************************************* File "./config/configure.py", line 293, in petsc_configure framework.configure(out = sys.stdout) File "/home/simpson/software/petsc-3.4.3/config/BuildSystem/config/framework.py", line 933, in configure child.configure() File "/home/simpson/software/petsc-3.4.3/config/BuildSystem/config/package.py", line 556, in configure self.executeTest(self.configureLibrary) File "/home/simpson/software/petsc-3.4.3/config/BuildSystem/config/base.py", line 115, in executeTest ret = apply(test, args,kargs) File "/home/simpson/software/petsc-3.4.3/config/BuildSystem/config/packages/MPI.py", line 748, in configureLibrary config.package.Package.configureLibrary(self) File "/home/simpson/software/petsc-3.4.3/config/BuildSystem/config/package.py", line 484, in configureLibrary for location, directory, lib, incl in self.generateGuesses(): File "/home/simpson/software/petsc-3.4.3/config/BuildSystem/config/package.py", line 232, in generateGuesses d = self.checkDownload(1) File "/home/simpson/software/petsc-3.4.3/config/BuildSystem/config/packages/MPI.py", line 351, in checkDownload return config.package.Package.checkDownload(self, requireDownload) File "/home/simpson/software/petsc-3.4.3/config/BuildSystem/config/package.py", line 340, in checkDownload return self.getInstallDir() File "/home/simpson/software/petsc-3.4.3/config/BuildSystem/config/package.py", line 187, in getInstallDir return os.path.abspath(self.Install()) File "/home/simpson/software/petsc-3.4.3/config/BuildSystem/config/packages/MPI.py", line 366, in Install return self.InstallMPICH() File "/home/simpson/software/petsc-3.4.3/config/BuildSystem/config/packages/MPI.py", line 555, in InstallMPICH raise RuntimeError('Error running configure on MPICH: '+str(e)) -gideon -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Tue Dec 3 13:53:09 2013 From: knepley at gmail.com (Matthew Knepley) Date: Tue, 3 Dec 2013 13:53:09 -0600 Subject: [petsc-users] error configuring petsc+mpich In-Reply-To: References: Message-ID: On Tue, Dec 3, 2013 at 1:16 PM, Gideon Simpson wrote: > I'm trying to get a petsc installation going on a Linux machine, and I'm > getting the following error when building with --download-mpich=yes > I am assuming you are on Linux? You are missing install, which should be part of the coreutils package. But install-sh should be created by autoconf. Not sure why this is going wrong. Can you send configure.log? Thanks, Matt > > ******************************************************************************* > UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log for > details): > > ------------------------------------------------------------------------------- > Error running configure on MPICH: Could not execute "cd > /home/simpson/software/petsc-3.4.3/externalpackages/mpich-3.0.4-106 && > ./configure --prefix=/home/simpson/software/petsc-3.4.3/arch-linux2-c-debug > CC="gcc" CFLAGS=" -Wall -Wwrite-strings -Wno-strict-aliasing > -Wno-unknown-pragmas -g3 -fno-inline -O0 " CXX="g++" CXXFLAGS=" -Wall > -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g " > FC="gfortran" FCFLAGS=" -Wall -Wno-unused-variable > -Wno-unused-dummy-argument -g " F77="gfortran" FFLAGS=" -Wall > -Wno-unused-variable -Wno-unused-dummy-argument -g " > --with-device=ch3:sock --without-mpe --with-pm=hydra --enable-g=meminit > --enable-fast": > Configuring MPICH version 3.0.4 with > '--prefix=/home/simpson/software/petsc-3.4.3/arch-linux2-c-debug' 'CC=gcc' > 'CFLAGS= -Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas > -g3 -fno-inline -O0 ' 'CXX=g++' 'CXXFLAGS= -Wall -Wwrite-strings > -Wno-strict-aliasing -Wno-unknown-pragmas -g ' 'FC=gfortran' 'FCFLAGS= > -Wall -Wno-unused-variable -Wno-unused-dummy-argument -g ' 'F77=gfortran' > 'FFLAGS= -Wall -Wno-unused-variable -Wno-unused-dummy-argument -g ' > '--with-device=ch3:sock' '--without-mpe' '--with-pm=hydra' > '--enable-g=meminit' '--enable-fast' > Running on system: Linux pegasus 3.2.0-41-generic #66-Ubuntu SMP Thu Apr > 25 03:27:11 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux > configure: error: cannot find install-sh, install.sh, or shtool in confdb > "."/confdb > > ******************************************************************************* > File "./config/configure.py", line 293, in petsc_configure > framework.configure(out = sys.stdout) > File > "/home/simpson/software/petsc-3.4.3/config/BuildSystem/config/framework.py", > line 933, in configure > child.configure() > File > "/home/simpson/software/petsc-3.4.3/config/BuildSystem/config/package.py", > line 556, in configure > self.executeTest(self.configureLibrary) > File > "/home/simpson/software/petsc-3.4.3/config/BuildSystem/config/base.py", > line 115, in executeTest > ret = apply(test, args,kargs) > File > "/home/simpson/software/petsc-3.4.3/config/BuildSystem/config/packages/MPI.py", > line 748, in configureLibrary > config.package.Package.configureLibrary(self) > File > "/home/simpson/software/petsc-3.4.3/config/BuildSystem/config/package.py", > line 484, in configureLibrary > for location, directory, lib, incl in self.generateGuesses(): > File > "/home/simpson/software/petsc-3.4.3/config/BuildSystem/config/package.py", > line 232, in generateGuesses > d = self.checkDownload(1) > File > "/home/simpson/software/petsc-3.4.3/config/BuildSystem/config/packages/MPI.py", > line 351, in checkDownload > return config.package.Package.checkDownload(self, requireDownload) > File > "/home/simpson/software/petsc-3.4.3/config/BuildSystem/config/package.py", > line 340, in checkDownload > return self.getInstallDir() > File > "/home/simpson/software/petsc-3.4.3/config/BuildSystem/config/package.py", > line 187, in getInstallDir > return os.path.abspath(self.Install()) > File > "/home/simpson/software/petsc-3.4.3/config/BuildSystem/config/packages/MPI.py", > line 366, in Install > return self.InstallMPICH() > File > "/home/simpson/software/petsc-3.4.3/config/BuildSystem/config/packages/MPI.py", > line 555, in InstallMPICH > raise RuntimeError('Error running configure on MPICH: '+str(e)) > > > > -gideon > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From jianjun.xiao at kit.edu Wed Dec 4 08:59:39 2013 From: jianjun.xiao at kit.edu (Xiao, Jianjun (IKET)) Date: Wed, 4 Dec 2013 15:59:39 +0100 Subject: [petsc-users] Convergence problem in solving a symmetric positive definite matrix in a CFD code Message-ID: <56D054AF2E93E044AC1D2685709D2868C7340018C7@KIT-MSX-07.kit.edu> Hello, I am using Petsc to solve the linear equation in a CFD code. The matrix is symmetric positive definite. Please find my input and output below. 1. When I used KSPCR solver, ||r(i)||/||b|| is inf. What is the reason? Does it mean ||b|| is zero? When I used the KSPLGMRES solver, it seems ||r(i)||/||b|| is OK. However, it seems the calculated results are not right. 2. I am not sure if I set the solver and matrix properly. Did I miss something? Thank you. **************************************** INPUT: CALL KSPCreate(PETSC_COMM_WORLD,solver,ierr) CALL DMSetMatType(da_ksp,MATMPISBAIJ,ierr) CALL DMCreateMatrix(da_ksp,mat,ierr) CALL MatMPISBAIJSetPreallocation(mat,1,4,PETSC_NULL_INTEGER,1,PETSC_NULL_INTEGER,ierr) CALL MatZeroEntries(mat,ierr) CALL MatAssemblyBegin(mat,MAT_FINAL_ASSEMBLY,ierr) CALL MatAssemblyEnd(mat,MAT_FINAL_ASSEMBLY,ierr) CALL KSPSetOperators(solver,mat,mat,SAME_NONZERO_PATTERN,ierr) ! CALL KSPSetType(solver,KSPCR,ierr) CALL KSPSetType(solver,KSPLGMRES,ierr) epsi = 1.0-e5 CALL KSPSetTolerances(solver,epsi,PETSC_DEFAULT_DOUBLE_PRECISION,& & PETSC_DEFAULT_DOUBLE_PRECISION,itmax,ierr) CALL KSPGetPC(solver,gfprec,ierr) CALL PCSetType(prec,PCBJACOBI,ierr) CALL KSPMonitorSet(solver,KSPMonitorTrueResidualNorm,& PETSC_NULL_OBJECT,PETSC_NULL_FUNCTION,ierr ) CALL KSPSetInitialGuessNonzero(solver,PETSC_TRUE,ierr) CALL KSPSolve(solver,bvec,xsol,ierr) CALL KSPGetIterationNumber(solver,iter,ierr) CALL KSPGetResidualNorm(solver,dmax,ierr) CALL KSPView(solver,PETSC_VIEWER_STDOUT_WORLD,ierr) ************************************** OUTPUT CALL KSPSetType(solver,KSPCR,ierr): 0 KSP preconditioned resid norm 2.226482634319e+05 true resid norm 1.204978940624e+07 ||r(i)||/||b|| inf 1 KSP preconditioned resid norm 1.684243557244e+05 true resid norm 6.742321430949e+06 ||r(i)||/||b|| inf 2 KSP preconditioned resid norm 1.039386033131e+05 true resid norm 5.094347016880e+06 ||r(i)||/||b|| inf 3 KSP preconditioned resid norm 3.767761162917e+04 true resid norm 2.085014289432e+06 ||r(i)||/||b|| inf 4 KSP preconditioned resid norm 2.220316358489e+04 true resid norm 1.039841616110e+06 ||r(i)||/||b|| inf 5 KSP preconditioned resid norm 1.009108756815e+04 true resid norm 6.764592620620e+05 ||r(i)||/||b|| inf 6 KSP preconditioned resid norm 7.266143334386e+03 true resid norm 4.713756053613e+05 ||r(i)||/||b|| inf 7 KSP preconditioned resid norm 4.925270026573e+03 true resid norm 3.276759177651e+05 ||r(i)||/||b|| inf 8 KSP preconditioned resid norm 2.595039666791e+03 true resid norm 1.774916597474e+05 ||r(i)||/||b|| inf 9 KSP preconditioned resid norm 1.970388137453e+03 true resid norm 1.449811653036e+05 ||r(i)||/||b|| inf 10 KSP preconditioned resid norm 1.455914234388e+03 true resid norm 7.916294162841e+04 ||r(i)||/||b|| inf 11 KSP preconditioned resid norm 8.335194818556e+02 true resid norm 4.530953608250e+04 ||r(i)||/||b|| inf 12 KSP preconditioned resid norm 3.022320555777e+02 true resid norm 1.728551635678e+04 ||r(i)||/||b|| inf 13 KSP preconditioned resid norm 7.190336024797e+01 true resid norm 4.186842086105e+03 ||r(i)||/||b|| inf 14 KSP preconditioned resid norm 1.718291655675e+01 true resid norm 1.089751055004e+03 ||r(i)||/||b|| inf 15 KSP preconditioned resid norm 1.150683059424e+01 true resid norm 8.672405273471e+02 ||r(i)||/||b|| inf 16 KSP preconditioned resid norm 8.663479440949e+00 true resid norm 5.776737380768e+02 ||r(i)||/||b|| inf 17 KSP preconditioned resid norm 5.282161990683e+00 true resid norm 2.977735906695e+02 ||r(i)||/||b|| inf 18 KSP preconditioned resid norm 3.802629315725e+00 true resid norm 2.789114564993e+02 ||r(i)||/||b|| inf 19 KSP preconditioned resid norm 1.722575171383e+00 true resid norm 1.051323829526e+02 ||r(i)||/||b|| inf KSP Object: 1 MPI processes type: cr maximum iterations=1000 tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using nonzero initial guess using PRECONDITIONED norm type for convergence test PC Object: 1 MPI processes type: bjacobi block Jacobi: number of blocks = 1 Local solve is same for all blocks, in the following KSP and PC objects: KSP Object: (sub_) 1 MPI processes type: preonly maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using NONE norm type for convergence test PC Object: (sub_) 1 MPI processes type: icc 0 levels of fill tolerance for zero pivot 2.22045e-14 using Manteuffel shift [POSITIVE_DEFINITE] matrix ordering: natural factor fill ratio given 1, needed 1 Factored matrix follows: Mat Object: 1 MPI processes type: seqsbaij rows=27, cols=27 package used to perform factorization: petsc total: nonzeros=81, allocated nonzeros=81 total number of mallocs used during MatSetValues calls =0 block size is 1 linear system matrix = precond matrix: Mat Object: 1 MPI processes type: seqsbaij rows=27, cols=27 total: nonzeros=81, allocated nonzeros=108 total number of mallocs used during MatSetValues calls =0 block size is 1 linear system matrix = precond matrix: Mat Object: 1 MPI processes type: mpisbaij rows=27, cols=27 total: nonzeros=81, allocated nonzeros=135 total number of mallocs used during MatSetValues calls =0 block size is 1 OUTPUT: CALL KSPSetType(solver,KSPLGMRES,ierr) 0 KSP preconditioned resid norm 2.362537325084e+04 true resid norm 1.138584383312e+06 ||r(i)||/||b|| 1.000000000000e+00 1 KSP preconditioned resid norm 8.501213683423e+03 true resid norm 3.655528853686e+05 ||r(i)||/||b|| 3.210591070162e-01 2 KSP preconditioned resid norm 5.487567253725e+03 true resid norm 3.005741194777e+05 ||r(i)||/||b|| 2.639893220769e-01 3 KSP preconditioned resid norm 2.470452880657e+03 true resid norm 1.545469272201e+05 ||r(i)||/||b|| 1.357360328187e-01 4 KSP preconditioned resid norm 1.750803325456e+03 true resid norm 1.182312309352e+05 ||r(i)||/||b|| 1.038405520646e-01 5 KSP preconditioned resid norm 1.123492053552e+03 true resid norm 6.754319630701e+04 ||r(i)||/||b|| 5.932208213726e-02 6 KSP preconditioned resid norm 5.150241959277e+02 true resid norm 3.689413898730e+04 ||r(i)||/||b|| 3.240351749775e-02 7 KSP preconditioned resid norm 4.182894544871e+02 true resid norm 3.052196222024e+04 ||r(i)||/||b|| 2.680693909701e-02 8 KSP preconditioned resid norm 2.520093155629e+02 true resid norm 1.880976788356e+04 ||r(i)||/||b|| 1.652031079932e-02 9 KSP preconditioned resid norm 1.186491314806e+02 true resid norm 6.797080217853e+03 ||r(i)||/||b|| 5.969764136483e-03 10 KSP preconditioned resid norm 5.851092821372e+01 true resid norm 2.973659280245e+03 ||r(i)||/||b|| 2.611716201127e-03 11 KSP preconditioned resid norm 1.669909189055e+01 true resid norm 5.658829814125e+02 ||r(i)||/||b|| 4.970057465277e-04 12 KSP preconditioned resid norm 3.090594692756e+00 true resid norm 2.161527454147e+02 ||r(i)||/||b|| 1.898434130864e-04 13 KSP preconditioned resid norm 2.164618839184e+00 true resid norm 1.620745991834e+02 ||r(i)||/||b|| 1.423474637093e-04 14 KSP preconditioned resid norm 1.291593952428e+00 true resid norm 9.095542547366e+01 ||r(i)||/||b|| 7.988465923722e-05 15 KSP preconditioned resid norm 6.100583411632e-01 true resid norm 4.021646656091e+01 ||r(i)||/||b|| 3.532146334551e-05 16 KSP preconditioned resid norm 2.723496807925e-01 true resid norm 1.676660466866e+01 ||r(i)||/||b|| 1.472583403954e-05 17 KSP preconditioned resid norm 1.377718471538e-01 true resid norm 8.551854245272e+00 ||r(i)||/||b|| 7.510953400221e-06 KSP Object: 1 MPI processes type: lgmres GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement GMRES: happy breakdown tolerance 1e-30 LGMRES: aug. dimension=2 LGMRES: number of matvecs=170 maximum iterations=1000 tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using nonzero initial guess using PRECONDITIONED norm type for convergence test PC Object: 1 MPI processes type: bjacobi block Jacobi: number of blocks = 1 Local solve is same for all blocks, in the following KSP and PC objects: KSP Object: (sub_) 1 MPI processes type: preonly maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using NONE norm type for convergence test PC Object: (sub_) 1 MPI processes type: icc 0 levels of fill tolerance for zero pivot 2.22045e-14 using Manteuffel shift [POSITIVE_DEFINITE] matrix ordering: natural factor fill ratio given 1, needed 1 Factored matrix follows: Mat Object: 1 MPI processes type: seqsbaij rows=27, cols=27 package used to perform factorization: petsc total: nonzeros=81, allocated nonzeros=81 total number of mallocs used during MatSetValues calls =0 block size is 1 linear system matrix = precond matrix: Mat Object: 1 MPI processes type: seqsbaij rows=27, cols=27 total: nonzeros=81, allocated nonzeros=108 total number of mallocs used during MatSetValues calls =0 block size is 1 linear system matrix = precond matrix: Mat Object: 1 MPI processes type: mpisbaij rows=27, cols=27 total: nonzeros=81, allocated nonzeros=135 total number of mallocs used during MatSetValues calls =0 From knepley at gmail.com Wed Dec 4 09:19:56 2013 From: knepley at gmail.com (Matthew Knepley) Date: Wed, 4 Dec 2013 09:19:56 -0600 Subject: [petsc-users] Convergence problem in solving a symmetric positive definite matrix in a CFD code In-Reply-To: <56D054AF2E93E044AC1D2685709D2868C7340018C7@KIT-MSX-07.kit.edu> References: <56D054AF2E93E044AC1D2685709D2868C7340018C7@KIT-MSX-07.kit.edu> Message-ID: On Wed, Dec 4, 2013 at 8:59 AM, Xiao, Jianjun (IKET) wrote: > Hello, > > I am using Petsc to solve the linear equation in a CFD code. The matrix is > symmetric positive definite. > > Please find my input and output below. > > 1. When I used KSPCR solver, ||r(i)||/||b|| is inf. What is the reason? > Does it mean ||b|| is zero? When I used the KSPLGMRES solver, it seems > ||r(i)||/||b|| is OK. However, it seems the calculated results are not > right. > It looks that way. You can check it with VecNorm() before calling KSPSolve(). Matt > 2. I am not sure if I set the solver and matrix properly. Did I miss > something? > > Thank you. > > > **************************************** > INPUT: > > CALL KSPCreate(PETSC_COMM_WORLD,solver,ierr) > CALL DMSetMatType(da_ksp,MATMPISBAIJ,ierr) > CALL DMCreateMatrix(da_ksp,mat,ierr) > CALL > MatMPISBAIJSetPreallocation(mat,1,4,PETSC_NULL_INTEGER,1,PETSC_NULL_INTEGER,ierr) > CALL MatZeroEntries(mat,ierr) > CALL MatAssemblyBegin(mat,MAT_FINAL_ASSEMBLY,ierr) > CALL MatAssemblyEnd(mat,MAT_FINAL_ASSEMBLY,ierr) > > CALL KSPSetOperators(solver,mat,mat,SAME_NONZERO_PATTERN,ierr) > > ! CALL KSPSetType(solver,KSPCR,ierr) > CALL KSPSetType(solver,KSPLGMRES,ierr) > > epsi = 1.0-e5 > CALL KSPSetTolerances(solver,epsi,PETSC_DEFAULT_DOUBLE_PRECISION,& > & PETSC_DEFAULT_DOUBLE_PRECISION,itmax,ierr) > > CALL KSPGetPC(solver,gfprec,ierr) > CALL PCSetType(prec,PCBJACOBI,ierr) > > CALL KSPMonitorSet(solver,KSPMonitorTrueResidualNorm,& > PETSC_NULL_OBJECT,PETSC_NULL_FUNCTION,ierr ) > > CALL KSPSetInitialGuessNonzero(solver,PETSC_TRUE,ierr) > CALL KSPSolve(solver,bvec,xsol,ierr) > > CALL KSPGetIterationNumber(solver,iter,ierr) > CALL KSPGetResidualNorm(solver,dmax,ierr) > CALL KSPView(solver,PETSC_VIEWER_STDOUT_WORLD,ierr) > > ************************************** > OUTPUT CALL KSPSetType(solver,KSPCR,ierr): > > 0 KSP preconditioned resid norm 2.226482634319e+05 true resid norm > 1.204978940624e+07 ||r(i)||/||b|| inf > 1 KSP preconditioned resid norm 1.684243557244e+05 true resid norm > 6.742321430949e+06 ||r(i)||/||b|| inf > 2 KSP preconditioned resid norm 1.039386033131e+05 true resid norm > 5.094347016880e+06 ||r(i)||/||b|| inf > 3 KSP preconditioned resid norm 3.767761162917e+04 true resid norm > 2.085014289432e+06 ||r(i)||/||b|| inf > 4 KSP preconditioned resid norm 2.220316358489e+04 true resid norm > 1.039841616110e+06 ||r(i)||/||b|| inf > 5 KSP preconditioned resid norm 1.009108756815e+04 true resid norm > 6.764592620620e+05 ||r(i)||/||b|| inf > 6 KSP preconditioned resid norm 7.266143334386e+03 true resid norm > 4.713756053613e+05 ||r(i)||/||b|| inf > 7 KSP preconditioned resid norm 4.925270026573e+03 true resid norm > 3.276759177651e+05 ||r(i)||/||b|| inf > 8 KSP preconditioned resid norm 2.595039666791e+03 true resid norm > 1.774916597474e+05 ||r(i)||/||b|| inf > 9 KSP preconditioned resid norm 1.970388137453e+03 true resid norm > 1.449811653036e+05 ||r(i)||/||b|| inf > 10 KSP preconditioned resid norm 1.455914234388e+03 true resid norm > 7.916294162841e+04 ||r(i)||/||b|| inf > 11 KSP preconditioned resid norm 8.335194818556e+02 true resid norm > 4.530953608250e+04 ||r(i)||/||b|| inf > 12 KSP preconditioned resid norm 3.022320555777e+02 true resid norm > 1.728551635678e+04 ||r(i)||/||b|| inf > 13 KSP preconditioned resid norm 7.190336024797e+01 true resid norm > 4.186842086105e+03 ||r(i)||/||b|| inf > 14 KSP preconditioned resid norm 1.718291655675e+01 true resid norm > 1.089751055004e+03 ||r(i)||/||b|| inf > 15 KSP preconditioned resid norm 1.150683059424e+01 true resid norm > 8.672405273471e+02 ||r(i)||/||b|| inf > 16 KSP preconditioned resid norm 8.663479440949e+00 true resid norm > 5.776737380768e+02 ||r(i)||/||b|| inf > 17 KSP preconditioned resid norm 5.282161990683e+00 true resid norm > 2.977735906695e+02 ||r(i)||/||b|| inf > 18 KSP preconditioned resid norm 3.802629315725e+00 true resid norm > 2.789114564993e+02 ||r(i)||/||b|| inf > 19 KSP preconditioned resid norm 1.722575171383e+00 true resid norm > 1.051323829526e+02 ||r(i)||/||b|| inf > KSP Object: 1 MPI processes > type: cr > maximum iterations=1000 > tolerances: relative=1e-05, absolute=1e-50, divergence=10000 > left preconditioning > using nonzero initial guess > using PRECONDITIONED norm type for convergence test > PC Object: 1 MPI processes > type: bjacobi > block Jacobi: number of blocks = 1 > Local solve is same for all blocks, in the following KSP and PC > objects: > KSP Object: (sub_) 1 MPI processes > type: preonly > maximum iterations=10000, initial guess is zero > tolerances: relative=1e-05, absolute=1e-50, divergence=10000 > left preconditioning > using NONE norm type for convergence test > PC Object: (sub_) 1 MPI processes > type: icc > 0 levels of fill > tolerance for zero pivot 2.22045e-14 > using Manteuffel shift [POSITIVE_DEFINITE] > matrix ordering: natural > factor fill ratio given 1, needed 1 > Factored matrix follows: > Mat Object: 1 MPI processes > type: seqsbaij > rows=27, cols=27 > package used to perform factorization: petsc > total: nonzeros=81, allocated nonzeros=81 > total number of mallocs used during MatSetValues calls =0 > block size is 1 > linear system matrix = precond matrix: > Mat Object: 1 MPI processes > type: seqsbaij > rows=27, cols=27 > total: nonzeros=81, allocated nonzeros=108 > total number of mallocs used during MatSetValues calls =0 > block size is 1 > linear system matrix = precond matrix: > Mat Object: 1 MPI processes > type: mpisbaij > rows=27, cols=27 > total: nonzeros=81, allocated nonzeros=135 > total number of mallocs used during MatSetValues calls =0 > block size is 1 > > OUTPUT: CALL KSPSetType(solver,KSPLGMRES,ierr) > > 0 KSP preconditioned resid norm 2.362537325084e+04 true resid norm > 1.138584383312e+06 ||r(i)||/||b|| 1.000000000000e+00 > 1 KSP preconditioned resid norm 8.501213683423e+03 true resid norm > 3.655528853686e+05 ||r(i)||/||b|| 3.210591070162e-01 > 2 KSP preconditioned resid norm 5.487567253725e+03 true resid norm > 3.005741194777e+05 ||r(i)||/||b|| 2.639893220769e-01 > 3 KSP preconditioned resid norm 2.470452880657e+03 true resid norm > 1.545469272201e+05 ||r(i)||/||b|| 1.357360328187e-01 > 4 KSP preconditioned resid norm 1.750803325456e+03 true resid norm > 1.182312309352e+05 ||r(i)||/||b|| 1.038405520646e-01 > 5 KSP preconditioned resid norm 1.123492053552e+03 true resid norm > 6.754319630701e+04 ||r(i)||/||b|| 5.932208213726e-02 > 6 KSP preconditioned resid norm 5.150241959277e+02 true resid norm > 3.689413898730e+04 ||r(i)||/||b|| 3.240351749775e-02 > 7 KSP preconditioned resid norm 4.182894544871e+02 true resid norm > 3.052196222024e+04 ||r(i)||/||b|| 2.680693909701e-02 > 8 KSP preconditioned resid norm 2.520093155629e+02 true resid norm > 1.880976788356e+04 ||r(i)||/||b|| 1.652031079932e-02 > 9 KSP preconditioned resid norm 1.186491314806e+02 true resid norm > 6.797080217853e+03 ||r(i)||/||b|| 5.969764136483e-03 > 10 KSP preconditioned resid norm 5.851092821372e+01 true resid norm > 2.973659280245e+03 ||r(i)||/||b|| 2.611716201127e-03 > 11 KSP preconditioned resid norm 1.669909189055e+01 true resid norm > 5.658829814125e+02 ||r(i)||/||b|| 4.970057465277e-04 > 12 KSP preconditioned resid norm 3.090594692756e+00 true resid norm > 2.161527454147e+02 ||r(i)||/||b|| 1.898434130864e-04 > 13 KSP preconditioned resid norm 2.164618839184e+00 true resid norm > 1.620745991834e+02 ||r(i)||/||b|| 1.423474637093e-04 > 14 KSP preconditioned resid norm 1.291593952428e+00 true resid norm > 9.095542547366e+01 ||r(i)||/||b|| 7.988465923722e-05 > 15 KSP preconditioned resid norm 6.100583411632e-01 true resid norm > 4.021646656091e+01 ||r(i)||/||b|| 3.532146334551e-05 > 16 KSP preconditioned resid norm 2.723496807925e-01 true resid norm > 1.676660466866e+01 ||r(i)||/||b|| 1.472583403954e-05 > 17 KSP preconditioned resid norm 1.377718471538e-01 true resid norm > 8.551854245272e+00 ||r(i)||/||b|| 7.510953400221e-06 > KSP Object: 1 MPI processes > type: lgmres > GMRES: restart=30, using Classical (unmodified) Gram-Schmidt > Orthogonalization with no iterative refinement > GMRES: happy breakdown tolerance 1e-30 > LGMRES: aug. dimension=2 > LGMRES: number of matvecs=170 > maximum iterations=1000 > tolerances: relative=1e-05, absolute=1e-50, divergence=10000 > left preconditioning > using nonzero initial guess > using PRECONDITIONED norm type for convergence test > PC Object: 1 MPI processes > type: bjacobi > block Jacobi: number of blocks = 1 > Local solve is same for all blocks, in the following KSP and PC > objects: > KSP Object: (sub_) 1 MPI processes > type: preonly > maximum iterations=10000, initial guess is zero > tolerances: relative=1e-05, absolute=1e-50, divergence=10000 > left preconditioning > using NONE norm type for convergence test > PC Object: (sub_) 1 MPI processes > type: icc > 0 levels of fill > tolerance for zero pivot 2.22045e-14 > using Manteuffel shift [POSITIVE_DEFINITE] > matrix ordering: natural > factor fill ratio given 1, needed 1 > Factored matrix follows: > Mat Object: 1 MPI processes > type: seqsbaij > rows=27, cols=27 > package used to perform factorization: petsc > total: nonzeros=81, allocated nonzeros=81 > total number of mallocs used during MatSetValues calls =0 > block size is 1 > linear system matrix = precond matrix: > Mat Object: 1 MPI processes > type: seqsbaij > rows=27, cols=27 > total: nonzeros=81, allocated nonzeros=108 > total number of mallocs used during MatSetValues calls =0 > block size is 1 > linear system matrix = precond matrix: > Mat Object: 1 MPI processes > type: mpisbaij > rows=27, cols=27 > total: nonzeros=81, allocated nonzeros=135 > total number of mallocs used during MatSetValues calls =0 > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From bisheshkh at gmail.com Wed Dec 4 10:34:25 2013 From: bisheshkh at gmail.com (Bishesh Khanal) Date: Wed, 4 Dec 2013 17:34:25 +0100 Subject: [petsc-users] solving stokes-like equation in 3D staggered grid in irregular domain with petscsection In-Reply-To: <87siu929q5.fsf@jedbrown.org> References: <874n7hb7zj.fsf@jedbrown.org> <87y54s57we.fsf@jedbrown.org> <87zjoi0z1r.fsf@jedbrown.org> <87siu929q5.fsf@jedbrown.org> Message-ID: On Tue, Dec 3, 2013 at 5:13 PM, Jed Brown wrote: > Bishesh Khanal writes: > > > With options ksp_converged_reason and ksp_monitor_true_residual set only > > for the fieldsplit_1 , I get the result as below. -ksp_view reports that > > the preconditioner for the schur complement is built from A11. > > preconditioned resid norm below looks to be the same and in the order of > > 10e+14 for every iteration. Does this imply the precondtioner is > singular? > > Does it mean that the problem is possibly in assembling the A11 block? > > I would suspect that, yes. The preconditioner is amplifying the > residual, but then it is not converging either, which indicates that the > original system may also be singular, with an inconsistent right hand > side. > I did some experiments and it seems that the matrix is assembled all right, because the solver converges for some RHS and not for some others. But I have not understood why RHS could be inconsistent. I present below 4 cases out of which 3 cases converge. The only difference between 2nd and 4th is the RHS. Do the following results give clues on what I should look at or consider about ? Experiments: The domain is a 11^3 sized cuboid partitioned into A and B where B is a sphere in the center with diameter 4 units. To make it easier to refer to, here are the equations I'm solving: div( mu (grad(u) )) - grad(p) = f(x) div(u) + c(x) p = g(x) WITH mu = 1 everywhere. The options I give: -pc_type fieldsplit -pc_fieldsplit_type schur -pc_fieldsplit_dm_splits 0 -pc_fieldsplit_0_fields 0,1,2 -pc_fieldsplit_1_fields 3 -fieldsplit_0_pc_type lu -fieldsplit_0_pc_factor_mat_solver_package superlu_dist -fieldsplit_1_ksp_converged_reason -ksp_converged_reason -fieldsplit_1_ksp_monitor_true_residual -ksp_monitor_true_residual Now, different cases starting with the ones where the solver converges. 1. A driven cavity test: c(x) = 0 everywhere; f(x) = g(x) = 0 everywhere; boundary condition ux = 5; It converges and results in the expected velocity field. Here is the output: Residual norms for fieldsplit_1_ solve. 0 KSP preconditioned resid norm 5.054223508734e+14 true resid norm 1.122436826478e+01 ||r(i)||/||b|| 1.000000000000e+00 1 KSP preconditioned resid norm 2.749535352327e+14 true resid norm 6.107840938579e+00 ||r(i)||/||b|| 5.441589935838e-01 2 KSP preconditioned resid norm 1.505961910867e+14 true resid norm 3.348977798988e+00 ||r(i)||/||b|| 2.983667071487e-01 3 KSP preconditioned resid norm 5.174359504409e+13 true resid norm 1.162887407911e+00 ||r(i)||/||b|| 1.036038180928e-01 4 KSP preconditioned resid norm 2.080045117083e+13 true resid norm 4.956790811875e-01 ||r(i)||/||b|| 4.416097810535e-02 5 KSP preconditioned resid norm 7.755059044492e+12 true resid norm 2.492038982077e-01 ||r(i)||/||b|| 2.220204222892e-02 6 KSP preconditioned resid norm 2.466612045429e+12 true resid norm 1.883096820091e-01 ||r(i)||/||b|| 1.677686240925e-02 7 KSP preconditioned resid norm 8.579343124832e+11 true resid norm 1.811484167702e-01 ||r(i)||/||b|| 1.613885187094e-02 8 KSP preconditioned resid norm 2.905820616232e+11 true resid norm 1.802566230397e-01 ||r(i)||/||b|| 1.605940029652e-02 9 KSP preconditioned resid norm 1.050448005003e+11 true resid norm 1.801573367439e-01 ||r(i)||/||b|| 1.605055469439e-02 10 KSP preconditioned resid norm 3.403790595963e+10 true resid norm 1.801439682968e-01 ||r(i)||/||b|| 1.604936367439e-02 11 KSP preconditioned resid norm 1.299335081666e+10 true resid norm 1.801425947866e-01 ||r(i)||/||b|| 1.604924130579e-02 12 KSP preconditioned resid norm 4.493652071254e+09 true resid norm 1.801423944430e-01 ||r(i)||/||b|| 1.604922345681e-02 Linear solve converged due to CONVERGED_RTOL iterations 12 0 KSP preconditioned resid norm 6.460680353553e+01 true resid norm 1.049466531148e+02 ||r(i)||/||b|| 1.000000000000e+00 Residual norms for fieldsplit_1_ solve. 0 KSP preconditioned resid norm 7.823051492951e+12 true resid norm 1.737068454898e-01 ||r(i)||/||b|| 1.000000000000e+00 1 KSP preconditioned resid norm 4.255798896540e+12 true resid norm 9.449771864126e-02 ||r(i)||/||b|| 5.440068776497e-01 2 KSP preconditioned resid norm 2.330965266450e+12 true resid norm 5.175786465381e-02 ||r(i)||/||b|| 2.979609957678e-01 3 KSP preconditioned resid norm 8.009006255864e+11 true resid norm 1.778356846780e-02 ||r(i)||/||b|| 1.023769006780e-01 4 KSP preconditioned resid norm 3.219545074418e+11 true resid norm 7.148826788310e-03 ||r(i)||/||b|| 4.115454844715e-02 5 KSP preconditioned resid norm 1.200344957869e+11 true resid norm 2.665301219518e-03 ||r(i)||/||b|| 1.534367406191e-02 6 KSP preconditioned resid norm 3.817859308610e+10 true resid norm 8.477351604027e-04 ||r(i)||/||b|| 4.880263400170e-03 7 KSP preconditioned resid norm 1.327911515975e+10 true resid norm 2.948555889112e-04 ||r(i)||/||b|| 1.697432177068e-03 8 KSP preconditioned resid norm 4.497669563823e+09 true resid norm 9.986832819219e-05 ||r(i)||/||b|| 5.749245397360e-04 9 KSP preconditioned resid norm 1.625897964247e+09 true resid norm 3.610218721966e-05 ||r(i)||/||b|| 2.078339924824e-04 10 KSP preconditioned resid norm 5.268465651593e+08 true resid norm 1.169834375386e-05 ||r(i)||/||b|| 6.734532378890e-05 11 KSP preconditioned resid norm 2.011138479503e+08 true resid norm 4.465624568191e-06 ||r(i)||/||b|| 2.570782144826e-05 12 KSP preconditioned resid norm 6.955390134082e+07 true resid norm 1.544407052087e-06 ||r(i)||/||b|| 8.890881920815e-06 Linear solve converged due to CONVERGED_RTOL iterations 12 1 KSP preconditioned resid norm 5.821935836469e-04 true resid norm 1.801423947884e-01 ||r(i)||/||b|| 1.716513956775e-03 Linear solve converged due to CONVERGED_RTOL iterations 1 2. Same as 1 but with c(x) = 1 in the sphere B, 0 everywhere else; It converges and results in expected velocity field. Interesting thing in the output is that preconditioned norm is exactly the same as true resid norm in this case: using user defined split Residual norms for fieldsplit_1_ solve. 0 KSP preconditioned resid norm 1.069474142891e+01 true resid norm 1.069474142891e+01 ||r(i)||/||b|| 1.000000000000e+00 1 KSP preconditioned resid norm 1.426854491662e+00 true resid norm 1.426854491662e+00 ||r(i)||/||b|| 1.334164552875e-01 2 KSP preconditioned resid norm 9.997206363303e-02 true resid norm 9.997206363303e-02 ||r(i)||/||b|| 9.347777531377e-03 3 KSP preconditioned resid norm 1.010655985247e-02 true resid norm 1.010655985247e-02 ||r(i)||/||b|| 9.450027305152e-04 4 KSP preconditioned resid norm 8.627873205985e-04 true resid norm 8.627873205984e-04 ||r(i)||/||b|| 8.067397667664e-05 5 KSP preconditioned resid norm 6.055739652458e-05 true resid norm 6.055739652441e-05 ||r(i)||/||b|| 5.662352561486e-06 Linear solve converged due to CONVERGED_RTOL iterations 5 0 KSP preconditioned resid norm 4.960608132875e+01 true resid norm 9.486832980505e+01 ||r(i)||/||b|| 1.000000000000e+00 Residual norms for fieldsplit_1_ solve. 0 KSP preconditioned resid norm 2.155933535217e-01 true resid norm 2.155933535217e-01 ||r(i)||/||b|| 1.000000000000e+00 1 KSP preconditioned resid norm 2.876369982498e-02 true resid norm 2.876369982498e-02 ||r(i)||/||b|| 1.334164497890e-01 2 KSP preconditioned resid norm 2.015316291302e-03 true resid norm 2.015316291302e-03 ||r(i)||/||b|| 9.347766331297e-03 3 KSP preconditioned resid norm 2.037356487804e-04 true resid norm 2.037356487804e-04 ||r(i)||/||b|| 9.449996739339e-04 4 KSP preconditioned resid norm 1.739275204864e-05 true resid norm 1.739275204864e-05 ||r(i)||/||b|| 8.067387869121e-05 5 KSP preconditioned resid norm 1.220766583893e-06 true resid norm 1.220766583895e-06 ||r(i)||/||b|| 5.662357229266e-06 Linear solve converged due to CONVERGED_RTOL iterations 5 1 KSP preconditioned resid norm 4.970924030301e-05 true resid norm 6.058440172041e-05 ||r(i)||/||b|| 6.386156670504e-07 Linear solve converged due to CONVERGED_RTOL iterations 1 3. Different RHS test: c(x) = 0 everywhere; g(x) = -0.2 in the sphere B, 0 everywhere else. f(x) = grad(g); boundary condition: ux = 5; other wall velocities 0. The solver converges; Residual norms for fieldsplit_1_ solve. 0 KSP preconditioned resid norm 5.054223508734e+14 true resid norm 1.122436826478e+01 ||r(i)||/||b|| 1.000000000000e+00 1 KSP preconditioned resid norm 2.749535352327e+14 true resid norm 6.107840938579e+00 ||r(i)||/||b|| 5.441589935838e-01 2 KSP preconditioned resid norm 1.505961910867e+14 true resid norm 3.348977798988e+00 ||r(i)||/||b|| 2.983667071487e-01 3 KSP preconditioned resid norm 5.174359504409e+13 true resid norm 1.162887407911e+00 ||r(i)||/||b|| 1.036038180928e-01 4 KSP preconditioned resid norm 2.080045117083e+13 true resid norm 4.956790811875e-01 ||r(i)||/||b|| 4.416097810535e-02 5 KSP preconditioned resid norm 7.755059044492e+12 true resid norm 2.492038982077e-01 ||r(i)||/||b|| 2.220204222892e-02 6 KSP preconditioned resid norm 2.466612045429e+12 true resid norm 1.883096820090e-01 ||r(i)||/||b|| 1.677686240925e-02 7 KSP preconditioned resid norm 8.579343124832e+11 true resid norm 1.811484167702e-01 ||r(i)||/||b|| 1.613885187094e-02 8 KSP preconditioned resid norm 2.905820616232e+11 true resid norm 1.802566230397e-01 ||r(i)||/||b|| 1.605940029652e-02 9 KSP preconditioned resid norm 1.050448005003e+11 true resid norm 1.801573367439e-01 ||r(i)||/||b|| 1.605055469439e-02 10 KSP preconditioned resid norm 3.403790595952e+10 true resid norm 1.801439682968e-01 ||r(i)||/||b|| 1.604936367439e-02 11 KSP preconditioned resid norm 1.299335081659e+10 true resid norm 1.801425947866e-01 ||r(i)||/||b|| 1.604924130579e-02 12 KSP preconditioned resid norm 4.493652072271e+09 true resid norm 1.801423944430e-01 ||r(i)||/||b|| 1.604922345681e-02 Linear solve converged due to CONVERGED_RTOL iterations 12 0 KSP preconditioned resid norm 6.298618568983e+01 true resid norm 9.494103433184e+01 ||r(i)||/||b|| 1.000000000000e+00 Residual norms for fieldsplit_1_ solve. 0 KSP preconditioned resid norm 8.024336532177e+12 true resid norm 1.781762765346e-01 ||r(i)||/||b|| 1.000000000000e+00 1 KSP preconditioned resid norm 4.365299472959e+12 true resid norm 9.692911986886e-02 ||r(i)||/||b|| 5.440068776497e-01 2 KSP preconditioned resid norm 2.390940384281e+12 true resid norm 5.308958077846e-02 ||r(i)||/||b|| 2.979609957678e-01 3 KSP preconditioned resid norm 8.215075861801e+11 true resid norm 1.824113496597e-02 ||r(i)||/||b|| 1.023769006780e-01 4 KSP preconditioned resid norm 3.302383114942e+11 true resid norm 7.332764204778e-03 ||r(i)||/||b|| 4.115454844715e-02 5 KSP preconditioned resid norm 1.231229515148e+11 true resid norm 2.733878712712e-03 ||r(i)||/||b|| 1.534367406191e-02 6 KSP preconditioned resid norm 3.916091815629e+10 true resid norm 8.695471611506e-04 ||r(i)||/||b|| 4.880263400170e-03 7 KSP preconditioned resid norm 1.362078326946e+10 true resid norm 3.024421449800e-04 ||r(i)||/||b|| 1.697432177068e-03 8 KSP preconditioned resid norm 4.613393408335e+09 true resid norm 1.024379137786e-04 ||r(i)||/||b|| 5.749245397361e-04 9 KSP preconditioned resid norm 1.667731887469e+09 true resid norm 3.703108691786e-05 ||r(i)||/||b|| 2.078339924825e-04 10 KSP preconditioned resid norm 5.404021874909e+08 true resid norm 1.199933903467e-05 ||r(i)||/||b|| 6.734532378861e-05 11 KSP preconditioned resid norm 2.062884538961e+08 true resid norm 4.580523903463e-06 ||r(i)||/||b|| 2.570782144823e-05 12 KSP preconditioned resid norm 7.134350475945e+07 true resid norm 1.584144235576e-06 ||r(i)||/||b|| 8.890881919785e-06 Linear solve converged due to CONVERGED_RTOL iterations 12 1 KSP preconditioned resid norm 5.821934702566e-04 true resid norm 1.801423947967e-01 ||r(i)||/||b|| 1.897413442612e-03 Linear solve converged due to CONVERGED_RTOL iterations 1 4. Same as 3, but with c(x) = 1 in the sphere B. using user defined split Residual norms for fieldsplit_1_ solve. 0 KSP preconditioned resid norm 5.174242038871e+13 true resid norm 1.076313011714e+01 ||r(i)||/||b|| 1.000000000000e+00 1 KSP preconditioned resid norm 5.174242038871e+13 true resid norm 1.075627696888e+01 ||r(i)||/||b|| 9.993632755352e-01 2 KSP preconditioned resid norm 5.174242038871e+13 true resid norm 1.075627696888e+01 ||r(i)||/||b|| 9.993632755352e-01 3 KSP preconditioned resid norm 5.174242038871e+13 true resid norm 1.075627696888e+01 ||r(i)||/||b|| 9.993632755352e-01 4 KSP preconditioned resid norm 5.174242038871e+13 true resid norm 1.075627696888e+01 ||r(i)||/||b|| 9.993632755352e-01 5 KSP preconditioned resid norm 5.174242038871e+13 true resid norm 1.075627696888e+01 ||r(i)||/||b|| 9.993632755352e-01 6 KSP preconditioned resid norm 5.174242038871e+13 true resid norm 1.075627696888e+01 ||r(i)||/||b|| 9.993632755352e-01 7 KSP preconditioned resid norm 5.174242038871e+13 true resid norm 1.075627696886e+01 ||r(i)||/||b|| 9.993632755341e-01 8 KSP preconditioned resid norm 5.174242038871e+13 true resid norm 1.075627696766e+01 ||r(i)||/||b|| 9.993632754221e-01 9 KSP preconditioned resid norm 5.174242038871e+13 true resid norm 1.075627641906e+01 ||r(i)||/||b|| 9.993632244519e-01 10 KSP preconditioned resid norm 5.174242038871e+13 true resid norm 1.075627067412e+01 ||r(i)||/||b|| 9.993626906904e-01 11 KSP preconditioned resid norm 5.174242038871e+13 true resid norm 1.410873813881e+01 ||r(i)||/||b|| 1.310839689315e+00 12 KSP preconditioned resid norm 5.174242038871e+13 true resid norm 1.215904758000e+01 ||r(i)||/||b|| 1.129694377720e+00 13 KSP preconditioned resid norm 5.174242038871e+13 true resid norm 1.597693687523e+01 ||r(i)||/||b|| 1.484413614009e+00 14 KSP preconditioned resid norm 5.174242038871e+13 true resid norm 2.266134843283e+01 ||r(i)||/||b|| 2.105460789399e+00 15 KSP preconditioned resid norm 5.174242038871e+13 true resid norm 2.035308798305e+01 ||r(i)||/||b|| 1.891000829827e+00 16 KSP preconditioned resid norm 5.174242038871e+13 true resid norm 2.407254169449e+01 ||r(i)||/||b|| 2.236574438152e+00 17 KSP preconditioned resid norm 5.174242038871e+13 true resid norm 3.477754796008e+01 ||r(i)||/||b|| 3.231174164168e+00 18 KSP preconditioned resid norm 5.174242038871e+13 true resid norm 3.156748925892e+01 ||r(i)||/||b|| 2.932928331755e+00 19 KSP preconditioned resid norm 5.174242038871e+13 true resid norm 3.324768650679e+01 ||r(i)||/||b|| 3.089035080403e+00 20 KSP preconditioned resid norm 5.174242038871e+13 true resid norm 3.488562155804e+01 ||r(i)||/||b|| 3.241215257863e+00 21 KSP preconditioned resid norm 5.174242038871e+13 true resid norm 5.078895151790e+01 ||r(i)||/||b|| 4.718790069909e+00 22 KSP preconditioned resid norm 5.174242038871e+13 true resid norm 4.939231093930e+01 ||r(i)||/||b|| 4.589028507669e+00 23 KSP preconditioned resid norm 5.174242038871e+13 true resid norm 5.086042832143e+01 ||r(i)||/||b|| 4.725430963659e+00 24 KSP preconditioned resid norm 5.174242038871e+13 true resid norm 6.737635727583e+01 ||r(i)||/||b|| 6.259922210597e+00 25 KSP preconditioned resid norm 5.174242038871e+13 true resid norm 6.509813461374e+01 ||r(i)||/||b|| 6.048253055129e+00 26 KSP preconditioned resid norm 5.174242038871e+13 true resid norm 6.471858831835e+01 ||r(i)||/||b|| 6.012989494133e+00 27 KSP preconditioned resid norm 5.174242038871e+13 true resid norm 1.033462891313e+02 ||r(i)||/||b|| 9.601880494476e+00 28 KSP preconditioned resid norm 5.174242038871e+13 true resid norm 7.685802125780e+01 ||r(i)||/||b|| 7.140861480008e+00 29 KSP preconditioned resid norm 5.174242038871e+13 true resid norm 8.469815519206e+01 ||r(i)||/||b|| 7.869286561646e+00 And it continues, it's not converging. When c(x) is zero everywhere, it is easy to think of a compatibility condition that total sum of g(x) should be zero if we enforce zero velocity on all the walls. But when we release the incompressibilty condition in some parts of the domain by putting non zero c(x), I think I do not need to take care about the total sum of g(x) being zero. But I'm not sure what is going wrong with this RHS. > > It is the diagonal of this block corresponding to the domain B cells > > that changes when I set c(x) = 1. Residual norms for fieldsplit_1_ > > solve. 0 KSP preconditioned resid norm 2.890735677419e+14 true resid > > norm 1.265820125363e+01 ||r(i)||/||b|| 1.000000000000e+00 1 KSP > > preconditioned resid norm 2.890735677419e+14 true resid norm > > 1.117075725063e+01 ||r(i)||/||b|| 8.824916768826e-01 2 KSP > > preconditioned resid norm 2.890735677419e+14 true resid norm > > 1.109990265188e+01 ||r(i)||/||b|| 8.768941518210e-01 3 KSP > > preconditioned resid norm 2.890735677419e+14 true resid norm > > 1.109895437193e+01 ||r(i)||/||b|| 8.768192375476e-01 4 KSP > > preconditioned resid norm 2.890735677419e+14 true resid norm > > 1.109892185160e+01 ||r(i)||/||b|| 8.768166684358e-01 5 KSP > > preconditioned resid norm 2.890735677419e+14 true resid norm > > 1.109889770615e+01 ||r(i)||/||b|| 8.768147609417e-01 6 KSP > > preconditioned resid norm 2.890735677419e+14 true resid norm > > 1.109886495655e+01 ||r(i)||/||b|| 8.768121737173e-01 7 KSP > > preconditioned resid norm 2.890735677419e+14 true resid norm > > 1.109886096749e+01 ||r(i)||/||b|| 8.768118585813e-01 8 KSP > > preconditioned resid norm 2.890735677419e+14 true resid norm > > 1.109885236160e+01 ||r(i)||/||b|| 8.768111787141e-01 9 KSP > > preconditioned resid norm 2.890735677419e+14 true resid norm > > 1.109882829958e+01 ||r(i)||/||b|| 8.768092778106e-01 10 KSP > > preconditioned resid norm 2.890735677419e+14 true resid norm > > 1.109882703198e+01 ||r(i)||/||b|| 8.768091776699e-01 11 KSP > > preconditioned resid norm 2.890735677419e+14 true resid norm > > 1.109882171007e+01 ||r(i)||/||b|| 8.768087572386e-01 12 KSP > > preconditioned resid norm 2.890735677419e+14 true resid norm > > 1.109882129003e+01 ||r(i)||/||b|| 8.768087240548e-01 13 KSP > > preconditioned resid norm 2.890735677419e+14 true resid norm > > 1.109880708177e+01 ||r(i)||/||b|| 8.768076016004e-01 14 KSP > > preconditioned resid norm 2.890735677419e+14 true resid norm > > 1.109884240385e+01 ||r(i)||/||b|| 8.768103920501e-01 15 KSP > > preconditioned resid norm 2.890735677419e+14 true resid norm > > 1.110342530238e+01 ||r(i)||/||b|| 8.771724417953e-01 16 KSP > > preconditioned resid norm 2.890735677419e+14 true resid norm > > 1.668559859400e+01 ||r(i)||/||b|| 1.318165058342e+00 17 KSP > > preconditioned resid norm 2.890735677419e+14 true resid norm > > 2.379573238198e+02 ||r(i)||/||b|| 1.879866807707e+01 18 KSP > > preconditioned resid norm 2.890735677419e+14 true resid norm > > 7.475251038198e+02 ||r(i)||/||b|| 5.905460727331e+01 19 KSP > > preconditioned resid norm 2.890735677419e+14 true resid norm > > 7.610489296073e+02 ||r(i)||/||b|| 6.012299175517e+01 20 KSP > > preconditioned resid norm 2.890735677419e+14 true resid norm > > 1.162871674016e+03 ||r(i)||/||b|| 9.186705525659e+01 21 KSP > > preconditioned resid norm 2.890735677419e+14 true resid norm > > 2.438552126509e+03 ||r(i)||/||b|| 1.926460227364e+02 22 KSP > > preconditioned resid norm 2.890735677419e+14 true resid norm > > 2.112154724517e+03 ||r(i)||/||b|| 1.668605738047e+02 23 KSP > > preconditioned resid norm 2.890735677419e+14 true resid norm > > 1.867895524623e+03 ||r(i)||/||b|| 1.475640564719e+02 24 KSP > > preconditioned resid norm 2.890735677419e+14 true resid norm > > 2.760528221023e+03 ||r(i)||/||b|| 2.180821876435e+02 25 KSP > > preconditioned resid norm 2.890735677419e+14 true resid norm > > 2.585072583244e+03 ||r(i)||/||b|| 2.042211631374e+02 and it continues. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zhenglun.wei at gmail.com Wed Dec 4 17:25:12 2013 From: zhenglun.wei at gmail.com (Alan Z. Wei) Date: Wed, 04 Dec 2013 17:25:12 -0600 Subject: [petsc-users] An issue of order of accuracy. Message-ID: <529FB9D8.8050708@gmail.com> Dear all, I hope you had a great Thanksgiving. Currently, I tested the order of accuracy for /src/ksp/ksp/tutorial/example/ex45.c. Since the 2nd-order discretization is used in this program and ksp solver is converged to 10^-7, I expected that the solution should provides a 2nd-order in L2 norm. However, as I tested (even with a Laplace equation), the L2 norm slope is much less than 2. Sometime, if the grid size is reduced, the L2 norm increases. Could anyone help me about this issue, please? Here is the L2 norm outputted: Grid L2 norm (10^-8) 0.05 4.36242 0.025 2.20794 0.0125 7.02749 0.00625 12.64 Once the grid size is reduced to half, the number of the grid will be multiplied by 8 in order to keep the same size of the computational domain. The code is also attached. It is from ex45.c with very little modifications. thanks in advance, Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- static char help[] = "Solves 3D Laplacian using multigrid.\n\n"; #include #include const double dx = 0.0125; const double dy = 0.0125; const double dz = 0.0125; const int imax = 40; const int jmax = 40; const int kmax = 40; extern PetscErrorCode ComputeMatrix(KSP,Mat,Mat,MatStructure*,void*); extern PetscErrorCode ComputeRHS(KSP,Vec,void*); extern PetscErrorCode ComputeInitialGuess(KSP,Vec,void*); extern PetscErrorCode PostP(DM, Vec, Vec); PetscInt i,j,k,mx,my,mz,xm,ym,zm,xs,ys,zs; #undef __FUNCT__ #define __FUNCT__ "main" int main(int argc,char **argv) { PetscErrorCode ierr; KSP ksp; PetscReal norm; DM da; Vec x,b,r; Mat A; double t1, t2; PetscInitialize(&argc,&argv,(char*)0,help); t1 = MPI_Wtime(); ierr = KSPCreate(PETSC_COMM_WORLD,&ksp);CHKERRQ(ierr); ierr = DMDACreate3d(PETSC_COMM_WORLD,DMDA_BOUNDARY_NONE,DMDA_BOUNDARY_NONE,DMDA_BOUNDARY_NONE,DMDA_STENCIL_STAR,-imax,-jmax,-kmax,PETSC_DECIDE,PETSC_DECIDE,PETSC_DECIDE,1,1,0,0,0,&da);CHKERRQ(ierr); ierr = KSPSetDM(ksp,da);CHKERRQ(ierr); ierr = KSPSetComputeInitialGuess(ksp,ComputeInitialGuess,NULL);CHKERRQ(ierr); ierr = KSPSetComputeRHS(ksp,ComputeRHS,NULL);CHKERRQ(ierr); ierr = KSPSetComputeOperators(ksp,ComputeMatrix,NULL);CHKERRQ(ierr); ierr = DMDestroy(&da);CHKERRQ(ierr); ierr = KSPSetFromOptions(ksp);CHKERRQ(ierr); ierr = KSPSolve(ksp,NULL,NULL);CHKERRQ(ierr); ierr = KSPGetSolution(ksp,&x);CHKERRQ(ierr); ierr = KSPGetRhs(ksp,&b);CHKERRQ(ierr); ierr = VecDuplicate(b,&r);CHKERRQ(ierr); ierr = KSPGetOperators(ksp,&A,NULL,NULL);CHKERRQ(ierr); ierr = MatMult(A,x,r);CHKERRQ(ierr); ierr = VecAXPY(r,-1.0,b);CHKERRQ(ierr); ierr = VecNorm(r,NORM_2,&norm);CHKERRQ(ierr); ierr = PetscPrintf(PETSC_COMM_WORLD,"Residual norm %G\n",norm);CHKERRQ(ierr); ierr = KSPGetDM(ksp, &da);CHKERRQ(ierr); ierr = PostP(da, x, b); ierr = VecDestroy(&r);CHKERRQ(ierr); ierr = KSPDestroy(&ksp);CHKERRQ(ierr); t2 = MPI_Wtime(); printf("Total Time Elapsed: %f\n", t2-t1); ierr = PetscFinalize(); return 0; } #undef __FUNCT__ #define __FUNCT__ "ComputeRHS" PetscErrorCode ComputeRHS(KSP ksp,Vec b,void *ctx) { PetscErrorCode ierr; PetscInt i,j,k,mx,my,mz,xm,ym,zm,xs,ys,zs, mm,nn,pp; DM dm; PetscScalar Hx,Hy,Hz,HxHydHz,HyHzdHx,HxHzdHy; PetscScalar ***barray; PetscFunctionBeginUser; ierr = KSPGetDM(ksp,&dm);CHKERRQ(ierr); ierr = DMDAGetInfo(dm,0,&mx,&my,&mz,&mm,&nn,&pp,0,0,0,0,0,0);CHKERRQ(ierr); printf("mx = %d, my = %d, mz =%d, mm = %d, nn = %d, pp = %d\n", mx, my, mz, mm, nn, pp); // Hx = 1.0 / (PetscReal)(mx-1); Hy = 1.0 / (PetscReal)(my-1); Hz = 1.0 / (PetscReal)(mz-1); Hx = dx; Hy = dy; Hz = dz; HxHydHz = Hx*Hy/Hz; HxHzdHy = Hx*Hz/Hy; HyHzdHx = Hy*Hz/Hx; ierr = DMDAGetCorners(dm,&xs,&ys,&zs,&xm,&ym,&zm);CHKERRQ(ierr); ierr = DMDAVecGetArray(dm,b,&barray);CHKERRQ(ierr); for (k=zs; k out & #/bio/work1/zlwei/PETSc/petsc-dev/linux-gnu-c-nodebug/bin/mpiexec -np 32 ./ex45 -pc_type mg -ksp_type cg -da_refine 5 -pc_mg_galerkin -ksp_rtol 1.0e-7 -mg_levels_pc_type jacobi -mg_levels_ksp_type chebyshev -dm_view -log_summary -pc_mg_log -pc_mg_monitor -ksp_view -ksp_monitor -pc_mg_log> out & /bio/work1/zlwei/PETSc/petsc-dev/linux-gnu-c-nodebug/bin/mpiexec -np 1 ./ex45 -pc_type gamg -ksp_type gmres -pc_gamg_agg_nsmooths 1 -mg_levels_ksp_max_it 1 -mg_levels_ksp_type richardson -ksp_rtol 1.0e-7 From bsmith at mcs.anl.gov Wed Dec 4 18:19:52 2013 From: bsmith at mcs.anl.gov (Barry Smith) Date: Wed, 4 Dec 2013 18:19:52 -0600 Subject: [petsc-users] An issue of order of accuracy. In-Reply-To: <529FB9D8.8050708@gmail.com> References: <529FB9D8.8050708@gmail.com> Message-ID: Alan, I changed your initial grid size to 10 to get faster solve times and get what is below. Note that your ?exact solution? is quadratic, since the method is second order this means that not only is the ?exact solution? an exact solution to the PDE, it is also an exact solution to the algebraic equations (for any grid size) hence the L2Norm of the error is only due to the round off of the algebraic solution, not due to any discretization error. In general, when testing for discretization error you always need to ?cook up? an exact solution that is not completely represented in the approximation space. You also need to use a really tight algebraic tolerance to eliminate the algebraic error from the computation ~/Src/petsc/test-dir master $ ./ex45 -pc_type mg -ksp_rtol 1e-12 mx = 10, my = 10, mz =10, mm = 1, nn = 1, pp = 1 Residual norm 1.81616e-12 L2Norm = 1.107359e-12 Total Time Elapsed: 0.048599 ~/Src/petsc/test-dir master $ ./ex45 -pc_type mg -ksp_rtol 1e-12 -da_refine 1 mx = 19, my = 19, mz =19, mm = 1, nn = 1, pp = 1 Residual norm 3.36741e-12 L2Norm = 1.037148e-12 Total Time Elapsed: 0.183398 ~/Src/petsc/test-dir master $ ./ex45 -pc_type mg -ksp_rtol 1e-12 -da_refine 2 mx = 37, my = 37, mz =37, mm = 1, nn = 1, pp = 1 Residual norm 1.09476e-11 L2Norm = 2.330658e-12 Total Time Elapsed: 1.180839 ~/Src/petsc/test-dir master $ ./ex45 -pc_type mg -ksp_rtol 1e-12 -da_refine 3 mx = 73, my = 73, mz =73, mm = 1, nn = 1, pp = 1 Residual norm 3.19809e-11 L2Norm = 2.278763e-12 Total Time Elapsed: 10.819450 ~/Src/petsc/test-dir master $ ./ex45 -pc_type mg -da_refine 3 mx = 73, my = 73, mz =73, mm = 1, nn = 1, pp = 1 Residual norm 0.000103197 L2Norm = 1.011806e-05 Total Time Elapsed: 7.250106 On Dec 4, 2013, at 5:25 PM, Alan Z. Wei wrote: > Dear all, > I hope you had a great Thanksgiving. > Currently, I tested the order of accuracy for /src/ksp/ksp/tutorial/example/ex45.c. Since the 2nd-order discretization is used in this program and ksp solver is converged to 10^-7, I expected that the solution should provides a 2nd-order in L2 norm. However, as I tested (even with a Laplace equation), the L2 norm slope is much less than 2. Sometime, if the grid size is reduced, the L2 norm increases. Could anyone help me about this issue, please? > > Here is the L2 norm outputted: > > Grid L2 norm (10^-8) > 0.05 4.36242 > 0.025 2.20794 > 0.0125 7.02749 > 0.00625 12.64 > Once the grid size is reduced to half, the number of the grid will be multiplied by 8 in order to keep the same size of the computational domain. > The code is also attached. It is from ex45.c with very little modifications. > > thanks in advance, > Alan > > From mirzadeh at gmail.com Wed Dec 4 20:43:41 2013 From: mirzadeh at gmail.com (Mohammad Mirzadeh) Date: Wed, 4 Dec 2013 18:43:41 -0800 Subject: [petsc-users] question about PetscLogEvents Message-ID: Hi guys, I'm gathering statistics for scaling analysis of my code and I'm making use of PetscLogEvent objects. Some of events I'm trying to log have run-times of few (1-5) seconds. Should I worry about any additional overheads? In general how long should an event be to not worry about overheads? Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: From jedbrown at mcs.anl.gov Wed Dec 4 22:28:27 2013 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Wed, 04 Dec 2013 21:28:27 -0700 Subject: [petsc-users] question about PetscLogEvents In-Reply-To: References: Message-ID: <87haanncpg.fsf@jedbrown.org> Mohammad Mirzadeh writes: > I'm gathering statistics for scaling analysis of my code and I'm making use > of PetscLogEvent objects. Some of events I'm trying to log have run-times > of few (1-5) seconds. Should I worry about any additional overheads? > > In general how long should an event be to not worry about overheads? When you run with -log_summary, you'll see a line like this: Average time to get PetscTime(): 9.53674e-08 This value of around 100 nanoseconds is typical, and is about the same as a full cache miss. You generally want your events to take significantly longer than this, say at least a microsecond. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From katyghantous at gmail.com Thu Dec 5 05:25:52 2013 From: katyghantous at gmail.com (Katy Ghantous) Date: Thu, 5 Dec 2013 12:25:52 +0100 Subject: [petsc-users] Stiff ODEs In-Reply-To: <87mwkh26yd.fsf@jedbrown.org> References: <87mwkh26yd.fsf@jedbrown.org> Message-ID: thank you! i used master from petsc-dev on mac and have TSRK working now. i am using an explicit scheme since its a set of coupled nonlinear equations: d?n/d t = an ?n-2?n-1 - bn ?n-1?n+1 + cn ?n+1?n+2 where n can go up to 24 or so.. I didnt think it is possible to solve it implicitly. and TSRK works for it now. But i need petsc on linux too, and testing to verify installation (same options on configure as mac expect arch) gave an error: /home/katy/petsc/linux-gnu-cxx-debug/bin/mpicxx -o ex19.o -c -Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g -O0 -fPIC -I/home/katy/petsc/include -I/home/katy/petsc/linux-gnu-cxx-debug/include `pwd`/ex19.c /home/katy/petsc/linux-gnu-cxx-debug/bin/mpicxx -Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g -O0 -o ex19 ex19.o -Wl,-rpath,/home/katy/petsc/linux-gnu-cxx-debug/lib -L/home/katy/petsc/linux-gnu-cxx-debug/lib -lpetsc -Wl,-rpath,/home/katy/petsc/linux-gnu-cxx-debug/lib -lf2clapack -lf2cblas -lm -lpthread -lm -Wl,-rpath,/usr/lib/gcc/x86_64-linux-gnu/4.6 -L/usr/lib/gcc/x86_64-linux-gnu/4.6 -Wl,-rpath,/usr/lib/x86_64-linux-gnu -L/usr/lib/x86_64-linux-gnu -Wl,-rpath,/lib/x86_64-linux-gnu -L/lib/x86_64-linux-gnu -lmpichcxx -lstdc++ -ldl -lmpich -lopa -lmpl -lrt -lpthread -lgcc_s -ldl /home/katy/petsc/linux-gnu-cxx-debug/lib/libpetsc.so: undefined reference to `SNESDefaultMatrixFreeCreate2(_p_SNES*, _p_Vec*, _p_Mat**)' /home/katy/petsc/linux-gnu-cxx-debug/lib/libpetsc.so: undefined reference to `KSPCreate_DGMRES' /home/katy/petsc/linux-gnu-cxx-debug/lib/libpetsc.so: undefined reference to `PCCreate_TFS' On Tue, Dec 3, 2013 at 6:13 PM, Jed Brown wrote: > Katy Ghantous writes: > > > Hi, > > I have just started with Petsc and have very elementary equations. > > I am trying to solve a set of stiff ODEs explicitly. > > If your ODEs are stiff, why are you trying to use an explicit method? > (A common definition of "stiff" is that explicit methods do not work > well.) > > > They are very stiff and was hoping to use TSRK with variable time > > steps, but it is throwing a segmentation fault > > Always show the full error message. Please also use 'master' from > petsc-dev, because TSRK was overhauled recently to be consistent with > the other integrators (and adding a bunch of new functionality). > > https://bitbucket.org/petsc/petsc/wiki/ > > > otherwise non existent with Euler or TSSSP. 1. should i look into the > > TSRK error or is it something with the new version of Petsc? 2. i > > read somewhere (but cant find the source again) that TSSSP is not > > recommended for stiff equations (even with the rk solvers set as the > > type). > > Yes, TSSSP is for non-stiff problems that require "strong stability" > (like controlling total variation or positivity in transport-dominated > applications). > > > Is that correct? if so, what would be most optimized to solve stiff > > odes explicitly. > > Stiff ODEs should be solved *implicitly*. > > > 3. If it is not correct and TSSSP is useful for stiff equations, how > > can i run it with variable time steps, since with TSStep it is taking > > equal time steps which is the time step that i am specifying as an > > initial time step TSSetInitialTimeStep. > > TSSSP does not have an adaptive controller. A user can set a post-check > function that adapts the time step based on a custom stability function > (e.g., CFL criteria). > > TSROSW and TSARKIMEX are IMEX methods that can handle stiff problems. > They have adaptive error estimators and the like. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Thu Dec 5 05:40:28 2013 From: knepley at gmail.com (Matthew Knepley) Date: Thu, 5 Dec 2013 05:40:28 -0600 Subject: [petsc-users] Stiff ODEs In-Reply-To: References: <87mwkh26yd.fsf@jedbrown.org> Message-ID: On Thu, Dec 5, 2013 at 5:25 AM, Katy Ghantous wrote: > thank you! > i used master from petsc-dev on mac and have TSRK working now. > > i am using an explicit scheme since its a set of coupled nonlinear > equations: > > d?n/d t = an ?n-2?n-1 - bn ?n-1?n+1 + cn ?n+1?n+2 > > where n can go up to 24 or so.. I didnt think it is possible to solve it > implicitly. and TSRK works for it now. > It is. > But i need petsc on linux too, and testing to verify installation (same > options on configure as mac expect arch) gave an error: > > > /home/katy/petsc/linux-gnu-cxx-debug/bin/mpicxx -o ex19.o -c -Wall > -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g -O0 -fPIC > -I/home/katy/petsc/include > -I/home/katy/petsc/linux-gnu-cxx-debug/include `pwd`/ex19.c > /home/katy/petsc/linux-gnu-cxx-debug/bin/mpicxx -Wall -Wwrite-strings > -Wno-strict-aliasing -Wno-unknown-pragmas -g -O0 -o ex19 ex19.o > -Wl,-rpath,/home/katy/petsc/linux-gnu-cxx-debug/lib > -L/home/katy/petsc/linux-gnu-cxx-debug/lib -lpetsc > -Wl,-rpath,/home/katy/petsc/linux-gnu-cxx-debug/lib -lf2clapack -lf2cblas > -lm -lpthread -lm -Wl,-rpath,/usr/lib/gcc/x86_64-linux-gnu/4.6 > -L/usr/lib/gcc/x86_64-linux-gnu/4.6 -Wl,-rpath,/usr/lib/x86_64-linux-gnu > -L/usr/lib/x86_64-linux-gnu -Wl,-rpath,/lib/x86_64-linux-gnu > -L/lib/x86_64-linux-gnu -lmpichcxx -lstdc++ -ldl -lmpich -lopa -lmpl -lrt > -lpthread -lgcc_s -ldl > /home/katy/petsc/linux-gnu-cxx-debug/lib/libpetsc.so: undefined reference > to `SNESDefaultMatrixFreeCreate2(_p_SNES*, _p_Vec*, _p_Mat**)' > /home/katy/petsc/linux-gnu-cxx-debug/lib/libpetsc.so: undefined reference > to `KSPCreate_DGMRES' > /home/katy/petsc/linux-gnu-cxx-debug/lib/libpetsc.so: undefined reference > to `PCCreate_TFS' > Send make.log. Matt > On Tue, Dec 3, 2013 at 6:13 PM, Jed Brown wrote: > >> Katy Ghantous writes: >> >> > Hi, >> > I have just started with Petsc and have very elementary equations. >> > I am trying to solve a set of stiff ODEs explicitly. >> >> If your ODEs are stiff, why are you trying to use an explicit method? >> (A common definition of "stiff" is that explicit methods do not work >> well.) >> >> > They are very stiff and was hoping to use TSRK with variable time >> > steps, but it is throwing a segmentation fault >> >> Always show the full error message. Please also use 'master' from >> petsc-dev, because TSRK was overhauled recently to be consistent with >> the other integrators (and adding a bunch of new functionality). >> >> https://bitbucket.org/petsc/petsc/wiki/ >> >> > otherwise non existent with Euler or TSSSP. 1. should i look into the >> > TSRK error or is it something with the new version of Petsc? 2. i >> > read somewhere (but cant find the source again) that TSSSP is not >> > recommended for stiff equations (even with the rk solvers set as the >> > type). >> >> Yes, TSSSP is for non-stiff problems that require "strong stability" >> (like controlling total variation or positivity in transport-dominated >> applications). >> >> > Is that correct? if so, what would be most optimized to solve stiff >> > odes explicitly. >> >> Stiff ODEs should be solved *implicitly*. >> >> > 3. If it is not correct and TSSSP is useful for stiff equations, how >> > can i run it with variable time steps, since with TSStep it is taking >> > equal time steps which is the time step that i am specifying as an >> > initial time step TSSetInitialTimeStep. >> >> TSSSP does not have an adaptive controller. A user can set a post-check >> function that adapts the time step based on a custom stability function >> (e.g., CFL criteria). >> >> TSROSW and TSARKIMEX are IMEX methods that can handle stiff problems. >> They have adaptive error estimators and the like. >> > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From katyghantous at gmail.com Thu Dec 5 07:21:58 2013 From: katyghantous at gmail.com (Katy Ghantous) Date: Thu, 5 Dec 2013 14:21:58 +0100 Subject: [petsc-users] Stiff ODEs In-Reply-To: References: <87mwkh26yd.fsf@jedbrown.org> Message-ID: On Thu, Dec 5, 2013 at 12:40 PM, Matthew Knepley wrote: > On Thu, Dec 5, 2013 at 5:25 AM, Katy Ghantous wrote: > >> thank you! >> i used master from petsc-dev on mac and have TSRK working now. >> >> i am using an explicit scheme since its a set of coupled nonlinear >> equations: >> >> d?n/d t = an ?n-2?n-1 - bn ?n-1?n+1 + cn ?n+1?n+2 >> >> where n can go up to 24 or so.. I didnt think it is possible to solve it >> implicitly. and TSRK works for it now. >> > > It is. > > >> But i need petsc on linux too, and testing to verify installation (same >> options on configure as mac expect arch) gave an error: >> >> >> /home/katy/petsc/linux-gnu-cxx-debug/bin/mpicxx -o ex19.o -c -Wall >> -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g -O0 -fPIC >> -I/home/katy/petsc/include >> -I/home/katy/petsc/linux-gnu-cxx-debug/include `pwd`/ex19.c >> /home/katy/petsc/linux-gnu-cxx-debug/bin/mpicxx -Wall -Wwrite-strings >> -Wno-strict-aliasing -Wno-unknown-pragmas -g -O0 -o ex19 ex19.o >> -Wl,-rpath,/home/katy/petsc/linux-gnu-cxx-debug/lib >> -L/home/katy/petsc/linux-gnu-cxx-debug/lib -lpetsc >> -Wl,-rpath,/home/katy/petsc/linux-gnu-cxx-debug/lib -lf2clapack -lf2cblas >> -lm -lpthread -lm -Wl,-rpath,/usr/lib/gcc/x86_64-linux-gnu/4.6 >> -L/usr/lib/gcc/x86_64-linux-gnu/4.6 -Wl,-rpath,/usr/lib/x86_64-linux-gnu >> -L/usr/lib/x86_64-linux-gnu -Wl,-rpath,/lib/x86_64-linux-gnu >> -L/lib/x86_64-linux-gnu -lmpichcxx -lstdc++ -ldl -lmpich -lopa -lmpl -lrt >> -lpthread -lgcc_s -ldl >> /home/katy/petsc/linux-gnu-cxx-debug/lib/libpetsc.so: undefined reference >> to `SNESDefaultMatrixFreeCreate2(_p_SNES*, _p_Vec*, _p_Mat**)' >> /home/katy/petsc/linux-gnu-cxx-debug/lib/libpetsc.so: undefined reference >> to `KSPCreate_DGMRES' >> /home/katy/petsc/linux-gnu-cxx-debug/lib/libpetsc.so: undefined reference >> to `PCCreate_TFS' >> > > Send make.log. > > Matt > > >> On Tue, Dec 3, 2013 at 6:13 PM, Jed Brown wrote: >> >>> Katy Ghantous writes: >>> >>> > Hi, >>> > I have just started with Petsc and have very elementary equations. >>> > I am trying to solve a set of stiff ODEs explicitly. >>> >>> If your ODEs are stiff, why are you trying to use an explicit method? >>> (A common definition of "stiff" is that explicit methods do not work >>> well.) >>> >>> > They are very stiff and was hoping to use TSRK with variable time >>> > steps, but it is throwing a segmentation fault >>> >>> Always show the full error message. Please also use 'master' from >>> petsc-dev, because TSRK was overhauled recently to be consistent with >>> the other integrators (and adding a bunch of new functionality). >>> >>> https://bitbucket.org/petsc/petsc/wiki/ >>> >>> > otherwise non existent with Euler or TSSSP. 1. should i look into the >>> > TSRK error or is it something with the new version of Petsc? 2. i >>> > read somewhere (but cant find the source again) that TSSSP is not >>> > recommended for stiff equations (even with the rk solvers set as the >>> > type). >>> >>> Yes, TSSSP is for non-stiff problems that require "strong stability" >>> (like controlling total variation or positivity in transport-dominated >>> applications). >>> >>> > Is that correct? if so, what would be most optimized to solve stiff >>> > odes explicitly. >>> >>> Stiff ODEs should be solved *implicitly*. >>> >>> > 3. If it is not correct and TSSSP is useful for stiff equations, how >>> > can i run it with variable time steps, since with TSStep it is taking >>> > equal time steps which is the time step that i am specifying as an >>> > initial time step TSSetInitialTimeStep. >>> >>> TSSSP does not have an adaptive controller. A user can set a post-check >>> function that adapts the time step based on a custom stability function >>> (e.g., CFL criteria). >>> >>> TSROSW and TSARKIMEX are IMEX methods that can handle stiff problems. >>> They have adaptive error estimators and the like. >>> >> >> > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: make.log Type: text/x-log Size: 50896 bytes Desc: not available URL: From knepley at gmail.com Thu Dec 5 07:35:50 2013 From: knepley at gmail.com (Matthew Knepley) Date: Thu, 5 Dec 2013 07:35:50 -0600 Subject: [petsc-users] Stiff ODEs In-Reply-To: References: <87mwkh26yd.fsf@jedbrown.org> Message-ID: On Thu, Dec 5, 2013 at 7:21 AM, Katy Ghantous wrote: > > On Thu, Dec 5, 2013 at 12:40 PM, Matthew Knepley wrote: > >> On Thu, Dec 5, 2013 at 5:25 AM, Katy Ghantous wrote: >> >>> thank you! >>> i used master from petsc-dev on mac and have TSRK working now. >>> >>> i am using an explicit scheme since its a set of coupled nonlinear >>> equations: >>> >>> d?n/d t = an ?n-2?n-1 - bn ?n-1?n+1 + cn ?n+1?n+2 >>> >>> where n can go up to 24 or so.. I didnt think it is possible to solve it >>> implicitly. and TSRK works for it now. >>> >> >> It is. >> >> >>> But i need petsc on linux too, and testing to verify installation (same >>> options on configure as mac expect arch) gave an error: >>> >>> >>> /home/katy/petsc/linux-gnu-cxx-debug/bin/mpicxx -o ex19.o -c -Wall >>> -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g -O0 -fPIC >>> -I/home/katy/petsc/include >>> -I/home/katy/petsc/linux-gnu-cxx-debug/include `pwd`/ex19.c >>> /home/katy/petsc/linux-gnu-cxx-debug/bin/mpicxx -Wall -Wwrite-strings >>> -Wno-strict-aliasing -Wno-unknown-pragmas -g -O0 -o ex19 ex19.o >>> -Wl,-rpath,/home/katy/petsc/linux-gnu-cxx-debug/lib >>> -L/home/katy/petsc/linux-gnu-cxx-debug/lib -lpetsc >>> -Wl,-rpath,/home/katy/petsc/linux-gnu-cxx-debug/lib -lf2clapack -lf2cblas >>> -lm -lpthread -lm -Wl,-rpath,/usr/lib/gcc/x86_64-linux-gnu/4.6 >>> -L/usr/lib/gcc/x86_64-linux-gnu/4.6 -Wl,-rpath,/usr/lib/x86_64-linux-gnu >>> -L/usr/lib/x86_64-linux-gnu -Wl,-rpath,/lib/x86_64-linux-gnu >>> -L/lib/x86_64-linux-gnu -lmpichcxx -lstdc++ -ldl -lmpich -lopa -lmpl -lrt >>> -lpthread -lgcc_s -ldl >>> /home/katy/petsc/linux-gnu-cxx-debug/lib/libpetsc.so: undefined >>> reference to `SNESDefaultMatrixFreeCreate2(_p_SNES*, _p_Vec*, _p_Mat**)' >>> /home/katy/petsc/linux-gnu-cxx-debug/lib/libpetsc.so: undefined >>> reference to `KSPCreate_DGMRES' >>> /home/katy/petsc/linux-gnu-cxx-debug/lib/libpetsc.so: undefined >>> reference to `PCCreate_TFS' >>> >> >> Send make.log. >> > It looks like you have corrupt source. Did you alter the source in any way. I assume you got it from git, so you can do 'git checkout master'. If you think it should be correct, send src/ksp/ksp/impls/gmres/makefile and src/ksp/ksp/impls/gmres/dgmres/makefile. Matt > Matt >> >> >>> On Tue, Dec 3, 2013 at 6:13 PM, Jed Brown wrote: >>> >>>> Katy Ghantous writes: >>>> >>>> > Hi, >>>> > I have just started with Petsc and have very elementary equations. >>>> > I am trying to solve a set of stiff ODEs explicitly. >>>> >>>> If your ODEs are stiff, why are you trying to use an explicit method? >>>> (A common definition of "stiff" is that explicit methods do not work >>>> well.) >>>> >>>> > They are very stiff and was hoping to use TSRK with variable time >>>> > steps, but it is throwing a segmentation fault >>>> >>>> Always show the full error message. Please also use 'master' from >>>> petsc-dev, because TSRK was overhauled recently to be consistent with >>>> the other integrators (and adding a bunch of new functionality). >>>> >>>> https://bitbucket.org/petsc/petsc/wiki/ >>>> >>>> > otherwise non existent with Euler or TSSSP. 1. should i look into the >>>> > TSRK error or is it something with the new version of Petsc? 2. i >>>> > read somewhere (but cant find the source again) that TSSSP is not >>>> > recommended for stiff equations (even with the rk solvers set as the >>>> > type). >>>> >>>> Yes, TSSSP is for non-stiff problems that require "strong stability" >>>> (like controlling total variation or positivity in transport-dominated >>>> applications). >>>> >>>> > Is that correct? if so, what would be most optimized to solve stiff >>>> > odes explicitly. >>>> >>>> Stiff ODEs should be solved *implicitly*. >>>> >>>> > 3. If it is not correct and TSSSP is useful for stiff equations, how >>>> > can i run it with variable time steps, since with TSStep it is taking >>>> > equal time steps which is the time step that i am specifying as an >>>> > initial time step TSSetInitialTimeStep. >>>> >>>> TSSSP does not have an adaptive controller. A user can set a post-check >>>> function that adapts the time step based on a custom stability function >>>> (e.g., CFL criteria). >>>> >>>> TSROSW and TSARKIMEX are IMEX methods that can handle stiff problems. >>>> They have adaptive error estimators and the like. >>>> >>> >>> >> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which their >> experiments lead. >> -- Norbert Wiener >> > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From katyghantous at gmail.com Thu Dec 5 07:38:08 2013 From: katyghantous at gmail.com (Katy Ghantous) Date: Thu, 5 Dec 2013 14:38:08 +0100 Subject: [petsc-users] Stiff ODEs In-Reply-To: References: <87mwkh26yd.fsf@jedbrown.org> Message-ID: it is already on 'master' upon doing git checkout master. On Thu, Dec 5, 2013 at 2:35 PM, Matthew Knepley wrote: > On Thu, Dec 5, 2013 at 7:21 AM, Katy Ghantous wrote: >> >> On Thu, Dec 5, 2013 at 12:40 PM, Matthew Knepley wrote: >> >>> On Thu, Dec 5, 2013 at 5:25 AM, Katy Ghantous wrote: >>> >>>> thank you! >>>> i used master from petsc-dev on mac and have TSRK working now. >>>> >>>> i am using an explicit scheme since its a set of coupled nonlinear >>>> equations: >>>> >>>> d?n/d t = an ?n-2?n-1 - bn ?n-1?n+1 + cn ?n+1?n+2 >>>> >>>> where n can go up to 24 or so.. I didnt think it is possible to solve >>>> it implicitly. and TSRK works for it now. >>>> >>> >>> It is. >>> >>> >>>> But i need petsc on linux too, and testing to verify installation (same >>>> options on configure as mac expect arch) gave an error: >>>> >>>> >>>> /home/katy/petsc/linux-gnu-cxx-debug/bin/mpicxx -o ex19.o -c -Wall >>>> -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g -O0 -fPIC >>>> -I/home/katy/petsc/include >>>> -I/home/katy/petsc/linux-gnu-cxx-debug/include `pwd`/ex19.c >>>> /home/katy/petsc/linux-gnu-cxx-debug/bin/mpicxx -Wall -Wwrite-strings >>>> -Wno-strict-aliasing -Wno-unknown-pragmas -g -O0 -o ex19 ex19.o >>>> -Wl,-rpath,/home/katy/petsc/linux-gnu-cxx-debug/lib >>>> -L/home/katy/petsc/linux-gnu-cxx-debug/lib -lpetsc >>>> -Wl,-rpath,/home/katy/petsc/linux-gnu-cxx-debug/lib -lf2clapack -lf2cblas >>>> -lm -lpthread -lm -Wl,-rpath,/usr/lib/gcc/x86_64-linux-gnu/4.6 >>>> -L/usr/lib/gcc/x86_64-linux-gnu/4.6 -Wl,-rpath,/usr/lib/x86_64-linux-gnu >>>> -L/usr/lib/x86_64-linux-gnu -Wl,-rpath,/lib/x86_64-linux-gnu >>>> -L/lib/x86_64-linux-gnu -lmpichcxx -lstdc++ -ldl -lmpich -lopa -lmpl -lrt >>>> -lpthread -lgcc_s -ldl >>>> /home/katy/petsc/linux-gnu-cxx-debug/lib/libpetsc.so: undefined >>>> reference to `SNESDefaultMatrixFreeCreate2(_p_SNES*, _p_Vec*, _p_Mat**)' >>>> /home/katy/petsc/linux-gnu-cxx-debug/lib/libpetsc.so: undefined >>>> reference to `KSPCreate_DGMRES' >>>> /home/katy/petsc/linux-gnu-cxx-debug/lib/libpetsc.so: undefined >>>> reference to `PCCreate_TFS' >>>> >>> >>> Send make.log. >>> >> > It looks like you have corrupt source. Did you alter the source in any > way. I assume you got it from > git, so you can do 'git checkout master'. > > If you think it should be correct, send src/ksp/ksp/impls/gmres/makefile > and src/ksp/ksp/impls/gmres/dgmres/makefile. > > Matt > > >> Matt >>> >>> >>>> On Tue, Dec 3, 2013 at 6:13 PM, Jed Brown wrote: >>>> >>>>> Katy Ghantous writes: >>>>> >>>>> > Hi, >>>>> > I have just started with Petsc and have very elementary equations. >>>>> > I am trying to solve a set of stiff ODEs explicitly. >>>>> >>>>> If your ODEs are stiff, why are you trying to use an explicit method? >>>>> (A common definition of "stiff" is that explicit methods do not work >>>>> well.) >>>>> >>>>> > They are very stiff and was hoping to use TSRK with variable time >>>>> > steps, but it is throwing a segmentation fault >>>>> >>>>> Always show the full error message. Please also use 'master' from >>>>> petsc-dev, because TSRK was overhauled recently to be consistent with >>>>> the other integrators (and adding a bunch of new functionality). >>>>> >>>>> https://bitbucket.org/petsc/petsc/wiki/ >>>>> >>>>> > otherwise non existent with Euler or TSSSP. 1. should i look into >>>>> the >>>>> > TSRK error or is it something with the new version of Petsc? 2. i >>>>> > read somewhere (but cant find the source again) that TSSSP is not >>>>> > recommended for stiff equations (even with the rk solvers set as the >>>>> > type). >>>>> >>>>> Yes, TSSSP is for non-stiff problems that require "strong stability" >>>>> (like controlling total variation or positivity in transport-dominated >>>>> applications). >>>>> >>>>> > Is that correct? if so, what would be most optimized to solve stiff >>>>> > odes explicitly. >>>>> >>>>> Stiff ODEs should be solved *implicitly*. >>>>> >>>>> > 3. If it is not correct and TSSSP is useful for stiff equations, how >>>>> > can i run it with variable time steps, since with TSStep it is taking >>>>> > equal time steps which is the time step that i am specifying as an >>>>> > initial time step TSSetInitialTimeStep. >>>>> >>>>> TSSSP does not have an adaptive controller. A user can set a >>>>> post-check >>>>> function that adapts the time step based on a custom stability function >>>>> (e.g., CFL criteria). >>>>> >>>>> TSROSW and TSARKIMEX are IMEX methods that can handle stiff problems. >>>>> They have adaptive error estimators and the like. >>>>> >>>> >>>> >>> >>> >>> -- >>> What most experimenters take for granted before they begin their >>> experiments is infinitely more interesting than any results to which their >>> experiments lead. >>> -- Norbert Wiener >>> >> >> > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: makefile Type: application/octet-stream Size: 345 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: makefile Type: application/octet-stream Size: 300 bytes Desc: not available URL: From knepley at gmail.com Thu Dec 5 07:50:13 2013 From: knepley at gmail.com (Matthew Knepley) Date: Thu, 5 Dec 2013 07:50:13 -0600 Subject: [petsc-users] Stiff ODEs In-Reply-To: References: <87mwkh26yd.fsf@jedbrown.org> Message-ID: On Thu, Dec 5, 2013 at 7:38 AM, Katy Ghantous wrote: > it is already on 'master' upon doing git checkout master. > This makes no sense. The makefiles appear correct, and the gmres makefile has the dgmres directory, but no record of it building in that directory is in the makefile. Jed, how could #requiresscalar be failing? Matt > On Thu, Dec 5, 2013 at 2:35 PM, Matthew Knepley wrote: > >> On Thu, Dec 5, 2013 at 7:21 AM, Katy Ghantous wrote: >>> >>> On Thu, Dec 5, 2013 at 12:40 PM, Matthew Knepley wrote: >>> >>>> On Thu, Dec 5, 2013 at 5:25 AM, Katy Ghantous wrote: >>>> >>>>> thank you! >>>>> i used master from petsc-dev on mac and have TSRK working now. >>>>> >>>>> i am using an explicit scheme since its a set of coupled nonlinear >>>>> equations: >>>>> >>>>> d?n/d t = an ?n-2?n-1 - bn ?n-1?n+1 + cn ?n+1?n+2 >>>>> >>>>> where n can go up to 24 or so.. I didnt think it is possible to solve >>>>> it implicitly. and TSRK works for it now. >>>>> >>>> >>>> It is. >>>> >>>> >>>>> But i need petsc on linux too, and testing to verify installation >>>>> (same options on configure as mac expect arch) gave an error: >>>>> >>>>> >>>>> /home/katy/petsc/linux-gnu-cxx-debug/bin/mpicxx -o ex19.o -c -Wall >>>>> -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g -O0 -fPIC >>>>> -I/home/katy/petsc/include >>>>> -I/home/katy/petsc/linux-gnu-cxx-debug/include `pwd`/ex19.c >>>>> /home/katy/petsc/linux-gnu-cxx-debug/bin/mpicxx -Wall -Wwrite-strings >>>>> -Wno-strict-aliasing -Wno-unknown-pragmas -g -O0 -o ex19 ex19.o >>>>> -Wl,-rpath,/home/katy/petsc/linux-gnu-cxx-debug/lib >>>>> -L/home/katy/petsc/linux-gnu-cxx-debug/lib -lpetsc >>>>> -Wl,-rpath,/home/katy/petsc/linux-gnu-cxx-debug/lib -lf2clapack -lf2cblas >>>>> -lm -lpthread -lm -Wl,-rpath,/usr/lib/gcc/x86_64-linux-gnu/4.6 >>>>> -L/usr/lib/gcc/x86_64-linux-gnu/4.6 -Wl,-rpath,/usr/lib/x86_64-linux-gnu >>>>> -L/usr/lib/x86_64-linux-gnu -Wl,-rpath,/lib/x86_64-linux-gnu >>>>> -L/lib/x86_64-linux-gnu -lmpichcxx -lstdc++ -ldl -lmpich -lopa -lmpl -lrt >>>>> -lpthread -lgcc_s -ldl >>>>> /home/katy/petsc/linux-gnu-cxx-debug/lib/libpetsc.so: undefined >>>>> reference to `SNESDefaultMatrixFreeCreate2(_p_SNES*, _p_Vec*, _p_Mat**)' >>>>> /home/katy/petsc/linux-gnu-cxx-debug/lib/libpetsc.so: undefined >>>>> reference to `KSPCreate_DGMRES' >>>>> /home/katy/petsc/linux-gnu-cxx-debug/lib/libpetsc.so: undefined >>>>> reference to `PCCreate_TFS' >>>>> >>>> >>>> Send make.log. >>>> >>> >> It looks like you have corrupt source. Did you alter the source in any >> way. I assume you got it from >> git, so you can do 'git checkout master'. >> >> If you think it should be correct, send src/ksp/ksp/impls/gmres/makefile >> and src/ksp/ksp/impls/gmres/dgmres/makefile. >> >> Matt >> >> >>> Matt >>>> >>>> >>>>> On Tue, Dec 3, 2013 at 6:13 PM, Jed Brown wrote: >>>>> >>>>>> Katy Ghantous writes: >>>>>> >>>>>> > Hi, >>>>>> > I have just started with Petsc and have very elementary equations. >>>>>> > I am trying to solve a set of stiff ODEs explicitly. >>>>>> >>>>>> If your ODEs are stiff, why are you trying to use an explicit method? >>>>>> (A common definition of "stiff" is that explicit methods do not work >>>>>> well.) >>>>>> >>>>>> > They are very stiff and was hoping to use TSRK with variable time >>>>>> > steps, but it is throwing a segmentation fault >>>>>> >>>>>> Always show the full error message. Please also use 'master' from >>>>>> petsc-dev, because TSRK was overhauled recently to be consistent with >>>>>> the other integrators (and adding a bunch of new functionality). >>>>>> >>>>>> https://bitbucket.org/petsc/petsc/wiki/ >>>>>> >>>>>> > otherwise non existent with Euler or TSSSP. 1. should i look into >>>>>> the >>>>>> > TSRK error or is it something with the new version of Petsc? 2. i >>>>>> > read somewhere (but cant find the source again) that TSSSP is not >>>>>> > recommended for stiff equations (even with the rk solvers set as the >>>>>> > type). >>>>>> >>>>>> Yes, TSSSP is for non-stiff problems that require "strong stability" >>>>>> (like controlling total variation or positivity in transport-dominated >>>>>> applications). >>>>>> >>>>>> > Is that correct? if so, what would be most optimized to solve stiff >>>>>> > odes explicitly. >>>>>> >>>>>> Stiff ODEs should be solved *implicitly*. >>>>>> >>>>>> > 3. If it is not correct and TSSSP is useful for stiff equations, how >>>>>> > can i run it with variable time steps, since with TSStep it is >>>>>> taking >>>>>> > equal time steps which is the time step that i am specifying as an >>>>>> > initial time step TSSetInitialTimeStep. >>>>>> >>>>>> TSSSP does not have an adaptive controller. A user can set a >>>>>> post-check >>>>>> function that adapts the time step based on a custom stability >>>>>> function >>>>>> (e.g., CFL criteria). >>>>>> >>>>>> TSROSW and TSARKIMEX are IMEX methods that can handle stiff problems. >>>>>> They have adaptive error estimators and the like. >>>>>> >>>>> >>>>> >>>> >>>> >>>> -- >>>> What most experimenters take for granted before they begin their >>>> experiments is infinitely more interesting than any results to which their >>>> experiments lead. >>>> -- Norbert Wiener >>>> >>> >>> >> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which their >> experiments lead. >> -- Norbert Wiener >> > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From nico.schloemer at gmail.com Thu Dec 5 09:18:37 2013 From: nico.schloemer at gmail.com (=?ISO-8859-1?Q?Nico_Schl=F6mer?=) Date: Thu, 5 Dec 2013 16:18:37 +0100 Subject: [petsc-users] fieldsplit: Unhandled case, must have at least two fields, not 1! In-Reply-To: <87siub6xzt.fsf@jedbrown.org> References: <87d2liblo7.fsf@jedbrown.org> <87haata2io.fsf@jedbrown.org> <87zjok8pha.fsf@jedbrown.org> <87r49w8d1x.fsf@jedbrown.org> <874n6r8i33.fsf@jedbrown.org> <871u1v8gbf.fsf@jedbrown.org> <87siub6xzt.fsf@jedbrown.org> Message-ID: For the record, the problem I had was caused by the fact that I * first created the solver, * then set PetscOptionsSetValue('pc_type', 'fieldsplit') Reverting the order of the two calls gives the expected results. Cheers, Nico e0319bd95940fc10e07f4fba29a91a8713afd825 On Mon, Dec 2, 2013 at 5:03 PM, Jed Brown wrote: > Mark Adams writes: >> My point was that this user got this message and so I suggesting a possible >> problem: giving fieldsplit parameters to a single field matrix/solver. > > Okay, I interpreted your original statement as being that this error > message has something to do with using monolithic matrix formats (such > as AIJ, versus Nest), which it does not, and that you thought the error > message was problematic behavior. > > Your very complicated test case (hundreds of processes, with multiple > solves, some intending to use fieldsplit, Schur, MatNest, tinkering with > the options database, huge differences between the two options files) > can be reduced to src/ksp/ksp/examples/tutorials/ex2.c. > > $ ./ex2 -pc_type fieldsplit > [0]PETSC ERROR: --------------------- Error Message ------------------------------------ > [0]PETSC ERROR: Petsc has generated inconsistent data! > [0]PETSC ERROR: Unhandled case, must have at least two fields, not 1! > [0]PETSC ERROR: ------------------------------------------------------------------------ > [0]PETSC ERROR: Petsc Development GIT revision: ad348534bf2da17dbc29a6a7895b244965ac0e39 GIT Date: 2013-11-13 12:43:46 -0700 > [0]PETSC ERROR: See docs/changes/index.html for recent updates. > [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. > [0]PETSC ERROR: See docs/index.html for manual pages. > [0]PETSC ERROR: ------------------------------------------------------------------------ > [0]PETSC ERROR: ./ex2 on a mpich named batura by jed Mon Dec 2 09:55:54 2013 > [0]PETSC ERROR: Libraries linked from /home/jed/petsc/mpich/lib > [0]PETSC ERROR: Configure run at Wed Nov 13 13:33:49 2013 > [0]PETSC ERROR: Configure options --download-blacs --download-chaco --download-ctetgen --download-elemental --download-generator --download-hypre --download-metis --download-ml --download-mumps --download-parmetis --download-scalapack --download-spai --download-spooles --download-sundials --download-superlu --download-superlu_dist --download-triangle --with-blas-lapack=/usr --with-c2html --with-cholmod-dir=/usr --with-cxx-dialect=C++11 --with-exodusii-dir=/usr --with-hdf5-dir=/opt/mpich --with-lgrind --with-mpi-dir=/home/jed/usr/ccache/mpich/ --with-netcdf-dir=/usr --with-openmp --with-pcbddc --with-pthread --with-shared-libraries --with-single-library=0 --with-sowing --with-threadcomm --with-umfpack-dir=/usr --with-x -PETSC_ARCH=mpich COPTFLAGS="-Og -g" CXXOPTFLAGS="-Og -g" FOPTFLAGS="-Og -g" > [0]PETSC ERROR: ------------------------------------------------------------------------ > [0]PETSC ERROR: PCFieldSplitSetDefaults() line 441 in src/ksp/pc/impls/fieldsplit/fieldsplit.c > [0]PETSC ERROR: PCSetUp_FieldSplit() line 459 in src/ksp/pc/impls/fieldsplit/fieldsplit.c > [0]PETSC ERROR: PCSetUp() line 888 in src/ksp/pc/interface/precon.c > [0]PETSC ERROR: KSPSetUp() line 278 in src/ksp/ksp/interface/itfunc.c > [0]PETSC ERROR: KSPSolve() line 399 in src/ksp/ksp/interface/itfunc.c > [0]PETSC ERROR: main() line 198 in /home/jed/petsc/src/ksp/ksp/examples/tutorials/ex2.c > application called MPI_Abort(MPI_COMM_WORLD, 77) - process 0 From jedbrown at mcs.anl.gov Thu Dec 5 09:48:56 2013 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Thu, 05 Dec 2013 08:48:56 -0700 Subject: [petsc-users] fieldsplit: Unhandled case, must have at least two fields, not 1! In-Reply-To: References: <87d2liblo7.fsf@jedbrown.org> <87haata2io.fsf@jedbrown.org> <87zjok8pha.fsf@jedbrown.org> <87r49w8d1x.fsf@jedbrown.org> <874n6r8i33.fsf@jedbrown.org> <871u1v8gbf.fsf@jedbrown.org> <87siub6xzt.fsf@jedbrown.org> Message-ID: <87wqjjl2mv.fsf@jedbrown.org> Nico Schl?mer writes: > For the record, the problem I had was caused by the fact that I > * first created the solver, > * then set > PetscOptionsSetValue('pc_type', 'fieldsplit') Please use PCSetType(pc,PCFIELDSPLIT) if you are setting this up from code, rather than tinkering with the options database. Also, we strongly recommend calling KSPSetFromOptions() *after* any manual settings, so that the solver can be modified at run-time. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From jedbrown at mcs.anl.gov Thu Dec 5 10:03:27 2013 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Thu, 05 Dec 2013 09:03:27 -0700 Subject: [petsc-users] Stiff ODEs In-Reply-To: References: <87mwkh26yd.fsf@jedbrown.org> Message-ID: <87txenl1yo.fsf@jedbrown.org> Matthew Knepley writes: > This makes no sense. The makefiles appear correct, and the gmres makefile > has the dgmres > directory, but no record of it building in that directory is in the > makefile. > > Jed, how could #requiresscalar be failing? Katy, please send linux-gnu-cxx-debug/conf/petscvariables. It should contain the line PETSC_SCALAR = real Could you try reconfiguring, and if the problem persists, send configure.log? -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From christophe.ortiz at ciemat.es Thu Dec 5 10:19:34 2013 From: christophe.ortiz at ciemat.es (Christophe Ortiz) Date: Thu, 5 Dec 2013 17:19:34 +0100 Subject: [petsc-users] Problem compiling with g++. Uses gcc instead of g++ Message-ID: Hi all, Since I would like to use classes, I am moving my C code to C++. I compiled PETSc with g++ using the following options for the configure: --with-mpi=0 --with-fc=ifort --with-cxx=g++ --with-debugging=1 --with-scalar-type=real --with-precision=double --download-f-blas-lapack It did not complain, everything went smooth. Then to compile I use: diffusion_c: diffusion.o chkopts -${CLINKER} -o diffusion_c diffusion.o ${PETSC_TS_LIB} ${RM} diffusion.o I declared a simple class at the beginning of the file and it complains. When I compile I get: gcc -o diffusion.o -c -fPIC -Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g3 -fno-inline -O0 -I/home/u5751/petsc-3.4.1/include -I/home/u5751/petsc-3.4.1/ifort-gpp-nompi-double-blas-debug/include -I/home/u5751/petsc-3.4.1/include/mpiuni -D__INSDIR__=src/ts/examples/tutorials/ diffusion.c diffusion.c:49: error: expected ?=?, ?,?, ?;?, ?asm? or ?__attribute__? before ?defect? .... .... Obviously, it is because it uses gcc instead of g++ to compile, so it does not recognize class. Is it correct to use CLINKER ? Is there another one for g++ ? Did I miss something ? Many thanks in advance for your help ! Christophe -------------- next part -------------- An HTML attachment was scrubbed... URL: From jedbrown at mcs.anl.gov Thu Dec 5 10:23:14 2013 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Thu, 05 Dec 2013 09:23:14 -0700 Subject: [petsc-users] Problem compiling with g++. Uses gcc instead of g++ In-Reply-To: References: Message-ID: <87eh5rl11p.fsf@jedbrown.org> Christophe Ortiz writes: > Hi all, > > Since I would like to use classes, I am moving my C code to C++. I compiled > PETSc with g++ using the following options for the configure: > > --with-mpi=0 --with-fc=ifort --with-cxx=g++ --with-debugging=1 > --with-scalar-type=real --with-precision=double --download-f-blas-lapack > > It did not complain, everything went smooth. > > Then to compile I use: > > diffusion_c: diffusion.o chkopts > -${CLINKER} -o diffusion_c diffusion.o ${PETSC_TS_LIB} > ${RM} diffusion.o > > > I declared a simple class at the beginning of the file and it complains. > When I compile I get: > > gcc -o diffusion.o -c -fPIC -Wall -Wwrite-strings -Wno-strict-aliasing > -Wno-unknown-pragmas -g3 -fno-inline -O0 > -I/home/u5751/petsc-3.4.1/include > -I/home/u5751/petsc-3.4.1/ifort-gpp-nompi-double-blas-debug/include > -I/home/u5751/petsc-3.4.1/include/mpiuni > -D__INSDIR__=src/ts/examples/tutorials/ diffusion.c > > diffusion.c:49: error: expected ?=?, ?,?, ?;?, ?asm? or ?__attribute__? > before ?defect? Name your source file diffusion.cxx -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From katyghantous at gmail.com Thu Dec 5 11:41:03 2013 From: katyghantous at gmail.com (Katy Ghantous) Date: Thu, 5 Dec 2013 18:41:03 +0100 Subject: [petsc-users] Stiff ODEs In-Reply-To: <87txenl1yo.fsf@jedbrown.org> References: <87mwkh26yd.fsf@jedbrown.org> <87txenl1yo.fsf@jedbrown.org> Message-ID: thanks! yep, it works.. but only after i removed the folder and cloned all over again before configure and make. it might have messed up because i had to install opengl after installing petsc.. and so had to do the configuration again to take it into account.. (but i did that the second time around without specifying petsc scalar, while i specified it as complex before..) but that is just a speculation. thanks for the help! On Thu, Dec 5, 2013 at 5:03 PM, Jed Brown wrote: > Matthew Knepley writes: > > This makes no sense. The makefiles appear correct, and the gmres makefile > > has the dgmres > > directory, but no record of it building in that directory is in the > > makefile. > > > > Jed, how could #requiresscalar be failing? > > Katy, please send linux-gnu-cxx-debug/conf/petscvariables. It should > contain the line > > PETSC_SCALAR = real > > Could you try reconfiguring, and if the problem persists, send > configure.log? > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lu_qin_2000 at yahoo.com Fri Dec 6 13:51:38 2013 From: lu_qin_2000 at yahoo.com (Qin Lu) Date: Fri, 6 Dec 2013 11:51:38 -0800 (PST) Subject: [petsc-users] CPU time of PETSc Message-ID: <1386359498.13687.YahooMailNeo@web160206.mail.bf1.yahoo.com> Hello, ? I am using serial PETSc linear solver in a serial Fortran program, which uses function CPU_TIME() to count cpu time. In Windows-7, the?cpu time turns out to be 2 to 3 times as much as elapse time. My questions are: ? 1. Does PETSc internally use multi-thread even though I configured it with --useThreads=0? I ask so since it is known that CPU_TIME()? adds the cpu time of all threads togather. 2. The same program gets consistent cpu time and elapse time in Linux. Do you have any idea why? ? Thanks for any suggestions. Qin -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Fri Dec 6 13:55:22 2013 From: knepley at gmail.com (Matthew Knepley) Date: Fri, 6 Dec 2013 13:55:22 -0600 Subject: [petsc-users] CPU time of PETSc In-Reply-To: <1386359498.13687.YahooMailNeo@web160206.mail.bf1.yahoo.com> References: <1386359498.13687.YahooMailNeo@web160206.mail.bf1.yahoo.com> Message-ID: On Fri, Dec 6, 2013 at 1:51 PM, Qin Lu wrote: > Hello, > > I am using serial PETSc linear solver in a serial Fortran program, which > uses function CPU_TIME() to count cpu time. In Windows-7, the cpu time > turns out to be 2 to 3 times as much as elapse time. My questions are: > > 1. Does PETSc internally use multi-thread even though I configured it with > --useThreads=0? I ask so since it is known that CPU_TIME() adds the cpu > time of all threads togather. > No. > 2. The same program gets consistent cpu time and elapse time in Linux. Do > you have any idea why? > I am guessing your normalization is wrong. Why not just use -log_summary? We test for these things. Matt > Thanks for any suggestions. > Qin > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From mlohry at gmail.com Sat Dec 7 13:26:25 2013 From: mlohry at gmail.com (Mark Lohry) Date: Sat, 7 Dec 2013 14:26:25 -0500 Subject: [petsc-users] Handling non-solution local vector data Message-ID: 1) Is there somewhere on the doc site with a single list of all examples and what they do? As a new user I hugely appreciate all the great examples and documentation, but it's a little unwieldy to try and find relevant examples by navigating the function documents. 2) I'm a little unclear as to the best-practice way to handle solution/grid related data that isn't actual solution DOFS or mesh points, and doesn't require global communication. Say I want to work on an (ni X nj ) cell-centered finite volume layout with one DOF per cell. I start with DMDACreate2d(PETSC_COMM_WORLD, DMDA_BOUNDARY_PERIODIC, DMDA_BOUNDARY_NONE, /* periodic in i, no conn in j */ DMDA_STENCIL_BOX, /* 5-point stencil */ ni,nj, /* number of cells */ PETSC_DECIDE,PETSC_DECIDE, /* let petsc determine number of procs */ 1, /* one degree of freedom per cell */ 1, /* stencil width */ NULL,NULL,&da); Now I can construct the solution vector like so: struct Field { PetscScalar u; }; Field **soln; Vec global_vec; DMDACreateGlobalVector(da,&global_vec); DMDAVecGetArray(da,global_vec,&soln); // converts global vector into Field** and access the local elements through soln[j][i]. But say I want to store some grid information locally, say the FaceNormal for a cell face: struct FaceNormal{ PetscScalar nx,ny; }; FaceNormal **IFace; Now it seems that the CreateGlobalVector/CreateLocalVector are just going to allocate for 1 DOF because of how the DMDA was defined, so there wouldn't be sufficient storage for the 2 variables in FaceNormal. What's an appropriate way to do this? Clearly I could just malloc that myself, but it'd be nice to keep the same indexing as the petsc vectors. Thanks in advance, -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Sat Dec 7 13:34:20 2013 From: knepley at gmail.com (Matthew Knepley) Date: Sat, 7 Dec 2013 13:34:20 -0600 Subject: [petsc-users] Handling non-solution local vector data In-Reply-To: References: Message-ID: On Sat, Dec 7, 2013 at 1:26 PM, Mark Lohry wrote: > 1) Is there somewhere on the doc site with a single list of all examples > and what they do? As a new user I hugely appreciate all the great examples > and documentation, but it's a little unwieldy to try and find relevant > examples by navigating the function documents. > No, we have not yet setup a nice system for this. > 2) I'm a little unclear as to the best-practice way to handle > solution/grid related data that isn't actual solution DOFS or mesh points, > and doesn't require global communication. > > Say I want to work on an (ni X nj ) cell-centered finite volume layout > with one DOF per cell. I start with > > DMDACreate2d(PETSC_COMM_WORLD, > DMDA_BOUNDARY_PERIODIC, DMDA_BOUNDARY_NONE, /* periodic in i, no > conn in j */ > DMDA_STENCIL_BOX, /* 5-point stencil */ > ni,nj, /* number of cells */ > PETSC_DECIDE,PETSC_DECIDE, /* let petsc determine number of procs */ > 1, /* one degree of freedom per cell */ > 1, /* stencil width */ > NULL,NULL,&da); > > Now I can construct the solution vector like so: > > struct Field { PetscScalar u; }; > Field **soln; > Vec global_vec; > DMDACreateGlobalVector(da,&global_vec); > DMDAVecGetArray(da,global_vec,&soln); // converts global vector into > Field** > > and access the local elements through soln[j][i]. But say I want to store > some grid information locally, say the FaceNormal for a cell face: > struct FaceNormal{ PetscScalar nx,ny; }; > FaceNormal **IFace; > > Now it seems that the CreateGlobalVector/CreateLocalVector are just going > to allocate for 1 DOF because of how the DMDA was defined, so there > wouldn't be sufficient storage for the 2 variables in FaceNormal. What's an > appropriate way to do this? Clearly I could just malloc that myself, but > it'd be nice to keep the same indexing as the petsc vectors. > There are at least two answers: 1) If you do not care about redundant storage, you could create another 2d DMDA using dof = 8, and store all the normals for each cell. DMDAs take almost no memory. Your storage would be the Vec you use to store the normals. 2) If you care about storage, then create another DMDA, but then create a PetscSection to assign 2 dof to every face, which you can do with DMDACreateSection() and use DMSetDefaultSection(). This will make DMCreateGlobalVector() create a Vec you can use to store all the face normals. Matt > Thanks in advance, > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From ross.c.brodie at ga.gov.au Mon Dec 9 18:14:23 2013 From: ross.c.brodie at ga.gov.au (ross.c.brodie at ga.gov.au) Date: Tue, 10 Dec 2013 11:14:23 +1100 Subject: [petsc-users] CG on Mixed dense sparse shell matrix [SEC=UNCLASSIFIED] Message-ID: <1F63AB19349C4F4DBC5A9435E8FF98B28580AF33B2@EXCCR01.agso.gov.au> Hello PETSc Users I have previously successfully tackled the problem of solving (J'DJ + aL'DL + cK'EK) x = b where real J, L, K were sparse matrices and D and E diagonal matrices and a and c real scalars using PETSc by using a shell matrix approach, i.e., never actually explicitly forming J'DJ etc but but supplying the function to KSP (PCG) to compute the products using PETSc functions with a vector on the fly. This did the trick for sparse J. I am pretty sure this can be done with a dense J, in PETSc as well just a lot slower. J could be the order of 10e5 x 10e6. I have considered using libElemental for the dense part (to make use of the grid cyclic layout) and PETSc for the sparse part of the PCG matvec product but I am not sure it is wise to mix packages. I also note that the recent release of PETSc includes an interface to Elemental via the MATELEMENTAL matrix type. Would this be a good choice? On a different note could you clarify the terminology for the sequential vector VECSEQ. Does it mean: (i) It is an independent uniprocessor object such that the data storage is not distributed and if you change an element on one process the change is NOT reflected on other processes, or (ii) does it mean the data storage is not distributed but it is not necessarily uniprocessor independent (ie such that each processor carries around redundant copies of the same data and changes on one process ARE reflected on other processes) depending on whether or not it is constructed using PETSC_COMM_SELF or PETSC_COMM_WORLD. Does it ever make sense to construct a VECSEQ with anything other than PETSC_COMM_SELF? I suspect it is (i) but want to make sure? Ross Geoscience Australia Disclaimer: This e-mail (and files transmitted with it) is intended only for the person or entity to which it is addressed. If you are not the intended recipient, then you have received this e-mail by mistake and any use, dissemination, forwarding, printing or copying of this e-mail and its file attachments is prohibited. The security of emails transmitted cannot be guaranteed; by forwarding or replying to this email, you acknowledge and accept these risks. ------------------------------------------------------------------------------------------------------------------------- From zhenglun.wei at gmail.com Tue Dec 10 00:45:50 2013 From: zhenglun.wei at gmail.com (Alan) Date: Tue, 10 Dec 2013 00:45:50 -0600 Subject: [petsc-users] An issue of order of accuracy. In-Reply-To: References: <529FB9D8.8050708@gmail.com> Message-ID: <52A6B89E.3060307@gmail.com> Thank you, Dr. Smith. I 'cooked up' a 4th-order polynomial and find out that the Poisson solver with uniform Cartesian mesh is 2nd order of accuracy. Also, if the solution is 3rd or less order polynomial, the L2Norm shows very small round-off error, which is usually 1e-7 if the -ksp_rtol is 1-e7. Following this idea, I re-derived the coefficients for the Poisson equation for solver with non-uniform grid, which is attached with a .docx file. From the equation, it indicates that at least 3rd or higher order polynomial should be constructed for the solution to detect the real order of accuracy. In other words, for solution of 2nd or lower order polynomials, the L2Norm should show very small round-off error. However, the reality is not and here is a brief description of my tests. The test focuses on the three-dimensional Poisson solver with non-uniform Cartesian mesh, which is modified from /src/ksp/ksp/example/tutorial/ex45.c. For simplicity, in this test, only x-direction has non-uniform grid, y- and z- are uniform grid. I have 3 sets of tests, which are recorded in TestRun attached. The difference among these 3 sets of tests are the -ksp_rtol. Each set has 3 different commands to run the program with different ksp solver (GMRES or richardson) and pc type (GAMG and Hypre boomeramg). As I recorded in TestRun, while the ksp_rtol is 1e-7, the L2Norm is fairly large. As Dr. Smith explained, tight algebraic tolerances are needed to eliminate the algebraic error. Therefore, I changed ksp_rtol to 10-e15. However, the L2Norm can only reach 1e-7. Compared with the Poisson solver with uniform Cartesian mesh (which the L2Norm exhibits 1e-7 round-off error as -ksp_rtol = 1e-7), the L2Norm from the non-uniform Poisson solver is relatively high. Is this normal? Moreover, the Residual norm shown for cases with ksp_rtol = 1e-15 is around 7, 28 or even 81, which are far away from the real ksp_rtol imported. I monitored the ksp iterations with ksp_monitor and found out that these solvers usually iterate around 20 times. Why wouldn't them continue to iterate until the 1e-15 is achieved? Based on my observation, neither solver/pc works fine for this Poisson solver with non-uniform mesh. Is there any other option to made some improvements? At last, for the Poisson solver with non-uniform Cartesian grid, is there any better way to prove its validity? sincerely appreciate, Alan > Alan, > > I changed your initial grid size to 10 to get faster solve times and get what is below. Note that your ?exact solution? is quadratic, since the method is second order this means that not only is the ?exact solution? an exact solution to the PDE, it is also an exact solution to the algebraic equations (for any grid size) hence the L2Norm of the error is only due to the round off of the algebraic solution, not due to any discretization error. In general, when testing for discretization error you always need to ?cook up? an exact solution that is not completely represented in the approximation space. You also need to use a really tight algebraic tolerance to eliminate the algebraic error from the computation > > > > ~/Src/petsc/test-dir master $ ./ex45 -pc_type mg -ksp_rtol 1e-12 > mx = 10, my = 10, mz =10, mm = 1, nn = 1, pp = 1 > Residual norm 1.81616e-12 > L2Norm = 1.107359e-12 > Total Time Elapsed: 0.048599 > ~/Src/petsc/test-dir master $ ./ex45 -pc_type mg -ksp_rtol 1e-12 -da_refine 1 > mx = 19, my = 19, mz =19, mm = 1, nn = 1, pp = 1 > Residual norm 3.36741e-12 > L2Norm = 1.037148e-12 > Total Time Elapsed: 0.183398 > ~/Src/petsc/test-dir master $ ./ex45 -pc_type mg -ksp_rtol 1e-12 -da_refine 2 > mx = 37, my = 37, mz =37, mm = 1, nn = 1, pp = 1 > Residual norm 1.09476e-11 > L2Norm = 2.330658e-12 > Total Time Elapsed: 1.180839 > ~/Src/petsc/test-dir master $ ./ex45 -pc_type mg -ksp_rtol 1e-12 -da_refine 3 > mx = 73, my = 73, mz =73, mm = 1, nn = 1, pp = 1 > Residual norm 3.19809e-11 > L2Norm = 2.278763e-12 > Total Time Elapsed: 10.819450 > ~/Src/petsc/test-dir master $ ./ex45 -pc_type mg -da_refine 3 > mx = 73, my = 73, mz =73, mm = 1, nn = 1, pp = 1 > Residual norm 0.000103197 > L2Norm = 1.011806e-05 > Total Time Elapsed: 7.250106 > > > On Dec 4, 2013, at 5:25 PM, Alan Z. Wei wrote: > >> Dear all, >> I hope you had a great Thanksgiving. >> Currently, I tested the order of accuracy for /src/ksp/ksp/tutorial/example/ex45.c. Since the 2nd-order discretization is used in this program and ksp solver is converged to 10^-7, I expected that the solution should provides a 2nd-order in L2 norm. However, as I tested (even with a Laplace equation), the L2 norm slope is much less than 2. Sometime, if the grid size is reduced, the L2 norm increases. Could anyone help me about this issue, please? >> >> Here is the L2 norm outputted: >> >> Grid L2 norm (10^-8) >> 0.05 4.36242 >> 0.025 2.20794 >> 0.0125 7.02749 >> 0.00625 12.64 >> Once the grid size is reduced to half, the number of the grid will be multiplied by 8 in order to keep the same size of the computational domain. >> The code is also attached. It is from ex45.c with very little modifications. >> >> thanks in advance, >> Alan >> >> -------------- next part -------------- /* Laplacian in 3D. Modeled by the partial differential equation - Laplacian u = 1,0 < x,y,z < 1, with boundary conditions u = 1 for x = 0, x = 1, y = 0, y = 1, z = 0, z = 1. This uses multigrid to solve the linear system See src/snes/examples/tutorials/ex50.c Can also be run with -pc_type exotic -ksp_type fgmres */ static char help[] = "Solves 3D Laplacian using multigrid.\n\n"; #include #include const double dx = 5.; const double dy = 5.; const double dz = 5.; const int imax = 20; const int jmax = 10; const int kmax = 10; const int seci1 = 5; const int seci2 = 10; const int seci3 = 5; const double rdx1 = 2.; const double rdx3 = 4.; extern PetscErrorCode ComputeMatrix(KSP,Mat,Mat,MatStructure*,void*); extern PetscErrorCode ComputeRHS(KSP,Vec,void*); extern PetscErrorCode ComputeInitialGuess(KSP,Vec,void*); extern PetscErrorCode PostP(DM, Vec, Vec); extern double RealDx(int); extern double RealPx(int); extern void CalTotalLength(); PetscInt i,j,k,mx,my,mz,xm,ym,zm,xs,ys,zs; double TotalLength1_x, TotalLength2_x, TotalLength3_x; #undef __FUNCT__ #define __FUNCT__ "main" int main(int argc,char **argv) { PetscErrorCode ierr; KSP ksp; PetscReal norm; DM da; Vec x,b,r; Mat A; double t1, t2; PetscInitialize(&argc,&argv,(char*)0,help); CalTotalLength(); t1 = MPI_Wtime(); ierr = KSPCreate(PETSC_COMM_WORLD,&ksp);CHKERRQ(ierr); ierr = DMDACreate3d(PETSC_COMM_WORLD,DMDA_BOUNDARY_NONE,DMDA_BOUNDARY_NONE,DMDA_BOUNDARY_NONE,DMDA_STENCIL_STAR,-imax,-jmax,-kmax,PETSC_DECIDE,PETSC_DECIDE,PETSC_DECIDE,1,1,0,0,0,&da);CHKERRQ(ierr); ierr = KSPSetDM(ksp,da);CHKERRQ(ierr); ierr = KSPSetComputeInitialGuess(ksp,ComputeInitialGuess,NULL);CHKERRQ(ierr); ierr = KSPSetComputeRHS(ksp,ComputeRHS,NULL);CHKERRQ(ierr); ierr = KSPSetComputeOperators(ksp,ComputeMatrix,NULL);CHKERRQ(ierr); ierr = DMDestroy(&da);CHKERRQ(ierr); ierr = KSPSetFromOptions(ksp);CHKERRQ(ierr); printf("Check #1!!\n"); ierr = KSPSolve(ksp,NULL,NULL);CHKERRQ(ierr); printf("Check #2!!\n"); ierr = KSPGetSolution(ksp,&x);CHKERRQ(ierr); ierr = KSPGetRhs(ksp,&b);CHKERRQ(ierr); ierr = VecDuplicate(b,&r);CHKERRQ(ierr); ierr = KSPGetOperators(ksp,&A,NULL,NULL);CHKERRQ(ierr); ierr = MatMult(A,x,r);CHKERRQ(ierr); ierr = VecAXPY(r,-1.0,b);CHKERRQ(ierr); ierr = VecNorm(r,NORM_2,&norm);CHKERRQ(ierr); ierr = PetscPrintf(PETSC_COMM_WORLD,"Residual norm %G\n",norm);CHKERRQ(ierr); ierr = KSPGetDM(ksp, &da);CHKERRQ(ierr); ierr = PostP(da, x, b); ierr = VecDestroy(&r);CHKERRQ(ierr); ierr = KSPDestroy(&ksp);CHKERRQ(ierr); t2 = MPI_Wtime(); printf("Total Time Elapsed: %f\n", t2-t1); ierr = PetscFinalize(); return 0; } #undef __FUNCT__ #define __FUNCT__ "ComputeRHS" PetscErrorCode ComputeRHS(KSP ksp,Vec b,void *ctx) { PetscErrorCode ierr; PetscInt i,j,k,mx,my,mz,xm,ym,zm,xs,ys,zs, mm,nn,pp; DM dm; PetscScalar ***barray; double af; PetscFunctionBeginUser; ierr = KSPGetDM(ksp,&dm);CHKERRQ(ierr); ierr = DMDAGetInfo(dm,0,&mx,&my,&mz,&mm,&nn,&pp,0,0,0,0,0,0);CHKERRQ(ierr); printf("mx = %d, my = %d, mz =%d, mm = %d, nn = %d, pp = %d\n", mx, my, mz, mm, nn, pp); ierr = DMDAGetCorners(dm,&xs,&ys,&zs,&xm,&ym,&zm);CHKERRQ(ierr); ierr = DMDAVecGetArray(dm,b,&barray);CHKERRQ(ierr); for (k=zs; k= seci1+seci2 && IndexI <= seci1+seci2+seci3) { DeltaCoord = dx*pow(rdx3,IndexI-seci1-seci2+1); } else if(IndexI >= -1 && IndexI <= seci1) { DeltaCoord = dx*(pow(rdx1,seci1-IndexI)); } else if(IndexI > seci1 && IndexI < seci1+seci2) { DeltaCoord = dx; } else { printf("Fetal Error in RealDx!! @ %d \n", IndexI); exit(0); } return DeltaCoord; } #undef __FUNCT__ #define __FUNCT__ "RealPx" double RealPx(int IndexI) { double PhyCoord; if(IndexI >= -1 && IndexI <= seci1) { PhyCoord = (rdx1==1.)?dx*(seci1-IndexI):dx*((pow(rdx1,seci1-IndexI+1)-1.0)/(rdx1-1.0)-1.); PhyCoord = TotalLength1_x - PhyCoord; } else if(IndexI > seci1 && IndexI <= seci1+seci2) { PhyCoord = TotalLength1_x + (IndexI-seci1)*dx; } else if(IndexI > seci1+seci2 && IndexI <= seci1+seci2+seci3) { PhyCoord = (rdx3==1.)?dx*(IndexI-seci1-seci2):dx*((pow(rdx3,IndexI-seci1-seci2+1)-1.0)/(rdx3-1.0)-1.); PhyCoord = TotalLength2_x + PhyCoord; } else { printf("Fetal Error in PhyX!!\n"); exit(0); } return PhyCoord; } -------------- next part -------------- CFLAGS = FFLAGS = CPPFLAGS = FPPFLAGS = #LOCDIR = src/ksp/ksp/examples/tutorials/ EXAMPLESC = ex1.c ex2.c ex3.c ex4.c ex5.c ex7.c ex8.c ex8g.c ex9.c \ ex10.c ex11.c ex12.c ex13.c ex15.c ex16.c ex18.c ex23.c \ ex25.c ex27.c ex28.c ex29.c ex30.c ex31.c ex32.c ex34.c \ ex38.c ex39.c ex40.c ex41.c ex42.c ex43.c \ ex45.c ex46.c ex49.c ex50.c ex51.c ex52.c ex53.c \ ex54.c ex55.c ex56.c ex58.c EXAMPLESF = ex1f.F ex2f.F ex6f.F ex11f.F ex13f90.F ex14f.F ex15f.F ex21f.F ex22f.F ex44f.F90 ex45f.F ex54f.F MANSEC = KSP CLEANFILES = rhs.vtk solution.vtk NP = 1 include ${PETSC_DIR}/conf/variables include ${PETSC_DIR}/conf/rules ex1: ex1.o chkopts -${CLINKER} -o ex1 ex1.o ${PETSC_KSP_LIB} ${RM} ex1.o ex1f: ex1f.o chkopts -${FLINKER} -o ex1f ex1f.o ${PETSC_KSP_LIB} ${RM} ex1f.o ex2: ex2.o chkopts -${CLINKER} -o ex2 ex2.o ${PETSC_KSP_LIB} ${RM} ex2.o ex2a: ex2a.o chkopts -${CLINKER} -o ex2a ex2a.o ${PETSC_KSP_LIB} ${RM} ex2a.o ex2f: ex2f.o chkopts -${FLINKER} -o ex2f ex2f.o ${PETSC_KSP_LIB} ${RM} ex2f.o ex3: ex3.o chkopts -${CLINKER} -o ex3 ex3.o ${PETSC_KSP_LIB} ${RM} ex3.o ex4: ex4.o chkopts -${CLINKER} -o ex4 ex4.o ${PETSC_KSP_LIB} ${RM} ex4.o ex5: ex5.o chkopts -${CLINKER} -o ex5 ex5.o ${PETSC_KSP_LIB} ${RM} ex5.o ex6: ex6.o chkopts -${CLINKER} -o ex6 ex6.o ${PETSC_KSP_LIB} ${RM} ex6.o ex6f: ex6f.o chkopts -${FLINKER} -o ex6f ex6f.o ${PETSC_KSP_LIB} ${RM} ex6f.o ex7: ex7.o chkopts -${CLINKER} -o ex7 ex7.o ${PETSC_KSP_LIB} ${RM} ex7.o ex8: ex8.o chkopts -${CLINKER} -o ex8 ex8.o ${PETSC_KSP_LIB} ${RM} ex8.o ex8g: ex8g.o chkopts -${CLINKER} -o ex8g ex8g.o ${PETSC_KSP_LIB} ${RM} ex8g.o ex9: ex9.o chkopts -${CLINKER} -o ex9 ex9.o ${PETSC_KSP_LIB} ${RM} ex9.o ex10: ex10.o chkopts -${CLINKER} -o ex10 ex10.o ${PETSC_KSP_LIB} ${RM} ex10.o ex11: ex11.o chkopts -${CLINKER} -o ex11 ex11.o ${PETSC_KSP_LIB} ${RM} ex11.o ex11f: ex11f.o chkopts -${FLINKER} -o ex11f ex11f.o ${PETSC_KSP_LIB} ${RM} ex11f.o ex12: ex12.o chkopts -${CLINKER} -o ex12 ex12.o ${PETSC_KSP_LIB} ${RM} ex12.o ex13: ex13.o chkopts -${CLINKER} -o ex13 ex13.o ${PETSC_KSP_LIB} ${RM} ex13.o ex13f90: ex13f90.o chkopts -${FLINKER} -o ex13f90 ex13f90.o ${PETSC_KSP_LIB} ${RM} ex13f90.o ex14: ex14.o chkopts -${CLINKER} -o ex14 ex14.o ${PETSC_KSP_LIB} ${RM} ex14.o ex14f: ex14f.o chkopts -${FLINKER} -o ex14f ex14f.o ${PETSC_KSP_LIB} ${RM} ex14f.o ex15: ex15.o chkopts -${CLINKER} -o ex15 ex15.o ${PETSC_KSP_LIB} ${RM} ex15.o ex15f: ex15f.o chkopts -${FLINKER} -o ex15f ex15f.o ${PETSC_KSP_LIB} ${RM} ex15f.o ex16: ex16.o chkopts -${CLINKER} -o ex16 ex16.o ${PETSC_KSP_LIB} ${RM} ex16.o ex17: ex17.o chkopts -${CLINKER} -o ex17 ex17.o ${PETSC_KSP_LIB} ${RM} ex17.o ex18: ex18.o chkopts -${CLINKER} -o ex18 ex18.o ${PETSC_KSP_LIB} ${RM} ex18.o ex20: ex20.o chkopts -${CLINKER} -o ex20 ex20.o ${PETSC_KSP_LIB} ${RM} ex20.o ex21f: ex21f.o chkopts -${FLINKER} -o ex21f ex21f.o ${PETSC_KSP_LIB} ${RM} ex21f.o ex22: ex22.o chkopts -${CLINKER} -o ex22 ex22.o ${PETSC_SNES_LIB} ${RM} ex22.o ex22f: ex22f.o chkopts -${FLINKER} -o ex22f ex22f.o ${PETSC_SNES_LIB} ${RM} ex22f.o ex23: ex23.o chkopts -${CLINKER} -o ex23 ex23.o ${PETSC_KSP_LIB} ${RM} ex23.o ex25: ex25.o chkopts -${CLINKER} -o ex25 ex25.o ${PETSC_SNES_LIB} ${RM} ex25.o ex26: ex26.o chkopts -${CLINKER} -o ex26 ex26.o ${PETSC_KSP_LIB} ${RM} ex26.o ex27: ex27.o chkopts -${CLINKER} -o ex27 ex27.o ${PETSC_KSP_LIB} ${RM} ex27.o ex28: ex28.o chkopts -${CLINKER} -o ex28 ex28.o ${PETSC_SNES_LIB} ${RM} ex28.o ex29: ex29.o chkopts -${CLINKER} -o ex29 ex29.o ${PETSC_SNES_LIB} ${RM} ex29.o ex30: ex30.o chkopts -${CLINKER} -o ex30 ex30.o ${PETSC_KSP_LIB} ${RM} ex30.o ex31: ex31.o chkopts -${CLINKER} -o ex31 ex31.o ${PETSC_SNES_LIB} ${RM} ex31.o ex32: ex32.o chkopts -${CLINKER} -o ex32 ex32.o ${PETSC_SNES_LIB} ${RM} ex32.o ex33: ex33.o chkopts -${CLINKER} -o ex33 ex33.o ${PETSC_SNES_LIB} ${RM} ex33.o ex34: ex34.o chkopts -${CLINKER} -o ex34 ex34.o ${PETSC_SNES_LIB} ${RM} ex34.o ex35: ex35.o chkopts -${CLINKER} -o ex35 ex35.o ${PETSC_SNES_LIB} ${RM} ex35.o ex36: ex36.o chkopts -${CLINKER} -o ex36 ex36.o ${PETSC_SNES_LIB} ${RM} ex36.o ex37: ex37.o chkopts -${CLINKER} -o ex37 ex37.o ${PETSC_SNES_LIB} ${RM} ex37.o ex38: ex38.o chkopts -${CLINKER} -o ex38 ex38.o ${PETSC_SNES_LIB} ${RM} ex38.o ex39: ex39.o chkopts -${CLINKER} -o ex39 ex39.o ${PETSC_SNES_LIB} ${RM} ex39.o ex40: ex40.o chkopts -${CLINKER} -o ex40 ex40.o ${PETSC_SNES_LIB} ${RM} ex40.o ex41: ex41.o chkopts -${CLINKER} -o ex41 ex41.o ${PETSC_SNES_LIB} ${RM} ex41.o ex42: ex42.o chkopts -${CLINKER} -o ex42 ex42.o ${PETSC_KSP_LIB} ${RM} ex42.o ex43: ex43.o chkopts -${CLINKER} -o ex43 ex43.o ${PETSC_KSP_LIB} ${RM} ex43.o # not tested in nightly builds because requires F90 compiler that handles long lines ex44f: ex44f.o chkopts -${FLINKER} -o ex44f ex44f.o ${PETSC_KSP_LIB} ${RM} ex44f.o ex45f: ex45f.o chkopts -${FLINKER} -o ex45f ex45f.o ${PETSC_KSP_LIB} ${RM} ex45f.o ex45: ex45.o chkopts -${CLINKER} -o ex45 ex45.o ${PETSC_KSP_LIB} ${RM} ex45.o ex46: ex46.o chkopts -${CLINKER} -o ex46 ex46.o ${PETSC_KSP_LIB} ${RM} ex46.o ex47: ex47.o chkopts -${CLINKER} -o ex47 ex47.o ${PETSC_KSP_LIB} ${RM} ex47.o ex49: ex49.o chkopts -${CLINKER} -o ex49 ex49.o ${PETSC_KSP_LIB} ${RM} ex49.o ex50: ex50.o chkopts -${CLINKER} -o ex50 ex50.o ${PETSC_KSP_LIB} ${RM} ex50.o ex51: ex51.o chkopts -${CLINKER} -o ex51 ex51.o ${PETSC_KSP_LIB} ${RM} ex51.o ex52: ex52.o chkopts -${CLINKER} -o ex52 ex52.o ${PETSC_KSP_LIB} ${RM} ex52.o ex53: ex53.o chkopts -${CLINKER} -o ex53 ex53.o ${PETSC_KSP_LIB} ${RM} ex53.o ex54: ex54.o chkopts -${CLINKER} -o ex54 ex54.o ${PETSC_KSP_LIB} ${RM} ex54.o ex54f: ex54f.o chkopts -${FLINKER} -o ex54f ex54f.o ${PETSC_KSP_LIB} ${RM} ex54f.o ex55: ex55.o chkopts -${CLINKER} -o ex55 ex55.o ${PETSC_KSP_LIB} ${RM} ex55.o ex56: ex56.o chkopts -${CLINKER} -o ex56 ex56.o ${PETSC_KSP_LIB} ${RM} ex56.o ex57f: ex57f.o chkopts -${FLINKER} -o ex57f ex57f.o ${PETSC_KSP_LIB} ${RM} ex57f.o ex58: ex58.o chkopts -${CLINKER} -o ex58 ex58.o ${PETSC_KSP_LIB} ${RM} ex58.o ex59: ex59.o chkopts -${CLINKER} -o ex59 ex59.o ${PETSC_KSP_LIB} ${RM} ex59.o #---------------------------------------------------------------------------- runex1: -@${MPIEXEC} -n 1 ./ex1 -ksp_monitor_short -ksp_gmres_cgs_refinement_type refine_always > ex1_1.tmp 2>&1; \ if (${DIFF} output/ex1_1.out ex1_1.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex1_1, diffs above \n========================================="; fi; \ ${RM} -f ex1_1.tmp runex1_2: -@${MPIEXEC} -n 1 ./ex1 -pc_type sor -pc_sor_symmetric -ksp_monitor_short -ksp_gmres_cgs_refinement_type refine_always >\ ex1_2.tmp 2>&1; \ if (${DIFF} output/ex1_2.out ex1_2.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex1_2, diffs above \n========================================="; fi; \ ${RM} -f ex1_2.tmp runex1_3: -@${MPIEXEC} -n 1 ./ex1 -pc_type eisenstat -ksp_monitor_short -ksp_gmres_cgs_refinement_type refine_always >\ ex1_3.tmp 2>&1; \ if (${DIFF} output/ex1_3.out ex1_3.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex1_3, diffs above \n========================================="; fi; \ ${RM} -f ex1_3.tmp runex1f: -@${MPIEXEC} -n 1 ./ex1f -ksp_monitor_short -ksp_gmres_cgs_refinement_type refine_always > ex1f_1.tmp 2>&1; \ if (${DIFF} output/ex1f_1.out ex1f_1.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex1f_1, diffs above \n========================================="; fi; \ ${RM} -f ex1f_1.tmp runex2: -@${MPIEXEC} -n 1 ./ex2 -ksp_monitor_short -m 5 -n 5 -ksp_gmres_cgs_refinement_type refine_always > ex2_1.tmp 2>&1; \ if (${DIFF} output/ex2_1.out ex2_1.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex2_1, diffs above \n========================================="; fi; \ ${RM} -f ex2_1.tmp runex2_2: -@${MPIEXEC} -n 2 ./ex2 -ksp_monitor_short -m 5 -n 5 -ksp_gmres_cgs_refinement_type refine_always > ex2_2.tmp 2>&1; \ if (${DIFF} output/ex2_2.out ex2_2.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex2_2, diffs above \n========================================="; fi; \ ${RM} -f ex2_2.tmp runex2_3: -@${MPIEXEC} -n 1 ./ex2 -pc_type sor -pc_sor_symmetric -ksp_monitor_short -ksp_gmres_cgs_refinement_type refine_always > \ ex2_3.tmp 2>&1; \ if (${DIFF} output/ex2_3.out ex2_3.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex2_3, diffs above \n========================================="; fi; \ ${RM} -f ex2_3.tmp runex2_4: -@${MPIEXEC} -n 1 ./ex2 -pc_type eisenstat -ksp_monitor_short -ksp_gmres_cgs_refinement_type refine_always >\ ex2_4.tmp 2>&1; \ if (${DIFF} output/ex2_4.out ex2_4.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex2_4, diffs above \n========================================="; fi; \ ${RM} -f ex2_4.tmp runex2_5: -@${MPIEXEC} -n 2 ./ex2 -ksp_monitor_short -m 5 -n 5 -mat_view draw -ksp_gmres_cgs_refinement_type refine_always -nox > ex2_5.tmp 2>&1; \ if (${DIFF} output/ex2_2.out ex2_5.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex2_5, diffs above \n========================================="; fi; \ ${RM} -f ex2_5.tmp runex2_7: -@${MPIEXEC} -n 1 ./ex2 -pc_type ilu -pc_factor_drop_tolerance 0.01,0.0,2 > ex2_7.tmp 2>&1; \ if (${DIFF} output/ex2_7.out ex2_7.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex2_7, diffs above \n========================================="; fi; \ ${RM} -f ex2_7.tmp runex2_bjacobi: -@${MPIEXEC} -n 4 ./ex2 -pc_type bjacobi -pc_bjacobi_blocks 1 -ksp_monitor_short -sub_pc_type jacobi -sub_ksp_type gmres > ex2.tmp 2>&1; \ if (${DIFF} output/ex2_bjacobi.out ex2.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex2_bjacobi, diffs above \n========================================="; fi; \ ${RM} -f ex2.tmp runex2_bjacobi_2: -@${MPIEXEC} -n 4 ./ex2 -pc_type bjacobi -pc_bjacobi_blocks 2 -ksp_monitor_short -sub_pc_type jacobi -sub_ksp_type gmres -ksp_view > ex2.tmp 2>&1; \ if (${DIFF} output/ex2_bjacobi_2.out ex2.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex2_bjacobi_2, diffs above \n========================================="; fi; \ ${RM} -f ex2.tmp runex2_bjacobi_3: -@${MPIEXEC} -n 4 ./ex2 -pc_type bjacobi -pc_bjacobi_blocks 4 -ksp_monitor_short -sub_pc_type jacobi -sub_ksp_type gmres > ex2.tmp 2>&1; \ if (${DIFF} output/ex2_bjacobi_3.out ex2.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex2_bjacobi_3, diffs above \n========================================="; fi; \ ${RM} -f ex2.tmp runex2_specest_1: -@${MPIEXEC} -n 1 ./ex2 -m 80 -n 80 -ksp_type fgmres -pc_type ksp -ksp_ksp_type specest -ksp_monitor_short > ex2.tmp 2>&1; \ ${DIFF} output/ex2_specest_1.out ex2.tmp || echo ${PWD} "\nPossible problem with with ex2_specest_1, diffs above \n========================================="; \ ${RM} -f ex2.tmp runex2_specest_2: -@${MPIEXEC} -n 1 ./ex2 -m 80 -n 80 -ksp_type fgmres -pc_type ksp -ksp_ksp_type specest -ksp_monitor_short -ksp_specest_ksp_type cg > ex2.tmp 2>&1; \ ${DIFF} output/ex2_specest_2.out ex2.tmp || echo ${PWD} "\nPossible problem with with ex2_specest_2, diffs above \n========================================="; \ ${RM} -f ex2.tmp runex2_chebyest_1: -@${MPIEXEC} -n 1 ./ex2 -m 80 -n 80 -ksp_pc_side right -pc_type ksp -ksp_ksp_type chebyshev -ksp_ksp_max_it 5 -ksp_ksp_chebyshev_estimate_eigenvalues 0.9,0,0,1.1 -ksp_monitor_short > ex2.tmp 2>&1; \ ${DIFF} output/ex2_chebyest_1.out ex2.tmp || echo ${PWD} "\nPossible problem with with ex2_chebyest_1, diffs above \n========================================="; \ ${RM} -f ex2.tmp runex2_chebyest_2: -@${MPIEXEC} -n 1 ./ex2 -m 80 -n 80 -ksp_pc_side right -pc_type ksp -ksp_ksp_type chebyshev -ksp_ksp_max_it 5 -ksp_ksp_chebyshev_estimate_eigenvalues 0.9,0,0,1.1 -ksp_est_ksp_type cg -ksp_monitor_short > ex2.tmp 2>&1; \ ${DIFF} output/ex2_chebyest_2.out ex2.tmp || echo ${PWD} "\nPossible problem with with ex2_chebyest_2, diffs above \n========================================="; \ ${RM} -f ex2.tmp runex2_chebyest_3: -@${MPIEXEC} -n 1 ./ex2 -ksp_type fgmres -pc_type ksp -ksp_ksp_type chebyshev -ksp_ksp_chebyshev_estimate_eigenvalues 0.1,1.1 -ksp_ksp_max_it 1 -ksp_ksp_norm_type none -m 100 -n 100 -ksp_ksp_chebyshev_hybrid -ksp_ksp_chebyshev_hybrid_purification 0 -ksp_ksp_chebyshev_hybrid_chebysteps 10 > ex2.tmp 2>&1; \ ${DIFF} output/ex2_chebyest_3.out ex2.tmp || echo ${PWD} "\nPossible problem with with ex2_chebyest_3, diffs above \n========================================="; \ ${RM} -f ex2.tmp runex2_chebyest_4: -@${MPIEXEC} -n 1 ./ex2 -ksp_type fgmres -pc_type ksp -ksp_ksp_type chebyshev -ksp_ksp_chebyshev_estimate_eigenvalues 0.1,1.1 -ksp_ksp_max_it 1 -ksp_ksp_norm_type none -m 100 -n 100 -ksp_ksp_chebyshev_hybrid -ksp_ksp_chebyshev_hybrid_purification 1 -ksp_ksp_chebyshev_hybrid_chebysteps 10 > ex2.tmp 2>&1; \ ${DIFF} output/ex2_chebyest_4.out ex2.tmp || echo ${PWD} "\nPossible problem with with ex2_chebyest_3, diffs above \n========================================="; \ ${RM} -f ex2.tmp runex2_umfpack: -@${MPIEXEC} -n 1 ./ex2 -ksp_type preonly -pc_type lu -pc_factor_mat_solver_package umfpack > ex2_umfpack.tmp 2>&1; \ if (${DIFF} output/ex2_umfpack.out ex2_umfpack.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex2_umfpack, diffs above \n========================================="; fi; \ ${RM} -f ex2_umfpack.tmp runex2_fbcgs: -@${MPIEXEC} -n 1 ./ex2 -ksp_type fbcgs -pc_type ilu > ex2.tmp 2>&1; \ if (${DIFF} output/ex2_fbcgs.out ex2.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex2_fbcgs, diffs above \n========================================="; fi; \ ${RM} -f ex2.tmp runex2_fbcgs_2: -@${MPIEXEC} -n 3 ./ex2 -ksp_type fbcgsr -pc_type bjacobi > ex2.tmp 2>&1; \ if (${DIFF} output/ex2_fbcgs_2.out ex2.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex2_fbcgs_2, diffs above \n========================================="; fi; \ ${RM} -f ex2.tmp runex2f: -@${MPIEXEC} -n 2 ./ex2f -pc_type jacobi -ksp_monitor_short -ksp_gmres_cgs_refinement_type refine_always > ex2f_1.tmp 2>&1; \ if (${DIFF} output/ex2f_1.out ex2f_1.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex2f_1, diffs above \n========================================="; fi; \ ${RM} -f ex2f_1.tmp runex5: -@${MPIEXEC} -n 1 ./ex5 -pc_type jacobi -ksp_monitor_short -ksp_gmres_cgs_refinement_type refine_always > ex5_1.tmp 2>&1; \ if (${DIFF} output/ex5_1.out ex5_1.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex5_1, diffs above \n========================================="; fi; \ ${RM} -f ex5_1.tmp runex5_2: -@${MPIEXEC} -n 2 ./ex5 -pc_type jacobi -ksp_monitor_short -ksp_gmres_cgs_refinement_type refine_always \ -ksp_rtol .000001 > ex5_2.tmp 2>&1; \ if (${DIFF} output/ex5_2.out ex5_2.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex5_2, diffs above \n========================================="; fi; \ ${RM} -f ex5_2.tmp runex5_5: -@${MPIEXEC} -n 2 ./ex5 -ksp_gmres_cgs_refinement_type refine_always > ex5_5.tmp 2>&1; \ if (${DIFF} output/ex5_5.out ex5_5.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex5_5, diffs above \n========================================="; fi; \ ${RM} -f ex5_5.tmp runex6f: -@${MPIEXEC} -n 1 ./ex6f -pc_type jacobi -mat_view -ksp_monitor_short -ksp_gmres_cgs_refinement_type refine_always > ex6f_1.tmp 2>&1; \ if (${DIFF} output/ex6f_1.out ex6f_1.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex6f_1, diffs above \n========================================="; fi; \ ${RM} -f ex6f_1.tmp runex7: -@${MPIEXEC} -n 2 ./ex7 -ksp_monitor_short -nokspview -ksp_gmres_cgs_refinement_type refine_always> ex7_1.tmp 2>&1; \ if (${DIFF} output/ex7_1.out ex7_1.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex7_1, diffs above \n========================================="; fi; \ ${RM} -f ex7_1.tmp runex9: -@${MPIEXEC} -n 1 ./ex9 -t 2 -pc_type jacobi -ksp_monitor_short -ksp_type gmres -ksp_gmres_cgs_refinement_type refine_always \ -s2_ksp_type bcgs -s2_pc_type jacobi -s2_ksp_monitor_short \ > ex9_1.tmp 2>&1; \ if (${DIFF} output/ex9_1.out ex9_1.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex9_1, diffs above \n========================================="; fi; \ ${RM} -f ex9_1.tmp NP = 1 M = 4 N = 5 MDOMAINS = 2 NDOMAINS = 1 OVERLAP=1 runex8: -@${MPIEXEC} -n ${NP} ./ex8 -m $M -n $N -user_set_subdomains -Mdomains ${MDOMAINS} -Ndomains ${NDOMAINS} -overlap ${OVERLAP} -print_error ${ARGS} VALGRIND= runex8g: -@${MPIEXEC} -n ${NP} ${VALGRIND} ./ex8g -M $M -N $N -user_set_subdomains -Mdomains ${MDOMAINS} -Ndomains ${NDOMAINS} -overlap ${OVERLAP} -print_error ${ARGS} BREAKPOINT= runex8g_debug: -@${MPIEXEC} -n ${NP} xterm -e gdb -ex 'set breakpoint pending on' -ex 'b ${BREAKPOINT}' -ex r -ex bt --args ./ex8g -M $M -N $N -user_set_subdomains -Mdomains ${MDOMAINS} -Ndomains ${NDOMAINS} -overlap ${OVERLAP} -print_error ${ARGS} runex8_1: -@${MPIEXEC} -n 1 ./ex8 -print_error > ex8_1.tmp 2>&1; \ if (${DIFF} output/ex8_1.out ex8_1.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex8_1, diffs above \n========================================="; fi; \ ${RM} -f ex8_1.tmp runex8g_1: -@${MPIEXEC} -n 1 ./ex8g -M 7 -N 9 -user_set_subdomains -Mdomains 1 -Ndomains 3 -overlap 1 -print_error -pc_gasm_print_subdomains > ex8g_1.tmp 2>&1; \ if (${DIFF} output/ex8g_1.out ex8g_1.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex8g_1, diffs above \n========================================="; fi; \ ${RM} -f ex8g_1.tmp runex8g_2: -@${MPIEXEC} -n 2 ./ex8g -M 7 -N 9 -user_set_subdomains -Mdomains 1 -Ndomains 3 -overlap 1 -print_error -pc_gasm_print_subdomains > ex8g_2.tmp 2>&1; \ if (${DIFF} output/ex8g_2.out ex8g_2.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex8g_2, diffs above \n========================================="; fi; \ ${RM} -f ex8g_2.tmp runex8g_3: -@${MPIEXEC} -n 3 ./ex8g -M 7 -N 9 -user_set_subdomains -Mdomains 1 -Ndomains 3 -overlap 1 -print_error -pc_gasm_print_subdomains > ex8g_3.tmp 2>&1; \ if (${DIFF} output/ex8g_3.out ex8g_3.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex8g_3, diffs above \n========================================="; fi; \ ${RM} -f ex8g_3.tmp runex10_basic: -@${MPIEXEC} -n 2 ./ex10 -f0 ${PETSC_DIR}/include/datafiles/matrices/spd-${PETSC_SCALAR_TYPE}-${PETSC_INDEX_SIZE}-${PETSC_SCALAR_SIZE} > ex10_1.tmp 2>&1; \ if (${DIFF} output/ex10_1.out ex10_1.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex10_1, diffs above \n========================================="; fi; \ ${RM} -f ex10_1.tmp # See http://www.mcs.anl.gov/petsc/documentation/faq.html#datafiles for how to obtain the datafiles used below runex10_2: -@${MPIEXEC} -n 2 ./ex10 -ksp_type bicg \ -f0 ${DATAFILESPATH}/matrices/medium > ex10_2.tmp 2>&1; \ if (${DIFF} output/ex10_2.out ex10_2.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex10_2, diffs above \n========================================="; fi; \ ${RM} -f ex10_2.tmp runex10_3: -@${MPIEXEC} -n 2 ./ex10 -ksp_type bicg -pc_type asm \ -f0 ${DATAFILESPATH}/matrices/medium > ex10_3.tmp 2>&1; \ if (${DIFF} output/ex10_3.out ex10_3.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex10_3, diffs above \n========================================="; fi; \ ${RM} -f ex10_3.tmp runex10_4: -@${MPIEXEC} -n 1 ./ex10 -ksp_type bicg -pc_type lu \ -f0 ${DATAFILESPATH}/matrices/medium > ex10_4.tmp 2>&1; \ if (${DIFF} output/ex10_4.out ex10_4.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex10_4, diffs above \n========================================="; fi; \ ${RM} -f ex10_4.tmp runex10_5: -@${MPIEXEC} -n 1 ./ex10 -ksp_type bicg \ -f0 ${DATAFILESPATH}/matrices/medium > ex10_5.tmp 2>&1; \ if (${DIFF} output/ex10_5.out ex10_5.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex10_5, diffs above \n========================================="; fi; \ ${RM} -f ex10_5.tmp runex10_6: -@${MPIEXEC} -n 1 ./ex10 -pc_factor_levels 2 -pc_factor_fill 1.73 -ksp_gmres_cgs_refinement_type refine_always \ -f0 ${DATAFILESPATH}/matrices/fem1 > ex10_6.tmp 2>&1; \ if (${DIFF} output/ex10_6.out ex10_6.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex10_6, diffs above \n========================================="; fi; \ ${RM} -f ex10_6.tmp # See http://www.mcs.anl.gov/petsc/documentation/faq.html#datafiles for how to obtain the datafiles used below BS = 2 3 4 5 6 7 8 runex10_7: - at touch ex10_7.tmp - at for bs in ${BS}; do \ ${MPIEXEC} -n 1 ./ex10 -f0 ${DATAFILESPATH}/matrices/medium -viewer_binary_skip_info \ -mat_type seqbaij -matload_block_size $$bs -ksp_max_it 100 -ksp_gmres_cgs_refinement_type refine_always -ksp_rtol \ 1.0e-15 -ksp_monitor_short >> ex10_7.tmp 2>&1 ; \ ${MPIEXEC} -n 1 ./ex10 -f0 ${DATAFILESPATH}/matrices/medium -viewer_binary_skip_info \ -mat_type seqbaij -matload_block_size $$bs -ksp_max_it 100 -ksp_gmres_cgs_refinement_type refine_always -ksp_rtol \ 1.0e-15 -ksp_monitor_short -pc_factor_mat_ordering_type nd >> ex10_7.tmp 2>&1 ; \ ${MPIEXEC} -n 1 ./ex10 -f0 ${DATAFILESPATH}/matrices/medium -viewer_binary_skip_info \ -mat_type seqbaij -matload_block_size $$bs -ksp_max_it 100 -ksp_gmres_cgs_refinement_type refine_always -ksp_rtol \ 1.0e-15 -ksp_monitor_short -pc_factor_levels 1 >> ex10_7.tmp 2>&1 ; \ ${MPIEXEC} -n 1 ./ex10 -f0 ${DATAFILESPATH}/matrices/medium -viewer_binary_skip_info \ -mat_type seqbaij -matload_block_size $$bs -ksp_type preonly \ -ksp_monitor_short -pc_type lu >> ex10_7.tmp 2>&1 ; \ done; - at if (${DIFF} output/ex10_7.out ex10_7.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex10_7, diffs above \n========================================="; fi; -@${RM} -f ex10_7.tmp # See http://www.mcs.anl.gov/petsc/documentation/faq.html#datafiles for how to obtain the datafiles used below runex10_8: -@${MPIEXEC} -n 1 ./ex10 -ksp_diagonal_scale -pc_type eisenstat -ksp_monitor_short -ksp_diagonal_scale_fix \ -f0 ${DATAFILESPATH}/matrices/medium -ksp_gmres_cgs_refinement_type refine_always -mat_no_inode > ex10_8.tmp 2>&1; \ if (${DIFF} output/ex10_8.out ex10_8.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex10_8, diffs above \n========================================="; fi; \ ${RM} -f ex10_8.tmp # See http://www.mcs.anl.gov/petsc/documentation/faq.html#datafiles for how to obtain the datafiles used below runex10_9: - at touch ex10_9.tmp - at for type in gmres; do \ for bs in 1 2 3 4 5 6 7; do \ ${MPIEXEC} -n 1 ./ex10 -f0 ${DATAFILESPATH}/matrices/medium -viewer_binary_skip_info \ -mat_type seqbaij -matload_block_size $$bs -ksp_max_it 100 -ksp_gmres_cgs_refinement_type refine_always -ksp_rtol \ 1.0e-15 -ksp_monitor_short >> ex10_9.tmp 2>&1 ; \ ${MPIEXEC} -n 1 ./ex10 -f0 ${DATAFILESPATH}/matrices/medium -ksp_gmres_cgs_refinement_type refine_always -viewer_binary_skip_info \ -mat_type seqbaij -matload_block_size $$bs -ksp_max_it 100 -ksp_rtol \ 1.0e-15 -ksp_monitor_short -trans >> ex10_9.tmp 2>&1 ; \ for np in 2 3; do \ ${MPIEXEC} -n $$np ./ex10 -f0 ${DATAFILESPATH}/matrices/medium -viewer_binary_skip_info \ -mat_type mpibaij -matload_block_size $$bs -ksp_max_it 100 -ksp_gmres_cgs_refinement_type refine_always -ksp_rtol \ 1.0e-15 -ksp_monitor_short >> ex10_9.tmp 2>&1 ; \ ${MPIEXEC} -n $$np ./ex10 -f0 ${DATAFILESPATH}/matrices/medium -ksp_gmres_cgs_refinement_type refine_always -viewer_binary_skip_info \ -mat_type mpibaij -matload_block_size $$bs -ksp_max_it 100 -ksp_rtol \ 1.0e-15 -ksp_monitor_short -trans >> ex10_9.tmp 2>&1 ; \ done; done; done; - at if (${DIFF} output/ex10_9.out ex10_9.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex10_9, diffs above \n========================================="; fi; -@${RM} -f ex10_9.tmp # See http://www.mcs.anl.gov/petsc/documentation/faq.html#datafiles for how to obtain the datafiles used below runex10_10: -@${MPIEXEC} -n 2 ./ex10 -ksp_type fgmres -pc_type ksp \ -f0 ${DATAFILESPATH}/matrices/medium -ksp_fgmres_modifypcksp -ksp_monitor_short> ex10_10.tmp 2>&1; \ if (${DIFF} output/ex10_10.out ex10_10.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex10_10, diffs above \n========================================="; fi; \ ${RM} -f ex10_10.tmp runex10_11: -@${MPIEXEC} -n 2 ./ex10 -f0 http://ftp.mcs.anl.gov/pub/petsc/matrices/testmatrix.gz > ex10_11.tmp 2>&1;\ if (${DIFF} output/ex10_11.out ex10_11.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex10_11, diffs above \n========================================="; fi; \ ${RM} -f ex10_11.tmp runex10_12: -@${MPIEXEC} -n 1 ./ex10 -mat_aij_matlab -pc_type lu -f0 ${DATAFILESPATH}/matrices/arco1 > ex10_12.tmp 2>&1;\ if (${DIFF} output/ex10_12.out ex10_12.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex10_12, diffs above \n========================================="; fi; \ ${RM} -f ex10_12.tmp runex10_13: -@${MPIEXEC} -n 1 ./ex10 -mat_type lusol -pc_type lu -f0 ${DATAFILESPATH}/matrices/arco1 > ex10_13.tmp 2>&1;\ if (${DIFF} output/ex10_13.out ex10_13.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex10_13, diffs above \n========================================="; fi; \ ${RM} -f ex10_13.tmp runex10_14: -@${MPIEXEC} -n 3 ./ex10 -pc_type spai -f0 ${DATAFILESPATH}/matrices/medium > ex10_14.tmp 2>&1; \ ${DIFF} output/ex10_14.out ex10_14.tmp || echo ${PWD} "\nPossible problem with with ex10_14, diffs above \n========================================="; \ ${RM} -f ex10_14.tmp runex10_15: -@${MPIEXEC} -n 3 ./ex10 -pc_type hypre -pc_hypre_type pilut -f0 ${DATAFILESPATH}/matrices/medium > ex10_15.tmp 2>&1;\ if (${DIFF} output/ex10_15.out ex10_15.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex10_15, diffs above \n========================================="; fi; \ ${RM} -f ex10_15.tmp runex10_16: -@${MPIEXEC} -n 3 ./ex10 -pc_type hypre -pc_hypre_type parasails -f0 ${DATAFILESPATH}/matrices/medium > ex10_16.tmp 2>&1;\ if (${DIFF} output/ex10_16.out ex10_16.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex10_16, diffs above \n========================================="; fi; \ ${RM} -f ex10_16.tmp runex10_17: -@${MPIEXEC} -n 3 ./ex10 -pc_type hypre -pc_hypre_type boomeramg -f0 ${DATAFILESPATH}/matrices/medium > ex10_17.tmp 2>&1;\ if (${DIFF} output/ex10_17.out ex10_17.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex10_17, diffs above \n========================================="; fi; \ ${RM} -f ex10_17.tmp # # The following leaves memory unfreed. It appears to be Euclid does something improper with its MPI_Comm runex10_18: -@${MPIEXEC} -n 3 ./ex10 -pc_type hypre -pc_hypre_type euclid -f0 ${DATAFILESPATH}/matrices/medium > ex10_18.tmp 2>&1;\ if (${DIFF} output/ex10_18.out ex10_18.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex10_18, diffs above \n========================================="; fi; \ ${RM} -f ex10_18.tmp # See http://www.mcs.anl.gov/petsc/documentation/faq.html#datafiles for how to obtain the datafiles used below LEVELS = 0 2 4 runex10_19: - at touch ex10_19aij.tmp - at touch ex10_19sbaij.tmp - at for levels in ${LEVELS}; do \ ${MPIEXEC} -n 1 ./ex10 -f0 ${DATAFILESPATH}/matrices/poisson1 -ksp_type cg -pc_type icc -pc_factor_levels $$levels >> ex10_19aij.tmp 2>&1; \ ${MPIEXEC} -n 1 ./ex10 -f0 ${DATAFILESPATH}/matrices/poisson1 -ksp_type cg -pc_type icc -pc_factor_levels $$levels -mat_type seqsbaij >> ex10_19sbaij.tmp 2>&1; \ done; - at if (${DIFF} ex10_19aij.tmp ex10_19sbaij.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex10_19, diffs above \n========================================="; fi; \ ${RM} -f ex10_19aij.tmp ex10_19sbaij.tmp # See http://www.mcs.anl.gov/petsc/documentation/faq.html#datafiles for how to obtain the datafiles used below # Start a sequential user code [on 1 node, assembling seqaij] and then # spwan a parallel solver on 2 procs. # See http://www.mcs.anl.gov/petsc/documentation/faq.html#datafiles for how to obtain the datafiles used below runex10_22: -@${MPIEXEC} -n 2 ./ex10 -hmpi_merge_size 2 -pc_type hmpi -ksp_type preonly -hmpi_pc_type ksp -f0 ${DATAFILESPATH}/matrices/medium > ex10_22.tmp 2>&1;\ if (${DIFF} output/ex10_22.out ex10_22.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex10_22, diffs above \n========================================="; fi; \ ${RM} -f ex10_22.tmp runex10_23: -@${MPIEXEC} -n 2 ./ex10 -hmpi_merge_size 2 -pc_type hmpi -ksp_type preonly -hmpi_pc_type ksp -hmpi_ksp_pc_type bjacobi -hmpi_ksp_ksp_type gmres -f0 ${DATAFILESPATH}/matrices/medium > ex10_23.tmp 2>&1;\ if (${DIFF} output/ex10_23.out ex10_23.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex10_23, diffs above \n========================================="; fi; \ ${RM} -f ex10_23.tmp runex10_24: -@${MPIEXEC} -n 2 ./ex10 -hmpi_merge_size 2 -pc_type hmpi -hmpi_pc_type sor -f0 ${DATAFILESPATH}/matrices/medium -hmpi_ksp_monitor_short -initialguess -ksp_type gmres -ksp_monitor_short -ksp_view > ex10_24.tmp 2>&1;\ if (${DIFF} output/ex10_24.out ex10_24.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex10_24, diffs above \n========================================="; fi; \ ${RM} -f ex10_24.tmp # Start a parallel user code [on 4 nodes, assembling MPIAIJ with # np=4] and then spwan a parallel sub-domain-solver on each node # [with np=2]. This emulates mixed MPI-shared memory model [MPI between # nodes, MPI within the nodes] # See http://www.mcs.anl.gov/petsc/documentation/faq.html#datafiles for how to obtain the datafiles used below runex10_25: -@${MPIEXEC} -n 8 ./ex10 -hmpi_merge_size 2 -sub_pc_type hmpi -f0 ${DATAFILESPATH}/matrices/medium -ksp_monitor_short> ex10_25.tmp 2>&1;\ if (${DIFF} output/ex10_25.out ex10_25.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex10_25, diffs above \n========================================="; fi; \ ${RM} -f ex10_25.tmp runex10_superlu_lu_1: -@${MPIEXEC} -n 1 ./ex10 -f0 ${DATAFILESPATH}/matrices/small -ksp_type preonly -pc_type lu -pc_factor_mat_solver_package superlu -num_numfac 2 -num_rhs 2 > ex10_superlu_lu_1.tmp 2>&1; \ if (${DIFF} output/ex10_mumps.out ex10_superlu_lu_1.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex10_superlu_lu_1, diffs above \n========================================="; fi; \ ${RM} -f ex10_superlu_lu_1.tmp runex10_superlu_dist_lu_1: -@${MPIEXEC} -n 1 ./ex10 -f0 ${DATAFILESPATH}/matrices/small -ksp_type preonly -pc_type lu -pc_factor_mat_solver_package superlu_dist -num_numfac 2 -num_rhs 2 > ex10_superlu_lu_2.tmp 2>&1; \ if (${DIFF} output/ex10_mumps.out ex10_superlu_lu_2.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex10_superlu_lu_2, diffs above \n========================================="; fi; \ ${RM} -f ex10_superlu_lu_2.tmp runex10_superlu_dist_lu_2: -@${MPIEXEC} -n 2 ./ex10 -f0 ${DATAFILESPATH}/matrices/small -ksp_type preonly -pc_type lu -pc_factor_mat_solver_package superlu_dist -num_numfac 2 -num_rhs 2 > ex10_superlu_lu_2.tmp 2>&1; \ if (${DIFF} output/ex10_mumps.out ex10_superlu_lu_2.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex10_superlu_lu_2, diffs above \n========================================="; fi; \ ${RM} -f ex10_superlu_lu_2.tmp runex10_umfpack: -@${MPIEXEC} -n 1 ./ex10 -f0 ${DATAFILESPATH}/matrices/small -ksp_type preonly -pc_type lu -mat_type seqaij -pc_factor_mat_solver_package umfpack -num_numfac 2 -num_rhs 2 > ex10_umfpack.tmp 2>&1; \ if (${DIFF} output/ex10_umfpack.out ex10_umfpack.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex10_umfpack, diffs above \n========================================="; fi; \ ${RM} -f ex10_umfpack.tmp runex10_mumps_lu_1: -@${MPIEXEC} -n 1 ./ex10 -f0 ${DATAFILESPATH}/matrices/small -ksp_type preonly -pc_type lu -mat_type seqaij -pc_factor_mat_solver_package mumps -num_numfac 2 -num_rhs 2 > ex10_mumps_lu_1.tmp 2>&1; \ if (${DIFF} output/ex10_mumps.out ex10_mumps_lu_1.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex10_mumps_lu_1, diffs above \n========================================="; fi; \ ${RM} -f ex10_mumps_lu_1.tmp runex10_mumps_lu_2: -@${MPIEXEC} -n 2 ./ex10 -f0 ${DATAFILESPATH}/matrices/small -ksp_type preonly -pc_type lu -mat_type mpiaij -pc_factor_mat_solver_package mumps -num_numfac 2 -num_rhs 2 > ex10_mumps_lu_2.tmp 2>&1; \ if (${DIFF} output/ex10_mumps.out ex10_mumps_lu_2.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex10_mumps_lu_2, diffs above \n========================================="; fi; \ ${RM} -f ex10_mumps_lu_2.tmp runex10_mumps_lu_3: -@${MPIEXEC} -n 1 ./ex10 -f0 ${DATAFILESPATH}/matrices/small -ksp_type preonly -pc_type lu -mat_type seqbaij -pc_factor_mat_solver_package mumps -num_numfac 2 -num_rhs 2 -matload_block_size 2 > ex10_mumps_lu_3.tmp 2>&1; \ if (${DIFF} output/ex10_mumps.out ex10_mumps_lu_3.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex10_mumps_lu_3, diffs above \n========================================="; fi; \ ${RM} -f ex10_mumps_lu_3.tmp runex10_mumps_lu_4: -@${MPIEXEC} -n 2 ./ex10 -f0 ${DATAFILESPATH}/matrices/small -ksp_type preonly -pc_type lu -mat_type mpibaij -pc_factor_mat_solver_package mumps -num_numfac 2 -num_rhs 2 -matload_block_size 2 > ex10_mumps_lu_4.tmp 2>&1; \ if (${DIFF} output/ex10_mumps.out ex10_mumps_lu_4.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex10_mumps_lu_4, diffs above \n========================================="; fi; \ ${RM} -f ex10_mumps_lu_4.tmp runex10_mumps_cholesky_1: -@${MPIEXEC} -n 1 ./ex10 -f0 ${DATAFILESPATH}/matrices/small -ksp_type preonly -pc_type cholesky -mat_type sbaij -pc_factor_mat_solver_package mumps -num_numfac 2 -num_rhs 2 -mat_ignore_lower_triangular > ex10_mumps_cholesky_1.tmp 2>&1; \ if (${DIFF} output/ex10_mumps.out ex10_mumps_cholesky_1.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex10_mumps_cholesky_1, diffs above \n========================================="; fi; \ ${RM} -f ex10_mumps_cholesky_1.tmp runex10_mumps_cholesky_2: -@${MPIEXEC} -n 2 ./ex10 -f0 ${DATAFILESPATH}/matrices/small -ksp_type preonly -pc_type cholesky -mat_type sbaij -pc_factor_mat_solver_package mumps -num_numfac 2 -num_rhs 2 -mat_ignore_lower_triangular > ex10_mumps_cholesky_2.tmp 2>&1; \ if (${DIFF} output/ex10_mumps.out ex10_mumps_cholesky_2.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex10_mumps_cholesky_2, diffs above \n========================================="; fi; \ ${RM} -f ex10_mumps_cholesky_2.tmp runex10_mumps_cholesky_3: -@${MPIEXEC} -n 1 ./ex10 -f0 ${DATAFILESPATH}/matrices/small -ksp_type preonly -pc_type cholesky -mat_type aij -pc_factor_mat_solver_package mumps -num_numfac 2 -num_rhs 2 > ex10_mumps_cholesky_3.tmp 2>&1; \ if (${DIFF} output/ex10_mumps.out ex10_mumps_cholesky_3.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex10_mumps_cholesky_3, diffs above \n========================================="; fi; \ ${RM} -f ex10_mumps_cholesky_3.tmp runex10_mumps_cholesky_4: -@${MPIEXEC} -n 2 ./ex10 -f0 ${DATAFILESPATH}/matrices/small -ksp_type preonly -pc_type cholesky -mat_type aij -pc_factor_mat_solver_package mumps -num_numfac 2 -num_rhs 2 > ex10_mumps_cholesky_4.tmp 2>&1; \ if (${DIFF} output/ex10_mumps.out ex10_mumps_cholesky_4.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex10_mumps_cholesky_4, diffs above \n========================================="; fi; \ ${RM} -f ex10_mumps_cholesky_4.tmp runex10_mumps_cholesky_spd_1: -@${MPIEXEC} -n 1 ./ex10 -f0 ${DATAFILESPATH}/matrices/small -ksp_type preonly -pc_type cholesky -mat_type aij -matload_spd -pc_factor_mat_solver_package mumps -num_numfac 2 -num_rhs 2 > ex10_mumps_cholesky_spd_1.tmp 2>&1; \ if (${DIFF} output/ex10_mumps.out ex10_mumps_cholesky_spd_1.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex10_mumps_cholesky_spd_1, diffs above \n========================================="; fi; \ ${RM} -f ex10_mumps_cholesky_spd_1.tmp runex10_mumps_cholesky_spd_2: -@${MPIEXEC} -n 2 ./ex10 -f0 ${DATAFILESPATH}/matrices/small -ksp_type preonly -pc_type cholesky -mat_type aij -matload_spd -pc_factor_mat_solver_package mumps -num_numfac 2 -num_rhs 2 > ex10_mumps_cholesky_spd_2.tmp 2>&1; \ if (${DIFF} output/ex10_mumps.out ex10_mumps_cholesky_spd_2.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex10_mumps_cholesky_spd_2, diffs above \n========================================="; fi; \ ${RM} -f ex10_mumps_cholesky_spd_2.tmp NSUBCOMM = 8 7 6 5 4 3 2 1 runex10_mumps_redundant: - at touch ex10_mumps_redundant.tmp - at for nsubcomm in ${NSUBCOMM}; do \ ${MPIEXEC} -n 8 ./ex10 -f0 ${DATAFILESPATH}/matrices/medium -ksp_type preonly -pc_type redundant -pc_redundant_number $$nsubcomm -redundant_pc_factor_mat_solver_package mumps -num_numfac 2 -num_rhs 2 >> ex10_mumps_redundant.tmp 2>&1; \ done; - at if (${DIFF} output/ex10_mumps_redundant.out ex10_mumps_redundant.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex10_mumps_redundant, diffs above \n========================================="; fi; \ ${RM} -f ex10_mumps_redundant.tmp; runex10_superlu_dist_redundant: - at touch ex10_superlu_dist_redundant.tmp - at for nsubcomm in ${NSUBCOMM}; do \ ${MPIEXEC} -n 8 ./ex10 -f0 ${DATAFILESPATH}/matrices/medium -ksp_type preonly -pc_type redundant -pc_redundant_number $$nsubcomm -redundant_pc_factor_mat_solver_package superlu_dist -num_numfac 2 -num_rhs 2 >> ex10_superlu_dist_redundant.tmp 2>&1; \ done; - at if (${DIFF} output/ex10_mumps_redundant.out ex10_superlu_dist_redundant.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex10_superlu_dist_redundant, diffs above \n========================================="; fi; \ ${RM} -f ex10_superlu_dist_redundant.tmp; runex10_ILU: # test ilu fill greater than zero -@${MPIEXEC} -n 1 ./ex10 -f0 ${DATAFILESPATH}/matrices/small -pc_factor_levels 1 > ex10_20.tmp 2>&1; \ if (${DIFF} output/ex10_ILU.out ex10_20.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex10_ILU, diffs above \n========================================="; fi; \ ${RM} -f ex10_20.tmp runex10_ILUBAIJ: # test ilu fill greater than zero -@${MPIEXEC} -n 1 ./ex10 -f0 ${DATAFILESPATH}/matrices/small -pc_factor_levels 1 -mat_type baij > ex10_20.tmp 2>&1; \ if (${DIFF} output/ex10_ILU.out ex10_20.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex10_ILU, diffs above \n========================================="; fi; \ ${RM} -f ex10_20.tmp runex10_cg: -@${MPIEXEC} -n 1 ./ex10 -f0 ${DATAFILESPATH}/matrices/small -mat_type mpisbaij -ksp_type cg -pc_type eisenstat -ksp_monitor_short -ksp_converged_reason > ex10_20.tmp 2>&1; \ if (${DIFF} output/ex10_cg_singlereduction.out ex10_20.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex10_cg, diffs above \n========================================="; fi; \ ${RM} -f ex10_20.tmp runex10_cg_singlereduction: -@${MPIEXEC} -n 1 ./ex10 -f0 ${DATAFILESPATH}/matrices/small -mat_type mpisbaij -ksp_type cg -pc_type eisenstat -ksp_monitor_short -ksp_converged_reason -ksp_cg_single_reduction > ex10_20.tmp 2>&1; \ if (${DIFF} output/ex10_cg_singlereduction.out ex10_20.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex10_cg_singlereduction, diffs above \n========================================="; fi; \ ${RM} -f ex10_20.tmp runex10_seqaijcrl: -@${MPIEXEC} -n 1 ./ex10 -f0 ${DATAFILESPATH}/matrices/small -ksp_monitor_short -ksp_view -mat_view ascii::ascii_info -mat_type seqaijcrl > ex10_seqaijcrl.tmp 2>&1; \ if (${DIFF} output/ex10_seqcrl.out ex10_seqaijcrl.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex10_seqaijcrl, diffs above \n========================================="; fi; \ ${RM} -f ex10_seqaijcrl.tmp runex10_mpiaijcrl: -@${MPIEXEC} -n 2 ./ex10 -f0 ${DATAFILESPATH}/matrices/small -ksp_monitor_short -ksp_view -mat_view ascii::ascii_info -mat_type mpiaijcrl > ex10_mpiaijcrl.tmp 2>&1; \ if (${DIFF} output/ex10_mpiaij.out ex10_mpiaijcrl.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex10_mpiaijcrl, diffs above \n========================================="; fi; \ ${RM} -f ex10_mpiaijcrl.tmp runex10_seqaijperm: -@${MPIEXEC} -n 1 ./ex10 -f0 ${DATAFILESPATH}/matrices/small -ksp_monitor_short -ksp_view -mat_view ascii::ascii_info -mat_type seqaijperm > ex10_seqaijperm.tmp 2>&1; \ if (${DIFF} output/ex10_seqcsrperm.out ex10_seqaijperm.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex10_seqaijperm, diffs above \n========================================="; fi; \ ${RM} -f ex10_seqaijperm.tmp runex10_mpiaijperm: -@${MPIEXEC} -n 2 ./ex10 -f0 ${DATAFILESPATH}/matrices/small -ksp_monitor_short -ksp_view -mat_view ascii::ascii_info -mat_type mpiaijperm > ex10_mpiaijperm.tmp 2>&1; \ if (${DIFF} output/ex10_mpicsrperm.out ex10_mpiaijperm.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex10_mpiaijperm, diffs above \n========================================="; fi; \ ${RM} -f ex10_mpiaijperm.tmp runex10_aijcusparse: -@${MPIEXEC} -n 1 ./ex10 -f0 ${DATAFILESPATH}/matrices/medium -ksp_monitor_short -ksp_view -mat_view ascii::ascii_info -mat_type aijcusparse > ex10_aijcusparse.tmp 2>&1; \ if (${DIFF} output/ex10_aijcusparse.out ex10_aijcusparse.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex10_aijcusparse, diffs above \n========================================="; fi; \ ${RM} -f ex10_aijcusparse.tmp runex11: -@${MPIEXEC} -n 1 ./ex11 -n 6 -norandom -pc_type none -ksp_monitor_short -ksp_gmres_cgs_refinement_type refine_always > ex11_1.tmp 2>&1; \ if (${DIFF} output/ex11_1.out ex11_1.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex11_1, diffs above \n========================================="; fi; \ ${RM} -f ex11_1.tmp runex11f: -@${MPIEXEC} -n 1 ./ex11f -n 6 -norandom -pc_type none -ksp_monitor_short -ksp_gmres_cgs_refinement_type refine_always > ex11f_1.tmp 2>&1; \ if (${DIFF} output/ex11f_1.out ex11f_1.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex11f_1, diffs above \n========================================="; fi; \ ${RM} -f ex11f_1.tmp runex12: -@${MPIEXEC} -n 1 ./ex12 -ksp_gmres_cgs_refinement_type refine_always > ex12_1.tmp 2>&1; \ if (${DIFF} output/ex12_1.out ex12_1.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex12_1, diffs above \n========================================="; fi; \ ${RM} -f ex12_1.tmp runex13: -@${MPIEXEC} -n 1 ./ex13 -m 19 -n 20 -ksp_gmres_cgs_refinement_type refine_always > ex13_1.tmp 2>&1; \ if (${DIFF} output/ex13_1.out ex13_1.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex13_1, diffs above \n========================================="; fi; \ ${RM} -f ex13_1.tmp runex13f90: -@${MPIEXEC} -n 1 ./ex13f90 -m 19 -n 20 -ksp_gmres_cgs_refinement_type refine_always > ex13f90_1.tmp 2>&1; \ if (${DIFF} output/ex13f90_1.out ex13f90_1.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex13f90_1, diffs above \n========================================="; fi; \ ${RM} -f ex13f90_1.tmp runex14f: -@${MPIEXEC} -n 1 ./ex14f -no_output -ksp_gmres_cgs_refinement_type refine_always > ex14_1.tmp 2>&1; \ if (${DIFF} output/ex14_1.out ex14_1.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex14f_1, diffs above \n========================================="; fi; \ ${RM} -f ex14_1.tmp runex15: -@${MPIEXEC} -n 2 ./ex15 -ksp_view -user_defined_pc -ksp_gmres_cgs_refinement_type refine_always > ex15_1.tmp 2>&1; \ if (${DIFF} output/ex15_1.out ex15_1.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex15_1, diffs above \n========================================="; fi; \ ${RM} -f ex15_1.tmp runex15f: -@${MPIEXEC} -n 2 ./ex15f -ksp_view -user_defined_pc -ksp_gmres_cgs_refinement_type refine_always > ex15f_1.tmp 2>&1; \ if (${DIFF} output/ex15f_1.out ex15f_1.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex15f_1, diffs above \n========================================="; fi; \ ${RM} -f ex15f_1.tmp runex16: -@${MPIEXEC} -n 2 ./ex16 -ntimes 4 -ksp_gmres_cgs_refinement_type refine_always > ex16_1.tmp 2>&1; \ if (${DIFF} output/ex16_1.out ex16_1.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex16_1, diffs above \n========================================="; fi; \ ${RM} -f ex16_1.tmp runex18: -@${MPIEXEC} -n 3 ./ex18 -m 39 -n 18 -ksp_monitor_short -permute nd > ex18_1.tmp 2>&1; \ ${DIFF} output/ex18_1.out ex18_1.tmp || printf "${PWD}\nPossible problem with with ex18_1, diffs above \n========================================="; \ ${RM} -f ex18_1.tmp runex18_2: -@${MPIEXEC} -n 3 ./ex18 -m 39 -n 18 -ksp_monitor_short -permute rcm > ex18_2.tmp 2>&1; \ ${DIFF} output/ex18_2.out ex18_2.tmp || printf "${PWD}\nPossible problem with with ex18_2, diffs above \n========================================="; \ ${RM} -f ex18_2.tmp runex21f: -@${MPIEXEC} -n 1 ./ex21f > ex21f_1.tmp 2>&1; \ if (${DIFF} output/ex21f_1.out ex21f_1.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex21f_1, diffs above \n========================================="; fi; \ ${RM} -f ex21f_1.tmp runex22f: -@${MPIEXEC} -n 1 ./ex22f -pc_mg_type full -ksp_monitor_short -mg_levels_ksp_monitor_short -mg_levels_ksp_norm_type preconditioned -pc_type mg -da_refine 2 -ksp_type fgmres > ex22_1.tmp 2>&1; \ if (${DIFF} output/ex22_1.out ex22_1.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex22f_1, diffs above \n========================================="; fi; \ ${RM} -f ex22_1.tmp runex23: -@${MPIEXEC} -n 1 ./ex23 -ksp_monitor_short -ksp_gmres_cgs_refinement_type refine_always > ex23_1.tmp 2>&1; \ if (${DIFF} output/ex23_1.out ex23_1.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex23_1, diffs above \n========================================="; fi; \ ${RM} -f ex23_1.tmp runex23_2: -@${MPIEXEC} -n 3 ./ex23 -ksp_monitor_short -ksp_gmres_cgs_refinement_type refine_always > ex23_2.tmp 2>&1; \ if (${DIFF} output/ex23_2.out ex23_2.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex23_2, diffs above \n========================================="; fi; \ ${RM} -f ex23_2.tmp runex25: -@${MPIEXEC} -n 1 ./ex25 -pc_type mg -ksp_type fgmres -da_refine 2 -ksp_monitor_short -mg_levels_ksp_monitor_short -mg_levels_ksp_norm_type unpreconditioned -ksp_view -pc_mg_type full > ex25_1.tmp 2>&1; \ if (${DIFF} output/ex25_1.out ex25_1.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex25_1, diffs above \n========================================="; fi; \ ${RM} -f ex25_1.tmp runex25_2: -@${MPIEXEC} -n 2 ./ex25 -pc_type mg -ksp_type fgmres -da_refine 2 -ksp_monitor_short -mg_levels_ksp_monitor_short -mg_levels_ksp_norm_type unpreconditioned -ksp_view -pc_mg_type full > ex25_2.tmp 2>&1; \ if (${DIFF} output/ex25_2.out ex25_2.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex25_2, diffs above \n========================================="; fi; \ ${RM} -f ex25_2.tmp runex28: -@${MPIEXEC} -n 1 ./ex28 -ksp_monitor_short -pc_type mg -pc_mg_type full -ksp_type fgmres -da_refine 2 -mg_levels_ksp_type gmres -mg_levels_ksp_max_it 1 -mg_levels_pc_type ilu > ex28_1.tmp 2>&1; \ if (${DIFF} output/ex28_1.out ex28_1.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex28_1, diffs above \n========================================="; fi; \ ${RM} -f ex28_1.tmp runex29: -@${MPIEXEC} -n 1 ./ex29 -pc_type mg -pc_mg_type full -ksp_type fgmres -ksp_monitor_short -da_refine 8 > ex29_1.tmp 2>&1; \ if (${DIFF} output/ex29_1.out ex29_1.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex29_1, diffs above \n========================================="; fi; \ ${RM} -f ex29_1.tmp runex29_2: -@${MPIEXEC} -n 1 ./ex29 -bc_type neumann -pc_type mg -pc_mg_type full -ksp_type fgmres -ksp_monitor_short -da_refine 8 -mg_levels_pc_factor_shift_type nonzero -mg_coarse_pc_factor_shift_type nonzero > ex29_2.tmp 2>&1; \ if (${DIFF} output/ex29_2.out ex29_2.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex29_2, diffs above \n========================================="; fi; \ ${RM} -f ex29_2.tmp runex30: -@${MPIEXEC} -n 1 ./ex30 > ex30_1.tmp 2>&1; \ if (${DIFF} output/ex30_1.out ex30_1.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex30_1, diffs above \n========================================="; fi; \ ${RM} -f ex30_1.tmp runex32: -@${MPIEXEC} -n 1 ./ex32 -pc_type mg -pc_mg_type full -ksp_type fgmres -ksp_monitor_short -pc_mg_levels 3 -mg_coarse_pc_factor_shift_type nonzero > ex32_1.tmp 2>&1; \ if (${DIFF} output/ex32_1.out ex32_1.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex32_1, diffs above \n========================================="; fi; \ ${RM} -f ex32_1.tmp runex34: -@${MPIEXEC} -n 1 ./ex34 -pc_type mg -pc_mg_type full -ksp_type fgmres -ksp_monitor_short -pc_mg_levels 3 -mg_coarse_pc_factor_shift_type nonzero -ksp_view > ex34_1.tmp 2>&1; \ if (${DIFF} output/ex34_1.out ex34_1.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex34_1, diffs above \n========================================="; fi; \ ${RM} -f ex34_1.tmp runex38: -@${MPIEXEC} -n 1 ./ex38 -ksp_monitor_short > ex38_1.tmp 2>&1; \ if (${DIFF} output/ex38_1.out ex38_1.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex38_1, diffs above \n========================================="; fi; \ ${RM} -f ex38_1.tmp runex39: -@${MPIEXEC} -n 1 ./ex39 -mat_no_inode -ksp_monitor_short > ex39_1.tmp 2>&1; \ if (${DIFF} output/ex39_1.out ex39_1.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex39_1, diffs above \n========================================="; fi; \ ${RM} -f ex39_1.tmp runex40: -@${MPIEXEC} -n 1 ./ex40 -mat_no_inode -ksp_monitor_short > ex40_1.tmp 2>&1; \ if (${DIFF} output/ex40_1.out ex40_1.tmp) then true; \ else echo ${PWD} ; echo "Possible problem with with ex40_1, diffs above \n========================================="; fi; \ ${RM} -f ex40_1.tmp runex43: -@${MPIEXEC} -n 1 ./ex43 -stokes_ksp_type fgmres -stokes_ksp_rtol 1e-8 -stokes_pc_type fieldsplit -stokes_pc_fieldsplit_block_size 3 -stokes_pc_fieldsplit_type SYMMETRIC_MULTIPLICATIVE -stokes_pc_fieldsplit_0_fields 0,1 -stokes_pc_fieldsplit_1_fields 2 -stokes_fieldsplit_0_ksp_type preonly -stokes_fieldsplit_0_pc_type lu -stokes_fieldsplit_1_ksp_type preonly -stokes_fieldsplit_1_pc_type jacobi -c_str 0 -solcx_eta0 1.0 -solcx_eta1 1.0e6 -solcx_xc 0.5 -solcx_nz 2 -mx 20 -my 20 -stokes_ksp_monitor_short > ex43_1.tmp 2>&1; \ ${DIFF} output/ex43_1.out ex43_1.tmp || echo ${PWD} "\nPossible problem with with ex43_1, diffs above \n========================================="; \ ${RM} -f ex43_1.tmp runex43_2: -@${MPIEXEC} -n 1 ./ex43 -stokes_ksp_type fgmres -stokes_ksp_rtol 1e-8 -stokes_pc_type fieldsplit -stokes_pc_fieldsplit_block_size 3 -stokes_pc_fieldsplit_type SYMMETRIC_MULTIPLICATIVE -stokes_fieldsplit_u_ksp_type preonly -stokes_fieldsplit_u_pc_type lu -stokes_fieldsplit_p_ksp_type preonly -stokes_fieldsplit_p_pc_type jacobi -c_str 0 -solcx_eta0 1.0 -solcx_eta1 1.0e6 -solcx_xc 0.5 -solcx_nz 2 -mx 20 -my 20 -stokes_ksp_monitor_short > ex43_2.tmp 2>&1; \ ${DIFF} output/ex43_1.out ex43_2.tmp || echo ${PWD} "\nPossible problem with with ex43_2, diffs above \n========================================="; \ ${RM} -f ex43_2.tmp runex43_3: -@${MPIEXEC} -n 4 ./ex43 -stokes_ksp_type gcr -stokes_ksp_gcr_restart 60 -stokes_ksp_norm_type unpreconditioned -stokes_ksp_rtol 1e-8 -c_str 3 -sinker_eta0 1.0 -sinker_eta1 100 -sinker_dx 0.4 -sinker_dy 0.3 -mx 128 -my 128 -stokes_ksp_monitor_short -stokes_pc_type mg -stokes_mg_levels_pc_type fieldsplit -stokes_pc_mg_galerkin -stokes_mg_levels_pc_fieldsplit_block_size 3 -stokes_mg_levels_pc_fieldsplit_0_fields 0,1 -stokes_mg_levels_pc_fieldsplit_1_fields 2 -stokes_mg_levels_fieldsplit_0_pc_type sor -stokes_mg_levels_fieldsplit_1_pc_type sor -stokes_mg_levels_ksp_type chebyshev -stokes_mg_levels_ksp_max_it 1 -stokes_mg_levels_ksp_chebyshev_estimate_eigenvalues 0,0.2,0,1.1 -stokes_pc_mg_levels 4 -stokes_ksp_view > ex43_3.tmp 2>&1; \ ${DIFF} output/ex43_3.out ex43_3.tmp || echo ${PWD} "\nPossible problem with with ex43_3, diffs above \n========================================="; \ ${RM} -f ex43_3.tmp runex43_bjacobi: -@${MPIEXEC} -n 4 ./ex43 -stokes_pc_type bjacobi -stokes_pc_bjacobi_blocks 2 -mat_type aij -stokes_ksp_converged_reason > ex43.tmp 2>&1; \ ${DIFF} output/ex43_bjacobi.out ex43.tmp || echo ${PWD} "\nPossible problem with with ex43_bjacobi, diffs above \n========================================="; \ ${RM} -f ex43.tmp runex43_bjacobi_baij: -@${MPIEXEC} -n 4 ./ex43 -stokes_pc_type bjacobi -stokes_pc_bjacobi_blocks 2 -mat_type baij -stokes_ksp_converged_reason > ex43.tmp 2>&1; \ ${DIFF} output/ex43_bjacobi.out ex43.tmp || echo ${PWD} "\nPossible problem with with ex43_bjacobi_baij, diffs above \n========================================="; \ ${RM} -f ex43.tmp runex45: -@${MPIEXEC} -n 4 ./ex45 -pc_type exotic -ksp_monitor_short -ksp_type fgmres -mg_levels_ksp_type gmres -mg_levels_ksp_max_it 1 -mg_levels_pc_type bjacobi > ex45_1.tmp 2>&1; \ ${DIFF} output/ex45_1.out ex45_1.tmp || echo ${PWD} "\nPossible problem with with ex45_1, diffs above \n========================================="; \ ${RM} -f ex45_1.tmp runex45_2: -@${MPIEXEC} -n 4 ./ex45 -ksp_monitor_short -da_grid_x 21 -da_grid_y 21 -da_grid_z 21 -pc_type mg -pc_mg_levels 3 -mg_levels_ksp_type richardson -mg_levels_ksp_max_it 1 -mg_levels_pc_type bjacobi > ex45_2.tmp 2>&1; \ ${DIFF} output/ex45_2.out ex45_2.tmp || echo ${PWD} "\nPossible problem with with ex45_2, diffs above \n========================================="; \ ${RM} -f ex45_2.tmp runex45f: -@${MPIEXEC} -n 4 ./ex45f -ksp_monitor_short -da_refine 5 -pc_type mg -pc_mg_levels 5 -mg_levels_ksp_type chebyshev -mg_levels_ksp_max_it 2 -mg_levels_pc_type jacobi -ksp_pc_side right > ex45f_1.tmp 2>&1; \ ${DIFF} output/ex45f_1.out ex45f_1.tmp || echo ${PWD} "\nPossible problem with with ex45f_1, diffs above \n========================================="; \ ${RM} -f ex45f_1.tmp runex49: -@${MPIEXEC} -n 1 ./ex49 -mx 20 -my 30 -elas_ksp_monitor_short -no_view -c_str 3 -sponge_E0 1 -sponge_E1 1000 -sponge_nu0 0.4 -sponge_nu1 0.2 -sponge_t 1 -sponge_w 8 > ex49_1.tmp 2>&1; \ ${DIFF} output/ex49_1.out ex49_1.tmp || echo ${PWD} "\nPossible problem with with ex49_1, diffs above \n========================================="; \ ${RM} -f ex49_1.tmp runex49_2: -@${MPIEXEC} -n 4 ./ex49 -mx 20 -my 30 -elas_ksp_monitor_short -no_view -c_str 3 -sponge_E0 1 -sponge_E1 1000 -sponge_nu0 0.4 -sponge_nu1 0.2 -sponge_t 1 -sponge_w 8 -elas_ksp_type gcr -elas_pc_type asm -elas_sub_pc_type lu > ex49_2.tmp 2>&1; \ ${DIFF} output/ex49_2.out ex49_2.tmp || echo ${PWD} "\nPossible problem with with ex49_2, diffs above \n========================================="; \ ${RM} -f ex49_2.tmp runex49_3: -@${MPIEXEC} -n 4 ./ex49 -mx 20 -my 30 -elas_ksp_monitor_short -no_view -c_str 2 -brick_E 1,10,1000,100 -brick_nu 0.4,0.2,0.3,0.1 -brick_span 3 -elas_pc_type asm -elas_sub_pc_type lu > ex49_3.tmp 2>&1; \ ${DIFF} output/ex49_3.out ex49_3.tmp || echo ${PWD} "\nPossible problem with with ex49_3, diffs above \n========================================="; \ ${RM} -f ex49_3.tmp runex49_4: -@${MPIEXEC} -n 4 ./ex49 -elas_ksp_monitor_short -elas_ksp_converged_reason -elas_ksp_type cg -elas_ksp_norm_type unpreconditioned -mx 40 -my 40 -c_str 2 -brick_E 1,1e-6,1e-2 -brick_nu .3,.2,.4 -brick_span 8 -elas_mg_levels_ksp_type chebyshev -elas_pc_type ml -elas_mg_levels_ksp_chebyshev_estimate_eigenvalues 0,0.2,0,1.1 -elas_mg_levels_pc_type pbjacobi -elas_mg_levels_ksp_max_it 2 -use_nonsymbc -elas_pc_ml_nullspace user > ex49_4.tmp 2>&1; \ ${DIFF} output/ex49_4.out ex49_4.tmp || echo ${PWD} "\nPossible problem with with ex49_4, diffs above \n========================================="; \ ${RM} -f ex49_4.tmp runex50: -@${MPIEXEC} -n 1 ./ex50 -pc_type mg -pc_mg_type full -ksp_type fgmres -ksp_monitor_short -da_refine 1 -mg_levels_pc_factor_shift_type nonzero -mg_coarse_pc_factor_shift_type nonzero -ksp_view > ex50.tmp 2>&1; \ ${DIFF} output/ex50.out ex50.tmp || echo ${PWD} "\nPossible problem with with ex50, diffs above \n========================================="; \ ${RM} -f ex50.tmp runex50_2: -@${MPIEXEC} -n 2 ./ex50 -pc_type mg -pc_mg_type full -ksp_type fgmres -ksp_monitor_short -da_refine 1 -mg_levels_sub_pc_factor_shift_type nonzero -mg_coarse_pc_type redundant -mg_coarse_redundant_pc_type svd -ksp_view > ex50_2.tmp 2>&1; \ ${DIFF} output/ex50_2.out ex50_2.tmp || echo ${PWD} "\nPossible problem with with ex50_2, diffs above \n========================================="; \ ${RM} -f ex50_2.tmp runex52_mumps: -@${MPIEXEC} -n 1 ./ex52 -use_mumps_lu > ex52.tmp 2>&1; \ ${DIFF} output/ex52_1.out ex52.tmp || echo ${PWD} "\nPossible problem with with ex52_mumps, diffs above \n========================================="; \ ${RM} -f ex52.tmp runex52_mumps_2: -@${MPIEXEC} -n 1 ./ex52 -use_mumps_ch > ex52.tmp 2>&1; \ ${DIFF} output/ex52_1.out ex52.tmp || echo ${PWD} "\nPossible problem with with ex52_mumps_2, diffs above \n========================================="; \ ${RM} -f ex52.tmp runex52_superlu: -@${MPIEXEC} -n 1 ./ex52 -use_superlu_ilu > ex52.tmp 2>&1; \ ${DIFF} output/ex52_1.out ex52.tmp || echo ${PWD} "\nPossible problem with with ex52, diffs above \n========================================="; \ ${RM} -f ex52.tmp runex53: -@${MPIEXEC} -n 1 ./ex53 > ex53.tmp 2>&1; \ ${DIFF} output/ex53.out ex53.tmp || echo ${PWD} "\nPossible problem with with ex53, diffs above \n========================================="; \ ${RM} -f ex53.tmp runex53_2: -@${MPIEXEC} -n 2 ./ex53 > ex53.tmp 2>&1; \ ${DIFF} output/ex53.out ex53.tmp || echo ${PWD} "\nPossible problem with with ex53_2, diffs above \n========================================="; \ ${RM} -f ex53.tmp runex54: -@${MPIEXEC} -n 4 ./ex54 -ne 109 -alpha 1.e-3 -ksp_monitor_short -ksp_type cg -pc_gamg_type geo -ksp_converged_reason -pc_gamg_coarse_eq_limit 80 -mg_levels_ksp_type chebyshev -mg_levels_ksp_chebyshev_estimate_eigenvalues 0,0.05,0,1.05 -mg_levels_pc_type jacobi > ex54.tmp 2>&1; \ ${DIFF} output/ex54_0.out ex54.tmp || echo ${PWD} "\nPossible problem with with ex54_0, diffs above \n========================================="; \ ${RM} -f ex54.tmp runex54_SA: -@${MPIEXEC} -n 4 ./ex54 -ne 109 -alpha 1.e-3 -ksp_monitor_short -ksp_type cg -pc_gamg_type agg -pc_gamg_agg_nsmooths 1 -ksp_converged_reason -pc_gamg_coarse_eq_limit 80 -mg_levels_ksp_type chebyshev -mg_levels_ksp_chebyshev_estimate_eigenvalues 0,0.05,0,1.05 -mg_levels_pc_type jacobi > ex54_sa.tmp 2>&1; \ ${DIFF} output/ex54_1.out ex54_sa.tmp || echo ${PWD} "\nPossible problem with with ex54_1, diffs above \n======================================"; \ ${RM} -f ex54_sa.tmp runex54f: -@${MPIEXEC} -n 4 ./ex54f -ne 59 -theta 30.0 -epsilon 1.e-1 -ksp_monitor_short -ksp_type cg -pc_type gamg -pc_gamg_type agg -pc_gamg_agg_nsmooths 1 -ksp_converged_reason -pc_gamg_coarse_eq_limit 80 -blob_center 0.,0. -mat_coarsen_type hem -pc_gamg_square_graph false > ex54f.tmp 2>&1; \ ${DIFF} output/ex54f.out ex54f.tmp || echo ${PWD} "\nPossible problem with with ex54f, diffs above \n======================================"; \ ${RM} -f ex54f.tmp runex55: -@${MPIEXEC} -n 4 ./ex55 -ne 29 -alpha 1.e-3 -ksp_monitor_short -ksp_type cg -pc_gamg_type geo -use_coordinates -ksp_converged_reason -pc_gamg_coarse_eq_limit 80 -mg_levels_ksp_type chebyshev -mg_levels_ksp_chebyshev_estimate_eigenvalues 0,0.05,0,1.05 -mg_levels_pc_type jacobi > ex55.tmp 2>&1; \ ${DIFF} output/ex55_0.out ex55.tmp || echo ${PWD} "\nPossible problem with with ex55_0, diffs above \n========================================="; \ ${RM} -f ex55.tmp runex55_SA: -@${MPIEXEC} -n 4 ./ex55 -ne 29 -alpha 1.e-3 -ksp_monitor_short -ksp_type cg -pc_gamg_type agg -pc_gamg_agg_nsmooths 1 -use_coordinates -ksp_converged_reason -pc_gamg_coarse_eq_limit 80 -mg_levels_ksp_type chebyshev -mg_levels_ksp_chebyshev_estimate_eigenvalues 0,0.05,0,1.05 -mg_levels_pc_type jacobi > ex55_sa.tmp 2>&1; \ ${DIFF} output/ex55_sa.out ex55_sa.tmp || echo ${PWD} "\nPossible problem with with ex55_SA, diffs above \n========================================="; \ ${RM} -f ex55_sa.tmp runex55_NC: -@${MPIEXEC} -n 4 ./ex55 -ne 29 -alpha 1.e-3 -ksp_monitor_short -ksp_type cg -pc_gamg_type agg -pc_gamg_agg_nsmooths 1 -ksp_converged_reason -pc_gamg_coarse_eq_limit 80 -mg_levels_ksp_type chebyshev -mg_levels_ksp_chebyshev_estimate_eigenvalues 0,0.05,0,1.05 -mg_levels_pc_type jacobi > ex55_nc.tmp 2>&1; \ ${DIFF} output/ex55_NC.out ex55_nc.tmp || echo ${PWD} "\nPossible problem with with ex55_NC, diffs above \n======================================"; \ ${RM} -f ex55_nc.tmp runex56: -@${MPIEXEC} -n 8 ./ex56 -ne 19 -alpha 1.e-3 -ksp_monitor_short -ksp_type cg -ksp_max_it 50 -pc_gamg_type agg -pc_gamg_agg_nsmooths 1 -ksp_converged_reason -pc_gamg_coarse_eq_limit 80 -mg_levels_ksp_type chebyshev -mg_levels_ksp_chebyshev_estimate_eigenvalues 0,0.05,0,1.05 -mg_levels_pc_type sor -pc_gamg_reuse_interpolation true -two_solves > ex56.tmp 2>&1; \ ${DIFF} output/ex56_0.out ex56.tmp || echo ${PWD} "\nPossible problem with with ex56_0, diffs above \n========================================="; \ ${RM} -f ex56.tmp runex56_ml: -@${MPIEXEC} -n 8 ./ex56 -ne 19 -alpha 1.e-3 -ksp_monitor_short -ksp_type cg -ksp_max_it 50 -pc_type ml -ksp_converged_reason -mg_levels_ksp_type chebyshev -mg_levels_ksp_chebyshev_estimate_eigenvalues 0,0.05,0,1.05 -mg_levels_pc_type jacobi > ex56.tmp 2>&1; \ ${DIFF} output/ex56_ml.out ex56.tmp || echo ${PWD} "\nPossible problem with with ex56_2, diffs above \n========================================="; \ ${RM} -f ex56.tmp runex58: -@${MPIEXEC} -n 1 ./ex58 -mat_type aij > ex58.tmp 2>&1; \ ${DIFF} output/ex58.out ex58.tmp || echo ${PWD} "\nPossible problem with with ex58, diffs above \n========================================="; \ ${RM} -f ex58.tmp runex58_baij: -@${MPIEXEC} -n 1 ./ex58 -mat_type baij > ex58.tmp 2>&1; \ ${DIFF} output/ex58.out ex58.tmp || echo ${PWD} "\nPossible problem with with ex58_baij, diffs above \n========================================="; \ ${RM} -f ex58.tmp runex58_sbaij: -@${MPIEXEC} -n 1 ./ex58 -mat_type sbaij > ex58.tmp 2>&1; \ ${DIFF} output/ex58.out ex58.tmp || echo ${PWD} "\nPossible problem with with ex58_sbaij, diffs above \n========================================="; \ ${RM} -f ex58.tmp TESTEXAMPLES_C = ex1.PETSc runex1 runex1_2 runex1_3 ex1.rm ex2.PETSc runex2 runex2_2 runex2_3 \ runex2_4 runex2_bjacobi runex2_bjacobi_2 runex2_bjacobi_3 runex2_specest_1 runex2_specest_2 \ runex2_chebyest_1 runex2_chebyest_2 runex2_chebyest_3 runex2_chebyest_4 runex2_fbcgs runex2_fbcgs_2 ex2.rm \ ex7.PETSc runex7 ex7.rm ex5.PETSc runex5 runex5_2 ex5.rm \ ex8g.PETSc runex8g_1 runex8g_2 runex8g_3 ex8g.rm \ ex9.PETSc runex9 ex9.rm ex12.PETSc runex12 ex12.rm ex13.PETSc runex13 ex13.rm \ ex15.PETSc runex15 ex15.rm ex16.PETSc runex16 ex16.rm \ ex23.PETSc runex23 runex23_2 ex23.rm ex25.PETSc runex25 runex25_2 ex25.rm \ ex27.PETSc ex27.rm ex28.PETSc ex28.rm ex29.PETSc ex29.rm \ ex31.PETSc ex31.rm ex32.PETSc runex32 ex32.rm ex34.PETSc runex34 ex34.rm ex38.PETSc runex38 ex38.rm \ ex43.PETSc runex43 runex43_2 runex43_3 runex43_bjacobi runex43_bjacobi_baij ex43.rm \ ex45.PETSc runex45 runex45_2 ex45.rm \ ex49.PETSc runex49 runex49_2 runex49_3 ex49.rm ex53.PETSc runex53 ex53.rm ex55.PETSc runex55_SA ex55.rm\ ex58.PETSc runex58 runex58_baij runex58_sbaij ex58.rm TESTEXAMPLES_C_X = ex2.PETSc runex2_5 ex2.rm ex5.PETSc runex5_5 ex5.rm ex8.PETSc ex8.rm ex28.PETSc runex28 ex28.rm TESTEXAMPLES_FORTRAN = ex1f.PETSc runex1f ex1f.rm ex2f.PETSc runex2f ex2f.rm ex6f.PETSc ex6f.rm \ ex14f.PETSc runex14f ex14f.rm ex15f.PETSc runex15f ex15f.rm ex22f.PETSc runex22f \ ex22f.rm ex21f.PETSc runex21f ex21f.rm ex45f.PETSc runex45f ex45f.rm TESTEXAMPLES_FORTRAN_MPIUNI = ex1f.PETSc runex1f ex1f.rm ex6f.PETSc runex6f ex6f.rm TESTEXAMPLES_C_X_MPIUNI = ex1.PETSc runex1 runex1_2 runex1_3 ex1.rm ex2.PETSc runex2 runex2_3 ex2.rm \ ex7.PETSc ex7.rm ex5.PETSc ex5.rm ex9.PETSc runex9 ex9.rm \ ex23.PETSc runex23 ex23.rm TESTEXAMPLES_C_COMPLEX = ex10.PETSc runex10 ex10.rm ex11.PETSc runex11 ex11.rm ex39.PETSc runex39 ex39.rm ex40.PETSc runex40 ex40.rm TESTEXAMPLES_DATAFILESPATH = ex10.PETSc runex10_2 runex10_3 runex10_4 runex10_5 runex10_6 runex10_7 runex10_8 \ runex10_9 runex10_10 runex10_19 runex10_22 runex10_23 runex10_24 runex10_25 runex10_ILU runex10_ILUBAIJ runex10_cg \ runex10_cg_singlereduction runex10_seqaijcrl runex10_mpiaijcrl runex10_seqaijperm runex10_mpiaijperm ex10.rm # even though ex10.c is -pc_mg_smoothdown na C example to run with -mat_type lusol requires a Fortran compiler, hence # we list it with the fortran examples TESTEXAMPLES_FORTRAN_NOCOMPLEX = TESTEXAMPLES_FORTRAN_COMPLEX = ex11f.PETSc runex11f ex11f.rm TESTEXAMPLES_F90 = ex13f90.PETSc runex13f90 ex13f90.rm TESTEXAMPLES_13 = ex3.PETSc ex3.rm ex14f.PETSc ex14f.rm TESTEXAMPLES_MATLAB_ENGINE = ex10.PETSc runex10_12 ex10.rm TESTEXAMPLES_17 = ex10.PETSc runex10_11 ex10.rm TESTEXAMPLES_18 = ex2.PETSc runex2_6 ex2.rm TESTEXAMPLES_SPAI = ex10.PETSc runex10_14 ex10.rm TESTEXAMPLES_HYPRE = ex10.PETSc runex10_15 runex10_16 runex10_17 runex10_18 ex10.rm TESTEXAMPLES_LUSOL = ex10.PETSc runex10_13 ex10.rm TESTEXAMPLES_MUMPS = ex10.PETSc runex10_mumps_lu_1 runex10_mumps_lu_2 runex10_mumps_lu_3 runex10_mumps_lu_4 \ runex10_mumps_cholesky_1 runex10_mumps_cholesky_2 runex10_mumps_cholesky_3 runex10_mumps_cholesky_4 runex10_mumps_redundant \ runex10_mumps_cholesky_spd_1 runex10_mumps_cholesky_spd_2 ex10.rm ex53.PETSc runex53 runex53_2 ex53.rm \ ex52.PETSc runex52_mumps runex52_mumps_2 ex52.rm TESTEXAMPLES_SUPERLU = ex10.PETSc runex10_superlu_lu_1 ex10.rm ex52.PETSc runex52_superlu ex52.rm TESTEXAMPLES_FFTW = TESTEXAMPLES_SUPERLU_DIST = ex10.PETSc runex10_superlu_dist_lu_1 runex10_superlu_dist_lu_2 runex10_superlu_dist_redundant ex10.rm TESTEXAMPLES_TXPETSCGPU = ex10.PETSc runex10_aijcusparse ex10.rm include ${PETSC_DIR}/conf/test -------------- next part -------------- rm -f Check.dat make ex45 #/work/zlwei/PETSc/petsc-dev/linux-gnu-c-nodebug/bin/mpiexec -np 1 ./ex45 -pc_type gamg -ksp_type gmres -pc_gamg_agg_nsmooths 1 -mg_levels_ksp_max_it 1 -mg_levels_ksp_type richardson -ksp_rtol 1.0e-7 -ksp_monitor #Residual norm 1.82355e+09 #L2Norm = 3.106827e+01 #/work/zlwei/PETSc/petsc-dev/linux-gnu-c-nodebug/bin/mpiexec -np 1 ./ex45 -ksp_type richardson -pc_type hypre -pc_hypre_type boomeramg -ksp_rtol 1.0e-7 -ksp_monitor #Residual norm 1.93203e+09 #L2Norm = 2.546364e+01 #/work/zlwei/PETSc/petsc-dev/linux-gnu-c-nodebug/bin/mpiexec -np 1 ./ex45 -ksp_type gmres -pc_type hypre -pc_hypre_type boomeramg -ksp_rtol 1.0e-7 -ksp_monitor #Residual norm 6.3083e+09 #L2Norm = 3.766155e+01 ################################################################################################################################################## #/work/zlwei/PETSc/petsc-dev/linux-gnu-c-nodebug/bin/mpiexec -np 1 ./ex45 -pc_type gamg -ksp_type gmres -pc_gamg_agg_nsmooths 1 -mg_levels_ksp_max_it 1 -mg_levels_ksp_type richardson -ksp_rtol 1.0e-10 -ksp_monitor #Residual norm 473554 #L2Norm = 1.573811e-02 #/work/zlwei/PETSc/petsc-dev/linux-gnu-c-nodebug/bin/mpiexec -np 1 ./ex45 -ksp_type richardson -pc_type hypre -pc_hypre_type boomeramg -ksp_rtol 1.0e-10 -ksp_monitor #Residual norm 1.39368e+06 #L2Norm = 1.846820e-02 #/work/zlwei/PETSc/petsc-dev/linux-gnu-c-nodebug/bin/mpiexec -np 1 ./ex45 -ksp_type gmres -pc_type hypre -pc_hypre_type boomeramg -ksp_rtol 1.0e-10 -ksp_monitor #Residual norm 703125 #L2Norm = 4.971447e-03 ################################################################################################################################################## #/work/zlwei/PETSc/petsc-dev/linux-gnu-c-nodebug/bin/mpiexec -np 1 ./ex45 -pc_type gamg -ksp_type gmres -pc_gamg_agg_nsmooths 1 -mg_levels_ksp_max_it 1 -mg_levels_ksp_type richardson -ksp_rtol 1.0e-15 -ksp_monitor #Residual norm 7.3617 #L2Norm = 3.444358e-07 #/work/zlwei/PETSc/petsc-dev/linux-gnu-c-nodebug/bin/mpiexec -np 1 ./ex45 -ksp_type richardson -pc_type hypre -pc_hypre_type boomeramg -ksp_rtol 1.0e-15 -ksp_monitor #Residual norm 28.3279 #L2Norm = 4.487079e-07 #/work/zlwei/PETSc/petsc-dev/linux-gnu-c-nodebug/bin/mpiexec -np 1 ./ex45 -ksp_type gmres -pc_type hypre -pc_hypre_type boomeramg -ksp_rtol 1.0e-15 -ksp_monitor #Residual norm 81.3155 #L2Norm = 5.140724e-07 -------------- next part -------------- A non-text attachment was scrubbed... Name: Non-Uniform Poisson.docx Type: application/vnd.openxmlformats-officedocument.wordprocessingml.document Size: 52683 bytes Desc: not available URL: From bsmith at mcs.anl.gov Tue Dec 10 07:53:04 2013 From: bsmith at mcs.anl.gov (Barry Smith) Date: Tue, 10 Dec 2013 07:53:04 -0600 Subject: [petsc-users] An issue of order of accuracy. In-Reply-To: <52A6B89E.3060307@gmail.com> References: <529FB9D8.8050708@gmail.com> <52A6B89E.3060307@gmail.com> Message-ID: <9B8546C2-672F-487D-A06B-6342EAA75C48@mcs.anl.gov> Alan, For non-uniform grid with the Poisson the order is no longer 2nd order. It is only for uniform grid that you get ?magic? cancelation that makes it second order. Barry On Dec 10, 2013, at 12:45 AM, Alan wrote: > Thank you, Dr. Smith. > I 'cooked up' a 4th-order polynomial and find out that the Poisson > solver with uniform Cartesian mesh is 2nd order of accuracy. Also, if > the solution is 3rd or less order polynomial, the L2Norm shows very > small round-off error, which is usually 1e-7 if the -ksp_rtol is 1-e7. > Following this idea, I re-derived the coefficients for the Poisson > equation for solver with non-uniform grid, which is attached with a > .docx file. From the equation, it indicates that at least 3rd or higher > order polynomial should be constructed for the solution to detect the > real order of accuracy. In other words, for solution of 2nd or lower > order polynomials, the L2Norm should show very small round-off error. > However, the reality is not and here is a brief description of my tests. > The test focuses on the three-dimensional Poisson solver with > non-uniform Cartesian mesh, which is modified from > /src/ksp/ksp/example/tutorial/ex45.c. For simplicity, in this test, only > x-direction has non-uniform grid, y- and z- are uniform grid. I have 3 > sets of tests, which are recorded in TestRun attached. The difference > among these 3 sets of tests are the -ksp_rtol. Each set has 3 different > commands to run the program with different ksp solver (GMRES or > richardson) and pc type (GAMG and Hypre boomeramg). > As I recorded in TestRun, while the ksp_rtol is 1e-7, the L2Norm is > fairly large. As Dr. Smith explained, tight algebraic tolerances are > needed to eliminate the algebraic error. Therefore, I changed ksp_rtol > to 10-e15. However, the L2Norm can only reach 1e-7. Compared with the > Poisson solver with uniform Cartesian mesh (which the L2Norm exhibits > 1e-7 round-off error as -ksp_rtol = 1e-7), the L2Norm from the > non-uniform Poisson solver is relatively high. Is this normal? > Moreover, the Residual norm shown for cases with ksp_rtol = 1e-15 is > around 7, 28 or even 81, which are far away from the real ksp_rtol > imported. I monitored the ksp iterations with ksp_monitor and found out > that these solvers usually iterate around 20 times. Why wouldn't them > continue to iterate until the 1e-15 is achieved? > Based on my observation, neither solver/pc works fine for this Poisson > solver with non-uniform mesh. Is there any other option to made some > improvements? > At last, for the Poisson solver with non-uniform Cartesian grid, is > there any better way to prove its validity? > > sincerely appreciate, > Alan > >> Alan, >> >> I changed your initial grid size to 10 to get faster solve times and get what is below. Note that your ?exact solution? is quadratic, since the method is second order this means that not only is the ?exact solution? an exact solution to the PDE, it is also an exact solution to the algebraic equations (for any grid size) hence the L2Norm of the error is only due to the round off of the algebraic solution, not due to any discretization error. In general, when testing for discretization error you always need to ?cook up? an exact solution that is not completely represented in the approximation space. You also need to use a really tight algebraic tolerance to eliminate the algebraic error from the computation >> >> >> >> ~/Src/petsc/test-dir master $ ./ex45 -pc_type mg -ksp_rtol 1e-12 >> mx = 10, my = 10, mz =10, mm = 1, nn = 1, pp = 1 >> Residual norm 1.81616e-12 >> L2Norm = 1.107359e-12 >> Total Time Elapsed: 0.048599 >> ~/Src/petsc/test-dir master $ ./ex45 -pc_type mg -ksp_rtol 1e-12 -da_refine 1 >> mx = 19, my = 19, mz =19, mm = 1, nn = 1, pp = 1 >> Residual norm 3.36741e-12 >> L2Norm = 1.037148e-12 >> Total Time Elapsed: 0.183398 >> ~/Src/petsc/test-dir master $ ./ex45 -pc_type mg -ksp_rtol 1e-12 -da_refine 2 >> mx = 37, my = 37, mz =37, mm = 1, nn = 1, pp = 1 >> Residual norm 1.09476e-11 >> L2Norm = 2.330658e-12 >> Total Time Elapsed: 1.180839 >> ~/Src/petsc/test-dir master $ ./ex45 -pc_type mg -ksp_rtol 1e-12 -da_refine 3 >> mx = 73, my = 73, mz =73, mm = 1, nn = 1, pp = 1 >> Residual norm 3.19809e-11 >> L2Norm = 2.278763e-12 >> Total Time Elapsed: 10.819450 >> ~/Src/petsc/test-dir master $ ./ex45 -pc_type mg -da_refine 3 >> mx = 73, my = 73, mz =73, mm = 1, nn = 1, pp = 1 >> Residual norm 0.000103197 >> L2Norm = 1.011806e-05 >> Total Time Elapsed: 7.250106 >> >> >> On Dec 4, 2013, at 5:25 PM, Alan Z. Wei wrote: >> >>> Dear all, >>> I hope you had a great Thanksgiving. >>> Currently, I tested the order of accuracy for /src/ksp/ksp/tutorial/example/ex45.c. Since the 2nd-order discretization is used in this program and ksp solver is converged to 10^-7, I expected that the solution should provides a 2nd-order in L2 norm. However, as I tested (even with a Laplace equation), the L2 norm slope is much less than 2. Sometime, if the grid size is reduced, the L2 norm increases. Could anyone help me about this issue, please? >>> >>> Here is the L2 norm outputted: >>> >>> Grid L2 norm (10^-8) >>> 0.05 4.36242 >>> 0.025 2.20794 >>> 0.0125 7.02749 >>> 0.00625 12.64 >>> Once the grid size is reduced to half, the number of the grid will be multiplied by 8 in order to keep the same size of the computational domain. >>> The code is also attached. It is from ex45.c with very little modifications. >>> >>> thanks in advance, >>> Alan >>> >>> > > From zhenglun.wei at gmail.com Tue Dec 10 10:33:16 2013 From: zhenglun.wei at gmail.com (Alan Z. Wei) Date: Tue, 10 Dec 2013 10:33:16 -0600 Subject: [petsc-users] An issue of order of accuracy. In-Reply-To: <9B8546C2-672F-487D-A06B-6342EAA75C48@mcs.anl.gov> References: <529FB9D8.8050708@gmail.com> <52A6B89E.3060307@gmail.com> <9B8546C2-672F-487D-A06B-6342EAA75C48@mcs.anl.gov> Message-ID: <52A7424C.904@gmail.com> Dear Dr. Smith, Indeed, non-uniform grid with the Poisson has only 1st order; my docx. file shows that also. However, it should recover to 'exact solution' with only small round-off error if the 'exact solution' is a polynomial function of degree 2 or less. In my tests, I constructed a quadratic function and the round-off error is quite large. I wonder if my usage of ksp solver or pc has some problems. thanks, Alan > > Alan, > > For non-uniform grid with the Poisson the order is no longer 2nd order. It is only for uniform grid that you get ?magic? cancelation that makes it second order. > > Barry > > On Dec 10, 2013, at 12:45 AM, Alan wrote: > >> Thank you, Dr. Smith. >> I 'cooked up' a 4th-order polynomial and find out that the Poisson >> solver with uniform Cartesian mesh is 2nd order of accuracy. Also, if >> the solution is 3rd or less order polynomial, the L2Norm shows very >> small round-off error, which is usually 1e-7 if the -ksp_rtol is 1-e7. >> Following this idea, I re-derived the coefficients for the Poisson >> equation for solver with non-uniform grid, which is attached with a >> .docx file. From the equation, it indicates that at least 3rd or higher >> order polynomial should be constructed for the solution to detect the >> real order of accuracy. In other words, for solution of 2nd or lower >> order polynomials, the L2Norm should show very small round-off error. >> However, the reality is not and here is a brief description of my tests. >> The test focuses on the three-dimensional Poisson solver with >> non-uniform Cartesian mesh, which is modified from >> /src/ksp/ksp/example/tutorial/ex45.c. For simplicity, in this test, only >> x-direction has non-uniform grid, y- and z- are uniform grid. I have 3 >> sets of tests, which are recorded in TestRun attached. The difference >> among these 3 sets of tests are the -ksp_rtol. Each set has 3 different >> commands to run the program with different ksp solver (GMRES or >> richardson) and pc type (GAMG and Hypre boomeramg). >> As I recorded in TestRun, while the ksp_rtol is 1e-7, the L2Norm is >> fairly large. As Dr. Smith explained, tight algebraic tolerances are >> needed to eliminate the algebraic error. Therefore, I changed ksp_rtol >> to 10-e15. However, the L2Norm can only reach 1e-7. Compared with the >> Poisson solver with uniform Cartesian mesh (which the L2Norm exhibits >> 1e-7 round-off error as -ksp_rtol = 1e-7), the L2Norm from the >> non-uniform Poisson solver is relatively high. Is this normal? >> Moreover, the Residual norm shown for cases with ksp_rtol = 1e-15 is >> around 7, 28 or even 81, which are far away from the real ksp_rtol >> imported. I monitored the ksp iterations with ksp_monitor and found out >> that these solvers usually iterate around 20 times. Why wouldn't them >> continue to iterate until the 1e-15 is achieved? >> Based on my observation, neither solver/pc works fine for this Poisson >> solver with non-uniform mesh. Is there any other option to made some >> improvements? >> At last, for the Poisson solver with non-uniform Cartesian grid, is >> there any better way to prove its validity? >> >> sincerely appreciate, >> Alan >> >>> Alan, >>> >>> I changed your initial grid size to 10 to get faster solve times and get what is below. Note that your ?exact solution? is quadratic, since the method is second order this means that not only is the ?exact solution? an exact solution to the PDE, it is also an exact solution to the algebraic equations (for any grid size) hence the L2Norm of the error is only due to the round off of the algebraic solution, not due to any discretization error. In general, when testing for discretization error you always need to ?cook up? an exact solution that is not completely represented in the approximation space. You also need to use a really tight algebraic tolerance to eliminate the algebraic error from the computation >>> >>> >>> >>> ~/Src/petsc/test-dir master $ ./ex45 -pc_type mg -ksp_rtol 1e-12 >>> mx = 10, my = 10, mz =10, mm = 1, nn = 1, pp = 1 >>> Residual norm 1.81616e-12 >>> L2Norm = 1.107359e-12 >>> Total Time Elapsed: 0.048599 >>> ~/Src/petsc/test-dir master $ ./ex45 -pc_type mg -ksp_rtol 1e-12 -da_refine 1 >>> mx = 19, my = 19, mz =19, mm = 1, nn = 1, pp = 1 >>> Residual norm 3.36741e-12 >>> L2Norm = 1.037148e-12 >>> Total Time Elapsed: 0.183398 >>> ~/Src/petsc/test-dir master $ ./ex45 -pc_type mg -ksp_rtol 1e-12 -da_refine 2 >>> mx = 37, my = 37, mz =37, mm = 1, nn = 1, pp = 1 >>> Residual norm 1.09476e-11 >>> L2Norm = 2.330658e-12 >>> Total Time Elapsed: 1.180839 >>> ~/Src/petsc/test-dir master $ ./ex45 -pc_type mg -ksp_rtol 1e-12 -da_refine 3 >>> mx = 73, my = 73, mz =73, mm = 1, nn = 1, pp = 1 >>> Residual norm 3.19809e-11 >>> L2Norm = 2.278763e-12 >>> Total Time Elapsed: 10.819450 >>> ~/Src/petsc/test-dir master $ ./ex45 -pc_type mg -da_refine 3 >>> mx = 73, my = 73, mz =73, mm = 1, nn = 1, pp = 1 >>> Residual norm 0.000103197 >>> L2Norm = 1.011806e-05 >>> Total Time Elapsed: 7.250106 >>> >>> >>> On Dec 4, 2013, at 5:25 PM, Alan Z. Wei wrote: >>> >>>> Dear all, >>>> I hope you had a great Thanksgiving. >>>> Currently, I tested the order of accuracy for /src/ksp/ksp/tutorial/example/ex45.c. Since the 2nd-order discretization is used in this program and ksp solver is converged to 10^-7, I expected that the solution should provides a 2nd-order in L2 norm. However, as I tested (even with a Laplace equation), the L2 norm slope is much less than 2. Sometime, if the grid size is reduced, the L2 norm increases. Could anyone help me about this issue, please? >>>> >>>> Here is the L2 norm outputted: >>>> >>>> Grid L2 norm (10^-8) >>>> 0.05 4.36242 >>>> 0.025 2.20794 >>>> 0.0125 7.02749 >>>> 0.00625 12.64 >>>> Once the grid size is reduced to half, the number of the grid will be multiplied by 8 in order to keep the same size of the computational domain. >>>> The code is also attached. It is from ex45.c with very little modifications. >>>> >>>> thanks in advance, >>>> Alan >>>> >>>> >> From bsmith at mcs.anl.gov Tue Dec 10 11:14:14 2013 From: bsmith at mcs.anl.gov (Barry Smith) Date: Tue, 10 Dec 2013 11:14:14 -0600 Subject: [petsc-users] An issue of order of accuracy. In-Reply-To: <52A7424C.904@gmail.com> References: <529FB9D8.8050708@gmail.com> <52A6B89E.3060307@gmail.com> <9B8546C2-672F-487D-A06B-6342EAA75C48@mcs.anl.gov> <52A7424C.904@gmail.com> Message-ID: <41722861-43B7-44ED-AF02-8A25A0E770ED@mcs.anl.gov> On Dec 10, 2013, at 10:33 AM, Alan Z. Wei wrote: > Dear Dr. Smith, > Indeed, non-uniform grid with the Poisson has only 1st order; my docx. > file shows that also. However, it should recover to 'exact solution' > with only small round-off error if the 'exact solution' is a polynomial > function of degree 2 or less. In my tests, I constructed a quadratic > function and the round-off error is quite large. I wonder if my usage of > ksp solver or pc has some problems. Are you sure that it reproduces a quadratic exactly? With pencil and paper apply the differencing to a quadratic function what does it generate for the right hand side? Barry > > thanks, > Alan >> >> Alan, >> >> For non-uniform grid with the Poisson the order is no longer 2nd order. It is only for uniform grid that you get ?magic? cancelation that makes it second order. >> >> Barry >> >> On Dec 10, 2013, at 12:45 AM, Alan wrote: >> >>> Thank you, Dr. Smith. >>> I 'cooked up' a 4th-order polynomial and find out that the Poisson >>> solver with uniform Cartesian mesh is 2nd order of accuracy. Also, if >>> the solution is 3rd or less order polynomial, the L2Norm shows very >>> small round-off error, which is usually 1e-7 if the -ksp_rtol is 1-e7. >>> Following this idea, I re-derived the coefficients for the Poisson >>> equation for solver with non-uniform grid, which is attached with a >>> .docx file. From the equation, it indicates that at least 3rd or higher >>> order polynomial should be constructed for the solution to detect the >>> real order of accuracy. In other words, for solution of 2nd or lower >>> order polynomials, the L2Norm should show very small round-off error. >>> However, the reality is not and here is a brief description of my tests. >>> The test focuses on the three-dimensional Poisson solver with >>> non-uniform Cartesian mesh, which is modified from >>> /src/ksp/ksp/example/tutorial/ex45.c. For simplicity, in this test, only >>> x-direction has non-uniform grid, y- and z- are uniform grid. I have 3 >>> sets of tests, which are recorded in TestRun attached. The difference >>> among these 3 sets of tests are the -ksp_rtol. Each set has 3 different >>> commands to run the program with different ksp solver (GMRES or >>> richardson) and pc type (GAMG and Hypre boomeramg). >>> As I recorded in TestRun, while the ksp_rtol is 1e-7, the L2Norm is >>> fairly large. As Dr. Smith explained, tight algebraic tolerances are >>> needed to eliminate the algebraic error. Therefore, I changed ksp_rtol >>> to 10-e15. However, the L2Norm can only reach 1e-7. Compared with the >>> Poisson solver with uniform Cartesian mesh (which the L2Norm exhibits >>> 1e-7 round-off error as -ksp_rtol = 1e-7), the L2Norm from the >>> non-uniform Poisson solver is relatively high. Is this normal? >>> Moreover, the Residual norm shown for cases with ksp_rtol = 1e-15 is >>> around 7, 28 or even 81, which are far away from the real ksp_rtol >>> imported. I monitored the ksp iterations with ksp_monitor and found out >>> that these solvers usually iterate around 20 times. Why wouldn't them >>> continue to iterate until the 1e-15 is achieved? >>> Based on my observation, neither solver/pc works fine for this Poisson >>> solver with non-uniform mesh. Is there any other option to made some >>> improvements? >>> At last, for the Poisson solver with non-uniform Cartesian grid, is >>> there any better way to prove its validity? >>> >>> sincerely appreciate, >>> Alan >>> >>>> Alan, >>>> >>>> I changed your initial grid size to 10 to get faster solve times and get what is below. Note that your ?exact solution? is quadratic, since the method is second order this means that not only is the ?exact solution? an exact solution to the PDE, it is also an exact solution to the algebraic equations (for any grid size) hence the L2Norm of the error is only due to the round off of the algebraic solution, not due to any discretization error. In general, when testing for discretization error you always need to ?cook up? an exact solution that is not completely represented in the approximation space. You also need to use a really tight algebraic tolerance to eliminate the algebraic error from the computation >>>> >>>> >>>> >>>> ~/Src/petsc/test-dir master $ ./ex45 -pc_type mg -ksp_rtol 1e-12 >>>> mx = 10, my = 10, mz =10, mm = 1, nn = 1, pp = 1 >>>> Residual norm 1.81616e-12 >>>> L2Norm = 1.107359e-12 >>>> Total Time Elapsed: 0.048599 >>>> ~/Src/petsc/test-dir master $ ./ex45 -pc_type mg -ksp_rtol 1e-12 -da_refine 1 >>>> mx = 19, my = 19, mz =19, mm = 1, nn = 1, pp = 1 >>>> Residual norm 3.36741e-12 >>>> L2Norm = 1.037148e-12 >>>> Total Time Elapsed: 0.183398 >>>> ~/Src/petsc/test-dir master $ ./ex45 -pc_type mg -ksp_rtol 1e-12 -da_refine 2 >>>> mx = 37, my = 37, mz =37, mm = 1, nn = 1, pp = 1 >>>> Residual norm 1.09476e-11 >>>> L2Norm = 2.330658e-12 >>>> Total Time Elapsed: 1.180839 >>>> ~/Src/petsc/test-dir master $ ./ex45 -pc_type mg -ksp_rtol 1e-12 -da_refine 3 >>>> mx = 73, my = 73, mz =73, mm = 1, nn = 1, pp = 1 >>>> Residual norm 3.19809e-11 >>>> L2Norm = 2.278763e-12 >>>> Total Time Elapsed: 10.819450 >>>> ~/Src/petsc/test-dir master $ ./ex45 -pc_type mg -da_refine 3 >>>> mx = 73, my = 73, mz =73, mm = 1, nn = 1, pp = 1 >>>> Residual norm 0.000103197 >>>> L2Norm = 1.011806e-05 >>>> Total Time Elapsed: 7.250106 >>>> >>>> >>>> On Dec 4, 2013, at 5:25 PM, Alan Z. Wei wrote: >>>> >>>>> Dear all, >>>>> I hope you had a great Thanksgiving. >>>>> Currently, I tested the order of accuracy for /src/ksp/ksp/tutorial/example/ex45.c. Since the 2nd-order discretization is used in this program and ksp solver is converged to 10^-7, I expected that the solution should provides a 2nd-order in L2 norm. However, as I tested (even with a Laplace equation), the L2 norm slope is much less than 2. Sometime, if the grid size is reduced, the L2 norm increases. Could anyone help me about this issue, please? >>>>> >>>>> Here is the L2 norm outputted: >>>>> >>>>> Grid L2 norm (10^-8) >>>>> 0.05 4.36242 >>>>> 0.025 2.20794 >>>>> 0.0125 7.02749 >>>>> 0.00625 12.64 >>>>> Once the grid size is reduced to half, the number of the grid will be multiplied by 8 in order to keep the same size of the computational domain. >>>>> The code is also attached. It is from ex45.c with very little modifications. >>>>> >>>>> thanks in advance, >>>>> Alan >>>>> >>>>> >>> > From zhenglun.wei at gmail.com Tue Dec 10 12:35:04 2013 From: zhenglun.wei at gmail.com (Alan Z. Wei) Date: Tue, 10 Dec 2013 12:35:04 -0600 Subject: [petsc-users] An issue of order of accuracy. In-Reply-To: <41722861-43B7-44ED-AF02-8A25A0E770ED@mcs.anl.gov> References: <529FB9D8.8050708@gmail.com> <52A6B89E.3060307@gmail.com> <9B8546C2-672F-487D-A06B-6342EAA75C48@mcs.anl.gov> <52A7424C.904@gmail.com> <41722861-43B7-44ED-AF02-8A25A0E770ED@mcs.anl.gov> Message-ID: <52A75ED8.3090704@gmail.com> Dear Dr. Smith, I derived and found that the leading truncation error is associated with a 3rd order derivative. This indicates that a quadratic solution should reproduced as a exact solution. Here attaches a pdf file with my procedure of derivation; it matches Ferziger and Peric's results with a small assumption, which is shown at the end of the derivation. thanks, Alan > On Dec 10, 2013, at 10:33 AM, Alan Z. Wei wrote: > >> Dear Dr. Smith, >> Indeed, non-uniform grid with the Poisson has only 1st order; my docx. >> file shows that also. However, it should recover to 'exact solution' >> with only small round-off error if the 'exact solution' is a polynomial >> function of degree 2 or less. In my tests, I constructed a quadratic >> function and the round-off error is quite large. I wonder if my usage of >> ksp solver or pc has some problems. > Are you sure that it reproduces a quadratic exactly? With pencil and paper apply the differencing to a quadratic function what does it generate for the right hand side? > > Barry > >> thanks, >> Alan >>> Alan, >>> >>> For non-uniform grid with the Poisson the order is no longer 2nd order. It is only for uniform grid that you get ?magic? cancelation that makes it second order. >>> >>> Barry >>> >>> On Dec 10, 2013, at 12:45 AM, Alan wrote: >>> >>>> Thank you, Dr. Smith. >>>> I 'cooked up' a 4th-order polynomial and find out that the Poisson >>>> solver with uniform Cartesian mesh is 2nd order of accuracy. Also, if >>>> the solution is 3rd or less order polynomial, the L2Norm shows very >>>> small round-off error, which is usually 1e-7 if the -ksp_rtol is 1-e7. >>>> Following this idea, I re-derived the coefficients for the Poisson >>>> equation for solver with non-uniform grid, which is attached with a >>>> .docx file. From the equation, it indicates that at least 3rd or higher >>>> order polynomial should be constructed for the solution to detect the >>>> real order of accuracy. In other words, for solution of 2nd or lower >>>> order polynomials, the L2Norm should show very small round-off error. >>>> However, the reality is not and here is a brief description of my tests. >>>> The test focuses on the three-dimensional Poisson solver with >>>> non-uniform Cartesian mesh, which is modified from >>>> /src/ksp/ksp/example/tutorial/ex45.c. For simplicity, in this test, only >>>> x-direction has non-uniform grid, y- and z- are uniform grid. I have 3 >>>> sets of tests, which are recorded in TestRun attached. The difference >>>> among these 3 sets of tests are the -ksp_rtol. Each set has 3 different >>>> commands to run the program with different ksp solver (GMRES or >>>> richardson) and pc type (GAMG and Hypre boomeramg). >>>> As I recorded in TestRun, while the ksp_rtol is 1e-7, the L2Norm is >>>> fairly large. As Dr. Smith explained, tight algebraic tolerances are >>>> needed to eliminate the algebraic error. Therefore, I changed ksp_rtol >>>> to 10-e15. However, the L2Norm can only reach 1e-7. Compared with the >>>> Poisson solver with uniform Cartesian mesh (which the L2Norm exhibits >>>> 1e-7 round-off error as -ksp_rtol = 1e-7), the L2Norm from the >>>> non-uniform Poisson solver is relatively high. Is this normal? >>>> Moreover, the Residual norm shown for cases with ksp_rtol = 1e-15 is >>>> around 7, 28 or even 81, which are far away from the real ksp_rtol >>>> imported. I monitored the ksp iterations with ksp_monitor and found out >>>> that these solvers usually iterate around 20 times. Why wouldn't them >>>> continue to iterate until the 1e-15 is achieved? >>>> Based on my observation, neither solver/pc works fine for this Poisson >>>> solver with non-uniform mesh. Is there any other option to made some >>>> improvements? >>>> At last, for the Poisson solver with non-uniform Cartesian grid, is >>>> there any better way to prove its validity? >>>> >>>> sincerely appreciate, >>>> Alan >>>> >>>>> Alan, >>>>> >>>>> I changed your initial grid size to 10 to get faster solve times and get what is below. Note that your ?exact solution? is quadratic, since the method is second order this means that not only is the ?exact solution? an exact solution to the PDE, it is also an exact solution to the algebraic equations (for any grid size) hence the L2Norm of the error is only due to the round off of the algebraic solution, not due to any discretization error. In general, when testing for discretization error you always need to ?cook up? an exact solution that is not completely represented in the approximation space. You also need to use a really tight algebraic tolerance to eliminate the algebraic error from the computation >>>>> >>>>> >>>>> >>>>> ~/Src/petsc/test-dir master $ ./ex45 -pc_type mg -ksp_rtol 1e-12 >>>>> mx = 10, my = 10, mz =10, mm = 1, nn = 1, pp = 1 >>>>> Residual norm 1.81616e-12 >>>>> L2Norm = 1.107359e-12 >>>>> Total Time Elapsed: 0.048599 >>>>> ~/Src/petsc/test-dir master $ ./ex45 -pc_type mg -ksp_rtol 1e-12 -da_refine 1 >>>>> mx = 19, my = 19, mz =19, mm = 1, nn = 1, pp = 1 >>>>> Residual norm 3.36741e-12 >>>>> L2Norm = 1.037148e-12 >>>>> Total Time Elapsed: 0.183398 >>>>> ~/Src/petsc/test-dir master $ ./ex45 -pc_type mg -ksp_rtol 1e-12 -da_refine 2 >>>>> mx = 37, my = 37, mz =37, mm = 1, nn = 1, pp = 1 >>>>> Residual norm 1.09476e-11 >>>>> L2Norm = 2.330658e-12 >>>>> Total Time Elapsed: 1.180839 >>>>> ~/Src/petsc/test-dir master $ ./ex45 -pc_type mg -ksp_rtol 1e-12 -da_refine 3 >>>>> mx = 73, my = 73, mz =73, mm = 1, nn = 1, pp = 1 >>>>> Residual norm 3.19809e-11 >>>>> L2Norm = 2.278763e-12 >>>>> Total Time Elapsed: 10.819450 >>>>> ~/Src/petsc/test-dir master $ ./ex45 -pc_type mg -da_refine 3 >>>>> mx = 73, my = 73, mz =73, mm = 1, nn = 1, pp = 1 >>>>> Residual norm 0.000103197 >>>>> L2Norm = 1.011806e-05 >>>>> Total Time Elapsed: 7.250106 >>>>> >>>>> >>>>> On Dec 4, 2013, at 5:25 PM, Alan Z. Wei wrote: >>>>> >>>>>> Dear all, >>>>>> I hope you had a great Thanksgiving. >>>>>> Currently, I tested the order of accuracy for /src/ksp/ksp/tutorial/example/ex45.c. Since the 2nd-order discretization is used in this program and ksp solver is converged to 10^-7, I expected that the solution should provides a 2nd-order in L2 norm. However, as I tested (even with a Laplace equation), the L2 norm slope is much less than 2. Sometime, if the grid size is reduced, the L2 norm increases. Could anyone help me about this issue, please? >>>>>> >>>>>> Here is the L2 norm outputted: >>>>>> >>>>>> Grid L2 norm (10^-8) >>>>>> 0.05 4.36242 >>>>>> 0.025 2.20794 >>>>>> 0.0125 7.02749 >>>>>> 0.00625 12.64 >>>>>> Once the grid size is reduced to half, the number of the grid will be multiplied by 8 in order to keep the same size of the computational domain. >>>>>> The code is also attached. It is from ex45.c with very little modifications. >>>>>> >>>>>> thanks in advance, >>>>>> Alan >>>>>> >>>>>> >>>> -------------- next part -------------- A non-text attachment was scrubbed... Name: Non Uniform Poisson - Accuracy.pdf Type: application/x-msword Size: 129929 bytes Desc: not available URL: From lu_qin_2000 at yahoo.com Wed Dec 11 12:22:26 2013 From: lu_qin_2000 at yahoo.com (Qin Lu) Date: Wed, 11 Dec 2013 10:22:26 -0800 (PST) Subject: [petsc-users] .petscrc in Windows Message-ID: <1386786146.84754.YahooMailNeo@web160203.mail.bf1.yahoo.com> Hello, ? PETSc can read options from .petscrc file at home directory in Linux. Can it do the same thing in Windows? What is the file name and where the file should be placed? ? Many thanks for your help, Qin -------------- next part -------------- An HTML attachment was scrubbed... URL: From balay at mcs.anl.gov Wed Dec 11 12:27:50 2013 From: balay at mcs.anl.gov (Satish Balay) Date: Wed, 11 Dec 2013 12:27:50 -0600 (CST) Subject: [petsc-users] .petscrc in Windows In-Reply-To: <1386786146.84754.YahooMailNeo@web160203.mail.bf1.yahoo.com> References: <1386786146.84754.YahooMailNeo@web160203.mail.bf1.yahoo.com> Message-ID: On Wed, 11 Dec 2013, Qin Lu wrote: > Hello, > ? > PETSc can read options from .petscrc file at home directory in Linux. Can it do the same thing in Windows? What is the file name and where the file should be placed? On windows - one can emulate this by setting "HOME" env variable to the location where you have .petscrc Also one can have .petscrc [or petscrc] in the location from which you run the petsc executable. Satish From lu_qin_2000 at yahoo.com Wed Dec 11 13:26:38 2013 From: lu_qin_2000 at yahoo.com (Qin Lu) Date: Wed, 11 Dec 2013 11:26:38 -0800 (PST) Subject: [petsc-users] .petscrc in Windows In-Reply-To: References: <1386786146.84754.YahooMailNeo@web160203.mail.bf1.yahoo.com> Message-ID: <1386789998.75789.YahooMailNeo@web160202.mail.bf1.yahoo.com> Thanks, Satish! ? Qin On Wednesday, December 11, 2013 12:27 PM, Satish Balay wrote: On Wed, 11 Dec 2013, Qin Lu wrote: > Hello, > ? > PETSc can read options from .petscrc file at home directory in Linux. Can it do the same thing in Windows? What is the file name and where the file should be placed? On windows - one can emulate this by setting "HOME" env variable to the location where you have .petscrc Also one can have .petscrc [or petscrc] in the location from which you run the petsc executable. Satish -------------- next part -------------- An HTML attachment was scrubbed... URL: From lu_qin_2000 at yahoo.com Wed Dec 11 17:23:52 2013 From: lu_qin_2000 at yahoo.com (Qin Lu) Date: Wed, 11 Dec 2013 15:23:52 -0800 (PST) Subject: [petsc-users] .petscrc in Windows In-Reply-To: <1386789998.75789.YahooMailNeo@web160202.mail.bf1.yahoo.com> References: <1386786146.84754.YahooMailNeo@web160203.mail.bf1.yahoo.com> <1386789998.75789.YahooMailNeo@web160202.mail.bf1.yahoo.com> Message-ID: <1386804232.53238.YahooMailNeo@web160201.mail.bf1.yahoo.com> Satish, ? I tried both ways that you suggested in Win-7 (and I tried to put the petscrc file in either the directory of executable or the directory where I run the executable), but it did not work. Note that I have to use file name "petscrc" instead of ".petscrc" since Windows does not allow the file name start with "." ? The content of the file is very simple: -pc_type jacobi On Wednesday, December 11, 2013 1:26 PM, Qin Lu wrote: Thanks, Satish! ? Qin On Wednesday, December 11, 2013 12:27 PM, Satish Balay wrote: On Wed, 11 Dec 2013, Qin Lu wrote: > Hello, > ? > PETSc can read options from .petscrc file at home directory in Linux. Can it do the same thing in Windows? What is the file name and where the file should be placed? On windows - one can emulate this by setting "HOME" env variable to the location where you have .petscrc Also one can have .petscrc [or petscrc] in the location from which you run the petsc executable. Satish Any suggestion? Thanks, Qin -------------- next part -------------- An HTML attachment was scrubbed... URL: From balay at mcs.anl.gov Wed Dec 11 17:34:07 2013 From: balay at mcs.anl.gov (Satish Balay) Date: Wed, 11 Dec 2013 17:34:07 -0600 (CST) Subject: [petsc-users] .petscrc in Windows In-Reply-To: <1386804232.53238.YahooMailNeo@web160201.mail.bf1.yahoo.com> References: <1386786146.84754.YahooMailNeo@web160203.mail.bf1.yahoo.com> <1386789998.75789.YahooMailNeo@web160202.mail.bf1.yahoo.com> <1386804232.53238.YahooMailNeo@web160201.mail.bf1.yahoo.com> Message-ID: Hm - petscrc in the run dir works for me. [and the other modes aswell] Satish ------- balay at msnehalem2 ~/petsc.clone/src/ksp/ksp/examples/tutorials $ make ex2 /home/balay/petsc.clone/bin/win32fe/win32fe cl -o ex2.o -c -MT -wd4996 -Z7 -I/home/balay/petsc.clone/include -I/home/balay/petsc.clone/arch-win32/include -I/home/balay/petsc.clone/include/mpiuni `pwd`/ex2.c ex2.c /home/balay/petsc.clone/bin/win32fe/win32fe cl -MT -wd4996 -Z7 -o ex2 ex2.o -L/home/balay/petsc.clone/arch-win32/lib -lpetsc -lflapack -lfblas Gdi32.lib User32.lib Advapi32.lib Kernel32.lib Ws2_32.lib /usr/bin/rm -f ex2.o balay at msnehalem2 ~/petsc.clone/src/ksp/ksp/examples/tutorials $ ./ex2 Norm of error 0.000156044 iterations 6 balay at msnehalem2 ~/petsc.clone/src/ksp/ksp/examples/tutorials $ echo "-ksp_view" > petscrc balay at msnehalem2 ~/petsc.clone/src/ksp/ksp/examples/tutorials $ cat petscrc -ksp_view balay at msnehalem2 ~/petsc.clone/src/ksp/ksp/examples/tutorials $ ./ex2 KSP Object: 1 MPI processes type: gmres GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement GMRES: happy breakdown tolerance 1e-030 maximum iterations=10000, initial guess is zero tolerances: relative=0.000138889, absolute=1e-050, divergence=10000 left preconditioning using PRECONDITIONED norm type for convergence test PC Object: 1 MPI processes type: icc 0 levels of fill tolerance for zero pivot 2.22045e-014 using Manteuffel shift [POSITIVE_DEFINITE] matrix ordering: natural factor fill ratio given 1, needed 1 Factored matrix follows: Mat Object: 1 MPI processes type: seqsbaij rows=56, cols=56 package used to perform factorization: petsc total: nonzeros=153, allocated nonzeros=153 total number of mallocs used during MatSetValues calls =0 block size is 1 linear system matrix = precond matrix: Mat Object: 1 MPI processes type: seqaij rows=56, cols=56 total: nonzeros=250, allocated nonzeros=280 total number of mallocs used during MatSetValues calls =0 not using I-node routines Norm of error 0.000156044 iterations 6 balay at msnehalem2 ~/petsc.clone/src/ksp/ksp/examples/tutorials $ On Wed, 11 Dec 2013, Qin Lu wrote: > Satish, > ? > I tried both ways that you suggested in Win-7 (and I tried to put the petscrc file in either the directory of executable or the directory where I run the executable), but it did not work. Note that I have to use file name "petscrc" instead of ".petscrc" since Windows does not allow the file name start with "." > ? > The content of the file is very simple: > -pc_type jacobi > > > > > On Wednesday, December 11, 2013 1:26 PM, Qin Lu wrote: > > Thanks, Satish! > ? > Qin > > > > On Wednesday, December 11, 2013 12:27 PM, Satish Balay wrote: > > On Wed, 11 Dec 2013, Qin Lu wrote: > > > > Hello, > > ? > > PETSc can read options from .petscrc file at home > directory in Linux. Can it do the same thing in Windows? What is the file name and where the file should be placed? > > On windows - one can emulate this by setting "HOME" env variable to the location where you have .petscrc > > Also one can have .petscrc [or petscrc] in the location from which you run the petsc executable. > > Satish > > > > Any suggestion? > > Thanks, > Qin From ross.c.brodie at icloud.com Thu Dec 12 00:53:09 2013 From: ross.c.brodie at icloud.com (Ross Brodie) Date: Thu, 12 Dec 2013 17:53:09 +1100 Subject: [petsc-users] Mixed sparse and dense CG Message-ID: <34D4BB3B-0068-4676-AEA0-6F1E7F7151AA@icloud.com> Hello PETSc Users I have previously successfully tackled the problem of solving (J'DJ + aL'DL + cK'EK) x = b where real J, L, K were sparse matrices and D and E diagonal matrices and a and c real scalars using PETSc by using a shell matrix approach, i.e., never actually explicitly forming J'DJ etc but but supplying the function to KSP (PCG) to compute the products using PETSc functions with a vector on the fly. This did the trick for sparse J. I am pretty sure this can be done with a dense J, in PETSc as well just a lot slower. J could be the order of 10e5 x 10e6. I have considered using libElemental for the dense part (to make use of the grid cyclic layout) and PETSc for the sparse part of the PCG matvec product but I am not sure it is wise to mix packages. I also note that the recent release of PETSc includes an interface to Elemental via the MATELEMENTAL matrix type. Would this be a good choice? On a different note could you clarify the terminology for the sequential vector VECSEQ. Does it mean: (i) It is an independent uniprocessor object such that the data storage is not distributed and if you change an element on one process the change is NOT reflected on other processes, or (ii) does it mean the data storage is not distributed but it is not necessarily uniprocessor independent (ie such that each processor carries around redundant copies of the same data and changes on one process ARE reflected on other processes) depending on whether or not it is constructed using PETSC_COMM_SELF or PETSC_COMM_WORLD. Does it ever make sense to construct a VECSEQ with anything other than PETSC_COMM_SELF? I suspect it is (i) but want to make sure? Ross From jedbrown at mcs.anl.gov Thu Dec 12 01:21:21 2013 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Wed, 11 Dec 2013 23:21:21 -0800 Subject: [petsc-users] Mixed sparse and dense CG In-Reply-To: <34D4BB3B-0068-4676-AEA0-6F1E7F7151AA@icloud.com> References: <34D4BB3B-0068-4676-AEA0-6F1E7F7151AA@icloud.com> Message-ID: <871u1izg9a.fsf@jedbrown.org> Ross Brodie writes: > I have considered using libElemental for the dense part (to make use > of the grid cyclic layout) and PETSc for the sparse part of the PCG > matvec product but I am not sure it is wise to mix packages. The PETSc interface to Elemental can do this. In principle, you don't have to deal with the cyclic ordering, but you should assemble the rows and columns returned by MatGetOwnershipIS(). > On a different note could you clarify the terminology for the > sequential vector VECSEQ. Does it mean: (i) It is an independent > uniprocessor object such that the data storage is not distributed and > if you change an element on one process the change is NOT reflected on > other processes, Yes, and only makes sense on a communicator of size 1 (effectively PETSC_COMM_SELF). -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From ross.c.brodie at icloud.com Thu Dec 12 02:31:26 2013 From: ross.c.brodie at icloud.com (Ross Brodie) Date: Thu, 12 Dec 2013 19:31:26 +1100 Subject: [petsc-users] Mixed sparse and dense CG In-Reply-To: <871u1izg9a.fsf@jedbrown.org> References: <34D4BB3B-0068-4676-AEA0-6F1E7F7151AA@icloud.com> <871u1izg9a.fsf@jedbrown.org> Message-ID: <95FF1564-130D-4B32-BCBE-0E949F31B080@icloud.com> Jed Thanks i will pursue it through MATELEMENTAL. Ross > On 12 Dec 2013, at 18:21, Jed Brown wrote: > > Ross Brodie writes: > >> I have considered using libElemental for the dense part (to make use >> of the grid cyclic layout) and PETSc for the sparse part of the PCG >> matvec product but I am not sure it is wise to mix packages. > > The PETSc interface to Elemental can do this. In principle, you don't > have to deal with the cyclic ordering, but you should assemble the rows > and columns returned by MatGetOwnershipIS(). > >> On a different note could you clarify the terminology for the >> sequential vector VECSEQ. Does it mean: (i) It is an independent >> uniprocessor object such that the data storage is not distributed and >> if you change an element on one process the change is NOT reflected on >> other processes, > > Yes, and only makes sense on a communicator of size 1 (effectively > PETSC_COMM_SELF). From gideon.simpson at gmail.com Thu Dec 12 11:50:23 2013 From: gideon.simpson at gmail.com (Gideon Simpson) Date: Thu, 12 Dec 2013 17:50:23 +0000 Subject: [petsc-users] variable reduction Message-ID: <45701E94-DC34-4BA3-8B97-3121F2129EA3@gmail.com> Are there any analogs of the MPI reduction routines implemented for vecs and other data structures? Or should I just use MPI_Reduce as appropriate? -gideon -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Thu Dec 12 12:22:15 2013 From: knepley at gmail.com (Matthew Knepley) Date: Thu, 12 Dec 2013 12:22:15 -0600 Subject: [petsc-users] variable reduction In-Reply-To: <45701E94-DC34-4BA3-8B97-3121F2129EA3@gmail.com> References: <45701E94-DC34-4BA3-8B97-3121F2129EA3@gmail.com> Message-ID: On Thu, Dec 12, 2013 at 11:50 AM, Gideon Simpson wrote: > Are there any analogs of the MPI reduction routines implemented for vecs > and other data structures? Or should I just use MPI_Reduce as appropriate? > Is this a linear algebraic operation, or something more general? Thanks, Matt > -gideon > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From gideon.simpson at gmail.com Thu Dec 12 12:43:39 2013 From: gideon.simpson at gmail.com (Gideon Simpson) Date: Thu, 12 Dec 2013 18:43:39 +0000 Subject: [petsc-users] variable reduction In-Reply-To: References: <45701E94-DC34-4BA3-8B97-3121F2129EA3@gmail.com> Message-ID: <1DA2BE1A-CC5D-4C2B-9B52-AAFF4DE18DC4@gmail.com> No, it's like I have a (local) object, such as an array, on each thread, and I want to compute the elementwise sum, average, etc. -gideon On Dec 12, 2013, at 6:22 PM, Matthew Knepley wrote: > On Thu, Dec 12, 2013 at 11:50 AM, Gideon Simpson wrote: > Are there any analogs of the MPI reduction routines implemented for vecs and other data structures? Or should I just use MPI_Reduce as appropriate? > > Is this a linear algebraic operation, or something more general? > > Thanks, > > Matt > > -gideon > > > > > -- > What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. > -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From jianjun.xiao at kit.edu Thu Dec 12 15:09:23 2013 From: jianjun.xiao at kit.edu (Xiao, Jianjun (IKET)) Date: Thu, 12 Dec 2013 22:09:23 +0100 Subject: [petsc-users] Convergence problem in solving a symmetric positive definite matrix in a CFD code In-Reply-To: References: <56D054AF2E93E044AC1D2685709D2868C7340018C7@KIT-MSX-07.kit.edu>, Message-ID: <56D054AF2E93E044AC1D2685709D2868C9EE571D48@KIT-MSX-07.kit.edu> Hi Matt, I am using PETSc-dev because only this version supports MatSetValuesBlockedStencil in Fortran. I tried many combinations of KSP and PC. None of them worked in my case. The same thing happened to PETSc-3.4. Then I changed back to 3.3-P7 and 3.2-P7 without using MatSetValuesBlockedStencil. I tried KSPCR+PCBJACOBI and many others, they worked fine. I read the changes of each version, and still could not find the reason. Did I miss something? Thank you. ________________________________________ From: Matthew Knepley [knepley at gmail.com] Sent: Wednesday, December 04, 2013 4:19 PM To: Xiao, Jianjun (IKET) Cc: petsc-users at mcs.anl.gov Subject: Re: [petsc-users] Convergence problem in solving a symmetric positive definite matrix in a CFD code On Wed, Dec 4, 2013 at 8:59 AM, Xiao, Jianjun (IKET) > wrote: Hello, I am using Petsc to solve the linear equation in a CFD code. The matrix is symmetric positive definite. Please find my input and output below. 1. When I used KSPCR solver, ||r(i)||/||b|| is inf. What is the reason? Does it mean ||b|| is zero? When I used the KSPLGMRES solver, it seems ||r(i)||/||b|| is OK. However, it seems the calculated results are not right. It looks that way. You can check it with VecNorm() before calling KSPSolve(). Matt 2. I am not sure if I set the solver and matrix properly. Did I miss something? Thank you. **************************************** INPUT: CALL KSPCreate(PETSC_COMM_WORLD,solver,ierr) CALL DMSetMatType(da_ksp,MATMPISBAIJ,ierr) CALL DMCreateMatrix(da_ksp,mat,ierr) CALL MatMPISBAIJSetPreallocation(mat,1,4,PETSC_NULL_INTEGER,1,PETSC_NULL_INTEGER,ierr) CALL MatZeroEntries(mat,ierr) CALL MatAssemblyBegin(mat,MAT_FINAL_ASSEMBLY,ierr) CALL MatAssemblyEnd(mat,MAT_FINAL_ASSEMBLY,ierr) CALL KSPSetOperators(solver,mat,mat,SAME_NONZERO_PATTERN,ierr) ! CALL KSPSetType(solver,KSPCR,ierr) CALL KSPSetType(solver,KSPLGMRES,ierr) epsi = 1.0-e5 CALL KSPSetTolerances(solver,epsi,PETSC_DEFAULT_DOUBLE_PRECISION,& & PETSC_DEFAULT_DOUBLE_PRECISION,itmax,ierr) CALL KSPGetPC(solver,gfprec,ierr) CALL PCSetType(prec,PCBJACOBI,ierr) CALL KSPMonitorSet(solver,KSPMonitorTrueResidualNorm,& PETSC_NULL_OBJECT,PETSC_NULL_FUNCTION,ierr ) CALL KSPSetInitialGuessNonzero(solver,PETSC_TRUE,ierr) CALL KSPSolve(solver,bvec,xsol,ierr) CALL KSPGetIterationNumber(solver,iter,ierr) CALL KSPGetResidualNorm(solver,dmax,ierr) CALL KSPView(solver,PETSC_VIEWER_STDOUT_WORLD,ierr) ************************************** OUTPUT CALL KSPSetType(solver,KSPCR,ierr): 0 KSP preconditioned resid norm 2.226482634319e+05 true resid norm 1.204978940624e+07 ||r(i)||/||b|| inf 1 KSP preconditioned resid norm 1.684243557244e+05 true resid norm 6.742321430949e+06 ||r(i)||/||b|| inf 2 KSP preconditioned resid norm 1.039386033131e+05 true resid norm 5.094347016880e+06 ||r(i)||/||b|| inf 3 KSP preconditioned resid norm 3.767761162917e+04 true resid norm 2.085014289432e+06 ||r(i)||/||b|| inf 4 KSP preconditioned resid norm 2.220316358489e+04 true resid norm 1.039841616110e+06 ||r(i)||/||b|| inf 5 KSP preconditioned resid norm 1.009108756815e+04 true resid norm 6.764592620620e+05 ||r(i)||/||b|| inf 6 KSP preconditioned resid norm 7.266143334386e+03 true resid norm 4.713756053613e+05 ||r(i)||/||b|| inf 7 KSP preconditioned resid norm 4.925270026573e+03 true resid norm 3.276759177651e+05 ||r(i)||/||b|| inf 8 KSP preconditioned resid norm 2.595039666791e+03 true resid norm 1.774916597474e+05 ||r(i)||/||b|| inf 9 KSP preconditioned resid norm 1.970388137453e+03 true resid norm 1.449811653036e+05 ||r(i)||/||b|| inf 10 KSP preconditioned resid norm 1.455914234388e+03 true resid norm 7.916294162841e+04 ||r(i)||/||b|| inf 11 KSP preconditioned resid norm 8.335194818556e+02 true resid norm 4.530953608250e+04 ||r(i)||/||b|| inf 12 KSP preconditioned resid norm 3.022320555777e+02 true resid norm 1.728551635678e+04 ||r(i)||/||b|| inf 13 KSP preconditioned resid norm 7.190336024797e+01 true resid norm 4.186842086105e+03 ||r(i)||/||b|| inf 14 KSP preconditioned resid norm 1.718291655675e+01 true resid norm 1.089751055004e+03 ||r(i)||/||b|| inf 15 KSP preconditioned resid norm 1.150683059424e+01 true resid norm 8.672405273471e+02 ||r(i)||/||b|| inf 16 KSP preconditioned resid norm 8.663479440949e+00 true resid norm 5.776737380768e+02 ||r(i)||/||b|| inf 17 KSP preconditioned resid norm 5.282161990683e+00 true resid norm 2.977735906695e+02 ||r(i)||/||b|| inf 18 KSP preconditioned resid norm 3.802629315725e+00 true resid norm 2.789114564993e+02 ||r(i)||/||b|| inf 19 KSP preconditioned resid norm 1.722575171383e+00 true resid norm 1.051323829526e+02 ||r(i)||/||b|| inf KSP Object: 1 MPI processes type: cr maximum iterations=1000 tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using nonzero initial guess using PRECONDITIONED norm type for convergence test PC Object: 1 MPI processes type: bjacobi block Jacobi: number of blocks = 1 Local solve is same for all blocks, in the following KSP and PC objects: KSP Object: (sub_) 1 MPI processes type: preonly maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using NONE norm type for convergence test PC Object: (sub_) 1 MPI processes type: icc 0 levels of fill tolerance for zero pivot 2.22045e-14 using Manteuffel shift [POSITIVE_DEFINITE] matrix ordering: natural factor fill ratio given 1, needed 1 Factored matrix follows: Mat Object: 1 MPI processes type: seqsbaij rows=27, cols=27 package used to perform factorization: petsc total: nonzeros=81, allocated nonzeros=81 total number of mallocs used during MatSetValues calls =0 block size is 1 linear system matrix = precond matrix: Mat Object: 1 MPI processes type: seqsbaij rows=27, cols=27 total: nonzeros=81, allocated nonzeros=108 total number of mallocs used during MatSetValues calls =0 block size is 1 linear system matrix = precond matrix: Mat Object: 1 MPI processes type: mpisbaij rows=27, cols=27 total: nonzeros=81, allocated nonzeros=135 total number of mallocs used during MatSetValues calls =0 block size is 1 OUTPUT: CALL KSPSetType(solver,KSPLGMRES,ierr) 0 KSP preconditioned resid norm 2.362537325084e+04 true resid norm 1.138584383312e+06 ||r(i)||/||b|| 1.000000000000e+00 1 KSP preconditioned resid norm 8.501213683423e+03 true resid norm 3.655528853686e+05 ||r(i)||/||b|| 3.210591070162e-01 2 KSP preconditioned resid norm 5.487567253725e+03 true resid norm 3.005741194777e+05 ||r(i)||/||b|| 2.639893220769e-01 3 KSP preconditioned resid norm 2.470452880657e+03 true resid norm 1.545469272201e+05 ||r(i)||/||b|| 1.357360328187e-01 4 KSP preconditioned resid norm 1.750803325456e+03 true resid norm 1.182312309352e+05 ||r(i)||/||b|| 1.038405520646e-01 5 KSP preconditioned resid norm 1.123492053552e+03 true resid norm 6.754319630701e+04 ||r(i)||/||b|| 5.932208213726e-02 6 KSP preconditioned resid norm 5.150241959277e+02 true resid norm 3.689413898730e+04 ||r(i)||/||b|| 3.240351749775e-02 7 KSP preconditioned resid norm 4.182894544871e+02 true resid norm 3.052196222024e+04 ||r(i)||/||b|| 2.680693909701e-02 8 KSP preconditioned resid norm 2.520093155629e+02 true resid norm 1.880976788356e+04 ||r(i)||/||b|| 1.652031079932e-02 9 KSP preconditioned resid norm 1.186491314806e+02 true resid norm 6.797080217853e+03 ||r(i)||/||b|| 5.969764136483e-03 10 KSP preconditioned resid norm 5.851092821372e+01 true resid norm 2.973659280245e+03 ||r(i)||/||b|| 2.611716201127e-03 11 KSP preconditioned resid norm 1.669909189055e+01 true resid norm 5.658829814125e+02 ||r(i)||/||b|| 4.970057465277e-04 12 KSP preconditioned resid norm 3.090594692756e+00 true resid norm 2.161527454147e+02 ||r(i)||/||b|| 1.898434130864e-04 13 KSP preconditioned resid norm 2.164618839184e+00 true resid norm 1.620745991834e+02 ||r(i)||/||b|| 1.423474637093e-04 14 KSP preconditioned resid norm 1.291593952428e+00 true resid norm 9.095542547366e+01 ||r(i)||/||b|| 7.988465923722e-05 15 KSP preconditioned resid norm 6.100583411632e-01 true resid norm 4.021646656091e+01 ||r(i)||/||b|| 3.532146334551e-05 16 KSP preconditioned resid norm 2.723496807925e-01 true resid norm 1.676660466866e+01 ||r(i)||/||b|| 1.472583403954e-05 17 KSP preconditioned resid norm 1.377718471538e-01 true resid norm 8.551854245272e+00 ||r(i)||/||b|| 7.510953400221e-06 KSP Object: 1 MPI processes type: lgmres GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement GMRES: happy breakdown tolerance 1e-30 LGMRES: aug. dimension=2 LGMRES: number of matvecs=170 maximum iterations=1000 tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using nonzero initial guess using PRECONDITIONED norm type for convergence test PC Object: 1 MPI processes type: bjacobi block Jacobi: number of blocks = 1 Local solve is same for all blocks, in the following KSP and PC objects: KSP Object: (sub_) 1 MPI processes type: preonly maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using NONE norm type for convergence test PC Object: (sub_) 1 MPI processes type: icc 0 levels of fill tolerance for zero pivot 2.22045e-14 using Manteuffel shift [POSITIVE_DEFINITE] matrix ordering: natural factor fill ratio given 1, needed 1 Factored matrix follows: Mat Object: 1 MPI processes type: seqsbaij rows=27, cols=27 package used to perform factorization: petsc total: nonzeros=81, allocated nonzeros=81 total number of mallocs used during MatSetValues calls =0 block size is 1 linear system matrix = precond matrix: Mat Object: 1 MPI processes type: seqsbaij rows=27, cols=27 total: nonzeros=81, allocated nonzeros=108 total number of mallocs used during MatSetValues calls =0 block size is 1 linear system matrix = precond matrix: Mat Object: 1 MPI processes type: mpisbaij rows=27, cols=27 total: nonzeros=81, allocated nonzeros=135 total number of mallocs used during MatSetValues calls =0 -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener From mfadams at lbl.gov Thu Dec 12 15:12:42 2013 From: mfadams at lbl.gov (Mark Adams) Date: Thu, 12 Dec 2013 16:12:42 -0500 Subject: [petsc-users] GPU local direct solve of penta-diagonal Message-ID: We have a lot of 5-point stencil operators on ~50x100 grids to solve. These are not symmetric and we have been using LU. We want to move this onto GPUs (Titan). What resources are there to do this? Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: From rupp at mcs.anl.gov Thu Dec 12 15:15:33 2013 From: rupp at mcs.anl.gov (Karl Rupp) Date: Thu, 12 Dec 2013 22:15:33 +0100 Subject: [petsc-users] GPU local direct solve of penta-diagonal In-Reply-To: References: Message-ID: <52AA2775.6070902@mcs.anl.gov> Hi Mark, > We have a lot of 5-point stencil operators on ~50x100 grids to solve. > These are not symmetric and we have been using LU. We want to move > this onto GPUs (Titan). What resources are there to do this? do you have lots of problems to solve simultaneously? Or any other feature that makes this problem expensive? 50x100 would mean a system size of about 5000 dofs, which is too small to really benefit from GPUs. Best regards, Karli From dazevedoef at ornl.gov Thu Dec 12 15:38:35 2013 From: dazevedoef at ornl.gov (Ed D'Azevedo) Date: Thu, 12 Dec 2013 16:38:35 -0500 Subject: [petsc-users] GPU local direct solve of penta-diagonal In-Reply-To: <52AA2775.6070902@mcs.anl.gov> References: <52AA2775.6070902@mcs.anl.gov> Message-ID: <52AA2CDB.9000400@ornl.gov> Hi Karli, Yes, each MPI process is responsible for solving a system of nonlinear equations on a number of grid cells. The nonlinear equations are solved by Picard iteration and the time consuming part is the formation and solution of the nonsymmetric sparse linear system arising from a rectangular grid with a regular finite difference stencil. All the linear systems have the same sparsity pattern but may have different numerical values. Since there are 16 cores on each node on Titan, there can be concurrently 16 separate independent linear systems to be solved. One may not want to batch or synchronize the solvers since different grid cells may require different number of Picard iterations. Ed On 12/12/2013 04:15 PM, Karl Rupp wrote: > Hi Mark, > > > We have a lot of 5-point stencil operators on ~50x100 grids to solve. >> These are not symmetric and we have been using LU. We want to move >> this onto GPUs (Titan). What resources are there to do this? > do you have lots of problems to solve simultaneously? Or any other > feature that makes this problem expensive? 50x100 would mean a system > size of about 5000 dofs, which is too small to really benefit from GPUs. > > Best regards, > Karli > From mfadams at lbl.gov Thu Dec 12 15:49:23 2013 From: mfadams at lbl.gov (Mark Adams) Date: Thu, 12 Dec 2013 16:49:23 -0500 Subject: [petsc-users] GPU local direct solve of penta-diagonal In-Reply-To: <52AA2CDB.9000400@ornl.gov> References: <52AA2775.6070902@mcs.anl.gov> <52AA2CDB.9000400@ornl.gov> Message-ID: On Thu, Dec 12, 2013 at 4:38 PM, Ed D'Azevedo wrote: > Hi Karli, > > Yes, each MPI process is responsible for solving a system of nonlinear > equations on a number of grid cells. > Just to elaborate, and Ed can correct me, each MPI process has a few 100 to a few 1000 (spacial) cells. We solve a (Folker-Plank) system in velocity space at each grid cell. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rupp at mcs.anl.gov Thu Dec 12 15:50:30 2013 From: rupp at mcs.anl.gov (Karl Rupp) Date: Thu, 12 Dec 2013 22:50:30 +0100 Subject: [petsc-users] GPU local direct solve of penta-diagonal In-Reply-To: <52AA2CDB.9000400@ornl.gov> References: <52AA2775.6070902@mcs.anl.gov> <52AA2CDB.9000400@ornl.gov> Message-ID: <52AA2FA6.8080802@mcs.anl.gov> Hi Ed, > Yes, each MPI process is responsible for solving a system of nonlinear > equations on a number of grid cells. > The nonlinear equations are solved by Picard iteration and the time > consuming part is the formation and solution of the nonsymmetric sparse > linear system arising from a rectangular grid with a regular finite > difference stencil. All the linear systems have the same sparsity > pattern but may have different numerical values. > > Since there are 16 cores on each node on Titan, there can be > concurrently 16 separate independent linear systems to be solved. > One may not want to batch or synchronize the solvers since different > grid cells may require different number of Picard iterations. Hmm, this does not sound like something I would consider a good fit for GPUs. With 16 MPI processes you have additional congestion of the one or two GPUs per node, so you would have the rethink the solution procedure as a whole. I can think of a procedure where each of these systems is solved on a separate streaming processor (or work group in OpenCL language), where synchronization is cheaper - however, this is not covered by standard functionality in PETSc. Either way, you would certainly trade robustness of the implementation and a substantial amount of development time for probably a 2x speedup (if you're lucky). If you want to give it a try nonetheless, try -vectype cusp -mattype aijcusp and some simple preconditioners like Jacobi in order to avoid host<->device communication. Best regards, Karli > > Ed > > > On 12/12/2013 04:15 PM, Karl Rupp wrote: >> Hi Mark, >> >> > We have a lot of 5-point stencil operators on ~50x100 grids to solve. >>> These are not symmetric and we have been using LU. We want to move >>> this onto GPUs (Titan). What resources are there to do this? >> do you have lots of problems to solve simultaneously? Or any other >> feature that makes this problem expensive? 50x100 would mean a system >> size of about 5000 dofs, which is too small to really benefit from GPUs. >> >> Best regards, >> Karli >> > From rupp at mcs.anl.gov Thu Dec 12 15:59:41 2013 From: rupp at mcs.anl.gov (Karl Rupp) Date: Thu, 12 Dec 2013 22:59:41 +0100 Subject: [petsc-users] GPU local direct solve of penta-diagonal In-Reply-To: References: <52AA2775.6070902@mcs.anl.gov> <52AA2CDB.9000400@ornl.gov> Message-ID: <52AA31CD.2080401@mcs.anl.gov> Hi, > Yes, each MPI process is responsible for solving a system of > nonlinear equations on a number of grid cells. > > > Just to elaborate, and Ed can correct me, each MPI process has a few 100 > to a few 1000 (spacial) cells. We solve a (Folker-Plank) system in > velocity space at each grid cell. Thanks, Mark, this helps. Is there any chance you can collect a couple of spatial cells together and solve a bigger system consisting of decoupled subsystems? Ideally you have more than 100k dofs for GPUs to perform well. Have a look at this figure here (cross-over at about 10k dofs for CUDA): http://viennacl.sourceforge.net/uploads/pics/cg-timings.png to get an idea about the saturation of GPU solves at smaller system sizes. PCI-Express latency is the limiting factor here. Best regards, Karli From dmeiser at txcorp.com Thu Dec 12 17:29:12 2013 From: dmeiser at txcorp.com (Dominic Meiser) Date: Thu, 12 Dec 2013 16:29:12 -0700 Subject: [petsc-users] GPU local direct solve of penta-diagonal In-Reply-To: <52AA2FA6.8080802@mcs.anl.gov> References: <52AA2775.6070902@mcs.anl.gov> <52AA2CDB.9000400@ornl.gov> <52AA2FA6.8080802@mcs.anl.gov> Message-ID: <52AA46C8.4030109@txcorp.com> Hi Karli, On 12/12/2013 02:50 PM, Karl Rupp wrote: > > Hmm, this does not sound like something I would consider a good fit > for GPUs. With 16 MPI processes you have additional congestion of the > one or two GPUs per node, so you would have the rethink the solution > procedure as a whole. > Are you sure about that for Titan? Supposedly the K20X's can deal with multiple MPI processes hitting a single GPU pretty well using Hyper-Q. Paul has seen pretty good speed up with small GPU kernels simply by over-subscribing each GPU with 4 MPI processes. See here: http://blogs.nvidia.com/blog/2012/08/23/unleash-legacy-mpi-codes-with-keplers-hyper-q/ Cheers, Dominic -- Dominic Meiser Tech-X Corporation 5621 Arapahoe Avenue Boulder, CO 80303 USA Telephone: 303-996-2036 Fax: 303-448-7756 www.txcorp.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From paulmullowney at gmail.com Thu Dec 12 17:38:02 2013 From: paulmullowney at gmail.com (Paul Mullowney) Date: Thu, 12 Dec 2013 16:38:02 -0700 Subject: [petsc-users] GPU local direct solve of penta-diagonal In-Reply-To: <52AA46C8.4030109@txcorp.com> References: <52AA2775.6070902@mcs.anl.gov> <52AA2CDB.9000400@ornl.gov> <52AA2FA6.8080802@mcs.anl.gov> <52AA46C8.4030109@txcorp.com> Message-ID: Provided you have a good parallel sparse direct solve for a single SM, you could unleash 32 direct solves (or perhaps 16) which run concurrently on the K20x. One only needs to set an environment variable to use Hypre Q I don't know of any good parallel sparse direct solver for small systems. -Paul On Thu, Dec 12, 2013 at 4:29 PM, Dominic Meiser wrote: > Hi Karli, > > > On 12/12/2013 02:50 PM, Karl Rupp wrote: > > > Hmm, this does not sound like something I would consider a good fit for > GPUs. With 16 MPI processes you have additional congestion of the one or > two GPUs per node, so you would have the rethink the solution procedure as > a whole. > > Are you sure about that for Titan? Supposedly the K20X's can deal with > multiple MPI processes hitting a single GPU pretty well using Hyper-Q. Paul > has seen pretty good speed up with small GPU kernels simply by > over-subscribing each GPU with 4 MPI processes. > > See here: > > http://blogs.nvidia.com/blog/2012/08/23/unleash-legacy-mpi-codes-with-keplers-hyper-q/ > > > Cheers, > Dominic > > > -- > Dominic Meiser > Tech-X Corporation > 5621 Arapahoe Avenue > Boulder, CO 80303 > USA > Telephone: 303-996-2036 > Fax: 303-448-7756www.txcorp.com > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmeiser at txcorp.com Thu Dec 12 18:08:56 2013 From: dmeiser at txcorp.com (Dominic Meiser) Date: Thu, 12 Dec 2013 17:08:56 -0700 Subject: [petsc-users] GPU local direct solve of penta-diagonal In-Reply-To: References: <52AA2775.6070902@mcs.anl.gov> <52AA2CDB.9000400@ornl.gov> <52AA2FA6.8080802@mcs.anl.gov> <52AA46C8.4030109@txcorp.com> Message-ID: <52AA5018.7020509@txcorp.com> On 12/12/2013 04:38 PM, Paul Mullowney wrote: > Provided you have a good parallel sparse direct solve for a single SM, > you could unleash 32 direct solves (or perhaps 16) which run > concurrently on the K20x. One only needs to set an environment > variable to use Hypre Q On Titan all you need to do is | $ export CRAY_CUDA_PROXY=1 | See here: https://www.olcf.ornl.gov/tutorials/cuda-proxy-managing-gpu-context/ Cheers, Dominic > > I don't know of any good parallel sparse direct solver for small systems. > > -Paul > > > On Thu, Dec 12, 2013 at 4:29 PM, Dominic Meiser > wrote: > > Hi Karli, > > > On 12/12/2013 02:50 PM, Karl Rupp wrote: >> >> Hmm, this does not sound like something I would consider a good >> fit for GPUs. With 16 MPI processes you have additional >> congestion of the one or two GPUs per node, so you would have the >> rethink the solution procedure as a whole. >> > Are you sure about that for Titan? Supposedly the K20X's can deal > with multiple MPI processes hitting a single GPU pretty well using > Hyper-Q. Paul has seen pretty good speed up with small GPU kernels > simply by over-subscribing each GPU with 4 MPI processes. > > See here: > http://blogs.nvidia.com/blog/2012/08/23/unleash-legacy-mpi-codes-with-keplers-hyper-q/ > > > Cheers, > Dominic > > > -- > Dominic Meiser > Tech-X Corporation > 5621 Arapahoe Avenue > Boulder, CO 80303 > USA > Telephone:303-996-2036 > Fax:303-448-7756 > www.txcorp.com > > -- Dominic Meiser Tech-X Corporation 5621 Arapahoe Avenue Boulder, CO 80303 USA Telephone: 303-996-2036 Fax: 303-448-7756 www.txcorp.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Thu Dec 12 20:05:28 2013 From: knepley at gmail.com (Matthew Knepley) Date: Thu, 12 Dec 2013 20:05:28 -0600 Subject: [petsc-users] variable reduction In-Reply-To: <1DA2BE1A-CC5D-4C2B-9B52-AAFF4DE18DC4@gmail.com> References: <45701E94-DC34-4BA3-8B97-3121F2129EA3@gmail.com> <1DA2BE1A-CC5D-4C2B-9B52-AAFF4DE18DC4@gmail.com> Message-ID: On Thu, Dec 12, 2013 at 12:43 PM, Gideon Simpson wrote: > No, it's like I have a (local) object, such as an array, on each thread, > and I want to compute the elementwise sum, average, etc. > You could have individual vectors with one element/proc, but it sound better just to use MPI directly. Matt > -gideon > > On Dec 12, 2013, at 6:22 PM, Matthew Knepley wrote: > > On Thu, Dec 12, 2013 at 11:50 AM, Gideon Simpson > wrote: > >> Are there any analogs of the MPI reduction routines implemented for vecs >> and other data structures? Or should I just use MPI_Reduce as appropriate? >> > > Is this a linear algebraic operation, or something more general? > > Thanks, > > Matt > > >> -gideon >> >> > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Thu Dec 12 20:12:30 2013 From: knepley at gmail.com (Matthew Knepley) Date: Thu, 12 Dec 2013 20:12:30 -0600 Subject: [petsc-users] Convergence problem in solving a symmetric positive definite matrix in a CFD code In-Reply-To: <56D054AF2E93E044AC1D2685709D2868C9EE571D48@KIT-MSX-07.kit.edu> References: <56D054AF2E93E044AC1D2685709D2868C7340018C7@KIT-MSX-07.kit.edu> <56D054AF2E93E044AC1D2685709D2868C9EE571D48@KIT-MSX-07.kit.edu> Message-ID: On Thu, Dec 12, 2013 at 3:09 PM, Xiao, Jianjun (IKET) wrote: > Hi Matt, > > I am using PETSc-dev because only this version supports > MatSetValuesBlockedStencil in Fortran. I tried many combinations of KSP > and PC. None of them worked in my case. The same thing happened to > PETSc-3.4. > > Then I changed back to 3.3-P7 and 3.2-P7 without using > MatSetValuesBlockedStencil. I tried KSPCR+PCBJACOBI and many others, they > worked fine. > > I read the changes of each version, and still could not find the reason. > Did I miss something? Thank you. > 1) "Did not work" is an insufficient description of the problem. If it did not converge, give the output of -ksp_monitor_true_residual -ksp_converged_reason for each run 2) You changed two things at once. That makes it impossible to debug. Use petsc-dev without MatSetValuesBlockedStencil(). Do you reproduce the results from 3.3? If so, I believe your blocks may be ordered incorrectly. Matt > ________________________________________ > From: Matthew Knepley [knepley at gmail.com] > Sent: Wednesday, December 04, 2013 4:19 PM > To: Xiao, Jianjun (IKET) > Cc: petsc-users at mcs.anl.gov > Subject: Re: [petsc-users] Convergence problem in solving a symmetric > positive definite matrix in a CFD code > > On Wed, Dec 4, 2013 at 8:59 AM, Xiao, Jianjun (IKET) > wrote: > Hello, > > I am using Petsc to solve the linear equation in a CFD code. The matrix is > symmetric positive definite. > > Please find my input and output below. > > 1. When I used KSPCR solver, ||r(i)||/||b|| is inf. What is the reason? > Does it mean ||b|| is zero? When I used the KSPLGMRES solver, it seems > ||r(i)||/||b|| is OK. However, it seems the calculated results are not > right. > > It looks that way. You can check it with VecNorm() before calling > KSPSolve(). > > Matt > > 2. I am not sure if I set the solver and matrix properly. Did I miss > something? > > Thank you. > > > **************************************** > INPUT: > > CALL KSPCreate(PETSC_COMM_WORLD,solver,ierr) > CALL DMSetMatType(da_ksp,MATMPISBAIJ,ierr) > CALL DMCreateMatrix(da_ksp,mat,ierr) > CALL > MatMPISBAIJSetPreallocation(mat,1,4,PETSC_NULL_INTEGER,1,PETSC_NULL_INTEGER,ierr) > CALL MatZeroEntries(mat,ierr) > CALL MatAssemblyBegin(mat,MAT_FINAL_ASSEMBLY,ierr) > CALL MatAssemblyEnd(mat,MAT_FINAL_ASSEMBLY,ierr) > > CALL KSPSetOperators(solver,mat,mat,SAME_NONZERO_PATTERN,ierr) > > ! CALL KSPSetType(solver,KSPCR,ierr) > CALL KSPSetType(solver,KSPLGMRES,ierr) > > epsi = 1.0-e5 > CALL KSPSetTolerances(solver,epsi,PETSC_DEFAULT_DOUBLE_PRECISION,& > & PETSC_DEFAULT_DOUBLE_PRECISION,itmax,ierr) > > CALL KSPGetPC(solver,gfprec,ierr) > CALL PCSetType(prec,PCBJACOBI,ierr) > > CALL KSPMonitorSet(solver,KSPMonitorTrueResidualNorm,& > PETSC_NULL_OBJECT,PETSC_NULL_FUNCTION,ierr ) > > CALL KSPSetInitialGuessNonzero(solver,PETSC_TRUE,ierr) > CALL KSPSolve(solver,bvec,xsol,ierr) > > CALL KSPGetIterationNumber(solver,iter,ierr) > CALL KSPGetResidualNorm(solver,dmax,ierr) > CALL KSPView(solver,PETSC_VIEWER_STDOUT_WORLD,ierr) > > ************************************** > OUTPUT CALL KSPSetType(solver,KSPCR,ierr): > > 0 KSP preconditioned resid norm 2.226482634319e+05 true resid norm > 1.204978940624e+07 ||r(i)||/||b|| inf > 1 KSP preconditioned resid norm 1.684243557244e+05 true resid norm > 6.742321430949e+06 ||r(i)||/||b|| inf > 2 KSP preconditioned resid norm 1.039386033131e+05 true resid norm > 5.094347016880e+06 ||r(i)||/||b|| inf > 3 KSP preconditioned resid norm 3.767761162917e+04 true resid norm > 2.085014289432e+06 ||r(i)||/||b|| inf > 4 KSP preconditioned resid norm 2.220316358489e+04 true resid norm > 1.039841616110e+06 ||r(i)||/||b|| inf > 5 KSP preconditioned resid norm 1.009108756815e+04 true resid norm > 6.764592620620e+05 ||r(i)||/||b|| inf > 6 KSP preconditioned resid norm 7.266143334386e+03 true resid norm > 4.713756053613e+05 ||r(i)||/||b|| inf > 7 KSP preconditioned resid norm 4.925270026573e+03 true resid norm > 3.276759177651e+05 ||r(i)||/||b|| inf > 8 KSP preconditioned resid norm 2.595039666791e+03 true resid norm > 1.774916597474e+05 ||r(i)||/||b|| inf > 9 KSP preconditioned resid norm 1.970388137453e+03 true resid norm > 1.449811653036e+05 ||r(i)||/||b|| inf > 10 KSP preconditioned resid norm 1.455914234388e+03 true resid norm > 7.916294162841e+04 ||r(i)||/||b|| inf > 11 KSP preconditioned resid norm 8.335194818556e+02 true resid norm > 4.530953608250e+04 ||r(i)||/||b|| inf > 12 KSP preconditioned resid norm 3.022320555777e+02 true resid norm > 1.728551635678e+04 ||r(i)||/||b|| inf > 13 KSP preconditioned resid norm 7.190336024797e+01 true resid norm > 4.186842086105e+03 ||r(i)||/||b|| inf > 14 KSP preconditioned resid norm 1.718291655675e+01 true resid norm > 1.089751055004e+03 ||r(i)||/||b|| inf > 15 KSP preconditioned resid norm 1.150683059424e+01 true resid norm > 8.672405273471e+02 ||r(i)||/||b|| inf > 16 KSP preconditioned resid norm 8.663479440949e+00 true resid norm > 5.776737380768e+02 ||r(i)||/||b|| inf > 17 KSP preconditioned resid norm 5.282161990683e+00 true resid norm > 2.977735906695e+02 ||r(i)||/||b|| inf > 18 KSP preconditioned resid norm 3.802629315725e+00 true resid norm > 2.789114564993e+02 ||r(i)||/||b|| inf > 19 KSP preconditioned resid norm 1.722575171383e+00 true resid norm > 1.051323829526e+02 ||r(i)||/||b|| inf > KSP Object: 1 MPI processes > type: cr > maximum iterations=1000 > tolerances: relative=1e-05, absolute=1e-50, divergence=10000 > left preconditioning > using nonzero initial guess > using PRECONDITIONED norm type for convergence test > PC Object: 1 MPI processes > type: bjacobi > block Jacobi: number of blocks = 1 > Local solve is same for all blocks, in the following KSP and PC > objects: > KSP Object: (sub_) 1 MPI processes > type: preonly > maximum iterations=10000, initial guess is zero > tolerances: relative=1e-05, absolute=1e-50, divergence=10000 > left preconditioning > using NONE norm type for convergence test > PC Object: (sub_) 1 MPI processes > type: icc > 0 levels of fill > tolerance for zero pivot 2.22045e-14 > using Manteuffel shift [POSITIVE_DEFINITE] > matrix ordering: natural > factor fill ratio given 1, needed 1 > Factored matrix follows: > Mat Object: 1 MPI processes > type: seqsbaij > rows=27, cols=27 > package used to perform factorization: petsc > total: nonzeros=81, allocated nonzeros=81 > total number of mallocs used during MatSetValues calls =0 > block size is 1 > linear system matrix = precond matrix: > Mat Object: 1 MPI processes > type: seqsbaij > rows=27, cols=27 > total: nonzeros=81, allocated nonzeros=108 > total number of mallocs used during MatSetValues calls =0 > block size is 1 > linear system matrix = precond matrix: > Mat Object: 1 MPI processes > type: mpisbaij > rows=27, cols=27 > total: nonzeros=81, allocated nonzeros=135 > total number of mallocs used during MatSetValues calls =0 > block size is 1 > > OUTPUT: CALL KSPSetType(solver,KSPLGMRES,ierr) > > 0 KSP preconditioned resid norm 2.362537325084e+04 true resid norm > 1.138584383312e+06 ||r(i)||/||b|| 1.000000000000e+00 > 1 KSP preconditioned resid norm 8.501213683423e+03 true resid norm > 3.655528853686e+05 ||r(i)||/||b|| 3.210591070162e-01 > 2 KSP preconditioned resid norm 5.487567253725e+03 true resid norm > 3.005741194777e+05 ||r(i)||/||b|| 2.639893220769e-01 > 3 KSP preconditioned resid norm 2.470452880657e+03 true resid norm > 1.545469272201e+05 ||r(i)||/||b|| 1.357360328187e-01 > 4 KSP preconditioned resid norm 1.750803325456e+03 true resid norm > 1.182312309352e+05 ||r(i)||/||b|| 1.038405520646e-01 > 5 KSP preconditioned resid norm 1.123492053552e+03 true resid norm > 6.754319630701e+04 ||r(i)||/||b|| 5.932208213726e-02 > 6 KSP preconditioned resid norm 5.150241959277e+02 true resid norm > 3.689413898730e+04 ||r(i)||/||b|| 3.240351749775e-02 > 7 KSP preconditioned resid norm 4.182894544871e+02 true resid norm > 3.052196222024e+04 ||r(i)||/||b|| 2.680693909701e-02 > 8 KSP preconditioned resid norm 2.520093155629e+02 true resid norm > 1.880976788356e+04 ||r(i)||/||b|| 1.652031079932e-02 > 9 KSP preconditioned resid norm 1.186491314806e+02 true resid norm > 6.797080217853e+03 ||r(i)||/||b|| 5.969764136483e-03 > 10 KSP preconditioned resid norm 5.851092821372e+01 true resid norm > 2.973659280245e+03 ||r(i)||/||b|| 2.611716201127e-03 > 11 KSP preconditioned resid norm 1.669909189055e+01 true resid norm > 5.658829814125e+02 ||r(i)||/||b|| 4.970057465277e-04 > 12 KSP preconditioned resid norm 3.090594692756e+00 true resid norm > 2.161527454147e+02 ||r(i)||/||b|| 1.898434130864e-04 > 13 KSP preconditioned resid norm 2.164618839184e+00 true resid norm > 1.620745991834e+02 ||r(i)||/||b|| 1.423474637093e-04 > 14 KSP preconditioned resid norm 1.291593952428e+00 true resid norm > 9.095542547366e+01 ||r(i)||/||b|| 7.988465923722e-05 > 15 KSP preconditioned resid norm 6.100583411632e-01 true resid norm > 4.021646656091e+01 ||r(i)||/||b|| 3.532146334551e-05 > 16 KSP preconditioned resid norm 2.723496807925e-01 true resid norm > 1.676660466866e+01 ||r(i)||/||b|| 1.472583403954e-05 > 17 KSP preconditioned resid norm 1.377718471538e-01 true resid norm > 8.551854245272e+00 ||r(i)||/||b|| 7.510953400221e-06 > KSP Object: 1 MPI processes > type: lgmres > GMRES: restart=30, using Classical (unmodified) Gram-Schmidt > Orthogonalization with no iterative refinement > GMRES: happy breakdown tolerance 1e-30 > LGMRES: aug. dimension=2 > LGMRES: number of matvecs=170 > maximum iterations=1000 > tolerances: relative=1e-05, absolute=1e-50, divergence=10000 > left preconditioning > using nonzero initial guess > using PRECONDITIONED norm type for convergence test > PC Object: 1 MPI processes > type: bjacobi > block Jacobi: number of blocks = 1 > Local solve is same for all blocks, in the following KSP and PC > objects: > KSP Object: (sub_) 1 MPI processes > type: preonly > maximum iterations=10000, initial guess is zero > tolerances: relative=1e-05, absolute=1e-50, divergence=10000 > left preconditioning > using NONE norm type for convergence test > PC Object: (sub_) 1 MPI processes > type: icc > 0 levels of fill > tolerance for zero pivot 2.22045e-14 > using Manteuffel shift [POSITIVE_DEFINITE] > matrix ordering: natural > factor fill ratio given 1, needed 1 > Factored matrix follows: > Mat Object: 1 MPI processes > type: seqsbaij > rows=27, cols=27 > package used to perform factorization: petsc > total: nonzeros=81, allocated nonzeros=81 > total number of mallocs used during MatSetValues calls =0 > block size is 1 > linear system matrix = precond matrix: > Mat Object: 1 MPI processes > type: seqsbaij > rows=27, cols=27 > total: nonzeros=81, allocated nonzeros=108 > total number of mallocs used during MatSetValues calls =0 > block size is 1 > linear system matrix = precond matrix: > Mat Object: 1 MPI processes > type: mpisbaij > rows=27, cols=27 > total: nonzeros=81, allocated nonzeros=135 > total number of mallocs used during MatSetValues calls =0 > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From mc0710 at gmail.com Thu Dec 12 21:24:08 2013 From: mc0710 at gmail.com (Mani Chandra) Date: Thu, 12 Dec 2013 21:24:08 -0600 Subject: [petsc-users] Optimal memory layout for finite differences Message-ID: Hi Everyone, Would it be a good idea to arrange the data in fastest direction in the following manner for the ease of aligned loads and vector operations? Total grid points = 4n 0, n, 2n, 3n, 1, n+1, 2n+1, 3n+1 and so on Ref: "Tuning a Finite Difference Computation for Parallel Vector Processors" http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=6341495 This change in the global memory layout would mix up the ghost zones in Petscs' DMDAs and I guess change the matrix structure seperating adjacent points by a distance = 4. One can even make the distance = 8 and load one full cacheline in one go. I was wondering if this memory layout can be used for computations using Petscs' DMDAs and if the preconditioners would be ok with this kind of an arrangement. Thanks, Mani -------------- next part -------------- An HTML attachment was scrubbed... URL: From jedbrown at mcs.anl.gov Fri Dec 13 01:34:17 2013 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Thu, 12 Dec 2013 23:34:17 -0800 Subject: [petsc-users] Convergence problem in solving a symmetric positive definite matrix in a CFD code In-Reply-To: <56D054AF2E93E044AC1D2685709D2868C9EE571D48@KIT-MSX-07.kit.edu> References: <56D054AF2E93E044AC1D2685709D2868C7340018C7@KIT-MSX-07.kit.edu> <56D054AF2E93E044AC1D2685709D2868C9EE571D48@KIT-MSX-07.kit.edu> Message-ID: <87vbyttdae.fsf@jedbrown.org> "Xiao, Jianjun (IKET)" writes: > Hi Matt, > > I am using PETSc-dev because only this version supports > MatSetValuesBlockedStencil in Fortran. I tried many combinations of > KSP and PC. None of them worked in my case. The same thing happened to > PETSc-3.4. > > Then I changed back to 3.3-P7 and 3.2-P7 without using > MatSetValuesBlockedStencil. I tried KSPCR+PCBJACOBI and many others, > they worked fine. We're going to need exact options and output for the cases you tried. You should be able to use the same algorithm with both versions (with suitable options). And if you have a modest-sized problem, you can run with -ksp_view_mat and send binaryoutput. We'll use src/ksp/ksp/examples/tutorials/ex10.c to reproduce the convergence you see. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From jedbrown at mcs.anl.gov Fri Dec 13 01:38:13 2013 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Thu, 12 Dec 2013 23:38:13 -0800 Subject: [petsc-users] GPU local direct solve of penta-diagonal In-Reply-To: References: <52AA2775.6070902@mcs.anl.gov> <52AA2CDB.9000400@ornl.gov> <52AA2FA6.8080802@mcs.anl.gov> <52AA46C8.4030109@txcorp.com> Message-ID: <87sitxtd3u.fsf@jedbrown.org> Paul Mullowney writes: > Provided you have a good parallel sparse direct solve for a single SM, you > could unleash 32 direct solves (or perhaps 16) which run concurrently on > the K20x. One only needs to set an environment variable to use Hypre Q Where is the "good" sparse direct solver that uses one SM and solves a problem with a few thousand degrees of freedom? -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From rupp at mcs.anl.gov Fri Dec 13 03:01:49 2013 From: rupp at mcs.anl.gov (Karl Rupp) Date: Fri, 13 Dec 2013 10:01:49 +0100 Subject: [petsc-users] GPU local direct solve of penta-diagonal In-Reply-To: References: <52AA2775.6070902@mcs.anl.gov> <52AA2CDB.9000400@ornl.gov> <52AA2FA6.8080802@mcs.anl.gov> <52AA46C8.4030109@txcorp.com> Message-ID: <52AACCFD.2080001@mcs.anl.gov> Hi guys, > Provided you have a good parallel sparse direct solve for a single SM, > you could unleash 32 direct solves (or perhaps 16) which run > concurrently on the K20x. One only needs to set an environment variable > to use Hypre Q Thanks for your inputs on HyperQ. I'm afraid this still won't give the good performance Marc and Ed are looking for, mostly because there is simply not enough parallelism in sparse direct solvers for systems of that size (cf. Jed's comment). They might actually work quite well on the CPU if a symbolic factorization is first carried out in a preprocessing step and then the actual numbers are computed in each Picard iteration. Best regards, Karli From jianjun.xiao at kit.edu Fri Dec 13 04:16:14 2013 From: jianjun.xiao at kit.edu (Xiao, Jianjun (IKET)) Date: Fri, 13 Dec 2013 11:16:14 +0100 Subject: [petsc-users] Convergence problem in solving a symmetric positive definite matrix in a CFD code In-Reply-To: <87vbyttdae.fsf@jedbrown.org> References: <56D054AF2E93E044AC1D2685709D2868C7340018C7@KIT-MSX-07.kit.edu> <56D054AF2E93E044AC1D2685709D2868C9EE571D48@KIT-MSX-07.kit.edu>, <87vbyttdae.fsf@jedbrown.org> Message-ID: <56D054AF2E93E044AC1D2685709D2868C9EE571D4B@KIT-MSX-07.kit.edu> Hello Jed and Matt, Thanks for your reply. I design a small 3D case with 10 cells in each direction. Please find the binary files generated with PETSc-dev in the attachment. The matrix should be symmetric and positive definite. The calculation did not converge at the first time step with PCR+JACOBI. With PCG+JACOBI, it seems it converges in this case. I have very limited experience in using PETSc. In the past, I felt more flexible to change the combination of linear solvers and preconditioners. I could get divergence anyhow, and the main difference was the speed. But now, it seems only PCG+JACOBI works fine for my case. Many other combinations, such as PCR+JACOBI, PCR+BJACOBI, PCG+BJACOBI, divergent at the first time step or take much longer time. I think there must be something wrong. Below is how I set up the matrix and the solver. CALL DMSetMatType(da_ksp,MATMPISBAIJ,ierr) CALL DMCreateMatrix(da_ksp,gmat,ierr) CALL DMSetMatrixPreallocateOnly(da_ksp,PETSC_TRUE,ierr) CALL MatMPISBAIJSetPreallocation(gmat,1,4,PETSC_NULL_INTEGER,3,PETSC_NULL_INTEGER,ierr) CALL MatZeroEntries(gmat,ierr) CALL KSPCreate(PETSC_COMM_WORLD,gsolver,ierr) ! CALL KSPSetType(gsolver,KSPCR,ierr) CALL KSPSetType(gsolver,KSPCG,ierr) CALL KSPGetPC(gsolver,gprec,ierr) CALL PCSetType(gprec,PCBJACOBI,ierr) ! CALL PCSetType(gprec,PCBJACOBI,ierr) ********************************** matsetvalues ... IF (k < kbar-1 .AND. j < jbar-1 .AND. i < ibar-1) THEN ! d-x-y-z valpm(1) = value_ad_gfpp(m) colpm(MatStencil_i,1) = i colpm(MatStencil_j,1) = j colpm(MatStencil_k,1) = k valpm(2) = value_ax_gfpp(m) colpm(MatStencil_i,2) = i+1 colpm(MatStencil_j,2) = j colpm(MatStencil_k,2) = k valpm(3) = value_ay_gfpp(m) colpm(MatStencil_i,3) = i colpm(MatStencil_j,3) = j+1 colpm(MatStencil_k,3) = k valpm(4) = value_az_gfpp(m) colpm(MatStencil_i,4) = i colpm(MatStencil_j,4) = j colpm(MatStencil_k,4) = k+1 CALL MatSetValuesBlockedStencil(gmat,1,rowpm,4,colpm,valpm,INSERT_VALUES,ierr) ... ... ********************************** CALL MatAssemblyBegin(gmat,MAT_FINAL_ASSEMBLY,ierr) CALL MatAssemblyEnd(gmat,MAT_FINAL_ASSEMBLY,ierr) CALL KSPSetOperators(gsolver,gmat,gmat,SAME_NONZERO_PATTERN,ierr) epsi_rtol = 1.0e-50_dble epsi_atol = 1.0e-5_dble CALL KSPSetTolerances(gsolver,epsi_rtol, epsi_atol,& & PETSC_DEFAULT_DOUBLE_PRECISION,itmax,ierr) CALL KSPSetNormType(gsolver,KSP_NORM_PRECONDITIONED,ierr) CALL KSPSolve(gsolver,bvec0_gfpp,xsol0_gfpp,ierr) Thank you for your help. JJ ________________________________________ From: Jed Brown [jedbrown at mcs.anl.gov] Sent: Friday, December 13, 2013 8:34 AM To: Xiao, Jianjun (IKET); Matthew Knepley Cc: petsc-users at mcs.anl.gov Subject: Re: [petsc-users] Convergence problem in solving a symmetric positive definite matrix in a CFD code "Xiao, Jianjun (IKET)" writes: > Hi Matt, > > I am using PETSc-dev because only this version supports > MatSetValuesBlockedStencil in Fortran. I tried many combinations of > KSP and PC. None of them worked in my case. The same thing happened to > PETSc-3.4. > > Then I changed back to 3.3-P7 and 3.2-P7 without using > MatSetValuesBlockedStencil. I tried KSPCR+PCBJACOBI and many others, > they worked fine. We're going to need exact options and output for the cases you tried. You should be able to use the same algorithm with both versions (with suitable options). And if you have a modest-sized problem, you can run with -ksp_view_mat and send binaryoutput. We'll use src/ksp/ksp/examples/tutorials/ex10.c to reproduce the convergence you see. -------------- next part -------------- A non-text attachment was scrubbed... Name: binaryoutput_PCR_JACOBI Type: application/octet-stream Size: 184320 bytes Desc: binaryoutput_PCR_JACOBI URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: binaryoutput_PCG_JACOBI Type: application/octet-stream Size: 184320 bytes Desc: binaryoutput_PCG_JACOBI URL: From knepley at gmail.com Fri Dec 13 10:25:57 2013 From: knepley at gmail.com (Matthew Knepley) Date: Fri, 13 Dec 2013 10:25:57 -0600 Subject: [petsc-users] Convergence problem in solving a symmetric positive definite matrix in a CFD code In-Reply-To: <56D054AF2E93E044AC1D2685709D2868C9EE571D4B@KIT-MSX-07.kit.edu> References: <56D054AF2E93E044AC1D2685709D2868C7340018C7@KIT-MSX-07.kit.edu> <56D054AF2E93E044AC1D2685709D2868C9EE571D48@KIT-MSX-07.kit.edu> <87vbyttdae.fsf@jedbrown.org> <56D054AF2E93E044AC1D2685709D2868C9EE571D4B@KIT-MSX-07.kit.edu> Message-ID: On Fri, Dec 13, 2013 at 4:16 AM, Xiao, Jianjun (IKET) wrote: > Hello Jed and Matt, > > Thanks for your reply. > > I design a small 3D case with 10 cells in each direction. Please find the > binary files generated with PETSc-dev in the attachment. The matrix should > be symmetric and positive definite. > > The calculation did not converge at the first time step with PCR+JACOBI. > With PCG+JACOBI, it seems it converges in this case. I have very limited > experience in using PETSc. In the past, I felt more flexible to change the > combination of linear solvers and preconditioners. I could get divergence > anyhow, and the main difference was the speed. But now, it seems only > PCG+JACOBI works fine for my case. Many other combinations, such as > PCR+JACOBI, PCR+BJACOBI, PCG+BJACOBI, divergent at the first time step or > take much longer time. I think there must be something wrong. > Can you see that the above paragraph gives NO IDEA what your problem is? 1) You completely failed to do the one thing I asked you to do. That is not the way to get help. *** Run the same solver with 3.3 and dev. Does it give the same answer *** Until you do this, why would I believe anything has changed. 2) Iterative solvers fail to converge all the time. That is routine, and the reason for preconditioning. Preconditioners are always tied to the specific problem. Thus telling us that something did not converge without outlining the problem, or providing the output I asked for (-ksp_monitor_true_residual -ksp_converged_reason) tells us nothing. Matt > Below is how I set up the matrix and the solver. > > CALL DMSetMatType(da_ksp,MATMPISBAIJ,ierr) > CALL DMCreateMatrix(da_ksp,gmat,ierr) > CALL DMSetMatrixPreallocateOnly(da_ksp,PETSC_TRUE,ierr) > CALL > MatMPISBAIJSetPreallocation(gmat,1,4,PETSC_NULL_INTEGER,3,PETSC_NULL_INTEGER,ierr) > CALL MatZeroEntries(gmat,ierr) > > CALL KSPCreate(PETSC_COMM_WORLD,gsolver,ierr) > ! CALL KSPSetType(gsolver,KSPCR,ierr) > CALL KSPSetType(gsolver,KSPCG,ierr) > > CALL KSPGetPC(gsolver,gprec,ierr) > CALL PCSetType(gprec,PCBJACOBI,ierr) > ! CALL PCSetType(gprec,PCBJACOBI,ierr) > > ********************************** matsetvalues > ... > IF (k < kbar-1 .AND. j < jbar-1 .AND. i < ibar-1) THEN ! d-x-y-z > > valpm(1) = value_ad_gfpp(m) > colpm(MatStencil_i,1) = i > colpm(MatStencil_j,1) = j > colpm(MatStencil_k,1) = k > > valpm(2) = value_ax_gfpp(m) > colpm(MatStencil_i,2) = i+1 > colpm(MatStencil_j,2) = j > colpm(MatStencil_k,2) = k > > valpm(3) = value_ay_gfpp(m) > colpm(MatStencil_i,3) = i > colpm(MatStencil_j,3) = j+1 > colpm(MatStencil_k,3) = k > > valpm(4) = value_az_gfpp(m) > colpm(MatStencil_i,4) = i > colpm(MatStencil_j,4) = j > colpm(MatStencil_k,4) = k+1 > > CALL > MatSetValuesBlockedStencil(gmat,1,rowpm,4,colpm,valpm,INSERT_VALUES,ierr) > ... > > ... > ********************************** > CALL MatAssemblyBegin(gmat,MAT_FINAL_ASSEMBLY,ierr) > CALL MatAssemblyEnd(gmat,MAT_FINAL_ASSEMBLY,ierr) > > CALL KSPSetOperators(gsolver,gmat,gmat,SAME_NONZERO_PATTERN,ierr) > > epsi_rtol = 1.0e-50_dble > epsi_atol = 1.0e-5_dble > > CALL KSPSetTolerances(gsolver,epsi_rtol, epsi_atol,& > & PETSC_DEFAULT_DOUBLE_PRECISION,itmax,ierr) > > CALL KSPSetNormType(gsolver,KSP_NORM_PRECONDITIONED,ierr) > > CALL KSPSolve(gsolver,bvec0_gfpp,xsol0_gfpp,ierr) > > Thank you for your help. > > JJ > > > ________________________________________ > From: Jed Brown [jedbrown at mcs.anl.gov] > Sent: Friday, December 13, 2013 8:34 AM > To: Xiao, Jianjun (IKET); Matthew Knepley > Cc: petsc-users at mcs.anl.gov > Subject: Re: [petsc-users] Convergence problem in solving a symmetric > positive definite matrix in a CFD code > > "Xiao, Jianjun (IKET)" writes: > > > Hi Matt, > > > > I am using PETSc-dev because only this version supports > > MatSetValuesBlockedStencil in Fortran. I tried many combinations of > > KSP and PC. None of them worked in my case. The same thing happened to > > PETSc-3.4. > > > > Then I changed back to 3.3-P7 and 3.2-P7 without using > > MatSetValuesBlockedStencil. I tried KSPCR+PCBJACOBI and many others, > > they worked fine. > > We're going to need exact options and output for the cases you tried. > You should be able to use the same algorithm with both versions (with > suitable options). And if you have a modest-sized problem, you can run > with -ksp_view_mat and send binaryoutput. We'll use > src/ksp/ksp/examples/tutorials/ex10.c to reproduce the convergence you > see. > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From jedbrown at mcs.anl.gov Fri Dec 13 10:37:42 2013 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Fri, 13 Dec 2013 08:37:42 -0800 Subject: [petsc-users] Convergence problem in solving a symmetric positive definite matrix in a CFD code In-Reply-To: <56D054AF2E93E044AC1D2685709D2868C9EE571D4B@KIT-MSX-07.kit.edu> References: <56D054AF2E93E044AC1D2685709D2868C7340018C7@KIT-MSX-07.kit.edu> <56D054AF2E93E044AC1D2685709D2868C9EE571D48@KIT-MSX-07.kit.edu> <87vbyttdae.fsf@jedbrown.org> <56D054AF2E93E044AC1D2685709D2868C9EE571D4B@KIT-MSX-07.kit.edu> Message-ID: <874n6cu2p5.fsf@jedbrown.org> "Xiao, Jianjun (IKET)" writes: > The calculation did not converge at the first time step with > PCR+JACOBI. Your matrix has a negative diagonal, but CR requires an SPD preconditioner. CG requires that too, but it automatically attempts to flip the sign to make up for people formulating as a negative-definite system. Here is what happens if you flip the sign in the file. ~/petsc/src/ksp/ksp/examples/tutorials$ ./ex10 -f /tmp/binaryoutput_PD -rhs 0 -ksp_type cr -pc_type jacobi -ksp_monitor_true_residual -ksp_converged_reason 0 KSP preconditioned resid norm 4.580179811550e-01 true resid norm 1.118033988750e+01 ||r(i)||/||b|| 1.000000000000e+00 1 KSP preconditioned resid norm 3.433059076126e-01 true resid norm 1.171499592651e+01 ||r(i)||/||b|| 1.047821089913e+00 2 KSP preconditioned resid norm 3.224273883310e-01 true resid norm 1.186428209805e+01 ||r(i)||/||b|| 1.061173651019e+00 3 KSP preconditioned resid norm 3.161490448951e-01 true resid norm 1.190487417460e+01 ||r(i)||/||b|| 1.064804316720e+00 4 KSP preconditioned resid norm 3.134497494248e-01 true resid norm 1.191266712565e+01 ||r(i)||/||b|| 1.065501339451e+00 5 KSP preconditioned resid norm 3.116324194377e-01 true resid norm 1.190750014933e+01 ||r(i)||/||b|| 1.065039191039e+00 6 KSP preconditioned resid norm 3.103182665201e-01 true resid norm 1.189333423670e+01 ||r(i)||/||b|| 1.063772153295e+00 7 KSP preconditioned resid norm 3.099146457154e-01 true resid norm 1.188235324536e+01 ||r(i)||/||b|| 1.062789983572e+00 8 KSP preconditioned resid norm 3.098484368758e-01 true resid norm 1.187621039989e+01 ||r(i)||/||b|| 1.062240550770e+00 9 KSP preconditioned resid norm 3.098427227728e-01 true resid norm 1.187506497491e+01 ||r(i)||/||b|| 1.062138100845e+00 10 KSP preconditioned resid norm 3.098403289528e-01 true resid norm 1.187404392635e+01 ||r(i)||/||b|| 1.062046775486e+00 11 KSP preconditioned resid norm 3.098364579073e-01 true resid norm 1.187232365639e+01 ||r(i)||/||b|| 1.061892909863e+00 12 KSP preconditioned resid norm 3.098258616063e-01 true resid norm 1.186975299234e+01 ||r(i)||/||b|| 1.061662982680e+00 13 KSP preconditioned resid norm 3.097972567200e-01 true resid norm 1.186780996443e+01 ||r(i)||/||b|| 1.061489192980e+00 14 KSP preconditioned resid norm 3.094048026767e-01 true resid norm 1.184190035808e+01 ||r(i)||/||b|| 1.059171767338e+00 15 KSP preconditioned resid norm 3.072833013214e-01 true resid norm 1.174521125483e+01 ||r(i)||/||b|| 1.050523631035e+00 16 KSP preconditioned resid norm 2.958951624767e-01 true resid norm 1.115235499681e+01 ||r(i)||/||b|| 9.974969552830e-01 17 KSP preconditioned resid norm 2.755553854179e-01 true resid norm 8.534143514965e+00 ||r(i)||/||b|| 7.633170011680e-01 18 KSP preconditioned resid norm 9.919182748576e-02 true resid norm 2.746916984686e+00 ||r(i)||/||b|| 2.456917242523e-01 19 KSP preconditioned resid norm 1.583757214911e-02 true resid norm 4.932287392439e-01 ||r(i)||/||b|| 4.411571957624e-02 20 KSP preconditioned resid norm 3.653926848384e-03 true resid norm 9.899702253503e-02 ||r(i)||/||b|| 8.854562878336e-03 21 KSP preconditioned resid norm 5.722006539625e-04 true resid norm 2.492171919704e-02 ||r(i)||/||b|| 2.229066329629e-03 22 KSP preconditioned resid norm 1.094837296062e-04 true resid norm 4.041397396052e-03 ||r(i)||/||b|| 3.614735720665e-04 23 KSP preconditioned resid norm 3.160350857762e-05 true resid norm 1.114526226356e-03 ||r(i)||/||b|| 9.968625619355e-05 24 KSP preconditioned resid norm 3.789869369652e-06 true resid norm 1.371099464070e-04 ||r(i)||/||b|| 1.226348642229e-05 Linear solve converged due to CONVERGED_RTOL iterations 24 Number of iterations = 24 Residual norm 0.00013711 That's pretty much the same as CG. If you think something is "wrong", you'll have to show us exactly what you think is wrong, the output you see and what you expect, and provide steps to reproduce. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From jedbrown at mcs.anl.gov Fri Dec 13 11:24:37 2013 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Fri, 13 Dec 2013 09:24:37 -0800 Subject: [petsc-users] Optimal memory layout for finite differences In-Reply-To: References: Message-ID: <87y53oslyi.fsf@jedbrown.org> Mani Chandra writes: > Hi Everyone, > > Would it be a good idea to arrange the data in fastest direction in the > following manner for the ease of aligned loads and vector operations? > > Total grid points = 4n > 0, n, 2n, 3n, 1, n+1, 2n+1, 3n+1 and so on > Ref: "Tuning a Finite Difference Computation for Parallel Vector Processors" > http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=6341495 This paper does not solve a problem. It performs AVX optimization for repeated application of a 1D stencil update using an s-step technique. There are many reasons why the present form of this work is impractical, including that it cannot be used with preconditioning, nor with standard Krylov methods, and that generalizing to multiple spatial dimensions, variable coefficients, etc., throws a wrench in the mix. I recommend skipping this premature optimization of 1% of the actual problem and actually get your science done. There are more practical ways to optimize when the time comes. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From mfadams at lbl.gov Fri Dec 13 21:10:02 2013 From: mfadams at lbl.gov (Mark Adams) Date: Fri, 13 Dec 2013 22:10:02 -0500 Subject: [petsc-users] GPU local direct solve of penta-diagonal In-Reply-To: <52AA46C8.4030109@txcorp.com> References: <52AA2775.6070902@mcs.anl.gov> <52AA2CDB.9000400@ornl.gov> <52AA2FA6.8080802@mcs.anl.gov> <52AA46C8.4030109@txcorp.com> Message-ID: This thread has lost the main developer Ed ... cc'ed along with the PI. Ed and CS, I will forward a few messages on this thread. On Thu, Dec 12, 2013 at 6:29 PM, Dominic Meiser wrote: > Hi Karli, > > > On 12/12/2013 02:50 PM, Karl Rupp wrote: > > > Hmm, this does not sound like something I would consider a good fit for > GPUs. With 16 MPI processes you have additional congestion of the one or > two GPUs per node, so you would have the rethink the solution procedure as > a whole. > > Are you sure about that for Titan? Supposedly the K20X's can deal with > multiple MPI processes hitting a single GPU pretty well using Hyper-Q. Paul > has seen pretty good speed up with small GPU kernels simply by > over-subscribing each GPU with 4 MPI processes. > > See here: > > http://blogs.nvidia.com/blog/2012/08/23/unleash-legacy-mpi-codes-with-keplers-hyper-q/ > > > Cheers, > Dominic > > > -- > Dominic Meiser > Tech-X Corporation > 5621 Arapahoe Avenue > Boulder, CO 80303 > USA > Telephone: 303-996-2036 > Fax: 303-448-7756www.txcorp.com > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mc0710 at gmail.com Fri Dec 13 22:27:51 2013 From: mc0710 at gmail.com (Mani Chandra) Date: Fri, 13 Dec 2013 22:27:51 -0600 Subject: [petsc-users] Seg fault when using DMDATSSetIFunctionLocal with DMComposite Message-ID: Hi, Can the function DMDATSSetIFunctionLocal be used with a global DM which has been constructed from many DMs' combined into a DMComposite? I put a single DM for each one of the variables that I wanted to solve for and combined them in a single one with DMComposite, but the timestepping seg faults with the following error 0x00007ffff69d3786 in VecScatterBegin_SGToSG (ctx=0x924400, x=0x90b640, y=0x920340, addv=INSERT_VALUES, mode=SCATTER_FORWARD) at /home/mc/Downloads/petsc/src/vec/vec/utils/vscat.c:370 370 for (i=0; i From torquil at gmail.com Sat Dec 14 07:46:15 2013 From: torquil at gmail.com (=?ISO-8859-1?Q?Torquil_Macdonald_S=F8rensen?=) Date: Sat, 14 Dec 2013 14:46:15 +0100 Subject: [petsc-users] Problem with TS and PCMG Message-ID: <52AC6127.2010407@gmail.com> Hi! I'm getting segfault problems when trying to use TSStep or TSSolve with TSBEULER and TSLINEAR, together with PCMG. So I have reduced my code to a minimal (and mathematically trivial) test case which displays the same error, and I'm hoping someone here will spot any errors that I've made in my setup of TS. I can use my PCMG restriction and interpolation matrices with success in a single linear solve with KSPSolve (and can see that MG is being used by adding -ksp_view), but for some reason I am not able to get PCMG working with TSStep. So I'm wondering if I'm forgetting a step when when I'm setting up the TS? My test case tries to solve Id*dU/dt = Id*U, where Id is a 5x5 identity matrix, and U is a 5-vector unknown, with initial condition U = 0. The PCMG construction uses an additional coarse grid of 3 points. The fine grid vertices are labelled 0, 1, 2, 3, 4, while the coarse grid is labelled 0, 1, 2, The coarse grid vertices coincide with the fine grid vertices 0, 2 and 4. The resulting interpolation and restriction matrices arise as interpolations matrices between the FEM spaces defined using piecewise linear shape functions on these grids. The GDB message I'm getting is: Program received signal SIGSEGV, Segmentation fault. 0x0000000000000000 in ?? () (gdb) where #0 0x0000000000000000 in ?? () #1 0x00007fa0e6bbd172 in DMCoarsen (dm=0x28e9170, comm=0x7fa0e638d9c0 , dmc=0x295a490) at dm.c:1811 #2 0x00007fa0e6ff1c1e in PCSetUp_MG (pc=0x28d71a0) at mg.c:593 #3 0x00007fa0e72eb6ce in PCSetUp (pc=0x28d71a0) at precon.c:890 #4 0x00007fa0e6e6f7e8 in KSPSetUp (ksp=0x28e1a50) at itfunc.c:278 #5 0x00007fa0e6e70a30 in KSPSolve (ksp=0x28e1a50, b=0x297bea0, x=0x2981970) at itfunc.c:399 #6 0x00007fa0e6e8ae0c in SNESSolve_KSPONLY (snes=0x28c7710) at ksponly.c:44 #7 0x00007fa0e757b642 in SNESSolve (snes=0x28c7710, b=0x0, x=0x296c7d0) at snes.c:3636 #8 0x00007fa0e76280cc in TSStep_Theta (ts=0x2894220) at theta.c:183 #9 0x00007fa0e766a202 in TSStep (ts=0x2894220) at ts.c:2507 #10 0x0000000000403363 in main (argc=3, argv=0x7fff08015378) at /home/tmac/research/schrodinger/source/testcase.cpp:97 My testcase code is: #include "petscts.h" int main(int argc, char ** argv) { PetscErrorCode ierr = PetscInitialize(&argc, &argv, 0, 0); CHKERRQ(ierr); int const nb_dof_fine = 5; int const nb_dof_coarse = 3; Vec x; // A vector to hold the solution on the fine grid ierr = VecCreateSeq(PETSC_COMM_SELF, nb_dof_fine, &x); CHKERRQ(ierr); Mat Id; // Identity matrix on the vector space of the fine grid ierr = MatCreateSeqAIJ(PETSC_COMM_SELF, nb_dof_fine, nb_dof_fine, 1, 0, &Id); CHKERRQ(ierr); for(int i = 0; i != nb_dof_fine; i++) { MatSetValue(Id, i, i, 1.0, INSERT_VALUES); } ierr = MatAssemblyBegin(Id, MAT_FINAL_ASSEMBLY); CHKERRQ(ierr); ierr = MatAssemblyEnd(Id, MAT_FINAL_ASSEMBLY); CHKERRQ(ierr); ierr = MatView(Id, PETSC_VIEWER_STDOUT_SELF); Mat interp; // Multigrid interpolation matrix ierr = MatCreateSeqAIJ(PETSC_COMM_SELF, nb_dof_fine, nb_dof_coarse, 2, 0, &interp); CHKERRQ(ierr); ierr = MatSetValue(interp, 0, 0, 1.0, INSERT_VALUES); CHKERRQ(ierr); ierr = MatSetValue(interp, 1, 0, 0.5, INSERT_VALUES); CHKERRQ(ierr); ierr = MatSetValue(interp, 1, 1, 0.5, INSERT_VALUES); CHKERRQ(ierr); ierr = MatSetValue(interp, 2, 1, 1.0, INSERT_VALUES); CHKERRQ(ierr); ierr = MatSetValue(interp, 3, 1, 0.5, INSERT_VALUES); CHKERRQ(ierr); ierr = MatSetValue(interp, 3, 2, 0.5, INSERT_VALUES); CHKERRQ(ierr); ierr = MatSetValue(interp, 4, 2, 1.0, INSERT_VALUES); CHKERRQ(ierr); ierr = MatAssemblyBegin(interp, MAT_FINAL_ASSEMBLY); CHKERRQ(ierr); ierr = MatAssemblyEnd(interp, MAT_FINAL_ASSEMBLY); CHKERRQ(ierr); ierr = MatView(interp, PETSC_VIEWER_STDOUT_SELF); Mat restrict; // Multigrid restriction matrix, not the transpose of "interp" ierr = MatCreateSeqAIJ(PETSC_COMM_SELF, nb_dof_coarse, nb_dof_fine, 1, 0, &restrict); CHKERRQ(ierr); ierr = MatSetValue(restrict, 0, 0, 1.0, INSERT_VALUES); CHKERRQ(ierr); ierr = MatSetValue(restrict, 1, 2, 1.0, INSERT_VALUES); CHKERRQ(ierr); ierr = MatSetValue(restrict, 2, 4, 1.0, INSERT_VALUES); CHKERRQ(ierr); ierr = MatAssemblyBegin(restrict, MAT_FINAL_ASSEMBLY); CHKERRQ(ierr); ierr = MatAssemblyEnd(restrict, MAT_FINAL_ASSEMBLY); CHKERRQ(ierr); ierr = MatView(restrict, PETSC_VIEWER_STDOUT_SELF); { // Solving the linear problem Id*x = 0 works KSP ksp; ierr = KSPCreate(PETSC_COMM_SELF, &ksp); CHKERRQ(ierr); ierr = KSPSetType(ksp, KSPFGMRES); CHKERRQ(ierr); ierr = KSPSetOperators(ksp, Id, Id, SAME_PRECONDITIONER); CHKERRQ(ierr); PC pc; ierr = KSPGetPC(ksp, &pc); CHKERRQ(ierr); ierr = PCSetType(pc, PCMG); CHKERRQ(ierr); ierr = PCMGSetLevels(pc, 2, 0); CHKERRQ(ierr); ierr = PCMGSetGalerkin(pc, PETSC_TRUE); CHKERRQ(ierr); ierr = PCMGSetInterpolation(pc, 1, interp); CHKERRQ(ierr); ierr = PCMGSetRestriction(pc, 1, restrict); CHKERRQ(ierr); ierr = KSPSetFromOptions(ksp); CHKERRQ(ierr); Vec b; // Trivial RHS for the linear problem ierr = VecDuplicate(x, &b); CHKERRQ(ierr); ierr = VecSet(b, 0); CHKERRQ(ierr); ierr = KSPSolve(ksp, b, x); CHKERRQ(ierr); ierr = VecView(x, PETSC_VIEWER_STDOUT_SELF); ierr = VecDestroy(&b); CHKERRQ(ierr); ierr = KSPDestroy(&ksp); CHKERRQ(ierr); ierr = PetscPrintf(PETSC_COMM_SELF, "KSPSolve worked\n"); CHKERRQ(ierr); } { // This block of code causes a segfault TS ts; ierr = TSCreate(PETSC_COMM_SELF, &ts); CHKERRQ(ierr); ierr = TSSetProblemType(ts, TS_LINEAR); CHKERRQ(ierr); ierr = TSSetType(ts, TSBEULER); // Use an implicit scheme ierr = TSSetInitialTimeStep(ts, 0.0, 0.1); CHKERRQ(ierr); ierr = TSSetSolution(ts, x); CHKERRQ(ierr); ierr = TSSetRHSFunction(ts, 0, TSComputeRHSFunctionLinear, 0); CHKERRQ(ierr); ierr = TSSetRHSJacobian(ts, Id, Id, TSComputeRHSJacobianConstant, 0); CHKERRQ(ierr); ierr = TSSetIFunction(ts, 0, TSComputeIFunctionLinear, 0); CHKERRQ(ierr); ierr = TSSetIJacobian(ts, Id, Id, TSComputeIJacobianConstant, 0); KSP ksp; ierr = TSGetKSP(ts, &ksp); CHKERRQ(ierr); ierr = KSPSetType(ksp, KSPFGMRES); CHKERRQ(ierr); PC pc; ierr = KSPGetPC(ksp, &pc); CHKERRQ(ierr); ierr = PCSetType(pc, PCMG); CHKERRQ(ierr); ierr = PCMGSetLevels(pc, 2, 0); CHKERRQ(ierr); ierr = PCMGSetGalerkin(pc, PETSC_TRUE); CHKERRQ(ierr); ierr = PCMGSetInterpolation(pc, 1, interp); CHKERRQ(ierr); ierr = PCMGSetRestriction(pc, 1, restrict); CHKERRQ(ierr); ierr = TSSetUp(ts); CHKERRQ(ierr); ierr = TSStep(ts); CHKERRQ(ierr); ierr = TSDestroy(&ts); CHKERRQ(ierr); } ierr = MatDestroy(&restrict); CHKERRQ(ierr); ierr = MatDestroy(&interp); CHKERRQ(ierr); ierr = VecDestroy(&x); CHKERRQ(ierr); ierr = MatDestroy(&Id); CHKERRQ(ierr); ierr = PetscFinalize(); CHKERRQ(ierr); return 0; } The actual output of the program when running without a debugger is: Matrix Object: 1 MPI processes type: seqaij row 0: (0, 1) row 1: (1, 1) row 2: (2, 1) row 3: (3, 1) row 4: (4, 1) Matrix Object: 1 MPI processes type: seqaij row 0: (0, 1) row 1: (0, 0.5) (1, 0.5) row 2: (1, 1) row 3: (1, 0.5) (2, 0.5) row 4: (2, 1) Matrix Object: 1 MPI processes type: seqaij row 0: (0, 1) row 1: (2, 1) row 2: (4, 1) Vector Object: 1 MPI processes type: seq 0 0 0 0 0 KSPSolve worked [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: Caught signal number 11 SEGV: Segmentation Violation, probably memory access out of range [0]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger [0]PETSC ERROR: or see http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind[0]PETSC ERROR: or try http://valgrind.org on GNU/linux and Apple Mac OS X to find memory corruption errors [0]PETSC ERROR: likely location of problem given in stack below [0]PETSC ERROR: --------------------- Stack Frames ------------------------------------ [0]PETSC ERROR: Note: The EXACT line numbers in the stack are not available, [0]PETSC ERROR: INSTEAD the line number of the start of the function [0]PETSC ERROR: is given. [0]PETSC ERROR: [0] DMCoarsen line 1809 src/dm/interface/dm.c [0]PETSC ERROR: [0] PCSetUp_MG line 546 src/ksp/pc/impls/mg/mg.c [0]PETSC ERROR: [0] PCSetUp line 868 src/ksp/pc/interface/precon.c [0]PETSC ERROR: [0] KSPSetUp line 192 src/ksp/ksp/interface/itfunc.c [0]PETSC ERROR: [0] KSPSolve line 356 src/ksp/ksp/interface/itfunc.c [0]PETSC ERROR: [0] SNESSolve_KSPONLY line 14 src/snes/impls/ksponly/ksponly.c [0]PETSC ERROR: [0] SNESSolve line 3589 src/snes/interface/snes.c [0]PETSC ERROR: [0] TSStep_Theta line 162 src/ts/impls/implicit/theta/theta.c [0]PETSC ERROR: [0] TSStep line 2499 src/ts/interface/ts.c [0]PETSC ERROR: --------------------- Error Message ------------------------------------ [0]PETSC ERROR: Signal received! [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: Petsc Release Version 3.4.3, Oct, 15, 2013 [0]PETSC ERROR: See docs/changes/index.html for recent updates. [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. [0]PETSC ERROR: See docs/index.html for manual pages. [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: ./testcase.elf on a linux-gnu-c-debug named lenovo by tmac Sat Dec 14 14:35:56 2013 [0]PETSC ERROR: Libraries linked from /tmp/petsc-3.4.3/linux-gnu-c-debug/lib [0]PETSC ERROR: Configure run at Fri Nov 22 11:07:41 2013 [0]PETSC ERROR: Configure options --with-shared-libraries --with-debugging=1 --useThreads 0 --with-clanguage=C++ --with-c-support --with-fortran-interfaces=1 --with-scalar-type=complex --with-mpi-dir=/usr/lib/openmpi --with-blas-lib=-lblas --with-lapack-lib=-llapack --with-blacs=1 --with-blacs-include=/usr/include --with-blacs-lib="[/usr/lib/libblacsCinit-openmpi.so,/usr/lib/libblacs-openmpi.so]" --with-scalapack=1 --with-scalapack-include=/usr/include --with-scalapack-lib=/usr/lib/libscalapack-openmpi.so --with-mumps=1 --with-mumps-include=/usr/include --with-mumps-lib="[/usr/lib/libdmumps.so,/usr/lib/libzmumps.so,/usr/lib/libsmumps.so,/usr/lib/libcmumps.so,/usr/lib/libmumps_common.so,/usr/lib/libpord.so]" --with-umfpack=1 --with-umfpack-include=/usr/include/suitesparse --with-umfpack-lib="[/usr/lib/libumfpack.so,/usr/lib/libamd.so]" --with-cholmod=1 --with-cholmod-include=/usr/include/suitesparse --with-cholmod-lib=/usr/lib/libcholmod.so --with-spooles=1 --with-spooles-include=/usr/include/spooles --with-spooles-lib=/usr/lib/libspooles.so --with-ptscotch=1 --with-ptscotch-include=/usr/include/scotch --with-ptscotch-lib="[/usr/lib/libptesmumps.so,/usr/lib/libptscotch.so,/usr/lib/libptscotcherr.so]" --with-fftw=1 --with-fftw-include=/usr/include --with-fftw-lib="[/usr/lib/x86_64-linux-gnu/libfftw3.so,/usr/lib/x86_64-linux-gnu/libfftw3_mpi.so]" --with-superlu=1 --with-superlu-include=/usr/include/superlu --with-superlu-lib=/usr/lib/libsuperlu.so --CXX_LINKER_FLAGS=-Wl,--no-as-needed [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: User provided function() line 0 in unknown directory unknown file -------------------------------------------------------------------------- MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD with errorcode 59. NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes. You may or may not see output from other processes, depending on exactly when Open MPI kills them. -------------------------------------------------------------------------- Best regards Torquil S?rensen From jedbrown at mcs.anl.gov Sat Dec 14 10:21:08 2013 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Sat, 14 Dec 2013 08:21:08 -0800 Subject: [petsc-users] Seg fault when using DMDATSSetIFunctionLocal with DMComposite In-Reply-To: References: Message-ID: <87r49fjte3.fsf@jedbrown.org> Mani Chandra writes: > Can the function DMDATSSetIFunctionLocal be used with a global DM which has > been constructed from many DMs' combined into a DMComposite? No, DMDA* functions can only be used with DMDA, not DMCOMPOSITE or any other DM implementation. > I put a single DM for each one of the variables that I wanted to solve > for and combined them in a single one with DMComposite, but the > timestepping seg faults with the following error > > 0x00007ffff69d3786 in VecScatterBegin_SGToSG (ctx=0x924400, x=0x90b640, > y=0x920340, addv=INSERT_VALUES, mode=SCATTER_FORWARD) at > /home/mc/Downloads/petsc/src/vec/vec/utils/vscat.c:370 > 370 for (i=0; i > Would the only way to use a DMComposite with TS be using TSSetIFunction and > then scatters the DMComposite to individual DMs' inside the function? Yes. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From adrimacortes at gmail.com Sat Dec 14 10:38:26 2013 From: adrimacortes at gmail.com (=?ISO-8859-1?Q?Adriano_C=F4rtes?=) Date: Sat, 14 Dec 2013 14:38:26 -0200 Subject: [petsc-users] Fwd: Questions using PCFIELDSPLIT In-Reply-To: References: Message-ID: Hi all, I'm trying to play with PCFIELDSPLIT, but after reading I'm still confused interpreting the monitors. I'm trying to solve a Stokes problem [ A B^t ] M = [ ] [ B 0 ] My first question is about the MINRES behaviour with pcfieldsplit. My options for pcfieldsplit are ierr = PCFieldSplitSetType(pc,PC_COMPOSITE_SCHUR);CHKERRQ(ierr); PCFieldSplitSchurFactType schurfact = PC_FIELDSPLIT_SCHUR_FACT_DIAG; ierr = PCFieldSplitSetSchurFactType(pc,schurfact);CHKERRQ(ierr); PCFieldSplitSchurPreType schurpre = PC_FIELDSPLIT_SCHUR_PRE_USER; where I'm providing the pressure mass matrix as a preconditioner for S. When I run with -snes_type ksponly -ksp_type minres -pc_type fieldsplit #Velocity -fieldsplit_u_ksp_type cg -fieldsplit_u_pc_type jacobi -fieldsplit_u_ksp_rtol 1e-05 -fieldsplit_u_ksp_monitor #Pressure -fieldsplit_p_ksp_type cg -fieldsplit_p_pc_type jacobi -fieldsplit_p_ksp_rtol 1e-05 -fieldsplit_p_ksp_monitor the solution does not converge (i'm testing an analytical one), but when I use -ksp_type gmres it converges. If you guys want I can dump both monitors outputs on a file an attach on the email. My second question is: what options I have to set up if I want to use as preconditionr for M the matrix [ A 0 ] P = [ ] [ 0 Q ] where again Q is the pressure mass matrix, and with A and Q being solved by CG with BJACOBI. Thanks in advance. Adriano. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jedbrown at mcs.anl.gov Sat Dec 14 11:29:30 2013 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Sat, 14 Dec 2013 09:29:30 -0800 Subject: [petsc-users] Fwd: Questions using PCFIELDSPLIT In-Reply-To: References: Message-ID: <87lhznjq85.fsf@jedbrown.org> Adriano C?rtes writes: > Hi all, > I'm trying to play with PCFIELDSPLIT, but after reading I'm still confused > interpreting the monitors. I'm trying to solve a Stokes problem > > [ A B^t ] > M = [ ] > [ B 0 ] > > My first question is about the MINRES behaviour with pcfieldsplit. My > options for pcfieldsplit are > > ierr = PCFieldSplitSetType(pc,PC_COMPOSITE_SCHUR);CHKERRQ(ierr); > > PCFieldSplitSchurFactType schurfact = PC_FIELDSPLIT_SCHUR_FACT_DIAG; > ierr = PCFieldSplitSetSchurFactType(pc,schurfact);CHKERRQ(ierr); > > PCFieldSplitSchurPreType schurpre = PC_FIELDSPLIT_SCHUR_PRE_USER; Use -ksp_view to check that these are being used. I recommend setting all these options using run-time options unless your code is taking active control over the choice of algorithm. > where I'm providing the pressure mass matrix as a preconditioner for S. Where are you putting it? > When I run with > > -snes_type ksponly > -ksp_type minres > -pc_type fieldsplit > > #Velocity > -fieldsplit_u_ksp_type cg > -fieldsplit_u_pc_type jacobi > -fieldsplit_u_ksp_rtol 1e-05 > -fieldsplit_u_ksp_monitor > > #Pressure > -fieldsplit_p_ksp_type cg > -fieldsplit_p_pc_type jacobi > -fieldsplit_p_ksp_rtol 1e-05 > -fieldsplit_p_ksp_monitor > > the solution does not converge "does not converge" is to imprecise to say anything. What exactly do you observe? > (i'm testing an analytical one), but when I use -ksp_type gmres it > converges. Are you sure your matrices are symmetric? (Including boundary conditions.) > If you guys want I can dump both monitors outputs on a file an attach > on the email. > > My second question is: what options I have to set up if I want to use as > preconditionr for M the matrix > > [ A 0 ] > P = [ ] > [ 0 Q ] > > where again Q is the pressure mass matrix, and with A and Q being solved by > CG with BJACOBI. I can't tell for sure from what you've sent, but PCFIELDSPLIT is likely iterating on the actual Schur complement S = - B A^{-1} B^T, preconditioned by your matrix Q. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From adrimacortes at gmail.com Sat Dec 14 13:08:50 2013 From: adrimacortes at gmail.com (=?ISO-8859-1?Q?Adriano_C=F4rtes?=) Date: Sat, 14 Dec 2013 17:08:50 -0200 Subject: [petsc-users] Fwd: Questions using PCFIELDSPLIT In-Reply-To: <87lhznjq85.fsf@jedbrown.org> References: <87lhznjq85.fsf@jedbrown.org> Message-ID: 2013/12/14 Jed Brown : > Adriano C?rtes writes: > >> Hi all, >> I'm trying to play with PCFIELDSPLIT, but after reading I'm still confused >> interpreting the monitors. I'm trying to solve a Stokes problem >> >> [ A B^t ] >> M = [ ] >> [ B 0 ] >> >> My first question is about the MINRES behaviour with pcfieldsplit. My >> options for pcfieldsplit are >> >> ierr = PCFieldSplitSetType(pc,PC_COMPOSITE_SCHUR);CHKERRQ(ierr); >> >> PCFieldSplitSchurFactType schurfact = PC_FIELDSPLIT_SCHUR_FACT_DIAG; >> ierr = PCFieldSplitSetSchurFactType(pc,schurfact);CHKERRQ(ierr); >> >> PCFieldSplitSchurPreType schurpre = PC_FIELDSPLIT_SCHUR_PRE_USER; > > Use -ksp_view to check that these are being used. I recommend setting > all these options using run-time options unless your code is taking > active control over the choice of algorithm. > Actually it is being done. Check the -snes_view bellow 0 SNES Function norm 9.644918978350e-02 0 KSP Residual norm 1.297271248442e-01 1 SNES Function norm 9.644918978350e-02 Nonlinear solve did not converge due to DIVERGED_LINEAR_SOLVE iterations 1 SNES Object: 1 MPI processes type: ksponly maximum iterations=50, maximum function evaluations=10000 tolerances: relative=1e-05, absolute=1e-50, solution=1e-08 total number of linear solver iterations=1 total number of function evaluations=2 SNESLineSearch Object: 1 MPI processes type: basic maxstep=1.000000e+08, minlambda=1.000000e-12 tolerances: relative=1.000000e-08, absolute=1.000000e-15, lambda=1.000000e-08 maximum iterations=1 KSP Object: 1 MPI processes type: minres maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using PRECONDITIONED norm type for convergence test PC Object: 1 MPI processes type: fieldsplit FieldSplit with Schur preconditioner, factorization DIAG Preconditioner for the Schur complement formed from user provided matrix Split info: Split number 0 Defined by IS Split number 1 Defined by IS KSP solver for A00 block KSP Object: (fieldsplit_u_) 1 MPI processes type: cg maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using PRECONDITIONED norm type for convergence test PC Object: (fieldsplit_u_) 1 MPI processes type: jacobi linear system matrix = precond matrix: Matrix Object: 1 MPI processes type: seqaij rows=220, cols=220 total: nonzeros=11552, allocated nonzeros=11552 total number of mallocs used during MatSetValues calls =0 not using I-node routines KSP solver for S = A11 - A10 inv(A00) A01 KSP Object: (fieldsplit_p_) 1 MPI processes type: cg maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using PRECONDITIONED norm type for convergence test PC Object: (fieldsplit_p_) 1 MPI processes type: jacobi linear system matrix followed by preconditioner matrix: Matrix Object: 1 MPI processes type: schurcomplement rows=100, cols=100 Schur complement A11 - A10 inv(A00) A01 A11 Matrix Object: 1 MPI processes type: seqaij rows=100, cols=100 total: nonzeros=1936, allocated nonzeros=1936 total number of mallocs used during MatSetValues calls =0 not using I-node routines A10 Matrix Object: 1 MPI processes type: seqaij rows=100, cols=220 total: nonzeros=4752, allocated nonzeros=4752 total number of mallocs used during MatSetValues calls =0 not using I-node routines KSP of A00 KSP Object: (fieldsplit_u_) 1 MPI processes type: cg maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using PRECONDITIONED norm type for convergence test PC Object: (fieldsplit_u_) 1 MPI processes type: jacobi linear system matrix = precond matrix: Matrix Object: 1 MPI processes type: seqaij rows=220, cols=220 total: nonzeros=11552, allocated nonzeros=11552 total number of mallocs used during MatSetValues calls =0 not using I-node routines A01 Matrix Object: 1 MPI processes type: seqaij rows=220, cols=100 total: nonzeros=4752, allocated nonzeros=4752 total number of mallocs used during MatSetValues calls =0 not using I-node routines Matrix Object: 1 MPI processes type: seqaij rows=100, cols=100 total: nonzeros=1936, allocated nonzeros=1936 total number of mallocs used during MatSetValues calls =0 not using I-node routines linear system matrix = precond matrix: Matrix Object: 1 MPI processes type: seqaij rows=320, cols=320 total: nonzeros=22992, allocated nonzeros=24996 total number of mallocs used during MatSetValues calls =1156 not using I-node routines L2 errors: u=0.0132378 p=0.0593741 >> where I'm providing the pressure mass matrix as a preconditioner for S. > > Where are you putting it? > I'm calling a callback function to compute Q (pressure mass matrix) and calling ierr = PCFieldSplitSchurPrecondition(pc,schurpre,Q);CHKERRQ(ierr); where schurpre = PC_FIELDSPLIT_SCHUR_PRE_USER; >> When I run with >> >> -snes_type ksponly >> -ksp_type minres >> -pc_type fieldsplit >> >> #Velocity >> -fieldsplit_u_ksp_type cg >> -fieldsplit_u_pc_type jacobi >> -fieldsplit_u_ksp_rtol 1e-05 >> -fieldsplit_u_ksp_monitor >> >> #Pressure >> -fieldsplit_p_ksp_type cg >> -fieldsplit_p_pc_type jacobi >> -fieldsplit_p_ksp_rtol 1e-05 >> -fieldsplit_p_ksp_monitor >> >> the solution does not converge > > "does not converge" is to imprecise to say anything. What exactly do > you observe? > Since it's an analytical solution I'm computing the L2 Norm. When I run with -ksp_type minres I get 0 SNES Function norm 9.644918978350e-02 0 KSP Residual norm 1.297271248442e-01 1 SNES Function norm 9.644918978350e-02 Nonlinear solve did not converge due to DIVERGED_LINEAR_SOLVE iterations 1 L2 errors: u=0.0132378 p=0.0593741 and when I run with -ksp_type gmres I get 0 SNES Function norm 9.644918978350e-02 0 KSP Residual norm 1.297271248442e-01 1 KSP Residual norm 1.269027428700e-01 2 KSP Residual norm 1.177913273071e-01 3 KSP Residual norm 4.746342719564e-05 4 KSP Residual norm 4.242617241067e-05 5 KSP Residual norm 9.541981683274e-07 1 SNES Function norm 1.468436883950e-06 Nonlinear solve converged due to CONVERGED_ITS iterations 1 L2 errors: u=4.097e-05 p=8.95462e-05 >> (i'm testing an analytical one), but when I use -ksp_type gmres it >> converges. > > Are you sure your matrices are symmetric? (Including boundary conditions.) > It supposed to be, but now that you asked I will check it, because this can be why minres isn't working for me. >> If you guys want I can dump both monitors outputs on a file an attach >> on the email. >> >> My second question is: what options I have to set up if I want to use as >> preconditionr for M the matrix >> >> [ A 0 ] >> P = [ ] >> [ 0 Q ] >> >> where again Q is the pressure mass matrix, and with A and Q being solved by >> CG with BJACOBI. > > I can't tell for sure from what you've sent, but PCFIELDSPLIT is likely > iterating on the actual Schur complement S = - B A^{-1} B^T, > preconditioned by your matrix Q. I guess I did not make myself clear. It was supposed to be a new question, that is, what I'm doing now is what you said above, but since I'm using an inf-sup stable discretization Q and S are spectral equivalent, what I'd like to do after is to precondition the Stokes system with A and Q, that is using ksp(A) and ksp(Q) to apply the block-diagonal preconditioner. Adriano. From jedbrown at mcs.anl.gov Sat Dec 14 13:19:36 2013 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Sat, 14 Dec 2013 11:19:36 -0800 Subject: [petsc-users] Fwd: Questions using PCFIELDSPLIT In-Reply-To: References: <87lhznjq85.fsf@jedbrown.org> Message-ID: <87y53ni6k7.fsf@jedbrown.org> Adriano C?rtes writes: > 2013/12/14 Jed Brown : >> Adriano C?rtes writes: >> Use -ksp_view to check that these are being used. I recommend setting >> all these options using run-time options unless your code is taking >> active control over the choice of algorithm. >> > > > Actually it is being done. Check the -snes_view bellow > > 0 SNES Function norm 9.644918978350e-02 > 0 KSP Residual norm 1.297271248442e-01 > 1 SNES Function norm 9.644918978350e-02 > Nonlinear solve did not converge due to DIVERGED_LINEAR_SOLVE iterations 1 > SNES Object: 1 MPI processes > type: ksponly > maximum iterations=50, maximum function evaluations=10000 > tolerances: relative=1e-05, absolute=1e-50, solution=1e-08 > total number of linear solver iterations=1 > total number of function evaluations=2 > SNESLineSearch Object: 1 MPI processes > type: basic > maxstep=1.000000e+08, minlambda=1.000000e-12 > tolerances: relative=1.000000e-08, absolute=1.000000e-15, > lambda=1.000000e-08 > maximum iterations=1 > KSP Object: 1 MPI processes > type: minres > maximum iterations=10000, initial guess is zero > tolerances: relative=1e-05, absolute=1e-50, divergence=10000 > left preconditioning > using PRECONDITIONED norm type for convergence test > PC Object: 1 MPI processes > type: fieldsplit > FieldSplit with Schur preconditioner, factorization DIAG > Preconditioner for the Schur complement formed from user provided matrix > Split info: > Split number 0 Defined by IS > Split number 1 Defined by IS > KSP solver for A00 block > KSP Object: (fieldsplit_u_) 1 MPI processes > type: cg > maximum iterations=10000, initial guess is zero > tolerances: relative=1e-05, absolute=1e-50, divergence=10000 > left preconditioning > using PRECONDITIONED norm type for convergence test > PC Object: (fieldsplit_u_) 1 MPI processes > type: jacobi > linear system matrix = precond matrix: > Matrix Object: 1 MPI processes > type: seqaij > rows=220, cols=220 > total: nonzeros=11552, allocated nonzeros=11552 > total number of mallocs used during MatSetValues calls =0 > not using I-node routines > KSP solver for S = A11 - A10 inv(A00) A01 > KSP Object: (fieldsplit_p_) 1 MPI processes > type: cg > maximum iterations=10000, initial guess is zero > tolerances: relative=1e-05, absolute=1e-50, divergence=10000 > left preconditioning > using PRECONDITIONED norm type for convergence test > PC Object: (fieldsplit_p_) 1 MPI processes > type: jacobi > linear system matrix followed by preconditioner matrix: > Matrix Object: 1 MPI processes > type: schurcomplement See that "fieldsplit_p_" uses a MATSCHURCOMPLEMENT and actually iterates on that reduced system. This is different from what is typically done in full-space iterative methods. > rows=100, cols=100 > Schur complement A11 - A10 inv(A00) A01 > A11 > Matrix Object: 1 MPI processes > type: seqaij > rows=100, cols=100 > total: nonzeros=1936, allocated nonzeros=1936 > total number of mallocs used during MatSetValues calls =0 > not using I-node routines > A10 > Matrix Object: 1 MPI processes > type: seqaij > rows=100, cols=220 > total: nonzeros=4752, allocated nonzeros=4752 > total number of mallocs used during MatSetValues calls =0 > not using I-node routines > KSP of A00 > KSP Object: (fieldsplit_u_) > 1 MPI processes > type: cg > maximum iterations=10000, initial guess is zero > tolerances: relative=1e-05, absolute=1e-50, divergence=10000 > left preconditioning > using PRECONDITIONED norm type for convergence test > PC Object: (fieldsplit_u_) > 1 MPI processes > type: jacobi > linear system matrix = precond matrix: > Matrix Object: 1 MPI processes > type: seqaij > rows=220, cols=220 > total: nonzeros=11552, allocated nonzeros=11552 > total number of mallocs used during MatSetValues calls =0 > not using I-node routines > A01 > Matrix Object: 1 MPI processes > type: seqaij > rows=220, cols=100 > total: nonzeros=4752, allocated nonzeros=4752 > total number of mallocs used during MatSetValues calls =0 > not using I-node routines > Matrix Object: 1 MPI processes > type: seqaij > rows=100, cols=100 > total: nonzeros=1936, allocated nonzeros=1936 > total number of mallocs used during MatSetValues calls =0 > not using I-node routines > linear system matrix = precond matrix: > Matrix Object: 1 MPI processes > type: seqaij > rows=320, cols=320 > total: nonzeros=22992, allocated nonzeros=24996 > total number of mallocs used during MatSetValues calls =1156 > not using I-node routines > L2 errors: u=0.0132378 p=0.0593741 > >>> where I'm providing the pressure mass matrix as a preconditioner for S. >> >> Where are you putting it? >> > > I'm calling a callback function to compute Q (pressure mass matrix) and calling > ierr = PCFieldSplitSchurPrecondition(pc,schurpre,Q);CHKERRQ(ierr); > where schurpre = PC_FIELDSPLIT_SCHUR_PRE_USER; > >>> When I run with >>> >>> -snes_type ksponly >>> -ksp_type minres >>> -pc_type fieldsplit >>> >>> #Velocity >>> -fieldsplit_u_ksp_type cg >>> -fieldsplit_u_pc_type jacobi >>> -fieldsplit_u_ksp_rtol 1e-05 >>> -fieldsplit_u_ksp_monitor >>> >>> #Pressure >>> -fieldsplit_p_ksp_type cg >>> -fieldsplit_p_pc_type jacobi >>> -fieldsplit_p_ksp_rtol 1e-05 >>> -fieldsplit_p_ksp_monitor >>> >>> the solution does not converge >> >> "does not converge" is to imprecise to say anything. What exactly do >> you observe? >> > > Since it's an analytical solution I'm computing the L2 Norm. When I > run with -ksp_type minres I get > > 0 SNES Function norm 9.644918978350e-02 > 0 KSP Residual norm 1.297271248442e-01 > 1 SNES Function norm 9.644918978350e-02 > Nonlinear solve did not converge due to DIVERGED_LINEAR_SOLVE iterations 1 Use -ksp_converged_reason to find out why the linear solve above did not converge. Maybe due to an indefinite preconditioner? > L2 errors: u=0.0132378 p=0.0593741 > > and when I run with -ksp_type gmres I get > > 0 SNES Function norm 9.644918978350e-02 > 0 KSP Residual norm 1.297271248442e-01 > 1 KSP Residual norm 1.269027428700e-01 > 2 KSP Residual norm 1.177913273071e-01 > 3 KSP Residual norm 4.746342719564e-05 > 4 KSP Residual norm 4.242617241067e-05 > 5 KSP Residual norm 9.541981683274e-07 > 1 SNES Function norm 1.468436883950e-06 > Nonlinear solve converged due to CONVERGED_ITS iterations 1 > L2 errors: u=4.097e-05 p=8.95462e-05 > > >>> (i'm testing an analytical one), but when I use -ksp_type gmres it >>> converges. >> >> Are you sure your matrices are symmetric? (Including boundary conditions.) >> > > It supposed to be, but now that you asked I will check it, because > this can be why minres isn't working for me. Yes that or if the preconditioner is not SPD. That could be as simple as getting the sign wrong in a preconditioning matrix. >>> If you guys want I can dump both monitors outputs on a file an attach >>> on the email. >>> >>> My second question is: what options I have to set up if I want to use as >>> preconditionr for M the matrix >>> >>> [ A 0 ] >>> P = [ ] >>> [ 0 Q ] >>> >>> where again Q is the pressure mass matrix, and with A and Q being solved by >>> CG with BJACOBI. >> >> I can't tell for sure from what you've sent, but PCFIELDSPLIT is likely >> iterating on the actual Schur complement S = - B A^{-1} B^T, >> preconditioned by your matrix Q. > > I guess I did not make myself clear. It was supposed to be a new > question, that is, what I'm doing now is what you said above, but > since I'm using an inf-sup stable discretization Q and S are spectral > equivalent, what I'd like to do after is to precondition the Stokes > system with A and Q, that is using ksp(A) and ksp(Q) to apply the > block-diagonal preconditioner. If you want to use an iterative method for Q, you can skip over the matrix-free Schur complement operator by using something like -fieldsplit_p_ksp_type preonly -fieldsplit_p_pc_type ksp, though usually one V-cycle is enough and keep the iteration linear regardless of tolerances. You might look at the src/ksp/ksp/examples/tutorials/ex42.c runs (see the makefile or ex42*.opts) for examples of solving similar systems. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From adrimacortes at gmail.com Sat Dec 14 14:00:11 2013 From: adrimacortes at gmail.com (=?ISO-8859-1?Q?Adriano_C=F4rtes?=) Date: Sat, 14 Dec 2013 18:00:11 -0200 Subject: [petsc-users] Fwd: Questions using PCFIELDSPLIT In-Reply-To: <87y53ni6k7.fsf@jedbrown.org> References: <87lhznjq85.fsf@jedbrown.org> <87y53ni6k7.fsf@jedbrown.org> Message-ID: 2013/12/14 Jed Brown > > Adriano C?rtes writes: > > > 2013/12/14 Jed Brown : > >> Adriano C?rtes writes: > >> Use -ksp_view to check that these are being used. I recommend setting > >> all these options using run-time options unless your code is taking > >> active control over the choice of algorithm. > >> > > > > > > Actually it is being done. Check the -snes_view bellow > > > > 0 SNES Function norm 9.644918978350e-02 > > 0 KSP Residual norm 1.297271248442e-01 > > 1 SNES Function norm 9.644918978350e-02 > > Nonlinear solve did not converge due to DIVERGED_LINEAR_SOLVE iterations 1 > > SNES Object: 1 MPI processes > > type: ksponly > > maximum iterations=50, maximum function evaluations=10000 > > tolerances: relative=1e-05, absolute=1e-50, solution=1e-08 > > total number of linear solver iterations=1 > > total number of function evaluations=2 > > SNESLineSearch Object: 1 MPI processes > > type: basic > > maxstep=1.000000e+08, minlambda=1.000000e-12 > > tolerances: relative=1.000000e-08, absolute=1.000000e-15, > > lambda=1.000000e-08 > > maximum iterations=1 > > KSP Object: 1 MPI processes > > type: minres > > maximum iterations=10000, initial guess is zero > > tolerances: relative=1e-05, absolute=1e-50, divergence=10000 > > left preconditioning > > using PRECONDITIONED norm type for convergence test > > PC Object: 1 MPI processes > > type: fieldsplit > > FieldSplit with Schur preconditioner, factorization DIAG > > Preconditioner for the Schur complement formed from user provided matrix > > Split info: > > Split number 0 Defined by IS > > Split number 1 Defined by IS > > KSP solver for A00 block > > KSP Object: (fieldsplit_u_) 1 MPI processes > > type: cg > > maximum iterations=10000, initial guess is zero > > tolerances: relative=1e-05, absolute=1e-50, divergence=10000 > > left preconditioning > > using PRECONDITIONED norm type for convergence test > > PC Object: (fieldsplit_u_) 1 MPI processes > > type: jacobi > > linear system matrix = precond matrix: > > Matrix Object: 1 MPI processes > > type: seqaij > > rows=220, cols=220 > > total: nonzeros=11552, allocated nonzeros=11552 > > total number of mallocs used during MatSetValues calls =0 > > not using I-node routines > > KSP solver for S = A11 - A10 inv(A00) A01 > > KSP Object: (fieldsplit_p_) 1 MPI processes > > type: cg > > maximum iterations=10000, initial guess is zero > > tolerances: relative=1e-05, absolute=1e-50, divergence=10000 > > left preconditioning > > using PRECONDITIONED norm type for convergence test > > PC Object: (fieldsplit_p_) 1 MPI processes > > type: jacobi > > linear system matrix followed by preconditioner matrix: > > Matrix Object: 1 MPI processes > > type: schurcomplement > > See that "fieldsplit_p_" uses a MATSCHURCOMPLEMENT and actually iterates > on that reduced system. This is different from what is typically done > in full-space iterative methods. > Dear Jed, Can you elaborate more this comment? What is a MATSCHURCOMPLEMENT object? You mean that S is built? Or is it matrix-free as you said bellow? > > rows=100, cols=100 > > Schur complement A11 - A10 inv(A00) A01 > > A11 > > Matrix Object: 1 MPI processes > > type: seqaij > > rows=100, cols=100 > > total: nonzeros=1936, allocated nonzeros=1936 > > total number of mallocs used during MatSetValues calls =0 > > not using I-node routines > > A10 > > Matrix Object: 1 MPI processes > > type: seqaij > > rows=100, cols=220 > > total: nonzeros=4752, allocated nonzeros=4752 > > total number of mallocs used during MatSetValues calls =0 > > not using I-node routines > > KSP of A00 > > KSP Object: (fieldsplit_u_) > > 1 MPI processes > > type: cg > > maximum iterations=10000, initial guess is zero > > tolerances: relative=1e-05, absolute=1e-50, divergence=10000 > > left preconditioning > > using PRECONDITIONED norm type for convergence test > > PC Object: (fieldsplit_u_) > > 1 MPI processes > > type: jacobi > > linear system matrix = precond matrix: > > Matrix Object: 1 MPI processes > > type: seqaij > > rows=220, cols=220 > > total: nonzeros=11552, allocated nonzeros=11552 > > total number of mallocs used during MatSetValues calls =0 > > not using I-node routines > > A01 > > Matrix Object: 1 MPI processes > > type: seqaij > > rows=220, cols=100 > > total: nonzeros=4752, allocated nonzeros=4752 > > total number of mallocs used during MatSetValues calls =0 > > not using I-node routines > > Matrix Object: 1 MPI processes > > type: seqaij > > rows=100, cols=100 > > total: nonzeros=1936, allocated nonzeros=1936 > > total number of mallocs used during MatSetValues calls =0 > > not using I-node routines > > linear system matrix = precond matrix: > > Matrix Object: 1 MPI processes > > type: seqaij > > rows=320, cols=320 > > total: nonzeros=22992, allocated nonzeros=24996 > > total number of mallocs used during MatSetValues calls =1156 > > not using I-node routines > > L2 errors: u=0.0132378 p=0.0593741 > > > >>> where I'm providing the pressure mass matrix as a preconditioner for S. > >> > >> Where are you putting it? > >> > > > > I'm calling a callback function to compute Q (pressure mass matrix) and calling > > ierr = PCFieldSplitSchurPrecondition(pc,schurpre,Q);CHKERRQ(ierr); > > where schurpre = PC_FIELDSPLIT_SCHUR_PRE_USER; > > > >>> When I run with > >>> > >>> -snes_type ksponly > >>> -ksp_type minres > >>> -pc_type fieldsplit > >>> > >>> #Velocity > >>> -fieldsplit_u_ksp_type cg > >>> -fieldsplit_u_pc_type jacobi > >>> -fieldsplit_u_ksp_rtol 1e-05 > >>> -fieldsplit_u_ksp_monitor > >>> > >>> #Pressure > >>> -fieldsplit_p_ksp_type cg > >>> -fieldsplit_p_pc_type jacobi > >>> -fieldsplit_p_ksp_rtol 1e-05 > >>> -fieldsplit_p_ksp_monitor > >>> > >>> the solution does not converge > >> > >> "does not converge" is to imprecise to say anything. What exactly do > >> you observe? > >> > > > > Since it's an analytical solution I'm computing the L2 Norm. When I > > run with -ksp_type minres I get > > > > 0 SNES Function norm 9.644918978350e-02 > > 0 KSP Residual norm 1.297271248442e-01 > > 1 SNES Function norm 9.644918978350e-02 > > Nonlinear solve did not converge due to DIVERGED_LINEAR_SOLVE iterations 1 > > Use -ksp_converged_reason to find out why the linear solve above did not > converge. Maybe due to an indefinite preconditioner? > This was there result of -ksp_converged_reason 0 SNES Function norm 9.644918978350e-02 0 KSP Residual norm 1.297271248442e-01 Linear solve did not converge due to DIVERGED_INDEFINITE_MAT iterations 1 1 SNES Function norm 9.644918978350e-02 Nonlinear solve did not converge due to DIVERGED_LINEAR_SOLVE iterations 1 SNES Object: 1 MPI processes This I didn't understand, because minres is supposed to work for symmetric indefinite matrices. Isn't it? What am I missing here? > > L2 errors: u=0.0132378 p=0.0593741 > > > > and when I run with -ksp_type gmres I get > > > > 0 SNES Function norm 9.644918978350e-02 > > 0 KSP Residual norm 1.297271248442e-01 > > 1 KSP Residual norm 1.269027428700e-01 > > 2 KSP Residual norm 1.177913273071e-01 > > 3 KSP Residual norm 4.746342719564e-05 > > 4 KSP Residual norm 4.242617241067e-05 > > 5 KSP Residual norm 9.541981683274e-07 > > 1 SNES Function norm 1.468436883950e-06 > > Nonlinear solve converged due to CONVERGED_ITS iterations 1 > > L2 errors: u=4.097e-05 p=8.95462e-05 > > > > > >>> (i'm testing an analytical one), but when I use -ksp_type gmres it > >>> converges. > >> > >> Are you sure your matrices are symmetric? (Including boundary conditions.) > >> > > > > It supposed to be, but now that you asked I will check it, because > > this can be why minres isn't working for me. > > Yes that or if the preconditioner is not SPD. That could be as simple > as getting the sign wrong in a preconditioning matrix. > > >>> If you guys want I can dump both monitors outputs on a file an attach > >>> on the email. > >>> > >>> My second question is: what options I have to set up if I want to use as > >>> preconditionr for M the matrix > >>> > >>> [ A 0 ] > >>> P = [ ] > >>> [ 0 Q ] > >>> > >>> where again Q is the pressure mass matrix, and with A and Q being solved by > >>> CG with BJACOBI. > >> > >> I can't tell for sure from what you've sent, but PCFIELDSPLIT is likely > >> iterating on the actual Schur complement S = - B A^{-1} B^T, > >> preconditioned by your matrix Q. > > > > I guess I did not make myself clear. It was supposed to be a new > > question, that is, what I'm doing now is what you said above, but > > since I'm using an inf-sup stable discretization Q and S are spectral > > equivalent, what I'd like to do after is to precondition the Stokes > > system with A and Q, that is using ksp(A) and ksp(Q) to apply the > > block-diagonal preconditioner. > > If you want to use an iterative method for Q, you can skip over the > matrix-free Schur complement operator by using something like > -fieldsplit_p_ksp_type preonly -fieldsplit_p_pc_type ksp, though usually > one V-cycle is enough and keep the iteration linear regardless of > tolerances. You might look at the src/ksp/ksp/examples/tutorials/ex42.c > runs (see the makefile or ex42*.opts) for examples of solving similar > systems. I will take a look. Thanks. But just to know if I set -fieldsplit_p_ksp_type preonly -fieldsplit_p_pc_type ksp, how do I choose the ksp? Multigrid V-cycle as you said is the default in this case? Thank you. Adriano From jedbrown at mcs.anl.gov Sat Dec 14 14:06:21 2013 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Sat, 14 Dec 2013 12:06:21 -0800 Subject: [petsc-users] Fwd: Questions using PCFIELDSPLIT In-Reply-To: References: <87lhznjq85.fsf@jedbrown.org> <87y53ni6k7.fsf@jedbrown.org> Message-ID: <87sitvi4ea.fsf@jedbrown.org> Adriano C?rtes writes: > 2013/12/14 Jed Brown >> >> See that "fieldsplit_p_" uses a MATSCHURCOMPLEMENT and actually iterates >> on that reduced system. This is different from what is typically done >> in full-space iterative methods. >> > > Dear Jed, > Can you elaborate more this comment? What is a MATSCHURCOMPLEMENT > object? You mean that S is built? Or is it matrix-free as you said > bellow? It is a special Mat type that applies the action of the Schur complement via the definition rather than by explicitly computing its entries. This is often referred to as "matrix-free", though it is still a Mat in PETSc (Mat in PETSc means "finite-dimensional linear operator"). >> Use -ksp_converged_reason to find out why the linear solve above did not >> converge. Maybe due to an indefinite preconditioner? >> > > This was there result of -ksp_converged_reason > > 0 SNES Function norm 9.644918978350e-02 > 0 KSP Residual norm 1.297271248442e-01 > Linear solve did not converge due to DIVERGED_INDEFINITE_MAT iterations 1 > 1 SNES Function norm 9.644918978350e-02 > Nonlinear solve did not converge due to DIVERGED_LINEAR_SOLVE iterations 1 > SNES Object: 1 MPI processes > > This I didn't understand, because minres is supposed to work for > symmetric indefinite matrices. Isn't it? What am I missing here? MINRES requires an SPD preconditioner. As I guessed earlier, you may have just gotten the sign wrong for the preconditioner. > I will take a look. Thanks. But just to know if I set > -fieldsplit_p_ksp_type preonly -fieldsplit_p_pc_type ksp, how do I > choose the ksp? -fieldsplit_p_ksp_ksp_type cg. > Multigrid V-cycle as you said is the default in this case? No, but it would be a good choice if the pressure Schur complement is similar to a Laplacian. For Stokes (as opposed to finite-time step or Navier-Stokes), the Schur complement is spectrally equivalent to a scaled mass matrix, so most people use Jacobi. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From arashm81 at gmail.com Sun Dec 15 17:47:11 2013 From: arashm81 at gmail.com (Arash Mehraban) Date: Sun, 15 Dec 2013 16:47:11 -0700 Subject: [petsc-users] vtk format Message-ID: Hi all, I was wondering if PETSc could write my (vector) solution into a *vtk*format file? I think DMDAVTKWriteAll might work, but I couldn't get it to work. For example, how could I store the solution in ex29.c on http://www.mcs.anl.gov/petsc/petsc-current/src/ksp/ksp/examples/tutorials/ex29.c.html into a vtk-format file? Thanks in advance, Arash -------------- next part -------------- An HTML attachment was scrubbed... URL: From adrimacortes at gmail.com Sun Dec 15 19:18:42 2013 From: adrimacortes at gmail.com (=?ISO-8859-1?Q?Adriano_C=F4rtes?=) Date: Sun, 15 Dec 2013 23:18:42 -0200 Subject: [petsc-users] Fwd: Questions using PCFIELDSPLIT In-Reply-To: <87sitvi4ea.fsf@jedbrown.org> References: <87lhznjq85.fsf@jedbrown.org> <87y53ni6k7.fsf@jedbrown.org> <87sitvi4ea.fsf@jedbrown.org> Message-ID: Dear Jed, Your comments helped a lot, and yes there was a sign problem on the linear system (that is the linear system was being computed as skew-symmetric), but I'm still confused of what is going on and what Petsc is doing inside. With the same setting of the last emails if I use the options -snes_type ksponly -ksp_type minres -pc_type fieldsplit -ksp_rtol 1e-10 -snes_rtol 1e-5 #Velocity -fieldsplit_u_ksp_type cg -fieldsplit_u_pc_type bjacobi -fieldsplit_u_ksp_rtol 1e-03 #Pressure -fieldsplit_p_ksp_type cg -fieldsplit_p_pc_type bjacobi -fieldsplit_p_ksp_rtol 1e-05 then I obtain the following output 0 SNES Function norm 9.644918978350e-02 0 KSP Residual norm 1.297267655138e-01 1 KSP Residual norm 4.149750591421e-02 2 KSP Residual norm 4.122610029985e-02 3 KSP Residual norm 4.791668571219e-06 4 KSP Residual norm 2.031018181616e-06 5 KSP Residual norm 1.909762237646e-06 6 KSP Residual norm 1.731835783849e-07 7 KSP Residual norm 1.341665413528e-07 8 KSP Residual norm 8.811364282590e-08 9 KSP Residual norm 4.886961323938e-08 10 KSP Residual norm 2.900680379628e-08 [...] 937 KSP Residual norm 1.313544933163e-11 938 KSP Residual norm 1.311355904193e-11 939 KSP Residual norm 1.309345645355e-11 940 KSP Residual norm 1.307133575538e-11 941 KSP Residual norm 1.305066151443e-11 942 KSP Residual norm 1.302844549705e-11 943 KSP Residual norm 1.300734852025e-11 944 KSP Residual norm 1.298516726546e-11 945 KSP Residual norm 1.296378673406e-11 Linear solve converged due to CONVERGED_RTOL iterations 945 1 SNES Function norm 6.472010982055e-07 Nonlinear solve converged due to CONVERGED_ITS iterations 1 L2 errors: u=4.09689e-05 p=8.94374e-05 BUT if I just change -fieldsplit_p_ksp_rtol to 1e-03 I get the output 0 SNES Function norm 9.644918978350e-02 0 KSP Residual norm 1.297267655138e-01 1 KSP Residual norm 4.149750203710e-02 2 KSP Residual norm 4.122609648562e-02 3 KSP Residual norm 6.532932382460e-06 4 KSP Residual norm 5.527048080699e-06 5 KSP Residual norm 3.343796902211e-06 6 KSP Residual norm 7.517943074624e-08 Linear solve did not converge due to DIVERGED_INDEFINITE_MAT iterations 7 1 SNES Function norm 6.833028247701e-07 Nonlinear solve did not converge due to DIVERGED_LINEAR_SOLVE iterations 1 L2 errors: u=4.09697e-05 p=8.95148e-05 That is in the first case MINRES is converging (very slowly) after 945 iterations, but in the second it is signaled as DIVERGED_INDEFINITE_MAT. Another information: I used the options -fieldsplit_u_ksp_converged_reason and -fieldsplit_p_ksp_converged_reason for both cases, and for both in every iteration I get Linear solve converged due to CONVERGED_RTOL iterations XXX Can you help interpret those behaviors? Any clue? Thanks again. Adriano. 2013/12/14 Jed Brown > > Adriano C?rtes writes: > > > 2013/12/14 Jed Brown > >> > >> See that "fieldsplit_p_" uses a MATSCHURCOMPLEMENT and actually iterates > >> on that reduced system. This is different from what is typically done > >> in full-space iterative methods. > >> > > > > Dear Jed, > > Can you elaborate more this comment? What is a MATSCHURCOMPLEMENT > > object? You mean that S is built? Or is it matrix-free as you said > > bellow? > > It is a special Mat type that applies the action of the Schur complement > via the definition rather than by explicitly computing its entries. > This is often referred to as "matrix-free", though it is still a Mat in > PETSc (Mat in PETSc means "finite-dimensional linear operator"). > > >> Use -ksp_converged_reason to find out why the linear solve above did not > >> converge. Maybe due to an indefinite preconditioner? > >> > > > > This was there result of -ksp_converged_reason > > > > 0 SNES Function norm 9.644918978350e-02 > > 0 KSP Residual norm 1.297271248442e-01 > > Linear solve did not converge due to DIVERGED_INDEFINITE_MAT iterations 1 > > 1 SNES Function norm 9.644918978350e-02 > > Nonlinear solve did not converge due to DIVERGED_LINEAR_SOLVE iterations 1 > > SNES Object: 1 MPI processes > > > > This I didn't understand, because minres is supposed to work for > > symmetric indefinite matrices. Isn't it? What am I missing here? > > MINRES requires an SPD preconditioner. As I guessed earlier, you may > have just gotten the sign wrong for the preconditioner. > > > I will take a look. Thanks. But just to know if I set > > -fieldsplit_p_ksp_type preonly -fieldsplit_p_pc_type ksp, how do I > > choose the ksp? > > -fieldsplit_p_ksp_ksp_type cg. > > > Multigrid V-cycle as you said is the default in this case? > > No, but it would be a good choice if the pressure Schur complement is > similar to a Laplacian. For Stokes (as opposed to finite-time step or > Navier-Stokes), the Schur complement is spectrally equivalent to a > scaled mass matrix, so most people use Jacobi. From qince168 at gmail.com Mon Dec 16 06:15:25 2013 From: qince168 at gmail.com (Ce Qin) Date: Mon, 16 Dec 2013 20:15:25 +0800 Subject: [petsc-users] Feature request: Origin QMR implementation. Message-ID: Dear all, Now I'm solving Maxwell's equations using finite difference method. In the beginning, I use BCGS to solve the linear system. It works, but converges slowly. And other solvers do not converge. Some others told me that QMR may be a good choice, so I tested it using matlab, the result shows that QMR works well than BCGS. I found that PETSc only provide TFQMR and TCQMR, so I wonder that why PETSc does not implement QMR solver. And is there a alternative solver that behaves like QMR? Thanks. Best regards, Ce Qin -------------- next part -------------- An HTML attachment was scrubbed... URL: From daveliu at mit.edu Mon Dec 16 12:02:11 2013 From: daveliu at mit.edu (David Liu) Date: Mon, 16 Dec 2013 13:02:11 -0500 Subject: [petsc-users] smallest SVD values in Slepc doesn't work for matrices larger than ~200 Message-ID: Hi, I'm having trouble using the SVD solver to find the smallest singular values. As a simple example: http://www.grycap.upv.es/slepc/handson/handson4.html When I run the command in the example: (./ex14 -file $SLEPC_DIR/share/slepc/datafiles/matrices/rdb200.petsc), it converges exactly as the page says it would, in 3 iterations. However, if I want the smallest values, I add the option "-svd_smallest" to the end of that command, and this time it doesn't converge at all. Is there some additional option I need to add, for example, something to do with the SVD object's EPS or ST, to make this work? best, David -------------- next part -------------- An HTML attachment was scrubbed... URL: From jroman at dsic.upv.es Mon Dec 16 12:21:35 2013 From: jroman at dsic.upv.es (Jose E. Roman) Date: Mon, 16 Dec 2013 19:21:35 +0100 Subject: [petsc-users] smallest SVD values in Slepc doesn't work for matrices larger than ~200 In-Reply-To: References: Message-ID: El 16/12/2013, a las 19:02, David Liu escribi?: > Hi, I'm having trouble using the SVD solver to find the smallest singular values. As a simple example: > > http://www.grycap.upv.es/slepc/handson/handson4.html > > When I run the command in the example: (./ex14 -file $SLEPC_DIR/share/slepc/datafiles/matrices/rdb200.petsc), it converges exactly as the page says it would, in 3 iterations. However, if I want the smallest values, I add the option > > "-svd_smallest" > > to the end of that command, and this time it doesn't converge at all. Is there some additional option I need to add, for example, something to do with the SVD object's EPS or ST, to make this work? > > best, > David In this example, it is enough to increase the dimension of the working subspace, for instance -svd_ncv 32 Usually, eigensolvers have a much harder time when computing smallest eigenvalues compared to largest, because the latter are dominant and hinder convergence of the small ones. In other problems, increasing the subspace may not be enough and other approaches might be required (e.g. shift-and-invert as described in the manual). It depends on the particular distribution of the spectrum. Jose From knepley at gmail.com Mon Dec 16 14:55:00 2013 From: knepley at gmail.com (Matthew Knepley) Date: Mon, 16 Dec 2013 14:55:00 -0600 Subject: [petsc-users] vtk format In-Reply-To: References: Message-ID: On Sun, Dec 15, 2013 at 5:47 PM, Arash Mehraban wrote: > Hi all, > > I was wondering if PETSc could write my (vector) solution into a *vtk*format file? > I think DMDAVTKWriteAll might work, but I couldn't get it to work. For > example, how could I store the solution in ex29.c on > http://www.mcs.anl.gov/petsc/petsc-current/src/ksp/ksp/examples/tutorials/ex29.c.html > into a vtk-format file? > You should be able to use -vec_view ::vtk from the command line or PetscViewerSetType(viewer, PETSCVIEWERVTK) for binary output, and PetscViewerPushFormat(viewer, PETSC_VIEWER_ASCII_VTK) for text. Thanks, Matt > Thanks in advance, > > Arash > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Mon Dec 16 19:08:42 2013 From: knepley at gmail.com (Matthew Knepley) Date: Mon, 16 Dec 2013 19:08:42 -0600 Subject: [petsc-users] vtk format In-Reply-To: References: Message-ID: On Mon, Dec 16, 2013 at 6:33 PM, Arash Mehraban wrote: > > Dear Matt, > > I added the following lines of code to ex29.c to get the solution in a vtk > file format: > > PetscViewer viewer; > > ierr = PetscViewerCreate(PETSC_COMM_WORLD, &viewer);CHKERRQ(ierr); > ierr = PetscViewerSetType(viewer, PETSCVIEWERVTK);CHKERRQ(ierr); > > ierr = PetscViewerFileSetName(viewer, "poisson.vtk");CHKERRQ(ierr); > ierr = PetscViewerSetFormat(viewer, > PETSC_VIEWER_ASCII_VTK);CHKERRQ(ierr); > DMDAVTKWriteAll((PetscObject)da, viewer); > This is mixing a few things. I think you want binary VTK, so ierr = PetscViewerCreate(PETSC_COMM_WORLD, &viewer);CHKERRQ(ierr); ierr = PetscViewerSetType(viewer, PETSCVIEWERVTK);CHKERRQ(ierr); ierr = PetscViewerFileSetName(viewer, "poisson.vtu");CHKERRQ(ierr); ierr = VecView(v);CHKERRQ(ierr); ierr = PetscViewerDestroy(&viewer);CHKERRQ(ierr); Matt > ierr = PetscViewerDestroy(&viewer);CHKERRQ(ierr); > > > I get errors. I have put a part of the list of errors I get in the > logfile: > > [37]PETSC ERROR: Configure run at Mon Sep 9 15:18:06 2013 > [37]PETSC ERROR: Configure options --with-mpi=1 --download-f-blas-lapack=1 > --with-clanguage=cxx > [37]PETSC ERROR: > ------------------------------------------------------------------------ > [37]PETSC ERROR: User provided function() line 0 in unknown directory > unknown file > or try http://valgrind.org on GNU/linux and Apple Mac OS X to find memory > corruption errors > See docs/index.html for manual pages. > [39]PETSC ERROR: > ------------------------------------------------------------------------ > [39]PETSC ERROR: ./ex29 on a arch-linux2-cxx-debug named node0310 by > arme5062 Mon Dec 16 17:20:55 2013 > [39]PETSC ERROR: Libraries linked from > /projects/arme5062/petsc/arch-linux2-cxx-debug/lib > [39]PETSC ERROR: Configure run at Mon Sep 9 15:18:06 2013 > [39]PETSC ERROR: Configure options --with-mpi=1 --download-f-blas-lapack=1 > --with-clanguage=cxx > [39]PETSC ERROR: > ------------------------------------------------------------------------ > [39]PETSC ERROR: User provided function() line 0 in unknown directory > unknown file > Note: The EXACT line numbers in the stack are not available, > [20]PETSC ERROR: INSTEAD the line number of the start of the function > [20]PETSC ERROR: is given. > [20]PETSC ERROR: [20] PetscTraceBackErrorHandler line 172 > src/sys/error/errtrace.c > [20]PETSC ERROR: [20] PetscError line 361 src/sys/error/err.c > [20]PETSC ERROR: [20] DMDAVTKWriteAll line 249 src/dm/impls/da/grvtk.c > [20]PETSC ERROR: --------------------- Error Message > ------------------------------------ > [20]PETSC ERROR: Signal received! > [20]PETSC ERROR: > ------------------------------------------------------------------------ > [20]PETSC ERROR: Petsc Release Version 3.4.2, unknown > [20]PETSC ERROR: See docs/changes/index.html for recent updates. > [20]PETSC ERROR: See docs/faq.html for hints about trouble shooting. > [20]PETSC ERROR: See docs/index.html for manual pages. > [20]PETSC ERROR: > ------------------------------------------------------------------------ > [20]PETSC ERROR: [40]PETSC ERROR: Configure run at Mon Sep 9 15:18:06 2013 > [40]PETSC ERROR: Configure options --with-mpi=1 --download-f-blas-lapack=1 > --with-clanguage=cxx > [40]PETSC ERROR: > ------------------------------------------------------------------------ > [40]PETSC ERROR: User provided function() line 0 in unknown directory > unknown file > ./ex29 on a arch-linux2-cxx-debug named node0311 by arme5062 Mon Dec 16 > 17:20:55 2013 > [20]PETSC ERROR: Libraries linked from > /projects/arme5062/petsc/arch-linux2-cxx-debug/lib > [20]PETSC ERROR: Configure run at Mon Sep 9 15:18:06 2013 > [20]PETSC ERROR: Configure options --with-mpi=1 --download-f-blas-lapack=1 > --with-clanguage=cxx > [20]PETSC ERROR: > ------------------------------------------------------------------------ > [20]PETSC ERROR: User provided function() line 0 in unknown directory > unknown file > [46]PETSC ERROR: > ------------------------------------------------------------------------ > [46]PETSC ERROR: Caught signal number 11 SEGV: Segmentation Violation, > probably memory access out of range > [46]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger > [46]PETSC ERROR: or see > http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind[46]PETSCERROR: or try > http://valgrind.org on GNU/linux and Apple Mac OS X to find memory > corruption errors > [19]PETSC ERROR: likely location of problem given in stack below > [19]PETSC ERROR: --------------------- Stack Frames > ------------------------------------ > [19]PETSC ERROR: Note: The EXACT line numbers in the stack are not > available, > [19]PETSC ERROR: INSTEAD the line number of the start of the function > [19]PETSC ERROR: is given. > [19]PETSC ERROR: [19] PetscTraceBackErrorHandler line 172 > src/sys/error/errtrace.c > [19]PETSC ERROR: [19] PetscError line 361 src/sys/error/err.c > [19]PETSC ERROR: [19] DMDAVTKWriteAll line 249 src/dm/impls/da/grvtk.c > [19]PETSC ERROR: --------------------- Error Message > ------------------------------------ > [19]PETSC ERROR: Signal received! > [19]PETSC ERROR: > ------------------------------------------------------------------------ > [19]PETSC ERROR: Petsc Release Version 3.4.2, unknown > [19]PETSC ERROR: See docs/changes/index.html for recent updates. > [19]PETSC ERROR: See docs/faq.html for hints about trouble shooting. > [19]PETSC ERROR: See docs/index.html for manual pages. > [19]PETSC ERROR: > ------------------------------------------------------------------------ > [19]PETSC ERROR: [42]PETSC ERROR: likely location of problem given in > stack below > [42]PETSC ERROR: --------------------- Stack Frames > ------------------------------------ > [42]PETSC ERROR: Note: The EXACT line numbers in the stack are not > available, > [42]PETSC ERROR: INSTEAD the line number of the start of the function > [42]PETSC ERROR: is given. > [42]PETSC ERROR: [42] PetscTraceBackErrorHandler line 172 > src/sys/error/errtrace.c > [42]PETSC ERROR: [42] PetscError line 361 src/sys/error/err.c > [42]PETSC ERROR: [42] DMDAVTKWriteAll line 249 src/dm/impls/da/grvtk.c > [42]PETSC ERROR: --------------------- Error Message > ------------------------------------ > [42]PETSC ERROR: Signal received! > [42]PETSC ERROR: > ------------------------------------------------------------------------ > [42]PETSC ERROR: Petsc Release Version 3.4.2, unknown > [42]PETSC ERROR: See docs/changes/index.html for recent updates. > [42]PETSC ERROR: See docs/faq.html for hints about trouble shooting. > [42]PETSC ERROR: See docs/index.html for manual pages. > > > > > > On Mon, Dec 16, 2013 at 1:55 PM, Matthew Knepley wrote: > >> On Sun, Dec 15, 2013 at 5:47 PM, Arash Mehraban wrote: >> >>> Hi all, >>> >>> I was wondering if PETSc could write my (vector) solution into a *vtk*format file? >>> I think DMDAVTKWriteAll might work, but I couldn't get it to work. For >>> example, how could I store the solution in ex29.c on >>> http://www.mcs.anl.gov/petsc/petsc-current/src/ksp/ksp/examples/tutorials/ex29.c.html >>> into a vtk-format file? >>> >> >> You should be able to use -vec_view ::vtk from the command line or >> PetscViewerSetType(viewer, PETSCVIEWERVTK) for binary >> output, and PetscViewerPushFormat(viewer, PETSC_VIEWER_ASCII_VTK) for >> text. >> >> Thanks, >> >> Matt >> >> >>> Thanks in advance, >>> >>> Arash >>> >> >> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which their >> experiments lead. >> -- Norbert Wiener >> > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From katyghantous at gmail.com Tue Dec 17 20:00:17 2013 From: katyghantous at gmail.com (Katy Ghantous) Date: Wed, 18 Dec 2013 03:00:17 +0100 Subject: [petsc-users] problems with Pets Complex Message-ID: I have configured Petsc to have the scalar type comlpex. It compliles and runs but the resulting imaginary part is always 0. i am just trying to do PetscScalar a = 1.+PETSC_i; printf("%G\n",PetscImaginaryPart(a)); but i get 0. -------------- next part -------------- An HTML attachment was scrubbed... URL: From torquil at gmail.com Wed Dec 18 04:14:57 2013 From: torquil at gmail.com (=?UTF-8?Q?Torquil_Macdonald_S=C3=B8rensen?=) Date: Wed, 18 Dec 2013 11:14:57 +0100 Subject: [petsc-users] Problem with TS and PCMG In-Reply-To: <52AC6127.2010407@gmail.com> References: <52AC6127.2010407@gmail.com> Message-ID: Hi! Just a second try in case the first email was unintentionally missed. In short, I'm wondering if it is possible to use TSSolve without using any DM structure: When I was debugging my program I tried to analyse the difference between the TSSolve-based time iteration program, and a version where I do the time-stepping manually using a for-loop which repeateed KSPSolve functions. The debugger showed that in the TSSolve program a pointer named ts->snes->ksp->pc->dm was nonzero, but in the for-loop program it was zero. In the TSSolve-program, and pc->dm != 0 made it enter a block of code within PCSetUp_MG that called DMCoarsen, which then crashed. So I guess I'm wondering how to tell PETSc that I'm not using any DM structure. Best regards Torquil S?rensen On 14 December 2013 14:46, Torquil Macdonald S?rensen wrote: > Hi! > > I'm getting segfault problems when trying to use TSStep or TSSolve with > TSBEULER and TSLINEAR, together with PCMG. So I have reduced my code to > a minimal (and mathematically trivial) test case which displays the same > error, and I'm hoping someone here will spot any errors that I've made > in my setup of TS. > > I can use my PCMG restriction and interpolation matrices with success in > a single linear solve with KSPSolve (and can see that MG is being used > by adding -ksp_view), but for some reason I am not able to get PCMG > working with TSStep. So I'm wondering if I'm forgetting a step when when > I'm setting up the TS? > > My test case tries to solve Id*dU/dt = Id*U, where Id is a 5x5 identity > matrix, and U is a 5-vector unknown, with initial condition U = 0. The > PCMG construction uses an additional coarse grid of 3 points. The fine > grid vertices are labelled 0, 1, 2, 3, 4, while the coarse grid is > labelled 0, 1, 2, The coarse grid vertices coincide with the fine grid > vertices 0, 2 and 4. The resulting interpolation and restriction > matrices arise as interpolations matrices between the FEM spaces defined > using piecewise linear shape functions on these grids. > > The GDB message I'm getting is: > > Program received signal SIGSEGV, Segmentation fault. > 0x0000000000000000 in ?? () > (gdb) where > #0 0x0000000000000000 in ?? () > #1 0x00007fa0e6bbd172 in DMCoarsen (dm=0x28e9170, comm=0x7fa0e638d9c0 > , dmc=0x295a490) at dm.c:1811 > #2 0x00007fa0e6ff1c1e in PCSetUp_MG (pc=0x28d71a0) at mg.c:593 > #3 0x00007fa0e72eb6ce in PCSetUp (pc=0x28d71a0) at precon.c:890 > #4 0x00007fa0e6e6f7e8 in KSPSetUp (ksp=0x28e1a50) at itfunc.c:278 > #5 0x00007fa0e6e70a30 in KSPSolve (ksp=0x28e1a50, b=0x297bea0, > x=0x2981970) at itfunc.c:399 > #6 0x00007fa0e6e8ae0c in SNESSolve_KSPONLY (snes=0x28c7710) at > ksponly.c:44 > #7 0x00007fa0e757b642 in SNESSolve (snes=0x28c7710, b=0x0, x=0x296c7d0) > at snes.c:3636 > #8 0x00007fa0e76280cc in TSStep_Theta (ts=0x2894220) at theta.c:183 > #9 0x00007fa0e766a202 in TSStep (ts=0x2894220) at ts.c:2507 > #10 0x0000000000403363 in main (argc=3, argv=0x7fff08015378) at > /home/tmac/research/schrodinger/source/testcase.cpp:97 > > My testcase code is: > > #include "petscts.h" > > int main(int argc, char ** argv) > { > PetscErrorCode ierr = PetscInitialize(&argc, &argv, 0, 0); > CHKERRQ(ierr); > > int const nb_dof_fine = 5; > int const nb_dof_coarse = 3; > > Vec x; // A vector to hold the solution on the fine grid > ierr = VecCreateSeq(PETSC_COMM_SELF, nb_dof_fine, &x); CHKERRQ(ierr); > > Mat Id; // Identity matrix on the vector space of the fine grid > ierr = MatCreateSeqAIJ(PETSC_COMM_SELF, nb_dof_fine, nb_dof_fine, 1, > 0, &Id); CHKERRQ(ierr); > for(int i = 0; i != nb_dof_fine; i++) { MatSetValue(Id, i, i, 1.0, > INSERT_VALUES); } > ierr = MatAssemblyBegin(Id, MAT_FINAL_ASSEMBLY); CHKERRQ(ierr); > ierr = MatAssemblyEnd(Id, MAT_FINAL_ASSEMBLY); CHKERRQ(ierr); > ierr = MatView(Id, PETSC_VIEWER_STDOUT_SELF); > > Mat interp; // Multigrid interpolation matrix > ierr = MatCreateSeqAIJ(PETSC_COMM_SELF, nb_dof_fine, nb_dof_coarse, > 2, 0, &interp); CHKERRQ(ierr); > ierr = MatSetValue(interp, 0, 0, 1.0, INSERT_VALUES); CHKERRQ(ierr); > ierr = MatSetValue(interp, 1, 0, 0.5, INSERT_VALUES); CHKERRQ(ierr); > ierr = MatSetValue(interp, 1, 1, 0.5, INSERT_VALUES); CHKERRQ(ierr); > ierr = MatSetValue(interp, 2, 1, 1.0, INSERT_VALUES); CHKERRQ(ierr); > ierr = MatSetValue(interp, 3, 1, 0.5, INSERT_VALUES); CHKERRQ(ierr); > ierr = MatSetValue(interp, 3, 2, 0.5, INSERT_VALUES); CHKERRQ(ierr); > ierr = MatSetValue(interp, 4, 2, 1.0, INSERT_VALUES); CHKERRQ(ierr); > ierr = MatAssemblyBegin(interp, MAT_FINAL_ASSEMBLY); CHKERRQ(ierr); > ierr = MatAssemblyEnd(interp, MAT_FINAL_ASSEMBLY); CHKERRQ(ierr); > ierr = MatView(interp, PETSC_VIEWER_STDOUT_SELF); > > Mat restrict; // Multigrid restriction matrix, not the transpose of > "interp" > ierr = MatCreateSeqAIJ(PETSC_COMM_SELF, nb_dof_coarse, nb_dof_fine, > 1, 0, &restrict); CHKERRQ(ierr); > ierr = MatSetValue(restrict, 0, 0, 1.0, INSERT_VALUES); CHKERRQ(ierr); > ierr = MatSetValue(restrict, 1, 2, 1.0, INSERT_VALUES); CHKERRQ(ierr); > ierr = MatSetValue(restrict, 2, 4, 1.0, INSERT_VALUES); CHKERRQ(ierr); > ierr = MatAssemblyBegin(restrict, MAT_FINAL_ASSEMBLY); CHKERRQ(ierr); > ierr = MatAssemblyEnd(restrict, MAT_FINAL_ASSEMBLY); CHKERRQ(ierr); > ierr = MatView(restrict, PETSC_VIEWER_STDOUT_SELF); > > { // Solving the linear problem Id*x = 0 works > KSP ksp; > ierr = KSPCreate(PETSC_COMM_SELF, &ksp); CHKERRQ(ierr); > ierr = KSPSetType(ksp, KSPFGMRES); CHKERRQ(ierr); > ierr = KSPSetOperators(ksp, Id, Id, SAME_PRECONDITIONER); > CHKERRQ(ierr); > > PC pc; > ierr = KSPGetPC(ksp, &pc); CHKERRQ(ierr); > ierr = PCSetType(pc, PCMG); CHKERRQ(ierr); > ierr = PCMGSetLevels(pc, 2, 0); CHKERRQ(ierr); > ierr = PCMGSetGalerkin(pc, PETSC_TRUE); CHKERRQ(ierr); > ierr = PCMGSetInterpolation(pc, 1, interp); CHKERRQ(ierr); > ierr = PCMGSetRestriction(pc, 1, restrict); CHKERRQ(ierr); > > ierr = KSPSetFromOptions(ksp); CHKERRQ(ierr); > > Vec b; // Trivial RHS for the linear problem > ierr = VecDuplicate(x, &b); CHKERRQ(ierr); > ierr = VecSet(b, 0); CHKERRQ(ierr); > ierr = KSPSolve(ksp, b, x); CHKERRQ(ierr); > ierr = VecView(x, PETSC_VIEWER_STDOUT_SELF); > > ierr = VecDestroy(&b); CHKERRQ(ierr); > ierr = KSPDestroy(&ksp); CHKERRQ(ierr); > > ierr = PetscPrintf(PETSC_COMM_SELF, "KSPSolve worked\n"); > CHKERRQ(ierr); > } > > { // This block of code causes a segfault > TS ts; > ierr = TSCreate(PETSC_COMM_SELF, &ts); CHKERRQ(ierr); > ierr = TSSetProblemType(ts, TS_LINEAR); CHKERRQ(ierr); > ierr = TSSetType(ts, TSBEULER); // Use an implicit scheme > ierr = TSSetInitialTimeStep(ts, 0.0, 0.1); CHKERRQ(ierr); > ierr = TSSetSolution(ts, x); CHKERRQ(ierr); > ierr = TSSetRHSFunction(ts, 0, TSComputeRHSFunctionLinear, 0); > CHKERRQ(ierr); > ierr = TSSetRHSJacobian(ts, Id, Id, > TSComputeRHSJacobianConstant, 0); CHKERRQ(ierr); > ierr = TSSetIFunction(ts, 0, TSComputeIFunctionLinear, 0); > CHKERRQ(ierr); > ierr = TSSetIJacobian(ts, Id, Id, TSComputeIJacobianConstant, 0); > > KSP ksp; > ierr = TSGetKSP(ts, &ksp); CHKERRQ(ierr); > ierr = KSPSetType(ksp, KSPFGMRES); CHKERRQ(ierr); > > PC pc; > ierr = KSPGetPC(ksp, &pc); CHKERRQ(ierr); > ierr = PCSetType(pc, PCMG); CHKERRQ(ierr); > ierr = PCMGSetLevels(pc, 2, 0); CHKERRQ(ierr); > ierr = PCMGSetGalerkin(pc, PETSC_TRUE); CHKERRQ(ierr); > ierr = PCMGSetInterpolation(pc, 1, interp); CHKERRQ(ierr); > ierr = PCMGSetRestriction(pc, 1, restrict); CHKERRQ(ierr); > > ierr = TSSetUp(ts); CHKERRQ(ierr); > ierr = TSStep(ts); CHKERRQ(ierr); > > ierr = TSDestroy(&ts); CHKERRQ(ierr); > } > > ierr = MatDestroy(&restrict); CHKERRQ(ierr); > ierr = MatDestroy(&interp); CHKERRQ(ierr); > ierr = VecDestroy(&x); CHKERRQ(ierr); > ierr = MatDestroy(&Id); CHKERRQ(ierr); > ierr = PetscFinalize(); CHKERRQ(ierr); > > return 0; > } > > The actual output of the program when running without a debugger is: > > Matrix Object: 1 MPI processes > type: seqaij > row 0: (0, 1) > row 1: (1, 1) > row 2: (2, 1) > row 3: (3, 1) > row 4: (4, 1) > Matrix Object: 1 MPI processes > type: seqaij > row 0: (0, 1) > row 1: (0, 0.5) (1, 0.5) > row 2: (1, 1) > row 3: (1, 0.5) (2, 0.5) > row 4: (2, 1) > Matrix Object: 1 MPI processes > type: seqaij > row 0: (0, 1) > row 1: (2, 1) > row 2: (4, 1) > Vector Object: 1 MPI processes > type: seq > 0 > 0 > 0 > 0 > 0 > KSPSolve worked > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: Caught signal number 11 SEGV: Segmentation Violation, > probably memory access out of range > [0]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger > [0]PETSC ERROR: or see > http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind[0]PETSC > ERROR: or try http://valgrind.org on GNU/linux and Apple Mac OS X to > find memory corruption errors > [0]PETSC ERROR: likely location of problem given in stack below > [0]PETSC ERROR: --------------------- Stack Frames > ------------------------------------ > [0]PETSC ERROR: Note: The EXACT line numbers in the stack are not > available, > [0]PETSC ERROR: INSTEAD the line number of the start of the function > [0]PETSC ERROR: is given. > [0]PETSC ERROR: [0] DMCoarsen line 1809 src/dm/interface/dm.c > [0]PETSC ERROR: [0] PCSetUp_MG line 546 src/ksp/pc/impls/mg/mg.c > [0]PETSC ERROR: [0] PCSetUp line 868 src/ksp/pc/interface/precon.c > [0]PETSC ERROR: [0] KSPSetUp line 192 src/ksp/ksp/interface/itfunc.c > [0]PETSC ERROR: [0] KSPSolve line 356 src/ksp/ksp/interface/itfunc.c > [0]PETSC ERROR: [0] SNESSolve_KSPONLY line 14 > src/snes/impls/ksponly/ksponly.c > [0]PETSC ERROR: [0] SNESSolve line 3589 src/snes/interface/snes.c > [0]PETSC ERROR: [0] TSStep_Theta line 162 > src/ts/impls/implicit/theta/theta.c > [0]PETSC ERROR: [0] TSStep line 2499 src/ts/interface/ts.c > [0]PETSC ERROR: --------------------- Error Message > ------------------------------------ > [0]PETSC ERROR: Signal received! > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: Petsc Release Version 3.4.3, Oct, 15, 2013 > [0]PETSC ERROR: See docs/changes/index.html for recent updates. > [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. > [0]PETSC ERROR: See docs/index.html for manual pages. > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: ./testcase.elf on a linux-gnu-c-debug named lenovo by > tmac Sat Dec 14 14:35:56 2013 > [0]PETSC ERROR: Libraries linked from > /tmp/petsc-3.4.3/linux-gnu-c-debug/lib > [0]PETSC ERROR: Configure run at Fri Nov 22 11:07:41 2013 > [0]PETSC ERROR: Configure options --with-shared-libraries > --with-debugging=1 --useThreads 0 --with-clanguage=C++ --with-c-support > --with-fortran-interfaces=1 --with-scalar-type=complex > --with-mpi-dir=/usr/lib/openmpi --with-blas-lib=-lblas > --with-lapack-lib=-llapack --with-blacs=1 > --with-blacs-include=/usr/include > > --with-blacs-lib="[/usr/lib/libblacsCinit-openmpi.so,/usr/lib/libblacs-openmpi.so]" > --with-scalapack=1 --with-scalapack-include=/usr/include > --with-scalapack-lib=/usr/lib/libscalapack-openmpi.so --with-mumps=1 > --with-mumps-include=/usr/include > > --with-mumps-lib="[/usr/lib/libdmumps.so,/usr/lib/libzmumps.so,/usr/lib/libsmumps.so,/usr/lib/libcmumps.so,/usr/lib/libmumps_common.so,/usr/lib/libpord.so]" > --with-umfpack=1 --with-umfpack-include=/usr/include/suitesparse > --with-umfpack-lib="[/usr/lib/libumfpack.so,/usr/lib/libamd.so]" > --with-cholmod=1 --with-cholmod-include=/usr/include/suitesparse > --with-cholmod-lib=/usr/lib/libcholmod.so --with-spooles=1 > --with-spooles-include=/usr/include/spooles > --with-spooles-lib=/usr/lib/libspooles.so --with-ptscotch=1 > --with-ptscotch-include=/usr/include/scotch > > --with-ptscotch-lib="[/usr/lib/libptesmumps.so,/usr/lib/libptscotch.so,/usr/lib/libptscotcherr.so]" > --with-fftw=1 --with-fftw-include=/usr/include > > --with-fftw-lib="[/usr/lib/x86_64-linux-gnu/libfftw3.so,/usr/lib/x86_64-linux-gnu/libfftw3_mpi.so]" > --with-superlu=1 --with-superlu-include=/usr/include/superlu > --with-superlu-lib=/usr/lib/libsuperlu.so > --CXX_LINKER_FLAGS=-Wl,--no-as-needed > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: User provided function() line 0 in unknown directory > unknown file > -------------------------------------------------------------------------- > MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD > with errorcode 59. > > NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes. > You may or may not see output from other processes, depending on > exactly when Open MPI kills them. > -------------------------------------------------------------------------- > > Best regards > Torquil S?rensen > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jedbrown at mcs.anl.gov Wed Dec 18 13:25:17 2013 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Wed, 18 Dec 2013 11:25:17 -0800 Subject: [petsc-users] vtk format In-Reply-To: References: Message-ID: <871u1a55cy.fsf@jedbrown.org> Matthew Knepley writes: >> ierr = PetscViewerCreate(PETSC_COMM_WORLD, &viewer);CHKERRQ(ierr); >> ierr = PetscViewerSetType(viewer, PETSCVIEWERVTK);CHKERRQ(ierr); >> >> ierr = PetscViewerFileSetName(viewer, "poisson.vtk");CHKERRQ(ierr); Please do not use the legacy ASCII format, except perhaps for test files so small you will be reading them by hand rather than viewing them with software. >> ierr = PetscViewerSetFormat(viewer, >> PETSC_VIEWER_ASCII_VTK);CHKERRQ(ierr); >> DMDAVTKWriteAll((PetscObject)da, viewer); >> > > This is mixing a few things. I think you want binary VTK, so > > ierr = PetscViewerCreate(PETSC_COMM_WORLD, &viewer);CHKERRQ(ierr); > ierr = PetscViewerSetType(viewer, PETSCVIEWERVTK);CHKERRQ(ierr); > ierr = PetscViewerFileSetName(viewer, "poisson.vtu");CHKERRQ(ierr); Matt means "poisson.vts" since you are on a structured grid. "poisson.vtr" should also work with petsc-dev (for rectangular grids, a special case of structured). > ierr = VecView(v);CHKERRQ(ierr); VecView(v,viewer); -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From jedbrown at mcs.anl.gov Wed Dec 18 14:11:45 2013 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Wed, 18 Dec 2013 12:11:45 -0800 Subject: [petsc-users] vtk format In-Reply-To: <52B1FBAA.3010503@utexas.edu> References: <871u1a55cy.fsf@jedbrown.org> <52B1FBAA.3010503@utexas.edu> Message-ID: <87k3f1537i.fsf@jedbrown.org> Please always use "reply-all" so that your messages go to the list. Khai Pham writes: > Hello Jed, > > May I join this topic with one question about vtk? I followed the > discussion on > http://lists.mcs.anl.gov/pipermail/petsc-users/2013-February/016506.html > to set the non-uniform grid coordinates. Then try to print out the vtk > file using: > ierr = PetscViewerCreate(PETSC_COMM_WORLD, &viewer);CHKERRQ(ierr); > ierr = PetscViewerSetType(viewer, PETSCVIEWERASCII);CHKERRQ(ierr); > ierr = PetscViewerFileSetName(viewer, file_name);CHKERRQ(ierr); > ierr = PetscViewerSetFormat(viewer,PETSC_VIEWER_ASCII_VTK);CHKERRQ(ierr); > > ierr = DMView(dm, viewer);CHKERRQ(ierr); > ierr = VecView(v, viewer);CHKERRQ(ierr); I don't know where you got this sequence, but it has been discouraged since we added proper VTK support. Use this: PetscViewerVTKOpen(comm,file_name,FILE_MODE_WRITE,&viewer); VecView(v,viewer); PetscViewerDestroy(&viewer); The file name should end in .vtr for "rectangular" or .vts for "structured" (when using DMDA), or with .vtu for "unstructured" (with DMPlex). -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From jedbrown at mcs.anl.gov Wed Dec 18 14:19:21 2013 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Wed, 18 Dec 2013 12:19:21 -0800 Subject: [petsc-users] Problem with TS and PCMG In-Reply-To: References: <52AC6127.2010407@gmail.com> Message-ID: <87eh5952uu.fsf@jedbrown.org> Torquil Macdonald S?rensen writes: > Just a second try in case the first email was unintentionally missed. I've been offline for the past few days. I'll be back for real on Friday. > In short, I'm wondering if it is possible to use TSSolve without using > any DM structure: > > When I was debugging my program I tried to analyse the difference between > the TSSolve-based time iteration program, and a version where I do the > time-stepping manually using a for-loop which repeateed KSPSolve functions. > The debugger showed that in the TSSolve program a pointer named > ts->snes->ksp->pc->dm was nonzero, but in the for-loop program it was zero. > In the TSSolve-program, and pc->dm != 0 made it enter a block of code > within PCSetUp_MG that called DMCoarsen, which then crashed. So I guess I'm > wondering how to tell PETSc that I'm not using any DM structure. Please show the entire trace. Callback pointers are stored on an "internal" DM if you don't provide one. This is important for nonlinear preconditioning and other types of composition. What preconditioner are you intending to use? -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From torquil at gmail.com Wed Dec 18 15:11:53 2013 From: torquil at gmail.com (=?UTF-8?B?VG9ycXVpbCBNYWNkb25hbGQgU8O4cmVuc2Vu?=) Date: Wed, 18 Dec 2013 22:11:53 +0100 Subject: [petsc-users] Problem with TS and PCMG In-Reply-To: <87eh5952uu.fsf@jedbrown.org> References: <52AC6127.2010407@gmail.com> <87eh5952uu.fsf@jedbrown.org> Message-ID: <52B20F99.3080104@gmail.com> On 18/12/13 21:19, Jed Brown wrote: >> In short, I'm wondering if it is possible to use TSSolve without using >> any DM structure: >> >> When I was debugging my program I tried to analyse the difference between >> the TSSolve-based time iteration program, and a version where I do the >> time-stepping manually using a for-loop which repeateed KSPSolve functions. >> The debugger showed that in the TSSolve program a pointer named >> ts->snes->ksp->pc->dm was nonzero, but in the for-loop program it was zero. >> In the TSSolve-program, and pc->dm != 0 made it enter a block of code >> within PCSetUp_MG that called DMCoarsen, which then crashed. So I guess I'm >> wondering how to tell PETSc that I'm not using any DM structure. > Please show the entire trace. Callback pointers are stored on an > "internal" DM if you don't provide one. This is important for nonlinear > preconditioning and other types of composition. What preconditioner are > you intending to use? If you mean the entire backtrace within GDB, I'm including it in this email. I'm using PCMG. My minimal testcase program has everything hardcoded. I'm not providing any runtime options other than -start_in_debugger, so everything can be seen from the code. I've since made a couple of small changes that might influence line number statements from GDB, so I'm including the newest version in this message. Here is the GDB output of "bt full" and the corresponding program code: (gdb) bt full #1 0x00007ffdc080d172 in DMCoarsen (dm=0x1fa7df0, comm=0x7ffdbffdd9c0 , dmc=0x2048f00) at dm.c:1811 ierr = 0 link = 0x7ffdc1af1640 __func__ = "DMCoarsen" #2 0x00007ffdc0c41c1e in PCSetUp_MG (pc=0x1ff7570) at mg.c:593 kdm = 0x1d5b370 p = 0xb453 rscale = 0x1ff7570 lu = PETSC_FALSE svd = (PETSC_TRUE | unknown: 32764) dB = 0x0 __func__ = "PCSetUp_MG" tvec = 0x0 viewer = 0x0 mglevels = 0x2006980 ierr = 0 preonly = PETSC_FALSE cholesky = (unknown: 3246112504) mg = 0x20060c0 i = 0 n = 2 redundant = (unknown: 1211234) use_amat = PETSC_TRUE dms = 0x2048f00 cpc = 0x200f620 dump = PETSC_FALSE opsset = PETSC_FALSE dA = 0x7ffdc1af1640 uflag = DIFFERENT_NONZERO_PATTERN #3 0x00007ffdc0f3b6ce in PCSetUp (pc=0x1ff7570) at precon.c:890 ierr = 0 def = 0x7ffdc1494e01 "User provided function" __func__ = "PCSetUp" #4 0x00007ffdc0abf7e8 in KSPSetUp (ksp=0x1fce550) at itfunc.c:278 ierr = 0 A = 0x55 __func__ = "KSPSetUp" B = 0x1fce550 stflg = SAME_NONZERO_PATTERN #5 0x00007ffdc0ac0a30 in KSPSolve (ksp=0x1fce550, b=0x2033dd0, x=0x204b120) at itfunc.c:399 rank = 0 flag1 = PETSC_FALSE guess_zero = (unknown: 33861920) viewer = 0x7fffbf50c290 ierr = 0 flag2 = PETSC_FALSE __func__ = "KSPSolve" flag3 = PETSC_TRUE flg = PETSC_FALSE inXisinB = PETSC_FALSE mat = 0x0 premat = 0x0 format = PETSC_VIEWER_DEFAULT #6 0x00007ffdc0adae0c in SNESSolve_KSPONLY (snes=0x1f86390) at ksponly.c:44 ierr = 0 lits = 32767 flg = SAME_PRECONDITIONER Y = 0x204b120 __func__ = "SNESSolve_KSPONLY" X = 0x2036b50 F = 0x2033dd0 kspreason = KSP_CONVERGED_ITERATING #7 0x00007ffdc11cb642 in SNESSolve (snes=0x1f86390, b=0x0, x=0x2036b50) at snes.c:3636 ierr = 0 flg = PETSC_FALSE viewer = 0x7ffdc18e5205 <_dl_runtime_resolve+53> grid = 0 xcreated = 0x0 __func__ = "SNESSolve" dm = 0x1fa7df0 format = PETSC_VIEWER_DEFAULT #8 0x00007ffdc12780cc in TSStep_Theta (ts=0x1f52ea0) at theta.c:183 shift = 10 th = 0x1fa3130 next_time_step = 0.10000000000000001 snesreason = 32765 adapt = 0x7ffdc18ded95 <_dl_fixup+245> accept = (PETSC_TRUE | unknown: 32766) its = 32765 reject = 0 next_scheme = 0 ierr = 0 __func__ = "TSStep_Theta" lits = -1071406416 #9 0x00007ffdc12ba202 in TSStep (ts=0x1f52ea0) at ts.c:2507 ptime_prev = 0 ierr = 0 __func__ = "TSStep" #10 0x00000000004026fd in main (argc=3, argv=0x7fffbf50c6d8) at /home/tmac/programming/petsc/source/tssolve_multigrid.cpp:65 ts = 0x1f52ea0 ksp = 0x1fce550 pc = 0x1ff7570 ierr = 0 __func__ = "main" nb_dof_fine = 5 nb_dof_coarse = 3 x = 0x1d769d0 Id = 0x1dffc30 interpolation = 0x1e24020 restriction = 0x1e48470 $ cat /home/tmac/programming/petsc/source/tssolve_multigrid.cpp // A trivial test case for TSBEULER and PCMG: Solve Id*dU/dt = 0, giving a segfault #include "petscts.h" int main(int argc, char ** argv) { PetscErrorCode ierr = PetscInitialize(&argc, &argv, 0, 0); CHKERRQ(ierr); int const nb_dof_fine = 5; int const nb_dof_coarse = 3; Vec x; // A vector to hold the solution on the fine grid ierr = VecCreateSeq(PETSC_COMM_SELF, nb_dof_fine, &x); CHKERRQ(ierr); Mat Id; // Identity matrix on the vector space of the fine grid ierr = MatCreateSeqAIJ(PETSC_COMM_SELF, nb_dof_fine, nb_dof_fine, 1, 0, &Id); CHKERRQ(ierr); for(int i = 0; i != nb_dof_fine; i++) { MatSetValue(Id, i, i, 1.0, INSERT_VALUES); } ierr = MatAssemblyBegin(Id, MAT_FINAL_ASSEMBLY); CHKERRQ(ierr); ierr = MatAssemblyEnd(Id, MAT_FINAL_ASSEMBLY); CHKERRQ(ierr); Mat interpolation; // Multigrid interpolation matrix ierr = MatCreateSeqAIJ(PETSC_COMM_SELF, nb_dof_fine, nb_dof_coarse, 2, 0, &interpolation); CHKERRQ(ierr); ierr = MatSetValue(interpolation, 0, 0, 1.0, INSERT_VALUES); CHKERRQ(ierr); ierr = MatSetValue(interpolation, 1, 0, 0.5, INSERT_VALUES); CHKERRQ(ierr); ierr = MatSetValue(interpolation, 1, 1, 0.5, INSERT_VALUES); CHKERRQ(ierr); ierr = MatSetValue(interpolation, 2, 1, 1.0, INSERT_VALUES); CHKERRQ(ierr); ierr = MatSetValue(interpolation, 3, 1, 0.5, INSERT_VALUES); CHKERRQ(ierr); ierr = MatSetValue(interpolation, 3, 2, 0.5, INSERT_VALUES); CHKERRQ(ierr); ierr = MatSetValue(interpolation, 4, 2, 1.0, INSERT_VALUES); CHKERRQ(ierr); ierr = MatAssemblyBegin(interpolation, MAT_FINAL_ASSEMBLY); CHKERRQ(ierr); ierr = MatAssemblyEnd(interpolation, MAT_FINAL_ASSEMBLY); CHKERRQ(ierr); Mat restriction; // Multigrid restriction matrix, not the transpose of "interpolation" ierr = MatCreateSeqAIJ(PETSC_COMM_SELF, nb_dof_coarse, nb_dof_fine, 1, 0, &restriction); CHKERRQ(ierr); ierr = MatSetValue(restriction, 0, 0, 1.0, INSERT_VALUES); CHKERRQ(ierr); ierr = MatSetValue(restriction, 1, 2, 1.0, INSERT_VALUES); CHKERRQ(ierr); ierr = MatSetValue(restriction, 2, 4, 1.0, INSERT_VALUES); CHKERRQ(ierr); ierr = MatAssemblyBegin(restriction, MAT_FINAL_ASSEMBLY); CHKERRQ(ierr); ierr = MatAssemblyEnd(restriction, MAT_FINAL_ASSEMBLY); CHKERRQ(ierr); { TS ts; ierr = TSCreate(PETSC_COMM_SELF, &ts); CHKERRQ(ierr); ierr = TSSetProblemType(ts, TS_LINEAR); CHKERRQ(ierr); ierr = TSSetType(ts, TSBEULER); // Use an implicit scheme ierr = TSSetInitialTimeStep(ts, 0.0, 0.1); CHKERRQ(ierr); ierr = TSSetSolution(ts, x); CHKERRQ(ierr); ierr = TSSetIFunction(ts, 0, TSComputeIFunctionLinear, 0); CHKERRQ(ierr); ierr = TSSetIJacobian(ts, Id, Id, TSComputeIJacobianConstant, 0); KSP ksp; ierr = TSGetKSP(ts, &ksp); CHKERRQ(ierr); ierr = KSPSetType(ksp, KSPFGMRES); CHKERRQ(ierr); PC pc; ierr = KSPGetPC(ksp, &pc); CHKERRQ(ierr); ierr = PCSetType(pc, PCMG); CHKERRQ(ierr); ierr = PCMGSetLevels(pc, 2, 0); CHKERRQ(ierr); ierr = PCMGSetGalerkin(pc, PETSC_TRUE); CHKERRQ(ierr); ierr = PCMGSetInterpolation(pc, 1, interpolation); CHKERRQ(ierr); ierr = PCMGSetRestriction(pc, 1, restriction); CHKERRQ(ierr); ierr = TSSetFromOptions(ts); CHKERRQ(ierr); ierr = TSSetUp(ts); CHKERRQ(ierr); ierr = TSStep(ts); CHKERRQ(ierr); ierr = TSDestroy(&ts); CHKERRQ(ierr); } ierr = MatDestroy(&restriction); CHKERRQ(ierr); ierr = MatDestroy(&interpolation); CHKERRQ(ierr); ierr = VecDestroy(&x); CHKERRQ(ierr); ierr = MatDestroy(&Id); CHKERRQ(ierr); ierr = PetscFinalize(); CHKERRQ(ierr); return 0; } Best regards Torquil S?rensen -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 897 bytes Desc: OpenPGP digital signature URL: From jedbrown at mcs.anl.gov Wed Dec 18 16:50:05 2013 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Wed, 18 Dec 2013 14:50:05 -0800 Subject: [petsc-users] Feature request: Origin QMR implementation. In-Reply-To: References: Message-ID: <8761ql3hb6.fsf@jedbrown.org> Ce Qin writes: > Dear all, > > Now I'm solving Maxwell's equations using finite difference method. In the > beginning, I use BCGS to solve the linear system. It works, but converges > slowly. And other solvers do not converge. Some others told me that QMR may > be a good choice, so I tested it using matlab, the result shows that QMR > works well than BCGS. I found that PETSc only provide TFQMR and TCQMR, so I > wonder that why PETSc does not implement QMR solver. What preconditioner were you using with MATLAB's QMR? My recollection is that the original QMR requires splitting the preconditioner, as with PC_SYMMETRIC, which is hard to do in general. Other than that, I think there just wasn't much demand. Is your system complex symmetric? Is QMR something you might be interested in implementing? (It's a very manageable task and we can advise; otherwise it's contingent on one of us finding "free" time to write it.) -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From jianchaoge at gmail.com Wed Dec 18 16:50:22 2013 From: jianchaoge at gmail.com (Gavin Ge) Date: Wed, 18 Dec 2013 16:50:22 -0600 Subject: [petsc-users] Issue about matsetvalues when turn off debugging Message-ID: <546FD27B-56BD-40BF-B129-D1EFB00A6F29@gmail.com> Hi All: Hope everyone enjoy a nice holiday. I just got a problem with matsetvalues function when I mute the debugging option. A segmentation violation error jumps out when I set value for the very first element in the matrix. There is no problem for me to run the program when turning on the debugging option. I just want to tune this for better performance. Or is it better to turn on the debugging option? Could anybody help me with this? Thanks! Regards, Gavin From jedbrown at mcs.anl.gov Wed Dec 18 17:16:59 2013 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Wed, 18 Dec 2013 15:16:59 -0800 Subject: [petsc-users] Issue about matsetvalues when turn off debugging In-Reply-To: <546FD27B-56BD-40BF-B129-D1EFB00A6F29@gmail.com> References: <546FD27B-56BD-40BF-B129-D1EFB00A6F29@gmail.com> Message-ID: <87wqj121hw.fsf@jedbrown.org> Gavin Ge writes: > Hi All: > > Hope everyone enjoy a nice holiday. I just got a problem with matsetvalues function when I mute the debugging option. A segmentation violation error jumps out when I set value for the very first element in the matrix. There is no problem for me to run the program when turning on the debugging option. I just want to tune this for better performance. Or is it better to turn on the debugging option? Could anybody help me with this? Thanks! Run in a debugger and get a stack trace. You can configure the non-debug PETSC_ARCH with COPTFLAGS='-O0 -g' to ensure that debugging symbols are available. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From bsmith at mcs.anl.gov Wed Dec 18 17:23:19 2013 From: bsmith at mcs.anl.gov (Barry Smith) Date: Wed, 18 Dec 2013 17:23:19 -0600 Subject: [petsc-users] Issue about matsetvalues when turn off debugging In-Reply-To: <546FD27B-56BD-40BF-B129-D1EFB00A6F29@gmail.com> References: <546FD27B-56BD-40BF-B129-D1EFB00A6F29@gmail.com> Message-ID: <958399D0-F65A-49E6-9E48-D18F0304BDDE@mcs.anl.gov> http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind On Dec 18, 2013, at 4:50 PM, Gavin Ge wrote: > Hi All: > > Hope everyone enjoy a nice holiday. I just got a problem with matsetvalues function when I mute the debugging option. A segmentation violation error jumps out when I set value for the very first element in the matrix. There is no problem for me to run the program when turning on the debugging option. I just want to tune this for better performance. Or is it better to turn on the debugging option? Could anybody help me with this? Thanks! > > Regards, > Gavin > > > > > From jedbrown at mcs.anl.gov Wed Dec 18 17:50:34 2013 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Wed, 18 Dec 2013 15:50:34 -0800 Subject: [petsc-users] Problem with TS and PCMG In-Reply-To: <52B20F99.3080104@gmail.com> References: <52AC6127.2010407@gmail.com> <87eh5952uu.fsf@jedbrown.org> <52B20F99.3080104@gmail.com> Message-ID: <87r4991zxx.fsf@jedbrown.org> Torquil Macdonald S?rensen writes: > If you mean the entire backtrace within GDB, I'm including it in this > email. Perfect, thank you. Cc'ing Barry and Peter who might also like to comment. > I'm using PCMG. My minimal testcase program has everything > hardcoded. I'm not providing any runtime options other than > -start_in_debugger, so everything can be seen from the code. I've > since made a couple of small changes that might influence line number > statements from GDB, so I'm including the newest version in this > message. > > Here is the GDB output of "bt full" and the corresponding program code: > > (gdb) bt full > #1 0x00007ffdc080d172 in DMCoarsen (dm=0x1fa7df0, comm=0x7ffdbffdd9c0 > , dmc=0x2048f00) at dm.c:1811 > ierr = 0 > link = 0x7ffdc1af1640 > __func__ = "DMCoarsen" > #2 0x00007ffdc0c41c1e in PCSetUp_MG (pc=0x1ff7570) at mg.c:593 > kdm = 0x1d5b370 > p = 0xb453 > rscale = 0x1ff7570 > lu = PETSC_FALSE > svd = (PETSC_TRUE | unknown: 32764) > dB = 0x0 > __func__ = "PCSetUp_MG" > tvec = 0x0 > viewer = 0x0 > mglevels = 0x2006980 > ierr = 0 > preonly = PETSC_FALSE > cholesky = (unknown: 3246112504) > mg = 0x20060c0 > i = 0 > n = 2 > redundant = (unknown: 1211234) > use_amat = PETSC_TRUE > dms = 0x2048f00 > cpc = 0x200f620 > dump = PETSC_FALSE > opsset = PETSC_FALSE > dA = 0x7ffdc1af1640 > uflag = DIFFERENT_NONZERO_PATTERN Okay, this is the relevant bit of code. /* Skipping this for galerkin==2 (externally managed hierarchy such as ML and GAMG). Cleaner logic here would be great. Wrap ML/GAMG as DMs? */ if (pc->dm && mg->galerkin != 2 && !pc->setupcalled) { /* construct the interpolation from the DMs */ Mat p; Vec rscale; ierr = PetscMalloc1(n,&dms);CHKERRQ(ierr); dms[n-1] = pc->dm; for (i=n-2; i>-1; i--) { DMKSP kdm; ierr = DMCoarsen(dms[i+1],MPI_COMM_NULL,&dms[i]);CHKERRQ(ierr); ierr = KSPSetDM(mglevels[i]->smoothd,dms[i]);CHKERRQ(ierr); if (mg->galerkin) {ierr = KSPSetDMActive(mglevels[i]->smoothd,PETSC_FALSE);CHKERRQ(ierr);} ierr = DMGetDMKSPWrite(dms[i],&kdm);CHKERRQ(ierr); /* Ugly hack so that the next KSPSetUp() will use the RHS that we set. A better fix is to change dmActive to take * a bitwise OR of computing the matrix, RHS, and initial iterate. */ kdm->ops->computerhs = NULL; kdm->rhsctx = NULL; if (!mglevels[i+1]->interpolate) { ierr = DMCreateInterpolation(dms[i],dms[i+1],&p,&rscale);CHKERRQ(ierr); I'm inclined to implement DMCoarsen_Shell() so that it carries through to the levels. It won't be used for interpolation because of the final conditional (the user has already set mglevels[i+1]->interpolate). This is slightly annoying because it's one more function that users have to implement. I think the patch below will also work, though I'm nervous about DMShellSetCreateMatrix(). diff --git i/src/dm/interface/dm.c w/src/dm/interface/dm.c index ec60763..c987200 100644 --- i/src/dm/interface/dm.c +++ w/src/dm/interface/dm.c @@ -1943,7 +1943,11 @@ PetscErrorCode DMCoarsen(DM dm, MPI_Comm comm, DM *dmc) PetscFunctionBegin; PetscValidHeaderSpecific(dm,DM_CLASSID,1); - ierr = (*dm->ops->coarsen)(dm, comm, dmc);CHKERRQ(ierr); + if (dm->ops->coarsen) { + ierr = (*dm->ops->coarsen)(dm, comm, dmc);CHKERRQ(ierr); + } else { + ierr = DMShellCreate(comm,dmc);CHKERRQ(ierr); + } (*dmc)->ops->creatematrix = dm->ops->creatematrix; ierr = PetscObjectCopyFortranFunctionPointers((PetscObject)dm,(PetscObject)*dmc);CHKERRQ(ierr); (*dmc)->ctx = dm->ctx; > Mat restriction; // Multigrid restriction matrix, not the transpose > of "interpolation" > ierr = MatCreateSeqAIJ(PETSC_COMM_SELF, nb_dof_coarse, nb_dof_fine, > 1, 0, &restriction); CHKERRQ(ierr); > ierr = MatSetValue(restriction, 0, 0, 1.0, INSERT_VALUES); > CHKERRQ(ierr); > ierr = MatSetValue(restriction, 1, 2, 1.0, INSERT_VALUES); > CHKERRQ(ierr); > ierr = MatSetValue(restriction, 2, 4, 1.0, INSERT_VALUES); > CHKERRQ(ierr); This "restriction" has primary order 0 (it aliases high-frequency harmonics without attenuation), and is thus not suitable for residual restriction. It is a great _state_ restriction operator, as are used by FAS methods. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From qince168 at gmail.com Thu Dec 19 05:13:51 2013 From: qince168 at gmail.com (Ce Qin) Date: Thu, 19 Dec 2013 19:13:51 +0800 Subject: [petsc-users] Feature request: Origin QMR implementation. In-Reply-To: <8761ql3hb6.fsf@jedbrown.org> References: <8761ql3hb6.fsf@jedbrown.org> Message-ID: Thanks, Jed. I used left Jacobi preconditioner (by setting M1 = diag(A)) with Matlab's QMR and BCGS. I am new to PETSc and I am not very fimilar with linear solvers. If I want to implement the QMR solver, what is the starting point? Thanks in advance. Best Regards, Ce Qin -------------- next part -------------- An HTML attachment was scrubbed... URL: From saeed.torabi at usask.ca Thu Dec 19 10:23:38 2013 From: saeed.torabi at usask.ca (Torabi Ziaratgahi, Saeed) Date: Thu, 19 Dec 2013 16:23:38 +0000 Subject: [petsc-users] Multiple right hand sides solver in PETSc or MUMPS In-Reply-To: <0DD2EDA41F02CB4DB70FAA4B8229F20D2890B0ED@CAMPUSMB4.usask.ca> References: <0DD2EDA41F02CB4DB70FAA4B8229F20D2890B0ED@CAMPUSMB4.usask.ca> Message-ID: <0DD2EDA41F02CB4DB70FAA4B8229F20D2890B116@CAMPUSMB4.usask.ca> Hello, I am using chaste software to solve a system including 2 linear systems with same coefficient matrix. Therefore, I want to use multiple right hand sides solver in PETSc or MUMPS. I have to solve the first linear system and use its solution to construct left hand side of the second system. I looked for PETSc and MUMPS documentations to figure out how to use multiple right hand sides solver. However, I could not find any clear instruction or example. Could you please help me on this? I use mpirun -np 64 $CHASTE_LOCATION/projects/SDIRK/build/intel/"$fileName"Runner -ksp_type preonly -ksp_initial_guess_nonzero 0 -pc_type lu -pc_factor_mat_solver_package mumps to run my code. Should I add another option to be able to use multiple right hand sides solver with 2 right hand sides? Or should I make change in my chaste code or PETSc codes? Bests, Saeed -------------- next part -------------- An HTML attachment was scrubbed... URL: From quecat001 at gmail.com Thu Dec 19 16:26:45 2013 From: quecat001 at gmail.com (Que Cat) Date: Thu, 19 Dec 2013 16:26:45 -0600 Subject: [petsc-users] Snes failled with reason -6 Message-ID: Hello, I use SNES to solve the non linear coupled equations. I have checked the hand-coded jacobian with -snes_type test and received the result: Norm of matrix ratio 1.39606e-23 difference 1.53372e-14 (user-defined state) Norm of matrix ratio 3.03157e-10 difference 0.33305 (constant state -1.0) Norm of matrix ratio 3.03157e-10 difference 0.33305 (constant state 1.0) It said that " if the ratio is O(1.e-8), the hand-coded Jacobian is probably correct." What is the second paramenter "difference 1.53372e-14 ...."? Does it play any role in checking the accuracy of hand-coded jacobian? The SNES solver for failed with the reason -6 "the line search failed". I tried with all other line search method stated in the PETSC manual, but they all failed. I tried to figure out what is wrong with my code? Could you please give me any recommendation to direct me how to check my code? I appreciate your help. QUe -------------- next part -------------- An HTML attachment was scrubbed... URL: From charlesj at purdue.edu Thu Dec 19 16:44:43 2013 From: charlesj at purdue.edu (James A Charles) Date: Thu, 19 Dec 2013 17:44:43 -0500 (EST) Subject: [petsc-users] Petsc Matrix Redistribution In-Reply-To: <92529557.83233.1387492885676.JavaMail.root@mailhub050.itcs.purdue.edu> Message-ID: <811778752.83245.1387493083300.JavaMail.root@mailhub050.itcs.purdue.edu> Hello, I want to redistribute a matrix across MPI processes (block tridiagonal of around size 20,000) for use with Arpack so that I can solve for eignepairs in parallel. Is this possible and if so what is the best way to do this? Thanks, James Charles From knepley at gmail.com Thu Dec 19 16:54:56 2013 From: knepley at gmail.com (Matthew Knepley) Date: Thu, 19 Dec 2013 16:54:56 -0600 Subject: [petsc-users] Snes failled with reason -6 In-Reply-To: References: Message-ID: On Thu, Dec 19, 2013 at 4:26 PM, Que Cat wrote: > Hello, > > I use SNES to solve the non linear coupled equations. I have checked the > hand-coded jacobian with -snes_type test and received the result: > Norm of matrix ratio 1.39606e-23 difference 1.53372e-14 (user-defined > state) > Norm of matrix ratio 3.03157e-10 difference 0.33305 (constant state -1.0) > Norm of matrix ratio 3.03157e-10 difference 0.33305 (constant state 1.0) > It said that " if the ratio is O(1.e-8), the hand-coded Jacobian is > probably correct." What is the second paramenter "difference 1.53372e-14 > ...."? Does it play any role in checking the accuracy of hand-coded > jacobian? > It appears that your Jacobian is correct, or at least has an error that is much smaller than most of the elements. > The SNES solver for failed with the reason -6 "the line search failed". I > tried with all other line search method stated in the PETSC manual, but > they all failed. > Do you know that the problem has a solution. Its possible that there is no real solution. Matt > I tried to figure out what is wrong with my code? Could you please give me > any recommendation to direct me how to check my code? I appreciate your > help. > > QUe > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Thu Dec 19 16:58:25 2013 From: knepley at gmail.com (Matthew Knepley) Date: Thu, 19 Dec 2013 16:58:25 -0600 Subject: [petsc-users] Petsc Matrix Redistribution In-Reply-To: <811778752.83245.1387493083300.JavaMail.root@mailhub050.itcs.purdue.edu> References: <92529557.83233.1387492885676.JavaMail.root@mailhub050.itcs.purdue.edu> <811778752.83245.1387493083300.JavaMail.root@mailhub050.itcs.purdue.edu> Message-ID: On Thu, Dec 19, 2013 at 4:44 PM, James A Charles wrote: > Hello, > > I want to redistribute a matrix across MPI processes (block tridiagonal of > around size 20,000) for use with Arpack so that I can solve for eignepairs > in parallel. Is this possible and if so what is the best way to do this? > The easiet way is probably http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatGetSubMatrix.html, but there is also http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatPermute.html . Matt > Thanks, > James Charles > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Thu Dec 19 17:01:41 2013 From: bsmith at mcs.anl.gov (Barry Smith) Date: Thu, 19 Dec 2013 17:01:41 -0600 Subject: [petsc-users] Snes failled with reason -6 In-Reply-To: References: Message-ID: <0C85C52E-7BA8-4C90-A6DA-9BFB6F5F2C78@mcs.anl.gov> ratio is || A - Afd || /||A|| where |||| is the frobenus norm and difference is || A - Afd ||. Try using -snes_mf -pc_type lu This can often converge even if the A you provide is wrong, so if this convergence sine then likely the Jacobian is wrong. It is also possible that the function evaluation has a bug. http://www.mcs.anl.gov/petsc/documentation/faq.html#newton Barry On Dec 19, 2013, at 4:26 PM, Que Cat wrote: > Hello, > > I use SNES to solve the non linear coupled equations. I have checked the hand-coded jacobian with -snes_type test and received the result: > Norm of matrix ratio 1.39606e-23 difference 1.53372e-14 (user-defined state) > Norm of matrix ratio 3.03157e-10 difference 0.33305 (constant state -1.0) > Norm of matrix ratio 3.03157e-10 difference 0.33305 (constant state 1.0) > It said that " if the ratio is O(1.e-8), the hand-coded Jacobian is probably correct." What is the second paramenter "difference 1.53372e-14 ...."? Does it play any role in checking the accuracy of hand-coded jacobian? > > The SNES solver for failed with the reason -6 "the line search failed". I tried with all other line search method stated in the PETSC manual, but they all failed. > > I tried to figure out what is wrong with my code? Could you please give me any recommendation to direct me how to check my code? I appreciate your help. > > QUe > From knepley at gmail.com Thu Dec 19 17:44:31 2013 From: knepley at gmail.com (Matthew Knepley) Date: Thu, 19 Dec 2013 17:44:31 -0600 Subject: [petsc-users] Petsc Matrix Redistribution In-Reply-To: <1236151554.83289.1387495232329.JavaMail.root@mailhub050.itcs.purdue.edu> References: <1236151554.83289.1387495232329.JavaMail.root@mailhub050.itcs.purdue.edu> Message-ID: On Thu, Dec 19, 2013 at 5:20 PM, James A Charles wrote: > My original matrix is serial. I then want to distribute this across MPI > processes for use with Arpack. MatGetSubMatrix wouldn't change > communicators right? > When you create the matrix, use PETSC_COMM_WORLD and 0 sizes on the other procs. Matt > ----- Original Message ----- > From: "Matthew Knepley" > To: "James A Charles" > Cc: petsc-users at mcs.anl.gov > Sent: Thursday, December 19, 2013 5:58:25 PM > Subject: Re: [petsc-users] Petsc Matrix Redistribution > > > > > On Thu, Dec 19, 2013 at 4:44 PM, James A Charles < charlesj at purdue.edu > > wrote: > > > Hello, > > I want to redistribute a matrix across MPI processes (block tridiagonal of > around size 20,000) for use with Arpack so that I can solve for eignepairs > in parallel. Is this possible and if so what is the best way to do this? > > > > The easiet way is probably > http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatGetSubMatrix.html, but there is also > > http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatPermute.html. > > > Matt > > > Thanks, > James Charles > > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From jianchaoge at gmail.com Thu Dec 19 22:45:31 2013 From: jianchaoge at gmail.com (Gavin Ge) Date: Thu, 19 Dec 2013 22:45:31 -0600 Subject: [petsc-users] Issue about matsetvalues when turn off debugging In-Reply-To: <87wqj121hw.fsf@jedbrown.org> References: <546FD27B-56BD-40BF-B129-D1EFB00A6F29@gmail.com> <87wqj121hw.fsf@jedbrown.org> Message-ID: <2AAD8B09-C3AD-4326-8D02-CE3667889A32@gmail.com> Thanks Jed. After configuring using this, how can I turn on the debugging option when I compile the code? On Dec 18, 2013, at 5:16 PM, Jed Brown wrote: > Gavin Ge writes: > >> Hi All: >> >> Hope everyone enjoy a nice holiday. I just got a problem with matsetvalues function when I mute the debugging option. A segmentation violation error jumps out when I set value for the very first element in the matrix. There is no problem for me to run the program when turning on the debugging option. I just want to tune this for better performance. Or is it better to turn on the debugging option? Could anybody help me with this? Thanks! > > Run in a debugger and get a stack trace. You can configure the > non-debug PETSC_ARCH with COPTFLAGS='-O0 -g' to ensure that debugging > symbols are available. From jianchaoge at gmail.com Thu Dec 19 22:47:42 2013 From: jianchaoge at gmail.com (Gavin Ge) Date: Thu, 19 Dec 2013 22:47:42 -0600 Subject: [petsc-users] Issue about matsetvalues when turn off debugging In-Reply-To: <958399D0-F65A-49E6-9E48-D18F0304BDDE@mcs.anl.gov> References: <546FD27B-56BD-40BF-B129-D1EFB00A6F29@gmail.com> <958399D0-F65A-49E6-9E48-D18F0304BDDE@mcs.anl.gov> Message-ID: Thanks Barry. The problem is that the debugging couldn't detect any problem with the debugger. The crash only show up when debugger is turned off. ----------------------------------------------- Gig' em Jianchao Ge Dept. of Geology and Geophysics Texas A&M University On Dec 18, 2013, at 5:23 PM, Barry Smith wrote: > > http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind > > On Dec 18, 2013, at 4:50 PM, Gavin Ge wrote: > >> Hi All: >> >> Hope everyone enjoy a nice holiday. I just got a problem with matsetvalues function when I mute the debugging option. A segmentation violation error jumps out when I set value for the very first element in the matrix. There is no problem for me to run the program when turning on the debugging option. I just want to tune this for better performance. Or is it better to turn on the debugging option? Could anybody help me with this? Thanks! >> >> Regards, >> Gavin >> >> >> >> >> > From bsmith at mcs.anl.gov Fri Dec 20 06:56:52 2013 From: bsmith at mcs.anl.gov (Barry Smith) Date: Fri, 20 Dec 2013 06:56:52 -0600 Subject: [petsc-users] Issue about matsetvalues when turn off debugging In-Reply-To: References: <546FD27B-56BD-40BF-B129-D1EFB00A6F29@gmail.com> <958399D0-F65A-49E6-9E48-D18F0304BDDE@mcs.anl.gov> Message-ID: On Dec 19, 2013, at 10:47 PM, Gavin Ge wrote: > Thanks Barry. The problem is that the debugging couldn't detect any problem with the debugger. The crash only show up when debugger is turned off. I understand. This kind of bug is often (usually?) a result of memory corruption earlier in the run. Valgrind is a very powerful tool for finding memory corruption, if the problem is memory corruption then valgrind will find it. Barry > > > > > ----------------------------------------------- > Gig' em > Jianchao Ge > Dept. of Geology and Geophysics > Texas A&M University > > > > > > On Dec 18, 2013, at 5:23 PM, Barry Smith wrote: > >> >> http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind >> >> On Dec 18, 2013, at 4:50 PM, Gavin Ge wrote: >> >>> Hi All: >>> >>> Hope everyone enjoy a nice holiday. I just got a problem with matsetvalues function when I mute the debugging option. A segmentation violation error jumps out when I set value for the very first element in the matrix. There is no problem for me to run the program when turning on the debugging option. I just want to tune this for better performance. Or is it better to turn on the debugging option? Could anybody help me with this? Thanks! >>> >>> Regards, >>> Gavin >>> >>> >>> >>> >>> >> > From quecat001 at gmail.com Fri Dec 20 07:51:47 2013 From: quecat001 at gmail.com (Que Cat) Date: Fri, 20 Dec 2013 07:51:47 -0600 Subject: [petsc-users] Snes failled with reason -6 In-Reply-To: References: Message-ID: Hi Matt, Thanks for your response. The problem I tried to test has analytical solution. I think something wrong with the matrix. On Thu, Dec 19, 2013 at 4:54 PM, Matthew Knepley wrote: > On Thu, Dec 19, 2013 at 4:26 PM, Que Cat wrote: > >> Hello, >> >> I use SNES to solve the non linear coupled equations. I have checked the >> hand-coded jacobian with -snes_type test and received the result: >> Norm of matrix ratio 1.39606e-23 difference 1.53372e-14 (user-defined >> state) >> Norm of matrix ratio 3.03157e-10 difference 0.33305 (constant state -1.0) >> Norm of matrix ratio 3.03157e-10 difference 0.33305 (constant state 1.0) >> It said that " if the ratio is O(1.e-8), the hand-coded Jacobian is >> probably correct." What is the second paramenter "difference 1.53372e-14 >> ...."? Does it play any role in checking the accuracy of hand-coded >> jacobian? >> > > It appears that your Jacobian is correct, or at least has an error that is > much smaller than most of the elements. > > >> The SNES solver for failed with the reason -6 "the line search failed". >> I tried with all other line search method stated in the PETSC manual, but >> they all failed. >> > > Do you know that the problem has a solution. Its possible that there is no > real solution. > > Matt > > >> I tried to figure out what is wrong with my code? Could you please give >> me any recommendation to direct me how to check my code? I appreciate your >> help. >> >> QUe >> >> > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > -------------- next part -------------- An HTML attachment was scrubbed... URL: From quecat001 at gmail.com Fri Dec 20 07:54:17 2013 From: quecat001 at gmail.com (Que Cat) Date: Fri, 20 Dec 2013 07:54:17 -0600 Subject: [petsc-users] Snes failled with reason -6 In-Reply-To: <0C85C52E-7BA8-4C90-A6DA-9BFB6F5F2C78@mcs.anl.gov> References: <0C85C52E-7BA8-4C90-A6DA-9BFB6F5F2C78@mcs.anl.gov> Message-ID: Hi Barry, Thanks for your exlanation. I used -snes_mf -pc_type lu and now it failed with reason -3 (/* the linear solve failed */). So, something is wrong with the function. On Thu, Dec 19, 2013 at 5:01 PM, Barry Smith wrote: > > ratio is || A - Afd || /||A|| where |||| is the frobenus norm and > difference is || A - Afd ||. > > Try using -snes_mf -pc_type lu > > This can often converge even if the A you provide is wrong, so if this > convergence sine then likely the Jacobian is wrong. > > It is also possible that the function evaluation has a bug. > > http://www.mcs.anl.gov/petsc/documentation/faq.html#newton > > > Barry > > > > On Dec 19, 2013, at 4:26 PM, Que Cat wrote: > > > Hello, > > > > I use SNES to solve the non linear coupled equations. I have checked the > hand-coded jacobian with -snes_type test and received the result: > > Norm of matrix ratio 1.39606e-23 difference 1.53372e-14 (user-defined > state) > > Norm of matrix ratio 3.03157e-10 difference 0.33305 (constant state -1.0) > > Norm of matrix ratio 3.03157e-10 difference 0.33305 (constant state 1.0) > > It said that " if the ratio is O(1.e-8), the hand-coded Jacobian is > probably correct." What is the second paramenter "difference 1.53372e-14 > ...."? Does it play any role in checking the accuracy of hand-coded > jacobian? > > > > The SNES solver for failed with the reason -6 "the line search failed". > I tried with all other line search method stated in the PETSC manual, but > they all failed. > > > > I tried to figure out what is wrong with my code? Could you please give > me any recommendation to direct me how to check my code? I appreciate your > help. > > > > QUe > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Fri Dec 20 08:30:09 2013 From: bsmith at mcs.anl.gov (Barry Smith) Date: Fri, 20 Dec 2013 08:30:09 -0600 Subject: [petsc-users] Snes failled with reason -6 In-Reply-To: References: <0C85C52E-7BA8-4C90-A6DA-9BFB6F5F2C78@mcs.anl.gov> Message-ID: Run with -snes_mf_operator -pc_type lu I?m sorry I sent the wrong instructions before. Barry On Dec 20, 2013, at 7:54 AM, Que Cat wrote: > Hi Barry, > > Thanks for your exlanation. I used -snes_mf -pc_type lu and now it failed with reason -3 (/* the linear solve failed */). So, something is wrong with the function. > > > On Thu, Dec 19, 2013 at 5:01 PM, Barry Smith wrote: > > ratio is || A - Afd || /||A|| where |||| is the frobenus norm and difference is || A - Afd ||. > > Try using -snes_mf -pc_type lu > > This can often converge even if the A you provide is wrong, so if this convergence sine then likely the Jacobian is wrong. > > It is also possible that the function evaluation has a bug. > > http://www.mcs.anl.gov/petsc/documentation/faq.html#newton > > > Barry > > > > On Dec 19, 2013, at 4:26 PM, Que Cat wrote: > > > Hello, > > > > I use SNES to solve the non linear coupled equations. I have checked the hand-coded jacobian with -snes_type test and received the result: > > Norm of matrix ratio 1.39606e-23 difference 1.53372e-14 (user-defined state) > > Norm of matrix ratio 3.03157e-10 difference 0.33305 (constant state -1.0) > > Norm of matrix ratio 3.03157e-10 difference 0.33305 (constant state 1.0) > > It said that " if the ratio is O(1.e-8), the hand-coded Jacobian is probably correct." What is the second paramenter "difference 1.53372e-14 ...."? Does it play any role in checking the accuracy of hand-coded jacobian? > > > > The SNES solver for failed with the reason -6 "the line search failed". I tried with all other line search method stated in the PETSC manual, but they all failed. > > > > I tried to figure out what is wrong with my code? Could you please give me any recommendation to direct me how to check my code? I appreciate your help. > > > > QUe > > > > From quecat001 at gmail.com Fri Dec 20 08:40:37 2013 From: quecat001 at gmail.com (Que Cat) Date: Fri, 20 Dec 2013 08:40:37 -0600 Subject: [petsc-users] Snes failled with reason -6 In-Reply-To: References: <0C85C52E-7BA8-4C90-A6DA-9BFB6F5F2C78@mcs.anl.gov> Message-ID: Barry, It said that "No support for this operation for this object type! Matrix format mpiaij does not have a built-in PETSc LU!". I run it with option "-snes_monitor -ksp_monitor_true_residual -snes_converged_reason -ksp_converged_reason." and received : 0 SNES Function norm 1.414213647226e-01 0 KSP preconditioned resid norm 8.424084241999e+08 true resid norm 1.414213647226e-01 ||r(i)||/||b|| 1.000000000000e+00 1 KSP preconditioned resid norm 1.354908290122e-02 true resid norm 1.424694873074e-01 ||r(i)||/||b|| 1.007411345428e+00 Nonlinear solve did not converge due to DIVERGED_LINE_SEARCH iterations 0 It looks like that the preconditioned residual converges, but the true residual does not. On Fri, Dec 20, 2013 at 8:30 AM, Barry Smith wrote: > > Run with > > -snes_mf_operator -pc_type lu > > I?m sorry I sent the wrong instructions before. > > Barry > > On Dec 20, 2013, at 7:54 AM, Que Cat wrote: > > > Hi Barry, > > > > Thanks for your exlanation. I used -snes_mf -pc_type lu and now it > failed with reason -3 (/* the linear solve failed */). So, something is > wrong with the function. > > > > > > On Thu, Dec 19, 2013 at 5:01 PM, Barry Smith wrote: > > > > ratio is || A - Afd || /||A|| where |||| is the frobenus norm and > difference is || A - Afd ||. > > > > Try using -snes_mf -pc_type lu > > > > This can often converge even if the A you provide is wrong, so if > this convergence sine then likely the Jacobian is wrong. > > > > It is also possible that the function evaluation has a bug. > > > > http://www.mcs.anl.gov/petsc/documentation/faq.html#newton > > > > > > Barry > > > > > > > > On Dec 19, 2013, at 4:26 PM, Que Cat wrote: > > > > > Hello, > > > > > > I use SNES to solve the non linear coupled equations. I have checked > the hand-coded jacobian with -snes_type test and received the result: > > > Norm of matrix ratio 1.39606e-23 difference 1.53372e-14 (user-defined > state) > > > Norm of matrix ratio 3.03157e-10 difference 0.33305 (constant state > -1.0) > > > Norm of matrix ratio 3.03157e-10 difference 0.33305 (constant state > 1.0) > > > It said that " if the ratio is O(1.e-8), the hand-coded Jacobian is > probably correct." What is the second paramenter "difference 1.53372e-14 > ...."? Does it play any role in checking the accuracy of hand-coded > jacobian? > > > > > > The SNES solver for failed with the reason -6 "the line search > failed". I tried with all other line search method stated in the PETSC > manual, but they all failed. > > > > > > I tried to figure out what is wrong with my code? Could you please > give me any recommendation to direct me how to check my code? I appreciate > your help. > > > > > > QUe > > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From popov at uni-mainz.de Fri Dec 20 08:54:25 2013 From: popov at uni-mainz.de (Anton Popov) Date: Fri, 20 Dec 2013 15:54:25 +0100 Subject: [petsc-users] Snes failled with reason -6 In-Reply-To: References: <0C85C52E-7BA8-4C90-A6DA-9BFB6F5F2C78@mcs.anl.gov> Message-ID: <52B45A21.2000801@uni-mainz.de> On 12/20/13 3:40 PM, Que Cat wrote: > Barry, > > It said that "No support for this operation for this object type! > Matrix format mpiaij does not have a built-in PETSc LU!". Try with -pc_factor_mat_solver_package mumps (superlu_dist) If you didn't install them reconfigure with: --download-scalapack=1 --download-mumps=1 or: --download-superlu_dist=1 Or just run sequentially Anton > > I run it with option "|-snes_monitor -ksp_monitor_true_residual > -snes_converged_reason -ksp_converged_reason|." and received : > 0 SNES Function norm 1.414213647226e-01 > 0 KSP preconditioned resid norm 8.424084241999e+08 true resid norm > 1.414213647226e-01 ||r(i)||/||b|| 1.000000000000e+00 > 1 KSP preconditioned resid norm 1.354908290122e-02 true resid norm > 1.424694873074e-01 ||r(i)||/||b|| 1.007411345428e+00 > Nonlinear solve did not converge due to DIVERGED_LINE_SEARCH iterations 0 > > It looks like that the preconditioned residual converges, but the true > residual does not. > > > On Fri, Dec 20, 2013 at 8:30 AM, Barry Smith > wrote: > > > Run with > > -snes_mf_operator -pc_type lu > > I?m sorry I sent the wrong instructions before. > > Barry > > On Dec 20, 2013, at 7:54 AM, Que Cat > wrote: > > > Hi Barry, > > > > Thanks for your exlanation. I used -snes_mf -pc_type lu and now > it failed with reason -3 (/* the linear solve failed */). So, > something is wrong with the function. > > > > > > On Thu, Dec 19, 2013 at 5:01 PM, Barry Smith > wrote: > > > > ratio is || A - Afd || /||A|| where |||| is the frobenus norm > and difference is || A - Afd ||. > > > > Try using -snes_mf -pc_type lu > > > > This can often converge even if the A you provide is wrong, > so if this convergence sine then likely the Jacobian is wrong. > > > > It is also possible that the function evaluation has a bug. > > > > http://www.mcs.anl.gov/petsc/documentation/faq.html#newton > > > > > > Barry > > > > > > > > On Dec 19, 2013, at 4:26 PM, Que Cat > wrote: > > > > > Hello, > > > > > > I use SNES to solve the non linear coupled equations. I have > checked the hand-coded jacobian with -snes_type test and received > the result: > > > Norm of matrix ratio 1.39606e-23 difference 1.53372e-14 > (user-defined state) > > > Norm of matrix ratio 3.03157e-10 difference 0.33305 (constant > state -1.0) > > > Norm of matrix ratio 3.03157e-10 difference 0.33305 (constant > state 1.0) > > > It said that " if the ratio is O(1.e-8), the hand-coded > Jacobian is probably correct." What is the second paramenter > "difference 1.53372e-14 ...."? Does it play any role in checking > the accuracy of hand-coded jacobian? > > > > > > The SNES solver for failed with the reason -6 "the line search > failed". I tried with all other line search method stated in the > PETSC manual, but they all failed. > > > > > > I tried to figure out what is wrong with my code? Could you > please give me any recommendation to direct me how to check my > code? I appreciate your help. > > > > > > QUe > > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Fri Dec 20 12:31:11 2013 From: bsmith at mcs.anl.gov (Barry Smith) Date: Fri, 20 Dec 2013 12:31:11 -0600 Subject: [petsc-users] Snes failled with reason -6 In-Reply-To: References: <0C85C52E-7BA8-4C90-A6DA-9BFB6F5F2C78@mcs.anl.gov> Message-ID: <3BBE82F6-778B-435B-AC10-1B0F18519385@mcs.anl.gov> Always run sequentially when debugging convergence issues etc (unless it converges fine sequentially and only has trouble in parallel). Was this with -snes_mf_operator ? I would recommend simplifying your ?function? and ?jacobian? by removing all the complicating terms until you get good convergence, and then start putting them back until you find the one causing the difficulties. Trying to directly bug a complicated function and jacobian is generally a nightmare. Barry On Dec 20, 2013, at 8:40 AM, Que Cat wrote: > Barry, > > It said that "No support for this operation for this object type! Matrix format mpiaij does not have a built-in PETSc LU!". > > I run it with option "-snes_monitor -ksp_monitor_true_residual -snes_converged_reason -ksp_converged_reason." and received : > 0 SNES Function norm 1.414213647226e-01 > 0 KSP preconditioned resid norm 8.424084241999e+08 true resid norm 1.414213647226e-01 ||r(i)||/||b|| 1.000000000000e+00 > 1 KSP preconditioned resid norm 1.354908290122e-02 true resid norm 1.424694873074e-01 ||r(i)||/||b|| 1.007411345428e+00 > Nonlinear solve did not converge due to DIVERGED_LINE_SEARCH iterations 0 > > It looks like that the preconditioned residual converges, but the true residual does not. > > > On Fri, Dec 20, 2013 at 8:30 AM, Barry Smith wrote: > > Run with > > -snes_mf_operator -pc_type lu > > I?m sorry I sent the wrong instructions before. > > Barry > > On Dec 20, 2013, at 7:54 AM, Que Cat wrote: > > > Hi Barry, > > > > Thanks for your exlanation. I used -snes_mf -pc_type lu and now it failed with reason -3 (/* the linear solve failed */). So, something is wrong with the function. > > > > > > On Thu, Dec 19, 2013 at 5:01 PM, Barry Smith wrote: > > > > ratio is || A - Afd || /||A|| where |||| is the frobenus norm and difference is || A - Afd ||. > > > > Try using -snes_mf -pc_type lu > > > > This can often converge even if the A you provide is wrong, so if this convergence sine then likely the Jacobian is wrong. > > > > It is also possible that the function evaluation has a bug. > > > > http://www.mcs.anl.gov/petsc/documentation/faq.html#newton > > > > > > Barry > > > > > > > > On Dec 19, 2013, at 4:26 PM, Que Cat wrote: > > > > > Hello, > > > > > > I use SNES to solve the non linear coupled equations. I have checked the hand-coded jacobian with -snes_type test and received the result: > > > Norm of matrix ratio 1.39606e-23 difference 1.53372e-14 (user-defined state) > > > Norm of matrix ratio 3.03157e-10 difference 0.33305 (constant state -1.0) > > > Norm of matrix ratio 3.03157e-10 difference 0.33305 (constant state 1.0) > > > It said that " if the ratio is O(1.e-8), the hand-coded Jacobian is probably correct." What is the second paramenter "difference 1.53372e-14 ...."? Does it play any role in checking the accuracy of hand-coded jacobian? > > > > > > The SNES solver for failed with the reason -6 "the line search failed". I tried with all other line search method stated in the PETSC manual, but they all failed. > > > > > > I tried to figure out what is wrong with my code? Could you please give me any recommendation to direct me how to check my code? I appreciate your help. > > > > > > QUe > > > > > > > > > From jedbrown at mcs.anl.gov Fri Dec 20 13:31:07 2013 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Fri, 20 Dec 2013 11:31:07 -0800 Subject: [petsc-users] Issue about matsetvalues when turn off debugging In-Reply-To: <2AAD8B09-C3AD-4326-8D02-CE3667889A32@gmail.com> References: <546FD27B-56BD-40BF-B129-D1EFB00A6F29@gmail.com> <87wqj121hw.fsf@jedbrown.org> <2AAD8B09-C3AD-4326-8D02-CE3667889A32@gmail.com> Message-ID: <877gaz7210.fsf@jedbrown.org> Gavin Ge writes: > Thanks Jed. After configuring using this, how can I turn on the debugging option when I compile the code? Compiling your code? If you use the PETSc makefiles, the same flag will be used. If you don't use PETSc makefiles, you're on your own to set compilation flags. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From jedbrown at mcs.anl.gov Fri Dec 20 14:50:24 2013 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Fri, 20 Dec 2013 12:50:24 -0800 Subject: [petsc-users] Feature request: Origin QMR implementation. In-Reply-To: References: <8761ql3hb6.fsf@jedbrown.org> Message-ID: <87bo0b5jsf.fsf@jedbrown.org> Ce Qin writes: > Thanks, Jed. > > I used left Jacobi preconditioner (by setting M1 = diag(A)) with Matlab's > QMR and BCGS. > I am new to PETSc and I am not very fimilar with linear solvers. If I want > to implement the QMR solver, what is the starting point? Copy a simple similar method, rename, register, and implement. Something like this: $ mkdir src/ksp/ksp/impls/qmr $ sed 's/tf\(qmr\)/\1/gi' src/ksp/ksp/impls/tfqmr/tfqmr.c > src/ksp/ksp/impls/qmr/qmr.c $ sed 's/tf\(qmr\)/\1/gi' src/ksp/ksp/impls/tfqmr/makefile > src/ksp/ksp/impls/qmr/makefile $ emacs src/ksp/ksp/impls/makefile # add new directory "qmr" to list $ emacs include/petscksp.h include/finclude/petsckspdef.h # add KSPQMR to both files $ emacs src/ksp/ksp/interface/itregis.c # add KSPQMR $ make # compile the new files # check that it works and commit starting point $ (cd src/ksp/ksp/examples/tutorials && make ex2 && ./ex2 -ksp_type qmr -ksp_monitor -ksp_view) $ git add src/ksp/ksp/impls/qmr/{qmr.c,makefile} $ git commit -am'KSPQMR: start new implementation, currently a copy of TFQMR' # change implementation from TFQMR to QMR, update references, etc. $ emacs src/ksp/ksp/impls/qmr/qmr.c -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From qince168 at gmail.com Sat Dec 21 02:39:04 2013 From: qince168 at gmail.com (Ce Qin) Date: Sat, 21 Dec 2013 16:39:04 +0800 Subject: [petsc-users] Feature request: Origin QMR implementation. In-Reply-To: <87bo0b5jsf.fsf@jedbrown.org> References: <8761ql3hb6.fsf@jedbrown.org> <87bo0b5jsf.fsf@jedbrown.org> Message-ID: Thanks for your detailed explanation. I have consulted the Templates book, I found that QMR requires splitting the preconditioner (M = M1 * M2). How to do this in PETSc's way? Best Regards, Ce Qin -------------- next part -------------- An HTML attachment was scrubbed... URL: From jedbrown at mcs.anl.gov Sat Dec 21 10:11:55 2013 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Sat, 21 Dec 2013 09:11:55 -0700 Subject: [petsc-users] Feature request: Origin QMR implementation. In-Reply-To: References: <8761ql3hb6.fsf@jedbrown.org> <87bo0b5jsf.fsf@jedbrown.org> Message-ID: <87haa29oac.fsf@jedbrown.org> Ce Qin writes: > Thanks for your detailed explanation. > > I have consulted the Templates book, I found that QMR requires splitting > the preconditioner (M = M1 * M2). > How to do this in PETSc's way? PCApplySymmetricLeft and PCApplySymmetricRight where PCSide must be PC_SYMMETRIC. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From potaman at outlook.com Mon Dec 23 22:23:17 2013 From: potaman at outlook.com (subramanya sadasiva) Date: Mon, 23 Dec 2013 23:23:17 -0500 Subject: [petsc-users] Extending PETSC-SNES-VI to linear inequality constraints, Message-ID: Hi, Is it possible to extend PETSC-SNES VI to handlie linear inequality constraints. My problem has 4 variables . 2 of them have bounds constraints, as well as a linear inequality. At present I've implemented an augmented lagrangian method to handle the linear inequality and I let SNES VI handle the bounds constraints. However, the convergence of this method is very poor. I'd like to know if there was an easy way to get SNES VI to handle the linear inequalities as well. Thanks, Subramanya -------------- next part -------------- An HTML attachment was scrubbed... URL: From jedbrown at mcs.anl.gov Mon Dec 23 22:27:57 2013 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Mon, 23 Dec 2013 21:27:57 -0700 Subject: [petsc-users] Extending PETSC-SNES-VI to linear inequality constraints, In-Reply-To: References: Message-ID: <87lhzaootu.fsf@jedbrown.org> subramanya sadasiva writes: > Hi, Is it possible to extend PETSC-SNES VI to handlie linear > inequality constraints. My problem has 4 variables . 2 of them have > bounds constraints, as well as a linear inequality. At present I've > implemented an augmented lagrangian method to handle the linear > inequality and I let SNES VI handle the bounds constraints. However, > the convergence of this method is very poor. I'd like to know if there > was an easy way to get SNES VI to handle the linear inequalities as > well. Thanks, This would be a useful extension, though I think nonlinear inequality constraints may ultimately be necessary. We are integrating TAO as a module in PETSc, which will help with problem formulation. Does your problem have an "energy" or objective functional, or is it a general variational inequality? -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From potaman at outlook.com Mon Dec 23 22:34:53 2013 From: potaman at outlook.com (subramanya sadasiva) Date: Mon, 23 Dec 2013 23:34:53 -0500 Subject: [petsc-users] Extending PETSC-SNES-VI to linear inequality constraints, In-Reply-To: <87lhzaootu.fsf@jedbrown.org> References: , <87lhzaootu.fsf@jedbrown.org> Message-ID: I am trying to solve a multiphase cahn hilliard equation with an obstacle potential. So I have an energy functional. However, the linear inequality arises when I eliminate one of the phase variables. So, the variables are phi1,phi2, phi3 , mu1,mu2 and mu3. with ,0 From: jedbrown at mcs.anl.gov > To: potaman at outlook.com; petsc-users at mcs.anl.gov > CC: tmunson at mcs.anl.gov > Subject: Re: [petsc-users] Extending PETSC-SNES-VI to linear inequality constraints, > Date: Mon, 23 Dec 2013 21:27:57 -0700 > > subramanya sadasiva writes: > > > Hi, Is it possible to extend PETSC-SNES VI to handlie linear > > inequality constraints. My problem has 4 variables . 2 of them have > > bounds constraints, as well as a linear inequality. At present I've > > implemented an augmented lagrangian method to handle the linear > > inequality and I let SNES VI handle the bounds constraints. However, > > the convergence of this method is very poor. I'd like to know if there > > was an easy way to get SNES VI to handle the linear inequalities as > > well. Thanks, > > This would be a useful extension, though I think nonlinear inequality > constraints may ultimately be necessary. We are integrating TAO as a > module in PETSc, which will help with problem formulation. > > Does your problem have an "energy" or objective functional, or is it a > general variational inequality? -------------- next part -------------- An HTML attachment was scrubbed... URL: From potaman at outlook.com Mon Dec 23 22:42:48 2013 From: potaman at outlook.com (subramanya sadasiva) Date: Mon, 23 Dec 2013 23:42:48 -0500 Subject: [petsc-users] Extending PETSC-SNES-VI to linear inequality constraints, In-Reply-To: References: , , <87lhzaootu.fsf@jedbrown.org>, Message-ID: Hi Jed, I was just looking through the SNES documentation, I came across SNESLineSearchSetVIFunctions. Is there an example of the use of this function to implement and arbitrary projection? Can I use this to implement a function that projects the solution back into the feasible region? I am assuming this should be able to handle simple linear inequality constraints of the type that I have, which can be applied pointwise. Thanks, Subramanya. From: potaman at outlook.com To: jedbrown at mcs.anl.gov; petsc-users at mcs.anl.gov Date: Mon, 23 Dec 2013 23:34:53 -0500 Subject: Re: [petsc-users] Extending PETSC-SNES-VI to linear inequality constraints, I am trying to solve a multiphase cahn hilliard equation with an obstacle potential. So I have an energy functional. However, the linear inequality arises when I eliminate one of the phase variables. So, the variables are phi1,phi2, phi3 , mu1,mu2 and mu3. with ,0 From: jedbrown at mcs.anl.gov > To: potaman at outlook.com; petsc-users at mcs.anl.gov > CC: tmunson at mcs.anl.gov > Subject: Re: [petsc-users] Extending PETSC-SNES-VI to linear inequality constraints, > Date: Mon, 23 Dec 2013 21:27:57 -0700 > > subramanya sadasiva writes: > > > Hi, Is it possible to extend PETSC-SNES VI to handlie linear > > inequality constraints. My problem has 4 variables . 2 of them have > > bounds constraints, as well as a linear inequality. At present I've > > implemented an augmented lagrangian method to handle the linear > > inequality and I let SNES VI handle the bounds constraints. However, > > the convergence of this method is very poor. I'd like to know if there > > was an easy way to get SNES VI to handle the linear inequalities as > > well. Thanks, > > This would be a useful extension, though I think nonlinear inequality > constraints may ultimately be necessary. We are integrating TAO as a > module in PETSc, which will help with problem formulation. > > Does your problem have an "energy" or objective functional, or is it a > general variational inequality? -------------- next part -------------- An HTML attachment was scrubbed... URL: From jedbrown at mcs.anl.gov Mon Dec 23 22:54:42 2013 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Mon, 23 Dec 2013 21:54:42 -0700 Subject: [petsc-users] Extending PETSC-SNES-VI to linear inequality constraints, In-Reply-To: References: <87lhzaootu.fsf@jedbrown.org> Message-ID: <87ha9yonl9.fsf@jedbrown.org> subramanya sadasiva writes: > Hi Jed, I was just looking through the SNES documentation, I came > across SNESLineSearchSetVIFunctions. Is there an example of the use of > this function to implement and arbitrary projection? Can I use this > to implement a function that projects the solution back into the > feasible region? I am assuming this should be able to handle simple > linear inequality constraints of the type that I have, which can be > applied pointwise. Hmm, that interface is currently used by SNESVI *implementations* to communicate information to the SNESLineSearch object. So while it may work to provide a custom projection, the current code will overwrite it, e.g., SNESSolve_VINEWTONRSLS in virs.c: ierr = SNESLineSearchSetVIFunctions(snes->linesearch, SNESVIProjectOntoBounds, SNESVIComputeInactiveSetFnorm);CHKERRQ(ierr); Maybe Peter has further thoughts about adding other constraints to the SNESVI interface. The restriction to simple bound constraints is a significant impediment, preventing its use for elastic contact and other applications. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From potaman at outlook.com Mon Dec 23 22:59:54 2013 From: potaman at outlook.com (subramanya sadasiva) Date: Mon, 23 Dec 2013 23:59:54 -0500 Subject: [petsc-users] Extending PETSC-SNES-VI to linear inequality constraints, In-Reply-To: <87ha9yonl9.fsf@jedbrown.org> References: <87lhzaootu.fsf@jedbrown.org> , <87ha9yonl9.fsf@jedbrown.org> Message-ID: Thanks Jed. Being able to apply other pointwise constraints will be really helpful, even through implementation of a custom project function .Augmented lagrangian for the one extra constraint really ruins my convergence. Subramanya > From: jedbrown at mcs.anl.gov > To: potaman at outlook.com; petsc-users at mcs.anl.gov > CC: brune at mcs.anl.gov > Subject: RE: [petsc-users] Extending PETSC-SNES-VI to linear inequality constraints, > Date: Mon, 23 Dec 2013 21:54:42 -0700 > > subramanya sadasiva writes: > > > Hi Jed, I was just looking through the SNES documentation, I came > > across SNESLineSearchSetVIFunctions. Is there an example of the use of > > this function to implement and arbitrary projection? Can I use this > > to implement a function that projects the solution back into the > > feasible region? I am assuming this should be able to handle simple > > linear inequality constraints of the type that I have, which can be > > applied pointwise. > > Hmm, that interface is currently used by SNESVI *implementations* to > communicate information to the SNESLineSearch object. So while it may > work to provide a custom projection, the current code will overwrite it, > e.g., SNESSolve_VINEWTONRSLS in virs.c: > > ierr = SNESLineSearchSetVIFunctions(snes->linesearch, SNESVIProjectOntoBounds, SNESVIComputeInactiveSetFnorm);CHKERRQ(ierr); > > Maybe Peter has further thoughts about adding other constraints to the > SNESVI interface. The restriction to simple bound constraints is a > significant impediment, preventing its use for elastic contact and other > applications. -------------- next part -------------- An HTML attachment was scrubbed... URL: From luchao at mail.iggcas.ac.cn Wed Dec 25 08:43:16 2013 From: luchao at mail.iggcas.ac.cn (=?GBK?B?wsCzrA==?=) Date: Wed, 25 Dec 2013 22:43:16 +0800 (GMT+08:00) Subject: [petsc-users] Vector Operations Message-ID: Hello! Recently, I am writing a program using Petsc, and I encouter a questions. the question is as follow: In matlab, if a=[1,2,3,4,5], b = [2,2,2,2,2], we can use c=a.*b to get c = [2,4,6,8,10]. This operation is called array multiply in matlab. But in Petsc' Vec operations, I can't find a counterpart. So, I want to kown how can I get c = a.*b in Petsc. Can you help me? I have tried to implement it by using the Petsc's function-VecGetValues(). But in parallel program, some erros like "undeclared here (not in a function)" always appear. I like the Petsc very much! Thank you for your work! Merry Christmas and Happy New Year! yours sincely. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jedbrown at mcs.anl.gov Wed Dec 25 09:26:30 2013 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Wed, 25 Dec 2013 08:26:30 -0700 Subject: [petsc-users] Vector Operations In-Reply-To: References: Message-ID: <87ha9xlzo9.fsf@jedbrown.org> ?? writes: > Hello! > > Recently, I am writing a program using Petsc, and I encouter a questions. > > the question is as follow: > > In matlab, if a=[1,2,3,4,5], b = [2,2,2,2,2], we can use c=a.*b to get c = [2,4,6,8,10]. This operation is called array multiply in matlab. > > But in Petsc' Vec operations, I can't find a counterpart. So, I want to kown how can I get c = a.*b in Petsc. Can you help me? http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Vec/VecPointwiseMult.html > I have tried to implement it by using the Petsc's function-VecGetValues(). > > But in parallel program, some erros like "undeclared here (not in a function)" always appear. Always send the entire error message. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From jedbrown at mcs.anl.gov Wed Dec 25 20:32:57 2013 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Wed, 25 Dec 2013 19:32:57 -0700 Subject: [petsc-users] 3d acoustic wave equation In-Reply-To: <19e14a1.7a3b.1432cae04cb.Coremail.luchao@mail.iggcas.ac.cn> References: <87ha9xlzo9.fsf@jedbrown.org> <19e14a1.7a3b.1432cae04cb.Coremail.luchao@mail.iggcas.ac.cn> Message-ID: <8761qcmjdy.fsf@jedbrown.org> ?? writes: > hi, Jed Brown: > > Thank you for your reply. > > I had met you in Changchun ,China, at about 2013/08. After > looking your operation of PETSC in your pc on that meeting, I > started to konw and study the Petsc. Do you have some Petsc's > program about the 3d acoustic wave equation(u_tt = ( c2 ) * > LAPLACIAN( u )). In Petsc's ts's tutorials, I just find the > heat equation(u_t = u_xx). Look at src/ts/examples/tutorials/ex12.c It could also be easily added as a first-order system to src/ts/examples/tutorials/ex11.c -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From dalcinl at gmail.com Thu Dec 26 08:53:08 2013 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Thu, 26 Dec 2013 12:53:08 -0200 Subject: [petsc-users] Vector Operations In-Reply-To: References: Message-ID: On 25 December 2013 11:43, ?? wrote: > So, I want to kown how can I get c = a.*b in Petsc. Can you help me? http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Vec/VecPointwiseMult.html -- Lisandro Dalcin --------------- CIMEC (UNL/CONICET) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1016) Tel/Fax: +54-342-4511169 From jedbrown at mcs.anl.gov Thu Dec 26 09:54:34 2013 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Thu, 26 Dec 2013 08:54:34 -0700 Subject: [petsc-users] Finite Element Method of 3d wave equation In-Reply-To: <1a979d4.a585.1432f9726b9.Coremail.luchao@mail.iggcas.ac.cn> References: <1a979d4.a585.1432f9726b9.Coremail.luchao@mail.iggcas.ac.cn> Message-ID: <87r48zli9x.fsf@jedbrown.org> Please always use "reply-all" so that your messages go to the list. This is standard mailing list etiquette. It is important to preserve threading for people who find this discussion later and so that we do not waste our time re-answering the same questions that have already been answered in private side-conversations. You'll likely get an answer faster that way too. ?? writes: > Hi Jed Brown: > > Thank you for your reply. My Phd's thesis is about 3d wave > equation solved by spectral element method(high order finite > element method). So I want to kown whether you have some docs > or Petsc's programs about my thesis direction. I am so sorry to > cause you so much trouble. Thanks! PETSc does not have special code for spectral element methods, though there is at least one example that uses spectral elements (src/ksp/ksp/examples/tutorials/ex59.c) and PetscFE (new in petsc-dev) has support for arbitrary order. The formulation in src/ts/examples/tutorials/ex12.c is a reasonable formulation of the time-dependent problem, though it is common to ask for a symplectic time integrator. Gauss methods are one way to get there (not necessarily the most efficient, but very high order and can be vectorized--new research topic), though there is a large literature on symplectic methods. Additional time integration methods are easy to add. > -----????----- > ???: "??" > ????: 2013?12?26? ??? > ???: "Jed Brown" > ??: > ??: Re: Re: 3d acoustic wave equation > > > >> -----????----- >> ???: "Jed Brown" >> ????: 2013?12?26? ??? >> ???: "??" >> ??: petsc-users >> ??: Re: 3d acoustic wave equation >> >> ?? writes: >> >> > hi, Jed Brown: >> > >> > Thank you for your reply. >> > >> > I had met you in Changchun ,China, at about 2013/08. After >> > looking your operation of PETSC in your pc on that meeting, I >> > started to konw and study the Petsc. Do you have some Petsc's >> > program about the 3d acoustic wave equation(u_tt = ( c2 ) * >> > LAPLACIAN( u )). In Petsc's ts's tutorials, I just find the >> > heat equation(u_t = u_xx). >> >> Look at src/ts/examples/tutorials/ex12.c >> >> It could also be easily added as a first-order system tojj >> src/ts/examples/tutorials/ex11.c -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From potaman at outlook.com Fri Dec 27 21:25:03 2013 From: potaman at outlook.com (subramanya sadasiva) Date: Fri, 27 Dec 2013 22:25:03 -0500 Subject: [petsc-users] fieldsplit error Message-ID: Hi, I have a nonlinear system with 6 variables grouped in two sets of 3. I am trying to get a fieldsplit preconditioner working . I have the following options set up -pc_type fieldsplit-pc_fieldsplit_type multiplicative-fieldsplit_0_fields 0,2,4-fieldsplit_1_fields 1,3,5-fieldsplit_0_ksp cg-fieldsplit_1_ksp cg-fieldsplit_0_pc hypre-fieldsplit_1_pc hypre However, I get the following error. My installation of libmesh has block matrices disabled, which I've seen to cause problems with fieldsplit. Any ideas what might cause this error?Thanks, Subramanya [0]PETSC ERROR: --------------------- Error Message ------------------------------------[0]PETSC ERROR: Petsc has generated inconsistent data![0]PETSC ERROR: Unhandled case, must have at least two fields, not 1![0]PETSC ERROR: ------------------------------------------------------------------------[0]PETSC ERROR: Petsc Release Version 3.4.3, unknown [0]PETSC ERROR: See docs/changes/index.html for recent updates.[0]PETSC ERROR: See docs/faq.html for hints about trouble shooting.[0]PETSC ERROR: See docs/index.html for manual pages. -------------- next part -------------- An HTML attachment was scrubbed... URL: From hzhang at mcs.anl.gov Fri Dec 27 23:05:56 2013 From: hzhang at mcs.anl.gov (Hong Zhang) Date: Fri, 27 Dec 2013 23:05:56 -0600 Subject: [petsc-users] fieldsplit error In-Reply-To: <6a84bbec095a46b0bb8c313561dd9320@NAGURSKI.anl.gov> References: <6a84bbec095a46b0bb8c313561dd9320@NAGURSKI.anl.gov> Message-ID: Add -pc_fieldsplit_block_size 2 See example petsc/src/snes/examples/tutorials/ex19.c: ./ex19 -snes_monitor_short -ksp_monitor_short -pc_type fieldsplit -pc_fieldsplit_block_size 2 -pc_fieldsplit_0_fields 0,1 -pc_fieldsplit_1_fields 0,1 -pc_fieldsplit_type multiplicative -snes_view -fieldsplit_pc_type lu -da_refine 1 -ksp_type fgmres (see runex19_8 in petsc/src/snes/examples/tutorials/makefile) Hong On Fri, Dec 27, 2013 at 9:25 PM, subramanya sadasiva wrote: > Hi, > I have a nonlinear system with 6 variables grouped in two sets of 3. I am > trying to get a fieldsplit preconditioner working . I have the following > options set up > > -pc_type fieldsplit > -pc_fieldsplit_type multiplicative > -fieldsplit_0_fields 0,2,4 > -fieldsplit_1_fields 1,3,5 > -fieldsplit_0_ksp cg > -fieldsplit_1_ksp cg > -fieldsplit_0_pc hypre > -fieldsplit_1_pc hypre > > However, I get the following error. My installation of libmesh has block > matrices disabled, which I've seen to cause problems with fieldsplit. Any > ideas what might cause this error? > Thanks, > Subramanya > > > > [0]PETSC ERROR: --------------------- Error Message > ------------------------------------ > [0]PETSC ERROR: Petsc has generated inconsistent data! > [0]PETSC ERROR: Unhandled case, must have at least two fields, not 1! > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: Petsc Release Version 3.4.3, unknown > [0]PETSC ERROR: See docs/changes/index.html for recent updates. > [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. > [0]PETSC ERROR: See docs/index.html for manual pages. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From potaman at outlook.com Fri Dec 27 23:15:00 2013 From: potaman at outlook.com (subramanya sadasiva) Date: Sat, 28 Dec 2013 00:15:00 -0500 Subject: [petsc-users] fieldsplit error Message-ID: Hi, I figured out that block size was missing. However I am not quite sure what the block size means. --- Original Message --- From: "Hong Zhang" Sent: December 28, 2013 12:06 AM To: "subramanya sadasiva" Cc: petsc-users at mcs.anl.gov, libmesh-users at lists.sourceforge.net Subject: Re: [petsc-users] fieldsplit error Add -pc_fieldsplit_block_size 2 See example petsc/src/snes/examples/tutorials/ex19.c: ./ex19 -snes_monitor_short -ksp_monitor_short -pc_type fieldsplit -pc_fieldsplit_block_size 2 -pc_fieldsplit_0_fields 0,1 -pc_fieldsplit_1_fields 0,1 -pc_fieldsplit_type multiplicative -snes_view -fieldsplit_pc_type lu -da_refine 1 -ksp_type fgmres (see runex19_8 in petsc/src/snes/examples/tutorials/makefile) Hong On Fri, Dec 27, 2013 at 9:25 PM, subramanya sadasiva wrote: > Hi, > I have a nonlinear system with 6 variables grouped in two sets of 3. I am > trying to get a fieldsplit preconditioner working . I have the following > options set up > > -pc_type fieldsplit > -pc_fieldsplit_type multiplicative > -fieldsplit_0_fields 0,2,4 > -fieldsplit_1_fields 1,3,5 > -fieldsplit_0_ksp cg > -fieldsplit_1_ksp cg > -fieldsplit_0_pc hypre > -fieldsplit_1_pc hypre > > However, I get the following error. My installation of libmesh has block > matrices disabled, which I've seen to cause problems with fieldsplit. Any > ideas what might cause this error? > Thanks, > Subramanya > > > > [0]PETSC ERROR: --------------------- Error Message > ------------------------------------ > [0]PETSC ERROR: Petsc has generated inconsistent data! > [0]PETSC ERROR: Unhandled case, must have at least two fields, not 1! > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: Petsc Release Version 3.4.3, unknown > [0]PETSC ERROR: See docs/changes/index.html for recent updates. > [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. > [0]PETSC ERROR: See docs/index.html for manual pages. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hzhang at mcs.anl.gov Sat Dec 28 11:28:57 2013 From: hzhang at mcs.anl.gov (Hong Zhang) Date: Sat, 28 Dec 2013 11:28:57 -0600 Subject: [petsc-users] fieldsplit error In-Reply-To: References: Message-ID: subramanya: > Hi, > I figured out that block size was missing. However I am not quite sure > what the block size means. > number of fields. See ./ex19 -snes_monitor_short -ksp_monitor_short -pc_type fieldsplit -pc_fieldsplit_block_size 2 -pc_fieldsplit_0_fields 0,1 -pc_fieldsplit_1_fields 0,1 -pc_fieldsplit_type multiplicative -snes_view -fieldsplit_pc_type lu -da_refine 1 -ksp_type fgmres -h |grep fieldsplit_block_size -pc_fieldsplit_block_size <-1>: Blocksize that defines number of fields (PCFieldSplitSetBlockSize) Hong > > --- Original Message --- > > From: "Hong Zhang" > Sent: December 28, 2013 12:06 AM > To: "subramanya sadasiva" > Cc: petsc-users at mcs.anl.gov, libmesh-users at lists.sourceforge.net > Subject: Re: [petsc-users] fieldsplit error > > Add > -pc_fieldsplit_block_size 2 > > See example > petsc/src/snes/examples/tutorials/ex19.c: > ./ex19 -snes_monitor_short -ksp_monitor_short -pc_type fieldsplit > -pc_fieldsplit_block_size 2 -pc_fieldsplit_0_fields 0,1 > -pc_fieldsplit_1_fields 0,1 -pc_fieldsplit_type multiplicative -snes_view > -fieldsplit_pc_type lu -da_refine 1 -ksp_type fgmres > > (see runex19_8 in petsc/src/snes/examples/tutorials/makefile) > > Hong > > > On Fri, Dec 27, 2013 at 9:25 PM, subramanya sadasiva wrote: > > Hi, > I have a nonlinear system with 6 variables grouped in two sets of 3. I am > trying to get a fieldsplit preconditioner working . I have the following > options set up > > -pc_type fieldsplit > -pc_fieldsplit_type multiplicative > -fieldsplit_0_fields 0,2,4 > -fieldsplit_1_fields 1,3,5 > -fieldsplit_0_ksp cg > -fieldsplit_1_ksp cg > -fieldsplit_0_pc hypre > -fieldsplit_1_pc hypre > > However, I get the following error. My installation of libmesh has block > matrices disabled, which I've seen to cause problems with fieldsplit. Any > ideas what might cause this error? > Thanks, > Subramanya > > > > [0]PETSC ERROR: --------------------- Error Message > ------------------------------------ > [0]PETSC ERROR: Petsc has generated inconsistent data! > [0]PETSC ERROR: Unhandled case, must have at least two fields, not 1! > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: Petsc Release Version 3.4.3, unknown > [0]PETSC ERROR: See docs/changes/index.html for recent updates. > [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. > [0]PETSC ERROR: See docs/index.html for manual pages. > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Sat Dec 28 15:06:40 2013 From: bsmith at mcs.anl.gov (Barry Smith) Date: Sat, 28 Dec 2013 15:06:40 -0600 Subject: [petsc-users] fieldsplit error In-Reply-To: References: Message-ID: <44762517-A66B-430D-881B-5F304A575B45@mcs.anl.gov> On Dec 27, 2013, at 9:25 PM, subramanya sadasiva wrote: > Hi, > I have a nonlinear system with 6 variables grouped in two sets of 3. I am trying to get a fieldsplit preconditioner working . I have the following options set up > > -pc_type fieldsplit > -pc_fieldsplit_type multiplicative > -fieldsplit_0_fields 0,2,4 > -fieldsplit_1_fields 1,3,5 These should be -pc_fieldsplit_0 or 1 _fields 0,2,4 or 1,3,5 the reason is these are options for pc_fieldsplit not for a particular sub solve > -fieldsplit_0_ksp cg > -fieldsplit_1_ksp cg These should be -fieldsplit_0 or 1 _ksp_type cg > -fieldsplit_0_pc hypre > -fieldsplit_1_pc hypre These should be -fieldsplit_0 or 1 _pc_type hypre But I don?t think hypre ensures a symmetric preconditioner so you probably use gmres above instead of cg Barry > > However, I get the following error. My installation of libmesh has block matrices disabled, which I've seen to cause problems with fieldsplit. Any ideas what might cause this error? > Thanks, > Subramanya > > > > [0]PETSC ERROR: --------------------- Error Message ------------------------------------ > [0]PETSC ERROR: Petsc has generated inconsistent data! > [0]PETSC ERROR: Unhandled case, must have at least two fields, not 1! > [0]PETSC ERROR: ------------------------------------------------------------------------ > [0]PETSC ERROR: Petsc Release Version 3.4.3, unknown > [0]PETSC ERROR: See docs/changes/index.html for recent updates. > [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. > [0]PETSC ERROR: See docs/index.html for manual pages. From gideon.simpson at icloud.com Sat Dec 28 19:44:00 2013 From: gideon.simpson at icloud.com (Gideon Simpson) Date: Sat, 28 Dec 2013 20:44:00 -0500 Subject: [petsc-users] dmda and structures Message-ID: I'm trying to use 1D DMDA's to manage distributed monte carlo computations, and I've got the following problem. If I know, in advance, how many floating points are needed for each realization, everything is fine. But, if I want to be able to set this at run time, I can't get it to work. What I have is: typedef struct { PetscScalar id; PetscScalar *x; } Realization; then, in the main part of the code: Realization *realization_array; DM sim_data_da; Vec sim_data_vec; PetscInt xsize=10, realization_dof, i; PetscOptionsGetInt(NULL, "-xsize", &xsize, NULL); realization_dof = xsize + 1; DMDACreate1d(PETSC_COMM_WORLD,DMDA_BOUNDARY_NONE, batch_size, realization_dof, 0, NULL, &sim_data_da); DMCreateGlobalVector(sim_data_da,&sim_data_vec); DMDAVecGetArray(sim_data_da, sim_data_vec, & realization_array); Up to this point, I have no problem, but, when I try to access realization_array[i].x[j], I get memory errors. Is this fundamentally unworkable, or is there a fix? -gideon From jedbrown at mcs.anl.gov Sun Dec 29 00:39:49 2013 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Sat, 28 Dec 2013 23:39:49 -0700 Subject: [petsc-users] fieldsplit error In-Reply-To: References: Message-ID: <87mwjkjh3e.fsf@jedbrown.org> Hong Zhang writes: > subramanya: > >> Hi, >> I figured out that block size was missing. However I am not quite sure >> what the block size means. >> > > number of fields. > See > ./ex19 -snes_monitor_short -ksp_monitor_short -pc_type fieldsplit > -pc_fieldsplit_block_size 2 -pc_fieldsplit_0_fields 0,1 > -pc_fieldsplit_1_fields 0,1 -pc_fieldsplit_type multiplicative -snes_view > -fieldsplit_pc_type lu -da_refine 1 -ksp_type fgmres -h |grep > fieldsplit_block_size Hong is correct, but this is a very confusing command-line to use as an explanation. If you read ex19, it defines four fields. The option is forcing PCFIELDSPLIT to interpret the array as having only two fields. Then both fields are being put into both splits, with a direct solver on both splits. I.e., the entire problem is being solved with a direct solver in both stages of a two-stage multiplicative preconditioner. This is a test to demonstrate that a weird code path works, but it's nothing you would actually use in practice. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From jedbrown at mcs.anl.gov Sun Dec 29 00:53:23 2013 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Sat, 28 Dec 2013 23:53:23 -0700 Subject: [petsc-users] dmda and structures In-Reply-To: References: Message-ID: <87k3eojggs.fsf@jedbrown.org> Gideon Simpson writes: > I'm trying to use 1D DMDA's to manage distributed monte carlo computations, and I've got the following problem. If I know, in advance, how many floating points are needed for each realization, everything is fine. But, if I want to be able to set this at run time, I can't get it to work. What I have is: > > typedef struct { > PetscScalar id; > PetscScalar *x; > } Realization; > > then, in the main part of the code: > > Realization *realization_array; > DM sim_data_da; > Vec sim_data_vec; > > PetscInt xsize=10, realization_dof, i; > PetscOptionsGetInt(NULL, "-xsize", &xsize, NULL); > > realization_dof = xsize + 1; > > > DMDACreate1d(PETSC_COMM_WORLD,DMDA_BOUNDARY_NONE, batch_size, realization_dof, 0, NULL, &sim_data_da); > DMCreateGlobalVector(sim_data_da,&sim_data_vec); > > DMDAVecGetArray(sim_data_da, sim_data_vec, & realization_array); Vec can only hold PetscScalar, not pointers. > Up to this point, I have no problem, but, when I try to access > realization_array[i].x[j], I get memory errors. Is this > fundamentally unworkable, or is there a fix? You can't have this syntax in C or C++. I suggest tiny accessor functions that use the realization_dof to return the correct entries. The simplest version would look like realization_access(ctx,realization_array,i)[j] where ctx holds the block size and realization_array is just a plain PetscScalar*. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From bsmith at mcs.anl.gov Sun Dec 29 13:33:50 2013 From: bsmith at mcs.anl.gov (Barry Smith) Date: Sun, 29 Dec 2013 13:33:50 -0600 Subject: [petsc-users] dmda and structures In-Reply-To: References: Message-ID: <54F1C412-70BE-40D1-AB52-A7A11F142E1A@mcs.anl.gov> DMDAVecGetArrayDOF() might serve your needs. You would access with realization_array[i][j] for the ith grid point and j th realization Barry On Dec 28, 2013, at 7:44 PM, Gideon Simpson wrote: > I'm trying to use 1D DMDA's to manage distributed monte carlo computations, and I've got the following problem. If I know, in advance, how many floating points are needed for each realization, everything is fine. But, if I want to be able to set this at run time, I can't get it to work. What I have is: > > typedef struct { > PetscScalar id; > PetscScalar *x; > } Realization; > > then, in the main part of the code: > > Realization *realization_array; > DM sim_data_da; > Vec sim_data_vec; > > PetscInt xsize=10, realization_dof, i; > PetscOptionsGetInt(NULL, "-xsize", &xsize, NULL); > > realization_dof = xsize + 1; > > > DMDACreate1d(PETSC_COMM_WORLD,DMDA_BOUNDARY_NONE, batch_size, realization_dof, 0, NULL, &sim_data_da); > DMCreateGlobalVector(sim_data_da,&sim_data_vec); > > DMDAVecGetArray(sim_data_da, sim_data_vec, & realization_array); > > Up to this point, I have no problem, but, when I try to access realization_array[i].x[j], I get memory errors. Is this fundamentally unworkable, or is there a fix? > > -gideon > From gideon.simpson at icloud.com Sun Dec 29 13:38:27 2013 From: gideon.simpson at icloud.com (Gideon Simpson) Date: Sun, 29 Dec 2013 14:38:27 -0500 Subject: [petsc-users] dmda and structures In-Reply-To: <54F1C412-70BE-40D1-AB52-A7A11F142E1A@mcs.anl.gov> References: <54F1C412-70BE-40D1-AB52-A7A11F142E1A@mcs.anl.gov> Message-ID: How would that work? I took Jed's advice, and set it up so that I have access via: realization_array[ realization_dof * i + 1 +j] = jth point of ith realization and then I further used a pointer: PetscScalar *x; x = & realization_array[ realization_dof * i + 1]; so that I have "clean" access to the elements of each realization through x[j]. -gideon On Dec 29, 2013, at 2:33 PM, Barry Smith wrote: > > DMDAVecGetArrayDOF() might serve your needs. You would access with realization_array[i][j] for the ith grid point and j th realization > > Barry > > > > On Dec 28, 2013, at 7:44 PM, Gideon Simpson wrote: > >> I'm trying to use 1D DMDA's to manage distributed monte carlo computations, and I've got the following problem. If I know, in advance, how many floating points are needed for each realization, everything is fine. But, if I want to be able to set this at run time, I can't get it to work. What I have is: >> >> typedef struct { >> PetscScalar id; >> PetscScalar *x; >> } Realization; >> >> then, in the main part of the code: >> >> Realization *realization_array; >> DM sim_data_da; >> Vec sim_data_vec; >> >> PetscInt xsize=10, realization_dof, i; >> PetscOptionsGetInt(NULL, "-xsize", &xsize, NULL); >> >> realization_dof = xsize + 1; >> >> >> DMDACreate1d(PETSC_COMM_WORLD,DMDA_BOUNDARY_NONE, batch_size, realization_dof, 0, NULL, &sim_data_da); >> DMCreateGlobalVector(sim_data_da,&sim_data_vec); >> >> DMDAVecGetArray(sim_data_da, sim_data_vec, & realization_array); >> >> Up to this point, I have no problem, but, when I try to access realization_array[i].x[j], I get memory errors. Is this fundamentally unworkable, or is there a fix? >> >> -gideon >> > From jedbrown at mcs.anl.gov Sun Dec 29 13:51:29 2013 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Sun, 29 Dec 2013 12:51:29 -0700 Subject: [petsc-users] dmda and structures In-Reply-To: References: <54F1C412-70BE-40D1-AB52-A7A11F142E1A@mcs.anl.gov> Message-ID: <87zjnjigfy.fsf@jedbrown.org> Gideon Simpson writes: > How would that work? I took Jed's advice, and set it up so that I have access via: > > realization_array[ realization_dof * i + 1 +j] = jth point of ith realization > > and then I further used a pointer: > > PetscScalar *x; > > x = & realization_array[ realization_dof * i + 1]; > > so that I have "clean" access to the elements of each realization through x[j]. DMDAVecGetArrayDOF sets up one extra level of pointer indirection. You can't have the reserved member "id" that way, but you can use a syntax closer to what you suggested. PetscScalar **realization_array; DMDAVecGetArrayDOF(sim_data_da, sim_data_vec, &realization_array); realization_array[i][j] = 1.234; -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From bsmith at mcs.anl.gov Sun Dec 29 13:53:18 2013 From: bsmith at mcs.anl.gov (Barry Smith) Date: Sun, 29 Dec 2013 13:53:18 -0600 Subject: [petsc-users] dmda and structures In-Reply-To: References: <54F1C412-70BE-40D1-AB52-A7A11F142E1A@mcs.anl.gov> Message-ID: <3C348792-7CEA-4ED3-8F7D-6071C078D11D@mcs.anl.gov> On Dec 29, 2013, at 1:38 PM, Gideon Simpson wrote: > How would that work? I?ve modified your code fragment below with how I think you could do it. > I took Jed's advice, and set it up so that I have access via: > > realization_array[ realization_dof * i + 1 +j] = jth point of ith realization > > and then I further used a pointer: > > PetscScalar *x; > > x = & realization_array[ realization_dof * i + 1]; > > so that I have "clean" access to the elements of each realization through x[j]. > > -gideon > > On Dec 29, 2013, at 2:33 PM, Barry Smith wrote: > >> >> DMDAVecGetArrayDOF() might serve your needs. You would access with realization_array[i][j] for the ith grid point and j th realization >> >> Barry >> >> >> >> On Dec 28, 2013, at 7:44 PM, Gideon Simpson wrote: >> >>> I'm trying to use 1D DMDA's to manage distributed monte carlo computations, and I've got the following problem. If I know, in advance, how many floating points are needed for each realization, everything is fine. But, if I want to be able to set this at run time, I can't get it to work. What I have is: >>> >>> >>> then, in the main part of the code: >>> >>> Realization *realization_array; >>> DM sim_data_da; >>> Vec sim_data_vec; >>> >>> PetscInt xsize=10, realization_dof, i; >>> PetscOptionsGetInt(NULL, "-xsize", &xsize, NULL); >>> >>> realization_dof = xsize + 1; >>> >>> >>> DMDACreate1d(PETSC_COMM_WORLD,DMDA_BOUNDARY_NONE, batch_size, realization_dof, 0, NULL, &sim_data_da); >>> DMCreateGlobalVector(sim_data_da,&sim_data_vec); >>> PetscScalar **realization_array; >>> DMDAVecGetArrayDOF(sim_data_da, sim_data_vec, & realization_array); something = realization_array[i][j]; here i varies from xs to xs+xm-1 and j varies from 0 to realization_dof-1 >>> >>> >>> -gideon >>> >> > From iwaddington at gmail.com Mon Dec 30 12:17:47 2013 From: iwaddington at gmail.com (iwaddington .) Date: Mon, 30 Dec 2013 16:17:47 -0200 Subject: [petsc-users] DMSetCoordinates(DM dm, Vec c) correct usage Message-ID: Hi everybody, I am trying to set the coordinates of a non-uniform dm, and to do so I found the function DMSetCoordinates(DM dm, Vec c), but the problem is I didn't find in the website manual any example of how to use it, so I don't know how to set the vector "c" in the function prototype, I mean, is this a common vector, or one declared with DMCreateGlobalVector (which doesn't make much sense but I don't know for sure ) ? And more importantly, how should I organize the coordinates I want to set inside the vector "c" ? Thanks in advance, and happy new year. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jedbrown at mcs.anl.gov Mon Dec 30 12:25:47 2013 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Mon, 30 Dec 2013 11:25:47 -0700 Subject: [petsc-users] DMSetCoordinates(DM dm, Vec c) correct usage In-Reply-To: References: Message-ID: <87y532gpqs.fsf@jedbrown.org> "iwaddington ." writes: > Hi everybody, I am trying to set the coordinates of a non-uniform > dm, and to do > so I found the function DMSetCoordinates(DM dm, Vec c), but the > problem is I didn't > > find in the website manual any example of how to use it, so I don't > know how to set > the vector "c" in the function prototype, I mean, is this a common > vector, or one > declared with DMCreateGlobalVector (which doesn't make much sense but > I don't know > > for sure ) ? DMGetCoordinateDM(dm,&cdm); DMCreateGlobalVector(cdm,&coords); // fill in the coordinates {x0,y0,x1,y1,...} DMSetCoordinates(dm,coords); VecDestroy(&coords); // assuming your code no longer needs this Vec -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From yaakoub at tacc.utexas.edu Mon Dec 30 19:23:17 2013 From: yaakoub at tacc.utexas.edu (Yaakoub El Khamra) Date: Mon, 30 Dec 2013 19:23:17 -0600 Subject: [petsc-users] DMRefine issue Message-ID: Greetings I am try to use DMRefine to refine a dm I create with DMPlexCreateBoxMesh with tonight's checkout. A quick glance at the DMView output from the DM's shows that the refinedMesh looks a lot like the original dm. DMGetRefineLevel returns 1 though. I must be missing something elementary here. Can someone please confirm if there are any glaring mistakes I made? Also if I want to refine and distribute, should I refine on a single processor then distribute or can I refine a couple of times, distribute, and refine some more? The test code looks as follows: program testRefine implicit none #include "finclude/petsc.h90" DM :: dm DM :: refinedMesh PetscInt :: r PetscInt :: dim PetscBool :: interpolate PetscErrorCode :: ierr call PetscInitialize(PETSC_NULL_CHARACTER, ierr) CHKERRQ(ierr) dim = 2 interpolate = PETSC_TRUE ! Create a mesh call DMPlexCreateBoxMesh(PETSC_COMM_WORLD, dim, interpolate, dm, $ ierr) CHKERRQ(ierr) call DMView(dm, PETSC_VIEWER_STDOUT_WORLD, ierr) CHKERRQ(ierr) call DMRefine(dm, PETSC_COMM_WORLD, refinedMesh, ierr) CHKERRQ(ierr) call DMView(refinedMesh, PETSC_VIEWER_STDOUT_WORLD, ierr) CHKERRQ(ierr) call DMGetRefineLevel(refinedMesh, r, ierr) write (*,*) "The value of r is:", r call DMDestroy(dm, ierr) call DMDestroy(refinedMesh, ierr) CHKERRQ(ierr) call PetscFinalize(ierr) end program testRefine And the output: DM_0x1122bb0_0 in 2 dimensions: 0-cells: 9 1-cells: 16 2-cells: 8 Labels: marker: 2 strata of sizes (16, 5) depth: 3 strata of sizes (9, 16, 8) DM_0x1122bb0_1 in 2 dimensions: 0-cells: 9 1-cells: 16 2-cells: 8 Labels: marker: 2 strata of sizes (16, 1) depth: 3 strata of sizes (9, 16, 8) The value of r is: 1 Regards Yaakoub El Khamra -------------- next part -------------- An HTML attachment was scrubbed... URL: From yaakoub at tacc.utexas.edu Mon Dec 30 19:34:35 2013 From: yaakoub at tacc.utexas.edu (Yaakoub El Khamra) Date: Mon, 30 Dec 2013 19:34:35 -0600 Subject: [petsc-users] DMPlexDistribute question Message-ID: If I create a dm with DMPlexCreateBoxMesh and immediately attempt to distribute it with DMPlexDistribute, the call to DMPlexDistribute never returns. I am working with a development checkout and the code looks as follows: call DMPlexCreateBoxMesh(PETSC_COMM_WORLD, 2, PETSC_TRUE, dm, $ ierr) CHKERRQ(ierr) call DMPlexDistribute(dm, "chaco", 0, $ PETSC_NULL_OBJECT, distributedMesh, ierr) CHKERRQ(ierr) Any thoughts? Regards Yaakoub El Khamra -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Mon Dec 30 22:58:36 2013 From: knepley at gmail.com (Matthew Knepley) Date: Mon, 30 Dec 2013 22:58:36 -0600 Subject: [petsc-users] DMPlexDistribute question In-Reply-To: References: Message-ID: On Mon, Dec 30, 2013 at 7:34 PM, Yaakoub El Khamra wrote: > > If I create a dm with DMPlexCreateBoxMesh and immediately attempt to > distribute it with DMPlexDistribute, the call to DMPlexDistribute never > returns. I am working with a development checkout and the code looks as > follows: > > call DMPlexCreateBoxMesh(PETSC_COMM_WORLD, 2, PETSC_TRUE, dm, > $ ierr) > CHKERRQ(ierr) > > call DMPlexDistribute(dm, "chaco", 0, > $ PETSC_NULL_OBJECT, distributedMesh, ierr) > CHKERRQ(ierr) > > Any thoughts? > Can you run SNES ex12? It does exactly this and gets run in the nightly tests. I can run it in parallel with no problems. My guess is that something is going wrong with the Fortran binding. Can you get a stack trace for the hang? If not, can you send a full Fortran program which exhibits your problem? Matt > Regards > Yaakoub El Khamra > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From yaakoub at tacc.utexas.edu Tue Dec 31 11:38:09 2013 From: yaakoub at tacc.utexas.edu (Yaakoub El Khamra) Date: Tue, 31 Dec 2013 11:38:09 -0600 Subject: [petsc-users] DMPlexDistribute question In-Reply-To: References: Message-ID: snes ex12 also hangs. Then I started trying all other examples, all MPI examples hang. Basic MPI code and benchmarks appear to compile and run fine. I recompiled PETSc and went carefully over the output, there are a lot of warnings (even from a stable build) If I compile petsc with openmpi 1.7.3 and gcc 4.8.2 and the configure line: /configure --with-mpi-compilers=yes --download-metis --download-parmetis --download-fiat --with-fortran=yes --download-scientificpython --download-f-blas-lapack=1 --download-triangle --download-chaco --download-exodusii --download-netcdf --download-hdf5 I get a lot of warnings from MPI that might significant. I can provide access to the fedora 20 box where I am working. /home/yye00/Work/Saswata/petsc/src/sys/fileio/fretrieve.c:159:5: warning: ?MPI_Keyval_create? is deprecated (declared at /usr/include/openmpi-x86_64/mpi.h:1506): MPI_Keyval_create is superseded by MPI_Comm_create_keyval in MPI-2.0 [-Wdeprecated-declarations] ierr = MPI_Keyval_create(MPI_NULL_COPY_FN,Petsc_DelTmpShared,&Petsc_Tmp_keyval,0);CHKERRQ(ierr); ^ /home/yye00/Work/Saswata/petsc/src/sys/fileio/fretrieve.c:159:5: warning: ?OMPI_C_MPI_NULL_COPY_FN? is deprecated (declared at /usr/include/openmpi-x86_64/mpi.h:828): MPI_NULL_COPY_FN is deprecated in MPI-2.0 [-Wdeprecated-declarations] /home/yye00/Work/Saswata/petsc/src/sys/fileio/fretrieve.c:162:3: warning: ?MPI_Attr_get? is deprecated (declared at /usr/include/openmpi-x86_64/mpi.h:1201): MPI_Attr_get is superseded by MPI_Comm_get_attr in MPI-2.0 [-Wdeprecated-declarations] ierr = MPI_Attr_get(comm,Petsc_Tmp_keyval,(void**)&tagvalp,(int*)&iflg);CHKERRQ(ierr); ^ /home/yye00/Work/Saswata/petsc/src/sys/fileio/fretrieve.c:168:5: warning: ?MPI_Attr_put? is deprecated (declared at /usr/include/openmpi-x86_64/mpi.h:1203): MPI_Attr_put is superseded by MPI_Comm_set_attr in MPI-2.0 [-Wdeprecated-declarations] ierr = MPI_Attr_put(comm,Petsc_Tmp_keyval,tagvalp);CHKERRQ(ierr); ^ /home/yye00/Work/Saswata/petsc/src/sys/fileio/fretrieve.c: In function ?PetscSharedWorkingDirectory?: /home/yye00/Work/Saswata/petsc/src/sys/fileio/fretrieve.c:282:5: warning: ?MPI_Keyval_create? is deprecated (declared at /usr/include/openmpi-x86_64/mpi.h:1506): MPI_Keyval_create is superseded by MPI_Comm_create_keyval in MPI-2.0 [-Wdeprecated-declarations] ierr = MPI_Keyval_create(MPI_NULL_COPY_FN,Petsc_DelTmpShared,&Petsc_WD_keyval,0);CHKERRQ(ierr); ^ /home/yye00/Work/Saswata/petsc/src/sys/fileio/fretrieve.c:282:5: warning: ?OMPI_C_MPI_NULL_COPY_FN? is deprecated (declared at /usr/include/openmpi-x86_64/mpi.h:828): MPI_NULL_COPY_FN is deprecated in MPI-2.0 [-Wdeprecated-declarations] /home/yye00/Work/Saswata/petsc/src/sys/fileio/fretrieve.c:285:3: warning: ?MPI_Attr_get? is deprecated (declared at /usr/include/openmpi-x86_64/mpi.h:1201): MPI_Attr_get is superseded by MPI_Comm_get_attr in MPI-2.0 [-Wdeprecated-declarations] ierr = MPI_Attr_get(comm,Petsc_WD_keyval,(void**)&tagvalp,(int*)&iflg);CHKERRQ(ierr); ^ /home/yye00/Work/Saswata/petsc/src/sys/fileio/fretrieve.c:291:5: warning: ?MPI_Attr_put? is deprecated (declared at /usr/include/openmpi-x86_64/mpi.h:1203): MPI_Attr_put is superseded by MPI_Comm_set_attr in MPI-2.0 [-Wdeprecated-declarations] ierr = MPI_Attr_put(comm,Petsc_WD_keyval,tagvalp);CHKERRQ(ierr); Regards Yaakoub El Khamra On Mon, Dec 30, 2013 at 10:58 PM, Matthew Knepley wrote: > On Mon, Dec 30, 2013 at 7:34 PM, Yaakoub El Khamra < > yaakoub at tacc.utexas.edu> wrote: > >> >> If I create a dm with DMPlexCreateBoxMesh and immediately attempt to >> distribute it with DMPlexDistribute, the call to DMPlexDistribute never >> returns. I am working with a development checkout and the code looks as >> follows: >> >> call DMPlexCreateBoxMesh(PETSC_COMM_WORLD, 2, PETSC_TRUE, dm, >> $ ierr) >> CHKERRQ(ierr) >> >> call DMPlexDistribute(dm, "chaco", 0, >> $ PETSC_NULL_OBJECT, distributedMesh, ierr) >> CHKERRQ(ierr) >> >> Any thoughts? >> > > Can you run SNES ex12? It does exactly this and gets run in the nightly > tests. I can run it in parallel > with no problems. My guess is that something is going wrong with the > Fortran binding. Can you > get a stack trace for the hang? If not, can you send a full Fortran > program which exhibits your problem? > > Matt > > >> Regards >> Yaakoub El Khamra >> > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Tue Dec 31 11:44:57 2013 From: knepley at gmail.com (Matthew Knepley) Date: Tue, 31 Dec 2013 11:44:57 -0600 Subject: [petsc-users] DMPlexDistribute question In-Reply-To: References: Message-ID: On Tue, Dec 31, 2013 at 11:38 AM, Yaakoub El Khamra wrote: > > snes ex12 also hangs. Then I started trying all other examples, all MPI > examples hang. Basic MPI code and benchmarks appear to compile and run > fine. I recompiled PETSc and went carefully over the output, there are a > lot of warnings (even from a stable build) > > If I compile petsc with openmpi 1.7.3 and gcc 4.8.2 and the configure line: > > /configure --with-mpi-compilers=yes --download-metis --download-parmetis > --download-fiat --with-fortran=yes --download-scientificpython > --download-f-blas-lapack=1 --download-triangle --download-chaco > --download-exodusii --download-netcdf --download-hdf5 > > I get a lot of warnings from MPI that might significant. I can provide > access to the fedora 20 box where I am working. > Either send login info to petsc-maint at mcs.anl.gov, or get a stack trace of the hang using gdb. Thanks, Matt > /home/yye00/Work/Saswata/petsc/src/sys/fileio/fretrieve.c:159:5: warning: > ?MPI_Keyval_create? is deprecated (declared at > /usr/include/openmpi-x86_64/mpi.h:1506): MPI_Keyval_create is superseded by > MPI_Comm_create_keyval in MPI-2.0 [-Wdeprecated-declarations] > ierr = > MPI_Keyval_create(MPI_NULL_COPY_FN,Petsc_DelTmpShared,&Petsc_Tmp_keyval,0);CHKERRQ(ierr); > ^ > /home/yye00/Work/Saswata/petsc/src/sys/fileio/fretrieve.c:159:5: warning: > ?OMPI_C_MPI_NULL_COPY_FN? is deprecated (declared at > /usr/include/openmpi-x86_64/mpi.h:828): MPI_NULL_COPY_FN is deprecated in > MPI-2.0 [-Wdeprecated-declarations] > /home/yye00/Work/Saswata/petsc/src/sys/fileio/fretrieve.c:162:3: warning: > ?MPI_Attr_get? is deprecated (declared at > /usr/include/openmpi-x86_64/mpi.h:1201): MPI_Attr_get is superseded by > MPI_Comm_get_attr in MPI-2.0 [-Wdeprecated-declarations] > ierr = > MPI_Attr_get(comm,Petsc_Tmp_keyval,(void**)&tagvalp,(int*)&iflg);CHKERRQ(ierr); > ^ > /home/yye00/Work/Saswata/petsc/src/sys/fileio/fretrieve.c:168:5: warning: > ?MPI_Attr_put? is deprecated (declared at > /usr/include/openmpi-x86_64/mpi.h:1203): MPI_Attr_put is superseded by > MPI_Comm_set_attr in MPI-2.0 [-Wdeprecated-declarations] > ierr = MPI_Attr_put(comm,Petsc_Tmp_keyval,tagvalp);CHKERRQ(ierr); > ^ > /home/yye00/Work/Saswata/petsc/src/sys/fileio/fretrieve.c: In function > ?PetscSharedWorkingDirectory?: > /home/yye00/Work/Saswata/petsc/src/sys/fileio/fretrieve.c:282:5: warning: > ?MPI_Keyval_create? is deprecated (declared at > /usr/include/openmpi-x86_64/mpi.h:1506): MPI_Keyval_create is superseded by > MPI_Comm_create_keyval in MPI-2.0 [-Wdeprecated-declarations] > ierr = > MPI_Keyval_create(MPI_NULL_COPY_FN,Petsc_DelTmpShared,&Petsc_WD_keyval,0);CHKERRQ(ierr); > ^ > /home/yye00/Work/Saswata/petsc/src/sys/fileio/fretrieve.c:282:5: warning: > ?OMPI_C_MPI_NULL_COPY_FN? is deprecated (declared at > /usr/include/openmpi-x86_64/mpi.h:828): MPI_NULL_COPY_FN is deprecated in > MPI-2.0 [-Wdeprecated-declarations] > /home/yye00/Work/Saswata/petsc/src/sys/fileio/fretrieve.c:285:3: warning: > ?MPI_Attr_get? is deprecated (declared at > /usr/include/openmpi-x86_64/mpi.h:1201): MPI_Attr_get is superseded by > MPI_Comm_get_attr in MPI-2.0 [-Wdeprecated-declarations] > ierr = > MPI_Attr_get(comm,Petsc_WD_keyval,(void**)&tagvalp,(int*)&iflg);CHKERRQ(ierr); > ^ > /home/yye00/Work/Saswata/petsc/src/sys/fileio/fretrieve.c:291:5: warning: > ?MPI_Attr_put? is deprecated (declared at > /usr/include/openmpi-x86_64/mpi.h:1203): MPI_Attr_put is superseded by > MPI_Comm_set_attr in MPI-2.0 [-Wdeprecated-declarations] > ierr = MPI_Attr_put(comm,Petsc_WD_keyval,tagvalp);CHKERRQ(ierr); > > > > Regards > Yaakoub El Khamra > > > On Mon, Dec 30, 2013 at 10:58 PM, Matthew Knepley wrote: > >> On Mon, Dec 30, 2013 at 7:34 PM, Yaakoub El Khamra < >> yaakoub at tacc.utexas.edu> wrote: >> >>> >>> If I create a dm with DMPlexCreateBoxMesh and immediately attempt to >>> distribute it with DMPlexDistribute, the call to DMPlexDistribute never >>> returns. I am working with a development checkout and the code looks as >>> follows: >>> >>> call DMPlexCreateBoxMesh(PETSC_COMM_WORLD, 2, PETSC_TRUE, dm, >>> $ ierr) >>> CHKERRQ(ierr) >>> >>> call DMPlexDistribute(dm, "chaco", 0, >>> $ PETSC_NULL_OBJECT, distributedMesh, ierr) >>> CHKERRQ(ierr) >>> >>> Any thoughts? >>> >> >> Can you run SNES ex12? It does exactly this and gets run in the nightly >> tests. I can run it in parallel >> with no problems. My guess is that something is going wrong with the >> Fortran binding. Can you >> get a stack trace for the hang? If not, can you send a full Fortran >> program which exhibits your problem? >> >> Matt >> >> >>> Regards >>> Yaakoub El Khamra >>> >> >> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which their >> experiments lead. >> -- Norbert Wiener >> > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From yaakoub at tacc.utexas.edu Tue Dec 31 11:59:19 2013 From: yaakoub at tacc.utexas.edu (Yaakoub El Khamra) Date: Tue, 31 Dec 2013 11:59:19 -0600 Subject: [petsc-users] DMPlexDistribute question In-Reply-To: References: Message-ID: Just sent credentials. I am also trying on stampede. Regards Yaakoub El Khamra On Tue, Dec 31, 2013 at 11:44 AM, Matthew Knepley wrote: > On Tue, Dec 31, 2013 at 11:38 AM, Yaakoub El Khamra < > yaakoub at tacc.utexas.edu> wrote: > >> >> snes ex12 also hangs. Then I started trying all other examples, all MPI >> examples hang. Basic MPI code and benchmarks appear to compile and run >> fine. I recompiled PETSc and went carefully over the output, there are a >> lot of warnings (even from a stable build) >> >> If I compile petsc with openmpi 1.7.3 and gcc 4.8.2 and the configure >> line: >> >> /configure --with-mpi-compilers=yes --download-metis --download-parmetis >> --download-fiat --with-fortran=yes --download-scientificpython >> --download-f-blas-lapack=1 --download-triangle --download-chaco >> --download-exodusii --download-netcdf --download-hdf5 >> >> I get a lot of warnings from MPI that might significant. I can provide >> access to the fedora 20 box where I am working. >> > > Either send login info to petsc-maint at mcs.anl.gov, or get a stack trace > of the hang using gdb. > > Thanks, > > Matt > > >> /home/yye00/Work/Saswata/petsc/src/sys/fileio/fretrieve.c:159:5: warning: >> ?MPI_Keyval_create? is deprecated (declared at >> /usr/include/openmpi-x86_64/mpi.h:1506): MPI_Keyval_create is superseded by >> MPI_Comm_create_keyval in MPI-2.0 [-Wdeprecated-declarations] >> ierr = >> MPI_Keyval_create(MPI_NULL_COPY_FN,Petsc_DelTmpShared,&Petsc_Tmp_keyval,0);CHKERRQ(ierr); >> ^ >> /home/yye00/Work/Saswata/petsc/src/sys/fileio/fretrieve.c:159:5: warning: >> ?OMPI_C_MPI_NULL_COPY_FN? is deprecated (declared at >> /usr/include/openmpi-x86_64/mpi.h:828): MPI_NULL_COPY_FN is deprecated in >> MPI-2.0 [-Wdeprecated-declarations] >> /home/yye00/Work/Saswata/petsc/src/sys/fileio/fretrieve.c:162:3: warning: >> ?MPI_Attr_get? is deprecated (declared at >> /usr/include/openmpi-x86_64/mpi.h:1201): MPI_Attr_get is superseded by >> MPI_Comm_get_attr in MPI-2.0 [-Wdeprecated-declarations] >> ierr = >> MPI_Attr_get(comm,Petsc_Tmp_keyval,(void**)&tagvalp,(int*)&iflg);CHKERRQ(ierr); >> ^ >> /home/yye00/Work/Saswata/petsc/src/sys/fileio/fretrieve.c:168:5: warning: >> ?MPI_Attr_put? is deprecated (declared at >> /usr/include/openmpi-x86_64/mpi.h:1203): MPI_Attr_put is superseded by >> MPI_Comm_set_attr in MPI-2.0 [-Wdeprecated-declarations] >> ierr = MPI_Attr_put(comm,Petsc_Tmp_keyval,tagvalp);CHKERRQ(ierr); >> ^ >> /home/yye00/Work/Saswata/petsc/src/sys/fileio/fretrieve.c: In function >> ?PetscSharedWorkingDirectory?: >> /home/yye00/Work/Saswata/petsc/src/sys/fileio/fretrieve.c:282:5: warning: >> ?MPI_Keyval_create? is deprecated (declared at >> /usr/include/openmpi-x86_64/mpi.h:1506): MPI_Keyval_create is superseded by >> MPI_Comm_create_keyval in MPI-2.0 [-Wdeprecated-declarations] >> ierr = >> MPI_Keyval_create(MPI_NULL_COPY_FN,Petsc_DelTmpShared,&Petsc_WD_keyval,0);CHKERRQ(ierr); >> ^ >> /home/yye00/Work/Saswata/petsc/src/sys/fileio/fretrieve.c:282:5: warning: >> ?OMPI_C_MPI_NULL_COPY_FN? is deprecated (declared at >> /usr/include/openmpi-x86_64/mpi.h:828): MPI_NULL_COPY_FN is deprecated in >> MPI-2.0 [-Wdeprecated-declarations] >> /home/yye00/Work/Saswata/petsc/src/sys/fileio/fretrieve.c:285:3: warning: >> ?MPI_Attr_get? is deprecated (declared at >> /usr/include/openmpi-x86_64/mpi.h:1201): MPI_Attr_get is superseded by >> MPI_Comm_get_attr in MPI-2.0 [-Wdeprecated-declarations] >> ierr = >> MPI_Attr_get(comm,Petsc_WD_keyval,(void**)&tagvalp,(int*)&iflg);CHKERRQ(ierr); >> ^ >> /home/yye00/Work/Saswata/petsc/src/sys/fileio/fretrieve.c:291:5: warning: >> ?MPI_Attr_put? is deprecated (declared at >> /usr/include/openmpi-x86_64/mpi.h:1203): MPI_Attr_put is superseded by >> MPI_Comm_set_attr in MPI-2.0 [-Wdeprecated-declarations] >> ierr = MPI_Attr_put(comm,Petsc_WD_keyval,tagvalp);CHKERRQ(ierr); >> >> >> >> Regards >> Yaakoub El Khamra >> >> >> On Mon, Dec 30, 2013 at 10:58 PM, Matthew Knepley wrote: >> >>> On Mon, Dec 30, 2013 at 7:34 PM, Yaakoub El Khamra < >>> yaakoub at tacc.utexas.edu> wrote: >>> >>>> >>>> If I create a dm with DMPlexCreateBoxMesh and immediately attempt to >>>> distribute it with DMPlexDistribute, the call to DMPlexDistribute never >>>> returns. I am working with a development checkout and the code looks as >>>> follows: >>>> >>>> call DMPlexCreateBoxMesh(PETSC_COMM_WORLD, 2, PETSC_TRUE, dm, >>>> $ ierr) >>>> CHKERRQ(ierr) >>>> >>>> call DMPlexDistribute(dm, "chaco", 0, >>>> $ PETSC_NULL_OBJECT, distributedMesh, ierr) >>>> CHKERRQ(ierr) >>>> >>>> Any thoughts? >>>> >>> >>> Can you run SNES ex12? It does exactly this and gets run in the nightly >>> tests. I can run it in parallel >>> with no problems. My guess is that something is going wrong with the >>> Fortran binding. Can you >>> get a stack trace for the hang? If not, can you send a full Fortran >>> program which exhibits your problem? >>> >>> Matt >>> >>> >>>> Regards >>>> Yaakoub El Khamra >>>> >>> >>> >>> >>> -- >>> What most experimenters take for granted before they begin their >>> experiments is infinitely more interesting than any results to which their >>> experiments lead. >>> -- Norbert Wiener >>> >> >> > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gideon.simpson at icloud.com Tue Dec 31 20:06:04 2013 From: gideon.simpson at icloud.com (Gideon Simpson) Date: Tue, 31 Dec 2013 21:06:04 -0500 Subject: [petsc-users] dmda and structures In-Reply-To: <3C348792-7CEA-4ED3-8F7D-6071C078D11D@mcs.anl.gov> References: <54F1C412-70BE-40D1-AB52-A7A11F142E1A@mcs.anl.gov> <3C348792-7CEA-4ED3-8F7D-6071C078D11D@mcs.anl.gov> Message-ID: <14A1B12D-BEE0-4737-89DC-369579433026@icloud.com> I believe I've gotten things to work in both cases, but I wanted to clarify what the substantial difference between DMDAVecGetArray and DMDAVecGetArrayDOF. Is it just that it allows me to do indexing of the form realization_array[i][j] instead of realization_array[i * num_dof +j], or is there something more here, in terms of how things are distributed? -gideon On Dec 29, 2013, at 2:53 PM, Barry Smith wrote: > > On Dec 29, 2013, at 1:38 PM, Gideon Simpson wrote: > >> How would that work? > > I?ve modified your code fragment below with how I think you could do it. > >> I took Jed's advice, and set it up so that I have access via: >> >> realization_array[ realization_dof * i + 1 +j] = jth point of ith realization >> >> and then I further used a pointer: >> >> PetscScalar *x; >> >> x = & realization_array[ realization_dof * i + 1]; >> >> so that I have "clean" access to the elements of each realization through x[j]. >> >> -gideon >> >> On Dec 29, 2013, at 2:33 PM, Barry Smith wrote: >> >>> >>> DMDAVecGetArrayDOF() might serve your needs. You would access with realization_array[i][j] for the ith grid point and j th realization >>> >>> Barry >>> >>> >>> >>> On Dec 28, 2013, at 7:44 PM, Gideon Simpson wrote: >>> >>>> I'm trying to use 1D DMDA's to manage distributed monte carlo computations, and I've got the following problem. If I know, in advance, how many floating points are needed for each realization, everything is fine. But, if I want to be able to set this at run time, I can't get it to work. What I have is: >>>> >>>> >>>> then, in the main part of the code: >>>> >>>> Realization *realization_array; >>>> DM sim_data_da; >>>> Vec sim_data_vec; >>>> >>>> PetscInt xsize=10, realization_dof, i; >>>> PetscOptionsGetInt(NULL, "-xsize", &xsize, NULL); >>>> >>>> realization_dof = xsize + 1; >>>> >>>> >>>> DMDACreate1d(PETSC_COMM_WORLD,DMDA_BOUNDARY_NONE, batch_size, realization_dof, 0, NULL, &sim_data_da); >>>> DMCreateGlobalVector(sim_data_da,&sim_data_vec); >>>> > PetscScalar **realization_array; > >>>> DMDAVecGetArrayDOF(sim_data_da, sim_data_vec, & realization_array); > > something = realization_array[i][j]; here i varies from xs to xs+xm-1 and j varies from 0 to realization_dof-1 > >>>> >>>> >>>> -gideon >>>> >>> >> > From jedbrown at mcs.anl.gov Tue Dec 31 20:41:53 2013 From: jedbrown at mcs.anl.gov (Jed Brown) Date: Tue, 31 Dec 2013 20:41:53 -0600 Subject: [petsc-users] dmda and structures In-Reply-To: <14A1B12D-BEE0-4737-89DC-369579433026@icloud.com> References: <54F1C412-70BE-40D1-AB52-A7A11F142E1A@mcs.anl.gov> <3C348792-7CEA-4ED3-8F7D-6071C078D11D@mcs.anl.gov> <14A1B12D-BEE0-4737-89DC-369579433026@icloud.com> Message-ID: <87sit8judq.fsf@jedbrown.org> Gideon Simpson writes: > I believe I've gotten things to work in both cases, but I wanted to > clarify what the substantial difference between DMDAVecGetArray and > DMDAVecGetArrayDOF. Is it just that it allows me to do indexing of > the form realization_array[i][j] instead of realization_array[i * > num_dof +j], That's all. It does that by using an extra set of pointers. > in terms of how things are distributed? No difference in distribution. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: