From tyoung at ippt.gov.pl Fri Jan 1 15:41:04 2010 From: tyoung at ippt.gov.pl (Toby D. Young) Date: Fri, 1 Jan 2010 22:41:04 +0100 (CET) Subject: [petsc-users] Unexpected error in compile In-Reply-To: References: <3124.124.16.137.114.1262056106.squirrel@webmail.iscas.ac.cn> Message-ID: > It looks like your MPI installation was not built with Fortran bindings. Was > it? Oops, no it wasn't. I made a couple of other related errors too, but PETSc configure system walked me through those.... All is good now. Thanks Matt. Best, Toby ----- Toby D. Young Assistant Professor Philosophy-Physics Polish Academy of Sciences Warszawa, Polska www: http://www.ippt.gov.pl/~tyoung skype: stenografia From torres.pedrozpk at gmail.com Sun Jan 3 10:29:17 2010 From: torres.pedrozpk at gmail.com (Pedro Torres) Date: Sun, 3 Jan 2010 13:29:17 -0300 Subject: [petsc-users] Question about MatConvert Message-ID: Hi, Can I use MatConvert for rectangular matrix?. Thanks in advance! Pedro -- Jonathan Swift - "May you live every day of your life." -------------- next part -------------- An HTML attachment was scrubbed... URL: From ventejuy at yahoo.es Sun Jan 3 15:19:17 2010 From: ventejuy at yahoo.es (Bentejui Medina) Date: Sun, 3 Jan 2010 21:19:17 +0000 (GMT) Subject: [petsc-users] ADDing a row to another one In-Reply-To: <87r5qb2nh8.fsf@59A2.org> References: <114247.16877.qm@web27003.mail.ukl.yahoo.com> <87r5qb2nh8.fsf@59A2.org> Message-ID: <661212.84547.qm@web27002.mail.ukl.yahoo.com> Thanks. Really faster now. ----- Mensaje original ---- De: Jed Brown Para: Sr. Yo ; PETSc users list ; petsc-users at mcs.anl.gov Enviado: jue,31 diciembre, 2009 15:05 Asunto: Re: [petsc-users] ADDing a row to another one On Thu, 31 Dec 2009 13:05:37 +0000 (GMT), "Sr. Yo" wrote: > Hi all, > I'm trying to ADD(sum) a row into another on sparce matrix. I was looking for a function on PETSC documentation but no luck, so now a get the entire row to a PetscScalar and them ADD it to another row. It works but is slow, not elegant and no good for a sparce matrix. So, is there a smart way to do this?. MatGetRow(A,source_row,&ncols,&cols,&vals); MatSetValues(A,1,&dest_row,ncols,cols,vals,ADD_VALUES); MatRestoreRow(A,source_row,&ncols,&cols,&vals); http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/Mat/MatGetRow.html Jed From bsmith at mcs.anl.gov Sun Jan 3 15:43:38 2010 From: bsmith at mcs.anl.gov (Barry Smith) Date: Sun, 3 Jan 2010 15:43:38 -0600 Subject: [petsc-users] Question about MatConvert In-Reply-To: References: Message-ID: In general MatConvert() should work for rectangular matrices but likely it is only tested for square matrices, please report specific problems you have (include code fragments or entire codes that demonstrate the problem) to petsc-maint at mcs.anl.gov and we'll get them working for you. Barry On Jan 3, 2010, at 10:29 AM, Pedro Torres wrote: > Hi, > > Can I use MatConvert for rectangular matrix?. > > Thanks in advance! > > Pedro > > > > > -- > > Jonathan Swift - "May you live every day of your life." From dave.mayhem23 at gmail.com Mon Jan 4 02:17:11 2010 From: dave.mayhem23 at gmail.com (Dave May) Date: Mon, 4 Jan 2010 09:17:11 +0100 Subject: [petsc-users] A question In-Reply-To: <944577.92523.qm@web52202.mail.re2.yahoo.com> References: <4B3878D5.10909@kit.edu> <944577.92523.qm@web52202.mail.re2.yahoo.com> Message-ID: <956373f1001040017u3693eb0cy419bc1db00da24f3@mail.gmail.com> Hey Farshid, Maybe you can check out Tim Davis's SuiteSparse package, http://www.cise.ufl.edu/research/sparse/SuiteSparse which includes both UMFPACK and CHOLMOD. For comparison papers, there is this one which compares the following solvers BCSLIB-EXT, CHOLMOD, MA57, MUMPS, Oblio, PARDISO, SPOOLES, SPRSBLKLLT, TAUCS, UMFPACK, and WSMP on 87 large symmetric positive definite matrices. @article{1236465, author = {Gould, Nicholas I. M. and Scott, Jennifer A. and Hu, Yifan}, title = {A numerical evaluation of sparse direct solvers for the solution of large sparse symmetric linear systems of equations}, journal = {ACM Trans. Math. Softw.}, volume = {33}, number = {2}, year = {2007}, issn = {0098-3500}, pages = {10}, doi = {http://doi.acm.org/10.1145/1236463.1236465}, publisher = {ACM}, address = {New York, NY, USA}, } and this one which compares MUMPS and SUPERLU @ARTICLE{Amestoy00analysisand, author = {Patrick R. Amestoy and Iain S. Duff and Jean-Yves L'Excellent and Xiaoye S. Li}, title = {Analysis and Comparison of Two General Sparse Solvers for Distributed Memory Computers}, journal = {ACM Transactions on Mathematical Software}, year = {2000}, volume = {27}, pages = {2001} } Cheers, Dave 2009/12/29 Farshid Mossaiby > > Thanks all for the replies and happy new year! > > I am also trying SuperLU MT for now. Anybody checked IBM Watson Sparse Matrix Package (not open source)? > > http://www.alphaworks.ibm.com/tech/wsmp/ > > Any hint on comparison between different solvers, in terms of performance and memory usage for FEM problems? > > Best regards, > Farshid Mossaiby > > --- On Mon, 12/28/09, Markus B?rg wrote: > > > From: Markus B?rg > > Subject: Re: [petsc-users] A question > > To: "PETSc users list" > > Date: Monday, December 28, 2009, 12:52 PM > > Hello Farshid, > > > > what about MUMPS, PARDISO or UMFPACK? > > > > Best Regards, > > Markus > > > > > > > > Farshid Mossaiby schrieb: > > > Hi all, > > > > > > Would somebody please suggest a *free* direct solver > > for a linux x64 SMP machine? Would be much better, if open > > source. > > > > > > Anybody has had success in compiling TAUCS with cilk > > support on linux x64? > > > > > > Best regards, > > > Farshid Mossaiby > > > > > > > > > > > > > > > > > > > > From mossaiby at yahoo.com Mon Jan 4 07:59:34 2010 From: mossaiby at yahoo.com (Farshid Mossaiby) Date: Mon, 4 Jan 2010 05:59:34 -0800 (PST) Subject: [petsc-users] A question In-Reply-To: <956373f1001040017u3693eb0cy419bc1db00da24f3@mail.gmail.com> Message-ID: <952715.73757.qm@web52204.mail.re2.yahoo.com> Dear Dave, Thanks for suggestion and paper. I am investigating the packages. Best regards, Farshid Mossaiby --- On Mon, 1/4/10, Dave May wrote: > From: Dave May > Subject: Re: [petsc-users] A question > To: "PETSc users list" > Date: Monday, January 4, 2010, 11:47 AM > Hey Farshid, > > Maybe you can check out Tim Davis's SuiteSparse package, > ? http://www.cise.ufl.edu/research/sparse/SuiteSparse > which includes both UMFPACK and CHOLMOD. > > For comparison papers, there is this one which compares the > following solvers > BCSLIB-EXT, CHOLMOD, MA57, MUMPS, Oblio, PARDISO, SPOOLES, > SPRSBLKLLT, > TAUCS, UMFPACK, and WSMP on 87 large symmetric positive > definite > matrices. > > @article{1236465, > author = {Gould, Nicholas I. M. and Scott, Jennifer A. and > Hu, Yifan}, > title = {A numerical evaluation of sparse direct solvers > for the > solution of large sparse symmetric linear systems of > equations}, > journal = {ACM Trans. Math. Softw.}, > volume = {33}, > number = {2}, > year = {2007}, > issn = {0098-3500}, > pages = {10}, > doi = {http://doi.acm.org/10.1145/1236463.1236465}, > publisher = {ACM}, > address = {New York, NY, USA}, > } > > and this one which compares MUMPS and SUPERLU > > @ARTICLE{Amestoy00analysisand, > ? ? author = {Patrick R. Amestoy and Iain S. Duff > and Jean-Yves > L'Excellent and Xiaoye S. Li}, > ? ? title = {Analysis and Comparison of Two > General Sparse Solvers for > Distributed Memory Computers}, > ? ? journal = {ACM Transactions on Mathematical > Software}, > ? ? year = {2000}, > ? ? volume = {27}, > ? ? pages = {2001} > } > > > Cheers, > ? Dave > > > > 2009/12/29 Farshid Mossaiby > > > > Thanks all for the replies and happy new year! > > > > I am also trying SuperLU MT for now. Anybody checked > IBM Watson Sparse Matrix Package (not open source)? > > > > http://www.alphaworks.ibm.com/tech/wsmp/ > > > > Any hint on comparison between different solvers, in > terms of performance and memory usage for FEM problems? > > > > Best regards, > > Farshid Mossaiby > > > > --- On Mon, 12/28/09, Markus B?rg > wrote: > > > > > From: Markus B?rg > > > Subject: Re: [petsc-users] A question > > > To: "PETSc users list" > > > Date: Monday, December 28, 2009, 12:52 PM > > > Hello Farshid, > > > > > > what about MUMPS, PARDISO or UMFPACK? > > > > > > Best Regards, > > > Markus > > > > > > > > > > > > Farshid Mossaiby schrieb: > > > > Hi all, > > > > > > > > Would somebody please suggest a *free* > direct solver > > > for a linux x64 SMP machine? Would be much > better, if open > > > source. > > > > > > > > Anybody has had success in compiling TAUCS > with cilk > > > support on linux x64? > > > > > > > > Best regards, > > > > Farshid Mossaiby > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > From hxie at umn.edu Mon Jan 4 13:25:58 2010 From: hxie at umn.edu (hxie at umn.edu) Date: 04 Jan 2010 13:25:58 -0600 Subject: [petsc-users] Any examples to use the external package? In-Reply-To: References: Message-ID: Hi, Where can I find the examples (command line options) to use the external packages like hypre, mumps, ml? Thanks. Bests, Hui From bsmith at mcs.anl.gov Mon Jan 4 13:28:11 2010 From: bsmith at mcs.anl.gov (Barry Smith) Date: Mon, 4 Jan 2010 13:28:11 -0600 Subject: [petsc-users] Any examples to use the external package? In-Reply-To: References: Message-ID: <89133C79-FE15-4A60-AC47-1EC0B06080CB@mcs.anl.gov> src/ksp/ksp/examples/tutorials/makefile On Jan 4, 2010, at 1:25 PM, hxie at umn.edu wrote: > Hi, > > Where can I find the examples (command line options) to use the > external packages like hypre, mumps, ml? Thanks. > > Bests, > Hui From hxie at umn.edu Mon Jan 4 15:07:56 2010 From: hxie at umn.edu (hxie at umn.edu) Date: 04 Jan 2010 15:07:56 -0600 Subject: [petsc-users] How to install the external packages? In-Reply-To: References: Message-ID: Hi, No problems with the configuration for hypre: ./config/configure.py --COPTFLAGS=-O3 --FOPTFLAGS=-O3 --with-debugging=0 --with-mpi-dir=/usr/local/mpich2-1.2.1 --with-fortran --with-shared=0 --PETSC_ARCH=mac-gnu-f-all --with-hypre=1 --download-hypre=1 But I met some problems when installing the superlu_dist: ./config/configure.py --COPTFLAGS=-O3 --FOPTFLAGS=-O3 --with-debugging=0 --with-mpi-dir=/usr/local/mpich2-1.2.1 --with-fortran --with-shared=0 --PETSC_ARCH=mac-gnu-f-all --with-superlu_dist=1 --download-superlu_dist=1 --download-parmetis=1 ----- error messages: /usr/bin/ranlib: file: /usr/local/petsc-3.0.0-p10/mac-gnu-f-all/lib/libml.a(MLAPI_LinearCombinations.o) has no symbols /usr/bin/ranlib: file: /usr/local/petsc-3.0.0-p10/mac-gnu-f-all/lib/libsmumps.a(smumps_struc_def.o) has no symbols ----- Are some options wrong? Bests, Hui From bsmith at mcs.anl.gov Mon Jan 4 15:09:59 2010 From: bsmith at mcs.anl.gov (Barry Smith) Date: Mon, 4 Jan 2010 15:09:59 -0600 Subject: [petsc-users] How to install the external packages? In-Reply-To: References: Message-ID: <7EF71CDC-D5D0-4940-A90C-5439BC6B382B@mcs.anl.gov> Those are not necessarily error messages. Does make test work? Barry On Jan 4, 2010, at 3:07 PM, hxie at umn.edu wrote: > Hi, > > No problems with the configuration for hypre: > ./config/configure.py --COPTFLAGS=-O3 --FOPTFLAGS=-O3 --with- > debugging=0 --with-mpi-dir=/usr/local/mpich2-1.2.1 --with-fortran -- > with-shared=0 --PETSC_ARCH=mac-gnu-f-all --with-hypre=1 --download- > hypre=1 > > But I met some problems when installing the superlu_dist: > ./config/configure.py --COPTFLAGS=-O3 --FOPTFLAGS=-O3 --with- > debugging=0 --with-mpi-dir=/usr/local/mpich2-1.2.1 --with-fortran -- > with-shared=0 --PETSC_ARCH=mac-gnu-f-all --with-superlu_dist=1 -- > download-superlu_dist=1 --download-parmetis=1 > ----- > error messages: > /usr/bin/ranlib: file: /usr/local/petsc-3.0.0-p10/mac-gnu-f-all/lib/ > libml.a(MLAPI_LinearCombinations.o) has no symbols > /usr/bin/ranlib: file: /usr/local/petsc-3.0.0-p10/mac-gnu-f-all/lib/ > libsmumps.a(smumps_struc_def.o) has no symbols > ----- > > Are some options wrong? > > Bests, > Hui From hxie at umn.edu Mon Jan 4 15:22:38 2010 From: hxie at umn.edu (hxie at umn.edu) Date: 04 Jan 2010 15:22:38 -0600 Subject: [petsc-users] How to install the external packages? In-Reply-To: References: Message-ID: 'make test' runs ok. But I cannot compile my own code. The problems I guess are the conflict of libmetis and libparmetis. The error messages while compiling my own code: Undefined symbols: "_METIS_mCPartGraphRecursive2", referenced from: _ParMETIS_V3_PartKway in libparmetis.a(kmetis.o) _PartitionSmallGraph__ in libparmetis.a(weird.o) _Moc_InitPartition_RB__ in libparmetis.a(initpart.o) _Moc_InitPartition_RB__ in libparmetis.a(initpart.o) _Balance_Partition__ in libparmetis.a(initbalance.o) _Balance_Partition__ in libparmetis.a(initbalance.o) ld: symbol(s) not found BTW, which example can I use to test the command line option "-mat_type superlu_dist -pc_type lu -ksp_type preonly"? Bests, Hui From bsmith at mcs.anl.gov Mon Jan 4 15:43:32 2010 From: bsmith at mcs.anl.gov (Barry Smith) Date: Mon, 4 Jan 2010 15:43:32 -0600 Subject: [petsc-users] How to install the external packages? In-Reply-To: References: Message-ID: <417B41C7-7154-46B6-853A-8D3505189688@mcs.anl.gov> Likely you do not have the correct link libraries for your own code. Are you using the PETSc makefile rule to link your code? If not you will have to change your makefile whenever you add new external packages. Does make getlinklibs in the PETSc directory list the libraries in the same order that you list them in your makefile? Barry On Jan 4, 2010, at 3:22 PM, hxie at umn.edu wrote: > > 'make test' runs ok. But I cannot compile my own code. The problems > I guess are the conflict of libmetis and libparmetis. The error > messages while compiling my own code: > > Undefined symbols: > "_METIS_mCPartGraphRecursive2", referenced from: > _ParMETIS_V3_PartKway in libparmetis.a(kmetis.o) > _PartitionSmallGraph__ in libparmetis.a(weird.o) > _Moc_InitPartition_RB__ in libparmetis.a(initpart.o) > _Moc_InitPartition_RB__ in libparmetis.a(initpart.o) > _Balance_Partition__ in libparmetis.a(initbalance.o) > _Balance_Partition__ in libparmetis.a(initbalance.o) > ld: symbol(s) not found > > BTW, which example can I use to test the command line option "- > mat_type superlu_dist -pc_type lu -ksp_type preonly"? > > Bests, > Hui From hxie at umn.edu Mon Jan 4 16:36:20 2010 From: hxie at umn.edu (hxie at umn.edu) Date: 04 Jan 2010 16:36:20 -0600 Subject: [petsc-users] How to install the external packages? In-Reply-To: References: Message-ID: I have fixed the link problem. Thanks. Bests, Hui From hxie at umn.edu Tue Jan 5 10:45:01 2010 From: hxie at umn.edu (hxie at umn.edu) Date: 05 Jan 2010 10:45:01 -0600 Subject: [petsc-users] command line option for superlu_dist In-Reply-To: References: Message-ID: Hi, My code works when I use mumps option like -ksp_type preonly -pc_type lu -pc_factor_mat_solver_package mumps . But it does not work with -ksp_type preonly -pc_type lu -pc_factor_mat_solver_package superlu_dist It runs but cannot get anything. Are the options correct? Or the superlu_dist solver does not work with my matrix. Bests, Hui From knepley at gmail.com Tue Jan 5 10:48:59 2010 From: knepley at gmail.com (Matthew Knepley) Date: Tue, 5 Jan 2010 10:48:59 -0600 Subject: [petsc-users] command line option for superlu_dist In-Reply-To: References: Message-ID: On Tue, Jan 5, 2010 at 10:45 AM, wrote: > Hi, > > My code works when I use mumps option like > -ksp_type preonly -pc_type lu -pc_factor_mat_solver_package mumps . > > But it does not work with -ksp_type preonly -pc_type lu > -pc_factor_mat_solver_package superlu_dist > It runs but cannot get anything. Are the options correct? Or the > superlu_dist solver does not work with my matrix. > Without any output, we have no idea what "cannot get anything" means. Run with -ksp_monitor -ksp_view -ksp_converged_reason and send the output. Matt > Bests, > Hui > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From hxie at umn.edu Tue Jan 5 12:01:21 2010 From: hxie at umn.edu (hxie at umn.edu) Date: 05 Jan 2010 12:01:21 -0600 Subject: [petsc-users] command line option for superlu_dist In-Reply-To: References: Message-ID: The information for superlu_dist Linear solve converged due to CONVERGED_ITS iterations 1 KSP Object: type: preonly maximum iterations=1000, initial guess is zero tolerances: relative=1e-06, absolute=1e-10, divergence=100 left preconditioning PC Object: type: lu LU: out-of-place factorization matrix ordering: natural LU: tolerance for zero pivot 1e-12 LU: factor fill ratio needed 0 Factored matrix follows Matrix Object: type=mpiaij, rows=291896, cols=291896 package used to perform factorization: superlu_dist total: nonzeros=0, allocated nonzeros=583792 SuperLU_DIST run parameters: Process grid nprow 2 x npcol 2 Equilibrate matrix TRUE Matrix input mode 1 Replace tiny pivots TRUE Use iterative refinement FALSE Processors in row 2 col partition 2 Row permutation LargeDiag Column permutation MMD_AT_PLUS_A Parallel symbolic factorization FALSE Repeated factorization SamePattern linear system matrix followed by preconditioner matrix: Matrix Object: type=mpiaij, rows=291896, cols=291896 total: nonzeros=33670280, allocated nonzeros=36641300 using I-node (on process 0) routines: found 21607 nodes, limit used is 5 Matrix Object: type=mpiaij, rows=291896, cols=291896 total: nonzeros=33670280, allocated nonzeros=36641300 using I-node (on process 0) routines: found 21607 nodes, limit used is 5 WARNING: Tolerance not met by convergence norms ----------------------------------------------------------------------------- Solution norm = NaN Total CPU time = 39.0 min For mumps, it takes only 2min to get some result "Solution norm = 0.255E+03". From knepley at gmail.com Tue Jan 5 12:13:45 2010 From: knepley at gmail.com (Matthew Knepley) Date: Tue, 5 Jan 2010 12:13:45 -0600 Subject: [petsc-users] command line option for superlu_dist In-Reply-To: References: Message-ID: 1) The solution norm you report is NaN, which indicates memory corruption in your code 2) The number of allocated nonzeros is really suspicious. There are only 2 nonzeros per row? Matt On Tue, Jan 5, 2010 at 12:01 PM, wrote: > > The information for superlu_dist > Linear solve converged due to CONVERGED_ITS iterations 1 > KSP Object: > type: preonly > maximum iterations=1000, initial guess is zero > tolerances: relative=1e-06, absolute=1e-10, divergence=100 > left preconditioning > PC Object: > type: lu > LU: out-of-place factorization > matrix ordering: natural > LU: tolerance for zero pivot 1e-12 > LU: factor fill ratio needed 0 > Factored matrix follows > Matrix Object: > type=mpiaij, rows=291896, cols=291896 > package used to perform factorization: superlu_dist > total: nonzeros=0, allocated nonzeros=583792 > SuperLU_DIST run parameters: > Process grid nprow 2 x npcol 2 Equilibrate matrix > TRUE Matrix input mode 1 Replace tiny pivots TRUE > Use iterative refinement FALSE Processors in row 2 col > partition 2 Row permutation LargeDiag Column > permutation MMD_AT_PLUS_A > Parallel symbolic factorization FALSE Repeated > factorization SamePattern > linear system matrix followed by preconditioner matrix: > Matrix Object: > type=mpiaij, rows=291896, cols=291896 > total: nonzeros=33670280, allocated nonzeros=36641300 > using I-node (on process 0) routines: found 21607 nodes, limit used is > 5 > Matrix Object: > type=mpiaij, rows=291896, cols=291896 > total: nonzeros=33670280, allocated nonzeros=36641300 > using I-node (on process 0) routines: found 21607 nodes, limit used is > 5 > > WARNING: Tolerance not met by convergence norms > > ----------------------------------------------------------------------------- > Solution norm = NaN > Total CPU time = 39.0 min > > For mumps, it takes only 2min to get some result "Solution norm = > 0.255E+03". > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Tue Jan 5 12:51:27 2010 From: bsmith at mcs.anl.gov (Barry Smith) Date: Tue, 5 Jan 2010 12:51:27 -0600 Subject: [petsc-users] command line option for superlu_dist In-Reply-To: References: Message-ID: <749E2563-3952-47E9-BBA5-879E52ADA2B9@mcs.anl.gov> Likely SuperLU_Dist is failing on the provided matrix. Could you use -ksp_view_binary on the run and copy the resulting file binaryoutput to the anonymous ftp site ftp.mcs.anl.gov in the directory incoming and tell us when the matrix is there. Sparse parallel factorizations are a bit fragile and can fail. Barry On Jan 5, 2010, at 12:01 PM, hxie at umn.edu wrote: > > The information for superlu_dist > Linear solve converged due to CONVERGED_ITS iterations 1 > KSP Object: > type: preonly > maximum iterations=1000, initial guess is zero > tolerances: relative=1e-06, absolute=1e-10, divergence=100 > left preconditioning > PC Object: > type: lu > LU: out-of-place factorization > matrix ordering: natural > LU: tolerance for zero pivot 1e-12 > LU: factor fill ratio needed 0 > Factored matrix follows > Matrix Object: > type=mpiaij, rows=291896, cols=291896 > package used to perform factorization: superlu_dist > total: nonzeros=0, allocated nonzeros=583792 > SuperLU_DIST run parameters: > Process grid nprow 2 x npcol 2 Equilibrate > matrix TRUE Matrix input mode 1 Replace > tiny pivots TRUE Use iterative refinement > FALSE Processors in row 2 col partition 2 > Row permutation LargeDiag Column permutation > MMD_AT_PLUS_A > Parallel symbolic factorization FALSE > Repeated factorization SamePattern > linear system matrix followed by preconditioner matrix: > Matrix Object: > type=mpiaij, rows=291896, cols=291896 > total: nonzeros=33670280, allocated nonzeros=36641300 > using I-node (on process 0) routines: found 21607 nodes, limit > used is 5 > Matrix Object: > type=mpiaij, rows=291896, cols=291896 > total: nonzeros=33670280, allocated nonzeros=36641300 > using I-node (on process 0) routines: found 21607 nodes, limit > used is 5 > > WARNING: Tolerance not met by convergence norms > ----------------------------------------------------------------------------- > Solution norm = NaN > Total CPU time = 39.0 min > > For mumps, it takes only 2min to get some result "Solution norm = > 0.255E+03". > > From hxie at umn.edu Tue Jan 5 13:00:23 2010 From: hxie at umn.edu (hxie at umn.edu) Date: 05 Jan 2010 13:00:23 -0600 Subject: [petsc-users] command line option for superlu_dist In-Reply-To: References: Message-ID: > 1) The solution norm you report is NaN, which indicates memory corruption > in your code I guess the reason may be a division by zero. The diagonals of the matrix have some zeros. >2) The number of allocated nonzeros is really suspicious. There are only 2 nonzeros per row? At least 5. Hui From bsmith at mcs.anl.gov Tue Jan 5 13:03:51 2010 From: bsmith at mcs.anl.gov (Barry Smith) Date: Tue, 5 Jan 2010 13:03:51 -0600 Subject: [petsc-users] command line option for superlu_dist In-Reply-To: References: Message-ID: <3F037CE4-18F6-4E2E-8C6E-3CDB2A458243@mcs.anl.gov> On Jan 5, 2010, at 1:00 PM, hxie at umn.edu wrote: > > >> 1) The solution norm you report is NaN, which indicates memory >> corruption in > your code > > I guess the reason may be a division by zero. The diagonals of the > matrix have some zeros. SuperLU_Dist should take this into account, but ... Send us the matrix as I requested via the ftp site. > > >> 2) The number of allocated nonzeros is really suspicious. There are >> only 2 > nonzeros per row? > > At least 5. Don't worry about this. This is an error in the way we print information about the "factored" matrix. > > Hui > From hxie at umn.edu Tue Jan 5 13:54:20 2010 From: hxie at umn.edu (hxie at umn.edu) Date: 05 Jan 2010 13:54:20 -0600 Subject: [petsc-users] command line option for superlu_dist In-Reply-To: References: Message-ID: I use ftp to upload the file matrix.gz to incoming. But I cannot see it using 'ls'. Hui ------------- Likely SuperLU_Dist is failing on the provided matrix. Could you use -ksp_view_binary on the run and copy the resulting file binaryoutput to the anonymous ftp site ftp.mcs.anl.gov in the directory incoming and tell us when the matrix is there. Sparse parallel factorizations are a bit fragile and can fail. Barry From bsmith at mcs.anl.gov Tue Jan 5 13:59:41 2010 From: bsmith at mcs.anl.gov (Barry Smith) Date: Tue, 5 Jan 2010 13:59:41 -0600 Subject: [petsc-users] command line option for superlu_dist In-Reply-To: References: Message-ID: <0FA592BE-D910-4176-A47B-FED2FE0F4B0E@mcs.anl.gov> Ok, we got the file. For "security reasons" ls is disabled in that directory. Barry On Jan 5, 2010, at 1:54 PM, hxie at umn.edu wrote: > > I use ftp to upload the file matrix.gz to incoming. But I cannot see > it using 'ls'. > > Hui > > ------------- > Likely SuperLU_Dist is failing on the provided matrix. Could you > use -ksp_view_binary on the run and copy the resulting file > binaryoutput to the anonymous ftp site ftp.mcs.anl.gov in the > directory incoming and tell us when the matrix is there. Sparse > parallel factorizations are a bit fragile and can fail. > > > Barry From hzhang at mcs.anl.gov Tue Jan 5 14:10:42 2010 From: hzhang at mcs.anl.gov (Hong Zhang) Date: Tue, 5 Jan 2010 14:10:42 -0600 Subject: [petsc-users] command line option for superlu_dist In-Reply-To: <0FA592BE-D910-4176-A47B-FED2FE0F4B0E@mcs.anl.gov> References: <0FA592BE-D910-4176-A47B-FED2FE0F4B0E@mcs.anl.gov> Message-ID: <9B198DB7-4F04-479D-A25B-07C813AFE713@mcs.anl.gov> I got the matrix and start testing it: - sequential mumps fails during numerical factorization [0]PETSC ERROR: [0] MatFactorNumeric_MUMPS line 314 src/mat/impls/aij/ mpi/mumps/mumps.c [0]PETSC ERROR: [0] MatLUFactorNumeric line 2378 src/mat/interface/ matrix.c - sequential superlu seems hang ... Something is wrong with your matrix. I'll further check it and let you know what I get. How did you get mumps work? Sequential or parallel (num of processors?)? Hong On Jan 5, 2010, at 1:59 PM, Barry Smith wrote: > > Ok, we got the file. For "security reasons" ls is disabled in that > directory. > > Barry > > On Jan 5, 2010, at 1:54 PM, hxie at umn.edu wrote: > >> >> I use ftp to upload the file matrix.gz to incoming. But I cannot >> see it using 'ls'. >> >> Hui >> >> ------------- >> Likely SuperLU_Dist is failing on the provided matrix. Could you >> use -ksp_view_binary on the run and copy the resulting file >> binaryoutput to the anonymous ftp site ftp.mcs.anl.gov in the >> directory incoming and tell us when the matrix is there. Sparse >> parallel factorizations are a bit fragile and can fail. >> >> >> Barry > From hxie at umn.edu Tue Jan 5 14:20:23 2010 From: hxie at umn.edu (hxie at umn.edu) Date: 05 Jan 2010 14:20:23 -0600 Subject: [petsc-users] command line option for superlu_dist In-Reply-To: References: Message-ID: I run with the following command: mpirun -np 4 ~/bin/cats3d.1 -ksp_type preonly -pc_type lu -pc_factor_mat_solver_package mumps From hxie at umn.edu Wed Jan 6 15:07:18 2010 From: hxie at umn.edu (hxie at umn.edu) Date: 06 Jan 2010 15:07:18 -0600 Subject: [petsc-users] command line option for external packages In-Reply-To: References: Message-ID: Hi, If I want to use hypre, should the runtime option be "-pc_type hypre -pc_type_hypre boomeramg -ksp_type gmres"? And I do not know how to use the ml package. Any example for this? Thanks. Bests, Hui From bsmith at mcs.anl.gov Wed Jan 6 15:13:27 2010 From: bsmith at mcs.anl.gov (Barry Smith) Date: Wed, 6 Jan 2010 15:13:27 -0600 Subject: [petsc-users] command line option for external packages In-Reply-To: References: Message-ID: On Jan 6, 2010, at 3:07 PM, hxie at umn.edu wrote: > Hi, > > If I want to use hypre, should the runtime option be > "-pc_type hypre -pc_type_hypre boomeramg -ksp_type gmres"? yes > And I do not > know how to use the ml package. Any example for this? Thanks. -pc_type ml BTW: if you run with -help it prints all the possible options > > Bests, > Hui From jed at 59A2.org Thu Jan 7 02:39:19 2010 From: jed at 59A2.org (Jed Brown) Date: Thu, 07 Jan 2010 02:39:19 -0600 Subject: [petsc-users] command line option for external packages In-Reply-To: References: Message-ID: <87vdfe1f7s.fsf@59A2.org> On 06 Jan 2010 15:07:18 -0600, hxie at umn.edu wrote: > Hi, > > If I want to use hypre, should the runtime option be > "-pc_type hypre -pc_type_hypre boomeramg -ksp_type gmres"? And I do not ^^^^^^^^^^^^^ should be -pc_hypre_type boomeramg (BoomerAMG is the default for Hypre from PETSc-3.0.0 onward). You can always run with -pc_type hypre -help |grep pc_hypre to find all Hypre-related options (similarly with ML and other PCs, KSPs, etc). Jed From u.tabak at tudelft.nl Thu Jan 7 15:35:23 2010 From: u.tabak at tudelft.nl (Umut Tabak) Date: Thu, 07 Jan 2010 22:35:23 +0100 Subject: [petsc-users] change the column order of a dense matrix Message-ID: <4B46539B.5070801@tudelft.nl> Is that possible to change the column order of a dense matrix with a built-in or combination of built-in function(s)? Best, Umut From bsmith at mcs.anl.gov Thu Jan 7 16:20:41 2010 From: bsmith at mcs.anl.gov (Barry Smith) Date: Thu, 7 Jan 2010 16:20:41 -0600 Subject: [petsc-users] change the column order of a dense matrix In-Reply-To: <4B46539B.5070801@tudelft.nl> References: <4B46539B.5070801@tudelft.nl> Message-ID: <1EA14256-C8E3-4CFE-B0B0-DF5A6F96D547@mcs.anl.gov> I assume you mean by this you want swap columns of numerical values? You can use MatGetArray() to access the values and do it with a regular loop. Barry On Jan 7, 2010, at 3:35 PM, Umut Tabak wrote: > Is that possible to change the column order of a dense matrix with a > built-in or combination of built-in function(s)? > Best, > Umut From vyan2000 at gmail.com Sun Jan 10 14:55:52 2010 From: vyan2000 at gmail.com (Ryan Yan) Date: Sun, 10 Jan 2010 15:55:52 -0500 Subject: [petsc-users] Snes behavior Message-ID: Hi All, I am solving a nonlinear system using snes. The -snes_monitor option has the following output: 0 SNES Function norm 2.640163923729e+09 1 SNES Function norm 1.047643565314e+08 2 SNES Function norm 1.712732074788e+06 3 SNES Function norm 1.002169173269e+04 4 SNES Function norm 1.655878303433e+03 5 SNES Function norm 3.746498305706e+02 6 SNES Function norm 8.317435704773e+01 7 SNES Function norm 1.857639969641e+01 8 SNES Function norm 4.149691057773e+00 9 SNES Function norm 9.265604042412e-01 10 SNES Function norm 2.069527103214e-01 11 SNES Function norm 4.624186491082e-02 12 SNES Function norm 1.035558432688e-02 13 SNES Function norm 2.341362958811e-03 14 SNES Function norm 5.507445427277e-04 15 SNES Function norm 1.485123568354e-04 16 SNES Function norm 5.180043781814e-05 17 SNES Function norm 2.341966514486e-05 18 SNES Function norm 1.344936158651e-05 19 SNES Function norm 1.054812641176e-05 Number of Newton iterations = 19 Converged reason is 4 It looks like the iterate never falls into a quadratic convergence region before it converges. Is there any hint to understand this behavior? Thanks a lot, Yan -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Sun Jan 10 14:59:07 2010 From: knepley at gmail.com (Matthew Knepley) Date: Sun, 10 Jan 2010 14:59:07 -0600 Subject: [petsc-users] Snes behavior In-Reply-To: References: Message-ID: It is possible for the radius of quadratic convergence to be very small. However, I would check your Jacobian, and maybe try -snes_mf. Matt On Sun, Jan 10, 2010 at 2:55 PM, Ryan Yan wrote: > Hi All, > I am solving a nonlinear system using snes. The -snes_monitor option has > the following output: > > 0 SNES Function norm 2.640163923729e+09 > 1 SNES Function norm 1.047643565314e+08 > 2 SNES Function norm 1.712732074788e+06 > 3 SNES Function norm 1.002169173269e+04 > 4 SNES Function norm 1.655878303433e+03 > 5 SNES Function norm 3.746498305706e+02 > 6 SNES Function norm 8.317435704773e+01 > 7 SNES Function norm 1.857639969641e+01 > 8 SNES Function norm 4.149691057773e+00 > 9 SNES Function norm 9.265604042412e-01 > 10 SNES Function norm 2.069527103214e-01 > 11 SNES Function norm 4.624186491082e-02 > 12 SNES Function norm 1.035558432688e-02 > 13 SNES Function norm 2.341362958811e-03 > 14 SNES Function norm 5.507445427277e-04 > 15 SNES Function norm 1.485123568354e-04 > 16 SNES Function norm 5.180043781814e-05 > 17 SNES Function norm 2.341966514486e-05 > 18 SNES Function norm 1.344936158651e-05 > 19 SNES Function norm 1.054812641176e-05 > Number of Newton iterations = 19 > Converged reason is 4 > > It looks like the iterate never falls into a quadratic convergence region > before it converges. Is there any hint to understand this behavior? > > Thanks a lot, > > Yan > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From vyan2000 at gmail.com Sun Jan 10 15:28:05 2010 From: vyan2000 at gmail.com (Ryan Yan) Date: Sun, 10 Jan 2010 16:28:05 -0500 Subject: [petsc-users] Snes behavior In-Reply-To: References: Message-ID: Hi Matt, Thank you very much for the suggestion. I was using DMMGSetSNESLocal(dmmg,FormFunctionLocal,0,ad_FormFunctionLocal,admf_FormFunctionLocal), so the Jacobian is calculated by automatic differentiation, right? For this instance, is there any way to check the correctness of the set up of the residual? After I tried the -snes_mf the linear solver failed( ): 0 SNES Function norm 1.578681107621e+08 1 SNES Function norm 1.343502549866e+08 2 SNES Function norm 1.211729760183e+08 3 SNES Function norm 1.211728837635e+08 4 SNES Function norm 1.211728837178e+08 5 SNES Function norm 1.211728837177e+08 0 SNES Function norm 1.999574234301e+08 0 SNES Function norm 1.677632378801e+08 Number of Newton iterations = 0 Converged reason is -3 Might it be helpful to call DMMGGetSNES and then setup the analytical jacobian for the preconditioner matrix? My residual is pretty straightforward though. Yan On Sun, Jan 10, 2010 at 3:59 PM, Matthew Knepley wrote: > It is possible for the radius of quadratic convergence to be very small. > However, I > would check your Jacobian, and maybe try -snes_mf. > > Matt > > > On Sun, Jan 10, 2010 at 2:55 PM, Ryan Yan wrote: > >> Hi All, >> I am solving a nonlinear system using snes. The -snes_monitor option has >> the following output: >> >> 0 SNES Function norm 2.640163923729e+09 >> 1 SNES Function norm 1.047643565314e+08 >> 2 SNES Function norm 1.712732074788e+06 >> 3 SNES Function norm 1.002169173269e+04 >> 4 SNES Function norm 1.655878303433e+03 >> 5 SNES Function norm 3.746498305706e+02 >> 6 SNES Function norm 8.317435704773e+01 >> 7 SNES Function norm 1.857639969641e+01 >> 8 SNES Function norm 4.149691057773e+00 >> 9 SNES Function norm 9.265604042412e-01 >> 10 SNES Function norm 2.069527103214e-01 >> 11 SNES Function norm 4.624186491082e-02 >> 12 SNES Function norm 1.035558432688e-02 >> 13 SNES Function norm 2.341362958811e-03 >> 14 SNES Function norm 5.507445427277e-04 >> 15 SNES Function norm 1.485123568354e-04 >> 16 SNES Function norm 5.180043781814e-05 >> 17 SNES Function norm 2.341966514486e-05 >> 18 SNES Function norm 1.344936158651e-05 >> 19 SNES Function norm 1.054812641176e-05 >> Number of Newton iterations = 19 >> Converged reason is 4 >> >> It looks like the iterate never falls into a quadratic convergence region >> before it converges. Is there any hint to understand this behavior? >> >> Thanks a lot, >> >> Yan >> >> >> > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Sun Jan 10 15:34:36 2010 From: knepley at gmail.com (Matthew Knepley) Date: Sun, 10 Jan 2010 15:34:36 -0600 Subject: [petsc-users] Snes behavior In-Reply-To: References: Message-ID: On Sun, Jan 10, 2010 at 3:28 PM, Ryan Yan wrote: > Hi Matt, > Thank you very much for the suggestion. I was using > DMMGSetSNESLocal(dmmg,FormFunctionLocal,0,ad_FormFunctionLocal,admf_FormFunctionLocal), > so the Jacobian is calculated by automatic differentiation, right? For this > instance, is there any way to check the correctness of the set up of the > residual? > Not unless you have ADIC configured. Do you see the AD code? > After I tried the -snes_mf the linear solver failed( ): > > 0 SNES Function norm 1.578681107621e+08 > 1 SNES Function norm 1.343502549866e+08 > 2 SNES Function norm 1.211729760183e+08 > 3 SNES Function norm 1.211728837635e+08 > 4 SNES Function norm 1.211728837178e+08 > 5 SNES Function norm 1.211728837177e+08 > 0 SNES Function norm 1.999574234301e+08 > 0 SNES Function norm 1.677632378801e+08 > Number of Newton iterations = 0 > Converged reason is -3 > Yes, since there is no preconditioner. However, just use GMRES and a very large subspace. Matt > Might it be helpful to call DMMGGetSNES and then setup the analytical > jacobian for the preconditioner matrix? My residual is pretty > straightforward though. > > Yan > > > > On Sun, Jan 10, 2010 at 3:59 PM, Matthew Knepley wrote: > >> It is possible for the radius of quadratic convergence to be very small. >> However, I >> would check your Jacobian, and maybe try -snes_mf. >> >> Matt >> >> >> On Sun, Jan 10, 2010 at 2:55 PM, Ryan Yan wrote: >> >>> Hi All, >>> I am solving a nonlinear system using snes. The -snes_monitor option has >>> the following output: >>> >>> 0 SNES Function norm 2.640163923729e+09 >>> 1 SNES Function norm 1.047643565314e+08 >>> 2 SNES Function norm 1.712732074788e+06 >>> 3 SNES Function norm 1.002169173269e+04 >>> 4 SNES Function norm 1.655878303433e+03 >>> 5 SNES Function norm 3.746498305706e+02 >>> 6 SNES Function norm 8.317435704773e+01 >>> 7 SNES Function norm 1.857639969641e+01 >>> 8 SNES Function norm 4.149691057773e+00 >>> 9 SNES Function norm 9.265604042412e-01 >>> 10 SNES Function norm 2.069527103214e-01 >>> 11 SNES Function norm 4.624186491082e-02 >>> 12 SNES Function norm 1.035558432688e-02 >>> 13 SNES Function norm 2.341362958811e-03 >>> 14 SNES Function norm 5.507445427277e-04 >>> 15 SNES Function norm 1.485123568354e-04 >>> 16 SNES Function norm 5.180043781814e-05 >>> 17 SNES Function norm 2.341966514486e-05 >>> 18 SNES Function norm 1.344936158651e-05 >>> 19 SNES Function norm 1.054812641176e-05 >>> Number of Newton iterations = 19 >>> Converged reason is 4 >>> >>> It looks like the iterate never falls into a quadratic convergence region >>> before it converges. Is there any hint to understand this behavior? >>> >>> Thanks a lot, >>> >>> Yan >>> >>> >>> >> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which their >> experiments lead. >> -- Norbert Wiener >> > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Sun Jan 10 15:35:40 2010 From: bsmith at mcs.anl.gov (Barry Smith) Date: Sun, 10 Jan 2010 15:35:40 -0600 Subject: [petsc-users] Snes behavior In-Reply-To: References: Message-ID: <0F96DD2E-AAB2-4D43-A985-33C043CA319F@mcs.anl.gov> You already got a 10^16 drop in the residual norm. It is not realistic to expect to get much more than that for double precision calculations. Perhaps your original F() has some funky scaling of different components that you can fix. Barry On Jan 10, 2010, at 2:55 PM, Ryan Yan wrote: > Hi All, > I am solving a nonlinear system using snes. The -snes_monitor option > has the following output: > > 0 SNES Function norm 2.640163923729e+09 > 1 SNES Function norm 1.047643565314e+08 > 2 SNES Function norm 1.712732074788e+06 > 3 SNES Function norm 1.002169173269e+04 > 4 SNES Function norm 1.655878303433e+03 > 5 SNES Function norm 3.746498305706e+02 > 6 SNES Function norm 8.317435704773e+01 > 7 SNES Function norm 1.857639969641e+01 > 8 SNES Function norm 4.149691057773e+00 > 9 SNES Function norm 9.265604042412e-01 > 10 SNES Function norm 2.069527103214e-01 > 11 SNES Function norm 4.624186491082e-02 > 12 SNES Function norm 1.035558432688e-02 > 13 SNES Function norm 2.341362958811e-03 > 14 SNES Function norm 5.507445427277e-04 > 15 SNES Function norm 1.485123568354e-04 > 16 SNES Function norm 5.180043781814e-05 > 17 SNES Function norm 2.341966514486e-05 > 18 SNES Function norm 1.344936158651e-05 > 19 SNES Function norm 1.054812641176e-05 > Number of Newton iterations = 19 > Converged reason is 4 > > It looks like the iterate never falls into a quadratic convergence > region before it converges. Is there any hint to understand this > behavior? > > Thanks a lot, > > Yan > > From vyan2000 at gmail.com Sun Jan 10 15:45:58 2010 From: vyan2000 at gmail.com (Ryan Yan) Date: Sun, 10 Jan 2010 16:45:58 -0500 Subject: [petsc-users] Snes behavior In-Reply-To: References: Message-ID: Hi Matt, Thanks, On Sun, Jan 10, 2010 at 4:34 PM, Matthew Knepley wrote: > On Sun, Jan 10, 2010 at 3:28 PM, Ryan Yan wrote: > >> Hi Matt, >> Thank you very much for the suggestion. I was using >> DMMGSetSNESLocal(dmmg,FormFunctionLocal,0,ad_FormFunctionLocal,admf_FormFunctionLocal), >> so the Jacobian is calculated by automatic differentiation, right? For this >> instance, is there any way to check the correctness of the set up of the >> residual? >> > > Not unless you have ADIC configured. Do you see the AD code? > > I did not have AD code configured when I was installing PETSc. I will leave this option till later for the test. > After I tried the -snes_mf the linear solver failed( ): >> >> 0 SNES Function norm 1.578681107621e+08 >> 1 SNES Function norm 1.343502549866e+08 >> 2 SNES Function norm 1.211729760183e+08 >> 3 SNES Function norm 1.211728837635e+08 >> 4 SNES Function norm 1.211728837178e+08 >> 5 SNES Function norm 1.211728837177e+08 >> 0 SNES Function norm 1.999574234301e+08 >> 0 SNES Function norm 1.677632378801e+08 >> Number of Newton iterations = 0 >> Converged reason is -3 >> > > Yes, since there is no preconditioner. However, just use GMRES and a very > large subspace. > > Sorry, but what do you mean by use a very large subspace? I would try this. Yan > Matt > > >> Might it be helpful to call DMMGGetSNES and then setup the analytical >> jacobian for the preconditioner matrix? My residual is pretty >> straightforward though. >> >> Yan >> >> >> >> On Sun, Jan 10, 2010 at 3:59 PM, Matthew Knepley wrote: >> >>> It is possible for the radius of quadratic convergence to be very small. >>> However, I >>> would check your Jacobian, and maybe try -snes_mf. >>> >>> Matt >>> >>> >>> On Sun, Jan 10, 2010 at 2:55 PM, Ryan Yan wrote: >>> >>>> Hi All, >>>> I am solving a nonlinear system using snes. The -snes_monitor option has >>>> the following output: >>>> >>>> 0 SNES Function norm 2.640163923729e+09 >>>> 1 SNES Function norm 1.047643565314e+08 >>>> 2 SNES Function norm 1.712732074788e+06 >>>> 3 SNES Function norm 1.002169173269e+04 >>>> 4 SNES Function norm 1.655878303433e+03 >>>> 5 SNES Function norm 3.746498305706e+02 >>>> 6 SNES Function norm 8.317435704773e+01 >>>> 7 SNES Function norm 1.857639969641e+01 >>>> 8 SNES Function norm 4.149691057773e+00 >>>> 9 SNES Function norm 9.265604042412e-01 >>>> 10 SNES Function norm 2.069527103214e-01 >>>> 11 SNES Function norm 4.624186491082e-02 >>>> 12 SNES Function norm 1.035558432688e-02 >>>> 13 SNES Function norm 2.341362958811e-03 >>>> 14 SNES Function norm 5.507445427277e-04 >>>> 15 SNES Function norm 1.485123568354e-04 >>>> 16 SNES Function norm 5.180043781814e-05 >>>> 17 SNES Function norm 2.341966514486e-05 >>>> 18 SNES Function norm 1.344936158651e-05 >>>> 19 SNES Function norm 1.054812641176e-05 >>>> Number of Newton iterations = 19 >>>> Converged reason is 4 >>>> >>>> It looks like the iterate never falls into a quadratic convergence >>>> region before it converges. Is there any hint to understand this behavior? >>>> >>>> Thanks a lot, >>>> >>>> Yan >>>> >>>> >>>> >>> >>> >>> -- >>> What most experimenters take for granted before they begin their >>> experiments is infinitely more interesting than any results to which their >>> experiments lead. >>> -- Norbert Wiener >>> >> >> > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vyan2000 at gmail.com Sun Jan 10 15:52:33 2010 From: vyan2000 at gmail.com (Ryan Yan) Date: Sun, 10 Jan 2010 16:52:33 -0500 Subject: [petsc-users] Snes behavior In-Reply-To: <0F96DD2E-AAB2-4D43-A985-33C043CA319F@mcs.anl.gov> References: <0F96DD2E-AAB2-4D43-A985-33C043CA319F@mcs.anl.gov> Message-ID: Hi Barry, Yes, exactly. The original multi-components system scale quite unevenly. I will try to rescale it. Could this be helpful to show some promise on quadratic convergence? Thanks a lot, Yan On Sun, Jan 10, 2010 at 4:35 PM, Barry Smith wrote: > > You already got a 10^16 drop in the residual norm. It is not realistic to > expect to get much more than that for double precision calculations. Perhaps > your original F() has some funky scaling of different components that you > can fix. > > > > Barry > > > On Jan 10, 2010, at 2:55 PM, Ryan Yan wrote: > > Hi All, >> I am solving a nonlinear system using snes. The -snes_monitor option has >> the following output: >> >> 0 SNES Function norm 2.640163923729e+09 >> 1 SNES Function norm 1.047643565314e+08 >> 2 SNES Function norm 1.712732074788e+06 >> 3 SNES Function norm 1.002169173269e+04 >> 4 SNES Function norm 1.655878303433e+03 >> 5 SNES Function norm 3.746498305706e+02 >> 6 SNES Function norm 8.317435704773e+01 >> 7 SNES Function norm 1.857639969641e+01 >> 8 SNES Function norm 4.149691057773e+00 >> 9 SNES Function norm 9.265604042412e-01 >> 10 SNES Function norm 2.069527103214e-01 >> 11 SNES Function norm 4.624186491082e-02 >> 12 SNES Function norm 1.035558432688e-02 >> 13 SNES Function norm 2.341362958811e-03 >> 14 SNES Function norm 5.507445427277e-04 >> 15 SNES Function norm 1.485123568354e-04 >> 16 SNES Function norm 5.180043781814e-05 >> 17 SNES Function norm 2.341966514486e-05 >> 18 SNES Function norm 1.344936158651e-05 >> 19 SNES Function norm 1.054812641176e-05 >> Number of Newton iterations = 19 >> Converged reason is 4 >> >> It looks like the iterate never falls into a quadratic convergence region >> before it converges. Is there any hint to understand this behavior? >> >> Thanks a lot, >> >> Yan >> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Sun Jan 10 15:57:01 2010 From: bsmith at mcs.anl.gov (Barry Smith) Date: Sun, 10 Jan 2010 15:57:01 -0600 Subject: [petsc-users] Snes behavior In-Reply-To: References: <0F96DD2E-AAB2-4D43-A985-33C043CA319F@mcs.anl.gov> Message-ID: On Jan 10, 2010, at 3:52 PM, Ryan Yan wrote: > Hi Barry, > Yes, exactly. The original multi-components system scale quite > unevenly. I will try to rescale it. > Could this be helpful to show some promise on quadratic convergence? I won't be concerned about "quadratic convergence" I'd only be concerned that it is converging to the correct answer and that you are getting close enough to the correct answer. You can run with -snes_mf_operator and -snes_type test to verify if the Jacobian being computed is accurate. Perhaps in your function evaluation you are not using the stencil that you set with the DA, this would cause the wrong Jacobian to be computed. Barry > > Thanks a lot, > > Yan > > On Sun, Jan 10, 2010 at 4:35 PM, Barry Smith > wrote: > > You already got a 10^16 drop in the residual norm. It is not > realistic to expect to get much more than that for double precision > calculations. Perhaps your original F() has some funky scaling of > different components that you can fix. > > > > Barry > > > On Jan 10, 2010, at 2:55 PM, Ryan Yan wrote: > > Hi All, > I am solving a nonlinear system using snes. The -snes_monitor option > has the following output: > > 0 SNES Function norm 2.640163923729e+09 > 1 SNES Function norm 1.047643565314e+08 > 2 SNES Function norm 1.712732074788e+06 > 3 SNES Function norm 1.002169173269e+04 > 4 SNES Function norm 1.655878303433e+03 > 5 SNES Function norm 3.746498305706e+02 > 6 SNES Function norm 8.317435704773e+01 > 7 SNES Function norm 1.857639969641e+01 > 8 SNES Function norm 4.149691057773e+00 > 9 SNES Function norm 9.265604042412e-01 > 10 SNES Function norm 2.069527103214e-01 > 11 SNES Function norm 4.624186491082e-02 > 12 SNES Function norm 1.035558432688e-02 > 13 SNES Function norm 2.341362958811e-03 > 14 SNES Function norm 5.507445427277e-04 > 15 SNES Function norm 1.485123568354e-04 > 16 SNES Function norm 5.180043781814e-05 > 17 SNES Function norm 2.341966514486e-05 > 18 SNES Function norm 1.344936158651e-05 > 19 SNES Function norm 1.054812641176e-05 > Number of Newton iterations = 19 > Converged reason is 4 > > It looks like the iterate never falls into a quadratic convergence > region before it converges. Is there any hint to understand this > behavior? > > Thanks a lot, > > Yan > > > > From vyan2000 at gmail.com Sun Jan 10 16:42:24 2010 From: vyan2000 at gmail.com (Ryan Yan) Date: Sun, 10 Jan 2010 17:42:24 -0500 Subject: [petsc-users] Snes behavior In-Reply-To: References: <0F96DD2E-AAB2-4D43-A985-33C043CA319F@mcs.anl.gov> Message-ID: Hi Barry, Please see reply below, On Sun, Jan 10, 2010 at 4:57 PM, Barry Smith wrote: > > On Jan 10, 2010, at 3:52 PM, Ryan Yan wrote: > > Hi Barry, >> Yes, exactly. The original multi-components system scale quite unevenly. I >> will try to rescale it. >> Could this be helpful to show some promise on quadratic convergence? >> > > I won't be concerned about "quadratic convergence" I'd only be concerned > that it is converging to the correct answer and that you are getting close > enough to the correct answer. > > Yes, I agree. > You can run with -snes_mf_operator and -snes_type test to verify if the > Jacobian being computed is accurate. Perhaps in your function evaluation you > are not using the stencil that you set with the DA, this would cause the > wrong Jacobian to be computed. > > After passing in -snes_mf_operator and -snes_type test as follows: vyan2000 at vyan2000-linux ~/NCproject/general : mpirun -np 2 ./HeatProfile1D -snes_mf_operator -snes_type test I got errors, as expected: [0]PETSC ERROR: --------------------- Error Message ------------------------------------ [0]PETSC ERROR: Invalid argument! [0]PETSC ERROR: Cannot test with alternative preconditioner! [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: Petsc Release Version 3.0.0, Patch 5, Mon Apr 13 09:15:37 CDT 2009 [0]PETSC ERROR: See docs/changes/index.html for recent updates. [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. [0]PETSC ERROR: See docs/index.html for manual pages. [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: ./HeatProfile1D on a O-hypre-p named vyan2000-linux by vyan2000 Sun Jan 10 17:16:00 2010 [0]PETSC ERROR: Libraries linked from /home/vyan2000/local/PPETSc/petsc-3.0.0-p5/O-hypre-prometheus/lib [0]PETSC ERROR: Configure run at Thu Jun 25 13:49:36 2009 [0]PETSC ERROR: Configure options --download-mpich=1 --with-debugger=gdb --download-hypre=1 --download-parmetis=1 --download-prometheus=1 --with-shared=0 [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: SNESSolve_Test() line 28 in src/snes/impls/test/snestest.c [0]PETSC ERROR: SNESSolve() line 2221 in src/snes/interface/snes.c [0]PETSC ERROR: DMMGSolveSNES() line 510 in src/snes/utils/damgsnes.c [0]PETSC ERROR: DMMGSolve() line 372 in src/snes/utils/damg.c [0]PETSC ERROR: main() line 270 in /home/vyan2000/local/PPETSc/petsc-2.3.3-p15/src/ksp/ksp/examples/tutorials/NCprojectHeatProfile1D.c application called MPI_Abort(MPI_COMM_WORLD, 62) - process 0[cli_0]: aborting job: application called MPI_Abort(MPI_COMM_WORLD, 62) - process 0 The error makes sense, since I did not pass in the analytical Jacobian for the preconditioner matrix. In stead, I was using DMMGSetSNESLocal(dmmg,FormFunctionLocal,0,ad_FormFunctionLocal,admf_FormFunctionLocal). I am going to change the code a bit, pass in the analytical Jacobian and do the -snes_type test. I will bear your suggestion in mind during the test. Thanks a lot, Yan > > Barry > > > >> Thanks a lot, >> >> Yan >> >> On Sun, Jan 10, 2010 at 4:35 PM, Barry Smith wrote: >> >> You already got a 10^16 drop in the residual norm. It is not realistic to >> expect to get much more than that for double precision calculations. Perhaps >> your original F() has some funky scaling of different components that you >> can fix. >> >> >> >> Barry >> >> >> On Jan 10, 2010, at 2:55 PM, Ryan Yan wrote: >> >> Hi All, >> I am solving a nonlinear system using snes. The -snes_monitor option has >> the following output: >> >> 0 SNES Function norm 2.640163923729e+09 >> 1 SNES Function norm 1.047643565314e+08 >> 2 SNES Function norm 1.712732074788e+06 >> 3 SNES Function norm 1.002169173269e+04 >> 4 SNES Function norm 1.655878303433e+03 >> 5 SNES Function norm 3.746498305706e+02 >> 6 SNES Function norm 8.317435704773e+01 >> 7 SNES Function norm 1.857639969641e+01 >> 8 SNES Function norm 4.149691057773e+00 >> 9 SNES Function norm 9.265604042412e-01 >> 10 SNES Function norm 2.069527103214e-01 >> 11 SNES Function norm 4.624186491082e-02 >> 12 SNES Function norm 1.035558432688e-02 >> 13 SNES Function norm 2.341362958811e-03 >> 14 SNES Function norm 5.507445427277e-04 >> 15 SNES Function norm 1.485123568354e-04 >> 16 SNES Function norm 5.180043781814e-05 >> 17 SNES Function norm 2.341966514486e-05 >> 18 SNES Function norm 1.344936158651e-05 >> 19 SNES Function norm 1.054812641176e-05 >> Number of Newton iterations = 19 >> Converged reason is 4 >> >> It looks like the iterate never falls into a quadratic convergence region >> before it converges. Is there any hint to understand this behavior? >> >> Thanks a lot, >> >> Yan >> >> >> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Sun Jan 10 16:57:05 2010 From: bsmith at mcs.anl.gov (Barry Smith) Date: Sun, 10 Jan 2010 16:57:05 -0600 Subject: [petsc-users] Snes behavior In-Reply-To: References: <0F96DD2E-AAB2-4D43-A985-33C043CA319F@mcs.anl.gov> Message-ID: JUST run with -snes_mf_operator and then with -snes_type test NOT together, Barry On Jan 10, 2010, at 4:42 PM, Ryan Yan wrote: > Hi Barry, > Please see reply below, > > On Sun, Jan 10, 2010 at 4:57 PM, Barry Smith > wrote: > > On Jan 10, 2010, at 3:52 PM, Ryan Yan wrote: > > Hi Barry, > Yes, exactly. The original multi-components system scale quite > unevenly. I will try to rescale it. > Could this be helpful to show some promise on quadratic convergence? > > I won't be concerned about "quadratic convergence" I'd only be > concerned that it is converging to the correct answer and that you > are getting close enough to the correct answer. > > Yes, I agree. > > You can run with -snes_mf_operator and -snes_type test to verify > if the Jacobian being computed is accurate. Perhaps in your function > evaluation you are not using the stencil that you set with the DA, > this would cause the wrong Jacobian to be computed. > > > After passing in -snes_mf_operator and -snes_type test as follows: > vyan2000 at vyan2000-linux ~/NCproject/general : mpirun -np 2 ./ > HeatProfile1D -snes_mf_operator -snes_type test > I got errors, as expected: > > [0]PETSC ERROR: --------------------- Error Message > ------------------------------------ > [0]PETSC ERROR: Invalid argument! > [0]PETSC ERROR: Cannot test with alternative preconditioner! > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: Petsc Release Version 3.0.0, Patch 5, Mon Apr 13 > 09:15:37 CDT 2009 > [0]PETSC ERROR: See docs/changes/index.html for recent updates. > [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. > [0]PETSC ERROR: See docs/index.html for manual pages. > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: ./HeatProfile1D on a O-hypre-p named vyan2000-linux > by vyan2000 Sun Jan 10 17:16:00 2010 > [0]PETSC ERROR: Libraries linked from /home/vyan2000/local/PPETSc/ > petsc-3.0.0-p5/O-hypre-prometheus/lib > [0]PETSC ERROR: Configure run at Thu Jun 25 13:49:36 2009 > [0]PETSC ERROR: Configure options --download-mpich=1 --with- > debugger=gdb --download-hypre=1 --download-parmetis=1 --download- > prometheus=1 --with-shared=0 > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: SNESSolve_Test() line 28 in src/snes/impls/test/ > snestest.c > [0]PETSC ERROR: SNESSolve() line 2221 in src/snes/interface/snes.c > [0]PETSC ERROR: DMMGSolveSNES() line 510 in src/snes/utils/damgsnes.c > [0]PETSC ERROR: DMMGSolve() line 372 in src/snes/utils/damg.c > [0]PETSC ERROR: main() line 270 in /home/vyan2000/local/PPETSc/ > petsc-2.3.3-p15/src/ksp/ksp/examples/tutorials/ > NCprojectHeatProfile1D.c > application called MPI_Abort(MPI_COMM_WORLD, 62) - process 0[cli_0]: > aborting job: > application called MPI_Abort(MPI_COMM_WORLD, 62) - process 0 > > The error makes sense, since I did not pass in the analytical > Jacobian for the preconditioner matrix. In stead, I was using > DMMGSetSNESLocal(dmmg,FormFunctionLocal, > 0,ad_FormFunctionLocal,admf_FormFunctionLocal). I am going to change > the code a bit, pass in the analytical Jacobian and do the - > snes_type test. > > I will bear your suggestion in mind during the test. > > Thanks a lot, > > Yan > > > > Barry > > > > Thanks a lot, > > Yan > > On Sun, Jan 10, 2010 at 4:35 PM, Barry Smith > wrote: > > You already got a 10^16 drop in the residual norm. It is not > realistic to expect to get much more than that for double precision > calculations. Perhaps your original F() has some funky scaling of > different components that you can fix. > > > > Barry > > > On Jan 10, 2010, at 2:55 PM, Ryan Yan wrote: > > Hi All, > I am solving a nonlinear system using snes. The -snes_monitor option > has the following output: > > 0 SNES Function norm 2.640163923729e+09 > 1 SNES Function norm 1.047643565314e+08 > 2 SNES Function norm 1.712732074788e+06 > 3 SNES Function norm 1.002169173269e+04 > 4 SNES Function norm 1.655878303433e+03 > 5 SNES Function norm 3.746498305706e+02 > 6 SNES Function norm 8.317435704773e+01 > 7 SNES Function norm 1.857639969641e+01 > 8 SNES Function norm 4.149691057773e+00 > 9 SNES Function norm 9.265604042412e-01 > 10 SNES Function norm 2.069527103214e-01 > 11 SNES Function norm 4.624186491082e-02 > 12 SNES Function norm 1.035558432688e-02 > 13 SNES Function norm 2.341362958811e-03 > 14 SNES Function norm 5.507445427277e-04 > 15 SNES Function norm 1.485123568354e-04 > 16 SNES Function norm 5.180043781814e-05 > 17 SNES Function norm 2.341966514486e-05 > 18 SNES Function norm 1.344936158651e-05 > 19 SNES Function norm 1.054812641176e-05 > Number of Newton iterations = 19 > Converged reason is 4 > > It looks like the iterate never falls into a quadratic convergence > region before it converges. Is there any hint to understand this > behavior? > > Thanks a lot, > > Yan > > > > > > From vyan2000 at gmail.com Sun Jan 10 17:09:12 2010 From: vyan2000 at gmail.com (Ryan Yan) Date: Sun, 10 Jan 2010 18:09:12 -0500 Subject: [petsc-users] Snes behavior In-Reply-To: References: <0F96DD2E-AAB2-4D43-A985-33C043CA319F@mcs.anl.gov> Message-ID: Hi Barry, I got the following result: vyan2000 at vyan2000-linux ~/NCproject/general $ mpirun -np 1 ./HeatProfile1D -snes_mf_operator Alarm clock vyan2000 at vyan2000-linux ~/NCproject/general $ mpirun -np 1 ./HeatProfile1D -snes_type test Alarm clock Are they normal responses and what do they indicate? Thanks a lot, Yan On Sun, Jan 10, 2010 at 5:57 PM, Barry Smith wrote: > > JUST run with -snes_mf_operator and then with -snes_type test NOT > together, > > Barry > > > On Jan 10, 2010, at 4:42 PM, Ryan Yan wrote: > > Hi Barry, >> Please see reply below, >> >> On Sun, Jan 10, 2010 at 4:57 PM, Barry Smith wrote: >> >> On Jan 10, 2010, at 3:52 PM, Ryan Yan wrote: >> >> Hi Barry, >> Yes, exactly. The original multi-components system scale quite unevenly. I >> will try to rescale it. >> Could this be helpful to show some promise on quadratic convergence? >> >> I won't be concerned about "quadratic convergence" I'd only be concerned >> that it is converging to the correct answer and that you are getting close >> enough to the correct answer. >> >> Yes, I agree. >> >> You can run with -snes_mf_operator and -snes_type test to verify if the >> Jacobian being computed is accurate. Perhaps in your function evaluation you >> are not using the stencil that you set with the DA, this would cause the >> wrong Jacobian to be computed. >> >> >> After passing in -snes_mf_operator and -snes_type test as follows: >> vyan2000 at vyan2000-linux ~/NCproject/general : mpirun -np 2 >> ./HeatProfile1D -snes_mf_operator -snes_type test >> I got errors, as expected: >> >> [0]PETSC ERROR: --------------------- Error Message >> ------------------------------------ >> [0]PETSC ERROR: Invalid argument! >> [0]PETSC ERROR: Cannot test with alternative preconditioner! >> [0]PETSC ERROR: >> ------------------------------------------------------------------------ >> [0]PETSC ERROR: Petsc Release Version 3.0.0, Patch 5, Mon Apr 13 09:15:37 >> CDT 2009 >> [0]PETSC ERROR: See docs/changes/index.html for recent updates. >> [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. >> [0]PETSC ERROR: See docs/index.html for manual pages. >> [0]PETSC ERROR: >> ------------------------------------------------------------------------ >> [0]PETSC ERROR: ./HeatProfile1D on a O-hypre-p named vyan2000-linux by >> vyan2000 Sun Jan 10 17:16:00 2010 >> [0]PETSC ERROR: Libraries linked from >> /home/vyan2000/local/PPETSc/petsc-3.0.0-p5/O-hypre-prometheus/lib >> [0]PETSC ERROR: Configure run at Thu Jun 25 13:49:36 2009 >> [0]PETSC ERROR: Configure options --download-mpich=1 --with-debugger=gdb >> --download-hypre=1 --download-parmetis=1 --download-prometheus=1 >> --with-shared=0 >> [0]PETSC ERROR: >> ------------------------------------------------------------------------ >> [0]PETSC ERROR: SNESSolve_Test() line 28 in src/snes/impls/test/snestest.c >> [0]PETSC ERROR: SNESSolve() line 2221 in src/snes/interface/snes.c >> [0]PETSC ERROR: DMMGSolveSNES() line 510 in src/snes/utils/damgsnes.c >> [0]PETSC ERROR: DMMGSolve() line 372 in src/snes/utils/damg.c >> [0]PETSC ERROR: main() line 270 in >> /home/vyan2000/local/PPETSc/petsc-2.3.3-p15/src/ksp/ksp/examples/tutorials/NCprojectHeatProfile1D.c >> application called MPI_Abort(MPI_COMM_WORLD, 62) - process 0[cli_0]: >> aborting job: >> application called MPI_Abort(MPI_COMM_WORLD, 62) - process 0 >> >> The error makes sense, since I did not pass in the analytical Jacobian for >> the preconditioner matrix. In stead, I was using >> DMMGSetSNESLocal(dmmg,FormFunctionLocal,0,ad_FormFunctionLocal,admf_FormFunctionLocal). >> I am going to change the code a bit, pass in the analytical Jacobian and do >> the -snes_type test. >> >> I will bear your suggestion in mind during the test. >> >> Thanks a lot, >> >> Yan >> >> >> >> Barry >> >> >> >> Thanks a lot, >> >> Yan >> >> On Sun, Jan 10, 2010 at 4:35 PM, Barry Smith wrote: >> >> You already got a 10^16 drop in the residual norm. It is not realistic to >> expect to get much more than that for double precision calculations. Perhaps >> your original F() has some funky scaling of different components that you >> can fix. >> >> >> >> Barry >> >> >> On Jan 10, 2010, at 2:55 PM, Ryan Yan wrote: >> >> Hi All, >> I am solving a nonlinear system using snes. The -snes_monitor option has >> the following output: >> >> 0 SNES Function norm 2.640163923729e+09 >> 1 SNES Function norm 1.047643565314e+08 >> 2 SNES Function norm 1.712732074788e+06 >> 3 SNES Function norm 1.002169173269e+04 >> 4 SNES Function norm 1.655878303433e+03 >> 5 SNES Function norm 3.746498305706e+02 >> 6 SNES Function norm 8.317435704773e+01 >> 7 SNES Function norm 1.857639969641e+01 >> 8 SNES Function norm 4.149691057773e+00 >> 9 SNES Function norm 9.265604042412e-01 >> 10 SNES Function norm 2.069527103214e-01 >> 11 SNES Function norm 4.624186491082e-02 >> 12 SNES Function norm 1.035558432688e-02 >> 13 SNES Function norm 2.341362958811e-03 >> 14 SNES Function norm 5.507445427277e-04 >> 15 SNES Function norm 1.485123568354e-04 >> 16 SNES Function norm 5.180043781814e-05 >> 17 SNES Function norm 2.341966514486e-05 >> 18 SNES Function norm 1.344936158651e-05 >> 19 SNES Function norm 1.054812641176e-05 >> Number of Newton iterations = 19 >> Converged reason is 4 >> >> It looks like the iterate never falls into a quadratic convergence region >> before it converges. Is there any hint to understand this behavior? >> >> Thanks a lot, >> >> Yan >> >> >> >> >> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Sun Jan 10 17:11:20 2010 From: bsmith at mcs.anl.gov (Barry Smith) Date: Sun, 10 Jan 2010 17:11:20 -0600 Subject: [petsc-users] Snes behavior In-Reply-To: References: <0F96DD2E-AAB2-4D43-A985-33C043CA319F@mcs.anl.gov> Message-ID: That message ain't from PETSc. Something is likely killing the job. Barry On Jan 10, 2010, at 5:09 PM, Ryan Yan wrote: > Hi Barry, > I got the following result: > > vyan2000 at vyan2000-linux ~/NCproject/general $ mpirun -np 1 ./ > HeatProfile1D -snes_mf_operator > Alarm clock > vyan2000 at vyan2000-linux ~/NCproject/general $ mpirun -np 1 ./ > HeatProfile1D -snes_type test > Alarm clock > > Are they normal responses and what do they indicate? > > Thanks a lot, > > Yan > > > On Sun, Jan 10, 2010 at 5:57 PM, Barry Smith > wrote: > > JUST run with -snes_mf_operator and then with -snes_type test NOT > together, > > Barry > > > On Jan 10, 2010, at 4:42 PM, Ryan Yan wrote: > > Hi Barry, > Please see reply below, > > On Sun, Jan 10, 2010 at 4:57 PM, Barry Smith > wrote: > > On Jan 10, 2010, at 3:52 PM, Ryan Yan wrote: > > Hi Barry, > Yes, exactly. The original multi-components system scale quite > unevenly. I will try to rescale it. > Could this be helpful to show some promise on quadratic convergence? > > I won't be concerned about "quadratic convergence" I'd only be > concerned that it is converging to the correct answer and that you > are getting close enough to the correct answer. > > Yes, I agree. > > You can run with -snes_mf_operator and -snes_type test to verify > if the Jacobian being computed is accurate. Perhaps in your function > evaluation you are not using the stencil that you set with the DA, > this would cause the wrong Jacobian to be computed. > > > After passing in -snes_mf_operator and -snes_type test as follows: > vyan2000 at vyan2000-linux ~/NCproject/general : mpirun -np 2 ./ > HeatProfile1D -snes_mf_operator -snes_type test > I got errors, as expected: > > [0]PETSC ERROR: --------------------- Error Message > ------------------------------------ > [0]PETSC ERROR: Invalid argument! > [0]PETSC ERROR: Cannot test with alternative preconditioner! > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: Petsc Release Version 3.0.0, Patch 5, Mon Apr 13 > 09:15:37 CDT 2009 > [0]PETSC ERROR: See docs/changes/index.html for recent updates. > [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. > [0]PETSC ERROR: See docs/index.html for manual pages. > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: ./HeatProfile1D on a O-hypre-p named vyan2000-linux > by vyan2000 Sun Jan 10 17:16:00 2010 > [0]PETSC ERROR: Libraries linked from /home/vyan2000/local/PPETSc/ > petsc-3.0.0-p5/O-hypre-prometheus/lib > [0]PETSC ERROR: Configure run at Thu Jun 25 13:49:36 2009 > [0]PETSC ERROR: Configure options --download-mpich=1 --with- > debugger=gdb --download-hypre=1 --download-parmetis=1 --download- > prometheus=1 --with-shared=0 > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: SNESSolve_Test() line 28 in src/snes/impls/test/ > snestest.c > [0]PETSC ERROR: SNESSolve() line 2221 in src/snes/interface/snes.c > [0]PETSC ERROR: DMMGSolveSNES() line 510 in src/snes/utils/damgsnes.c > [0]PETSC ERROR: DMMGSolve() line 372 in src/snes/utils/damg.c > [0]PETSC ERROR: main() line 270 in /home/vyan2000/local/PPETSc/ > petsc-2.3.3-p15/src/ksp/ksp/examples/tutorials/ > NCprojectHeatProfile1D.c > application called MPI_Abort(MPI_COMM_WORLD, 62) - process 0[cli_0]: > aborting job: > application called MPI_Abort(MPI_COMM_WORLD, 62) - process 0 > > The error makes sense, since I did not pass in the analytical > Jacobian for the preconditioner matrix. In stead, I was using > DMMGSetSNESLocal(dmmg,FormFunctionLocal, > 0,ad_FormFunctionLocal,admf_FormFunctionLocal). I am going to change > the code a bit, pass in the analytical Jacobian and do the - > snes_type test. > > I will bear your suggestion in mind during the test. > > Thanks a lot, > > Yan > > > > Barry > > > > Thanks a lot, > > Yan > > On Sun, Jan 10, 2010 at 4:35 PM, Barry Smith > wrote: > > You already got a 10^16 drop in the residual norm. It is not > realistic to expect to get much more than that for double precision > calculations. Perhaps your original F() has some funky scaling of > different components that you can fix. > > > > Barry > > > On Jan 10, 2010, at 2:55 PM, Ryan Yan wrote: > > Hi All, > I am solving a nonlinear system using snes. The -snes_monitor option > has the following output: > > 0 SNES Function norm 2.640163923729e+09 > 1 SNES Function norm 1.047643565314e+08 > 2 SNES Function norm 1.712732074788e+06 > 3 SNES Function norm 1.002169173269e+04 > 4 SNES Function norm 1.655878303433e+03 > 5 SNES Function norm 3.746498305706e+02 > 6 SNES Function norm 8.317435704773e+01 > 7 SNES Function norm 1.857639969641e+01 > 8 SNES Function norm 4.149691057773e+00 > 9 SNES Function norm 9.265604042412e-01 > 10 SNES Function norm 2.069527103214e-01 > 11 SNES Function norm 4.624186491082e-02 > 12 SNES Function norm 1.035558432688e-02 > 13 SNES Function norm 2.341362958811e-03 > 14 SNES Function norm 5.507445427277e-04 > 15 SNES Function norm 1.485123568354e-04 > 16 SNES Function norm 5.180043781814e-05 > 17 SNES Function norm 2.341966514486e-05 > 18 SNES Function norm 1.344936158651e-05 > 19 SNES Function norm 1.054812641176e-05 > Number of Newton iterations = 19 > Converged reason is 4 > > It looks like the iterate never falls into a quadratic convergence > region before it converges. Is there any hint to understand this > behavior? > > Thanks a lot, > > Yan > > > > > > > > From vyan2000 at gmail.com Sun Jan 10 18:09:01 2010 From: vyan2000 at gmail.com (Ryan Yan) Date: Sun, 10 Jan 2010 19:09:01 -0500 Subject: [petsc-users] Snes behavior In-Reply-To: References: <0F96DD2E-AAB2-4D43-A985-33C043CA319F@mcs.anl.gov> Message-ID: Hi Barry, I run the code on another machine. yy2250 at sci-m8n6 ~/test/1d $ srun -p sci-comp -N 2 -n 2 ./HeatProfile1D -snes_type test > out.snes_test [0]PETSC ERROR: SNESSolve() line 2221 in src/snes/interface/snes.c [1]PETSC ERROR: SNESSolve() line 2221 in src/snes/interface/snes.c [0]PETSC ERROR: DMMGSolveSNES() line 510 in src/snes/utils/damgsnes.c [0]PETSC ERROR: DMMGSolve() line 372 in src/snes/utils/damg.c [0]PETSC ERROR: main() line 270 in /home/vyan2000/local/PPETSc/petsc-2.3.3-p15/src/ksp/ksp/examples/tutorials/NCprojectHeatProfile1D.c application called MPI_Abort(MPI_COMM_WORLD, 73) - process 0 [1]PETSC ERROR: DMMGSolveSNES() line 510 in src/snes/utils/damgsnes.c [1]PETSC ERROR: DMMGSolve() line 372 in src/snes/utils/damg.c [1]PETSC ERROR: main() line 270 in /home/vyan2000/local/PPETSc/petsc-2.3.3-p15/src/ksp/ksp/examples/tutorials/NCprojectHeatProfile1D.c In: PMI_Abort(73, application called MPI_Abort(MPI_COMM_WORLD, 73) - process 0) application called MPI_Abort(MPI_COMM_WORLD, 73) - process 1 In: PMI_Abort(73, application called MPI_Abort(MPI_COMM_WORLD, 73) - process 1) srun: error: task 1: Exited with exit code 73 srun: error: task 0: Exited with exit code 73 I was using 3 level grids. The error comes out during the snes solving on the fine level grid. The output in the file out.snes_test is : yy2250 at sci-m8n6 ~/test/1d $ cat out.snes_test Testing hand-coded Jacobian, if the ratio is O(1.e-8), the hand-coded Jacobian is probably correct. Run with -snes_test_display to show difference of hand-coded and finite difference Jacobian. Norm of matrix ratio 1.21314e-10 difference 1.41447 Norm of matrix ratio 4.30115e-06 difference 1.41427 Norm of matrix ratio 4.30108e-06 difference 1.41427 For the -snes_mf_operator, yy2250 at sci-m8n6 ~/test/1d $ srun -p sci-comp -N 2 -n 2 ./HeatProfile1D -snes_mf_operator the program is still running. I suspect that the program hanged. Is there any useful information in the file "out.snes_test"? BTW, I did not hand-code any Jacobian, since I passed 0 for the Jacobian evaluation subroutine in the DMMGSetSNESLocal(). Thank you very much, Yan On Sun, Jan 10, 2010 at 6:11 PM, Barry Smith wrote: > > That message ain't from PETSc. Something is likely killing the job. > > Barry > > > On Jan 10, 2010, at 5:09 PM, Ryan Yan wrote: > > Hi Barry, >> I got the following result: >> >> vyan2000 at vyan2000-linux ~/NCproject/general $ mpirun -np 1 >> ./HeatProfile1D -snes_mf_operator >> Alarm clock >> vyan2000 at vyan2000-linux ~/NCproject/general $ mpirun -np 1 >> ./HeatProfile1D -snes_type test >> Alarm clock >> >> Are they normal responses and what do they indicate? >> >> Thanks a lot, >> >> Yan >> >> >> On Sun, Jan 10, 2010 at 5:57 PM, Barry Smith wrote: >> >> JUST run with -snes_mf_operator and then with -snes_type test NOT >> together, >> >> Barry >> >> >> On Jan 10, 2010, at 4:42 PM, Ryan Yan wrote: >> >> Hi Barry, >> Please see reply below, >> >> On Sun, Jan 10, 2010 at 4:57 PM, Barry Smith wrote: >> >> On Jan 10, 2010, at 3:52 PM, Ryan Yan wrote: >> >> Hi Barry, >> Yes, exactly. The original multi-components system scale quite unevenly. I >> will try to rescale it. >> Could this be helpful to show some promise on quadratic convergence? >> >> I won't be concerned about "quadratic convergence" I'd only be concerned >> that it is converging to the correct answer and that you are getting close >> enough to the correct answer. >> >> Yes, I agree. >> >> You can run with -snes_mf_operator and -snes_type test to verify if the >> Jacobian being computed is accurate. Perhaps in your function evaluation you >> are not using the stencil that you set with the DA, this would cause the >> wrong Jacobian to be computed. >> >> >> After passing in -snes_mf_operator and -snes_type test as follows: >> vyan2000 at vyan2000-linux ~/NCproject/general : mpirun -np 2 >> ./HeatProfile1D -snes_mf_operator -snes_type test >> I got errors, as expected: >> >> [0]PETSC ERROR: --------------------- Error Message >> ------------------------------------ >> [0]PETSC ERROR: Invalid argument! >> [0]PETSC ERROR: Cannot test with alternative preconditioner! >> [0]PETSC ERROR: >> ------------------------------------------------------------------------ >> [0]PETSC ERROR: Petsc Release Version 3.0.0, Patch 5, Mon Apr 13 09:15:37 >> CDT 2009 >> [0]PETSC ERROR: See docs/changes/index.html for recent updates. >> [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. >> [0]PETSC ERROR: See docs/index.html for manual pages. >> [0]PETSC ERROR: >> ------------------------------------------------------------------------ >> [0]PETSC ERROR: ./HeatProfile1D on a O-hypre-p named vyan2000-linux by >> vyan2000 Sun Jan 10 17:16:00 2010 >> [0]PETSC ERROR: Libraries linked from >> /home/vyan2000/local/PPETSc/petsc-3.0.0-p5/O-hypre-prometheus/lib >> [0]PETSC ERROR: Configure run at Thu Jun 25 13:49:36 2009 >> [0]PETSC ERROR: Configure options --download-mpich=1 --with-debugger=gdb >> --download-hypre=1 --download-parmetis=1 --download-prometheus=1 >> --with-shared=0 >> [0]PETSC ERROR: >> ------------------------------------------------------------------------ >> [0]PETSC ERROR: SNESSolve_Test() line 28 in src/snes/impls/test/snestest.c >> [0]PETSC ERROR: SNESSolve() line 2221 in src/snes/interface/snes.c >> [0]PETSC ERROR: DMMGSolveSNES() line 510 in src/snes/utils/damgsnes.c >> [0]PETSC ERROR: DMMGSolve() line 372 in src/snes/utils/damg.c >> [0]PETSC ERROR: main() line 270 in >> /home/vyan2000/local/PPETSc/petsc-2.3.3-p15/src/ksp/ksp/examples/tutorials/NCprojectHeatProfile1D.c >> application called MPI_Abort(MPI_COMM_WORLD, 62) - process 0[cli_0]: >> aborting job: >> application called MPI_Abort(MPI_COMM_WORLD, 62) - process 0 >> >> The error makes sense, since I did not pass in the analytical Jacobian for >> the preconditioner matrix. In stead, I was using >> DMMGSetSNESLocal(dmmg,FormFunctionLocal,0,ad_FormFunctionLocal,admf_FormFunctionLocal). >> I am going to change the code a bit, pass in the analytical Jacobian and do >> the -snes_type test. >> >> I will bear your suggestion in mind during the test. >> >> Thanks a lot, >> >> Yan >> >> >> >> Barry >> >> >> >> Thanks a lot, >> >> Yan >> >> On Sun, Jan 10, 2010 at 4:35 PM, Barry Smith wrote: >> >> You already got a 10^16 drop in the residual norm. It is not realistic to >> expect to get much more than that for double precision calculations. Perhaps >> your original F() has some funky scaling of different components that you >> can fix. >> >> >> >> Barry >> >> >> On Jan 10, 2010, at 2:55 PM, Ryan Yan wrote: >> >> Hi All, >> I am solving a nonlinear system using snes. The -snes_monitor option has >> the following output: >> >> 0 SNES Function norm 2.640163923729e+09 >> 1 SNES Function norm 1.047643565314e+08 >> 2 SNES Function norm 1.712732074788e+06 >> 3 SNES Function norm 1.002169173269e+04 >> 4 SNES Function norm 1.655878303433e+03 >> 5 SNES Function norm 3.746498305706e+02 >> 6 SNES Function norm 8.317435704773e+01 >> 7 SNES Function norm 1.857639969641e+01 >> 8 SNES Function norm 4.149691057773e+00 >> 9 SNES Function norm 9.265604042412e-01 >> 10 SNES Function norm 2.069527103214e-01 >> 11 SNES Function norm 4.624186491082e-02 >> 12 SNES Function norm 1.035558432688e-02 >> 13 SNES Function norm 2.341362958811e-03 >> 14 SNES Function norm 5.507445427277e-04 >> 15 SNES Function norm 1.485123568354e-04 >> 16 SNES Function norm 5.180043781814e-05 >> 17 SNES Function norm 2.341966514486e-05 >> 18 SNES Function norm 1.344936158651e-05 >> 19 SNES Function norm 1.054812641176e-05 >> Number of Newton iterations = 19 >> Converged reason is 4 >> >> It looks like the iterate never falls into a quadratic convergence region >> before it converges. Is there any hint to understand this behavior? >> >> Thanks a lot, >> >> Yan >> >> >> >> >> >> >> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Sun Jan 10 18:25:18 2010 From: bsmith at mcs.anl.gov (Barry Smith) Date: Sun, 10 Jan 2010 18:25:18 -0600 Subject: [petsc-users] Snes behavior In-Reply-To: References: <0F96DD2E-AAB2-4D43-A985-33C043CA319F@mcs.anl.gov> Message-ID: On Jan 10, 2010, at 6:09 PM, Ryan Yan wrote: > Hi Barry, > I run the code on another machine. > > yy2250 at sci-m8n6 ~/test/1d $ srun -p sci-comp -N 2 -n 2 ./ > HeatProfile1D -snes_type test > out.snes_test > [0]PETSC ERROR: SNESSolve() line 2221 in src/snes/interface/snes.c > [1]PETSC ERROR: SNESSolve() line 2221 in src/snes/interface/snes.c > [0]PETSC ERROR: DMMGSolveSNES() line 510 in src/snes/utils/damgsnes.c > [0]PETSC ERROR: DMMGSolve() line 372 in src/snes/utils/damg.c > [0]PETSC ERROR: main() line 270 in /home/vyan2000/local/PPETSc/ > petsc-2.3.3-p15/src/ksp/ksp/examples/tutorials/ > NCprojectHeatProfile1D.c > application called MPI_Abort(MPI_COMM_WORLD, 73) - process 0 > [1]PETSC ERROR: DMMGSolveSNES() line 510 in src/snes/utils/damgsnes.c > [1]PETSC ERROR: DMMGSolve() line 372 in src/snes/utils/damg.c > [1]PETSC ERROR: main() line 270 in /home/vyan2000/local/PPETSc/ > petsc-2.3.3-p15/src/ksp/ksp/examples/tutorials/ > NCprojectHeatProfile1D.c > In: PMI_Abort(73, application called MPI_Abort(MPI_COMM_WORLD, 73) - > process 0) > application called MPI_Abort(MPI_COMM_WORLD, 73) - process 1 > In: PMI_Abort(73, application called MPI_Abort(MPI_COMM_WORLD, 73) - > process 1) > srun: error: task 1: Exited with exit code 73 > srun: error: task 0: Exited with exit code 73 > > I was using 3 level grids. The error comes out during the snes > solving on the fine level grid. The output in the file out.snes_test > is : > yy2250 at sci-m8n6 ~/test/1d $ cat out.snes_test > Testing hand-coded Jacobian, if the ratio is > O(1.e-8), the hand-coded Jacobian is probably correct. > Run with -snes_test_display to show difference > of hand-coded and finite difference Jacobian. > Norm of matrix ratio 1.21314e-10 difference 1.41447 > Norm of matrix ratio 4.30115e-06 difference 1.41427 > Norm of matrix ratio 4.30108e-06 difference 1.41427 > > Ok, Jacobians being used are probably ok. > > For the -snes_mf_operator, > yy2250 at sci-m8n6 ~/test/1d $ srun -p sci-comp -N 2 -n 2 ./ > HeatProfile1D -snes_mf_operator > the program is still running. I suspect that the program hanged. It's not hanging. It is just running. Use -snes_monitor -ksp_rtol 1.e-12 -ksp_converged_reason -snes_converged_reason and run again Barry > > Is there any useful information in the file "out.snes_test"? > > BTW, I did not hand-code any Jacobian, since I passed 0 for the > Jacobian evaluation subroutine in the DMMGSetSNESLocal(). > > Thank you very much, > > Yan > > > > On Sun, Jan 10, 2010 at 6:11 PM, Barry Smith > wrote: > > That message ain't from PETSc. Something is likely killing the job. > > Barry > > > On Jan 10, 2010, at 5:09 PM, Ryan Yan wrote: > > Hi Barry, > I got the following result: > > vyan2000 at vyan2000-linux ~/NCproject/general $ mpirun -np 1 ./ > HeatProfile1D -snes_mf_operator > Alarm clock > vyan2000 at vyan2000-linux ~/NCproject/general $ mpirun -np 1 ./ > HeatProfile1D -snes_type test > Alarm clock > > Are they normal responses and what do they indicate? > > Thanks a lot, > > Yan > > > On Sun, Jan 10, 2010 at 5:57 PM, Barry Smith > wrote: > > JUST run with -snes_mf_operator and then with -snes_type test NOT > together, > > Barry > > > On Jan 10, 2010, at 4:42 PM, Ryan Yan wrote: > > Hi Barry, > Please see reply below, > > On Sun, Jan 10, 2010 at 4:57 PM, Barry Smith > wrote: > > On Jan 10, 2010, at 3:52 PM, Ryan Yan wrote: > > Hi Barry, > Yes, exactly. The original multi-components system scale quite > unevenly. I will try to rescale it. > Could this be helpful to show some promise on quadratic convergence? > > I won't be concerned about "quadratic convergence" I'd only be > concerned that it is converging to the correct answer and that you > are getting close enough to the correct answer. > > Yes, I agree. > > You can run with -snes_mf_operator and -snes_type test to verify if > the Jacobian being computed is accurate. Perhaps in your function > evaluation you are not using the stencil that you set with the DA, > this would cause the wrong Jacobian to be computed. > > > After passing in -snes_mf_operator and -snes_type test as follows: > vyan2000 at vyan2000-linux ~/NCproject/general : mpirun -np 2 ./ > HeatProfile1D -snes_mf_operator -snes_type test > I got errors, as expected: > > [0]PETSC ERROR: --------------------- Error Message > ------------------------------------ > [0]PETSC ERROR: Invalid argument! > [0]PETSC ERROR: Cannot test with alternative preconditioner! > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: Petsc Release Version 3.0.0, Patch 5, Mon Apr 13 > 09:15:37 CDT 2009 > [0]PETSC ERROR: See docs/changes/index.html for recent updates. > [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. > [0]PETSC ERROR: See docs/index.html for manual pages. > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: ./HeatProfile1D on a O-hypre-p named vyan2000-linux > by vyan2000 Sun Jan 10 17:16:00 2010 > [0]PETSC ERROR: Libraries linked from /home/vyan2000/local/PPETSc/ > petsc-3.0.0-p5/O-hypre-prometheus/lib > [0]PETSC ERROR: Configure run at Thu Jun 25 13:49:36 2009 > [0]PETSC ERROR: Configure options --download-mpich=1 --with- > debugger=gdb --download-hypre=1 --download-parmetis=1 --download- > prometheus=1 --with-shared=0 > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: SNESSolve_Test() line 28 in src/snes/impls/test/ > snestest.c > [0]PETSC ERROR: SNESSolve() line 2221 in src/snes/interface/snes.c > [0]PETSC ERROR: DMMGSolveSNES() line 510 in src/snes/utils/damgsnes.c > [0]PETSC ERROR: DMMGSolve() line 372 in src/snes/utils/damg.c > [0]PETSC ERROR: main() line 270 in /home/vyan2000/local/PPETSc/ > petsc-2.3.3-p15/src/ksp/ksp/examples/tutorials/ > NCprojectHeatProfile1D.c > application called MPI_Abort(MPI_COMM_WORLD, 62) - process 0[cli_0]: > aborting job: > application called MPI_Abort(MPI_COMM_WORLD, 62) - process 0 > > The error makes sense, since I did not pass in the analytical > Jacobian for the preconditioner matrix. In stead, I was using > DMMGSetSNESLocal(dmmg,FormFunctionLocal, > 0,ad_FormFunctionLocal,admf_FormFunctionLocal). I am going to change > the code a bit, pass in the analytical Jacobian and do the - > snes_type test. > > I will bear your suggestion in mind during the test. > > Thanks a lot, > > Yan > > > > Barry > > > > Thanks a lot, > > Yan > > On Sun, Jan 10, 2010 at 4:35 PM, Barry Smith > wrote: > > You already got a 10^16 drop in the residual norm. It is not > realistic to expect to get much more than that for double precision > calculations. Perhaps your original F() has some funky scaling of > different components that you can fix. > > > > Barry > > > On Jan 10, 2010, at 2:55 PM, Ryan Yan wrote: > > Hi All, > I am solving a nonlinear system using snes. The -snes_monitor option > has the following output: > > 0 SNES Function norm 2.640163923729e+09 > 1 SNES Function norm 1.047643565314e+08 > 2 SNES Function norm 1.712732074788e+06 > 3 SNES Function norm 1.002169173269e+04 > 4 SNES Function norm 1.655878303433e+03 > 5 SNES Function norm 3.746498305706e+02 > 6 SNES Function norm 8.317435704773e+01 > 7 SNES Function norm 1.857639969641e+01 > 8 SNES Function norm 4.149691057773e+00 > 9 SNES Function norm 9.265604042412e-01 > 10 SNES Function norm 2.069527103214e-01 > 11 SNES Function norm 4.624186491082e-02 > 12 SNES Function norm 1.035558432688e-02 > 13 SNES Function norm 2.341362958811e-03 > 14 SNES Function norm 5.507445427277e-04 > 15 SNES Function norm 1.485123568354e-04 > 16 SNES Function norm 5.180043781814e-05 > 17 SNES Function norm 2.341966514486e-05 > 18 SNES Function norm 1.344936158651e-05 > 19 SNES Function norm 1.054812641176e-05 > Number of Newton iterations = 19 > Converged reason is 4 > > It looks like the iterate never falls into a quadratic convergence > region before it converges. Is there any hint to understand this > behavior? > > Thanks a lot, > > Yan > > > > > > > > > > From vyan2000 at gmail.com Sun Jan 10 21:13:28 2010 From: vyan2000 at gmail.com (Ryan Yan) Date: Sun, 10 Jan 2010 22:13:28 -0500 Subject: [petsc-users] Snes behavior In-Reply-To: References: <0F96DD2E-AAB2-4D43-A985-33C043CA319F@mcs.anl.gov> Message-ID: Hi Barry, Here is the result for -snes_mf_operator: yy2250 at sci-m8n6 ~/test/1d $ srun -p sci-comp -N 2 -n 2 ./HeatProfile1D -snes_mf_operator -dmmg_grid_sequence -pc_type bjacobi -snes_rtol 1e-15 -snes_monitor -ksp_rtol 1.e-12 -ksp_converged_reason -snes_converged_reason > out.mf_operator & yy2250 at sci-m8n6 ~/test/1d $ cat out.mf_operator 0 SNES Function norm 1.411468156752e+08 Linear solve converged due to CONVERGED_ITS iterations 1 1 SNES Function norm 1.396727866424e+08 Linear solve converged due to CONVERGED_ITS iterations 1 Nonlinear solve did not converge due to DIVERGED_LS_FAILURE 0 SNES Function norm 1.045985796073e+08 Linear solve converged due to CONVERGED_RTOL iterations 44 1 SNES Function norm 7.912999174650e+07 Linear solve converged due to CONVERGED_RTOL iterations 38 2 SNES Function norm 6.079225520436e+07 Linear solve converged due to CONVERGED_RTOL iterations 66 3 SNES Function norm 4.610252725173e+07 Linear solve converged due to CONVERGED_RTOL iterations 66 4 SNES Function norm 3.333587574896e+07 Linear solve converged due to CONVERGED_RTOL iterations 46 5 SNES Function norm 2.326587775240e+07 Linear solve converged due to CONVERGED_RTOL iterations 59 6 SNES Function norm 1.233942497170e+05 Linear solve converged due to CONVERGED_RTOL iterations 38 7 SNES Function norm 2.309536978272e+03 Linear solve converged due to CONVERGED_RTOL iterations 38 8 SNES Function norm 7.064055492974e-03 Linear solve converged due to CONVERGED_RTOL iterations 38 9 SNES Function norm 3.419662119536e-06 Nonlinear solve converged due to CONVERGED_PNORM_RELATIVE 0 SNES Function norm 3.420486720202e+09 Linear solve converged due to CONVERGED_RTOL iterations 10 1 SNES Function norm 1.426355492941e+08 Linear solve converged due to CONVERGED_RTOL iterations 36 2 SNES Function norm 1.429830907344e+06 Linear solve converged due to CONVERGED_RTOL iterations 37 3 SNES Function norm 1.755275678702e+03 Linear solve converged due to CONVERGED_RTOL iterations 37 4 SNES Function norm 1.216161531652e-03 Linear solve converged due to CONVERGED_RTOL iterations 36 5 SNES Function norm 4.309776935609e-06 Nonlinear solve converged due to CONVERGED_PNORM_RELATIVE Number of Newton iterations = 5 Converged reason is 4 This time the convergence is very aggressive, and it is quadratic from iteration 0 to iteration 4. The solution vector x from this run is the same as the run without the option "-snes_mf_operator". Any hint for why this is totally quadratic convergence? Thank you very much, Yan On Sun, Jan 10, 2010 at 7:25 PM, Barry Smith wrote: > > On Jan 10, 2010, at 6:09 PM, Ryan Yan wrote: > > Hi Barry, >> >> I run the code on another machine. >> >> yy2250 at sci-m8n6 ~/test/1d $ srun -p sci-comp -N 2 -n 2 ./HeatProfile1D >> -snes_type test > out.snes_test >> [0]PETSC ERROR: SNESSolve() line 2221 in src/snes/interface/snes.c >> [1]PETSC ERROR: SNESSolve() line 2221 in src/snes/interface/snes.c >> [0]PETSC ERROR: DMMGSolveSNES() line 510 in src/snes/utils/damgsnes.c >> [0]PETSC ERROR: DMMGSolve() line 372 in src/snes/utils/damg.c >> [0]PETSC ERROR: main() line 270 in >> /home/vyan2000/local/PPETSc/petsc-2.3.3-p15/src/ksp/ksp/examples/tutorials/NCprojectHeatProfile1D.c >> application called MPI_Abort(MPI_COMM_WORLD, 73) - process 0 >> [1]PETSC ERROR: DMMGSolveSNES() line 510 in src/snes/utils/damgsnes.c >> [1]PETSC ERROR: DMMGSolve() line 372 in src/snes/utils/damg.c >> [1]PETSC ERROR: main() line 270 in >> /home/vyan2000/local/PPETSc/petsc-2.3.3-p15/src/ksp/ksp/examples/tutorials/NCprojectHeatProfile1D.c >> In: PMI_Abort(73, application called MPI_Abort(MPI_COMM_WORLD, 73) - >> process 0) >> application called MPI_Abort(MPI_COMM_WORLD, 73) - process 1 >> In: PMI_Abort(73, application called MPI_Abort(MPI_COMM_WORLD, 73) - >> process 1) >> srun: error: task 1: Exited with exit code 73 >> srun: error: task 0: Exited with exit code 73 >> >> I was using 3 level grids. The error comes out during the snes solving on >> the fine level grid. The output in the file out.snes_test is : >> yy2250 at sci-m8n6 ~/test/1d $ cat out.snes_test >> Testing hand-coded Jacobian, if the ratio is >> O(1.e-8), the hand-coded Jacobian is probably correct. >> Run with -snes_test_display to show difference >> of hand-coded and finite difference Jacobian. >> Norm of matrix ratio 1.21314e-10 difference 1.41447 >> Norm of matrix ratio 4.30115e-06 difference 1.41427 >> Norm of matrix ratio 4.30108e-06 difference 1.41427 >> >> >> Ok, Jacobians being used are probably ok. > > > >> For the -snes_mf_operator, >> yy2250 at sci-m8n6 ~/test/1d $ srun -p sci-comp -N 2 -n 2 ./HeatProfile1D >> -snes_mf_operator >> the program is still running. I suspect that the program hanged. >> > > It's not hanging. It is just running. Use -snes_monitor -ksp_rtol 1.e-12 > -ksp_converged_reason -snes_converged_reason and run again > > Barry > > > >> Is there any useful information in the file "out.snes_test"? >> >> BTW, I did not hand-code any Jacobian, since I passed 0 for the Jacobian >> evaluation subroutine in the DMMGSetSNESLocal(). >> >> Thank you very much, >> >> Yan >> >> >> >> On Sun, Jan 10, 2010 at 6:11 PM, Barry Smith wrote: >> >> That message ain't from PETSc. Something is likely killing the job. >> >> Barry >> >> >> On Jan 10, 2010, at 5:09 PM, Ryan Yan wrote: >> >> Hi Barry, >> I got the following result: >> >> vyan2000 at vyan2000-linux ~/NCproject/general $ mpirun -np 1 >> ./HeatProfile1D -snes_mf_operator >> Alarm clock >> vyan2000 at vyan2000-linux ~/NCproject/general $ mpirun -np 1 >> ./HeatProfile1D -snes_type test >> Alarm clock >> >> Are they normal responses and what do they indicate? >> >> Thanks a lot, >> >> Yan >> >> >> On Sun, Jan 10, 2010 at 5:57 PM, Barry Smith wrote: >> >> JUST run with -snes_mf_operator and then with -snes_type test NOT >> together, >> >> Barry >> >> >> On Jan 10, 2010, at 4:42 PM, Ryan Yan wrote: >> >> Hi Barry, >> Please see reply below, >> >> On Sun, Jan 10, 2010 at 4:57 PM, Barry Smith wrote: >> >> On Jan 10, 2010, at 3:52 PM, Ryan Yan wrote: >> >> Hi Barry, >> Yes, exactly. The original multi-components system scale quite unevenly. I >> will try to rescale it. >> Could this be helpful to show some promise on quadratic convergence? >> >> I won't be concerned about "quadratic convergence" I'd only be concerned >> that it is converging to the correct answer and that you are getting close >> enough to the correct answer. >> >> Yes, I agree. >> >> You can run with -snes_mf_operator and -snes_type test to verify if the >> Jacobian being computed is accurate. Perhaps in your function evaluation you >> are not using the stencil that you set with the DA, this would cause the >> wrong Jacobian to be computed. >> >> >> After passing in -snes_mf_operator and -snes_type test as follows: >> vyan2000 at vyan2000-linux ~/NCproject/general : mpirun -np 2 >> ./HeatProfile1D -snes_mf_operator -snes_type test >> I got errors, as expected: >> >> [0]PETSC ERROR: --------------------- Error Message >> ------------------------------------ >> [0]PETSC ERROR: Invalid argument! >> [0]PETSC ERROR: Cannot test with alternative preconditioner! >> [0]PETSC ERROR: >> ------------------------------------------------------------------------ >> [0]PETSC ERROR: Petsc Release Version 3.0.0, Patch 5, Mon Apr 13 09:15:37 >> CDT 2009 >> [0]PETSC ERROR: See docs/changes/index.html for recent updates. >> [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. >> [0]PETSC ERROR: See docs/index.html for manual pages. >> [0]PETSC ERROR: >> ------------------------------------------------------------------------ >> [0]PETSC ERROR: ./HeatProfile1D on a O-hypre-p named vyan2000-linux by >> vyan2000 Sun Jan 10 17:16:00 2010 >> [0]PETSC ERROR: Libraries linked from >> /home/vyan2000/local/PPETSc/petsc-3.0.0-p5/O-hypre-prometheus/lib >> [0]PETSC ERROR: Configure run at Thu Jun 25 13:49:36 2009 >> [0]PETSC ERROR: Configure options --download-mpich=1 --with-debugger=gdb >> --download-hypre=1 --download-parmetis=1 --download-prometheus=1 >> --with-shared=0 >> [0]PETSC ERROR: >> ------------------------------------------------------------------------ >> [0]PETSC ERROR: SNESSolve_Test() line 28 in src/snes/impls/test/snestest.c >> [0]PETSC ERROR: SNESSolve() line 2221 in src/snes/interface/snes.c >> [0]PETSC ERROR: DMMGSolveSNES() line 510 in src/snes/utils/damgsnes.c >> [0]PETSC ERROR: DMMGSolve() line 372 in src/snes/utils/damg.c >> [0]PETSC ERROR: main() line 270 in >> /home/vyan2000/local/PPETSc/petsc-2.3.3-p15/src/ksp/ksp/examples/tutorials/NCprojectHeatProfile1D.c >> application called MPI_Abort(MPI_COMM_WORLD, 62) - process 0[cli_0]: >> aborting job: >> application called MPI_Abort(MPI_COMM_WORLD, 62) - process 0 >> >> The error makes sense, since I did not pass in the analytical Jacobian for >> the preconditioner matrix. In stead, I was using >> DMMGSetSNESLocal(dmmg,FormFunctionLocal,0,ad_FormFunctionLocal,admf_FormFunctionLocal). >> I am going to change the code a bit, pass in the analytical Jacobian and do >> the -snes_type test. >> >> I will bear your suggestion in mind during the test. >> >> Thanks a lot, >> >> Yan >> >> >> >> Barry >> >> >> >> Thanks a lot, >> >> Yan >> >> On Sun, Jan 10, 2010 at 4:35 PM, Barry Smith wrote: >> >> You already got a 10^16 drop in the residual norm. It is not realistic to >> expect to get much more than that for double precision calculations. Perhaps >> your original F() has some funky scaling of different components that you >> can fix. >> >> >> >> Barry >> >> >> On Jan 10, 2010, at 2:55 PM, Ryan Yan wrote: >> >> Hi All, >> I am solving a nonlinear system using snes. The -snes_monitor option has >> the following output: >> >> 0 SNES Function norm 2.640163923729e+09 >> 1 SNES Function norm 1.047643565314e+08 >> 2 SNES Function norm 1.712732074788e+06 >> 3 SNES Function norm 1.002169173269e+04 >> 4 SNES Function norm 1.655878303433e+03 >> 5 SNES Function norm 3.746498305706e+02 >> 6 SNES Function norm 8.317435704773e+01 >> 7 SNES Function norm 1.857639969641e+01 >> 8 SNES Function norm 4.149691057773e+00 >> 9 SNES Function norm 9.265604042412e-01 >> 10 SNES Function norm 2.069527103214e-01 >> 11 SNES Function norm 4.624186491082e-02 >> 12 SNES Function norm 1.035558432688e-02 >> 13 SNES Function norm 2.341362958811e-03 >> 14 SNES Function norm 5.507445427277e-04 >> 15 SNES Function norm 1.485123568354e-04 >> 16 SNES Function norm 5.180043781814e-05 >> 17 SNES Function norm 2.341966514486e-05 >> 18 SNES Function norm 1.344936158651e-05 >> 19 SNES Function norm 1.054812641176e-05 >> Number of Newton iterations = 19 >> Converged reason is 4 >> >> It looks like the iterate never falls into a quadratic convergence region >> before it converges. Is there any hint to understand this behavior? >> >> Thanks a lot, >> >> Yan >> >> >> >> >> >> >> >> >> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Sun Jan 10 22:22:00 2010 From: bsmith at mcs.anl.gov (Barry Smith) Date: Sun, 10 Jan 2010 22:22:00 -0600 Subject: [petsc-users] Snes behavior In-Reply-To: References: <0F96DD2E-AAB2-4D43-A985-33C043CA319F@mcs.anl.gov> Message-ID: <18837D43-E96A-4C5A-B3E4-2C6DAB896DFC@mcs.anl.gov> Like I said in my previous mail this means that the Jacobian being computing before was not accurate enough for some reason. This will happen if your function evaluation does NOT respect the stencil you provided when you created the DA. For example if you told the DA to use a stencil width of 1 but somewhere your function evaluation grabs a value 2 cells away and uses it in the computation. For example on some boundary condition. Or you told it was a box stencil but the function evaluation grabbed a value from the diagonal stencil. There is a 99% probability this is your problem. If you cannot find the bug then send your function evaluation and how you form the DA to petsc-maint at mcs.anl.gov and we'll find it. Barry On Jan 10, 2010, at 9:13 PM, Ryan Yan wrote: > Hi Barry, > Here is the result for -snes_mf_operator: > > yy2250 at sci-m8n6 ~/test/1d $ srun -p sci-comp -N 2 -n 2 ./ > HeatProfile1D -snes_mf_operator -dmmg_grid_sequence -pc_type bjacobi > -snes_rtol 1e-15 -snes_monitor -ksp_rtol 1.e-12 - > ksp_converged_reason -snes_converged_reason > out.mf_operator & > > > yy2250 at sci-m8n6 ~/test/1d $ cat out.mf_operator > 0 SNES Function norm 1.411468156752e+08 > Linear solve converged due to CONVERGED_ITS iterations 1 > 1 SNES Function norm 1.396727866424e+08 > Linear solve converged due to CONVERGED_ITS iterations 1 > Nonlinear solve did not converge due to DIVERGED_LS_FAILURE > 0 SNES Function norm 1.045985796073e+08 > Linear solve converged due to CONVERGED_RTOL iterations 44 > 1 SNES Function norm 7.912999174650e+07 > Linear solve converged due to CONVERGED_RTOL iterations 38 > 2 SNES Function norm 6.079225520436e+07 > Linear solve converged due to CONVERGED_RTOL iterations 66 > 3 SNES Function norm 4.610252725173e+07 > Linear solve converged due to CONVERGED_RTOL iterations 66 > 4 SNES Function norm 3.333587574896e+07 > Linear solve converged due to CONVERGED_RTOL iterations 46 > 5 SNES Function norm 2.326587775240e+07 > Linear solve converged due to CONVERGED_RTOL iterations 59 > 6 SNES Function norm 1.233942497170e+05 > Linear solve converged due to CONVERGED_RTOL iterations 38 > 7 SNES Function norm 2.309536978272e+03 > Linear solve converged due to CONVERGED_RTOL iterations 38 > 8 SNES Function norm 7.064055492974e-03 > Linear solve converged due to CONVERGED_RTOL iterations 38 > 9 SNES Function norm 3.419662119536e-06 > Nonlinear solve converged due to CONVERGED_PNORM_RELATIVE > 0 SNES Function norm 3.420486720202e+09 > Linear solve converged due to CONVERGED_RTOL iterations 10 > 1 SNES Function norm 1.426355492941e+08 > Linear solve converged due to CONVERGED_RTOL iterations 36 > 2 SNES Function norm 1.429830907344e+06 > Linear solve converged due to CONVERGED_RTOL iterations 37 > 3 SNES Function norm 1.755275678702e+03 > Linear solve converged due to CONVERGED_RTOL iterations 37 > 4 SNES Function norm 1.216161531652e-03 > Linear solve converged due to CONVERGED_RTOL iterations 36 > 5 SNES Function norm 4.309776935609e-06 > Nonlinear solve converged due to CONVERGED_PNORM_RELATIVE > Number of Newton iterations = 5 > Converged reason is 4 > > This time the convergence is very aggressive, and it is quadratic > from iteration 0 to iteration 4. > > The solution vector x from this run is the same as the run without > the option "-snes_mf_operator". Any hint for why this is totally > quadratic convergence? > > Thank you very much, > > Yan > > > On Sun, Jan 10, 2010 at 7:25 PM, Barry Smith > wrote: > > On Jan 10, 2010, at 6:09 PM, Ryan Yan wrote: > > Hi Barry, > > I run the code on another machine. > > yy2250 at sci-m8n6 ~/test/1d $ srun -p sci-comp -N 2 -n 2 ./ > HeatProfile1D -snes_type test > out.snes_test > [0]PETSC ERROR: SNESSolve() line 2221 in src/snes/interface/snes.c > [1]PETSC ERROR: SNESSolve() line 2221 in src/snes/interface/snes.c > [0]PETSC ERROR: DMMGSolveSNES() line 510 in src/snes/utils/damgsnes.c > [0]PETSC ERROR: DMMGSolve() line 372 in src/snes/utils/damg.c > [0]PETSC ERROR: main() line 270 in /home/vyan2000/local/PPETSc/ > petsc-2.3.3-p15/src/ksp/ksp/examples/tutorials/ > NCprojectHeatProfile1D.c > application called MPI_Abort(MPI_COMM_WORLD, 73) - process 0 > [1]PETSC ERROR: DMMGSolveSNES() line 510 in src/snes/utils/damgsnes.c > [1]PETSC ERROR: DMMGSolve() line 372 in src/snes/utils/damg.c > [1]PETSC ERROR: main() line 270 in /home/vyan2000/local/PPETSc/ > petsc-2.3.3-p15/src/ksp/ksp/examples/tutorials/ > NCprojectHeatProfile1D.c > In: PMI_Abort(73, application called MPI_Abort(MPI_COMM_WORLD, 73) - > process 0) > application called MPI_Abort(MPI_COMM_WORLD, 73) - process 1 > In: PMI_Abort(73, application called MPI_Abort(MPI_COMM_WORLD, 73) - > process 1) > srun: error: task 1: Exited with exit code 73 > srun: error: task 0: Exited with exit code 73 > > I was using 3 level grids. The error comes out during the snes > solving on the fine level grid. The output in the file out.snes_test > is : > yy2250 at sci-m8n6 ~/test/1d $ cat out.snes_test > Testing hand-coded Jacobian, if the ratio is > O(1.e-8), the hand-coded Jacobian is probably correct. > Run with -snes_test_display to show difference > of hand-coded and finite difference Jacobian. > Norm of matrix ratio 1.21314e-10 difference 1.41447 > Norm of matrix ratio 4.30115e-06 difference 1.41427 > Norm of matrix ratio 4.30108e-06 difference 1.41427 > > > Ok, Jacobians being used are probably ok. > > > > For the -snes_mf_operator, > yy2250 at sci-m8n6 ~/test/1d $ srun -p sci-comp -N 2 -n 2 ./ > HeatProfile1D -snes_mf_operator > the program is still running. I suspect that the program hanged. > > It's not hanging. It is just running. Use -snes_monitor -ksp_rtol > 1.e-12 -ksp_converged_reason -snes_converged_reason and run again > > Barry > > > > Is there any useful information in the file "out.snes_test"? > > BTW, I did not hand-code any Jacobian, since I passed 0 for the > Jacobian evaluation subroutine in the DMMGSetSNESLocal(). > > Thank you very much, > > Yan > > > > On Sun, Jan 10, 2010 at 6:11 PM, Barry Smith > wrote: > > That message ain't from PETSc. Something is likely killing the job. > > Barry > > > On Jan 10, 2010, at 5:09 PM, Ryan Yan wrote: > > Hi Barry, > I got the following result: > > vyan2000 at vyan2000-linux ~/NCproject/general $ mpirun -np 1 ./ > HeatProfile1D -snes_mf_operator > Alarm clock > vyan2000 at vyan2000-linux ~/NCproject/general $ mpirun -np 1 ./ > HeatProfile1D -snes_type test > Alarm clock > > Are they normal responses and what do they indicate? > > Thanks a lot, > > Yan > > > On Sun, Jan 10, 2010 at 5:57 PM, Barry Smith > wrote: > > JUST run with -snes_mf_operator and then with -snes_type test NOT > together, > > Barry > > > On Jan 10, 2010, at 4:42 PM, Ryan Yan wrote: > > Hi Barry, > Please see reply below, > > On Sun, Jan 10, 2010 at 4:57 PM, Barry Smith > wrote: > > On Jan 10, 2010, at 3:52 PM, Ryan Yan wrote: > > Hi Barry, > Yes, exactly. The original multi-components system scale quite > unevenly. I will try to rescale it. > Could this be helpful to show some promise on quadratic convergence? > > I won't be concerned about "quadratic convergence" I'd only be > concerned that it is converging to the correct answer and that you > are getting close enough to the correct answer. > > Yes, I agree. > > You can run with -snes_mf_operator and -snes_type test to verify if > the Jacobian being computed is accurate. Perhaps in your function > evaluation you are not using the stencil that you set with the DA, > this would cause the wrong Jacobian to be computed. > > > After passing in -snes_mf_operator and -snes_type test as follows: > vyan2000 at vyan2000-linux ~/NCproject/general : mpirun -np 2 ./ > HeatProfile1D -snes_mf_operator -snes_type test > I got errors, as expected: > > [0]PETSC ERROR: --------------------- Error Message > ------------------------------------ > [0]PETSC ERROR: Invalid argument! > [0]PETSC ERROR: Cannot test with alternative preconditioner! > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: Petsc Release Version 3.0.0, Patch 5, Mon Apr 13 > 09:15:37 CDT 2009 > [0]PETSC ERROR: See docs/changes/index.html for recent updates. > [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. > [0]PETSC ERROR: See docs/index.html for manual pages. > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: ./HeatProfile1D on a O-hypre-p named vyan2000-linux > by vyan2000 Sun Jan 10 17:16:00 2010 > [0]PETSC ERROR: Libraries linked from /home/vyan2000/local/PPETSc/ > petsc-3.0.0-p5/O-hypre-prometheus/lib > [0]PETSC ERROR: Configure run at Thu Jun 25 13:49:36 2009 > [0]PETSC ERROR: Configure options --download-mpich=1 --with- > debugger=gdb --download-hypre=1 --download-parmetis=1 --download- > prometheus=1 --with-shared=0 > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: SNESSolve_Test() line 28 in src/snes/impls/test/ > snestest.c > [0]PETSC ERROR: SNESSolve() line 2221 in src/snes/interface/snes.c > [0]PETSC ERROR: DMMGSolveSNES() line 510 in src/snes/utils/damgsnes.c > [0]PETSC ERROR: DMMGSolve() line 372 in src/snes/utils/damg.c > [0]PETSC ERROR: main() line 270 in /home/vyan2000/local/PPETSc/ > petsc-2.3.3-p15/src/ksp/ksp/examples/tutorials/ > NCprojectHeatProfile1D.c > application called MPI_Abort(MPI_COMM_WORLD, 62) - process 0[cli_0]: > aborting job: > application called MPI_Abort(MPI_COMM_WORLD, 62) - process 0 > > The error makes sense, since I did not pass in the analytical > Jacobian for the preconditioner matrix. In stead, I was using > DMMGSetSNESLocal(dmmg,FormFunctionLocal, > 0,ad_FormFunctionLocal,admf_FormFunctionLocal). I am going to change > the code a bit, pass in the analytical Jacobian and do the - > snes_type test. > > I will bear your suggestion in mind during the test. > > Thanks a lot, > > Yan > > > > Barry > > > > Thanks a lot, > > Yan > > On Sun, Jan 10, 2010 at 4:35 PM, Barry Smith > wrote: > > You already got a 10^16 drop in the residual norm. It is not > realistic to expect to get much more than that for double precision > calculations. Perhaps your original F() has some funky scaling of > different components that you can fix. > > > > Barry > > > On Jan 10, 2010, at 2:55 PM, Ryan Yan wrote: > > Hi All, > I am solving a nonlinear system using snes. The -snes_monitor option > has the following output: > > 0 SNES Function norm 2.640163923729e+09 > 1 SNES Function norm 1.047643565314e+08 > 2 SNES Function norm 1.712732074788e+06 > 3 SNES Function norm 1.002169173269e+04 > 4 SNES Function norm 1.655878303433e+03 > 5 SNES Function norm 3.746498305706e+02 > 6 SNES Function norm 8.317435704773e+01 > 7 SNES Function norm 1.857639969641e+01 > 8 SNES Function norm 4.149691057773e+00 > 9 SNES Function norm 9.265604042412e-01 > 10 SNES Function norm 2.069527103214e-01 > 11 SNES Function norm 4.624186491082e-02 > 12 SNES Function norm 1.035558432688e-02 > 13 SNES Function norm 2.341362958811e-03 > 14 SNES Function norm 5.507445427277e-04 > 15 SNES Function norm 1.485123568354e-04 > 16 SNES Function norm 5.180043781814e-05 > 17 SNES Function norm 2.341966514486e-05 > 18 SNES Function norm 1.344936158651e-05 > 19 SNES Function norm 1.054812641176e-05 > Number of Newton iterations = 19 > Converged reason is 4 > > It looks like the iterate never falls into a quadratic convergence > region before it converges. Is there any hint to understand this > behavior? > > Thanks a lot, > > Yan > > > > > > > > > > > > From vyan2000 at gmail.com Mon Jan 11 09:32:18 2010 From: vyan2000 at gmail.com (Ryan Yan) Date: Mon, 11 Jan 2010 10:32:18 -0500 Subject: [petsc-users] Snes behavior In-Reply-To: <18837D43-E96A-4C5A-B3E4-2C6DAB896DFC@mcs.anl.gov> References: <18837D43-E96A-4C5A-B3E4-2C6DAB896DFC@mcs.anl.gov> Message-ID: Hi Barry, This is my case: In 1-d, for the interior pts the stencil width is 1, but for boundary pts the stencil width is 2. Should the stencil width in DACreate1d be 1 or 2? If I switch to stencil width 2, then I can get quadratic convergence. Thank you very much, Yan On Sun, Jan 10, 2010 at 11:22 PM, Barry Smith wrote: > > Like I said in my previous mail this means that the Jacobian being > computing before was not accurate enough for some reason. This will happen > if your function evaluation does NOT respect the stencil you provided when > you created the DA. For example if you told the DA to use a stencil width of > 1 but somewhere your function evaluation grabs a value 2 cells away and uses > it in the computation. For example on some boundary condition. Or you told > it was a box stencil but the function evaluation grabbed a value from the > diagonal stencil. > There is a 99% probability this is your problem. > > If you cannot find the bug then send your function evaluation and how you > form the DA to petsc-maint at mcs.anl.gov and we'll find it. > > > Barry > > > > On Jan 10, 2010, at 9:13 PM, Ryan Yan wrote: > > Hi Barry, >> Here is the result for -snes_mf_operator: >> >> yy2250 at sci-m8n6 ~/test/1d $ srun -p sci-comp -N 2 -n 2 ./HeatProfile1D >> -snes_mf_operator -dmmg_grid_sequence -pc_type bjacobi -snes_rtol 1e-15 >> -snes_monitor -ksp_rtol 1.e-12 -ksp_converged_reason -snes_converged_reason >> > out.mf_operator & >> >> >> yy2250 at sci-m8n6 ~/test/1d $ cat out.mf_operator >> 0 SNES Function norm 1.411468156752e+08 >> Linear solve converged due to CONVERGED_ITS iterations 1 >> 1 SNES Function norm 1.396727866424e+08 >> Linear solve converged due to CONVERGED_ITS iterations 1 >> Nonlinear solve did not converge due to DIVERGED_LS_FAILURE >> 0 SNES Function norm 1.045985796073e+08 >> Linear solve converged due to CONVERGED_RTOL iterations 44 >> 1 SNES Function norm 7.912999174650e+07 >> Linear solve converged due to CONVERGED_RTOL iterations 38 >> 2 SNES Function norm 6.079225520436e+07 >> Linear solve converged due to CONVERGED_RTOL iterations 66 >> 3 SNES Function norm 4.610252725173e+07 >> Linear solve converged due to CONVERGED_RTOL iterations 66 >> 4 SNES Function norm 3.333587574896e+07 >> Linear solve converged due to CONVERGED_RTOL iterations 46 >> 5 SNES Function norm 2.326587775240e+07 >> Linear solve converged due to CONVERGED_RTOL iterations 59 >> 6 SNES Function norm 1.233942497170e+05 >> Linear solve converged due to CONVERGED_RTOL iterations 38 >> 7 SNES Function norm 2.309536978272e+03 >> Linear solve converged due to CONVERGED_RTOL iterations 38 >> 8 SNES Function norm 7.064055492974e-03 >> Linear solve converged due to CONVERGED_RTOL iterations 38 >> 9 SNES Function norm 3.419662119536e-06 >> Nonlinear solve converged due to CONVERGED_PNORM_RELATIVE >> 0 SNES Function norm 3.420486720202e+09 >> Linear solve converged due to CONVERGED_RTOL iterations 10 >> 1 SNES Function norm 1.426355492941e+08 >> Linear solve converged due to CONVERGED_RTOL iterations 36 >> 2 SNES Function norm 1.429830907344e+06 >> Linear solve converged due to CONVERGED_RTOL iterations 37 >> 3 SNES Function norm 1.755275678702e+03 >> Linear solve converged due to CONVERGED_RTOL iterations 37 >> 4 SNES Function norm 1.216161531652e-03 >> Linear solve converged due to CONVERGED_RTOL iterations 36 >> 5 SNES Function norm 4.309776935609e-06 >> Nonlinear solve converged due to CONVERGED_PNORM_RELATIVE >> Number of Newton iterations = 5 >> Converged reason is 4 >> >> This time the convergence is very aggressive, and it is quadratic from >> iteration 0 to iteration 4. >> >> The solution vector x from this run is the same as the run without the >> option "-snes_mf_operator". Any hint for why this is totally quadratic >> convergence? >> >> Thank you very much, >> >> Yan >> >> >> On Sun, Jan 10, 2010 at 7:25 PM, Barry Smith wrote: >> >> On Jan 10, 2010, at 6:09 PM, Ryan Yan wrote: >> >> Hi Barry, >> >> I run the code on another machine. >> >> yy2250 at sci-m8n6 ~/test/1d $ srun -p sci-comp -N 2 -n 2 ./HeatProfile1D >> -snes_type test > out.snes_test >> [0]PETSC ERROR: SNESSolve() line 2221 in src/snes/interface/snes.c >> [1]PETSC ERROR: SNESSolve() line 2221 in src/snes/interface/snes.c >> [0]PETSC ERROR: DMMGSolveSNES() line 510 in src/snes/utils/damgsnes.c >> [0]PETSC ERROR: DMMGSolve() line 372 in src/snes/utils/damg.c >> [0]PETSC ERROR: main() line 270 in >> /home/vyan2000/local/PPETSc/petsc-2.3.3-p15/src/ksp/ksp/examples/tutorials/NCprojectHeatProfile1D.c >> application called MPI_Abort(MPI_COMM_WORLD, 73) - process 0 >> [1]PETSC ERROR: DMMGSolveSNES() line 510 in src/snes/utils/damgsnes.c >> [1]PETSC ERROR: DMMGSolve() line 372 in src/snes/utils/damg.c >> [1]PETSC ERROR: main() line 270 in >> /home/vyan2000/local/PPETSc/petsc-2.3.3-p15/src/ksp/ksp/examples/tutorials/NCprojectHeatProfile1D.c >> In: PMI_Abort(73, application called MPI_Abort(MPI_COMM_WORLD, 73) - >> process 0) >> application called MPI_Abort(MPI_COMM_WORLD, 73) - process 1 >> In: PMI_Abort(73, application called MPI_Abort(MPI_COMM_WORLD, 73) - >> process 1) >> srun: error: task 1: Exited with exit code 73 >> srun: error: task 0: Exited with exit code 73 >> >> I was using 3 level grids. The error comes out during the snes solving on >> the fine level grid. The output in the file out.snes_test is : >> yy2250 at sci-m8n6 ~/test/1d $ cat out.snes_test >> Testing hand-coded Jacobian, if the ratio is >> O(1.e-8), the hand-coded Jacobian is probably correct. >> Run with -snes_test_display to show difference >> of hand-coded and finite difference Jacobian. >> Norm of matrix ratio 1.21314e-10 difference 1.41447 >> Norm of matrix ratio 4.30115e-06 difference 1.41427 >> Norm of matrix ratio 4.30108e-06 difference 1.41427 >> >> >> Ok, Jacobians being used are probably ok. >> >> >> >> For the -snes_mf_operator, >> yy2250 at sci-m8n6 ~/test/1d $ srun -p sci-comp -N 2 -n 2 ./HeatProfile1D >> -snes_mf_operator >> the program is still running. I suspect that the program hanged. >> >> It's not hanging. It is just running. Use -snes_monitor -ksp_rtol 1.e-12 >> -ksp_converged_reason -snes_converged_reason and run again >> >> Barry >> >> >> >> Is there any useful information in the file "out.snes_test"? >> >> BTW, I did not hand-code any Jacobian, since I passed 0 for the Jacobian >> evaluation subroutine in the DMMGSetSNESLocal(). >> >> Thank you very much, >> >> Yan >> >> >> >> On Sun, Jan 10, 2010 at 6:11 PM, Barry Smith wrote: >> >> That message ain't from PETSc. Something is likely killing the job. >> >> Barry >> >> >> On Jan 10, 2010, at 5:09 PM, Ryan Yan wrote: >> >> Hi Barry, >> I got the following result: >> >> vyan2000 at vyan2000-linux ~/NCproject/general $ mpirun -np 1 >> ./HeatProfile1D -snes_mf_operator >> Alarm clock >> vyan2000 at vyan2000-linux ~/NCproject/general $ mpirun -np 1 >> ./HeatProfile1D -snes_type test >> Alarm clock >> >> Are they normal responses and what do they indicate? >> >> Thanks a lot, >> >> Yan >> >> >> On Sun, Jan 10, 2010 at 5:57 PM, Barry Smith wrote: >> >> JUST run with -snes_mf_operator and then with -snes_type test NOT >> together, >> >> Barry >> >> >> On Jan 10, 2010, at 4:42 PM, Ryan Yan wrote: >> >> Hi Barry, >> Please see reply below, >> >> On Sun, Jan 10, 2010 at 4:57 PM, Barry Smith wrote: >> >> On Jan 10, 2010, at 3:52 PM, Ryan Yan wrote: >> >> Hi Barry, >> Yes, exactly. The original multi-components system scale quite unevenly. I >> will try to rescale it. >> Could this be helpful to show some promise on quadratic convergence? >> >> I won't be concerned about "quadratic convergence" I'd only be concerned >> that it is converging to the correct answer and that you are getting close >> enough to the correct answer. >> >> Yes, I agree. >> >> You can run with -snes_mf_operator and -snes_type test to verify if the >> Jacobian being computed is accurate. Perhaps in your function evaluation you >> are not using the stencil that you set with the DA, this would cause the >> wrong Jacobian to be computed. >> >> >> After passing in -snes_mf_operator and -snes_type test as follows: >> vyan2000 at vyan2000-linux ~/NCproject/general : mpirun -np 2 >> ./HeatProfile1D -snes_mf_operator -snes_type test >> I got errors, as expected: >> >> [0]PETSC ERROR: --------------------- Error Message >> ------------------------------------ >> [0]PETSC ERROR: Invalid argument! >> [0]PETSC ERROR: Cannot test with alternative preconditioner! >> [0]PETSC ERROR: >> ------------------------------------------------------------------------ >> [0]PETSC ERROR: Petsc Release Version 3.0.0, Patch 5, Mon Apr 13 09:15:37 >> CDT 2009 >> [0]PETSC ERROR: See docs/changes/index.html for recent updates. >> [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. >> [0]PETSC ERROR: See docs/index.html for manual pages. >> [0]PETSC ERROR: >> ------------------------------------------------------------------------ >> [0]PETSC ERROR: ./HeatProfile1D on a O-hypre-p named vyan2000-linux by >> vyan2000 Sun Jan 10 17:16:00 2010 >> [0]PETSC ERROR: Libraries linked from >> /home/vyan2000/local/PPETSc/petsc-3.0.0-p5/O-hypre-prometheus/lib >> [0]PETSC ERROR: Configure run at Thu Jun 25 13:49:36 2009 >> [0]PETSC ERROR: Configure options --download-mpich=1 --with-debugger=gdb >> --download-hypre=1 --download-parmetis=1 --download-prometheus=1 >> --with-shared=0 >> [0]PETSC ERROR: >> ------------------------------------------------------------------------ >> [0]PETSC ERROR: SNESSolve_Test() line 28 in src/snes/impls/test/snestest.c >> [0]PETSC ERROR: SNESSolve() line 2221 in src/snes/interface/snes.c >> [0]PETSC ERROR: DMMGSolveSNES() line 510 in src/snes/utils/damgsnes.c >> [0]PETSC ERROR: DMMGSolve() line 372 in src/snes/utils/damg.c >> [0]PETSC ERROR: main() line 270 in >> /home/vyan2000/local/PPETSc/petsc-2.3.3-p15/src/ksp/ksp/examples/tutorials/NCprojectHeatProfile1D.c >> application called MPI_Abort(MPI_COMM_WORLD, 62) - process 0[cli_0]: >> aborting job: >> application called MPI_Abort(MPI_COMM_WORLD, 62) - process 0 >> >> The error makes sense, since I did not pass in the analytical Jacobian for >> the preconditioner matrix. In stead, I was using >> DMMGSetSNESLocal(dmmg,FormFunctionLocal,0,ad_FormFunctionLocal,admf_FormFunctionLocal). >> I am going to change the code a bit, pass in the analytical Jacobian and do >> the -snes_type test. >> >> I will bear your suggestion in mind during the test. >> >> Thanks a lot, >> >> Yan >> >> >> >> Barry >> >> >> >> Thanks a lot, >> >> Yan >> >> On Sun, Jan 10, 2010 at 4:35 PM, Barry Smith wrote: >> >> You already got a 10^16 drop in the residual norm. It is not realistic to >> expect to get much more than that for double precision calculations. Perhaps >> your original F() has some funky scaling of different components that you >> can fix. >> >> >> >> Barry >> >> >> On Jan 10, 2010, at 2:55 PM, Ryan Yan wrote: >> >> Hi All, >> I am solving a nonlinear system using snes. The -snes_monitor option has >> the following output: >> >> 0 SNES Function norm 2.640163923729e+09 >> 1 SNES Function norm 1.047643565314e+08 >> 2 SNES Function norm 1.712732074788e+06 >> 3 SNES Function norm 1.002169173269e+04 >> 4 SNES Function norm 1.655878303433e+03 >> 5 SNES Function norm 3.746498305706e+02 >> 6 SNES Function norm 8.317435704773e+01 >> 7 SNES Function norm 1.857639969641e+01 >> 8 SNES Function norm 4.149691057773e+00 >> 9 SNES Function norm 9.265604042412e-01 >> 10 SNES Function norm 2.069527103214e-01 >> 11 SNES Function norm 4.624186491082e-02 >> 12 SNES Function norm 1.035558432688e-02 >> 13 SNES Function norm 2.341362958811e-03 >> 14 SNES Function norm 5.507445427277e-04 >> 15 SNES Function norm 1.485123568354e-04 >> 16 SNES Function norm 5.180043781814e-05 >> 17 SNES Function norm 2.341966514486e-05 >> 18 SNES Function norm 1.344936158651e-05 >> 19 SNES Function norm 1.054812641176e-05 >> Number of Newton iterations = 19 >> Converged reason is 4 >> >> It looks like the iterate never falls into a quadratic convergence region >> before it converges. Is there any hint to understand this behavior? >> >> Thanks a lot, >> >> Yan >> >> >> >> >> >> >> >> >> >> >> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vyan2000 at gmail.com Mon Jan 11 09:34:18 2010 From: vyan2000 at gmail.com (Ryan Yan) Date: Mon, 11 Jan 2010 10:34:18 -0500 Subject: [petsc-users] Snes behavior In-Reply-To: References: <18837D43-E96A-4C5A-B3E4-2C6DAB896DFC@mcs.anl.gov> Message-ID: Never mind, I saught boundary conditions in your last email. Thank you very much, Yan On Mon, Jan 11, 2010 at 10:32 AM, Ryan Yan wrote: > Hi Barry, > This is my case: In 1-d, for the interior pts the stencil width is 1, but > for boundary pts the stencil width is 2. > > Should the stencil width in DACreate1d be 1 or 2? > > If I switch to stencil width 2, then I can get quadratic convergence. > > > Thank you very much, > > Yan > > On Sun, Jan 10, 2010 at 11:22 PM, Barry Smith wrote: > >> >> Like I said in my previous mail this means that the Jacobian being >> computing before was not accurate enough for some reason. This will happen >> if your function evaluation does NOT respect the stencil you provided when >> you created the DA. For example if you told the DA to use a stencil width of >> 1 but somewhere your function evaluation grabs a value 2 cells away and uses >> it in the computation. For example on some boundary condition. Or you told >> it was a box stencil but the function evaluation grabbed a value from the >> diagonal stencil. >> There is a 99% probability this is your problem. >> >> If you cannot find the bug then send your function evaluation and how you >> form the DA to petsc-maint at mcs.anl.gov and we'll find it. >> >> >> Barry >> >> >> >> On Jan 10, 2010, at 9:13 PM, Ryan Yan wrote: >> >> Hi Barry, >>> Here is the result for -snes_mf_operator: >>> >>> yy2250 at sci-m8n6 ~/test/1d $ srun -p sci-comp -N 2 -n 2 ./HeatProfile1D >>> -snes_mf_operator -dmmg_grid_sequence -pc_type bjacobi -snes_rtol 1e-15 >>> -snes_monitor -ksp_rtol 1.e-12 -ksp_converged_reason -snes_converged_reason >>> > out.mf_operator & >>> >>> >>> yy2250 at sci-m8n6 ~/test/1d $ cat out.mf_operator >>> 0 SNES Function norm 1.411468156752e+08 >>> Linear solve converged due to CONVERGED_ITS iterations 1 >>> 1 SNES Function norm 1.396727866424e+08 >>> Linear solve converged due to CONVERGED_ITS iterations 1 >>> Nonlinear solve did not converge due to DIVERGED_LS_FAILURE >>> 0 SNES Function norm 1.045985796073e+08 >>> Linear solve converged due to CONVERGED_RTOL iterations 44 >>> 1 SNES Function norm 7.912999174650e+07 >>> Linear solve converged due to CONVERGED_RTOL iterations 38 >>> 2 SNES Function norm 6.079225520436e+07 >>> Linear solve converged due to CONVERGED_RTOL iterations 66 >>> 3 SNES Function norm 4.610252725173e+07 >>> Linear solve converged due to CONVERGED_RTOL iterations 66 >>> 4 SNES Function norm 3.333587574896e+07 >>> Linear solve converged due to CONVERGED_RTOL iterations 46 >>> 5 SNES Function norm 2.326587775240e+07 >>> Linear solve converged due to CONVERGED_RTOL iterations 59 >>> 6 SNES Function norm 1.233942497170e+05 >>> Linear solve converged due to CONVERGED_RTOL iterations 38 >>> 7 SNES Function norm 2.309536978272e+03 >>> Linear solve converged due to CONVERGED_RTOL iterations 38 >>> 8 SNES Function norm 7.064055492974e-03 >>> Linear solve converged due to CONVERGED_RTOL iterations 38 >>> 9 SNES Function norm 3.419662119536e-06 >>> Nonlinear solve converged due to CONVERGED_PNORM_RELATIVE >>> 0 SNES Function norm 3.420486720202e+09 >>> Linear solve converged due to CONVERGED_RTOL iterations 10 >>> 1 SNES Function norm 1.426355492941e+08 >>> Linear solve converged due to CONVERGED_RTOL iterations 36 >>> 2 SNES Function norm 1.429830907344e+06 >>> Linear solve converged due to CONVERGED_RTOL iterations 37 >>> 3 SNES Function norm 1.755275678702e+03 >>> Linear solve converged due to CONVERGED_RTOL iterations 37 >>> 4 SNES Function norm 1.216161531652e-03 >>> Linear solve converged due to CONVERGED_RTOL iterations 36 >>> 5 SNES Function norm 4.309776935609e-06 >>> Nonlinear solve converged due to CONVERGED_PNORM_RELATIVE >>> Number of Newton iterations = 5 >>> Converged reason is 4 >>> >>> This time the convergence is very aggressive, and it is quadratic from >>> iteration 0 to iteration 4. >>> >>> The solution vector x from this run is the same as the run without the >>> option "-snes_mf_operator". Any hint for why this is totally quadratic >>> convergence? >>> >>> Thank you very much, >>> >>> Yan >>> >>> >>> On Sun, Jan 10, 2010 at 7:25 PM, Barry Smith wrote: >>> >>> On Jan 10, 2010, at 6:09 PM, Ryan Yan wrote: >>> >>> Hi Barry, >>> >>> I run the code on another machine. >>> >>> yy2250 at sci-m8n6 ~/test/1d $ srun -p sci-comp -N 2 -n 2 ./HeatProfile1D >>> -snes_type test > out.snes_test >>> [0]PETSC ERROR: SNESSolve() line 2221 in src/snes/interface/snes.c >>> [1]PETSC ERROR: SNESSolve() line 2221 in src/snes/interface/snes.c >>> [0]PETSC ERROR: DMMGSolveSNES() line 510 in src/snes/utils/damgsnes.c >>> [0]PETSC ERROR: DMMGSolve() line 372 in src/snes/utils/damg.c >>> [0]PETSC ERROR: main() line 270 in >>> /home/vyan2000/local/PPETSc/petsc-2.3.3-p15/src/ksp/ksp/examples/tutorials/NCprojectHeatProfile1D.c >>> application called MPI_Abort(MPI_COMM_WORLD, 73) - process 0 >>> [1]PETSC ERROR: DMMGSolveSNES() line 510 in src/snes/utils/damgsnes.c >>> [1]PETSC ERROR: DMMGSolve() line 372 in src/snes/utils/damg.c >>> [1]PETSC ERROR: main() line 270 in >>> /home/vyan2000/local/PPETSc/petsc-2.3.3-p15/src/ksp/ksp/examples/tutorials/NCprojectHeatProfile1D.c >>> In: PMI_Abort(73, application called MPI_Abort(MPI_COMM_WORLD, 73) - >>> process 0) >>> application called MPI_Abort(MPI_COMM_WORLD, 73) - process 1 >>> In: PMI_Abort(73, application called MPI_Abort(MPI_COMM_WORLD, 73) - >>> process 1) >>> srun: error: task 1: Exited with exit code 73 >>> srun: error: task 0: Exited with exit code 73 >>> >>> I was using 3 level grids. The error comes out during the snes solving on >>> the fine level grid. The output in the file out.snes_test is : >>> yy2250 at sci-m8n6 ~/test/1d $ cat out.snes_test >>> Testing hand-coded Jacobian, if the ratio is >>> O(1.e-8), the hand-coded Jacobian is probably correct. >>> Run with -snes_test_display to show difference >>> of hand-coded and finite difference Jacobian. >>> Norm of matrix ratio 1.21314e-10 difference 1.41447 >>> Norm of matrix ratio 4.30115e-06 difference 1.41427 >>> Norm of matrix ratio 4.30108e-06 difference 1.41427 >>> >>> >>> Ok, Jacobians being used are probably ok. >>> >>> >>> >>> For the -snes_mf_operator, >>> yy2250 at sci-m8n6 ~/test/1d $ srun -p sci-comp -N 2 -n 2 ./HeatProfile1D >>> -snes_mf_operator >>> the program is still running. I suspect that the program hanged. >>> >>> It's not hanging. It is just running. Use -snes_monitor -ksp_rtol 1.e-12 >>> -ksp_converged_reason -snes_converged_reason and run again >>> >>> Barry >>> >>> >>> >>> Is there any useful information in the file "out.snes_test"? >>> >>> BTW, I did not hand-code any Jacobian, since I passed 0 for the Jacobian >>> evaluation subroutine in the DMMGSetSNESLocal(). >>> >>> Thank you very much, >>> >>> Yan >>> >>> >>> >>> On Sun, Jan 10, 2010 at 6:11 PM, Barry Smith wrote: >>> >>> That message ain't from PETSc. Something is likely killing the job. >>> >>> Barry >>> >>> >>> On Jan 10, 2010, at 5:09 PM, Ryan Yan wrote: >>> >>> Hi Barry, >>> I got the following result: >>> >>> vyan2000 at vyan2000-linux ~/NCproject/general $ mpirun -np 1 >>> ./HeatProfile1D -snes_mf_operator >>> Alarm clock >>> vyan2000 at vyan2000-linux ~/NCproject/general $ mpirun -np 1 >>> ./HeatProfile1D -snes_type test >>> Alarm clock >>> >>> Are they normal responses and what do they indicate? >>> >>> Thanks a lot, >>> >>> Yan >>> >>> >>> On Sun, Jan 10, 2010 at 5:57 PM, Barry Smith wrote: >>> >>> JUST run with -snes_mf_operator and then with -snes_type test NOT >>> together, >>> >>> Barry >>> >>> >>> On Jan 10, 2010, at 4:42 PM, Ryan Yan wrote: >>> >>> Hi Barry, >>> Please see reply below, >>> >>> On Sun, Jan 10, 2010 at 4:57 PM, Barry Smith wrote: >>> >>> On Jan 10, 2010, at 3:52 PM, Ryan Yan wrote: >>> >>> Hi Barry, >>> Yes, exactly. The original multi-components system scale quite unevenly. >>> I will try to rescale it. >>> Could this be helpful to show some promise on quadratic convergence? >>> >>> I won't be concerned about "quadratic convergence" I'd only be concerned >>> that it is converging to the correct answer and that you are getting close >>> enough to the correct answer. >>> >>> Yes, I agree. >>> >>> You can run with -snes_mf_operator and -snes_type test to verify if the >>> Jacobian being computed is accurate. Perhaps in your function evaluation you >>> are not using the stencil that you set with the DA, this would cause the >>> wrong Jacobian to be computed. >>> >>> >>> After passing in -snes_mf_operator and -snes_type test as follows: >>> vyan2000 at vyan2000-linux ~/NCproject/general : mpirun -np 2 >>> ./HeatProfile1D -snes_mf_operator -snes_type test >>> I got errors, as expected: >>> >>> [0]PETSC ERROR: --------------------- Error Message >>> ------------------------------------ >>> [0]PETSC ERROR: Invalid argument! >>> [0]PETSC ERROR: Cannot test with alternative preconditioner! >>> [0]PETSC ERROR: >>> ------------------------------------------------------------------------ >>> [0]PETSC ERROR: Petsc Release Version 3.0.0, Patch 5, Mon Apr 13 09:15:37 >>> CDT 2009 >>> [0]PETSC ERROR: See docs/changes/index.html for recent updates. >>> [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. >>> [0]PETSC ERROR: See docs/index.html for manual pages. >>> [0]PETSC ERROR: >>> ------------------------------------------------------------------------ >>> [0]PETSC ERROR: ./HeatProfile1D on a O-hypre-p named vyan2000-linux by >>> vyan2000 Sun Jan 10 17:16:00 2010 >>> [0]PETSC ERROR: Libraries linked from >>> /home/vyan2000/local/PPETSc/petsc-3.0.0-p5/O-hypre-prometheus/lib >>> [0]PETSC ERROR: Configure run at Thu Jun 25 13:49:36 2009 >>> [0]PETSC ERROR: Configure options --download-mpich=1 --with-debugger=gdb >>> --download-hypre=1 --download-parmetis=1 --download-prometheus=1 >>> --with-shared=0 >>> [0]PETSC ERROR: >>> ------------------------------------------------------------------------ >>> [0]PETSC ERROR: SNESSolve_Test() line 28 in >>> src/snes/impls/test/snestest.c >>> [0]PETSC ERROR: SNESSolve() line 2221 in src/snes/interface/snes.c >>> [0]PETSC ERROR: DMMGSolveSNES() line 510 in src/snes/utils/damgsnes.c >>> [0]PETSC ERROR: DMMGSolve() line 372 in src/snes/utils/damg.c >>> [0]PETSC ERROR: main() line 270 in >>> /home/vyan2000/local/PPETSc/petsc-2.3.3-p15/src/ksp/ksp/examples/tutorials/NCprojectHeatProfile1D.c >>> application called MPI_Abort(MPI_COMM_WORLD, 62) - process 0[cli_0]: >>> aborting job: >>> application called MPI_Abort(MPI_COMM_WORLD, 62) - process 0 >>> >>> The error makes sense, since I did not pass in the analytical Jacobian >>> for the preconditioner matrix. In stead, I was using >>> DMMGSetSNESLocal(dmmg,FormFunctionLocal,0,ad_FormFunctionLocal,admf_FormFunctionLocal). >>> I am going to change the code a bit, pass in the analytical Jacobian and do >>> the -snes_type test. >>> >>> I will bear your suggestion in mind during the test. >>> >>> Thanks a lot, >>> >>> Yan >>> >>> >>> >>> Barry >>> >>> >>> >>> Thanks a lot, >>> >>> Yan >>> >>> On Sun, Jan 10, 2010 at 4:35 PM, Barry Smith wrote: >>> >>> You already got a 10^16 drop in the residual norm. It is not realistic >>> to expect to get much more than that for double precision calculations. >>> Perhaps your original F() has some funky scaling of different components >>> that you can fix. >>> >>> >>> >>> Barry >>> >>> >>> On Jan 10, 2010, at 2:55 PM, Ryan Yan wrote: >>> >>> Hi All, >>> I am solving a nonlinear system using snes. The -snes_monitor option has >>> the following output: >>> >>> 0 SNES Function norm 2.640163923729e+09 >>> 1 SNES Function norm 1.047643565314e+08 >>> 2 SNES Function norm 1.712732074788e+06 >>> 3 SNES Function norm 1.002169173269e+04 >>> 4 SNES Function norm 1.655878303433e+03 >>> 5 SNES Function norm 3.746498305706e+02 >>> 6 SNES Function norm 8.317435704773e+01 >>> 7 SNES Function norm 1.857639969641e+01 >>> 8 SNES Function norm 4.149691057773e+00 >>> 9 SNES Function norm 9.265604042412e-01 >>> 10 SNES Function norm 2.069527103214e-01 >>> 11 SNES Function norm 4.624186491082e-02 >>> 12 SNES Function norm 1.035558432688e-02 >>> 13 SNES Function norm 2.341362958811e-03 >>> 14 SNES Function norm 5.507445427277e-04 >>> 15 SNES Function norm 1.485123568354e-04 >>> 16 SNES Function norm 5.180043781814e-05 >>> 17 SNES Function norm 2.341966514486e-05 >>> 18 SNES Function norm 1.344936158651e-05 >>> 19 SNES Function norm 1.054812641176e-05 >>> Number of Newton iterations = 19 >>> Converged reason is 4 >>> >>> It looks like the iterate never falls into a quadratic convergence region >>> before it converges. Is there any hint to understand this behavior? >>> >>> Thanks a lot, >>> >>> Yan >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Mon Jan 11 11:27:32 2010 From: bsmith at mcs.anl.gov (Barry Smith) Date: Mon, 11 Jan 2010 11:27:32 -0600 Subject: [petsc-users] Snes behavior In-Reply-To: References: <18837D43-E96A-4C5A-B3E4-2C6DAB896DFC@mcs.anl.gov> Message-ID: On Jan 11, 2010, at 9:32 AM, Ryan Yan wrote: > Hi Barry, > This is my case: In 1-d, for the interior pts the stencil width is > 1, but for boundary pts the stencil width is 2. > > Should the stencil width in DACreate1d be 1 or 2? 2 > > If I switch to stencil width 2, then I can get quadratic convergence. Yes > > Thank you very much, > > Yan > > On Sun, Jan 10, 2010 at 11:22 PM, Barry Smith > wrote: > > Like I said in my previous mail this means that the Jacobian being > computing before was not accurate enough for some reason. This will > happen if your function evaluation does NOT respect the stencil you > provided when you created the DA. For example if you told the DA to > use a stencil width of 1 but somewhere your function evaluation > grabs a value 2 cells away and uses it in the computation. For > example on some boundary condition. Or you told it was a box stencil > but the function evaluation grabbed a value from the diagonal stencil. > There is a 99% probability this is your problem. > > If you cannot find the bug then send your function evaluation and > how you form the DA to petsc-maint at mcs.anl.gov and we'll find it. > > > Barry > > > > On Jan 10, 2010, at 9:13 PM, Ryan Yan wrote: > > Hi Barry, > Here is the result for -snes_mf_operator: > > yy2250 at sci-m8n6 ~/test/1d $ srun -p sci-comp -N 2 -n 2 ./ > HeatProfile1D -snes_mf_operator -dmmg_grid_sequence -pc_type bjacobi > -snes_rtol 1e-15 -snes_monitor -ksp_rtol 1.e-12 - > ksp_converged_reason -snes_converged_reason > out.mf_operator & > > > yy2250 at sci-m8n6 ~/test/1d $ cat out.mf_operator > 0 SNES Function norm 1.411468156752e+08 > Linear solve converged due to CONVERGED_ITS iterations 1 > 1 SNES Function norm 1.396727866424e+08 > Linear solve converged due to CONVERGED_ITS iterations 1 > Nonlinear solve did not converge due to DIVERGED_LS_FAILURE > 0 SNES Function norm 1.045985796073e+08 > Linear solve converged due to CONVERGED_RTOL iterations 44 > 1 SNES Function norm 7.912999174650e+07 > Linear solve converged due to CONVERGED_RTOL iterations 38 > 2 SNES Function norm 6.079225520436e+07 > Linear solve converged due to CONVERGED_RTOL iterations 66 > 3 SNES Function norm 4.610252725173e+07 > Linear solve converged due to CONVERGED_RTOL iterations 66 > 4 SNES Function norm 3.333587574896e+07 > Linear solve converged due to CONVERGED_RTOL iterations 46 > 5 SNES Function norm 2.326587775240e+07 > Linear solve converged due to CONVERGED_RTOL iterations 59 > 6 SNES Function norm 1.233942497170e+05 > Linear solve converged due to CONVERGED_RTOL iterations 38 > 7 SNES Function norm 2.309536978272e+03 > Linear solve converged due to CONVERGED_RTOL iterations 38 > 8 SNES Function norm 7.064055492974e-03 > Linear solve converged due to CONVERGED_RTOL iterations 38 > 9 SNES Function norm 3.419662119536e-06 > Nonlinear solve converged due to CONVERGED_PNORM_RELATIVE > 0 SNES Function norm 3.420486720202e+09 > Linear solve converged due to CONVERGED_RTOL iterations 10 > 1 SNES Function norm 1.426355492941e+08 > Linear solve converged due to CONVERGED_RTOL iterations 36 > 2 SNES Function norm 1.429830907344e+06 > Linear solve converged due to CONVERGED_RTOL iterations 37 > 3 SNES Function norm 1.755275678702e+03 > Linear solve converged due to CONVERGED_RTOL iterations 37 > 4 SNES Function norm 1.216161531652e-03 > Linear solve converged due to CONVERGED_RTOL iterations 36 > 5 SNES Function norm 4.309776935609e-06 > Nonlinear solve converged due to CONVERGED_PNORM_RELATIVE > Number of Newton iterations = 5 > Converged reason is 4 > > This time the convergence is very aggressive, and it is quadratic > from iteration 0 to iteration 4. > > The solution vector x from this run is the same as the run without > the option "-snes_mf_operator". Any hint for why this is totally > quadratic convergence? > > Thank you very much, > > Yan > > > On Sun, Jan 10, 2010 at 7:25 PM, Barry Smith > wrote: > > On Jan 10, 2010, at 6:09 PM, Ryan Yan wrote: > > Hi Barry, > > I run the code on another machine. > > yy2250 at sci-m8n6 ~/test/1d $ srun -p sci-comp -N 2 -n 2 ./ > HeatProfile1D -snes_type test > out.snes_test > [0]PETSC ERROR: SNESSolve() line 2221 in src/snes/interface/snes.c > [1]PETSC ERROR: SNESSolve() line 2221 in src/snes/interface/snes.c > [0]PETSC ERROR: DMMGSolveSNES() line 510 in src/snes/utils/damgsnes.c > [0]PETSC ERROR: DMMGSolve() line 372 in src/snes/utils/damg.c > [0]PETSC ERROR: main() line 270 in /home/vyan2000/local/PPETSc/ > petsc-2.3.3-p15/src/ksp/ksp/examples/tutorials/ > NCprojectHeatProfile1D.c > application called MPI_Abort(MPI_COMM_WORLD, 73) - process 0 > [1]PETSC ERROR: DMMGSolveSNES() line 510 in src/snes/utils/damgsnes.c > [1]PETSC ERROR: DMMGSolve() line 372 in src/snes/utils/damg.c > [1]PETSC ERROR: main() line 270 in /home/vyan2000/local/PPETSc/ > petsc-2.3.3-p15/src/ksp/ksp/examples/tutorials/ > NCprojectHeatProfile1D.c > In: PMI_Abort(73, application called MPI_Abort(MPI_COMM_WORLD, 73) - > process 0) > application called MPI_Abort(MPI_COMM_WORLD, 73) - process 1 > In: PMI_Abort(73, application called MPI_Abort(MPI_COMM_WORLD, 73) - > process 1) > srun: error: task 1: Exited with exit code 73 > srun: error: task 0: Exited with exit code 73 > > I was using 3 level grids. The error comes out during the snes > solving on the fine level grid. The output in the file out.snes_test > is : > yy2250 at sci-m8n6 ~/test/1d $ cat out.snes_test > Testing hand-coded Jacobian, if the ratio is > O(1.e-8), the hand-coded Jacobian is probably correct. > Run with -snes_test_display to show difference > of hand-coded and finite difference Jacobian. > Norm of matrix ratio 1.21314e-10 difference 1.41447 > Norm of matrix ratio 4.30115e-06 difference 1.41427 > Norm of matrix ratio 4.30108e-06 difference 1.41427 > > > Ok, Jacobians being used are probably ok. > > > > For the -snes_mf_operator, > yy2250 at sci-m8n6 ~/test/1d $ srun -p sci-comp -N 2 -n 2 ./ > HeatProfile1D -snes_mf_operator > the program is still running. I suspect that the program hanged. > > It's not hanging. It is just running. Use -snes_monitor -ksp_rtol > 1.e-12 -ksp_converged_reason -snes_converged_reason and run again > > Barry > > > > Is there any useful information in the file "out.snes_test"? > > BTW, I did not hand-code any Jacobian, since I passed 0 for the > Jacobian evaluation subroutine in the DMMGSetSNESLocal(). > > Thank you very much, > > Yan > > > > On Sun, Jan 10, 2010 at 6:11 PM, Barry Smith > wrote: > > That message ain't from PETSc. Something is likely killing the job. > > Barry > > > On Jan 10, 2010, at 5:09 PM, Ryan Yan wrote: > > Hi Barry, > I got the following result: > > vyan2000 at vyan2000-linux ~/NCproject/general $ mpirun -np 1 ./ > HeatProfile1D -snes_mf_operator > Alarm clock > vyan2000 at vyan2000-linux ~/NCproject/general $ mpirun -np 1 ./ > HeatProfile1D -snes_type test > Alarm clock > > Are they normal responses and what do they indicate? > > Thanks a lot, > > Yan > > > On Sun, Jan 10, 2010 at 5:57 PM, Barry Smith > wrote: > > JUST run with -snes_mf_operator and then with -snes_type test NOT > together, > > Barry > > > On Jan 10, 2010, at 4:42 PM, Ryan Yan wrote: > > Hi Barry, > Please see reply below, > > On Sun, Jan 10, 2010 at 4:57 PM, Barry Smith > wrote: > > On Jan 10, 2010, at 3:52 PM, Ryan Yan wrote: > > Hi Barry, > Yes, exactly. The original multi-components system scale quite > unevenly. I will try to rescale it. > Could this be helpful to show some promise on quadratic convergence? > > I won't be concerned about "quadratic convergence" I'd only be > concerned that it is converging to the correct answer and that you > are getting close enough to the correct answer. > > Yes, I agree. > > You can run with -snes_mf_operator and -snes_type test to verify if > the Jacobian being computed is accurate. Perhaps in your function > evaluation you are not using the stencil that you set with the DA, > this would cause the wrong Jacobian to be computed. > > > After passing in -snes_mf_operator and -snes_type test as follows: > vyan2000 at vyan2000-linux ~/NCproject/general : mpirun -np 2 ./ > HeatProfile1D -snes_mf_operator -snes_type test > I got errors, as expected: > > [0]PETSC ERROR: --------------------- Error Message > ------------------------------------ > [0]PETSC ERROR: Invalid argument! > [0]PETSC ERROR: Cannot test with alternative preconditioner! > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: Petsc Release Version 3.0.0, Patch 5, Mon Apr 13 > 09:15:37 CDT 2009 > [0]PETSC ERROR: See docs/changes/index.html for recent updates. > [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. > [0]PETSC ERROR: See docs/index.html for manual pages. > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: ./HeatProfile1D on a O-hypre-p named vyan2000-linux > by vyan2000 Sun Jan 10 17:16:00 2010 > [0]PETSC ERROR: Libraries linked from /home/vyan2000/local/PPETSc/ > petsc-3.0.0-p5/O-hypre-prometheus/lib > [0]PETSC ERROR: Configure run at Thu Jun 25 13:49:36 2009 > [0]PETSC ERROR: Configure options --download-mpich=1 --with- > debugger=gdb --download-hypre=1 --download-parmetis=1 --download- > prometheus=1 --with-shared=0 > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: SNESSolve_Test() line 28 in src/snes/impls/test/ > snestest.c > [0]PETSC ERROR: SNESSolve() line 2221 in src/snes/interface/snes.c > [0]PETSC ERROR: DMMGSolveSNES() line 510 in src/snes/utils/damgsnes.c > [0]PETSC ERROR: DMMGSolve() line 372 in src/snes/utils/damg.c > [0]PETSC ERROR: main() line 270 in /home/vyan2000/local/PPETSc/ > petsc-2.3.3-p15/src/ksp/ksp/examples/tutorials/ > NCprojectHeatProfile1D.c > application called MPI_Abort(MPI_COMM_WORLD, 62) - process 0[cli_0]: > aborting job: > application called MPI_Abort(MPI_COMM_WORLD, 62) - process 0 > > The error makes sense, since I did not pass in the analytical > Jacobian for the preconditioner matrix. In stead, I was using > DMMGSetSNESLocal(dmmg,FormFunctionLocal, > 0,ad_FormFunctionLocal,admf_FormFunctionLocal). I am going to change > the code a bit, pass in the analytical Jacobian and do the - > snes_type test. > > I will bear your suggestion in mind during the test. > > Thanks a lot, > > Yan > > > > Barry > > > > Thanks a lot, > > Yan > > On Sun, Jan 10, 2010 at 4:35 PM, Barry Smith > wrote: > > You already got a 10^16 drop in the residual norm. It is not > realistic to expect to get much more than that for double precision > calculations. Perhaps your original F() has some funky scaling of > different components that you can fix. > > > > Barry > > > On Jan 10, 2010, at 2:55 PM, Ryan Yan wrote: > > Hi All, > I am solving a nonlinear system using snes. The -snes_monitor option > has the following output: > > 0 SNES Function norm 2.640163923729e+09 > 1 SNES Function norm 1.047643565314e+08 > 2 SNES Function norm 1.712732074788e+06 > 3 SNES Function norm 1.002169173269e+04 > 4 SNES Function norm 1.655878303433e+03 > 5 SNES Function norm 3.746498305706e+02 > 6 SNES Function norm 8.317435704773e+01 > 7 SNES Function norm 1.857639969641e+01 > 8 SNES Function norm 4.149691057773e+00 > 9 SNES Function norm 9.265604042412e-01 > 10 SNES Function norm 2.069527103214e-01 > 11 SNES Function norm 4.624186491082e-02 > 12 SNES Function norm 1.035558432688e-02 > 13 SNES Function norm 2.341362958811e-03 > 14 SNES Function norm 5.507445427277e-04 > 15 SNES Function norm 1.485123568354e-04 > 16 SNES Function norm 5.180043781814e-05 > 17 SNES Function norm 2.341966514486e-05 > 18 SNES Function norm 1.344936158651e-05 > 19 SNES Function norm 1.054812641176e-05 > Number of Newton iterations = 19 > Converged reason is 4 > > It looks like the iterate never falls into a quadratic convergence > region before it converges. Is there any hint to understand this > behavior? > > Thanks a lot, > > Yan > > > > > > > > > > > > > > From vyan2000 at gmail.com Mon Jan 11 11:38:17 2010 From: vyan2000 at gmail.com (Ryan Yan) Date: Mon, 11 Jan 2010 12:38:17 -0500 Subject: [petsc-users] Snes behavior In-Reply-To: References: <18837D43-E96A-4C5A-B3E4-2C6DAB896DFC@mcs.anl.gov> Message-ID: Thanks a lot for the confirm, Yan On Mon, Jan 11, 2010 at 12:27 PM, Barry Smith wrote: > > On Jan 11, 2010, at 9:32 AM, Ryan Yan wrote: > > Hi Barry, >> >> This is my case: In 1-d, for the interior pts the stencil width is 1, but >> for boundary pts the stencil width is 2. >> >> Should the stencil width in DACreate1d be 1 or 2? >> > > 2 > > >> If I switch to stencil width 2, then I can get quadratic convergence. >> > > Yes > > > >> Thank you very much, >> >> Yan >> >> On Sun, Jan 10, 2010 at 11:22 PM, Barry Smith wrote: >> >> Like I said in my previous mail this means that the Jacobian being >> computing before was not accurate enough for some reason. This will happen >> if your function evaluation does NOT respect the stencil you provided when >> you created the DA. For example if you told the DA to use a stencil width of >> 1 but somewhere your function evaluation grabs a value 2 cells away and uses >> it in the computation. For example on some boundary condition. Or you told >> it was a box stencil but the function evaluation grabbed a value from the >> diagonal stencil. >> There is a 99% probability this is your problem. >> >> If you cannot find the bug then send your function evaluation and how you >> form the DA to petsc-maint at mcs.anl.gov and we'll find it. >> >> >> Barry >> >> >> >> On Jan 10, 2010, at 9:13 PM, Ryan Yan wrote: >> >> Hi Barry, >> Here is the result for -snes_mf_operator: >> >> yy2250 at sci-m8n6 ~/test/1d $ srun -p sci-comp -N 2 -n 2 ./HeatProfile1D >> -snes_mf_operator -dmmg_grid_sequence -pc_type bjacobi -snes_rtol 1e-15 >> -snes_monitor -ksp_rtol 1.e-12 -ksp_converged_reason -snes_converged_reason >> > out.mf_operator & >> >> >> yy2250 at sci-m8n6 ~/test/1d $ cat out.mf_operator >> 0 SNES Function norm 1.411468156752e+08 >> Linear solve converged due to CONVERGED_ITS iterations 1 >> 1 SNES Function norm 1.396727866424e+08 >> Linear solve converged due to CONVERGED_ITS iterations 1 >> Nonlinear solve did not converge due to DIVERGED_LS_FAILURE >> 0 SNES Function norm 1.045985796073e+08 >> Linear solve converged due to CONVERGED_RTOL iterations 44 >> 1 SNES Function norm 7.912999174650e+07 >> Linear solve converged due to CONVERGED_RTOL iterations 38 >> 2 SNES Function norm 6.079225520436e+07 >> Linear solve converged due to CONVERGED_RTOL iterations 66 >> 3 SNES Function norm 4.610252725173e+07 >> Linear solve converged due to CONVERGED_RTOL iterations 66 >> 4 SNES Function norm 3.333587574896e+07 >> Linear solve converged due to CONVERGED_RTOL iterations 46 >> 5 SNES Function norm 2.326587775240e+07 >> Linear solve converged due to CONVERGED_RTOL iterations 59 >> 6 SNES Function norm 1.233942497170e+05 >> Linear solve converged due to CONVERGED_RTOL iterations 38 >> 7 SNES Function norm 2.309536978272e+03 >> Linear solve converged due to CONVERGED_RTOL iterations 38 >> 8 SNES Function norm 7.064055492974e-03 >> Linear solve converged due to CONVERGED_RTOL iterations 38 >> 9 SNES Function norm 3.419662119536e-06 >> Nonlinear solve converged due to CONVERGED_PNORM_RELATIVE >> 0 SNES Function norm 3.420486720202e+09 >> Linear solve converged due to CONVERGED_RTOL iterations 10 >> 1 SNES Function norm 1.426355492941e+08 >> Linear solve converged due to CONVERGED_RTOL iterations 36 >> 2 SNES Function norm 1.429830907344e+06 >> Linear solve converged due to CONVERGED_RTOL iterations 37 >> 3 SNES Function norm 1.755275678702e+03 >> Linear solve converged due to CONVERGED_RTOL iterations 37 >> 4 SNES Function norm 1.216161531652e-03 >> Linear solve converged due to CONVERGED_RTOL iterations 36 >> 5 SNES Function norm 4.309776935609e-06 >> Nonlinear solve converged due to CONVERGED_PNORM_RELATIVE >> Number of Newton iterations = 5 >> Converged reason is 4 >> >> This time the convergence is very aggressive, and it is quadratic from >> iteration 0 to iteration 4. >> >> The solution vector x from this run is the same as the run without the >> option "-snes_mf_operator". Any hint for why this is totally quadratic >> convergence? >> >> Thank you very much, >> >> Yan >> >> >> On Sun, Jan 10, 2010 at 7:25 PM, Barry Smith wrote: >> >> On Jan 10, 2010, at 6:09 PM, Ryan Yan wrote: >> >> Hi Barry, >> >> I run the code on another machine. >> >> yy2250 at sci-m8n6 ~/test/1d $ srun -p sci-comp -N 2 -n 2 ./HeatProfile1D >> -snes_type test > out.snes_test >> [0]PETSC ERROR: SNESSolve() line 2221 in src/snes/interface/snes.c >> [1]PETSC ERROR: SNESSolve() line 2221 in src/snes/interface/snes.c >> [0]PETSC ERROR: DMMGSolveSNES() line 510 in src/snes/utils/damgsnes.c >> [0]PETSC ERROR: DMMGSolve() line 372 in src/snes/utils/damg.c >> [0]PETSC ERROR: main() line 270 in >> /home/vyan2000/local/PPETSc/petsc-2.3.3-p15/src/ksp/ksp/examples/tutorials/NCprojectHeatProfile1D.c >> application called MPI_Abort(MPI_COMM_WORLD, 73) - process 0 >> [1]PETSC ERROR: DMMGSolveSNES() line 510 in src/snes/utils/damgsnes.c >> [1]PETSC ERROR: DMMGSolve() line 372 in src/snes/utils/damg.c >> [1]PETSC ERROR: main() line 270 in >> /home/vyan2000/local/PPETSc/petsc-2.3.3-p15/src/ksp/ksp/examples/tutorials/NCprojectHeatProfile1D.c >> In: PMI_Abort(73, application called MPI_Abort(MPI_COMM_WORLD, 73) - >> process 0) >> application called MPI_Abort(MPI_COMM_WORLD, 73) - process 1 >> In: PMI_Abort(73, application called MPI_Abort(MPI_COMM_WORLD, 73) - >> process 1) >> srun: error: task 1: Exited with exit code 73 >> srun: error: task 0: Exited with exit code 73 >> >> I was using 3 level grids. The error comes out during the snes solving on >> the fine level grid. The output in the file out.snes_test is : >> yy2250 at sci-m8n6 ~/test/1d $ cat out.snes_test >> Testing hand-coded Jacobian, if the ratio is >> O(1.e-8), the hand-coded Jacobian is probably correct. >> Run with -snes_test_display to show difference >> of hand-coded and finite difference Jacobian. >> Norm of matrix ratio 1.21314e-10 difference 1.41447 >> Norm of matrix ratio 4.30115e-06 difference 1.41427 >> Norm of matrix ratio 4.30108e-06 difference 1.41427 >> >> >> Ok, Jacobians being used are probably ok. >> >> >> >> For the -snes_mf_operator, >> yy2250 at sci-m8n6 ~/test/1d $ srun -p sci-comp -N 2 -n 2 ./HeatProfile1D >> -snes_mf_operator >> the program is still running. I suspect that the program hanged. >> >> It's not hanging. It is just running. Use -snes_monitor -ksp_rtol 1.e-12 >> -ksp_converged_reason -snes_converged_reason and run again >> >> Barry >> >> >> >> Is there any useful information in the file "out.snes_test"? >> >> BTW, I did not hand-code any Jacobian, since I passed 0 for the Jacobian >> evaluation subroutine in the DMMGSetSNESLocal(). >> >> Thank you very much, >> >> Yan >> >> >> >> On Sun, Jan 10, 2010 at 6:11 PM, Barry Smith wrote: >> >> That message ain't from PETSc. Something is likely killing the job. >> >> Barry >> >> >> On Jan 10, 2010, at 5:09 PM, Ryan Yan wrote: >> >> Hi Barry, >> I got the following result: >> >> vyan2000 at vyan2000-linux ~/NCproject/general $ mpirun -np 1 >> ./HeatProfile1D -snes_mf_operator >> Alarm clock >> vyan2000 at vyan2000-linux ~/NCproject/general $ mpirun -np 1 >> ./HeatProfile1D -snes_type test >> Alarm clock >> >> Are they normal responses and what do they indicate? >> >> Thanks a lot, >> >> Yan >> >> >> On Sun, Jan 10, 2010 at 5:57 PM, Barry Smith wrote: >> >> JUST run with -snes_mf_operator and then with -snes_type test NOT >> together, >> >> Barry >> >> >> On Jan 10, 2010, at 4:42 PM, Ryan Yan wrote: >> >> Hi Barry, >> Please see reply below, >> >> On Sun, Jan 10, 2010 at 4:57 PM, Barry Smith wrote: >> >> On Jan 10, 2010, at 3:52 PM, Ryan Yan wrote: >> >> Hi Barry, >> Yes, exactly. The original multi-components system scale quite unevenly. I >> will try to rescale it. >> Could this be helpful to show some promise on quadratic convergence? >> >> I won't be concerned about "quadratic convergence" I'd only be concerned >> that it is converging to the correct answer and that you are getting close >> enough to the correct answer. >> >> Yes, I agree. >> >> You can run with -snes_mf_operator and -snes_type test to verify if the >> Jacobian being computed is accurate. Perhaps in your function evaluation you >> are not using the stencil that you set with the DA, this would cause the >> wrong Jacobian to be computed. >> >> >> After passing in -snes_mf_operator and -snes_type test as follows: >> vyan2000 at vyan2000-linux ~/NCproject/general : mpirun -np 2 >> ./HeatProfile1D -snes_mf_operator -snes_type test >> I got errors, as expected: >> >> [0]PETSC ERROR: --------------------- Error Message >> ------------------------------------ >> [0]PETSC ERROR: Invalid argument! >> [0]PETSC ERROR: Cannot test with alternative preconditioner! >> [0]PETSC ERROR: >> ------------------------------------------------------------------------ >> [0]PETSC ERROR: Petsc Release Version 3.0.0, Patch 5, Mon Apr 13 09:15:37 >> CDT 2009 >> [0]PETSC ERROR: See docs/changes/index.html for recent updates. >> [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. >> [0]PETSC ERROR: See docs/index.html for manual pages. >> [0]PETSC ERROR: >> ------------------------------------------------------------------------ >> [0]PETSC ERROR: ./HeatProfile1D on a O-hypre-p named vyan2000-linux by >> vyan2000 Sun Jan 10 17:16:00 2010 >> [0]PETSC ERROR: Libraries linked from >> /home/vyan2000/local/PPETSc/petsc-3.0.0-p5/O-hypre-prometheus/lib >> [0]PETSC ERROR: Configure run at Thu Jun 25 13:49:36 2009 >> [0]PETSC ERROR: Configure options --download-mpich=1 --with-debugger=gdb >> --download-hypre=1 --download-parmetis=1 --download-prometheus=1 >> --with-shared=0 >> [0]PETSC ERROR: >> ------------------------------------------------------------------------ >> [0]PETSC ERROR: SNESSolve_Test() line 28 in src/snes/impls/test/snestest.c >> [0]PETSC ERROR: SNESSolve() line 2221 in src/snes/interface/snes.c >> [0]PETSC ERROR: DMMGSolveSNES() line 510 in src/snes/utils/damgsnes.c >> [0]PETSC ERROR: DMMGSolve() line 372 in src/snes/utils/damg.c >> [0]PETSC ERROR: main() line 270 in >> /home/vyan2000/local/PPETSc/petsc-2.3.3-p15/src/ksp/ksp/examples/tutorials/NCprojectHeatProfile1D.c >> application called MPI_Abort(MPI_COMM_WORLD, 62) - process 0[cli_0]: >> aborting job: >> application called MPI_Abort(MPI_COMM_WORLD, 62) - process 0 >> >> The error makes sense, since I did not pass in the analytical Jacobian for >> the preconditioner matrix. In stead, I was using >> DMMGSetSNESLocal(dmmg,FormFunctionLocal,0,ad_FormFunctionLocal,admf_FormFunctionLocal). >> I am going to change the code a bit, pass in the analytical Jacobian and do >> the -snes_type test. >> >> I will bear your suggestion in mind during the test. >> >> Thanks a lot, >> >> Yan >> >> >> >> Barry >> >> >> >> Thanks a lot, >> >> Yan >> >> On Sun, Jan 10, 2010 at 4:35 PM, Barry Smith wrote: >> >> You already got a 10^16 drop in the residual norm. It is not realistic to >> expect to get much more than that for double precision calculations. Perhaps >> your original F() has some funky scaling of different components that you >> can fix. >> >> >> >> Barry >> >> >> On Jan 10, 2010, at 2:55 PM, Ryan Yan wrote: >> >> Hi All, >> I am solving a nonlinear system using snes. The -snes_monitor option has >> the following output: >> >> 0 SNES Function norm 2.640163923729e+09 >> 1 SNES Function norm 1.047643565314e+08 >> 2 SNES Function norm 1.712732074788e+06 >> 3 SNES Function norm 1.002169173269e+04 >> 4 SNES Function norm 1.655878303433e+03 >> 5 SNES Function norm 3.746498305706e+02 >> 6 SNES Function norm 8.317435704773e+01 >> 7 SNES Function norm 1.857639969641e+01 >> 8 SNES Function norm 4.149691057773e+00 >> 9 SNES Function norm 9.265604042412e-01 >> 10 SNES Function norm 2.069527103214e-01 >> 11 SNES Function norm 4.624186491082e-02 >> 12 SNES Function norm 1.035558432688e-02 >> 13 SNES Function norm 2.341362958811e-03 >> 14 SNES Function norm 5.507445427277e-04 >> 15 SNES Function norm 1.485123568354e-04 >> 16 SNES Function norm 5.180043781814e-05 >> 17 SNES Function norm 2.341966514486e-05 >> 18 SNES Function norm 1.344936158651e-05 >> 19 SNES Function norm 1.054812641176e-05 >> Number of Newton iterations = 19 >> Converged reason is 4 >> >> It looks like the iterate never falls into a quadratic convergence region >> before it converges. Is there any hint to understand this behavior? >> >> Thanks a lot, >> >> Yan >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vasoula_emp at hotmail.com Mon Jan 11 12:38:46 2010 From: vasoula_emp at hotmail.com (Vasia Kalavri) Date: Mon, 11 Jan 2010 20:38:46 +0200 Subject: [petsc-users] Merge two MATMPIAIJ into one Message-ID: Hello. I could use some help regarding the following issues: 1. I have created 2 structurally identical MATMPIJ matrices A1 and A2 with N rows and M columns. I want to merge them in order to create a matrix of the double size A3 with 2*N rows and M columns so that the first row of A2 will be the N+1-th row of the new matrix. Is there any function I could use to do this? 2. Is there a way to split a matrix after its values have been set by 1 process? What I have done is to create a MATMPIJ and make that process send the data to the other processes in order to get the distributed matrix. But I hope there is a smarter way to do this! Any idea is welcome, Vasia. _________________________________________________________________ Windows Live: Keep your friends up to date with what you do online. http://www.microsoft.com/middleeast/windows/windowslive/see-it-in-action/social-network-basics.aspx?ocid=PID23461::T:WLMTAGL:ON:WL:en-xm:SI_SB_1:092010 -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Mon Jan 11 13:21:04 2010 From: bsmith at mcs.anl.gov (Barry Smith) Date: Mon, 11 Jan 2010 13:21:04 -0600 Subject: [petsc-users] Merge two MATMPIAIJ into one In-Reply-To: References: Message-ID: On Jan 11, 2010, at 12:38 PM, Vasia Kalavri wrote: > Hello. > I could use some help regarding the following issues: > > 1. I have created 2 structurally identical MATMPIJ matrices A1 and > A2 with N rows and M columns. I want to merge them in order to > create a matrix of the double size A3 with 2*N rows and M columns so > that the first row of A2 will be the N+1-th row of the new matrix. > Is there any function I could use to do this? Since PETSc lays out the matrix rows across processors by having the first set of rows on the first process, the next set of rows on the next process etc. where is your "appended matrix" rows going to live? Do you want the new matrix to have the A1 matrix on the first half of the processors and the A2 matrix on the second half of the processors? PETSc doesn't have tools for "putting together" collections of sparse parallel matrices, so there is some direct access to the sparse matrix you need to do to get what you want. > > 2. Is there a way to split a matrix after its values have been set > by 1 process? What I have done is to create a MATMPIJ and make that > process send the data to the other processes in order to get the > distributed matrix. But I hope there is a smarter way to do this! In general one should create the entries to a PETSc matrix on the same number of processors as where the matrix lives so one does not need to "spread out" the matrix entries. Again, there isn't code to spread around sparse matrix entries. Barry > > Any idea is welcome, > Vasia. > > Windows Live: Keep your friends up to date with what you do online. From knepley at gmail.com Mon Jan 11 13:41:20 2010 From: knepley at gmail.com (Matthew Knepley) Date: Mon, 11 Jan 2010 13:41:20 -0600 Subject: [petsc-users] Merge two MATMPIAIJ into one In-Reply-To: References: Message-ID: On Mon, Jan 11, 2010 at 1:21 PM, Barry Smith wrote: > > On Jan 11, 2010, at 12:38 PM, Vasia Kalavri wrote: > > Hello. >> I could use some help regarding the following issues: >> >> 1. I have created 2 structurally identical MATMPIJ matrices A1 and A2 with >> N rows and M columns. I want to merge them in order to create a matrix of >> the double size A3 with 2*N rows and M columns so that the first row of A2 >> will be the N+1-th row of the new matrix. Is there any function I could use >> to do this? >> > > Since PETSc lays out the matrix rows across processors by having the > first set of rows on the first process, the next set of rows on the next > process etc. where is your "appended matrix" rows going to live? Do you want > the new matrix to have the A1 matrix on the first half of the processors and > the A2 matrix on the second half of the processors? > > PETSc doesn't have tools for "putting together" collections of sparse > parallel matrices, so there is some direct access to the sparse matrix you > need to do to get what you want. > Dave May at ETHZ had a package called PetscExt that does compose sparse matrices. Matt > >> 2. Is there a way to split a matrix after its values have been set by 1 >> process? What I have done is to create a MATMPIJ and make that process send >> the data to the other processes in order to get the distributed matrix. But >> I hope there is a smarter way to do this! >> > > In general one should create the entries to a PETSc matrix on the same > number of processors as where the matrix lives so one does not need to > "spread out" the matrix entries. > Again, there isn't code to spread around sparse matrix entries. > > > Barry > > > >> Any idea is welcome, >> Vasia. >> >> Windows Live: Keep your friends up to date with what you do online. >> > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From sekikawa at msi.co.jp Wed Jan 13 02:07:24 2010 From: sekikawa at msi.co.jp (Takuya Sekikawa) Date: Wed, 13 Jan 2010 17:07:24 +0900 Subject: [petsc-users] On what condition is useful MPI-based solution? Message-ID: <20100113164840.42C6.SEKIKAWA@msi.co.jp> Dear SLEPc/PETSc team, I tried to run SLEPc's samples program ex1 in MPI-based multi-PC environment. (1-PC vs 2-PCs) (ex) [running ex1 in only 1 PC] ---------------------------------------------------------- $ time -p mpiexec -n 1 ./ex1 -n 4000 -eps_max_it 10000 ... real 76.2 ---------------------------------------------------------- says, ex1 took 76.2 seconds. next, I run same sample in 2 PCs environment. [running ex1 in 2 PCs] ---------------------------------------------------------- $ time -p mpiexec -n 2 ./ex1 -n 4000 -eps_max_it 10000 ... real 265.54 ---------------------------------------------------------- I got 265.54 seconds. (slower than single PC) [Q1] Can ex1 sample speed up with MPI?, if so, generally on what condition? [Q2] Generally, Is MPI only useful in very large matrix? Now I have to solve eigenvalue problem of 1M x 1M matrix, Should I use MPI-based system? Thanks, Takuya --------------------------------------------------------------- Takuya Sekikawa Mathematical Systems, Inc sekikawa at msi.co.jp --------------------------------------------------------------- From jroman at dsic.upv.es Wed Jan 13 02:50:57 2010 From: jroman at dsic.upv.es (Jose E. Roman) Date: Wed, 13 Jan 2010 09:50:57 +0100 Subject: [petsc-users] On what condition is useful MPI-based solution? In-Reply-To: <20100113164840.42C6.SEKIKAWA@msi.co.jp> References: <20100113164840.42C6.SEKIKAWA@msi.co.jp> Message-ID: On 13/01/2010, Takuya Sekikawa wrote: > Dear SLEPc/PETSc team, > > I tried to run SLEPc's samples program ex1 in MPI-based multi-PC > environment. (1-PC vs 2-PCs) > > (ex) > [running ex1 in only 1 PC] > ---------------------------------------------------------- > $ time -p mpiexec -n 1 ./ex1 -n 4000 -eps_max_it 10000 > ... > real 76.2 > ---------------------------------------------------------- > says, ex1 took 76.2 seconds. > > next, I run same sample in 2 PCs environment. > > [running ex1 in 2 PCs] > ---------------------------------------------------------- > $ time -p mpiexec -n 2 ./ex1 -n 4000 -eps_max_it 10000 > ... > real 265.54 > ---------------------------------------------------------- > I got 265.54 seconds. (slower than single PC) > > [Q1] > Can ex1 sample speed up with MPI?, if so, generally on what condition? Yes. The same example on my desktop computer (Intel Core 2 Duo): With -n 1 --> real 33.99 With -n 2 --> real 21.90 If you simply have two PCs connected via a slow network, then you cannot expect good speedup. Try in a cluster with fast network. On the other hand, a better way to measure the parallel execution time is to edit the source file and put PetscGetTime around the Solve call. > > [Q2] > Generally, Is MPI only useful in very large matrix? > Now I have to solve eigenvalue problem of 1M x 1M matrix, > Should I use MPI-based system? For a 1 million matrix I would suggest to run in parallel on an MPI cluster. However, a single computer might be enough if the matrix is very sparse, you need very few eigenvalues, and/or the system has enough memory (but in that case, be prepared for very long response times, depending on how your problem converges). Jose > > Thanks, > Takuya > --------------------------------------------------------------- > Takuya Sekikawa > Mathematical Systems, Inc > sekikawa at msi.co.jp > --------------------------------------------------------------- > From sekikawa at msi.co.jp Wed Jan 13 03:21:06 2010 From: sekikawa at msi.co.jp (Takuya Sekikawa) Date: Wed, 13 Jan 2010 18:21:06 +0900 Subject: [petsc-users] On what condition is useful MPI-based solution? In-Reply-To: References: <20100113164840.42C6.SEKIKAWA@msi.co.jp> Message-ID: <20100113181637.42D2.SEKIKAWA@msi.co.jp> Hi Jose, Thank you for very quick reply. On Wed, 13 Jan 2010 09:50:57 +0100 "Jose E. Roman" wrote: > On 13/01/2010, Takuya Sekikawa wrote: > > > Dear SLEPc/PETSc team, > > > > I tried to run SLEPc's samples program ex1 in MPI-based multi-PC > > environment. (1-PC vs 2-PCs) > > > > (ex) > > [running ex1 in only 1 PC] > > ---------------------------------------------------------- > > $ time -p mpiexec -n 1 ./ex1 -n 4000 -eps_max_it 10000 > > ... > > real 76.2 > > ---------------------------------------------------------- > > says, ex1 took 76.2 seconds. > > > > next, I run same sample in 2 PCs environment. > > > > [running ex1 in 2 PCs] > > ---------------------------------------------------------- > > $ time -p mpiexec -n 2 ./ex1 -n 4000 -eps_max_it 10000 > > ... > > real 265.54 > > ---------------------------------------------------------- > > I got 265.54 seconds. (slower than single PC) > > > > [Q1] > > Can ex1 sample speed up with MPI?, if so, generally on what condition? > > Yes. The same example on my desktop computer (Intel Core 2 Duo): > With -n 1 --> real 33.99 > With -n 2 --> real 21.90 > If you simply have two PCs connected via a slow network, then you cannot expect good speedup. Try in a cluster with fast network. > On the other hand, a better way to measure the parallel execution time is to edit the source file and put PetscGetTime around the Solve call. Thank you for benchmarking on your environment. that result is quite useful for me. Certainly my current test network is not fast. Probably that is the problem. thank you. > > [Q2] > > Generally, Is MPI only useful in very large matrix? > > Now I have to solve eigenvalue problem of 1M x 1M matrix, > > Should I use MPI-based system? > > For a 1 million matrix I would suggest to run in parallel on an MPI cluster. However, a single computer might be enough if the matrix is very sparse, you need very few eigenvalues, and/or the system has enough memory (but in that case, be prepared for very long response times, depending on how your problem converges). Also this advice is really meaningful to me, thank you so much! Takuya > Jose > > > > > Thanks, > > Takuya > > --------------------------------------------------------------- > > Takuya Sekikawa > > Mathematical Systems, Inc > > sekikawa at msi.co.jp > > --------------------------------------------------------------- > > > --------------------------------------------------------------- Takuya Sekikawa Mathematical Systems, Inc sekikawa at msi.co.jp --------------------------------------------------------------- From sekikawa at msi.co.jp Wed Jan 13 03:33:13 2010 From: sekikawa at msi.co.jp (Takuya Sekikawa) Date: Wed, 13 Jan 2010 18:33:13 +0900 Subject: [petsc-users] On what condition is useful MPI-based solution? In-Reply-To: References: <20100113164840.42C6.SEKIKAWA@msi.co.jp> Message-ID: <20100113182234.42D5.SEKIKAWA@msi.co.jp> I have a few more questions. Just in case I want to ask. [Q1] In general, like ex1, Don't we need any change to source code to run program with MPI-based system? in other words, to run SLEPc-based program in MPI-based system, all I need to do is changing ./configure option (--with-mpi=1, etc) and re-compiling (without changing source) ? [Q2] Can I use intel-compiler (icc/icpc) on MPI-based SLEPc application? (because intel compiler is really fast so I want to use it on MPI-based system too) Takuya On Wed, 13 Jan 2010 09:50:57 +0100 "Jose E. Roman" wrote: > > On 13/01/2010, Takuya Sekikawa wrote: > > > Dear SLEPc/PETSc team, > > > > I tried to run SLEPc's samples program ex1 in MPI-based multi-PC > > environment. (1-PC vs 2-PCs) > > > > (ex) > > [running ex1 in only 1 PC] > > ---------------------------------------------------------- > > $ time -p mpiexec -n 1 ./ex1 -n 4000 -eps_max_it 10000 > > ... > > real 76.2 > > ---------------------------------------------------------- > > says, ex1 took 76.2 seconds. > > > > next, I run same sample in 2 PCs environment. > > > > [running ex1 in 2 PCs] > > ---------------------------------------------------------- > > $ time -p mpiexec -n 2 ./ex1 -n 4000 -eps_max_it 10000 > > ... > > real 265.54 > > ---------------------------------------------------------- > > I got 265.54 seconds. (slower than single PC) > > > > [Q1] > > Can ex1 sample speed up with MPI?, if so, generally on what condition? > > Yes. The same example on my desktop computer (Intel Core 2 Duo): > With -n 1 --> real 33.99 > With -n 2 --> real 21.90 > If you simply have two PCs connected via a slow network, then you cannot expect good speedup. Try in a cluster with fast network. > On the other hand, a better way to measure the parallel execution time is to edit the source file and put PetscGetTime around the Solve call. > > > > > [Q2] > > Generally, Is MPI only useful in very large matrix? > > Now I have to solve eigenvalue problem of 1M x 1M matrix, > > Should I use MPI-based system? > > For a 1 million matrix I would suggest to run in parallel on an MPI cluster. However, a single computer might be enough if the matrix is very sparse, you need very few eigenvalues, and/or the system has enough memory (but in that case, be prepared for very long response times, depending on how your problem converges). > Jose > > > > > Thanks, > > Takuya > > --------------------------------------------------------------- > > Takuya Sekikawa > > Mathematical Systems, Inc > > sekikawa at msi.co.jp > > --------------------------------------------------------------- > > > --------------------------------------------------------------- ? Takuya Sekikawa ??? Mathematical Systems, Inc ? sekikawa at msi.co.jp --------------------------------------------------------------- From jroman at dsic.upv.es Wed Jan 13 03:43:35 2010 From: jroman at dsic.upv.es (Jose E. Roman) Date: Wed, 13 Jan 2010 10:43:35 +0100 Subject: [petsc-users] On what condition is useful MPI-based solution? In-Reply-To: <20100113182234.42D5.SEKIKAWA@msi.co.jp> References: <20100113164840.42C6.SEKIKAWA@msi.co.jp> <20100113182234.42D5.SEKIKAWA@msi.co.jp> Message-ID: <482CBF44-606A-4C98-B6CA-AD042E63D0AB@dsic.upv.es> On 13/01/2010, Takuya Sekikawa wrote: > I have a few more questions. > > Just in case I want to ask. > > [Q1] > In general, like ex1, Don't we need any change to source code to > run program with MPI-based system? > in other words, to run SLEPc-based program in MPI-based system, > all I need to do is changing ./configure option (--with-mpi=1, etc) > and re-compiling (without changing source) ? No change in the program. Just enable MPI in the installation. > > [Q2] > Can I use intel-compiler (icc/icpc) on MPI-based SLEPc application? > (because intel compiler is really fast so I want to use it on MPI- > based > system too) > > Takuya Yes, no problem. From jed at 59A2.org Wed Jan 13 06:47:55 2010 From: jed at 59A2.org (Jed Brown) Date: Wed, 13 Jan 2010 13:47:55 +0100 Subject: [petsc-users] On what condition is useful MPI-based solution? In-Reply-To: References: <20100113164840.42C6.SEKIKAWA@msi.co.jp> Message-ID: <87ockycgsk.fsf@59A2.org> On Wed, 13 Jan 2010 09:50:57 +0100, "Jose E. Roman" wrote: > On the other hand, a better way to measure the parallel execution time > is to edit the source file and put PetscGetTime around the Solve call. Or use -log_summary Jed From ildar.kk at gmail.com Wed Jan 13 08:12:00 2010 From: ildar.kk at gmail.com (Ildar Rakhmanov) Date: Wed, 13 Jan 2010 09:12:00 -0500 Subject: [petsc-users] petsc 3.0 vs petsc 2.3 Message-ID: <6cf9cc2d1001130612r329f01dan75d3344e59d1982b@mail.gmail.com> Hi: We have piece of code which is using Petsc linear equations solver in serial fashion. The version linked to Petsc 2.3.3.15 works fine. When we try to use Petsc 3.0.0.10 we start getting errors like this [0]PETSC ERROR: --------------------- Error Message ------------------------------------ [0]PETSC ERROR: Corrupted Petsc object! [0]PETSC ERROR: Missing Inode Structure! [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: Petsc Release Version 3.0.0, Patch 10, Tue Nov 24 16:38:09 CST 2009 [0]PETSC ERROR: See docs/changes/index.html for recent updates. [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. [0]PETSC ERROR: See docs/index.html for manual pages. [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: ****/driverFlow on a i686-pc-l named ******* by **** Mon Jan 4 18:49:59 2010 [0]PETSC ERROR: Libraries linked from =/home/eni/local1/hpc/lib [0]PETSC ERROR: Configure run at Tue Dec 29 12:44:48 2009 [0]PETSC ERROR: Configure options --prefix==****** --with-mpi=0 --with-cc=gcc --with-fc=gfortran -with-cxx=g++ --download-f-blas-lapack=1 --download-lusol=1 --download-superlu=1 --download-umfpack=1 --with-shared=0 [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: MatSolve_Inode() line 793 in src/mat/impls/aij/seq/inode.c [0]PETSC ERROR: MatSolve() line 2624 in src/mat/interface/matrix.c [0]PETSC ERROR: PCApply_ILU() line 346 in src/ksp/pc/impls/factor/ilu/ilu.c [0]PETSC ERROR: PCApply() line 357 in src/ksp/pc/interface/precon.c [0]PETSC ERROR: KSPInitialResidual() line 57 in src/ksp/ksp/interface/itres.c [0]PETSC ERROR: KSPSolve_GMRES() line 241 in src/ksp/ksp/impls/gmres/gmres.c [0]PETSC ERROR: KSPSolve() line 385 in src/ksp/ksp/interface/itfunc.c Could you please advise what can fix this problem? Ildar Khabibrakhmanov -------------- next part -------------- An HTML attachment was scrubbed... URL: From balay at mcs.anl.gov Wed Jan 13 08:27:53 2010 From: balay at mcs.anl.gov (Satish Balay) Date: Wed, 13 Jan 2010 08:27:53 -0600 (CST) Subject: [petsc-users] petsc 3.0 vs petsc 2.3 In-Reply-To: <6cf9cc2d1001130612r329f01dan75d3344e59d1982b@mail.gmail.com> References: <6cf9cc2d1001130612r329f01dan75d3344e59d1982b@mail.gmail.com> Message-ID: Is this application making PETSc calls from fortran? You'll have to go through the changes file - and make sure the prototype changes are applied to your code [a fortran compiler doesn't help with prototype checks here]. http://www.mcs.anl.gov/petsc/petsc-as/documentation/changes/300.html [If the above doesn't catch all changes] use valgrind to figureout which function/s are corrupting data - and then check if the code is using the function prototype correctly [by verifying with the manpage] http://www.mcs.anl.gov/petsc/petsc-as/documentation/troubleshooting.html#Corrupt Satish On Wed, 13 Jan 2010, Ildar Rakhmanov wrote: > Hi: > > We have piece of code which is using Petsc linear equations solver in serial > fashion. > The version linked to Petsc 2.3.3.15 works fine. > When we try to use Petsc 3.0.0.10 we start getting errors like this > > > [0]PETSC ERROR: --------------------- Error Message > ------------------------------------ > [0]PETSC ERROR: Corrupted Petsc object! > [0]PETSC ERROR: Missing Inode Structure! > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: Petsc Release Version 3.0.0, Patch 10, Tue Nov 24 16:38:09 > CST 2009 > [0]PETSC ERROR: See docs/changes/index.html for recent updates. > [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. > [0]PETSC ERROR: See docs/index.html for manual pages. > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: ****/driverFlow on a i686-pc-l named ******* by **** Mon > Jan 4 18:49:59 2010 > [0]PETSC ERROR: Libraries linked from =/home/eni/local1/hpc/lib > [0]PETSC ERROR: Configure run at Tue Dec 29 12:44:48 2009 > [0]PETSC ERROR: Configure options --prefix==****** --with-mpi=0 > --with-cc=gcc --with-fc=gfortran -with-cxx=g++ --download-f-blas-lapack=1 > --download-lusol=1 --download-superlu=1 --download-umfpack=1 --with-shared=0 > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: MatSolve_Inode() line 793 in src/mat/impls/aij/seq/inode.c > [0]PETSC ERROR: MatSolve() line 2624 in src/mat/interface/matrix.c > [0]PETSC ERROR: PCApply_ILU() line 346 in src/ksp/pc/impls/factor/ilu/ilu.c > [0]PETSC ERROR: PCApply() line 357 in src/ksp/pc/interface/precon.c > [0]PETSC ERROR: KSPInitialResidual() line 57 in > src/ksp/ksp/interface/itres.c > [0]PETSC ERROR: KSPSolve_GMRES() line 241 in src/ksp/ksp/impls/gmres/gmres.c > [0]PETSC ERROR: KSPSolve() line 385 in src/ksp/ksp/interface/itfunc.c > > > Could you please advise what can fix this problem? > > Ildar Khabibrakhmanov > From balay at mcs.anl.gov Wed Jan 13 08:44:20 2010 From: balay at mcs.anl.gov (Satish Balay) Date: Wed, 13 Jan 2010 08:44:20 -0600 (CST) Subject: [petsc-users] On what condition is useful MPI-based solution? In-Reply-To: <482CBF44-606A-4C98-B6CA-AD042E63D0AB@dsic.upv.es> References: <20100113164840.42C6.SEKIKAWA@msi.co.jp> <20100113182234.42D5.SEKIKAWA@msi.co.jp> <482CBF44-606A-4C98-B6CA-AD042E63D0AB@dsic.upv.es> Message-ID: On Wed, 13 Jan 2010, Jose E. Roman wrote: > > On 13/01/2010, Takuya Sekikawa wrote: > > > I have a few more questions. > > > > Just in case I want to ask. > > > > [Q1] > > In general, like ex1, Don't we need any change to source code to > > run program with MPI-based system? > > in other words, to run SLEPc-based program in MPI-based system, > > all I need to do is changing ./configure option (--with-mpi=1, etc) > > and re-compiling (without changing source) ? > > No change in the program. Just enable MPI in the installation. Note that the examples are written to be parallel - and run with or without MPI. You cannot take any random sequential code, compile it with MPI - and expect it to run parallely. Eventhough SLEPc/PETSc hide most of the MPI related stuff from the user - there is generally some user code that should be MPI aware. [in ex1 - the matrix assembly is written to be MPI aware - and the data is distrubuted and assembled parallely] > > > > [Q2] > > Can I use intel-compiler (icc/icpc) on MPI-based SLEPc application? > > (because intel compiler is really fast so I want to use it on MPI-based > > system too) > > Yes, Note: Faster compilers speedup sequential part of the code [hence the overall runtime for the parallel run aswell]. But your bottleneck is MPI/communication cost - which won't change. So your parallel scalability will be skewed further. To improve parallel scalability - you'll have to get the fastest network you can - between the nodes. And then install MPI that can perform well on the given network setup. [and as Jed mentioned -log_summary is a better tool to compare performance] Satish From u.tabak at tudelft.nl Wed Jan 13 09:16:06 2010 From: u.tabak at tudelft.nl (Umut Tabak) Date: Wed, 13 Jan 2010 16:16:06 +0100 Subject: [petsc-users] MatSetValues for nonzeros of a sparse matrix Message-ID: <4B4DE3B6.2010708@tudelft.nl> Dear all, I am reading some FE system matrices from a commercial FE code. I wrote an interface where I export them in matrix market format. Also I keep the row and column indices of the non-zero values and as well the non-zeros values in arrays of proper type. The question which I could not conclude is that how to use MatSetValues for sparse matrix assignments, I mean should I keep all the values(non-zeros also) in my inputs?( There are also some pointers for block assignments but they work in the same way I guess) I have tested with the below code. If I supply only the non-zero diagonal entries, I get the first row set as expected but this did not seem so practical to me, because to get what I want accomplished I should keep all the zero entries as well. And I thought that there should be a far more efficient way to apply this. (Diagonal assignment is just an example.) I am sorry if this question is too naive :) Any comments is appreciated. Best regards, Umut Mat A; /* linear system matrix */ PetscErrorCode ierr; PetscInt n = 4; PetscInt rI=0,cI=0; PetscViewer viewer; PetscInt index; // // // row indices,m column indices,n of Matrix A const int idxm[] = { 0,1,2,3 }; const int idxn[] = { 0,1,2,3 }; // values in the matrix A, row major order PetscScalar valuesMat[]={ 1,0,0,0,0,2,0,0,0,0,3,0,0,0,0,4 }; //PetscScalar valuesMat[]={ 1,2,3,4 }; /* should be the 1st call after the declaration and definition * block */ PetscInitialize(&argc,&args,(char *)0,help); // added by U.Tabak // intermediate assigments are // set the values of the Matrix A ierr = MatSetValues(A, n, idxm, n, idxn, valuesMat,INSERT_VALUES); CHKERRQ(ierr); // done so the 2nd argument should be // MAT_FLUSH_ASSEMBLY ierr = MatAssemblyBegin(A,MAT_FINAL_ASSEMBLY); CHKERRQ(ierr); ierr = MatAssemblyEnd(A,MAT_FINAL_ASSEMBLY); CHKERRQ(ierr); ierr = MatView(A,PETSC_VIEWER_STDOUT_WORLD); CHKERRQ(ierr); ierr = PetscFinalize();CHKERRQ(ierr); return 0; From knepley at gmail.com Wed Jan 13 09:21:20 2010 From: knepley at gmail.com (Matthew Knepley) Date: Wed, 13 Jan 2010 09:21:20 -0600 Subject: [petsc-users] MatSetValues for nonzeros of a sparse matrix In-Reply-To: <4B4DE3B6.2010708@tudelft.nl> References: <4B4DE3B6.2010708@tudelft.nl> Message-ID: On Wed, Jan 13, 2010 at 9:16 AM, Umut Tabak wrote: > Dear all, > > I am reading some FE system matrices from a commercial FE code. I wrote an > interface where I export them in matrix market format. Also I keep the row > and column indices of the non-zero values and as well the non-zeros values > in arrays of proper type. > > The question which I could not conclude is that how to use MatSetValues for > sparse matrix assignments, I mean should I keep all the values(non-zeros > also) in my inputs?( There are also some pointers for block assignments but > they work in the same way I guess) > > I have tested with the below code. If I supply only the non-zero diagonal > entries, I get the first row set as expected but this did not seem so > practical to me, because to get what I want accomplished I should keep all > the zero entries as well. And I thought that there should be a far more > efficient way to apply this. (Diagonal assignment is just an example.) I am > sorry if this question is too naive :) > > Any comments is appreciated. > Best regards, > Umut > > Mat A; /* linear system matrix */ > PetscErrorCode ierr; > PetscInt n = 4; > PetscInt rI=0,cI=0; > PetscViewer viewer; > PetscInt index; > // > // > // row indices,m column indices,n of Matrix A > const int idxm[] = { 0,1,2,3 }; > const int idxn[] = { 0,1,2,3 }; > // values in the matrix A, row major order > PetscScalar valuesMat[]={ 1,0,0,0,0,2,0,0,0,0,3,0,0,0,0,4 }; > //PetscScalar valuesMat[]={ 1,2,3,4 }; > /* should be the 1st call after the declaration and definition > * block > */ > PetscInitialize(&argc,&args,(char *)0,help); > // added by U.Tabak > // intermediate assigments are > // set the values of the Matrix A > ierr = MatSetValues(A, n, idxm, n, idxn, valuesMat,INSERT_VALUES); > CHKERRQ(ierr); > Should be for(int r = 0; r < 4; ++r) { MatSetValues(A, 1, idxm[r], 1, idxn[r], &valuesMat[r*4+r], INSERT_VALUES); } The simplest way to use MatSetValues is to call it once per row. Matt > // done so the 2nd argument should be > // MAT_FLUSH_ASSEMBLY > ierr = MatAssemblyBegin(A,MAT_FINAL_ASSEMBLY); > CHKERRQ(ierr); > ierr = MatAssemblyEnd(A,MAT_FINAL_ASSEMBLY); > CHKERRQ(ierr); > ierr = MatView(A,PETSC_VIEWER_STDOUT_WORLD); > CHKERRQ(ierr); > ierr = PetscFinalize();CHKERRQ(ierr); > return 0; > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From jed at 59A2.org Wed Jan 13 09:33:46 2010 From: jed at 59A2.org (Jed Brown) Date: Wed, 13 Jan 2010 16:33:46 +0100 Subject: [petsc-users] MatSetValues for nonzeros of a sparse matrix In-Reply-To: References: <4B4DE3B6.2010708@tudelft.nl> Message-ID: <87fx6ac945.fsf@59A2.org> On Wed, 13 Jan 2010 09:21:20 -0600, Matthew Knepley wrote: > Should be > > for(int r = 0; r < 4; ++r) { > MatSetValues(A, 1, idxm[r], 1, idxn[r], &valuesMat[r*4+r], INSERT_VALUES); actually MatSetValues(A, 1, &idxm[r], 1, &idxn[r], &valuesMat[r*4+r], INSERT_VALUES); or MatSetValue(A, idxm[r], idxn[r], valuesMat[r*4+r], INSERT_VALUES); But considering that > Also I keep the row and column indices of the non-zero values and as > well the non-zeros values in arrays of proper type. you should loop over rows and insert all entries of that row at once. Jed From u.tabak at tudelft.nl Wed Jan 13 09:53:20 2010 From: u.tabak at tudelft.nl (Umut Tabak) Date: Wed, 13 Jan 2010 16:53:20 +0100 Subject: [petsc-users] MatSetValues for nonzeros of a sparse matrix In-Reply-To: <87fx6ac945.fsf@59A2.org> References: <4B4DE3B6.2010708@tudelft.nl> <87fx6ac945.fsf@59A2.org> Message-ID: <4B4DEC70.9010000@tudelft.nl> Jed Brown wrote: > MatSetValues(A, 1, &idxm[r], 1, &idxn[r], &valuesMat[r*4+r], INSERT_VALUES); > > or > > MatSetValue(A, idxm[r], idxn[r], valuesMat[r*4+r], INSERT_VALUES); > > Thanks for the quick replies, the above, I guess, still means to keep the zeros but below explanation is more practical from an efficiency point of view. >> Also I keep the row and column indices of the non-zero values and as >> well the non-zeros values in arrays of proper type. >> > > you should loop over rows and insert all entries of that row at once. > This was where I was confused :), I should keep the non-zeros of the row and set them with the loop. Thanks Jed. Umut From jed at 59A2.org Wed Jan 13 10:01:03 2010 From: jed at 59A2.org (Jed Brown) Date: Wed, 13 Jan 2010 17:01:03 +0100 Subject: [petsc-users] MatSetValues for nonzeros of a sparse matrix In-Reply-To: <4B4DEC70.9010000@tudelft.nl> References: <4B4DE3B6.2010708@tudelft.nl> <87fx6ac945.fsf@59A2.org> <4B4DEC70.9010000@tudelft.nl> Message-ID: <87eiluc7uo.fsf@59A2.org> On Wed, 13 Jan 2010 16:53:20 +0100, Umut Tabak wrote: > Jed Brown wrote: > > MatSetValues(A, 1, &idxm[r], 1, &idxn[r], &valuesMat[r*4+r], INSERT_VALUES); > > > > or > > > > MatSetValue(A, idxm[r], idxn[r], valuesMat[r*4+r], INSERT_VALUES); > > > > > Thanks for the quick replies, the above, I guess, still means to keep > the zeros but below explanation is more practical from an efficiency > point of view. No, the index r*4+r is selecting only the diagonal (nonzero entries) from the example array that you provided. > >> Also I keep the row and column indices of the non-zero values and as > >> well the non-zeros values in arrays of proper type. > >> > > > > you should loop over rows and insert all entries of that row at once. > > > This was where I was confused :), I should keep the non-zeros of the row > and set them with the loop. You should call MatSetValues once per row. For example, suppose we have row = 8; cols[] = {2,7,8,9,14}; vals[] = {-1.0,-2.0,6.0,-1.5,-1.2}; then we would call MatSetValues(A,1,&row,5,cols,vals,INSERT_VALUES); to set these 5 values on row 8. I hope this helps. Jed From sekikawa at msi.co.jp Wed Jan 13 18:04:50 2010 From: sekikawa at msi.co.jp (Takuya Sekikawa) Date: Thu, 14 Jan 2010 09:04:50 +0900 Subject: [petsc-users] On what condition is useful MPI-based solution? In-Reply-To: References: <20100113164840.42C6.SEKIKAWA@msi.co.jp> Message-ID: <20100114085409.E42D.SEKIKAWA@msi.co.jp> Dear SLEPc/PETSc team, On Wed, 13 Jan 2010 09:50:57 +0100 "Jose E. Roman" wrote: > > [Q2] > > Generally, Is MPI only useful in very large matrix? > > Now I have to solve eigenvalue problem of 1M x 1M matrix, > > Should I use MPI-based system? > > For a 1 million matrix I would suggest to run in parallel on an MPI cluster. However, a single computer might be enough if the matrix is very sparse, you need very few eigenvalues, and/or the system has enough memory (but in that case, be prepared for very long response times, depending on how your problem converges). > Jose Do you have any example of how many PCs need to solve this level of problem? and also how many memories do each PC should have? I would like to know how much resources do I need (PCs, memories) and how long it takes to solve. (not precisely, rough estimation is enough) Problem is 1M x 1M symmetric sparse matrix, and only a few eigenpairs (at least 1) I need. so currently I plan to use lanczos or krylov-schur method, with EPS_NEV=1. Takuya --------------------------------------------------------------- Takuya Sekikawa Mathematical Systems, Inc sekikawa at msi.co.jp --------------------------------------------------------------- From jroman at dsic.upv.es Thu Jan 14 02:41:04 2010 From: jroman at dsic.upv.es (Jose E. Roman) Date: Thu, 14 Jan 2010 09:41:04 +0100 Subject: [petsc-users] On what condition is useful MPI-based solution? In-Reply-To: <20100114085409.E42D.SEKIKAWA@msi.co.jp> References: <20100113164840.42C6.SEKIKAWA@msi.co.jp> <20100114085409.E42D.SEKIKAWA@msi.co.jp> Message-ID: El 14/01/2010, a las 01:04, Takuya Sekikawa escribi?: > Dear SLEPc/PETSc team, > > On Wed, 13 Jan 2010 09:50:57 +0100 > "Jose E. Roman" wrote: > >>> [Q2] >>> Generally, Is MPI only useful in very large matrix? >>> Now I have to solve eigenvalue problem of 1M x 1M matrix, >>> Should I use MPI-based system? >> >> For a 1 million matrix I would suggest to run in parallel on an MPI cluster. However, a single computer might be enough if the matrix is very sparse, you need very few eigenvalues, and/or the system has enough memory (but in that case, be prepared for very long response times, depending on how your problem converges). >> Jose > > Do you have any example of how many PCs need to solve this level of > problem? and also how many memories do each PC should have? > > I would like to know how much resources do I need (PCs, memories) > and how long it takes to solve. (not precisely, rough estimation > is enough) > > Problem is 1M x 1M symmetric sparse matrix, and only a few eigenpairs > (at least 1) I need. so currently I plan to use lanczos or krylov-schur > method, with EPS_NEV=1. For nev=1, the workspace used by the solver is moderate. Maybe 20 vectors of length 1M (i.e. 160 Mbytes). If the matrix is really sparse, say 30 nonzero elements per row, then the matrix is not a problem either (roughly 364 Mbytes). So one PC may be enough. If the matrix is much denser, or you have convergence problems, or you need to do shift-and-invert, then things get worse. The execution time basically depends on convergence. For instance, ex1 with n=1M will have very bad convergence, but your problem may not. Run with -eps_monitor or -eps_monitor_draw to see how the solver is progressing. Jose > > Takuya > --------------------------------------------------------------- > Takuya Sekikawa > Mathematical Systems, Inc > sekikawa at msi.co.jp > --------------------------------------------------------------- > > From xy2102 at columbia.edu Thu Jan 14 09:14:07 2010 From: xy2102 at columbia.edu ((Rebecca) Xuefei YUAN) Date: Thu, 14 Jan 2010 10:14:07 -0500 Subject: [petsc-users] A flag for FormFunction. In-Reply-To: References: <20100113164840.42C6.SEKIKAWA@msi.co.jp> <20100114085409.E42D.SEKIKAWA@msi.co.jp> Message-ID: <20100114101407.f4fsupgozockk4ss@cubmail.cc.columbia.edu> Dear all, I would like to setup a flag for the FormFunction like: if (FormFunction is called by SNESComputeFunction){ flag = 1; }else{ flag = 0; } Is there any way I can know which function called FormFunction? I can tell very clear in debug mode, but I need this information in the code. Thanks very much! Rebecca From jed at 59A2.org Thu Jan 14 09:25:19 2010 From: jed at 59A2.org (Jed Brown) Date: Thu, 14 Jan 2010 16:25:19 +0100 Subject: [petsc-users] A flag for FormFunction. In-Reply-To: <20100114101407.f4fsupgozockk4ss@cubmail.cc.columbia.edu> References: <20100114101407.f4fsupgozockk4ss@cubmail.cc.columbia.edu> Message-ID: <87vdf4bteo.fsf@59A2.org> On Thu, 14 Jan 2010 10:14:07 -0500, "(Rebecca) Xuefei YUAN" wrote: > Dear all, > > I would like to setup a flag for the FormFunction like: > > if (FormFunction is called by SNESComputeFunction){ > flag = 1; > }else{ > flag = 0; > } > > Is there any way I can know which function called FormFunction? I can > tell very clear in debug mode, but I need this information in the code. Why do you need it in the code? If your FormFunction does something different when called in a different context, it's very easy to end up with incorrect results as you change the algorithm. In any case, PETSc only calls it through SNESComputeFunction(), if you want it to do something different when *you* call it, you can put the flag in your user context. Jed From bsmith at mcs.anl.gov Thu Jan 14 09:29:33 2010 From: bsmith at mcs.anl.gov (Barry Smith) Date: Thu, 14 Jan 2010 09:29:33 -0600 Subject: [petsc-users] A flag for FormFunction. In-Reply-To: <20100114101407.f4fsupgozockk4ss@cubmail.cc.columbia.edu> References: <20100113164840.42C6.SEKIKAWA@msi.co.jp> <20100114085409.E42D.SEKIKAWA@msi.co.jp> <20100114101407.f4fsupgozockk4ss@cubmail.cc.columbia.edu> Message-ID: <85DF5225-C2DC-4C80-B5C2-3B7D698579C7@mcs.anl.gov> There really isn't a mechanism to know by whom a function is called. Do you want this so you know when the function is being called directly vs when it is being called in computing a Jacobian derivative with finite differencing? Do you want the function to do something different in the two cases? Barry On Jan 14, 2010, at 9:14 AM, (Rebecca) Xuefei YUAN wrote: > Dear all, > > I would like to setup a flag for the FormFunction like: > > if (FormFunction is called by SNESComputeFunction){ > flag = 1; > }else{ > flag = 0; > } > > Is there any way I can know which function called FormFunction? I > can tell very clear in debug mode, but I need this information in > the code. > > Thanks very much! > > Rebecca > From jed at 59A2.org Thu Jan 14 10:02:22 2010 From: jed at 59A2.org (Jed Brown) Date: Thu, 14 Jan 2010 17:02:22 +0100 Subject: [petsc-users] A flag for FormFunction. In-Reply-To: <85DF5225-C2DC-4C80-B5C2-3B7D698579C7@mcs.anl.gov> References: <20100113164840.42C6.SEKIKAWA@msi.co.jp> <20100114085409.E42D.SEKIKAWA@msi.co.jp> <20100114101407.f4fsupgozockk4ss@cubmail.cc.columbia.edu> <85DF5225-C2DC-4C80-B5C2-3B7D698579C7@mcs.anl.gov> Message-ID: <87ska8brox.fsf@59A2.org> On Thu, 14 Jan 2010 09:29:33 -0600, Barry Smith wrote: > Do you want this so you know when the function is being called > directly vs when it is being called in computing a Jacobian derivative > with finite differencing? Do you want the function to do something > different in the two cases? Actually, here's a use case where it would be good to know. Suppose that F does some caching of intermediate quantities needed to apply J, or that make assembly of J cheaper. For example, I apply full-accuracy Jacobians unassembled, and assemble something else for preconditioning. But suppose I wanted to use a preconditioning matrix computed by coloring, that perhaps also discards some coupling. For example, we might precondition a 3-component STENCIL_BOX discretization with STENCIL_STAR where inter-field coupling has been discarded [1]. Since I have a fast full-accuracy Jacobian, I wouldn't want to use the inexact thing as the real Jacobian, and caching the intermediate terms isn't a lot of work, but it might add 20% to the cost of evaluating F(x) and thus be worthwhile if the line search typically needs less than 5 steps or so. But perhaps more importantly, if I'm recaching J while computing the preconditioner with FD, then J will be evaluated at x+eps*y instead of at x which may hurt quadratic convergence. Jed [1] STENCIL_STAR preconditioning isn't just hypothetical, I have a nice 2D p-Laplacian example where -snes_mf_operator with STENCIL_STAR preconditioning beats the analytic (STENCIL_BOX) Jacobian in every measure. From bsmith at mcs.anl.gov Thu Jan 14 10:49:11 2010 From: bsmith at mcs.anl.gov (Barry Smith) Date: Thu, 14 Jan 2010 10:49:11 -0600 Subject: [petsc-users] A flag for FormFunction. In-Reply-To: <87ska8brox.fsf@59A2.org> References: <20100113164840.42C6.SEKIKAWA@msi.co.jp> <20100114085409.E42D.SEKIKAWA@msi.co.jp> <20100114101407.f4fsupgozockk4ss@cubmail.cc.columbia.edu> <85DF5225-C2DC-4C80-B5C2-3B7D698579C7@mcs.anl.gov> <87ska8brox.fsf@59A2.org> Message-ID: <43611BED-C4F9-4915-A084-386B35CB7EC0@mcs.anl.gov> Jed, In this case I think one would provide a different function to the code that computes the Jacobian then one provides to the SNES as the function evaluation. So, for example, one forms a Mat with MatFDColoringCreate() and then provides the other function with MatFDColoringSetFunction()? (Rather than having one function and try to figure out how it is being calling and doing different things). Does this handle all situations you are thinking of? Barry On Jan 14, 2010, at 10:02 AM, Jed Brown wrote: > On Thu, 14 Jan 2010 09:29:33 -0600, Barry Smith > wrote: >> Do you want this so you know when the function is being called >> directly vs when it is being called in computing a Jacobian >> derivative >> with finite differencing? Do you want the function to do something >> different in the two cases? > > Actually, here's a use case where it would be good to know. > > Suppose that F does some caching of intermediate quantities needed to > apply J, or that make assembly of J cheaper. For example, I apply > full-accuracy Jacobians unassembled, and assemble something else for > preconditioning. > > But suppose I wanted to use a preconditioning matrix computed by > coloring, that perhaps also discards some coupling. For example, we > might precondition a 3-component STENCIL_BOX discretization with > STENCIL_STAR where inter-field coupling has been discarded [1]. > > Since I have a fast full-accuracy Jacobian, I wouldn't want to use the > inexact thing as the real Jacobian, and caching the intermediate terms > isn't a lot of work, but it might add 20% to the cost of evaluating > F(x) > and thus be worthwhile if the line search typically needs less than 5 > steps or so. But perhaps more importantly, if I'm recaching J while > computing the preconditioner with FD, then J will be evaluated at > x+eps*y instead of at x which may hurt quadratic convergence. > > Jed > > [1] STENCIL_STAR preconditioning isn't just hypothetical, I have a > nice > 2D p-Laplacian example where -snes_mf_operator with STENCIL_STAR > preconditioning beats the analytic (STENCIL_BOX) Jacobian in every > measure. From bsmith at mcs.anl.gov Thu Jan 14 11:01:25 2010 From: bsmith at mcs.anl.gov (Barry Smith) Date: Thu, 14 Jan 2010 11:01:25 -0600 Subject: [petsc-users] A flag for FormFunction. In-Reply-To: <20100114115038.137f6tleqso8koc8@cubmail.cc.columbia.edu> References: <20100113164840.42C6.SEKIKAWA@msi.co.jp> <20100114085409.E42D.SEKIKAWA@msi.co.jp> <20100114101407.f4fsupgozockk4ss@cubmail.cc.columbia.edu> <85DF5225-C2DC-4C80-B5C2-3B7D698579C7@mcs.anl.gov> <20100114115038.137f6tleqso8koc8@cubmail.cc.columbia.edu> Message-ID: <18D8CBD9-8120-4F96-B47B-18B856659376@mcs.anl.gov> This isn't really a good solution. Could you explain what you want to do differently with the function and when, then we may have a better solution. Barry On Jan 14, 2010, at 10:50 AM, (Rebecca) Xuefei YUAN wrote: > Dear Barry, > > Yes, I would like the FormFunction to do different things depending > on if it is called by SNESComputeFunction. > > I found a solution to it, but I am not sure whether it is good/easy > enough to track. > > I put up a integer "callNumber" into my user-defined contents, and > set it to be 0 at the beginning of each time iteration. > > When FormFunction is called, it is first called by > SNESComputeFunction, followed by other PETSc functions, so my if > condition is > if (callNumber==0){ > do A > }else{ > do B > } > > Before PetscFunctionReturn(0), > > callNumber + = 1; > > So far it is good, and it did what I would like to. > > Is there a better way of doing it? > > Thanks a lot! > > Rebecca > > Quoting Barry Smith : > >> >> There really isn't a mechanism to know by whom a function is called. >> >> Do you want this so you know when the function is being called >> directly vs when it is being called in computing a Jacobian >> derivative >> with finite differencing? Do you want the function to do something >> different in the two cases? >> >> Barry >> >> On Jan 14, 2010, at 9:14 AM, (Rebecca) Xuefei YUAN wrote: >> >>> Dear all, >>> >>> I would like to setup a flag for the FormFunction like: >>> >>> if (FormFunction is called by SNESComputeFunction){ >>> flag = 1; >>> }else{ >>> flag = 0; >>> } >>> >>> Is there any way I can know which function called FormFunction? I >>> can tell very clear in debug mode, but I need this information in >>> the code. >>> >>> Thanks very much! >>> >>> Rebecca >>> > > > > -- > (Rebecca) Xuefei YUAN > Department of Applied Physics and Applied Mathematics > Columbia University > Tel:917-399-8032 > www.columbia.edu/~xy2102 > From jed at 59A2.org Thu Jan 14 11:18:51 2010 From: jed at 59A2.org (Jed Brown) Date: Thu, 14 Jan 2010 18:18:51 +0100 Subject: [petsc-users] A flag for FormFunction. In-Reply-To: <43611BED-C4F9-4915-A084-386B35CB7EC0@mcs.anl.gov> References: <20100113164840.42C6.SEKIKAWA@msi.co.jp> <20100114085409.E42D.SEKIKAWA@msi.co.jp> <20100114101407.f4fsupgozockk4ss@cubmail.cc.columbia.edu> <85DF5225-C2DC-4C80-B5C2-3B7D698579C7@mcs.anl.gov> <87ska8brox.fsf@59A2.org> <43611BED-C4F9-4915-A084-386B35CB7EC0@mcs.anl.gov> Message-ID: <87ockwbo5g.fsf@59A2.org> On Thu, 14 Jan 2010 10:49:11 -0600, Barry Smith wrote: > Does this handle all situations you are thinking of? This is definitely the right thing for my example. The other case would be -snes_mf_operator where part of the preconditioner is cached while evaluating F. But this is also not a big deal, since again, the user can SNESGetJacobian(snes,&A,0,0,0); PetscTypeCompare(A,MFFD,&flg); if (flg) MatMFFDSetFunction(A,FormFunctionNoCache,ctx); Doing something different based on the call stack is a really bad thing, I don't think FormFunction should be allowed to find out, just that any cases that seem to call for it should have an alternate mechanism (which they do). Jed From xy2102 at columbia.edu Thu Jan 14 11:39:35 2010 From: xy2102 at columbia.edu ((Rebecca) Xuefei YUAN) Date: Thu, 14 Jan 2010 12:39:35 -0500 Subject: [petsc-users] A flag for FormFunction. In-Reply-To: <18D8CBD9-8120-4F96-B47B-18B856659376@mcs.anl.gov> References: <20100113164840.42C6.SEKIKAWA@msi.co.jp> <20100114085409.E42D.SEKIKAWA@msi.co.jp> <20100114101407.f4fsupgozockk4ss@cubmail.cc.columbia.edu> <85DF5225-C2DC-4C80-B5C2-3B7D698579C7@mcs.anl.gov> <20100114115038.137f6tleqso8koc8@cubmail.cc.columbia.edu> <18D8CBD9-8120-4F96-B47B-18B856659376@mcs.anl.gov> Message-ID: <20100114123935.zqdplzigow0og0ks@cubmail.cc.columbia.edu> Dear Barry, For example, I am solving a time-dependent problem as dfdt = F(f(xi,eta),g(xi,eta)); dgdt = G(f(xi,eta),g(xi,eta)) on some moving grid, i.e., at each time step, the grid is not the same as the one in the previous step. In FormFunction, there are two parts, the first part is giving the spatial only residual function F(f,g) and G(f,g); the second part is to add the time-dependent part into it. There are some quantity h I would like to save, for example, the velocity of the moving grid, h = h(x(xi,eta),y(xi,eta),f(xi,eta)) and I would like to save this h as an output file for tracking purpose. This h is calculated and stored in the second part, but if the FormFunction is called by functions other than SNESComputeJacobian, f,x,y are changed from previous nonliner/linear iterations and h has been changed, too. I use the flag "callNumber" to open two different files in which this h is stored. if (callNumber==0){ open "useful file" and save current h. }else{ open "abandon file" and save current h. } If FormFunction is called by SNESComputeFunction, it means the f,x,y are just the content from previous time iteration, on the contrary, they are the content from previous nonlinear/linear iterations. The short version of the code for FormFunction is: #undef __FUNCT__ #define __FUNCT__ "FormFunction" PetscErrorCode FormFunction(SNES snes,Vec X,Vec F,void*dummg) { // begin of FormFunctionLocal // begin of part 1: prepare FormFunctionLocal without temporal terms GETTING SPATIAL TERMS FOR RESIDUAL FUNCTION // end of part 1: prepare FormFunctionLocal without temporal terms // begin of part 2: add temporal terms { if (parameters->timeAccuracyOrder == 1){ // getting the velocity of the grids char v_fileName[127]; PetscViewer v_viewer; FILE *v_fp; if(parameters->callNumber==0){ sprintf(v_fileName, "twgcqt2unffni_grid_velocity_tx%i_ty%i_x%i_y%i_nl%i_nt%i.m", info.mx, info.my,parameters->mxfield,parameters->myfield,parameters->numberOfLevels,parameters->currentTimeStep); }else{ sprintf(v_fileName, "unwanted_twgcqt2unffni_grid_velocity_tx%i_ty%i_x%i_y%i_nl%i_nt%i.m", info.mx, info.my,parameters->mxfield,parameters->myfield,parameters->numberOfLevels,parameters->currentTimeStep); } ierr = PetscViewerASCIIOpen (PETSC_COMM_WORLD,v_fileName , &v_viewer); CHKERRQ (ierr); ierr = PetscViewerSetFormat (v_viewer, PETSC_VIEWER_ASCII_MATLAB); CHKERRQ (ierr); ierr = PetscViewerASCIIPrintf(v_viewer,"function [V1,V2,Vr1,Vr2,Vc1,Vc2,ct] = twgcqt2unffni_grid_velocity_tx%i_ty%i_x%i_y%i_nl%i_nt%i", info.mx, info.my,parameters->mxfield,parameters->myfield,parameters->numberOfLevels,parameters->currentTimeStep);CHKERRQ(ierr); ierr = PetscFOpen(PETSC_COMM_WORLD,v_fileName,"a",&v_fp); CHKERRQ(ierr); if(v_fp){ PetscViewerASCIIPrintf(v_viewer,"ct = %1.5f;\n",parameters->currentTime); for (j = jFirst; j <= jLast; j++){ for (i = iFirst; i <= iLast; i++){ if(j!=pye){ V1 = tempiJ*hx*hy*(x_xi *0.5*ihy*(field[j+1][i].phi - field[j-1][i].phi) - x_eta*0.5*ihx*(field[j][i+1].phi - field[j][i-1].phi)); V2 = -tempiJ*hx*hy*(y_eta*0.5*ihx*(field[j][i+1].phi - field[j][i-1].phi) - y_xi *0.5*ihy*(field[j+1][i].phi - field[j-1][i].phi)); // begin of adding temporal terms ADDING TEMPORAL TERMS // end of adding temporal terms PetscViewerASCIIPrintf(v_viewer,"V1(%i,%i)=%1.20f;\n",j+1,i+1,V1); PetscViewerASCIIPrintf(v_viewer,"V2(%i,%i)=%1.20f;\n",j+1,i+1,V2); }else if (j==pye){ V1 = tempiJ*hx*hy*(x_xi *0.5*ihy*((2*field[j][i].phi - field[j-1][i].phi) - field[j-1][i].phi) - x_eta*0.5*ihx*(field[j][i+1].phi - field[j][i-1].phi)); V2 = -tempiJ*hx*hy*(y_eta*0.5*ihx*(field[j][i+1].phi - field[j][i-1].phi) - y_xi *0.5*ihy*((2*field[j][i].phi - field[j-1][i].phi) - field[j-1][i].phi)); PetscViewerASCIIPrintf(v_viewer,"V1(%i,%i)=%1.20f;\n",j+1,i+1,V1); PetscViewerASCIIPrintf(v_viewer,"V2(%i,%i)=%1.20f;\n",j+1,i+1,V2); } } } } ierr = PetscFClose(PETSC_COMM_WORLD,v_fp);CHKERRQ(ierr); PetscViewerDestroy(v_viewer); } } // end of part 2: add temporal terms // end of FormFunctionLocal parameters->callNumber = parameters->callNumber+1; PetscFunctionReturn(0); } And in main loop, the callNumber has been set to zero at the beginning of each time iteration. The analytic Jacobian is not provided as ierr = DMMGSetSNES(dmmg, FormFunction,0);CHKERRQ(ierr); Thanks very much! Rebecca Quoting Barry Smith : > > This isn't really a good solution. > > Could you explain what you want to do differently with the function > and when, then we may have a better solution. > > Barry > > On Jan 14, 2010, at 10:50 AM, (Rebecca) Xuefei YUAN wrote: > >> Dear Barry, >> >> Yes, I would like the FormFunction to do different things depending >> on if it is called by SNESComputeFunction. >> >> I found a solution to it, but I am not sure whether it is good/easy >> enough to track. >> >> I put up a integer "callNumber" into my user-defined contents, and >> set it to be 0 at the beginning of each time iteration. >> >> When FormFunction is called, it is first called by >> SNESComputeFunction, followed by other PETSc functions, so my if >> condition is >> if (callNumber==0){ >> do A >> }else{ >> do B >> } >> >> Before PetscFunctionReturn(0), >> >> callNumber + = 1; >> >> So far it is good, and it did what I would like to. >> >> Is there a better way of doing it? >> >> Thanks a lot! >> >> Rebecca >> >> Quoting Barry Smith : >> >>> >>> There really isn't a mechanism to know by whom a function is called. >>> >>> Do you want this so you know when the function is being called >>> directly vs when it is being called in computing a Jacobian derivative >>> with finite differencing? Do you want the function to do something >>> different in the two cases? >>> >>> Barry >>> >>> On Jan 14, 2010, at 9:14 AM, (Rebecca) Xuefei YUAN wrote: >>> >>>> Dear all, >>>> >>>> I would like to setup a flag for the FormFunction like: >>>> >>>> if (FormFunction is called by SNESComputeFunction){ >>>> flag = 1; >>>> }else{ >>>> flag = 0; >>>> } >>>> >>>> Is there any way I can know which function called FormFunction? I >>>> can tell very clear in debug mode, but I need this information >>>> in the code. >>>> >>>> Thanks very much! >>>> >>>> Rebecca >>>> >> >> >> >> -- >> (Rebecca) Xuefei YUAN >> Department of Applied Physics and Applied Mathematics >> Columbia University >> Tel:917-399-8032 >> www.columbia.edu/~xy2102 >> -- (Rebecca) Xuefei YUAN Department of Applied Physics and Applied Mathematics Columbia University Tel:917-399-8032 www.columbia.edu/~xy2102 From bsmith at mcs.anl.gov Thu Jan 14 12:39:46 2010 From: bsmith at mcs.anl.gov (Barry Smith) Date: Thu, 14 Jan 2010 12:39:46 -0600 Subject: [petsc-users] A flag for FormFunction. In-Reply-To: <20100114123935.zqdplzigow0og0ks@cubmail.cc.columbia.edu> References: <20100113164840.42C6.SEKIKAWA@msi.co.jp> <20100114085409.E42D.SEKIKAWA@msi.co.jp> <20100114101407.f4fsupgozockk4ss@cubmail.cc.columbia.edu> <85DF5225-C2DC-4C80-B5C2-3B7D698579C7@mcs.anl.gov> <20100114115038.137f6tleqso8koc8@cubmail.cc.columbia.edu> <18D8CBD9-8120-4F96-B47B-18B856659376@mcs.anl.gov> <20100114123935.zqdplzigow0og0ks@cubmail.cc.columbia.edu> Message-ID: <0EE08341-7BEC-4799-B215-7DAFBA075C0B@mcs.anl.gov> I would never have that file output from the FormFunction(); it doesn't have much meaning. because FormFunction() can be called for many reasons. If you want to, for example, save the solution at the end of each Newton step then use SNESMonitorSet() and provide a function that does the output, this will be called after each Newton step. On Jan 14, 2010, at 11:39 AM, (Rebecca) Xuefei YUAN wrote: > Dear Barry, > > For example, I am solving a time-dependent problem as > > dfdt = F(f(xi,eta),g(xi,eta)); > dgdt = G(f(xi,eta),g(xi,eta)) > > on some moving grid, i.e., at each time step, the grid is not the > same as the one in the previous step. > > In FormFunction, there are two parts, the first part is giving the > spatial only residual function F(f,g) and G(f,g); the second part is > to add the time-dependent part into it. > > There are some quantity h I would like to save, for example, the > velocity of the moving grid, > > h = h(x(xi,eta),y(xi,eta),f(xi,eta)) > > and I would like to save this h as an output file for tracking > purpose. > > This h is calculated and stored in the second part, but if the > FormFunction is called by functions other than SNESComputeJacobian, This is not really accurate. If the line search is needed then the FormFunction() will be called by SNESComputeFunction() several times during a single Newton step. The input to SNESComputeFunction() is not always meaningful. Barry > f,x,y are changed from previous nonliner/linear iterations and h has > been changed, too. > > I use the flag "callNumber" to open two different files in which > this h is stored. > > if (callNumber==0){ > open "useful file" and save current h. > }else{ > open "abandon file" and save current h. > } > > If FormFunction is called by SNESComputeFunction, it means the f,x,y > are just the content from previous time iteration, on the contrary, > they are the content from previous nonlinear/linear iterations. > > The short version of the code for FormFunction is: > > > #undef __FUNCT__ > #define __FUNCT__ "FormFunction" > PetscErrorCode FormFunction(SNES snes,Vec X,Vec F,void*dummg) > { > // begin of FormFunctionLocal > // begin of part 1: prepare FormFunctionLocal without temporal terms > GETTING SPATIAL TERMS FOR RESIDUAL FUNCTION > // end of part 1: prepare FormFunctionLocal without temporal terms > // begin of part 2: add temporal terms > { > if (parameters->timeAccuracyOrder == 1){ > // getting the velocity of the grids > char v_fileName[127]; > PetscViewer v_viewer; > FILE *v_fp; > if(parameters->callNumber==0){ > sprintf(v_fileName, "twgcqt2unffni_grid_velocity_tx%i_ty%i_x%i_y > %i_nl%i_nt%i.m", info.mx, info.my,parameters->mxfield,parameters- > >myfield,parameters->numberOfLevels,parameters->currentTimeStep); > }else{ > sprintf(v_fileName, "unwanted_twgcqt2unffni_grid_velocity_tx%i_ty > %i_x%i_y%i_nl%i_nt%i.m", info.mx, info.my,parameters- > >mxfield,parameters->myfield,parameters->numberOfLevels,parameters- > >currentTimeStep); > } > ierr = PetscViewerASCIIOpen (PETSC_COMM_WORLD,v_fileName , > &v_viewer); CHKERRQ (ierr); > ierr = PetscViewerSetFormat (v_viewer, > PETSC_VIEWER_ASCII_MATLAB); CHKERRQ (ierr); > ierr = PetscViewerASCIIPrintf(v_viewer,"function > [V1,V2,Vr1,Vr2,Vc1,Vc2,ct] = twgcqt2unffni_grid_velocity_tx%i_ty%i_x > %i_y%i_nl%i_nt%i", info.mx, info.my,parameters->mxfield,parameters- > >myfield,parameters->numberOfLevels,parameters- > >currentTimeStep);CHKERRQ(ierr); > ierr = PetscFOpen(PETSC_COMM_WORLD,v_fileName,"a",&v_fp); > CHKERRQ(ierr); > if(v_fp){ > PetscViewerASCIIPrintf(v_viewer,"ct = %1.5f;\n",parameters- > >currentTime); > for (j = jFirst; j <= jLast; j++){ > for (i = iFirst; i <= iLast; i++){ > if(j!=pye){ > V1 = tempiJ*hx*hy*(x_xi *0.5*ihy*(field[j+1] > [i].phi - field[j-1][i].phi) - x_eta*0.5*ihx*(field[j][i+1].phi - > field[j][i-1].phi)); > V2 = -tempiJ*hx*hy*(y_eta*0.5*ihx*(field[j][i > +1].phi - field[j][i-1].phi) - y_xi *0.5*ihy*(field[j+1][i].phi - > field[j-1][i].phi)); > > // begin of adding temporal terms > ADDING TEMPORAL TERMS > // end of adding temporal terms > > PetscViewerASCIIPrintf(v_viewer,"V1(%i, > %i)=%1.20f;\n",j+1,i+1,V1); > PetscViewerASCIIPrintf(v_viewer,"V2(%i, > %i)=%1.20f;\n",j+1,i+1,V2); > > }else if (j==pye){ > V1 = tempiJ*hx*hy*(x_xi > *0.5*ihy*((2*field[j][i].phi - field[j-1][i].phi) - field[j-1] > [i].phi) - x_eta*0.5*ihx*(field[j][i+1].phi - field[j][i-1].phi)); > V2 = - > tempiJ*hx*hy*(y_eta*0.5*ihx*(field[j][i+1].phi - field[j][i-1].phi) > - y_xi *0.5*ihy*((2*field[j][i].phi - field[j-1][i].phi) - field[j-1] > [i].phi)); > > PetscViewerASCIIPrintf(v_viewer,"V1(%i, > %i)=%1.20f;\n",j+1,i+1,V1); > PetscViewerASCIIPrintf(v_viewer,"V2(%i, > %i)=%1.20f;\n",j+1,i+1,V2); > } > } > } > } > ierr = PetscFClose(PETSC_COMM_WORLD,v_fp);CHKERRQ(ierr); > PetscViewerDestroy(v_viewer); > } > } > // end of part 2: add temporal terms > // end of FormFunctionLocal > parameters->callNumber = parameters->callNumber+1; > PetscFunctionReturn(0); > } > > And in main loop, the callNumber has been set to zero at the > beginning of each time iteration. > > The analytic Jacobian is not provided as > > ierr = DMMGSetSNES(dmmg, FormFunction,0);CHKERRQ(ierr); > > Thanks very much! > > Rebecca > > > > > > > Quoting Barry Smith : > >> >> This isn't really a good solution. >> >> Could you explain what you want to do differently with the function >> and when, then we may have a better solution. >> >> Barry >> >> On Jan 14, 2010, at 10:50 AM, (Rebecca) Xuefei YUAN wrote: >> >>> Dear Barry, >>> >>> Yes, I would like the FormFunction to do different things >>> depending on if it is called by SNESComputeFunction. >>> >>> I found a solution to it, but I am not sure whether it is good/ >>> easy enough to track. >>> >>> I put up a integer "callNumber" into my user-defined contents, >>> and set it to be 0 at the beginning of each time iteration. >>> >>> When FormFunction is called, it is first called by >>> SNESComputeFunction, followed by other PETSc functions, so my if >>> condition is >>> if (callNumber==0){ >>> do A >>> }else{ >>> do B >>> } >>> >>> Before PetscFunctionReturn(0), >>> >>> callNumber + = 1; >>> >>> So far it is good, and it did what I would like to. >>> >>> Is there a better way of doing it? >>> >>> Thanks a lot! >>> >>> Rebecca >>> >>> Quoting Barry Smith : >>> >>>> >>>> There really isn't a mechanism to know by whom a function is >>>> called. >>>> >>>> Do you want this so you know when the function is being called >>>> directly vs when it is being called in computing a Jacobian >>>> derivative >>>> with finite differencing? Do you want the function to do something >>>> different in the two cases? >>>> >>>> Barry >>>> >>>> On Jan 14, 2010, at 9:14 AM, (Rebecca) Xuefei YUAN wrote: >>>> >>>>> Dear all, >>>>> >>>>> I would like to setup a flag for the FormFunction like: >>>>> >>>>> if (FormFunction is called by SNESComputeFunction){ >>>>> flag = 1; >>>>> }else{ >>>>> flag = 0; >>>>> } >>>>> >>>>> Is there any way I can know which function called FormFunction? >>>>> I can tell very clear in debug mode, but I need this >>>>> information in the code. >>>>> >>>>> Thanks very much! >>>>> >>>>> Rebecca >>>>> >>> >>> >>> >>> -- >>> (Rebecca) Xuefei YUAN >>> Department of Applied Physics and Applied Mathematics >>> Columbia University >>> Tel:917-399-8032 >>> www.columbia.edu/~xy2102 >>> > > > > -- > (Rebecca) Xuefei YUAN > Department of Applied Physics and Applied Mathematics > Columbia University > Tel:917-399-8032 > www.columbia.edu/~xy2102 > From vyan2000 at gmail.com Fri Jan 15 13:25:58 2010 From: vyan2000 at gmail.com (Ryan Yan) Date: Fri, 15 Jan 2010 14:25:58 -0500 Subject: [petsc-users] scalar abs() Message-ID: Hi All, I am solving nonlinear BVP on structure grid using DM. For the variable v, I have the following term included in my residual: abs(v)*v My question is: In DMMGSetSNESLocal(), for the FormFunctionLocal(), at interior node i, What is the right PETSc function to use for the abs in: abs(x[i].v) * x[i].v Thanks, Yan -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Fri Jan 15 13:29:14 2010 From: knepley at gmail.com (Matthew Knepley) Date: Fri, 15 Jan 2010 13:29:14 -0600 Subject: [petsc-users] scalar abs() In-Reply-To: References: Message-ID: fabs(). Matt On Fri, Jan 15, 2010 at 1:25 PM, Ryan Yan wrote: > Hi All, > I am solving nonlinear BVP on structure grid using DM. > > For the variable v, I have the following term included in my residual: > abs(v)*v > > My question is: > In DMMGSetSNESLocal(), for the FormFunctionLocal(), at interior node i, > What is the right PETSc function to use for the abs in: > abs(x[i].v) * x[i].v > > Thanks, > Yan > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Fri Jan 15 13:29:28 2010 From: knepley at gmail.com (Matthew Knepley) Date: Fri, 15 Jan 2010 13:29:28 -0600 Subject: [petsc-users] scalar abs() In-Reply-To: References: Message-ID: You can also use PetscAbsScalar(). Matt On Fri, Jan 15, 2010 at 1:29 PM, Matthew Knepley wrote: > fabs(). > > Matt > > > On Fri, Jan 15, 2010 at 1:25 PM, Ryan Yan wrote: > >> Hi All, >> I am solving nonlinear BVP on structure grid using DM. >> >> For the variable v, I have the following term included in my residual: >> abs(v)*v >> >> My question is: >> In DMMGSetSNESLocal(), for the FormFunctionLocal(), at interior node i, >> What is the right PETSc function to use for the abs in: >> abs(x[i].v) * x[i].v >> >> Thanks, >> Yan >> > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From vyan2000 at gmail.com Fri Jan 15 13:33:49 2010 From: vyan2000 at gmail.com (Ryan Yan) Date: Fri, 15 Jan 2010 14:33:49 -0500 Subject: [petsc-users] scalar abs() In-Reply-To: References: Message-ID: Thanks a lot, Yan On Fri, Jan 15, 2010 at 2:29 PM, Matthew Knepley wrote: > You can also use PetscAbsScalar(). > > Matt > > > On Fri, Jan 15, 2010 at 1:29 PM, Matthew Knepley wrote: > >> fabs(). >> >> Matt >> >> >> On Fri, Jan 15, 2010 at 1:25 PM, Ryan Yan wrote: >> >>> Hi All, >>> I am solving nonlinear BVP on structure grid using DM. >>> >>> For the variable v, I have the following term included in my residual: >>> abs(v)*v >>> >>> My question is: >>> In DMMGSetSNESLocal(), for the FormFunctionLocal(), at interior node i, >>> What is the right PETSc function to use for the abs in: >>> abs(x[i].v) * x[i].v >>> >>> Thanks, >>> Yan >>> >> >> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which their >> experiments lead. >> -- Norbert Wiener >> > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Robert.G.Ellis at Shaw.ca Sun Jan 17 19:07:35 2010 From: Robert.G.Ellis at Shaw.ca (Rob Ellis) Date: Sun, 17 Jan 2010 17:07:35 -0800 Subject: [petsc-users] x64 build problem Message-ID: <009601ca97da$9f3e10b0$ddba3210$@G.Ellis@Shaw.ca> Hi Petsc Experts I've been running Petsc/TAO successfully under CVF VC but need to move to 64 bit. Now, I'm trying to compile Petsc on a Windows XP64, VS2008 C/C++ and IVF 11 from the IVF compiler shell with Cygwin bash using the command ./config/configure.py \ --with-cc='win32fe cl' \ --with-cxx='win32fe cl' \ --with-fc=ifort \ --with-mpi-dir=/cygdrive/c/MPICH2 \ --download-f-blas-lapack=1 \ --with-clanguage=C++ CXXFLAGS='-DMPICH_SKIP_MPICXX -MT -GR -EHsc' and get the following error: ============================================================================ ===== Configuring PETSc to compile on your system ============================================================================ ===== TESTING: checkCCompiler from config.setCompilers(/cygdrive/z/ai/petsc-3.0.0-p9/config/BuildSystem/config/ setCompilers.py:394) UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log for details): ---------------------------------------------------------------------------- ----------- C compiler you provided with -with-cc=win32fe cl does not work **************************************************************************** ***** However if I type cl at the command line, cl seems to work $ cl Microsoft (R) C/C++ Optimizing Compiler Version 15.00.21022.08 for x64 Copyright (C) Microsoft Corporation. All rights reserved. usage: cl [ option... ] filename... [ /link linkoption... ] Any help would be most appreciated. Cheers, Rob -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Sun Jan 17 19:10:15 2010 From: knepley at gmail.com (Matthew Knepley) Date: Sun, 17 Jan 2010 19:10:15 -0600 Subject: [petsc-users] x64 build problem In-Reply-To: <4124141812321442201@unknownmsgid> References: <4124141812321442201@unknownmsgid> Message-ID: You have to send configure.log (send it to petsc-maint at mcs.anl.gov) or we have absolutely no idea what is going on. Matt On Sun, Jan 17, 2010 at 7:07 PM, Rob Ellis wrote: > Hi Petsc Experts > > > > I've been running Petsc/TAO successfully under CVF VC but need to move to > 64 bit. Now, I'm trying to compile Petsc on a Windows XP64, VS2008 C/C++ and > IVF 11 from the IVF compiler shell with Cygwin bash using the command > > > > ./config/configure.py \ > > --with-cc='win32fe cl' \ > > --with-cxx='win32fe cl' \ > > --with-fc=ifort \ > > --with-mpi-dir=/cygdrive/c/MPICH2 \ > > --download-f-blas-lapack=1 \ > > --with-clanguage=C++ CXXFLAGS='-DMPICH_SKIP_MPICXX -MT -GR -EHsc' > > > > and get the following error: > > > > > ================================================================================= > > Configuring PETSc to compile on your system > > > ================================================================================= > > TESTING: checkCCompiler from > config.setCompilers(/cygdrive/z/ai/petsc-3.0.0-p9/config/BuildSystem/config/setCompilers.py:394) > > UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log for > details): > > > --------------------------------------------------------------------------------------- > > C compiler you provided with -with-cc=win32fe cl does not work > > > ********************************************************************************* > > > > However if I type cl at the command line, cl seems to work > > > > $ cl > > Microsoft (R) C/C++ Optimizing Compiler Version 15.00.21022.08 for x64 > > Copyright (C) Microsoft Corporation. All rights reserved. > > > > usage: cl [ option... ] filename... [ /link linkoption... ] > > > > > > Any help would be most appreciated. > > Cheers, > > Rob > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From Robert.G.Ellis at Shaw.ca Sun Jan 17 19:15:13 2010 From: Robert.G.Ellis at Shaw.ca (Rob Ellis) Date: Sun, 17 Jan 2010 17:15:13 -0800 Subject: [petsc-users] x64 build problem In-Reply-To: References: <4124141812321442201@unknownmsgid> Message-ID: <00a101ca97db$b02112f0$106338d0$@G.Ellis@Shaw.ca> Hi Matt, Of course, my apologies, here it is. Thanks, Rob From: petsc-users-bounces at mcs.anl.gov [mailto:petsc-users-bounces at mcs.anl.gov] On Behalf Of Matthew Knepley Sent: Sunday, January 17, 2010 5:10 PM To: PETSc users list Subject: Re: [petsc-users] x64 build problem You have to send configure.log (send it to petsc-maint at mcs.anl.gov) or we have absolutely no idea what is going on. Matt On Sun, Jan 17, 2010 at 7:07 PM, Rob Ellis wrote: Hi Petsc Experts I've been running Petsc/TAO successfully under CVF VC but need to move to 64 bit. Now, I'm trying to compile Petsc on a Windows XP64, VS2008 C/C++ and IVF 11 from the IVF compiler shell with Cygwin bash using the command ./config/configure.py \ --with-cc='win32fe cl' \ --with-cxx='win32fe cl' \ --with-fc=ifort \ --with-mpi-dir=/cygdrive/c/MPICH2 \ --download-f-blas-lapack=1 \ --with-clanguage=C++ CXXFLAGS='-DMPICH_SKIP_MPICXX -MT -GR -EHsc' and get the following error: ============================================================================ ===== Configuring PETSc to compile on your system ============================================================================ ===== TESTING: checkCCompiler from config.setCompilers(/cygdrive/z/ai/petsc-3.0.0-p9/config/BuildSystem/config/ setCompilers.py:394) UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log for details): ---------------------------------------------------------------------------- ----------- C compiler you provided with -with-cc=win32fe cl does not work **************************************************************************** ***** However if I type cl at the command line, cl seems to work $ cl Microsoft (R) C/C++ Optimizing Compiler Version 15.00.21022.08 for x64 Copyright (C) Microsoft Corporation. All rights reserved. usage: cl [ option... ] filename... [ /link linkoption... ] Any help would be most appreciated. Cheers, Rob -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: configure.log Type: application/octet-stream Size: 33073 bytes Desc: not available URL: From bsmith at mcs.anl.gov Sun Jan 17 20:26:58 2010 From: bsmith at mcs.anl.gov (Barry Smith) Date: Sun, 17 Jan 2010 20:26:58 -0600 Subject: [petsc-users] x64 build problem In-Reply-To: <00a101ca97db$b02112f0$106338d0$@G.Ellis@Shaw.ca> References: <4124141812321442201@unknownmsgid> <00a101ca97db$b02112f0$106338d0$@G.Ellis@Shaw.ca> Message-ID: <776665AA-D718-4A1B-9B11-2A7055EECCC3@mcs.anl.gov> Try using the options --with-cc="win32fe --nodetect cl" --with- cxx="win32fe --nodetect cl" instead This business has to be fixed up very soon in petsc-dev for the next release, Barry On Jan 17, 2010, at 7:15 PM, Rob Ellis wrote: > Hi Matt, > Of course, my apologies, here it is. > Thanks, > Rob > > From: petsc-users-bounces at mcs.anl.gov [mailto:petsc-users-bounces at mcs.anl.gov > ] On Behalf Of Matthew Knepley > Sent: Sunday, January 17, 2010 5:10 PM > To: PETSc users list > Subject: Re: [petsc-users] x64 build problem > > You have to send configure.log (send it to petsc-maint at mcs.anl.gov) > or we have absolutely > no idea what is going on. > > Matt > > On Sun, Jan 17, 2010 at 7:07 PM, Rob Ellis > wrote: > Hi Petsc Experts > > I've been running Petsc/TAO successfully under CVF VC but need to > move to 64 bit. Now, I'm trying to compile Petsc on a Windows XP64, > VS2008 C/C++ and IVF 11 from the IVF compiler shell with Cygwin bash > using the command > > ./config/configure.py \ > --with-cc='win32fe cl' \ > --with-cxx='win32fe cl' \ > --with-fc=ifort \ > --with-mpi-dir=/cygdrive/c/MPICH2 \ > --download-f-blas-lapack=1 \ > --with-clanguage=C++ CXXFLAGS='-DMPICH_SKIP_MPICXX -MT -GR -EHsc' > > and get the following error: > > = > = > = > = > = > = > = > = > = > = > = > ====================================================================== > Configuring PETSc to compile on your system > = > = > = > = > = > = > = > = > = > = > = > ====================================================================== > TESTING: checkCCompiler from config.setCompilers(/cygdrive/z/ai/ > petsc-3.0.0-p9/config/BuildSystem/config/setCompilers.py:394) > UNABLE to CONFIGURE with GIVEN OPTIONS (see > configure.log for details): > --------------------------------------------------------------------------------------- > C compiler you provided with -with-cc=win32fe cl does not work > ********************************************************************************* > > However if I type cl at the command line, cl seems to work > > $ cl > Microsoft (R) C/C++ Optimizing Compiler Version 15.00.21022.08 for x64 > Copyright (C) Microsoft Corporation. All rights reserved. > > usage: cl [ option... ] filename... [ /link linkoption... ] > > > Any help would be most appreciated. > Cheers, > Rob > > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which > their experiments lead. > -- Norbert Wiener > From balay at mcs.anl.gov Sun Jan 17 22:50:15 2010 From: balay at mcs.anl.gov (Satish Balay) Date: Sun, 17 Jan 2010 22:50:15 -0600 (CST) Subject: [petsc-users] x64 build problem In-Reply-To: <776665AA-D718-4A1B-9B11-2A7055EECCC3@mcs.anl.gov> References: <4124141812321442201@unknownmsgid> <00a101ca97db$b02112f0$106338d0$@G.Ellis@Shaw.ca> <776665AA-D718-4A1B-9B11-2A7055EECCC3@mcs.anl.gov> Message-ID: Actually its --with-cc='win32fe cl --nodetect' --with-fc='win32fe ifort --nodetect' etc.. check 'Using Visual Studio 2005:' section of installation instructions http://www.mcs.anl.gov/petsc/petsc-as/documentation/installation.html Satish On Sun, 17 Jan 2010, Barry Smith wrote: > > Try using the options --with-cc="win32fe --nodetect cl" --with-cxx="win32fe > --nodetect cl" instead > > This business has to be fixed up very soon in petsc-dev for the next > release, > > > > Barry > > On Jan 17, 2010, at 7:15 PM, Rob Ellis wrote: > > > Hi Matt, > > Of course, my apologies, here it is. > > Thanks, > > Rob > > > > From: petsc-users-bounces at mcs.anl.gov > > [mailto:petsc-users-bounces at mcs.anl.gov] On Behalf Of Matthew Knepley > > Sent: Sunday, January 17, 2010 5:10 PM > > To: PETSc users list > > Subject: Re: [petsc-users] x64 build problem > > > > You have to send configure.log (send it to petsc-maint at mcs.anl.gov) or we > > have absolutely > > no idea what is going on. > > > > Matt > > > > On Sun, Jan 17, 2010 at 7:07 PM, Rob Ellis wrote: > > Hi Petsc Experts > > > > I've been running Petsc/TAO successfully under CVF VC but need to move to 64 > > bit. Now, I'm trying to compile Petsc on a Windows XP64, VS2008 C/C++ and > > IVF 11 from the IVF compiler shell with Cygwin bash using the command > > > > ./config/configure.py \ > > --with-cc='win32fe cl' \ > > --with-cxx='win32fe cl' \ > > --with-fc=ifort \ > > --with-mpi-dir=/cygdrive/c/MPICH2 \ > > --download-f-blas-lapack=1 \ > > --with-clanguage=C++ CXXFLAGS='-DMPICH_SKIP_MPICXX -MT -GR -EHsc' > > > > and get the following error: > > > > ================================================================================= > > Configuring PETSc to compile on your system > > ================================================================================= > > TESTING: checkCCompiler from > > config.setCompilers(/cygdrive/z/ai/petsc-3.0.0-p9/config/BuildSystem/config/setCompilers.py:394) > > UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log for > > details): > > --------------------------------------------------------------------------------------- > > C compiler you provided with -with-cc=win32fe cl does not work > > ********************************************************************************* > > > > However if I type cl at the command line, cl seems to work > > > > $ cl > > Microsoft (R) C/C++ Optimizing Compiler Version 15.00.21022.08 for x64 > > Copyright (C) Microsoft Corporation. All rights reserved. > > > > usage: cl [ option... ] filename... [ /link linkoption... ] > > > > > > Any help would be most appreciated. > > Cheers, > > Rob > > > > > > > > > > -- > > What most experimenters take for granted before they begin their experiments > > is infinitely more interesting than any results to which their experiments > > lead. > > -- Norbert Wiener > > > From u.tabak at tudelft.nl Mon Jan 18 09:41:51 2010 From: u.tabak at tudelft.nl (Umut Tabak) Date: Mon, 18 Jan 2010 16:41:51 +0100 Subject: [petsc-users] setting block matrices with preallocation Message-ID: <4B54813F.5060805@tudelft.nl> Dear all, Some time ago, I asked some questions about setting entries of a sparse matrix efficiently. I got some response from Jed Brown and Barry Smith. I can set the matrix entries of one sparse matrix very efficiently for the moment. I use MatSetValues and preallocate the array as explained in the manual for performance and set the rows in once in a loop. My actual goal was to set the entries of a matrix in blocks. Let me rephrase, say I would like to set the blocks of a matrix like 1 1 1 [ Kaa Kas ] or [ Kaa 0 ] does not matter. 2 [ 0 Kss ] [ Ksa Kss ] Since I can read the matrices Kaa, Ksa, Kss, and their non-zeros, row and column indices of these matrices alone. I thought that to get the same efficient setting of values, I can do some kind of block selection on a 'switch' where I can allocate the nnz, and values for the rows of this big matrix and set the row and column indices with some offsets. I mean sth like if blocks are on the same global row, say 1, change nnz for those rows in that block and allocate the necessary space and set the values of this array. But I thought this should be done before me by someone else, so just wanted some advice before coding. All blocks are pretty sparse if that helps from the point of view of ideas. If I do not do this preallocation, the performance decreases, as explained in the manual, drastically(not for this case but for stand alone matrices for instance.) I appreciate any brighter ideas. Umut From knepley at gmail.com Mon Jan 18 09:48:00 2010 From: knepley at gmail.com (Matthew Knepley) Date: Mon, 18 Jan 2010 09:48:00 -0600 Subject: [petsc-users] setting block matrices with preallocation In-Reply-To: <4B54813F.5060805@tudelft.nl> References: <4B54813F.5060805@tudelft.nl> Message-ID: I would probably do what Barry & Jed have suggested. However, if you really want to work with blocks, you can also consider what Dave has done. It is much closer to what you are asking for http://jupiter.ethz.ch/~dmay/Research/PetscExt/index.html Matt On Mon, Jan 18, 2010 at 9:41 AM, Umut Tabak wrote: > Dear all, > > Some time ago, I asked some questions about setting entries of a sparse > matrix efficiently. I got some response from Jed Brown and Barry Smith. I > can set the matrix entries of one sparse matrix very efficiently for the > moment. I use MatSetValues and preallocate the array as explained in the > manual for performance and set the rows in once in a loop. My actual goal > was to set the entries of a matrix in blocks. Let me rephrase, say I would > like to set the blocks of a matrix like > 1 1 > 1 [ Kaa Kas ] or [ Kaa 0 ] does not > matter. > 2 [ 0 Kss ] [ Ksa Kss ] > > Since I can read the matrices Kaa, Ksa, Kss, and their non-zeros, row and > column indices of these matrices alone. > > I thought that to get the same efficient setting of values, I can do some > kind of block selection on a 'switch' where I can allocate the nnz, and > values for the rows of this big matrix and set the row and column indices > with some offsets. I mean sth like if blocks are on the same global row, say > 1, change nnz for those rows in that block and allocate the necessary space > and set the values of this array. But I thought this should be done before > me by someone else, so just wanted some advice before coding. All blocks are > pretty sparse if that helps from the point of view of ideas. If I do not do > this preallocation, the performance decreases, as explained in the manual, > drastically(not for this case but for stand alone matrices for instance.) > > I appreciate any brighter ideas. > > Umut > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From u.tabak at tudelft.nl Mon Jan 18 09:52:38 2010 From: u.tabak at tudelft.nl (Umut Tabak) Date: Mon, 18 Jan 2010 16:52:38 +0100 Subject: [petsc-users] setting block matrices with preallocation In-Reply-To: References: <4B54813F.5060805@tudelft.nl> Message-ID: <4B5483C6.50606@tudelft.nl> Matthew Knepley wrote: > I would probably do what Barry & Jed have suggested. However, if you > really want to work with blocks, > you can also consider what Dave has done. It is much closer to what > you are asking for > > http://jupiter.ethz.ch/~dmay/Research/PetscExt/index.html > > > Matt Dear Matthew, Thanks for the link, it looks like what I was looking for. Umut From brockp at umich.edu Mon Jan 18 09:56:03 2010 From: brockp at umich.edu (Brock Palen) Date: Mon, 18 Jan 2010 10:56:03 -0500 Subject: [petsc-users] PETSc Podcast Available (And Thank You) Message-ID: <01C9C440-0388-4EB9-B636-642DEA639CEB@umich.edu> Thank you to Matt Knepley and Jed Brown for taking some time to talk to Jeff and I about PETSc on our HPC Podcast www.rce-cast.com, You can find the show at: http://www.rce-cast.com/index.php/Podcast/rce-24-petsc.html There is an RSS feed and iTunes link there for those who wish to follow the show, also feel free to email me with topics you would like to hear on the show. Thanks again Matt and Jed! Brock Palen www.umich.edu/~brockp Center for Advanced Computing brockp at umich.edu (734)936-1985 From xy2102 at columbia.edu Mon Jan 18 14:12:18 2010 From: xy2102 at columbia.edu ((Rebecca) Xuefei YUAN) Date: Mon, 18 Jan 2010 15:12:18 -0500 Subject: [petsc-users] valgrind errors. Message-ID: <20100118151218.isbvg6ssgw8cs0kw@cubmail.cc.columbia.edu> Hi, I got some errors from valgrind showing some errors from PETSc defined functions, but at the end of the valgrind report, it seems fine except the first two that valgrind always has for PETSc. Is this really fine or not? -------------------------------------------------------------- THERE ARE LOTS OF SIMILAR ONES BEFORE.... ==9537== Conditional jump or move depends on uninitialised value(s) ==9537== at 0x858CE93: VecMaxPointwiseDivide_Seq (dvec2.c:903) ==9537== by 0x856DB50: VecMaxPointwiseDivide (rvector.c:47) ==9537== by 0x80BFB09: SNESLineSearchCubic (ls.c:550) ==9537== by 0x80BD9F4: SNESSolve_LS (ls.c:219) ==9537== by 0x809E77F: SNESSolve (snes.c:2221) ==9537== by 0x80B7280: DMMGSolveSNES (damgsnes.c:510) ==9537== by 0x80B0ABF: DMMGSolve (damg.c:372) ==9537== by 0x8050EE0: Solve (twgcqt2unffnidgp.c:579) ==9537== by 0x804DC7F: main (twgcqt2unffnidgp.c:287) ==9537== ==9537== Conditional jump or move depends on uninitialised value(s) ==9537== at 0x80A2B4E: SNESKSPEW_PreSolve (snes.c:2854) ==9537== by 0x80A330F: SNES_KSPSolve (snes.c:2898) ==9537== by 0x80BD598: SNESSolve_LS (ls.c:191) ==9537== by 0x809E77F: SNESSolve (snes.c:2221) ==9537== by 0x80B7280: DMMGSolveSNES (damgsnes.c:510) ==9537== by 0x80B0ABF: DMMGSolve (damg.c:372) ==9537== by 0x8050EE0: Solve (twgcqt2unffnidgp.c:579) ==9537== by 0x804DC7F: main (twgcqt2unffnidgp.c:287) ==9537== ==9537== Conditional jump or move depends on uninitialised value(s) ==9537== at 0x816A0A5: KSPSetTolerances (itfunc.c:816) ==9537== by 0x80A2B96: SNESKSPEW_PreSolve (snes.c:2855) ==9537== by 0x80A330F: SNES_KSPSolve (snes.c:2898) ==9537== by 0x80BD598: SNESSolve_LS (ls.c:191) ==9537== by 0x809E77F: SNESSolve (snes.c:2221) ==9537== by 0x80B7280: DMMGSolveSNES (damgsnes.c:510) ==9537== by 0x80B0ABF: DMMGSolve (damg.c:372) ==9537== by 0x8050EE0: Solve (twgcqt2unffnidgp.c:579) ==9537== by 0x804DC7F: main (twgcqt2unffnidgp.c:287) ==9537== ==9537== Conditional jump or move depends on uninitialised value(s) ==9537== at 0x816A0C9: KSPSetTolerances (itfunc.c:817) ==9537== by 0x80A2B96: SNESKSPEW_PreSolve (snes.c:2855) ==9537== by 0x80A330F: SNES_KSPSolve (snes.c:2898) ==9537== by 0x80BD598: SNESSolve_LS (ls.c:191) ==9537== by 0x809E77F: SNESSolve (snes.c:2221) ==9537== by 0x80B7280: DMMGSolveSNES (damgsnes.c:510) ==9537== by 0x80B0ABF: DMMGSolve (damg.c:372) ==9537== by 0x8050EE0: Solve (twgcqt2unffnidgp.c:579) ==9537== by 0x804DC7F: main (twgcqt2unffnidgp.c:287) ==9537== ==9537== Conditional jump or move depends on uninitialised value(s) ==9537== at 0x817C76E: KSPDefaultConverged (iterativ.c:669) ==9537== by 0x81B5432: FGMREScycle (fgmres.c:294) ==9537== by 0x81B5AB4: KSPSolve_FGMRES (fgmres.c:370) ==9537== by 0x816554A: KSPSolve (itfunc.c:385) ==9537== by 0x80A3374: SNES_KSPSolve (snes.c:2899) ==9537== by 0x80BD598: SNESSolve_LS (ls.c:191) ==9537== by 0x809E77F: SNESSolve (snes.c:2221) ==9537== by 0x80B7280: DMMGSolveSNES (damgsnes.c:510) ==9537== by 0x80B0ABF: DMMGSolve (damg.c:372) ==9537== by 0x8050EE0: Solve (twgcqt2unffnidgp.c:579) ==9537== by 0x804DC7F: main (twgcqt2unffnidgp.c:287) ==9537== ==9537== ERROR SUMMARY: 220920 errors from 280 contexts (suppressed: 41 from 1) ==9537== malloc/free: in use at exit: 156 bytes in 11 blocks. ==9537== malloc/free: 5,714 allocs, 5,703 frees, 21,737,471 bytes allocated. ==9537== For counts of detected errors, rerun with: -v ==9537== searching for pointers to 11 not-freed blocks. ==9537== checked 1,068,204 bytes. ==9537== ==9537== LEAK SUMMARY: ==9537== definitely lost: 156 bytes in 11 blocks. ==9537== possibly lost: 0 bytes in 0 blocks. ==9537== still reachable: 0 bytes in 0 blocks. ==9537== suppressed: 0 bytes in 0 blocks. ==9537== Rerun with --leak-check=full to see details of leaked memory. Thanks, -- (Rebecca) Xuefei YUAN Department of Applied Physics and Applied Mathematics Columbia University Tel:917-399-8032 www.columbia.edu/~xy2102 From jed at 59A2.org Mon Jan 18 14:22:44 2010 From: jed at 59A2.org (Jed Brown) Date: Mon, 18 Jan 2010 21:22:44 +0100 Subject: [petsc-users] valgrind errors. In-Reply-To: <20100118151218.isbvg6ssgw8cs0kw@cubmail.cc.columbia.edu> References: <20100118151218.isbvg6ssgw8cs0kw@cubmail.cc.columbia.edu> Message-ID: <87my0b88ob.fsf@59A2.org> On Mon, 18 Jan 2010 15:12:18 -0500, "(Rebecca) Xuefei YUAN" wrote: > Hi, > > I got some errors from valgrind showing some errors from PETSc defined > functions, but at the end of the valgrind report, it seems fine except > the first two that valgrind always has for PETSc. These don't look "fine", perhaps you're passing in uninitialized values at some point. You can run with valgrind --track-origins=yes to get some info about where the uninitialized values come from. None of these show up in the examples for me. All I get are a couple places where memory (cached attributes in communicators) isn't completely cleaned up. Jed From jed at 59A2.org Mon Jan 18 14:37:35 2010 From: jed at 59A2.org (Jed Brown) Date: Mon, 18 Jan 2010 21:37:35 +0100 Subject: [petsc-users] valgrind errors. In-Reply-To: <20100118153321.2hdea0mkcg040ck0@cubmail.cc.columbia.edu> References: <20100118151218.isbvg6ssgw8cs0kw@cubmail.cc.columbia.edu> <87my0b88ob.fsf@59A2.org> <20100118153321.2hdea0mkcg040ck0@cubmail.cc.columbia.edu> Message-ID: <87ljfv87zk.fsf@59A2.org> On Mon, 18 Jan 2010 15:33:21 -0500, "(Rebecca) Xuefei YUAN" wrote: > Dear Jed, > > Thanks for the reply. > > However, there was an error message for this option: It was introduced in valgrind-3.4.0 (and it's worth upgrading just for this feature). Jed From xy2102 at columbia.edu Mon Jan 18 15:32:22 2010 From: xy2102 at columbia.edu ((Rebecca) Xuefei YUAN) Date: Mon, 18 Jan 2010 16:32:22 -0500 Subject: [petsc-users] valgrind errors. In-Reply-To: <87ljfv87zk.fsf@59A2.org> References: <20100118151218.isbvg6ssgw8cs0kw@cubmail.cc.columbia.edu> <87my0b88ob.fsf@59A2.org> <20100118153321.2hdea0mkcg040ck0@cubmail.cc.columbia.edu> <87ljfv87zk.fsf@59A2.org> Message-ID: <20100118163222.s8gzpi76v4ss0so8@cubmail.cc.columbia.edu> Dear Jed, I got some origins, but I still do not understand what is going on here? Here is a piece of uninitialised value from ==9725== Conditional jump or move depends on uninitialised value(s) ==9725== at 0x86EFB20: dnrm2_ (dnrm2.f:46) ==9725== by 0x857ECB1: VecNorm_Seq (bvec2.c:60) ==9725== by 0x8556C5B: VecNormBegin (comb.c:507) ==9725== by 0x80BCE2D: SNESSolve_LS (ls.c:164) ==9725== by 0x809E6E3: SNESSolve (snes.c:2221) ==9725== by 0x80B71E4: DMMGSolveSNES (damgsnes.c:510) ==9725== by 0x80B0A23: DMMGSolve (damg.c:372) ==9725== by 0x8050E41: Solve (twgcqt2unffnidgp.c:580) ==9725== by 0x804DBE0: main (twgcqt2unffnidgp.c:288) ==9725== Uninitialised value was created by a heap allocation ==9725== at 0x4022BF3: malloc (vg_replace_malloc.c:195) ==9725== by 0x8628840: PetscMallocAlign (mal.c:40) ==9725== by 0x862992F: PetscTrMallocDefault (mtr.c:194) ==9725== by 0x8583503: VecCreate_Seq (bvec2.c:809) ==9725== by 0x8569F43: VecSetType (vecreg.c:54) ==9725== by 0x858E01B: VecCreateSeq (vseqcr.c:40) ==9725== by 0x821F345: DACreateLocalVector (dalocal.c:82) ==9725== by 0x825E26C: DMCreateLocalVector (dm.c:115) ==9725== by 0x821F7FF: DMGetLocalVector (dalocal.c:139) ==9725== by 0x8055EBD: FormFunction (twgcqt2unffnidgp.c:1033) ==9725== by 0x80953E8: SNESComputeFunction (snes.c:1016) ==9725== by 0x80BCD06: SNESSolve_LS (ls.c:159) What would it mean by "Uninitialised value was created by a heap allocation"? Thanks a lot! Rebecca Quoting Jed Brown : > On Mon, 18 Jan 2010 15:33:21 -0500, "(Rebecca) Xuefei YUAN" > wrote: >> Dear Jed, >> >> Thanks for the reply. >> >> However, there was an error message for this option: > > It was introduced in valgrind-3.4.0 (and it's worth upgrading just for > this feature). > > Jed > > -- (Rebecca) Xuefei YUAN Department of Applied Physics and Applied Mathematics Columbia University Tel:917-399-8032 www.columbia.edu/~xy2102 From knepley at gmail.com Mon Jan 18 15:53:22 2010 From: knepley at gmail.com (Matthew Knepley) Date: Mon, 18 Jan 2010 15:53:22 -0600 Subject: [petsc-users] valgrind errors. In-Reply-To: <20100118163222.s8gzpi76v4ss0so8@cubmail.cc.columbia.edu> References: <20100118151218.isbvg6ssgw8cs0kw@cubmail.cc.columbia.edu> <87my0b88ob.fsf@59A2.org> <20100118153321.2hdea0mkcg040ck0@cubmail.cc.columbia.edu> <87ljfv87zk.fsf@59A2.org> <20100118163222.s8gzpi76v4ss0so8@cubmail.cc.columbia.edu> Message-ID: On Mon, Jan 18, 2010 at 3:32 PM, (Rebecca) Xuefei YUAN wrote: > Dear Jed, > > I got some origins, but I still do not understand what is going on here? > > Here is a piece of uninitialised value from > > ==9725== Conditional jump or move depends on uninitialised value(s) > ==9725== at 0x86EFB20: dnrm2_ (dnrm2.f:46) > ==9725== by 0x857ECB1: VecNorm_Seq (bvec2.c:60) > ==9725== by 0x8556C5B: VecNormBegin (comb.c:507) > ==9725== by 0x80BCE2D: SNESSolve_LS (ls.c:164) > ==9725== by 0x809E6E3: SNESSolve (snes.c:2221) > ==9725== by 0x80B71E4: DMMGSolveSNES (damgsnes.c:510) > ==9725== by 0x80B0A23: DMMGSolve (damg.c:372) > ==9725== by 0x8050E41: Solve (twgcqt2unffnidgp.c:580) > ==9725== by 0x804DBE0: main (twgcqt2unffnidgp.c:288) > ==9725== Uninitialised value was created by a heap allocation > ==9725== at 0x4022BF3: malloc (vg_replace_malloc.c:195) > ==9725== by 0x8628840: PetscMallocAlign (mal.c:40) > ==9725== by 0x862992F: PetscTrMallocDefault (mtr.c:194) > ==9725== by 0x8583503: VecCreate_Seq (bvec2.c:809) > ==9725== by 0x8569F43: VecSetType (vecreg.c:54) > ==9725== by 0x858E01B: VecCreateSeq (vseqcr.c:40) > ==9725== by 0x821F345: DACreateLocalVector (dalocal.c:82) > ==9725== by 0x825E26C: DMCreateLocalVector (dm.c:115) > ==9725== by 0x821F7FF: DMGetLocalVector (dalocal.c:139) > ==9725== by 0x8055EBD: FormFunction (twgcqt2unffnidgp.c:1033) > ==9725== by 0x80953E8: SNESComputeFunction (snes.c:1016) > ==9725== by 0x80BCD06: SNESSolve_LS (ls.c:159) > > What would it mean by "Uninitialised value was created by a heap > allocation"? > Heap allocation means calling malloc(). It looks like you get a vector but do not set some components. Matt > Thanks a lot! > > Rebecca > > Quoting Jed Brown : > > On Mon, 18 Jan 2010 15:33:21 -0500, "(Rebecca) Xuefei YUAN" < >> xy2102 at columbia.edu> wrote: >> >>> Dear Jed, >>> >>> Thanks for the reply. >>> >>> However, there was an error message for this option: >>> >> >> It was introduced in valgrind-3.4.0 (and it's worth upgrading just for >> this feature). >> >> Jed >> >> >> > > > -- > (Rebecca) Xuefei YUAN > Department of Applied Physics and Applied Mathematics > Columbia University > Tel:917-399-8032 > www.columbia.edu/~xy2102 > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From jed at 59A2.org Mon Jan 18 16:03:04 2010 From: jed at 59A2.org (Jed Brown) Date: Mon, 18 Jan 2010 23:03:04 +0100 Subject: [petsc-users] valgrind errors. In-Reply-To: <20100118163222.s8gzpi76v4ss0so8@cubmail.cc.columbia.edu> References: <20100118151218.isbvg6ssgw8cs0kw@cubmail.cc.columbia.edu> <87my0b88ob.fsf@59A2.org> <20100118153321.2hdea0mkcg040ck0@cubmail.cc.columbia.edu> <87ljfv87zk.fsf@59A2.org> <20100118163222.s8gzpi76v4ss0so8@cubmail.cc.columbia.edu> Message-ID: <87k4vf8413.fsf@59A2.org> On Mon, 18 Jan 2010 16:32:22 -0500, "(Rebecca) Xuefei YUAN" wrote: > Dear Jed, > > I got some origins, but I still do not understand what is going on here? > > Here is a piece of uninitialised value from I think you have an indexing error while reading from this local vector in your residual evaluation. The contents of the array are explicitly initialized to zero in VecCreate_Seq, but when memalign is not available (your trace indicates this is so) PetscMallocAlign allocate a little extra space and aligns the result, leaving a few bytes uninitialized (memory further away may also be uninitialized, but this error isn't saying complaining about those places). Jed From bsmith at mcs.anl.gov Mon Jan 18 17:34:06 2010 From: bsmith at mcs.anl.gov (Barry Smith) Date: Mon, 18 Jan 2010 17:34:06 -0600 Subject: [petsc-users] valgrind errors. In-Reply-To: <20100118163222.s8gzpi76v4ss0so8@cubmail.cc.columbia.edu> References: <20100118151218.isbvg6ssgw8cs0kw@cubmail.cc.columbia.edu> <87my0b88ob.fsf@59A2.org> <20100118153321.2hdea0mkcg040ck0@cubmail.cc.columbia.edu> <87ljfv87zk.fsf@59A2.org> <20100118163222.s8gzpi76v4ss0so8@cubmail.cc.columbia.edu> Message-ID: When using valgrind it is best to use the additional PETSc option - malloc off this turns off PETSc's malloc tracking and usually makes the problem code clearer. I am updating the troubleshooting.html guide to indicate this. Barry On Jan 18, 2010, at 3:32 PM, (Rebecca) Xuefei YUAN wrote: > Dear Jed, > > I got some origins, but I still do not understand what is going on > here? > > Here is a piece of uninitialised value from > > > ==9725== Conditional jump or move depends on uninitialised value(s) > ==9725== at 0x86EFB20: dnrm2_ (dnrm2.f:46) > ==9725== by 0x857ECB1: VecNorm_Seq (bvec2.c:60) > ==9725== by 0x8556C5B: VecNormBegin (comb.c:507) > ==9725== by 0x80BCE2D: SNESSolve_LS (ls.c:164) > ==9725== by 0x809E6E3: SNESSolve (snes.c:2221) > ==9725== by 0x80B71E4: DMMGSolveSNES (damgsnes.c:510) > ==9725== by 0x80B0A23: DMMGSolve (damg.c:372) > ==9725== by 0x8050E41: Solve (twgcqt2unffnidgp.c:580) > ==9725== by 0x804DBE0: main (twgcqt2unffnidgp.c:288) > ==9725== Uninitialised value was created by a heap allocation > ==9725== at 0x4022BF3: malloc (vg_replace_malloc.c:195) > ==9725== by 0x8628840: PetscMallocAlign (mal.c:40) > ==9725== by 0x862992F: PetscTrMallocDefault (mtr.c:194) > ==9725== by 0x8583503: VecCreate_Seq (bvec2.c:809) > ==9725== by 0x8569F43: VecSetType (vecreg.c:54) > ==9725== by 0x858E01B: VecCreateSeq (vseqcr.c:40) > ==9725== by 0x821F345: DACreateLocalVector (dalocal.c:82) > ==9725== by 0x825E26C: DMCreateLocalVector (dm.c:115) > ==9725== by 0x821F7FF: DMGetLocalVector (dalocal.c:139) > ==9725== by 0x8055EBD: FormFunction (twgcqt2unffnidgp.c:1033) > ==9725== by 0x80953E8: SNESComputeFunction (snes.c:1016) > ==9725== by 0x80BCD06: SNESSolve_LS (ls.c:159) > > What would it mean by "Uninitialised value was created by a heap > allocation"? > > Thanks a lot! > > Rebecca > > > > > > > > > > > > > Quoting Jed Brown : > >> On Mon, 18 Jan 2010 15:33:21 -0500, "(Rebecca) Xuefei YUAN" > > wrote: >>> Dear Jed, >>> >>> Thanks for the reply. >>> >>> However, there was an error message for this option: >> >> It was introduced in valgrind-3.4.0 (and it's worth upgrading just >> for >> this feature). >> >> Jed >> >> > > > > -- > (Rebecca) Xuefei YUAN > Department of Applied Physics and Applied Mathematics > Columbia University > Tel:917-399-8032 > www.columbia.edu/~xy2102 > From xy2102 at columbia.edu Mon Jan 18 20:24:36 2010 From: xy2102 at columbia.edu ((Rebecca) Xuefei YUAN) Date: Mon, 18 Jan 2010 21:24:36 -0500 Subject: [petsc-users] valgrind errors. In-Reply-To: References: <20100118151218.isbvg6ssgw8cs0kw@cubmail.cc.columbia.edu> <87my0b88ob.fsf@59A2.org> <20100118153321.2hdea0mkcg040ck0@cubmail.cc.columbia.edu> <87ljfv87zk.fsf@59A2.org> <20100118163222.s8gzpi76v4ss0so8@cubmail.cc.columbia.edu> Message-ID: <20100118212436.6gbe2jx80s4k8wgs@cubmail.cc.columbia.edu> Dear all, Thanks very much for kind replies. I get a better idea where to look into when debugging. Cheers, Rebecca Quoting Barry Smith : > > When using valgrind it is best to use the additional PETSc option > -malloc off this turns off PETSc's malloc tracking and usually makes > the problem code clearer. > > I am updating the troubleshooting.html guide to indicate this. > > Barry > > On Jan 18, 2010, at 3:32 PM, (Rebecca) Xuefei YUAN wrote: > >> Dear Jed, >> >> I got some origins, but I still do not understand what is going on here? >> >> Here is a piece of uninitialised value from >> >> >> ==9725== Conditional jump or move depends on uninitialised value(s) >> ==9725== at 0x86EFB20: dnrm2_ (dnrm2.f:46) >> ==9725== by 0x857ECB1: VecNorm_Seq (bvec2.c:60) >> ==9725== by 0x8556C5B: VecNormBegin (comb.c:507) >> ==9725== by 0x80BCE2D: SNESSolve_LS (ls.c:164) >> ==9725== by 0x809E6E3: SNESSolve (snes.c:2221) >> ==9725== by 0x80B71E4: DMMGSolveSNES (damgsnes.c:510) >> ==9725== by 0x80B0A23: DMMGSolve (damg.c:372) >> ==9725== by 0x8050E41: Solve (twgcqt2unffnidgp.c:580) >> ==9725== by 0x804DBE0: main (twgcqt2unffnidgp.c:288) >> ==9725== Uninitialised value was created by a heap allocation >> ==9725== at 0x4022BF3: malloc (vg_replace_malloc.c:195) >> ==9725== by 0x8628840: PetscMallocAlign (mal.c:40) >> ==9725== by 0x862992F: PetscTrMallocDefault (mtr.c:194) >> ==9725== by 0x8583503: VecCreate_Seq (bvec2.c:809) >> ==9725== by 0x8569F43: VecSetType (vecreg.c:54) >> ==9725== by 0x858E01B: VecCreateSeq (vseqcr.c:40) >> ==9725== by 0x821F345: DACreateLocalVector (dalocal.c:82) >> ==9725== by 0x825E26C: DMCreateLocalVector (dm.c:115) >> ==9725== by 0x821F7FF: DMGetLocalVector (dalocal.c:139) >> ==9725== by 0x8055EBD: FormFunction (twgcqt2unffnidgp.c:1033) >> ==9725== by 0x80953E8: SNESComputeFunction (snes.c:1016) >> ==9725== by 0x80BCD06: SNESSolve_LS (ls.c:159) >> >> What would it mean by "Uninitialised value was created by a heap >> allocation"? >> >> Thanks a lot! >> >> Rebecca >> >> >> >> >> >> >> >> >> >> >> >> >> Quoting Jed Brown : >> >>> On Mon, 18 Jan 2010 15:33:21 -0500, "(Rebecca) Xuefei YUAN" >>> wrote: >>>> Dear Jed, >>>> >>>> Thanks for the reply. >>>> >>>> However, there was an error message for this option: >>> >>> It was introduced in valgrind-3.4.0 (and it's worth upgrading just for >>> this feature). >>> >>> Jed >>> >>> >> >> >> >> -- >> (Rebecca) Xuefei YUAN >> Department of Applied Physics and Applied Mathematics >> Columbia University >> Tel:917-399-8032 >> www.columbia.edu/~xy2102 >> -- (Rebecca) Xuefei YUAN Department of Applied Physics and Applied Mathematics Columbia University Tel:917-399-8032 www.columbia.edu/~xy2102 k From u.tabak at tudelft.nl Tue Jan 19 09:38:46 2010 From: u.tabak at tudelft.nl (Umut Tabak) Date: Tue, 19 Jan 2010 16:38:46 +0100 Subject: [petsc-users] a difference between user-manual and online documentation Message-ID: <4B55D206.3030004@tudelft.nl> Dear developers, I guess there is a small difference between the documentation and pdf user-manual. On page 174 of the manual, for MatLUFactor function, the last argument is printed as a pointer to MatLUInfo, MatLUInfo*, however on line documentation documents that function as(small typo I guess) PetscErrorCode MatLUFactor(Mat mat,IS row,IS col,const MatFactorInfo *info) Best regards, Umut From jed at 59A2.org Tue Jan 19 09:49:28 2010 From: jed at 59A2.org (Jed Brown) Date: Tue, 19 Jan 2010 16:49:28 +0100 Subject: [petsc-users] a difference between user-manual and online documentation In-Reply-To: <4B55D206.3030004@tudelft.nl> References: <4B55D206.3030004@tudelft.nl> Message-ID: <87vdeyxfg7.fsf@59A2.org> On Tue, 19 Jan 2010 16:38:46 +0100, Umut Tabak wrote: > Dear developers, > > I guess there is a small difference between the documentation and pdf > user-manual. On page 174 of the manual, for MatLUFactor function, the > last argument is printed as a pointer to MatLUInfo, MatLUInfo*, however > on line documentation documents that function as(small typo I guess) > > PetscErrorCode MatLUFactor(Mat mat,IS row,IS col,const MatFactorInfo *info) The online documentation is correct (it is automatically generated from the source). The manual was not updated when the change was made. Jed From xy2102 at columbia.edu Tue Jan 19 11:24:07 2010 From: xy2102 at columbia.edu ((Rebecca) Xuefei YUAN) Date: Tue, 19 Jan 2010 12:24:07 -0500 Subject: [petsc-users] valgrind errors. In-Reply-To: <87k4vf8413.fsf@59A2.org> References: <20100118151218.isbvg6ssgw8cs0kw@cubmail.cc.columbia.edu> <87my0b88ob.fsf@59A2.org> <20100118153321.2hdea0mkcg040ck0@cubmail.cc.columbia.edu> <87ljfv87zk.fsf@59A2.org> <20100118163222.s8gzpi76v4ss0so8@cubmail.cc.columbia.edu> <87k4vf8413.fsf@59A2.org> Message-ID: <20100119122407.i30ocbavc40ccc4c@cubmail.cc.columbia.edu> Dear Jed, I found the source of these uninitialised values. In my FormFunction(), there is a user-defined structure called FieldOther and I will use this as a temp values to build up my residual function. The main loop is like: typedef struct { PetscReal x1,x2,x3,x4; } FieldOther; #undef __FUNCT__ #define __FUNCT__ "FormFunction" PetscErrorCode FormFunction(SNES snes,Vec X,Vec F,void*dummg) { // ... FieldOther *fieldother; // ... PetscFunctionBegin; // ... ierr = PetscMalloc(sizeof(PetscReal)*(info.mx)*(info.my)*sizeof(FieldOther), &fieldother);CHKERRQ(ierr); // ... for (j = jFirst; j <= jLast; j++){ for (i = iFirst; i <= iLast; i++){ fieldother[(i)+(j)*(info.mx)].x1 = 0; fieldother[(i)+(j)*(info.mx)].x2 = 0; fieldother[(i)+(j)*(info.mx)].x3 = 0; fieldother[(i)+(j)*(info.mx)].x4 = 0; } } // ... for (j = jFirst; j <= jLast; j++){ for (i = iFirst; i <= iLast; i++){ // errors coming out from valgrind if fieldother[] is used in residual function evaluation. // ffield[j][i].x1 = fieldother[(i)+(j)*(info.mx)].x1; // everything looks fine from valgrind if fieldother[] is not used in residual function evaluation. ffield[j][i].x1 = field[j][i].x1; } } // ... ierr = PetscFree(fieldother);CHKERRQ(ierr); // ... PetscFunctionReturn(0); } If I will use those fieldother[] in my residual evaluation functions, errors from valgrind come out like: ==24486== ==24486== Conditional jump or move depends on uninitialised value(s) ==24486== at 0x4024D9B: memcpy (mc_replace_strmem.c:482) ==24486== by 0x41F6387: __printf_fp (in /lib/tls/i686/cmov/libc-2.7.so) ==24486== by 0x41F1563: vfprintf (in /lib/tls/i686/cmov/libc-2.7.so) ==24486== by 0x41F5581: ??? (in /lib/tls/i686/cmov/libc-2.7.so) ==24486== by 0x41F1229: vfprintf (in /lib/tls/i686/cmov/libc-2.7.so) ==24486== by 0x8643996: PetscVFPrintfDefault (mprint.c:193) ==24486== by 0x85F3F00: PetscViewerASCIIMonitorPrintf (filev.c:988) ==24486== by 0x80C814A: SNESMonitorDefault (snesut.c:150) ==24486== by 0x80E0450: SNESSolve_LS (ls.c:173) ==24486== by 0x80C199F: SNESSolve (snes.c:2221) ==24486== by 0x80DA4A0: DMMGSolveSNES (damgsnes.c:510) ==24486== by 0x80D3CDF: DMMGSolve (damg.c:372) ==24486== Uninitialised value was created by a heap allocation ==24486== at 0x4022BF3: malloc (vg_replace_malloc.c:195) ==24486== by 0x864BAFC: PetscMallocAlign (mal.c:40) ==24486== by 0x864CBEB: PetscTrMallocDefault (mtr.c:194) ==24486== by 0x8055FB8: FormFunction (twgcqt2unff.c:941) ==24486== by 0x80B86A4: SNESComputeFunction (snes.c:1016) ==24486== by 0x80DFFC2: SNESSolve_LS (ls.c:159) ==24486== by 0x80C199F: SNESSolve (snes.c:2221) ==24486== by 0x80DA4A0: DMMGSolveSNES (damgsnes.c:510) ==24486== by 0x80D3CDF: DMMGSolve (damg.c:372) ==24486== by 0x804F539: Solve (twgcqt2unff.c:437) ==24486== by 0x804DC92: main (twgcqt2unff.c:283) FormFunction (twgcqt2unff.c:941) is located at the line of ierr = PetscMalloc(sizeof(PetscReal)*(info.mx)*(info.my)*sizeof(FieldOther), &fieldother);CHKERRQ(ierr); I think I did give values of those fieldother and free them after usage, but why this uninitialised message comes out all the time? Thanks very much! Rebecca Quoting Jed Brown : > On Mon, 18 Jan 2010 16:32:22 -0500, "(Rebecca) Xuefei YUAN" > wrote: >> Dear Jed, >> >> I got some origins, but I still do not understand what is going on here? >> >> Here is a piece of uninitialised value from > > I think you have an indexing error while reading from this local vector > in your residual evaluation. The contents of the array are explicitly > initialized to zero in VecCreate_Seq, but when memalign is not available > (your trace indicates this is so) PetscMallocAlign allocate a little > extra space and aligns the result, leaving a few bytes uninitialized > (memory further away may also be uninitialized, but this error isn't > saying complaining about those places). > > Jed > > -- (Rebecca) Xuefei YUAN Department of Applied Physics and Applied Mathematics Columbia University Tel:917-399-8032 www.columbia.edu/~xy2102 From bsmith at mcs.anl.gov Tue Jan 19 11:37:54 2010 From: bsmith at mcs.anl.gov (Barry Smith) Date: Tue, 19 Jan 2010 11:37:54 -0600 Subject: [petsc-users] valgrind errors. In-Reply-To: <20100119122407.i30ocbavc40ccc4c@cubmail.cc.columbia.edu> References: <20100118151218.isbvg6ssgw8cs0kw@cubmail.cc.columbia.edu> <87my0b88ob.fsf@59A2.org> <20100118153321.2hdea0mkcg040ck0@cubmail.cc.columbia.edu> <87ljfv87zk.fsf@59A2.org> <20100118163222.s8gzpi76v4ss0so8@cubmail.cc.columbia.edu> <87k4vf8413.fsf@59A2.org> <20100119122407.i30ocbavc40ccc4c@cubmail.cc.columbia.edu> Message-ID: > err = > PetscMalloc > (sizeof(PetscReal)*(info.mx)*(info.my)*sizeof(FieldOther), > &fieldother);CHKERRQ(ierr); This line is wrong, but is not the cause of the problem. sizeof(FieldOther) takes into account the size of the real values inside so you do not need the sizeof(PetscReal)* What are iFirst and iLast? If they are not 0 then you indexing into fieldother[] wrong since that array starts at 0. Barry On Jan 19, 2010, at 11:24 AM, (Rebecca) Xuefei YUAN wrote: > Dear Jed, > > I found the source of these uninitialised values. > > In my FormFunction(), there is a user-defined structure called > FieldOther and I will use this as a temp values to build up my > residual function. The main loop is like: > > > typedef struct { > PetscReal x1,x2,x3,x4; > } FieldOther; > > #undef __FUNCT__ > #define __FUNCT__ "FormFunction" > PetscErrorCode FormFunction(SNES snes,Vec X,Vec F,void*dummg) > { > // ... > FieldOther *fieldother; > // ... > PetscFunctionBegin; > // ... > ierr = > PetscMalloc > (sizeof(PetscReal)*(info.mx)*(info.my)*sizeof(FieldOther), > &fieldother);CHKERRQ(ierr); > // ... > for (j = jFirst; j <= jLast; j++){ > for (i = iFirst; i <= iLast; i++){ > fieldother[(i)+(j)*(info.mx)].x1 = 0; > fieldother[(i)+(j)*(info.mx)].x2 = 0; > fieldother[(i)+(j)*(info.mx)].x3 = 0; > fieldother[(i)+(j)*(info.mx)].x4 = 0; > } > } > // ... > for (j = jFirst; j <= jLast; j++){ > for (i = iFirst; i <= iLast; i++){ > // errors coming out from valgrind if fieldother[] is used in > residual function evaluation. > // ffield[j][i].x1 = fieldother[(i)+(j)*(info.mx)].x1; > // everything looks fine from valgrind if fieldother[] is not used > in residual function evaluation. > ffield[j][i].x1 = field[j][i].x1; > } > } > // ... > ierr = PetscFree(fieldother);CHKERRQ(ierr); > // ... > PetscFunctionReturn(0); > > } > > If I will use those fieldother[] in my residual evaluation > functions, errors from valgrind come out like: > > ==24486== > ==24486== Conditional jump or move depends on uninitialised value(s) > ==24486== at 0x4024D9B: memcpy (mc_replace_strmem.c:482) > ==24486== by 0x41F6387: __printf_fp (in /lib/tls/i686/cmov/ > libc-2.7.so) > ==24486== by 0x41F1563: vfprintf (in /lib/tls/i686/cmov/ > libc-2.7.so) > ==24486== by 0x41F5581: ??? (in /lib/tls/i686/cmov/libc-2.7.so) > ==24486== by 0x41F1229: vfprintf (in /lib/tls/i686/cmov/ > libc-2.7.so) > ==24486== by 0x8643996: PetscVFPrintfDefault (mprint.c:193) > ==24486== by 0x85F3F00: PetscViewerASCIIMonitorPrintf (filev.c:988) > ==24486== by 0x80C814A: SNESMonitorDefault (snesut.c:150) > ==24486== by 0x80E0450: SNESSolve_LS (ls.c:173) > ==24486== by 0x80C199F: SNESSolve (snes.c:2221) > ==24486== by 0x80DA4A0: DMMGSolveSNES (damgsnes.c:510) > ==24486== by 0x80D3CDF: DMMGSolve (damg.c:372) > ==24486== Uninitialised value was created by a heap allocation > ==24486== at 0x4022BF3: malloc (vg_replace_malloc.c:195) > ==24486== by 0x864BAFC: PetscMallocAlign (mal.c:40) > ==24486== by 0x864CBEB: PetscTrMallocDefault (mtr.c:194) > ==24486== by 0x8055FB8: FormFunction (twgcqt2unff.c:941) > ==24486== by 0x80B86A4: SNESComputeFunction (snes.c:1016) > ==24486== by 0x80DFFC2: SNESSolve_LS (ls.c:159) > ==24486== by 0x80C199F: SNESSolve (snes.c:2221) > ==24486== by 0x80DA4A0: DMMGSolveSNES (damgsnes.c:510) > ==24486== by 0x80D3CDF: DMMGSolve (damg.c:372) > ==24486== by 0x804F539: Solve (twgcqt2unff.c:437) > ==24486== by 0x804DC92: main (twgcqt2unff.c:283) > > FormFunction (twgcqt2unff.c:941) is located at the line of > > ierr = > PetscMalloc > (sizeof(PetscReal)*(info.mx)*(info.my)*sizeof(FieldOther), > &fieldother);CHKERRQ(ierr); > > I think I did give values of those fieldother and free them after > usage, but why this uninitialised message comes out all the time? > > Thanks very much! > > Rebecca > > > Quoting Jed Brown : > >> On Mon, 18 Jan 2010 16:32:22 -0500, "(Rebecca) Xuefei YUAN" > > wrote: >>> Dear Jed, >>> >>> I got some origins, but I still do not understand what is going on >>> here? >>> >>> Here is a piece of uninitialised value from >> >> I think you have an indexing error while reading from this local >> vector >> in your residual evaluation. The contents of the array are >> explicitly >> initialized to zero in VecCreate_Seq, but when memalign is not >> available >> (your trace indicates this is so) PetscMallocAlign allocate a little >> extra space and aligns the result, leaving a few bytes uninitialized >> (memory further away may also be uninitialized, but this error isn't >> saying complaining about those places). >> >> Jed >> >> > > > > -- > (Rebecca) Xuefei YUAN > Department of Applied Physics and Applied Mathematics > Columbia University > Tel:917-399-8032 > www.columbia.edu/~xy2102 > From jed at 59A2.org Tue Jan 19 11:44:38 2010 From: jed at 59A2.org (Jed Brown) Date: Tue, 19 Jan 2010 18:44:38 +0100 Subject: [petsc-users] valgrind errors. In-Reply-To: <20100119122407.i30ocbavc40ccc4c@cubmail.cc.columbia.edu> References: <20100118151218.isbvg6ssgw8cs0kw@cubmail.cc.columbia.edu> <87my0b88ob.fsf@59A2.org> <20100118153321.2hdea0mkcg040ck0@cubmail.cc.columbia.edu> <87ljfv87zk.fsf@59A2.org> <20100118163222.s8gzpi76v4ss0so8@cubmail.cc.columbia.edu> <87k4vf8413.fsf@59A2.org> <20100119122407.i30ocbavc40ccc4c@cubmail.cc.columbia.edu> Message-ID: <87ska2xa49.fsf@59A2.org> On Tue, 19 Jan 2010 12:24:07 -0500, "(Rebecca) Xuefei YUAN" wrote: > Dear Jed, > > I found the source of these uninitialised values. > > In my FormFunction(), there is a user-defined structure called > FieldOther and I will use this as a temp values to build up my > residual function. The main loop is like: > > > typedef struct { > PetscReal x1,x2,x3,x4; > } FieldOther; > > #undef __FUNCT__ > #define __FUNCT__ "FormFunction" > PetscErrorCode FormFunction(SNES snes,Vec X,Vec F,void*dummg) > { > // ... > FieldOther *fieldother; > // ... > PetscFunctionBegin; > // ... > ierr = > PetscMalloc(sizeof(PetscReal)*(info.mx)*(info.my)*sizeof(FieldOther), > &fieldother);CHKERRQ(ierr); This allocates more memory than you need (probably 8 times). PetscMalloc(info.mx*info.my*sizeof(PetscReal),&fieldother); would give you what you want. Note that you can also zero it with PetscMemzero(fieldother,info.mx*info.my*sizeof(PetscReal)); instead of the explicit loop. You are now allowed to index fieldother[0+0*info.mx] up to fieldother[info.mx*info.my-1]. What values are jFirst,jLast,iFirst, and iLast? In particular, they will not start at 0 in parallel, and perhaps they include ghost values?. The usual procedure when you need such temporary arrays is to clone a DA of whatever size you need, but with different values of dof (and maybe overlap), then use DAGetArray() or DAGetLocalVector()+DAVecGetArray() so that you can index it as fieldother[j][i].x2 (even in parallel). Jed From jed at 59A2.org Tue Jan 19 11:46:07 2010 From: jed at 59A2.org (Jed Brown) Date: Tue, 19 Jan 2010 18:46:07 +0100 Subject: [petsc-users] valgrind errors. In-Reply-To: <87ska2xa49.fsf@59A2.org> References: <20100118151218.isbvg6ssgw8cs0kw@cubmail.cc.columbia.edu> <87my0b88ob.fsf@59A2.org> <20100118153321.2hdea0mkcg040ck0@cubmail.cc.columbia.edu> <87ljfv87zk.fsf@59A2.org> <20100118163222.s8gzpi76v4ss0so8@cubmail.cc.columbia.edu> <87k4vf8413.fsf@59A2.org> <20100119122407.i30ocbavc40ccc4c@cubmail.cc.columbia.edu> <87ska2xa49.fsf@59A2.org> Message-ID: <87r5pmxa1s.fsf@59A2.org> On Tue, 19 Jan 2010 18:44:38 +0100, Jed Brown wrote: > PetscMalloc(info.mx*info.my*sizeof(PetscReal),&fieldother); Oops, I meant PetscMalloc(info.mx*info.my*sizeof(FieldOther),&fieldother); Jed From xy2102 at columbia.edu Tue Jan 19 11:52:21 2010 From: xy2102 at columbia.edu ((Rebecca) Xuefei YUAN) Date: Tue, 19 Jan 2010 12:52:21 -0500 Subject: [petsc-users] valgrind errors. In-Reply-To: <87r5pmxa1s.fsf@59A2.org> References: <20100118151218.isbvg6ssgw8cs0kw@cubmail.cc.columbia.edu> <87my0b88ob.fsf@59A2.org> <20100118153321.2hdea0mkcg040ck0@cubmail.cc.columbia.edu> <87ljfv87zk.fsf@59A2.org> <20100118163222.s8gzpi76v4ss0so8@cubmail.cc.columbia.edu> <87k4vf8413.fsf@59A2.org> <20100119122407.i30ocbavc40ccc4c@cubmail.cc.columbia.edu> <87ska2xa49.fsf@59A2.org> <87r5pmxa1s.fsf@59A2.org> Message-ID: <20100119125221.lz7sfe8y8cooc48s@cubmail.cc.columbia.edu> Dear Jed, This makes a lot of sense! I made the following change: ierr = PetscMalloc((info.mx)*(info.my)*sizeof(FieldOther), &fieldother);CHKERRQ(ierr); and the values of iFirst, iLast, jFirst, jLast in one single processor is from iFirst = info.xs; iLast = info.xs + info.xm - 1; jFirst = info.ys; jLast = info.ys + info.ym - 1; where ierr = DAGetLocalInfo((DA)dmmg->dm,&info);CHKERRQ(ierr); . However, I still have the same errors as before in valgrind. I think it is better for me to do the second way of cloning DA to set up these temp arrays. More later. Thanks so much! Rebecca Quoting Jed Brown : > On Tue, 19 Jan 2010 18:44:38 +0100, Jed Brown wrote: >> PetscMalloc(info.mx*info.my*sizeof(PetscReal),&fieldother); > > Oops, I meant > > PetscMalloc(info.mx*info.my*sizeof(FieldOther),&fieldother); > > Jed > > -- (Rebecca) Xuefei YUAN Department of Applied Physics and Applied Mathematics Columbia University Tel:917-399-8032 www.columbia.edu/~xy2102 From xy2102 at columbia.edu Tue Jan 19 11:56:30 2010 From: xy2102 at columbia.edu ((Rebecca) Xuefei YUAN) Date: Tue, 19 Jan 2010 12:56:30 -0500 Subject: [petsc-users] valgrind errors. In-Reply-To: <87ska2xa49.fsf@59A2.org> References: <20100118151218.isbvg6ssgw8cs0kw@cubmail.cc.columbia.edu> <87my0b88ob.fsf@59A2.org> <20100118153321.2hdea0mkcg040ck0@cubmail.cc.columbia.edu> <87ljfv87zk.fsf@59A2.org> <20100118163222.s8gzpi76v4ss0so8@cubmail.cc.columbia.edu> <87k4vf8413.fsf@59A2.org> <20100119122407.i30ocbavc40ccc4c@cubmail.cc.columbia.edu> <87ska2xa49.fsf@59A2.org> Message-ID: <20100119125630.yvn09ibg9c8c844g@cubmail.cc.columbia.edu> Dear Jed, I have one more question, what is the difference or how could I know whether I need DAGetArray() or DAGetLocalVector()+DAVecGetArray() to get my array? Thanks a ton! Rebecca Quoting Jed Brown : > On Tue, 19 Jan 2010 12:24:07 -0500, "(Rebecca) Xuefei YUAN" > wrote: >> Dear Jed, >> >> I found the source of these uninitialised values. >> >> In my FormFunction(), there is a user-defined structure called >> FieldOther and I will use this as a temp values to build up my >> residual function. The main loop is like: >> >> >> typedef struct { >> PetscReal x1,x2,x3,x4; >> } FieldOther; >> >> #undef __FUNCT__ >> #define __FUNCT__ "FormFunction" >> PetscErrorCode FormFunction(SNES snes,Vec X,Vec F,void*dummg) >> { >> // ... >> FieldOther *fieldother; >> // ... >> PetscFunctionBegin; >> // ... >> ierr = >> PetscMalloc(sizeof(PetscReal)*(info.mx)*(info.my)*sizeof(FieldOther), >> &fieldother);CHKERRQ(ierr); > > This allocates more memory than you need (probably 8 times). > > PetscMalloc(info.mx*info.my*sizeof(PetscReal),&fieldother); > > would give you what you want. Note that you can also zero it with > > PetscMemzero(fieldother,info.mx*info.my*sizeof(PetscReal)); > > instead of the explicit loop. > > > You are now allowed to index fieldother[0+0*info.mx] up to > fieldother[info.mx*info.my-1]. What values are jFirst,jLast,iFirst, and > iLast? In particular, they will not start at 0 in parallel, and perhaps > they include ghost values?. > > The usual procedure when you need such temporary arrays is to clone a DA > of whatever size you need, but with different values of dof (and maybe > overlap), then use DAGetArray() or DAGetLocalVector()+DAVecGetArray() so > that you can index it as fieldother[j][i].x2 (even in parallel). > > Jed > > -- (Rebecca) Xuefei YUAN Department of Applied Physics and Applied Mathematics Columbia University Tel:917-399-8032 www.columbia.edu/~xy2102 From jed at 59A2.org Tue Jan 19 12:05:34 2010 From: jed at 59A2.org (Jed Brown) Date: Tue, 19 Jan 2010 19:05:34 +0100 Subject: [petsc-users] valgrind errors. In-Reply-To: <20100119125630.yvn09ibg9c8c844g@cubmail.cc.columbia.edu> References: <20100118151218.isbvg6ssgw8cs0kw@cubmail.cc.columbia.edu> <87my0b88ob.fsf@59A2.org> <20100118153321.2hdea0mkcg040ck0@cubmail.cc.columbia.edu> <87ljfv87zk.fsf@59A2.org> <20100118163222.s8gzpi76v4ss0so8@cubmail.cc.columbia.edu> <87k4vf8413.fsf@59A2.org> <20100119122407.i30ocbavc40ccc4c@cubmail.cc.columbia.edu> <87ska2xa49.fsf@59A2.org> <20100119125630.yvn09ibg9c8c844g@cubmail.cc.columbia.edu> Message-ID: <87pr56x95d.fsf@59A2.org> On Tue, 19 Jan 2010 12:56:30 -0500, "(Rebecca) Xuefei YUAN" wrote: > Dear Jed, > > I have one more question, what is the difference or how could I know > whether I need DAGetArray() or DAGetLocalVector()+DAVecGetArray() to > get my array? You can use the former unless you need to use it when packed up in a vector. In a mulitlevel scheme, I would probably put something like this in my FormFunction: DA da = (DA)dmmg->dm,eda; PetscObjectQuery((PetscObject)da,"ExtraDA",(PetscObject*)&eda); if (!eda) { // create eda based on the layout of da PetscObjectCompose((PetscObject)da,"ExtraDA",eda); DADestroy(eda); // give away ownership } DAGetArray(eda,PETSC_TRUE,&extrafield); // or FALSE if you don't need ghost values ... DARestoreArray(eda,PETSC_TRUE,&extrafield); This way the memory is only allocated once, it will work with multiple levels, and get cleaned up when you are done. Jed From xy2102 at columbia.edu Tue Jan 19 12:27:14 2010 From: xy2102 at columbia.edu ((Rebecca) Xuefei YUAN) Date: Tue, 19 Jan 2010 13:27:14 -0500 Subject: [petsc-users] valgrind errors. In-Reply-To: <87pr56x95d.fsf@59A2.org> References: <20100118151218.isbvg6ssgw8cs0kw@cubmail.cc.columbia.edu> <87my0b88ob.fsf@59A2.org> <20100118153321.2hdea0mkcg040ck0@cubmail.cc.columbia.edu> <87ljfv87zk.fsf@59A2.org> <20100118163222.s8gzpi76v4ss0so8@cubmail.cc.columbia.edu> <87k4vf8413.fsf@59A2.org> <20100119122407.i30ocbavc40ccc4c@cubmail.cc.columbia.edu> <87ska2xa49.fsf@59A2.org> <20100119125630.yvn09ibg9c8c844g@cubmail.cc.columbia.edu> <87pr56x95d.fsf@59A2.org> Message-ID: <20100119132714.yy40n8q2kg8g8ko4@cubmail.cc.columbia.edu> Dear Jed, I switched the code as below and it is working now without valgrind errors. I am using multilevel, will the following approach be fine? I will modify the code a little more to get sth like yours. typedef struct { PetscReal x1,x2,x3,x4; } FieldOther; #undef __FUNCT__ #define __FUNCT__ "FormFunction" PetscErrorCode FormFunction(SNES snes,Vec X,Vec F,void*dummg) { // ... DA dafieldother; FieldOther **fieldother; Vec FIELDOTHER; // ... PetscFunctionBegin; // ... ierr = DAGetLocalInfo((DA)dmmg->dm,&info);CHKERRQ(ierr); // ... ierr = DACreate2d(PETSC_COMM_WORLD,DA_NONPERIODIC,DA_STENCIL_BOX, info.mx, info.my, PETSC_DECIDE, PETSC_DECIDE, 4, 2, 0, 0, &dafieldother);CHKERRQ(ierr); ierr = DASetFieldName(dafieldother, 0, "x1");CHKERRQ(ierr); ierr = DASetFieldName(dafieldother, 1, "x2");CHKERRQ(ierr); ierr = DASetFieldName(dafieldother, 2, "x3");CHKERRQ(ierr); ierr = DASetFieldName(dafieldother, 3, "x4");CHKERRQ(ierr); ierr = DAGetLocalVector(dafieldother,&FIELDOTHER);CHKERRQ(ierr); ierr = DAVecGetArray(dafieldother,FIELDOTHER,&fieldother);CHKERRQ(ierr); for (j = jFirst; j <= jLast; j++){ for (i = iFirst; i <= iLast; i++){ fieldother[j][i].x1 = 0; fieldother[j][i].x2 = 0; fieldother[j][i].x3 = 0; fieldother[j][i].x4 = 0; } } // ... for (j = jFirst; j <= jLast; j++){ for (i = iFirst; i <= iLast; i++){ ffield[j][i].x1 = fieldother[j][i].x1; } } // ... ierr = DAVecRestoreArray(dafieldother,FIELDOTHER,&fieldother);CHKERRQ(ierr); ierr = DARestoreLocalVector(dafieldother,&FIELDOTHER);CHKERRQ(ierr); ierr = DADestroy(dafieldother);CHKERRQ(ierr); PetscFunctionReturn(0); } Thanks so much! Rebecca Quoting Jed Brown : > On Tue, 19 Jan 2010 12:56:30 -0500, "(Rebecca) Xuefei YUAN" > wrote: >> Dear Jed, >> >> I have one more question, what is the difference or how could I know >> whether I need DAGetArray() or DAGetLocalVector()+DAVecGetArray() to >> get my array? > > You can use the former unless you need to use it when packed up in a > vector. > > In a mulitlevel scheme, I would probably put something like this in my > FormFunction: > > DA da = (DA)dmmg->dm,eda; > > PetscObjectQuery((PetscObject)da,"ExtraDA",(PetscObject*)&eda); > if (!eda) { > // create eda based on the layout of da > PetscObjectCompose((PetscObject)da,"ExtraDA",eda); > DADestroy(eda); // give away ownership > } > DAGetArray(eda,PETSC_TRUE,&extrafield); // or FALSE if you don't > need ghost values > > ... > > DARestoreArray(eda,PETSC_TRUE,&extrafield); > > > This way the memory is only allocated once, it will work with multiple > levels, and get cleaned up when you are done. > > Jed > > -- (Rebecca) Xuefei YUAN Department of Applied Physics and Applied Mathematics Columbia University Tel:917-399-8032 www.columbia.edu/~xy2102 From xy2102 at columbia.edu Tue Jan 19 12:33:02 2010 From: xy2102 at columbia.edu ((Rebecca) Xuefei YUAN) Date: Tue, 19 Jan 2010 13:33:02 -0500 Subject: [petsc-users] valgrind errors. In-Reply-To: <87pr56x95d.fsf@59A2.org> References: <20100118151218.isbvg6ssgw8cs0kw@cubmail.cc.columbia.edu> <87my0b88ob.fsf@59A2.org> <20100118153321.2hdea0mkcg040ck0@cubmail.cc.columbia.edu> <87ljfv87zk.fsf@59A2.org> <20100118163222.s8gzpi76v4ss0so8@cubmail.cc.columbia.edu> <87k4vf8413.fsf@59A2.org> <20100119122407.i30ocbavc40ccc4c@cubmail.cc.columbia.edu> <87ska2xa49.fsf@59A2.org> <20100119125630.yvn09ibg9c8c844g@cubmail.cc.columbia.edu> <87pr56x95d.fsf@59A2.org> Message-ID: <20100119133302.bot470zpmos48o48@cubmail.cc.columbia.edu> What if the dof of da is 4 and the dof of eda is 6? Quoting Jed Brown : > On Tue, 19 Jan 2010 12:56:30 -0500, "(Rebecca) Xuefei YUAN" > wrote: >> Dear Jed, >> >> I have one more question, what is the difference or how could I know >> whether I need DAGetArray() or DAGetLocalVector()+DAVecGetArray() to >> get my array? > > You can use the former unless you need to use it when packed up in a > vector. > > In a mulitlevel scheme, I would probably put something like this in my > FormFunction: > > DA da = (DA)dmmg->dm,eda; > > PetscObjectQuery((PetscObject)da,"ExtraDA",(PetscObject*)&eda); > if (!eda) { > // create eda based on the layout of da > PetscObjectCompose((PetscObject)da,"ExtraDA",eda); > DADestroy(eda); // give away ownership > } > DAGetArray(eda,PETSC_TRUE,&extrafield); // or FALSE if you don't > need ghost values > > ... > > DARestoreArray(eda,PETSC_TRUE,&extrafield); > > > This way the memory is only allocated once, it will work with multiple > levels, and get cleaned up when you are done. > > Jed > > -- (Rebecca) Xuefei YUAN Department of Applied Physics and Applied Mathematics Columbia University Tel:917-399-8032 www.columbia.edu/~xy2102 From jed at 59A2.org Tue Jan 19 12:44:46 2010 From: jed at 59A2.org (Jed Brown) Date: Tue, 19 Jan 2010 19:44:46 +0100 Subject: [petsc-users] valgrind errors. In-Reply-To: <20100119132714.yy40n8q2kg8g8ko4@cubmail.cc.columbia.edu> References: <20100118151218.isbvg6ssgw8cs0kw@cubmail.cc.columbia.edu> <87my0b88ob.fsf@59A2.org> <20100118153321.2hdea0mkcg040ck0@cubmail.cc.columbia.edu> <87ljfv87zk.fsf@59A2.org> <20100118163222.s8gzpi76v4ss0so8@cubmail.cc.columbia.edu> <87k4vf8413.fsf@59A2.org> <20100119122407.i30ocbavc40ccc4c@cubmail.cc.columbia.edu> <87ska2xa49.fsf@59A2.org> <20100119125630.yvn09ibg9c8c844g@cubmail.cc.columbia.edu> <87pr56x95d.fsf@59A2.org> <20100119132714.yy40n8q2kg8g8ko4@cubmail.cc.columbia.edu> Message-ID: <87ockqx7c1.fsf@59A2.org> On Tue, 19 Jan 2010 13:27:14 -0500, "(Rebecca) Xuefei YUAN" wrote: > Dear Jed, > > I switched the code as below and it is working now without valgrind errors. Great! > I am using multilevel, will the following approach be fine? I will > modify the code a little more to get sth like yours. It looks fine, but it does a bit of allocation/setup on each function evaluation. The snippet I gave would only do the setup once and do no allocations afterwards. Also note that you can VecZeroEntries() instead of your explicit zeroing loop. > What if the dof of da is 4 and the dof of eda is 6? No problem, just change 4 to 6 in: > ierr = DACreate2d(PETSC_COMM_WORLD,DA_NONPERIODIC,DA_STENCIL_BOX, > info.mx, info.my, PETSC_DECIDE, PETSC_DECIDE, 4, 2, 0, 0, > &dafieldother);CHKERRQ(ierr); I would use dof = sizeof(FieldOther)/sizeof(PetscScalar); so that it's always the correct size if I go change FieldOther. (And the entries in FieldOther should by PetscScalar if you want them to work correctly on a PETSc build with complex support.) Jed From xy2102 at columbia.edu Tue Jan 19 12:49:36 2010 From: xy2102 at columbia.edu ((Rebecca) Xuefei YUAN) Date: Tue, 19 Jan 2010 13:49:36 -0500 Subject: [petsc-users] valgrind errors. Message-ID: <20100119134936.t9d993iqisoo84ok@cubmail.cc.columbia.edu> Dear Barry, I switched from using temp arrays via PetscMalloc() and PetscFree() to creating a DA and using DAGetLocalVector()+DAVecGetArray() to get fieldother as a temp array. However, I do not think DAGetLocalVector()+DAVecGetArray() is a must in my case, I tried DAGetArray() to take over of the above two. However, I have the error message saying twgcqt2unff.c:2150: warning: passing argument 3 of ?DAGetArray? from incompatible pointer type twgcqt2unff.c:3213: warning: passing argument 3 of ?DARestoreArray? from incompatible pointer type . The code looks like: typedef struct { PetscReal x1,x2,x3,x4; } FieldOther; #undef __FUNCT__ #define __FUNCT__ "FormFunction" PetscErrorCode FormFunction(SNES snes,Vec X,Vec F,void*dummg) { // ... DA dafieldother; FieldOther **fieldother; Vec FIELDOTHER; // ... PetscFunctionBegin; // ... ierr = DAGetLocalInfo((DA)dmmg->dm,&info);CHKERRQ(ierr); // ... ierr = DACreate2d(PETSC_COMM_WORLD,DA_NONPERIODIC,DA_STENCIL_BOX, info.mx, info.my, PETSC_DECIDE, PETSC_DECIDE, 4, 2, 0, 0, &dafieldother);CHKERRQ(ierr); ierr = DASetFieldName(dafieldother, 0, "x1");CHKERRQ(ierr); ierr = DASetFieldName(dafieldother, 1, "x2");CHKERRQ(ierr); ierr = DASetFieldName(dafieldother, 2, "x3");CHKERRQ(ierr); ierr = DASetFieldName(dafieldother, 3, "x4");CHKERRQ(ierr); // ierr = DAGetLocalVector(dafieldother,&FIELDOTHER);CHKERRQ(ierr); // ierr = DAVecGetArray(dafieldother,FIELDOTHER,&fieldother);CHKERRQ(ierr); ierr = DAGetArray(dafieldother,PETSC_TRUTH,&fieldother);CHKERRQ(ierr); for (j = jFirst; j <= jLast; j++){ for (i = iFirst; i <= iLast; i++){ fieldother[j][i].x1 = 0; fieldother[j][i].x2 = 0; fieldother[j][i].x3 = 0; fieldother[j][i].x4 = 0; } } // ... for (j = jFirst; j <= jLast; j++){ for (i = iFirst; i <= iLast; i++){ ffield[j][i].x1 = fieldother[j][i].x1; } } // ... // ierr = DAVecRestoreArray(dafieldother,FIELDOTHER,&fieldother);CHKERRQ(ierr); // ierr = DARestoreLocalVector(dafieldother,&FIELDOTHER);CHKERRQ(ierr); ierr = DARestoreArray(dafieldother,PETSC_TRUTH,&fieldother);CHKERRQ(ierr); ierr = DADestroy(dafieldother);CHKERRQ(ierr); PetscFunctionReturn(0); } What could be wrong with it? Thanks very much! Rebecca Quoting Barry Smith : > >> err = >> PetscMalloc(sizeof(PetscReal)*(info.mx)*(info.my)*sizeof(FieldOther), >> &fieldother);CHKERRQ(ierr); > > This line is wrong, but is not the cause of the problem. > sizeof(FieldOther) takes into account the size of the real values > inside so you do not need the sizeof(PetscReal)* > > What are iFirst and iLast? If they are not 0 then you indexing into > fieldother[] wrong since that array starts at 0. > > Barry > > On Jan 19, 2010, at 11:24 AM, (Rebecca) Xuefei YUAN wrote: > >> Dear Jed, >> >> I found the source of these uninitialised values. >> >> In my FormFunction(), there is a user-defined structure called >> FieldOther and I will use this as a temp values to build up my >> residual function. The main loop is like: >> >> >> typedef struct { >> PetscReal x1,x2,x3,x4; >> } FieldOther; >> >> #undef __FUNCT__ >> #define __FUNCT__ "FormFunction" >> PetscErrorCode FormFunction(SNES snes,Vec X,Vec F,void*dummg) >> { >> // ... >> FieldOther *fieldother; >> // ... >> PetscFunctionBegin; >> // ... >> ierr = >> PetscMalloc(sizeof(PetscReal)*(info.mx)*(info.my)*sizeof(FieldOther), >> &fieldother);CHKERRQ(ierr); >> // ... >> for (j = jFirst; j <= jLast; j++){ >> for (i = iFirst; i <= iLast; i++){ >> fieldother[(i)+(j)*(info.mx)].x1 = 0; >> fieldother[(i)+(j)*(info.mx)].x2 = 0; >> fieldother[(i)+(j)*(info.mx)].x3 = 0; >> fieldother[(i)+(j)*(info.mx)].x4 = 0; >> } >> } >> // ... >> for (j = jFirst; j <= jLast; j++){ >> for (i = iFirst; i <= iLast; i++){ >> // errors coming out from valgrind if fieldother[] is used in >> residual function evaluation. >> // ffield[j][i].x1 = fieldother[(i)+(j)*(info.mx)].x1; >> // everything looks fine from valgrind if fieldother[] is not used >> in residual function evaluation. >> ffield[j][i].x1 = field[j][i].x1; >> } >> } >> // ... >> ierr = PetscFree(fieldother);CHKERRQ(ierr); >> // ... >> PetscFunctionReturn(0); >> >> } >> >> If I will use those fieldother[] in my residual evaluation >> functions, errors from valgrind come out like: >> >> ==24486== >> ==24486== Conditional jump or move depends on uninitialised value(s) >> ==24486== at 0x4024D9B: memcpy (mc_replace_strmem.c:482) >> ==24486== by 0x41F6387: __printf_fp (in /lib/tls/i686/cmov/libc-2.7.so) >> ==24486== by 0x41F1563: vfprintf (in /lib/tls/i686/cmov/libc-2.7.so) >> ==24486== by 0x41F5581: ??? (in /lib/tls/i686/cmov/libc-2.7.so) >> ==24486== by 0x41F1229: vfprintf (in /lib/tls/i686/cmov/libc-2.7.so) >> ==24486== by 0x8643996: PetscVFPrintfDefault (mprint.c:193) >> ==24486== by 0x85F3F00: PetscViewerASCIIMonitorPrintf (filev.c:988) >> ==24486== by 0x80C814A: SNESMonitorDefault (snesut.c:150) >> ==24486== by 0x80E0450: SNESSolve_LS (ls.c:173) >> ==24486== by 0x80C199F: SNESSolve (snes.c:2221) >> ==24486== by 0x80DA4A0: DMMGSolveSNES (damgsnes.c:510) >> ==24486== by 0x80D3CDF: DMMGSolve (damg.c:372) >> ==24486== Uninitialised value was created by a heap allocation >> ==24486== at 0x4022BF3: malloc (vg_replace_malloc.c:195) >> ==24486== by 0x864BAFC: PetscMallocAlign (mal.c:40) >> ==24486== by 0x864CBEB: PetscTrMallocDefault (mtr.c:194) >> ==24486== by 0x8055FB8: FormFunction (twgcqt2unff.c:941) >> ==24486== by 0x80B86A4: SNESComputeFunction (snes.c:1016) >> ==24486== by 0x80DFFC2: SNESSolve_LS (ls.c:159) >> ==24486== by 0x80C199F: SNESSolve (snes.c:2221) >> ==24486== by 0x80DA4A0: DMMGSolveSNES (damgsnes.c:510) >> ==24486== by 0x80D3CDF: DMMGSolve (damg.c:372) >> ==24486== by 0x804F539: Solve (twgcqt2unff.c:437) >> ==24486== by 0x804DC92: main (twgcqt2unff.c:283) >> >> FormFunction (twgcqt2unff.c:941) is located at the line of >> >> ierr = >> PetscMalloc(sizeof(PetscReal)*(info.mx)*(info.my)*sizeof(FieldOther), >> &fieldother);CHKERRQ(ierr); >> >> I think I did give values of those fieldother and free them after >> usage, but why this uninitialised message comes out all the time? >> >> Thanks very much! >> >> Rebecca >> >> >> Quoting Jed Brown : >> >>> On Mon, 18 Jan 2010 16:32:22 -0500, "(Rebecca) Xuefei YUAN" >>> wrote: >>>> Dear Jed, >>>> >>>> I got some origins, but I still do not understand what is going on here? >>>> >>>> Here is a piece of uninitialised value from >>> >>> I think you have an indexing error while reading from this local vector >>> in your residual evaluation. The contents of the array are explicitly >>> initialized to zero in VecCreate_Seq, but when memalign is not available >>> (your trace indicates this is so) PetscMallocAlign allocate a little >>> extra space and aligns the result, leaving a few bytes uninitialized >>> (memory further away may also be uninitialized, but this error isn't >>> saying complaining about those places). >>> >>> Jed >>> >>> >> >> >> >> -- >> (Rebecca) Xuefei YUAN >> Department of Applied Physics and Applied Mathematics >> Columbia University >> Tel:917-399-8032 >> www.columbia.edu/~xy2102 >> -- (Rebecca) Xuefei YUAN Department of Applied Physics and Applied Mathematics Columbia University Tel:917-399-8032 www.columbia.edu/~xy2102 From jed at 59A2.org Tue Jan 19 12:55:07 2010 From: jed at 59A2.org (Jed Brown) Date: Tue, 19 Jan 2010 19:55:07 +0100 Subject: [petsc-users] valgrind errors. In-Reply-To: <20100119134936.t9d993iqisoo84ok@cubmail.cc.columbia.edu> References: <20100119134936.t9d993iqisoo84ok@cubmail.cc.columbia.edu> Message-ID: <87my09ylf8.fsf@59A2.org> On Tue, 19 Jan 2010 13:49:36 -0500, "(Rebecca) Xuefei YUAN" wrote: > However, I do not think DAGetLocalVector()+DAVecGetArray() is a must > in my case, I tried DAGetArray() to take over of the above two. > However, I have the error message saying These are just warnings, you can avoid them by using the cast ierr = DAGetArray(dafieldother,PETSC_TRUTH,(void**)&fieldother);CHKERRQ(ierr); But PETSc should just change the type signature so the third argument is void* so that no cast is required. Jed From xy2102 at columbia.edu Tue Jan 19 12:57:06 2010 From: xy2102 at columbia.edu ((Rebecca) Xuefei YUAN) Date: Tue, 19 Jan 2010 13:57:06 -0500 Subject: [petsc-users] valgrind errors. In-Reply-To: <87ockqx7c1.fsf@59A2.org> References: <20100118151218.isbvg6ssgw8cs0kw@cubmail.cc.columbia.edu> <87my0b88ob.fsf@59A2.org> <20100118153321.2hdea0mkcg040ck0@cubmail.cc.columbia.edu> <87ljfv87zk.fsf@59A2.org> <20100118163222.s8gzpi76v4ss0so8@cubmail.cc.columbia.edu> <87k4vf8413.fsf@59A2.org> <20100119122407.i30ocbavc40ccc4c@cubmail.cc.columbia.edu> <87ska2xa49.fsf@59A2.org> <20100119125630.yvn09ibg9c8c844g@cubmail.cc.columbia.edu> <87pr56x95d.fsf@59A2.org> <20100119132714.yy40n8q2kg8g8ko4@cubmail.cc.columbia.edu> <87ockqx7c1.fsf@59A2.org> Message-ID: <20100119135706.rjyostek0s4c8gss@cubmail.cc.columbia.edu> Quoting Jed Brown : > On Tue, 19 Jan 2010 13:27:14 -0500, "(Rebecca) Xuefei YUAN" > wrote: >> Dear Jed, >> >> I switched the code as below and it is working now without valgrind errors. > > Great! > >> I am using multilevel, will the following approach be fine? I will >> modify the code a little more to get sth like yours. > > It looks fine, but it does a bit of allocation/setup on each function > evaluation. The snippet I gave would only do the setup once and do no > allocations afterwards. > If I want to do it like your way, how could I handle the different dof in the call if (!eda) { // create eda based on the layout of da PetscObjectCompose((PetscObject)da,"ExtraDA",eda); DADestroy(eda); // give away ownership } DAGetArray(eda,PETSC_TRUE,&extrafield); // or FALSE if you don't need ghost values ... DARestoreArray(eda,PETSC_TRUE,&extrafield); Why eda is destroyed before I can use it in DAGetArray? > Also note that you can VecZeroEntries() instead of your explicit zeroing > loop. > >> What if the dof of da is 4 and the dof of eda is 6? > > No problem, just change 4 to 6 in: > >> ierr = DACreate2d(PETSC_COMM_WORLD,DA_NONPERIODIC,DA_STENCIL_BOX, >> info.mx, info.my, PETSC_DECIDE, PETSC_DECIDE, 4, 2, 0, 0, >> &dafieldother);CHKERRQ(ierr); > > I would use > > dof = sizeof(FieldOther)/sizeof(PetscScalar); > > so that it's always the correct size if I go change FieldOther. (And > the entries in FieldOther should by PetscScalar if you want them to work > correctly on a PETSc build with complex support.) Is that mean PetscReal and PetscScalar are the same if it is not complex, however, for a better compatible reason, should I change all my PetscReal to PetscScalar? > > Jed > > -- (Rebecca) Xuefei YUAN Department of Applied Physics and Applied Mathematics Columbia University Tel:917-399-8032 www.columbia.edu/~xy2102 From jed at 59A2.org Tue Jan 19 13:09:47 2010 From: jed at 59A2.org (Jed Brown) Date: Tue, 19 Jan 2010 20:09:47 +0100 Subject: [petsc-users] valgrind errors. In-Reply-To: <20100119135706.rjyostek0s4c8gss@cubmail.cc.columbia.edu> References: <20100118151218.isbvg6ssgw8cs0kw@cubmail.cc.columbia.edu> <87my0b88ob.fsf@59A2.org> <20100118153321.2hdea0mkcg040ck0@cubmail.cc.columbia.edu> <87ljfv87zk.fsf@59A2.org> <20100118163222.s8gzpi76v4ss0so8@cubmail.cc.columbia.edu> <87k4vf8413.fsf@59A2.org> <20100119122407.i30ocbavc40ccc4c@cubmail.cc.columbia.edu> <87ska2xa49.fsf@59A2.org> <20100119125630.yvn09ibg9c8c844g@cubmail.cc.columbia.edu> <87pr56x95d.fsf@59A2.org> <20100119132714.yy40n8q2kg8g8ko4@cubmail.cc.columbia.edu> <87ockqx7c1.fsf@59A2.org> <20100119135706.rjyostek0s4c8gss@cubmail.cc.columbia.edu> Message-ID: <87k4vdykqs.fsf@59A2.org> On Tue, 19 Jan 2010 13:57:06 -0500, "(Rebecca) Xuefei YUAN" wrote: > Quoting Jed Brown : > If I want to do it like your way, how could I handle the different dof > in the call > if (!eda) { > // create eda based on the layout of da ^^^^ This is where your DACreate2D(...,sizeof(FieldOther)/sizeof(PetscScalar),...,&eda) goes. > PetscObjectCompose((PetscObject)da,"ExtraDA",eda); > DADestroy(eda); // give away ownership > Why eda is destroyed before I can use it in DAGetArray? DADestroy(eda) could also be written PetscObjectDereference((PetscObject)eda). When you compose eda with dmmg->dm, it increases the reference count. The DADestroy(eda) just decreases the reference count so that it will properly be destroyed inside DMMGDestroy() at end of your program. So it's just giving away ownership. > Is that mean PetscReal and PetscScalar are the same if it is not > complex, however, for a better compatible reason, should I change all > my PetscReal to PetscScalar? Yes, PetscScalar is for all "solution-like" variables, PetscReal is mostly used for parameters (length scales, Reynold's number, material constants) and norms. Jed From jack.lee at sjc.ox.ac.uk Wed Jan 20 16:50:47 2010 From: jack.lee at sjc.ox.ac.uk (Jack Lee) Date: Wed, 20 Jan 2010 22:50:47 +0000 Subject: [petsc-users] using external direct linear solvers Message-ID: <4B5788C7.20902@sjc.ox.ac.uk> Hi there, Apologies if this is an obvious question, but is there an example showing how to call an external solver? I am trying to get the snes solver to use MUMPS. As the calls are made from a fortran application, I cannot use the commandline options. Jack From knepley at gmail.com Wed Jan 20 17:14:16 2010 From: knepley at gmail.com (Matthew Knepley) Date: Wed, 20 Jan 2010 17:14:16 -0600 Subject: [petsc-users] using external direct linear solvers In-Reply-To: <4B5788C7.20902@sjc.ox.ac.uk> References: <4B5788C7.20902@sjc.ox.ac.uk> Message-ID: On Wed, Jan 20, 2010 at 4:50 PM, Jack Lee wrote: > Hi there, > > Apologies if this is an obvious question, but is there an example showing > how to call an external solver? I am trying to get the snes solver to use > MUMPS. As the calls are made from a fortran application, I cannot use the > commandline options. > 1) MUMPS is a linear solver 2) http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/PC/PCFactorSetMatSolverPackage.html 3) There is a manual section on this 4) You can use command line options in Fortran Matt Jack > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From Z.Sheng at tudelft.nl Thu Jan 21 10:41:31 2010 From: Z.Sheng at tudelft.nl (Zhifeng Sheng - EWI) Date: Thu, 21 Jan 2010 17:41:31 +0100 Subject: [petsc-users] using PETSC with visual studio 2005? References: <4B5788C7.20902@sjc.ox.ac.uk> Message-ID: <947C04CD618D16429440FED56EAE47BA06B2FF@SRV564.tudelft.net> Dear all Does anyone use Petsc in visual studio 2005? I have used petsc under linux before, but now I need to switch to windows visual studio 2005. And I am wondering how I can link to the petsc library. If you know how to do it, please let me know. Thanks and best regards Zhifeng -------------- next part -------------- An HTML attachment was scrubbed... URL: From balay at mcs.anl.gov Thu Jan 21 11:03:18 2010 From: balay at mcs.anl.gov (Satish Balay) Date: Thu, 21 Jan 2010 11:03:18 -0600 (CST) Subject: [petsc-users] using PETSC with visual studio 2005? In-Reply-To: <947C04CD618D16429440FED56EAE47BA06B2FF@SRV564.tudelft.net> References: <4B5788C7.20902@sjc.ox.ac.uk> <947C04CD618D16429440FED56EAE47BA06B2FF@SRV564.tudelft.net> Message-ID: Check the installation instructions - specifically - the section: 'Using Visual Studio 2005' http://www.mcs.anl.gov/petsc/petsc-as/documentation/installation.html Satish On Thu, 21 Jan 2010, Zhifeng Sheng - EWI wrote: > Dear all > > Does anyone use Petsc in visual studio 2005? I have used petsc under linux before, but now I need to switch to windows visual studio 2005. And I am wondering how I can link to the petsc library. If you know how to do it, please let me know. > > Thanks and best regards > > Zhifeng > > From jack.lee at sjc.ox.ac.uk Thu Jan 21 14:07:31 2010 From: jack.lee at sjc.ox.ac.uk (Jack Lee) Date: Thu, 21 Jan 2010 20:07:31 +0000 Subject: [petsc-users] petsc-users Digest, Vol 13, Issue 28 In-Reply-To: References: Message-ID: <4B58B403.3090007@sjc.ox.ac.uk> Thanks for the reply. Perhaps I should have explained a little further: > 1) MUMPS is a linear solver I am aware of that, what I meant was that instead of the default iterative solver which snes employs, I'd like the option of being able to use an external solver for each linear solution of the non-linear loop. > 2) > http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/PC/PCFactorSetMatSolverPackage.html Currently I am applying these steps: 1. set ksp context extracted from snes to pre only 2. set pc type to lu 3. set pc factor solver package to MUMPS, as you've explained above. However, having no examples to compare to, I'm not 100% sure if this is the way it's supposed to be done. > 3) There is a manual section on this if you're talking about page 83, it doesn't spell out the case which I'm trying to apply to, so thought I'd better double-check. > 4) You can use command line options in Fortran Not in the custom fortran application we're building, which is compiled as a library. We're linking in petsc to exploit its fantastic solver capabilities.... Cheers, Jack petsc-users-request at mcs.anl.gov wrote: > Send petsc-users mailing list submissions to > petsc-users at mcs.anl.gov > > To subscribe or unsubscribe via the World Wide Web, visit > https://lists.mcs.anl.gov/mailman/listinfo/petsc-users > or, via email, send a message with subject or body 'help' to > petsc-users-request at mcs.anl.gov > > You can reach the person managing the list at > petsc-users-owner at mcs.anl.gov > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of petsc-users digest..." > > > Today's Topics: > > 1. using external direct linear solvers (Jack Lee) > 2. Re: using external direct linear solvers (Matthew Knepley) > 3. using PETSC with visual studio 2005? (Zhifeng Sheng - EWI) > 4. Re: using PETSC with visual studio 2005? (Satish Balay) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Wed, 20 Jan 2010 22:50:47 +0000 > From: Jack Lee > Subject: [petsc-users] using external direct linear solvers > To: petsc-users at mcs.anl.gov > Message-ID: <4B5788C7.20902 at sjc.ox.ac.uk> > Content-Type: text/plain; charset=ISO-8859-1; format=flowed > > Hi there, > > Apologies if this is an obvious question, but is there an example > showing how to call an external solver? I am trying to get the snes > solver to use MUMPS. As the calls are made from a fortran application, I > cannot use the commandline options. > > Jack > > > ------------------------------ > > Message: 2 > Date: Wed, 20 Jan 2010 17:14:16 -0600 > From: Matthew Knepley > Subject: Re: [petsc-users] using external direct linear solvers > To: PETSc users list > Message-ID: > > Content-Type: text/plain; charset="iso-8859-1" > > On Wed, Jan 20, 2010 at 4:50 PM, Jack Lee wrote: > >> Hi there, >> >> Apologies if this is an obvious question, but is there an example showing >> how to call an external solver? I am trying to get the snes solver to use >> MUMPS. As the calls are made from a fortran application, I cannot use the >> commandline options. >> > > 1) MUMPS is a linear solver > > 2) > http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/PC/PCFactorSetMatSolverPackage.html > > 3) There is a manual section on this > > 4) You can use command line options in Fortran > > Matt > > Jack From knepley at gmail.com Thu Jan 21 14:11:16 2010 From: knepley at gmail.com (Matthew Knepley) Date: Thu, 21 Jan 2010 14:11:16 -0600 Subject: [petsc-users] petsc-users Digest, Vol 13, Issue 28 In-Reply-To: <4B58B403.3090007@sjc.ox.ac.uk> References: <4B58B403.3090007@sjc.ox.ac.uk> Message-ID: On Thu, Jan 21, 2010 at 2:07 PM, Jack Lee wrote: > Thanks for the reply. Perhaps I should have explained a little further: > > > 1) MUMPS is a linear solver > I am aware of that, what I meant was that instead of the default iterative > solver which snes employs, I'd like the option of being able to use an > external solver for each linear solution of the non-linear loop. > Okay > > 2) > > > http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/PC/PCFactorSetMatSolverPackage.html > Currently I am applying these steps: > 1. set ksp context extracted from snes to pre only > 2. set pc type to lu > 3. set pc factor solver package to MUMPS, as you've explained above. > However, having no examples to compare to, I'm not 100% sure if this is the > way it's supposed to be done. > Yep, that is it > > 3) There is a manual section on this > if you're talking about page 83, it doesn't spell out the case which I'm > trying to apply to, so thought I'd better double-check. > > > 4) You can use command line options in Fortran > Not in the custom fortran application we're building, which is compiled as > a library. We're linking in petsc to exploit its fantastic solver > capabilities.... > Well, when you call PetscInitialize(), in Fortran, it is reading the command line arguments. You, of course, do not have to use them, but they are there. Matt > Cheers, > Jack > > > > petsc-users-request at mcs.anl.gov wrote: > >> Send petsc-users mailing list submissions to >> petsc-users at mcs.anl.gov >> >> To subscribe or unsubscribe via the World Wide Web, visit >> https://lists.mcs.anl.gov/mailman/listinfo/petsc-users >> or, via email, send a message with subject or body 'help' to >> petsc-users-request at mcs.anl.gov >> >> You can reach the person managing the list at >> petsc-users-owner at mcs.anl.gov >> >> When replying, please edit your Subject line so it is more specific >> than "Re: Contents of petsc-users digest..." >> >> >> Today's Topics: >> >> 1. using external direct linear solvers (Jack Lee) >> 2. Re: using external direct linear solvers (Matthew Knepley) >> 3. using PETSC with visual studio 2005? (Zhifeng Sheng - EWI) >> 4. Re: using PETSC with visual studio 2005? (Satish Balay) >> >> >> ---------------------------------------------------------------------- >> >> Message: 1 >> Date: Wed, 20 Jan 2010 22:50:47 +0000 >> From: Jack Lee >> Subject: [petsc-users] using external direct linear solvers >> To: petsc-users at mcs.anl.gov >> Message-ID: <4B5788C7.20902 at sjc.ox.ac.uk> >> Content-Type: text/plain; charset=ISO-8859-1; format=flowed >> >> Hi there, >> >> Apologies if this is an obvious question, but is there an example showing >> how to call an external solver? I am trying to get the snes solver to use >> MUMPS. As the calls are made from a fortran application, I cannot use the >> commandline options. >> >> Jack >> >> >> ------------------------------ >> >> Message: 2 >> Date: Wed, 20 Jan 2010 17:14:16 -0600 >> From: Matthew Knepley >> Subject: Re: [petsc-users] using external direct linear solvers >> To: PETSc users list >> Message-ID: >> >> Content-Type: text/plain; charset="iso-8859-1" >> >> On Wed, Jan 20, 2010 at 4:50 PM, Jack Lee wrote: >> >> Hi there, >>> >>> Apologies if this is an obvious question, but is there an example showing >>> how to call an external solver? I am trying to get the snes solver to use >>> MUMPS. As the calls are made from a fortran application, I cannot use the >>> commandline options. >>> >>> >> 1) MUMPS is a linear solver >> >> 2) >> >> http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/PC/PCFactorSetMatSolverPackage.html >> >> 3) There is a manual section on this >> >> 4) You can use command line options in Fortran >> >> Matt >> >> Jack >> > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Thu Jan 21 14:22:56 2010 From: bsmith at mcs.anl.gov (Barry Smith) Date: Thu, 21 Jan 2010 14:22:56 -0600 Subject: [petsc-users] petsc-users Digest, Vol 13, Issue 28 In-Reply-To: <4B58B403.3090007@sjc.ox.ac.uk> References: <4B58B403.3090007@sjc.ox.ac.uk> Message-ID: <6AAC458F-63C8-42AE-A44E-46146728FCE0@mcs.anl.gov> On Jan 21, 2010, at 2:07 PM, Jack Lee wrote: > Thanks for the reply. Perhaps I should have explained a little > further: > > > 1) MUMPS is a linear solver > I am aware of that, what I meant was that instead of the default > iterative solver which snes employs, I'd like the option of being > able to use an external solver for each linear solution of the non- > linear loop. > > > 2) > > http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/PC/PCFactorSetMatSolverPackage.html > Currently I am applying these steps: > 1. set ksp context extracted from snes to pre only > 2. set pc type to lu > 3. set pc factor solver package to MUMPS, as you've explained above. > However, having no examples to compare to, I'm not 100% sure if this > is the way it's supposed to be done. > > > 3) There is a manual section on this > if you're talking about page 83, it doesn't spell out the case which > I'm trying to apply to, so thought I'd better double-check. > > > 4) You can use command line options in Fortran > Not in the custom fortran application we're building, which is > compiled as a library. We're linking in petsc to exploit its > fantastic solver capabilities.... You can also put the options into a file called .petscrc, petscrc or ~/.petscrc or a file of your own choosing and pass the filename into PetscInitialize() or the environmental variable PETSC_OPTIONS The ability to change options without recompiling/relinking is what keeps me sane :-) Barry > > Cheers, > Jack > > > > petsc-users-request at mcs.anl.gov wrote: >> Send petsc-users mailing list submissions to >> petsc-users at mcs.anl.gov >> To subscribe or unsubscribe via the World Wide Web, visit >> https://lists.mcs.anl.gov/mailman/listinfo/petsc-users >> or, via email, send a message with subject or body 'help' to >> petsc-users-request at mcs.anl.gov >> You can reach the person managing the list at >> petsc-users-owner at mcs.anl.gov >> When replying, please edit your Subject line so it is more specific >> than "Re: Contents of petsc-users digest..." >> Today's Topics: >> 1. using external direct linear solvers (Jack Lee) >> 2. Re: using external direct linear solvers (Matthew Knepley) >> 3. using PETSC with visual studio 2005? (Zhifeng Sheng - EWI) >> 4. Re: using PETSC with visual studio 2005? (Satish Balay) >> ---------------------------------------------------------------------- >> Message: 1 >> Date: Wed, 20 Jan 2010 22:50:47 +0000 >> From: Jack Lee >> Subject: [petsc-users] using external direct linear solvers >> To: petsc-users at mcs.anl.gov >> Message-ID: <4B5788C7.20902 at sjc.ox.ac.uk> >> Content-Type: text/plain; charset=ISO-8859-1; format=flowed >> Hi there, >> Apologies if this is an obvious question, but is there an example >> showing how to call an external solver? I am trying to get the snes >> solver to use MUMPS. As the calls are made from a fortran >> application, I cannot use the commandline options. >> Jack >> ------------------------------ >> Message: 2 >> Date: Wed, 20 Jan 2010 17:14:16 -0600 >> From: Matthew Knepley >> Subject: Re: [petsc-users] using external direct linear solvers >> To: PETSc users list >> Message-ID: >> >> Content-Type: text/plain; charset="iso-8859-1" >> On Wed, Jan 20, 2010 at 4:50 PM, Jack Lee >> wrote: >>> Hi there, >>> >>> Apologies if this is an obvious question, but is there an example >>> showing >>> how to call an external solver? I am trying to get the snes solver >>> to use >>> MUMPS. As the calls are made from a fortran application, I cannot >>> use the >>> commandline options. >>> >> 1) MUMPS is a linear solver >> 2) >> http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/PC/PCFactorSetMatSolverPackage.html >> 3) There is a manual section on this >> 4) You can use command line options in Fortran >> Matt >> Jack From jed at 59A2.org Thu Jan 21 14:24:45 2010 From: jed at 59A2.org (Jed Brown) Date: Thu, 21 Jan 2010 21:24:45 +0100 Subject: [petsc-users] petsc-users Digest, Vol 13, Issue 28 In-Reply-To: References: <4B58B403.3090007@sjc.ox.ac.uk> Message-ID: <87aaw7fboy.fsf@59A2.org> On Thu, 21 Jan 2010 14:11:16 -0600, Matthew Knepley wrote: > Well, when you call PetscInitialize(), in Fortran, it is reading the > command line arguments. You, of course, do not have to use them, but > they are there. Note that you can also specify options in ~/.petscrc, $PWD/petscrc, and in a user-defined file. I highly recommend providing a way for your users to get runtime options to PETSc, it's never beneficial to hard-code the whole solution process. Jed From hzhang at mcs.anl.gov Thu Jan 21 17:28:29 2010 From: hzhang at mcs.anl.gov (Hong Zhang) Date: Thu, 21 Jan 2010 17:28:29 -0600 Subject: [petsc-users] petsc-users Digest, Vol 13, Issue 28 In-Reply-To: <4B58B403.3090007@sjc.ox.ac.uk> References: <4B58B403.3090007@sjc.ox.ac.uk> Message-ID: <417B126C-12B6-4FA5-B024-7E225A4CC67C@mcs.anl.gov> Jack, > > > 2) > > http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/PC/PCFactorSetMatSolverPackage.html > Currently I am applying these steps: > 1. set ksp context extracted from snes to pre only > 2. set pc type to lu > 3. set pc factor solver package to MUMPS, as you've explained above. > However, having no examples to compare to, I'm not 100% sure if this > is the way it's supposed to be done. I just wrote a Fortran example petsc-dev/src/snes/examples/tests/ex14f.F few days ago which uses MUMPS as a linear solver within snes non-linear loop. The example is attached, or can be obtained through petsc-dev. petsc-dev/src/snes/examples/tests/ex12f.F is a simpler example which you can run it with options ./ex12f -pc_type lu -pc_factor_mat_solver_package mumps to invoke mumps' LU. Use '-snes_view' to check which solvers are used and '-ksp_monitor -snes_monitor' for monitoring iterations. Hong > > > 3) There is a manual section on this > if you're talking about page 83, it doesn't spell out the case which > I'm trying to apply to, so thought I'd better double-check. > > > 4) You can use command line options in Fortran > Not in the custom fortran application we're building, which is > compiled as a library. We're linking in petsc to exploit its > fantastic solver capabilities.... > > Cheers, > Jack > > > > petsc-users-request at mcs.anl.gov wrote: >> Send petsc-users mailing list submissions to >> petsc-users at mcs.anl.gov >> To subscribe or unsubscribe via the World Wide Web, visit >> https://lists.mcs.anl.gov/mailman/listinfo/petsc-users >> or, via email, send a message with subject or body 'help' to >> petsc-users-request at mcs.anl.gov >> You can reach the person managing the list at >> petsc-users-owner at mcs.anl.gov >> When replying, please edit your Subject line so it is more specific >> than "Re: Contents of petsc-users digest..." >> Today's Topics: >> 1. using external direct linear solvers (Jack Lee) >> 2. Re: using external direct linear solvers (Matthew Knepley) >> 3. using PETSC with visual studio 2005? (Zhifeng Sheng - EWI) >> 4. Re: using PETSC with visual studio 2005? (Satish Balay) >> ---------------------------------------------------------------------- >> Message: 1 >> Date: Wed, 20 Jan 2010 22:50:47 +0000 >> From: Jack Lee >> Subject: [petsc-users] using external direct linear solvers >> To: petsc-users at mcs.anl.gov >> Message-ID: <4B5788C7.20902 at sjc.ox.ac.uk> >> Content-Type: text/plain; charset=ISO-8859-1; format=flowed >> Hi there, >> Apologies if this is an obvious question, but is there an example >> showing how to call an external solver? I am trying to get the snes >> solver to use MUMPS. As the calls are made from a fortran >> application, I cannot use the commandline options. >> Jack >> ------------------------------ >> Message: 2 >> Date: Wed, 20 Jan 2010 17:14:16 -0600 >> From: Matthew Knepley >> Subject: Re: [petsc-users] using external direct linear solvers >> To: PETSc users list >> Message-ID: >> >> Content-Type: text/plain; charset="iso-8859-1" >> On Wed, Jan 20, 2010 at 4:50 PM, Jack Lee >> wrote: >>> Hi there, >>> >>> Apologies if this is an obvious question, but is there an example >>> showing >>> how to call an external solver? I am trying to get the snes solver >>> to use >>> MUMPS. As the calls are made from a fortran application, I cannot >>> use the >>> commandline options. >>> >> 1) MUMPS is a linear solver >> 2) >> http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/PC/PCFactorSetMatSolverPackage.html >> 3) There is a manual section on this >> 4) You can use command line options in Fortran >> Matt >> Jack -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ex14f.F Type: application/octet-stream Size: 12434 bytes Desc: not available URL: -------------- next part -------------- An HTML attachment was scrubbed... URL: From recrusader at gmail.com Fri Jan 22 11:04:22 2010 From: recrusader at gmail.com (Yujie) Date: Fri, 22 Jan 2010 11:04:22 -0600 Subject: [petsc-users] matrix with complex number Message-ID: <7ff0ee011001220904h6b3af7f8k57f22044ab41d66d@mail.gmail.com> Dear PETSc Developers, Recently, I am trying to find some complex number-based solvers and preconditioners. However, it is difficult to find a general framework to include some solvers and preconditioners. Trilinos is developing a package, komplex, to use the real-number-based solver to solve complex number -based problem. I don't know whether PETSc wants to develop such the function for complex number-based problem. I think it will significantly increase the application range of PETSc. After all, in PETSc, lots of solvers and preconditioners have been developed. Thanks a lot. Regards, Yujie -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Fri Jan 22 14:35:18 2010 From: bsmith at mcs.anl.gov (Barry Smith) Date: Fri, 22 Jan 2010 14:35:18 -0600 Subject: [petsc-users] matrix with complex number In-Reply-To: <7ff0ee011001220904h6b3af7f8k57f22044ab41d66d@mail.gmail.com> References: <7ff0ee011001220904h6b3af7f8k57f22044ab41d66d@mail.gmail.com> Message-ID: The way we handle complex numbered linear systems in PETSc is to compile all of PETSc with complex numbers and then just use the solvers on those complex numbers. The current drawback to this is that PETSc can only be built with support for complex numbers or for real numbers. We cannot build a PETSc where some matrices are complex and some are real. We don't have any interest in solving complex systems as larger real systems. Barry On Jan 22, 2010, at 11:04 AM, Yujie wrote: > Dear PETSc Developers, > > Recently, I am trying to find some complex number-based solvers and > preconditioners. However, it is difficult to find a general > framework to include some solvers and preconditioners. Trilinos is > developing a package, komplex, to use the real-number-based solver > to solve complex number -based problem. I don't know whether PETSc > wants to develop such the function for complex number-based problem. > I think it will significantly increase the application range of > PETSc. After all, in PETSc, lots of solvers and preconditioners have > been developed. Thanks a lot. > > Regards, > Yujie From recrusader at gmail.com Fri Jan 22 14:43:00 2010 From: recrusader at gmail.com (Yujie) Date: Fri, 22 Jan 2010 14:43:00 -0600 Subject: [petsc-users] matrix with complex number In-Reply-To: References: <7ff0ee011001220904h6b3af7f8k57f22044ab41d66d@mail.gmail.com> Message-ID: <7ff0ee011001221243q495dc036wd85fd6b2ba3b86ef@mail.gmail.com> Dear Barry, What are the difficult things if PETSc is revised with matrices of complex and real numbers? It should be more flexible for a general scientific toolbox. I am curious almost all the packages don't support both simultaneously. Thanks a lot. Regards, Yujie On Fri, Jan 22, 2010 at 2:35 PM, Barry Smith wrote: > > The way we handle complex numbered linear systems in PETSc is to compile > all of PETSc with complex numbers and then just use the solvers on those > complex numbers. The current drawback to this is that PETSc can only be > built with support for complex numbers or for real numbers. We cannot build > a PETSc where some matrices are complex and some are real. > We don't have any interest in solving complex systems as larger real > systems. > > Barry > > > On Jan 22, 2010, at 11:04 AM, Yujie wrote: > > Dear PETSc Developers, >> >> Recently, I am trying to find some complex number-based solvers and >> preconditioners. However, it is difficult to find a general framework to >> include some solvers and preconditioners. Trilinos is developing a package, >> komplex, to use the real-number-based solver to solve complex number -based >> problem. I don't know whether PETSc wants to develop such the function for >> complex number-based problem. I think it will significantly increase the >> application range of PETSc. After all, in PETSc, lots of solvers and >> preconditioners have been developed. Thanks a lot. >> >> Regards, >> Yujie >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Fri Jan 22 14:49:04 2010 From: bsmith at mcs.anl.gov (Barry Smith) Date: Fri, 22 Jan 2010 14:49:04 -0600 Subject: [petsc-users] matrix with complex number In-Reply-To: <7ff0ee011001221243q495dc036wd85fd6b2ba3b86ef@mail.gmail.com> References: <7ff0ee011001220904h6b3af7f8k57f22044ab41d66d@mail.gmail.com> <7ff0ee011001221243q495dc036wd85fd6b2ba3b86ef@mail.gmail.com> Message-ID: The issue is that PETSc is written in C and there are not C++ templates in C. We do not want to write TWO copies of each file, one with dcomplex everywhere and another with double everywhere and give different names to the functions in each language. It would be a pain to write and maintain and a pain to use. Who wants to write MatSolve() to solve with real numbers and MatSolveComplex() to solve with complex? C++ templates can be used to make all this possible but they are not perfect to the task. Basically we need a better language to allow easy mixing of dcomplex and double. Barry On Jan 22, 2010, at 2:43 PM, Yujie wrote: > Dear Barry, > > What are the difficult things if PETSc is revised with matrices of > complex and real numbers? It should be more flexible for a general > scientific toolbox. I am curious almost all the packages don't > support both simultaneously. Thanks a lot. > > Regards, > Yujie > > On Fri, Jan 22, 2010 at 2:35 PM, Barry Smith > wrote: > > The way we handle complex numbered linear systems in PETSc is to > compile all of PETSc with complex numbers and then just use the > solvers on those complex numbers. The current drawback to this is > that PETSc can only be built with support for complex numbers or for > real numbers. We cannot build a PETSc where some matrices are > complex and some are real. > We don't have any interest in solving complex systems as larger real > systems. > > Barry > > > On Jan 22, 2010, at 11:04 AM, Yujie wrote: > > Dear PETSc Developers, > > Recently, I am trying to find some complex number-based solvers and > preconditioners. However, it is difficult to find a general > framework to include some solvers and preconditioners. Trilinos is > developing a package, komplex, to use the real-number-based solver > to solve complex number -based problem. I don't know whether PETSc > wants to develop such the function for complex number-based problem. > I think it will significantly increase the application range of > PETSc. After all, in PETSc, lots of solvers and preconditioners have > been developed. Thanks a lot. > > Regards, > Yujie > > From recrusader at gmail.com Fri Jan 22 14:57:49 2010 From: recrusader at gmail.com (Yujie) Date: Fri, 22 Jan 2010 14:57:49 -0600 Subject: [petsc-users] matrix with complex number In-Reply-To: References: <7ff0ee011001220904h6b3af7f8k57f22044ab41d66d@mail.gmail.com> <7ff0ee011001221243q495dc036wd85fd6b2ba3b86ef@mail.gmail.com> Message-ID: <7ff0ee011001221257m7e24d918gf85f59f79ec12a8e@mail.gmail.com> In order to using PETSc with complex and real numbers, a simple method I am considering is to build PETSc with complex numbers. I can control the matrices with complex or real numbers by myself. However, I can't built PETSc with some external packages and some packages developed based on PETSc. If I want to do it successfully, do you have any advice for me? Thanks a lot. Regards, Yujie On Fri, Jan 22, 2010 at 2:49 PM, Barry Smith wrote: > > The issue is that PETSc is written in C and there are not C++ templates > in C. We do not want to write TWO copies of each file, one with dcomplex > everywhere and another with double everywhere and give different names to > the functions in each language. It would be a pain to write and maintain and > a pain to use. Who wants to write MatSolve() to solve with real numbers and > MatSolveComplex() to solve with complex? > > C++ templates can be used to make all this possible but they are not > perfect to the task. Basically we need a better language to allow easy > mixing of dcomplex and double. > > Barry > > > > > On Jan 22, 2010, at 2:43 PM, Yujie wrote: > > Dear Barry, >> >> What are the difficult things if PETSc is revised with matrices of complex >> and real numbers? It should be more flexible for a general scientific >> toolbox. I am curious almost all the packages don't support both >> simultaneously. Thanks a lot. >> >> Regards, >> Yujie >> >> On Fri, Jan 22, 2010 at 2:35 PM, Barry Smith wrote: >> >> The way we handle complex numbered linear systems in PETSc is to compile >> all of PETSc with complex numbers and then just use the solvers on those >> complex numbers. The current drawback to this is that PETSc can only be >> built with support for complex numbers or for real numbers. We cannot build >> a PETSc where some matrices are complex and some are real. >> We don't have any interest in solving complex systems as larger real >> systems. >> >> Barry >> >> >> On Jan 22, 2010, at 11:04 AM, Yujie wrote: >> >> Dear PETSc Developers, >> >> Recently, I am trying to find some complex number-based solvers and >> preconditioners. However, it is difficult to find a general framework to >> include some solvers and preconditioners. Trilinos is developing a package, >> komplex, to use the real-number-based solver to solve complex number -based >> problem. I don't know whether PETSc wants to develop such the function for >> complex number-based problem. I think it will significantly increase the >> application range of PETSc. After all, in PETSc, lots of solvers and >> preconditioners have been developed. Thanks a lot. >> >> Regards, >> Yujie >> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Fri Jan 22 15:03:07 2010 From: bsmith at mcs.anl.gov (Barry Smith) Date: Fri, 22 Jan 2010 15:03:07 -0600 Subject: [petsc-users] matrix with complex number In-Reply-To: <7ff0ee011001221257m7e24d918gf85f59f79ec12a8e@mail.gmail.com> References: <7ff0ee011001220904h6b3af7f8k57f22044ab41d66d@mail.gmail.com> <7ff0ee011001221243q495dc036wd85fd6b2ba3b86ef@mail.gmail.com> <7ff0ee011001221257m7e24d918gf85f59f79ec12a8e@mail.gmail.com> Message-ID: On Jan 22, 2010, at 2:57 PM, Yujie wrote: > In order to using PETSc with complex and real numbers, a simple > method I am considering is to build PETSc with complex numbers. I > can control the matrices with complex or real numbers by myself. > However, I can't built PETSc with some external packages and some > packages developed based on PETSc. If I want to do it successfully, > do you have any advice for me? Thanks a lot. Some external packages simply don't support complex numbers, there is nothing we can do about this. Similarly if some writes a package with PETSc if they are not careful to use PetscScalar it won't be usable with complex, again there is nothing we can do about this. Barry > > Regards, > Yujie > > On Fri, Jan 22, 2010 at 2:49 PM, Barry Smith > wrote: > > The issue is that PETSc is written in C and there are not C++ > templates in C. We do not want to write TWO copies of each file, one > with dcomplex everywhere and another with double everywhere and give > different names to the functions in each language. It would be a > pain to write and maintain and a pain to use. Who wants to write > MatSolve() to solve with real numbers and MatSolveComplex() to solve > with complex? > > C++ templates can be used to make all this possible but they are > not perfect to the task. Basically we need a better language to > allow easy mixing of dcomplex and double. > > Barry > > > > > On Jan 22, 2010, at 2:43 PM, Yujie wrote: > > Dear Barry, > > What are the difficult things if PETSc is revised with matrices of > complex and real numbers? It should be more flexible for a general > scientific toolbox. I am curious almost all the packages don't > support both simultaneously. Thanks a lot. > > Regards, > Yujie > > On Fri, Jan 22, 2010 at 2:35 PM, Barry Smith > wrote: > > The way we handle complex numbered linear systems in PETSc is to > compile all of PETSc with complex numbers and then just use the > solvers on those complex numbers. The current drawback to this is > that PETSc can only be built with support for complex numbers or for > real numbers. We cannot build a PETSc where some matrices are > complex and some are real. > We don't have any interest in solving complex systems as larger real > systems. > > Barry > > > On Jan 22, 2010, at 11:04 AM, Yujie wrote: > > Dear PETSc Developers, > > Recently, I am trying to find some complex number-based solvers and > preconditioners. However, it is difficult to find a general > framework to include some solvers and preconditioners. Trilinos is > developing a package, komplex, to use the real-number-based solver > to solve complex number -based problem. I don't know whether PETSc > wants to develop such the function for complex number-based problem. > I think it will significantly increase the application range of > PETSc. After all, in PETSc, lots of solvers and preconditioners have > been developed. Thanks a lot. > > Regards, > Yujie > > > > From recrusader at gmail.com Fri Jan 22 15:14:05 2010 From: recrusader at gmail.com (Yujie) Date: Fri, 22 Jan 2010 15:14:05 -0600 Subject: [petsc-users] matrix with complex number In-Reply-To: References: <7ff0ee011001220904h6b3af7f8k57f22044ab41d66d@mail.gmail.com> <7ff0ee011001221243q495dc036wd85fd6b2ba3b86ef@mail.gmail.com> <7ff0ee011001221257m7e24d918gf85f59f79ec12a8e@mail.gmail.com> Message-ID: <7ff0ee011001221314y421a778bi8d94d2390f265bfd@mail.gmail.com> Dear Barry, Thank you very much for your information. I am considering to do it by myself. To the latter, if the package with PETSc can be avoided to use PetscScalar and to check this when it is built. Should it work? to my understanding, if one is using real number, PetscScalar should be "double" type generally? Thanks again. Regards, Yujie On Fri, Jan 22, 2010 at 3:03 PM, Barry Smith wrote: > > On Jan 22, 2010, at 2:57 PM, Yujie wrote: > > In order to using PETSc with complex and real numbers, a simple method I >> am considering is to build PETSc with complex numbers. I can control the >> matrices with complex or real numbers by myself. However, I can't built >> PETSc with some external packages and some packages developed based on >> PETSc. If I want to do it successfully, do you have any advice for me? >> Thanks a lot. >> > > Some external packages simply don't support complex numbers, there is > nothing we can do about this. Similarly if some writes a package with PETSc > if they are not careful to use PetscScalar it won't be usable with complex, > again there is nothing we can do about this. > > Barry > > > >> Regards, >> Yujie >> >> On Fri, Jan 22, 2010 at 2:49 PM, Barry Smith wrote: >> >> The issue is that PETSc is written in C and there are not C++ templates >> in C. We do not want to write TWO copies of each file, one with dcomplex >> everywhere and another with double everywhere and give different names to >> the functions in each language. It would be a pain to write and maintain and >> a pain to use. Who wants to write MatSolve() to solve with real numbers and >> MatSolveComplex() to solve with complex? >> >> C++ templates can be used to make all this possible but they are not >> perfect to the task. Basically we need a better language to allow easy >> mixing of dcomplex and double. >> >> Barry >> >> >> >> >> On Jan 22, 2010, at 2:43 PM, Yujie wrote: >> >> Dear Barry, >> >> What are the difficult things if PETSc is revised with matrices of complex >> and real numbers? It should be more flexible for a general scientific >> toolbox. I am curious almost all the packages don't support both >> simultaneously. Thanks a lot. >> >> Regards, >> Yujie >> >> On Fri, Jan 22, 2010 at 2:35 PM, Barry Smith wrote: >> >> The way we handle complex numbered linear systems in PETSc is to compile >> all of PETSc with complex numbers and then just use the solvers on those >> complex numbers. The current drawback to this is that PETSc can only be >> built with support for complex numbers or for real numbers. We cannot build >> a PETSc where some matrices are complex and some are real. >> We don't have any interest in solving complex systems as larger real >> systems. >> >> Barry >> >> >> On Jan 22, 2010, at 11:04 AM, Yujie wrote: >> >> Dear PETSc Developers, >> >> Recently, I am trying to find some complex number-based solvers and >> preconditioners. However, it is difficult to find a general framework to >> include some solvers and preconditioners. Trilinos is developing a package, >> komplex, to use the real-number-based solver to solve complex number -based >> problem. I don't know whether PETSc wants to develop such the function for >> complex number-based problem. I think it will significantly increase the >> application range of PETSc. After all, in PETSc, lots of solvers and >> preconditioners have been developed. Thanks a lot. >> >> Regards, >> Yujie >> >> >> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hal.finkel at yale.edu Fri Jan 22 22:23:42 2010 From: hal.finkel at yale.edu (Hal Finkel) Date: Fri, 22 Jan 2010 23:23:42 -0500 Subject: [petsc-users] matrix with complex number In-Reply-To: References: <7ff0ee011001220904h6b3af7f8k57f22044ab41d66d@mail.gmail.com> <7ff0ee011001221243q495dc036wd85fd6b2ba3b86ef@mail.gmail.com> Message-ID: <4B5A79CE.9080909@yale.edu> Barry, I've used the following process to achieve a similar effect in C projects: Modify the make files to build each source file multiple times, once for each base type (float, double, double complex, etc.). The type is selected via a preprocessor definition provided on the command line (-DUSE_FLOAT, -DUSE_DOUBLE, etc.). A prefix or suffix specific to each type is appended to the name of the resulting object file (so that they're unique). Also, the function names (and all other global symbols) in each file are wrapped with a function-like macro which appends a prefix or suffix depending on the current type selected. In this way, everything can be included in one library without conflict or unnecessary code duplication. -Hal Barry Smith wrote: > > The issue is that PETSc is written in C and there are not C++ > templates in C. We do not want to write TWO copies of each file, one > with dcomplex everywhere and another with double everywhere and give > different names to the functions in each language. It would be a pain to > write and maintain and a pain to use. Who wants to write MatSolve() to > solve with real numbers and MatSolveComplex() to solve with complex? > > C++ templates can be used to make all this possible but they are not > perfect to the task. Basically we need a better language to allow easy > mixing of dcomplex and double. > > Barry > > > > On Jan 22, 2010, at 2:43 PM, Yujie wrote: > >> Dear Barry, >> >> What are the difficult things if PETSc is revised with matrices of >> complex and real numbers? It should be more flexible for a general >> scientific toolbox. I am curious almost all the packages don't support >> both simultaneously. Thanks a lot. >> >> Regards, >> Yujie >> >> On Fri, Jan 22, 2010 at 2:35 PM, Barry Smith wrote: >> >> The way we handle complex numbered linear systems in PETSc is to >> compile all of PETSc with complex numbers and then just use the >> solvers on those complex numbers. The current drawback to this is that >> PETSc can only be built with support for complex numbers or for real >> numbers. We cannot build a PETSc where some matrices are complex and >> some are real. >> We don't have any interest in solving complex systems as larger real >> systems. >> >> Barry >> >> >> On Jan 22, 2010, at 11:04 AM, Yujie wrote: >> >> Dear PETSc Developers, >> >> Recently, I am trying to find some complex number-based solvers and >> preconditioners. However, it is difficult to find a general framework >> to include some solvers and preconditioners. Trilinos is developing a >> package, komplex, to use the real-number-based solver to solve complex >> number -based problem. I don't know whether PETSc wants to develop >> such the function for complex number-based problem. I think it will >> significantly increase the application range of PETSc. After all, in >> PETSc, lots of solvers and preconditioners have been developed. >> Thanks a lot. >> >> Regards, >> Yujie >> >> From bsmith at mcs.anl.gov Fri Jan 22 22:29:40 2010 From: bsmith at mcs.anl.gov (Barry Smith) Date: Fri, 22 Jan 2010 22:29:40 -0600 Subject: [petsc-users] matrix with complex number In-Reply-To: <4B5A79CE.9080909@yale.edu> References: <7ff0ee011001220904h6b3af7f8k57f22044ab41d66d@mail.gmail.com> <7ff0ee011001221243q495dc036wd85fd6b2ba3b86ef@mail.gmail.com> <4B5A79CE.9080909@yale.edu> Message-ID: <116D4926-2294-4A7B-8DF5-A3384ED6276B@mcs.anl.gov> Hal, Thanks for the suggestion. I've played around with this approach over the years, but given the size of PETSc I've always been too frightened by the uglyness and complexity of maintaining such code. It also doesn't cleanly handle mixing objects of different types together. For example, from a user perspective it is completely reasonable to try to multiply a complex vector by a real matrix, or conversely a real vector by a complex matrix, or to put real values into a complex matrix or vector but providing support for this requires even more "templating-like" stuff with macros. Barry On Jan 22, 2010, at 10:23 PM, Hal Finkel wrote: > Barry, > > I've used the following process to achieve a similar effect in C > projects: > > Modify the make files to build each source file multiple times, once > for each base type (float, double, double complex, etc.). The type > is selected via a preprocessor definition provided on the command > line (-DUSE_FLOAT, -DUSE_DOUBLE, etc.). A prefix or suffix specific > to each type is appended to the name of the resulting object file > (so that they're unique). Also, the function names (and all other > global symbols) in each file are wrapped with a function-like macro > which appends a prefix or suffix depending on the current type > selected. In this way, everything can be included in one library > without conflict or unnecessary code duplication. > > -Hal > > Barry Smith wrote: >> The issue is that PETSc is written in C and there are not C++ >> templates in C. We do not want to write TWO copies of each file, >> one with dcomplex everywhere and another with double everywhere and >> give different names to the functions in each language. It would be >> a pain to write and maintain and a pain to use. Who wants to write >> MatSolve() to solve with real numbers and MatSolveComplex() to >> solve with complex? >> C++ templates can be used to make all this possible but they are >> not perfect to the task. Basically we need a better language to >> allow easy mixing of dcomplex and double. >> Barry >> On Jan 22, 2010, at 2:43 PM, Yujie wrote: >>> Dear Barry, >>> >>> What are the difficult things if PETSc is revised with matrices of >>> complex and real numbers? It should be more flexible for a general >>> scientific toolbox. I am curious almost all the packages don't >>> support both simultaneously. Thanks a lot. >>> >>> Regards, >>> Yujie >>> >>> On Fri, Jan 22, 2010 at 2:35 PM, Barry Smith >>> wrote: >>> >>> The way we handle complex numbered linear systems in PETSc is to >>> compile all of PETSc with complex numbers and then just use the >>> solvers on those complex numbers. The current drawback to this is >>> that PETSc can only be built with support for complex numbers or >>> for real numbers. We cannot build a PETSc where some matrices are >>> complex and some are real. >>> We don't have any interest in solving complex systems as larger >>> real systems. >>> >>> Barry >>> >>> >>> On Jan 22, 2010, at 11:04 AM, Yujie wrote: >>> >>> Dear PETSc Developers, >>> >>> Recently, I am trying to find some complex number-based solvers >>> and preconditioners. However, it is difficult to find a general >>> framework to include some solvers and preconditioners. Trilinos is >>> developing a package, komplex, to use the real-number-based solver >>> to solve complex number -based problem. I don't know whether PETSc >>> wants to develop such the function for complex number-based >>> problem. I think it will significantly increase the application >>> range of PETSc. After all, in PETSc, lots of solvers and >>> preconditioners have been developed. Thanks a lot. >>> >>> Regards, >>> Yujie >>> >>> From hal.finkel at yale.edu Sat Jan 23 09:04:16 2010 From: hal.finkel at yale.edu (Hal Finkel) Date: Sat, 23 Jan 2010 10:04:16 -0500 Subject: [petsc-users] matrix with complex number In-Reply-To: <116D4926-2294-4A7B-8DF5-A3384ED6276B@mcs.anl.gov> References: <7ff0ee011001220904h6b3af7f8k57f22044ab41d66d@mail.gmail.com> <7ff0ee011001221243q495dc036wd85fd6b2ba3b86ef@mail.gmail.com> <4B5A79CE.9080909@yale.edu> <116D4926-2294-4A7B-8DF5-A3384ED6276B@mcs.anl.gov> Message-ID: <4B5B0FF0.3020004@yale.edu> Barry, I'd think that adding efficient cross-type operations in C also has a combinatorial explosion problem :) At the expense of a more complicated build system, the ugliness can be addressed in the following way: prior to compiling, ctags (or some sed script, etc.) is run against the source files to pull out the global symbol names. The list is then used to generate a header file which looks like: #if defined(USE_FLOAT) #define Sym1 Sym1_FLT ... #elif defined(USE_DOUBLE) #define Sym1 Sym1_DBL ... #endif I've done this kind of thing too, so I figured I'd mention it. It has the advantage that the source files are left essentially unchanged. -Hal Barry Smith wrote: > > Hal, > > Thanks for the suggestion. I've played around with this approach over > the years, but given the size of PETSc I've always been too frightened > by the uglyness and complexity of maintaining such code. It also doesn't > cleanly handle mixing objects of different types together. For example, > from a user perspective it is completely reasonable to try to multiply a > complex vector by a real matrix, or conversely a real vector by a > complex matrix, or to put real values into a complex matrix or vector > but providing support for this requires even more "templating-like" > stuff with macros. > > Barry > > On Jan 22, 2010, at 10:23 PM, Hal Finkel wrote: > >> Barry, >> >> I've used the following process to achieve a similar effect in C >> projects: >> >> Modify the make files to build each source file multiple times, once >> for each base type (float, double, double complex, etc.). The type is >> selected via a preprocessor definition provided on the command line >> (-DUSE_FLOAT, -DUSE_DOUBLE, etc.). A prefix or suffix specific to each >> type is appended to the name of the resulting object file (so that >> they're unique). Also, the function names (and all other global >> symbols) in each file are wrapped with a function-like macro which >> appends a prefix or suffix depending on the current type selected. In >> this way, everything can be included in one library without conflict >> or unnecessary code duplication. >> >> -Hal >> >> Barry Smith wrote: >>> The issue is that PETSc is written in C and there are not C++ >>> templates in C. We do not want to write TWO copies of each file, one >>> with dcomplex everywhere and another with double everywhere and give >>> different names to the functions in each language. It would be a pain >>> to write and maintain and a pain to use. Who wants to write >>> MatSolve() to solve with real numbers and MatSolveComplex() to solve >>> with complex? >>> C++ templates can be used to make all this possible but they are not >>> perfect to the task. Basically we need a better language to allow >>> easy mixing of dcomplex and double. >>> Barry >>> On Jan 22, 2010, at 2:43 PM, Yujie wrote: >>>> Dear Barry, >>>> >>>> What are the difficult things if PETSc is revised with matrices of >>>> complex and real numbers? It should be more flexible for a general >>>> scientific toolbox. I am curious almost all the packages don't >>>> support both simultaneously. Thanks a lot. >>>> >>>> Regards, >>>> Yujie >>>> >>>> On Fri, Jan 22, 2010 at 2:35 PM, Barry Smith >>>> wrote: >>>> >>>> The way we handle complex numbered linear systems in PETSc is to >>>> compile all of PETSc with complex numbers and then just use the >>>> solvers on those complex numbers. The current drawback to this is >>>> that PETSc can only be built with support for complex numbers or for >>>> real numbers. We cannot build a PETSc where some matrices are >>>> complex and some are real. >>>> We don't have any interest in solving complex systems as larger real >>>> systems. >>>> >>>> Barry >>>> >>>> >>>> On Jan 22, 2010, at 11:04 AM, Yujie wrote: >>>> >>>> Dear PETSc Developers, >>>> >>>> Recently, I am trying to find some complex number-based solvers and >>>> preconditioners. However, it is difficult to find a general >>>> framework to include some solvers and preconditioners. Trilinos is >>>> developing a package, komplex, to use the real-number-based solver >>>> to solve complex number -based problem. I don't know whether PETSc >>>> wants to develop such the function for complex number-based problem. >>>> I think it will significantly increase the application range of >>>> PETSc. After all, in PETSc, lots of solvers and preconditioners have >>>> been developed. Thanks a lot. >>>> >>>> Regards, >>>> Yujie >>>> >>>> From don at drexel.edu Mon Jan 25 11:23:57 2010 From: don at drexel.edu (Don Pellegrino) Date: Mon, 25 Jan 2010 12:23:57 -0500 Subject: [petsc-users] Persistence of sparse matrices. Message-ID: <3d5932651001250923m53049729ge87c8b77166e0b9a@mail.gmail.com> I would like to create a persistent sparse matrix. PETSc's MatCreateSeqAIJ appears to be a useful way to instantiate a sparse matrix structure in memory [http://cluster.ischool.drexel.edu/~st96wym4/flumap/cgit/cgit.cgi/exp007/tree/src/assign/assign_blast_scores.c?id=72edf83cbac32ea26f056ae85373ad7179707333]. I would like to be able to write the structure out to disk and reload it during later runs so that I can grow it incrementally. My project makes use of HDF5 and that seems to be a feasible technology for saving and loading the matrix. Are routines for saving and loading a sparse matrix already available or would I need to write my own? From knepley at gmail.com Mon Jan 25 11:25:52 2010 From: knepley at gmail.com (Matthew Knepley) Date: Mon, 25 Jan 2010 11:25:52 -0600 Subject: [petsc-users] Persistence of sparse matrices. In-Reply-To: <3d5932651001250923m53049729ge87c8b77166e0b9a@mail.gmail.com> References: <3d5932651001250923m53049729ge87c8b77166e0b9a@mail.gmail.com> Message-ID: I think you want to Viewer technology. Matt On Mon, Jan 25, 2010 at 11:23 AM, Don Pellegrino wrote: > I would like to create a persistent sparse matrix. PETSc's > MatCreateSeqAIJ appears to be a useful way to instantiate a sparse > matrix structure in memory > [ > http://cluster.ischool.drexel.edu/~st96wym4/flumap/cgit/cgit.cgi/exp007/tree/src/assign/assign_blast_scores.c?id=72edf83cbac32ea26f056ae85373ad7179707333 > ]. > I would like to be able to write the structure out to disk and reload > it during later runs so that I can grow it incrementally. My project > makes use of HDF5 and that seems to be a feasible technology for > saving and loading the matrix. Are routines for saving and loading a > sparse matrix already available or would I need to write my own? > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Mon Jan 25 11:27:39 2010 From: bsmith at mcs.anl.gov (Barry Smith) Date: Mon, 25 Jan 2010 11:27:39 -0600 Subject: [petsc-users] Persistence of sparse matrices. In-Reply-To: <3d5932651001250923m53049729ge87c8b77166e0b9a@mail.gmail.com> References: <3d5932651001250923m53049729ge87c8b77166e0b9a@mail.gmail.com> Message-ID: <24E5963C-CCB9-43E6-98F1-16EECC18C207@mcs.anl.gov> You can use MatView() with a PETSc binary viewer and MatLoad() to get it back in. This is pretty efficient for sequential matrices and works ok for parallel matrices with a few processors. If it becomes a bottleneck (which is unlikely) the sequential view/load can likely be improved a bit more. Barry On Jan 25, 2010, at 11:23 AM, Don Pellegrino wrote: > I would like to create a persistent sparse matrix. PETSc's > MatCreateSeqAIJ appears to be a useful way to instantiate a sparse > matrix structure in memory > [http://cluster.ischool.drexel.edu/~st96wym4/flumap/cgit/cgit.cgi/exp007/tree/src/assign/assign_blast_scores.c?id=72edf83cbac32ea26f056ae85373ad7179707333 > ]. > I would like to be able to write the structure out to disk and reload > it during later runs so that I can grow it incrementally. My project > makes use of HDF5 and that seems to be a feasible technology for > saving and loading the matrix. Are routines for saving and loading a > sparse matrix already available or would I need to write my own? From don at drexel.edu Mon Jan 25 17:40:05 2010 From: don at drexel.edu (Don Pellegrino) Date: Mon, 25 Jan 2010 18:40:05 -0500 Subject: [petsc-users] Persistence of sparse matrices. In-Reply-To: References: <3d5932651001250923m53049729ge87c8b77166e0b9a@mail.gmail.com> Message-ID: <3d5932651001251540k20723980l282a859647fb9402@mail.gmail.com> On Mon, Jan 25, 2010 at 12:25 PM, Matthew Knepley wrote: > I think you want to Viewer technology. Thanks for pointing me at the PetscViewer functions. I wrote a quick test case to write a sparse matrix to an HDF5 file [http://cluster.ischool.drexel.edu/~st96wym4/flumap/cgit/cgit.cgi/exp007/tree/src/assign/assign_blast_scores.c?id=899b82178d3bcc365b07447a5686635c5856c39e]. Unfortunately I receive the following run-time error: [0]PETSC ERROR: --------------------- Error Message ------------------------------------ [0]PETSC ERROR: No support for this operation for this object type! [0]PETSC ERROR: Viewer type hdf5 not supported by SeqAIJ matrices! [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: Petsc Release Version 3.0.0, Patch 10, Tue Nov 24 16:38:09 CST 2009 [0]PETSC ERROR: See docs/changes/index.html for recent updates. [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. [0]PETSC ERROR: See docs/index.html for manual pages. [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: ./updator on a linux-gnu named coffee by don Mon Jan 25 18:28:31 2010 [0]PETSC ERROR: Libraries linked from /home/don/Apps/Installers/petsc-3.0.0-p10/linux-gnu-c-debug/lib [0]PETSC ERROR: Configure run at Mon Jan 25 17:57:35 2010 [0]PETSC ERROR: Configure options --with-hdf5=1 --with-hdf5-dir=/usr --with-shared [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: MatView_SeqAIJ() line 625 in src/mat/impls/aij/seq/aij.c [0]PETSC ERROR: MatView() line 643 in src/mat/interface/matrix.c So it seems that writing sparse matrices to HDF5 is not yet implemented. I suppose such an implementation should be possible. For now I'll try a binary viewer as suggested by Barry Smith. If I were to implement the writer using HDF5 where should I start in the PETSc source? From knepley at gmail.com Mon Jan 25 18:08:44 2010 From: knepley at gmail.com (Matthew Knepley) Date: Mon, 25 Jan 2010 18:08:44 -0600 Subject: [petsc-users] Persistence of sparse matrices. In-Reply-To: <3d5932651001251540k20723980l282a859647fb9402@mail.gmail.com> References: <3d5932651001250923m53049729ge87c8b77166e0b9a@mail.gmail.com> <3d5932651001251540k20723980l282a859647fb9402@mail.gmail.com> Message-ID: On Mon, Jan 25, 2010 at 5:40 PM, Don Pellegrino wrote: > On Mon, Jan 25, 2010 at 12:25 PM, Matthew Knepley > wrote: > > I think you want to Viewer technology. > > Thanks for pointing me at the PetscViewer functions. I wrote a quick > test case to write a sparse matrix to an HDF5 file > [ > http://cluster.ischool.drexel.edu/~st96wym4/flumap/cgit/cgit.cgi/exp007/tree/src/assign/assign_blast_scores.c?id=899b82178d3bcc365b07447a5686635c5856c39e > ]. > Unfortunately I receive the following run-time error: > Yes, so far people have just been saving Vecs (fields). > [0]PETSC ERROR: --------------------- Error Message > ------------------------------------ > [0]PETSC ERROR: No support for this operation for this object type! > [0]PETSC ERROR: Viewer type hdf5 not supported by SeqAIJ matrices! > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: Petsc Release Version 3.0.0, Patch 10, Tue Nov 24 > 16:38:09 CST 2009 > [0]PETSC ERROR: See docs/changes/index.html for recent updates. > [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. > [0]PETSC ERROR: See docs/index.html for manual pages. > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: ./updator on a linux-gnu named coffee by don Mon Jan > 25 18:28:31 2010 > [0]PETSC ERROR: Libraries linked from > /home/don/Apps/Installers/petsc-3.0.0-p10/linux-gnu-c-debug/lib > [0]PETSC ERROR: Configure run at Mon Jan 25 17:57:35 2010 > [0]PETSC ERROR: Configure options --with-hdf5=1 --with-hdf5-dir=/usr > --with-shared > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: MatView_SeqAIJ() line 625 in src/mat/impls/aij/seq/aij.c > [0]PETSC ERROR: MatView() line 643 in src/mat/interface/matrix.c > > So it seems that writing sparse matrices to HDF5 is not yet > implemented. I suppose such an implementation should be possible. > For now I'll try a binary viewer as suggested by Barry Smith. > > If I were to implement the writer using HDF5 where should I start in > the PETSc source? > I would look at the binary viewer. Matt -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Mon Jan 25 20:12:08 2010 From: bsmith at mcs.anl.gov (Barry Smith) Date: Mon, 25 Jan 2010 20:12:08 -0600 Subject: [petsc-users] Persistence of sparse matrices. In-Reply-To: <3d5932651001251540k20723980l282a859647fb9402@mail.gmail.com> References: <3d5932651001250923m53049729ge87c8b77166e0b9a@mail.gmail.com> <3d5932651001251540k20723980l282a859647fb9402@mail.gmail.com> Message-ID: <9E37C202-E282-49AB-885D-A33AB7377A80@mcs.anl.gov> What does HDF5 buy you in this situation? It won't buy you any speed, disk space, or portability. Barry On Jan 25, 2010, at 5:40 PM, Don Pellegrino wrote: > On Mon, Jan 25, 2010 at 12:25 PM, Matthew Knepley > wrote: >> I think you want to Viewer technology. > > Thanks for pointing me at the PetscViewer functions. I wrote a quick > test case to write a sparse matrix to an HDF5 file > [http://cluster.ischool.drexel.edu/~st96wym4/flumap/cgit/cgit.cgi/exp007/tree/src/assign/assign_blast_scores.c?id=899b82178d3bcc365b07447a5686635c5856c39e > ]. > Unfortunately I receive the following run-time error: > > [0]PETSC ERROR: --------------------- Error Message > ------------------------------------ > [0]PETSC ERROR: No support for this operation for this object type! > [0]PETSC ERROR: Viewer type hdf5 not supported by SeqAIJ matrices! > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: Petsc Release Version 3.0.0, Patch 10, Tue Nov 24 > 16:38:09 CST 2009 > [0]PETSC ERROR: See docs/changes/index.html for recent updates. > [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. > [0]PETSC ERROR: See docs/index.html for manual pages. > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: ./updator on a linux-gnu named coffee by don Mon Jan > 25 18:28:31 2010 > [0]PETSC ERROR: Libraries linked from > /home/don/Apps/Installers/petsc-3.0.0-p10/linux-gnu-c-debug/lib > [0]PETSC ERROR: Configure run at Mon Jan 25 17:57:35 2010 > [0]PETSC ERROR: Configure options --with-hdf5=1 --with-hdf5-dir=/usr > --with-shared > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: MatView_SeqAIJ() line 625 in src/mat/impls/aij/seq/ > aij.c > [0]PETSC ERROR: MatView() line 643 in src/mat/interface/matrix.c > > So it seems that writing sparse matrices to HDF5 is not yet > implemented. I suppose such an implementation should be possible. > For now I'll try a binary viewer as suggested by Barry Smith. > > If I were to implement the writer using HDF5 where should I start in > the PETSc source? From xy2102 at columbia.edu Mon Jan 25 21:01:26 2010 From: xy2102 at columbia.edu ((Rebecca) Xuefei YUAN) Date: Mon, 25 Jan 2010 22:01:26 -0500 Subject: [petsc-users] Errors while install pets-dev Message-ID: <20100125220126.0u8u28zhko8k80kc@cubmail.cc.columbia.edu> Hi,All, I try to install a pets-dev locally according to the instruction http://www.mcs.anl.gov/petsc/petsc-as/developers/index.html I have installed mercurial, but when I get the second step, I get some error like: rebecca at YuanWork:~/soft$ hg clone http://petsc.cs.iit.edu/petsc/petsc-dev destination directory: petsc-dev abort: error: Name or service not known Is there anything wrong? Thanks! -- (Rebecca) Xuefei YUAN Department of Applied Physics and Applied Mathematics Columbia University Tel:917-399-8032 www.columbia.edu/~xy2102 From vyan2000 at gmail.com Mon Jan 25 21:03:57 2010 From: vyan2000 at gmail.com (Ryan Yan) Date: Mon, 25 Jan 2010 22:03:57 -0500 Subject: [petsc-users] How to find a good initial guess for a BVP Message-ID: Hi All, Hopefully, this is not considered as an off-topic thread. Since I got snes converged reason -6. Can anyone please share some hints on how to find a good inital guess? I am solving a nonlinear BVP(steady-states) extracted from a time-dependent problem by setting d/dt=0. The equation is conservational law for mass, momentum, internal energy, with an algebraic heating source. After all, v(velocity), P(pressure), U(fluid internal energy), can be solved from this coupled system. I am using Newton's method for this nonlinear system. Frankly, I do not have a good initial guess for the solver. The only information that I have is the inital condition for the time-dependent problem, where the BVP comes from. I have tried my solver with different inital guess for many times, but with no luck of a satisfying residual reduction. So, do I have to solve the time-dependent problem after a long time stepping to get a steady solution? Or is there any better way of finding a good initial guess? Any suggestion is highly appreciated. Yan -------------- next part -------------- An HTML attachment was scrubbed... URL: From balay at mcs.anl.gov Mon Jan 25 21:46:53 2010 From: balay at mcs.anl.gov (Satish Balay) Date: Mon, 25 Jan 2010 21:46:53 -0600 (CST) Subject: [petsc-users] Errors while install pets-dev In-Reply-To: <20100125220126.0u8u28zhko8k80kc@cubmail.cc.columbia.edu> References: <20100125220126.0u8u28zhko8k80kc@cubmail.cc.columbia.edu> Message-ID: Do you have to use proxy for your http connections? If so - perhaps you can try adding the settings to ~/.hgrc - as this url indicates and see if it fixes the problem. Satish On Mon, 25 Jan 2010, (Rebecca) Xuefei YUAN wrote: > Hi,All, > > I try to install a pets-dev locally according to the instruction > > http://www.mcs.anl.gov/petsc/petsc-as/developers/index.html > > I have installed mercurial, but when I get the second step, I get some error > like: > > rebecca at YuanWork:~/soft$ hg clone http://petsc.cs.iit.edu/petsc/petsc-dev > destination directory: petsc-dev > abort: error: Name or service not known > > Is there anything wrong? > > Thanks! > > From balay at mcs.anl.gov Mon Jan 25 21:47:21 2010 From: balay at mcs.anl.gov (Satish Balay) Date: Mon, 25 Jan 2010 21:47:21 -0600 (CST) Subject: [petsc-users] Errors while install pets-dev In-Reply-To: References: <20100125220126.0u8u28zhko8k80kc@cubmail.cc.columbia.edu> Message-ID: Sorry - forgot to mention the url http://mail.openjdk.java.net/pipermail/build-dev/2007-December/000628.html Satish On Mon, 25 Jan 2010, Satish Balay wrote: > Do you have to use proxy for your http connections? If so - perhaps > you can try adding the settings to ~/.hgrc - as this url indicates and > see if it fixes the problem. > > Satish > > On Mon, 25 Jan 2010, (Rebecca) Xuefei YUAN wrote: > > > Hi,All, > > > > I try to install a pets-dev locally according to the instruction > > > > http://www.mcs.anl.gov/petsc/petsc-as/developers/index.html > > > > I have installed mercurial, but when I get the second step, I get some error > > like: > > > > rebecca at YuanWork:~/soft$ hg clone http://petsc.cs.iit.edu/petsc/petsc-dev > > destination directory: petsc-dev > > abort: error: Name or service not known > > > > Is there anything wrong? > > > > Thanks! > > > > > > From xy2102 at columbia.edu Mon Jan 25 21:58:46 2010 From: xy2102 at columbia.edu ((Rebecca) Xuefei YUAN) Date: Mon, 25 Jan 2010 22:58:46 -0500 Subject: [petsc-users] Errors while install pets-dev In-Reply-To: References: <20100125220126.0u8u28zhko8k80kc@cubmail.cc.columbia.edu> Message-ID: <20100125225846.ktzxtioeo8wsgwwg@cubmail.cc.columbia.edu> Dear Satish, I switched to install it manually and encounter an error when "make test". Here are error messages: Completed building libraries ========================================= Now to check if the libraries are working do: make test ========================================= rebecca at YuanWork:~/soft/petsc-dev$ make test Running test examples to verify correct installation C/C++ example src/snes/examples/tutorials/ex19 run successfully with 1 MPI process C/C++ example src/snes/examples/tutorials/ex19 run successfully with 2 MPI processes --------------Error detected during compile or link!----------------------- See http://www.mcs.anl.gov/petsc/petsc-2/documentation/troubleshooting.html /home/rebecca/soft/petsc-dev/linux-gnu-c-debug/bin/mpif90 -c -Wall -Wno-unused-variable -g -I/home/rebecca/soft/petsc-dev/linux-gnu-c-debug/include -I/home/rebecca/soft/petsc-dev/include -I/home/rebecca/soft/petsc-dev/linux-gnu-c-debug/include -I/home/rebecca/soft/petsc-dev/linux-gnu-c-debug/include -I/home/rebecca/soft/petsc-dev/linux-gnu-c-debug/include -o ex5f.o ex5f.F /home/rebecca/soft/petsc-dev/linux-gnu-c-debug/bin/mpif90 -Wall -Wno-unused-variable -g -o ex5f ex5f.o -Wl,-rpath,/home/rebecca/soft/petsc-dev/linux-gnu-c-debug/lib -L/home/rebecca/soft/petsc-dev/linux-gnu-c-debug/lib -lpetscsnes -lpetscksp -lpetscdm -lpetscmat -lpetscvec -lpetsc -lX11 -Wl,-rpath,/home/rebecca/soft/petsc-dev/linux-gnu-c-debug/lib -L/home/rebecca/soft/petsc-dev/linux-gnu-c-debug/lib -lflapack -lfblas -lnsl -lrt -lm -L/home/rebecca/soft/petsc-dev/linux-gnu-c-debug/lib -L/usr/lib/gcc/i486-linux-gnu/4.2.4 -ldl -lmpich -lopa -lpthread -lrt -lgcc_s -lmpichf90 -lgfortran -lm -lm -ldl -lmpich -lopa -lpthread -lrt -lgcc_s -ldl ex5f.o: In function `MAIN__': /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:103: undefined reference to `snescreate_' /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:120: undefined reference to `dacreateglobalvector_' /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:121: undefined reference to `vecduplicate_' /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:131: undefined reference to `dagetcorners_' /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:133: undefined reference to `dagetghostcorners_' /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:211: undefined reference to `snessetfromoptions_' /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:223: undefined reference to `snesgetiterationnumber_' /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:235: undefined reference to `matdestroy_' /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:236: undefined reference to `matdestroy_' /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:237: undefined reference to `vecdestroy_' /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:238: undefined reference to `vecdestroy_' /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:239: undefined reference to `snesdestroy_' /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:240: undefined reference to `dadestroy_' ex5f.o: In function `forminitialguess_': /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:286: undefined reference to `dmgetlocalvector_' /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:299: undefined reference to `dalocaltoglobal_' /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:300: undefined reference to `dmrestorelocalvector_' ex5f.o: In function `formjacobianlocal_': /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:523: undefined reference to `matsetvalueslocal_' /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:538: undefined reference to `matsetvalueslocal_' /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:542: undefined reference to `matassemblybegin_' /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:543: undefined reference to `matassemblyend_' collect2: ld returned 1 exit status make[3]: [ex5f] Error 1 (ignored) /bin/rm -f ex5f.o Completed test examples What could possibly wrong? Thanks! Rebecca Quoting Satish Balay : > Sorry - forgot to mention the url > > http://mail.openjdk.java.net/pipermail/build-dev/2007-December/000628.html > > Satish > > On Mon, 25 Jan 2010, Satish Balay wrote: > >> Do you have to use proxy for your http connections? If so - perhaps >> you can try adding the settings to ~/.hgrc - as this url indicates and >> see if it fixes the problem. >> >> Satish >> >> On Mon, 25 Jan 2010, (Rebecca) Xuefei YUAN wrote: >> >> > Hi,All, >> > >> > I try to install a pets-dev locally according to the instruction >> > >> > http://www.mcs.anl.gov/petsc/petsc-as/developers/index.html >> > >> > I have installed mercurial, but when I get the second step, I get >> some error >> > like: >> > >> > rebecca at YuanWork:~/soft$ hg clone http://petsc.cs.iit.edu/petsc/petsc-dev >> > destination directory: petsc-dev >> > abort: error: Name or service not known >> > >> > Is there anything wrong? >> > >> > Thanks! >> > >> > >> >> > > > -- (Rebecca) Xuefei YUAN Department of Applied Physics and Applied Mathematics Columbia University Tel:917-399-8032 www.columbia.edu/~xy2102 From balay at mcs.anl.gov Mon Jan 25 22:19:20 2010 From: balay at mcs.anl.gov (Satish Balay) Date: Mon, 25 Jan 2010 22:19:20 -0600 (CST) Subject: [petsc-users] Errors while install pets-dev In-Reply-To: <20100125225846.ktzxtioeo8wsgwwg@cubmail.cc.columbia.edu> References: <20100125220126.0u8u28zhko8k80kc@cubmail.cc.columbia.edu> <20100125225846.ktzxtioeo8wsgwwg@cubmail.cc.columbia.edu> Message-ID: On Mon, 25 Jan 2010, (Rebecca) Xuefei YUAN wrote: > Dear Satish, > > I switched to install it manually What do you mean? Did you get the nightly tarball? > and encounter an error when "make test". Did the build go through fine without errors?? Send over configure.log and make.log to petsc-maint. Satish > Here are error messages: > > Completed building libraries > ========================================= > Now to check if the libraries are working do: make test > ========================================= > rebecca at YuanWork:~/soft/petsc-dev$ make test > Running test examples to verify correct installation > C/C++ example src/snes/examples/tutorials/ex19 run successfully with 1 MPI > process > C/C++ example src/snes/examples/tutorials/ex19 run successfully with 2 MPI > processes > --------------Error detected during compile or link!----------------------- > See http://www.mcs.anl.gov/petsc/petsc-2/documentation/troubleshooting.html > /home/rebecca/soft/petsc-dev/linux-gnu-c-debug/bin/mpif90 -c -Wall > -Wno-unused-variable -g > -I/home/rebecca/soft/petsc-dev/linux-gnu-c-debug/include > -I/home/rebecca/soft/petsc-dev/include > -I/home/rebecca/soft/petsc-dev/linux-gnu-c-debug/include > -I/home/rebecca/soft/petsc-dev/linux-gnu-c-debug/include > -I/home/rebecca/soft/petsc-dev/linux-gnu-c-debug/include -o ex5f.o ex5f.F > /home/rebecca/soft/petsc-dev/linux-gnu-c-debug/bin/mpif90 -Wall > -Wno-unused-variable -g -o ex5f ex5f.o > -Wl,-rpath,/home/rebecca/soft/petsc-dev/linux-gnu-c-debug/lib > -L/home/rebecca/soft/petsc-dev/linux-gnu-c-debug/lib -lpetscsnes -lpetscksp > -lpetscdm -lpetscmat -lpetscvec -lpetsc -lX11 > -Wl,-rpath,/home/rebecca/soft/petsc-dev/linux-gnu-c-debug/lib > -L/home/rebecca/soft/petsc-dev/linux-gnu-c-debug/lib -lflapack -lfblas -lnsl > -lrt -lm -L/home/rebecca/soft/petsc-dev/linux-gnu-c-debug/lib > -L/usr/lib/gcc/i486-linux-gnu/4.2.4 -ldl -lmpich -lopa -lpthread -lrt -lgcc_s > -lmpichf90 -lgfortran -lm -lm -ldl -lmpich -lopa -lpthread -lrt -lgcc_s -ldl > ex5f.o: In function `MAIN__': > /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:103: undefined > reference to `snescreate_' > /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:120: undefined > reference to `dacreateglobalvector_' > /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:121: undefined > reference to `vecduplicate_' > /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:131: undefined > reference to `dagetcorners_' > /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:133: undefined > reference to `dagetghostcorners_' > /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:211: undefined > reference to `snessetfromoptions_' > /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:223: undefined > reference to `snesgetiterationnumber_' > /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:235: undefined > reference to `matdestroy_' > /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:236: undefined > reference to `matdestroy_' > /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:237: undefined > reference to `vecdestroy_' > /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:238: undefined > reference to `vecdestroy_' > /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:239: undefined > reference to `snesdestroy_' > /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:240: undefined > reference to `dadestroy_' > ex5f.o: In function `forminitialguess_': > /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:286: undefined > reference to `dmgetlocalvector_' > /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:299: undefined > reference to `dalocaltoglobal_' > /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:300: undefined > reference to `dmrestorelocalvector_' > ex5f.o: In function `formjacobianlocal_': > /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:523: undefined > reference to `matsetvalueslocal_' > /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:538: undefined > reference to `matsetvalueslocal_' > /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:542: undefined > reference to `matassemblybegin_' > /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:543: undefined > reference to `matassemblyend_' > collect2: ld returned 1 exit status > make[3]: [ex5f] Error 1 (ignored) > /bin/rm -f ex5f.o > Completed test examples > > What could possibly wrong? > > Thanks! > > Rebecca > > > > > Quoting Satish Balay : > > > Sorry - forgot to mention the url > > > > http://mail.openjdk.java.net/pipermail/build-dev/2007-December/000628.html > > > > Satish > > > > On Mon, 25 Jan 2010, Satish Balay wrote: > > > > > Do you have to use proxy for your http connections? If so - perhaps > > > you can try adding the settings to ~/.hgrc - as this url indicates and > > > see if it fixes the problem. > > > > > > Satish > > > > > > On Mon, 25 Jan 2010, (Rebecca) Xuefei YUAN wrote: > > > > > > > Hi,All, > > > > > > > > I try to install a pets-dev locally according to the instruction > > > > > > > > http://www.mcs.anl.gov/petsc/petsc-as/developers/index.html > > > > > > > > I have installed mercurial, but when I get the second step, I get some > > > > error > > > > like: > > > > > > > > rebecca at YuanWork:~/soft$ hg clone > > > > http://petsc.cs.iit.edu/petsc/petsc-dev > > > > destination directory: petsc-dev > > > > abort: error: Name or service not known > > > > > > > > Is there anything wrong? > > > > > > > > Thanks! > > > > > > > > > > > > > > > > > > > > > > > > From xy2102 at columbia.edu Mon Jan 25 22:20:59 2010 From: xy2102 at columbia.edu ((Rebecca) Xuefei YUAN) Date: Mon, 25 Jan 2010 23:20:59 -0500 Subject: [petsc-users] Errors while install pets-dev In-Reply-To: References: <20100125220126.0u8u28zhko8k80kc@cubmail.cc.columbia.edu> Message-ID: <20100125232059.x71hex77yo0wsws4@cubmail.cc.columbia.edu> Dear Satish, I tried to find ~/.hgrc file but there is none. Instead, I found a global hgrc file located at /etc/mercurial/hgrc and its content is one sentence of comments: # system-wide mercurial configuration file # See hgrc(5) for more information Shall I create a new .hgrc file as indicated in url: http://www.selenic.com/pipermail/mercurial/2006-June/008690.html I tried to find my proxy host name and port, and in Firefox, it is said "Use system proxy settings" under "Connection Settings" tab under "Preferences" in Firefox, if this is the case, what is my real proxh host name and port? Is [http_proxy] host=proxy:8080 a default one? Thanks very much! Rebecca Quoting Satish Balay : > Sorry - forgot to mention the url > > http://mail.openjdk.java.net/pipermail/build-dev/2007-December/000628.html > > Satish > > On Mon, 25 Jan 2010, Satish Balay wrote: > >> Do you have to use proxy for your http connections? If so - perhaps >> you can try adding the settings to ~/.hgrc - as this url indicates and >> see if it fixes the problem. >> >> Satish >> >> On Mon, 25 Jan 2010, (Rebecca) Xuefei YUAN wrote: >> >> > Hi,All, >> > >> > I try to install a pets-dev locally according to the instruction >> > >> > http://www.mcs.anl.gov/petsc/petsc-as/developers/index.html >> > >> > I have installed mercurial, but when I get the second step, I get >> some error >> > like: >> > >> > rebecca at YuanWork:~/soft$ hg clone http://petsc.cs.iit.edu/petsc/petsc-dev >> > destination directory: petsc-dev >> > abort: error: Name or service not known >> > >> > Is there anything wrong? >> > >> > Thanks! >> > >> > >> >> > > > -- (Rebecca) Xuefei YUAN Department of Applied Physics and Applied Mathematics Columbia University Tel:917-399-8032 www.columbia.edu/~xy2102 From balay at mcs.anl.gov Mon Jan 25 22:32:19 2010 From: balay at mcs.anl.gov (Satish Balay) Date: Mon, 25 Jan 2010 22:32:19 -0600 (CST) Subject: [petsc-users] Errors while install pets-dev In-Reply-To: <20100125232059.x71hex77yo0wsws4@cubmail.cc.columbia.edu> References: <20100125220126.0u8u28zhko8k80kc@cubmail.cc.columbia.edu> <20100125232059.x71hex77yo0wsws4@cubmail.cc.columbia.edu> Message-ID: On Mon, 25 Jan 2010, (Rebecca) Xuefei YUAN wrote: > Dear Satish, > > I tried to find ~/.hgrc file but there is none. Instead, I found a global hgrc > file located at > /etc/mercurial/hgrc > and its content is one sentence of comments: > # system-wide mercurial configuration file > # See hgrc(5) for more information > > Shall I create a new .hgrc file as indicated in url: > > http://www.selenic.com/pipermail/mercurial/2006-June/008690.html Yes - ~/.hgrc is created by the user. You can just have the 2 lines needed for proxy. > I tried to find my proxy host name and port, and in Firefox, it is > said "Use system proxy settings" under "Connection Settings" tab > under "Preferences" in Firefox, if this is the case, what is my real > proxh host name and port? For sure firefox must be using http_proxy [otherwise it would be set to 'no proxy'. But I'm not sure how you can find the settings. What do you have for env |grep -i proxy env |grep -i http Also what OS is this? If you are using gnome - there can be proxy settings there. On my Fedora 12 box with Gnome - the settings can be at: 'System -> Preferences -> Network Proxy' Satish > Is > > [http_proxy] > host=proxy:8080 > > a default one? > > Thanks very much! > > Rebecca > > > > > > Quoting Satish Balay : > > > Sorry - forgot to mention the url > > > > http://mail.openjdk.java.net/pipermail/build-dev/2007-December/000628.html > > > > Satish > > > > On Mon, 25 Jan 2010, Satish Balay wrote: > > > > > Do you have to use proxy for your http connections? If so - perhaps > > > you can try adding the settings to ~/.hgrc - as this url indicates and > > > see if it fixes the problem. > > > > > > Satish > > > > > > On Mon, 25 Jan 2010, (Rebecca) Xuefei YUAN wrote: > > > > > > > Hi,All, > > > > > > > > I try to install a pets-dev locally according to the instruction > > > > > > > > http://www.mcs.anl.gov/petsc/petsc-as/developers/index.html > > > > > > > > I have installed mercurial, but when I get the second step, I get some > > > > error > > > > like: > > > > > > > > rebecca at YuanWork:~/soft$ hg clone > > > > http://petsc.cs.iit.edu/petsc/petsc-dev > > > > destination directory: petsc-dev > > > > abort: error: Name or service not known > > > > > > > > Is there anything wrong? > > > > > > > > Thanks! > > > > > > > > > > > > > > > > > > > > > > > > From xy2102 at columbia.edu Mon Jan 25 22:40:17 2010 From: xy2102 at columbia.edu ((Rebecca) Xuefei YUAN) Date: Mon, 25 Jan 2010 23:40:17 -0500 Subject: [petsc-users] Errors while install pets-dev In-Reply-To: References: <20100125220126.0u8u28zhko8k80kc@cubmail.cc.columbia.edu> <20100125232059.x71hex77yo0wsws4@cubmail.cc.columbia.edu> Message-ID: <20100125234017.q2zin6o8g0o80ooo@cubmail.cc.columbia.edu> Dear Satish, I got these from those two commands: rebecca at YuanWork:~/soft/petsc-dev$ env |grep -i proxy http_proxy=http://:8080/ no_proxy=localhost,127.0.0.0/8,*.local rebecca at YuanWork:~/soft/petsc-dev$ env |grep -i http http_proxy=http://:8080/ And my System -> Preferences -> Network Proxy shows three options: * Direct internet connection * Manual proxy configuration * Automatic proxy configuration I think it is the third one with "Autoconfiguration URL" blank. Does this mean my .hgrc file should have two lines of [http_proxy] host=http://:8080/ Thanks a lot! Rebecca Quoting Satish Balay : > On Mon, 25 Jan 2010, (Rebecca) Xuefei YUAN wrote: > >> Dear Satish, >> >> I tried to find ~/.hgrc file but there is none. Instead, I found a >> global hgrc >> file located at >> /etc/mercurial/hgrc >> and its content is one sentence of comments: >> # system-wide mercurial configuration file >> # See hgrc(5) for more information >> >> Shall I create a new .hgrc file as indicated in url: >> >> http://www.selenic.com/pipermail/mercurial/2006-June/008690.html > > Yes - ~/.hgrc is created by the user. You can just have the 2 lines > needed for proxy. > >> I tried to find my proxy host name and port, and in Firefox, it is >> said "Use system proxy settings" under "Connection Settings" tab >> under "Preferences" in Firefox, if this is the case, what is my real >> proxh host name and port? > > For sure firefox must be using http_proxy [otherwise it would be set > to 'no proxy'. But I'm not sure how you can find the settings. What > do you have for > > env |grep -i proxy > env |grep -i http > > Also what OS is this? If you are using gnome - there can be proxy > settings there. On my Fedora 12 box with Gnome - the settings can be > at: > > 'System -> Preferences -> Network Proxy' > > > Satish > >> Is >> >> [http_proxy] >> host=proxy:8080 >> >> a default one? >> >> Thanks very much! >> >> Rebecca >> >> >> >> >> >> Quoting Satish Balay : >> >> > Sorry - forgot to mention the url >> > >> > http://mail.openjdk.java.net/pipermail/build-dev/2007-December/000628.html >> > >> > Satish >> > >> > On Mon, 25 Jan 2010, Satish Balay wrote: >> > >> > > Do you have to use proxy for your http connections? If so - perhaps >> > > you can try adding the settings to ~/.hgrc - as this url indicates and >> > > see if it fixes the problem. >> > > >> > > Satish >> > > >> > > On Mon, 25 Jan 2010, (Rebecca) Xuefei YUAN wrote: >> > > >> > > > Hi,All, >> > > > >> > > > I try to install a pets-dev locally according to the instruction >> > > > >> > > > http://www.mcs.anl.gov/petsc/petsc-as/developers/index.html >> > > > >> > > > I have installed mercurial, but when I get the second step, I >> get some >> > > > error >> > > > like: >> > > > >> > > > rebecca at YuanWork:~/soft$ hg clone >> > > > http://petsc.cs.iit.edu/petsc/petsc-dev >> > > > destination directory: petsc-dev >> > > > abort: error: Name or service not known >> > > > >> > > > Is there anything wrong? >> > > > >> > > > Thanks! >> > > > >> > > > >> > > >> > > >> > >> > >> > >> >> >> >> > > > -- (Rebecca) Xuefei YUAN Department of Applied Physics and Applied Mathematics Columbia University Tel:917-399-8032 www.columbia.edu/~xy2102 From balay at mcs.anl.gov Mon Jan 25 22:44:50 2010 From: balay at mcs.anl.gov (Satish Balay) Date: Mon, 25 Jan 2010 22:44:50 -0600 (CST) Subject: [petsc-users] Errors while install pets-dev In-Reply-To: <20100125234017.q2zin6o8g0o80ooo@cubmail.cc.columbia.edu> References: <20100125220126.0u8u28zhko8k80kc@cubmail.cc.columbia.edu> <20100125232059.x71hex77yo0wsws4@cubmail.cc.columbia.edu> <20100125234017.q2zin6o8g0o80ooo@cubmail.cc.columbia.edu> Message-ID: On Mon, 25 Jan 2010, (Rebecca) Xuefei YUAN wrote: > Dear Satish, > > I got these from those two commands: > > rebecca at YuanWork:~/soft/petsc-dev$ env |grep -i proxy > http_proxy=http://:8080/ > no_proxy=localhost,127.0.0.0/8,*.local > rebecca at YuanWork:~/soft/petsc-dev$ env |grep -i http > http_proxy=http://:8080/ > > And my System -> Preferences -> Network Proxy shows three options: > > * Direct internet connection > * Manual proxy configuration > * Automatic proxy configuration > > I think it is the third one with "Autoconfiguration URL" blank. > > Does this mean my .hgrc file should have two lines of > [http_proxy] > host=http://:8080/ Give it a try and see if it works. I'm not sure how all this proxy stuff works though.. Satish From balay at mcs.anl.gov Mon Jan 25 22:57:45 2010 From: balay at mcs.anl.gov (Satish Balay) Date: Mon, 25 Jan 2010 22:57:45 -0600 (CST) Subject: [petsc-users] Errors while install pets-dev In-Reply-To: References: <20100125220126.0u8u28zhko8k80kc@cubmail.cc.columbia.edu> <20100125225846.ktzxtioeo8wsgwwg@cubmail.cc.columbia.edu> Message-ID: Ok - is your manual install '2. access tarballs directly from our mercurial server.? http://www.mcs.anl.gov/petsc/petsc-as/developers/index.html And do you have problem with --download-pacakge options due to the proxy issues? If so - that might explain the problem. petsc-dev - with 'hg' - or with 'tarballs from mercurial' server require sowing package for the fortran interface. If configure is unable to install sowing - then fortran interface is not generated. Thus unresolved symbol errors when compiling fortran examples. This can be verified by looking at configure.log and make.log for this build. Satish On Mon, 25 Jan 2010, Satish Balay wrote: > > On Mon, 25 Jan 2010, (Rebecca) Xuefei YUAN wrote: > > > Dear Satish, > > > > I switched to install it manually > > What do you mean? Did you get the nightly tarball? > > > and encounter an error when "make test". > > Did the build go through fine without errors?? > > Send over configure.log and make.log to petsc-maint. > > Satish > > > Here are error messages: > > > > Completed building libraries > > ========================================= > > Now to check if the libraries are working do: make test > > ========================================= > > rebecca at YuanWork:~/soft/petsc-dev$ make test > > Running test examples to verify correct installation > > C/C++ example src/snes/examples/tutorials/ex19 run successfully with 1 MPI > > process > > C/C++ example src/snes/examples/tutorials/ex19 run successfully with 2 MPI > > processes > > --------------Error detected during compile or link!----------------------- > > See http://www.mcs.anl.gov/petsc/petsc-2/documentation/troubleshooting.html > > /home/rebecca/soft/petsc-dev/linux-gnu-c-debug/bin/mpif90 -c -Wall > > -Wno-unused-variable -g > > -I/home/rebecca/soft/petsc-dev/linux-gnu-c-debug/include > > -I/home/rebecca/soft/petsc-dev/include > > -I/home/rebecca/soft/petsc-dev/linux-gnu-c-debug/include > > -I/home/rebecca/soft/petsc-dev/linux-gnu-c-debug/include > > -I/home/rebecca/soft/petsc-dev/linux-gnu-c-debug/include -o ex5f.o ex5f.F > > /home/rebecca/soft/petsc-dev/linux-gnu-c-debug/bin/mpif90 -Wall > > -Wno-unused-variable -g -o ex5f ex5f.o > > -Wl,-rpath,/home/rebecca/soft/petsc-dev/linux-gnu-c-debug/lib > > -L/home/rebecca/soft/petsc-dev/linux-gnu-c-debug/lib -lpetscsnes -lpetscksp > > -lpetscdm -lpetscmat -lpetscvec -lpetsc -lX11 > > -Wl,-rpath,/home/rebecca/soft/petsc-dev/linux-gnu-c-debug/lib > > -L/home/rebecca/soft/petsc-dev/linux-gnu-c-debug/lib -lflapack -lfblas -lnsl > > -lrt -lm -L/home/rebecca/soft/petsc-dev/linux-gnu-c-debug/lib > > -L/usr/lib/gcc/i486-linux-gnu/4.2.4 -ldl -lmpich -lopa -lpthread -lrt -lgcc_s > > -lmpichf90 -lgfortran -lm -lm -ldl -lmpich -lopa -lpthread -lrt -lgcc_s -ldl > > ex5f.o: In function `MAIN__': > > /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:103: undefined > > reference to `snescreate_' > > /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:120: undefined > > reference to `dacreateglobalvector_' > > /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:121: undefined > > reference to `vecduplicate_' > > /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:131: undefined > > reference to `dagetcorners_' > > /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:133: undefined > > reference to `dagetghostcorners_' > > /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:211: undefined > > reference to `snessetfromoptions_' > > /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:223: undefined > > reference to `snesgetiterationnumber_' > > /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:235: undefined > > reference to `matdestroy_' > > /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:236: undefined > > reference to `matdestroy_' > > /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:237: undefined > > reference to `vecdestroy_' > > /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:238: undefined > > reference to `vecdestroy_' > > /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:239: undefined > > reference to `snesdestroy_' > > /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:240: undefined > > reference to `dadestroy_' > > ex5f.o: In function `forminitialguess_': > > /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:286: undefined > > reference to `dmgetlocalvector_' > > /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:299: undefined > > reference to `dalocaltoglobal_' > > /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:300: undefined > > reference to `dmrestorelocalvector_' > > ex5f.o: In function `formjacobianlocal_': > > /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:523: undefined > > reference to `matsetvalueslocal_' > > /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:538: undefined > > reference to `matsetvalueslocal_' > > /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:542: undefined > > reference to `matassemblybegin_' > > /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:543: undefined > > reference to `matassemblyend_' > > collect2: ld returned 1 exit status > > make[3]: [ex5f] Error 1 (ignored) > > /bin/rm -f ex5f.o > > Completed test examples > > > > What could possibly wrong? > > > > Thanks! > > > > Rebecca > > > > > > > > > > Quoting Satish Balay : > > > > > Sorry - forgot to mention the url > > > > > > http://mail.openjdk.java.net/pipermail/build-dev/2007-December/000628.html > > > > > > Satish > > > > > > On Mon, 25 Jan 2010, Satish Balay wrote: > > > > > > > Do you have to use proxy for your http connections? If so - perhaps > > > > you can try adding the settings to ~/.hgrc - as this url indicates and > > > > see if it fixes the problem. > > > > > > > > Satish > > > > > > > > On Mon, 25 Jan 2010, (Rebecca) Xuefei YUAN wrote: > > > > > > > > > Hi,All, > > > > > > > > > > I try to install a pets-dev locally according to the instruction > > > > > > > > > > http://www.mcs.anl.gov/petsc/petsc-as/developers/index.html > > > > > > > > > > I have installed mercurial, but when I get the second step, I get some > > > > > error > > > > > like: > > > > > > > > > > rebecca at YuanWork:~/soft$ hg clone > > > > > http://petsc.cs.iit.edu/petsc/petsc-dev > > > > > destination directory: petsc-dev > > > > > abort: error: Name or service not known > > > > > > > > > > Is there anything wrong? > > > > > > > > > > Thanks! > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > From xy2102 at columbia.edu Mon Jan 25 22:59:49 2010 From: xy2102 at columbia.edu ((Rebecca) Xuefei YUAN) Date: Mon, 25 Jan 2010 23:59:49 -0500 Subject: [petsc-users] Errors while install pets-dev In-Reply-To: References: <20100125220126.0u8u28zhko8k80kc@cubmail.cc.columbia.edu> <20100125232059.x71hex77yo0wsws4@cubmail.cc.columbia.edu> <20100125234017.q2zin6o8g0o80ooo@cubmail.cc.columbia.edu> Message-ID: <20100125235949.35jgm123wok04gsk@cubmail.cc.columbia.edu> Dear Satish, There is no such an error showed up this time. Thanks very much! Rebecca Quoting Satish Balay : > On Mon, 25 Jan 2010, (Rebecca) Xuefei YUAN wrote: > >> Dear Satish, >> >> I got these from those two commands: >> >> rebecca at YuanWork:~/soft/petsc-dev$ env |grep -i proxy >> http_proxy=http://:8080/ >> no_proxy=localhost,127.0.0.0/8,*.local >> rebecca at YuanWork:~/soft/petsc-dev$ env |grep -i http >> http_proxy=http://:8080/ >> >> And my System -> Preferences -> Network Proxy shows three options: >> >> * Direct internet connection >> * Manual proxy configuration >> * Automatic proxy configuration >> >> I think it is the third one with "Autoconfiguration URL" blank. >> >> Does this mean my .hgrc file should have two lines of >> [http_proxy] >> host=http://:8080/ > > > Give it a try and see if it works. I'm not sure how all this proxy > stuff works though.. > > Satish > > -- (Rebecca) Xuefei YUAN Department of Applied Physics and Applied Mathematics Columbia University Tel:917-399-8032 www.columbia.edu/~xy2102 From xy2102 at columbia.edu Mon Jan 25 23:03:00 2010 From: xy2102 at columbia.edu ((Rebecca) Xuefei YUAN) Date: Tue, 26 Jan 2010 00:03:00 -0500 Subject: [petsc-users] Errors while install pets-dev In-Reply-To: References: <20100125220126.0u8u28zhko8k80kc@cubmail.cc.columbia.edu> <20100125225846.ktzxtioeo8wsgwwg@cubmail.cc.columbia.edu> Message-ID: <20100126000300.mkdqxia2v4g40c08@cubmail.cc.columbia.edu> Dear Satish, Yes, I do have problems with external packages, so I downloaded them manually and move them into petsc-dev. So the configure has options like rebecca at YuanWork:~/soft/petsc-dev$ ./config/configure.py --with-cc=gcc --with-fc=gfortran --download-f-blas-lapack=/home/rebecca/soft/petsc-dev/fblaslapack-3.1.1.tar.gz --download-mpich=/home/rebecca/soft/petsc-dev/mpich2-1.2.1.tar.gz --download-sowing=/home/rebecca/soft/petsc-dev/sowing-1.1.15.tar.gz --download-c2html=/home/rebecca/soft/petsc-dev/c2html.tar.gz The Fortran compiler I have is g77 and gfortran. If I do not use Fortran, will this be fine? Cheers, Rebecca Quoting Satish Balay : > Ok - is your manual install '2. access tarballs directly from our > mercurial server.? > http://www.mcs.anl.gov/petsc/petsc-as/developers/index.html > > > And do you have problem with --download-pacakge options due to the > proxy issues? If so - that might explain the problem. > > petsc-dev - with 'hg' - or with 'tarballs from mercurial' server > require sowing package for the fortran interface. If configure is > unable to install sowing - then fortran interface is not generated. > > Thus unresolved symbol errors when compiling fortran examples. > > This can be verified by looking at configure.log and make.log for this > build. > > Satish > > > On Mon, 25 Jan 2010, Satish Balay wrote: > >> >> On Mon, 25 Jan 2010, (Rebecca) Xuefei YUAN wrote: >> >> > Dear Satish, >> > >> > I switched to install it manually >> >> What do you mean? Did you get the nightly tarball? >> >> > and encounter an error when "make test". >> >> Did the build go through fine without errors?? >> >> Send over configure.log and make.log to petsc-maint. >> >> Satish >> >> > Here are error messages: >> > >> > Completed building libraries >> > ========================================= >> > Now to check if the libraries are working do: make test >> > ========================================= >> > rebecca at YuanWork:~/soft/petsc-dev$ make test >> > Running test examples to verify correct installation >> > C/C++ example src/snes/examples/tutorials/ex19 run successfully with 1 MPI >> > process >> > C/C++ example src/snes/examples/tutorials/ex19 run successfully with 2 MPI >> > processes >> > --------------Error detected during compile or >> link!----------------------- >> > See >> http://www.mcs.anl.gov/petsc/petsc-2/documentation/troubleshooting.html >> > /home/rebecca/soft/petsc-dev/linux-gnu-c-debug/bin/mpif90 -c -Wall >> > -Wno-unused-variable -g >> > -I/home/rebecca/soft/petsc-dev/linux-gnu-c-debug/include >> > -I/home/rebecca/soft/petsc-dev/include >> > -I/home/rebecca/soft/petsc-dev/linux-gnu-c-debug/include >> > -I/home/rebecca/soft/petsc-dev/linux-gnu-c-debug/include >> > -I/home/rebecca/soft/petsc-dev/linux-gnu-c-debug/include -o >> ex5f.o ex5f.F >> > /home/rebecca/soft/petsc-dev/linux-gnu-c-debug/bin/mpif90 -Wall >> > -Wno-unused-variable -g -o ex5f ex5f.o >> > -Wl,-rpath,/home/rebecca/soft/petsc-dev/linux-gnu-c-debug/lib >> > -L/home/rebecca/soft/petsc-dev/linux-gnu-c-debug/lib -lpetscsnes >> -lpetscksp >> > -lpetscdm -lpetscmat -lpetscvec -lpetsc -lX11 >> > -Wl,-rpath,/home/rebecca/soft/petsc-dev/linux-gnu-c-debug/lib >> > -L/home/rebecca/soft/petsc-dev/linux-gnu-c-debug/lib -lflapack >> -lfblas -lnsl >> > -lrt -lm -L/home/rebecca/soft/petsc-dev/linux-gnu-c-debug/lib >> > -L/usr/lib/gcc/i486-linux-gnu/4.2.4 -ldl -lmpich -lopa -lpthread >> -lrt -lgcc_s >> > -lmpichf90 -lgfortran -lm -lm -ldl -lmpich -lopa -lpthread -lrt >> -lgcc_s -ldl >> > ex5f.o: In function `MAIN__': >> > >> /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:103: >> undefined >> > reference to `snescreate_' >> > >> /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:120: >> undefined >> > reference to `dacreateglobalvector_' >> > >> /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:121: >> undefined >> > reference to `vecduplicate_' >> > >> /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:131: >> undefined >> > reference to `dagetcorners_' >> > >> /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:133: >> undefined >> > reference to `dagetghostcorners_' >> > >> /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:211: >> undefined >> > reference to `snessetfromoptions_' >> > >> /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:223: >> undefined >> > reference to `snesgetiterationnumber_' >> > >> /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:235: >> undefined >> > reference to `matdestroy_' >> > >> /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:236: >> undefined >> > reference to `matdestroy_' >> > >> /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:237: >> undefined >> > reference to `vecdestroy_' >> > >> /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:238: >> undefined >> > reference to `vecdestroy_' >> > >> /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:239: >> undefined >> > reference to `snesdestroy_' >> > >> /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:240: >> undefined >> > reference to `dadestroy_' >> > ex5f.o: In function `forminitialguess_': >> > >> /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:286: >> undefined >> > reference to `dmgetlocalvector_' >> > >> /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:299: >> undefined >> > reference to `dalocaltoglobal_' >> > >> /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:300: >> undefined >> > reference to `dmrestorelocalvector_' >> > ex5f.o: In function `formjacobianlocal_': >> > >> /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:523: >> undefined >> > reference to `matsetvalueslocal_' >> > >> /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:538: >> undefined >> > reference to `matsetvalueslocal_' >> > >> /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:542: >> undefined >> > reference to `matassemblybegin_' >> > >> /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:543: >> undefined >> > reference to `matassemblyend_' >> > collect2: ld returned 1 exit status >> > make[3]: [ex5f] Error 1 (ignored) >> > /bin/rm -f ex5f.o >> > Completed test examples >> > >> > What could possibly wrong? >> > >> > Thanks! >> > >> > Rebecca >> > >> > >> > >> > >> > Quoting Satish Balay : >> > >> > > Sorry - forgot to mention the url >> > > >> > > >> http://mail.openjdk.java.net/pipermail/build-dev/2007-December/000628.html >> > > >> > > Satish >> > > >> > > On Mon, 25 Jan 2010, Satish Balay wrote: >> > > >> > > > Do you have to use proxy for your http connections? If so - perhaps >> > > > you can try adding the settings to ~/.hgrc - as this url indicates and >> > > > see if it fixes the problem. >> > > > >> > > > Satish >> > > > >> > > > On Mon, 25 Jan 2010, (Rebecca) Xuefei YUAN wrote: >> > > > >> > > > > Hi,All, >> > > > > >> > > > > I try to install a pets-dev locally according to the instruction >> > > > > >> > > > > http://www.mcs.anl.gov/petsc/petsc-as/developers/index.html >> > > > > >> > > > > I have installed mercurial, but when I get the second step, >> I get some >> > > > > error >> > > > > like: >> > > > > >> > > > > rebecca at YuanWork:~/soft$ hg clone >> > > > > http://petsc.cs.iit.edu/petsc/petsc-dev >> > > > > destination directory: petsc-dev >> > > > > abort: error: Name or service not known >> > > > > >> > > > > Is there anything wrong? >> > > > > >> > > > > Thanks! >> > > > > >> > > > > >> > > > >> > > > >> > > >> > > >> > > >> > >> > >> > >> > >> >> > > > -- (Rebecca) Xuefei YUAN Department of Applied Physics and Applied Mathematics Columbia University Tel:917-399-8032 www.columbia.edu/~xy2102 From balay at mcs.anl.gov Mon Jan 25 23:04:06 2010 From: balay at mcs.anl.gov (Satish Balay) Date: Mon, 25 Jan 2010 23:04:06 -0600 (CST) Subject: [petsc-users] Errors while install pets-dev In-Reply-To: <20100125235949.35jgm123wok04gsk@cubmail.cc.columbia.edu> References: <20100125220126.0u8u28zhko8k80kc@cubmail.cc.columbia.edu> <20100125232059.x71hex77yo0wsws4@cubmail.cc.columbia.edu> <20100125234017.q2zin6o8g0o80ooo@cubmail.cc.columbia.edu> <20100125235949.35jgm123wok04gsk@cubmail.cc.columbia.edu> Message-ID: Gald it works now. BTW: What OS/version do you have? And what version of mercurial is this? [just for completeness] Satish On Mon, 25 Jan 2010, (Rebecca) Xuefei YUAN wrote: > Dear Satish, > > There is no such an error showed up this time. > > Thanks very much! > > Rebecca > > > > > Quoting Satish Balay : > > > On Mon, 25 Jan 2010, (Rebecca) Xuefei YUAN wrote: > > > > > Dear Satish, > > > > > > I got these from those two commands: > > > > > > rebecca at YuanWork:~/soft/petsc-dev$ env |grep -i proxy > > > http_proxy=http://:8080/ > > > no_proxy=localhost,127.0.0.0/8,*.local > > > rebecca at YuanWork:~/soft/petsc-dev$ env |grep -i http > > > http_proxy=http://:8080/ > > > > > > And my System -> Preferences -> Network Proxy shows three options: > > > > > > * Direct internet connection > > > * Manual proxy configuration > > > * Automatic proxy configuration > > > > > > I think it is the third one with "Autoconfiguration URL" blank. > > > > > > Does this mean my .hgrc file should have two lines of > > > [http_proxy] > > > host=http://:8080/ > > > > > > Give it a try and see if it works. I'm not sure how all this proxy > > stuff works though.. > > > > Satish > > > > > > > > From balay at mcs.anl.gov Mon Jan 25 23:08:22 2010 From: balay at mcs.anl.gov (Satish Balay) Date: Mon, 25 Jan 2010 23:08:22 -0600 (CST) Subject: [petsc-users] Errors while install pets-dev In-Reply-To: <20100126000300.mkdqxia2v4g40c08@cubmail.cc.columbia.edu> References: <20100125220126.0u8u28zhko8k80kc@cubmail.cc.columbia.edu> <20100125225846.ktzxtioeo8wsgwwg@cubmail.cc.columbia.edu> <20100126000300.mkdqxia2v4g40c08@cubmail.cc.columbia.edu> Message-ID: The fortran stubs should have been generated with --download-sowing=/home/rebecca/soft/petsc-dev/sowing-1.1.15.tar.gz But - if you don't use fortran - you can ignore fortran errors - or simply use '--with-fc=0' [unless you need an externalpackage that need fortran. For blas you can use --download-c-blas-lapack] Satish On Tue, 26 Jan 2010, (Rebecca) Xuefei YUAN wrote: > Dear Satish, > > Yes, I do have problems with external packages, so I downloaded them manually > and move them into petsc-dev. > > So the configure has options like > > rebecca at YuanWork:~/soft/petsc-dev$ ./config/configure.py --with-cc=gcc > --with-fc=gfortran > --download-f-blas-lapack=/home/rebecca/soft/petsc-dev/fblaslapack-3.1.1.tar.gz > --download-mpich=/home/rebecca/soft/petsc-dev/mpich2-1.2.1.tar.gz > --download-sowing=/home/rebecca/soft/petsc-dev/sowing-1.1.15.tar.gz > --download-c2html=/home/rebecca/soft/petsc-dev/c2html.tar.gz > > The Fortran compiler I have is g77 and gfortran. > > If I do not use Fortran, will this be fine? > > Cheers, > > Rebecca > > Quoting Satish Balay : > > > Ok - is your manual install '2. access tarballs directly from our > > mercurial server.? > > http://www.mcs.anl.gov/petsc/petsc-as/developers/index.html > > > > > > And do you have problem with --download-pacakge options due to the > > proxy issues? If so - that might explain the problem. > > > > petsc-dev - with 'hg' - or with 'tarballs from mercurial' server > > require sowing package for the fortran interface. If configure is > > unable to install sowing - then fortran interface is not generated. > > > > Thus unresolved symbol errors when compiling fortran examples. > > > > This can be verified by looking at configure.log and make.log for this > > build. > > > > Satish > > > > > > On Mon, 25 Jan 2010, Satish Balay wrote: > > > > > > > > On Mon, 25 Jan 2010, (Rebecca) Xuefei YUAN wrote: > > > > > > > Dear Satish, > > > > > > > > I switched to install it manually > > > > > > What do you mean? Did you get the nightly tarball? > > > > > > > and encounter an error when "make test". > > > > > > Did the build go through fine without errors?? > > > > > > Send over configure.log and make.log to petsc-maint. > > > > > > Satish > > > > > > > Here are error messages: > > > > > > > > Completed building libraries > > > > ========================================= > > > > Now to check if the libraries are working do: make test > > > > ========================================= > > > > rebecca at YuanWork:~/soft/petsc-dev$ make test > > > > Running test examples to verify correct installation > > > > C/C++ example src/snes/examples/tutorials/ex19 run successfully with 1 > > > > MPI > > > > process > > > > C/C++ example src/snes/examples/tutorials/ex19 run successfully with 2 > > > > MPI > > > > processes > > > > --------------Error detected during compile or > > > > link!----------------------- > > > > See > > > > http://www.mcs.anl.gov/petsc/petsc-2/documentation/troubleshooting.html > > > > /home/rebecca/soft/petsc-dev/linux-gnu-c-debug/bin/mpif90 -c -Wall > > > > -Wno-unused-variable -g > > > > -I/home/rebecca/soft/petsc-dev/linux-gnu-c-debug/include > > > > -I/home/rebecca/soft/petsc-dev/include > > > > -I/home/rebecca/soft/petsc-dev/linux-gnu-c-debug/include > > > > -I/home/rebecca/soft/petsc-dev/linux-gnu-c-debug/include > > > > -I/home/rebecca/soft/petsc-dev/linux-gnu-c-debug/include -o ex5f.o > > > > ex5f.F > > > > /home/rebecca/soft/petsc-dev/linux-gnu-c-debug/bin/mpif90 -Wall > > > > -Wno-unused-variable -g -o ex5f ex5f.o > > > > -Wl,-rpath,/home/rebecca/soft/petsc-dev/linux-gnu-c-debug/lib > > > > -L/home/rebecca/soft/petsc-dev/linux-gnu-c-debug/lib -lpetscsnes > > > > -lpetscksp > > > > -lpetscdm -lpetscmat -lpetscvec -lpetsc -lX11 > > > > -Wl,-rpath,/home/rebecca/soft/petsc-dev/linux-gnu-c-debug/lib > > > > -L/home/rebecca/soft/petsc-dev/linux-gnu-c-debug/lib -lflapack -lfblas > > > > -lnsl > > > > -lrt -lm -L/home/rebecca/soft/petsc-dev/linux-gnu-c-debug/lib > > > > -L/usr/lib/gcc/i486-linux-gnu/4.2.4 -ldl -lmpich -lopa -lpthread -lrt > > > > -lgcc_s > > > > -lmpichf90 -lgfortran -lm -lm -ldl -lmpich -lopa -lpthread -lrt -lgcc_s > > > > -ldl > > > > ex5f.o: In function `MAIN__': > > > > /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:103: > > > > undefined > > > > reference to `snescreate_' > > > > /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:120: > > > > undefined > > > > reference to `dacreateglobalvector_' > > > > /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:121: > > > > undefined > > > > reference to `vecduplicate_' > > > > /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:131: > > > > undefined > > > > reference to `dagetcorners_' > > > > /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:133: > > > > undefined > > > > reference to `dagetghostcorners_' > > > > /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:211: > > > > undefined > > > > reference to `snessetfromoptions_' > > > > /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:223: > > > > undefined > > > > reference to `snesgetiterationnumber_' > > > > /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:235: > > > > undefined > > > > reference to `matdestroy_' > > > > /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:236: > > > > undefined > > > > reference to `matdestroy_' > > > > /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:237: > > > > undefined > > > > reference to `vecdestroy_' > > > > /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:238: > > > > undefined > > > > reference to `vecdestroy_' > > > > /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:239: > > > > undefined > > > > reference to `snesdestroy_' > > > > /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:240: > > > > undefined > > > > reference to `dadestroy_' > > > > ex5f.o: In function `forminitialguess_': > > > > /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:286: > > > > undefined > > > > reference to `dmgetlocalvector_' > > > > /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:299: > > > > undefined > > > > reference to `dalocaltoglobal_' > > > > /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:300: > > > > undefined > > > > reference to `dmrestorelocalvector_' > > > > ex5f.o: In function `formjacobianlocal_': > > > > /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:523: > > > > undefined > > > > reference to `matsetvalueslocal_' > > > > /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:538: > > > > undefined > > > > reference to `matsetvalueslocal_' > > > > /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:542: > > > > undefined > > > > reference to `matassemblybegin_' > > > > /home/rebecca/soft/petsc-dev/src/snes/examples/tutorials/ex5f.F:543: > > > > undefined > > > > reference to `matassemblyend_' > > > > collect2: ld returned 1 exit status > > > > make[3]: [ex5f] Error 1 (ignored) > > > > /bin/rm -f ex5f.o > > > > Completed test examples > > > > > > > > What could possibly wrong? > > > > > > > > Thanks! > > > > > > > > Rebecca > > > > > > > > > > > > > > > > > > > > Quoting Satish Balay : > > > > > > > > > Sorry - forgot to mention the url > > > > > > > > > > > > > > http://mail.openjdk.java.net/pipermail/build-dev/2007-December/000628.html > > > > > > > > > > Satish > > > > > > > > > > On Mon, 25 Jan 2010, Satish Balay wrote: > > > > > > > > > > > Do you have to use proxy for your http connections? If so - perhaps > > > > > > you can try adding the settings to ~/.hgrc - as this url indicates > > > > and > > > > > > see if it fixes the problem. > > > > > > > > > > > > Satish > > > > > > > > > > > > On Mon, 25 Jan 2010, (Rebecca) Xuefei YUAN wrote: > > > > > > > > > > > > > Hi,All, > > > > > > > > > > > > > > I try to install a pets-dev locally according to the instruction > > > > > > > > > > > > > > http://www.mcs.anl.gov/petsc/petsc-as/developers/index.html > > > > > > > > > > > > > > I have installed mercurial, but when I get the second step, I get > > > > some > > > > > > > error > > > > > > > like: > > > > > > > > > > > > > > rebecca at YuanWork:~/soft$ hg clone > > > > > > > http://petsc.cs.iit.edu/petsc/petsc-dev > > > > > > > destination directory: petsc-dev > > > > > > > abort: error: Name or service not known > > > > > > > > > > > > > > Is there anything wrong? > > > > > > > > > > > > > > Thanks! > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > From xy2102 at columbia.edu Mon Jan 25 23:09:29 2010 From: xy2102 at columbia.edu ((Rebecca) Xuefei YUAN) Date: Tue, 26 Jan 2010 00:09:29 -0500 Subject: [petsc-users] Errors while install pets-dev In-Reply-To: <20100125235949.35jgm123wok04gsk@cubmail.cc.columbia.edu> References: <20100125220126.0u8u28zhko8k80kc@cubmail.cc.columbia.edu> <20100125232059.x71hex77yo0wsws4@cubmail.cc.columbia.edu> <20100125234017.q2zin6o8g0o80ooo@cubmail.cc.columbia.edu> <20100125235949.35jgm123wok04gsk@cubmail.cc.columbia.edu> Message-ID: <20100126000929.ltmslbguvwwss48c@cubmail.cc.columbia.edu> oops, an error comes up: rebecca at YuanWork:~/soft$ hg clone http://petsc.cs.iit.edu/petsc/petsc-dev destination directory: petsc-dev abort: error: Connection timed out The problem might still be there. If I ignore the "make test" error happened to ex5.f, I am able to make petsc-dev/src/snes/examples/tutorials/ex48.c successfully. Thanks again very much! Rebecca Quoting "(Rebecca) Xuefei YUAN" : > Dear Satish, > > There is no such an error showed up this time. > > Thanks very much! > > Rebecca > > > > > Quoting Satish Balay : > >> On Mon, 25 Jan 2010, (Rebecca) Xuefei YUAN wrote: >> >>> Dear Satish, >>> >>> I got these from those two commands: >>> >>> rebecca at YuanWork:~/soft/petsc-dev$ env |grep -i proxy >>> http_proxy=http://:8080/ >>> no_proxy=localhost,127.0.0.0/8,*.local >>> rebecca at YuanWork:~/soft/petsc-dev$ env |grep -i http >>> http_proxy=http://:8080/ >>> >>> And my System -> Preferences -> Network Proxy shows three options: >>> >>> * Direct internet connection >>> * Manual proxy configuration >>> * Automatic proxy configuration >>> >>> I think it is the third one with "Autoconfiguration URL" blank. >>> >>> Does this mean my .hgrc file should have two lines of >>> [http_proxy] >>> host=http://:8080/ >> >> >> Give it a try and see if it works. I'm not sure how all this proxy >> stuff works though.. >> >> Satish >> >> > > > > -- > (Rebecca) Xuefei YUAN > Department of Applied Physics and Applied Mathematics > Columbia University > Tel:917-399-8032 > www.columbia.edu/~xy2102 -- (Rebecca) Xuefei YUAN Department of Applied Physics and Applied Mathematics Columbia University Tel:917-399-8032 www.columbia.edu/~xy2102 From xy2102 at columbia.edu Mon Jan 25 23:12:34 2010 From: xy2102 at columbia.edu ((Rebecca) Xuefei YUAN) Date: Tue, 26 Jan 2010 00:12:34 -0500 Subject: [petsc-users] Errors while install pets-dev In-Reply-To: References: <20100125220126.0u8u28zhko8k80kc@cubmail.cc.columbia.edu> <20100125232059.x71hex77yo0wsws4@cubmail.cc.columbia.edu> <20100125234017.q2zin6o8g0o80ooo@cubmail.cc.columbia.edu> <20100125235949.35jgm123wok04gsk@cubmail.cc.columbia.edu> Message-ID: <20100126001234.o66zu3me4gs48k88@cubmail.cc.columbia.edu> I do not know why the connection time is out, but for completeness, I am using Ubuntu 8.04, and the mercurial version is Mercurial Distributed SCM (version 0.9.5). Cheers, Rebecca Quoting Satish Balay : > Gald it works now. > > BTW: What OS/version do you have? And what version of mercurial is > this? [just for completeness] > > Satish > > On Mon, 25 Jan 2010, (Rebecca) Xuefei YUAN wrote: > >> Dear Satish, >> >> There is no such an error showed up this time. >> >> Thanks very much! >> >> Rebecca >> >> >> >> >> Quoting Satish Balay : >> >> > On Mon, 25 Jan 2010, (Rebecca) Xuefei YUAN wrote: >> > >> > > Dear Satish, >> > > >> > > I got these from those two commands: >> > > >> > > rebecca at YuanWork:~/soft/petsc-dev$ env |grep -i proxy >> > > http_proxy=http://:8080/ >> > > no_proxy=localhost,127.0.0.0/8,*.local >> > > rebecca at YuanWork:~/soft/petsc-dev$ env |grep -i http >> > > http_proxy=http://:8080/ >> > > >> > > And my System -> Preferences -> Network Proxy shows three options: >> > > >> > > * Direct internet connection >> > > * Manual proxy configuration >> > > * Automatic proxy configuration >> > > >> > > I think it is the third one with "Autoconfiguration URL" blank. >> > > >> > > Does this mean my .hgrc file should have two lines of >> > > [http_proxy] >> > > host=http://:8080/ >> > >> > >> > Give it a try and see if it works. I'm not sure how all this proxy >> > stuff works though.. >> > >> > Satish >> > >> > >> >> >> >> > > > -- (Rebecca) Xuefei YUAN Department of Applied Physics and Applied Mathematics Columbia University Tel:917-399-8032 www.columbia.edu/~xy2102 From balay at mcs.anl.gov Mon Jan 25 23:22:49 2010 From: balay at mcs.anl.gov (Satish Balay) Date: Mon, 25 Jan 2010 23:22:49 -0600 (CST) Subject: [petsc-users] Errors while install pets-dev In-Reply-To: <20100126001234.o66zu3me4gs48k88@cubmail.cc.columbia.edu> References: <20100125220126.0u8u28zhko8k80kc@cubmail.cc.columbia.edu> <20100125232059.x71hex77yo0wsws4@cubmail.cc.columbia.edu> <20100125234017.q2zin6o8g0o80ooo@cubmail.cc.columbia.edu> <20100125235949.35jgm123wok04gsk@cubmail.cc.columbia.edu> <20100126001234.o66zu3me4gs48k88@cubmail.cc.columbia.edu> Message-ID: Ok. My guess is - 0.9.5 is a bit too old - and perhaps newer versions might work automatically with the 'system proxy' settings. The proxy url 'http://:8080/' looks non-standard [Doesn't list a machine] - so that might be something mercurial proxy is unable to handle. Satish On Tue, 26 Jan 2010, (Rebecca) Xuefei YUAN wrote: > I do not know why the connection time is out, but for completeness, I am using > Ubuntu 8.04, and the mercurial version is Mercurial Distributed SCM (version > 0.9.5). > > Cheers, > > Rebecca > > > > > Quoting Satish Balay : > > > Gald it works now. > > > > BTW: What OS/version do you have? And what version of mercurial is > > this? [just for completeness] > > > > Satish > > > > On Mon, 25 Jan 2010, (Rebecca) Xuefei YUAN wrote: > > > > > Dear Satish, > > > > > > There is no such an error showed up this time. > > > > > > Thanks very much! > > > > > > Rebecca > > > > > > > > > > > > > > > Quoting Satish Balay : > > > > > > > On Mon, 25 Jan 2010, (Rebecca) Xuefei YUAN wrote: > > > > > > > > > Dear Satish, > > > > > > > > > > I got these from those two commands: > > > > > > > > > > rebecca at YuanWork:~/soft/petsc-dev$ env |grep -i proxy > > > > > http_proxy=http://:8080/ > > > > > no_proxy=localhost,127.0.0.0/8,*.local > > > > > rebecca at YuanWork:~/soft/petsc-dev$ env |grep -i http > > > > > http_proxy=http://:8080/ > > > > > > > > > > And my System -> Preferences -> Network Proxy shows three options: > > > > > > > > > > * Direct internet connection > > > > > * Manual proxy configuration > > > > > * Automatic proxy configuration > > > > > > > > > > I think it is the third one with "Autoconfiguration URL" blank. > > > > > > > > > > Does this mean my .hgrc file should have two lines of > > > > > [http_proxy] > > > > > host=http://:8080/ > > > > > > > > > > > > Give it a try and see if it works. I'm not sure how all this proxy > > > > stuff works though.. > > > > > > > > Satish > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > From knepley at gmail.com Tue Jan 26 04:46:47 2010 From: knepley at gmail.com (Matthew Knepley) Date: Tue, 26 Jan 2010 04:46:47 -0600 Subject: [petsc-users] How to find a good initial guess for a BVP In-Reply-To: References: Message-ID: You might try the pseudo-transient continuation code. Matt On Mon, Jan 25, 2010 at 9:03 PM, Ryan Yan wrote: > Hi All, > Hopefully, this is not considered as an off-topic thread. Since I got snes > converged reason -6. > > Can anyone please share some hints on how to find a good inital guess? > > I am solving a nonlinear BVP(steady-states) extracted from a time-dependent > problem by setting d/dt=0. The equation is conservational law for mass, > momentum, internal energy, with an algebraic heating source. After all, > v(velocity), P(pressure), U(fluid internal energy), can be solved from this > coupled system. I am using Newton's method for this nonlinear system. > Frankly, I do not have a good initial guess for the solver. The only > information that I have is the inital condition for the time-dependent > problem, where the BVP comes from. I have tried my solver with different > inital guess for many times, but with no luck of a satisfying residual > reduction. > > So, do I have to solve the time-dependent problem after a long time > stepping to get a steady solution? Or is there any better way of finding a > good initial guess? > > Any suggestion is highly appreciated. > > Yan > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From jed at 59A2.org Tue Jan 26 04:49:21 2010 From: jed at 59A2.org (Jed Brown) Date: Tue, 26 Jan 2010 11:49:21 +0100 Subject: [petsc-users] How to find a good initial guess for a BVP In-Reply-To: References: Message-ID: <878wblqgy6.fsf@59A2.org> On Mon, 25 Jan 2010 22:03:57 -0500, Ryan Yan wrote: > I am solving a nonlinear BVP(steady-states) extracted from a time-dependent > problem by setting d/dt=0. Globalization of steady-state problems is notoriously difficult, it's very likely that you will need to perform a continuation, of which there are at least two kinds to consider. 1. Pseudotransient continuation, I like this paper http://www.cs.odu.edu/~keyes/papers/ptc03.pdf which explains snes/examples/tutorials/ex27.c. This can be done with TSPSEUDO, but not currently for differential algebraic systems. If you would like it to work with DAEs, or ODEs written in implicit form (f(t,x,x')=0 instead of x' = f(t,x)), let me know and I'll add such support to PETSc-dev. 2. Grid sequencing: solve the problem on coarser grids to get an initial guess on the finer grids. If you use DMMG, this is -dmmg_grid_sequence. > So, do I have to solve the time-dependent problem after a long time stepping > to get a steady solution? Or is there any better way of finding a good > initial guess? Pseudotransient continuation is somewhat like this, but does it in a clever and adaptive way. Jed From vyan2000 at gmail.com Tue Jan 26 10:13:18 2010 From: vyan2000 at gmail.com (Ryan Yan) Date: Tue, 26 Jan 2010 11:13:18 -0500 Subject: [petsc-users] How to find a good initial guess for a BVP In-Reply-To: References: Message-ID: Hi Matt, Thanks a lot for the suggestion. Yan On Tue, Jan 26, 2010 at 5:46 AM, Matthew Knepley wrote: > You might try the pseudo-transient continuation code. > > Matt > > > On Mon, Jan 25, 2010 at 9:03 PM, Ryan Yan wrote: > >> Hi All, >> Hopefully, this is not considered as an off-topic thread. Since I got snes >> converged reason -6. >> >> Can anyone please share some hints on how to find a good inital guess? >> >> I am solving a nonlinear BVP(steady-states) extracted from a >> time-dependent problem by setting d/dt=0. The equation is conservational law >> for mass, momentum, internal energy, with an algebraic heating source. After >> all, v(velocity), P(pressure), U(fluid internal energy), can be solved from >> this coupled system. I am using Newton's method for this nonlinear system. >> Frankly, I do not have a good initial guess for the solver. The only >> information that I have is the inital condition for the time-dependent >> problem, where the BVP comes from. I have tried my solver with different >> inital guess for many times, but with no luck of a satisfying residual >> reduction. >> >> So, do I have to solve the time-dependent problem after a long time >> stepping to get a steady solution? Or is there any better way of finding a >> good initial guess? >> >> Any suggestion is highly appreciated. >> >> Yan >> > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vyan2000 at gmail.com Tue Jan 26 10:27:01 2010 From: vyan2000 at gmail.com (Ryan Yan) Date: Tue, 26 Jan 2010 11:27:01 -0500 Subject: [petsc-users] How to find a good initial guess for a BVP In-Reply-To: <878wblqgy6.fsf@59A2.org> References: <878wblqgy6.fsf@59A2.org> Message-ID: Hi Jed, Thank you very much for the suggestion and providing the reference and examples. I have tried the grid sequencing, but with little luck on that. I will try the approach 1 for the explicit form of the conservation system. Yan On Tue, Jan 26, 2010 at 5:49 AM, Jed Brown wrote: > On Mon, 25 Jan 2010 22:03:57 -0500, Ryan Yan wrote: > > I am solving a nonlinear BVP(steady-states) extracted from a > time-dependent > > problem by setting d/dt=0. > > Globalization of steady-state problems is notoriously difficult, it's > very likely that you will need to perform a continuation, of which there > are at least two kinds to consider. > > 1. Pseudotransient continuation, I like this paper > > http://www.cs.odu.edu/~keyes/papers/ptc03.pdf > > which explains snes/examples/tutorials/ex27.c. > > This can be done with TSPSEUDO, but not currently for differential > algebraic systems. If you would like it to work with DAEs, or ODEs > written in implicit form (f(t,x,x')=0 instead of x' = f(t,x)), let me > know and I'll add such support to PETSc-dev. > > 2. Grid sequencing: solve the problem on coarser grids to get an initial > guess on the finer grids. If you use DMMG, this is -dmmg_grid_sequence. > > > So, do I have to solve the time-dependent problem after a long time > stepping > > to get a steady solution? Or is there any better way of finding a good > > initial guess? > > Pseudotransient continuation is somewhat like this, but does it in a > clever and adaptive way. > > Jed > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vyan2000 at gmail.com Tue Jan 26 16:57:38 2010 From: vyan2000 at gmail.com (Ryan Yan) Date: Tue, 26 Jan 2010 17:57:38 -0500 Subject: [petsc-users] Is the staggered grid good for SNES? or only for SIMPLE scheme? Message-ID: Hi All, I am re-thinking my discretization of a problem: is the stagger grid only good for SIMPLE scheme? I am wondering if I am using a stagger grid with FVM to discretize the conservation system(velocity, pressure, internal energy), is there any drawback of using newton's method to solve it, comparing with the SIMPLE scheme for which the staggered grid is suit for? I am not sure whether the question is asked properly, please feel free to ask more information if needed. Any comment is welcome. Thanks a lot, Yan -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Tue Jan 26 17:08:42 2010 From: bsmith at mcs.anl.gov (Barry Smith) Date: Tue, 26 Jan 2010 17:08:42 -0600 Subject: [petsc-users] Is the staggered grid good for SNES? or only for SIMPLE scheme? In-Reply-To: References: Message-ID: Typing "why use staggered grids?" into google gave me the first answer: http://www.cfd-online.com/Wiki/Staggered_grid Though a bit terse and far from complete it is more or less correct. As it states, coding a non-staggered grid is generally easier, You can use or not use staggered grids for such problems but you should understand something about the issues before you do anything. This isn't really a question about PDEs but a question about PDE discretizations for which there is a huge literature. You should talk to a fluid dynamics person about your particular problem. Barry On Jan 26, 2010, at 4:57 PM, Ryan Yan wrote: > Hi All, > I am re-thinking my discretization of a problem: is the stagger grid > only good for SIMPLE scheme? > > I am wondering if I am using a stagger grid with FVM to discretize > the conservation system(velocity, pressure, internal energy), is > there any drawback of using newton's method to solve it, comparing > with the SIMPLE scheme for which the staggered grid is suit for? I > am not sure whether the question is asked properly, please feel free > to ask more information if needed. Any comment is welcome. > > Thanks a lot, > > Yan From vyan2000 at gmail.com Tue Jan 26 19:11:50 2010 From: vyan2000 at gmail.com (Ryan Yan) Date: Tue, 26 Jan 2010 20:11:50 -0500 Subject: [petsc-users] Is the staggered grid good for SNES? or only for SIMPLE scheme? In-Reply-To: References: Message-ID: Hi Barry, On Tue, Jan 26, 2010 at 6:08 PM, Barry Smith wrote: > > Typing "why use staggered grids?" into google gave me the first answer: > http://www.cfd-online.com/Wiki/Staggered_grid > > I did see that webpage and it is where I started. > Though a bit terse and far from complete it is more or less correct. As it > states, coding a non-staggered grid is generally easier, > > Agreed. > You can use or not use staggered grids for such problems but you should > understand something about the issues before you do anything. This isn't > really a question about PDEs but a question about PDE discretizations for > which there is a huge literature. You should talk to a fluid dynamics person > about your particular problem. > > Agreed. Thank you very much for the suggestion. Sorry for this off-topic thread. Yan > Barry > > > On Jan 26, 2010, at 4:57 PM, Ryan Yan wrote: > > Hi All, >> I am re-thinking my discretization of a problem: is the stagger grid only >> good for SIMPLE scheme? >> >> I am wondering if I am using a stagger grid with FVM to discretize the >> conservation system(velocity, pressure, internal energy), is there any >> drawback of using newton's method to solve it, comparing with the SIMPLE >> scheme for which the staggered grid is suit for? I am not sure whether the >> question is asked properly, please feel free to ask more information if >> needed. Any comment is welcome. >> >> Thanks a lot, >> >> Yan >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From michel.cancelliere at polito.it Fri Jan 29 08:54:36 2010 From: michel.cancelliere at polito.it (Michel Cancelliere) Date: Fri, 29 Jan 2010 15:54:36 +0100 Subject: [petsc-users] Question about matrix permutation Message-ID: <7f18de3b1001290654l7132fd07k71b88b43041e3395@mail.gmail.com> Hi all, I'm implementing a preconditioner that involves to extract some submatrices, the system matrix is of type seqbaij, and i need to permute some rows and columns before extract the submatrices. I'm trying to use MatPermute() but i should need create and additional matrix that i don't want (and it seems not to work with baij matrices), I took a look to MatGetOrdering but not sure if it fit to my case. there is any alternative to use something like MatPermute() in a seqbaij matrix but without create a new matrix? Thank you! Michel Cancelliere -------------- next part -------------- An HTML attachment was scrubbed... URL: From abhyshr at mcs.anl.gov Fri Jan 29 09:15:20 2010 From: abhyshr at mcs.anl.gov (abhyshr at mcs.anl.gov) Date: Fri, 29 Jan 2010 09:15:20 -0600 (CST) Subject: [petsc-users] Question about matrix permutation In-Reply-To: <25932678.116301264778055323.JavaMail.root@zimbra> Message-ID: <21866499.116341264778120071.JavaMail.root@zimbra> You can use MatGetSubMatrix. http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/Mat/MatGetSubMatrix.html ----- "Michel Cancelliere" wrote: > Hi all, > I'm implementing a preconditioner that involves to extract some submatrices, the system matrix is of type seqbaij, and i need to permute some rows and columns before extract the submatrices. I'm trying to use MatPermute() but i should need create and additional matrix that i don't want (and it seems not to work with baij matrices), I took a look to MatGetOrdering but not sure if it fit to my case. there is any alternative to use something like MatPermute() in a seqbaij matrix but without create a new matrix? > Thank you! > Michel Cancelliere > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jed at 59A2.org Fri Jan 29 09:28:54 2010 From: jed at 59A2.org (Jed Brown) Date: Fri, 29 Jan 2010 16:28:54 +0100 Subject: [petsc-users] Question about matrix permutation In-Reply-To: <7f18de3b1001290654l7132fd07k71b88b43041e3395@mail.gmail.com> References: <7f18de3b1001290654l7132fd07k71b88b43041e3395@mail.gmail.com> Message-ID: <87tyu5x749.fsf@59A2.org> On Fri, 29 Jan 2010 15:54:36 +0100, Michel Cancelliere wrote: > Hi all, > > I'm implementing a preconditioner that involves to extract some submatrices, > the system matrix is of type seqbaij, and i need to permute some rows and > columns before extract the submatrices. How big are these submatrices relative to the whole thing? The way MatGetSubMatrix is implemented, it needs sorted index sets, but you could extract the submatrix and then sort. It's not clear to me if this optimization is important for performance in the context that MatGetSubMatrix is normally used (i.e. a reasonable amount of work is done with the submatrix). And it would certainly not have a performance impact to branch to a generic implementation when the index sets are unsorted. > I'm trying to use MatPermute() but i should need create and additional > matrix If the submatrix is much smaller than the original, this is probably not a problem. If it's large, and there isn't a better way to solve the problem, then maybe MatGetSubMatrix should support unsorted index sets. > and it seems not to work with baij matrices It should work with MPIBAIJ matrices, it looks like it was never implemented for SeqBAIJ, but this could be done fairly easily. Jed From abhyshr at mcs.anl.gov Fri Jan 29 09:43:05 2010 From: abhyshr at mcs.anl.gov (abhyshr at mcs.anl.gov) Date: Fri, 29 Jan 2010 09:43:05 -0600 (CST) Subject: [petsc-users] Question about matrix permutation In-Reply-To: <25100129.117431264779588613.JavaMail.root@zimbra> Message-ID: <15232620.117881264779785465.JavaMail.root@zimbra> Instead of calling MatGetSubMatrix to get each submatrix you can use MatGetSubMatrices to extract several submatrices at once. See http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/Mat/MatGetSubMatrices.html#MatGetSubMatrices. ----- "Jed Brown" wrote: > On Fri, 29 Jan 2010 15:54:36 +0100, Michel Cancelliere > wrote: > > Hi all, > > > > I'm implementing a preconditioner that involves to extract some > submatrices, > > the system matrix is of type seqbaij, and i need to permute some > rows and > > columns before extract the submatrices. > > How big are these submatrices relative to the whole thing? The way > MatGetSubMatrix is implemented, it needs sorted index sets, but you > could extract the submatrix and then sort. It's not clear to me if > this > optimization is important for performance in the context that > MatGetSubMatrix is normally used (i.e. a reasonable amount of work is > done with the submatrix). And it would certainly not have a > performance > impact to branch to a generic implementation when the index sets are > unsorted. > > > I'm trying to use MatPermute() but i should need create and > additional > > matrix > > If the submatrix is much smaller than the original, this is probably > not > a problem. If it's large, and there isn't a better way to solve the > problem, then maybe MatGetSubMatrix should support unsorted index > sets. > > > and it seems not to work with baij matrices > > It should work with MPIBAIJ matrices, it looks like it was never > implemented for SeqBAIJ, but this could be done fairly easily. > > Jed From jed at 59A2.org Fri Jan 29 10:01:34 2010 From: jed at 59A2.org (Jed Brown) Date: Fri, 29 Jan 2010 17:01:34 +0100 Subject: [petsc-users] Question about matrix permutation In-Reply-To: <15232620.117881264779785465.JavaMail.root@zimbra> References: <15232620.117881264779785465.JavaMail.root@zimbra> Message-ID: <87sk9oyk69.fsf@59A2.org> On Fri, 29 Jan 2010 09:43:05 -0600 (CST), abhyshr at mcs.anl.gov wrote: > Instead of calling MatGetSubMatrix to get each submatrix you can use MatGetSubMatrices to extract several submatrices at once. See http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/Mat/MatGetSubMatrices.html#MatGetSubMatrices. I don't think Michel was asking to extract multiple matrices. In serial, MatGetSubMatrices is implemented by calling MatGetSubMatrix so there is no weaker assumption on index set sortedness. Jed From fernandez858 at gmail.com Fri Jan 29 11:29:02 2010 From: fernandez858 at gmail.com (Michel Cancelliere) Date: Fri, 29 Jan 2010 18:29:02 +0100 Subject: [petsc-users] Question about matrix permutation In-Reply-To: <87sk9oyk69.fsf@59A2.org> References: <15232620.117881264779785465.JavaMail.root@zimbra> <87sk9oyk69.fsf@59A2.org> Message-ID: <7f18de3b1001290929q75304049l7bd5a98c2ce8bed0@mail.gmail.com> Thank you for your promptly response, i will use unsorted inex set as you suggest (because my submatrices are very large (4 submatrices N/2 x N/2 from the original one Nx N), I realize that i cannot use block matrices because the reordering split within the blocks and the resulting matrices are not block matrices anymore. In terms of performance how much can affect this to the code, in the sense if i apply for example an ILU preconditioner to a block matrix (baij) and solve the same system but this time using an aij matrix ? not sure if it is clear Thanks Michel On 1/29/10, Jed Brown wrote: > On Fri, 29 Jan 2010 09:43:05 -0600 (CST), abhyshr at mcs.anl.gov wrote: >> Instead of calling MatGetSubMatrix to get each submatrix you can use >> MatGetSubMatrices to extract several submatrices at once. See >> http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/Mat/MatGetSubMatrices.html#MatGetSubMatrices. > > I don't think Michel was asking to extract multiple matrices. In > serial, MatGetSubMatrices is implemented by calling MatGetSubMatrix so > there is no weaker assumption on index set sortedness. > > Jed > From knepley at gmail.com Fri Jan 29 11:39:35 2010 From: knepley at gmail.com (Matthew Knepley) Date: Fri, 29 Jan 2010 11:39:35 -0600 Subject: [petsc-users] Question about matrix permutation In-Reply-To: <7f18de3b1001290929q75304049l7bd5a98c2ce8bed0@mail.gmail.com> References: <15232620.117881264779785465.JavaMail.root@zimbra> <87sk9oyk69.fsf@59A2.org> <7f18de3b1001290929q75304049l7bd5a98c2ce8bed0@mail.gmail.com> Message-ID: If what you want to do is apply a block preconditioner to a matrix arising from a regular grid, DA can already do this automatically. This uses the PCFieldSplit, and can be completely controlled from the command line. It is designed to do these sorts of things. I do not understand your last question. Matt On Fri, Jan 29, 2010 at 11:29 AM, Michel Cancelliere wrote: > Thank you for your promptly response, i will use unsorted inex set as > you suggest (because my submatrices are very large (4 submatrices N/2 > x N/2 from the original one Nx N), > I realize that i cannot use block matrices because the reordering > split within the blocks and the resulting matrices are not block > matrices anymore. In terms of performance how much can affect this to > the code, in the sense if i apply for example an ILU preconditioner to > a block matrix (baij) and solve the same system but this time using an > aij matrix ? not sure if it is clear > > Thanks > Michel > > > > > On 1/29/10, Jed Brown wrote: > > On Fri, 29 Jan 2010 09:43:05 -0600 (CST), abhyshr at mcs.anl.gov wrote: > >> Instead of calling MatGetSubMatrix to get each submatrix you can use > >> MatGetSubMatrices to extract several submatrices at once. See > >> > http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/Mat/MatGetSubMatrices.html#MatGetSubMatrices > . > > > > I don't think Michel was asking to extract multiple matrices. In > > serial, MatGetSubMatrices is implemented by calling MatGetSubMatrix so > > there is no weaker assumption on index set sortedness. > > > > Jed > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From jed at 59A2.org Fri Jan 29 11:44:32 2010 From: jed at 59A2.org (Jed Brown) Date: Fri, 29 Jan 2010 18:44:32 +0100 Subject: [petsc-users] Question about matrix permutation In-Reply-To: <7f18de3b1001290929q75304049l7bd5a98c2ce8bed0@mail.gmail.com> References: <15232620.117881264779785465.JavaMail.root@zimbra> <87sk9oyk69.fsf@59A2.org> <7f18de3b1001290929q75304049l7bd5a98c2ce8bed0@mail.gmail.com> Message-ID: <87ockcyfen.fsf@59A2.org> On Fri, 29 Jan 2010 18:29:02 +0100, Michel Cancelliere wrote: > Thank you for your promptly response, i will use unsorted inex set as > you suggest (because my submatrices are very large (4 submatrices N/2 > x N/2 from the original one Nx N), Wait, what are you planning to do with these submatrices? It's likely that PCFieldSplit can already do what you want. Could you describe the algorithm you want to implement? > In terms of performance how much can affect this to the code, in the > sense if i apply for example an ILU preconditioner to a block matrix > (baij) and solve the same system but this time using an aij matrix ? You have to benchmark to be sure, inodes should be used so that gives you some of the benefits of BAIJ. But if your algorithm is like FieldSplit, you'll be spending time solving with the submatrices. Jed From bsmith at mcs.anl.gov Fri Jan 29 13:48:57 2010 From: bsmith at mcs.anl.gov (Barry Smith) Date: Fri, 29 Jan 2010 13:48:57 -0600 Subject: [petsc-users] Question about matrix permutation In-Reply-To: <7f18de3b1001290929q75304049l7bd5a98c2ce8bed0@mail.gmail.com> References: <15232620.117881264779785465.JavaMail.root@zimbra> <87sk9oyk69.fsf@59A2.org> <7f18de3b1001290929q75304049l7bd5a98c2ce8bed0@mail.gmail.com> Message-ID: <7BAAA0C6-3468-4B88-94E0-DB6EFE1E3FE9@mcs.anl.gov> On Jan 29, 2010, at 11:29 AM, Michel Cancelliere wrote: > Thank you for your promptly response, i will use unsorted inex set as > you suggest (because my submatrices are very large (4 submatrices N/2 > x N/2 from the original one Nx N), > I realize that i cannot use block matrices because the reordering > split within the blocks and the resulting matrices are not block > matrices anymore. In terms of performance how much can affect this to > the code, in the sense if i apply for example an ILU preconditioner to > a block matrix (baij) and solve the same system but this time using an > aij matrix ? not sure if it is clear It is not clear what you are asking. **If you have a matrix with say block size of 4 and store it with BAIJ and do factorizations and solvers and it takes N seconds then solve the same system with factorization and solvers with AIJ it will take at least 1.5 * N seconds. The block BAIJ format uses a point block ILU factorization so usually gives better convergence rate than AIJ depending on the problem. But you state that the submatrix you select does not have the same block structure so my answer ** is irrelevent. In general using a smart composite preconditioner (as Jed is taking about with PCFiledSplit or friends) is better then using a dumb global preconditioner like ILU(0) on the whole problem. Yes, the flop rate may be lower because you cannot do block ILU on the BAIJ matrix but there should be so fewer flops because of the better preconditioner that the smart preconditioner wins. If the smart preconditioner does not win then it is not a smart preconditioner for the problem :-) Barry > > Thanks > Michel > > > > > On 1/29/10, Jed Brown wrote: >> On Fri, 29 Jan 2010 09:43:05 -0600 (CST), abhyshr at mcs.anl.gov wrote: >>> Instead of calling MatGetSubMatrix to get each submatrix you can use >>> MatGetSubMatrices to extract several submatrices at once. See >>> http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/Mat/MatGetSubMatrices.html#MatGetSubMatrices >>> . >> >> I don't think Michel was asking to extract multiple matrices. In >> serial, MatGetSubMatrices is implemented by calling MatGetSubMatrix >> so >> there is no weaker assumption on index set sortedness. >> >> Jed >> From jed at 59A2.org Fri Jan 29 14:46:04 2010 From: jed at 59A2.org (Jed Brown) Date: Fri, 29 Jan 2010 21:46:04 +0100 Subject: [petsc-users] Question about matrix permutation In-Reply-To: <7BAAA0C6-3468-4B88-94E0-DB6EFE1E3FE9@mcs.anl.gov> References: <15232620.117881264779785465.JavaMail.root@zimbra> <87sk9oyk69.fsf@59A2.org> <7f18de3b1001290929q75304049l7bd5a98c2ce8bed0@mail.gmail.com> <7BAAA0C6-3468-4B88-94E0-DB6EFE1E3FE9@mcs.anl.gov> Message-ID: <87k4v0y703.fsf@59A2.org> On Fri, 29 Jan 2010 13:48:57 -0600, Barry Smith wrote: > **If you have a matrix with say block size of 4 and store it with > BAIJ and do factorizations and solvers and it takes N seconds then > solve the same system with factorization and solvers with AIJ it will > take at least 1.5 * N seconds. If the AIJ system uses full blocks, then it's much less clear since inodes would be used. I've still found this number to be about right, but not due to the standard "memory bandwidth required for the matrix entries and column indices" performance model. I have a hard time explaining it since the vector access is almost identical. Perhaps it is due to poor prefetch of matrix entries? Jed From bsmith at mcs.anl.gov Fri Jan 29 14:57:51 2010 From: bsmith at mcs.anl.gov (Barry Smith) Date: Fri, 29 Jan 2010 14:57:51 -0600 Subject: [petsc-users] Question about matrix permutation In-Reply-To: <87k4v0y703.fsf@59A2.org> References: <15232620.117881264779785465.JavaMail.root@zimbra> <87sk9oyk69.fsf@59A2.org> <7f18de3b1001290929q75304049l7bd5a98c2ce8bed0@mail.gmail.com> <7BAAA0C6-3468-4B88-94E0-DB6EFE1E3FE9@mcs.anl.gov> <87k4v0y703.fsf@59A2.org> Message-ID: <7DCA6A2F-D1F6-4D55-9C48-1E202474366B@mcs.anl.gov> On Jan 29, 2010, at 2:46 PM, Jed Brown wrote: > On Fri, 29 Jan 2010 13:48:57 -0600, Barry Smith > wrote: >> **If you have a matrix with say block size of 4 and store it with >> BAIJ and do factorizations and solvers and it takes N seconds then >> solve the same system with factorization and solvers with AIJ it will >> take at least 1.5 * N seconds. > > If the AIJ system uses full blocks, then it's much less clear since > inodes would be used. This is why I have the 1.5 there instead of 2.5 or 3 or more you might see without inodes. > I've still found this number to be about right, > but not due to the standard "memory bandwidth required for the matrix > entries and column indices" performance model. I have a hard time > explaining it since the vector access is almost identical. Perhaps it > is due to poor prefetch of matrix entries? If you look at the code, for example, MatMult_SeqAIJ_Inode() you will see a couple of things that keep it from being as good as the block version. The "blocking" of column indices is only taken advantage of for the rows, not for columns. So if the block size is 4, with BAIJ there is one column index loaded per block while with Inodes there are 4. (without inodes there are 16). In additional all the "extra" column entries are still stored in the a->j array. Thus when we move to each new set of rows we skip over those entries, thus partial cache lines of a->i are constantly wasted. With BAIJ the a->i array is accessed sequentially and linearly and all entries are used. Whenever someone suggests the performance loss is possibly coming from "poor prefetch", it is clear they don't understand enough about the details of the data access, in my experience prefetch is or lack there of is purely an academic exercise and never really happens :-) Barry inodes are essentially blocking in one direction only, while BAIJ does in both. > > Jed From jed at 59A2.org Fri Jan 29 15:58:02 2010 From: jed at 59A2.org (Jed Brown) Date: Fri, 29 Jan 2010 22:58:02 +0100 Subject: [petsc-users] Question about matrix permutation In-Reply-To: <7DCA6A2F-D1F6-4D55-9C48-1E202474366B@mcs.anl.gov> References: <15232620.117881264779785465.JavaMail.root@zimbra> <87sk9oyk69.fsf@59A2.org> <7f18de3b1001290929q75304049l7bd5a98c2ce8bed0@mail.gmail.com> <7BAAA0C6-3468-4B88-94E0-DB6EFE1E3FE9@mcs.anl.gov> <87k4v0y703.fsf@59A2.org> <7DCA6A2F-D1F6-4D55-9C48-1E202474366B@mcs.anl.gov> Message-ID: <87iqaky3o5.fsf@59A2.org> On Fri, 29 Jan 2010 14:57:51 -0600, Barry Smith wrote: > This is why I have the 1.5 there instead of 2.5 or 3 or more you > might see without inodes. Okay, so suppose that the ordering is identical. With no inodes, each entry effectively costs sizeof double + sizeof int (=12). With inodes, it's sizeof double + 1/4*sizeof int (=9). With BAIJ, it's sizeof double + 1/16*sizeof int (=8.25). If the ordering is different, e.g. [u0,u1,...,v0,v1,...] as seems popular for unknown reasons, then cache reuse of the vector goes out the window and it's going to be really bad. > In additional all the "extra" column entries are still stored in the > a->j array. Thus when we move to each new set of rows we skip over > those entries, thus partial cache lines of a->i are constantly > wasted. So a nontrivial matrix with bs=4 will have over 100 nonzeros per row, thus moving to the next block involves skipping 300*sizeof int. This is more than 8 lines ahead so it's almost guaranteed to be a completely cold cache miss at all levels, which means 250 clocks to memory. During the block line, we are using around 400 doubles that also need to come all the way from memory, and cost a bit under 4 clocks each (assuming fully saturated bus). So the miss due only to stepping over these column indices could cost over 15% if everything else was running smoothly (big assumption, I know). Add to that the fact that a stream of matrix entries steps over a 4kb page boundary (which hardware prefetch doesn't cross) four times as often as with BAIJ where the entries are a single contiguous stream, and that the hardware prefetcher only follows one stream per 4kB page. So with this naive analysis, it seems possible for deficient hardware prefetch to be at fault for a nontrivial amount of the observed difference between Inode and BAIJ. Nobody's sparse mat-vecs are especially close to saturating the memory bus with useful stuff, even for matrices that are good at reusing the vector. So there must be more to the story than pure bandwidth and reuse of the vector. Jed From bsmith at mcs.anl.gov Fri Jan 29 16:52:51 2010 From: bsmith at mcs.anl.gov (Barry Smith) Date: Fri, 29 Jan 2010 16:52:51 -0600 Subject: [petsc-users] Question about matrix permutation In-Reply-To: <87iqaky3o5.fsf@59A2.org> References: <15232620.117881264779785465.JavaMail.root@zimbra> <87sk9oyk69.fsf@59A2.org> <7f18de3b1001290929q75304049l7bd5a98c2ce8bed0@mail.gmail.com> <7BAAA0C6-3468-4B88-94E0-DB6EFE1E3FE9@mcs.anl.gov> <87k4v0y703.fsf@59A2.org> <7DCA6A2F-D1F6-4D55-9C48-1E202474366B@mcs.anl.gov> <87iqaky3o5.fsf@59A2.org> Message-ID: Jed, It is possible some times to "turn hardware prefetching off", possibly with Intel compiler options. Take something like a PETSc matrix vector product compile with it turned on and turned off. See if you have a difference in flop rates. I owe you a beer if the difference is more than say 2 percent. Barry On Jan 29, 2010, at 3:58 PM, Jed Brown wrote: > On Fri, 29 Jan 2010 14:57:51 -0600, Barry Smith > wrote: >> This is why I have the 1.5 there instead of 2.5 or 3 or more you >> might see without inodes. > > Okay, so suppose that the ordering is identical. With no inodes, each > entry effectively costs sizeof double + sizeof int (=12). With > inodes, > it's sizeof double + 1/4*sizeof int (=9). With BAIJ, it's sizeof > double > + 1/16*sizeof int (=8.25). > > If the ordering is different, e.g. [u0,u1,...,v0,v1,...] as seems > popular for unknown reasons, then cache reuse of the vector goes out > the > window and it's going to be really bad. > >> In additional all the "extra" column entries are still stored in the >> a->j array. Thus when we move to each new set of rows we skip over >> those entries, thus partial cache lines of a->i are constantly >> wasted. > > So a nontrivial matrix with bs=4 will have over 100 nonzeros per row, > thus moving to the next block involves skipping 300*sizeof int. > This is > more than 8 lines ahead so it's almost guaranteed to be a completely > cold cache miss at all levels, which means 250 clocks to memory. > During > the block line, we are using around 400 doubles that also need to come > all the way from memory, and cost a bit under 4 clocks each (assuming > fully saturated bus). So the miss due only to stepping over these > column indices could cost over 15% if everything else was running > smoothly (big assumption, I know). > > Add to that the fact that a stream of matrix entries steps over a 4kb > page boundary (which hardware prefetch doesn't cross) four times as > often as with BAIJ where the entries are a single contiguous stream, > and > that the hardware prefetcher only follows one stream per 4kB page. So > with this naive analysis, it seems possible for deficient hardware > prefetch to be at fault for a nontrivial amount of the observed > difference between Inode and BAIJ. > > Nobody's sparse mat-vecs are especially close to saturating the memory > bus with useful stuff, even for matrices that are good at reusing the > vector. So there must be more to the story than pure bandwidth and > reuse of the vector. > > Jed From cmay at phys.ethz.ch Sat Jan 30 04:57:31 2010 From: cmay at phys.ethz.ch (Christian May) Date: Sat, 30 Jan 2010 11:57:31 +0100 (CET) Subject: [petsc-users] where is PCREDISTRIBUTE? Message-ID: Hi everybody, I want to use the redistribute preconditioner to keep a matrix symmetric as explained in the FAQ. However, the source is not contained in my up-to-date installation of petsc-3.0.0-p10. Is this intended? Do I have to install the development version or how can I obtain this additional preconditioner? Thanks Christian From knepley at gmail.com Sat Jan 30 08:26:37 2010 From: knepley at gmail.com (Matthew Knepley) Date: Sat, 30 Jan 2010 08:26:37 -0600 Subject: [petsc-users] where is PCREDISTRIBUTE? In-Reply-To: References: Message-ID: Yes, you will need petsc-dev. Note that we will release 3.1 in just a few weeks. Thanks, Matt On Sat, Jan 30, 2010 at 4:57 AM, Christian May wrote: > Hi everybody, > > I want to use the redistribute preconditioner to keep a matrix symmetric as > explained in the FAQ. However, the source is not contained in my up-to-date > installation of petsc-3.0.0-p10. Is this intended? > Do I have to install the development version or how can I obtain this > additional preconditioner? > > Thanks > Christian > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From jed at 59A2.org Sat Jan 30 13:38:04 2010 From: jed at 59A2.org (Jed Brown) Date: Sat, 30 Jan 2010 20:38:04 +0100 Subject: [petsc-users] Question about matrix permutation In-Reply-To: References: <15232620.117881264779785465.JavaMail.root@zimbra> <87sk9oyk69.fsf@59A2.org> <7f18de3b1001290929q75304049l7bd5a98c2ce8bed0@mail.gmail.com> <7BAAA0C6-3468-4B88-94E0-DB6EFE1E3FE9@mcs.anl.gov> <87k4v0y703.fsf@59A2.org> <7DCA6A2F-D1F6-4D55-9C48-1E202474366B@mcs.anl.gov> <87iqaky3o5.fsf@59A2.org> Message-ID: <87bpgbxu1v.fsf@59A2.org> On Fri, 29 Jan 2010 16:52:51 -0600, Barry Smith wrote: > It is possible some times to "turn hardware prefetching off", > possibly with Intel compiler options. Hmm, I had never heard of this. Apparently the instruction can only be executed in ring 0, and though the Linux kernel exposes this. resulting thing. http://stackoverflow.com/questions/784041/how-do-i-programatically-disable-hardware-prefetching Anyway, I didn't bother with this. > I owe you a beer if the difference is more than say 2 percent. Would you accept a 30 percent speedup instead of a 2 percent slowdown? Apply the attached patch, compile with GCC (I don't know if other compilers have the same __builtin_prefetch), and compare the following (top result is before the patch). ./ex19 -ksp_type cgs -pc_type none -ksp_monitor -ksp_max_it 1000 -snes_max_it 1 -da_grid_x 50 -da_grid_y 50 -log_summary MatMult 2001 1.0 5.3909e+00 1.0 3.03e+09 1.0 0.0e+00 0.0e+00 0.0e+00 41 41 0 0 0 82 81 0 0 0 563 MatMult 2001 1.0 3.9953e+00 1.0 3.03e+09 1.0 0.0e+00 0.0e+00 0.0e+00 38 41 0 0 0 77 81 0 0 0 759 ./ex19 -ksp_type cgs -pc_type none -ksp_monitor -ksp_max_it 100 -snes_max_it 1 -da_grid_x 200 -da_grid_y 200 -log_summary MatMult 201 1.0 7.9618e+00 1.0 4.98e+09 1.0 0.0e+00 0.0e+00 0.0e+00 28 38 0 0 0 60 77 0 0 0 626 MatMult 201 1.0 6.1575e+00 1.0 4.98e+09 1.0 0.0e+00 0.0e+00 0.0e+00 24 38 0 0 0 54 77 0 0 0 809 ./ex19 -ksp_type cgs -pc_type none -ksp_monitor -ksp_max_it 100 -snes_max_it 1 -da_grid_x 300 -da_grid_y 300 -log_summary MatMult 201 1.0 1.7829e+01 1.0 1.12e+10 1.0 0.0e+00 0.0e+00 0.0e+00 27 38 0 0 0 60 77 0 0 0 630 MatMult 201 1.0 1.3561e+01 1.0 1.12e+10 1.0 0.0e+00 0.0e+00 0.0e+00 24 38 0 0 0 53 77 0 0 0 828 This blows me away. Jed -------------- next part -------------- A non-text attachment was scrubbed... Name: inode-prefetch.patch Type: text/x-patch Size: 1131 bytes Desc: not available URL: From bsmith at mcs.anl.gov Sat Jan 30 14:41:03 2010 From: bsmith at mcs.anl.gov (Barry Smith) Date: Sat, 30 Jan 2010 14:41:03 -0600 Subject: [petsc-users] Question about matrix permutation In-Reply-To: <87bpgbxu1v.fsf@59A2.org> References: <15232620.117881264779785465.JavaMail.root@zimbra> <87sk9oyk69.fsf@59A2.org> <7f18de3b1001290929q75304049l7bd5a98c2ce8bed0@mail.gmail.com> <7BAAA0C6-3468-4B88-94E0-DB6EFE1E3FE9@mcs.anl.gov> <87k4v0y703.fsf@59A2.org> <7DCA6A2F-D1F6-4D55-9C48-1E202474366B@mcs.anl.gov> <87iqaky3o5.fsf@59A2.org> <87bpgbxu1v.fsf@59A2.org> Message-ID: On Jan 30, 2010, at 1:38 PM, Jed Brown wrote: > >> I owe you a beer if the difference is more than say 2 percent. > > Would you accept a 30 percent speedup instead of a 2 percent slowdown? > Apply the attached patch, compile with GCC (I don't know if other > compilers have the same __builtin_prefetch), and compare the following > (top result is before the patch). > > ./ex19 -ksp_type cgs -pc_type none -ksp_monitor -ksp_max_it 1000 - > snes_max_it 1 -da_grid_x 50 -da_grid_y 50 -log_summary > MatMult 2001 1.0 5.3909e+00 1.0 3.03e+09 1.0 0.0e+00 0.0e > +00 0.0e+00 41 41 0 0 0 82 81 0 0 0 563 > MatMult 2001 1.0 3.9953e+00 1.0 3.03e+09 1.0 0.0e+00 0.0e > +00 0.0e+00 38 41 0 0 0 77 81 0 0 0 759 > > ./ex19 -ksp_type cgs -pc_type none -ksp_monitor -ksp_max_it 100 - > snes_max_it 1 -da_grid_x 200 -da_grid_y 200 -log_summary > MatMult 201 1.0 7.9618e+00 1.0 4.98e+09 1.0 0.0e+00 0.0e > +00 0.0e+00 28 38 0 0 0 60 77 0 0 0 626 > MatMult 201 1.0 6.1575e+00 1.0 4.98e+09 1.0 0.0e+00 0.0e > +00 0.0e+00 24 38 0 0 0 54 77 0 0 0 809 > > ./ex19 -ksp_type cgs -pc_type none -ksp_monitor -ksp_max_it 100 - > snes_max_it 1 -da_grid_x 300 -da_grid_y 300 -log_summary > MatMult 201 1.0 1.7829e+01 1.0 1.12e+10 1.0 0.0e+00 0.0e > +00 0.0e+00 27 38 0 0 0 60 77 0 0 0 630 > MatMult 201 1.0 1.3561e+01 1.0 1.12e+10 1.0 0.0e+00 0.0e > +00 0.0e+00 24 38 0 0 0 53 77 0 0 0 828 > > > This blows me away. This looks great; push it to petsc-dev if it really works. BUT, this is NOT hardware prefetching, this is specifically software prefetching, you are telling it exactly what to prefetch and when. I would call it hardware prefetching only when the hardware detects a particular pattern of access and then automatically extrapolates that pattern to prefetch further along the pattern. Barry > > Jed > > diff --git a/src/mat/impls/aij/seq/inode.c b/src/mat/impls/aij/seq/ > inode.c > --- a/src/mat/impls/aij/seq/inode.c > +++ b/src/mat/impls/aij/seq/inode.c > @@ -392,6 +392,11 @@ > > /* ----------------------------------------------------------- */ > > +#define PetscPrefetchReadOnly(addr,len,loc) do > { \ > + char *__p = (char*)(addr),*__end = (char*)((addr)+ > (len)); \ > + for ( ; __p < __end; __p += 64) __builtin_prefetch(__p, > 0,loc); \ > + } while(0) > + > #undef __FUNCT__ > #define __FUNCT__ "MatMult_SeqAIJ_Inode" > static PetscErrorCode MatMult_SeqAIJ_Inode(Mat A,Vec xx,Vec yy) > @@ -423,6 +428,8 @@ > n = ii[1] - ii[0]; > nonzerorow += (n>0)*nsz; > ii += nsz; > + PetscPrefetchReadOnly(idx+nsz*n,n,0); /* Prefetch the > indices for the block row after the current one */ > + PetscPrefetchReadOnly(v1+nsz*n,nsz*n,0); /* Prefetch the values > for the block row after the current one */ > sz = n; /* No of non zeros in this row */ > /* Switch on the size of Node */ > switch (nsz){ /* Each loop in 'case' is unrolled */ From cmay at phys.ethz.ch Sun Jan 31 07:10:09 2010 From: cmay at phys.ethz.ch (Christian May) Date: Sun, 31 Jan 2010 14:10:09 +0100 (CET) Subject: [petsc-users] where is PCREDISTRIBUTE? In-Reply-To: References: Message-ID: Thanks, Matt. The preconditioner is now available, but when I invoke it with -ksp_type preonly -pc_type redistribute -redistribute_ksp_type cg -redistribute_pc_type bjacobi -redistribute_sub_pc_type icc I get the following error: [0]PETSC ERROR: --------------------- Error Message ------------------------------------ [0]PETSC ERROR: Invalid argument! [0]PETSC ERROR: Wrong type of object: Parameter # 5! [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: Petsc Development HG revision: cec5db6729c23bbdb9cc80092ac1dc4ccc7e9069 HG Date: Fri Jan 29 22:04:34 2010 -0600 [0]PETSC ERROR: See docs/changes/index.html for recent updates. [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. [0]PETSC ERROR: See docs/index.html for manual pages. [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: ./SPsolver2D-petsc on a linux-gnu named a6256.hpc-net.ethz.ch by cmay Sun Jan 31 11:10:52 2010 [0]PETSC ERROR: Libraries linked from /cluster/work/phys/cmay/petsc/lib [0]PETSC ERROR: Configure run at Sat Jan 30 11:48:43 2010 [0]PETSC ERROR: Configure options --prefix=/cluster/work/phys/cmay/petsc/ --with-c++-support --with-precision=double --with-sha red=0 --download-superlu_dist=1 --with-superlu_dist --with-parmetis-lib="[/cluster/work/phys/cmay/petsc/petsc-3.0.0-p3/external packages/ParMetis-3.1.1/libparmetis.a,/cluster/work/phys/cmay/petsc/petsc-3.0.0-p3/externalpackages/ParMetis-3.1.1/libmetis.a]" --with-parmetis-include=/cluster/work/phys/cmay/petsc/petsc-3.0.0-p3/externalpackages/ParMetis-3.1.1/ --with-x=0 --with-mpi-di r=/cluster/apps/openmpi/1.2.7/x86_64/intel/ --with-fc=0 --download-c-blas-lapack=yes [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: MatGetSubMatrix() line 6519 in src/mat/interface/matrix.c Thanks Christian >Yes, you will need petsc-dev. Note that we will release 3.1 in just a few >weeks. > > Thanks, > > Matt > >On Sat, Jan 30, 2010 at 4:57 AM, Christian May wrote: > >> Hi everybody, >> >> I want to use the redistribute preconditioner to keep a matrix symmetric as >> explained in the FAQ. However, the source is not contained in my up-to-date >> installation of petsc-3.0.0-p10. Is this intended? >> Do I have to install the development version or how can I obtain this >> additional preconditioner? >> >> Thanks >> Christian >> >> > > From jed at 59A2.org Sun Jan 31 15:26:45 2010 From: jed at 59A2.org (Jed Brown) Date: Sun, 31 Jan 2010 22:26:45 +0100 Subject: [petsc-users] Question about matrix permutation In-Reply-To: References: <15232620.117881264779785465.JavaMail.root@zimbra> <87sk9oyk69.fsf@59A2.org> <7f18de3b1001290929q75304049l7bd5a98c2ce8bed0@mail.gmail.com> <7BAAA0C6-3468-4B88-94E0-DB6EFE1E3FE9@mcs.anl.gov> <87k4v0y703.fsf@59A2.org> <7DCA6A2F-D1F6-4D55-9C48-1E202474366B@mcs.anl.gov> <87iqaky3o5.fsf@59A2.org> <87bpgbxu1v.fsf@59A2.org> Message-ID: <877hqyx8x6.fsf@59A2.org> On Sat, 30 Jan 2010 14:41:03 -0600, Barry Smith wrote: > This looks great; push it to petsc-dev if it really works. I will, but it needs to be cleaned up a bit before it can be pushed (e.g. so that it works with other compilers). > BUT, this is NOT hardware prefetching, this is specifically > software prefetching, you are telling it exactly what to prefetch and > when. Absolutely, but my original statement was it seems possible for deficient hardware prefetch to be at fault for a nontrivial amount of the observed difference between Inode and BAIJ. I never claimed that hardware prefetch was doing anything useful (although the tests below show that it is), but that prefetch was the reason why Inode was not as fast as our standard performance model would indicate. > I would call it hardware prefetching only when the hardware detects a > particular pattern of access and then automatically extrapolates that > pattern to prefetch further along the pattern. Right, and that extrapolation was not very good, for the reasons described earlier in this thread. But, to investigate how much good hardware prefetch is in this case, I ran the following tests. First I disabled hardware prefetch. This is poorly documented on the web, so here's the scoop: With Core2 on Linux, you install msr-tools and check register 0x1a0. # rdmsr -p 1 0x1a0 1364970489 The details are processor dependent, but on both 32 and 64-bit intel chips, you need to flip bit 9 to disable hardware prefetch. # wrmsr -p 1 0x1a0 0x1364970689 to turn it on again, just disable that bit # wrmsr -p 1 0x1a0 0x1364970489 Note that this all needs to be run as root. For details on MSR and which bits to use for chips other than Core 2, see Appendix B of Intel? 64 and IA-32 Architectures Software Developer's Manual Volume 3B: System Programming Guide All the manuals are here http://www.intel.com/products/processor/manuals/ I then ran the benchmarks using taskset to bind to the correct core and nice -n -10 to ensure that mostly-idle background processes would always run on the inactive core. I checked that the results were consistent regardless of which core was being used, and ran each test twice. All of these are *without* the software prefetch patch. It appears that the difference is comfortably over 2 percent in all cases. ./ex19 -ksp_type cgs -pc_type none -ksp_monitor -ksp_max_it 1000 -snes_max_it 1 -da_grid_x 50 -da_grid_y 50 -log_summary on: MatMult 2001 1.0 5.3418e+00 1.0 3.03e+09 1.0 0.0e+00 0.0e+00 0.0e+00 41 41 0 0 0 82 81 0 0 0 568 MatMult 2001 1.0 5.3506e+00 1.0 3.03e+09 1.0 0.0e+00 0.0e+00 0.0e+00 41 41 0 0 0 82 81 0 0 0 567 off: MatMult 2001 1.0 5.9154e+00 1.0 3.03e+09 1.0 0.0e+00 0.0e+00 0.0e+00 37 41 0 0 0 76 81 0 0 0 513 MatMult 2001 1.0 5.9271e+00 1.0 3.03e+09 1.0 0.0e+00 0.0e+00 0.0e+00 37 41 0 0 0 76 81 0 0 0 512 ./ex19 -ksp_type cgs -pc_type none -ksp_monitor -ksp_max_it 100 -snes_max_it 1 -da_grid_x 200 -da_grid_y 200 -log_summary on: MatMult 201 1.0 7.9634e+00 1.0 4.98e+09 1.0 0.0e+00 0.0e+00 0.0e+00 28 38 0 0 0 61 77 0 0 0 626 MatMult 201 1.0 7.9570e+00 1.0 4.98e+09 1.0 0.0e+00 0.0e+00 0.0e+00 28 38 0 0 0 61 77 0 0 0 626 off: MatMult 201 1.0 9.2377e+00 1.0 4.98e+09 1.0 0.0e+00 0.0e+00 0.0e+00 27 38 0 0 0 59 77 0 0 0 539 MatMult 201 1.0 9.2294e+00 1.0 4.98e+09 1.0 0.0e+00 0.0e+00 0.0e+00 27 38 0 0 0 59 77 0 0 0 540 ./ex19 -ksp_type cgs -pc_type none -ksp_monitor -ksp_max_it 100 -snes_max_it 1 -da_grid_x 200 -da_grid_y 200 -log_summary on: MatMult 201 1.0 1.7588e+01 1.0 1.12e+10 1.0 0.0e+00 0.0e+00 0.0e+00 27 38 0 0 0 59 77 0 0 0 639 MatMult 201 1.0 1.7642e+01 1.0 1.12e+10 1.0 0.0e+00 0.0e+00 0.0e+00 27 38 0 0 0 60 77 0 0 0 637 off: MatMult 201 1.0 2.0519e+01 1.0 1.12e+10 1.0 0.0e+00 0.0e+00 0.0e+00 26 38 0 0 0 58 77 0 0 0 548 MatMult 201 1.0 2.0601e+01 1.0 1.12e+10 1.0 0.0e+00 0.0e+00 0.0e+00 26 38 0 0 0 58 77 0 0 0 545 Then I did one comparison *with* the software prefetch patch: ./ex19 -ksp_type cgs -pc_type none -ksp_monitor -ksp_max_it 100 -snes_max_it 1 -da_grid_x 200 -da_grid_y 200 -log_summary on: MatMult 201 1.0 6.1305e+00 1.0 4.98e+09 1.0 0.0e+00 0.0e+00 0.0e+00 24 38 0 0 0 54 77 0 0 0 813 MatMult 201 1.0 6.1379e+00 1.0 4.98e+09 1.0 0.0e+00 0.0e+00 0.0e+00 24 38 0 0 0 54 77 0 0 0 812 off: MatMult 201 1.0 7.8732e+00 1.0 4.98e+09 1.0 0.0e+00 0.0e+00 0.0e+00 25 38 0 0 0 55 77 0 0 0 633 MatMult 201 1.0 7.8841e+00 1.0 4.98e+09 1.0 0.0e+00 0.0e+00 0.0e+00 25 38 0 0 0 55 77 0 0 0 632 So the performance loss from disabling hardware prefetch is *greater* when using explicit software prefetch than when no software prefetch is done. Finally, I reverted the software prefetch patch and disabled DCU prefetch (bit 37). I'm not clear on the details of the difference between this and the streaming prefetch that the tests above use, but the manual says The DCU prefetcher is an L1 data cache prefetcher. When the DCU prefetcher detects multiple loads from the same line done within a time limit, the DCU prefetcher assumes the next line will be required. The next line is prefetched in to the L1 data cache from memory or L2. Compare these numbers to the ones above (~638 flop/s with both on, ~547 with stream prefetch off). ./ex19 -ksp_type cgs -pc_type none -ksp_monitor -ksp_max_it 100 -snes_max_it 1 -da_grid_x 200 -da_grid_y 200 -log_summary dcu-off: MatMult 201 1.0 8.2517e+00 1.0 4.98e+09 1.0 0.0e+00 0.0e+00 0.0e+00 28 38 0 0 0 61 77 0 0 0 604 both-off: MatMult 201 1.0 9.6107e+00 1.0 4.98e+09 1.0 0.0e+00 0.0e+00 0.0e+00 27 38 0 0 0 59 77 0 0 0 518 On my system, these were achieved with # wrmsr -p 1 0x1a0 0x3364970489 # stream on, DCU off # wrmsr -p 1 0x1a0 0x3364970689 # stream off, DCU off My system details: $ uname -a Linux kunyang 2.6.32-ARCH #1 SMP PREEMPT Mon Jan 25 20:33:50 CET 2010 x86_64 Intel(R) Core(TM)2 Duo CPU P8700 @ 2.53GHz GenuineIntel GNU/Linux $ gcc -v Using built-in specs. Target: x86_64-unknown-linux-gnu Configured with: ../configure --prefix=/usr --enable-shared --enable-languages=c,c++,fortran,objc,obj-c++,ada --enable-threads=posix --mandir=/usr/share/man --infodir=/usr/share/info --enable-__cxa_atexit --disable-multilib --libdir=/usr/lib --libexecdir=/usr/lib --enable-clocale=gnu --disable-libstdcxx-pch --with-tune=generic Thread model: posix gcc version 4.4.3 (GCC) $ cat /proc/cpuinfo processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 23 model name : Intel(R) Core(TM)2 Duo CPU P8700 @ 2.53GHz stepping : 10 cpu MHz : 2534.000 cache size : 3072 KB physical id : 0 siblings : 2 core id : 0 cpu cores : 2 apicid : 0 initial apicid : 0 fpu : yes fpu_exception : yes cpuid level : 13 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good aperfmperf pni dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm sse4_1 xsave lahf_lm ida tpr_shadow vnmi flexpriority bogomips : 5057.73 clflush size : 64 cache_alignment : 64 address sizes : 36 bits physical, 48 bits virtual power management: processor : 1 vendor_id : GenuineIntel cpu family : 6 model : 23 model name : Intel(R) Core(TM)2 Duo CPU P8700 @ 2.53GHz stepping : 10 cpu MHz : 2534.000 cache size : 3072 KB physical id : 0 siblings : 2 core id : 1 cpu cores : 2 apicid : 1 initial apicid : 1 fpu : yes fpu_exception : yes cpuid level : 13 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good aperfmperf pni dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm sse4_1 xsave lahf_lm ida tpr_shadow vnmi flexpriority bogomips : 5059.12 clflush size : 64 cache_alignment : 64 address sizes : 36 bits physical, 48 bits virtual power management: Jed From bsmith at mcs.anl.gov Sun Jan 31 15:52:03 2010 From: bsmith at mcs.anl.gov (Barry Smith) Date: Sun, 31 Jan 2010 15:52:03 -0600 Subject: [petsc-users] where is PCREDISTRIBUTE? In-Reply-To: References: Message-ID: <75D841B7-7039-4A48-9214-AF37CDDBE593@mcs.anl.gov> Sorry, there was an error in the case when the preconditioner was rebuilt with the same nonzero pattern, an internal matrix was mistakenly freed early. Drop the attached src/ksp/ps/impls/redistribute.c and run make in that directory and it should run correctly. Barry On Jan 31, 2010, at 7:10 AM, Christian May wrote: > Thanks, Matt. The preconditioner is now available, but when I invoke > it with > > -ksp_type preonly -pc_type redistribute -redistribute_ksp_type cg - > redistribute_pc_type bjacobi -redistribute_sub_pc_type icc > > I get the following error: > > [0]PETSC ERROR: --------------------- Error Message > ------------------------------------ > [0]PETSC ERROR: Invalid argument! > [0]PETSC ERROR: Wrong type of object: Parameter # 5! > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: Petsc Development HG revision: > cec5db6729c23bbdb9cc80092ac1dc4ccc7e9069 HG Date: Fri Jan 29 > 22:04:34 2010 -0600 > [0]PETSC ERROR: See docs/changes/index.html for recent updates. > [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. > [0]PETSC ERROR: See docs/index.html for manual pages. > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: ./SPsolver2D-petsc on a linux-gnu named a6256.hpc- > net.ethz.ch by cmay Sun Jan 31 11:10:52 2010 > [0]PETSC ERROR: Libraries linked from /cluster/work/phys/cmay/petsc/ > lib > [0]PETSC ERROR: Configure run at Sat Jan 30 11:48:43 2010 > [0]PETSC ERROR: Configure options --prefix=/cluster/work/phys/cmay/ > petsc/ --with-c++-support --with-precision=double --with-sha > red=0 --download-superlu_dist=1 --with-superlu_dist --with-parmetis- > lib="[/cluster/work/phys/cmay/petsc/petsc-3.0.0-p3/external > packages/ParMetis-3.1.1/libparmetis.a,/cluster/work/phys/cmay/petsc/ > petsc-3.0.0-p3/externalpackages/ParMetis-3.1.1/libmetis.a]" > --with-parmetis-include=/cluster/work/phys/cmay/petsc/petsc-3.0.0-p3/ > externalpackages/ParMetis-3.1.1/ --with-x=0 --with-mpi-di > r=/cluster/apps/openmpi/1.2.7/x86_64/intel/ --with-fc=0 --download-c- > blas-lapack=yes > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: MatGetSubMatrix() line 6519 in src/mat/interface/ > matrix.c > > > Thanks > Christian > > >> Yes, you will need petsc-dev. Note that we will release 3.1 in just >> a few >> weeks. >> >> Thanks, >> >> Matt >> >> On Sat, Jan 30, 2010 at 4:57 AM, Christian May > wrote: >> >>> Hi everybody, >>> >>> I want to use the redistribute preconditioner to keep a matrix > symmetric as >>> explained in the FAQ. However, the source is not contained in my > up-to-date >>> installation of petsc-3.0.0-p10. Is this intended? >>> Do I have to install the development version or how can I obtain >>> this >>> additional preconditioner? >>> >>> Thanks >>> Christian >>> >>> >> >> -------------- next part -------------- A non-text attachment was scrubbed... Name: redistribute.c Type: application/octet-stream Size: 14112 bytes Desc: not available URL: From bsmith at mcs.anl.gov Sun Jan 31 16:02:17 2010 From: bsmith at mcs.anl.gov (Barry Smith) Date: Sun, 31 Jan 2010 16:02:17 -0600 Subject: [petsc-users] Question about matrix permutation In-Reply-To: <877hqyx8x6.fsf@59A2.org> References: <15232620.117881264779785465.JavaMail.root@zimbra> <87sk9oyk69.fsf@59A2.org> <7f18de3b1001290929q75304049l7bd5a98c2ce8bed0@mail.gmail.com> <7BAAA0C6-3468-4B88-94E0-DB6EFE1E3FE9@mcs.anl.gov> <87k4v0y703.fsf@59A2.org> <7DCA6A2F-D1F6-4D55-9C48-1E202474366B@mcs.anl.gov> <87iqaky3o5.fsf@59A2.org> <87bpgbxu1v.fsf@59A2.org> <877hqyx8x6.fsf@59A2.org> Message-ID: <092174A2-41CB-4087-8847-1C341B694DF6@mcs.anl.gov> Ok, I owe you! > This looks great; push it to petsc-dev if it really works. I will, but it needs to be cleaned up a bit before it can be pushed (e.g. so that it works with other compilers). Check config/PETSc/Configure.py configurePrefix() for the current support in PETSc. It is used, for example, in src/mat/impls/sbaij/seq/ relax.h; you may have found better ways of doing this so feel free to change the current support if you have something better. Barry On Jan 31, 2010, at 3:26 PM, Jed Brown wrote: > On Sat, 30 Jan 2010 14:41:03 -0600, Barry Smith > wrote: >> This looks great; push it to petsc-dev if it really works. > > I will, but it needs to be cleaned up a bit before it can be pushed > (e.g. so that it works with other compilers). > >> BUT, this is NOT hardware prefetching, this is specifically >> software prefetching, you are telling it exactly what to prefetch and >> when. > > Absolutely, but my original statement was > > it seems possible for deficient hardware prefetch to be at fault > for a > nontrivial amount of the observed difference between Inode and BAIJ. > > I never claimed that hardware prefetch was doing anything useful > (although the tests below show that it is), but that prefetch was the > reason why Inode was not as fast as our standard performance model > would indicate. > >> I would call it hardware prefetching only when the hardware detects a >> particular pattern of access and then automatically extrapolates that >> pattern to prefetch further along the pattern. > > Right, and that extrapolation was not very good, for the reasons > described earlier in this thread. But, to investigate how much good > hardware prefetch is in this case, I ran the following tests. > > First I disabled hardware prefetch. This is poorly documented on the > web, so here's the scoop: > > With Core2 on Linux, you install msr-tools and check register 0x1a0. > > # rdmsr -p 1 0x1a0 > 1364970489 > > The details are processor dependent, but on both 32 and 64-bit intel > chips, you need to flip bit 9 to disable hardware prefetch. > > # wrmsr -p 1 0x1a0 0x1364970689 > > to turn it on again, just disable that bit > > # wrmsr -p 1 0x1a0 0x1364970489 > > Note that this all needs to be run as root. For details on MSR and > which bits to use for chips other than Core 2, see Appendix B of > > Intel? 64 and IA-32 Architectures Software Developer's Manual > Volume 3B: System Programming Guide > > All the manuals are here > > http://www.intel.com/products/processor/manuals/ > > > I then ran the benchmarks using taskset to bind to the correct core > and > nice -n -10 to ensure that mostly-idle background processes would > always > run on the inactive core. I checked that the results were consistent > regardless of which core was being used, and ran each test twice. All > of these are *without* the software prefetch patch. It appears that > the > difference is comfortably over 2 percent in all cases. > > ./ex19 -ksp_type cgs -pc_type none -ksp_monitor -ksp_max_it 1000 - > snes_max_it 1 -da_grid_x 50 -da_grid_y 50 -log_summary > on: MatMult 2001 1.0 5.3418e+00 1.0 3.03e+09 1.0 0.0e > +00 0.0e+00 0.0e+00 41 41 0 0 0 82 81 0 0 0 568 > MatMult 2001 1.0 5.3506e+00 1.0 3.03e+09 1.0 0.0e+00 > 0.0e+00 0.0e+00 41 41 0 0 0 82 81 0 0 0 567 > off: MatMult 2001 1.0 5.9154e+00 1.0 3.03e+09 1.0 0.0e > +00 0.0e+00 0.0e+00 37 41 0 0 0 76 81 0 0 0 513 > MatMult 2001 1.0 5.9271e+00 1.0 3.03e+09 1.0 0.0e+00 > 0.0e+00 0.0e+00 37 41 0 0 0 76 81 0 0 0 512 > > ./ex19 -ksp_type cgs -pc_type none -ksp_monitor -ksp_max_it 100 - > snes_max_it 1 -da_grid_x 200 -da_grid_y 200 -log_summary > on: MatMult 201 1.0 7.9634e+00 1.0 4.98e+09 1.0 0.0e > +00 0.0e+00 0.0e+00 28 38 0 0 0 61 77 0 0 0 626 > MatMult 201 1.0 7.9570e+00 1.0 4.98e+09 1.0 0.0e+00 > 0.0e+00 0.0e+00 28 38 0 0 0 61 77 0 0 0 626 > off: MatMult 201 1.0 9.2377e+00 1.0 4.98e+09 1.0 0.0e > +00 0.0e+00 0.0e+00 27 38 0 0 0 59 77 0 0 0 539 > MatMult 201 1.0 9.2294e+00 1.0 4.98e+09 1.0 0.0e+00 > 0.0e+00 0.0e+00 27 38 0 0 0 59 77 0 0 0 540 > > ./ex19 -ksp_type cgs -pc_type none -ksp_monitor -ksp_max_it 100 - > snes_max_it 1 -da_grid_x 200 -da_grid_y 200 -log_summary > on: MatMult 201 1.0 1.7588e+01 1.0 1.12e+10 1.0 0.0e > +00 0.0e+00 0.0e+00 27 38 0 0 0 59 77 0 0 0 639 > MatMult 201 1.0 1.7642e+01 1.0 1.12e+10 1.0 0.0e+00 > 0.0e+00 0.0e+00 27 38 0 0 0 60 77 0 0 0 637 > off: MatMult 201 1.0 2.0519e+01 1.0 1.12e+10 1.0 0.0e > +00 0.0e+00 0.0e+00 26 38 0 0 0 58 77 0 0 0 548 > MatMult 201 1.0 2.0601e+01 1.0 1.12e+10 1.0 0.0e+00 > 0.0e+00 0.0e+00 26 38 0 0 0 58 77 0 0 0 545 > > > Then I did one comparison *with* the software prefetch patch: > > ./ex19 -ksp_type cgs -pc_type none -ksp_monitor -ksp_max_it 100 - > snes_max_it 1 -da_grid_x 200 -da_grid_y 200 -log_summary > on: MatMult 201 1.0 6.1305e+00 1.0 4.98e+09 1.0 0.0e > +00 0.0e+00 0.0e+00 24 38 0 0 0 54 77 0 0 0 813 > MatMult 201 1.0 6.1379e+00 1.0 4.98e+09 1.0 0.0e+00 > 0.0e+00 0.0e+00 24 38 0 0 0 54 77 0 0 0 812 > off: MatMult 201 1.0 7.8732e+00 1.0 4.98e+09 1.0 0.0e > +00 0.0e+00 0.0e+00 25 38 0 0 0 55 77 0 0 0 633 > MatMult 201 1.0 7.8841e+00 1.0 4.98e+09 1.0 0.0e+00 > 0.0e+00 0.0e+00 25 38 0 0 0 55 77 0 0 0 632 > > So the performance loss from disabling hardware prefetch is *greater* > when using explicit software prefetch than when no software prefetch > is > done. > > > Finally, I reverted the software prefetch patch and disabled DCU > prefetch (bit 37). I'm not clear on the details of the difference > between this and the streaming prefetch that the tests above use, but > the manual says > > The DCU prefetcher is an L1 data cache prefetcher. When the DCU > prefetcher detects multiple loads from the same line done within a > time limit, the DCU prefetcher assumes the next line will be > required. > The next line is prefetched in to the L1 data cache from memory or > L2. > > Compare these numbers to the ones above (~638 flop/s with both on, > ~547 > with stream prefetch off). > > ./ex19 -ksp_type cgs -pc_type none -ksp_monitor -ksp_max_it 100 - > snes_max_it 1 -da_grid_x 200 -da_grid_y 200 -log_summary > dcu-off: MatMult 201 1.0 8.2517e+00 1.0 4.98e+09 1.0 > 0.0e+00 0.0e+00 0.0e+00 28 38 0 0 0 61 77 0 0 0 604 > both-off: MatMult 201 1.0 9.6107e+00 1.0 4.98e+09 1.0 > 0.0e+00 0.0e+00 0.0e+00 27 38 0 0 0 59 77 0 0 0 518 > > On my system, these were achieved with > > # wrmsr -p 1 0x1a0 0x3364970489 # stream on, DCU off > # wrmsr -p 1 0x1a0 0x3364970689 # stream off, DCU off > > > My system details: > > $ uname -a > Linux kunyang 2.6.32-ARCH #1 SMP PREEMPT Mon Jan 25 20:33:50 CET > 2010 x86_64 Intel(R) Core(TM)2 Duo CPU P8700 @ 2.53GHz GenuineIntel > GNU/Linux > $ gcc -v > Using built-in specs. > Target: x86_64-unknown-linux-gnu > Configured with: ../configure --prefix=/usr --enable-shared -- > enable-languages=c,c++,fortran,objc,obj-c++,ada --enable- > threads=posix --mandir=/usr/share/man --infodir=/usr/share/info -- > enable-__cxa_atexit --disable-multilib --libdir=/usr/lib -- > libexecdir=/usr/lib --enable-clocale=gnu --disable-libstdcxx-pch -- > with-tune=generic > Thread model: posix > gcc version 4.4.3 (GCC) > $ cat /proc/cpuinfo > processor : 0 > vendor_id : GenuineIntel > cpu family : 6 > model : 23 > model name : Intel(R) Core(TM)2 Duo CPU P8700 @ 2.53GHz > stepping : 10 > cpu MHz : 2534.000 > cache size : 3072 KB > physical id : 0 > siblings : 2 > core id : 0 > cpu cores : 2 > apicid : 0 > initial apicid : 0 > fpu : yes > fpu_exception : yes > cpuid level : 13 > wp : yes > flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr > pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm > pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good > aperfmperf pni dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr > pdcm sse4_1 xsave lahf_lm ida tpr_shadow vnmi flexpriority > bogomips : 5057.73 > clflush size : 64 > cache_alignment : 64 > address sizes : 36 bits physical, 48 bits virtual > power management: > > processor : 1 > vendor_id : GenuineIntel > cpu family : 6 > model : 23 > model name : Intel(R) Core(TM)2 Duo CPU P8700 @ 2.53GHz > stepping : 10 > cpu MHz : 2534.000 > cache size : 3072 KB > physical id : 0 > siblings : 2 > core id : 1 > cpu cores : 2 > apicid : 1 > initial apicid : 1 > fpu : yes > fpu_exception : yes > cpuid level : 13 > wp : yes > flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr > pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm > pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good > aperfmperf pni dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr > pdcm sse4_1 xsave lahf_lm ida tpr_shadow vnmi flexpriority > bogomips : 5059.12 > clflush size : 64 > cache_alignment : 64 > address sizes : 36 bits physical, 48 bits virtual > power management: > > > Jed From jed at 59A2.org Sun Jan 31 16:44:38 2010 From: jed at 59A2.org (Jed Brown) Date: Sun, 31 Jan 2010 23:44:38 +0100 Subject: [petsc-users] Question about matrix permutation In-Reply-To: <092174A2-41CB-4087-8847-1C341B694DF6@mcs.anl.gov> References: <15232620.117881264779785465.JavaMail.root@zimbra> <87sk9oyk69.fsf@59A2.org> <7f18de3b1001290929q75304049l7bd5a98c2ce8bed0@mail.gmail.com> <7BAAA0C6-3468-4B88-94E0-DB6EFE1E3FE9@mcs.anl.gov> <87k4v0y703.fsf@59A2.org> <7DCA6A2F-D1F6-4D55-9C48-1E202474366B@mcs.anl.gov> <87iqaky3o5.fsf@59A2.org> <87bpgbxu1v.fsf@59A2.org> <877hqyx8x6.fsf@59A2.org> <092174A2-41CB-4087-8847-1C341B694DF6@mcs.anl.gov> Message-ID: <87636hyjvt.fsf@59A2.org> On Sun, 31 Jan 2010 16:02:17 -0600, Barry Smith wrote: > Check config/PETSc/Configure.py configurePrefix() for the current > support in PETSc. It is used, for example, in src/mat/impls/sbaij/seq/ > relax.h; you may have found better ways of doing this so feel free to > change the current support if you have something better. Aha, but PETSC_Prefetch (naming violation?) just wraps __builtin_prefetch or _mm_prefetch, which means it's only fetching one cache line of the preceding row. From what I can tell, it's bad to software prefetch part of a row and rely on hardware prefetch to pick up the rest. If you're using software prefetch for a particular access pattern, you want to ask for exactly what you need, which normally means calling prefetch more than once. The Intel optimization manual says that you should overlap the prefetch calls with computation (because the prefetch instruction occupies the same execution unit as loads) but since we're so far beyond actually being CPU bound, I don't think there is a real penalty to issuing several prefetch instructions at once and then going to work on the block that should already be in cache while the prefetch results trickle in. Also, I'm pretty sure we want to be using _MM_HINT_NTA (0) instead of _MM_HINT_T2 (1) since the latter brings the values into all levels of cache. Since it's the next row that we'll work with, it's very rare (i.e. only matrices with an absurd number of nonzeros per row) that the row would actually be evicted from L1 before we get to it. We don't want to pollute the higher levels because we want as much as possible for the vector. (At least, using _MM_HINT_T2 took several percent off the MatMult flops, and I would anticipate the same effect for MatSolve and MatSOR.) Do we currently detect the cache line size anywhere? I don't know how to do that kind of thing portably, though this claims to do it http://www.open-mpi.org/projects/hwloc/ Jed From bsmith at mcs.anl.gov Sun Jan 31 21:34:21 2010 From: bsmith at mcs.anl.gov (Barry Smith) Date: Sun, 31 Jan 2010 21:34:21 -0600 Subject: [petsc-users] Question about matrix permutation In-Reply-To: <87636hyjvt.fsf@59A2.org> References: <15232620.117881264779785465.JavaMail.root@zimbra> <87sk9oyk69.fsf@59A2.org> <7f18de3b1001290929q75304049l7bd5a98c2ce8bed0@mail.gmail.com> <7BAAA0C6-3468-4B88-94E0-DB6EFE1E3FE9@mcs.anl.gov> <87k4v0y703.fsf@59A2.org> <7DCA6A2F-D1F6-4D55-9C48-1E202474366B@mcs.anl.gov> <87iqaky3o5.fsf@59A2.org> <87bpgbxu1v.fsf@59A2.org> <877hqyx8x6.fsf@59A2.org> <092174A2-41CB-4087-8847-1C341B694DF6@mcs.anl.gov> <87636hyjvt.fsf@59A2.org> Message-ID: <903BF008-CC9F-4972-B07B-5966BC25C6F8@mcs.anl.gov> Move this to petsc-dev at mcs.anl.gov On Jan 31, 2010, at 4:44 PM, Jed Brown wrote: > On Sun, 31 Jan 2010 16:02:17 -0600, Barry Smith > wrote: >> Check config/PETSc/Configure.py configurePrefix() for the current >> support in PETSc. It is used, for example, in src/mat/impls/sbaij/ >> seq/ >> relax.h; you may have found better ways of doing this so feel free to >> change the current support if you have something better. > > Aha, but PETSC_Prefetch (naming violation?) Yes, it uses the config addDefine() method that prepends the silly PETSC_, I was to lazy to figure out a nicer way to do it. > just wraps > __builtin_prefetch or _mm_prefetch, which means it's only fetching one > cache line of the preceding row. From what I can tell, it's bad to > software prefetch part of a row and rely on hardware prefetch to > pick up > the rest. If you're using software prefetch for a particular access > pattern, you want to ask for exactly what you need, which normally > means > calling prefetch more than once. The Intel optimization manual says > that you should overlap the prefetch calls with computation (because > the > prefetch instruction occupies the same execution unit as loads) but > since we're so far beyond actually being CPU bound, I don't think > there > is a real penalty to issuing several prefetch instructions at once and > then going to work on the block that should already be in cache while > the prefetch results trickle in. Just doing that one gave me a surprising large jump in performance so I didn't purse it further. As you may have guessed I'm pretty pessimistic about the whole business :-( > > Also, I'm pretty sure we want to be using _MM_HINT_NTA (0) instead of > _MM_HINT_T2 (1) since the latter brings the values into all levels of > cache. Since it's the next row that we'll work with, it's very rare > (i.e. only matrices with an absurd number of nonzeros per row) that > the > row would actually be evicted from L1 before we get to it. We don't > want to pollute the higher levels because we want as much as possible > for the vector. (At least, using _MM_HINT_T2 took several percent off > the MatMult flops, and I would anticipate the same effect for MatSolve > and MatSOR.) I didn't have a clue what to use for this option. > > Do we currently detect the cache line size anywhere? Nope, another job for ./configure Barry > I don't know how > to do that kind of thing portably, though this claims to do it > > http://www.open-mpi.org/projects/hwloc/ > > > Jed