From tyoung at ippt.gov.pl Tue Sep 4 07:08:45 2007 From: tyoung at ippt.gov.pl (Toby D. Young) Date: Tue, 4 Sep 2007 14:08:45 +0200 Subject: MatZeroColsIS question. Message-ID: <20070904140845.3369ea22@rav.ippt.gov.pl> Hello, Being relatively new to Petsc I wanted to ask other users if this is an appropriate approach to a problem of applying boundary conditions to a matrix. I am trying to apply Dirichlet boundary conditions on a PetSc matrix by zeroing out the rows and columns in the usual way. The reason to zero out the columns is to preserve the symmetrey of the matrix. I know the row numbers I need to zero out. While zeroing out rows is simple using MatZeroRowsIS() there does not seem to be a synomonous MatZeroColsIS(). The problem I have here is to construct one from the other routines available in Petsc. In that case I hope to get the nonzero column indices of a specific row and then to forcively set these nonzero column entries to zero. I am assuming this is possible using MatGetRow(matrix, row, ncols, colnums, values); where "colnums" will give me the column numbers of nonzero matrix elements for that row, if I understand this routine correctly?? :-) followed by MatSetValues (matrix, 1, row, 1, colnums[j], 0, INSERT_VALUES); and loop for all rows that were zeroed out in the previous step. Although this is a rather inefficient I hope it should work. Has anyone concieved of or could suggest a more effective approach for constructing a MatZeroColsIS? I welcome any response / advice and thank you in advance. Best, Toby I am using c++ (gcc 4.1.0) and petsc 2.3.0. -- Toby D. Young ( Adiunkt ) Department of Computational Science Institute of Fundamental Technological Research, Polish Academy of Sciences, Room 206, Swietokrzyska 21, 00-049 Warsaw, POLAND From ps at cs.caltech.edu Tue Sep 4 08:01:50 2007 From: ps at cs.caltech.edu (=?ISO-8859-1?Q?Peter_Schr=F6der?=) Date: Tue, 04 Sep 2007 06:01:50 -0700 Subject: MatZeroColsIS question. In-Reply-To: <20070904140845.3369ea22@rav.ippt.gov.pl> References: <20070904140845.3369ea22@rav.ippt.gov.pl> Message-ID: <46DD573E.10303@cs.caltech.edu> Why not just construct the matrix without the corresponding columns and rows? That preserves the symmetry (as it should of course), gives you a smaller matrix, and you don't have the hassle with the zero columns. Peter From gtg100n at mail.gatech.edu Tue Sep 4 09:18:57 2007 From: gtg100n at mail.gatech.edu (Alejandro Garzon) Date: Tue, 04 Sep 2007 10:18:57 -0400 Subject: incomplete LU with drop tolerance Message-ID: <1188915537.46dd695174188@webmail.mail.gatech.edu> Hi, I'm trying to find out what is the performance advantage of Petsc over Matlab when solving a particular linear system. For the comparison to be meaningful I have to use the exact same linear solver and preconditioner in both Petsc and Matlab. The one problem I have found in this attempt is that I haven't been able to identify which options should I give to my Petsc code so that it uses the same preconditioner as Matlab's function "luinc" when used in the syntax "[L,U]=luinc(A,droptol)". In the Matlab documentation it is said that in this syntax the function uses incomplete LU factorization with drop tolerance. A similar preconditioner is available in Petsc through the option "-pc_factor_use_drop_tolerance
,, ". To see if they were exactly the same I tried to know about the internal working of each preconditioner. In the manual page of the Petsc function "PCFactorSetUseDropTolerance" (that sets the same preconditioner as the option above) it is said that this preconditioner uses the function "iludt" from SPARSKIT. I downloaded SPARSKIT from www-users.cs.umn.edu/~saad/software/SPARSKIT/sparskit.html and when searching through the source code I couldn't find any function with that exact name. My fist question is if the function's name is not misspelled in the Petsc documentation both the manual page and the Petsc user's manual and if it is not rather "ilutp". "ilutp" is documented in Saad's book "Iterative methods for sparse linear systems" and at least two of its arguments are similar to those of the "-pc_factor_use_drop_tolerance" option, that is,
, the threshold or drop tolerance (the "t" in "ilutp") and the number of nonzeros per row . The remaining argument, , the "tolerance for column pivot", sounds related to the "p" in "ilutp" that stands for "pivoting". There is a main difference between "ilutp" and "luinc" (Matlab) and is that the dropping rule is applied by rows in "ilutp" (Saad's book) while it is applied by columns in "luinc" (Matlab help). If the function used by Petsc is "ilutp" I have to conclude that the preconditioner implemented by the Matlab function "luinc(A,droptol)" is not available in Petsc. Could you validate or deny this? Through the PCSHELL preconditioner a custom preconditioner can be implemented. But this looks like difficult job. Could I generate the preconditioner matrices with Matlab, read them on a Petsc program and use them as preconditioner? Is there a quick way to do this by using PCSHELL? Thanks. -- Alejandro From dalcinl at gmail.com Tue Sep 4 09:20:55 2007 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Tue, 4 Sep 2007 11:20:55 -0300 Subject: MatZeroColsIS question. In-Reply-To: <46DD573E.10303@cs.caltech.edu> References: <20070904140845.3369ea22@rav.ippt.gov.pl> <46DD573E.10303@cs.caltech.edu> Message-ID: On 9/4/07, Peter Schr?der wrote: > Why not just construct the matrix without the corresponding columns and > rows? That preserves the symmetry (as it should of course), gives you a > smaller matrix, and you don't have the hassle with the zero columns. The former is far simpler in some contexts, and the size (of the matrix) is not an issue if the appropriate MatOption is set regarding sucessive assemblies. Anyway, you proposed solution, altough more difficult to implement in practice for general cases, is actually the best; especially in the context of iterative methods, as you iterate using shorter vectors. I was discussing on petsc-dev about implementing a general object PetscMapping. This would enable to set a general mapping on Vec's and Mat's for using with {Vec|Mat}SetValues in order to 'filter and map' indices, but this is a non-trivial feature that will require a fair amount of time to implement. -- Lisandro Dalc?n --------------- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 From knepley at gmail.com Tue Sep 4 09:32:56 2007 From: knepley at gmail.com (Matthew Knepley) Date: Tue, 4 Sep 2007 09:32:56 -0500 Subject: incomplete LU with drop tolerance In-Reply-To: <1188915537.46dd695174188@webmail.mail.gatech.edu> References: <1188915537.46dd695174188@webmail.mail.gatech.edu> Message-ID: There are a huge number of variants of ILU. Our interface to SPARSKIT may be old as we do not have a lot of call for it. Most people are interested in parallel versions, such as the ILUDT in Hypre. I am also not sure how meaningful this comparison will really be. Matlab uses LAPACK, etc. in the backend to achieve fairly good serial performance. As long as you are careful, it should be hard to beat it. However, it is proprietary, not scalable, and not very portable to typical large scale computing hardware. Performance on a serial benchmark seems to me to miss the point. Thanks, Matt On 9/4/07, Alejandro Garzon wrote: > Hi, I'm trying to find out what is the performance advantage of Petsc > over Matlab when solving a particular linear system. For the > comparison to be meaningful I have to use the exact same linear solver > and preconditioner in both Petsc and Matlab. The one problem I > have found in this attempt is that I haven't been able to identify > which options should I give to my Petsc code so that it uses the same > preconditioner as Matlab's function "luinc" when used in the syntax > "[L,U]=luinc(A,droptol)". In the Matlab documentation it is said that > in this syntax the function uses incomplete LU factorization with drop > tolerance. A similar preconditioner is available > in Petsc through the option "-pc_factor_use_drop_tolerance >
,, ". To see if they were exactly the same I tried > to know about the internal working of each preconditioner. In > the manual page of the Petsc function "PCFactorSetUseDropTolerance" > (that sets the same preconditioner as the option above) it is said > that this preconditioner uses the function "iludt" from SPARSKIT. > I downloaded SPARSKIT from > www-users.cs.umn.edu/~saad/software/SPARSKIT/sparskit.html and when > searching through the source code I couldn't find any function with that > exact name. My fist question is if the function's name is not > misspelled in the Petsc documentation both the manual page and the > Petsc user's manual and if it is not rather "ilutp". > "ilutp" is documented in Saad's book "Iterative methods for sparse > linear systems" and at least two of its arguments are similar to those > of the "-pc_factor_use_drop_tolerance" option, that is,
, the threshold > or drop tolerance (the "t" in "ilutp") and the number of nonzeros per > row . The remaining argument, , the "tolerance for > column pivot", sounds related to the "p" in "ilutp" that stands for > "pivoting". > > There is a main difference between "ilutp" and "luinc" (Matlab) and is > that the dropping rule is applied by rows in "ilutp" (Saad's book) > while it is applied by columns in "luinc" (Matlab help). > > If the function used by Petsc is "ilutp" I have to conclude that the > preconditioner implemented by the Matlab function "luinc(A,droptol)" is not > available in Petsc. Could you validate or deny this? > > Through the PCSHELL preconditioner a custom preconditioner can be > implemented. But this looks like difficult job. Could I generate the > preconditioner matrices with Matlab, read them on a Petsc program > and use them as preconditioner? Is there a quick way to do this by > using PCSHELL? > > Thanks. > > -- > Alejandro > > > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener From amjad11 at gmail.com Tue Sep 4 09:32:57 2007 From: amjad11 at gmail.com (amjad ali) Date: Tue, 4 Sep 2007 19:32:57 +0500 Subject: [PETSC #16590] PETSc on Dual Opteron Cluster In-Reply-To: References: <428810f20708162112g11b23c6er57c5f07b517b8777@mail.gmail.com> <428810f20708192347i45df22ek43c49c2a78a65d7d@mail.gmail.com> <428810f20708202155t7a2e96edhdc2214a716d51666@mail.gmail.com> <428810f20709030025wd522459xe5b872f786e3435f@mail.gmail.com> <428810f20709040446n4e6e7781w37d85d581e3d6556@mail.gmail.com> Message-ID: <428810f20709040732t5966142dw6b69fae3693b71e5@mail.gmail.com> Hello all in PETSc team and others, I tried to install/configure PETSc 2.3.3-p5 on my FC6 desktop computer. Following Happend: [petsc1 at localhost petsc-2.3.3-p5]$ ./config/configure.py --with-cc=gcc --with-fc=gfortran --with-cxx=g++ --download-f-blas-lapack=1 --download-mpich=1 --with-scalar-type=complex --with-clanguage=cxx ================================================================================= Configuring PETSc to compile on your system ================================================================================= TESTING: configureLibrary from config.packages.BlasLapack (python/BuildSystem/con********************************************************************************* UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log for details): --------------------------------------------------------------------------------------- Error downloading fblaslapack.tar.gz requested with -with-f-blas-lapack option ********************************************************************************* How to tackle this situation? At the moment I want to develop & run PARALLEL codes on my FC6 Desktop (single machine). Should I use the option --download-mpich=1 or --download-mpich=0 Regards to All. Amjad Ali. On 9/4/07, Matthew Knepley wrote: > > On 9/4/07, amjad ali wrote: > > Hello Satish, hoping you will be fine and happy. > > > > Is it necessary to have MPICH2 already installed (seaprately), before > > installing/configuring PETSc. > > If this is Windows, yes it is necessary (if you want to run in > parallel). If not, > you can use --with-mpi=0. > > > Do we need at all MPICH2 separately to wprk with PETSc or installing > PETSc > > will automatically have MPI implementation within that? > > PETSc can automatically build MPI, but not on Windows unfortunately. > > Thanks, > > Matt > > > regards, > > Amjad Ali. > > > > Also please answer my Yesterday's question: > > > > What is your opinion about using PETSc for developing/running > appliactions > > on MS Windows (Compute Cluster server) based cluster? > > > > Is it better/equal to Linux based system in performance? > > > > regards, > > Amjad Ali. > > > > > > > > On 8/21/07, amjad ali wrote: > > > > > > Hello Satish, > > > Thanks a lot for your kind answers. These are definitely helpful in > > clearifying concepts. > > > > > > Regards, > > > Amjad Ali > > > > > > > > > > > > On 8/21/07, Satish Balay wrote: > > > > On Mon, 20 Aug 2007, amjad ali wrote: > > > > > > > > > Hello, > > > > > Thanks Balay for your kind reply. > > > > > > > > > > As my system is: > > > > > Total cpu cores =4x4 = 16. And there are 2 memory banks per node, > so > > total > > > > > = 8 memory banks for the complete cluster. > > > > > > > > > > Let me ask my question in a better way: > > > > > *Is this system is suffiently good (giving decent performance) to > RUN > > > > > my "PETSc based Parallel CFD/FEM code" on this system *? > > > > > > > > it should be fine. > > > > > > > > > Can you please suggest any other *BETTER* PC-cluster for my PETSc > > code? > > > > > > > > Inte Core Duos appear to have much higher mermory bandwidth - so > they > > > > can potentially do better [this is unverified/speculation. Also the > > > > latencies are higher - so I'm not sure if this hurts performance > > > > much]. > > > > > > > > > > Also - we recommend doing the development on machines where its > easy > > > > > > to run debuggers [your desktop or laptop etc..] - and not a > parallel > > > > > > machine - where such things are hard to do. > > > > > > > > > > How to install and use PETSc on desktop (single machine)? Is it > > straight > > > > > forward? > > > > > > > > Yes [assuming your desktop is linux] its straight forward. Just use > > > > the configure option --download-mpich=1. You'll be able to run > > > > parallel codes on this machine [by default MPICH will use multiple > > > > processes for the parallel run, and all these processes will run on > > > > your desktop]. > > > > > > > > When developing code - performance is not critical, so the above > model > > > > will be more convinent than developing on a cluster. > > > > > > > > Satish > > > > > > > > > > > > > > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which > their experiments lead. > -- Norbert Wiener > -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Tue Sep 4 09:34:34 2007 From: knepley at gmail.com (Matthew Knepley) Date: Tue, 4 Sep 2007 09:34:34 -0500 Subject: MatZeroColsIS question. In-Reply-To: References: <20070904140845.3369ea22@rav.ippt.gov.pl> <46DD573E.10303@cs.caltech.edu> Message-ID: However, there is a very simple scheme, which I use. PETSc can ignore negative indices. I map all degrees of freedom associated to BC to negative indices when using PETSc Mat and Vec objects. Matt On 9/4/07, Lisandro Dalcin wrote: > On 9/4/07, Peter Schr?der wrote: > > Why not just construct the matrix without the corresponding columns and > > rows? That preserves the symmetry (as it should of course), gives you a > > smaller matrix, and you don't have the hassle with the zero columns. > > The former is far simpler in some contexts, and the size (of the > matrix) is not an issue if the appropriate MatOption is set regarding > sucessive assemblies. > > Anyway, you proposed solution, altough more difficult to implement in > practice for general cases, is actually the best; especially in the > context of iterative methods, as you iterate using shorter vectors. > > I was discussing on petsc-dev about implementing a general object > PetscMapping. This would enable to set a general mapping on Vec's and > Mat's for using with {Vec|Mat}SetValues in order to 'filter and map' > indices, but this is a non-trivial feature that will require a fair > amount of time to implement. > > > -- > Lisandro Dalc?n > --------------- > Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) > Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) > Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) > PTLC - G?emes 3450, (3000) Santa Fe, Argentina > Tel/Fax: +54-(0)342-451.1594 > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener From ps at cs.caltech.edu Tue Sep 4 09:39:41 2007 From: ps at cs.caltech.edu (=?ISO-8859-1?Q?Peter_Schr=F6der?=) Date: Tue, 04 Sep 2007 07:39:41 -0700 Subject: MatZeroColsIS question. In-Reply-To: References: <20070904140845.3369ea22@rav.ippt.gov.pl> <46DD573E.10303@cs.caltech.edu> Message-ID: <46DD6E2D.1040200@cs.caltech.edu> An HTML attachment was scrubbed... URL: From amjad11 at gmail.com Tue Sep 4 09:50:33 2007 From: amjad11 at gmail.com (amjad ali) Date: Tue, 4 Sep 2007 19:50:33 +0500 Subject: [PETSC #16590] PETSc on Dual Opteron Cluster In-Reply-To: References: <428810f20708162112g11b23c6er57c5f07b517b8777@mail.gmail.com> <428810f20708192347i45df22ek43c49c2a78a65d7d@mail.gmail.com> <428810f20708202155t7a2e96edhdc2214a716d51666@mail.gmail.com> <428810f20709030025wd522459xe5b872f786e3435f@mail.gmail.com> <428810f20709040446n4e6e7781w37d85d581e3d6556@mail.gmail.com> <428810f20709040732t5966142dw6b69fae3693b71e5@mail.gmail.com> Message-ID: <428810f20709040750l81e46cfr2eea5ac50edd0a83@mail.gmail.com> Hello Matt... 1) configure.log attached 2) Yes, I am online. 3) At the moment I want to develop & run PARALLEL codes on my FC6 Desktop (single machine). Should I use the option --download-mpich=1 or --download-mpich=0 regards, Amjad Ali. On 9/4/07, Matthew Knepley wrote: > > 1) Always send configure.log > > 2) It appears that you cannot reach the PETSc ftp site. Are you online? > > Matt > > On 9/4/07, amjad ali wrote: > > Hello all in PETSc team and others, > > > > I tried to install/configure PETSc 2.3.3-p5 on my FC6 desktop computer. > > Following Happend: > > > > [petsc1 at localhost petsc-2.3.3-p5]$ ./config/configure.py --with-cc=gcc > > --with-fc=gfortran --with-cxx=g++ --download-f-blas-lapack=1 > > --download-mpich=1 --with-scalar-type=complex --with-clanguage=cxx > > > ================================================================================= > > Configuring PETSc to compile on your system > > > > > ================================================================================= > > TESTING: configureLibrary from > > config.packages.BlasLapack > (python/BuildSystem/con********************************************************************************* > > UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.logfor > > details): > > > --------------------------------------------------------------------------------------- > > Error downloading fblaslapack.tar.gz requested with -with-f-blas-lapack > > option > > > ********************************************************************************* > > > > How to tackle this situation? > > > > At the moment I want to develop & run PARALLEL codes on my FC6 Desktop > > (single machine). Should I use the option --download-mpich=1 or > > --download-mpich=0 > > > > Regards to All. > > Amjad Ali. > > > > On 9/4/07, Matthew Knepley wrote: > > > On 9/4/07, amjad ali wrote: > > > > Hello Satish, hoping you will be fine and happy. > > > > > > > > Is it necessary to have MPICH2 already installed (seaprately), > before > > > > installing/configuring PETSc. > > > > > > If this is Windows, yes it is necessary (if you want to run in > > > parallel). If not, > > > you can use --with-mpi=0. > > > > > > > Do we need at all MPICH2 separately to wprk with PETSc or installing > > PETSc > > > > will automatically have MPI implementation within that? > > > > > > PETSc can automatically build MPI, but not on Windows unfortunately. > > > > > > Thanks, > > > > > > Matt > > > > > > > regards, > > > > Amjad Ali. > > > > > > > > Also please answer my Yesterday's question: > > > > > > > > What is your opinion about using PETSc for developing/running > > appliactions > > > > on MS Windows (Compute Cluster server) based cluster? > > > > > > > > Is it better/equal to Linux based system in performance? > > > > > > > > regards, > > > > Amjad Ali. > > > > > > > > > > > > > > > > On 8/21/07, amjad ali wrote: > > > > > > > > > > Hello Satish, > > > > > Thanks a lot for your kind answers. These are definitely helpful > in > > > > clearifying concepts. > > > > > > > > > > Regards, > > > > > Amjad Ali > > > > > > > > > > > > > > > > > > > > On 8/21/07, Satish Balay wrote: > > > > > > On Mon, 20 Aug 2007, amjad ali wrote: > > > > > > > > > > > > > Hello, > > > > > > > Thanks Balay for your kind reply. > > > > > > > > > > > > > > As my system is: > > > > > > > Total cpu cores =4x4 = 16. And there are 2 memory banks per > node, > > so > > > > total > > > > > > > = 8 memory banks for the complete cluster. > > > > > > > > > > > > > > Let me ask my question in a better way: > > > > > > > *Is this system is suffiently good (giving decent performance) > to > > RUN > > > > > > > my "PETSc based Parallel CFD/FEM code" on this system *? > > > > > > > > > > > > it should be fine. > > > > > > > > > > > > > Can you please suggest any other *BETTER* PC-cluster for my > PETSc > > > > code? > > > > > > > > > > > > Inte Core Duos appear to have much higher mermory bandwidth - so > > they > > > > > > can potentially do better [this is unverified/speculation. Also > the > > > > > > latencies are higher - so I'm not sure if this hurts performance > > > > > > much]. > > > > > > > > > > > > > > Also - we recommend doing the development on machines where > its > > easy > > > > > > > > to run debuggers [your desktop or laptop etc..] - and not a > > parallel > > > > > > > > machine - where such things are hard to do. > > > > > > > > > > > > > > How to install and use PETSc on desktop (single machine)? Is > it > > > > straight > > > > > > > forward? > > > > > > > > > > > > Yes [assuming your desktop is linux] its straight forward. Just > use > > > > > > the configure option --download-mpich=1. You'll be able to run > > > > > > parallel codes on this machine [by default MPICH will use > multiple > > > > > > processes for the parallel run, and all these processes will run > on > > > > > > your desktop]. > > > > > > > > > > > > When developing code - performance is not critical, so the above > > model > > > > > > will be more convinent than developing on a cluster. > > > > > > > > > > > > Satish > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > What most experimenters take for granted before they begin their > > > experiments is infinitely more interesting than any results to which > > > their experiments lead. > > > -- Norbert Wiener > > > > > > > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which > their experiments lead. > -- Norbert Wiener > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jeff.science at gmail.com Tue Sep 4 09:52:52 2007 From: jeff.science at gmail.com (Jeff Hammond) Date: Tue, 4 Sep 2007 09:52:52 -0500 Subject: [PETSC #16590] PETSc on Dual Opteron Cluster In-Reply-To: <428810f20709040732t5966142dw6b69fae3693b71e5@mail.gmail.com> References: <428810f20708162112g11b23c6er57c5f07b517b8777@mail.gmail.com> <428810f20708192347i45df22ek43c49c2a78a65d7d@mail.gmail.com> <428810f20708202155t7a2e96edhdc2214a716d51666@mail.gmail.com> <428810f20709030025wd522459xe5b872f786e3435f@mail.gmail.com> <428810f20709040446n4e6e7781w37d85d581e3d6556@mail.gmail.com> <428810f20709040732t5966142dw6b69fae3693b71e5@mail.gmail.com> Message-ID: <96f4bb620709040752r6d83c34r3e0621886d2f05e3@mail.gmail.com> I had no trouble configuring PETSc 2.3.3-p5 on my machine (granted it's Suse 10.2 not FC6) with your settings. The problem seems to be rather obvious: "Error downloading fblaslapack.tar.gz". Are you sure your firewall is configured properly as to permit you to access the appropriate repositories? Why don't you just point PETSc to libblas.a and liblapack.a, which come with FC6? If they aren't there now ('locate libblas') you can get them from the package manager. As for the value of --download-mpich=n (n=0,1) that would depend on whether or not you already have MPICH on your system. Jeff From amjad11 at gmail.com Tue Sep 4 10:28:29 2007 From: amjad11 at gmail.com (amjad ali) Date: Tue, 4 Sep 2007 20:28:29 +0500 Subject: [PETSC #16669] PETSc on Dual Opteron Cluster In-Reply-To: References: <428810f20709040754v3b347a2bg60248fb7989ef9e5@mail.gmail.com> Message-ID: <428810f20709040828v5e279153j2a9f5fa39b156876@mail.gmail.com> Thanks Matt. I will do so for fblaclapack.Now as perhaps I am unable to reach PETSc website then is it suitable to give --download-mpich. as you said? So I think I should likewise download mpich2 and give its path in Option. Sorry, but my confusion is still there: At the moment I want to develop & run PARALLEL codes on my FC6 Desktop (single machine). Should I use the option --download-mpich=1 or --download-mpich=0 When downloaded mpich2 (instead of mpich) should I give --download-mpich2=n or simply --download-mpich=n (n = 0 or 1). Could you please write the complete command for me (that gives proper option for using downloaded MPICH2) like you gave following: When configuring, use --download-f-blas-lapack=/path/to/fblaslapack.tar.gz regards, Amjad Ali. On 9/4/07, Matthew Knepley wrote: > > You appear to have a problem reaching the PETSc website, or your Python > installation is incomplete. However, you can do this: > > 1) Go to ftp.mcs.anl.gov > > 2) cd pub/petsc/externalpackages > > 3) get fblaslapack.tar.gz > > 4) When configuring, use > --download-f-blas-lapack=/path/to/fblaslapack.tar.gz > > Matt > > On 9/4/07, amjad ali wrote: > > Hello Matt, > > Sorry, > > > > configure.log attached this time. > > > > regrads > > > > > > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which > their experiments lead. > -- Norbert Wiener > -------------- next part -------------- An HTML attachment was scrubbed... URL: From li76pan at yahoo.com Tue Sep 4 11:09:48 2007 From: li76pan at yahoo.com (li pan) Date: Tue, 4 Sep 2007 09:09:48 -0700 (PDT) Subject: memory problem at parallel on a linux cluster Message-ID: <216464.42791.qm@web36811.mail.mud.yahoo.com> Dear all, I recently installed Petsc on a linux cluster and tried to solve a linear equation in parallel way. I used 3D Hex mesh. Mesh dimension is 181, 181, 41. The number of Dofs are 1343201. In serial run, there was no problem. But at parallel run, there was memory allocation problem. ----------------------------------------------------------------------- [0]PETSC ERROR: PetscMallocAlign() line 62 in src/sys/src/memory/mal.c [0]PETSC ERROR: Out of memory. This could be due to allocating [0]PETSC ERROR: too large an object or bleeding by not properly [0]PETSC ERROR: destroying unneeded objects. [3]PETSC ERROR: MatSetValues() line 702 in src/mat/interface/matrix.c [3]PETSC ERROR: User provided function() line 312 in unknowndirectory/src/numerics/petsc_matrix.C [cli_3]: aborting job: application called MPI_Abort(comm=0x84000000, 55) - process 3 [0]PETSC ERROR: Memory allocated 865987336 Memory used by process 1591005184 [0]PETSC ERROR: Try running with -malloc_dump or -malloc_log for info. [0]PETSC ERROR: Memory requested 1310720296! [0]PETSC ERROR: PetscTrMallocDefault() line 188 in src/sys/src/memory/mtr.c [0]PETSC ERROR: MatStashExpand_Private() line 240 in src/mat/utils/matstash.c [0]PETSC ERROR: MatStashValuesRow_Private() line 276 in src/mat/utils/matstash.c [0]PETSC ERROR: MatSetValues_MPIAIJ() line 199 in src/mat/impls/aij/mpi/mpiaij.c [0]PETSC ERROR: MatSetValues() line 702 in src/mat/interface/matrix.c [0]PETSC ERROR: User provided function() line 312 in unknowndirectory/src/numerics/petsc_matrix.C [cli_0]: aborting job: application called MPI_Abort(comm=0x84000000, 55) - process 0 rank 3 in job 1 hpc16_44261 caused collective abort of all ranks exit status of rank 3: return code 55 I checked memory on all the nodes. Each of them has more than 2.5 GB before program starts. What could be the reason? thanx pan ____________________________________________________________________________________ Building a website is a piece of cake. Yahoo! Small Business gives you all the tools to get online. http://smallbusiness.yahoo.com/webhosting From knepley at gmail.com Tue Sep 4 12:06:36 2007 From: knepley at gmail.com (Matthew Knepley) Date: Tue, 4 Sep 2007 12:06:36 -0500 Subject: memory problem at parallel on a linux cluster In-Reply-To: <216464.42791.qm@web36811.mail.mud.yahoo.com> References: <216464.42791.qm@web36811.mail.mud.yahoo.com> Message-ID: Are you trying to set all the values from a single processor? Matt On 9/4/07, li pan wrote: > Dear all, > I recently installed Petsc on a linux cluster and > tried to solve a linear equation in parallel way. I > used 3D Hex mesh. Mesh dimension is 181, 181, 41. The > number of Dofs are 1343201. > In serial run, there was no problem. But at parallel > run, there was memory allocation problem. > ----------------------------------------------------------------------- > [0]PETSC ERROR: PetscMallocAlign() line 62 in > src/sys/src/memory/mal.c > [0]PETSC ERROR: Out of memory. This could be due to > allocating > [0]PETSC ERROR: too large an object or bleeding by not > properly > [0]PETSC ERROR: destroying unneeded objects. > [3]PETSC ERROR: MatSetValues() line 702 in > src/mat/interface/matrix.c > [3]PETSC ERROR: User provided function() line 312 in > unknowndirectory/src/numerics/petsc_matrix.C > [cli_3]: aborting job: > application called MPI_Abort(comm=0x84000000, 55) - > process 3 > [0]PETSC ERROR: Memory allocated 865987336 Memory used > by process 1591005184 > [0]PETSC ERROR: Try running with -malloc_dump or > -malloc_log for info. > [0]PETSC ERROR: Memory requested 1310720296! > [0]PETSC ERROR: PetscTrMallocDefault() line 188 in > src/sys/src/memory/mtr.c > [0]PETSC ERROR: MatStashExpand_Private() line 240 in > src/mat/utils/matstash.c > [0]PETSC ERROR: MatStashValuesRow_Private() line 276 > in src/mat/utils/matstash.c > [0]PETSC ERROR: MatSetValues_MPIAIJ() line 199 in > src/mat/impls/aij/mpi/mpiaij.c > [0]PETSC ERROR: MatSetValues() line 702 in > src/mat/interface/matrix.c > [0]PETSC ERROR: User provided function() line 312 in > unknowndirectory/src/numerics/petsc_matrix.C > [cli_0]: aborting job: > application called MPI_Abort(comm=0x84000000, 55) - > process 0 > rank 3 in job 1 hpc16_44261 caused collective abort > of all ranks > exit status of rank 3: return code 55 > > > I checked memory on all the nodes. Each of them has > more than 2.5 GB before program starts. > What could be the reason? > > thanx > > pan > > > > > > ____________________________________________________________________________________ > Building a website is a piece of cake. Yahoo! Small Business gives you all the tools to get online. > http://smallbusiness.yahoo.com/webhosting > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener From yaronkretchmer at gmail.com Tue Sep 4 19:29:21 2007 From: yaronkretchmer at gmail.com (Yaron Kretchmer) Date: Tue, 4 Sep 2007 17:29:21 -0700 Subject: where can i find the matrix examples for ex10 under src/ksp/ksp/examples/tutorials? Message-ID: Thanks Yaron -------------- next part -------------- An HTML attachment was scrubbed... URL: From li76pan at yahoo.com Wed Sep 5 02:10:33 2007 From: li76pan at yahoo.com (li pan) Date: Wed, 5 Sep 2007 00:10:33 -0700 (PDT) Subject: memory problem at parallel on a linux cluster In-Reply-To: Message-ID: <759590.95163.qm@web36811.mail.mud.yahoo.com> hi Matt, I'm using libmesh. So I have no idea how the values were set. Before, I was connecting several computers in my office. And I didn't have this problem. Recently, I tried to install all libraries to a linux cluster. And I've got this problem. I don't know why. mpdtrace shows all the connected nodes I want. The only one difference is, all the nodes are mounted to a headnode. In my office I didn't use mount. Could this be the reason? thanx pan --- Matthew Knepley wrote: > Are you trying to set all the values from a single > processor? > > Matt > > On 9/4/07, li pan wrote: > > Dear all, > > I recently installed Petsc on a linux cluster and > > tried to solve a linear equation in parallel way. > I > > used 3D Hex mesh. Mesh dimension is 181, 181, 41. > The > > number of Dofs are 1343201. > > In serial run, there was no problem. But at > parallel > > run, there was memory allocation problem. > > > ----------------------------------------------------------------------- > > [0]PETSC ERROR: PetscMallocAlign() line 62 in > > src/sys/src/memory/mal.c > > [0]PETSC ERROR: Out of memory. This could be due > to > > allocating > > [0]PETSC ERROR: too large an object or bleeding by > not > > properly > > [0]PETSC ERROR: destroying unneeded objects. > > [3]PETSC ERROR: MatSetValues() line 702 in > > src/mat/interface/matrix.c > > [3]PETSC ERROR: User provided function() line 312 > in > > unknowndirectory/src/numerics/petsc_matrix.C > > [cli_3]: aborting job: > > application called MPI_Abort(comm=0x84000000, 55) > - > > process 3 > > [0]PETSC ERROR: Memory allocated 865987336 Memory > used > > by process 1591005184 > > [0]PETSC ERROR: Try running with -malloc_dump or > > -malloc_log for info. > > [0]PETSC ERROR: Memory requested 1310720296! > > [0]PETSC ERROR: PetscTrMallocDefault() line 188 in > > src/sys/src/memory/mtr.c > > [0]PETSC ERROR: MatStashExpand_Private() line 240 > in > > src/mat/utils/matstash.c > > [0]PETSC ERROR: MatStashValuesRow_Private() line > 276 > > in src/mat/utils/matstash.c > > [0]PETSC ERROR: MatSetValues_MPIAIJ() line 199 in > > src/mat/impls/aij/mpi/mpiaij.c > > [0]PETSC ERROR: MatSetValues() line 702 in > > src/mat/interface/matrix.c > > [0]PETSC ERROR: User provided function() line 312 > in > > unknowndirectory/src/numerics/petsc_matrix.C > > [cli_0]: aborting job: > > application called MPI_Abort(comm=0x84000000, 55) > - > > process 0 > > rank 3 in job 1 hpc16_44261 caused collective > abort > > of all ranks > > exit status of rank 3: return code 55 > > > > > > I checked memory on all the nodes. Each of them > has > > more than 2.5 GB before program starts. > > What could be the reason? > > > > thanx > > > > pan > > > > > > > > > > > > > ____________________________________________________________________________________ > > Building a website is a piece of cake. Yahoo! > Small Business gives you all the tools to get > online. > > http://smallbusiness.yahoo.com/webhosting > > > > > > > -- > What most experimenters take for granted before they > begin their > experiments is infinitely more interesting than any > results to which > their experiments lead. > -- Norbert Wiener > > ____________________________________________________________________________________ Sick sense of humor? Visit Yahoo! TV's Comedy with an Edge to see what's on, when. http://tv.yahoo.com/collections/222 From knepley at gmail.com Wed Sep 5 06:11:17 2007 From: knepley at gmail.com (Matthew Knepley) Date: Wed, 5 Sep 2007 06:11:17 -0500 Subject: memory problem at parallel on a linux cluster In-Reply-To: <759590.95163.qm@web36811.mail.mud.yahoo.com> References: <759590.95163.qm@web36811.mail.mud.yahoo.com> Message-ID: Here is the trace: 0]PETSC ERROR: Memory allocated 865987336 Memory used by process 1591005184 [0]PETSC ERROR: Try running with -malloc_dump or -malloc_log for info. [0]PETSC ERROR: Memory requested 1310720296! [0]PETSC ERROR: PetscTrMallocDefault() line 188 in src/sys/src/memory/mtr.c [0]PETSC ERROR: MatStashExpand_Private() line 240 in src/mat/utils/matstash.c [0]PETSC ERROR: MatStashValuesRow_Private() line 276 in src/mat/utils/matstash.c [0]PETSC ERROR: MatSetValues_MPIAIJ() line 199 in src/mat/impls/aij/mpi/mpiaij.c [0]PETSC ERROR: MatSetValues() line 702 in src/mat/interface/matrix.c [0]PETSC ERROR: User provided function() line 312 in unknowndirectory/src/numerics/petsc_matrix.C So, you did not write petsc_matrix? What is happening here is off-processor values are being set with MatSetValues(). That means we have to stash them. This is not inherently bad, but the stash space grows so large that memory on the node is exhausted. This is very rare with a PDE problem on a mesh. That what leads me to think that too many values are being generated on a single proc. Matt On 9/5/07, li pan wrote: > hi Matt, > I'm using libmesh. So I have no idea how the values > were set. Before, I was connecting several computers > in my office. And I didn't have this problem. > Recently, I tried to install all libraries to a linux > cluster. And I've got this problem. I don't know why. > mpdtrace shows all the connected nodes I want. The > only one difference is, all the nodes are mounted to a > headnode. In my office I didn't use mount. > Could this be the reason? > > thanx > > pan > > > --- Matthew Knepley wrote: > > > Are you trying to set all the values from a single > > processor? > > > > Matt > > > > On 9/4/07, li pan wrote: > > > Dear all, > > > I recently installed Petsc on a linux cluster and > > > tried to solve a linear equation in parallel way. > > I > > > used 3D Hex mesh. Mesh dimension is 181, 181, 41. > > The > > > number of Dofs are 1343201. > > > In serial run, there was no problem. But at > > parallel > > > run, there was memory allocation problem. > > > > > > ----------------------------------------------------------------------- > > > [0]PETSC ERROR: PetscMallocAlign() line 62 in > > > src/sys/src/memory/mal.c > > > [0]PETSC ERROR: Out of memory. This could be due > > to > > > allocating > > > [0]PETSC ERROR: too large an object or bleeding by > > not > > > properly > > > [0]PETSC ERROR: destroying unneeded objects. > > > [3]PETSC ERROR: MatSetValues() line 702 in > > > src/mat/interface/matrix.c > > > [3]PETSC ERROR: User provided function() line 312 > > in > > > unknowndirectory/src/numerics/petsc_matrix.C > > > [cli_3]: aborting job: > > > application called MPI_Abort(comm=0x84000000, 55) > > - > > > process 3 > > > [0]PETSC ERROR: Memory allocated 865987336 Memory > > used > > > by process 1591005184 > > > [0]PETSC ERROR: Try running with -malloc_dump or > > > -malloc_log for info. > > > [0]PETSC ERROR: Memory requested 1310720296! > > > [0]PETSC ERROR: PetscTrMallocDefault() line 188 in > > > src/sys/src/memory/mtr.c > > > [0]PETSC ERROR: MatStashExpand_Private() line 240 > > in > > > src/mat/utils/matstash.c > > > [0]PETSC ERROR: MatStashValuesRow_Private() line > > 276 > > > in src/mat/utils/matstash.c > > > [0]PETSC ERROR: MatSetValues_MPIAIJ() line 199 in > > > src/mat/impls/aij/mpi/mpiaij.c > > > [0]PETSC ERROR: MatSetValues() line 702 in > > > src/mat/interface/matrix.c > > > [0]PETSC ERROR: User provided function() line 312 > > in > > > unknowndirectory/src/numerics/petsc_matrix.C > > > [cli_0]: aborting job: > > > application called MPI_Abort(comm=0x84000000, 55) > > - > > > process 0 > > > rank 3 in job 1 hpc16_44261 caused collective > > abort > > > of all ranks > > > exit status of rank 3: return code 55 > > > > > > > > > I checked memory on all the nodes. Each of them > > has > > > more than 2.5 GB before program starts. > > > What could be the reason? > > > > > > thanx > > > > > > pan > > > > > > > > > > > > > > > > > > > > > ____________________________________________________________________________________ > > > Building a website is a piece of cake. Yahoo! > > Small Business gives you all the tools to get > > online. > > > http://smallbusiness.yahoo.com/webhosting > > > > > > > > > > > > -- > > What most experimenters take for granted before they > > begin their > > experiments is infinitely more interesting than any > > results to which > > their experiments lead. > > -- Norbert Wiener > > > > > > > > > ____________________________________________________________________________________ > Sick sense of humor? Visit Yahoo! TV's > Comedy with an Edge to see what's on, when. > http://tv.yahoo.com/collections/222 > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener From li76pan at yahoo.com Wed Sep 5 06:36:47 2007 From: li76pan at yahoo.com (li pan) Date: Wed, 5 Sep 2007 04:36:47 -0700 (PDT) Subject: memory problem at parallel on a linux cluster In-Reply-To: Message-ID: <811819.17698.qm@web36804.mail.mud.yahoo.com> hi Matt, I found the error. It's my problem. I used a serial code. My program tried to write value for other process. As you said, off-process values. After I corrected the code to parallel mode, it works now. Sorry for wasting your time. :-) thanx pan --- Matthew Knepley wrote: > Here is the trace: > > 0]PETSC ERROR: Memory allocated 865987336 Memory > used > by process 1591005184 > [0]PETSC ERROR: Try running with -malloc_dump or > -malloc_log for info. > [0]PETSC ERROR: Memory requested 1310720296! > [0]PETSC ERROR: PetscTrMallocDefault() line 188 in > src/sys/src/memory/mtr.c > [0]PETSC ERROR: MatStashExpand_Private() line 240 in > src/mat/utils/matstash.c > [0]PETSC ERROR: MatStashValuesRow_Private() line 276 > in src/mat/utils/matstash.c > [0]PETSC ERROR: MatSetValues_MPIAIJ() line 199 in > src/mat/impls/aij/mpi/mpiaij.c > [0]PETSC ERROR: MatSetValues() line 702 in > src/mat/interface/matrix.c > [0]PETSC ERROR: User provided function() line 312 in > unknowndirectory/src/numerics/petsc_matrix.C > > So, you did not write petsc_matrix? What is > happening here is > off-processor values are being set with > MatSetValues(). That means > we have to stash them. This is not inherently bad, > but the stash space > grows so large that memory on the node is exhausted. > This is very rare > with a PDE problem on a mesh. That what leads me to > think that too many > values are being generated on a single proc. > > Matt > > On 9/5/07, li pan wrote: > > hi Matt, > > I'm using libmesh. So I have no idea how the > values > > were set. Before, I was connecting several > computers > > in my office. And I didn't have this problem. > > Recently, I tried to install all libraries to a > linux > > cluster. And I've got this problem. I don't know > why. > > mpdtrace shows all the connected nodes I want. The > > only one difference is, all the nodes are mounted > to a > > headnode. In my office I didn't use mount. > > Could this be the reason? > > > > thanx > > > > pan > > > > > > --- Matthew Knepley wrote: > > > > > Are you trying to set all the values from a > single > > > processor? > > > > > > Matt > > > > > > On 9/4/07, li pan wrote: > > > > Dear all, > > > > I recently installed Petsc on a linux cluster > and > > > > tried to solve a linear equation in parallel > way. > > > I > > > > used 3D Hex mesh. Mesh dimension is 181, 181, > 41. > > > The > > > > number of Dofs are 1343201. > > > > In serial run, there was no problem. But at > > > parallel > > > > run, there was memory allocation problem. > > > > > > > > > > ----------------------------------------------------------------------- > > > > [0]PETSC ERROR: PetscMallocAlign() line 62 in > > > > src/sys/src/memory/mal.c > > > > [0]PETSC ERROR: Out of memory. This could be > due > > > to > > > > allocating > > > > [0]PETSC ERROR: too large an object or > bleeding by > > > not > > > > properly > > > > [0]PETSC ERROR: destroying unneeded objects. > > > > [3]PETSC ERROR: MatSetValues() line 702 in > > > > src/mat/interface/matrix.c > > > > [3]PETSC ERROR: User provided function() line > 312 > > > in > > > > unknowndirectory/src/numerics/petsc_matrix.C > > > > [cli_3]: aborting job: > > > > application called MPI_Abort(comm=0x84000000, > 55) > > > - > > > > process 3 > > > > [0]PETSC ERROR: Memory allocated 865987336 > Memory > > > used > > > > by process 1591005184 > > > > [0]PETSC ERROR: Try running with -malloc_dump > or > > > > -malloc_log for info. > > > > [0]PETSC ERROR: Memory requested 1310720296! > > > > [0]PETSC ERROR: PetscTrMallocDefault() line > 188 in > > > > src/sys/src/memory/mtr.c > > > > [0]PETSC ERROR: MatStashExpand_Private() line > 240 > > > in > > > > src/mat/utils/matstash.c > > > > [0]PETSC ERROR: MatStashValuesRow_Private() > line > > > 276 > > > > in src/mat/utils/matstash.c > > > > [0]PETSC ERROR: MatSetValues_MPIAIJ() line 199 > in > > > > src/mat/impls/aij/mpi/mpiaij.c > > > > [0]PETSC ERROR: MatSetValues() line 702 in > > > > src/mat/interface/matrix.c > > > > [0]PETSC ERROR: User provided function() line > 312 > > > in > > > > unknowndirectory/src/numerics/petsc_matrix.C > > > > [cli_0]: aborting job: > > > > application called MPI_Abort(comm=0x84000000, > 55) > > > - > > > > process 0 > > > > rank 3 in job 1 hpc16_44261 caused > collective > > > abort > > > > of all ranks > > > > exit status of rank 3: return code 55 > > > > > > > > > > > > I checked memory on all the nodes. Each of > them > > > has > > > > more than 2.5 GB before program starts. > > > > What could be the reason? > > > > > > > > thanx > > > > > > > > pan > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > ____________________________________________________________________________________ > > > > Building a website is a piece of cake. Yahoo! > > > Small Business gives you all the tools to get > > > online. > > > > http://smallbusiness.yahoo.com/webhosting > > > > > > > > > > > > > > > > > -- > > > What most experimenters take for granted before > they > > > begin their > > > experiments is infinitely more interesting than > any > > > results to which > > > their experiments lead. > > > -- Norbert Wiener > > > > > > > > > > > > > > > > > ____________________________________________________________________________________ > > Sick sense of humor? Visit Yahoo! TV's > > Comedy with an Edge to see what's on, when. > > http://tv.yahoo.com/collections/222 > > > > > > > -- > What most experimenters take for granted before they > begin their > experiments is infinitely more interesting than any > results to which > their experiments lead. > === message truncated === ____________________________________________________________________________________Ready for the edge of your seat? Check out tonight's top picks on Yahoo! TV. http://tv.yahoo.com/ From refac at bristol.ac.uk Mon Sep 10 10:59:47 2007 From: refac at bristol.ac.uk (Felipe Cruz) Date: Mon, 10 Sep 2007 16:59:47 +0100 Subject: OpenMPI, PETSc and petsc4py in a Mac Intel Message-ID: <93318264-2BD1-410F-A2C1-E089DA3A5DAB@bris.ac.uk> Hi all, During the past weeks I was trying to install the Python bindings for PETSc (petsc4py) for Mac Intel (a black Macbook and an Aluminium iMac, both with OS X 10.4.10) I succeeded and "how I did it" might be helpful for some others Python-PETSc-Mac-Users. Before to start the installation you'll need to have installed: * XcodeTools: It is IMPORTANT to get the latest version directly from Apple's website (http://connect.apple.com), the version that comes with the installation CD have some issues (or not so nice "features".) I recommend to upgrade your XcodeTools before trying to install anything. * Python 2.5, Numpy, Scipy (optional - scientific lib) and Matplotlib (optional -matlab style graphics): I followed the installation of this packages from: http:// www.scipy.org/Download I installed MacPython2.5 and the SuperPack. For some reason Matplotlib didn't work after the installation of the superpack, so I manually installed it again from: http:// matplotlib.sourceforge.net/ INSTALLATION I'll describe the whole installation of OpenMPI, PETSc and petsc4py. OpenMpi ------------- Download OpenMPI source from: http://www.open-mpi.org/software/ompi/v1.2/ Decompress the downloaded tar.gz file and from a terminal "cd" into the openMpi folder. '$' indicates a terminal command, e.g; $ tar -xzvf openmpi-1.2.3.tar.gz $ cd openmpi-1.2.3 Now you can compile openmpi $ ./configure --prefix=/usr/local/openmpi --with-fortran --enable- shared --disable-static --with-xgrid $ sudo make $ sudo make install Add the next line at the END of /etc/profile, so the path will be available for all the users: export PATH=$PATH:/usr/local/openmpi/bin PETSc ------------- First you need to make sure that OpenMPI is properly installed! check that mpicc and mpif77 are in your $PATH before running the PETSc configure.py! (if you added the export PATH to /etc/profile you can close your terminal and open a new terminal, the PATH should be working now, otherwise, you can just type in your terminal: export PATH=$PATH:/usr/local/openmpi/bin I installed PETSc on /opt/petsc $ sudo mkdir /opt $ sudo mkdir /opt/petsc Set permissions for user and group. $ sudo chown user:group /opt/petsc In my case I added the user 'felipe' and the group 'admin' (sudo chown felipe:admin /opt/petsc) Download PETSc from: http://www-unix.mcs.anl.gov/petsc/petsc-as/download/index.html Save the downloaded tar.gz on /opt/petsc Decompress the file $ cd /opt/petsc $ tar -xzvf petsc-2.3.3-p5.tar.gz I installed petscy for shared libraries and without X, beside this everything is normal: $ cd petsc-2.3.3-p5 $ export PETSC_ARCH=darwin8.10.1-c-debug $ export PETSC_DIR=/opt/petsc/petsc-2.3.3-p5 $ ./config/configure.py --with-mpi-dir=/usr/local/openmpi --with- shared=1 --with-x=0 $ make all test At the end of /etc/profile add (so it will be available for all the users): # variables for petsc export PETSC_ARCH=darwin8.10.1-c-debug export PETSC_DIR=/opt/petsc/petsc-2.3.3-p5 petsc4py ------------- Open a new terminal so the variables PETSC_ARCH and PETSC_DIR will be available. I installed petsc4py in /opt/petsc/petsc4py In a terminal $ cd /opt/petsc/petsc4py Get a copy of petsc4py from: http://code.google.com/p/petsc4py/ and save it at /opt/petsc/petsc4py $ tar -xvf petsc4py-0.7.5.tar $ cd petsc4py-0.7.5 $ python setup.py install --home=/opt/petsc/petsc4py # Add at the end of /etc/profile (it will be available for all the users) export PYTHONPATH=/opt/petsc/petsc4py/lib/python/ Now is possible to open a new terminal and test petsc4py: $python >>> from petsc4py import PETSc >>> A = PETSc.Mat() >>> A >>> A.createSeqAIJ(1000) Hope that this info is helpful! I tested the installation in my MacBook (Dual-core), in an aluminium iMac (Dual-core), and I hope to test the installation on a Mac pro (with two quad-core) some time this week! I'm quite new to the list so I'm sorry if you have already discussed this topic. Best regards, Felipe Cruz From mspieg at ldeo.columbia.edu Mon Sep 10 12:01:36 2007 From: mspieg at ldeo.columbia.edu (mspieg) Date: Mon, 10 Sep 2007 13:01:36 -0400 Subject: Binary dump of SNES residuals Message-ID: <81A61241-017A-4EB8-B142-C39A672C01B1@ldeo.columbia.edu> Apologies if this is obvious, but is there a clean way to dump out SNES residuals (and or updates) (inside a DMMG) as binary vectors for debugging. I'm currently using -snes_monitor_residual which is lovely for visualizing the residuals but uses VecView with the default Viewer set to draw and it would be useful to have a command line option to change the viewer type to binary (if it doesn't already exist) I can certainly use SNESMonitorSet with a custom monitor that sets up a binary viewer for SNESMonitorResidual, but I'm being lazy. Any and all help greatly appreciated. cheers marc ---------------------------------------------------- Marc Spiegelman Lamont-Doherty Earth Observatory Dept. of Applied Physics/Applied Math Columbia University http://www.ldeo.columbia.edu/~mspieg ---------------------------------------------------- -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Mon Sep 10 14:09:06 2007 From: knepley at gmail.com (Matthew Knepley) Date: Mon, 10 Sep 2007 14:09:06 -0500 Subject: OpenMPI, PETSc and petsc4py in a Mac Intel In-Reply-To: <93318264-2BD1-410F-A2C1-E089DA3A5DAB@bris.ac.uk> References: <93318264-2BD1-410F-A2C1-E089DA3A5DAB@bris.ac.uk> Message-ID: Thanks for your thorough directions. I have incorporated them into the installation page for dev, which will become public on the upcoming release. Matt On 9/10/07, Felipe Cruz wrote: > Hi all, > > During the past weeks I was trying to install the Python bindings for > PETSc (petsc4py) for > Mac Intel (a black Macbook and an Aluminium iMac, both with OS X > 10.4.10) > > I succeeded and "how I did it" might be helpful for some others > Python-PETSc-Mac-Users. > > Before to start the installation you'll need to have installed: > > * XcodeTools: > > It is IMPORTANT to get the latest version directly from Apple's > website (http://connect.apple.com), the version > that comes with the installation CD have some issues (or not so nice > "features".) I recommend to upgrade > your XcodeTools before trying to install anything. > > * Python 2.5, Numpy, Scipy (optional - scientific lib) and Matplotlib > (optional -matlab style graphics): > > I followed the installation of this packages from: http:// > www.scipy.org/Download > I installed MacPython2.5 and the SuperPack. For some reason > Matplotlib didn't work after the installation > of the superpack, so I manually installed it again from: http:// > matplotlib.sourceforge.net/ > > > INSTALLATION > > I'll describe the whole installation of OpenMPI, PETSc and petsc4py. > > OpenMpi > ------------- > > Download OpenMPI source from: > http://www.open-mpi.org/software/ompi/v1.2/ > > Decompress the downloaded tar.gz file and from a terminal "cd" into > the openMpi folder. > '$' indicates a terminal command, e.g; > $ tar -xzvf openmpi-1.2.3.tar.gz > $ cd openmpi-1.2.3 > > Now you can compile openmpi > $ ./configure --prefix=/usr/local/openmpi --with-fortran --enable- > shared --disable-static --with-xgrid > $ sudo make > $ sudo make install > > Add the next line at the END of /etc/profile, so the path will be > available for all the users: > export PATH=$PATH:/usr/local/openmpi/bin > > > > PETSc > ------------- > > First you need to make sure that OpenMPI is properly installed! check > that > mpicc and mpif77 are in your $PATH before running the PETSc > configure.py! > (if you added the export PATH to /etc/profile you can close your > terminal and > open a new terminal, the PATH should be working now, otherwise, you > can just type > in your terminal: > export PATH=$PATH:/usr/local/openmpi/bin > > > I installed PETSc on /opt/petsc > $ sudo mkdir /opt > $ sudo mkdir /opt/petsc > > Set permissions for user and group. > $ sudo chown user:group /opt/petsc > In my case I added the user 'felipe' and the group 'admin' (sudo > chown felipe:admin /opt/petsc) > > Download PETSc from: > http://www-unix.mcs.anl.gov/petsc/petsc-as/download/index.html > Save the downloaded tar.gz on /opt/petsc > > Decompress the file > $ cd /opt/petsc > $ tar -xzvf petsc-2.3.3-p5.tar.gz > > I installed petscy for shared libraries and without X, beside this > everything is normal: > $ cd petsc-2.3.3-p5 > $ export PETSC_ARCH=darwin8.10.1-c-debug > $ export PETSC_DIR=/opt/petsc/petsc-2.3.3-p5 > $ ./config/configure.py --with-mpi-dir=/usr/local/openmpi --with- > shared=1 --with-x=0 > $ make all test > > At the end of /etc/profile add (so it will be available for all the > users): > # variables for petsc > export PETSC_ARCH=darwin8.10.1-c-debug > export PETSC_DIR=/opt/petsc/petsc-2.3.3-p5 > > > > petsc4py > ------------- > Open a new terminal so the variables PETSC_ARCH and PETSC_DIR will be > available. > > I installed petsc4py in /opt/petsc/petsc4py > > In a terminal > $ cd /opt/petsc/petsc4py > > Get a copy of petsc4py from: http://code.google.com/p/petsc4py/ > and save it at /opt/petsc/petsc4py > > $ tar -xvf petsc4py-0.7.5.tar > $ cd petsc4py-0.7.5 > $ python setup.py install --home=/opt/petsc/petsc4py > > # Add at the end of /etc/profile (it will be available for all the > users) > export PYTHONPATH=/opt/petsc/petsc4py/lib/python/ > > Now is possible to open a new terminal and test petsc4py: > $python > >>> from petsc4py import PETSc > >>> A = PETSc.Mat() > >>> A > >>> A.createSeqAIJ(1000) > > > > > Hope that this info is helpful! I tested the installation in my > MacBook (Dual-core), in an aluminium iMac (Dual-core), > and I hope to test the installation on a Mac pro (with two quad-core) > some time this week! > > I'm quite new to the list so I'm sorry if you have already discussed > this topic. > > > Best regards, > > Felipe Cruz > > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener From petsc-maint at mcs.anl.gov Mon Sep 10 15:38:39 2007 From: petsc-maint at mcs.anl.gov (Barry Smith) Date: Mon, 10 Sep 2007 15:38:39 -0500 (CDT) Subject: [PETSC #16696] Binary dump of SNES residuals In-Reply-To: <81A61241-017A-4EB8-B142-C39A672C01B1@ldeo.columbia.edu> References: <81A61241-017A-4EB8-B142-C39A672C01B1@ldeo.columbia.edu> Message-ID: > I can certainly use SNESMonitorSet with a custom monitor that sets up a > binary viewer for SNESMonitorResidual, but I'm being lazy. This is currently the way to go: should require 3 lines of code, create the binary viewer, call SNESMonitorSet() and then destroy the viewer at the end. You can make it one line of code by passing PETSC_BINARY_VIEWER_WORLD into the the SNESMonitorSet() call. It would be nice to have a very generally solution to this, so one could easily direct to many different types of viewers from the options database. Maybe a rework of VecView_Private() code being used in conjunction with the various -xxx_monitor_residual/solution/update options, for KSP, SNES and TS. Do I have time to write it? Heck no, I don't even have time to read my email :-) Barry On Mon, 10 Sep 2007, mspieg wrote: > Apologies if this is obvious, but is there a clean way to dump out SNES > residuals (and or updates) (inside a DMMG) as binary vectors for debugging. > > > I'm currently using > > -snes_monitor_residual > > which is lovely for visualizing the residuals but uses VecView with the > default Viewer set to draw and it would be useful to have a command line > option to change the viewer type to binary (if it doesn't already exist) > > I can certainly use SNESMonitorSet with a custom monitor that sets up a > binary viewer for SNESMonitorResidual, but I'm being lazy. > > Any and all help greatly appreciated. > > cheers > marc > > > > ---------------------------------------------------- > Marc Spiegelman > Lamont-Doherty Earth Observatory > Dept. of Applied Physics/Applied Math > Columbia University > http://www.ldeo.columbia.edu/~mspieg > ---------------------------------------------------- > > From mspieg at ldeo.columbia.edu Mon Sep 10 16:42:08 2007 From: mspieg at ldeo.columbia.edu (mspieg) Date: Mon, 10 Sep 2007 17:42:08 -0400 Subject: [PETSC #16696] Binary dump of SNES residuals In-Reply-To: References: <81A61241-017A-4EB8-B142-C39A672C01B1@ldeo.columbia.edu> Message-ID: <58AFD67D-6851-43FB-B6A7-9AD0B94E8891@ldeo.columbia.edu> Thanks Barry, I hacked up something using SNESMonitorSet and a binary viewer that works (although not quite 1-3 lines of code). This will work great...thanks... cheers marc On Sep 10, 2007, at 4:38 PM, Barry Smith wrote: > > >> I can certainly use SNESMonitorSet with a custom monitor that >> sets up a >> binary viewer for SNESMonitorResidual, but I'm being lazy. > > This is currently the way to go: should require 3 lines of code, > create the binary > viewer, call SNESMonitorSet() and then destroy the viewer at the > end. You > can make it one line of code by passing PETSC_BINARY_VIEWER_WORLD > into the > the SNESMonitorSet() call. > > It would be nice to have a very generally solution to this, so > one could > easily direct to many different types of viewers from the options > database. > Maybe a rework of VecView_Private() code being used in conjunction > with the > various -xxx_monitor_residual/solution/update options, for KSP, > SNES and TS. > Do I have time to write it? Heck no, I don't even have time to read > my email :-) > > Barry > > > > On Mon, 10 Sep 2007, mspieg wrote: > >> Apologies if this is obvious, but is there a clean way to dump >> out SNES >> residuals (and or updates) (inside a DMMG) as binary vectors for >> debugging. >> >> >> I'm currently using >> >> -snes_monitor_residual >> >> which is lovely for visualizing the residuals but uses VecView >> with the >> default Viewer set to draw and it would be useful to have a >> command line >> option to change the viewer type to binary (if it doesn't already >> exist) >> >> I can certainly use SNESMonitorSet with a custom monitor that >> sets up a >> binary viewer for SNESMonitorResidual, but I'm being lazy. >> >> Any and all help greatly appreciated. >> >> cheers >> marc >> >> >> >> ---------------------------------------------------- >> Marc Spiegelman >> Lamont-Doherty Earth Observatory >> Dept. of Applied Physics/Applied Math >> Columbia University >> http://www.ldeo.columbia.edu/~mspieg >> ---------------------------------------------------- >> >> ---------------------------------------------------- Marc Spiegelman Lamont-Doherty Earth Observatory Dept. of Applied Physics/Applied Math Columbia University http://www.ldeo.columbia.edu/~mspieg ---------------------------------------------------- -------------- next part -------------- An HTML attachment was scrubbed... URL: From ondrej at certik.cz Tue Sep 11 09:20:24 2007 From: ondrej at certik.cz (Ondrej Certik) Date: Tue, 11 Sep 2007 16:20:24 +0200 Subject: OpenMPI, PETSc and petsc4py in a Mac Intel In-Reply-To: References: <93318264-2BD1-410F-A2C1-E089DA3A5DAB@bris.ac.uk> Message-ID: <85b5c3130709110720l77d1f1f0v84f9fb21d33187f3@mail.gmail.com> Hi, thanks for your instructions. For those using Debian, that want to have exactly the same configuration (petsc2.3.3 with superlu and umfpack, openmpi, petsc4py), just add unstable into your sources.list and do apt-get install python-petsc4py and that's it. Ondrej On 9/10/07, Matthew Knepley wrote: > Thanks for your thorough directions. I have incorporated them into the > installation page > for dev, which will become public on the upcoming release. > > Matt > > On 9/10/07, Felipe Cruz wrote: > > Hi all, > > > > During the past weeks I was trying to install the Python bindings for > > PETSc (petsc4py) for > > Mac Intel (a black Macbook and an Aluminium iMac, both with OS X > > 10.4.10) > > > > I succeeded and "how I did it" might be helpful for some others > > Python-PETSc-Mac-Users. > > > > Before to start the installation you'll need to have installed: > > > > * XcodeTools: > > > > It is IMPORTANT to get the latest version directly from Apple's > > website (http://connect.apple.com), the version > > that comes with the installation CD have some issues (or not so nice > > "features".) I recommend to upgrade > > your XcodeTools before trying to install anything. > > > > * Python 2.5, Numpy, Scipy (optional - scientific lib) and Matplotlib > > (optional -matlab style graphics): > > > > I followed the installation of this packages from: http:// > > www.scipy.org/Download > > I installed MacPython2.5 and the SuperPack. For some reason > > Matplotlib didn't work after the installation > > of the superpack, so I manually installed it again from: http:// > > matplotlib.sourceforge.net/ > > > > > > INSTALLATION > > > > I'll describe the whole installation of OpenMPI, PETSc and petsc4py. > > > > OpenMpi > > ------------- > > > > Download OpenMPI source from: > > http://www.open-mpi.org/software/ompi/v1.2/ > > > > Decompress the downloaded tar.gz file and from a terminal "cd" into > > the openMpi folder. > > '$' indicates a terminal command, e.g; > > $ tar -xzvf openmpi-1.2.3.tar.gz > > $ cd openmpi-1.2.3 > > > > Now you can compile openmpi > > $ ./configure --prefix=/usr/local/openmpi --with-fortran --enable- > > shared --disable-static --with-xgrid > > $ sudo make > > $ sudo make install > > > > Add the next line at the END of /etc/profile, so the path will be > > available for all the users: > > export PATH=$PATH:/usr/local/openmpi/bin > > > > > > > > PETSc > > ------------- > > > > First you need to make sure that OpenMPI is properly installed! check > > that > > mpicc and mpif77 are in your $PATH before running the PETSc > > configure.py! > > (if you added the export PATH to /etc/profile you can close your > > terminal and > > open a new terminal, the PATH should be working now, otherwise, you > > can just type > > in your terminal: > > export PATH=$PATH:/usr/local/openmpi/bin > > > > > > I installed PETSc on /opt/petsc > > $ sudo mkdir /opt > > $ sudo mkdir /opt/petsc > > > > Set permissions for user and group. > > $ sudo chown user:group /opt/petsc > > In my case I added the user 'felipe' and the group 'admin' (sudo > > chown felipe:admin /opt/petsc) > > > > Download PETSc from: > > http://www-unix.mcs.anl.gov/petsc/petsc-as/download/index.html > > Save the downloaded tar.gz on /opt/petsc > > > > Decompress the file > > $ cd /opt/petsc > > $ tar -xzvf petsc-2.3.3-p5.tar.gz > > > > I installed petscy for shared libraries and without X, beside this > > everything is normal: > > $ cd petsc-2.3.3-p5 > > $ export PETSC_ARCH=darwin8.10.1-c-debug > > $ export PETSC_DIR=/opt/petsc/petsc-2.3.3-p5 > > $ ./config/configure.py --with-mpi-dir=/usr/local/openmpi --with- > > shared=1 --with-x=0 > > $ make all test > > > > At the end of /etc/profile add (so it will be available for all the > > users): > > # variables for petsc > > export PETSC_ARCH=darwin8.10.1-c-debug > > export PETSC_DIR=/opt/petsc/petsc-2.3.3-p5 > > > > > > > > petsc4py > > ------------- > > Open a new terminal so the variables PETSC_ARCH and PETSC_DIR will be > > available. > > > > I installed petsc4py in /opt/petsc/petsc4py > > > > In a terminal > > $ cd /opt/petsc/petsc4py > > > > Get a copy of petsc4py from: http://code.google.com/p/petsc4py/ > > and save it at /opt/petsc/petsc4py > > > > $ tar -xvf petsc4py-0.7.5.tar > > $ cd petsc4py-0.7.5 > > $ python setup.py install --home=/opt/petsc/petsc4py > > > > # Add at the end of /etc/profile (it will be available for all the > > users) > > export PYTHONPATH=/opt/petsc/petsc4py/lib/python/ > > > > Now is possible to open a new terminal and test petsc4py: > > $python > > >>> from petsc4py import PETSc > > >>> A = PETSc.Mat() > > >>> A > > >>> A.createSeqAIJ(1000) > > > > > > > > > > Hope that this info is helpful! I tested the installation in my > > MacBook (Dual-core), in an aluminium iMac (Dual-core), > > and I hope to test the installation on a Mac pro (with two quad-core) > > some time this week! > > > > I'm quite new to the list so I'm sorry if you have already discussed > > this topic. > > > > > > Best regards, > > > > Felipe Cruz > > > > > > > > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which > their experiments lead. > -- Norbert Wiener > > From mossaiby at yahoo.com Wed Sep 12 05:07:32 2007 From: mossaiby at yahoo.com (Farshid Mossaiby) Date: Wed, 12 Sep 2007 03:07:32 -0700 (PDT) Subject: Sample code for Prometheus Message-ID: <65165.53736.qm@web52210.mail.re2.yahoo.com> Dear users / developers, Is there any sample code about Prometheus? And is it just a preconditioner, or it can be used as a solver? Or I am totally wrong? Regards, Farshid Mossaiby ____________________________________________________________________________________ Yahoo! oneSearch: Finally, mobile search that gives answers, not web links. http://mobile.yahoo.com/mobileweb/onesearch?refer=1ONXIC From bsmith at mcs.anl.gov Thu Sep 13 21:45:43 2007 From: bsmith at mcs.anl.gov (Barry Smith) Date: Thu, 13 Sep 2007 21:45:43 -0500 (CDT) Subject: Sample code for Prometheus In-Reply-To: <65165.53736.qm@web52210.mail.re2.yahoo.com> References: <65165.53736.qm@web52210.mail.re2.yahoo.com> Message-ID: On Wed, 12 Sep 2007, Farshid Mossaiby wrote: > Dear users / developers, > > Is there any sample code about Prometheus? It can be used just like any preconditioner in PETSc so does not need a particular example. You can also use PCSetCoordinates() to provide the coordinates of the grid points which Prometheus can use to generate a better preconditioner. >And is it > just a preconditioner, or it can be used as a solver? If you wish to use it "as a solver" that is, without a Krylov method, you simply use -ksp_type richardson. It is very unlikely this will do better in time then using it with GMRES or Bi-CG-stab but it is trivial to experiment to determine what is best for your problem. Barry > Or I am totally wrong? > > Regards, > Farshid Mossaiby > > > > ____________________________________________________________________________________ > Yahoo! oneSearch: Finally, mobile search > that gives answers, not web links. > http://mobile.yahoo.com/mobileweb/onesearch?refer=1ONXIC > > From bsmith at mcs.anl.gov Sat Sep 15 22:23:45 2007 From: bsmith at mcs.anl.gov (Barry Smith) Date: Sat, 15 Sep 2007 22:23:45 -0500 (CDT) Subject: petsc and matlab In-Reply-To: <1187698745.46cad8397378c@webmail.mail.gatech.edu> References: <1187641537.46c9f8c135de4@webmail.mail.gatech.edu> <1187698745.46cad8397378c@webmail.mail.gatech.edu> Message-ID: Alejandro, That is really weird, the convergence usually does not depend much on exactly what the right hand side is. Are you using a nonzero initial guess with Matlab? With PETSc? (With PETSc call KSPSetInitialGuessNonzero(). Can you run the PETSc one with -ksp_monitor so that you see how many iterations are needed each time, and get that information from Matlab to compare? Barry On Tue, 21 Aug 2007, Alejandro Garzon wrote: > Yes, the "A" matrix is the same. > -- > Alejandro > > > Quoting Barry Smith : > > > > > Is the matrix "A" the same for all groups (and the only thing > > different for each group is b)? > > > > Barry > > > > > > On Mon, 20 Aug 2007, Alejandro Garzon wrote: > > > > > Hi, I wrote a code for solving a time dependent pde. In each time step > > > I have to solve a linear system. I first wrote a prototype in matlab > > > and then a C version using Petsc. To my surprise for some input > > > parameters the matlab version runs faster than the Petsc code (on the > > > same single processor). I did > > > the performance comparison by timing 20 groups of 40 iterations. The > > > times for Petsc and matlab are shown bellow > > > > > > group Petsc matlab > > > > > > 1 0.244981 1.110283 > > > 2 0.244995 1.112919 > > > 3 0.241305 1.113608 > > > 4 0.244542 1.114669 > > > 5 7.534417 1.112450 > > > 6 0.242212 1.115867 > > > 7 0.246327 1.111135 > > > 8 0.241105 1.113442 > > > 9 0.244468 1.111215 > > > 10 0.241113 1.111334 > > > 11 0.244541 1.112467 > > > 12 0.241400 1.113525 > > > 13 0.245020 1.114077 > > > 14 0.241303 1.113409 > > > 15 0.244380 1.116238 > > > 16 0.241372 1.109931 > > > 17 0.244108 1.100667 > > > 18 0.240419 1.096030 > > > 19 0.244337 1.096293 > > > 20 17.139999 1.097120 > > > --------- -------- > > > total 29.0523 22.1967 > > > time > > > > > > As can be seen in the table, although in most of the groups the time > > > spent by the Petsc code is lower than that of matlab > > > (0.24 compared to 1.1) there are two groups (5 and 20) in the Petsc column > > that > > > take a long time and make the total for the Petsc code bigger than > > > that for matlab. > > > > > > In Petsc and matlab the method used is bicg and the relative residual > > > is the same: 1e-8. The preconditioners are different, though. In petsc > > > I used the default preconditioner and in matlab I used incomplete LU > > > decomposition with drop tolerance. The code that solves the linear > > > system in matlab is > > > > > > [L,U] = luinc(A,droptol); <---- this is done only once before the > > > first iteration, droptol = 1e-12 > > > > > > x = bicg(A,b,relres,maxsteps,L,U); > > > > > > I know incomplete LU decomposition as a preconditioner is available in > > > Petsc but in order to use it one must provide two arguments in > > > addition to the drop tolerance and I didn't know what values to give > > > to them. > > > > > > My question is this: what options should be chosen so that the > > > preconditioner and method use by Petsc are the same as those shown in > > > the two lines of matlab code above? (could you contact the authors of the > > > matlab functions?) A comparison of the performace of Petsc and matlab > > > makes sense only if they are using the exact same methods. > > > > > > > > > Thanks. > > > > > > -- > > > Alejandro > > > > > > > > > > > > > > > > > > > > > From bsmith at mcs.anl.gov Sat Sep 15 22:28:38 2007 From: bsmith at mcs.anl.gov (Barry Smith) Date: Sat, 15 Sep 2007 22:28:38 -0500 (CDT) Subject: petsc and matlab In-Reply-To: References: <1187641537.46c9f8c135de4@webmail.mail.gatech.edu> <1187698745.46cad8397378c@webmail.mail.gatech.edu> Message-ID: Forgot to mention, given all the odd tweaks people put into drop tolerance ILU's there is no way we could have exactly the same preconditioner as PETSc. It is possible using the "Matlab engine" to use Matlab to apply its own drop tolerance ILU(), this would not be useful for production, but could be used to confirm that the two codes do work the same. You must build PETSc with matlab engine (which is painful, run PETSc's config/configure.py with --help to see what options to use) then use -mat_type matlab to run. I don't recommend doing this unless you really care and have spare time on your hands. Barry On Sat, 15 Sep 2007, Barry Smith wrote: > > Alejandro, > > That is really weird, the convergence usually does > not depend much on exactly what the right hand side is. > > Are you using a nonzero initial guess with Matlab? > With PETSc? (With PETSc call KSPSetInitialGuessNonzero(). > > Can you run the PETSc one with -ksp_monitor so that > you see how many iterations are needed each time, and get that > information from Matlab to compare? > > Barry > > > On Tue, 21 Aug 2007, Alejandro Garzon wrote: > > > Yes, the "A" matrix is the same. > > -- > > Alejandro > > > > > > Quoting Barry Smith : > > > > > > > > Is the matrix "A" the same for all groups (and the only thing > > > different for each group is b)? > > > > > > Barry > > > > > > > > > On Mon, 20 Aug 2007, Alejandro Garzon wrote: > > > > > > > Hi, I wrote a code for solving a time dependent pde. In each time step > > > > I have to solve a linear system. I first wrote a prototype in matlab > > > > and then a C version using Petsc. To my surprise for some input > > > > parameters the matlab version runs faster than the Petsc code (on the > > > > same single processor). I did > > > > the performance comparison by timing 20 groups of 40 iterations. The > > > > times for Petsc and matlab are shown bellow > > > > > > > > group Petsc matlab > > > > > > > > 1 0.244981 1.110283 > > > > 2 0.244995 1.112919 > > > > 3 0.241305 1.113608 > > > > 4 0.244542 1.114669 > > > > 5 7.534417 1.112450 > > > > 6 0.242212 1.115867 > > > > 7 0.246327 1.111135 > > > > 8 0.241105 1.113442 > > > > 9 0.244468 1.111215 > > > > 10 0.241113 1.111334 > > > > 11 0.244541 1.112467 > > > > 12 0.241400 1.113525 > > > > 13 0.245020 1.114077 > > > > 14 0.241303 1.113409 > > > > 15 0.244380 1.116238 > > > > 16 0.241372 1.109931 > > > > 17 0.244108 1.100667 > > > > 18 0.240419 1.096030 > > > > 19 0.244337 1.096293 > > > > 20 17.139999 1.097120 > > > > --------- -------- > > > > total 29.0523 22.1967 > > > > time > > > > > > > > As can be seen in the table, although in most of the groups the time > > > > spent by the Petsc code is lower than that of matlab > > > > (0.24 compared to 1.1) there are two groups (5 and 20) in the Petsc column > > > that > > > > take a long time and make the total for the Petsc code bigger than > > > > that for matlab. > > > > > > > > In Petsc and matlab the method used is bicg and the relative residual > > > > is the same: 1e-8. The preconditioners are different, though. In petsc > > > > I used the default preconditioner and in matlab I used incomplete LU > > > > decomposition with drop tolerance. The code that solves the linear > > > > system in matlab is > > > > > > > > [L,U] = luinc(A,droptol); <---- this is done only once before the > > > > first iteration, droptol = 1e-12 > > > > > > > > x = bicg(A,b,relres,maxsteps,L,U); > > > > > > > > I know incomplete LU decomposition as a preconditioner is available in > > > > Petsc but in order to use it one must provide two arguments in > > > > addition to the drop tolerance and I didn't know what values to give > > > > to them. > > > > > > > > My question is this: what options should be chosen so that the > > > > preconditioner and method use by Petsc are the same as those shown in > > > > the two lines of matlab code above? (could you contact the authors of the > > > > matlab functions?) A comparison of the performace of Petsc and matlab > > > > makes sense only if they are using the exact same methods. > > > > > > > > > > > > Thanks. > > > > > > > > -- > > > > Alejandro > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > From amjad11 at gmail.com Sun Sep 16 06:07:10 2007 From: amjad11 at gmail.com (amjad ali) Date: Sun, 16 Sep 2007 16:07:10 +0500 Subject: Problem in Using PETSc examples In-Reply-To: References: <428810f20709092257s46f491f9q37079d4d5b0c43@mail.gmail.com> <428810f20709110032y379676d9rf19fca5917a4da84@mail.gmail.com> <428810f20709130319y490877c4y8f10ebe2928a13a5@mail.gmail.com> <428810f20709131057t4f4470a3m1fbde83aeec4800d@mail.gmail.com> <428810f20709131112w617224a5u17dfe582977ac05f@mail.gmail.com> Message-ID: <428810f20709160407u18c4e88ek7ba3f00b40362e5d@mail.gmail.com> Hello Dr. Matt and others at PETSc, I read few tutorials on makefile as you advised. It improved my understanding. Now I do the following to run the test example ex1: 1)cd ${PETSC_DIR}/src/ksp/ksp/examples/tutorials/ 2)make ex1 (it produces executable ex1) 3)[petsc1 at localhost tutorials]$ mpiexec ex1 bash: mpiexec: command not found 4)[petsc1 at localhost tutorials]$ ${PETSC_DIR}/externalpackages/ mpich2-1.0.5p4/${PETSC_ARCH}/bin/mpiexec ex1 Error in system call mpiexec: No such file or directory mpiexec could not exec ex1 I am puzzled with this situation. What should be the remedy? In added the following two lines (although somewhat extra) in the makefile : MPIEXEC = ${PETSC_DIR}/externalpackages/mpich2-1.0.5p4 /${PETSC_ARCH}/bin/mpiexec include ${PETSC_DIR}/bmake/${PETSC_ARCH}/petscconf Please tell me that is there any effect of placing "include" lines on different locations in the makefile (at the top or bottom of the file)? with best regards, Amjad Ali. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jeff.science at gmail.com Sun Sep 16 11:29:39 2007 From: jeff.science at gmail.com (Jeff Hammond) Date: Sun, 16 Sep 2007 09:29:39 -0700 Subject: Problem in Using PETSc examples In-Reply-To: <428810f20709160407u18c4e88ek7ba3f00b40362e5d@mail.gmail.com> References: <428810f20709092257s46f491f9q37079d4d5b0c43@mail.gmail.com> <428810f20709110032y379676d9rf19fca5917a4da84@mail.gmail.com> <428810f20709130319y490877c4y8f10ebe2928a13a5@mail.gmail.com> <428810f20709131057t4f4470a3m1fbde83aeec4800d@mail.gmail.com> <428810f20709131112w617224a5u17dfe582977ac05f@mail.gmail.com> <428810f20709160407u18c4e88ek7ba3f00b40362e5d@mail.gmail.com> Message-ID: <96f4bb620709160929n66d7d196je56400758b62fe9b@mail.gmail.com> Try this instead: > mpiexec ./ex1 It works for me: ------------------------------------------------------------------------------------------------ > mpiexec ex1 problem with execution of ex1 on davidson: [Errno 2] No such file or directory > mpiexec ./ex1 > mpiexec ex10 problem with execution of ex10 on davidson: [Errno 2] No such file or directory > mpiexec ./ex10 m = 1, N=81 Matrix Object: type=seqbaij, rows=36, cols=36 total: nonzeros=1200, allocated nonzeros=1200 block size is 1 matrix 1 norm = 34.0627 Norm of error < 1.e-12, Number of iterations 9 ------------------------------------------------------------------------------------------------ Best, Jeff From dalcinl at gmail.com Mon Sep 17 18:48:28 2007 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Mon, 17 Sep 2007 20:48:28 -0300 Subject: flexible TS implementation for user-defined timestepping Message-ID: Current TS implementations never fited my needs. Why? My target application is solving incompressible NS equations with FEM and fully implicit schemes (trapezoidal rule with theta in [0.5, 1.0] ) and residual-based stabilization (SUPG/PSPG). Then I need to solve at each time step a nonlinear problem like F(t, u, t0, u0), where F is nonlinear in u. I'm surelly naive, and I could not accomodate my function and Jacobian code for the beuler/cn TS implementations. Futhermore, I've recently convinced some coworkers to take advance of SNES/TS features and my Python wrappers (petsc4py) to manage the the time evolution of some problems related to fluid-structure interaction and mesh movement (formulated as an optimization problem, only nodal positions change, not remeshing needed for many timesteps). In order to support those applications setups, I've wrote a new TS implemetation, which enables users to (almost) completely define the temporal evolution of their problems, with pre/post solve/step methods and support for accept/reject steps and implementing timestep size control. Some of these features are available in some TS types, but not all-in-one. This code is available for review in petsc4py SVN repo (link below), an python example of this in action (very simple, just for testing all is working) is attached. Currently it only support implicit schemes and nonlinear problems, but I believe it can be extended to support explicit schemes and linear problems. http://petsc4py.googlecode.com/svn/trunk/petsc/lib/ext/src/ts/impls/implicit/user/ I want to know the opinion of PETSc core developers and potential users ot this, and I hope anyone can provide suggestions for improvements. After that, I can push this to petsc-dev for general availability (in fact, the code is almost ready to be integrated in petsc). Perhaps in the long term, all this can be integrated in the generic TS interface if that is appropriate. Of course, this would require some (I hope minor) changes in TS interface, some additions, and a reimplementation of some default TS types Regards, and I'm waiting for your comments. -- Lisandro Dalc?n --------------- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 -------------- next part -------------- A non-text attachment was scrubbed... Name: test_tsuser.py Type: text/x-python Size: 4014 bytes Desc: not available URL: From gtg100n at mail.gatech.edu Wed Sep 19 09:56:11 2007 From: gtg100n at mail.gatech.edu (Alejandro Garzon) Date: Wed, 19 Sep 2007 10:56:11 -0400 Subject: combining matrices In-Reply-To: References: <1187641537.46c9f8c135de4@webmail.mail.gatech.edu> <1187698745.46cad8397378c@webmail.mail.gatech.edu> Message-ID: <1190213771.46f1388be889c@webmail.mail.gatech.edu> Hi, Is there a function to combine two matrices, sizes m x n1 and m x n2, in a single matrix size m x (n1 + n2) or the analog case for rows? Thanks. Alejandro. From hzhang at mcs.anl.gov Wed Sep 19 10:07:23 2007 From: hzhang at mcs.anl.gov (Hong Zhang) Date: Wed, 19 Sep 2007 10:07:23 -0500 (CDT) Subject: combining matrices In-Reply-To: <1190213771.46f1388be889c@webmail.mail.gatech.edu> References: <1187641537.46c9f8c135de4@webmail.mail.gatech.edu> <1187698745.46cad8397378c@webmail.mail.gatech.edu> <1190213771.46f1388be889c@webmail.mail.gatech.edu> Message-ID: We do not have such function. You can create a new matrix C of size m x (n1 + n2), and do a loop: for each row: MatGetRow(A, row,...) MatSetValues(C,1,&row,...) MatGetRow(B, ) MatSetValues(C,1,&row,...) or create A and B with the size m x (n1 + n2), and get B=A+B by calling MatAXPY(), Hong On Wed, 19 Sep 2007, Alejandro Garzon wrote: > > Hi, Is there a function to combine two matrices, sizes m x n1 and m x n2, in a > single matrix size m x (n1 + n2) or the analog case for rows? Thanks. > > Alejandro. > > From knepley at gmail.com Wed Sep 19 11:33:22 2007 From: knepley at gmail.com (Matthew Knepley) Date: Wed, 19 Sep 2007 11:33:22 -0500 Subject: combining matrices In-Reply-To: <1190213771.46f1388be889c@webmail.mail.gatech.edu> References: <1187641537.46c9f8c135de4@webmail.mail.gatech.edu> <1187698745.46cad8397378c@webmail.mail.gatech.edu> <1190213771.46f1388be889c@webmail.mail.gatech.edu> Message-ID: This is, in general, not an efficient parallel operation. We do offer MatGetSubMatrix(), but for simple situations I would avoid it. Matt On 9/19/07, Alejandro Garzon wrote: > > Hi, Is there a function to combine two matrices, sizes m x n1 and m x n2, in a > single matrix size m x (n1 + n2) or the analog case for rows? Thanks. > > Alejandro. > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener From randy at geosystem.us Fri Sep 21 09:58:10 2007 From: randy at geosystem.us (Randall Mackie) Date: Fri, 21 Sep 2007 07:58:10 -0700 Subject: matrix inversion Message-ID: <46F3DC02.20503@geosystem.us> I have a collegue who is considering to use PETSc for programming his problem. He would like to know if you can get the actual inverse of a matrix, because this inverse is embedded in another larger system that is then solved. I suppose that if you solve the system directly, say with SuperLU or MUMPS, you could get the actual inverse? I looked briefly through the manual and documentation, but it didn't jump out at me. Any advice here? Thanks, Randy -- Randall Mackie GSY-USA, Inc. PMB# 643 2261 Market St., San Francisco, CA 94114-1600 Tel (415) 469-8649 Fax (415) 469-5044 California Registered Geophysicist License No. GP 1034 From knepley at gmail.com Fri Sep 21 10:06:47 2007 From: knepley at gmail.com (Matthew Knepley) Date: Fri, 21 Sep 2007 10:06:47 -0500 Subject: matrix inversion In-Reply-To: <46F3DC02.20503@geosystem.us> References: <46F3DC02.20503@geosystem.us> Message-ID: On 9/21/07, Randall Mackie wrote: > I have a collegue who is considering to use PETSc for programming > his problem. He would like to know if you can get the actual > inverse of a matrix, because this inverse is embedded in another > larger system that is then solved. > > I suppose that if you solve the system directly, say with SuperLU > or MUMPS, you could get the actual inverse? I looked briefly through > the manual and documentation, but it didn't jump out at me. This makes several time this week people have asked for inverses. Never never never use the inverse. Its not stable. The storage blows up. That is why factorizations were invented. Numerical analysis courses have failed the country entirely. Just use a KSP and tell it -ksp_type preonly -pc_type lu and then you can customize the matrix type for different solvers. KSPSolve() will apply the matrix. If you want to get the action on an entire matrix, this is dense and we would have to put in the BLAS3 triangular solve calls. Matt > Any advice here? > > Thanks, Randy > > -- > Randall Mackie > GSY-USA, Inc. > PMB# 643 > 2261 Market St., > San Francisco, CA 94114-1600 > Tel (415) 469-8649 > Fax (415) 469-5044 > > California Registered Geophysicist > License No. GP 1034 > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener From randy at geosystem.us Fri Sep 21 10:10:23 2007 From: randy at geosystem.us (Randall Mackie) Date: Fri, 21 Sep 2007 08:10:23 -0700 Subject: matrix inversion In-Reply-To: References: <46F3DC02.20503@geosystem.us> Message-ID: <46F3DEDF.9080307@geosystem.us> Matt, There may be very valid reasons why one would want a matrix inverse, say in the context of solving optimizations where you need the inverse of the covariance matrix, and you need those values to embed in another system. In this case, just getting the action of the inverse times a vector is not what is needed. Randy Matthew Knepley wrote: > On 9/21/07, Randall Mackie wrote: >> I have a collegue who is considering to use PETSc for programming >> his problem. He would like to know if you can get the actual >> inverse of a matrix, because this inverse is embedded in another >> larger system that is then solved. >> >> I suppose that if you solve the system directly, say with SuperLU >> or MUMPS, you could get the actual inverse? I looked briefly through >> the manual and documentation, but it didn't jump out at me. > > This makes several time this week people have asked for inverses. Never > never never use the inverse. Its not stable. The storage blows up. That is > why factorizations were invented. Numerical analysis courses have failed > the country entirely. Just use a KSP and tell it -ksp_type preonly -pc_type lu > and then you can customize the matrix type for different solvers. KSPSolve() > will apply the matrix. If you want to get the action on an entire matrix, this > is dense and we would have to put in the BLAS3 triangular solve calls. > > Matt > >> Any advice here? >> >> Thanks, Randy >> >> -- >> Randall Mackie >> GSY-USA, Inc. >> PMB# 643 >> 2261 Market St., >> San Francisco, CA 94114-1600 >> Tel (415) 469-8649 >> Fax (415) 469-5044 >> >> California Registered Geophysicist >> License No. GP 1034 >> >> > > -- Randall Mackie GSY-USA, Inc. PMB# 643 2261 Market St., San Francisco, CA 94114-1600 Tel (415) 469-8649 Fax (415) 469-5044 California Registered Geophysicist License No. GP 1034 From knepley at gmail.com Fri Sep 21 10:13:01 2007 From: knepley at gmail.com (Matthew Knepley) Date: Fri, 21 Sep 2007 10:13:01 -0500 Subject: matrix inversion In-Reply-To: <46F3DEDF.9080307@geosystem.us> References: <46F3DC02.20503@geosystem.us> <46F3DEDF.9080307@geosystem.us> Message-ID: On 9/21/07, Randall Mackie wrote: > Matt, > > There may be very valid reasons why one would want a matrix inverse, > say in the context of solving optimizations where you need the > inverse of the covariance matrix, and you need those values to > embed in another system. I do not agree. What are you going to do with the inverse that cannot be done with LU? Backsolves against a matrix are commonplace. Matt > In this case, just getting the action of the inverse times a vector > is not what is needed. > > Randy > > > Matthew Knepley wrote: > > On 9/21/07, Randall Mackie wrote: > >> I have a collegue who is considering to use PETSc for programming > >> his problem. He would like to know if you can get the actual > >> inverse of a matrix, because this inverse is embedded in another > >> larger system that is then solved. > >> > >> I suppose that if you solve the system directly, say with SuperLU > >> or MUMPS, you could get the actual inverse? I looked briefly through > >> the manual and documentation, but it didn't jump out at me. > > > > This makes several time this week people have asked for inverses. Never > > never never use the inverse. Its not stable. The storage blows up. That is > > why factorizations were invented. Numerical analysis courses have failed > > the country entirely. Just use a KSP and tell it -ksp_type preonly -pc_type lu > > and then you can customize the matrix type for different solvers. KSPSolve() > > will apply the matrix. If you want to get the action on an entire matrix, this > > is dense and we would have to put in the BLAS3 triangular solve calls. > > > > Matt > > > >> Any advice here? > >> > >> Thanks, Randy > >> > >> -- > >> Randall Mackie > >> GSY-USA, Inc. > >> PMB# 643 > >> 2261 Market St., > >> San Francisco, CA 94114-1600 > >> Tel (415) 469-8649 > >> Fax (415) 469-5044 > >> > >> California Registered Geophysicist > >> License No. GP 1034 > >> > >> > > > > > > -- > Randall Mackie > GSY-USA, Inc. > PMB# 643 > 2261 Market St., > San Francisco, CA 94114-1600 > Tel (415) 469-8649 > Fax (415) 469-5044 > > California Registered Geophysicist > License No. GP 1034 > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener From randy at geosystem.us Fri Sep 21 10:16:56 2007 From: randy at geosystem.us (Randall Mackie) Date: Fri, 21 Sep 2007 08:16:56 -0700 Subject: matrix inversion In-Reply-To: References: <46F3DC02.20503@geosystem.us> <46F3DEDF.9080307@geosystem.us> Message-ID: <46F3E068.6030708@geosystem.us> Matt, I suspect it is my collegue who asked the earlier questions about matrix inverses this week, although I didn't realize that until after I sent my original message. I also suggested he just get the LU factorizations, but he insists he needs the actual values to embed in another larger system he'll solve. I suppose that with the LU factorizations, you can just do successive solves with a vector with zeros and a 1 in the appropriate place to generate the elements you need, no? I've asked him for more clarification on what he needs exactly. Thanks, Randy Matthew Knepley wrote: > On 9/21/07, Randall Mackie wrote: >> Matt, >> >> There may be very valid reasons why one would want a matrix inverse, >> say in the context of solving optimizations where you need the >> inverse of the covariance matrix, and you need those values to >> embed in another system. > > I do not agree. What are you going to do with the inverse that cannot be > done with LU? Backsolves against a matrix are commonplace. > > Matt > >> In this case, just getting the action of the inverse times a vector >> is not what is needed. >> >> Randy >> >> >> Matthew Knepley wrote: >>> On 9/21/07, Randall Mackie wrote: >>>> I have a collegue who is considering to use PETSc for programming >>>> his problem. He would like to know if you can get the actual >>>> inverse of a matrix, because this inverse is embedded in another >>>> larger system that is then solved. >>>> >>>> I suppose that if you solve the system directly, say with SuperLU >>>> or MUMPS, you could get the actual inverse? I looked briefly through >>>> the manual and documentation, but it didn't jump out at me. >>> This makes several time this week people have asked for inverses. Never >>> never never use the inverse. Its not stable. The storage blows up. That is >>> why factorizations were invented. Numerical analysis courses have failed >>> the country entirely. Just use a KSP and tell it -ksp_type preonly -pc_type lu >>> and then you can customize the matrix type for different solvers. KSPSolve() >>> will apply the matrix. If you want to get the action on an entire matrix, this >>> is dense and we would have to put in the BLAS3 triangular solve calls. >>> >>> Matt >>> >>>> Any advice here? >>>> >>>> Thanks, Randy >>>> >>>> -- >>>> Randall Mackie >>>> GSY-USA, Inc. >>>> PMB# 643 >>>> 2261 Market St., >>>> San Francisco, CA 94114-1600 >>>> Tel (415) 469-8649 >>>> Fax (415) 469-5044 >>>> >>>> California Registered Geophysicist >>>> License No. GP 1034 >>>> >>>> >>> >> -- >> Randall Mackie >> GSY-USA, Inc. >> PMB# 643 >> 2261 Market St., >> San Francisco, CA 94114-1600 >> Tel (415) 469-8649 >> Fax (415) 469-5044 >> >> California Registered Geophysicist >> License No. GP 1034 >> >> > > -- Randall Mackie GSY-USA, Inc. PMB# 643 2261 Market St., San Francisco, CA 94114-1600 Tel (415) 469-8649 Fax (415) 469-5044 California Registered Geophysicist License No. GP 1034 From hzhang at mcs.anl.gov Fri Sep 21 10:19:01 2007 From: hzhang at mcs.anl.gov (Hong Zhang) Date: Fri, 21 Sep 2007 10:19:01 -0500 (CDT) Subject: matrix inversion In-Reply-To: <46F3DC02.20503@geosystem.us> References: <46F3DC02.20503@geosystem.us> Message-ID: On Fri, 21 Sep 2007, Randall Mackie wrote: > I have a collegue who is considering to use PETSc for programming > his problem. He would like to know if you can get the actual > inverse of a matrix, because this inverse is embedded in another > larger system that is then solved. Then you may have to - create a dense matrix Ainv - call MatLUFactorSymbolic() and MatLUFactorNumeric() to get A=LU - loop over matrix columns: MatSolve(LU,A(:,j),Ainv(:,j)); Note, if A is in row-compressed format, getting its columns is VERY inefficient. Creating A as a dense matrix and using Lapack or scalapack would give better result. > > I suppose that if you solve the system directly, say with SuperLU > or MUMPS, you could get the actual inverse? I looked briefly through > the manual and documentation, but it didn't jump out at me. You get matrix factors: A = LU. Then A*x = b is solved using MatSolve(): L*y = b and U*x = y. Again, sparse matrix A gives dense matrix inverse inv(A). For computational efficiency and stability, we try to avoid computing inv(A). Hong > > Any advice here? > > Thanks, Randy > > -- > Randall Mackie > GSY-USA, Inc. > PMB# 643 > 2261 Market St., > San Francisco, CA 94114-1600 > Tel (415) 469-8649 > Fax (415) 469-5044 > > California Registered Geophysicist > License No. GP 1034 > > From hzhang at mcs.anl.gov Fri Sep 21 10:22:30 2007 From: hzhang at mcs.anl.gov (Hong Zhang) Date: Fri, 21 Sep 2007 10:22:30 -0500 (CDT) Subject: matrix inversion In-Reply-To: <46F3E068.6030708@geosystem.us> References: <46F3DC02.20503@geosystem.us> <46F3DEDF.9080307@geosystem.us> <46F3E068.6030708@geosystem.us> Message-ID: On Fri, 21 Sep 2007, Randall Mackie wrote: > Matt, > > I suspect it is my collegue who asked the earlier questions about > matrix inverses this week, although I didn't realize that until > after I sent my original message. > > I also suggested he just get the LU factorizations, but he insists > he needs the actual values to embed in another larger system > he'll solve. Yes. In my previous email MatSolve(LU,A(:,j),Ainv(:,j)); should be MatSolve(LU,bj,Ainv(:,j)) where bj is a vector with zeros and a 1 in the j-th place. Hong > > I suppose that with the LU factorizations, you can just do > successive solves with a vector with zeros and a 1 in the appropriate > place to generate the elements you need, no? > > I've asked him for more clarification on what he needs exactly. > > Thanks, Randy > > > Matthew Knepley wrote: > > On 9/21/07, Randall Mackie wrote: > >> Matt, > >> > >> There may be very valid reasons why one would want a matrix inverse, > >> say in the context of solving optimizations where you need the > >> inverse of the covariance matrix, and you need those values to > >> embed in another system. > > > > I do not agree. What are you going to do with the inverse that cannot be > > done with LU? Backsolves against a matrix are commonplace. > > > > Matt > > > >> In this case, just getting the action of the inverse times a vector > >> is not what is needed. > >> > >> Randy > >> > >> > >> Matthew Knepley wrote: > >>> On 9/21/07, Randall Mackie wrote: > >>>> I have a collegue who is considering to use PETSc for programming > >>>> his problem. He would like to know if you can get the actual > >>>> inverse of a matrix, because this inverse is embedded in another > >>>> larger system that is then solved. > >>>> > >>>> I suppose that if you solve the system directly, say with SuperLU > >>>> or MUMPS, you could get the actual inverse? I looked briefly through > >>>> the manual and documentation, but it didn't jump out at me. > >>> This makes several time this week people have asked for inverses. Never > >>> never never use the inverse. Its not stable. The storage blows up. That is > >>> why factorizations were invented. Numerical analysis courses have failed > >>> the country entirely. Just use a KSP and tell it -ksp_type preonly -pc_type lu > >>> and then you can customize the matrix type for different solvers. KSPSolve() > >>> will apply the matrix. If you want to get the action on an entire matrix, this > >>> is dense and we would have to put in the BLAS3 triangular solve calls. > >>> > >>> Matt > >>> > >>>> Any advice here? > >>>> > >>>> Thanks, Randy > >>>> > >>>> -- > >>>> Randall Mackie > >>>> GSY-USA, Inc. > >>>> PMB# 643 > >>>> 2261 Market St., > >>>> San Francisco, CA 94114-1600 > >>>> Tel (415) 469-8649 > >>>> Fax (415) 469-5044 > >>>> > >>>> California Registered Geophysicist > >>>> License No. GP 1034 > >>>> > >>>> > >>> > >> -- > >> Randall Mackie > >> GSY-USA, Inc. > >> PMB# 643 > >> 2261 Market St., > >> San Francisco, CA 94114-1600 > >> Tel (415) 469-8649 > >> Fax (415) 469-5044 > >> > >> California Registered Geophysicist > >> License No. GP 1034 > >> > >> > > > > > > -- > Randall Mackie > GSY-USA, Inc. > PMB# 643 > 2261 Market St., > San Francisco, CA 94114-1600 > Tel (415) 469-8649 > Fax (415) 469-5044 > > California Registered Geophysicist > License No. GP 1034 > > From errbj at yahoo.com.br Fri Sep 21 15:11:11 2007 From: errbj at yahoo.com.br (Eduardo Brito Jr.) Date: Fri, 21 Sep 2007 13:11:11 -0700 (PDT) Subject: Help using PETSC Message-ID: <107329.74063.qm@web50911.mail.re2.yahoo.com> Dear colleagues, I'm starting to use the PETSC but I'm having problem when I'm trying to creat a simple C/C++ program, does exist any tutorial that explain how I creat a new C/C++ program ? Thanks for the attention, Eduardo Brito Junior === "It's something unpredictable, but in the end is right. I hope you had the time of your life." Eduardo Roberto Rodrigues de Brito Junior Contato:errbj at yahoo.com.br ICQ: 124914262 Flickr agora em portugu?s. Voc? clica, todo mundo v?. http://www.flickr.com.br/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From balay at mcs.anl.gov Fri Sep 21 15:30:48 2007 From: balay at mcs.anl.gov (Satish Balay) Date: Fri, 21 Sep 2007 15:30:48 -0500 (CDT) Subject: Help using PETSC In-Reply-To: <107329.74063.qm@web50911.mail.re2.yahoo.com> References: <107329.74063.qm@web50911.mail.re2.yahoo.com> Message-ID: There is extensive doucmentation on PETSc, and a users manual at: http://www-unix.mcs.anl.gov/petsc/petsc-as/documentation/index.html You might start with a relavent PETSc example - and start modifying it according to your needs. For eg: you might want find a relavent example from: http://www-unix.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/help.html Satish On Fri, 21 Sep 2007, Eduardo Brito Jr. wrote: > Dear colleagues, I'm starting to use the PETSC but I'm having problem when I'm trying to creat a simple C/C++ program, does exist any tutorial that explain how I creat a new C/C++ program ? Thanks for the attention, Eduardo Brito Junior === "It's something unpredictable, but in the end is right. I hope you had the time of your life." Eduardo Roberto Rodrigues de Brito Junior Contato:errbj at yahoo.com.br ICQ: 124914262 Flickr agora em portugu?s. Voc? clica, todo mundo v?. http://www.flickr.com.br/ From bsmith at mcs.anl.gov Fri Sep 21 15:53:52 2007 From: bsmith at mcs.anl.gov (Barry Smith) Date: Fri, 21 Sep 2007 15:53:52 -0500 (CDT) Subject: matrix inversion In-Reply-To: <46F3E068.6030708@geosystem.us> References: <46F3DC02.20503@geosystem.us> <46F3DEDF.9080307@geosystem.us> <46F3E068.6030708@geosystem.us> Message-ID: Randy, I have sent your user an example of how it may be done sequentially. We can add support for doing it in parallel but I doubt that his matrix is big enough to need it (more than say 2000 by 2000). Barry On Fri, 21 Sep 2007, Randall Mackie wrote: > Matt, > > I suspect it is my collegue who asked the earlier questions about > matrix inverses this week, although I didn't realize that until > after I sent my original message. > > I also suggested he just get the LU factorizations, but he insists > he needs the actual values to embed in another larger system > he'll solve. > > I suppose that with the LU factorizations, you can just do > successive solves with a vector with zeros and a 1 in the appropriate > place to generate the elements you need, no? > > I've asked him for more clarification on what he needs exactly. > > Thanks, Randy > > > Matthew Knepley wrote: > > On 9/21/07, Randall Mackie wrote: > > > Matt, > > > > > > There may be very valid reasons why one would want a matrix inverse, > > > say in the context of solving optimizations where you need the > > > inverse of the covariance matrix, and you need those values to > > > embed in another system. > > > > I do not agree. What are you going to do with the inverse that cannot be > > done with LU? Backsolves against a matrix are commonplace. > > > > Matt > > > > > In this case, just getting the action of the inverse times a vector > > > is not what is needed. > > > > > > Randy > > > > > > > > > Matthew Knepley wrote: > > > > On 9/21/07, Randall Mackie wrote: > > > > > I have a collegue who is considering to use PETSc for programming > > > > > his problem. He would like to know if you can get the actual > > > > > inverse of a matrix, because this inverse is embedded in another > > > > > larger system that is then solved. > > > > > > > > > > I suppose that if you solve the system directly, say with SuperLU > > > > > or MUMPS, you could get the actual inverse? I looked briefly through > > > > > the manual and documentation, but it didn't jump out at me. > > > > This makes several time this week people have asked for inverses. Never > > > > never never use the inverse. Its not stable. The storage blows up. That > > > > is > > > > why factorizations were invented. Numerical analysis courses have failed > > > > the country entirely. Just use a KSP and tell it -ksp_type preonly > > > > -pc_type lu > > > > and then you can customize the matrix type for different solvers. > > > > KSPSolve() > > > > will apply the matrix. If you want to get the action on an entire > > > > matrix, this > > > > is dense and we would have to put in the BLAS3 triangular solve calls. > > > > > > > > Matt > > > > > > > > > Any advice here? > > > > > > > > > > Thanks, Randy > > > > > > > > > > -- > > > > > Randall Mackie > > > > > GSY-USA, Inc. > > > > > PMB# 643 > > > > > 2261 Market St., > > > > > San Francisco, CA 94114-1600 > > > > > Tel (415) 469-8649 > > > > > Fax (415) 469-5044 > > > > > > > > > > California Registered Geophysicist > > > > > License No. GP 1034 > > > > > > > > > > > > > > > > > -- > > > Randall Mackie > > > GSY-USA, Inc. > > > PMB# 643 > > > 2261 Market St., > > > San Francisco, CA 94114-1600 > > > Tel (415) 469-8649 > > > Fax (415) 469-5044 > > > > > > California Registered Geophysicist > > > License No. GP 1034 > > > > > > > > > > > > From randy at geosystem.us Fri Sep 21 15:59:33 2007 From: randy at geosystem.us (Randall Mackie) Date: Fri, 21 Sep 2007 13:59:33 -0700 Subject: matrix inversion In-Reply-To: References: <46F3DC02.20503@geosystem.us> <46F3DEDF.9080307@geosystem.us> <46F3E068.6030708@geosystem.us> Message-ID: <46F430B5.6050306@geosystem.us> Thanks Barry, I didn't realize you guys had already been in communication, but now that I know that, perhaps I can help him by looking at what you sent. Randy Barry Smith wrote: > Randy, > > I have sent your user an example of how it may be done sequentially. > We can add support for doing it in parallel but I doubt that his matrix > is big enough to need it (more than say 2000 by 2000). > > Barry > > > On Fri, 21 Sep 2007, Randall Mackie wrote: > >> Matt, >> >> I suspect it is my collegue who asked the earlier questions about >> matrix inverses this week, although I didn't realize that until >> after I sent my original message. >> >> I also suggested he just get the LU factorizations, but he insists >> he needs the actual values to embed in another larger system >> he'll solve. >> >> I suppose that with the LU factorizations, you can just do >> successive solves with a vector with zeros and a 1 in the appropriate >> place to generate the elements you need, no? >> >> I've asked him for more clarification on what he needs exactly. >> >> Thanks, Randy >> >> >> Matthew Knepley wrote: >>> On 9/21/07, Randall Mackie wrote: >>>> Matt, >>>> >>>> There may be very valid reasons why one would want a matrix inverse, >>>> say in the context of solving optimizations where you need the >>>> inverse of the covariance matrix, and you need those values to >>>> embed in another system. >>> I do not agree. What are you going to do with the inverse that cannot be >>> done with LU? Backsolves against a matrix are commonplace. >>> >>> Matt >>> >>>> In this case, just getting the action of the inverse times a vector >>>> is not what is needed. >>>> >>>> Randy >>>> >>>> >>>> Matthew Knepley wrote: >>>>> On 9/21/07, Randall Mackie wrote: >>>>>> I have a collegue who is considering to use PETSc for programming >>>>>> his problem. He would like to know if you can get the actual >>>>>> inverse of a matrix, because this inverse is embedded in another >>>>>> larger system that is then solved. >>>>>> >>>>>> I suppose that if you solve the system directly, say with SuperLU >>>>>> or MUMPS, you could get the actual inverse? I looked briefly through >>>>>> the manual and documentation, but it didn't jump out at me. >>>>> This makes several time this week people have asked for inverses. Never >>>>> never never use the inverse. Its not stable. The storage blows up. That >>>>> is >>>>> why factorizations were invented. Numerical analysis courses have failed >>>>> the country entirely. Just use a KSP and tell it -ksp_type preonly >>>>> -pc_type lu >>>>> and then you can customize the matrix type for different solvers. >>>>> KSPSolve() >>>>> will apply the matrix. If you want to get the action on an entire >>>>> matrix, this >>>>> is dense and we would have to put in the BLAS3 triangular solve calls. >>>>> >>>>> Matt >>>>> >>>>>> Any advice here? >>>>>> >>>>>> Thanks, Randy >>>>>> >>>>>> -- >>>>>> Randall Mackie >>>>>> GSY-USA, Inc. >>>>>> PMB# 643 >>>>>> 2261 Market St., >>>>>> San Francisco, CA 94114-1600 >>>>>> Tel (415) 469-8649 >>>>>> Fax (415) 469-5044 >>>>>> >>>>>> California Registered Geophysicist >>>>>> License No. GP 1034 >>>>>> >>>>>> >>>> -- >>>> Randall Mackie >>>> GSY-USA, Inc. >>>> PMB# 643 >>>> 2261 Market St., >>>> San Francisco, CA 94114-1600 >>>> Tel (415) 469-8649 >>>> Fax (415) 469-5044 >>>> >>>> California Registered Geophysicist >>>> License No. GP 1034 >>>> >>>> >>> >> > -- Randall Mackie GSY-USA, Inc. PMB# 643 2261 Market St., San Francisco, CA 94114-1600 Tel (415) 469-8649 Fax (415) 469-5044 California Registered Geophysicist License No. GP 1034 From tyoung at ippt.gov.pl Thu Sep 27 07:52:53 2007 From: tyoung at ippt.gov.pl (Toby D. Young) Date: Thu, 27 Sep 2007 14:52:53 +0200 Subject: wierd behaviour from MatGetRow? Message-ID: <20070927145253.570e1747@rav.ippt.gov.pl> Hello, I apologize in advance if this message is garbled. I'm trying to set non-zero values of a column to zero, so I do the following: MatGetRow() <- get the column numbers of non-zero elements MatSetValues() <- set those elements to zero MatGetRow() <- check that element is set to zero The output from my program gives: num = 0x56f024 <- pointer to column number values = 0.666667 <- it's value num = 0x56f024 <- pointer to column number values = 0 <- it's value So it looks like it has worked the matrix value is zero! ...but if the matrix element has been set to zero, why does MatGetRow tell mne this? Should not MatGetRow see that it is zero and return nothing? Is this behaviour be correct? I'd be interested to know it seems like a riddle to me. Thanks. Best, Toby -- Toby D. Young ( Adiunkt ) Department of Computational Science Institute of Fundamental Technological Research, Polish Academy of Sciences, Room 206, Swietokrzyska 21, 00-049 Warsaw, POLAND From knepley at gmail.com Thu Sep 27 08:14:37 2007 From: knepley at gmail.com (Matthew Knepley) Date: Thu, 27 Sep 2007 08:14:37 -0500 Subject: wierd behaviour from MatGetRow? In-Reply-To: <20070927145253.570e1747@rav.ippt.gov.pl> References: <20070927145253.570e1747@rav.ippt.gov.pl> Message-ID: On 9/27/07, Toby D. Young wrote: > Hello, > > I apologize in advance if this message is garbled. > > I'm trying to set non-zero values of a column to zero, so I do the > following: > > MatGetRow() <- get the column numbers of non-zero elements > MatSetValues() <- set those elements to zero > MatGetRow() <- check that element is set to zero > > The output from my program gives: > > num = 0x56f024 <- pointer to column number > values = 0.666667 <- it's value > num = 0x56f024 <- pointer to column number > values = 0 <- it's value > > So it looks like it has worked the matrix value is zero! > > ...but if the matrix element has been set to zero, why does MatGetRow > tell mne this? Should not MatGetRow see that it is zero and return > nothing? > > Is this behaviour be correct? I'd be interested to know it seems like > a riddle to me. Yes, this is correct. The default matrix type, AIJ, distinguishes between a) Structural zeros: Elements not present in the sparse matrix graph b) Numerical zeros: Elements that have value 0.0 PETSc matrices are not designed to change the nonzero structure once set. This optimizes the MatMult time at the expense of more complicated assembly. This tradeoff is usually the correct one for PDEs. If you are trying to impose Dirichlet conditions, I suggest the following. Execute a first loop to calculate just the nonzero structure of the operator, incorporating all boundary conditions. Then execute a second loop to calculate and store the values. The time increase is usually negligible compared to the total simulation. Thanks, Matt > Thanks. > > Best, > Toby > > > > -- > > Toby D. Young ( Adiunkt ) > Department of Computational Science > Institute of Fundamental Technological Research, Polish Academy of > Sciences, Room 206, Swietokrzyska 21, 00-049 Warsaw, POLAND > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener From bsmith at mcs.anl.gov Thu Sep 27 09:40:43 2007 From: bsmith at mcs.anl.gov (Barry Smith) Date: Thu, 27 Sep 2007 09:40:43 -0500 (CDT) Subject: wierd behaviour from MatGetRow? In-Reply-To: <20070927145253.570e1747@rav.ippt.gov.pl> References: <20070927145253.570e1747@rav.ippt.gov.pl> Message-ID: Toby, From the manual page for MatGetRow() Caution: Do not try to change the contents of the output arrays (cols and vals). In some cases, this may corrupt the matrix. You can use MatZeroRows() to zero out some rows. Similarly you can use MatGetSubmatrix() to extract out certain rows and columns that you wish to solve with. Barry On Thu, 27 Sep 2007, Toby D. Young wrote: > > > > > Hello, > > I apologize in advance if this message is garbled. > > I'm trying to set non-zero values of a column to zero, so I do the > following: > > MatGetRow() <- get the column numbers of non-zero elements > MatSetValues() <- set those elements to zero > MatGetRow() <- check that element is set to zero > > The output from my program gives: > > num = 0x56f024 <- pointer to column number > values = 0.666667 <- it's value > num = 0x56f024 <- pointer to column number > values = 0 <- it's value > > So it looks like it has worked the matrix value is zero! > > ...but if the matrix element has been set to zero, why does MatGetRow > tell mne this? Should not MatGetRow see that it is zero and return > nothing? > > Is this behaviour be correct? I'd be interested to know it seems like > a riddle to me. > > Thanks. > > Best, > Toby > > > > From li76pan at yahoo.com Fri Sep 28 05:53:44 2007 From: li76pan at yahoo.com (li pan) Date: Fri, 28 Sep 2007 03:53:44 -0700 (PDT) Subject: problem with install Message-ID: <631646.37309.qm@web36812.mail.mud.yahoo.com> Dear developers, I've got an error at installation. "Could not use downloaded f-blas-lapack?" I sent the configure file to the maintaning list, hope you can help me. thanx pan ____________________________________________________________________________________ Catch up on fall's hot new shows on Yahoo! TV. Watch previews, get listings, and more! http://tv.yahoo.com/collections/3658 From tyoung at ippt.gov.pl Fri Sep 28 08:47:37 2007 From: tyoung at ippt.gov.pl (Toby D. Young) Date: Fri, 28 Sep 2007 15:47:37 +0200 Subject: Simple question Message-ID: <20070928154737.00b8a175@rav.ippt.gov.pl> Hello, For the function MatIsSymmetric(), is it correct to state that PetscTruth returns: 1 - true (symmetric) 0 - false (nonsymmetric) This is the output I get from test on an Identity matrix. A simple question perhaps, but I have really confusing results for other matrices which are needed to be symmetry verified. Thanks! Best, Toby -- Toby D. Young ( Adiunkt ) Department of Computational Science Institute of Fundamental Technological Research, Polish Academy of Sciences, Room 206, Swietokrzyska 21, 00-049 Warsaw, POLAND From knepley at gmail.com Fri Sep 28 08:52:06 2007 From: knepley at gmail.com (Matthew Knepley) Date: Fri, 28 Sep 2007 08:52:06 -0500 Subject: Simple question In-Reply-To: <20070928154737.00b8a175@rav.ippt.gov.pl> References: <20070928154737.00b8a175@rav.ippt.gov.pl> Message-ID: On 9/28/07, Toby D. Young wrote: > Hello, > > For the function MatIsSymmetric(), > is it correct to state that PetscTruth returns: > 1 - true (symmetric) > 0 - false (nonsymmetric) Technically, PETSC_TRUE and PETSC_FALSE. > This is the output I get from test on an Identity matrix. > > A simple question perhaps, but I have really confusing results for other > matrices which are needed to be symmetry verified. I assume that you have (supposedly) symmetric matrices that are coming up nonsymmetric? You could put a break point in MatIsTranspose_SeqAIJ() and see which row is disputed. Alternatively, you could use MatTranspose() and then output them both with the ASCII viewer and diff. Perhaps your tolerance is too low. Matt > Thanks! > > Best, > Toby > > -- > > Toby D. Young ( Adiunkt ) > Department of Computational Science > Institute of Fundamental Technological Research, Polish Academy of > Sciences, Room 206, Swietokrzyska 21, 00-049 Warsaw, POLAND > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener From mfhoel at ifi.uio.no Sun Sep 30 12:06:50 2007 From: mfhoel at ifi.uio.no (Mads Hoel) Date: Sun, 30 Sep 2007 19:06:50 +0200 Subject: Constructing a PETSc MatMPIAIJ matrix from partitioned arrays Message-ID: Hi all, I have a partitioned matrix on CRS format across several processors. I have 4 arrays describing the parallel matrix; the typical CRS arrays irow, jcol and values, and an array describing the mapping of local nodes/rows to global nodes/rows. jcol is in local indices. 1. Does Petsc support overlap? 2. What is the optimal way of creating a PETSc MatMPIAIJ from these arrays? I looked at MatInsertValues, but would not that create overhead in re-partitioning/distributing the arrays? Thanks in advance. -- Mads Hoel From knepley at gmail.com Sun Sep 30 13:48:12 2007 From: knepley at gmail.com (Matthew Knepley) Date: Sun, 30 Sep 2007 13:48:12 -0500 Subject: Constructing a PETSc MatMPIAIJ matrix from partitioned arrays In-Reply-To: References: Message-ID: On 9/30/07, Mads Hoel wrote: > Hi all, > > I have a partitioned matrix on CRS format across several processors. I > have 4 arrays describing the parallel matrix; the typical CRS arrays irow, > jcol and values, and an array describing the mapping of local nodes/rows > to global nodes/rows. jcol is in local indices. > > 1. Does Petsc support overlap? I do not understand what you mean by "overlap". > 2. What is the optimal way of creating a PETSc MatMPIAIJ from these > arrays? I looked at MatInsertValues, but would not that create overhead in > re-partitioning/distributing the arrays? PETSc partitions rows across processors. You can specify the local sizes when you create the matrix. You may have to renumber your representation in order to get contiguous row blocks on each process. Then preallocate the matrix since you know the row lengths. Then call MatSetValues() for each row. Note that PETSc will want global row/col numbers. Thanks, Matt > Thanks in advance. > > -- > Mads Hoel > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener