From neckel at in.tum.de Tue Jul 1 05:53:07 2008 From: neckel at in.tum.de (Tobias Neckel) Date: Tue, 01 Jul 2008 12:53:07 +0200 Subject: Question on MatPtAP Message-ID: <486A0C93.8040804@in.tum.de> Hello, when moving from PETSc 2.3.2-p10 to 2.3.3-p13, I recently encountered a strange behaviour (segmentation fault) of the new version concerning the usage of MatPtAP. I tried to pull everything down to a simple test case. For square matrices (A as well as P), everything seems to work fine. But when I set up a simple non-square example (A being the 3x3-identity, P being a 3x5 matrix, see attached file PETScLibTest.cpp), I encounter severe problems: As soon as P gets entries outside its 3x3 block (an entry P(0,3), e.g.), PETSc is telling me about Memory corruption while using MatPtAP (see part of command line output in the attached file output.txt, run with -info). I used valgrind to check if sth. weird happens, but it is giving nothing until the PETSc error message. Using the same test case with PETSc 2.3.2-p10 (with identic configuring options on the same machine) does not show any problems at all. So I am a bit confused. Is it not allowed any longer to use MatPtAP with non-square matrices? I checked the online documentation but did not find anything ... Next idea was to use MatMatMult directly to see if that works. So I used MatMatMultTranspose(P, A, MAT_INITIAL_MATRIX, 1.0, &C1) and then MatMatMult(C1, P, MAT_INITIAL_MATRIX, 1.0, &C) to get C=C1*P=P^T*A*P in two steps. This time, everything works fine for the small test case from above, also with the new version 2.3.3-p13. Best regards Tobias Neckel -- Dipl.-Tech. Math. Tobias Neckel Institut f?r Informatik V, TU M?nchen Boltzmannstr. 3, 85748 Garching Tel.: 089/289-18602 Email: neckel at in.tum.de URL: http://www5.in.tum.de/persons/neckel.html -------------- next part -------------- A non-text attachment was scrubbed... Name: PETScLibTest.cpp Type: text/x-c++src Size: 3257 bytes Desc: not available URL: -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: output.txt URL: From knepley at gmail.com Tue Jul 1 08:07:37 2008 From: knepley at gmail.com (Matthew Knepley) Date: Tue, 1 Jul 2008 08:07:37 -0500 Subject: Question on MatPtAP In-Reply-To: <486A0C93.8040804@in.tum.de> References: <486A0C93.8040804@in.tum.de> Message-ID: I just ran this with petsc-dev and valgrind, and got no errors. Matt On Tue, Jul 1, 2008 at 5:53 AM, Tobias Neckel wrote: > Hello, > > when moving from PETSc 2.3.2-p10 to 2.3.3-p13, I recently encountered a > strange behaviour (segmentation fault) of the new version concerning the > usage of MatPtAP. I tried to pull everything down to a simple test case. For > square matrices (A as well as P), everything seems to work fine. > > But when I set up a simple non-square example (A being the 3x3-identity, P > being a 3x5 matrix, see attached file PETScLibTest.cpp), I encounter severe > problems: As soon as P gets entries outside its 3x3 block (an entry P(0,3), > e.g.), PETSc is telling me about Memory corruption while using MatPtAP (see > part of command line output in the attached file output.txt, run with > -info). > > I used valgrind to check if sth. weird happens, but it is giving nothing > until the PETSc error message. > > Using the same test case with PETSc 2.3.2-p10 (with identic configuring > options on the same machine) does not show any problems at all. > > So I am a bit confused. Is it not allowed any longer to use MatPtAP with > non-square matrices? I checked the online documentation but did not find > anything ... > > Next idea was to use MatMatMult directly to see if that works. So I used > MatMatMultTranspose(P, A, MAT_INITIAL_MATRIX, 1.0, &C1) and then > MatMatMult(C1, P, MAT_INITIAL_MATRIX, 1.0, &C) > to get C=C1*P=P^T*A*P in two steps. This time, everything works fine for the > small test case from above, also with the new version 2.3.3-p13. > > > Best regards > Tobias Neckel > > -- > Dipl.-Tech. Math. Tobias Neckel > > Institut f?r Informatik V, TU M?nchen > Boltzmannstr. 3, 85748 Garching > > Tel.: 089/289-18602 > Email: neckel at in.tum.de > URL: http://www5.in.tum.de/persons/neckel.html > > 11:16:11 info UserInterface::writeHeader() > peano - jafs (Just Another Fluid Solver) > 11:16:11 info UserInterface::writeHeader() > 2d, loaded components: stacks geometry plotter trivialgrid fluid > fluid-scenario steady-state(fluid) turbulence(fluid) grid petsc > 11:16:11 info UserInterface::writeHeader() > $Revision: 1.82 $ > 11:16:11 info UserInterface::writeHeader() > 2005 - 2008 Tobias Neckel, Tobias Weinzierl, TUM, Informatik V > 11:16:11 debug /::validateProgramArguments(int,char**) > Using configuration file config-files/configurationTests.xml > --32741-- REDIR: 0x4861F5C (operator new[](unsigned)) redirected to > 0x40219AB (operator new[](unsigned)) > --32741-- REDIR: 0x4860B08 (operator delete[](void*)) redirected to > 0x4020ABA (operator delete[](void*)) > 11:16:12 debug petsc::PETScLibTest::testMatPreallocation() > start PETSc mat preallocation test > 11:16:12 debug petsc::PETScLibTest::testMatPreallocation() > stop PETSc mat preallocation test > 11:16:12 debug petsc::PETScLibTest::testPTAPQuadratic() > start PETSc PTAP test 1 > 11:16:13 debug petsc::PETScLibTest::testPTAPQuadratic() > start test 2 > 11:16:13 debug petsc::PETScLibTest::testPTAPQuadratic() > stop PETSc PTAP test > 11:16:13 debug petsc::PETScLibTest::testPTAPRectangular() > start PETSc PTAP rectangular test > [0] PetscCommDuplicate(): Duplicating a communicator 1140850688 -2080374784 > max tags = 2147483647 > [0] PetscCommDuplicate(): returning tag 2147483647 > [0] MatAssemblyEnd_SeqAIJ(): Matrix size: 3 X 3; storage space: 0 unneeded,3 > used > [0] MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues() is 0 > [0] MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 1 > [0] Mat_CheckInode(): Found 3 nodes out of 3 rows. Not using Inode routines > [0] PetscCommDuplicate(): Using internal PETSc communicator 1140850688 > -2080374784 > [0] PetscCommDuplicate(): returning tag 2147483646 > [0] MatAssemblyEnd_SeqAIJ(): Matrix size: 3 X 5; storage space: 5 > unneeded,10 used > [0] MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues() is 0 > [0] MatAssemblyEnd_SeqAIJ(): Maximum nonzeros in any row is 4 > [0] Mat_CheckInode(): Found 2 nodes of 3. Limit used: 5. Using Inode > routines > [0] MatGetSymbolicTranspose_SeqIJ(): Getting Symbolic Transpose. > [0]PETSC ERROR: PetscTrFreeDefault() called from PetscFreeSpaceContiguous() > line 43 in src/mat/utils/freespace.c > [0]PETSC ERROR: Block [id=0(12)] at address 0x4e9cd20 is corrupted (probably > write past end of array) > [0]PETSC ERROR: Block allocated in PetscFreeSpaceGet() line 14 in > src/mat/utils/freespace.c > [0]PETSC ERROR: --------------------- Error Message > ------------------------------------ > [0]PETSC ERROR: Memory corruption! > [0]PETSC ERROR: Corrupted memory! > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: Petsc Release Version 2.3.3, Patch 13, Thu May 15 17:29:26 > CDT 2008 HG revision: 4466c6289a0922df26e20626fd4a0b4dd03c8124 > [0]PETSC ERROR: See docs/changes/index.html for recent updates. > [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. > [0]PETSC ERROR: See docs/index.html for manual pages. > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: Debug/peano_stud on a linux-gnu named atsccs29 by neckel Tue > Jul 1 11:16:11 2008 > [0]PETSC ERROR: Libraries linked from > /home/neckel/Programme/petsc-2.3.3-p13/lib/linux-gnu-c-real-debug-gentoo-NOmpi-080523 > [0]PETSC ERROR: Configure run at Fri May 23 13:44:27 2008 > [0]PETSC ERROR: Configure options > --PETSC_ARCH=linux-gnu-c-real-debug-gentoo-NOmpi-080523 > --prefix=/home/neckel/Programme/petsc-2.3.3-p13 --with-mpi-dir=/usr/ > --with-mpi=0 --with-debugging=yes --with-clanguage=C++ --with-cxx=g++ > --with-shared=0 --with-fortran-kernels=generic > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: PetscTrFreeDefault() line 303 in src/sys/memory/mtr.c > [0]PETSC ERROR: PetscFreeSpaceContiguous() line 43 in > src/mat/utils/freespace.c > [0]PETSC ERROR: MatPtAPSymbolic_SeqAIJ_SeqAIJ() line 129 in > src/mat/impls/aij/seq/matptap.c > [0]PETSC ERROR: MatPtAPSymbolic_SeqAIJ() line 22 in > src/mat/impls/aij/seq/matptap.c > [0]PETSC ERROR: MatPtAPSymbolic() line 6689 in src/mat/interface/matrix.c > [0]PETSC ERROR: MatPtAP_Basic() line 13 in src/mat/utils/ptap.c > [0]PETSC ERROR: MatPtAP() line 6582 in src/mat/interface/matrix.c > 11:16:13 warning petscError::logPETScError > Attention: no PetscErrorBaseMessage imitated! > 11:16:13 error petscError::logPetscTraceBackErrorHandler > Major PETSc error occured in function > petsc::PETScLibTest::testPTAPRectangular, line 591, dir. petsc/tests, file > ../src/petsc/tests/PETScLibTest.cpp. > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener From hzhang at mcs.anl.gov Tue Jul 1 09:33:27 2008 From: hzhang at mcs.anl.gov (Hong Zhang) Date: Tue, 1 Jul 2008 09:33:27 -0500 (CDT) Subject: Question on MatPtAP In-Reply-To: <486A0C93.8040804@in.tum.de> References: <486A0C93.8040804@in.tum.de> Message-ID: MatPtAP() should work for rectangular P. I'll test your code and let you know what I get, hopefully later this evening, Hong On Tue, 1 Jul 2008, Tobias Neckel wrote: > Hello, > > when moving from PETSc 2.3.2-p10 to 2.3.3-p13, I recently encountered a > strange behaviour (segmentation fault) of the new version concerning the > usage of MatPtAP. I tried to pull everything down to a simple test case. For > square matrices (A as well as P), everything seems to work fine. > > But when I set up a simple non-square example (A being the 3x3-identity, P > being a 3x5 matrix, see attached file PETScLibTest.cpp), I encounter severe > problems: As soon as P gets entries outside its 3x3 block (an entry P(0,3), > e.g.), PETSc is telling me about Memory corruption while using MatPtAP (see > part of command line output in the attached file output.txt, run with -info). > > I used valgrind to check if sth. weird happens, but it is giving nothing > until the PETSc error message. > > Using the same test case with PETSc 2.3.2-p10 (with identic configuring > options on the same machine) does not show any problems at all. > > So I am a bit confused. Is it not allowed any longer to use MatPtAP with > non-square matrices? I checked the online documentation but did not find > anything ... > > Next idea was to use MatMatMult directly to see if that works. So I used > MatMatMultTranspose(P, A, MAT_INITIAL_MATRIX, 1.0, &C1) and then > MatMatMult(C1, P, MAT_INITIAL_MATRIX, 1.0, &C) > to get C=C1*P=P^T*A*P in two steps. This time, everything works fine for the > small test case from above, also with the new version 2.3.3-p13. > > > Best regards > Tobias Neckel > > -- > Dipl.-Tech. Math. Tobias Neckel > > Institut f?r Informatik V, TU M?nchen > Boltzmannstr. 3, 85748 Garching > > Tel.: 089/289-18602 > Email: neckel at in.tum.de > URL: http://www5.in.tum.de/persons/neckel.html > From zonexo at gmail.com Tue Jul 1 20:04:23 2008 From: zonexo at gmail.com (Ben Tay) Date: Wed, 02 Jul 2008 09:04:23 +0800 Subject: Using ex2f to determine server's MPI performance Message-ID: <486AD417.6020908@gmail.com> Hi, I'm thinking of using the example code ex2f to determine my school server's MPI performance. Since my own code uses a 1200x1080 grid, I let m and n in ex2f.F be equal to 1200 and 1080 respectively. I ran the code on 4 processors and found that it's taking a very long time to complete for this job (> 0.5hr). I then use Hypre as the preconditioner and it completed in a few min or so. I decided to use this to gauge my server's scaling performance. My own code is also using Hypre. However, in order to increase the runtime to make the timing less prone to random timing errors, I added do k=1,100 call KSPSolve(ksp,b,x,ierr) end do to repeat the solving 100 times. Is this fine (will there be any cache related problems)? Else, is there a better way? I need to keep the grid size fixed so that I can compared with my own code which uses the same no. of grids. Anyway, I ran the code and I got this result: processors ksp_solve time (from log_summary) wall time min/sec 1 4.6e2 7/50 2 2.4e2 4/9 4 1.3e2 2/20 8 1.1e2 2/23 12 7.7e1 1/24 Does this mean that my school server scales poorly for >4 processors? I repeated running the code using the 8 processor and the new timing is even longer. Thank you very much. From acolombi at gmail.com Wed Jul 2 02:52:09 2008 From: acolombi at gmail.com (Andrew Colombi) Date: Wed, 2 Jul 2008 02:52:09 -0500 Subject: "Diagonal" Columns of MPIAIJ Matrix? Message-ID: <9dc10d950807020052m5a0819efy73f7a6ab2bf8fc02@mail.gmail.com> I'm letting PETSC_DECIDE how to distribute the rows and columns of my non-square matrix, and I want to retrieve the columns it's decided to make the "Diagonal" for a particular process. How can I do that? I'm probably missing something obvious here, but I checked the docs. In case some context helps (or inspires a work-around): I want this information to count the number of d_nnz and o_nnz to preallocate perfectly. This shouldn't be hard to do, but I need to know the boundary between off-diagonal and diagonal to do it. Thanks, -Andrew From knepley at gmail.com Wed Jul 2 06:35:06 2008 From: knepley at gmail.com (Matthew Knepley) Date: Wed, 2 Jul 2008 06:35:06 -0500 Subject: "Diagonal" Columns of MPIAIJ Matrix? In-Reply-To: <9dc10d950807020052m5a0819efy73f7a6ab2bf8fc02@mail.gmail.com> References: <9dc10d950807020052m5a0819efy73f7a6ab2bf8fc02@mail.gmail.com> Message-ID: By definition, the diagonal portion is square. Thus you retrieve the local rows [start, end), and then the diagonal block is [start, end) x [start, end) which you can do with http://www-unix.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/Mat/MatGetOwnershipRange.html Matt On Wed, Jul 2, 2008 at 2:52 AM, Andrew Colombi wrote: > I'm letting PETSC_DECIDE how to distribute the rows and columns of my > non-square matrix, and I want to retrieve the columns it's decided to > make the "Diagonal" for a particular process. How can I do that? I'm > probably missing something obvious here, but I checked the docs. > > In case some context helps (or inspires a work-around): > > I want this information to count the number of d_nnz and o_nnz to > preallocate perfectly. This shouldn't be hard to do, but I need to > know the boundary between off-diagonal and diagonal to do it. > > Thanks, > -Andrew > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener From acolombi at gmail.com Wed Jul 2 11:58:42 2008 From: acolombi at gmail.com (Andrew Colombi) Date: Wed, 2 Jul 2008 11:58:42 -0500 Subject: "Diagonal" Columns of MPIAIJ Matrix? In-Reply-To: References: <9dc10d950807020052m5a0819efy73f7a6ab2bf8fc02@mail.gmail.com> Message-ID: <9dc10d950807020958t5547982s6e2f3adf489db4f1@mail.gmail.com> Hmm... this has not been my experience. When A is not square I'm finding the that the diagonal portion is also not square. Instead it assigns columns to the diagonal portion by evenly distributing the columns among processors. For example, a 4 by 8 would look like: (please excuse the embedded HTML, I need it to specify a fixed-width font) c0 c1 c2 c3 c4 c5 c6 c7 p0 r0 X X X X p0 r1 X X X X p1 r2 X X X X p1 r3 X X X X X marks the diagonal portions for each processor. I have attempted to verify this using the program below. If you run this code it will complain that processor 1 must allocate new memory for "(0, 3)" (which I am interpreting to mean the local (0, 3), otherwise known as (2, 3)). Further more you can stop the complaint by incrementing the off-diagonal preallocation, and decrementing the diagonal preallocation. Note, that specifying 0 is some special value that isn't 0. If you specify 0 you won't get any malloc errors. Mat A; int rows = 4; int cols = 8; if (mpi_rank() == 0) { MatCreateMPIAIJ(PETSC_COMM_WORLD, PETSC_DECIDE, PETSC_DECIDE, rows, cols, 2, PETSC_NULL, 1, PETSC_NULL, &A); MatSetOption(A, MAT_NEW_NONZERO_ALLOCATION_ERR); static const int R = 2; int set_rows[R] = {0, 1}; static const int C = 2; int set_cols[C] = {0, 1}; double vals[C*R] = {1., 1., 1., 1.}; MatSetValues(A, R, set_rows, C, set_cols, vals, ADD_VALUES); } else { MatCreateMPIAIJ(PETSC_COMM_WORLD, PETSC_DECIDE, PETSC_DECIDE,rows, cols, 2, PETSC_NULL, 1, PETSC_NULL, &A); MatSetOption(A, MAT_NEW_NONZERO_ALLOCATION_ERR); static const int R = 2; int set_rows[R] = {2,3}; static const int C = 2; int set_cols[C] = {2,3}; double vals[C*R] = {1., 1., 1., 1.}; MatSetValues(A, R, set_rows, C, set_cols, vals, ADD_VALUES); } MatAssemblyBegin(A, MAT_FINAL_ASSEMBLY); MatAssemblyEnd(A, MAT_FINAL_ASSEMBLY); On Wed, Jul 2, 2008 at 6:35 AM, Matthew Knepley wrote: > By definition, the diagonal portion is square. Thus you retrieve the > local rows [start, end), and then the diagonal block is > > [start, end) x [start, end) > > which you can do with > > http://www-unix.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/Mat/MatGetOwnershipRange.html > > Matt > > On Wed, Jul 2, 2008 at 2:52 AM, Andrew Colombi wrote: >> I'm letting PETSC_DECIDE how to distribute the rows and columns of my >> non-square matrix, and I want to retrieve the columns it's decided to >> make the "Diagonal" for a particular process. How can I do that? I'm >> probably missing something obvious here, but I checked the docs. >> >> In case some context helps (or inspires a work-around): >> >> I want this information to count the number of d_nnz and o_nnz to >> preallocate perfectly. This shouldn't be hard to do, but I need to >> know the boundary between off-diagonal and diagonal to do it. >> >> Thanks, >> -Andrew >> >> > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which > their experiments lead. > -- Norbert Wiener > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Wed Jul 2 13:52:34 2008 From: bsmith at mcs.anl.gov (Barry Smith) Date: Wed, 2 Jul 2008 13:52:34 -0500 Subject: "Diagonal" Columns of MPIAIJ Matrix? In-Reply-To: <9dc10d950807020958t5547982s6e2f3adf489db4f1@mail.gmail.com> References: <9dc10d950807020052m5a0819efy73f7a6ab2bf8fc02@mail.gmail.com> <9dc10d950807020958t5547982s6e2f3adf489db4f1@mail.gmail.com> Message-ID: <43000180-0A80-4E7B-BBFF-A274D1832740@mcs.anl.gov> MatGetOwnershipRangeColumn() will give you the needed information. If you are coding from C/C++ you might consider using the macro MatPreallocateInitialize() and friends. These utilities make it somewhat easier, in some circumstances, to count the entries in the diagonal and off-diagonal blocks. Barry On Jul 2, 2008, at 11:58 AM, Andrew Colombi wrote: > Hmm... this has not been my experience. When A is not square I'm > finding the that the diagonal portion is also not square. Instead > it assigns columns to the diagonal portion by evenly distributing > the columns among processors. For example, a 4 by 8 would look like: > > (please excuse the embedded HTML, I need it to specify a fixed-width > font) > > c0 c1 c2 c3 c4 c5 c6 c7 > p0 r0 X X X X > p0 r1 X X X X > p1 r2 X X X X > p1 r3 X X X X > > > X marks the diagonal portions for each processor. I have attempted > to verify this using the program below. If you run this code it > will complain that processor 1 must allocate new memory for "(0, > 3)" (which I am interpreting to mean the local (0, 3), otherwise > known as (2, 3)). Further more you can stop the complaint by > incrementing the off-diagonal preallocation, and decrementing the > diagonal preallocation. Note, that specifying 0 is some special > value that isn't 0. If you specify 0 you won't get any malloc errors. > > Mat A; > int rows = 4; > int cols = 8; > > if (mpi_rank() == 0) > { > MatCreateMPIAIJ(PETSC_COMM_WORLD, PETSC_DECIDE, PETSC_DECIDE, > rows, cols, > 2, PETSC_NULL, 1, PETSC_NULL, &A); > MatSetOption(A, MAT_NEW_NONZERO_ALLOCATION_ERR); > > static const int R = 2; > int set_rows[R] = {0, 1}; > static const int C = 2; > int set_cols[C] = {0, 1}; > double vals[C*R] = {1., 1., 1., 1.}; > MatSetValues(A, R, set_rows, C, set_cols, vals, ADD_VALUES); > } > else > { > MatCreateMPIAIJ(PETSC_COMM_WORLD, PETSC_DECIDE, > PETSC_DECIDE,rows, cols, > 2, PETSC_NULL, 1, PETSC_NULL, &A); > MatSetOption(A, MAT_NEW_NONZERO_ALLOCATION_ERR); > > static const int R = 2; > int set_rows[R] = {2,3}; > static const int C = 2; > int set_cols[C] = {2,3}; > double vals[C*R] = {1., 1., 1., 1.}; > MatSetValues(A, R, set_rows, C, set_cols, vals, ADD_VALUES); > } > > MatAssemblyBegin(A, MAT_FINAL_ASSEMBLY); > MatAssemblyEnd(A, MAT_FINAL_ASSEMBLY); > > On Wed, Jul 2, 2008 at 6:35 AM, Matthew Knepley > wrote: > > By definition, the diagonal portion is square. Thus you retrieve the > > local rows [start, end), and then the diagonal block is > > > > [start, end) x [start, end) > > > > which you can do with > > > > http://www-unix.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/Mat/MatGetOwnershipRange.html > > > > Matt > > > > On Wed, Jul 2, 2008 at 2:52 AM, Andrew Colombi > wrote: > >> I'm letting PETSC_DECIDE how to distribute the rows and columns > of my > >> non-square matrix, and I want to retrieve the columns it's > decided to > >> make the "Diagonal" for a particular process. How can I do > that? I'm > >> probably missing something obvious here, but I checked the docs. > >> > >> In case some context helps (or inspires a work-around): > >> > >> I want this information to count the number of d_nnz and o_nnz to > >> preallocate perfectly. This shouldn't be hard to do, but I need to > >> know the boundary between off-diagonal and diagonal to do it. > >> > >> Thanks, > >> -Andrew > >> > >> > > > > > > > > -- > > What most experimenters take for granted before they begin their > > experiments is infinitely more interesting than any results to which > > their experiments lead. > > -- Norbert Wiener > > > > > From acolombi at gmail.com Wed Jul 2 15:25:51 2008 From: acolombi at gmail.com (Andrew Colombi) Date: Wed, 2 Jul 2008 15:25:51 -0500 Subject: "Diagonal" Columns of MPIAIJ Matrix? In-Reply-To: <43000180-0A80-4E7B-BBFF-A274D1832740@mcs.anl.gov> References: <9dc10d950807020052m5a0819efy73f7a6ab2bf8fc02@mail.gmail.com> <9dc10d950807020958t5547982s6e2f3adf489db4f1@mail.gmail.com> <43000180-0A80-4E7B-BBFF-A274D1832740@mcs.anl.gov> Message-ID: <9dc10d950807021325h220e45f0me13d8c60bc924ec4@mail.gmail.com> Is MatGetOwnershipRangeColumn a new function? because I don't think it I have it. I'm using petsc-2.3.3-p13. I searched petsc/include for RangeColumn and didn't find anything. I could switch over to MatPreallocateInitialize, but I'm not entirely sure how it works. The docs reference a chapter in the user manual about performance, "See the chapter in the users manual on performance for more details" But I can't find anything in the PDF manual that mentions PreallocateInitialize or PreallocateSet (I searched the text for that string). My first question is, how does MatPreallocateInitialize allocate dnz and onz? It says we should not malloc or free them, but it takes PetscInt *dnz not PetscInt **dnz? My understanding of how these utilities work is that you do your normal matrix assembly, but use MatPreallocateSet instead of MatSetValues. Then it just does some counting and when you're done dnz and onz are correct. Am I correct? Thanks, -Andrew On Wed, Jul 2, 2008 at 1:52 PM, Barry Smith wrote: > > MatGetOwnershipRangeColumn() will give you the needed information. If you are coding from C/C++ > you might consider using the macro MatPreallocateInitialize() and friends. These utilities make it > somewhat easier, in some circumstances, to count the entries in the diagonal and off-diagonal blocks. > > Barry > > On Jul 2, 2008, at 11:58 AM, Andrew Colombi wrote: > >> Hmm... this has not been my experience. When A is not square I'm finding the that the diagonal portion is also not square. Instead it assigns columns to the diagonal portion by evenly distributing the columns among processors. For example, a 4 by 8 would look like: >> >> (please excuse the embedded HTML, I need it to specify a fixed-width font) >> >> c0 c1 c2 c3 c4 c5 c6 c7 >> p0 r0 X X X X >> p0 r1 X X X X >> p1 r2 X X X X >> p1 r3 X X X X >> >> >> X marks the diagonal portions for each processor. I have attempted to verify this using the program below. If you run this code it will complain that processor 1 must allocate new memory for "(0, 3)" (which I am interpreting to mean the local (0, 3), otherwise known as (2, 3)). Further more you can stop the complaint by incrementing the off-diagonal preallocation, and decrementing the diagonal preallocation. Note, that specifying 0 is some special value that isn't 0. If you specify 0 you won't get any malloc errors. >> >> Mat A; >> int rows = 4; >> int cols = 8; >> >> if (mpi_rank() == 0) >> { >> MatCreateMPIAIJ(PETSC_COMM_WORLD, PETSC_DECIDE, PETSC_DECIDE, rows, cols, >> 2, PETSC_NULL, 1, PETSC_NULL, &A); >> MatSetOption(A, MAT_NEW_NONZERO_ALLOCATION_ERR); >> >> static const int R = 2; >> int set_rows[R] = {0, 1}; >> static const int C = 2; >> int set_cols[C] = {0, 1}; >> double vals[C*R] = {1., 1., 1., 1.}; >> MatSetValues(A, R, set_rows, C, set_cols, vals, ADD_VALUES); >> } >> else >> { >> MatCreateMPIAIJ(PETSC_COMM_WORLD, PETSC_DECIDE, PETSC_DECIDE,rows, cols, >> 2, PETSC_NULL, 1, PETSC_NULL, &A); >> MatSetOption(A, MAT_NEW_NONZERO_ALLOCATION_ERR); >> >> static const int R = 2; >> int set_rows[R] = {2,3}; >> static const int C = 2; >> int set_cols[C] = {2,3}; >> double vals[C*R] = {1., 1., 1., 1.}; >> MatSetValues(A, R, set_rows, C, set_cols, vals, ADD_VALUES); >> } >> >> MatAssemblyBegin(A, MAT_FINAL_ASSEMBLY); >> MatAssemblyEnd(A, MAT_FINAL_ASSEMBLY); >> >> On Wed, Jul 2, 2008 at 6:35 AM, Matthew Knepley wrote: >> > By definition, the diagonal portion is square. Thus you retrieve the >> > local rows [start, end), and then the diagonal block is >> > >> > [start, end) x [start, end) >> > >> > which you can do with >> > >> > http://www-unix.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/Mat/MatGetOwnershipRange.html >> > >> > Matt >> > >> > On Wed, Jul 2, 2008 at 2:52 AM, Andrew Colombi wrote: >> >> I'm letting PETSC_DECIDE how to distribute the rows and columns of my >> >> non-square matrix, and I want to retrieve the columns it's decided to >> >> make the "Diagonal" for a particular process. How can I do that? I'm >> >> probably missing something obvious here, but I checked the docs. >> >> >> >> In case some context helps (or inspires a work-around): >> >> >> >> I want this information to count the number of d_nnz and o_nnz to >> >> preallocate perfectly. This shouldn't be hard to do, but I need to >> >> know the boundary between off-diagonal and diagonal to do it. >> >> >> >> Thanks, >> >> -Andrew >> >> >> >> >> > >> > >> > >> > -- >> > What most experimenters take for granted before they begin their >> > experiments is infinitely more interesting than any results to which >> > their experiments lead. >> > -- Norbert Wiener >> > >> > >> > From bsmith at mcs.anl.gov Wed Jul 2 15:34:32 2008 From: bsmith at mcs.anl.gov (Barry Smith) Date: Wed, 2 Jul 2008 15:34:32 -0500 Subject: "Diagonal" Columns of MPIAIJ Matrix? In-Reply-To: <9dc10d950807021325h220e45f0me13d8c60bc924ec4@mail.gmail.com> References: <9dc10d950807020052m5a0819efy73f7a6ab2bf8fc02@mail.gmail.com> <9dc10d950807020958t5547982s6e2f3adf489db4f1@mail.gmail.com> <43000180-0A80-4E7B-BBFF-A274D1832740@mcs.anl.gov> <9dc10d950807021325h220e45f0me13d8c60bc924ec4@mail.gmail.com> Message-ID: <355B21C9-1F7E-43B5-8FBD-A5CDB34C7012@mcs.anl.gov> On Jul 2, 2008, at 3:25 PM, Andrew Colombi wrote: > Is MatGetOwnershipRangeColumn a new function? Looks like it. > because I don't think it > I have it. I'm using petsc-2.3.3-p13. I searched petsc/include for > RangeColumn and didn't find anything. > > I could switch over to MatPreallocateInitialize, but I'm not entirely > sure how it works. The docs reference a chapter in the user manual > about performance, > > "See the chapter in the users manual on performance for more details" > > But I can't find anything in the PDF manual that mentions > PreallocateInitialize or PreallocateSet (I searched the text for that > string). The manual pages for these macros are pretty extensive; they are not discussed in the manual. > > > My first question is, how does MatPreallocateInitialize allocate dnz > and onz? It says we should not malloc or free them, but it takes > PetscInt *dnz not PetscInt **dnz? My understanding of how these > utilities work is that you do your normal matrix assembly, but use > MatPreallocateSet instead of MatSetValues. Then it just does some > counting and when you're done dnz and onz are correct. Am I correct? Yes. See src/dm/da/src/utils/fdda.c for many examples of its use. Barry > > Thanks, > -Andrew > > On Wed, Jul 2, 2008 at 1:52 PM, Barry Smith > wrote: >> >> MatGetOwnershipRangeColumn() will give you the needed information. >> If you are coding from C/C++ >> you might consider using the macro MatPreallocateInitialize() and >> friends. These utilities make it >> somewhat easier, in some circumstances, to count the entries in the >> diagonal and off-diagonal blocks. >> >> Barry >> >> On Jul 2, 2008, at 11:58 AM, Andrew Colombi wrote: >> >>> Hmm... this has not been my experience. When A is not square I'm >>> finding the that the diagonal portion is also not square. Instead >>> it assigns columns to the diagonal portion by evenly distributing >>> the columns among processors. For example, a 4 by 8 would look >>> like: >>> >>> (please excuse the embedded HTML, I need it to specify a fixed- >>> width font) >>> >>> c0 c1 c2 c3 c4 c5 c6 c7 >>> p0 r0 X X X X >>> p0 r1 X X X X >>> p1 r2 X X X X >>> p1 r3 X X X X >>> >>> >>> X marks the diagonal portions for each processor. I have >>> attempted to verify this using the program below. If you run this >>> code it will complain that processor 1 must allocate new memory >>> for "(0, 3)" (which I am interpreting to mean the local (0, 3), >>> otherwise known as (2, 3)). Further more you can stop the >>> complaint by incrementing the off-diagonal preallocation, and >>> decrementing the diagonal preallocation. Note, that specifying 0 >>> is some special value that isn't 0. If you specify 0 you won't get >>> any malloc errors. >>> >>> Mat A; >>> int rows = 4; >>> int cols = 8; >>> >>> if (mpi_rank() == 0) >>> { >>> MatCreateMPIAIJ(PETSC_COMM_WORLD, PETSC_DECIDE, PETSC_DECIDE, >>> rows, cols, >>> 2, PETSC_NULL, 1, PETSC_NULL, &A); >>> MatSetOption(A, MAT_NEW_NONZERO_ALLOCATION_ERR); >>> >>> static const int R = 2; >>> int set_rows[R] = {0, 1}; >>> static const int C = 2; >>> int set_cols[C] = {0, 1}; >>> double vals[C*R] = {1., 1., 1., 1.}; >>> MatSetValues(A, R, set_rows, C, set_cols, vals, ADD_VALUES); >>> } >>> else >>> { >>> MatCreateMPIAIJ(PETSC_COMM_WORLD, PETSC_DECIDE, >>> PETSC_DECIDE,rows, cols, >>> 2, PETSC_NULL, 1, PETSC_NULL, &A); >>> MatSetOption(A, MAT_NEW_NONZERO_ALLOCATION_ERR); >>> >>> static const int R = 2; >>> int set_rows[R] = {2,3}; >>> static const int C = 2; >>> int set_cols[C] = {2,3}; >>> double vals[C*R] = {1., 1., 1., 1.}; >>> MatSetValues(A, R, set_rows, C, set_cols, vals, ADD_VALUES); >>> } >>> >>> MatAssemblyBegin(A, MAT_FINAL_ASSEMBLY); >>> MatAssemblyEnd(A, MAT_FINAL_ASSEMBLY); >>> >>> On Wed, Jul 2, 2008 at 6:35 AM, Matthew Knepley >>> wrote: >>>> By definition, the diagonal portion is square. Thus you retrieve >>>> the >>>> local rows [start, end), and then the diagonal block is >>>> >>>> [start, end) x [start, end) >>>> >>>> which you can do with >>>> >>>> http://www-unix.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/Mat/MatGetOwnershipRange.html >>>> >>>> Matt >>>> >>>> On Wed, Jul 2, 2008 at 2:52 AM, Andrew Colombi >>>> wrote: >>>>> I'm letting PETSC_DECIDE how to distribute the rows and columns >>>>> of my >>>>> non-square matrix, and I want to retrieve the columns it's >>>>> decided to >>>>> make the "Diagonal" for a particular process. How can I do >>>>> that? I'm >>>>> probably missing something obvious here, but I checked the docs. >>>>> >>>>> In case some context helps (or inspires a work-around): >>>>> >>>>> I want this information to count the number of d_nnz and o_nnz to >>>>> preallocate perfectly. This shouldn't be hard to do, but I need >>>>> to >>>>> know the boundary between off-diagonal and diagonal to do it. >>>>> >>>>> Thanks, >>>>> -Andrew >>>>> >>>>> >>>> >>>> >>>> >>>> -- >>>> What most experimenters take for granted before they begin their >>>> experiments is infinitely more interesting than any results to >>>> which >>>> their experiments lead. >>>> -- Norbert Wiener >>>> >>>> >>> >> > From acolombi at gmail.com Wed Jul 2 16:33:41 2008 From: acolombi at gmail.com (Andrew Colombi) Date: Wed, 2 Jul 2008 16:33:41 -0500 Subject: "Diagonal" Columns of MPIAIJ Matrix? In-Reply-To: <355B21C9-1F7E-43B5-8FBD-A5CDB34C7012@mcs.anl.gov> References: <9dc10d950807020052m5a0819efy73f7a6ab2bf8fc02@mail.gmail.com> <9dc10d950807020958t5547982s6e2f3adf489db4f1@mail.gmail.com> <43000180-0A80-4E7B-BBFF-A274D1832740@mcs.anl.gov> <9dc10d950807021325h220e45f0me13d8c60bc924ec4@mail.gmail.com> <355B21C9-1F7E-43B5-8FBD-A5CDB34C7012@mcs.anl.gov> Message-ID: <9dc10d950807021433p3b86cb78n488f330741193806@mail.gmail.com> Is it fair to say that MatPreallocateInitialize and MatPreallocateSet need to be in the same scope for this to work? If so, that will never work with my code. (Unless I can call repeatedly without having it reinitialize things.) I already had everything setup to work with my own bookkeeping (STL vector and binary_search, in my tests STL set took too much memory). But I need get the diagonal columns! So be it. I'm willing to hack. May I assume PETSc distributes columns with the following algorithm: int diag_cols = total_cols / processors; if (mpi_rank() < total_cols % processors) diag_cols += 1; Then assume the offsets follow the MPI rank? (i.e. higher rank has later columns) Thanks, -Andrew On Wed, Jul 2, 2008 at 3:34 PM, Barry Smith wrote: > > On Jul 2, 2008, at 3:25 PM, Andrew Colombi wrote: > >> Is MatGetOwnershipRangeColumn a new function? > > Looks like it. > >> because I don't think it >> I have it. I'm using petsc-2.3.3-p13. I searched petsc/include for >> RangeColumn and didn't find anything. >> >> I could switch over to MatPreallocateInitialize, but I'm not entirely >> sure how it works. The docs reference a chapter in the user manual >> about performance, >> >> "See the chapter in the users manual on performance for more details" >> >> But I can't find anything in the PDF manual that mentions >> PreallocateInitialize or PreallocateSet (I searched the text for that >> string). > > The manual pages for these macros are pretty extensive; they are not > discussed in the manual. >> >> >> My first question is, how does MatPreallocateInitialize allocate dnz >> and onz? It says we should not malloc or free them, but it takes >> PetscInt *dnz not PetscInt **dnz? My understanding of how these >> utilities work is that you do your normal matrix assembly, but use >> MatPreallocateSet instead of MatSetValues. Then it just does some >> counting and when you're done dnz and onz are correct. Am I correct? > > Yes. See src/dm/da/src/utils/fdda.c for many examples of its use. > > Barry > >> >> Thanks, >> -Andrew >> >> On Wed, Jul 2, 2008 at 1:52 PM, Barry Smith wrote: >>> >>> MatGetOwnershipRangeColumn() will give you the needed information. If you >>> are coding from C/C++ >>> you might consider using the macro MatPreallocateInitialize() and >>> friends. These utilities make it >>> somewhat easier, in some circumstances, to count the entries in the >>> diagonal and off-diagonal blocks. >>> >>> Barry >>> >>> On Jul 2, 2008, at 11:58 AM, Andrew Colombi wrote: >>> >>>> Hmm... this has not been my experience. When A is not square I'm >>>> finding the that the diagonal portion is also not square. Instead it >>>> assigns columns to the diagonal portion by evenly distributing the columns >>>> among processors. For example, a 4 by 8 would look like: >>>> >>>> (please excuse the embedded HTML, I need it to specify a fixed-width >>>> font) >>>> >>>> c0 c1 c2 c3 c4 c5 c6 c7 >>>> p0 r0 X X X X >>>> p0 r1 X X X X >>>> p1 r2 X X X X >>>> p1 r3 X X X X >>>> >>>> >>>> X marks the diagonal portions for each processor. I have attempted to >>>> verify this using the program below. If you run this code it will complain >>>> that processor 1 must allocate new memory for "(0, 3)" (which I am >>>> interpreting to mean the local (0, 3), otherwise known as (2, 3)). Further >>>> more you can stop the complaint by incrementing the off-diagonal >>>> preallocation, and decrementing the diagonal preallocation. Note, that >>>> specifying 0 is some special value that isn't 0. If you specify 0 you won't >>>> get any malloc errors. >>>> >>>> Mat A; >>>> int rows = 4; >>>> int cols = 8; >>>> >>>> if (mpi_rank() == 0) >>>> { >>>> MatCreateMPIAIJ(PETSC_COMM_WORLD, PETSC_DECIDE, PETSC_DECIDE, rows, >>>> cols, >>>> 2, PETSC_NULL, 1, PETSC_NULL, &A); >>>> MatSetOption(A, MAT_NEW_NONZERO_ALLOCATION_ERR); >>>> >>>> static const int R = 2; >>>> int set_rows[R] = {0, 1}; >>>> static const int C = 2; >>>> int set_cols[C] = {0, 1}; >>>> double vals[C*R] = {1., 1., 1., 1.}; >>>> MatSetValues(A, R, set_rows, C, set_cols, vals, ADD_VALUES); >>>> } >>>> else >>>> { >>>> MatCreateMPIAIJ(PETSC_COMM_WORLD, PETSC_DECIDE, PETSC_DECIDE,rows, >>>> cols, >>>> 2, PETSC_NULL, 1, PETSC_NULL, &A); >>>> MatSetOption(A, MAT_NEW_NONZERO_ALLOCATION_ERR); >>>> >>>> static const int R = 2; >>>> int set_rows[R] = {2,3}; >>>> static const int C = 2; >>>> int set_cols[C] = {2,3}; >>>> double vals[C*R] = {1., 1., 1., 1.}; >>>> MatSetValues(A, R, set_rows, C, set_cols, vals, ADD_VALUES); >>>> } >>>> >>>> MatAssemblyBegin(A, MAT_FINAL_ASSEMBLY); >>>> MatAssemblyEnd(A, MAT_FINAL_ASSEMBLY); >>>> >>>> On Wed, Jul 2, 2008 at 6:35 AM, Matthew Knepley >>>> wrote: >>>>> >>>>> By definition, the diagonal portion is square. Thus you retrieve the >>>>> local rows [start, end), and then the diagonal block is >>>>> >>>>> [start, end) x [start, end) >>>>> >>>>> which you can do with >>>>> >>>>> >>>>> http://www-unix.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/Mat/MatGetOwnershipRange.html >>>>> >>>>> Matt >>>>> >>>>> On Wed, Jul 2, 2008 at 2:52 AM, Andrew Colombi >>>>> wrote: >>>>>> >>>>>> I'm letting PETSC_DECIDE how to distribute the rows and columns of my >>>>>> non-square matrix, and I want to retrieve the columns it's decided to >>>>>> make the "Diagonal" for a particular process. How can I do that? I'm >>>>>> probably missing something obvious here, but I checked the docs. >>>>>> >>>>>> In case some context helps (or inspires a work-around): >>>>>> >>>>>> I want this information to count the number of d_nnz and o_nnz to >>>>>> preallocate perfectly. This shouldn't be hard to do, but I need to >>>>>> know the boundary between off-diagonal and diagonal to do it. >>>>>> >>>>>> Thanks, >>>>>> -Andrew >>>>>> >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> What most experimenters take for granted before they begin their >>>>> experiments is infinitely more interesting than any results to which >>>>> their experiments lead. >>>>> -- Norbert Wiener >>>>> >>>>> >>>> >>> >> > > From bsmith at mcs.anl.gov Wed Jul 2 16:43:09 2008 From: bsmith at mcs.anl.gov (Barry Smith) Date: Wed, 2 Jul 2008 16:43:09 -0500 Subject: "Diagonal" Columns of MPIAIJ Matrix? In-Reply-To: <9dc10d950807021433p3b86cb78n488f330741193806@mail.gmail.com> References: <9dc10d950807020052m5a0819efy73f7a6ab2bf8fc02@mail.gmail.com> <9dc10d950807020958t5547982s6e2f3adf489db4f1@mail.gmail.com> <43000180-0A80-4E7B-BBFF-A274D1832740@mcs.anl.gov> <9dc10d950807021325h220e45f0me13d8c60bc924ec4@mail.gmail.com> <355B21C9-1F7E-43B5-8FBD-A5CDB34C7012@mcs.anl.gov> <9dc10d950807021433p3b86cb78n488f330741193806@mail.gmail.com> Message-ID: The code used to divide it up by default is PetscMapSetUp() in src/ vec/impls/mpi/pmap.c Barry On Jul 2, 2008, at 4:33 PM, Andrew Colombi wrote: > Is it fair to say that MatPreallocateInitialize and MatPreallocateSet > need to be in the same scope for this to work? If so, that will never > work with my code. (Unless I can call repeatedly without having it > reinitialize things.) > > I already had everything setup to work with my own bookkeeping (STL > vector and binary_search, in my tests STL set took too much memory). > But I need get the diagonal columns! > > So be it. I'm willing to hack. May I assume PETSc distributes > columns with the following algorithm: > > int diag_cols = total_cols / processors; > if (mpi_rank() < total_cols % processors) > diag_cols += 1; > > Then assume the offsets follow the MPI rank? (i.e. higher rank has > later columns) > > Thanks, > -Andrew > > On Wed, Jul 2, 2008 at 3:34 PM, Barry Smith > wrote: >> >> On Jul 2, 2008, at 3:25 PM, Andrew Colombi wrote: >> >>> Is MatGetOwnershipRangeColumn a new function? >> >> Looks like it. >> >>> because I don't think it >>> I have it. I'm using petsc-2.3.3-p13. I searched petsc/include for >>> RangeColumn and didn't find anything. >>> >>> I could switch over to MatPreallocateInitialize, but I'm not >>> entirely >>> sure how it works. The docs reference a chapter in the user manual >>> about performance, >>> >>> "See the chapter in the users manual on performance for more >>> details" >>> >>> But I can't find anything in the PDF manual that mentions >>> PreallocateInitialize or PreallocateSet (I searched the text for >>> that >>> string). >> >> The manual pages for these macros are pretty extensive; they are not >> discussed in the manual. >>> >>> >>> My first question is, how does MatPreallocateInitialize allocate dnz >>> and onz? It says we should not malloc or free them, but it takes >>> PetscInt *dnz not PetscInt **dnz? My understanding of how these >>> utilities work is that you do your normal matrix assembly, but use >>> MatPreallocateSet instead of MatSetValues. Then it just does some >>> counting and when you're done dnz and onz are correct. Am I >>> correct? >> >> Yes. See src/dm/da/src/utils/fdda.c for many examples of its use. >> >> Barry >> >>> >>> Thanks, >>> -Andrew >>> >>> On Wed, Jul 2, 2008 at 1:52 PM, Barry Smith >>> wrote: >>>> >>>> MatGetOwnershipRangeColumn() will give you the needed >>>> information. If you >>>> are coding from C/C++ >>>> you might consider using the macro MatPreallocateInitialize() and >>>> friends. These utilities make it >>>> somewhat easier, in some circumstances, to count the entries in the >>>> diagonal and off-diagonal blocks. >>>> >>>> Barry >>>> >>>> On Jul 2, 2008, at 11:58 AM, Andrew Colombi wrote: >>>> >>>>> Hmm... this has not been my experience. When A is not square I'm >>>>> finding the that the diagonal portion is also not square. >>>>> Instead it >>>>> assigns columns to the diagonal portion by evenly distributing >>>>> the columns >>>>> among processors. For example, a 4 by 8 would look like: >>>>> >>>>> (please excuse the embedded HTML, I need it to specify a fixed- >>>>> width >>>>> font) >>>>> >>>>> c0 c1 c2 c3 c4 c5 c6 c7 >>>>> p0 r0 X X X X >>>>> p0 r1 X X X X >>>>> p1 r2 X X X X >>>>> p1 r3 X X X X >>>>> >>>>> >>>>> X marks the diagonal portions for each processor. I have >>>>> attempted to >>>>> verify this using the program below. If you run this code it >>>>> will complain >>>>> that processor 1 must allocate new memory for "(0, 3)" (which I am >>>>> interpreting to mean the local (0, 3), otherwise known as (2, >>>>> 3)). Further >>>>> more you can stop the complaint by incrementing the off-diagonal >>>>> preallocation, and decrementing the diagonal preallocation. >>>>> Note, that >>>>> specifying 0 is some special value that isn't 0. If you specify >>>>> 0 you won't >>>>> get any malloc errors. >>>>> >>>>> Mat A; >>>>> int rows = 4; >>>>> int cols = 8; >>>>> >>>>> if (mpi_rank() == 0) >>>>> { >>>>> MatCreateMPIAIJ(PETSC_COMM_WORLD, PETSC_DECIDE, PETSC_DECIDE, >>>>> rows, >>>>> cols, >>>>> 2, PETSC_NULL, 1, PETSC_NULL, &A); >>>>> MatSetOption(A, MAT_NEW_NONZERO_ALLOCATION_ERR); >>>>> >>>>> static const int R = 2; >>>>> int set_rows[R] = {0, 1}; >>>>> static const int C = 2; >>>>> int set_cols[C] = {0, 1}; >>>>> double vals[C*R] = {1., 1., 1., 1.}; >>>>> MatSetValues(A, R, set_rows, C, set_cols, vals, ADD_VALUES); >>>>> } >>>>> else >>>>> { >>>>> MatCreateMPIAIJ(PETSC_COMM_WORLD, PETSC_DECIDE, PETSC_DECIDE,rows, >>>>> cols, >>>>> 2, PETSC_NULL, 1, PETSC_NULL, &A); >>>>> MatSetOption(A, MAT_NEW_NONZERO_ALLOCATION_ERR); >>>>> >>>>> static const int R = 2; >>>>> int set_rows[R] = {2,3}; >>>>> static const int C = 2; >>>>> int set_cols[C] = {2,3}; >>>>> double vals[C*R] = {1., 1., 1., 1.}; >>>>> MatSetValues(A, R, set_rows, C, set_cols, vals, ADD_VALUES); >>>>> } >>>>> >>>>> MatAssemblyBegin(A, MAT_FINAL_ASSEMBLY); >>>>> MatAssemblyEnd(A, MAT_FINAL_ASSEMBLY); >>>>> >>>>> On Wed, Jul 2, 2008 at 6:35 AM, Matthew Knepley >>>>> >>>>> wrote: >>>>>> >>>>>> By definition, the diagonal portion is square. Thus you >>>>>> retrieve the >>>>>> local rows [start, end), and then the diagonal block is >>>>>> >>>>>> [start, end) x [start, end) >>>>>> >>>>>> which you can do with >>>>>> >>>>>> >>>>>> http://www-unix.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/Mat/MatGetOwnershipRange.html >>>>>> >>>>>> Matt >>>>>> >>>>>> On Wed, Jul 2, 2008 at 2:52 AM, Andrew Colombi >>>>> > >>>>>> wrote: >>>>>>> >>>>>>> I'm letting PETSC_DECIDE how to distribute the rows and >>>>>>> columns of my >>>>>>> non-square matrix, and I want to retrieve the columns it's >>>>>>> decided to >>>>>>> make the "Diagonal" for a particular process. How can I do >>>>>>> that? I'm >>>>>>> probably missing something obvious here, but I checked the docs. >>>>>>> >>>>>>> In case some context helps (or inspires a work-around): >>>>>>> >>>>>>> I want this information to count the number of d_nnz and o_nnz >>>>>>> to >>>>>>> preallocate perfectly. This shouldn't be hard to do, but I >>>>>>> need to >>>>>>> know the boundary between off-diagonal and diagonal to do it. >>>>>>> >>>>>>> Thanks, >>>>>>> -Andrew >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> What most experimenters take for granted before they begin their >>>>>> experiments is infinitely more interesting than any results to >>>>>> which >>>>>> their experiments lead. >>>>>> -- Norbert Wiener >>>>>> >>>>>> >>>>> >>>> >>> >> >> > From thierrylaurent.tonellot at aramco.com Sun Jul 6 03:22:34 2008 From: thierrylaurent.tonellot at aramco.com (Tonellot, Thierry-Laurent D) Date: Sun, 6 Jul 2008 11:22:34 +0300 Subject: how to initialize a parallel vector from one process? Message-ID: <9A0F2BE67CAC54448B6227A347B1B07C26BFF754@EMAILA.aramco.com> Hi, At the very beginning of my application I need to read data in a database to initialize a 3D vector distributed over several processes. The database I'm using can only be accessed by one process (for instance the process 0). Moreover, due to performance issues, we need to limit the request to the database. Therefore the data need to be read by slices, for instance (z,x) slices. A typical initialization would then consist in the following pseudo code: Loop over y If (rank=0) Read slice y Send to each process the appropriate part of the data slice Else Receive data Endif End loop This process is quite heavy and its performances will probably depend on the way it is implemented. I'm wondering if there is any way to perform this initialization efficiently using Petsc? I'm also considering other packages to handle distributed arrays and I'm wondering how a package like global arrays compares with petsc/DA? For instance global arrays seem to have a feature which is partly solving my problem above using the function "ga_fill_patch" which fills only a region of the parallel vector and can be called by any process... Thank you in advance, Thierry -------------- next part -------------- An HTML attachment was scrubbed... URL: From jed at 59A2.org Sun Jul 6 04:27:10 2008 From: jed at 59A2.org (Jed Brown) Date: Sun, 6 Jul 2008 11:27:10 +0200 Subject: how to initialize a parallel vector from one process? In-Reply-To: <9A0F2BE67CAC54448B6227A347B1B07C26BFF754@EMAILA.aramco.com> References: <9A0F2BE67CAC54448B6227A347B1B07C26BFF754@EMAILA.aramco.com> Message-ID: <20080706092710.GC7361@brakk.ethz.ch> I've done something similar when reading and writing netCDF in parallel where the files are too large to be stored on a single processor. NetCDF-4 makes this obsolete, but here's the idea: * The new parallel job makes a PETSc DA and uses PETSC_DECIDE to for partitioning. * Process zero reads the header and broadcasts the dimensions. * Each process determines the index range it needs to interpolate the file data onto the locally owned computational grid. Send this index range to rank zero. * Rank zero reads each of the blocks sequentially (the netCDF API has a read (imin,imax)x(jmin,jmax)x(kmin,kmax)) and sends it to the appropriate process. * Each process does the necessary interpolation locally. I've found that this performs just fine for many GiB of state and hundreds of processors. You have to get your hands a bit dirty with MPI. Of course, there are simpler (pure PETSc solutions) if you can fit the whole state on process zero or if you can use the PETSc binary format. Jed On Sun 2008-07-06 11:22, Tonellot, Thierry-Laurent D wrote: > Hi, > > > > At the very beginning of my application I need to read data in a database to > initialize a 3D vector distributed over several processes. > > The database I?m using can only be accessed by one process (for instance the > process 0). Moreover, due to performance issues, we need to limit the request > to the database. Therefore the data need to be read by slices, for instance > (z,x) slices. > > A typical initialization would then consist in the following pseudo code: > > > > Loop over y > > If (rank=0) > > Read slice y > > Send to each process the appropriate part of the data > slice > > Else > > Receive data > > Endif > > End loop > > > > This process is quite heavy and its performances will probably depend on the > way it is implemented. > > > > I?m wondering if there is any way to perform this initialization efficiently > using Petsc? > > > > I?m also considering other packages to handle distributed arrays and I?m > wondering how a package like global arrays compares with petsc/DA? > > > > For instance global arrays seem to have a feature which is partly solving my > problem above using the function ?ga_fill_patch? which fills only a region of > the parallel vector and can be called by any process? > > > > Thank you in advance, > > > > Thierry > -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: not available URL: From thierrylaurent.tonellot at aramco.com Sun Jul 6 05:01:31 2008 From: thierrylaurent.tonellot at aramco.com (Tonellot, Thierry-Laurent D) Date: Sun, 6 Jul 2008 13:01:31 +0300 Subject: how to initialize a parallel vector from one process? In-Reply-To: <20080706092710.GC7361@brakk.ethz.ch> References: <9A0F2BE67CAC54448B6227A347B1B07C26BFF754@EMAILA.aramco.com> <20080706092710.GC7361@brakk.ethz.ch> Message-ID: <9A0F2BE67CAC54448B6227A347B1B07C26BFF876@EMAILA.aramco.com> Thank you Jed, I'm in the same situation where I cannot assume that the data can fit in one process memory space. I will certainly end by implementing it in the dirty way. It would be neat if a function could take care of distributing the data to each process based on the global indices range. Maybe such a function already exists... Another solution would be to write a new viewer similar to binary viewer... Except that instead of reading in a file one would perform a request to a database. Did you consider this option while implementing your own initialization? Do you know if there is any documentation on how to implement a new viewer? Thierry -----Original Message----- From: owner-petsc-users at mcs.anl.gov [mailto:owner-petsc-users at mcs.anl.gov] On Behalf Of Jed Brown Sent: Sunday, July 06, 2008 12:27 PM To: petsc-users at mcs.anl.gov Subject: Re: how to initialize a parallel vector from one process? I've done something similar when reading and writing netCDF in parallel where the files are too large to be stored on a single processor. NetCDF-4 makes this obsolete, but here's the idea: * The new parallel job makes a PETSc DA and uses PETSC_DECIDE to for partitioning. * Process zero reads the header and broadcasts the dimensions. * Each process determines the index range it needs to interpolate the file data onto the locally owned computational grid. Send this index range to rank zero. * Rank zero reads each of the blocks sequentially (the netCDF API has a read (imin,imax)x(jmin,jmax)x(kmin,kmax)) and sends it to the appropriate process. * Each process does the necessary interpolation locally. I've found that this performs just fine for many GiB of state and hundreds of processors. You have to get your hands a bit dirty with MPI. Of course, there are simpler (pure PETSc solutions) if you can fit the whole state on process zero or if you can use the PETSc binary format. Jed On Sun 2008-07-06 11:22, Tonellot, Thierry-Laurent D wrote: > Hi, > > > > At the very beginning of my application I need to read data in a database to > initialize a 3D vector distributed over several processes. > > The database I'm using can only be accessed by one process (for instance the > process 0). Moreover, due to performance issues, we need to limit the request > to the database. Therefore the data need to be read by slices, for instance > (z,x) slices. > > A typical initialization would then consist in the following pseudo code: > > > > Loop over y > > If (rank=0) > > Read slice y > > Send to each process the appropriate part of the data > slice > > Else > > Receive data > > Endif > > End loop > > > > This process is quite heavy and its performances will probably depend on the > way it is implemented. > > > > I'm wondering if there is any way to perform this initialization efficiently > using Petsc? > > > > I'm also considering other packages to handle distributed arrays and I'm > wondering how a package like global arrays compares with petsc/DA? > > > > For instance global arrays seem to have a feature which is partly solving my > problem above using the function "ga_fill_patch" which fills only a region of > the parallel vector and can be called by any process... > > > > Thank you in advance, > > > > Thierry > From zonexo at gmail.com Mon Jul 7 19:31:09 2008 From: zonexo at gmail.com (Ben Tay) Date: Tue, 08 Jul 2008 08:31:09 +0800 Subject: Using ex2f to determine server's MPI performance (resend) Message-ID: <4872B54D.50105@gmail.com> Hi, I sent this question a while ago. However, it was not answered. Anyway, if the question below is not as relevant, maybe just 1 simple question: should the ex2f code with about 1000000 unknowns scale properly, maybe up to 8 or 16 processors if the server distributes the MPI load properly? Thank you very much. Here's the previous email: I'm thinking of using the example code ex2f to determine my school server's MPI performance. Since my own code uses a 1200x1080 grid, I let m and n in ex2f.F be equal to 1200 and 1080 respectively. I ran the code on 4 processors and found that it's taking a very long time to complete for this job (> 0.5hr). I then use Hypre as the preconditioner and it completed in a few min or so. I decided to use this to gauge my server's scaling performance. My own code is also using Hypre. However, in order to increase the runtime to make the timing less prone to random timing errors, I added do k=1,100 call KSPSolve(ksp,b,x,ierr) end do to repeat the solving 100 times. Is this fine (will there be any cache related problems)? Else, is there a better way? I need to keep the grid size fixed so that I can compared with my own code which uses the same no. of grids. Anyway, I ran the code and I got this result: processors ksp_solve time (from log_summary) wall time min/sec 1 4.6e2 7/50 2 2.4e2 4/9 4 1.3e2 2/20 8 1.1e2 2/23 12 7.7e1 1/24 Does this mean that my school server scales poorly for >4 processors? I repeated running the code using the 8 processor and the new timing is even longer. Thank you very much. From acolombi at gmail.com Tue Jul 8 10:57:49 2008 From: acolombi at gmail.com (Andrew Colombi) Date: Tue, 8 Jul 2008 10:57:49 -0500 Subject: how to initialize a parallel vector from one process? In-Reply-To: <9A0F2BE67CAC54448B6227A347B1B07C26BFF876@EMAILA.aramco.com> References: <9A0F2BE67CAC54448B6227A347B1B07C26BFF754@EMAILA.aramco.com> <20080706092710.GC7361@brakk.ethz.ch> <9A0F2BE67CAC54448B6227A347B1B07C26BFF876@EMAILA.aramco.com> Message-ID: <9dc10d950807080857t59a6ce66q7a7029d438ceffc@mail.gmail.com> PETSc will automatically distribute data to the appropriate processor. When you use VecSetValues it will cache values that belong to other processors and distribute when you call VecAssemblyBegin/VecAssemblyEnd. So you can do this: Loop over y: if rank == 0: read slice y VecSetValues( y ) VecAssemblyBegin() VecAssemblyEnd() Note that you can run out of memory on rank 0 process if you try to cache too many values before calling the Assembly methods. -Andrew On Sun, Jul 6, 2008 at 5:01 AM, Tonellot, Thierry-Laurent D wrote: > > Thank you Jed, > > I'm in the same situation where I cannot assume that the data can fit in one process memory space. > I will certainly end by implementing it in the dirty way. > It would be neat if a function could take care of distributing the data to each process based on the global indices range. Maybe such a function already exists... > > Another solution would be to write a new viewer similar to binary viewer... Except that instead of reading in a file one would perform a request to a database. > Did you consider this option while implementing your own initialization? > Do you know if there is any documentation on how to implement a new viewer? > > Thierry > > -----Original Message----- > From: owner-petsc-users at mcs.anl.gov [mailto:owner-petsc-users at mcs.anl.gov] On Behalf Of Jed Brown > Sent: Sunday, July 06, 2008 12:27 PM > To: petsc-users at mcs.anl.gov > Subject: Re: how to initialize a parallel vector from one process? > > I've done something similar when reading and writing netCDF in parallel where > the files are too large to be stored on a single processor. NetCDF-4 makes this > obsolete, but here's the idea: > > * The new parallel job makes a PETSc DA and uses PETSC_DECIDE to for partitioning. > > * Process zero reads the header and broadcasts the dimensions. > > * Each process determines the index range it needs to interpolate the file data > onto the locally owned computational grid. Send this index range to rank > zero. > > * Rank zero reads each of the blocks sequentially (the netCDF API has a read > (imin,imax)x(jmin,jmax)x(kmin,kmax)) and sends it to the appropriate process. > > * Each process does the necessary interpolation locally. > > I've found that this performs just fine for many GiB of state and hundreds of > processors. You have to get your hands a bit dirty with MPI. Of course, there > are simpler (pure PETSc solutions) if you can fit the whole state on process > zero or if you can use the PETSc binary format. > > Jed > > > On Sun 2008-07-06 11:22, Tonellot, Thierry-Laurent D wrote: >> Hi, >> >> >> >> At the very beginning of my application I need to read data in a database to >> initialize a 3D vector distributed over several processes. >> >> The database I'm using can only be accessed by one process (for instance the >> process 0). Moreover, due to performance issues, we need to limit the request >> to the database. Therefore the data need to be read by slices, for instance >> (z,x) slices. >> >> A typical initialization would then consist in the following pseudo code: >> >> >> >> Loop over y >> >> If (rank=0) >> >> Read slice y >> >> Send to each process the appropriate part of the data >> slice >> >> Else >> >> Receive data >> >> Endif >> >> End loop >> >> >> >> This process is quite heavy and its performances will probably depend on the >> way it is implemented. >> >> >> >> I'm wondering if there is any way to perform this initialization efficiently >> using Petsc? >> >> >> >> I'm also considering other packages to handle distributed arrays and I'm >> wondering how a package like global arrays compares with petsc/DA? >> >> >> >> For instance global arrays seem to have a feature which is partly solving my >> problem above using the function "ga_fill_patch" which fills only a region of >> the parallel vector and can be called by any process... >> >> >> >> Thank you in advance, >> >> >> >> Thierry >> > > From w_subber at yahoo.com Sun Jul 13 18:11:24 2008 From: w_subber at yahoo.com (Waad Subber) Date: Sun, 13 Jul 2008 16:11:24 -0700 (PDT) Subject: MatView Message-ID: <120963.15905.qm@web38207.mail.mud.yahoo.com> Hi I am trying to visualize the nonzero structure of a sparse matrix. I am calling MatView with the viewer context is PETSC_VIEWER_DRAW_SELF. I understand the colored points are the nonzero entries. Then what is the difference between the red color point and the blue point in the draw ? Thanks Waad -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Sun Jul 13 19:09:23 2008 From: bsmith at mcs.anl.gov (Barry Smith) Date: Sun, 13 Jul 2008 19:09:23 -0500 Subject: MatView In-Reply-To: <120963.15905.qm@web38207.mail.mud.yahoo.com> References: <120963.15905.qm@web38207.mail.mud.yahoo.com> Message-ID: They indicate the sign of the value. Barry On Jul 13, 2008, at 6:11 PM, Waad Subber wrote: > Hi > > I am trying to visualize the nonzero structure of a sparse matrix. I > am calling MatView with the viewer context is > PETSC_VIEWER_DRAW_SELF. I understand the colored points are the > nonzero entries. Then what is the difference between the red color > point and the blue point in the draw ? > > Thanks > > Waad > From sosentos at cs.umass.edu Mon Jul 14 12:23:17 2008 From: sosentos at cs.umass.edu (Sarah Osentoski) Date: Mon, 14 Jul 2008 13:23:17 -0400 Subject: MatTranspose Question Message-ID: Hi, I'll start out by saying I'm a new user to PETSc. I'm actually using SLEPc which is a package for eigenvalue problems and is built on top of PETSc and I have a question about the MatTranspose function. I have a very large, very sparse matrix with dimensions on the order of 3 million by 1 million. I created my matrix using MatCreateMPIAIJ and it seems to be evenly distributed across my machines. I've hit a bottleneck when running some code to compute the SVD of the matrix and the bottleneck seems to be using the MatTranspose operation on my matrix. This operation runs for a very long time. I ran a job over the weekend and it still seems to be in this function. (Running for on the order of 60 hours at this point.) It also seems to only being using one of the processors available to it. When MatTranspose runs does it go through every element and then send messages between the process? Or for sparse matrices does it only reorder the elements that are not zero? Or is there something else going on that explains this behavior? Thanks, Sarah Osentoski From tyoung at ippt.gov.pl Mon Jul 14 13:21:18 2008 From: tyoung at ippt.gov.pl (Toby D. Young) Date: Mon, 14 Jul 2008 20:21:18 +0200 (CEST) Subject: MatTranspose Question In-Reply-To: References: Message-ID: > I'll start out by saying I'm a new user to PETSc. I'm actually using > SLEPc Me too. :-) > I have a very large, very sparse matrix with dimensions on the order > of 3 million by 1 million. I created my matrix using MatCreateMPIAIJ > and it seems to be evenly distributed across my machines. I've hit a > bottleneck when running some code to compute the SVD of the matrix > and the bottleneck seems to be using the MatTranspose operation on my > matrix. I can not directly help you with your questions Sarah, sorry. Out of sheer academic interest in your problem (I had a similar problem once and found a work-around): Have you tried to run your code for very small matrices (say, 300x100)? Just to check that the bottleneck problem is in fact not something else! Transposing a HUGE matrix such as the one you have is an expensive operation in anyone's book. As far as I know, PETSc does not store zero-components of a matrix but uses a sparsity pattern to "remember" them... (Someone please correct me if I am wrong). Hope that is somehow helpful for you. Best, Toby ----- Toby D. Young - Adiunkt (Assistant Professor) Department of Computational Science Institute of Fundamental Technological Research Polish Academy of Sciences Room 206, ul. Swietokrzyska 21 00-049 Warszawa, Polska +48 22 826 12 81 ext. 184 http://rav.ippt.gov.pl/~tyoung From sosentos at cs.umass.edu Mon Jul 14 14:23:42 2008 From: sosentos at cs.umass.edu (Sarah Osentoski) Date: Mon, 14 Jul 2008 15:23:42 -0400 Subject: MatTranspose Question In-Reply-To: References: Message-ID: <986B3CEE-E41F-41A7-82A7-C9C1FB3D23D3@cs.umass.edu> Hi Toby, The first thing I did was to run my code to check for correctness on a matrix of size 202x200. It gave me, what looked to be, the singular values i was expecting on that matrix. The code ran extremely quickly on this size of matrix. I suppose could keep generating larger and larger fake matrices until I get stuck but not too stuck. But at some point I think I will have to solve this specific problem. Theoretically I might be able to work around this problem by creating the transpose of my matrix when I read in my original matrix from several files. (I could just create another matrix and do another call to MatSetValue w/ the row and column values switched.) This would just require hacking some code in SLEPc but it seems possible. And my very very naive way of loading the data into the matrix only takes about an hour or so which is substantially better than the times that I'm seeing. However, I am wondering if there is something I'm doing that is really wrong in some way. I am trying to dig through the transpose code to understand but any input anyone can give me would be great. Sarah On Jul 14, 2008, at 2:21 PM, Toby D. Young wrote: > > >> I'll start out by saying I'm a new user to PETSc. I'm actually using >> SLEPc > > Me too. :-) > >> I have a very large, very sparse matrix with dimensions on the order >> of 3 million by 1 million. I created my matrix using MatCreateMPIAIJ >> and it seems to be evenly distributed across my machines. I've hit a >> bottleneck when running some code to compute the SVD of the matrix >> and the bottleneck seems to be using the MatTranspose operation on my >> matrix. > > I can not directly help you with your questions Sarah, sorry. Out of > sheer academic interest in your problem (I had a similar problem > once and > found a work-around): Have you tried to run your code for very small > matrices (say, 300x100)? Just to check that the bottleneck problem > is in > fact not something else! > > Transposing a HUGE matrix such as the one you have is an expensive > operation in anyone's book. As far as I know, PETSc does not store > zero-components of a matrix but uses a sparsity pattern to "remember" > them... (Someone please correct me if I am wrong). > > Hope that is somehow helpful for you. > > Best, > Toby > > > ----- > > Toby D. Young - Adiunkt (Assistant Professor) > Department of Computational Science > Institute of Fundamental Technological Research > Polish Academy of Sciences > Room 206, ul. Swietokrzyska 21 > 00-049 Warszawa, Polska > > +48 22 826 12 81 ext. 184 > http://rav.ippt.gov.pl/~tyoung From tyoung at ippt.gov.pl Mon Jul 14 15:02:32 2008 From: tyoung at ippt.gov.pl (Toby D. Young) Date: Mon, 14 Jul 2008 22:02:32 +0200 (CEST) Subject: MatTranspose Question In-Reply-To: <986B3CEE-E41F-41A7-82A7-C9C1FB3D23D3@cs.umass.edu> References: <986B3CEE-E41F-41A7-82A7-C9C1FB3D23D3@cs.umass.edu> Message-ID: Hej Sarah, > The first thing I did was to run my code to check for correctness on > a matrix of size 202x200. It gave me, what looked to be, the > singular values i was expecting on that matrix. Ok, looks like good news - nothing is seriously wrong. So we can guess you are looking at a matrix "size-problem"(?) > But at some point I think I will have to solve this > specific problem. Sure. :-) > Theoretically I might be able to work around this problem by creating > the transpose of my matrix when I read in my original matrix from > several files. You may have misunderstood me. I meant, as a work around, can you recast your equations in a way that avoids performing the transpose of a matrix? Sometimes this is possible and is almost always desirable. Note: Are you able to use the "inline matrix transpose" rather than by creating a new matrix? If this is possible it will definately save you some time. > However, I am wondering if there is something I'm doing that is > really wrong in some way. I don't suppose you are doing anything particuarily wrong... I can not guess without first seeing your particular problem and code first. best, Toby ----- Toby D. Young - Adiunkt (Assistant Professor) Department of Computational Science Institute of Fundamental Technological Research Polish Academy of Sciences Room 206, ul. Swietokrzyska 21 00-049 Warszawa, Polska +48 22 826 12 81 ext. 184 http://rav.ippt.gov.pl/~tyoung From hzhang at mcs.anl.gov Mon Jul 14 16:08:49 2008 From: hzhang at mcs.anl.gov (Hong Zhang) Date: Mon, 14 Jul 2008 16:08:49 -0500 (CDT) Subject: MatTranspose Question In-Reply-To: References: <986B3CEE-E41F-41A7-82A7-C9C1FB3D23D3@cs.umass.edu> Message-ID: Sarah, You can use runtime option '-log_summary' to find out which function calls dominate execution time. For Large matrices, MatTranspose() is very expensive. PETSc stores mpiaij format in row blocks. Transposing it requires inserting matrix columns one by one and redistributing the matrix entries among processors, which should be avoied whenever possible. You may peek into our implementation MatTranspose_MPIAIJ() in src/mat/impls/aij/mpi/mpiaij.c. If you know the processor distribution and matrix entries, you may create Atranspose and call MatSetValues() for inserting local rows of Atranspose directly. In this way, the expensive processor communications can be avoided. Good luck, Hong On Mon, 14 Jul 2008, Toby D. Young wrote: > > Hej Sarah, > >> The first thing I did was to run my code to check for correctness on >> a matrix of size 202x200. It gave me, what looked to be, the >> singular values i was expecting on that matrix. > > Ok, looks like good news - nothing is seriously wrong. So we can guess you > are looking at a matrix "size-problem"(?) > >> But at some point I think I will have to solve this >> specific problem. > > Sure. :-) > >> Theoretically I might be able to work around this problem by creating >> the transpose of my matrix when I read in my original matrix from >> several files. > > You may have misunderstood me. I meant, as a work around, can you recast > your equations in a way that avoids performing the transpose of a matrix? > Sometimes this is possible and is almost always desirable. Note: Are you > able to use the "inline matrix transpose" rather than by creating a new > matrix? If this is possible it will definately save you some time. > >> However, I am wondering if there is something I'm doing that is >> really wrong in some way. > > I don't suppose you are doing anything particuarily wrong... I can > not guess without first seeing your particular problem and code first. > > best, > Toby > > > ----- > > Toby D. Young - Adiunkt (Assistant Professor) > Department of Computational Science > Institute of Fundamental Technological Research > Polish Academy of Sciences > Room 206, ul. Swietokrzyska 21 > 00-049 Warszawa, Polska > > +48 22 826 12 81 ext. 184 > http://rav.ippt.gov.pl/~tyoung > > From mafunk at nmsu.edu Tue Jul 15 14:44:43 2008 From: mafunk at nmsu.edu (mafunk at nmsu.edu) Date: Tue, 15 Jul 2008 13:44:43 -0600 (MDT) Subject: problems with KSPComputeEigenvalues Message-ID: <52052.128.123.130.224.1216151083.squirrel@webmail.nmsu.edu> Hi, i am trying to get some idea on the spectrum of my system. In order to do that i am calling KSPSetComputeEigenvalues right before calling KSPSetUp. Right after the call to KSPSolve i call KSPComputeEigenvalues as such: ... PetscInt n = -1; PetscReal * realpart; PetscReal * complexpart; PetscInt * neig; KSPComputeEigenvalues(m_ksp, n, realpart, complexpart, neig); ... (I am not sure whether i have to allocate the memory for the arrays, however, i tried that and it made no difference). Anyway, the issue is when it enters the KSPComputeEigenvalues call. The error i get is: ... [0]PETSC ERROR: --------------------- Error Message ------------------------------------ [0]PETSC ERROR: Nonconforming object sizes! [0]PETSC ERROR: Not enough room in work space r and c for eigenvalues! ... What does that mean/how do i debug that? I.e. am i doing something wrong in terms of usage? thanks matt From bsmith at mcs.anl.gov Tue Jul 15 15:23:27 2008 From: bsmith at mcs.anl.gov (Barry Smith) Date: Tue, 15 Jul 2008 15:23:27 -0500 Subject: problems with KSPComputeEigenvalues In-Reply-To: <52052.128.123.130.224.1216151083.squirrel@webmail.nmsu.edu> References: <52052.128.123.130.224.1216151083.squirrel@webmail.nmsu.edu> Message-ID: From the manual page: Notes: The number of eigenvalues estimated depends on the size of the Krylov space generated during the KSPSolve() ; for example, with CG it corresponds to the number of CG iterations, for GMRES it is the number of GMRES iterations SINCE the last restart. Any extra space in r[] and c[] will be ignored. Likely you just have n too small by 1 or something, just make n and the sizes of realpart and complex part a bit longer. Barry On Jul 15, 2008, at 2:44 PM, mafunk at nmsu.edu wrote: > Hi, > > i am trying to get some idea on the spectrum of my system. In order > to do > that i am calling KSPSetComputeEigenvalues right before calling > KSPSetUp. > Right after the call to KSPSolve i call KSPComputeEigenvalues as such: > > ... > > > PetscInt n = -1; > PetscReal * realpart; > PetscReal * complexpart; > PetscInt * neig; > > KSPComputeEigenvalues(m_ksp, n, realpart, complexpart, neig); > > ... > > (I am not sure whether i have to allocate the memory for the arrays, > however, i tried that and it made no difference). > > Anyway, the issue is when it enters the KSPComputeEigenvalues call. > The error i get is: > > ... > > [0]PETSC ERROR: --------------------- Error Message > ------------------------------------ > [0]PETSC ERROR: Nonconforming object sizes! > [0]PETSC ERROR: Not enough room in work space r and c for eigenvalues! > > ... > > What does that mean/how do i debug that? I.e. am i doing something > wrong > in terms of usage? > > > thanks > matt > From ykwok2 at miners.utep.edu Wed Jul 16 19:20:57 2008 From: ykwok2 at miners.utep.edu (Yipkei Kwok) Date: Wed, 16 Jul 2008 18:20:57 -0600 Subject: test cases for GMRES and CG Message-ID: <487E9069.3010708@miners.utep.edu> Hi, Question 1: Test case results for GMRES, one of the Krylov Subspace Methods, do not match the standard results. Question 2: Where can I found test cases for CG, another Krylov Subspace Method? [Question 1: Test case results for GMRES, one of the Krylov Subspace Methods, do not match the standard results.] I have installed PETSc 2.3.3 on an Itanium 2 machine. Then, I tried to verify the correctness of the installation by running test cases in the following directory. src/ksp/ksp/examples/tests I got the following error when running example 17 $ make runex17 17c17 < 16 KSP Residual norm 4.19038e-08 --- > 16 KSP Residual norm 2.9736e-07 29c29 < type=aij, rows=36, cols=36 --- > type=seqaij, rows=36, cols=36 32c32 < Norm of error 9.90874e-10, Iterations 16 --- > Norm of error 7.03149e-09,Iterations 16 Possible problem with ex17_1, diffs above And, I got the following error when running the example 19. $ make runex19 3,21c3,22 < 0 KSP Residual norm 2.34801 < 1 KSP Residual norm 1.10991 < 2 KSP Residual norm 0.905352 < 3 KSP Residual norm 0.719343 < 4 KSP Residual norm 0.542882 < 5 KSP Residual norm 0.378135 < 6 KSP Residual norm 0.2063 < 7 KSP Residual norm 0.0785186 < 8 KSP Residual norm 0.0271851 < 9 KSP Residual norm 0.0154188 < 10 KSP Residual norm 0.00984534 < 11 KSP Residual norm 0.00604218 < 12 KSP Residual norm 0.00189778 < 13 KSP Residual norm 0.000708898 < 14 KSP Residual norm 0.000230128 < 15 KSP Residual norm 9.31276e-05 < 16 KSP Residual norm 3.83561e-05 < 17 KSP Residual norm 2.04223e-05 < Number of iterations = 17 --- > 0 KSP Residual norm 2.90126 > 1 KSP Residual norm 1.45832 > 2 KSP Residual norm 1.18005 > 3 KSP Residual norm 0.941127 > 4 KSP Residual norm 0.696937 > 5 KSP Residual norm 0.484776 > 6 KSP Residual norm 0.245944 > 7 KSP Residual norm 0.102339 > 8 KSP Residual norm 0.0501583 > 9 KSP Residual norm 0.0346147 > 10 KSP Residual norm 0.0196042 > 11 KSP Residual norm 0.0102261 > 12 KSP Residual norm 0.00331366 > 13 KSP Residual norm 0.00153735 > 14 KSP Residual norm 0.000698171 > 15 KSP Residual norm 0.000356882 > 16 KSP Residual norm 0.000136588 > 17 KSP Residual norm 5.23453e-05 > 18 KSP Residual norm 1.50308e-05 > Number of iterations = 18 Possible problem with ex19_1, diffs above Does anyone have a clue why the results are different? Note that I also have PETSc installed on a Xeon machine. The results generated on the Xeon match the results generated on the Itanium 2 machine for ex19. But, for ex17, the results generated on the Xeon machine match neither the standard results nor the results generated on the Itanium 2 machine. [Question 2: Where can I find test cases for CG, another Krylov Subspace Method?] ex17 and ex19 in the src/ksp/ksp/examples/tests directory are using the GMRES method. We want to make sure that the CG method is working correctly. The option `-ksp_type cg' does not appear in any test case of that directory. Where can I find test cases for CG? Or, is there anyway that I can verify the correctness of the CG installation? [Extra information] Here is the machine configuration. Four Intel Itanium 2 processor 8GB memory Red Hat Enterprise Linux AS release 4 (Nahant Update 5) gcc 3.4.6 gfortran 4.1.1 PETSc Configure Options: --with-mpi=0 --prefix=/usr/local/EE/ --with-x=0 --with-x11=0 -download-c-blas-lapack=ifneeded -download-f-blas-lapack=ifneeded --with-shared= 0 --with-scalar-type=complex --configModules=PETSc.Configure --optionsModule=PETSc.compilerOptions Please note that the option `--with-mpi=0' is used as we want a sequential version of PETSc. Thank you very much! Regards, Yipkei -- Join the We Campaign! (http://www.wecansolveit.org/) ******************************************** Yipkei Kwok Ph.D. Student Research Assistant HiPerSys Lab Department of Computer Science The University of Texas at El Paso Phone: 915 747 6433 (O) E-mail: ykwok2 at miners dot utep dot edu ******************************************** From knepley at gmail.com Wed Jul 16 19:29:00 2008 From: knepley at gmail.com (Matthew Knepley) Date: Wed, 16 Jul 2008 19:29:00 -0500 Subject: test cases for GMRES and CG In-Reply-To: <487E9069.3010708@miners.utep.edu> References: <487E9069.3010708@miners.utep.edu> Message-ID: On Wed, Jul 16, 2008 at 7:20 PM, Yipkei Kwok wrote: > Hi, > > Question 1: Test case results for GMRES, one of the Krylov Subspace Methods, > do not match the standard results. These appear to be normal floating point roundoff effects. > Question 2: Where can I found test cases for CG, another Krylov Subspace > Method? You can run all tests using 'make testexamples' Matt > [Question 1: Test case results for GMRES, one of the Krylov Subspace > Methods, do not match the standard results.] > I have installed PETSc 2.3.3 on an Itanium 2 machine. > > Then, I tried to verify the correctness of the installation by running test > cases in the following directory. > src/ksp/ksp/examples/tests > > I got the following error when running example 17 > $ make runex17 > 17c17 > < 16 KSP Residual norm 4.19038e-08 > --- >> 16 KSP Residual norm 2.9736e-07 > 29c29 > < type=aij, rows=36, cols=36 > --- >> type=seqaij, rows=36, cols=36 > 32c32 > < Norm of error 9.90874e-10, Iterations 16 > --- >> Norm of error 7.03149e-09,Iterations 16 > Possible problem with ex17_1, diffs above > > > > > > And, I got the following error when running the example 19. > $ make runex19 > 3,21c3,22 > < 0 KSP Residual norm 2.34801 > < 1 KSP Residual norm 1.10991 > < 2 KSP Residual norm 0.905352 > < 3 KSP Residual norm 0.719343 > < 4 KSP Residual norm 0.542882 > < 5 KSP Residual norm 0.378135 > < 6 KSP Residual norm 0.2063 > < 7 KSP Residual norm 0.0785186 > < 8 KSP Residual norm 0.0271851 > < 9 KSP Residual norm 0.0154188 > < 10 KSP Residual norm 0.00984534 > < 11 KSP Residual norm 0.00604218 > < 12 KSP Residual norm 0.00189778 > < 13 KSP Residual norm 0.000708898 > < 14 KSP Residual norm 0.000230128 > < 15 KSP Residual norm 9.31276e-05 > < 16 KSP Residual norm 3.83561e-05 > < 17 KSP Residual norm 2.04223e-05 > < Number of iterations = 17 > --- >> 0 KSP Residual norm 2.90126 >> 1 KSP Residual norm 1.45832 >> 2 KSP Residual norm 1.18005 >> 3 KSP Residual norm 0.941127 >> 4 KSP Residual norm 0.696937 >> 5 KSP Residual norm 0.484776 >> 6 KSP Residual norm 0.245944 >> 7 KSP Residual norm 0.102339 >> 8 KSP Residual norm 0.0501583 >> 9 KSP Residual norm 0.0346147 >> 10 KSP Residual norm 0.0196042 >> 11 KSP Residual norm 0.0102261 >> 12 KSP Residual norm 0.00331366 >> 13 KSP Residual norm 0.00153735 >> 14 KSP Residual norm 0.000698171 >> 15 KSP Residual norm 0.000356882 >> 16 KSP Residual norm 0.000136588 >> 17 KSP Residual norm 5.23453e-05 >> 18 KSP Residual norm 1.50308e-05 >> Number of iterations = 18 > Possible problem with ex19_1, diffs above > > Does anyone have a clue why the results are different? > > Note that I also have PETSc installed on a Xeon machine. The results > generated on the Xeon match the results generated on the Itanium 2 machine > for ex19. But, for ex17, the results generated on the Xeon machine match > neither the standard results nor the results generated on the Itanium 2 > machine. > > > > > > > [Question 2: Where can I find test cases for CG, another Krylov Subspace > Method?] > > ex17 and ex19 in the src/ksp/ksp/examples/tests directory are using the > GMRES method. We want to make sure that the CG method is working correctly. > The option `-ksp_type cg' does not appear in any test case of that > directory. Where can I find test cases for CG? Or, is there anyway that I > can verify the correctness of the CG installation? > > > > [Extra information] > Here is the machine configuration. > Four Intel Itanium 2 processor > 8GB memory > Red Hat Enterprise Linux AS release 4 (Nahant Update 5) > gcc 3.4.6 > gfortran 4.1.1 > PETSc Configure Options: --with-mpi=0 --prefix=/usr/local/EE/ --with-x=0 > --with-x11=0 > -download-c-blas-lapack=ifneeded -download-f-blas-lapack=ifneeded > --with-shared= > 0 --with-scalar-type=complex --configModules=PETSc.Configure > --optionsModule=PETSc.compilerOptions > > Please note that the option `--with-mpi=0' is used as we want a sequential > version of PETSc. > > > > Thank you very much! > > Regards, > Yipkei > > -- > Join the We Campaign! (http://www.wecansolveit.org/) > > ******************************************** > Yipkei Kwok > Ph.D. Student > Research Assistant > HiPerSys Lab > Department of Computer Science > The University of Texas at El Paso > Phone: 915 747 6433 (O) > E-mail: ykwok2 at miners dot utep dot edu > ******************************************** > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener From dave.mayhem23 at gmail.com Wed Jul 16 22:16:09 2008 From: dave.mayhem23 at gmail.com (Dave May) Date: Thu, 17 Jul 2008 13:16:09 +1000 Subject: semi coarsening with DA's Message-ID: <956373f0807162016u5549463bveba3f8d2fb950233@mail.gmail.com> Hello, Does anyone know the best way to perform semi coarsening using DA's? I was trying to use the following grid sequence level [2]: 20 x 20 level [1]: 11 x 11 level [0]: 6 x 6 where the numbers M x N corresponds to the number of nodes in the x and y direction respectively. However I was unable to generate the interpolation operator using DAGetInterpolation(DA dac,DA daf,Mat *A,Vec *scale) Doing so caused the following error [0]PETSC ERROR: Arguments are incompatible! [0]PETSC ERROR: Ratio between levels: (mx - 1)/(Mx - 1) must be integer: mx 20 Mx 11! Is there a more general way to construct the interpolation operator within petsc for this type of grid sequence? Thanks, Dave. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Thu Jul 17 08:03:31 2008 From: bsmith at mcs.anl.gov (Barry Smith) Date: Thu, 17 Jul 2008 08:03:31 -0500 Subject: semi coarsening with DA's In-Reply-To: <956373f0807162016u5549463bveba3f8d2fb950233@mail.gmail.com> References: <956373f0807162016u5549463bveba3f8d2fb950233@mail.gmail.com> Message-ID: <88D8C0A1-4302-4AE9-BFCA-1026E6309981@mcs.anl.gov> On Jul 16, 2008, at 10:16 PM, Dave May wrote: > Hello, > Does anyone know the best way to perform semi coarsening using > DA's? > I was trying to use the following grid sequence > level [2]: 20 x 20 > level [1]: 11 x 11 > level [0]: 6 x 6 > where the numbers M x N corresponds to the number of nodes in the x > and y > direction respectively. > However I was unable to generate the interpolation operator using > DAGetInterpolation(DA dac,DA daf,Mat *A,Vec *scale) > Doing so caused the following error > [0]PETSC ERROR: Arguments are incompatible! > [0]PETSC ERROR: Ratio between levels: (mx - 1)/(Mx - 1) must be > integer: mx 20 Mx 11! In this case mx = 21, Mx = 11 gives (21 - 1)/(11-1) = 2 works. > > > Is there a more general way to construct the interpolation operator > within petsc for this type of grid sequence? What I usually do is start with a coarse DA, then use DARefine() to generate the finer ones, this makes sure the interpolation will work. You can use DASetRefinementFactor() to control which directions get refined. Barry > > > Thanks, > Dave. > From knepley at gmail.com Thu Jul 17 08:16:32 2008 From: knepley at gmail.com (Matthew Knepley) Date: Thu, 17 Jul 2008 08:16:32 -0500 Subject: semi coarsening with DA's In-Reply-To: <88D8C0A1-4302-4AE9-BFCA-1026E6309981@mcs.anl.gov> References: <956373f0807162016u5549463bveba3f8d2fb950233@mail.gmail.com> <88D8C0A1-4302-4AE9-BFCA-1026E6309981@mcs.anl.gov> Message-ID: On Thu, Jul 17, 2008 at 8:03 AM, Barry Smith wrote: > > On Jul 16, 2008, at 10:16 PM, Dave May wrote: > >> Hello, >> Does anyone know the best way to perform semi coarsening using DA's? >> I was trying to use the following grid sequence >> level [2]: 20 x 20 >> level [1]: 11 x 11 >> level [0]: 6 x 6 >> where the numbers M x N corresponds to the number of nodes in the x and y >> direction respectively. >> However I was unable to generate the interpolation operator using >> DAGetInterpolation(DA dac,DA daf,Mat *A,Vec *scale) >> Doing so caused the following error >> [0]PETSC ERROR: Arguments are incompatible! >> [0]PETSC ERROR: Ratio between levels: (mx - 1)/(Mx - 1) must be integer: >> mx 20 Mx 11! > > In this case mx = 21, Mx = 11 gives (21 - 1)/(11-1) = 2 works. >> >> >> Is there a more general way to construct the interpolation operator >> within petsc for this type of grid sequence? > > What I usually do is start with a coarse DA, then use DARefine() to > generate the finer > ones, this makes sure the interpolation will work. You can use > DASetRefinementFactor() > to control which directions get refined. DAve's question is different. He is trying to use DA for doing staggered methods for Stokes where both A and B^T B are solved with MG, meaning that he cannot adjust grid sizes like that. Is there a way to specify coarse grids himself, but have PETSc calculate the interpolation operators automatically? I would look, but I am on travel :) Matt > Barry > >> >> >> Thanks, >> Dave. -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener From bsmith at mcs.anl.gov Thu Jul 17 08:26:10 2008 From: bsmith at mcs.anl.gov (Barry Smith) Date: Thu, 17 Jul 2008 08:26:10 -0500 Subject: semi coarsening with DA's In-Reply-To: References: <956373f0807162016u5549463bveba3f8d2fb950233@mail.gmail.com> <88D8C0A1-4302-4AE9-BFCA-1026E6309981@mcs.anl.gov> Message-ID: <8D87F45E-3A66-4777-B2FB-0CA653BDAE26@mcs.anl.gov> On Jul 17, 2008, at 8:16 AM, Matthew Knepley wrote: > On Thu, Jul 17, 2008 at 8:03 AM, Barry Smith > wrote: >> >> On Jul 16, 2008, at 10:16 PM, Dave May wrote: >> >>> Hello, >>> Does anyone know the best way to perform semi coarsening using >>> DA's? >>> I was trying to use the following grid sequence >>> level [2]: 20 x 20 >>> level [1]: 11 x 11 >>> level [0]: 6 x 6 >>> where the numbers M x N corresponds to the number of nodes in the >>> x and y >>> direction respectively. >>> However I was unable to generate the interpolation operator using >>> DAGetInterpolation(DA dac,DA daf,Mat *A,Vec *scale) >>> Doing so caused the following error >>> [0]PETSC ERROR: Arguments are incompatible! >>> [0]PETSC ERROR: Ratio between levels: (mx - 1)/(Mx - 1) must be >>> integer: >>> mx 20 Mx 11! >> >> In this case mx = 21, Mx = 11 gives (21 - 1)/(11-1) = 2 works. >>> >>> >>> Is there a more general way to construct the interpolation operator >>> within petsc for this type of grid sequence? >> >> What I usually do is start with a coarse DA, then use DARefine() to >> generate the finer >> ones, this makes sure the interpolation will work. You can use >> DASetRefinementFactor() >> to control which directions get refined. > > DAve's question is different. He is trying to use DA for doing > staggered methods > for Stokes where both A and B^T B are solved with MG, meaning that > he cannot > adjust grid sizes like that. Is there a way to specify coarse grids > himself, but have > PETSc calculate the interpolation operators automatically? Not if they don't satisfy the (mx - 1)/(Mx - 1) = integer rule. It just doesn't make sense. Note that if a staggered grid with like velocities and pressures then you need to use 2 DA's one for the velocity (vertex centered) that uses the (mx - 1) (Mx - 1) rule and one for pressures that uses the mx/Mx = integer rule. (You cannot "hack" and use the same DA for all the variables and "ignore" some of the pressure points). Barry > I would > look, but I am > on travel :) > > Matt > >> Barry >> >>> >>> >>> Thanks, >>> Dave. > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which > their experiments lead. > -- Norbert Wiener > From Nicolas.Flipo at ensmp.fr Thu Jul 17 10:06:26 2008 From: Nicolas.Flipo at ensmp.fr (Nicolas Flipo) Date: Thu, 17 Jul 2008 08:06:26 -0700 Subject: problem with bison Message-ID: <4f6604290807170806t256c717va9afcf6e009cec37@mail.gmail.com> I've got a problem when compiling my code if I include petscksp.h in a *.h Here is the error message (translated in english) user at linux-46jw> make program bison -y -dv newent.y gcc -g -Wall -DYYDEBUG -c -o newent.o y.tab.c In file included from newent.y:166: function.h:27:56: error: petscksp.h: Aucun fichier ou r?pertoire de ce type (no file or folder found) I have no problem to compile PETSc examples, so the environment variable PETSC_DIR is correctely set in my system. I also made a test including petscksp.h directly in the .y and it gave me exactly the same error message, that's why I am suspecting something with bison... FYI here is my Makefile: PROGRAMME= program SOURCES.c = verouille.c hydraulique.c flux_apport.c init_hyd.c matrice.c SOURCES.l = lexique.l SOURCES.y = newent.y SOURCES.h = parametres.h structurs.h var_globals.h functions.h SOURCES.f= CFLAGS_GLOBAUX= -g -Wall -DYYDEBUG CFLAGS= $(CFLAGS_GLOBAUX) YACC= bison -y LEX= flex YFLAGS= -dv LFLAGS= -v CC=gcc OBJ= $(SOURCES.y:.y=.o) $(SOURCES.l:.l=.o) $(SOURCES.c:.c=.o) CLEANFILES = $(OBJ) core y.* include ${PETSC_DIR}/bmake/common/base $(PROGRAMME) : $(OBJ) chkopts -$(CLINKER) -o $(PROGRAMME) $(OBJ) ${PETSC_KSP_LIB} # ${RM} $(OBJ) # rajouter une fois que tout marchera correctement Thanks for your help Nico From balay at mcs.anl.gov Thu Jul 17 11:40:38 2008 From: balay at mcs.anl.gov (Satish Balay) Date: Thu, 17 Jul 2008 11:40:38 -0500 (CDT) Subject: problem with bison In-Reply-To: <4f6604290807170806t256c717va9afcf6e009cec37@mail.gmail.com> References: <4f6604290807170806t256c717va9afcf6e009cec37@mail.gmail.com> Message-ID: Hmm - perhaps you need to specify a '.y.c:' target in your makefile? It loks like make is using a '.y.o:' target - thus skipping '.c.o:' target as defined in PETSc makefiles. > gcc -g -Wall -DYYDEBUG -c -o newent.o y.tab.c If PETSc target for .c file is used - you should see -I$PETSC_DIR/include etc options aswell. Satish On Thu, 17 Jul 2008, Nicolas Flipo wrote: > I've got a problem when compiling my code if I include petscksp.h in a *.h > > Here is the error message (translated in english) > > user at linux-46jw> make program > bison -y -dv newent.y > gcc -g -Wall -DYYDEBUG -c -o newent.o y.tab.c > In file included from newent.y:166: > function.h:27:56: error: petscksp.h: Aucun fichier ou r?pertoire de ce > type (no file or folder found) > > I have no problem to compile PETSc examples, so the environment > variable PETSC_DIR is correctely set in my system. > > I also made a test including petscksp.h directly in the .y and it gave > me exactly the same error message, that's why I am suspecting > something with bison... > > FYI here is my Makefile: > > PROGRAMME= program > SOURCES.c = verouille.c hydraulique.c flux_apport.c init_hyd.c matrice.c > > SOURCES.l = lexique.l > > SOURCES.y = newent.y > > SOURCES.h = parametres.h structurs.h var_globals.h functions.h > > SOURCES.f= > > CFLAGS_GLOBAUX= -g -Wall -DYYDEBUG > CFLAGS= $(CFLAGS_GLOBAUX) > > YACC= bison -y > LEX= flex > YFLAGS= -dv > LFLAGS= -v > > CC=gcc > > OBJ= $(SOURCES.y:.y=.o) $(SOURCES.l:.l=.o) $(SOURCES.c:.c=.o) > > CLEANFILES = $(OBJ) core y.* > > include ${PETSC_DIR}/bmake/common/base > > $(PROGRAMME) : $(OBJ) chkopts > -$(CLINKER) -o $(PROGRAMME) $(OBJ) ${PETSC_KSP_LIB} > # ${RM} $(OBJ) # rajouter une fois que tout marchera correctement > > > Thanks for your help > Nico > > From Nicolas.Flipo at ensmp.fr Thu Jul 17 13:10:26 2008 From: Nicolas.Flipo at ensmp.fr (Nicolas Flipo) Date: Thu, 17 Jul 2008 11:10:26 -0700 Subject: problem with bison In-Reply-To: <4f6604290807171108s2eb6e935h6345006038024ac3@mail.gmail.com> References: <4f6604290807170806t256c717va9afcf6e009cec37@mail.gmail.com> <4f6604290807171108s2eb6e935h6345006038024ac3@mail.gmail.com> Message-ID: <4f6604290807171110p474542b9tb6dea119d31191bf@mail.gmail.com> Hi Satish, I'm not sure to understand exactly what you mean. I tried to use the -I option . It seems that include works out now but I still have a problem with vecimpl.h user at linux-46jw:> make program bison -y -dv newent.y gcc -g -Wall -DYYDEBUG -I/usr/local/bin/petsc-2.3.3-p13/include -I/usr/local/bin/petsc-2.3.3-p13/include/petsc -I/usr/local/bin/petsc-2.3.3-p13/include/mpiuni -I/usr/local/bin/petsc-2.3.3-p13/include/private -I/usr/local/bin/petsc-2.3.3-p13/bmake/linux-gnu-c-debug -c -o newent.o y.tab.c In file included from /usr/local/bin/petsc-2.3.3-p13/include/petscvec.h:427, from /usr/local/bin/petsc-2.3.3-p13/include/petscmat.h:6, from /usr/local/bin/petsc-2.3.3-p13/include/petscpc.h:6, from /usr/local/bin/petsc-2.3.3-p13/include/petscksp.h:6, from function.h:27, from newent.y:166: /usr/local/bin/petsc-2.3.3-p13/include/private/vecimpl.h:236: error: expected specifier-qualifier-list before 'MPI_Win' newent.y: In function 'yyparse': newent.y:740: warning: implicit declaration of function 'calcule_pk_apports' newent.y:1900: warning: unused variable 'lu_fprod' make: *** [newent.o] Error 1 The last thing is that I don't understand why include ${PETSC_DIR}/bmake/common/base doesn't work for bison? Nico On Thu, Jul 17, 2008 at 9:40 AM, Satish Balay wrote: > Hmm - perhaps you need to specify a '.y.c:' target in your makefile? > > It loks like make is using a '.y.o:' target - thus skipping '.c.o:' > target as defined in PETSc makefiles. > >> gcc -g -Wall -DYYDEBUG -c -o newent.o y.tab.c > > If PETSc target for .c file is used - you should see > -I$PETSC_DIR/include etc options aswell. > > Satish > > > > On Thu, 17 Jul 2008, Nicolas Flipo wrote: > >> I've got a problem when compiling my code if I include petscksp.h in a *.h >> >> Here is the error message (translated in english) >> >> user at linux-46jw> make program >> bison -y -dv newent.y >> gcc -g -Wall -DYYDEBUG -c -o newent.o y.tab.c >> In file included from newent.y:166: >> function.h:27:56: error: petscksp.h: Aucun fichier ou r?pertoire de ce >> type (no file or folder found) >> >> I have no problem to compile PETSc examples, so the environment >> variable PETSC_DIR is correctely set in my system. >> >> I also made a test including petscksp.h directly in the .y and it gave >> me exactly the same error message, that's why I am suspecting >> something with bison... >> >> FYI here is my Makefile: >> >> PROGRAMME= program >> SOURCES.c = verouille.c hydraulique.c flux_apport.c init_hyd.c matrice.c >> >> SOURCES.l = lexique.l >> >> SOURCES.y = newent.y >> >> SOURCES.h = parametres.h structurs.h var_globals.h functions.h >> >> SOURCES.f= >> >> CFLAGS_GLOBAUX= -g -Wall -DYYDEBUG >> CFLAGS= $(CFLAGS_GLOBAUX) >> >> YACC= bison -y >> LEX= flex >> YFLAGS= -dv >> LFLAGS= -v >> >> CC=gcc >> >> OBJ= $(SOURCES.y:.y=.o) $(SOURCES.l:.l=.o) $(SOURCES.c:.c=.o) >> >> CLEANFILES = $(OBJ) core y.* >> >> include ${PETSC_DIR}/bmake/common/base >> >> $(PROGRAMME) : $(OBJ) chkopts >> -$(CLINKER) -o $(PROGRAMME) $(OBJ) ${PETSC_KSP_LIB} >> # ${RM} $(OBJ) # rajouter une fois que tout marchera correctement >> >> >> Thanks for your help >> Nico >> >> > From balay at mcs.anl.gov Thu Jul 17 13:49:48 2008 From: balay at mcs.anl.gov (Satish Balay) Date: Thu, 17 Jul 2008 13:49:48 -0500 (CDT) Subject: problem with bison In-Reply-To: <4f6604290807171110p474542b9tb6dea119d31191bf@mail.gmail.com> References: <4f6604290807170806t256c717va9afcf6e009cec37@mail.gmail.com> <4f6604290807171108s2eb6e935h6345006038024ac3@mail.gmail.com> <4f6604290807171110p474542b9tb6dea119d31191bf@mail.gmail.com> Message-ID: Can you send the make output - when you compile a PETSc example? Satish On Thu, 17 Jul 2008, Nicolas Flipo wrote: > Hi Satish, > > I'm not sure to understand exactly what you mean. I tried to use the > -I option . It seems that include works out now but I still have a > problem with vecimpl.h > > user at linux-46jw:> make program > bison -y -dv newent.y > gcc -g -Wall -DYYDEBUG -I/usr/local/bin/petsc-2.3.3-p13/include > -I/usr/local/bin/petsc-2.3.3-p13/include/petsc > -I/usr/local/bin/petsc-2.3.3-p13/include/mpiuni > -I/usr/local/bin/petsc-2.3.3-p13/include/private > -I/usr/local/bin/petsc-2.3.3-p13/bmake/linux-gnu-c-debug -c -o > newent.o y.tab.c > In file included from /usr/local/bin/petsc-2.3.3-p13/include/petscvec.h:427, > from /usr/local/bin/petsc-2.3.3-p13/include/petscmat.h:6, > from /usr/local/bin/petsc-2.3.3-p13/include/petscpc.h:6, > from /usr/local/bin/petsc-2.3.3-p13/include/petscksp.h:6, > from function.h:27, > from newent.y:166: > /usr/local/bin/petsc-2.3.3-p13/include/private/vecimpl.h:236: error: > expected specifier-qualifier-list before 'MPI_Win' > newent.y: In function 'yyparse': > newent.y:740: warning: implicit declaration of function 'calcule_pk_apports' > newent.y:1900: warning: unused variable 'lu_fprod' > make: *** [newent.o] Error 1 > > The last thing is that I don't understand why include > ${PETSC_DIR}/bmake/common/base doesn't work for bison? > Nico > > > On Thu, Jul 17, 2008 at 9:40 AM, Satish Balay wrote: > > Hmm - perhaps you need to specify a '.y.c:' target in your makefile? > > > > It loks like make is using a '.y.o:' target - thus skipping '.c.o:' > > target as defined in PETSc makefiles. > > > >> gcc -g -Wall -DYYDEBUG -c -o newent.o y.tab.c > > > > If PETSc target for .c file is used - you should see > > -I$PETSC_DIR/include etc options aswell. > > > > Satish > > > > > > > > On Thu, 17 Jul 2008, Nicolas Flipo wrote: > > > >> I've got a problem when compiling my code if I include petscksp.h in a *.h > >> > >> Here is the error message (translated in english) > >> > >> user at linux-46jw> make program > >> bison -y -dv newent.y > >> gcc -g -Wall -DYYDEBUG -c -o newent.o y.tab.c > >> In file included from newent.y:166: > >> function.h:27:56: error: petscksp.h: Aucun fichier ou r?pertoire de ce > >> type (no file or folder found) > >> > >> I have no problem to compile PETSc examples, so the environment > >> variable PETSC_DIR is correctely set in my system. > >> > >> I also made a test including petscksp.h directly in the .y and it gave > >> me exactly the same error message, that's why I am suspecting > >> something with bison... > >> > >> FYI here is my Makefile: > >> > >> PROGRAMME= program > >> SOURCES.c = verouille.c hydraulique.c flux_apport.c init_hyd.c matrice.c > >> > >> SOURCES.l = lexique.l > >> > >> SOURCES.y = newent.y > >> > >> SOURCES.h = parametres.h structurs.h var_globals.h functions.h > >> > >> SOURCES.f= > >> > >> CFLAGS_GLOBAUX= -g -Wall -DYYDEBUG > >> CFLAGS= $(CFLAGS_GLOBAUX) > >> > >> YACC= bison -y > >> LEX= flex > >> YFLAGS= -dv > >> LFLAGS= -v > >> > >> CC=gcc > >> > >> OBJ= $(SOURCES.y:.y=.o) $(SOURCES.l:.l=.o) $(SOURCES.c:.c=.o) > >> > >> CLEANFILES = $(OBJ) core y.* > >> > >> include ${PETSC_DIR}/bmake/common/base > >> > >> $(PROGRAMME) : $(OBJ) chkopts > >> -$(CLINKER) -o $(PROGRAMME) $(OBJ) ${PETSC_KSP_LIB} > >> # ${RM} $(OBJ) # rajouter une fois que tout marchera correctement > >> > >> > >> Thanks for your help > >> Nico > >> > >> > > > > From balay at mcs.anl.gov Thu Jul 17 14:00:42 2008 From: balay at mcs.anl.gov (Satish Balay) Date: Thu, 17 Jul 2008 14:00:42 -0500 (CDT) Subject: problem with bison In-Reply-To: <4f6604290807171110p474542b9tb6dea119d31191bf@mail.gmail.com> References: <4f6604290807170806t256c717va9afcf6e009cec37@mail.gmail.com> <4f6604290807171108s2eb6e935h6345006038024ac3@mail.gmail.com> <4f6604290807171110p474542b9tb6dea119d31191bf@mail.gmail.com> Message-ID: What I mean by not having the correct make targets is: PETSc provides a make target for compiling .c files into .o files. i.e .c.o: ${CC} -c -I${PETSC_DIR} -I ${PETSC_DIR}/include $< However when your sourcefile is .y - and not .c, make is using a .y.o target such as: .y.o: ${YACC} $< ${CC} -c ${CFLAGS} -o $*.o y.tab.c This '.y.o:' target might be internal to make. So I was speculating that becuase this other target is used to create .y.o - instead of the '.c.o' from PETSc - the compile is failing. I was sugesting - perhaps you can add a '.y.c' target in your makefile - this way make uses .y -> .c and .c -> .o targets [instead of .y->.o target] .y.c: ${YACC} $< [or whatever the correct syntax for invoking yacc to convert foo.y to foo.c is] Satish On Thu, 17 Jul 2008, Nicolas Flipo wrote: > Hi Satish, > > I'm not sure to understand exactly what you mean. > The last thing is that I don't understand why include > ${PETSC_DIR}/bmake/common/base doesn't work for bison? > Nico > On Thu, Jul 17, 2008 at 9:40 AM, Satish Balay wrote: > > Hmm - perhaps you need to specify a '.y.c:' target in your makefile? > > > > It loks like make is using a '.y.o:' target - thus skipping '.c.o:' > > target as defined in PETSc makefiles. > > > >> gcc -g -Wall -DYYDEBUG -c -o newent.o y.tab.c > > > > If PETSc target for .c file is used - you should see > > -I$PETSC_DIR/include etc options aswell. > > > > Satish From Nicolas.Flipo at ensmp.fr Fri Jul 18 07:38:05 2008 From: Nicolas.Flipo at ensmp.fr (Nicolas Flipo) Date: Fri, 18 Jul 2008 14:38:05 +0200 Subject: problem with bison In-Reply-To: References: <4f6604290807170806t256c717va9afcf6e009cec37@mail.gmail.com> <4f6604290807171108s2eb6e935h6345006038024ac3@mail.gmail.com> <4f6604290807171110p474542b9tb6dea119d31191bf@mail.gmail.com> Message-ID: <4f6604290807180538h192411fcr35a02195e2fe4743@mail.gmail.com> Hi Satish, Thanks for your quick answer but I don't manage to make it works because I don't have enough expertise with makefiles. Thus here is the Makefile of my initial problem (sorry I just saw that I forgot to include the last lines concerning the .y.o in my first message): PROGRAMME= program SOURCES.c = verouille.c \ hydraulique.c debug.c flux_apport.c init_hyd.c\ adv_bott.c adv_decentre.c transport.c dispersion.c adv_caract.c pk.c \ reaere.c tubes.c rayons.c traceur.c \ bact_nit.c timing.c GMcatch_main.c sortie.c verification_basin.c output_basin.c messages.c surf-zns.c \ phyto.c oxygene.c element_nut.c zoo.c bact.c\ geometrie.c calculs_courants.c\ maillage.c init.c biologie.c\ matrice.c SOURCES.l = lexique.l SOURCES.y = newent.y SOURCES.h = parametres.h structurs.h var_globals.h functions.h CFLAGS_GLOBAUX= -g -Wall -DYYDEBUG CFLAGS= $(CFLAGS_GLOBAUX) YACC= bison -y LEX= flex YFLAGS= -dv LFLAGS= -v CC=gcc OBJ= $(SOURCES.y:.y=.o) $(SOURCES.l:.l=.o) $(SOURCES.c:.c=.o) CLEANFILES = $(OBJ) core y.* include ${PETSC_DIR}/bmake/common/base $(PROGRAMME) : $(OBJ) chkopts -$(CLINKER) -o $(PROGRAMME) $(OBJ) ${PETSC_KSP_LIB} .y.o : $(YACC.y) $< $(COMPILE.c) -o $@ y.tab.c giving this error [user at horsdeprix ]$ make program In file included from newent.y:166: var_globals.h:8:35: petscksp.h : File or folder not found I then tried to replace the target .y.o by a .y.c one. I erased the target .y.o and replace it by what you said: .y.c : $(YACC.y) $< what led to the following error [user at horsdeprix ]$ make program bison -y -dv newent.y mpicxx -o newent.o -c -Wall -Wwrite-strings -Wno-strict-aliasing -g -g -Wall -DYYDEBUG -I/usr/local/PETSc/petsc-2.3.3-p13 -I/usr/local/PETSc/petsc-2.3.3-p13/bmake/linux-gnu-c-debug -I/usr/local/PETSc/petsc-2.3.3-p13/include -I. -I/usr/local/include -I/usr/local/include -I/usr/X11R6/include -D__SDIR__="" newent.c c++: newent.c : No files or folder of this type c++: pas de fichier ? l'entr?(c)e make: *** [newent.o] Erreur 1 The thing is that bison -y -dv newent.y creates three files called y.tab.c y.tab.h and y.output If I don't set the -y option I gonna also have three files called newent.tab.c newent.tab.h and newent.output And I don't know how to force the makefile to take y.tab.c and y.tab.h instead of newent.c which does not exist. Maybe it comes from the lines OBJ= $(SOURCES.y:.y=.o) $(SOURCES.l:.l=.o) $(SOURCES.c:.c=.o) and $(PROGRAMME) : $(OBJ) chkopts -$(CLINKER) -o $(PROGRAMME) $(OBJ) ${PETSC_KSP_LIB} Maybe I also have to include the .tab.h at one point of the compiling process. Thanks for your help Nico On Thu, Jul 17, 2008 at 9:00 PM, Satish Balay wrote: > What I mean by not having the correct make targets is: > > PETSc provides a make target for compiling .c files into .o files. i.e > > .c.o: > ${CC} -c -I${PETSC_DIR} -I ${PETSC_DIR}/include $< > > However when your sourcefile is .y - and not .c, make is using a .y.o > target such as: > > .y.o: > ${YACC} $< > ${CC} -c ${CFLAGS} -o $*.o y.tab.c > > > This '.y.o:' target might be internal to make. So I was speculating > that becuase this other target is used to create .y.o - instead of the > '.c.o' from PETSc - the compile is failing. > > I was sugesting - perhaps you can add a '.y.c' target in your makefile > - this way make uses .y -> .c and .c -> .o targets [instead of .y->.o > target] > > .y.c: > ${YACC} $< > > > [or whatever the correct syntax for invoking yacc to convert foo.y to foo.c is] > > Satish > > On Thu, 17 Jul 2008, Nicolas Flipo wrote: > >> Hi Satish, >> >> I'm not sure to understand exactly what you mean. > >> The last thing is that I don't understand why include >> ${PETSC_DIR}/bmake/common/base doesn't work for bison? >> Nico > >> On Thu, Jul 17, 2008 at 9:40 AM, Satish Balay wrote: >> > Hmm - perhaps you need to specify a '.y.c:' target in your makefile? >> > >> > It loks like make is using a '.y.o:' target - thus skipping '.c.o:' >> > target as defined in PETSc makefiles. >> > >> >> gcc -g -Wall -DYYDEBUG -c -o newent.o y.tab.c >> > >> > If PETSc target for .c file is used - you should see >> > -I$PETSC_DIR/include etc options aswell. >> > >> > Satish > > -- ------------------------------------------------------- Nicolas Flipo Centre de G?osciences Syst?mes Hydrologiques et R?servoirs Ecole Nationale Sup?rieure des Mines de Paris 35, rue Saint-Honor? 77305 Fontainebleau t?l.: (33) (0)1 64 69 47 52 e-mail: Nicolas.Flipo at ensmp.fr http://cig.ensmp.fr/~flipo ------------------------------------------------------- From balay at mcs.anl.gov Fri Jul 18 08:49:22 2008 From: balay at mcs.anl.gov (Satish Balay) Date: Fri, 18 Jul 2008 08:49:22 -0500 (CDT) Subject: problem with bison In-Reply-To: <4f6604290807180538h192411fcr35a02195e2fe4743@mail.gmail.com> References: <4f6604290807170806t256c717va9afcf6e009cec37@mail.gmail.com> <4f6604290807171108s2eb6e935h6345006038024ac3@mail.gmail.com> <4f6604290807171110p474542b9tb6dea119d31191bf@mail.gmail.com> <4f6604290807180538h192411fcr35a02195e2fe4743@mail.gmail.com> Message-ID: On Fri, 18 Jul 2008, Nicolas Flipo wrote: > Hi Satish, > > Thanks for your quick answer but I don't manage to make it works > because I don't have enough expertise with makefiles. > > Thus here is the Makefile of my initial problem (sorry I just saw that > I forgot to include the last lines concerning the .y.o in my first > message): > > PROGRAMME= program > SOURCES.c = verouille.c \ > hydraulique.c debug.c flux_apport.c init_hyd.c\ > adv_bott.c adv_decentre.c transport.c dispersion.c adv_caract.c pk.c \ > reaere.c tubes.c rayons.c traceur.c \ > bact_nit.c timing.c GMcatch_main.c sortie.c verification_basin.c > output_basin.c messages.c surf-zns.c \ > phyto.c oxygene.c element_nut.c zoo.c bact.c\ > geometrie.c calculs_courants.c\ > maillage.c init.c biologie.c\ > matrice.c > > SOURCES.l = lexique.l > > SOURCES.y = newent.y > > SOURCES.h = parametres.h structurs.h var_globals.h functions.h > > CFLAGS_GLOBAUX= -g -Wall -DYYDEBUG > > CFLAGS= $(CFLAGS_GLOBAUX) > > YACC= bison -y > LEX= flex > YFLAGS= -dv > LFLAGS= -v > > CC=gcc > > OBJ= $(SOURCES.y:.y=.o) $(SOURCES.l:.l=.o) $(SOURCES.c:.c=.o) > > CLEANFILES = $(OBJ) core y.* > > include ${PETSC_DIR}/bmake/common/base > > $(PROGRAMME) : $(OBJ) chkopts > -$(CLINKER) -o $(PROGRAMME) $(OBJ) ${PETSC_KSP_LIB} > .y.o : > $(YACC.y) $< > $(COMPILE.c) -o $@ y.tab.c > > giving this error > > [user at horsdeprix ]$ make program > In file included from newent.y:166: > var_globals.h:8:35: petscksp.h : File or folder not found > > I then tried to replace the target .y.o by a .y.c one. I erased the > target .y.o and replace it by what you said: > > .y.c : > $(YACC.y) $< > > what led to the following error > > [user at horsdeprix ]$ make program > bison -y -dv newent.y > mpicxx -o newent.o -c -Wall -Wwrite-strings -Wno-strict-aliasing -g -g > -Wall -DYYDEBUG -I/usr/local/PETSc/petsc-2.3.3-p13 > -I/usr/local/PETSc/petsc-2.3.3-p13/bmake/linux-gnu-c-debug > -I/usr/local/PETSc/petsc-2.3.3-p13/include -I. -I/usr/local/include > -I/usr/local/include -I/usr/X11R6/include -D__SDIR__="" newent.c > c++: newent.c : No files or folder of this type > c++: pas de fichier ? l'entr?(c)e > make: *** [newent.o] Erreur 1 > > The thing is that bison -y -dv newent.y creates three files called > y.tab.c y.tab.h and y.output You porbably have to: bison newent.y -o newent.c [I don't know if you need the -y -dv options]. So the target can probably be: .y.c : $(YACC.y) $< -o $*.c Satish > > If I don't set the -y option I gonna also have three files called > newent.tab.c newent.tab.h and newent.output > > And I don't know how to force the makefile to take y.tab.c and y.tab.h > instead of newent.c which does not exist. Maybe it comes from the > lines > > OBJ= $(SOURCES.y:.y=.o) $(SOURCES.l:.l=.o) $(SOURCES.c:.c=.o) > > and > > $(PROGRAMME) : $(OBJ) chkopts > -$(CLINKER) -o $(PROGRAMME) $(OBJ) ${PETSC_KSP_LIB} > > Maybe I also have to include the .tab.h at one point of the compiling process. > > Thanks for your help > Nico > > > On Thu, Jul 17, 2008 at 9:00 PM, Satish Balay wrote: > > What I mean by not having the correct make targets is: > > > > PETSc provides a make target for compiling .c files into .o files. i.e > > > > .c.o: > > ${CC} -c -I${PETSC_DIR} -I ${PETSC_DIR}/include $< > > > > However when your sourcefile is .y - and not .c, make is using a .y.o > > target such as: > > > > .y.o: > > ${YACC} $< > > ${CC} -c ${CFLAGS} -o $*.o y.tab.c > > > > > > This '.y.o:' target might be internal to make. So I was speculating > > that becuase this other target is used to create .y.o - instead of the > > '.c.o' from PETSc - the compile is failing. > > > > I was sugesting - perhaps you can add a '.y.c' target in your makefile > > - this way make uses .y -> .c and .c -> .o targets [instead of .y->.o > > target] > > > > .y.c: > > ${YACC} $< > > > > > > [or whatever the correct syntax for invoking yacc to convert foo.y to foo.c is] > > > > Satish > > > > On Thu, 17 Jul 2008, Nicolas Flipo wrote: > > > >> Hi Satish, > >> > >> I'm not sure to understand exactly what you mean. > > > >> The last thing is that I don't understand why include > >> ${PETSC_DIR}/bmake/common/base doesn't work for bison? > >> Nico > > > >> On Thu, Jul 17, 2008 at 9:40 AM, Satish Balay wrote: > >> > Hmm - perhaps you need to specify a '.y.c:' target in your makefile? > >> > > >> > It loks like make is using a '.y.o:' target - thus skipping '.c.o:' > >> > target as defined in PETSc makefiles. > >> > > >> >> gcc -g -Wall -DYYDEBUG -c -o newent.o y.tab.c > >> > > >> > If PETSc target for .c file is used - you should see > >> > -I$PETSC_DIR/include etc options aswell. > >> > > >> > Satish > > > > > > > > From icksa1 at gmail.com Mon Jul 21 11:39:42 2008 From: icksa1 at gmail.com (Milad Fatenejad) Date: Mon, 21 Jul 2008 11:39:42 -0500 Subject: Integer DA's Message-ID: Hello: I was wondering if it was possible to have distributed arrays that contain integers, instead of floating point numbers? If so, is it possible to have DA's represent arrays of any type? Thanks Milad Fatenejad From bsmith at mcs.anl.gov Mon Jul 21 12:50:38 2008 From: bsmith at mcs.anl.gov (Barry Smith) Date: Mon, 21 Jul 2008 12:50:38 -0500 Subject: Integer DA's In-Reply-To: References: Message-ID: <33C9A295-CC43-4213-8A0B-8BF263CE22C4@mcs.anl.gov> You can compile PETSc so vectors can contain with double precision, single precision or double complex. There is no support for integers. If you are using these as just part of some code then you can just use the double vector and store integers in them. Unless the bulk of your code involves moving integers between processes then you wouldn't even notice any difference in performance. If you entire code involves only moving integers between processes then PETSc is likely not the right tool for you. Barry On Jul 21, 2008, at 11:39 AM, Milad Fatenejad wrote: > Hello: > I was wondering if it was possible to have distributed arrays that > contain integers, instead of floating point numbers? If so, is it > possible to have DA's represent arrays of any type? > > Thanks > Milad Fatenejad > From icksa1 at gmail.com Mon Jul 21 14:43:24 2008 From: icksa1 at gmail.com (Milad Fatenejad) Date: Mon, 21 Jul 2008 14:43:24 -0500 Subject: Integer DA's In-Reply-To: <33C9A295-CC43-4213-8A0B-8BF263CE22C4@mcs.anl.gov> References: <33C9A295-CC43-4213-8A0B-8BF263CE22C4@mcs.anl.gov> Message-ID: Thanks, that answers my question Milad On Mon, Jul 21, 2008 at 12:50 PM, Barry Smith wrote: > > You can compile PETSc so vectors can contain with double precision, single > precision or double > complex. There is no support for integers. If you are using these as just > part of some code > then you can just use the double vector and store integers in them. Unless > the bulk of your > code involves moving integers between processes then you wouldn't even > notice any difference > in performance. If you entire code involves only moving integers between > processes then > PETSc is likely not the right tool for you. > > Barry > > On Jul 21, 2008, at 11:39 AM, Milad Fatenejad wrote: > >> Hello: >> I was wondering if it was possible to have distributed arrays that >> contain integers, instead of floating point numbers? If so, is it >> possible to have DA's represent arrays of any type? >> >> Thanks >> Milad Fatenejad >> > > From icksa1 at gmail.com Mon Jul 21 18:31:16 2008 From: icksa1 at gmail.com (Milad Fatenejad) Date: Mon, 21 Jul 2008 18:31:16 -0500 Subject: Question about DAVecGetArray and DAVecRestoreArray Message-ID: Hello: The documentation states that to access elements of a vector created with a DA you should use the DAVecGetArray function and you should return the pointer when you are finished with the DAVecRestoreArray function. For my code, I would really like to call DAVecGetArray as soon as the vector is created, and call DAVecRestoreArray right before the vector is destroyed. In other words, I would like to be able to access the elements during the vector's entire life and not have to call these two functions every time I need to access individual elements. I would like to know if having access to the elements of the vector prevents me from using it in some way? I am concerned that there might be some operation that won't work while the array is being accessed, i.e. that DAVecGetArray somehow locks the vector so some other function cannot use it until DAVecRestoreArray is called. Is this the case? Thanks for your help. Milad From knepley at gmail.com Mon Jul 21 18:52:54 2008 From: knepley at gmail.com (Matthew Knepley) Date: Mon, 21 Jul 2008 18:52:54 -0500 Subject: Question about DAVecGetArray and DAVecRestoreArray In-Reply-To: References: Message-ID: On Mon, Jul 21, 2008 at 6:31 PM, Milad Fatenejad wrote: > Hello: > The documentation states that to access elements of a vector created > with a DA you should use the DAVecGetArray function and you should > return the pointer when you are finished with the DAVecRestoreArray > function. > > For my code, I would really like to call DAVecGetArray as soon as the > vector is created, and call DAVecRestoreArray right before the vector > is destroyed. In other words, I would like to be able to access the > elements during the vector's entire life and not have to call these > two functions every time I need to access individual elements. > > I would like to know if having access to the elements of the vector > prevents me from using it in some way? I am concerned that there might > be some operation that won't work while the array is being accessed, > i.e. that DAVecGetArray somehow locks the vector so some other > function cannot use it until DAVecRestoreArray is called. Is this the > case? Yes, there are some operations which need to know that you are not changing the vector while they are operating. There is no overhead for calling DAVecGetArray(). Why not call it only when you need access to the internals? Matt > Thanks for your help. > Milad -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener From fernandez858 at gmail.com Mon Jul 21 19:57:27 2008 From: fernandez858 at gmail.com (Michel Cancelliere) Date: Tue, 22 Jul 2008 02:57:27 +0200 Subject: Matlab and PETSc communication Message-ID: <7f18de3b0807211757g4be0dce6vc25e7c39add03c67@mail.gmail.com> Hi, I am a new user of PETSc. I am working in Reservoir Simulation and I have been developing the simulator inside Matlab. I have some question in order to understand better my possibilities of success in what I want to do: - I want to solve the linear system obtained from the inner iterations in the newton method using PETSc, is it possible to communicate in an efficient way PETSc with Matlab to do that? I now that I can write binary files and then read with PETSc but due the size of the matrix it is a very time-expensive way. Where i can find some examples? I look at the examples within the package but I could not find it. \ - It is possible to call PETSc library inside Matlab? Using the Mex files and Matlab compiler? Thank you very much for your time, Michel Cancelliere -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Mon Jul 21 22:14:43 2008 From: bsmith at mcs.anl.gov (Barry Smith) Date: Mon, 21 Jul 2008 22:14:43 -0500 Subject: Matlab and PETSc communication In-Reply-To: <7f18de3b0807211757g4be0dce6vc25e7c39add03c67@mail.gmail.com> References: <7f18de3b0807211757g4be0dce6vc25e7c39add03c67@mail.gmail.com> Message-ID: <7C3139A5-983E-4774-A943-DE1C926C7634@mcs.anl.gov> On Jul 21, 2008, at 7:57 PM, Michel Cancelliere wrote: > Hi, I am a new user of PETSc. I am working in Reservoir Simulation > and I have been developing the simulator inside Matlab. I have some > question in order to understand better my possibilities of success > in what I want to do: > > ? I want to solve the linear system obtained from the inner > iterations in the newton method using PETSc, is it possible to > communicate in an efficient way PETSc with Matlab to do that? I now > that I can write binary files and then read with PETSc but due the > size of the matrix it is a very time-expensive way. Where i can find > some examples? I look at the examples within the package but I could > not find it. \ > ? It is possible to call PETSc library inside Matlab? Using the Mex > files and Matlab compiler? There is no code to do this. It is possible but pretty complicated to write the appropriate Mex code. (Because each Mex function is a separate shared library you cannot just write a Mex function for each PETSc function since they would not share the PETSc global variables. You would have to write one Mex function that is a "gatekeeper" and calls the requested PETSc function underneath. I've monkeyed with this a few times but did not have the time/energy/intellect to write code to automate this process. Give me 300,000 dollars and we could hire someone to write this :-) You might look at the "newly improved" socket interface in petsc- dev (http://www-unix.mcs.anl.gov/petsc/petsc-as/developers/index.html). With this you write a stand alone C PETSc program that waits at a socket, receive the matrix and right hand side and then sends back the solution. The code for marshalling the matrices and vector is common between the sockets and binary files. On the Matlab side you create a "file" that is actually a socket connection. See src/sys/viewer/impls/socket/matlab This may take a little poking around and you asking us a couple of questions to get it. Note there is no inherent support for parallelism on the PETSc side with this setup but I think it is possible. Barry > Thank you very much for your time, > > Michel Cancelliere From knepley at gmail.com Mon Jul 21 23:12:00 2008 From: knepley at gmail.com (Matthew Knepley) Date: Mon, 21 Jul 2008 23:12:00 -0500 Subject: Matlab and PETSc communication In-Reply-To: <7C3139A5-983E-4774-A943-DE1C926C7634@mcs.anl.gov> References: <7f18de3b0807211757g4be0dce6vc25e7c39add03c67@mail.gmail.com> <7C3139A5-983E-4774-A943-DE1C926C7634@mcs.anl.gov> Message-ID: On Mon, Jul 21, 2008 at 10:14 PM, Barry Smith wrote: > > On Jul 21, 2008, at 7:57 PM, Michel Cancelliere wrote: > >> Hi, I am a new user of PETSc. I am working in Reservoir Simulation and I >> have been developing the simulator inside Matlab. I have some question in >> order to understand better my possibilities of success in what I want to do: >> >> ? I want to solve the linear system obtained from the inner >> iterations in the newton method using PETSc, is it possible to communicate >> in an efficient way PETSc with Matlab to do that? I now that I can write >> binary files and then read with PETSc but due the size of the matrix it is a >> very time-expensive way. Where i can find some examples? I look at the >> examples within the package but I could not find it. \ >> ? It is possible to call PETSc library inside Matlab? Using the Mex >> files and Matlab compiler? > > There is no code to do this. It is possible but pretty complicated to > write the appropriate Mex code. (Because > each Mex function is a separate shared library you cannot just write a Mex > function for each PETSc function since they > would not share the PETSc global variables. You would have to write one Mex > function that is a "gatekeeper" and calls > the requested PETSc function underneath. I've monkeyed with this a few times > but did not have the time/energy/intellect > to write code to automate this process. Give me 300,000 dollars and we could > hire someone to write this :-) > > You might look at the "newly improved" socket interface in petsc-dev > (http://www-unix.mcs.anl.gov/petsc/petsc-as/developers/index.html). > With this you write a stand alone C PETSc program that waits at a socket, > receive the matrix and right hand side and then > sends back the solution. The code for marshalling the matrices and vector is > common between the sockets and binary files. > On the Matlab side you create a "file" that is actually a socket connection. > See src/sys/viewer/impls/socket/matlab This may > take a little poking around and you asking us a couple of questions to get > it. > Note there is no inherent support for parallelism on the PETSc side with > this setup but I think it is possible. I personally think this would be much easier in Sage than in Matlab proper. In fact, with Sage you could use petsc4py directly, and directly access the data structures as numpy arrays if necessary. Matt > Barry > > >> Thank you very much for your time, >> >> Michel Cancelliere > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener From icksa1 at gmail.com Tue Jul 22 00:09:28 2008 From: icksa1 at gmail.com (Milad Fatenejad) Date: Tue, 22 Jul 2008 00:09:28 -0500 Subject: Question about DAVecGetArray and DAVecRestoreArray In-Reply-To: References: Message-ID: Hello: Thanks for the reply. That is what I was currently doing, but I thought that if it didn't make any difference, I could cut down on the steps necessary to perform these operations. But I guess I will keep it the way it is, thanks! Thanks Milad From aja2111 at columbia.edu Tue Jul 22 13:56:47 2008 From: aja2111 at columbia.edu (Aron Ahmadia) Date: Tue, 22 Jul 2008 14:56:47 -0400 Subject: Matlab and PETSc communication In-Reply-To: References: <7f18de3b0807211757g4be0dce6vc25e7c39add03c67@mail.gmail.com> <7C3139A5-983E-4774-A943-DE1C926C7634@mcs.anl.gov> Message-ID: <37604ab40807221156i48fa2832rcd5d6ab94371559f@mail.gmail.com> Michel, I would recommend investing the time to write your C/C++ wrapper code a little higher up around the Newton iteration, since PETSc provides a great abstraction interface for it. Then you could write code to build the matrix (or assemble a matrix-free routine!) in C/C++, and pass the parameters in from there. ~Aron On Tue, Jul 22, 2008 at 12:12 AM, Matthew Knepley wrote: > On Mon, Jul 21, 2008 at 10:14 PM, Barry Smith wrote: > > > > On Jul 21, 2008, at 7:57 PM, Michel Cancelliere wrote: > > > >> Hi, I am a new user of PETSc. I am working in Reservoir Simulation and I > >> have been developing the simulator inside Matlab. I have some question > in > >> order to understand better my possibilities of success in what I want to > do: > >> > >> ? I want to solve the linear system obtained from the inner > >> iterations in the newton method using PETSc, is it possible to > communicate > >> in an efficient way PETSc with Matlab to do that? I now that I can > write > >> binary files and then read with PETSc but due the size of the matrix it > is a > >> very time-expensive way. Where i can find some examples? I look at the > >> examples within the package but I could not find it. \ > >> ? It is possible to call PETSc library inside Matlab? Using the > Mex > >> files and Matlab compiler? > > > > There is no code to do this. It is possible but pretty complicated to > > write the appropriate Mex code. (Because > > each Mex function is a separate shared library you cannot just write a > Mex > > function for each PETSc function since they > > would not share the PETSc global variables. You would have to write one > Mex > > function that is a "gatekeeper" and calls > > the requested PETSc function underneath. I've monkeyed with this a few > times > > but did not have the time/energy/intellect > > to write code to automate this process. Give me 300,000 dollars and we > could > > hire someone to write this :-) > > > > You might look at the "newly improved" socket interface in petsc-dev > > (http://www-unix.mcs.anl.gov/petsc/petsc-as/developers/index.html). > > With this you write a stand alone C PETSc program that waits at a socket, > > receive the matrix and right hand side and then > > sends back the solution. The code for marshalling the matrices and vector > is > > common between the sockets and binary files. > > On the Matlab side you create a "file" that is actually a socket > connection. > > See src/sys/viewer/impls/socket/matlab This may > > take a little poking around and you asking us a couple of questions to > get > > it. > > Note there is no inherent support for parallelism on the PETSc side with > > this setup but I think it is possible. > > I personally think this would be much easier in Sage than in Matlab > proper. In fact, > with Sage you could use petsc4py directly, and directly access the > data structures > as numpy arrays if necessary. > > Matt > > > Barry > > > > > >> Thank you very much for your time, > >> > >> Michel Cancelliere > > > > > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which > their experiments lead. > -- Norbert Wiener > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dimitri.lecas at c-s.fr Tue Jul 22 17:09:54 2008 From: dimitri.lecas at c-s.fr (LECAS Dimitri) Date: Wed, 23 Jul 2008 00:09:54 +0200 Subject: MAT_FLUSH_ASSEMBLY Message-ID: <20080723000954.xpd7wlsjuoswc8cc@messagerie.si.c-s.fr> Hi I'am building a matrice (MPISBAIJ) in two steps, the first one i add many contributions with MatSetValue and the flag ADD_VALUES, and the last step i put boundary conditions using the flag INSERT_VALUES. I want to understand how to use MatAssemblyBegin and MatAssemblyEnd. Do I have to put between the two steps call to MatAssemblyBegin and MatAssemblyEnd with MAT_FLUSH_ASSEMBLY flag and after the last step call to MatAssemblyBegin and MatAssemblyEnd with MAT_FINAL_ASSEMBLY flag ? Thanks -- Dimitri Lecas ---------------------------------------------------------------- Ce message electronique et tous les fichiers joints qu'il contient (ci-apres "le message") sont confidentiels et destines exclusivement a l'usage des destinataires indiques ou des personnes dument habilitees a les recevoir a leur place. Si vous recevez ce message par erreur, merci de bien vouloir le detruire et d'en avertir l'emetteur. Toute utilisation de ce message non conforme a sa destination, toute diffusion ou toute publication totale ou partielle est interdite sauf autorisation expresse de l'emetteur. Les idees et opinions exprimees dans ce message sont celles de son auteur et ne representent pas necessairement celles de CS Communication & Systemes ou de ses filiales. Malgre toutes les dispositions prises par CS Communication & Systemes et ses filiales pour minimiser les risques de virus, les fichiers joints a ce message peuvent contenir des virus qui pourraient endommager votre systeme informatique. Il vous appartient d'effectuer vos propres controles anti-virus avant d'ouvrir les fichiers joints. CS Communication & Systemes et ses filiales declinent toute responsabilite pour toute perte ou dommage resultant de l'utilisation de ce message et/ou des fichiers joints. This e-mail and any file attached hereto (hereinafter 'the e-mail') are confidential and intended solely for the use of the adressees indicated below or the persons duly entitled to receive them in their place. If you receive this e-mail in error, please delete it and notify the sender. Any use of this e-mail not in accordance with its purpose, any dissemination or disclosure, either whole or partial, is prohibited, unless formally approved by the sender. The ideas or opinions expressed in this e-mail are solely those of its author and do not necessarily represent those of CS Communication & Systeme or its affiliates. Despite all the measures taken by CS Communication & Systeme and its affiliates in order to minimize the risks of virus, the files attached to this e-mail may contain virus which could damage your information system. You are responsible for processing your own anti-virus checking before opening any file attached hereto. Neither CS Communication & Systemes, nor its affiliates, shall be held liable for any loss or damage due to the use of this e-mail or any file attached hereto. From knepley at gmail.com Tue Jul 22 17:51:17 2008 From: knepley at gmail.com (Matthew Knepley) Date: Tue, 22 Jul 2008 17:51:17 -0500 Subject: MAT_FLUSH_ASSEMBLY In-Reply-To: <20080723000954.xpd7wlsjuoswc8cc@messagerie.si.c-s.fr> References: <20080723000954.xpd7wlsjuoswc8cc@messagerie.si.c-s.fr> Message-ID: 2008/7/22 LECAS Dimitri : > Hi > > I'am building a matrice (MPISBAIJ) in two steps, the first one i add many > contributions with MatSetValue and the flag ADD_VALUES, and the last step i > put boundary conditions using the flag INSERT_VALUES. > > I want to understand how to use MatAssemblyBegin and MatAssemblyEnd. Do I > have to put between the two steps call to MatAssemblyBegin and > MatAssemblyEnd with MAT_FLUSH_ASSEMBLY flag and after the last step call to > MatAssemblyBegin and MatAssemblyEnd with MAT_FINAL_ASSEMBLY flag ? 1) FLUSH allows you to change the insertion mode, and flushes the communication buffers. FINAL does the same and compresses the matrix, etc. What you propose should work. 2) If you are just setting some rows to the identity, you might consider MatZeroRows(). Matt > Thanks > -- > Dimitri Lecas -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener From pbauman at ices.utexas.edu Wed Jul 23 12:41:37 2008 From: pbauman at ices.utexas.edu (Paul T. Bauman) Date: Wed, 23 Jul 2008 12:41:37 -0500 Subject: CHKMEMQ Message-ID: <48876D51.8070905@ices.utexas.edu> Hello, Quick (and probably stupid) question: Does CHKMEMQ check *all* of memory used by the program for corruption or just the memory PETSc allocated? I have a code that I'm assigned to work on that doesn't use PETSc (yet), but I was hoping to use this macro to track down what I think is a memory corruption bug. Thanks, Paul From balay at mcs.anl.gov Wed Jul 23 13:04:28 2008 From: balay at mcs.anl.gov (Satish Balay) Date: Wed, 23 Jul 2008 13:04:28 -0500 (CDT) Subject: CHKMEMQ In-Reply-To: <48876D51.8070905@ices.utexas.edu> References: <48876D51.8070905@ices.utexas.edu> Message-ID: On Wed, 23 Jul 2008, Paul T. Bauman wrote: > Hello, > > Quick (and probably stupid) question: Does CHKMEMQ check *all* of memory used > by the program for corruption or just the memory PETSc allocated? I have a > code that I'm assigned to work on that doesn't use PETSc (yet), but I was > hoping to use this macro to track down what I think is a memory corruption > bug. CHKMEMQ checks memory allocated by PetscMalloc() - for out-of-bounds-write errors. Valgrind is a general tool for detecting many more types of memory corruption. Satish From bsmith at mcs.anl.gov Wed Jul 23 13:25:59 2008 From: bsmith at mcs.anl.gov (Barry Smith) Date: Wed, 23 Jul 2008 13:25:59 -0500 Subject: CHKMEMQ In-Reply-To: References: <48876D51.8070905@ices.utexas.edu> Message-ID: To emphasis Satish's point you should definitely use www.valgrind.org to remove all the memory bugs before starting to use it with PETSc. valgrind is the closest thing to a magic bullet I have ever seen in computing. Barry On Jul 23, 2008, at 1:04 PM, Satish Balay wrote: > On Wed, 23 Jul 2008, Paul T. Bauman wrote: > >> Hello, >> >> Quick (and probably stupid) question: Does CHKMEMQ check *all* of >> memory used >> by the program for corruption or just the memory PETSc allocated? I >> have a >> code that I'm assigned to work on that doesn't use PETSc (yet), but >> I was >> hoping to use this macro to track down what I think is a memory >> corruption >> bug. > > CHKMEMQ checks memory allocated by PetscMalloc() - for > out-of-bounds-write errors. > > Valgrind is a general tool for detecting many more types of memory > corruption. > > Satish > From pbauman at ices.utexas.edu Wed Jul 23 13:31:48 2008 From: pbauman at ices.utexas.edu (Paul T. Bauman) Date: Wed, 23 Jul 2008 13:31:48 -0500 Subject: CHKMEMQ In-Reply-To: References: <48876D51.8070905@ices.utexas.edu> Message-ID: <48877914.4070706@ices.utexas.edu> Thanks guys. Barry Smith wrote: > > To emphasis Satish's point you should definitely use > www.valgrind.org to remove > all the memory bugs before starting to use it with PETSc. > > valgrind is the closest thing to a magic bullet I have ever seen in > computing. > > Barry > > > On Jul 23, 2008, at 1:04 PM, Satish Balay wrote: > >> On Wed, 23 Jul 2008, Paul T. Bauman wrote: >> >>> Hello, >>> >>> Quick (and probably stupid) question: Does CHKMEMQ check *all* of >>> memory used >>> by the program for corruption or just the memory PETSc allocated? I >>> have a >>> code that I'm assigned to work on that doesn't use PETSc (yet), but >>> I was >>> hoping to use this macro to track down what I think is a memory >>> corruption >>> bug. >> >> CHKMEMQ checks memory allocated by PetscMalloc() - for >> out-of-bounds-write errors. >> >> Valgrind is a general tool for detecting many more types of memory >> corruption. >> >> Satish >> > > From aja2111 at columbia.edu Wed Jul 23 13:35:37 2008 From: aja2111 at columbia.edu (Aron Ahmadia) Date: Wed, 23 Jul 2008 14:35:37 -0400 Subject: CHKMEMQ In-Reply-To: References: <48876D51.8070905@ices.utexas.edu> Message-ID: <37604ab40807231135w45da9dc4rd9069164730e9d03@mail.gmail.com> unless you're on an OS X machine, in which case you should use libgmalloc: http://developer.apple.com/documentation/Darwin/Reference/ManPages/man3/libgmalloc.3.html ~A On Wed, Jul 23, 2008 at 2:25 PM, Barry Smith wrote: > > To emphasis Satish's point you should definitely use www.valgrind.org to > remove > all the memory bugs before starting to use it with PETSc. > > valgrind is the closest thing to a magic bullet I have ever seen in > computing. > > Barry > > > > On Jul 23, 2008, at 1:04 PM, Satish Balay wrote: > > On Wed, 23 Jul 2008, Paul T. Bauman wrote: >> >> Hello, >>> >>> Quick (and probably stupid) question: Does CHKMEMQ check *all* of memory >>> used >>> by the program for corruption or just the memory PETSc allocated? I have >>> a >>> code that I'm assigned to work on that doesn't use PETSc (yet), but I was >>> hoping to use this macro to track down what I think is a memory >>> corruption >>> bug. >>> >> >> CHKMEMQ checks memory allocated by PetscMalloc() - for >> out-of-bounds-write errors. >> >> Valgrind is a general tool for detecting many more types of memory >> corruption. >> >> Satish >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Wed Jul 23 13:37:35 2008 From: bsmith at mcs.anl.gov (Barry Smith) Date: Wed, 23 Jul 2008 13:37:35 -0500 Subject: CHKMEMQ In-Reply-To: <37604ab40807231135w45da9dc4rd9069164730e9d03@mail.gmail.com> References: <48876D51.8070905@ices.utexas.edu> <37604ab40807231135w45da9dc4rd9069164730e9d03@mail.gmail.com> Message-ID: <1126DDEF-6870-47D1-9B55-19FE4DE2D7ED@mcs.anl.gov> Sadly libgmalloc doesn't hold a candle to valgrind. Apparently valgrind for the Mac is in the works. Barry If you have the choice between using libgmalloc on your mac, or rebuilding the code on linux and then using valgrind I'd go with valgrind any day. On Jul 23, 2008, at 1:35 PM, Aron Ahmadia wrote: > unless you're on an OS X machine, in which case you should use > libgmalloc: http://developer.apple.com/documentation/Darwin/Reference/ManPages/man3/libgmalloc.3.html > > ~A > > On Wed, Jul 23, 2008 at 2:25 PM, Barry Smith > wrote: > > To emphasis Satish's point you should definitely use > www.valgrind.org to remove > all the memory bugs before starting to use it with PETSc. > > valgrind is the closest thing to a magic bullet I have ever seen in > computing. > > Barry > > > > On Jul 23, 2008, at 1:04 PM, Satish Balay wrote: > > On Wed, 23 Jul 2008, Paul T. Bauman wrote: > > Hello, > > Quick (and probably stupid) question: Does CHKMEMQ check *all* of > memory used > by the program for corruption or just the memory PETSc allocated? I > have a > code that I'm assigned to work on that doesn't use PETSc (yet), but > I was > hoping to use this macro to track down what I think is a memory > corruption > bug. > > CHKMEMQ checks memory allocated by PetscMalloc() - for > out-of-bounds-write errors. > > Valgrind is a general tool for detecting many more types of memory > corruption. > > Satish > > > From kellwood at ameritech.net Fri Jul 25 16:12:07 2008 From: kellwood at ameritech.net (Kevin Ellwood) Date: Fri, 25 Jul 2008 17:12:07 -0400 Subject: Puzzling error Message-ID: Hi All, I am trying to resurrect an old FEM code of mine that was based on a very early version of PETSC (ie. 2.0.x). I am go through all the changes and I am pretty sure I have some issue to get through. However, this set of errors below from an early test run are puzzling me. The relevant lines of code are shown for setting two type of solvers. Both generate an error message indicating the Petsc can't find the KSP type once the KSPSolve is called. It seems odd that I can get such an error since KSPBCGS and KSPGMRES are define in petsc header files and can't be typos or miss-spellings. I know I messed up somewhere. Does anyone have advice as to where I should start looking? Thanks Kevin KSPSetType(ksp_solver,KSPBCGS); ierr = KSPSolve(ksp_solver, rhs, b); CHKERRQ(ierr); [0]PETSC ERROR: --------------------- Error Message ------------------------------------ [0]PETSC ERROR: Unknown type. Check for miss-spelling or missing external package needed for type! [0]PETSC ERROR: Unable to find requested KSP type bcgs! KSPSetType(ksp_solver,KSPGMRES); ierr = KSPSolve(ksp_solver, rhs, b); CHKERRQ(ierr); [0]PETSC ERROR: --------------------- Error Message ------------------------------------ [0]PETSC ERROR: Unknown type. Check for miss-spelling or missing external package needed for type! [0]PETSC ERROR: Unable to find requested KSP type gmres! [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: Petsc Release Version 2.3.3, Patch 3, Fri Jun 15 16:51:25 CDT 2007 HG revision: f051789beadcd36f77fb6111d20225e26ed7cc0d -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Fri Jul 25 16:21:12 2008 From: bsmith at mcs.anl.gov (Barry Smith) Date: Fri, 25 Jul 2008 16:21:12 -0500 Subject: Puzzling error In-Reply-To: References: Message-ID: <58CB1CFD-F211-494E-9E2F-78630F64D70D@mcs.anl.gov> Kevin, Make sure KSPRegisterAll() is called and that KSPRegister() is called specifically for KSPGMRES Barry On Jul 25, 2008, at 4:12 PM, Kevin Ellwood wrote: > Hi All, > > I am trying to resurrect an old FEM code of mine that was based on a > very early version of PETSC (ie. 2.0.x). I am go through all the > changes and I am pretty sure I have some issue to get through. > However, this set of errors below from an early test run are > puzzling me. The relevant lines of code are shown for setting two > type of solvers. Both generate an error message indicating the > Petsc can't find the KSP type once the KSPSolve is called. It > seems odd that I can get such an error since KSPBCGS and KSPGMRES > are define in petsc header files and can't be typos or miss-spellings. > > I know I messed up somewhere. Does anyone have advice as to where I > should start looking? > > Thanks > Kevin > > > KSPSetType(ksp_solver,KSPBCGS); > ierr = KSPSolve(ksp_solver, rhs, b); CHKERRQ(ierr); > > [0]PETSC ERROR: --------------------- Error Message > ------------------------------------ > [0]PETSC ERROR: Unknown type. Check for miss-spelling or missing > external package needed for type! > [0]PETSC ERROR: Unable to find requested KSP type bcgs! > > > KSPSetType(ksp_solver,KSPGMRES); > ierr = KSPSolve(ksp_solver, rhs, b); CHKERRQ(ierr); > > [0]PETSC ERROR: --------------------- Error Message > ------------------------------------ > [0]PETSC ERROR: Unknown type. Check for miss-spelling or missing > external package needed for type! > [0]PETSC ERROR: Unable to find requested KSP type gmres! > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: Petsc Release Version 2.3.3, Patch 3, Fri Jun 15 > 16:51:25 CDT 2007 HG revision: > f051789beadcd36f77fb6111d20225e26ed7cc0d > From kellwood at gmail.com Fri Jul 25 20:56:30 2008 From: kellwood at gmail.com (Kevin Ellwood) Date: Fri, 25 Jul 2008 21:56:30 -0400 Subject: Puzzling error In-Reply-To: <58CB1CFD-F211-494E-9E2F-78630F64D70D@mcs.anl.gov> References: <58CB1CFD-F211-494E-9E2F-78630F64D70D@mcs.anl.gov> Message-ID: Hi and thanks. Are there any example for the usage of these calls? On Fri, Jul 25, 2008 at 5:21 PM, Barry Smith wrote: > > Kevin, > > Make sure KSPRegisterAll() is called and that KSPRegister() is called > specifically for KSPGMRES > > Barry > > > On Jul 25, 2008, at 4:12 PM, Kevin Ellwood wrote: > > Hi All, >> >> I am trying to resurrect an old FEM code of mine that was based on a very >> early version of PETSC (ie. 2.0.x). I am go through all the changes and I >> am pretty sure I have some issue to get through. However, this set of >> errors below from an early test run are puzzling me. The relevant lines of >> code are shown for setting two type of solvers. Both generate an error >> message indicating the Petsc can't find the KSP type once the KSPSolve is >> called. It seems odd that I can get such an error since KSPBCGS and >> KSPGMRES are define in petsc header files and can't be typos or >> miss-spellings. >> >> I know I messed up somewhere. Does anyone have advice as to where I >> should start looking? >> >> Thanks >> Kevin >> >> >> KSPSetType(ksp_solver,KSPBCGS); >> ierr = KSPSolve(ksp_solver, rhs, b); CHKERRQ(ierr); >> >> [0]PETSC ERROR: --------------------- Error Message >> ------------------------------------ >> [0]PETSC ERROR: Unknown type. Check for miss-spelling or missing external >> package needed for type! >> [0]PETSC ERROR: Unable to find requested KSP type bcgs! >> >> >> KSPSetType(ksp_solver,KSPGMRES); >> ierr = KSPSolve(ksp_solver, rhs, b); CHKERRQ(ierr); >> >> [0]PETSC ERROR: --------------------- Error Message >> ------------------------------------ >> [0]PETSC ERROR: Unknown type. Check for miss-spelling or missing external >> package needed for type! >> [0]PETSC ERROR: Unable to find requested KSP type gmres! >> [0]PETSC ERROR: >> ------------------------------------------------------------------------ >> [0]PETSC ERROR: Petsc Release Version 2.3.3, Patch 3, Fri Jun 15 16:51:25 >> CDT 2007 HG revision: f051789beadcd36f77fb6111d20225e26ed7cc0d >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Sat Jul 26 09:14:50 2008 From: knepley at gmail.com (Matthew Knepley) Date: Sat, 26 Jul 2008 09:14:50 -0500 Subject: Puzzling error In-Reply-To: References: <58CB1CFD-F211-494E-9E2F-78630F64D70D@mcs.anl.gov> Message-ID: On Fri, Jul 25, 2008 at 8:56 PM, Kevin Ellwood wrote: > Hi and thanks. Are there any example for the usage of these calls? There is not really a "usage". They are supposed to be called automatically and you never know. However, when using ancient versions of petsc, sometimes things go wrong. I really encourage upgrading to the latest version. Thanks, Matt > On Fri, Jul 25, 2008 at 5:21 PM, Barry Smith wrote: >> >> Kevin, >> >> Make sure KSPRegisterAll() is called and that KSPRegister() is called >> specifically for KSPGMRES >> >> Barry >> >> On Jul 25, 2008, at 4:12 PM, Kevin Ellwood wrote: >> >>> Hi All, >>> >>> I am trying to resurrect an old FEM code of mine that was based on a very >>> early version of PETSC (ie. 2.0.x). I am go through all the changes and I >>> am pretty sure I have some issue to get through. However, this set of >>> errors below from an early test run are puzzling me. The relevant lines of >>> code are shown for setting two type of solvers. Both generate an error >>> message indicating the Petsc can't find the KSP type once the KSPSolve is >>> called. It seems odd that I can get such an error since KSPBCGS and >>> KSPGMRES are define in petsc header files and can't be typos or >>> miss-spellings. >>> >>> I know I messed up somewhere. Does anyone have advice as to where I >>> should start looking? >>> >>> Thanks >>> Kevin >>> >>> >>> KSPSetType(ksp_solver,KSPBCGS); >>> ierr = KSPSolve(ksp_solver, rhs, b); CHKERRQ(ierr); >>> >>> [0]PETSC ERROR: --------------------- Error Message >>> ------------------------------------ >>> [0]PETSC ERROR: Unknown type. Check for miss-spelling or missing external >>> package needed for type! >>> [0]PETSC ERROR: Unable to find requested KSP type bcgs! >>> >>> >>> KSPSetType(ksp_solver,KSPGMRES); >>> ierr = KSPSolve(ksp_solver, rhs, b); CHKERRQ(ierr); >>> >>> [0]PETSC ERROR: --------------------- Error Message >>> ------------------------------------ >>> [0]PETSC ERROR: Unknown type. Check for miss-spelling or missing external >>> package needed for type! >>> [0]PETSC ERROR: Unable to find requested KSP type gmres! >>> [0]PETSC ERROR: >>> ------------------------------------------------------------------------ >>> [0]PETSC ERROR: Petsc Release Version 2.3.3, Patch 3, Fri Jun 15 16:51:25 >>> CDT 2007 HG revision: f051789beadcd36f77fb6111d20225e26ed7cc0d >>> >> > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener From balay at mcs.anl.gov Sat Jul 26 09:23:46 2008 From: balay at mcs.anl.gov (Satish Balay) Date: Sat, 26 Jul 2008 09:23:46 -0500 (CDT) Subject: Puzzling error In-Reply-To: References: <58CB1CFD-F211-494E-9E2F-78630F64D70D@mcs.anl.gov> Message-ID: Things to check while upgrading to latest petsc: - Does a ksp example work? - say src/ksp/ksp/examples/tutorials/ex2.c - If the example works - and yours does not - whats the difference in usage? For ex2, KSPRegisterAll() getrs called from KSPCreate() (gdb) where #0 KSPRegisterAll (path=0x0) at itregis.c:54 #1 0x00007f1ee1ef262e in KSPInitializePackage (path=0x0) at dlregisksp.c:107 #2 0x00007f1ee1f3b031 in KSPCreate (comm=1140850688, inksp=0x7fffea348188) at itcreate.c:383 #3 0x0000000000402259 in main (argc=1, args=0x7fffea3482b8) at ex2.c:157 Satish On Sat, 26 Jul 2008, Matthew Knepley wrote: > On Fri, Jul 25, 2008 at 8:56 PM, Kevin Ellwood wrote: > > Hi and thanks. Are there any example for the usage of these calls? > > There is not really a "usage". They are supposed to be called automatically > and you never know. However, when using ancient versions of petsc, sometimes > things go wrong. I really encourage upgrading to the latest version. > > Thanks, > > Matt > > > On Fri, Jul 25, 2008 at 5:21 PM, Barry Smith wrote: > >> > >> Kevin, > >> > >> Make sure KSPRegisterAll() is called and that KSPRegister() is called > >> specifically for KSPGMRES > >> > >> Barry > >> > >> On Jul 25, 2008, at 4:12 PM, Kevin Ellwood wrote: > >> > >>> Hi All, > >>> > >>> I am trying to resurrect an old FEM code of mine that was based on a very > >>> early version of PETSC (ie. 2.0.x). I am go through all the changes and I > >>> am pretty sure I have some issue to get through. However, this set of > >>> errors below from an early test run are puzzling me. The relevant lines of > >>> code are shown for setting two type of solvers. Both generate an error > >>> message indicating the Petsc can't find the KSP type once the KSPSolve is > >>> called. It seems odd that I can get such an error since KSPBCGS and > >>> KSPGMRES are define in petsc header files and can't be typos or > >>> miss-spellings. > >>> > >>> I know I messed up somewhere. Does anyone have advice as to where I > >>> should start looking? > >>> > >>> Thanks > >>> Kevin > >>> > >>> > >>> KSPSetType(ksp_solver,KSPBCGS); > >>> ierr = KSPSolve(ksp_solver, rhs, b); CHKERRQ(ierr); > >>> > >>> [0]PETSC ERROR: --------------------- Error Message > >>> ------------------------------------ > >>> [0]PETSC ERROR: Unknown type. Check for miss-spelling or missing external > >>> package needed for type! > >>> [0]PETSC ERROR: Unable to find requested KSP type bcgs! > >>> > >>> > >>> KSPSetType(ksp_solver,KSPGMRES); > >>> ierr = KSPSolve(ksp_solver, rhs, b); CHKERRQ(ierr); > >>> > >>> [0]PETSC ERROR: --------------------- Error Message > >>> ------------------------------------ > >>> [0]PETSC ERROR: Unknown type. Check for miss-spelling or missing external > >>> package needed for type! > >>> [0]PETSC ERROR: Unable to find requested KSP type gmres! > >>> [0]PETSC ERROR: > >>> ------------------------------------------------------------------------ > >>> [0]PETSC ERROR: Petsc Release Version 2.3.3, Patch 3, Fri Jun 15 16:51:25 > >>> CDT 2007 HG revision: f051789beadcd36f77fb6111d20225e26ed7cc0d > >>> > >> > > > > > > > > From kellwood at ameritech.net Sat Jul 26 09:26:53 2008 From: kellwood at ameritech.net (Kevin Ellwood) Date: Sat, 26 Jul 2008 10:26:53 -0400 Subject: Puzzling error In-Reply-To: <58CB1CFD-F211-494E-9E2F-78630F64D70D@mcs.anl.gov> References: <58CB1CFD-F211-494E-9E2F-78630F64D70D@mcs.anl.gov> Message-ID: Hi and thanks. I can't really find anything in the examples and this seems to be new since Petsc 2.0.X. Are there any example for the usage of these calls? On Fri, Jul 25, 2008 at 5:21 PM, Barry Smith wrote: > > Kevin, > > Make sure KSPRegisterAll() is called and that KSPRegister() is called > specifically for KSPGMRES > > Barry > > > On Jul 25, 2008, at 4:12 PM, Kevin Ellwood wrote: > > Hi All, >> >> I am trying to resurrect an old FEM code of mine that was based on a very >> early version of PETSC (ie. 2.0.x). I am go through all the changes and I >> am pretty sure I have some issue to get through. However, this set of >> errors below from an early test run are puzzling me. The relevant lines of >> code are shown for setting two type of solvers. Both generate an error >> message indicating the Petsc can't find the KSP type once the KSPSolve is >> called. It seems odd that I can get such an error since KSPBCGS and >> KSPGMRES are define in petsc header files and can't be typos or >> miss-spellings. >> >> I know I messed up somewhere. Does anyone have advice as to where I >> should start looking? >> >> Thanks >> Kevin >> >> >> KSPSetType(ksp_solver,KSPBCGS); >> ierr = KSPSolve(ksp_solver, rhs, b); CHKERRQ(ierr); >> >> [0]PETSC ERROR: --------------------- Error Message >> ------------------------------------ >> [0]PETSC ERROR: Unknown type. Check for miss-spelling or missing external >> package needed for type! >> [0]PETSC ERROR: Unable to find requested KSP type bcgs! >> >> >> KSPSetType(ksp_solver,KSPGMRES); >> ierr = KSPSolve(ksp_solver, rhs, b); CHKERRQ(ierr); >> >> [0]PETSC ERROR: --------------------- Error Message >> ------------------------------------ >> [0]PETSC ERROR: Unknown type. Check for miss-spelling or missing external >> package needed for type! >> [0]PETSC ERROR: Unable to find requested KSP type gmres! >> [0]PETSC ERROR: >> ------------------------------------------------------------------------ >> [0]PETSC ERROR: Petsc Release Version 2.3.3, Patch 3, Fri Jun 15 16:51:25 >> CDT 2007 HG revision: f051789beadcd36f77fb6111d20225e26ed7cc0d >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kellwood at ameritech.net Sat Jul 26 10:40:52 2008 From: kellwood at ameritech.net (Kevin Ellwood) Date: Sat, 26 Jul 2008 11:40:52 -0400 Subject: Puzzling error In-Reply-To: References: <58CB1CFD-F211-494E-9E2F-78630F64D70D@mcs.anl.gov> Message-ID: Ah. I think I have confused you guys. I am using the most recent version pf Petsc (ie. 2.3.3 p13). I am trying to get my old code (which I made using 2.0.22) up and running. I have updated most everything and I am trying to run for the first time. Given what you said, I am guessing that I have messed up in a way the causes KSPRegisterAll and KSPRegister not to be called. Thanks Kev On Sat, Jul 26, 2008 at 10:14 AM, Matthew Knepley wrote: > On Fri, Jul 25, 2008 at 8:56 PM, Kevin Ellwood wrote: > > Hi and thanks. Are there any example for the usage of these calls? > > There is not really a "usage". They are supposed to be called automatically > and you never know. However, when using ancient versions of petsc, > sometimes > things go wrong. I really encourage upgrading to the latest version. > > Thanks, > > Matt > > > On Fri, Jul 25, 2008 at 5:21 PM, Barry Smith wrote: > >> > >> Kevin, > >> > >> Make sure KSPRegisterAll() is called and that KSPRegister() is called > >> specifically for KSPGMRES > >> > >> Barry > >> > >> On Jul 25, 2008, at 4:12 PM, Kevin Ellwood wrote: > >> > >>> Hi All, > >>> > >>> I am trying to resurrect an old FEM code of mine that was based on a > very > >>> early version of PETSC (ie. 2.0.x). I am go through all the changes > and I > >>> am pretty sure I have some issue to get through. However, this set of > >>> errors below from an early test run are puzzling me. The relevant > lines of > >>> code are shown for setting two type of solvers. Both generate an error > >>> message indicating the Petsc can't find the KSP type once the KSPSolve > is > >>> called. It seems odd that I can get such an error since KSPBCGS and > >>> KSPGMRES are define in petsc header files and can't be typos or > >>> miss-spellings. > >>> > >>> I know I messed up somewhere. Does anyone have advice as to where I > >>> should start looking? > >>> > >>> Thanks > >>> Kevin > >>> > >>> > >>> KSPSetType(ksp_solver,KSPBCGS); > >>> ierr = KSPSolve(ksp_solver, rhs, b); CHKERRQ(ierr); > >>> > >>> [0]PETSC ERROR: --------------------- Error Message > >>> ------------------------------------ > >>> [0]PETSC ERROR: Unknown type. Check for miss-spelling or missing > external > >>> package needed for type! > >>> [0]PETSC ERROR: Unable to find requested KSP type bcgs! > >>> > >>> > >>> KSPSetType(ksp_solver,KSPGMRES); > >>> ierr = KSPSolve(ksp_solver, rhs, b); CHKERRQ(ierr); > >>> > >>> [0]PETSC ERROR: --------------------- Error Message > >>> ------------------------------------ > >>> [0]PETSC ERROR: Unknown type. Check for miss-spelling or missing > external > >>> package needed for type! > >>> [0]PETSC ERROR: Unable to find requested KSP type gmres! > >>> [0]PETSC ERROR: > >>> > ------------------------------------------------------------------------ > >>> [0]PETSC ERROR: Petsc Release Version 2.3.3, Patch 3, Fri Jun 15 > 16:51:25 > >>> CDT 2007 HG revision: f051789beadcd36f77fb6111d20225e26ed7cc0d > >>> > >> > > > > > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which > their experiments lead. > -- Norbert Wiener > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Sat Jul 26 11:01:34 2008 From: knepley at gmail.com (Matthew Knepley) Date: Sat, 26 Jul 2008 11:01:34 -0500 Subject: Puzzling error In-Reply-To: References: <58CB1CFD-F211-494E-9E2F-78630F64D70D@mcs.anl.gov> Message-ID: On Sat, Jul 26, 2008 at 10:40 AM, Kevin Ellwood wrote: > Ah. I think I have confused you guys. I am using the most recent version > pf Petsc (ie. 2.3.3 p13). I am trying to get my old code (which I made > using 2.0.22) up and running. I have updated most everything and I am > trying to run for the first time. Given what you said, I am guessing that I > have messed up in a way the causes KSPRegisterAll and KSPRegister not to be > called. Okay, this can happen if somehow you configure using --with-shared or --with-dynamic, but then link with static libraries. Can you send configure.log to petsc-maint? Also, the code would help as well. Matt > Thanks > Kev > > > On Sat, Jul 26, 2008 at 10:14 AM, Matthew Knepley wrote: >> >> On Fri, Jul 25, 2008 at 8:56 PM, Kevin Ellwood wrote: >> > Hi and thanks. Are there any example for the usage of these calls? >> >> There is not really a "usage". They are supposed to be called >> automatically >> and you never know. However, when using ancient versions of petsc, >> sometimes >> things go wrong. I really encourage upgrading to the latest version. >> >> Thanks, >> >> Matt >> >> > On Fri, Jul 25, 2008 at 5:21 PM, Barry Smith wrote: >> >> >> >> Kevin, >> >> >> >> Make sure KSPRegisterAll() is called and that KSPRegister() is called >> >> specifically for KSPGMRES >> >> >> >> Barry >> >> >> >> On Jul 25, 2008, at 4:12 PM, Kevin Ellwood wrote: >> >> >> >>> Hi All, >> >>> >> >>> I am trying to resurrect an old FEM code of mine that was based on a >> >>> very >> >>> early version of PETSC (ie. 2.0.x). I am go through all the changes >> >>> and I >> >>> am pretty sure I have some issue to get through. However, this set of >> >>> errors below from an early test run are puzzling me. The relevant >> >>> lines of >> >>> code are shown for setting two type of solvers. Both generate an >> >>> error >> >>> message indicating the Petsc can't find the KSP type once the KSPSolve >> >>> is >> >>> called. It seems odd that I can get such an error since KSPBCGS and >> >>> KSPGMRES are define in petsc header files and can't be typos or >> >>> miss-spellings. >> >>> >> >>> I know I messed up somewhere. Does anyone have advice as to where I >> >>> should start looking? >> >>> >> >>> Thanks >> >>> Kevin >> >>> >> >>> >> >>> KSPSetType(ksp_solver,KSPBCGS); >> >>> ierr = KSPSolve(ksp_solver, rhs, b); CHKERRQ(ierr); >> >>> >> >>> [0]PETSC ERROR: --------------------- Error Message >> >>> ------------------------------------ >> >>> [0]PETSC ERROR: Unknown type. Check for miss-spelling or missing >> >>> external >> >>> package needed for type! >> >>> [0]PETSC ERROR: Unable to find requested KSP type bcgs! >> >>> >> >>> >> >>> KSPSetType(ksp_solver,KSPGMRES); >> >>> ierr = KSPSolve(ksp_solver, rhs, b); CHKERRQ(ierr); >> >>> >> >>> [0]PETSC ERROR: --------------------- Error Message >> >>> ------------------------------------ >> >>> [0]PETSC ERROR: Unknown type. Check for miss-spelling or missing >> >>> external >> >>> package needed for type! >> >>> [0]PETSC ERROR: Unable to find requested KSP type gmres! >> >>> [0]PETSC ERROR: >> >>> >> >>> ------------------------------------------------------------------------ >> >>> [0]PETSC ERROR: Petsc Release Version 2.3.3, Patch 3, Fri Jun 15 >> >>> 16:51:25 >> >>> CDT 2007 HG revision: f051789beadcd36f77fb6111d20225e26ed7cc0d >> >>> >> >> >> > >> > >> >> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which >> their experiments lead. >> -- Norbert Wiener >> > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener From kellwood at ameritech.net Sat Jul 26 11:27:24 2008 From: kellwood at ameritech.net (Kevin Ellwood) Date: Sat, 26 Jul 2008 09:27:24 -0700 (PDT) Subject: Puzzling error Message-ID: <256990.31277.qm@web81302.mail.mud.yahoo.com> Hi The petsc build was done by the ubuntu (or debian) guys. I think it may be something to do with my Makefile. I should be able to link to the *.a libs and, I am guessing, that this issue will go away. Before wasting you time, ley me look into that and I will get back to you. Thanks for the tips. Kev ----- Original Message ---- From: Matthew Knepley To: petsc-users at mcs.anl.gov Sent: Saturday, July 26, 2008 12:01:34 PM Subject: Re: Puzzling error On Sat, Jul 26, 2008 at 10:40 AM, Kevin Ellwood wrote: > Ah. I think I have confused you guys. I am using the most recent version > pf Petsc (ie. 2.3.3 p13). I am trying to get my old code (which I made > using 2.0.22) up and running. I have updated most everything and I am > trying to run for the first time. Given what you said, I am guessing that I > have messed up in a way the causes KSPRegisterAll and KSPRegister not to be > called. Okay, this can happen if somehow you configure using --with-shared or --with-dynamic, but then link with static libraries. Can you send configure.log to petsc-maint? Also, the code would help as well. Matt > Thanks > Kev > > > On Sat, Jul 26, 2008 at 10:14 AM, Matthew Knepley wrote: >> >> On Fri, Jul 25, 2008 at 8:56 PM, Kevin Ellwood wrote: >> > Hi and thanks. Are there any example for the usage of these calls? >> >> There is not really a "usage". They are supposed to be called >> automatically >> and you never know. However, when using ancient versions of petsc, >> sometimes >> things go wrong. I really encourage upgrading to the latest version. >> >> Thanks, >> >> Matt >> >> > On Fri, Jul 25, 2008 at 5:21 PM, Barry Smith wrote: >> >> >> >> Kevin, >> >> >> >> Make sure KSPRegisterAll() is called and that KSPRegister() is called >> >> specifically for KSPGMRES >> >> >> >> Barry >> >> >> >> On Jul 25, 2008, at 4:12 PM, Kevin Ellwood wrote: >> >> >> >>> Hi All, >> >>> >> >>> I am trying to resurrect an old FEM code of mine that was based on a >> >>> very >> >>> early version of PETSC (ie. 2.0.x). I am go through all the changes >> >>> and I >> >>> am pretty sure I have some issue to get through. However, this set of >> >>> errors below from an early test run are puzzling me. The relevant >> >>> lines of >> >>> code are shown for setting two type of solvers. Both generate an >> >>> error >> >>> message indicating the Petsc can't find the KSP type once the KSPSolve >> >>> is >> >>> called. It seems odd that I can get such an error since KSPBCGS and >> >>> KSPGMRES are define in petsc header files and can't be typos or >> >>> miss-spellings. >> >>> >> >>> I know I messed up somewhere. Does anyone have advice as to where I >> >>> should start looking? >> >>> >> >>> Thanks >> >>> Kevin >> >>> >> >>> >> >>> KSPSetType(ksp_solver,KSPBCGS); >> >>> ierr = KSPSolve(ksp_solver, rhs, b); CHKERRQ(ierr); >> >>> >> >>> [0]PETSC ERROR: --------------------- Error Message >> >>> ------------------------------------ >> >>> [0]PETSC ERROR: Unknown type. Check for miss-spelling or missing >> >>> external >> >>> package needed for type! >> >>> [0]PETSC ERROR: Unable to find requested KSP type bcgs! >> >>> >> >>> >> >>> KSPSetType(ksp_solver,KSPGMRES); >> >>> ierr = KSPSolve(ksp_solver, rhs, b); CHKERRQ(ierr); >> >>> >> >>> [0]PETSC ERROR: --------------------- Error Message >> >>> ------------------------------------ >> >>> [0]PETSC ERROR: Unknown type. Check for miss-spelling or missing >> >>> external >> >>> package needed for type! >> >>> [0]PETSC ERROR: Unable to find requested KSP type gmres! >> >>> [0]PETSC ERROR: >> >>> >> >>> ------------------------------------------------------------------------ >> >>> [0]PETSC ERROR: Petsc Release Version 2.3.3, Patch 3, Fri Jun 15 >> >>> 16:51:25 >> >>> CDT 2007 HG revision: f051789beadcd36f77fb6111d20225e26ed7cc0d >> >>> >> >> >> > >> > >> >> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which >> their experiments lead. >> -- Norbert Wiener >> > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Sat Jul 26 11:53:56 2008 From: bsmith at mcs.anl.gov (Barry Smith) Date: Sat, 26 Jul 2008 11:53:56 -0500 Subject: Puzzling error In-Reply-To: <256990.31277.qm@web81302.mail.mud.yahoo.com> References: <256990.31277.qm@web81302.mail.mud.yahoo.com> Message-ID: If the build is for dynamic libraries then you CANNOT just link against the .a static libraries. It don't work that way. Recommend just compiling PETSc yourself. For linux this is no big deal. Barry On Jul 26, 2008, at 11:27 AM, Kevin Ellwood wrote: > Hi > > The petsc build was done by the ubuntu (or debian) guys. I think it > may be something to do with my Makefile. I should be able to link > to the *.a libs and, I am guessing, that this issue will go away. > Before wasting you time, ley me look into that and I will get back > to you. > > Thanks for the tips. > Kev > > > ----- Original Message ---- > From: Matthew Knepley > To: petsc-users at mcs.anl.gov > Sent: Saturday, July 26, 2008 12:01:34 PM > Subject: Re: Puzzling error > > On Sat, Jul 26, 2008 at 10:40 AM, Kevin Ellwood > wrote: > > Ah. I think I have confused you guys. I am using the most recent > version > > pf Petsc (ie. 2.3.3 p13). I am trying to get my old code (which I > made > > using 2.0.22) up and running. I have updated most everything and > I am > > trying to run for the first time. Given what you said, I am > guessing that I > > have messed up in a way the causes KSPRegisterAll and KSPRegister > not to be > > called. > > Okay, this can happen if somehow you configure using --with-shared or > --with-dynamic, > but then link with static libraries. Can you send configure.log to > petsc-maint? Also, the > code would help as well. > > Matt > > > Thanks > > Kev > > > > > > On Sat, Jul 26, 2008 at 10:14 AM, Matthew Knepley > wrote: > >> > >> On Fri, Jul 25, 2008 at 8:56 PM, Kevin Ellwood > wrote: > >> > Hi and thanks. Are there any example for the usage of these > calls? > >> > >> There is not really a "usage". They are supposed to be called > >> automatically > >> and you never know. However, when using ancient versions of petsc, > >> sometimes > >> things go wrong. I really encourage upgrading to the latest > version. > >> > >> Thanks, > >> > >> Matt > >> > >> > On Fri, Jul 25, 2008 at 5:21 PM, Barry Smith > wrote: > >> >> > >> >> Kevin, > >> >> > >> >> Make sure KSPRegisterAll() is called and that KSPRegister() > is called > >> >> specifically for KSPGMRES > >> >> > >> >> Barry > >> >> > >> >> On Jul 25, 2008, at 4:12 PM, Kevin Ellwood wrote: > >> >> > >> >>> Hi All, > >> >>> > >> >>> I am trying to resurrect an old FEM code of mine that was > based on a > >> >>> very > >> >>> early version of PETSC (ie. 2.0.x). I am go through all the > changes > >> >>> and I > >> >>> am pretty sure I have some issue to get through. However, > this set of > >> >>> errors below from an early test run are puzzling me. The > relevant > >> >>> lines of > >> >>> code are shown for setting two type of solvers. Both > generate an > >> >>> error > >> >>> message indicating the Petsc can't find the KSP type once the > KSPSolve > >> >>> is > >> >>> called. It seems odd that I can get such an error since > KSPBCGS and > >> >>> KSPGMRES are define in petsc header files and can't be typos or > >> >>> miss-spellings. > >> >>> > >> >>> I know I messed up somewhere. Does anyone have advice as to > where I > >> >>> should start looking? > >> >>> > >> >>> Thanks > >> >>> Kevin > >> >>> > >> >>> > >> >>> KSPSetType(ksp_solver,KSPBCGS); > >> >>> ierr = KSPSolve(ksp_solver, rhs, b); CHKERRQ(ierr); > >> >>> > >> >>> [0]PETSC ERROR: --------------------- Error Message > >> >>> ------------------------------------ > >> >>> [0]PETSC ERROR: Unknown type. Check for miss-spelling or > missing > >> >>> external > >> >>> package needed for type! > >> >>> [0]PETSC ERROR: Unable to find requested KSP type bcgs! > >> >>> > >> >>> > >> >>> KSPSetType(ksp_solver,KSPGMRES); > >> >>> ierr = KSPSolve(ksp_solver, rhs, b); CHKERRQ(ierr); > >> >>> > >> >>> [0]PETSC ERROR: --------------------- Error Message > >> >>> ------------------------------------ > >> >>> [0]PETSC ERROR: Unknown type. Check for miss-spelling or > missing > >> >>> external > >> >>> package needed for type! > >> >>> [0]PETSC ERROR: Unable to find requested KSP type gmres! > >> >>> [0]PETSC ERROR: > >> >>> > >> >>> > ------------------------------------------------------------------------ > >> >>> [0]PETSC ERROR: Petsc Release Version 2.3.3, Patch 3, Fri Jun > 15 > >> >>> 16:51:25 > >> >>> CDT 2007 HG revision: f051789beadcd36f77fb6111d20225e26ed7cc0d > >> >>> > >> >> > >> > > >> > > >> > >> > >> > >> -- > >> What most experimenters take for granted before they begin their > >> experiments is infinitely more interesting than any results to > which > >> their experiments lead. > >> -- Norbert Wiener > >> > > > > > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which > their experiments lead. > -- Norbert Wiener > From rafaelsantoscoelho at gmail.com Mon Jul 28 00:16:14 2008 From: rafaelsantoscoelho at gmail.com (Rafael Santos Coelho) Date: Mon, 28 Jul 2008 02:16:14 -0300 Subject: Help in understanding -log_summary output Message-ID: <3b6f83d40807272216s145758d5s42e29f42d512b47e@mail.gmail.com> Hi all Lately, I've been working on this article which covers the design principles that guide the development of the PETSc library and also the main functionalities it offers for the solution of PDEs. One of the things I've pointed out are the profiling capabilities that PETSc includes. I need some help to properly interpret the output from the '-log_summary' command line option, especially its "phase summaries section". I've read the documentation a few times but I still have difficulties in grasping the meaning of the percentages of the statistics (T, R, L and M). For instance, the following line ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Phase Count Time (sec) Flops/sec --- Global --- --- Stage --- Total Max Ratio Max Ratio Mess Avg len Reduct %T %F %M %L %R %T %F %M %L %R Mflop/s ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- (...) --- Event Stage 5: KSPSolve 1 MatMult 56 1.199e+00 1.1 5.3e+07 1.0 1.1e+03 4.2e+03 0.0e+00 5 28 99 23 0 30 28 99 99 0 201 (...) indicates that the 'MatMult' routine was called 56 times, a total of 1.1e+03 messages were sent, with an average length of 4.2e+03 bytes, and there were no global reductions, right? Now, what about the "--- Global ---" and "--- Stage ---" columns? What do those percentages really say? Is it possible to estimate how much time the overall program execution spent performing floating point operations and inter-process communication by looking exclusively at the output? Thanks in advance, Rafael -------------- next part -------------- An HTML attachment was scrubbed... URL: From ajvogel at tuks.co.za Mon Jul 28 06:29:32 2008 From: ajvogel at tuks.co.za (Adolph J. Vogel) Date: Mon, 28 Jul 2008 13:29:32 +0200 Subject: SIGSERV in MPI_Comm_rank Message-ID: <200807281329.33336.ajvogel@tuks.co.za> Hi, I'm having trouble when running a hello world example program with PETSc. Running the same type of program with plain MPI works. This lead me to believe that the error lies with my PETSc setup. I get the following error: $ ./hello MPI process rank 0 (n0, p28410) caught a SIGSEGV in MPI_Comm_rank. Rank (0, MPI_COMM_WORLD): Call stack within LAM: Rank (0, MPI_COMM_WORLD): - MPI_Comm_rank() Rank (0, MPI_COMM_WORLD): - main() and: $ mpirun -np 2 hello ----------------------------------------------------------------------------- One of the processes started by mpirun has exited with a nonzero exit code. This typically indicates that the process finished in error. If your process did not finish in error, be sure to include a "return 0" or "exit(0)" in your C code before exiting the application. PID 28430 failed on node n0 (127.0.0.1) due to signal 11. ----------------------------------------------------------------------------- Does anyone have any idea what the error might be? Thank you in advance. Adolph PS: hello.c: #include "petsc.h" #include int main( int argc, char *argv[] ) { PetscInitialize( &argc, &argv, 0, 0 ); PetscPrintf( PETSC_COMM_WORLD, "Hello World\n" ); PetscFinalize(); return 0; } -- _/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/ Adolph J. Vogel ajvogel at tuks.co.za Department Mechanical and Aeronautical Engineering University of Pretoria, South Africa _/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/ From aja2111 at columbia.edu Mon Jul 28 08:29:02 2008 From: aja2111 at columbia.edu (Aron Ahmadia) Date: Mon, 28 Jul 2008 09:29:02 -0400 Subject: SIGSERV in MPI_Comm_rank In-Reply-To: <200807281329.33336.ajvogel@tuks.co.za> References: <200807281329.33336.ajvogel@tuks.co.za> Message-ID: <37604ab40807280629x26999730sa6fd8e2222d646de@mail.gmail.com> Hi Adolph, What are the results of running make test after you've completed PETSc installation? Do those tests pass? If so, you should try copying over an example makefile and using that to build your code. If they fail, something is wrong with your PETSc installation and you should send a copy of your configure.log to petsc-maint at mcs.anl.gov for more help. http://www-unix.mcs.anl.gov/petsc/petsc-as/documentation/installation.html A SIGSEGV is a segmentation violation, usually caused by your program attempting to access a zero'd memory location. You can run it through the debugger to get a better feel for what's not being initialized, but I think in this case it wouldn't be too helpful. ~A On Mon, Jul 28, 2008 at 7:29 AM, Adolph J. Vogel wrote: > Hi, > > I'm having trouble when running a hello world example program with PETSc. > Running the same type of program with plain MPI works. This lead me to > believe that the error lies with my PETSc setup. > > I get the following error: > > $ ./hello > MPI process rank 0 (n0, p28410) caught a SIGSEGV in MPI_Comm_rank. > Rank (0, MPI_COMM_WORLD): Call stack within LAM: > Rank (0, MPI_COMM_WORLD): - MPI_Comm_rank() > Rank (0, MPI_COMM_WORLD): - main() > > and: > > $ mpirun -np 2 hello > > ----------------------------------------------------------------------------- > One of the processes started by mpirun has exited with a nonzero exit > code. This typically indicates that the process finished in error. > If your process did not finish in error, be sure to include a "return > 0" or "exit(0)" in your C code before exiting the application. > > PID 28430 failed on node n0 (127.0.0.1) due to signal 11. > > ----------------------------------------------------------------------------- > > Does anyone have any idea what the error might be? > > Thank you in advance. > > Adolph > > PS: hello.c: > #include "petsc.h" > #include > > int main( int argc, char *argv[] ) > { > PetscInitialize( &argc, &argv, 0, 0 ); > PetscPrintf( PETSC_COMM_WORLD, "Hello World\n" ); > PetscFinalize(); > return 0; > } > > -- > _/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/ > Adolph J. Vogel ajvogel at tuks.co.za > > Department Mechanical and Aeronautical Engineering > University of Pretoria, South Africa > _/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/ > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fernandez858 at gmail.com Mon Jul 28 12:33:54 2008 From: fernandez858 at gmail.com (Michel Cancelliere) Date: Mon, 28 Jul 2008 19:33:54 +0200 Subject: Matlab and PETSc communication In-Reply-To: <37604ab40807221156i48fa2832rcd5d6ab94371559f@mail.gmail.com> References: <7f18de3b0807211757g4be0dce6vc25e7c39add03c67@mail.gmail.com> <7C3139A5-983E-4774-A943-DE1C926C7634@mcs.anl.gov> <37604ab40807221156i48fa2832rcd5d6ab94371559f@mail.gmail.com> Message-ID: <7f18de3b0807281033h2af92238ha1561944fc12eedc@mail.gmail.com> Hi, Thank you for your response, I think the socket communication between Matlab and PETSc is what I need, I took a look to the example ex12.c and ex12.m (Simple example to show how to PETSc programs can be run from Matlab) but in this case PETSc send the Vector solution that matlab "receive", is it possible to make the inverse? Send the variable from matlab to PETSc using socket communication? can PetscMatlabEngineGet do that ? How it works? Aron, I was studying your recommendation but it means that I have to write all the function from my Matlab Code to C?, because it should take me a lot of time and this work is part of my MSc Thesis and maybe I need a less time consuming solution. Thank you in advance, Michel On Tue, Jul 22, 2008 at 8:56 PM, Aron Ahmadia wrote: > Michel, > > I would recommend investing the time to write your C/C++ wrapper code a > little higher up around the Newton iteration, since PETSc provides a great > abstraction interface for it. Then you could write code to build the matrix > (or assemble a matrix-free routine!) in C/C++, and pass the parameters in > from there. > > ~Aron > > > On Tue, Jul 22, 2008 at 12:12 AM, Matthew Knepley > wrote: > >> On Mon, Jul 21, 2008 at 10:14 PM, Barry Smith wrote: >> > >> > On Jul 21, 2008, at 7:57 PM, Michel Cancelliere wrote: >> > >> >> Hi, I am a new user of PETSc. I am working in Reservoir Simulation and >> I >> >> have been developing the simulator inside Matlab. I have some question >> in >> >> order to understand better my possibilities of success in what I want >> to do: >> >> >> >> ? I want to solve the linear system obtained from the inner >> >> iterations in the newton method using PETSc, is it possible to >> communicate >> >> in an efficient way PETSc with Matlab to do that? I now that I can >> write >> >> binary files and then read with PETSc but due the size of the matrix it >> is a >> >> very time-expensive way. Where i can find some examples? I look at the >> >> examples within the package but I could not find it. \ >> >> ? It is possible to call PETSc library inside Matlab? Using the >> Mex >> >> files and Matlab compiler? >> > >> > There is no code to do this. It is possible but pretty complicated to >> > write the appropriate Mex code. (Because >> > each Mex function is a separate shared library you cannot just write a >> Mex >> > function for each PETSc function since they >> > would not share the PETSc global variables. You would have to write one >> Mex >> > function that is a "gatekeeper" and calls >> > the requested PETSc function underneath. I've monkeyed with this a few >> times >> > but did not have the time/energy/intellect >> > to write code to automate this process. Give me 300,000 dollars and we >> could >> > hire someone to write this :-) >> > >> > You might look at the "newly improved" socket interface in petsc-dev >> > (http://www-unix.mcs.anl.gov/petsc/petsc-as/developers/index.html). >> > With this you write a stand alone C PETSc program that waits at a >> socket, >> > receive the matrix and right hand side and then >> > sends back the solution. The code for marshalling the matrices and >> vector is >> > common between the sockets and binary files. >> > On the Matlab side you create a "file" that is actually a socket >> connection. >> > See src/sys/viewer/impls/socket/matlab This may >> > take a little poking around and you asking us a couple of questions to >> get >> > it. >> > Note there is no inherent support for parallelism on the PETSc side with >> > this setup but I think it is possible. >> >> I personally think this would be much easier in Sage than in Matlab >> proper. In fact, >> with Sage you could use petsc4py directly, and directly access the >> data structures >> as numpy arrays if necessary. >> >> Matt >> >> > Barry >> > >> > >> >> Thank you very much for your time, >> >> >> >> Michel Cancelliere >> > >> > >> >> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which >> their experiments lead. >> -- Norbert Wiener >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fernandez858 at gmail.com Mon Jul 28 12:33:54 2008 From: fernandez858 at gmail.com (Michel Cancelliere) Date: Mon, 28 Jul 2008 19:33:54 +0200 Subject: Matlab and PETSc communication In-Reply-To: <37604ab40807221156i48fa2832rcd5d6ab94371559f@mail.gmail.com> References: <7f18de3b0807211757g4be0dce6vc25e7c39add03c67@mail.gmail.com> <7C3139A5-983E-4774-A943-DE1C926C7634@mcs.anl.gov> <37604ab40807221156i48fa2832rcd5d6ab94371559f@mail.gmail.com> Message-ID: <7f18de3b0807281033h2af92238ha1561944fc12eedc@mail.gmail.com> Hi, Thank you for your response, I think the socket communication between Matlab and PETSc is what I need, I took a look to the example ex12.c and ex12.m (Simple example to show how to PETSc programs can be run from Matlab) but in this case PETSc send the Vector solution that matlab "receive", is it possible to make the inverse? Send the variable from matlab to PETSc using socket communication? can PetscMatlabEngineGet do that ? How it works? Aron, I was studying your recommendation but it means that I have to write all the function from my Matlab Code to C?, because it should take me a lot of time and this work is part of my MSc Thesis and maybe I need a less time consuming solution. Thank you in advance, Michel On Tue, Jul 22, 2008 at 8:56 PM, Aron Ahmadia wrote: > Michel, > > I would recommend investing the time to write your C/C++ wrapper code a > little higher up around the Newton iteration, since PETSc provides a great > abstraction interface for it. Then you could write code to build the matrix > (or assemble a matrix-free routine!) in C/C++, and pass the parameters in > from there. > > ~Aron > > > On Tue, Jul 22, 2008 at 12:12 AM, Matthew Knepley > wrote: > >> On Mon, Jul 21, 2008 at 10:14 PM, Barry Smith wrote: >> > >> > On Jul 21, 2008, at 7:57 PM, Michel Cancelliere wrote: >> > >> >> Hi, I am a new user of PETSc. I am working in Reservoir Simulation and >> I >> >> have been developing the simulator inside Matlab. I have some question >> in >> >> order to understand better my possibilities of success in what I want >> to do: >> >> >> >> ? I want to solve the linear system obtained from the inner >> >> iterations in the newton method using PETSc, is it possible to >> communicate >> >> in an efficient way PETSc with Matlab to do that? I now that I can >> write >> >> binary files and then read with PETSc but due the size of the matrix it >> is a >> >> very time-expensive way. Where i can find some examples? I look at the >> >> examples within the package but I could not find it. \ >> >> ? It is possible to call PETSc library inside Matlab? Using the >> Mex >> >> files and Matlab compiler? >> > >> > There is no code to do this. It is possible but pretty complicated to >> > write the appropriate Mex code. (Because >> > each Mex function is a separate shared library you cannot just write a >> Mex >> > function for each PETSc function since they >> > would not share the PETSc global variables. You would have to write one >> Mex >> > function that is a "gatekeeper" and calls >> > the requested PETSc function underneath. I've monkeyed with this a few >> times >> > but did not have the time/energy/intellect >> > to write code to automate this process. Give me 300,000 dollars and we >> could >> > hire someone to write this :-) >> > >> > You might look at the "newly improved" socket interface in petsc-dev >> > (http://www-unix.mcs.anl.gov/petsc/petsc-as/developers/index.html). >> > With this you write a stand alone C PETSc program that waits at a >> socket, >> > receive the matrix and right hand side and then >> > sends back the solution. The code for marshalling the matrices and >> vector is >> > common between the sockets and binary files. >> > On the Matlab side you create a "file" that is actually a socket >> connection. >> > See src/sys/viewer/impls/socket/matlab This may >> > take a little poking around and you asking us a couple of questions to >> get >> > it. >> > Note there is no inherent support for parallelism on the PETSc side with >> > this setup but I think it is possible. >> >> I personally think this would be much easier in Sage than in Matlab >> proper. In fact, >> with Sage you could use petsc4py directly, and directly access the >> data structures >> as numpy arrays if necessary. >> >> Matt >> >> > Barry >> > >> > >> >> Thank you very much for your time, >> >> >> >> Michel Cancelliere >> > >> > >> >> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which >> their experiments lead. >> -- Norbert Wiener >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Mon Jul 28 13:45:37 2008 From: bsmith at mcs.anl.gov (Barry Smith) Date: Mon, 28 Jul 2008 13:45:37 -0500 Subject: Matlab and PETSc communication In-Reply-To: <7f18de3b0807281033h2af92238ha1561944fc12eedc@mail.gmail.com> References: <7f18de3b0807211757g4be0dce6vc25e7c39add03c67@mail.gmail.com> <7C3139A5-983E-4774-A943-DE1C926C7634@mcs.anl.gov> <37604ab40807221156i48fa2832rcd5d6ab94371559f@mail.gmail.com> <7f18de3b0807281033h2af92238ha1561944fc12eedc@mail.gmail.com> Message-ID: On Jul 28, 2008, at 12:33 PM, Michel Cancelliere wrote: > Hi, > Thank you for your response, I think the socket communication > between Matlab and PETSc is what I need, I took a look to the > example ex12.c and ex12.m (Simple example to show how to PETSc > programs can be run from Matlab) but in this case PETSc send the > Vector solution that matlab "receive", is it possible to make the > inverse? Send the variable from matlab to PETSc using socket > communication? can PetscMatlabEngineGet do that ? How it works? You don't really want to use the Matlab Engine code, because with this approach the PETSc program is "in charge" and just sends work requests to Matlab. Instead you want Matlab in charge and to send work requests to PETSc. The basic idea is that both Matlab and PETSc open a socket connection to each other. On the PETSc side use PetscViewerSocketOpen() on the Matlab side use the bin/matlab/@sreader/sreader.m class to create a (Matlab) socket object. Now on the PETSc side you can make calls to VecView() to send vectors to Matlab and VecLoad() and MatLoad() to read vectors and matrices from Matlab. On the Matlab side use PetscBinaryRead() and PetscBinaryWrite() to read vectors from PETSc and to write vectors and matrices to PETSc. Your PETSc program may end up looking something like PetscViewerSocketOpen(..... &socket); do forever: VecLoad(socket, &b) MatLoad(socket,....,&A) KSPSetOperators(ksp,A,A,....) KSPSolve(ksp,b,x) VecView(socket,x) You Matlab program could look something like socket = sread() do forever: Create your vector and matrix PetscBinaryWrite(socket,b) PetscBinaryWrite(socket,A) x = PetscBinaryRead(socket) The details and correct syntax are obviously not given. Good luck, Barry > > > Aron, I was studying your recommendation but it means that I have to > write all the function from my Matlab Code to C?, because it should > take me a lot of time and this work is part of my MSc Thesis and > maybe I need a less time consuming solution. > > Thank you in advance, > > Michel > > On Tue, Jul 22, 2008 at 8:56 PM, Aron Ahmadia > wrote: > Michel, > > I would recommend investing the time to write your C/C++ wrapper > code a little higher up around the Newton iteration, since PETSc > provides a great abstraction interface for it. Then you could write > code to build the matrix (or assemble a matrix-free routine!) in C/C+ > +, and pass the parameters in from there. > > ~Aron > > > On Tue, Jul 22, 2008 at 12:12 AM, Matthew Knepley > wrote: > On Mon, Jul 21, 2008 at 10:14 PM, Barry Smith > wrote: > > > > On Jul 21, 2008, at 7:57 PM, Michel Cancelliere wrote: > > > >> Hi, I am a new user of PETSc. I am working in Reservoir > Simulation and I > >> have been developing the simulator inside Matlab. I have some > question in > >> order to understand better my possibilities of success in what I > want to do: > >> > >> ? I want to solve the linear system obtained from the inner > >> iterations in the newton method using PETSc, is it possible to > communicate > >> in an efficient way PETSc with Matlab to do that? I now that I > can write > >> binary files and then read with PETSc but due the size of the > matrix it is a > >> very time-expensive way. Where i can find some examples? I look > at the > >> examples within the package but I could not find it. \ > >> ? It is possible to call PETSc library inside Matlab? > Using the Mex > >> files and Matlab compiler? > > > > There is no code to do this. It is possible but pretty > complicated to > > write the appropriate Mex code. (Because > > each Mex function is a separate shared library you cannot just > write a Mex > > function for each PETSc function since they > > would not share the PETSc global variables. You would have to > write one Mex > > function that is a "gatekeeper" and calls > > the requested PETSc function underneath. I've monkeyed with this a > few times > > but did not have the time/energy/intellect > > to write code to automate this process. Give me 300,000 dollars > and we could > > hire someone to write this :-) > > > > You might look at the "newly improved" socket interface in petsc- > dev > > (http://www-unix.mcs.anl.gov/petsc/petsc-as/developers/index.html). > > With this you write a stand alone C PETSc program that waits at a > socket, > > receive the matrix and right hand side and then > > sends back the solution. The code for marshalling the matrices and > vector is > > common between the sockets and binary files. > > On the Matlab side you create a "file" that is actually a socket > connection. > > See src/sys/viewer/impls/socket/matlab This may > > take a little poking around and you asking us a couple of > questions to get > > it. > > Note there is no inherent support for parallelism on the PETSc > side with > > this setup but I think it is possible. > > I personally think this would be much easier in Sage than in Matlab > proper. In fact, > with Sage you could use petsc4py directly, and directly access the > data structures > as numpy arrays if necessary. > > Matt > > > Barry > > > > > >> Thank you very much for your time, > >> > >> Michel Cancelliere > > > > > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which > their experiments lead. > -- Norbert Wiener > > > From bsmith at mcs.anl.gov Mon Jul 28 14:28:09 2008 From: bsmith at mcs.anl.gov (Barry Smith) Date: Mon, 28 Jul 2008 14:28:09 -0500 Subject: Help in understanding -log_summary output In-Reply-To: <3b6f83d40807272216s145758d5s42e29f42d512b47e@mail.gmail.com> References: <3b6f83d40807272216s145758d5s42e29f42d512b47e@mail.gmail.com> Message-ID: <13A48AC6-4277-44B2-8FBA-EE09B2035642@mcs.anl.gov> On Jul 28, 2008, at 12:16 AM, Rafael Santos Coelho wrote: > Hi all > > Lately, I've been working on this article which covers the design > principles that guide the development of the PETSc library and also > the main functionalities it offers for the solution of PDEs. One of > the things I've pointed out are the profiling capabilities that > PETSc includes. > > I need some help to properly interpret the output from the '- > log_summary' command line option, especially its "phase summaries > section". I've read the documentation a few times but I still have > difficulties in grasping the meaning of the percentages of the > statistics (T, R, L and M). For instance, the following line > > ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ > Phase Count Time (sec) Flops/ > sec --- Global > --- --- Stage --- Total > Max Ratio Max Ratio > Mess Avg len Reduct %T %F %M %L %R %T %F %M %L %R Mflop/s > ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > (...) > > --- Event Stage 5: KSPSolve 1 > MatMult 56 1.199e+00 1.1 5.3e+07 1.0 1.1e+03 4.2e+03 0.0e > +00 5 28 99 23 0 30 28 99 99 0 201 > > (...) > > > indicates that the 'MatMult' routine was called 56 times, a total of > 1.1e+03 messages were sent, with an average length of 4.2e+03 bytes, > and there were no global reductions, right? > > Now, what about the "--- Global ---" and "--- Stage ---" columns? > What do those percentages really say? Global gives the percentage of the time/flops/messages/total message data/reductions in this event out of the TOTAL RUNTIME (basically from PetscInitialize() to PETScFinalize(). So 5 percent of the total runtime was in this event and 99 percent of the messages (99 is rounded down from 100 so it is probably essentially 100%). The stage gives the percentage of these things out of the total of these things in that stage. For example the MatMult() is taking 30 percent of the time in the stage. (The numbers in the stage will be at least as large as in the global). > Is it possible to estimate how much time the overall program > execution spent performing floating point operations and inter- > process communication by looking exclusively at the output? Not always, but sometimes pretty close. Looking just at the MatMult() line doesn't tell use how much of that time is communication and how much is floating point. But if you know that MatMult() does ALL its communication inside VecScatterBegin/End() you can say the communication time is the time inside VecScatterBegin/ End() and the floating point time is the MatMult() time minus the VecScatter time. This assumes that the VecScatters are not being used for something else also. For example with the ASM preconditioners there is also VecScatters. So if you used ASM you could not know how much time was spent in the multiply for communication but you would know the communication time of the multiply AND ASM is the time in the scatters and the floating point time in the multiply AND ASM is the time for multiply + asm - the time for the scatters. Barry > > > Thanks in advance, > > Rafael > > > > From rafaelsantoscoelho at gmail.com Mon Jul 28 15:13:00 2008 From: rafaelsantoscoelho at gmail.com (Rafael Santos Coelho) Date: Mon, 28 Jul 2008 17:13:00 -0300 Subject: Help in understanding -log_summary output In-Reply-To: <13A48AC6-4277-44B2-8FBA-EE09B2035642@mcs.anl.gov> References: <3b6f83d40807272216s145758d5s42e29f42d512b47e@mail.gmail.com> <13A48AC6-4277-44B2-8FBA-EE09B2035642@mcs.anl.gov> Message-ID: <3b6f83d40807281313x34e5daa7l516b4817cb3944d5@mail.gmail.com> > Global gives the percentage of the time/flops/messages/total message > data/reductions in this event out of the TOTAL RUNTIME (basically from > PetscInitialize() to PETScFinalize(). So 5 percent of the total runtime was > in this event and 99 percent of the messages (99 is rounded down from 100 > so it is probably essentially 100%). I see, but then how come the sum of all those global percentages don`t equal 100%? -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Mon Jul 28 15:34:30 2008 From: bsmith at mcs.anl.gov (Barry Smith) Date: Mon, 28 Jul 2008 15:34:30 -0500 Subject: Help in understanding -log_summary output In-Reply-To: <3b6f83d40807281313x34e5daa7l516b4817cb3944d5@mail.gmail.com> References: <3b6f83d40807272216s145758d5s42e29f42d512b47e@mail.gmail.com> <13A48AC6-4277-44B2-8FBA-EE09B2035642@mcs.anl.gov> <3b6f83d40807281313x34e5daa7l516b4817cb3944d5@mail.gmail.com> Message-ID: On Jul 28, 2008, at 3:13 PM, Rafael Santos Coelho wrote: > > Global gives the percentage of the time/flops/messages/total > message data/reductions in this event out of the TOTAL RUNTIME > (basically from > PetscInitialize() to PETScFinalize(). So 5 percent of the total > runtime was in this event and 99 percent of the messages (99 is > rounded down from 100 > so it is probably essentially 100%). > > I see, but then how come the sum of all those global percentages > don`t equal 100%? You mean, for example, why doesn't the global %T of MatMult() + global %T of VecNorm() + global %T of VecScatterBegin() etc etc add up to 100%? This is because many of the events are imbedded inside other events (sometimes in complicated ways). For a simple example MatMult() in parallel uses VecScatterBegin/End().... Barry From ajvogel at tuks.co.za Tue Jul 29 01:31:57 2008 From: ajvogel at tuks.co.za (Adolph J. Vogel) Date: Tue, 29 Jul 2008 08:31:57 +0200 Subject: SIGSERV in MPI_Comm_rank In-Reply-To: <37604ab40807280629x26999730sa6fd8e2222d646de@mail.gmail.com> References: <200807281329.33336.ajvogel@tuks.co.za> <37604ab40807280629x26999730sa6fd8e2222d646de@mail.gmail.com> Message-ID: <200807290831.57797.ajvogel@tuks.co.za> Hi Aron, It seems you where correct. There was something wrong with my installation. After removing every thing and reinstalling it worked. All the tests also pass now. Thank you for your assistance. Adolph On Monday 28 July 2008 15:29:02 Aron Ahmadia wrote: > Hi Adolph, > > What are the results of running make test after you've completed PETSc > installation? Do those tests pass? If so, you should try copying over an > example makefile and using that to build your code. If they fail, > something is wrong with your PETSc installation and you should send a copy > of your configure.log to petsc-maint at mcs.anl.gov for more help. > > http://www-unix.mcs.anl.gov/petsc/petsc-as/documentation/installation.html > > A SIGSEGV is a segmentation violation, usually caused by your program > attempting to access a zero'd memory location. You can run it through the > debugger to get a better feel for what's not being initialized, but I think > in this case it wouldn't be too helpful. > > ~A > > On Mon, Jul 28, 2008 at 7:29 AM, Adolph J. Vogel wrote: > > Hi, > > > > I'm having trouble when running a hello world example program with PETSc. > > Running the same type of program with plain MPI works. This lead me to > > believe that the error lies with my PETSc setup. > > > > I get the following error: > > > > $ ./hello > > MPI process rank 0 (n0, p28410) caught a SIGSEGV in MPI_Comm_rank. > > Rank (0, MPI_COMM_WORLD): Call stack within LAM: > > Rank (0, MPI_COMM_WORLD): - MPI_Comm_rank() > > Rank (0, MPI_COMM_WORLD): - main() > > > > and: > > > > $ mpirun -np 2 hello > > > > ------------------------------------------------------------------------- > >---- One of the processes started by mpirun has exited with a nonzero exit > > code. This typically indicates that the process finished in error. If > > your process did not finish in error, be sure to include a "return 0" or > > "exit(0)" in your C code before exiting the application. > > > > PID 28430 failed on node n0 (127.0.0.1) due to signal 11. > > > > ------------------------------------------------------------------------- > >---- > > > > Does anyone have any idea what the error might be? > > > > Thank you in advance. > > > > Adolph > > > > PS: hello.c: > > #include "petsc.h" > > #include > > > > int main( int argc, char *argv[] ) > > { > > PetscInitialize( &argc, &argv, 0, 0 ); > > PetscPrintf( PETSC_COMM_WORLD, "Hello World\n" ); > > PetscFinalize(); > > return 0; > > } > > > > -- > > _/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/ > > Adolph J. Vogel ajvogel at tuks.co.za > > > > Department Mechanical and Aeronautical Engineering > > University of Pretoria, South Africa > > _/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/ -- _/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/ Adolph J. Vogel ajvogel at tuks.co.za Department Mechanical and Aeronautical Engineering University of Pretoria, South Africa _/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/ From harpomaxx at gmail.com Wed Jul 30 03:08:48 2008 From: harpomaxx at gmail.com (Carlos A. Catania (AKA Harpo)) Date: Wed, 30 Jul 2008 10:08:48 +0200 Subject: petsc for 64 bits Message-ID: <44dbb0dd0807300108q1dc0ad76g3295de107f12c7cf@mail.gmail.com> Hello, Perhaps this is not the right place to ask this question because I am using PETSC as a library inside a third-part software. I am using feap[1] which uses PETSC and also Metis for parallel execution support. I have a question related to petsc under 64bits. Feap is written in Fortran and uses 8 bytes integer data type. As far I know this is the normal behavior for AMD 64bit arch (perhaps this is not true). However when feap invokes PETSC libraries, a segfault error messages is shown. I read a little bit about it and the problems seems to be that Petsc uses 4 bytes integers (normal behaviour for IA32 arch) and there are some problems with pointer references which cause the segfault error message. Reading petsc documentation I found an option to compile petsc with 64 bit integer support --with-64-bit-indices=1. I tried it but when compiling several incompatibles error messages are shown. Moreover when I run the test I got a lot of errors like: ------------------------------------------------------------- Possible error running C/C++ src/snes/examples/tutorials/ex19 with 2 MPI processes See http://www.mcs.anl.gov/petsc/petsc-as/documentation/troubleshooting.html [1]PETSC ERROR: --------------------- Error Message ------------------------------------ [1]PETSC ERROR: Null argument, when expecting valid pointer! [1]PETSC ERROR: Null Object: Parameter # 1! [1]PETSC ERROR: ------------------------------------------------------------------------ [1]PETSC ERROR: Petsc Release Version 2.3.3, Patch 3, Fri Jun 15 16:51:25 CDT 2007 HG revision: f051789beadcd36f77fb6111d20225e26ed7cc0d [1]PETSC ERROR: See docs/changes/index.html for recent updates. [1]PETSC ERROR: See docs/faq.html for hints about trouble shooting. [1]PETSC ERROR: See docs/index.html for manual pages. [1]PETSC ERROR: ------------------------------------------------------------------------ So my question is if it is posible to compile petsc with 64bits integer support? Besides --with-64-bit-indices option, is there another option I should add to get petsc tests work? My configuration is: AMD 64 petsc version 2.3.3-p3 Intel C an Fortran compilers v 9.1 MPICH2 1.0.7 I configured petsc with this options: /config/configure.py PETSC_ARCH=GNU/Linux-gnu-amd64 CC=icc FC=ifort --with-mpi-dir=/opt/mpich2/ --download-f-blas-lapack=1 --with-fc=ifort --with-64-bit-indices=1 Thanks in advance [1] Feap:http://www.ce.berkeley.edu/~rlt/feap/ -- Carlos A. Catania (AKA Harpo+) [FP] F152 8D50 6939 E21A 7A90 CDD4 AEE4 FA48 DF70 4D68 From knepley at gmail.com Wed Jul 30 08:08:31 2008 From: knepley at gmail.com (Matthew Knepley) Date: Wed, 30 Jul 2008 08:08:31 -0500 Subject: petsc for 64 bits In-Reply-To: <44dbb0dd0807300108q1dc0ad76g3295de107f12c7cf@mail.gmail.com> References: <44dbb0dd0807300108q1dc0ad76g3295de107f12c7cf@mail.gmail.com> Message-ID: 1) On a 64-bit architecture, ONLY pointers are 64 bits, not integers. 2) The option --with-64-bit-indices is for people with extremely long arrays, not for 64 bit machines. 3) On a 64 bit machine, PETSc will automatically switch to long pointers if the 64 bit compilers are used. 4) I suspect the problem is that when you compile FEAP, It explicitly manipulates the size of an integer, and does not tell us. The correct way to interact with PETSc is to use PetscInt, which is always the right size. Matt On Wed, Jul 30, 2008 at 3:08 AM, Carlos A. Catania (AKA Harpo) wrote: > Hello, > Perhaps this is not the right place to ask this question because I am using > PETSC as a library inside a third-part software. I am using feap[1] which > uses PETSC and also Metis for parallel execution support. > > I have a question related to petsc under 64bits. Feap is written in > Fortran and uses 8 bytes integer data type. As far I know this is the normal > behavior for AMD 64bit arch (perhaps this is not true). However when > feap invokes PETSC libraries, > a segfault error messages is shown. I read a little bit about it and > the problems seems to be that Petsc uses 4 bytes integers (normal > behaviour for IA32 arch) and there > are some problems with pointer references which cause the segfault error > message. > > Reading petsc documentation I found an option to compile petsc with 64 > bit integer support --with-64-bit-indices=1. I tried it but when compiling > several incompatibles error messages are shown. Moreover when I > run the test I got a lot of errors like: > > ------------------------------------------------------------- > Possible error running C/C++ src/snes/examples/tutorials/ex19 with 2 MPI > processes > See > http://www.mcs.anl.gov/petsc/petsc-as/documentation/troubleshooting.html > [1]PETSC ERROR: --------------------- Error Message > ------------------------------------ > [1]PETSC ERROR: Null argument, when expecting valid pointer! > [1]PETSC ERROR: Null Object: Parameter # 1! > [1]PETSC ERROR: > ------------------------------------------------------------------------ > [1]PETSC ERROR: Petsc Release Version 2.3.3, Patch 3, Fri Jun 15 > 16:51:25 CDT 2007 HG revision: f051789beadcd36f77fb6111d20225e26ed7cc0d > [1]PETSC ERROR: See docs/changes/index.html for recent updates. > [1]PETSC ERROR: See docs/faq.html for hints about trouble shooting. > [1]PETSC ERROR: See docs/index.html for manual pages. > [1]PETSC ERROR: > ------------------------------------------------------------------------ > > > So my question is if it is posible to compile petsc with 64bits > integer support? > Besides --with-64-bit-indices option, is there another option I should > add to get petsc tests work? > > My configuration is: > AMD 64 > petsc version 2.3.3-p3 > Intel C an Fortran compilers v 9.1 > MPICH2 1.0.7 > > I configured petsc with this options: > /config/configure.py PETSC_ARCH=GNU/Linux-gnu-amd64 CC=icc FC=ifort > --with-mpi-dir=/opt/mpich2/ --download-f-blas-lapack=1 --with-fc=ifort > --with-64-bit-indices=1 > > Thanks in advance > > [1] Feap:http://www.ce.berkeley.edu/~rlt/feap/ > -- > Carlos A. Catania (AKA Harpo+) > [FP] F152 8D50 6939 E21A 7A90 CDD4 AEE4 FA48 DF70 4D68 > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener From Hung.V.Nguyen at usace.army.mil Wed Jul 30 08:56:57 2008 From: Hung.V.Nguyen at usace.army.mil (Nguyen, Hung V ERDC-ITL-MS) Date: Wed, 30 Jul 2008 08:56:57 -0500 Subject: Petsc with hypre In-Reply-To: References: <44dbb0dd0807300108q1dc0ad76g3295de107f12c7cf@mail.gmail.com> Message-ID: Dear, I tried to run PETSC with hypre on CrayXT3 system and got the error message below. I didn't install PETSC so I don't know whether it was installed with hypre option or not. So if I need to install PETSC with using hypre option, then please send me the instruction for installation (and also using parMeTiS). I appreciate your help, Regards, -hung hvnguyen:sapphire09% yod -np 16 ./fw -ksp_type richardson -pc_type hypre -pc_hypre_type boomeramg -pc_hypre_boomeramg_max_iter 4 -pc_hypre_boomeramg_tol 1.0e-15 [10]PETSC ERROR: [9]PETSC ERROR: [13]PETSC ERROR: [11]PETSC ERROR: [3]PETSC ERROR: [2]PETSC ERROR: [8]PETSC ERROR: [7]PETSC ERROR: [14]PETSC ERROR: [0]PETSC ERROR: [5]PETSC ERROR: [12]PETSC ERROR: [15]PETSC ERROR: [6]PETSC ERROR: [1]PETSC ERROR: [4]PETSC ERROR: --------------------- Error Message ------------------------------------ --------------------- Error Message ------------------------------------ --------------------- Error Message ------------------------------------ --------------------- Error Message ------------------------------------ --------------------- Error Message ------------------------------------ --------------------- Error Message ------------------------------------ --------------------- Error Message ------------------------------------ --------------------- Error Message ------------------------------------ --------------------- Error Message ------------------------------------ --------------------- Error Message ------------------------------------ --------------------- Error Message ------------------------------------ --------------------- Error Message ------------------------------------ --------------------- Error Message ------------------------------------ --------------------- Error Message ------------------------------------ --------------------- Error Message ------------------------------------ --------------------- Error Message ------------------------------------ [10]PETSC ERROR: [9]PETSC ERROR: [13]PETSC ERROR: [11]PETSC ERROR: [3]PETSC ERROR: [2]PETSC ERROR: [8]PETSC ERROR: [7]PETSC ERROR: [14]PETSC ERROR: [0]PETSC ERROR: [5]PETSC ERROR: [12]PETSC ERROR: [15]PETSC ERROR: [6]PETSC ERROR: [1]PETSC ERROR: [4]PETSC ERROR: Unknown type. Check for miss-spelling or missing external package needed for type! Unknown type. Check for miss-spelling or missing external package needed for type! Unknown type. Check for miss-spelling or missing external package needed for type! Unknown type. Check for miss-spelling or missing external package needed for type! [10]PETSC ERROR: [9]PETSC ERROR: [13]PETSC ERROR: [11]PETSC ERROR: [3]PETSC ERROR: [2]PETSC ERROR: [8]PETSC ERROR: [7]PETSC ERROR: [14]PETSC ERROR: [0]PETSC ERROR: [5]PETSC ERROR: [12]PETSC ERROR: [15]PETSC ERROR: [6]PETSC ERROR: [1]PETSC ERROR: [4]PETSC ERROR: Unable to find requested PC type hypre! Unable to find requested PC type hypre! Unable to find requested PC type hypre! Unable to find requested PC type hypre! Unable to find requested PC type hypre! Unable to find requested PC type hypre! Unable to find requested PC type hypre! Unable to find requested PC type hypre! Unable to find requested PC type hypre! Unable to find requested PC type hypre! Unable to find requested PC type hypre! Unable to find requested PC type hypre! Unable to find requested PC type hypre! Unable to find requested PC type hypre! Unable to find requested PC type hypre! Unable to find requested PC type hypre! [10]PETSC ERROR: [9]PETSC ERROR: [13]PETSC ERROR: [11]PETSC ERROR: [3]PETSC ERROR: [2]PETSC ERROR: [8]PETSC ERROR: [7]PETSC ERROR: [14]PETSC ERROR: [0]PETSC ERROR: [5]PETSC ERROR: [12]PETSC ERROR: [15]PETSC ERROR: [6]PETSC ERROR: [1]PETSC ERROR: [4]PETSC ERROR: ------------------------------------------------------------------------ From keita at cray.com Wed Jul 30 09:08:55 2008 From: keita at cray.com (Keita Teranishi) Date: Wed, 30 Jul 2008 09:08:55 -0500 Subject: Petsc with hypre In-Reply-To: References: <44dbb0dd0807300108q1dc0ad76g3295de107f12c7cf@mail.gmail.com> Message-ID: <925346A443D4E340BEB20248BAFCDBDF06A5F685@CFEVS1-IP.americas.cray.com> Hi Hung, Did this error happen with Cray's PETSc? Our PETSc supports both hypre and ParMetis. Please let me know more details with your error (such as module environment). Thanks, ================================ Keita Teranishi Math Software Group Cray, Inc. keita at cray.com ================================ -----Original Message----- From: owner-petsc-users at mcs.anl.gov [mailto:owner-petsc-users at mcs.anl.gov] On Behalf Of Nguyen, Hung V ERDC-ITL-MS Sent: Wednesday, July 30, 2008 8:57 AM To: petsc-users at mcs.anl.gov Subject: Petsc with hypre Dear, I tried to run PETSC with hypre on CrayXT3 system and got the error message below. I didn't install PETSC so I don't know whether it was installed with hypre option or not. So if I need to install PETSC with using hypre option, then please send me the instruction for installation (and also using parMeTiS). I appreciate your help, Regards, -hung hvnguyen:sapphire09% yod -np 16 ./fw -ksp_type richardson -pc_type hypre -pc_hypre_type boomeramg -pc_hypre_boomeramg_max_iter 4 -pc_hypre_boomeramg_tol 1.0e-15 [10]PETSC ERROR: [9]PETSC ERROR: [13]PETSC ERROR: [11]PETSC ERROR: [3]PETSC ERROR: [2]PETSC ERROR: [8]PETSC ERROR: [7]PETSC ERROR: [14]PETSC ERROR: [0]PETSC ERROR: [5]PETSC ERROR: [12]PETSC ERROR: [15]PETSC ERROR: [6]PETSC ERROR: [1]PETSC ERROR: [4]PETSC ERROR: --------------------- Error Message ------------------------------------ --------------------- Error Message ------------------------------------ --------------------- Error Message ------------------------------------ --------------------- Error Message ------------------------------------ --------------------- Error Message ------------------------------------ --------------------- Error Message ------------------------------------ --------------------- Error Message ------------------------------------ --------------------- Error Message ------------------------------------ --------------------- Error Message ------------------------------------ --------------------- Error Message ------------------------------------ --------------------- Error Message ------------------------------------ --------------------- Error Message ------------------------------------ --------------------- Error Message ------------------------------------ --------------------- Error Message ------------------------------------ --------------------- Error Message ------------------------------------ --------------------- Error Message ------------------------------------ [10]PETSC ERROR: [9]PETSC ERROR: [13]PETSC ERROR: [11]PETSC ERROR: [3]PETSC ERROR: [2]PETSC ERROR: [8]PETSC ERROR: [7]PETSC ERROR: [14]PETSC ERROR: [0]PETSC ERROR: [5]PETSC ERROR: [12]PETSC ERROR: [15]PETSC ERROR: [6]PETSC ERROR: [1]PETSC ERROR: [4]PETSC ERROR: Unknown type. Check for miss-spelling or missing external package needed for type! Unknown type. Check for miss-spelling or missing external package needed for type! Unknown type. Check for miss-spelling or missing external package needed for type! Unknown type. Check for miss-spelling or missing external package needed for type! [10]PETSC ERROR: [9]PETSC ERROR: [13]PETSC ERROR: [11]PETSC ERROR: [3]PETSC ERROR: [2]PETSC ERROR: [8]PETSC ERROR: [7]PETSC ERROR: [14]PETSC ERROR: [0]PETSC ERROR: [5]PETSC ERROR: [12]PETSC ERROR: [15]PETSC ERROR: [6]PETSC ERROR: [1]PETSC ERROR: [4]PETSC ERROR: Unable to find requested PC type hypre! Unable to find requested PC type hypre! Unable to find requested PC type hypre! Unable to find requested PC type hypre! Unable to find requested PC type hypre! Unable to find requested PC type hypre! Unable to find requested PC type hypre! Unable to find requested PC type hypre! Unable to find requested PC type hypre! Unable to find requested PC type hypre! Unable to find requested PC type hypre! Unable to find requested PC type hypre! Unable to find requested PC type hypre! Unable to find requested PC type hypre! Unable to find requested PC type hypre! Unable to find requested PC type hypre! [10]PETSC ERROR: [9]PETSC ERROR: [13]PETSC ERROR: [11]PETSC ERROR: [3]PETSC ERROR: [2]PETSC ERROR: [8]PETSC ERROR: [7]PETSC ERROR: [14]PETSC ERROR: [0]PETSC ERROR: [5]PETSC ERROR: [12]PETSC ERROR: [15]PETSC ERROR: [6]PETSC ERROR: [1]PETSC ERROR: [4]PETSC ERROR: ------------------------------------------------------------------------ From sanjay at ce.berkeley.edu Wed Jul 30 09:02:00 2008 From: sanjay at ce.berkeley.edu (Sanjay Govindjee) Date: Wed, 30 Jul 2008 16:02:00 +0200 Subject: petsc for 64 bits In-Reply-To: References: <44dbb0dd0807300108q1dc0ad76g3295de107f12c7cf@mail.gmail.com> Message-ID: <48907458.4050209@ce.berkeley.edu> Dear Carlos, I happen to have written this bit of code for FEAP. If configured properly it works just fine with PETSc on a 64-bit machine. For example my configure line for PETSc on a 64-bit cluster looks like: ./config/configure.py --with-batch=1 --download-parmetis=1 --download-prometheus=1 --download-spooles=1 --download-hypre=1 --download-blacs=1 --download-scalapack=1 --download-mumps=1 --with-debugging=0 --with-mpi-dir=$MPI_ROOT --with-mpi-shared=0 Note that you do not have to tell PETSc anything special. With FEAP you must follow the instructions in the installation manual which in particular ask that you use the include/integer8 include files as opposed to the include/integer4 ones. Other than that there is nothing special that you need to do yourself. -sg Matthew Knepley wrote: > 1) On a 64-bit architecture, ONLY pointers are 64 bits, not integers. > > 2) The option --with-64-bit-indices is for people with extremely long > arrays, not for 64 bit machines. > > 3) On a 64 bit machine, PETSc will automatically switch to long pointers > if the 64 bit compilers are used. > > 4) I suspect the problem is that when you compile FEAP, It explicitly > manipulates the size of an integer, and does not tell us. The correct > way to interact with PETSc is to use PetscInt, which is always the > right size. > > Matt > > On Wed, Jul 30, 2008 at 3:08 AM, Carlos A. Catania (AKA Harpo) > wrote: > >> Hello, >> Perhaps this is not the right place to ask this question because I am using >> PETSC as a library inside a third-part software. I am using feap[1] which >> uses PETSC and also Metis for parallel execution support. >> >> I have a question related to petsc under 64bits. Feap is written in >> Fortran and uses 8 bytes integer data type. As far I know this is the normal >> behavior for AMD 64bit arch (perhaps this is not true). However when >> feap invokes PETSC libraries, >> a segfault error messages is shown. I read a little bit about it and >> the problems seems to be that Petsc uses 4 bytes integers (normal >> behaviour for IA32 arch) and there >> are some problems with pointer references which cause the segfault error >> message. >> >> Reading petsc documentation I found an option to compile petsc with 64 >> bit integer support --with-64-bit-indices=1. I tried it but when compiling >> several incompatibles error messages are shown. Moreover when I >> run the test I got a lot of errors like: >> >> ------------------------------------------------------------- >> Possible error running C/C++ src/snes/examples/tutorials/ex19 with 2 MPI >> processes >> See >> http://www.mcs.anl.gov/petsc/petsc-as/documentation/troubleshooting.html >> [1]PETSC ERROR: --------------------- Error Message >> ------------------------------------ >> [1]PETSC ERROR: Null argument, when expecting valid pointer! >> [1]PETSC ERROR: Null Object: Parameter # 1! >> [1]PETSC ERROR: >> ------------------------------------------------------------------------ >> [1]PETSC ERROR: Petsc Release Version 2.3.3, Patch 3, Fri Jun 15 >> 16:51:25 CDT 2007 HG revision: f051789beadcd36f77fb6111d20225e26ed7cc0d >> [1]PETSC ERROR: See docs/changes/index.html for recent updates. >> [1]PETSC ERROR: See docs/faq.html for hints about trouble shooting. >> [1]PETSC ERROR: See docs/index.html for manual pages. >> [1]PETSC ERROR: >> ------------------------------------------------------------------------ >> >> >> So my question is if it is posible to compile petsc with 64bits >> integer support? >> Besides --with-64-bit-indices option, is there another option I should >> add to get petsc tests work? >> >> My configuration is: >> AMD 64 >> petsc version 2.3.3-p3 >> Intel C an Fortran compilers v 9.1 >> MPICH2 1.0.7 >> >> I configured petsc with this options: >> /config/configure.py PETSC_ARCH=GNU/Linux-gnu-amd64 CC=icc FC=ifort >> --with-mpi-dir=/opt/mpich2/ --download-f-blas-lapack=1 --with-fc=ifort >> --with-64-bit-indices=1 >> >> Thanks in advance >> >> [1] Feap:http://www.ce.berkeley.edu/~rlt/feap/ >> -- >> Carlos A. Catania (AKA Harpo+) >> [FP] F152 8D50 6939 E21A 7A90 CDD4 AEE4 FA48 DF70 4D68 >> >> >> > > > > -- ----------------------------------------------- Sanjay Govindjee, PhD, PE Professor of Civil Engineering and Chancellor's Professor 709 Davis Hall Structural Engineering, Mechanics and Materials Department of Civil Engineering University of California Berkeley, CA 94720-1710 Voice: +1 510 642 6060 FAX: +1 510 643 8928 s_g at berkeley.edu http://www.ce.berkeley.edu/~sanjay ----------------------------------------------- From Hung.V.Nguyen at usace.army.mil Wed Jul 30 09:18:58 2008 From: Hung.V.Nguyen at usace.army.mil (Nguyen, Hung V ERDC-ITL-MS) Date: Wed, 30 Jul 2008 09:18:58 -0500 Subject: Petsc with hypre In-Reply-To: <925346A443D4E340BEB20248BAFCDBDF06A5F685@CFEVS1-IP.americas.cray.com> References: <44dbb0dd0807300108q1dc0ad76g3295de107f12c7cf@mail.gmail.com> <925346A443D4E340BEB20248BAFCDBDF06A5F685@CFEVS1-IP.americas.cray.com> Message-ID: Hello Keita, >Did this error happen with Cray's PETSc? I am running my code in Cray XT3 (at ERDC). Please find the current and available modules in the system below. I think I amn't using Cray's PETSC since I am using the one from the location /usr/local/usp/PETtools/CE/MATH/petsc-2.3.3-p3. How do I use Cray's PETSC? Which module I need to load? Thanks, -Hung --- current module: hvnguyen:sapphire12% module list Currently Loaded Modulefiles: 1) modules/3.1.6 5) xt-libsci/1.5.52 9) xt-pbs/5.3.5 13) xt-catamount/1.5.52 17) Base-opts/1.5.52 2) MySQL/4.0.27 6) xt-mpt/1.5.52 10) xt-service/1.5.52 14) xt-boot/1.5.52 3) acml/3.6 7) xt-pe/1.5.52 11) xt-libc/1.5.52 15) xt-crms/1.5.52 4) pgi/7.0.6 8) PrgEnv-pgi/1.5.52 12) xt-os/1.5.52 16) xt-lustre-ss/1.5.52 hvnguyen:sapphire12% module avai --------------------------------------------------------------- /opt/modulefiles ---------------------------------------------------------------Base-opts/1.4. 10 craypat/3.2.3 xt-catamount/1.4.43 xt-mpt/1.4.10 Base-opts/1.4.38 dwarf/6.10.0 xt-catamount/1.5.16 xt-mpt/1.4.38 Base-opts/1.4.43 elf/0.8.6(default) xt-catamount/1.5.27 xt-mpt/1.4.43 Base-opts/1.5.16 fftw/2.1.5(default) xt-catamount/1.5.39 xt-mpt/1.5.16 Base-opts/1.5.27 fftw/3.1.1 xt-catamount/1.5.39.nic10 xt-mpt/1.5.27 Base-opts/1.5.39 gcc/3.2.3 xt-catamount/1.5.52 xt-mpt/1.5.39 Base-opts/1.5.52(default) gcc/4.1.1(default) xt-craypat/4.0(default) xt-mpt/1.5.52 MySQL/4.0.27 glib/2.4.2 xt-crms/1.4.10 xt-mpt-gnu/1.4.10 PrgEnv/1.4.10 gmalloc xt-crms/1.4.38 xt-mpt-gnu/1.4.38 PrgEnv/1.4.38 gnet/2.0.5 xt-crms/1.4.43 xt-mpt-gnu/1.4.43 PrgEnv/1.4.43 iobuf/1.0.2 xt-crms/1.5.16 xt-mpt-gnu/1.5.16 PrgEnv-gnu/1.4.10 iobuf/1.0.5(default) xt-crms/1.5.27 xt-mpt-gnu/1.5.27 PrgEnv-gnu/1.4.38 iobuf/1.0.6 xt-crms/1.5.39 xt-mpt-gnu/1.5.39 PrgEnv-gnu/1.4.43 libscifft-pgi/1.0.0(default) xt-crms/1.5.52 xt-mpt-gnu/1.5.52 PrgEnv-gnu/1.5.16 modules/3.1.6 xt-libc/1.4.10 xt-mpt-pathscale/1.5.39 PrgEnv-gnu/1.5.27 papi/3.2.1(default) xt-libc/1.4.38 xt-mpt-pathscale/1.5.52 PrgEnv-gnu/1.5.39 papi/3.5.0C.1 xt-libc/1.4.43 xt-os/1.4.10 PrgEnv-gnu/1.5.52(default) pathscale/2.5 xt-libc/1.5.16 xt-os/1.4.38 PrgEnv-pathscale/1.5.39 pathscale/3.0(default) xt-libc/1.5.27 xt-os/1.4.43 PrgEnv-pathscale/1.5.52 pgi/6.2.5 xt-libc/1.5.39 xt-os/1.5.16 PrgEnv-pgi/1.4.10 pgi/7.0.2 xt-libc/1.5.52 xt-os/1.5.27 PrgEnv-pgi/1.4.38 pgi/7.0.3 xt-libsci/1.4.10 xt-os/1.5.39 PrgEnv-pgi/1.4.43 pgi/7.0.4 xt-libsci/1.4.38 xt-os/1.5.52 PrgEnv-pgi/1.5.16 pgi/7.0.5 xt-libsci/1.4.43 xt-papi/3.5.99a(default) PrgEnv-pgi/1.5.27 pgi/7.0.6(default) xt-libsci/1.5.16 xt-pbs/5.3.5 PrgEnv-pgi/1.5.39 pgi/7.0.7 xt-libsci/1.5.27 xt-pe/1.4.38 PrgEnv-pgi/1.5.52(default) pgi/7.1.2 xt-libsci/1.5.39 xt-pe/1.4.43 acml/3.0 pgi/7.1.4 xt-libsci/1.5.52(default) xt-pe/1.5.16 acml/3.6(default) pgi32/6.1.1 xt-libsci/10.2.0 xt-pe/1.5.27 acml/4.0.1a pkg-config/0.15.0 xt-lsfhpc/6.1 xt-pe/1.5.39 acml-gnu/3.0 xt-boot/1.4.10 xt-lustre-ss/1.4.10 xt-pe/1.5.52 acml-large_arrays/3.0 xt-boot/1.4.38 xt-lustre-ss/1.4.38 xt-service/1.4.10 acml-mp/3.0 xt-boot/1.4.43 xt-lustre-ss/1.4.43 xt-service/1.4.38 apprentice2/3.0.1(default) xt-boot/1.5.16 xt-lustre-ss/1.5.16 xt-service/1.4.43 apprentice2/3.2.3 xt-boot/1.5.27 xt-lustre-ss/1.5.27 xt-service/1.5.16 apprentice2/4.0 xt-boot/1.5.39 xt-lustre-ss/1.5.39 xt-service/1.5.27 craypat/3.0.1 xt-boot/1.5.52 xt-lustre-ss/1.5.39.bogl1 xt-service/1.5.39 craypat/3.1.1 xt-catamount/1.4.10 xt-lustre-ss/1.5.39.nic10 xt-service/1.5.52 craypat/3.1.2(default) xt-catamount/1.4.38 xt-lustre-ss/1.5.52 -----Original Message----- From: Keita Teranishi [mailto:keita at cray.com] Sent: Wednesday, July 30, 2008 9:09 AM To: petsc-users at mcs.anl.gov Cc: Nguyen, Hung V ERDC-ITL-MS Subject: RE: Petsc with hypre Hi Hung, Did this error happen with Cray's PETSc? Our PETSc supports both hypre and ParMetis. Please let me know more details with your error (such as module environment). Thanks, ================================ Keita Teranishi Math Software Group Cray, Inc. keita at cray.com ================================ -----Original Message----- From: owner-petsc-users at mcs.anl.gov [mailto:owner-petsc-users at mcs.anl.gov] On Behalf Of Nguyen, Hung V ERDC-ITL-MS Sent: Wednesday, July 30, 2008 8:57 AM To: petsc-users at mcs.anl.gov Subject: Petsc with hypre Dear, I tried to run PETSC with hypre on CrayXT3 system and got the error message below. I didn't install PETSC so I don't know whether it was installed with hypre option or not. So if I need to install PETSC with using hypre option, then please send me the instruction for installation (and also using parMeTiS). I appreciate your help, Regards, -hung hvnguyen:sapphire09% yod -np 16 ./fw -ksp_type richardson -pc_type hypre -pc_hypre_type boomeramg -pc_hypre_boomeramg_max_iter 4 -pc_hypre_boomeramg_tol 1.0e-15 [10]PETSC ERROR: [9]PETSC ERROR: [13]PETSC ERROR: [11]PETSC ERROR: [3]PETSC ERROR: [2]PETSC ERROR: [8]PETSC ERROR: [7]PETSC ERROR: [14]PETSC ERROR: [0]PETSC ERROR: [5]PETSC ERROR: [12]PETSC ERROR: [15]PETSC ERROR: [6]PETSC ERROR: [1]PETSC ERROR: [4]PETSC ERROR: --------------------- Error Message ------------------------------------ --------------------- Error Message ------------------------------------ --------------------- Error Message ------------------------------------ --------------------- Error Message ------------------------------------ --------------------- Error Message ------------------------------------ --------------------- Error Message ------------------------------------ --------------------- Error Message ------------------------------------ --------------------- Error Message ------------------------------------ --------------------- Error Message ------------------------------------ --------------------- Error Message ------------------------------------ --------------------- Error Message ------------------------------------ --------------------- Error Message ------------------------------------ --------------------- Error Message ------------------------------------ --------------------- Error Message ------------------------------------ --------------------- Error Message ------------------------------------ --------------------- Error Message ------------------------------------ [10]PETSC ERROR: [9]PETSC ERROR: [13]PETSC ERROR: [11]PETSC ERROR: [3]PETSC ERROR: [2]PETSC ERROR: [8]PETSC ERROR: [7]PETSC ERROR: [14]PETSC ERROR: [0]PETSC ERROR: [5]PETSC ERROR: [12]PETSC ERROR: [15]PETSC ERROR: [6]PETSC ERROR: [1]PETSC ERROR: [4]PETSC ERROR: Unknown type. Check for miss-spelling or missing external package needed for type! Unknown type. Check for miss-spelling or missing external package needed for type! Unknown type. Check for miss-spelling or missing external package needed for type! Unknown type. Check for miss-spelling or missing external package needed for type! [10]PETSC ERROR: [9]PETSC ERROR: [13]PETSC ERROR: [11]PETSC ERROR: [3]PETSC ERROR: [2]PETSC ERROR: [8]PETSC ERROR: [7]PETSC ERROR: [14]PETSC ERROR: [0]PETSC ERROR: [5]PETSC ERROR: [12]PETSC ERROR: [15]PETSC ERROR: [6]PETSC ERROR: [1]PETSC ERROR: [4]PETSC ERROR: Unable to find requested PC type hypre! Unable to find requested PC type hypre! Unable to find requested PC type hypre! Unable to find requested PC type hypre! Unable to find requested PC type hypre! Unable to find requested PC type hypre! Unable to find requested PC type hypre! Unable to find requested PC type hypre! Unable to find requested PC type hypre! Unable to find requested PC type hypre! Unable to find requested PC type hypre! Unable to find requested PC type hypre! Unable to find requested PC type hypre! Unable to find requested PC type hypre! Unable to find requested PC type hypre! Unable to find requested PC type hypre! [10]PETSC ERROR: [9]PETSC ERROR: [13]PETSC ERROR: [11]PETSC ERROR: [3]PETSC ERROR: [2]PETSC ERROR: [8]PETSC ERROR: [7]PETSC ERROR: [14]PETSC ERROR: [0]PETSC ERROR: [5]PETSC ERROR: [12]PETSC ERROR: [15]PETSC ERROR: [6]PETSC ERROR: [1]PETSC ERROR: [4]PETSC ERROR: ------------------------------------------------------------------------ From bsmith at mcs.anl.gov Wed Jul 30 10:27:36 2008 From: bsmith at mcs.anl.gov (Barry Smith) Date: Wed, 30 Jul 2008 10:27:36 -0500 Subject: Petsc with hypre In-Reply-To: References: <44dbb0dd0807300108q1dc0ad76g3295de107f12c7cf@mail.gmail.com> <925346A443D4E340BEB20248BAFCDBDF06A5F685@CFEVS1-IP.americas.cray.com> Message-ID: <4C3C67D6-9EDC-4CB3-9185-C33FD7D7AAFE@mcs.anl.gov> If PETSc is built from source then it, by default, does not include hypre and other external packages. You should install PETSc yourself and include the additional config/configure.py option --download-hypre Barry On Jul 30, 2008, at 9:18 AM, Nguyen, Hung V ERDC-ITL-MS wrote: > > Hello Keita, > >> Did this error happen with Cray's PETSc? > > I am running my code in Cray XT3 (at ERDC). Please find the current > and > available modules in the system below. > > I think I amn't using Cray's PETSC since I am using the one from the > location > /usr/local/usp/PETtools/CE/MATH/petsc-2.3.3-p3. > > How do I use Cray's PETSC? Which module I need to load? > > Thanks, > > -Hung > > --- current module: > hvnguyen:sapphire12% module list > Currently Loaded Modulefiles: > 1) modules/3.1.6 5) xt-libsci/1.5.52 9) xt-pbs/5.3.5 > 13) xt-catamount/1.5.52 17) Base-opts/1.5.52 > 2) MySQL/4.0.27 6) xt-mpt/1.5.52 10) xt-service/ > 1.5.52 > 14) xt-boot/1.5.52 > 3) acml/3.6 7) xt-pe/1.5.52 11) xt-libc/1.5.52 > 15) xt-crms/1.5.52 > 4) pgi/7.0.6 8) PrgEnv-pgi/1.5.52 12) xt-os/1.5.52 > 16) xt-lustre-ss/1.5.52 > hvnguyen:sapphire12% module avai > > --------------------------------------------------------------- > /opt/modulefiles > ---------------------------------------------------------------Base- > opts/1.4. > 10 craypat/3.2.3 xt-catamount/1.4.43 > xt-mpt/1.4.10 > Base-opts/1.4.38 dwarf/6.10.0 xt- > catamount/1.5.16 > xt-mpt/1.4.38 > Base-opts/1.4.43 elf/0.8.6(default) xt- > catamount/1.5.27 > xt-mpt/1.4.43 > Base-opts/1.5.16 fftw/2.1.5(default) xt- > catamount/1.5.39 > xt-mpt/1.5.16 > Base-opts/1.5.27 fftw/3.1.1 > xt-catamount/1.5.39.nic10 xt-mpt/1.5.27 > Base-opts/1.5.39 gcc/3.2.3 xt- > catamount/1.5.52 > xt-mpt/1.5.39 > Base-opts/1.5.52(default) gcc/4.1.1(default) > xt-craypat/4.0(default) xt-mpt/1.5.52 > MySQL/4.0.27 glib/2.4.2 xt-crms/ > 1.4.10 > xt-mpt-gnu/1.4.10 > PrgEnv/1.4.10 gmalloc xt-crms/ > 1.4.38 > xt-mpt-gnu/1.4.38 > PrgEnv/1.4.38 gnet/2.0.5 xt-crms/ > 1.4.43 > xt-mpt-gnu/1.4.43 > PrgEnv/1.4.43 iobuf/1.0.2 xt-crms/ > 1.5.16 > xt-mpt-gnu/1.5.16 > PrgEnv-gnu/1.4.10 iobuf/1.0.5(default) xt-crms/ > 1.5.27 > xt-mpt-gnu/1.5.27 > PrgEnv-gnu/1.4.38 iobuf/1.0.6 xt-crms/ > 1.5.39 > xt-mpt-gnu/1.5.39 > PrgEnv-gnu/1.4.43 libscifft-pgi/1.0.0(default) xt-crms/ > 1.5.52 > xt-mpt-gnu/1.5.52 > PrgEnv-gnu/1.5.16 modules/3.1.6 xt-libc/ > 1.4.10 > xt-mpt-pathscale/1.5.39 > PrgEnv-gnu/1.5.27 papi/3.2.1(default) xt-libc/ > 1.4.38 > xt-mpt-pathscale/1.5.52 > PrgEnv-gnu/1.5.39 papi/3.5.0C.1 xt-libc/ > 1.4.43 > xt-os/1.4.10 > PrgEnv-gnu/1.5.52(default) pathscale/2.5 xt-libc/ > 1.5.16 > xt-os/1.4.38 > PrgEnv-pathscale/1.5.39 pathscale/3.0(default) xt-libc/ > 1.5.27 > xt-os/1.4.43 > PrgEnv-pathscale/1.5.52 pgi/6.2.5 xt-libc/ > 1.5.39 > xt-os/1.5.16 > PrgEnv-pgi/1.4.10 pgi/7.0.2 xt-libc/ > 1.5.52 > xt-os/1.5.27 > PrgEnv-pgi/1.4.38 pgi/7.0.3 xt-libsci/ > 1.4.10 > xt-os/1.5.39 > PrgEnv-pgi/1.4.43 pgi/7.0.4 xt-libsci/ > 1.4.38 > xt-os/1.5.52 > PrgEnv-pgi/1.5.16 pgi/7.0.5 xt-libsci/ > 1.4.43 > xt-papi/3.5.99a(default) > PrgEnv-pgi/1.5.27 pgi/7.0.6(default) xt-libsci/ > 1.5.16 > xt-pbs/5.3.5 > PrgEnv-pgi/1.5.39 pgi/7.0.7 xt-libsci/ > 1.5.27 > xt-pe/1.4.38 > PrgEnv-pgi/1.5.52(default) pgi/7.1.2 xt-libsci/ > 1.5.39 > xt-pe/1.4.43 > acml/3.0 pgi/7.1.4 > xt-libsci/1.5.52(default) xt-pe/1.5.16 > acml/3.6(default) pgi32/6.1.1 xt-libsci/ > 10.2.0 > xt-pe/1.5.27 > acml/4.0.1a pkg-config/0.15.0 xt-lsfhpc/ > 6.1 > xt-pe/1.5.39 > acml-gnu/3.0 xt-boot/1.4.10 xt-lustre- > ss/1.4.10 > xt-pe/1.5.52 > acml-large_arrays/3.0 xt-boot/1.4.38 xt-lustre- > ss/1.4.38 > xt-service/1.4.10 > acml-mp/3.0 xt-boot/1.4.43 xt-lustre- > ss/1.4.43 > xt-service/1.4.38 > apprentice2/3.0.1(default) xt-boot/1.5.16 xt-lustre- > ss/1.5.16 > xt-service/1.4.43 > apprentice2/3.2.3 xt-boot/1.5.27 xt-lustre- > ss/1.5.27 > xt-service/1.5.16 > apprentice2/4.0 xt-boot/1.5.39 xt-lustre- > ss/1.5.39 > xt-service/1.5.27 > craypat/3.0.1 xt-boot/1.5.52 > xt-lustre-ss/1.5.39.bogl1 xt-service/1.5.39 > craypat/3.1.1 xt-catamount/1.4.10 > xt-lustre-ss/1.5.39.nic10 xt-service/1.5.52 > craypat/3.1.2(default) xt-catamount/1.4.38 xt-lustre- > ss/1.5.52 > > > > -----Original Message----- > From: Keita Teranishi [mailto:keita at cray.com] > Sent: Wednesday, July 30, 2008 9:09 AM > To: petsc-users at mcs.anl.gov > Cc: Nguyen, Hung V ERDC-ITL-MS > Subject: RE: Petsc with hypre > > Hi Hung, > > Did this error happen with Cray's PETSc? Our PETSc supports both > hypre and > ParMetis. > Please let me know more details with your error (such as module > environment). > > > Thanks, > > ================================ > Keita Teranishi > Math Software Group > Cray, Inc. > keita at cray.com > ================================ > > > -----Original Message----- > From: owner-petsc-users at mcs.anl.gov [mailto:owner-petsc-users at mcs.anl.gov > ] On > Behalf Of Nguyen, Hung V ERDC-ITL-MS > Sent: Wednesday, July 30, 2008 8:57 AM > To: petsc-users at mcs.anl.gov > Subject: Petsc with hypre > > > Dear, > > I tried to run PETSC with hypre on CrayXT3 system and got the error > message > below. I didn't install PETSC so I don't know whether it was > installed with > hypre option or not. So if I need to install PETSC with using hypre > option, > then please send me the instruction for installation (and also using > parMeTiS). > > I appreciate your help, > > Regards, > > -hung > > hvnguyen:sapphire09% yod -np 16 ./fw -ksp_type richardson -pc_type > hypre > -pc_hypre_type boomeramg -pc_hypre_boomeramg_max_iter 4 > -pc_hypre_boomeramg_tol 1.0e-15 > > > [10]PETSC ERROR: [9]PETSC ERROR: [13]PETSC ERROR: [11]PETSC ERROR: > [3]PETSC > ERROR: [2]PETSC ERROR: [8]PETSC ERROR: [7]PETSC ERROR: [14]PETSC > ERROR: > [0]PETSC ERROR: [5]PETSC ERROR: [12]PETSC ERROR: [15]PETSC ERROR: > [6]PETSC > ERROR: [1]PETSC ERROR: [4]PETSC ERROR: --------------------- Error > Message > ------------------------------------ > --------------------- Error Message > ------------------------------------ > --------------------- Error Message > ------------------------------------ > --------------------- Error Message > ------------------------------------ > --------------------- Error Message > ------------------------------------ > --------------------- Error Message > ------------------------------------ > --------------------- Error Message > ------------------------------------ > --------------------- Error Message > ------------------------------------ > --------------------- Error Message > ------------------------------------ > --------------------- Error Message > ------------------------------------ > --------------------- Error Message > ------------------------------------ > --------------------- Error Message > ------------------------------------ > --------------------- Error Message > ------------------------------------ > --------------------- Error Message > ------------------------------------ > --------------------- Error Message > ------------------------------------ > --------------------- Error Message > ------------------------------------ > [10]PETSC ERROR: [9]PETSC ERROR: [13]PETSC ERROR: [11]PETSC ERROR: > [3]PETSC > ERROR: [2]PETSC ERROR: [8]PETSC ERROR: [7]PETSC ERROR: [14]PETSC > ERROR: > [0]PETSC ERROR: [5]PETSC ERROR: [12]PETSC ERROR: [15]PETSC ERROR: > [6]PETSC > ERROR: [1]PETSC ERROR: [4]PETSC ERROR: Unknown type. Check for miss- > spelling > or missing external package needed for type! > Unknown type. Check for miss-spelling or missing external package > needed for > type! > Unknown type. Check for miss-spelling or missing external package > needed for > type! > > Unknown type. Check for miss-spelling or missing external package > needed for > type! > [10]PETSC ERROR: [9]PETSC ERROR: [13]PETSC ERROR: [11]PETSC ERROR: > [3]PETSC > ERROR: [2]PETSC ERROR: [8]PETSC ERROR: [7]PETSC ERROR: [14]PETSC > ERROR: > [0]PETSC ERROR: [5]PETSC ERROR: [12]PETSC ERROR: [15]PETSC ERROR: > [6]PETSC > ERROR: [1]PETSC ERROR: [4]PETSC ERROR: Unable to find requested PC > type > hypre! > Unable to find requested PC type hypre! > Unable to find requested PC type hypre! > Unable to find requested PC type hypre! > Unable to find requested PC type hypre! > Unable to find requested PC type hypre! > Unable to find requested PC type hypre! > Unable to find requested PC type hypre! > Unable to find requested PC type hypre! > Unable to find requested PC type hypre! > Unable to find requested PC type hypre! > Unable to find requested PC type hypre! > Unable to find requested PC type hypre! > Unable to find requested PC type hypre! > Unable to find requested PC type hypre! > Unable to find requested PC type hypre! > [10]PETSC ERROR: [9]PETSC ERROR: [13]PETSC ERROR: [11]PETSC ERROR: > [3]PETSC > ERROR: [2]PETSC ERROR: [8]PETSC ERROR: [7]PETSC ERROR: [14]PETSC > ERROR: > [0]PETSC ERROR: [5]PETSC ERROR: [12]PETSC ERROR: [15]PETSC ERROR: > [6]PETSC > ERROR: [1]PETSC ERROR: [4]PETSC ERROR: > ------------------------------------------------------------------------ > From hashhsah at gmail.com Thu Jul 31 06:08:47 2008 From: hashhsah at gmail.com (Chen Shen) Date: Thu, 31 Jul 2008 19:08:47 +0800 Subject: a new release soon? Message-ID: <48af9a710807310408m1ac365a6kb6760b91efbfbe31@mail.gmail.com> Hi all, I remember reading in a thread in the list, talking about an upcoming release. Is it coming soon? If that is the case, we shall start tracking the tip of the petsc-devel tree. Thank you very much. regards, shenchen -------------- next part -------------- An HTML attachment was scrubbed... URL: From w_subber at yahoo.com Thu Jul 31 16:33:50 2008 From: w_subber at yahoo.com (Waad Subber) Date: Thu, 31 Jul 2008 14:33:50 -0700 (PDT) Subject: Performance Log Question Message-ID: <107234.55767.qm@web38205.mail.mud.yahoo.com> Hello, I am currently solving a 1.2 million by 1.2 million linear system using PETSc 2.3.3, Patch 13 using domain decomposition (iterative substructuring using a Krylov Subspace Solver). I'm using a 120 CPU cluster with InfiniBand interconnect; each node has 8 cores -- 2 quad-core Xeon CPUs (X5365) at 3.0 GHz, each node having 32 GB of RAM. After running my code, I generate a log using the following: ?????? CALL PetscLogPrintSummary(PETSC_COMM_WORLD,"log.txt",ierr) When looking at the log output, I noticed that the peak and average number of Flops seems fairly low -- in the case of one run, a peak of 6.0e7 flops with an average value of 5.2e7. The exact log output is: ???????????????????????? Max?????? Max/Min??????? Avg????? Total Time (sec):?????????? 5.283e+01????? 1.02357?? 5.169e+01 Objects:????????????? 2.600e+02????? 1.00000?? 2.600e+02 Flops:??????????????? 3.187e+09????? 1.69853?? 2.721e+09? 3.265e+11 Flops/sec:??????????? 6.165e+07????? 1.69865?? 5.264e+07? 6.317e+09 Memory:?????????????? 6.081e+07????? 1.39801????????????? 6.608e+09 MPI Messages:???????? 1.067e+05????? 1.00000?? 1.067e+05? 1.281e+07 MPI Message Lengths:? 5.205e+08????? 1.00081?? 4.875e+03? 6.245e+10 MPI Reductions:?????? 1.898e+01????? 1.00000 Is my interpretation correct about this being a fairly low flop count? Does this mean there's an issue with my code? I am attaching my log file Thanks Waad -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: log.txt URL: From rlmackie862 at gmail.com Thu Jul 31 16:35:18 2008 From: rlmackie862 at gmail.com (Randall Mackie) Date: Thu, 31 Jul 2008 14:35:18 -0700 Subject: Question on using MUMPS in PETSC Message-ID: <48923016.2000401@gmail.com> I've compiled PETSc with MUMPS support, and I'm trying to run a small test problem, but I'm having some problems. It seems to begin just fine, but what I notice is that on one process (out of 64), the memory just keeps going up and up and up until it crashes, while on the other processes, the memory usage is reasonable. I'm wondering if anyone might have any idea why? By the way, my command file is like this: -ksp_type preonly -pc_type lu -mat_type aijmumps -mat_mumps_cntl_4 3 -mat_mumps_cntl_9 1 Randy ps. This happens after the analysis stage and in the factorization stage. From knepley at gmail.com Thu Jul 31 17:03:33 2008 From: knepley at gmail.com (Matthew Knepley) Date: Thu, 31 Jul 2008 17:03:33 -0500 Subject: Question on using MUMPS in PETSC In-Reply-To: <48923016.2000401@gmail.com> References: <48923016.2000401@gmail.com> Message-ID: Does it work on KSP ex2? Matt On Thu, Jul 31, 2008 at 4:35 PM, Randall Mackie wrote: > I've compiled PETSc with MUMPS support, and I'm trying to run a small test > problem, but I'm having some problems. It seems to begin just fine, but > what I notice is that on one process (out of 64), the memory just keeps > going up and up and up until it crashes, while on the other processes, > the memory usage is reasonable. I'm wondering if anyone might have any idea > why? By the way, my command file is like this: > > -ksp_type preonly > -pc_type lu > -mat_type aijmumps > -mat_mumps_cntl_4 3 > -mat_mumps_cntl_9 1 > > > Randy > > ps. This happens after the analysis stage and in the factorization stage. > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener From rlmackie862 at gmail.com Thu Jul 31 17:56:04 2008 From: rlmackie862 at gmail.com (Randall Mackie) Date: Thu, 31 Jul 2008 15:56:04 -0700 Subject: Question on using MUMPS in PETSC In-Reply-To: References: <48923016.2000401@gmail.com> Message-ID: <48924304.7070107@gmail.com> When m = n = small (like 50), it works fine. When I set m=n=5000, I see the same thing, where one process on the localhost is taking >4 G of RAM, while all other processes are taking 137 M. Is this the standard behavior for MUMPS? It seems strange to me. Randy Matthew Knepley wrote: > Does it work on KSP ex2? > > Matt > > On Thu, Jul 31, 2008 at 4:35 PM, Randall Mackie wrote: >> I've compiled PETSc with MUMPS support, and I'm trying to run a small test >> problem, but I'm having some problems. It seems to begin just fine, but >> what I notice is that on one process (out of 64), the memory just keeps >> going up and up and up until it crashes, while on the other processes, >> the memory usage is reasonable. I'm wondering if anyone might have any idea >> why? By the way, my command file is like this: >> >> -ksp_type preonly >> -pc_type lu >> -mat_type aijmumps >> -mat_mumps_cntl_4 3 >> -mat_mumps_cntl_9 1 >> >> >> Randy >> >> ps. This happens after the analysis stage and in the factorization stage. >> >> > > > From knepley at gmail.com Thu Jul 31 18:17:47 2008 From: knepley at gmail.com (Matthew Knepley) Date: Thu, 31 Jul 2008 18:17:47 -0500 Subject: Question on using MUMPS in PETSC In-Reply-To: <48924304.7070107@gmail.com> References: <48923016.2000401@gmail.com> <48924304.7070107@gmail.com> Message-ID: On Thu, Jul 31, 2008 at 5:56 PM, Randall Mackie wrote: > When m = n = small (like 50), it works fine. When I set m=n=5000, I see > the same thing, where one process on the localhost is taking >4 G of RAM, > while all other processes are taking 137 M. > > Is this the standard behavior for MUMPS? It seems strange to me. No, this is a problem. I do not see this with ex2. Matt > Randy > > > Matthew Knepley wrote: >> >> Does it work on KSP ex2? >> >> Matt >> >> On Thu, Jul 31, 2008 at 4:35 PM, Randall Mackie >> wrote: >>> >>> I've compiled PETSc with MUMPS support, and I'm trying to run a small >>> test >>> problem, but I'm having some problems. It seems to begin just fine, but >>> what I notice is that on one process (out of 64), the memory just keeps >>> going up and up and up until it crashes, while on the other processes, >>> the memory usage is reasonable. I'm wondering if anyone might have any >>> idea >>> why? By the way, my command file is like this: >>> >>> -ksp_type preonly >>> -pc_type lu >>> -mat_type aijmumps >>> -mat_mumps_cntl_4 3 >>> -mat_mumps_cntl_9 1 >>> >>> >>> Randy >>> >>> ps. This happens after the analysis stage and in the factorization stage. >>> >>> >> >> >> > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener From bsmith at mcs.anl.gov Thu Jul 31 20:24:33 2008 From: bsmith at mcs.anl.gov (Barry Smith) Date: Thu, 31 Jul 2008 20:24:33 -0500 Subject: Question on using MUMPS in PETSC In-Reply-To: <48924304.7070107@gmail.com> References: <48923016.2000401@gmail.com> <48924304.7070107@gmail.com> Message-ID: If m and n are the number of rows and columns of the sparse matrix (i.e. it is tiny problem) then please send us matrix so we can experiment with it to petsc-maint at mcs.anl.log You can send us the matrix by simply running with -ksp_view_binary and sending us the file binaryoutput. Barry On Jul 31, 2008, at 5:56 PM, Randall Mackie wrote: > When m = n = small (like 50), it works fine. When I set m=n=5000, I > see > the same thing, where one process on the localhost is taking >4 G of > RAM, > while all other processes are taking 137 M. > > Is this the standard behavior for MUMPS? It seems strange to me. > > Randy > > > Matthew Knepley wrote: >> Does it work on KSP ex2? >> Matt >> On Thu, Jul 31, 2008 at 4:35 PM, Randall Mackie > > wrote: >>> I've compiled PETSc with MUMPS support, and I'm trying to run a >>> small test >>> problem, but I'm having some problems. It seems to begin just >>> fine, but >>> what I notice is that on one process (out of 64), the memory just >>> keeps >>> going up and up and up until it crashes, while on the other >>> processes, >>> the memory usage is reasonable. I'm wondering if anyone might have >>> any idea >>> why? By the way, my command file is like this: >>> >>> -ksp_type preonly >>> -pc_type lu >>> -mat_type aijmumps >>> -mat_mumps_cntl_4 3 >>> -mat_mumps_cntl_9 1 >>> >>> >>> Randy >>> >>> ps. This happens after the analysis stage and in the factorization >>> stage. >>> >>> > From bsmith at mcs.anl.gov Thu Jul 31 20:27:22 2008 From: bsmith at mcs.anl.gov (Barry Smith) Date: Thu, 31 Jul 2008 20:27:22 -0500 Subject: Performance Log Question In-Reply-To: <107234.55767.qm@web38205.mail.mud.yahoo.com> References: <107234.55767.qm@web38205.mail.mud.yahoo.com> Message-ID: From the file, it seems you never make an nontrivial PETSc calls. Do you get the same log if you just run with -log_summary? Barry Event Count Time (sec) Flops/ sec --- Global --- --- Stage --- Total Max Ratio Max Ratio Max Ratio Mess Avg len Reduct %T %F %M %L %R %T %F %M %L %R Mflop/s ------------------------------------------------------------------------------------------------------------------------ --- Event Stage 0: Main Stage ------------------------------------------------------------------------------------------------------------------------ On Jul 31, 2008, at 4:33 PM, Waad Subber wrote: > Hello, > > I am currently solving a 1.2 million by 1.2 million linear system > using PETSc 2.3.3, Patch 13 using domain decomposition (iterative > substructuring using a Krylov Subspace Solver). I'm using a 120 CPU > cluster with InfiniBand interconnect; each node has 8 cores -- 2 > quad-core Xeon CPUs (X5365) at 3.0 GHz, each node having 32 GB of RAM. > > After running my code, I generate a log using the following: > CALL PetscLogPrintSummary(PETSC_COMM_WORLD,"log.txt",ierr) > > When looking at the log output, I noticed that the peak and average > number of Flops seems fairly low -- in the case of one run, a peak > of 6.0e7 flops with an average value of 5.2e7. The exact log output > is: > > Max Max/Min Avg Total > Time (sec): 5.283e+01 1.02357 5.169e+01 > Objects: 2.600e+02 1.00000 2.600e+02 > Flops: 3.187e+09 1.69853 2.721e+09 3.265e+11 > Flops/sec: 6.165e+07 1.69865 5.264e+07 6.317e+09 > Memory: 6.081e+07 1.39801 6.608e+09 > MPI Messages: 1.067e+05 1.00000 1.067e+05 1.281e+07 > MPI Message Lengths: 5.205e+08 1.00081 4.875e+03 6.245e+10 > MPI Reductions: 1.898e+01 1.00000 > > Is my interpretation correct about this being a fairly low flop > count? Does this mean there's an issue with my code? > > I am attaching my log file > > Thanks > > Waad > > > From w_subber at yahoo.com Thu Jul 31 22:09:10 2008 From: w_subber at yahoo.com (Waad Subber) Date: Thu, 31 Jul 2008 20:09:10 -0700 (PDT) Subject: Performance Log Question In-Reply-To: Message-ID: <983023.46888.qm@web38203.mail.mud.yahoo.com> Thank you Barry for the replay. Yes I get more or less the same log when I run with the option -log_summary. Please have a look at the attached file. Thanks Waad --- On Thu, 7/31/08, Barry Smith wrote: From: Barry Smith Subject: Re: Performance Log Question To: petsc-users at mcs.anl.gov Date: Thursday, July 31, 2008, 9:27 PM From the file, it seems you never make an nontrivial PETSc calls. Do you get the same log if you just run with -log_summary? Barry Event Count Time (sec) Flops/ sec --- Global --- --- Stage --- Total Max Ratio Max Ratio Max Ratio Mess Avg len Reduct %T %F %M %L %R %T %F %M %L %R Mflop/s ------------------------------------------------------------------------------------------------------------------------ --- Event Stage 0: Main Stage ------------------------------------------------------------------------------------------------------------------------ On Jul 31, 2008, at 4:33 PM, Waad Subber wrote: > Hello, > > I am currently solving a 1.2 million by 1.2 million linear system > using PETSc 2.3.3, Patch 13 using domain decomposition (iterative > substructuring using a Krylov Subspace Solver). I'm using a 120 CPU > cluster with InfiniBand interconnect; each node has 8 cores -- 2 > quad-core Xeon CPUs (X5365) at 3.0 GHz, each node having 32 GB of RAM. > > After running my code, I generate a log using the following: > CALL PetscLogPrintSummary(PETSC_COMM_WORLD,"log.txt",ierr) > > When looking at the log output, I noticed that the peak and average > number of Flops seems fairly low -- in the case of one run, a peak > of 6.0e7 flops with an average value of 5.2e7. The exact log output > is: > > Max Max/Min Avg Total > Time (sec): 5.283e+01 1.02357 5.169e+01 > Objects: 2.600e+02 1.00000 2.600e+02 > Flops: 3.187e+09 1.69853 2.721e+09 3.265e+11 > Flops/sec: 6.165e+07 1.69865 5.264e+07 6.317e+09 > Memory: 6.081e+07 1.39801 6.608e+09 > MPI Messages: 1.067e+05 1.00000 1.067e+05 1.281e+07 > MPI Message Lengths: 5.205e+08 1.00081 4.875e+03 6.245e+10 > MPI Reductions: 1.898e+01 1.00000 > > Is my interpretation correct about this being a fairly low flop > count? Does this mean there's an issue with my code? > > I am attaching my log file > > Thanks > > Waad > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: log_summary Type: application/octet-stream Size: 12914 bytes Desc: not available URL: From w_subber at yahoo.com Thu Jul 31 22:40:53 2008 From: w_subber at yahoo.com (Waad Subber) Date: Thu, 31 Jul 2008 20:40:53 -0700 (PDT) Subject: Performance Log Question In-Reply-To: Message-ID: <714437.78026.qm@web38201.mail.mud.yahoo.com> Thank you Barry for the reply. Yes I get more or less the same log when I run with the option -log_summary. Please have a look at the attached file. Thanks Waad --- On Thu, 7/31/08, Barry Smith wrote: From: Barry Smith Subject: Re: Performance Log Question To: petsc-users at mcs.anl.gov Date: Thursday, July 31, 2008, 9:27 PM From the file, it seems you never make an nontrivial PETSc calls. Do you get the same log if you just run with -log_summary? Barry Event Count Time (sec) Flops/ sec --- Global --- --- Stage --- Total Max Ratio Max Ratio Max Ratio Mess Avg len Reduct %T %F %M %L %R %T %F %M %L %R Mflop/s ------------------------------------------------------------------------------------------------------------------------ --- Event Stage 0: Main Stage ------------------------------------------------------------------------------------------------------------------------ On Jul 31, 2008, at 4:33 PM, Waad Subber wrote: > Hello, > > I am currently solving a 1.2 million by 1.2 million linear system > using PETSc 2.3.3, Patch 13 using domain decomposition (iterative > substructuring using a Krylov Subspace Solver). I'm using a 120 CPU > cluster with InfiniBand interconnect; each node has 8 cores -- 2 > quad-core Xeon CPUs (X5365) at 3.0 GHz, each node having 32 GB of RAM. > > After running my code, I generate a log using the following: > CALL PetscLogPrintSummary(PETSC_COMM_WORLD,"log.txt",ierr) > > When looking at the log output, I noticed that the peak and average > number of Flops seems fairly low -- in the case of one run, a peak > of 6.0e7 flops with an average value of 5.2e7. The exact log output > is: > > Max Max/Min Avg Total > Time (sec): 5.283e+01 1.02357 5.169e+01 > Objects: 2.600e+02 1.00000 2.600e+02 > Flops: 3.187e+09 1.69853 2.721e+09 3.265e+11 > Flops/sec: 6.165e+07 1.69865 5.264e+07 6.317e+09 > Memory: 6.081e+07 1.39801 6.608e+09 > MPI Messages: 1.067e+05 1.00000 1.067e+05 1.281e+07 > MPI Message Lengths: 5.205e+08 1.00081 4.875e+03 6.245e+10 > MPI Reductions: 1.898e+01 1.00000 > > Is my interpretation correct about this being a fairly low flop > count? Does this mean there's an issue with my code? > > I am attaching my log file > > Thanks > > Waad > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Thu Jul 31 22:43:48 2008 From: bsmith at mcs.anl.gov (Barry Smith) Date: Thu, 31 Jul 2008 22:43:48 -0500 Subject: Performance Log Question In-Reply-To: <714437.78026.qm@web38201.mail.mud.yahoo.com> References: <714437.78026.qm@web38201.mail.mud.yahoo.com> Message-ID: <3BD4DE71-C511-4E49-8685-EB6823748A07@mcs.anl.gov> Code is not using PETSc to do anything. Therefore there isn't going to be much in the way of logging. PETSc's logging only logs our own code. It doesn't magically log all flops in the code. Barry On Jul 31, 2008, at 10:40 PM, Waad Subber wrote: > Thank you Barry for the reply. Yes I get more or less the same log > when I run with the option -log_summary. Please have a look at the > attached file. > > Thanks > > Waad > > --- On Thu, 7/31/08, Barry Smith wrote: > From: Barry Smith > Subject: Re: Performance Log Question > To: petsc-users at mcs.anl.gov > Date: Thursday, July 31, 2008, 9:27 PM > > From the file, it seems you never make an nontrivial PETSc calls. > Do you get the same > log if you just run with -log_summary? > > Barry > > > > Event Count Time (sec) Flops/ > sec --- Global --- --- Stage --- Total > Max Ratio Max Ratio Max Ratio Mess Avg > len Reduct %T %F %M %L %R %T %F %M %L %R Mflop/s > ------------------------------------------------------------------------------------------------------------------------ > > --- Event Stage > 0: Main Stage > > ------------------------------------------------------------------------------------------------------------------------ > > > > On Jul 31, 2008, at 4:33 PM, Waad Subber wrote: > > > Hello, > > > > I am currently solving a 1.2 million by 1.2 million linear system > > using PETSc 2.3.3, Patch 13 using domain decomposition (iterative > > substructuring using a Krylov Subspace Solver). I'm using a 120 CPU > > cluster with InfiniBand interconnect; each node has 8 cores -- 2 > > quad-core Xeon CPUs (X5365) at 3.0 GHz, each node having 32 GB of > RAM. > > > > After running my code, I generate a log using the following: > > CALL > PetscLogPrintSummary(PETSC_COMM_WORLD,"log.txt",ierr) > > > > When looking at the log output, I noticed that the peak and average > > number of Flops seems fairly low -- in the case of one run, a peak > > of 6.0e7 flops with an > average value of 5.2e7. The exact log output > > is: > > > > Max Max/Min Avg Total > > Time (sec): 5.283e+01 1.02357 5.169e+01 > > Objects: 2.600e+02 1.00000 2.600e+02 > > Flops: 3.187e+09 1.69853 2.721e+09 3.265e+11 > > Flops/sec: 6.165e+07 1.69865 5.264e+07 6.317e+09 > > Memory: 6.081e+07 1.39801 6.608e+09 > > MPI Messages: 1.067e+05 1.00000 1.067e+05 1.281e+07 > > MPI Message Lengths: 5.205e+08 1.00081 4.875e+03 6.245e+10 > > MPI Reductions: 1.898e+01 1.00000 > > > > Is my interpretation correct about this being a fairly low flop > > count? Does this mean there's an issue with my code? > > > > I am attaching my log file > > > > Thanks > > > > Waad > > > > > > > > > From rlmackie862 at gmail.com Thu Jul 31 23:56:12 2008 From: rlmackie862 at gmail.com (Randall Mackie) Date: Thu, 31 Jul 2008 21:56:12 -0700 Subject: Question on using MUMPS in PETSC In-Reply-To: References: <48923016.2000401@gmail.com> <48924304.7070107@gmail.com> Message-ID: <4892976C.1000605@gmail.com> Barry, I don't think it's the matrix - I saw the same behavior when I ran your ex2.c program and set m=n=5000. Randy Barry Smith wrote: > > If m and n are the number of rows and columns of the sparse matrix > (i.e. it is > tiny problem) then please > send us matrix so we can experiment with it to petsc-maint at mcs.anl.log > > You can send us the matrix by simply running with -ksp_view_binary and > sending us the file binaryoutput. > > Barry > > On Jul 31, 2008, at 5:56 PM, Randall Mackie wrote: > >> When m = n = small (like 50), it works fine. When I set m=n=5000, I see >> the same thing, where one process on the localhost is taking >4 G of RAM, >> while all other processes are taking 137 M. >> >> Is this the standard behavior for MUMPS? It seems strange to me. >> >> Randy >> >> >> Matthew Knepley wrote: >>> Does it work on KSP ex2? >>> Matt >>> On Thu, Jul 31, 2008 at 4:35 PM, Randall Mackie >>> wrote: >>>> I've compiled PETSc with MUMPS support, and I'm trying to run a >>>> small test >>>> problem, but I'm having some problems. It seems to begin just fine, but >>>> what I notice is that on one process (out of 64), the memory just keeps >>>> going up and up and up until it crashes, while on the other processes, >>>> the memory usage is reasonable. I'm wondering if anyone might have >>>> any idea >>>> why? By the way, my command file is like this: >>>> >>>> -ksp_type preonly >>>> -pc_type lu >>>> -mat_type aijmumps >>>> -mat_mumps_cntl_4 3 >>>> -mat_mumps_cntl_9 1 >>>> >>>> >>>> Randy >>>> >>>> ps. This happens after the analysis stage and in the factorization >>>> stage. >>>> >>>> >> >