From bsmith at mcs.anl.gov Thu Mar 1 15:59:00 2007 From: bsmith at mcs.anl.gov (Barry Smith) Date: Thu, 1 Mar 2007 15:59:00 -0600 (CST) Subject: Non-uniform 2D mesh questions In-Reply-To: <20070207063325.20285.qmail@s402.sureserver.com> References: <20070207063325.20285.qmail@s402.sureserver.com> Message-ID: Yaron, We don't really have tools to help do this efficiently (especially in parallel). The DA, IS and AO are not related to this. It is all doable and not terribly difficult (I think) but we just don't have the pieces to 1) get the local to global index map (you need to do this based on your geometry) 2) given a sparse matrix and a mapping from its local rows/columns to global rows/columns stick the values into the larger map. To start you can simply loop over the rows of the small matrix and call matsetvalues onto the big matrix after mapping the row number and the column numbers for that row. This will be really slow until you preallocate the larger matrix. Figuring out the preallocation for the larger matrix is not terribly hard, you again loop over the rows in the small matrix count how many entries will be in the "diagonal part" of the large parallel matrix and how many in the "off-diagonal part". Once you have all this call MatCreateMPIAIJ() with the computed row lengths. All of this mapping stuff would just be done with arrays of integers. I'm sorry but this is so different from what our "paying customers" need that we cannot write the code. But, as always, we'll do our best to answer questions that may come up. Good luck, Barry On Tue, 6 Feb 2007, yaron at oak-research.com wrote: > Barry- > Maybe i'd better provide more details on what I'm trying to do > > *) I'm modeling current flowing through several different "block types", > each of which describes a section of a semiconductor device. Each block > type has a different geometry, which is triangulated to create an AIJ > matrix (Each row/column in the matrix represents a coordinate, and the > matrix values represent electrical admittance). There are about 100 > different types of these blocks, and since they have quite convoluted > geometries , their triangulation takes quite a while. > > *) My complete problem is composed of many (up to 10K) tiles, each of > which is one of the 100 blocks . I want to reuse the triangulation which > was done for each of the block, do I'd like to have a way of taking the > matrix objects of the individual blcks, and combine them into a large > matrix, taking into account their relative locations. > > *) This means that for each block instance, I would need to to translate > every internal coordinate/node, and map it to a global coordinate/node. > > *) Once I have a mapping of local to global indices, I'd like to take the > matrix values of the instances, and combine them to form a large matrix > which describes the complete problem. > > > So my question is : > *) What data structures (DA/IS/AO/???) should I use to achieve the above? > > Best Regards > Yaron > > > -------Original Message------- > From: Barry Smith > Subject: Re: Non-uniform 2D mesh questions > Sent: 02 Feb '07 13:38 > > > Yaron, > > Anything is possible :-) and maybe not terribly difficult to get > started. > > You could use DAGetMatrx() to give you the properly pre-allocated "huge" > Mat. > > Have each process loop over the "rectangular portion[s] of the domain" > that > it mostly owns (that is if a rectangular portion lies across two > processes just > assign it to one of them for this loop.) > > Then loop over the locations inside the rectangular portion calling > MatSetValuesStencil() for that row of the huge matrix to put the entries > from > the smaller matrix INTO the huge matrix using the natural grid i,j > coordindates > (so not have to map the coordinates from the grid location to the > location in > the matrix). > > This may require some thought to get right but should require little > coding > (if you are writting hundreds and hundreds of lines of code then likely > something is wrong). > > Good luck, > > Barry > > > On Tue, 30 Jan 2007, [LINK: > http://webmail.oak-research.com/compose.php?to=yaron at oak-research.com] > yaron at oak-research.com wrote: > > > Barry- > > So far I only thought of having a single large sparse matrix. > > > > Yaron > > > > > > -------Original Message------- > > From: Barry Smith > > Subject: Re: Non-uniform 2D mesh questions > > Sent: 30 Jan '07 10:58 > > > > > > Yaron, > > > > Do you want to end up generating a single large sparse matrix? Like a > > MPIAIJ > > matrix? Or do you want to somehow not store the entire huge matrix but > > still > > be able to solve with the composed matrix? Or both? > > > > Barry > > > > > > On Mon, 29 Jan 2007, [LINK: > > [LINK: > http://webmail.oak-research.com/compose.php?to=yaron at oak-research.com] > http://webmail.oak-research.com/compose.php?to=yaron at oak-research.com] > > [LINK: > http://webmail.oak-research.com/compose.php?to=yaron at oak-research.com] > yaron at oak-research.com wrote: > > > > > Barry- > > > Yes, each block is a rectangular portion of the domain. Not so small > > > though (more like 100 x 100 nodes) > > > > > > Yaron > > > > > > > > > -------Original Message------- > > > From: Barry Smith > > > Subject: Re: Non-uniform 2D mesh questions > > > Sent: 29 Jan '07 19:40 > > > > > > > > > Yaron, > > > > > > Is each one of these "blocks" a small rectangular part of the > > > domain (like a 4 by 5 set of nodes)? I don't understand what you > > > want to do. > > > > > > Barry > > > > > > > > > On Mon, 29 Jan 2007, [LINK: > > > [LINK: > > [LINK: > http://webmail.oak-research.com/compose.php?to=yaron at oak-research.com] > http://webmail.oak-research.com/compose.php?to=yaron at oak-research.com] > > [LINK: > http://webmail.oak-research.com/compose.php?to=yaron at oak-research.com] > http://webmail.oak-research.com/compose.php?to=yaron at oak-research.com] > > > [LINK: > > [LINK: > http://webmail.oak-research.com/compose.php?to=yaron at oak-research.com] > http://webmail.oak-research.com/compose.php?to=yaron at oak-research.com] > > [LINK: > http://webmail.oak-research.com/compose.php?to=yaron at oak-research.com] > yaron at oak-research.com wrote: > > > > > > > Hi all > > > > I have a laplace-type problem that's physically built from > repeating > > > > instances of the same block. > > > > I'm creaing matrices for the individual blocks, and I'd like to > > reuse > > > > the individual block matrices in order to compose the complete > > > problem. > > > > (i.e if there 10K instances of 20 blocks, I'd like to build 20 > > > matrices, > > > > then use them to compose the large complete matrix) > > > > Is a 2D DA the right object to do that? And if so, where can I > find > > a > > > > small example of building the DA object in parallel, then using > the > > > > different (for every instance) mappings of local nodes to global > > nodes > > > in > > > > order to build the complete matrix? > > > > > > > > > > > > Thanks > > > > Yaron > > > > > > > > > > From palott at ipst.umd.edu Thu Mar 1 16:43:01 2007 From: palott at ipst.umd.edu (P. Aaron Lott) Date: Thu, 1 Mar 2007 17:43:01 -0500 Subject: need help with makefile Message-ID: <27727123-941B-41D3-9C11-BEC596A6D638@ipst.umd.edu> Hi, I have a fortran90 code and I would like to use petsc's preconditioners inside of it. I'm having trouble setting up a makefile that will allow me include a petsc. It seems the problem is from the use of the c-preprocessor. My old makefile looked something like this: $(CMD) : $(SOBJS) $(F90) $(FLAGS) -o $(EXENAME) $(SOBJS) where SOBJ is a list of all of my object files. I was hoping to be able to make a petsc object file and include it in this list. Is this possible? Is there a tutorial on how to get petsc setup inside of your existing code? Thanks, -Aaron P. Aaron Lott Ph.D. Candidate 4239 Computer and Space Sciences Building University of Maryland College Park, MD 20742-4015 palott at ipst.umd.edu http://www.lcv.umd.edu/~palott Office: 301.405.4894 Fax: 301.314.0827 From knepley at gmail.com Thu Mar 1 16:55:05 2007 From: knepley at gmail.com (Matthew Knepley) Date: Thu, 1 Mar 2007 16:55:05 -0600 Subject: need help with makefile In-Reply-To: <27727123-941B-41D3-9C11-BEC596A6D638@ipst.umd.edu> References: <27727123-941B-41D3-9C11-BEC596A6D638@ipst.umd.edu> Message-ID: You usually need the PETSc Fortran includes in order to define variables and setup F90 interfaces. I would suggest looking at our examples: cd src/snes/examples/tutorials make ex5f90 Matt On 3/1/07, P. Aaron Lott wrote: > > > Hi, > > I have a fortran90 code and I would like to use petsc's > preconditioners inside of it. I'm having trouble setting up a > makefile that will allow me include a petsc. It seems the problem is > from the use of the c-preprocessor. > > My old makefile looked something like this: > > $(CMD) : $(SOBJS) > $(F90) $(FLAGS) -o $(EXENAME) $(SOBJS) > > where SOBJ is a list of all of my object files. > > I was hoping to be able to make a petsc object file and include it in > this list. Is this possible? Is there a tutorial on how to get petsc > setup inside of your existing code? > > Thanks, > > -Aaron > > > > P. Aaron Lott > Ph.D. Candidate > 4239 Computer and Space Sciences Building > University of Maryland > College Park, MD 20742-4015 > > palott at ipst.umd.edu > http://www.lcv.umd.edu/~palott > Office: 301.405.4894 > Fax: 301.314.0827 > > > -- One trouble is that despite this system, anyone who reads journals widely and critically is forced to realize that there are scarcely any bars to eventual publication. There seems to be no study too fragmented, no hypothesis too trivial, no literature citation too biased or too egotistical, no design too warped, no methodology too bungled, no presentation of results too inaccurate, too obscure, and too contradictory, no analysis too self-serving, no argument too circular, no conclusions too trifling or too unjustified, and no grammar and syntax too offensive for a paper to end up in print. -- Drummond Rennie From balay at mcs.anl.gov Thu Mar 1 16:56:29 2007 From: balay at mcs.anl.gov (Satish Balay) Date: Thu, 1 Mar 2007 16:56:29 -0600 (CST) Subject: need help with makefile In-Reply-To: <27727123-941B-41D3-9C11-BEC596A6D638@ipst.umd.edu> References: <27727123-941B-41D3-9C11-BEC596A6D638@ipst.umd.edu> Message-ID: The idea is to use a PETSc example makefile - and modify it appropriately. And with fortran codes - we require preprocessing [i.e source files that call PETSc routines should be .F] A minimal PETSc makefile is as follows: ------------------------------------------------ CFLAGS = FFLAGS = CPPFLAGS = FPPFLAGS = CLEANFILES = include ${PETSC_DIR}/bmake/common/base ex1f: ex1f.o chkopts -${FLINKER} -o ex1f ex1f.o ${PETSC_KSP_LIB} ${RM} ex1f.o ----------------------------------------------------- So you'll just make the change [with the variables CMD and SOBJS properly defined] ex1f -> $(CMD) ex1f.o -> $(SOBJS) Note: Necessary FFLAGS should already be PETSc config files. Some additional flags can be specified with FFLAGS variable Satish On Thu, 1 Mar 2007, P. Aaron Lott wrote: > > > Hi, > > I have a fortran90 code and I would like to use petsc's preconditioners > inside of it. I'm having trouble setting up a makefile that will allow me > include a petsc. It seems the problem is from the use of the c-preprocessor. > > My old makefile looked something like this: > > $(CMD) : $(SOBJS) > $(F90) $(FLAGS) -o $(EXENAME) $(SOBJS) > > where SOBJ is a list of all of my object files. > > I was hoping to be able to make a petsc object file and include it in this > list. Is this possible? Is there a tutorial on how to get petsc setup inside > of your existing code? > > Thanks, > > -Aaron > > > > P. Aaron Lott > Ph.D. Candidate > 4239 Computer and Space Sciences Building > University of Maryland > College Park, MD 20742-4015 > > palott at ipst.umd.edu > http://www.lcv.umd.edu/~palott > Office: 301.405.4894 > Fax: 301.314.0827 > > From palott at ipst.umd.edu Fri Mar 2 10:29:30 2007 From: palott at ipst.umd.edu (P. Aaron Lott) Date: Fri, 2 Mar 2007 11:29:30 -0500 Subject: need help with makefile In-Reply-To: References: <27727123-941B-41D3-9C11-BEC596A6D638@ipst.umd.edu> Message-ID: <5BC5FB72-7FF4-4A8E-9123-12211E4305A5@ipst.umd.edu> Thanks for this information. I think I setup things almost right, but something is causing preprocessor to complain. The errors it is giving are referring to lines of my main program file that doesn't exist. I would appreciate any recommendations about how to fix this problem. I've list some of the details of my setup below. Thanks, -Aaron e.g. macbeth% make unsteady_ex /usr/local/mpich-1.2.5.2/bin/mpicc -I/Users/palott/research/codes/ petsc/petsc-2.3.2-p8 -I/Users/palott/research/codes/pets c/petsc-2.3.2-p8/bmake/darwin-mpich-g -I/Users/palott/research/codes/ petsc/petsc-2.3.2-p8/include -I/usr/local/mpich-1.2. 5.2/include -E -traditional-cpp __unsteady_ex.c | /usr/bin/ grep -v '^ *#' > __unsteady_ex.F /usr/local/mpich-1.2.5.2/bin/mpif90 -c -I. -g __unsteady_ex.F -o unsteady_ex.o "__unsteady_ex.F", line 965.2: 1515-019 (S) Syntax is incorrect. But there are only 207 lines in this file..... In my current makefile I have F90=mpif90 F77=mpif77 F77FLAGS= -C -g -qdpc -qautodbl=dbl4 -qinitauto -Wl,-framework - Wl,vecLib FLAGS= -C -g -qdpc -qautodbl=dbl4 -qinitauto -Wl,-framework -Wl,vecLib EXENAME = unsteady_ex CMD:=EXENAME SOBJS = mpi.o quicksort.o routines.o binsort.o coordinates.o comm_maps.o elements.o globaldata.o sem.o comp_inv.o d1ops.o maps.o pardss.o dss.o stiffness.o convection.o Mass.o pressure.o buildU.o wind.o forcing.o boundary_cond.o navier_stokes.o runge_kutta.o out.o gmres.o pcg.o stream.o uzawa.o print_matrices.o projection.o bdf.o stokes_solvers.o initialize_mpi.o unsteady_ex.o $(CMD) : $(SOBJS) -${FLINKER} $(FLAGS) -o $(EXENAME) $(SOBJS) $(PETSC_SYS_LIB) mpi.o : ../Comm/mpi.f90 ;$(F90) $(FLAGS) -c ../Comm/mpi.f90 gmres.o : ../Solvers/gmres.f90 ;$(F90) $(FLAGS) -c ../Solvers/ gmres.f90 etc.... for all the object files except unsteady_ex.o unsteady_ex.F is my main program. In the example you gave, ex1f.F would be the main program, and at the top of the file it would have the statement: #include "include/finclude/petsc.h" So in my case, I've put that in the top of my unsteady_ex.F On Mar 1, 2007, at 5:56 PM, Satish Balay wrote: > The idea is to use a PETSc example makefile - and modify it > appropriately. > > And with fortran codes - we require preprocessing [i.e source files > that call PETSc routines should be .F] > > A minimal PETSc makefile is as follows: > ------------------------------------------------ > CFLAGS = > FFLAGS = > CPPFLAGS = > FPPFLAGS = > CLEANFILES = > > include ${PETSC_DIR}/bmake/common/base > > ex1f: ex1f.o chkopts > -${FLINKER} -o ex1f ex1f.o ${PETSC_KSP_LIB} > ${RM} ex1f.o > ----------------------------------------------------- > > So you'll just make the change [with the variables CMD and SOBJS > properly defined] > > ex1f -> $(CMD) > ex1f.o -> $(SOBJS) > > Note: Necessary FFLAGS should already be PETSc config files. Some > additional flags can be specified with FFLAGS variable > > Satish > > > On Thu, 1 Mar 2007, P. Aaron Lott wrote: > >> >> >> Hi, >> >> I have a fortran90 code and I would like to use petsc's >> preconditioners >> inside of it. I'm having trouble setting up a makefile that will >> allow me >> include a petsc. It seems the problem is from the use of the c- >> preprocessor. >> >> My old makefile looked something like this: >> >> $(CMD) : $(SOBJS) >> $(F90) $(FLAGS) -o $(EXENAME) $(SOBJS) >> >> where SOBJ is a list of all of my object files. >> >> I was hoping to be able to make a petsc object file and include it >> in this >> list. Is this possible? Is there a tutorial on how to get petsc >> setup inside >> of your existing code? >> >> Thanks, >> >> -Aaron >> >> >> >> P. Aaron Lott >> Ph.D. Candidate >> 4239 Computer and Space Sciences Building >> University of Maryland >> College Park, MD 20742-4015 >> >> palott at ipst.umd.edu >> http://www.lcv.umd.edu/~palott >> Office: 301.405.4894 >> Fax: 301.314.0827 >> >> > P. Aaron Lott Ph.D. Candidate 4239 Computer and Space Sciences Building University of Maryland College Park, MD 20742-4015 palott at ipst.umd.edu http://www.lcv.umd.edu/~palott Office: 301.405.4894 Fax: 301.314.0827 From balay at mcs.anl.gov Fri Mar 2 10:38:33 2007 From: balay at mcs.anl.gov (Satish Balay) Date: Fri, 2 Mar 2007 10:38:33 -0600 (CST) Subject: need help with makefile In-Reply-To: <5BC5FB72-7FF4-4A8E-9123-12211E4305A5@ipst.umd.edu> References: <27727123-941B-41D3-9C11-BEC596A6D638@ipst.umd.edu> <5BC5FB72-7FF4-4A8E-9123-12211E4305A5@ipst.umd.edu> Message-ID: On Fri, 2 Mar 2007, P. Aaron Lott wrote: > > Thanks for this information. I think I setup things almost right, but > something is causing preprocessor to complain. The errors it is giving are > referring to lines of my main program file that doesn't exist. The line numbers refer to the temoprary file created - __unsteady_ex.F You can run the following command to preserve the temporary file - and check it for the correct line number make unsteady_ex SKIP_RM=yes Preprocessing fortran code with C compiler can have some false positives. For eg: if you have quotes in comments - then cpp can misbehave. > > I would appreciate any recommendations about how to fix this problem. I've > list some of the details of my setup below. > > Thanks, > > -Aaron > > > > e.g. > > macbeth% make unsteady_ex > /usr/local/mpich-1.2.5.2/bin/mpicc > -I/Users/palott/research/codes/petsc/petsc-2.3.2-p8 > -I/Users/palott/research/codes/pets > c/petsc-2.3.2-p8/bmake/darwin-mpich-g > -I/Users/palott/research/codes/petsc/petsc-2.3.2-p8/include > -I/usr/local/mpich-1.2. > 5.2/include -E -traditional-cpp __unsteady_ex.c | /usr/bin/grep -v '^ > *#' > __unsteady_ex.F > /usr/local/mpich-1.2.5.2/bin/mpif90 -c -I. -g __unsteady_ex.F -o > unsteady_ex.o > "__unsteady_ex.F", line 965.2: 1515-019 (S) Syntax is incorrect. > > > But there are only 207 lines in this file..... > > > > In my current makefile I have > > F90=mpif90 > F77=mpif77 These don't need to be specified as PETSc .F.o targets don't use them.. [and you don't want to redefine whats already configured inside PETSc] > F77FLAGS= -C -g -qdpc -qautodbl=dbl4 -qinitauto -Wl,-framework -Wl,vecLib This variable is not used by PETSc target > FLAGS= -C -g -qdpc -qautodbl=dbl4 -qinitauto -Wl,-framework -Wl,vecLib PETSc should already use a blas implementation - so no need to specify vecLib again. > > > EXENAME = unsteady_ex > CMD:=EXENAME > > SOBJS = mpi.o quicksort.o routines.o binsort.o coordinates.o comm_maps.o > elements.o globaldata.o sem.o comp_inv.o d1ops.o maps.o pardss.o dss.o > stiffness.o convection.o Mass.o pressure.o buildU.o wind.o forcing.o > boundary_cond.o navier_stokes.o runge_kutta.o out.o gmres.o pcg.o stream.o > uzawa.o print_matrices.o projection.o bdf.o stokes_solvers.o > initialize_mpi.o unsteady_ex.o > > > $(CMD) : $(SOBJS) > -${FLINKER} $(FLAGS) -o $(EXENAME) $(SOBJS) $(PETSC_SYS_LIB) No need to specify FLAGS here - as the default targets pick it up [in this case the flags get added to FLINKER Satish > > > mpi.o : ../Comm/mpi.f90 ;$(F90) $(FLAGS) -c > ../Comm/mpi.f90 > gmres.o : ../Solvers/gmres.f90 ;$(F90) $(FLAGS) -c > ../Solvers/gmres.f90 > etc.... for all the object files except unsteady_ex.o > > unsteady_ex.F is my main program. > > In the example you gave, ex1f.F would be the main program, and at the top of > the file it would have the statement: > > #include "include/finclude/petsc.h" > > So in my case, I've put that in the top of my unsteady_ex.F > > > > > > > On Mar 1, 2007, at 5:56 PM, Satish Balay wrote: > > > The idea is to use a PETSc example makefile - and modify it > > appropriately. > > > > And with fortran codes - we require preprocessing [i.e source files > > that call PETSc routines should be .F] > > > > A minimal PETSc makefile is as follows: > > ------------------------------------------------ > > CFLAGS = > > FFLAGS = > > CPPFLAGS = > > FPPFLAGS = > > CLEANFILES = > > > > include ${PETSC_DIR}/bmake/common/base > > > > ex1f: ex1f.o chkopts > > -${FLINKER} -o ex1f ex1f.o ${PETSC_KSP_LIB} > > ${RM} ex1f.o > > ----------------------------------------------------- > > > > So you'll just make the change [with the variables CMD and SOBJS properly > > defined] > > > > ex1f -> $(CMD) > > ex1f.o -> $(SOBJS) > > > > Note: Necessary FFLAGS should already be PETSc config files. Some > > additional flags can be specified with FFLAGS variable > > > > Satish > > > > > > On Thu, 1 Mar 2007, P. Aaron Lott wrote: > > > > > > > > > > > Hi, > > > > > > I have a fortran90 code and I would like to use petsc's preconditioners > > > inside of it. I'm having trouble setting up a makefile that will allow me > > > include a petsc. It seems the problem is from the use of the > > > c-preprocessor. > > > > > > My old makefile looked something like this: > > > > > > $(CMD) : $(SOBJS) > > > $(F90) $(FLAGS) -o $(EXENAME) $(SOBJS) > > > > > > where SOBJ is a list of all of my object files. > > > > > > I was hoping to be able to make a petsc object file and include it in this > > > list. Is this possible? Is there a tutorial on how to get petsc setup > > > inside > > > of your existing code? > > > > > > Thanks, > > > > > > -Aaron > > > > > > > > > > > > P. Aaron Lott > > > Ph.D. Candidate > > > 4239 Computer and Space Sciences Building > > > University of Maryland > > > College Park, MD 20742-4015 > > > > > > palott at ipst.umd.edu > > > http://www.lcv.umd.edu/~palott > > > Office: 301.405.4894 > > > Fax: 301.314.0827 > > > > > > > > > > P. Aaron Lott > Ph.D. Candidate > 4239 Computer and Space Sciences Building > University of Maryland > College Park, MD 20742-4015 > > palott at ipst.umd.edu > http://www.lcv.umd.edu/~palott > Office: 301.405.4894 > Fax: 301.314.0827 > > From palott at ipst.umd.edu Fri Mar 2 10:58:54 2007 From: palott at ipst.umd.edu (P. Aaron Lott) Date: Fri, 2 Mar 2007 11:58:54 -0500 Subject: need help with makefile In-Reply-To: References: <27727123-941B-41D3-9C11-BEC596A6D638@ipst.umd.edu> <5BC5FB72-7FF4-4A8E-9123-12211E4305A5@ipst.umd.edu> Message-ID: <845FB810-2733-48D2-A255-D09F8DD68330@ipst.umd.edu> Thanks Satish, I checked the __unsteady_ex.F file. The preprocessor doesn't seem to like any of my fortran code. It starts complaining about the use of the use statements, and basically all the lines of code in this file. Any ideas? -Aaron Here are some of the statements the preprocessor complains about: use initialize use coordinates use maps use boundaries use elements use buildU use matrix_ops use output use d1ops use globaldata use dss use par_dss use forces use winds use stokes_solvers implicit none integer n,i,j real(kind=8) dt,gam integer nsteps type(coord) coords type(coord) proc_coords type(solution) GUXY,LUXY real(kind=8),allocatable, dimension(:):: force,wind integer nelements integer nely,nelx integer twod integer nglobaldofs integer irc,myrank,nprocs, ierr, status(MPI_STATUS_SIZE) call mpi_init(ierr) ! Get myrank call mpi_comm_rank(MPI_COMM_WORLD,myrank,ierr) call mpi_comm_size(MPI_COMM_WORLD,nprocs,ierr) call PetscInitialize( PETSC_NULL_CHARACTER, ierr ) call MPI_Comm_rank( PETSC_COMM_WORLD, myrank, ierr ) if (myrank .eq. 0) then print *, ?Hello World? endif call PetscFinalize(ierr) stop On Mar 2, 2007, at 11:38 AM, Satish Balay wrote: > > > On Fri, 2 Mar 2007, P. Aaron Lott wrote: > >> >> Thanks for this information. I think I setup things almost right, but >> something is causing preprocessor to complain. The errors it is >> giving are >> referring to lines of my main program file that doesn't exist. > > The line numbers refer to the temoprary file created - __unsteady_ex.F > > You can run the following command to preserve the temporary file - and > check it for the correct line number > > make unsteady_ex SKIP_RM=yes > > Preprocessing fortran code with C compiler can have some false > positives. For eg: if you have quotes in comments - then cpp can > misbehave. > >> >> I would appreciate any recommendations about how to fix this >> problem. I've >> list some of the details of my setup below. >> >> Thanks, >> >> -Aaron >> >> >> >> e.g. >> >> macbeth% make unsteady_ex >> /usr/local/mpich-1.2.5.2/bin/mpicc >> -I/Users/palott/research/codes/petsc/petsc-2.3.2-p8 >> -I/Users/palott/research/codes/pets >> c/petsc-2.3.2-p8/bmake/darwin-mpich-g >> -I/Users/palott/research/codes/petsc/petsc-2.3.2-p8/include >> -I/usr/local/mpich-1.2. >> 5.2/include -E -traditional-cpp __unsteady_ex.c | /usr/bin/ >> grep -v '^ >> *#' > __unsteady_ex.F >> /usr/local/mpich-1.2.5.2/bin/mpif90 -c -I. -g __unsteady_ex.F -o >> unsteady_ex.o >> "__unsteady_ex.F", line 965.2: 1515-019 (S) Syntax is incorrect. >> >> >> But there are only 207 lines in this file..... >> >> >> >> In my current makefile I have >> >> F90=mpif90 >> F77=mpif77 > > These don't need to be specified as PETSc .F.o targets don't use > them.. [and > you don't want to redefine whats already configured inside PETSc] > >> F77FLAGS= -C -g -qdpc -qautodbl=dbl4 -qinitauto -Wl,-framework - >> Wl,vecLib > > This variable is not used by PETSc target > >> FLAGS= -C -g -qdpc -qautodbl=dbl4 -qinitauto -Wl,-framework - >> Wl,vecLib > > PETSc should already use a blas implementation - so no need to specify > vecLib again. > >> >> >> EXENAME = unsteady_ex >> CMD:=EXENAME >> >> SOBJS = mpi.o quicksort.o routines.o binsort.o coordinates.o >> comm_maps.o >> elements.o globaldata.o sem.o comp_inv.o d1ops.o maps.o pardss.o >> dss.o >> stiffness.o convection.o Mass.o pressure.o buildU.o wind.o >> forcing.o >> boundary_cond.o navier_stokes.o runge_kutta.o out.o gmres.o >> pcg.o stream.o >> uzawa.o print_matrices.o projection.o bdf.o stokes_solvers.o >> initialize_mpi.o unsteady_ex.o >> >> >> $(CMD) : $(SOBJS) >> -${FLINKER} $(FLAGS) -o $(EXENAME) $(SOBJS) $(PETSC_SYS_LIB) > > No need to specify FLAGS here - as the default targets pick it up > [in this case > the flags get added to FLINKER > > Satish > >> >> >> mpi.o : ../Comm/mpi.f90 ;$(F90) $(FLAGS) -c >> ../Comm/mpi.f90 >> gmres.o : ../Solvers/gmres.f90 ;$(F90) $(FLAGS) -c >> ../Solvers/gmres.f90 >> etc.... for all the object files except unsteady_ex.o >> >> unsteady_ex.F is my main program. >> >> In the example you gave, ex1f.F would be the main program, and at >> the top of >> the file it would have the statement: >> >> #include "include/finclude/petsc.h" >> >> So in my case, I've put that in the top of my unsteady_ex.F >> >> >> >> >> >> >> On Mar 1, 2007, at 5:56 PM, Satish Balay wrote: >> >>> The idea is to use a PETSc example makefile - and modify it >>> appropriately. >>> >>> And with fortran codes - we require preprocessing [i.e source files >>> that call PETSc routines should be .F] >>> >>> A minimal PETSc makefile is as follows: >>> ------------------------------------------------ >>> CFLAGS = >>> FFLAGS = >>> CPPFLAGS = >>> FPPFLAGS = >>> CLEANFILES = >>> >>> include ${PETSC_DIR}/bmake/common/base >>> >>> ex1f: ex1f.o chkopts >>> -${FLINKER} -o ex1f ex1f.o ${PETSC_KSP_LIB} >>> ${RM} ex1f.o >>> ----------------------------------------------------- >>> >>> So you'll just make the change [with the variables CMD and SOBJS >>> properly >>> defined] >>> >>> ex1f -> $(CMD) >>> ex1f.o -> $(SOBJS) >>> >>> Note: Necessary FFLAGS should already be PETSc config files. Some >>> additional flags can be specified with FFLAGS variable >>> >>> Satish >>> >>> >>> On Thu, 1 Mar 2007, P. Aaron Lott wrote: >>> >>>> >>>> >>>> Hi, >>>> >>>> I have a fortran90 code and I would like to use petsc's >>>> preconditioners >>>> inside of it. I'm having trouble setting up a makefile that will >>>> allow me >>>> include a petsc. It seems the problem is from the use of the >>>> c-preprocessor. >>>> >>>> My old makefile looked something like this: >>>> >>>> $(CMD) : $(SOBJS) >>>> $(F90) $(FLAGS) -o $(EXENAME) $(SOBJS) >>>> >>>> where SOBJ is a list of all of my object files. >>>> >>>> I was hoping to be able to make a petsc object file and include >>>> it in this >>>> list. Is this possible? Is there a tutorial on how to get petsc >>>> setup >>>> inside >>>> of your existing code? >>>> >>>> Thanks, >>>> >>>> -Aaron >>>> >>>> >>>> >>>> P. Aaron Lott >>>> Ph.D. Candidate >>>> 4239 Computer and Space Sciences Building >>>> University of Maryland >>>> College Park, MD 20742-4015 >>>> >>>> palott at ipst.umd.edu >>>> http://www.lcv.umd.edu/~palott >>>> Office: 301.405.4894 >>>> Fax: 301.314.0827 >>>> >>>> >>> >> >> P. Aaron Lott >> Ph.D. Candidate >> 4239 Computer and Space Sciences Building >> University of Maryland >> College Park, MD 20742-4015 >> >> palott at ipst.umd.edu >> http://www.lcv.umd.edu/~palott >> Office: 301.405.4894 >> Fax: 301.314.0827 >> >> > P. Aaron Lott Ph.D. Candidate 4239 Computer and Space Sciences Building University of Maryland College Park, MD 20742-4015 palott at ipst.umd.edu http://www.lcv.umd.edu/~palott Office: 301.405.4894 Fax: 301.314.0827 From balay at mcs.anl.gov Fri Mar 2 11:16:12 2007 From: balay at mcs.anl.gov (Satish Balay) Date: Fri, 2 Mar 2007 11:16:12 -0600 (CST) Subject: need help with makefile In-Reply-To: <845FB810-2733-48D2-A255-D09F8DD68330@ipst.umd.edu> References: <27727123-941B-41D3-9C11-BEC596A6D638@ipst.umd.edu> <5BC5FB72-7FF4-4A8E-9123-12211E4305A5@ipst.umd.edu> <845FB810-2733-48D2-A255-D09F8DD68330@ipst.umd.edu> Message-ID: >>>/usr/local/mpich-1.2.5.2/bin/mpif90 -c -I. -g __unsteady_ex.F -o unsteady_ex.o "__unsteady_ex.F", line 965.2: 1515-019 (S) Syntax is incorrect. <<< The error is from mpif90 - not the preprocessor. Maybe its interpreting the code as fixed form? Try adding '-qfree' to FFLAGS and see if it makes a difference. Satish On Fri, 2 Mar 2007, P. Aaron Lott wrote: > > Thanks Satish, > > I checked the __unsteady_ex.F file. The preprocessor doesn't seem to like any > of my fortran code. It starts complaining about the use of the use statements, > and basically all the lines of code in this file. > > Any ideas? > > -Aaron > > Here are some of the statements the preprocessor complains about: > > use initialize > use coordinates > use maps > use boundaries > use elements > use buildU > use matrix_ops > use output > use d1ops > use globaldata > use dss > use par_dss > use forces > use winds > use stokes_solvers > > implicit none > > integer n,i,j > real(kind=8) dt,gam > integer nsteps > type(coord) coords > type(coord) proc_coords > type(solution) GUXY,LUXY > real(kind=8),allocatable, dimension(:):: force,wind > integer nelements > integer nely,nelx > integer twod > integer nglobaldofs > integer irc,myrank,nprocs, ierr, status(MPI_STATUS_SIZE) > > call mpi_init(ierr) > ! Get myrank > call mpi_comm_rank(MPI_COMM_WORLD,myrank,ierr) > call mpi_comm_size(MPI_COMM_WORLD,nprocs,ierr) > > call PetscInitialize( PETSC_NULL_CHARACTER, ierr ) > call MPI_Comm_rank( PETSC_COMM_WORLD, myrank, ierr ) > if (myrank .eq. 0) then > print *, ?Hello World? > endif > call PetscFinalize(ierr) > > stop > > > > > > > On Mar 2, 2007, at 11:38 AM, Satish Balay wrote: > > > > > > > On Fri, 2 Mar 2007, P. Aaron Lott wrote: > > > > > > > > Thanks for this information. I think I setup things almost right, but > > > something is causing preprocessor to complain. The errors it is giving are > > > referring to lines of my main program file that doesn't exist. > > > > The line numbers refer to the temoprary file created - __unsteady_ex.F > > > > You can run the following command to preserve the temporary file - and > > check it for the correct line number > > > > make unsteady_ex SKIP_RM=yes > > > > Preprocessing fortran code with C compiler can have some false > > positives. For eg: if you have quotes in comments - then cpp can > > misbehave. > > > > > > > > I would appreciate any recommendations about how to fix this problem. I've > > > list some of the details of my setup below. > > > > > > Thanks, > > > > > > -Aaron > > > > > > > > > > > > e.g. > > > > > > macbeth% make unsteady_ex > > > /usr/local/mpich-1.2.5.2/bin/mpicc > > > -I/Users/palott/research/codes/petsc/petsc-2.3.2-p8 > > > -I/Users/palott/research/codes/pets > > > c/petsc-2.3.2-p8/bmake/darwin-mpich-g > > > -I/Users/palott/research/codes/petsc/petsc-2.3.2-p8/include > > > -I/usr/local/mpich-1.2. > > > 5.2/include -E -traditional-cpp __unsteady_ex.c | /usr/bin/grep -v > > > '^ > > > *#' > __unsteady_ex.F > > > /usr/local/mpich-1.2.5.2/bin/mpif90 -c -I. -g __unsteady_ex.F -o > > > unsteady_ex.o > > > "__unsteady_ex.F", line 965.2: 1515-019 (S) Syntax is incorrect. > > > > > > > > > But there are only 207 lines in this file..... > > > > > > > > > > > > In my current makefile I have > > > > > > F90=mpif90 > > > F77=mpif77 > > > > These don't need to be specified as PETSc .F.o targets don't use them.. [and > > you don't want to redefine whats already configured inside PETSc] > > > > > F77FLAGS= -C -g -qdpc -qautodbl=dbl4 -qinitauto -Wl,-framework -Wl,vecLib > > > > This variable is not used by PETSc target > > > > > FLAGS= -C -g -qdpc -qautodbl=dbl4 -qinitauto -Wl,-framework -Wl,vecLib > > > > PETSc should already use a blas implementation - so no need to specify > > vecLib again. > > > > > > > > > > > EXENAME = unsteady_ex > > > CMD:=EXENAME > > > > > > SOBJS = mpi.o quicksort.o routines.o binsort.o coordinates.o comm_maps.o > > > elements.o globaldata.o sem.o comp_inv.o d1ops.o maps.o pardss.o dss.o > > > stiffness.o convection.o Mass.o pressure.o buildU.o wind.o forcing.o > > > boundary_cond.o navier_stokes.o runge_kutta.o out.o gmres.o pcg.o > > > stream.o > > > uzawa.o print_matrices.o projection.o bdf.o stokes_solvers.o > > > initialize_mpi.o unsteady_ex.o > > > > > > > > > $(CMD) : $(SOBJS) > > > -${FLINKER} $(FLAGS) -o $(EXENAME) $(SOBJS) $(PETSC_SYS_LIB) > > > > No need to specify FLAGS here - as the default targets pick it up [in this > > case > > the flags get added to FLINKER > > > > Satish > > > > > > > > > > > mpi.o : ../Comm/mpi.f90 ;$(F90) $(FLAGS) -c > > > ../Comm/mpi.f90 > > > gmres.o : ../Solvers/gmres.f90 ;$(F90) $(FLAGS) -c > > > ../Solvers/gmres.f90 > > > etc.... for all the object files except unsteady_ex.o > > > > > > unsteady_ex.F is my main program. > > > > > > In the example you gave, ex1f.F would be the main program, and at the top > > > of > > > the file it would have the statement: > > > > > > #include "include/finclude/petsc.h" > > > > > > So in my case, I've put that in the top of my unsteady_ex.F > > > > > > > > > > > > > > > > > > > > > On Mar 1, 2007, at 5:56 PM, Satish Balay wrote: > > > > > > > The idea is to use a PETSc example makefile - and modify it > > > > appropriately. > > > > > > > > And with fortran codes - we require preprocessing [i.e source files > > > > that call PETSc routines should be .F] > > > > > > > > A minimal PETSc makefile is as follows: > > > > ------------------------------------------------ > > > > CFLAGS = > > > > FFLAGS = > > > > CPPFLAGS = > > > > FPPFLAGS = > > > > CLEANFILES = > > > > > > > > include ${PETSC_DIR}/bmake/common/base > > > > > > > > ex1f: ex1f.o chkopts > > > > -${FLINKER} -o ex1f ex1f.o ${PETSC_KSP_LIB} > > > > ${RM} ex1f.o > > > > ----------------------------------------------------- > > > > > > > > So you'll just make the change [with the variables CMD and SOBJS > > > > properly > > > > defined] > > > > > > > > ex1f -> $(CMD) > > > > ex1f.o -> $(SOBJS) > > > > > > > > Note: Necessary FFLAGS should already be PETSc config files. Some > > > > additional flags can be specified with FFLAGS variable > > > > > > > > Satish > > > > > > > > > > > > On Thu, 1 Mar 2007, P. Aaron Lott wrote: > > > > > > > > > > > > > > > > > > > Hi, > > > > > > > > > > I have a fortran90 code and I would like to use petsc's > > > > > preconditioners > > > > > inside of it. I'm having trouble setting up a makefile that will allow > > > > > me > > > > > include a petsc. It seems the problem is from the use of the > > > > > c-preprocessor. > > > > > > > > > > My old makefile looked something like this: > > > > > > > > > > $(CMD) : $(SOBJS) > > > > > $(F90) $(FLAGS) -o $(EXENAME) $(SOBJS) > > > > > > > > > > where SOBJ is a list of all of my object files. > > > > > > > > > > I was hoping to be able to make a petsc object file and include it in > > > > > this > > > > > list. Is this possible? Is there a tutorial on how to get petsc setup > > > > > inside > > > > > of your existing code? > > > > > > > > > > Thanks, > > > > > > > > > > -Aaron > > > > > > > > > > > > > > > > > > > > P. Aaron Lott > > > > > Ph.D. Candidate > > > > > 4239 Computer and Space Sciences Building > > > > > University of Maryland > > > > > College Park, MD 20742-4015 > > > > > > > > > > palott at ipst.umd.edu > > > > > http://www.lcv.umd.edu/~palott > > > > > Office: 301.405.4894 > > > > > Fax: 301.314.0827 > > > > > > > > > > > > > > > > > > > > P. Aaron Lott > > > Ph.D. Candidate > > > 4239 Computer and Space Sciences Building > > > University of Maryland > > > College Park, MD 20742-4015 > > > > > > palott at ipst.umd.edu > > > http://www.lcv.umd.edu/~palott > > > Office: 301.405.4894 > > > Fax: 301.314.0827 > > > > > > > > > > P. Aaron Lott > Ph.D. Candidate > 4239 Computer and Space Sciences Building > University of Maryland > College Park, MD 20742-4015 > > palott at ipst.umd.edu > http://www.lcv.umd.edu/~palott > Office: 301.405.4894 > Fax: 301.314.0827 > > From jianings at gmail.com Fri Mar 2 13:34:55 2007 From: jianings at gmail.com (Jianing Shi) Date: Fri, 2 Mar 2007 13:34:55 -0600 Subject: about Unstructured Meshes In-Reply-To: References: Message-ID: <63516a2e0703021134q136b96edl34f3c3c62532a263@mail.gmail.com> Actually, I have a very similar question for unstructured meshes. I took a look at the tutorial in PETSc using meshes. Looks like there is a mixture programming of meshes (written in OOP) and the PETSc solver. Looks very complicated, I must say. Besides, what would be a golden packages using meshes that would work with PETSc? Say, PETSc (solver) + ParMetis (partition meshes) + Some sort of mesh generators (I don't what)? Jianing From knepley at gmail.com Fri Mar 2 13:42:16 2007 From: knepley at gmail.com (Matthew Knepley) Date: Fri, 2 Mar 2007 13:42:16 -0600 Subject: about Unstructured Meshes In-Reply-To: <63516a2e0703021134q136b96edl34f3c3c62532a263@mail.gmail.com> References: <63516a2e0703021134q136b96edl34f3c3c62532a263@mail.gmail.com> Message-ID: On 3/2/07, Jianing Shi wrote: > Actually, I have a very similar question for unstructured meshes. > > I took a look at the tutorial in PETSc using meshes. Looks like there > is a mixture programming of meshes (written in OOP) and the PETSc > solver. Looks very complicated, I must say. Well, in my experience, implementing this is usually 100 times more complicated, so I invite you prove me wrong. > Besides, what would be a golden packages using meshes that would work > with PETSc? Say, PETSc (solver) + ParMetis (partition meshes) + Some > sort of mesh generators (I don't what)? I do not understand the question. Matt > Jianing > > -- One trouble is that despite this system, anyone who reads journals widely and critically is forced to realize that there are scarcely any bars to eventual publication. There seems to be no study too fragmented, no hypothesis too trivial, no literature citation too biased or too egotistical, no design too warped, no methodology too bungled, no presentation of results too inaccurate, too obscure, and too contradictory, no analysis too self-serving, no argument too circular, no conclusions too trifling or too unjustified, and no grammar and syntax too offensive for a paper to end up in print. -- Drummond Rennie From jianings at gmail.com Fri Mar 2 14:21:32 2007 From: jianings at gmail.com (Jianing Shi) Date: Fri, 2 Mar 2007 14:21:32 -0600 Subject: about Unstructured Meshes In-Reply-To: References: <63516a2e0703021134q136b96edl34f3c3c62532a263@mail.gmail.com> Message-ID: <63516a2e0703021221o582a9ad1wf0ca3eb4f8a51450@mail.gmail.com> > > I took a look at the tutorial in PETSc using meshes. Looks like there > > is a mixture programming of meshes (written in OOP) and the PETSc > > solver. Looks very complicated, I must say. > > Well, in my experience, implementing this is usually 100 times more > complicated, so I invite you prove me wrong. I can see it is a lot of work to implement it. I am a beginner in PETSc, even using it seems complicated to me : ) > > Besides, what would be a golden packages using meshes that would work > > with PETSc? Say, PETSc (solver) + ParMetis (partition meshes) + Some > > sort of mesh generators (I don't what)? > > I do not understand the question. Well, let me rephrase my question. So the mesh support in PETSc already includes the functionality of partitioning meshes, I guess, using ParMetis, is that the case? Something that an end user need to worry about is really how to generate a mesh that is tailored towards his/her application. I am trying to write a library on top of PETSc meshes that will generate meshes according to some neurophysiology. I would like to know what are the mesh generate softwares out there that will interface nicely with PETSc, or if it makes sense for me to write my own? Just would like to understand more about the data structure in the PETSc ALE::Mesh classes. Is there any tutorial out there apart from looking at the source code in the mesh directory? I am currently using the petsc-2.3.2-p3. Is there any new functionality about meshes in the development version? Jianing From knepley at gmail.com Fri Mar 2 14:38:08 2007 From: knepley at gmail.com (Matthew Knepley) Date: Fri, 2 Mar 2007 14:38:08 -0600 Subject: about Unstructured Meshes In-Reply-To: <63516a2e0703021221o582a9ad1wf0ca3eb4f8a51450@mail.gmail.com> References: <63516a2e0703021134q136b96edl34f3c3c62532a263@mail.gmail.com> <63516a2e0703021221o582a9ad1wf0ca3eb4f8a51450@mail.gmail.com> Message-ID: On 3/2/07, Jianing Shi wrote: > Well, let me rephrase my question. So the mesh support in PETSc > already includes the functionality of partitioning meshes, I guess, > using ParMetis, is that the case? Something that an end user need to > worry about is really how to generate a mesh that is tailored towards > his/her application. It can use a range of partioners, like Chaco for instance. > I am trying to write a library on top of PETSc meshes that will > generate meshes according to some neurophysiology. I would like to > know what are the mesh generate softwares out there that will > interface nicely with PETSc, or if it makes sense for me to write my > own? Just would like to understand more about the data structure in > the PETSc ALE::Mesh classes. Is there any tutorial out there apart > from looking at the source code in the mesh directory? 1) No, it makes no sense for you to write a mesh generator 2) In 2D, Triangle. In 3d, the only free things are TetGen and Netgen. I support TetGen. Hopefully, CMU will release its MG soon. There is a tutorial on the website. > I am currently using the petsc-2.3.2-p3. Is there any new > functionality about meshes in the development version? All the working stuff is in petsc-dev. Matt > Jianing -- One trouble is that despite this system, anyone who reads journals widely and critically is forced to realize that there are scarcely any bars to eventual publication. There seems to be no study too fragmented, no hypothesis too trivial, no literature citation too biased or too egotistical, no design too warped, no methodology too bungled, no presentation of results too inaccurate, too obscure, and too contradictory, no analysis too self-serving, no argument too circular, no conclusions too trifling or too unjustified, and no grammar and syntax too offensive for a paper to end up in print. -- Drummond Rennie From zonexo at gmail.com Fri Mar 2 20:26:04 2007 From: zonexo at gmail.com (Ben Tay) Date: Sat, 3 Mar 2007 10:26:04 +0800 Subject: Deleting hypre files other than the library Message-ID: <804ab5d40703021826k55e039edl4d64fbe6307053df@mail.gmail.com> Hi, I've used --download-hypre=1 to integrate hypre into PETSc. I've found the hypre directory in externalpackages. hypre takes up about 136mb. Can I delete the files other than the library files (such as src) after I've compile them? Thanks alot! -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Fri Mar 2 20:28:56 2007 From: bsmith at mcs.anl.gov (Barry Smith) Date: Fri, 2 Mar 2007 20:28:56 -0600 (CST) Subject: Deleting hypre files other than the library In-Reply-To: <804ab5d40703021826k55e039edl4d64fbe6307053df@mail.gmail.com> References: <804ab5d40703021826k55e039edl4d64fbe6307053df@mail.gmail.com> Message-ID: You can delete src and docs Barry On Sat, 3 Mar 2007, Ben Tay wrote: > Hi, > > I've used --download-hypre=1 to integrate hypre into PETSc. I've found the > hypre directory in externalpackages. hypre takes up about 136mb. Can I > delete the files other than the library files (such as src) after I've > compile them? > > Thanks alot! > From zonexo at gmail.com Sat Mar 3 03:41:55 2007 From: zonexo at gmail.com (Ben Tay) Date: Sat, 3 Mar 2007 17:41:55 +0800 Subject: Problem with using --download-f-blas-lapack=1 in windows Message-ID: <804ab5d40703030141l272d166ao1cc945caf6bfad97@mail.gmail.com> Hi, I tried to compile PETSc with compaq visual fortran in cygwin. My command is $ ./config/configure.py --with-cc='win32fe cl' --with-fc='win32fe f90' --download-f-blas-lapack=1 --with-f90-interface=win32 --with-x=0 --with-mpi-dir=/cygdrive/c/program\ files/MPICH/SDK I then got this error message: ================================================================================ Compiling FBLASLAPACK; this may take several minutes ================================================================================ TESTING: checkLib from config.packages.BlasLapack (python/BuildSystem/config/pack ******************************************************************************** * UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log for detail s): -------------------------------------------------------------------------------- ------- Could not use downloaded f-blas-lapack? ******************************************************************************** * Can someone enlighten me what is wrong? Btw, I found the win32 libraries of blas/lapack in the externalpackages/fblaslapack/win32 directory Thank you. -------------- next part -------------- An HTML attachment was scrubbed... URL: From palott at ipst.umd.edu Sat Mar 3 08:56:36 2007 From: palott at ipst.umd.edu (P. Aaron Lott) Date: Sat, 3 Mar 2007 09:56:36 -0500 Subject: need help with makefile In-Reply-To: References: <27727123-941B-41D3-9C11-BEC596A6D638@ipst.umd.edu> <5BC5FB72-7FF4-4A8E-9123-12211E4305A5@ipst.umd.edu> <845FB810-2733-48D2-A255-D09F8DD68330@ipst.umd.edu> Message-ID: <55AF300D-692C-405B-8698-9014A5642819@ipst.umd.edu> Hi Satish, You're right the problem is from mpif90, I tried mpif90 -qfree -c -I. -g __unsteady_ex.F -o unsteady_ex.o but I'm getting the same error messages. The file unsteady_ex compiled fine before using petsc and the preprocessor, so I really don't know what the problem could be. Any ideas? -Aaron On Mar 2, 2007, at 12:16 PM, Satish Balay wrote: >>>> /usr/local/mpich-1.2.5.2/bin/mpif90 -c -I. -g __unsteady_ex.F - >>>> o unsteady_ex.o > "__unsteady_ex.F", line 965.2: 1515-019 (S) Syntax is incorrect. > <<< > > The error is from mpif90 - not the preprocessor. Maybe its > interpreting > the code as fixed form? Try adding '-qfree' to FFLAGS and see if it > makes > a difference. > > Satish > > On Fri, 2 Mar 2007, P. Aaron Lott wrote: > >> >> Thanks Satish, >> >> I checked the __unsteady_ex.F file. The preprocessor doesn't seem >> to like any >> of my fortran code. It starts complaining about the use of the use >> statements, >> and basically all the lines of code in this file. >> >> Any ideas? >> >> -Aaron >> >> Here are some of the statements the preprocessor complains about: >> >> use initialize >> use coordinates >> use maps >> use boundaries >> use elements >> use buildU >> use matrix_ops >> use output >> use d1ops >> use globaldata >> use dss >> use par_dss >> use forces >> use winds >> use stokes_solvers >> >> implicit none >> >> integer n,i,j >> real(kind=8) dt,gam >> integer nsteps >> type(coord) coords >> type(coord) proc_coords >> type(solution) GUXY,LUXY >> real(kind=8),allocatable, dimension(:):: force,wind >> integer nelements >> integer nely,nelx >> integer twod >> integer nglobaldofs >> integer irc,myrank,nprocs, ierr, status(MPI_STATUS_SIZE) >> >> call mpi_init(ierr) >> ! Get myrank >> call mpi_comm_rank(MPI_COMM_WORLD,myrank,ierr) >> call mpi_comm_size(MPI_COMM_WORLD,nprocs,ierr) >> >> call PetscInitialize( PETSC_NULL_CHARACTER, ierr ) >> call MPI_Comm_rank( PETSC_COMM_WORLD, myrank, ierr ) >> if (myrank .eq. 0) then >> print *, ?Hello World? >> endif >> call PetscFinalize(ierr) >> >> stop >> >> >> >> >> >> >> On Mar 2, 2007, at 11:38 AM, Satish Balay wrote: >> >>> >>> >>> On Fri, 2 Mar 2007, P. Aaron Lott wrote: >>> >>>> >>>> Thanks for this information. I think I setup things almost >>>> right, but >>>> something is causing preprocessor to complain. The errors it is >>>> giving are >>>> referring to lines of my main program file that doesn't exist. >>> >>> The line numbers refer to the temoprary file created - >>> __unsteady_ex.F >>> >>> You can run the following command to preserve the temporary file >>> - and >>> check it for the correct line number >>> >>> make unsteady_ex SKIP_RM=yes >>> >>> Preprocessing fortran code with C compiler can have some false >>> positives. For eg: if you have quotes in comments - then cpp can >>> misbehave. >>> >>>> >>>> I would appreciate any recommendations about how to fix this >>>> problem. I've >>>> list some of the details of my setup below. >>>> >>>> Thanks, >>>> >>>> -Aaron >>>> >>>> >>>> >>>> e.g. >>>> >>>> macbeth% make unsteady_ex >>>> /usr/local/mpich-1.2.5.2/bin/mpicc >>>> -I/Users/palott/research/codes/petsc/petsc-2.3.2-p8 >>>> -I/Users/palott/research/codes/pets >>>> c/petsc-2.3.2-p8/bmake/darwin-mpich-g >>>> -I/Users/palott/research/codes/petsc/petsc-2.3.2-p8/include >>>> -I/usr/local/mpich-1.2. >>>> 5.2/include -E -traditional-cpp __unsteady_ex.c | /usr/ >>>> bin/grep -v >>>> '^ >>>> *#' > __unsteady_ex.F >>>> /usr/local/mpich-1.2.5.2/bin/mpif90 -c -I. -g __unsteady_ex.F -o >>>> unsteady_ex.o >>>> "__unsteady_ex.F", line 965.2: 1515-019 (S) Syntax is incorrect. >>>> >>>> >>>> But there are only 207 lines in this file..... >>>> >>>> >>>> >>>> In my current makefile I have >>>> >>>> F90=mpif90 >>>> F77=mpif77 >>> >>> These don't need to be specified as PETSc .F.o targets don't use >>> them.. [and >>> you don't want to redefine whats already configured inside PETSc] >>> >>>> F77FLAGS= -C -g -qdpc -qautodbl=dbl4 -qinitauto -Wl,-framework - >>>> Wl,vecLib >>> >>> This variable is not used by PETSc target >>> >>>> FLAGS= -C -g -qdpc -qautodbl=dbl4 -qinitauto -Wl,-framework - >>>> Wl,vecLib >>> >>> PETSc should already use a blas implementation - so no need to >>> specify >>> vecLib again. >>> >>>> >>>> >>>> EXENAME = unsteady_ex >>>> CMD:=EXENAME >>>> >>>> SOBJS = mpi.o quicksort.o routines.o binsort.o coordinates.o >>>> comm_maps.o >>>> elements.o globaldata.o sem.o comp_inv.o d1ops.o maps.o >>>> pardss.o dss.o >>>> stiffness.o convection.o Mass.o pressure.o buildU.o wind.o >>>> forcing.o >>>> boundary_cond.o navier_stokes.o runge_kutta.o out.o gmres.o pcg.o >>>> stream.o >>>> uzawa.o print_matrices.o projection.o bdf.o stokes_solvers.o >>>> initialize_mpi.o unsteady_ex.o >>>> >>>> >>>> $(CMD) : $(SOBJS) >>>> -${FLINKER} $(FLAGS) -o $(EXENAME) $(SOBJS) $(PETSC_SYS_LIB) >>> >>> No need to specify FLAGS here - as the default targets pick it up >>> [in this >>> case >>> the flags get added to FLINKER >>> >>> Satish >>> >>>> >>>> >>>> mpi.o : ../Comm/mpi.f90 ;$(F90) $(FLAGS) -c >>>> ../Comm/mpi.f90 >>>> gmres.o : ../Solvers/gmres.f90 ;$(F90) $(FLAGS) -c >>>> ../Solvers/gmres.f90 >>>> etc.... for all the object files except unsteady_ex.o >>>> >>>> unsteady_ex.F is my main program. >>>> >>>> In the example you gave, ex1f.F would be the main program, and >>>> at the top >>>> of >>>> the file it would have the statement: >>>> >>>> #include "include/finclude/petsc.h" >>>> >>>> So in my case, I've put that in the top of my unsteady_ex.F >>>> >>>> >>>> >>>> >>>> >>>> >>>> On Mar 1, 2007, at 5:56 PM, Satish Balay wrote: >>>> >>>>> The idea is to use a PETSc example makefile - and modify it >>>>> appropriately. >>>>> >>>>> And with fortran codes - we require preprocessing [i.e source >>>>> files >>>>> that call PETSc routines should be .F] >>>>> >>>>> A minimal PETSc makefile is as follows: >>>>> ------------------------------------------------ >>>>> CFLAGS = >>>>> FFLAGS = >>>>> CPPFLAGS = >>>>> FPPFLAGS = >>>>> CLEANFILES = >>>>> >>>>> include ${PETSC_DIR}/bmake/common/base >>>>> >>>>> ex1f: ex1f.o chkopts >>>>> -${FLINKER} -o ex1f ex1f.o ${PETSC_KSP_LIB} >>>>> ${RM} ex1f.o >>>>> ----------------------------------------------------- >>>>> >>>>> So you'll just make the change [with the variables CMD and SOBJS >>>>> properly >>>>> defined] >>>>> >>>>> ex1f -> $(CMD) >>>>> ex1f.o -> $(SOBJS) >>>>> >>>>> Note: Necessary FFLAGS should already be PETSc config files. Some >>>>> additional flags can be specified with FFLAGS variable >>>>> >>>>> Satish >>>>> >>>>> >>>>> On Thu, 1 Mar 2007, P. Aaron Lott wrote: >>>>> >>>>>> >>>>>> >>>>>> Hi, >>>>>> >>>>>> I have a fortran90 code and I would like to use petsc's >>>>>> preconditioners >>>>>> inside of it. I'm having trouble setting up a makefile that >>>>>> will allow >>>>>> me >>>>>> include a petsc. It seems the problem is from the use of the >>>>>> c-preprocessor. >>>>>> >>>>>> My old makefile looked something like this: >>>>>> >>>>>> $(CMD) : $(SOBJS) >>>>>> $(F90) $(FLAGS) -o $(EXENAME) $(SOBJS) >>>>>> >>>>>> where SOBJ is a list of all of my object files. >>>>>> >>>>>> I was hoping to be able to make a petsc object file and >>>>>> include it in >>>>>> this >>>>>> list. Is this possible? Is there a tutorial on how to get >>>>>> petsc setup >>>>>> inside >>>>>> of your existing code? >>>>>> >>>>>> Thanks, >>>>>> >>>>>> -Aaron >>>>>> >>>>>> >>>>>> >>>>>> P. Aaron Lott >>>>>> Ph.D. Candidate >>>>>> 4239 Computer and Space Sciences Building >>>>>> University of Maryland >>>>>> College Park, MD 20742-4015 >>>>>> >>>>>> palott at ipst.umd.edu >>>>>> http://www.lcv.umd.edu/~palott >>>>>> Office: 301.405.4894 >>>>>> Fax: 301.314.0827 >>>>>> >>>>>> >>>>> >>>> >>>> P. Aaron Lott >>>> Ph.D. Candidate >>>> 4239 Computer and Space Sciences Building >>>> University of Maryland >>>> College Park, MD 20742-4015 >>>> >>>> palott at ipst.umd.edu >>>> http://www.lcv.umd.edu/~palott >>>> Office: 301.405.4894 >>>> Fax: 301.314.0827 >>>> >>>> >>> >> >> P. Aaron Lott >> Ph.D. Candidate >> 4239 Computer and Space Sciences Building >> University of Maryland >> College Park, MD 20742-4015 >> >> palott at ipst.umd.edu >> http://www.lcv.umd.edu/~palott >> Office: 301.405.4894 >> Fax: 301.314.0827 >> >> > P. Aaron Lott Ph.D. Candidate 4239 Computer and Space Sciences Building University of Maryland College Park, MD 20742-4015 palott at ipst.umd.edu http://www.lcv.umd.edu/~palott Office: 301.405.4894 Fax: 301.314.0827 From balay at mcs.anl.gov Sat Mar 3 10:28:45 2007 From: balay at mcs.anl.gov (Satish Balay) Date: Sat, 3 Mar 2007 10:28:45 -0600 (CST) Subject: Problem with using --download-f-blas-lapack=1 in windows In-Reply-To: <804ab5d40703030141l272d166ao1cc945caf6bfad97@mail.gmail.com> References: <804ab5d40703030141l272d166ao1cc945caf6bfad97@mail.gmail.com> Message-ID: if you have installation problems - send us configure.log at petsc-maint at mcs.anl.gov Satish On Sat, 3 Mar 2007, Ben Tay wrote: > Hi, > > I tried to compile PETSc with compaq visual fortran in cygwin. My command is > > $ ./config/configure.py --with-cc='win32fe cl' --with-fc='win32fe f90' > --download-f-blas-lapack=1 --with-f90-interface=win32 --with-x=0 > --with-mpi-dir=/cygdrive/c/program\ files/MPICH/SDK > > I then got this error message: > > ================================================================================ > Compiling FBLASLAPACK; this may take several minutes > ================================================================================ > TESTING: checkLib from config.packages.BlasLapack > (python/BuildSystem/config/pack > ******************************************************************************** > * > UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log for > detail > s): > -------------------------------------------------------------------------------- > ------- > Could not use downloaded f-blas-lapack? > ******************************************************************************** > * > > Can someone enlighten me what is wrong? Btw, I found the win32 libraries of > blas/lapack in the externalpackages/fblaslapack/win32 directory > > Thank you. > From balay at mcs.anl.gov Sat Mar 3 10:29:50 2007 From: balay at mcs.anl.gov (Satish Balay) Date: Sat, 3 Mar 2007 10:29:50 -0600 (CST) Subject: need help with makefile In-Reply-To: <55AF300D-692C-405B-8698-9014A5642819@ipst.umd.edu> References: <27727123-941B-41D3-9C11-BEC596A6D638@ipst.umd.edu> <5BC5FB72-7FF4-4A8E-9123-12211E4305A5@ipst.umd.edu> <845FB810-2733-48D2-A255-D09F8DD68330@ipst.umd.edu> <55AF300D-692C-405B-8698-9014A5642819@ipst.umd.edu> Message-ID: Can you send us the preprocessed __unsteady_ex.F file? Satish On Sat, 3 Mar 2007, P. Aaron Lott wrote: > > Hi Satish, > > You're right the problem is from mpif90, I tried > > mpif90 -qfree -c -I. -g __unsteady_ex.F -o unsteady_ex.o > > but I'm getting the same error messages. The file unsteady_ex compiled fine > before using petsc and the preprocessor, so I really don't know what the > problem could be. > > Any ideas? > > -Aaron > > > > > > > > On Mar 2, 2007, at 12:16 PM, Satish Balay wrote: > > > > > > /usr/local/mpich-1.2.5.2/bin/mpif90 -c -I. -g __unsteady_ex.F -o > > > > > unsteady_ex.o > > "__unsteady_ex.F", line 965.2: 1515-019 (S) Syntax is incorrect. > > <<< > > > > The error is from mpif90 - not the preprocessor. Maybe its interpreting > > the code as fixed form? Try adding '-qfree' to FFLAGS and see if it makes > > a difference. > > > > Satish > > > > On Fri, 2 Mar 2007, P. Aaron Lott wrote: > > > > > > > > Thanks Satish, > > > > > > I checked the __unsteady_ex.F file. The preprocessor doesn't seem to like > > > any > > > of my fortran code. It starts complaining about the use of the use > > > statements, > > > and basically all the lines of code in this file. > > > > > > Any ideas? > > > > > > -Aaron > > > > > > Here are some of the statements the preprocessor complains about: > > > > > > use initialize > > > use coordinates > > > use maps > > > use boundaries > > > use elements > > > use buildU > > > use matrix_ops > > > use output > > > use d1ops > > > use globaldata > > > use dss > > > use par_dss > > > use forces > > > use winds > > > use stokes_solvers > > > > > > implicit none > > > > > > integer n,i,j > > > real(kind=8) dt,gam > > > integer nsteps > > > type(coord) coords > > > type(coord) proc_coords > > > type(solution) GUXY,LUXY > > > real(kind=8),allocatable, dimension(:):: force,wind > > > integer nelements > > > integer nely,nelx > > > integer twod > > > integer nglobaldofs > > > integer irc,myrank,nprocs, ierr, status(MPI_STATUS_SIZE) > > > > > > call mpi_init(ierr) > > > ! Get myrank > > > call mpi_comm_rank(MPI_COMM_WORLD,myrank,ierr) > > > call mpi_comm_size(MPI_COMM_WORLD,nprocs,ierr) > > > > > > call PetscInitialize( PETSC_NULL_CHARACTER, ierr ) > > > call MPI_Comm_rank( PETSC_COMM_WORLD, myrank, ierr ) > > > if (myrank .eq. 0) then > > > print *, ?Hello World? > > > endif > > > call PetscFinalize(ierr) > > > > > > stop > > > > > > > > > > > > > > > > > > > > > On Mar 2, 2007, at 11:38 AM, Satish Balay wrote: > > > > > > > > > > > > > > > On Fri, 2 Mar 2007, P. Aaron Lott wrote: > > > > > > > > > > > > > > Thanks for this information. I think I setup things almost right, but > > > > > something is causing preprocessor to complain. The errors it is giving > > > > > are > > > > > referring to lines of my main program file that doesn't exist. > > > > > > > > The line numbers refer to the temoprary file created - __unsteady_ex.F > > > > > > > > You can run the following command to preserve the temporary file - and > > > > check it for the correct line number > > > > > > > > make unsteady_ex SKIP_RM=yes > > > > > > > > Preprocessing fortran code with C compiler can have some false > > > > positives. For eg: if you have quotes in comments - then cpp can > > > > misbehave. > > > > > > > > > > > > > > I would appreciate any recommendations about how to fix this problem. > > > > > I've > > > > > list some of the details of my setup below. > > > > > > > > > > Thanks, > > > > > > > > > > -Aaron > > > > > > > > > > > > > > > > > > > > e.g. > > > > > > > > > > macbeth% make unsteady_ex > > > > > /usr/local/mpich-1.2.5.2/bin/mpicc > > > > > -I/Users/palott/research/codes/petsc/petsc-2.3.2-p8 > > > > > -I/Users/palott/research/codes/pets > > > > > c/petsc-2.3.2-p8/bmake/darwin-mpich-g > > > > > -I/Users/palott/research/codes/petsc/petsc-2.3.2-p8/include > > > > > -I/usr/local/mpich-1.2. > > > > > 5.2/include -E -traditional-cpp __unsteady_ex.c | /usr/bin/grep > > > > > -v > > > > > '^ > > > > > *#' > __unsteady_ex.F > > > > > /usr/local/mpich-1.2.5.2/bin/mpif90 -c -I. -g __unsteady_ex.F -o > > > > > unsteady_ex.o > > > > > "__unsteady_ex.F", line 965.2: 1515-019 (S) Syntax is incorrect. > > > > > > > > > > > > > > > But there are only 207 lines in this file..... > > > > > > > > > > > > > > > > > > > > In my current makefile I have > > > > > > > > > > F90=mpif90 > > > > > F77=mpif77 > > > > > > > > These don't need to be specified as PETSc .F.o targets don't use them.. > > > > [and > > > > you don't want to redefine whats already configured inside PETSc] > > > > > > > > > F77FLAGS= -C -g -qdpc -qautodbl=dbl4 -qinitauto -Wl,-framework > > > > > -Wl,vecLib > > > > > > > > This variable is not used by PETSc target > > > > > > > > > FLAGS= -C -g -qdpc -qautodbl=dbl4 -qinitauto -Wl,-framework > > > > > -Wl,vecLib > > > > > > > > PETSc should already use a blas implementation - so no need to specify > > > > vecLib again. > > > > > > > > > > > > > > > > > > > EXENAME = unsteady_ex > > > > > CMD:=EXENAME > > > > > > > > > > SOBJS = mpi.o quicksort.o routines.o binsort.o coordinates.o > > > > > comm_maps.o > > > > > elements.o globaldata.o sem.o comp_inv.o d1ops.o maps.o pardss.o > > > > > dss.o > > > > > stiffness.o convection.o Mass.o pressure.o buildU.o wind.o forcing.o > > > > > boundary_cond.o navier_stokes.o runge_kutta.o out.o gmres.o pcg.o > > > > > stream.o > > > > > uzawa.o print_matrices.o projection.o bdf.o stokes_solvers.o > > > > > initialize_mpi.o unsteady_ex.o > > > > > > > > > > > > > > > $(CMD) : $(SOBJS) > > > > > -${FLINKER} $(FLAGS) -o $(EXENAME) $(SOBJS) $(PETSC_SYS_LIB) > > > > > > > > No need to specify FLAGS here - as the default targets pick it up [in > > > > this > > > > case > > > > the flags get added to FLINKER > > > > > > > > Satish > > > > > > > > > > > > > > > > > > > mpi.o : ../Comm/mpi.f90 ;$(F90) $(FLAGS) -c > > > > > ../Comm/mpi.f90 > > > > > gmres.o : ../Solvers/gmres.f90 ;$(F90) $(FLAGS) -c > > > > > ../Solvers/gmres.f90 > > > > > etc.... for all the object files except unsteady_ex.o > > > > > > > > > > unsteady_ex.F is my main program. > > > > > > > > > > In the example you gave, ex1f.F would be the main program, and at the > > > > > top > > > > > of > > > > > the file it would have the statement: > > > > > > > > > > #include "include/finclude/petsc.h" > > > > > > > > > > So in my case, I've put that in the top of my unsteady_ex.F > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Mar 1, 2007, at 5:56 PM, Satish Balay wrote: > > > > > > > > > > > The idea is to use a PETSc example makefile - and modify it > > > > > > appropriately. > > > > > > > > > > > > And with fortran codes - we require preprocessing [i.e source files > > > > > > that call PETSc routines should be .F] > > > > > > > > > > > > A minimal PETSc makefile is as follows: > > > > > > ------------------------------------------------ > > > > > > CFLAGS = > > > > > > FFLAGS = > > > > > > CPPFLAGS = > > > > > > FPPFLAGS = > > > > > > CLEANFILES = > > > > > > > > > > > > include ${PETSC_DIR}/bmake/common/base > > > > > > > > > > > > ex1f: ex1f.o chkopts > > > > > > -${FLINKER} -o ex1f ex1f.o ${PETSC_KSP_LIB} > > > > > > ${RM} ex1f.o > > > > > > ----------------------------------------------------- > > > > > > > > > > > > So you'll just make the change [with the variables CMD and SOBJS > > > > > > properly > > > > > > defined] > > > > > > > > > > > > ex1f -> $(CMD) > > > > > > ex1f.o -> $(SOBJS) > > > > > > > > > > > > Note: Necessary FFLAGS should already be PETSc config files. Some > > > > > > additional flags can be specified with FFLAGS variable > > > > > > > > > > > > Satish > > > > > > > > > > > > > > > > > > On Thu, 1 Mar 2007, P. Aaron Lott wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > I have a fortran90 code and I would like to use petsc's > > > > > > > preconditioners > > > > > > > inside of it. I'm having trouble setting up a makefile that will > > > > > > > allow > > > > > > > me > > > > > > > include a petsc. It seems the problem is from the use of the > > > > > > > c-preprocessor. > > > > > > > > > > > > > > My old makefile looked something like this: > > > > > > > > > > > > > > $(CMD) : $(SOBJS) > > > > > > > $(F90) $(FLAGS) -o $(EXENAME) $(SOBJS) > > > > > > > > > > > > > > where SOBJ is a list of all of my object files. > > > > > > > > > > > > > > I was hoping to be able to make a petsc object file and include it > > > > > > > in > > > > > > > this > > > > > > > list. Is this possible? Is there a tutorial on how to get petsc > > > > > > > setup > > > > > > > inside > > > > > > > of your existing code? > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > -Aaron > > > > > > > > > > > > > > > > > > > > > > > > > > > > P. Aaron Lott > > > > > > > Ph.D. Candidate > > > > > > > 4239 Computer and Space Sciences Building > > > > > > > University of Maryland > > > > > > > College Park, MD 20742-4015 > > > > > > > > > > > > > > palott at ipst.umd.edu > > > > > > > http://www.lcv.umd.edu/~palott > > > > > > > Office: 301.405.4894 > > > > > > > Fax: 301.314.0827 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > P. Aaron Lott > > > > > Ph.D. Candidate > > > > > 4239 Computer and Space Sciences Building > > > > > University of Maryland > > > > > College Park, MD 20742-4015 > > > > > > > > > > palott at ipst.umd.edu > > > > > http://www.lcv.umd.edu/~palott > > > > > Office: 301.405.4894 > > > > > Fax: 301.314.0827 > > > > > > > > > > > > > > > > > > > > P. Aaron Lott > > > Ph.D. Candidate > > > 4239 Computer and Space Sciences Building > > > University of Maryland > > > College Park, MD 20742-4015 > > > > > > palott at ipst.umd.edu > > > http://www.lcv.umd.edu/~palott > > > Office: 301.405.4894 > > > Fax: 301.314.0827 > > > > > > > > > > P. Aaron Lott > Ph.D. Candidate > 4239 Computer and Space Sciences Building > University of Maryland > College Park, MD 20742-4015 > > palott at ipst.umd.edu > http://www.lcv.umd.edu/~palott > Office: 301.405.4894 > Fax: 301.314.0827 > > From palott at ipst.umd.edu Sat Mar 3 10:52:56 2007 From: palott at ipst.umd.edu (P. Aaron Lott) Date: Sat, 3 Mar 2007 11:52:56 -0500 Subject: need help with makefile In-Reply-To: References: <27727123-941B-41D3-9C11-BEC596A6D638@ipst.umd.edu> <5BC5FB72-7FF4-4A8E-9123-12211E4305A5@ipst.umd.edu> <845FB810-2733-48D2-A255-D09F8DD68330@ipst.umd.edu> <55AF300D-692C-405B-8698-9014A5642819@ipst.umd.edu> Message-ID: <0629CF3F-A8F0-44CB-9352-CC4D038FC890@ipst.umd.edu> Hi Satish, I'm attaching the file to this e-mail. Thanks, -Aaron -------------- next part -------------- A non-text attachment was scrubbed... Name: __unsteady_ex.F Type: application/octet-stream Size: 22954 bytes Desc: not available URL: -------------- next part -------------- On Mar 3, 2007, at 11:29 AM, Satish Balay wrote: > Can you send us the preprocessed __unsteady_ex.F file? > > Satish > > On Sat, 3 Mar 2007, P. Aaron Lott wrote: > >> >> Hi Satish, >> >> You're right the problem is from mpif90, I tried >> >> mpif90 -qfree -c -I. -g __unsteady_ex.F -o unsteady_ex.o >> >> but I'm getting the same error messages. The file unsteady_ex >> compiled fine >> before using petsc and the preprocessor, so I really don't know >> what the >> problem could be. >> >> Any ideas? >> >> -Aaron >> >> >> >> >> >> >> >> On Mar 2, 2007, at 12:16 PM, Satish Balay wrote: >> >>>>>> /usr/local/mpich-1.2.5.2/bin/mpif90 -c -I. -g >>>>>> __unsteady_ex.F -o >>>>>> unsteady_ex.o >>> "__unsteady_ex.F", line 965.2: 1515-019 (S) Syntax is incorrect. >>> <<< >>> >>> The error is from mpif90 - not the preprocessor. Maybe its >>> interpreting >>> the code as fixed form? Try adding '-qfree' to FFLAGS and see if >>> it makes >>> a difference. >>> >>> Satish >>> >>> On Fri, 2 Mar 2007, P. Aaron Lott wrote: >>> >>>> >>>> Thanks Satish, >>>> >>>> I checked the __unsteady_ex.F file. The preprocessor doesn't >>>> seem to like >>>> any >>>> of my fortran code. It starts complaining about the use of the use >>>> statements, >>>> and basically all the lines of code in this file. >>>> >>>> Any ideas? >>>> >>>> -Aaron >>>> >>>> Here are some of the statements the preprocessor complains about: >>>> >>>> use initialize >>>> use coordinates >>>> use maps >>>> use boundaries >>>> use elements >>>> use buildU >>>> use matrix_ops >>>> use output >>>> use d1ops >>>> use globaldata >>>> use dss >>>> use par_dss >>>> use forces >>>> use winds >>>> use stokes_solvers >>>> >>>> implicit none >>>> >>>> integer n,i,j >>>> real(kind=8) dt,gam >>>> integer nsteps >>>> type(coord) coords >>>> type(coord) proc_coords >>>> type(solution) GUXY,LUXY >>>> real(kind=8),allocatable, dimension(:):: force,wind >>>> integer nelements >>>> integer nely,nelx >>>> integer twod >>>> integer nglobaldofs >>>> integer irc,myrank,nprocs, ierr, status(MPI_STATUS_SIZE) >>>> >>>> call mpi_init(ierr) >>>> ! Get myrank >>>> call mpi_comm_rank(MPI_COMM_WORLD,myrank,ierr) >>>> call mpi_comm_size(MPI_COMM_WORLD,nprocs,ierr) >>>> >>>> call PetscInitialize( PETSC_NULL_CHARACTER, ierr ) >>>> call MPI_Comm_rank( PETSC_COMM_WORLD, myrank, ierr ) >>>> if (myrank .eq. 0) then >>>> print *, ?Hello World? >>>> endif >>>> call PetscFinalize(ierr) >>>> >>>> stop >>>> >>>> >>>> >>>> >>>> >>>> >>>> On Mar 2, 2007, at 11:38 AM, Satish Balay wrote: >>>> >>>>> >>>>> >>>>> On Fri, 2 Mar 2007, P. Aaron Lott wrote: >>>>> >>>>>> >>>>>> Thanks for this information. I think I setup things almost >>>>>> right, but >>>>>> something is causing preprocessor to complain. The errors it >>>>>> is giving >>>>>> are >>>>>> referring to lines of my main program file that doesn't exist. >>>>> >>>>> The line numbers refer to the temoprary file created - >>>>> __unsteady_ex.F >>>>> >>>>> You can run the following command to preserve the temporary >>>>> file - and >>>>> check it for the correct line number >>>>> >>>>> make unsteady_ex SKIP_RM=yes >>>>> >>>>> Preprocessing fortran code with C compiler can have some false >>>>> positives. For eg: if you have quotes in comments - then cpp can >>>>> misbehave. >>>>> >>>>>> >>>>>> I would appreciate any recommendations about how to fix this >>>>>> problem. >>>>>> I've >>>>>> list some of the details of my setup below. >>>>>> >>>>>> Thanks, >>>>>> >>>>>> -Aaron >>>>>> >>>>>> >>>>>> >>>>>> e.g. >>>>>> >>>>>> macbeth% make unsteady_ex >>>>>> /usr/local/mpich-1.2.5.2/bin/mpicc >>>>>> -I/Users/palott/research/codes/petsc/petsc-2.3.2-p8 >>>>>> -I/Users/palott/research/codes/pets >>>>>> c/petsc-2.3.2-p8/bmake/darwin-mpich-g >>>>>> -I/Users/palott/research/codes/petsc/petsc-2.3.2-p8/include >>>>>> -I/usr/local/mpich-1.2. >>>>>> 5.2/include -E -traditional-cpp __unsteady_ex.c | /usr/ >>>>>> bin/grep >>>>>> -v >>>>>> '^ >>>>>> *#' > __unsteady_ex.F >>>>>> /usr/local/mpich-1.2.5.2/bin/mpif90 -c -I. -g >>>>>> __unsteady_ex.F -o >>>>>> unsteady_ex.o >>>>>> "__unsteady_ex.F", line 965.2: 1515-019 (S) Syntax is incorrect. >>>>>> >>>>>> >>>>>> But there are only 207 lines in this file..... >>>>>> >>>>>> >>>>>> >>>>>> In my current makefile I have >>>>>> >>>>>> F90=mpif90 >>>>>> F77=mpif77 >>>>> >>>>> These don't need to be specified as PETSc .F.o targets don't >>>>> use them.. >>>>> [and >>>>> you don't want to redefine whats already configured inside PETSc] >>>>> >>>>>> F77FLAGS= -C -g -qdpc -qautodbl=dbl4 -qinitauto -Wl,-framework >>>>>> -Wl,vecLib >>>>> >>>>> This variable is not used by PETSc target >>>>> >>>>>> FLAGS= -C -g -qdpc -qautodbl=dbl4 -qinitauto -Wl,-framework >>>>>> -Wl,vecLib >>>>> >>>>> PETSc should already use a blas implementation - so no need to >>>>> specify >>>>> vecLib again. >>>>> >>>>>> >>>>>> >>>>>> EXENAME = unsteady_ex >>>>>> CMD:=EXENAME >>>>>> >>>>>> SOBJS = mpi.o quicksort.o routines.o binsort.o coordinates.o >>>>>> comm_maps.o >>>>>> elements.o globaldata.o sem.o comp_inv.o d1ops.o maps.o pardss.o >>>>>> dss.o >>>>>> stiffness.o convection.o Mass.o pressure.o buildU.o wind.o >>>>>> forcing.o >>>>>> boundary_cond.o navier_stokes.o runge_kutta.o out.o gmres.o >>>>>> pcg.o >>>>>> stream.o >>>>>> uzawa.o print_matrices.o projection.o bdf.o stokes_solvers.o >>>>>> initialize_mpi.o unsteady_ex.o >>>>>> >>>>>> >>>>>> $(CMD) : $(SOBJS) >>>>>> -${FLINKER} $(FLAGS) -o $(EXENAME) $(SOBJS) $(PETSC_SYS_LIB) >>>>> >>>>> No need to specify FLAGS here - as the default targets pick it >>>>> up [in >>>>> this >>>>> case >>>>> the flags get added to FLINKER >>>>> >>>>> Satish >>>>> >>>>>> >>>>>> >>>>>> mpi.o : ../Comm/mpi.f90 ;$(F90) $(FLAGS) -c >>>>>> ../Comm/mpi.f90 >>>>>> gmres.o : ../Solvers/gmres.f90 ;$(F90) $(FLAGS) -c >>>>>> ../Solvers/gmres.f90 >>>>>> etc.... for all the object files except unsteady_ex.o >>>>>> >>>>>> unsteady_ex.F is my main program. >>>>>> >>>>>> In the example you gave, ex1f.F would be the main program, and >>>>>> at the >>>>>> top >>>>>> of >>>>>> the file it would have the statement: >>>>>> >>>>>> #include "include/finclude/petsc.h" >>>>>> >>>>>> So in my case, I've put that in the top of my unsteady_ex.F >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Mar 1, 2007, at 5:56 PM, Satish Balay wrote: >>>>>> >>>>>>> The idea is to use a PETSc example makefile - and modify it >>>>>>> appropriately. >>>>>>> >>>>>>> And with fortran codes - we require preprocessing [i.e source >>>>>>> files >>>>>>> that call PETSc routines should be .F] >>>>>>> >>>>>>> A minimal PETSc makefile is as follows: >>>>>>> ------------------------------------------------ >>>>>>> CFLAGS = >>>>>>> FFLAGS = >>>>>>> CPPFLAGS = >>>>>>> FPPFLAGS = >>>>>>> CLEANFILES = >>>>>>> >>>>>>> include ${PETSC_DIR}/bmake/common/base >>>>>>> >>>>>>> ex1f: ex1f.o chkopts >>>>>>> -${FLINKER} -o ex1f ex1f.o ${PETSC_KSP_LIB} >>>>>>> ${RM} ex1f.o >>>>>>> ----------------------------------------------------- >>>>>>> >>>>>>> So you'll just make the change [with the variables CMD and SOBJS >>>>>>> properly >>>>>>> defined] >>>>>>> >>>>>>> ex1f -> $(CMD) >>>>>>> ex1f.o -> $(SOBJS) >>>>>>> >>>>>>> Note: Necessary FFLAGS should already be PETSc config files. >>>>>>> Some >>>>>>> additional flags can be specified with FFLAGS variable >>>>>>> >>>>>>> Satish >>>>>>> >>>>>>> >>>>>>> On Thu, 1 Mar 2007, P. Aaron Lott wrote: >>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> I have a fortran90 code and I would like to use petsc's >>>>>>>> preconditioners >>>>>>>> inside of it. I'm having trouble setting up a makefile that >>>>>>>> will >>>>>>>> allow >>>>>>>> me >>>>>>>> include a petsc. It seems the problem is from the use of the >>>>>>>> c-preprocessor. >>>>>>>> >>>>>>>> My old makefile looked something like this: >>>>>>>> >>>>>>>> $(CMD) : $(SOBJS) >>>>>>>> $(F90) $(FLAGS) -o $(EXENAME) $(SOBJS) >>>>>>>> >>>>>>>> where SOBJ is a list of all of my object files. >>>>>>>> >>>>>>>> I was hoping to be able to make a petsc object file and >>>>>>>> include it >>>>>>>> in >>>>>>>> this >>>>>>>> list. Is this possible? Is there a tutorial on how to get petsc >>>>>>>> setup >>>>>>>> inside >>>>>>>> of your existing code? >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> -Aaron >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> P. Aaron Lott >>>>>>>> Ph.D. Candidate >>>>>>>> 4239 Computer and Space Sciences Building >>>>>>>> University of Maryland >>>>>>>> College Park, MD 20742-4015 >>>>>>>> >>>>>>>> palott at ipst.umd.edu >>>>>>>> http://www.lcv.umd.edu/~palott >>>>>>>> Office: 301.405.4894 >>>>>>>> Fax: 301.314.0827 >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>>> P. Aaron Lott >>>>>> Ph.D. Candidate >>>>>> 4239 Computer and Space Sciences Building >>>>>> University of Maryland >>>>>> College Park, MD 20742-4015 >>>>>> >>>>>> palott at ipst.umd.edu >>>>>> http://www.lcv.umd.edu/~palott >>>>>> Office: 301.405.4894 >>>>>> Fax: 301.314.0827 >>>>>> >>>>>> >>>>> >>>> >>>> P. Aaron Lott >>>> Ph.D. Candidate >>>> 4239 Computer and Space Sciences Building >>>> University of Maryland >>>> College Park, MD 20742-4015 >>>> >>>> palott at ipst.umd.edu >>>> http://www.lcv.umd.edu/~palott >>>> Office: 301.405.4894 >>>> Fax: 301.314.0827 >>>> >>>> >>> >> >> P. Aaron Lott >> Ph.D. Candidate >> 4239 Computer and Space Sciences Building >> University of Maryland >> College Park, MD 20742-4015 >> >> palott at ipst.umd.edu >> http://www.lcv.umd.edu/~palott >> Office: 301.405.4894 >> Fax: 301.314.0827 >> >> > P. Aaron Lott Ph.D. Candidate 4239 Computer and Space Sciences Building University of Maryland College Park, MD 20742-4015 palott at ipst.umd.edu http://www.lcv.umd.edu/~palott Office: 301.405.4894 Fax: 301.314.0827 From balay at mcs.anl.gov Sat Mar 3 11:07:34 2007 From: balay at mcs.anl.gov (Satish Balay) Date: Sat, 3 Mar 2007 11:07:34 -0600 (CST) Subject: need help with makefile In-Reply-To: <0629CF3F-A8F0-44CB-9352-CC4D038FC890@ipst.umd.edu> References: <27727123-941B-41D3-9C11-BEC596A6D638@ipst.umd.edu> <5BC5FB72-7FF4-4A8E-9123-12211E4305A5@ipst.umd.edu> <845FB810-2733-48D2-A255-D09F8DD68330@ipst.umd.edu> <55AF300D-692C-405B-8698-9014A5642819@ipst.umd.edu> <0629CF3F-A8F0-44CB-9352-CC4D038FC890@ipst.umd.edu> Message-ID: I can't spot any obvious issues here.. Is it possible that the #include should go after the 'use statements'? [I don't know enough about f90 issues here] Satish On Sat, 3 Mar 2007, P. Aaron Lott wrote: > > Hi Satish, > > I'm attaching the file to this e-mail. > > Thanks, > > -Aaron > > > From palott at ipst.umd.edu Sat Mar 3 11:34:14 2007 From: palott at ipst.umd.edu (P. Aaron Lott) Date: Sat, 3 Mar 2007 12:34:14 -0500 Subject: need help with makefile In-Reply-To: References: <27727123-941B-41D3-9C11-BEC596A6D638@ipst.umd.edu> <5BC5FB72-7FF4-4A8E-9123-12211E4305A5@ipst.umd.edu> <845FB810-2733-48D2-A255-D09F8DD68330@ipst.umd.edu> <55AF300D-692C-405B-8698-9014A5642819@ipst.umd.edu> <0629CF3F-A8F0-44CB-9352-CC4D038FC890@ipst.umd.edu> Message-ID: <8CF86E6F-AF32-4E2F-A92C-B093026C4088@ipst.umd.edu> Hi Satish, I tried the order of the use statements and the include, but it didn't seem to matter. I really don't know much about make files or the linking process with compilers but it seems like the compiler statement /usr/local/mpich-1.2.5.2/bin/mpif90 -c -I. -g __unsteady_ex.F -o unsteady_ex.o is trying to make an object file unsteady_ex.o However, unsteady_ex.F depends on many other object files before this can be compiled. During the make process the first file dealt with is unsteady_ex.F, which before using petsc, unsteady_ex.f90 was the last to be compiled. It seems like there should be some way to compile the rest of my program first and then perform this step at the end. I'm not sure if there's some way to rearrange things in the makefile command in order to do this or not. Do you have any ideas? Thanks, -Aaron $(CMD) : $(SOBJS) -${FLINKER} $(FLAGS) -o $(EXENAME) $(SOBJS) On Mar 3, 2007, at 12:07 PM, Satish Balay wrote: > I can't spot any obvious issues here.. Is it possible that the > #include should go after the 'use statements'? [I don't know enough > about f90 issues here] > > Satish > > On Sat, 3 Mar 2007, P. Aaron Lott wrote: > >> >> Hi Satish, >> >> I'm attaching the file to this e-mail. >> >> Thanks, >> >> -Aaron >> >> >> > P. Aaron Lott Ph.D. Candidate 4239 Computer and Space Sciences Building University of Maryland College Park, MD 20742-4015 palott at ipst.umd.edu http://www.lcv.umd.edu/~palott Office: 301.405.4894 Fax: 301.314.0827 From balay at mcs.anl.gov Sat Mar 3 11:36:41 2007 From: balay at mcs.anl.gov (Satish Balay) Date: Sat, 3 Mar 2007 11:36:41 -0600 (CST) Subject: need help with makefile In-Reply-To: <8CF86E6F-AF32-4E2F-A92C-B093026C4088@ipst.umd.edu> References: <27727123-941B-41D3-9C11-BEC596A6D638@ipst.umd.edu> <5BC5FB72-7FF4-4A8E-9123-12211E4305A5@ipst.umd.edu> <845FB810-2733-48D2-A255-D09F8DD68330@ipst.umd.edu> <55AF300D-692C-405B-8698-9014A5642819@ipst.umd.edu> <0629CF3F-A8F0-44CB-9352-CC4D038FC890@ipst.umd.edu> <8CF86E6F-AF32-4E2F-A92C-B093026C4088@ipst.umd.edu> Message-ID: What is the command that correctly compiles this file? [from your original makefile] Satish On Sat, 3 Mar 2007, P. Aaron Lott wrote: > > Hi Satish, > > I tried the order of the use statements and the include, but it didn't seem to > matter. I really don't know much about make files or the linking process with > compilers but it seems like the compiler statement > > /usr/local/mpich-1.2.5.2/bin/mpif90 -c -I. -g __unsteady_ex.F -o > unsteady_ex.o > > is trying to make an object file unsteady_ex.o > > However, unsteady_ex.F depends on many other object files before this can be > compiled. During the make process the first file dealt with is unsteady_ex.F, > which before using petsc, unsteady_ex.f90 was the last to be compiled. It > seems like there should be some way to compile the rest of my program first > and then perform this step at the end. > > > I'm not sure if there's some way to rearrange things in the makefile command > in order to do this or not. Do you have any ideas? > > Thanks, > > -Aaron > > > > $(CMD) : $(SOBJS) > -${FLINKER} $(FLAGS) -o $(EXENAME) $(SOBJS) > > > > > > On Mar 3, 2007, at 12:07 PM, Satish Balay wrote: > > > I can't spot any obvious issues here.. Is it possible that the > > #include should go after the 'use statements'? [I don't know enough > > about f90 issues here] > > > > Satish > > > > On Sat, 3 Mar 2007, P. Aaron Lott wrote: > > > > > > > > Hi Satish, > > > > > > I'm attaching the file to this e-mail. > > > > > > Thanks, > > > > > > -Aaron > > > > > > > > > > > > > P. Aaron Lott > Ph.D. Candidate > 4239 Computer and Space Sciences Building > University of Maryland > College Park, MD 20742-4015 > > palott at ipst.umd.edu > http://www.lcv.umd.edu/~palott > Office: 301.405.4894 > Fax: 301.314.0827 > > From palott at ipst.umd.edu Sat Mar 3 12:32:24 2007 From: palott at ipst.umd.edu (P. Aaron Lott) Date: Sat, 3 Mar 2007 13:32:24 -0500 Subject: need help with makefile In-Reply-To: References: <27727123-941B-41D3-9C11-BEC596A6D638@ipst.umd.edu> <5BC5FB72-7FF4-4A8E-9123-12211E4305A5@ipst.umd.edu> <845FB810-2733-48D2-A255-D09F8DD68330@ipst.umd.edu> <55AF300D-692C-405B-8698-9014A5642819@ipst.umd.edu> <0629CF3F-A8F0-44CB-9352-CC4D038FC890@ipst.umd.edu> <8CF86E6F-AF32-4E2F-A92C-B093026C4088@ipst.umd.edu> Message-ID: All of the source files are first compiled using mpif90 -C -g -qdpc -qautodbl=dbl4 -qinitauto -Wl,-framework - Wl,vecLib -c file.f90 On each file, this creates the appropriate .o and .mod files. Also, these are compiled in order such that if a file B depends on a module defined in another file A, then A would be compiled before B Then the executable is created by running mpif90 -C -g -qdpc -qautodbl=dbl4 -qinitauto -Wl,-framework - Wl,vecLib -o executable all_object_files.o I'm attaching the Makefile here -------------- next part -------------- A non-text attachment was scrubbed... Name: Makefile_no_petsc Type: application/octet-stream Size: 3558 bytes Desc: not available URL: -------------- next part -------------- -Aaron On Mar 3, 2007, at 12:36 PM, Satish Balay wrote: > What is the command that correctly compiles this file? [from your > original makefile] > > Satish > > On Sat, 3 Mar 2007, P. Aaron Lott wrote: > >> >> Hi Satish, >> >> I tried the order of the use statements and the include, but it >> didn't seem to >> matter. I really don't know much about make files or the linking >> process with >> compilers but it seems like the compiler statement >> >> /usr/local/mpich-1.2.5.2/bin/mpif90 -c -I. -g __unsteady_ex.F -o >> unsteady_ex.o >> >> is trying to make an object file unsteady_ex.o >> >> However, unsteady_ex.F depends on many other object files before >> this can be >> compiled. During the make process the first file dealt with is >> unsteady_ex.F, >> which before using petsc, unsteady_ex.f90 was the last to be >> compiled. It >> seems like there should be some way to compile the rest of my >> program first >> and then perform this step at the end. >> >> >> I'm not sure if there's some way to rearrange things in the >> makefile command >> in order to do this or not. Do you have any ideas? >> >> Thanks, >> >> -Aaron >> >> >> >> $(CMD) : $(SOBJS) >> -${FLINKER} $(FLAGS) -o $(EXENAME) $(SOBJS) >> >> >> >> >> >> On Mar 3, 2007, at 12:07 PM, Satish Balay wrote: >> >>> I can't spot any obvious issues here.. Is it possible that the >>> #include should go after the 'use statements'? [I don't know enough >>> about f90 issues here] >>> >>> Satish >>> >>> On Sat, 3 Mar 2007, P. Aaron Lott wrote: >>> >>>> >>>> Hi Satish, >>>> >>>> I'm attaching the file to this e-mail. >>>> >>>> Thanks, >>>> >>>> -Aaron >>>> >>>> >>>> >>> >> >> P. Aaron Lott >> Ph.D. Candidate >> 4239 Computer and Space Sciences Building >> University of Maryland >> College Park, MD 20742-4015 >> >> palott at ipst.umd.edu >> http://www.lcv.umd.edu/~palott >> Office: 301.405.4894 >> Fax: 301.314.0827 >> >> > P. Aaron Lott Ph.D. Candidate 4239 Computer and Space Sciences Building University of Maryland College Park, MD 20742-4015 palott at ipst.umd.edu http://www.lcv.umd.edu/~palott Office: 301.405.4894 Fax: 301.314.0827 From pbauman at ices.utexas.edu Sat Mar 3 13:43:05 2007 From: pbauman at ices.utexas.edu (Paul T. Bauman) Date: Sat, 03 Mar 2007 13:43:05 -0600 Subject: need help with makefile In-Reply-To: References: <27727123-941B-41D3-9C11-BEC596A6D638@ipst.umd.edu> <5BC5FB72-7FF4-4A8E-9123-12211E4305A5@ipst.umd.edu> <845FB810-2733-48D2-A255-D09F8DD68330@ipst.umd.edu> <55AF300D-692C-405B-8698-9014A5642819@ipst.umd.edu> <0629CF3F-A8F0-44CB-9352-CC4D038FC890@ipst.umd.edu> <8CF86E6F-AF32-4E2F-A92C-B093026C4088@ipst.umd.edu> Message-ID: <45E9CFC9.7060407@ices.utexas.edu> Aaron, Don't want to be rude and interject, but I looked at the unsteady.F file - I've always had trouble compiling when 'use statements' come after variable declarations. Give a try putting things in this order: program use modules implicit none include statements variable declarations code end program This always works for me, not matter what compiler. Also, since you've named the file .F, you MUST have -qfree (or the correct statement, intel compilers it's -fpp) otherwise you will get errors since, by default, .F will use f77 fixed format and you are using free format, although, now that I think about, some compilers will override if you use mpif90 version, but I've had mixed luck with that. If you name the files .f90, they will be compiled in free format by default (similar for .F90, but now with pre processing). Again, just my two cents in trying to help. Paul P. Aaron Lott wrote: > > All of the source files are first compiled using > > mpif90 -C -g -qdpc -qautodbl=dbl4 -qinitauto -Wl,-framework > -Wl,vecLib -c file.f90 > > On each file, this creates the appropriate .o and .mod files. Also, > these are compiled in order such that if a file B depends on a module > defined in another file A, then A would be compiled before B > > Then the executable is created by running > > mpif90 -C -g -qdpc -qautodbl=dbl4 -qinitauto -Wl,-framework > -Wl,vecLib -o executable all_object_files.o > > > I'm attaching the Makefile here > > > > > -Aaron > > > > > On Mar 3, 2007, at 12:36 PM, Satish Balay wrote: > >> What is the command that correctly compiles this file? [from your >> original makefile] >> >> Satish >> >> On Sat, 3 Mar 2007, P. Aaron Lott wrote: >> >>> >>> Hi Satish, >>> >>> I tried the order of the use statements and the include, but it >>> didn't seem to >>> matter. I really don't know much about make files or the linking >>> process with >>> compilers but it seems like the compiler statement >>> >>> /usr/local/mpich-1.2.5.2/bin/mpif90 -c -I. -g __unsteady_ex.F -o >>> unsteady_ex.o >>> >>> is trying to make an object file unsteady_ex.o >>> >>> However, unsteady_ex.F depends on many other object files before >>> this can be >>> compiled. During the make process the first file dealt with is >>> unsteady_ex.F, >>> which before using petsc, unsteady_ex.f90 was the last to be >>> compiled. It >>> seems like there should be some way to compile the rest of my >>> program first >>> and then perform this step at the end. >>> >>> >>> I'm not sure if there's some way to rearrange things in the makefile >>> command >>> in order to do this or not. Do you have any ideas? >>> >>> Thanks, >>> >>> -Aaron >>> >>> >>> >>> $(CMD) : $(SOBJS) >>> -${FLINKER} $(FLAGS) -o $(EXENAME) $(SOBJS) >>> >>> >>> >>> >>> >>> On Mar 3, 2007, at 12:07 PM, Satish Balay wrote: >>> >>>> I can't spot any obvious issues here.. Is it possible that the >>>> #include should go after the 'use statements'? [I don't know enough >>>> about f90 issues here] >>>> >>>> Satish >>>> >>>> On Sat, 3 Mar 2007, P. Aaron Lott wrote: >>>> >>>>> >>>>> Hi Satish, >>>>> >>>>> I'm attaching the file to this e-mail. >>>>> >>>>> Thanks, >>>>> >>>>> -Aaron >>>>> >>>>> >>>>> >>>> >>> >>> P. Aaron Lott >>> Ph.D. Candidate >>> 4239 Computer and Space Sciences Building >>> University of Maryland >>> College Park, MD 20742-4015 >>> >>> palott at ipst.umd.edu >>> http://www.lcv.umd.edu/~palott >>> Office: 301.405.4894 >>> Fax: 301.314.0827 >>> >>> >> > > P. Aaron Lott > Ph.D. Candidate > 4239 Computer and Space Sciences Building > University of Maryland > College Park, MD 20742-4015 > > palott at ipst.umd.edu > http://www.lcv.umd.edu/~palott > Office: 301.405.4894 > Fax: 301.314.0827 > > From palott at ipst.umd.edu Sat Mar 3 14:33:27 2007 From: palott at ipst.umd.edu (P. Aaron Lott) Date: Sat, 3 Mar 2007 15:33:27 -0500 Subject: need help with makefile In-Reply-To: <45E9CFC9.7060407@ices.utexas.edu> References: <27727123-941B-41D3-9C11-BEC596A6D638@ipst.umd.edu> <5BC5FB72-7FF4-4A8E-9123-12211E4305A5@ipst.umd.edu> <845FB810-2733-48D2-A255-D09F8DD68330@ipst.umd.edu> <55AF300D-692C-405B-8698-9014A5642819@ipst.umd.edu> <0629CF3F-A8F0-44CB-9352-CC4D038FC890@ipst.umd.edu> <8CF86E6F-AF32-4E2F-A92C-B093026C4088@ipst.umd.edu> <45E9CFC9.7060407@ices.utexas.edu> Message-ID: <83CAB50D-B521-426C-BADE-82CE34C83B8C@ipst.umd.edu> Hi Paul & Satish, This order works. Thank you so much for your help. I really appreciate it. -Aaron On Mar 3, 2007, at 2:43 PM, Paul T. Bauman wrote: > Aaron, > > Don't want to be rude and interject, but I looked at the unsteady.F > file - I've always had trouble compiling when 'use statements' come > after variable declarations. Give a try putting things in this order: > > program > > use modules > > implicit none > > include statements > > variable declarations > > code > > end program > > This always works for me, not matter what compiler. Also, since > you've named the file .F, you MUST have -qfree (or the correct > statement, intel compilers it's -fpp) otherwise you will get errors > since, by default, .F will use f77 fixed format and you are using > free format, although, now that I think about, some compilers will > override if you use mpif90 version, but I've had mixed luck with > that. If you name the files .f90, they will be compiled in free > format by default (similar for .F90, but now with pre processing). > > Again, just my two cents in trying to help. > > Paul > > P. Aaron Lott wrote: >> >> All of the source files are first compiled using >> >> mpif90 -C -g -qdpc -qautodbl=dbl4 -qinitauto -Wl,-framework - >> Wl,vecLib -c file.f90 >> >> On each file, this creates the appropriate .o and .mod files. >> Also, these are compiled in order such that if a file B depends on >> a module defined in another file A, then A would be compiled before B >> >> Then the executable is created by running >> >> mpif90 -C -g -qdpc -qautodbl=dbl4 -qinitauto -Wl,-framework - >> Wl,vecLib -o executable all_object_files.o >> >> >> I'm attaching the Makefile here >> >> >> >> >> -Aaron >> >> >> >> >> On Mar 3, 2007, at 12:36 PM, Satish Balay wrote: >> >>> What is the command that correctly compiles this file? [from your >>> original makefile] >>> >>> Satish >>> >>> On Sat, 3 Mar 2007, P. Aaron Lott wrote: >>> >>>> >>>> Hi Satish, >>>> >>>> I tried the order of the use statements and the include, but it >>>> didn't seem to >>>> matter. I really don't know much about make files or the linking >>>> process with >>>> compilers but it seems like the compiler statement >>>> >>>> /usr/local/mpich-1.2.5.2/bin/mpif90 -c -I. -g __unsteady_ex.F -o >>>> unsteady_ex.o >>>> >>>> is trying to make an object file unsteady_ex.o >>>> >>>> However, unsteady_ex.F depends on many other object files before >>>> this can be >>>> compiled. During the make process the first file dealt with is >>>> unsteady_ex.F, >>>> which before using petsc, unsteady_ex.f90 was the last to be >>>> compiled. It >>>> seems like there should be some way to compile the rest of my >>>> program first >>>> and then perform this step at the end. >>>> >>>> >>>> I'm not sure if there's some way to rearrange things in the >>>> makefile command >>>> in order to do this or not. Do you have any ideas? >>>> >>>> Thanks, >>>> >>>> -Aaron >>>> >>>> >>>> >>>> $(CMD) : $(SOBJS) >>>> -${FLINKER} $(FLAGS) -o $(EXENAME) $(SOBJS) >>>> >>>> >>>> >>>> >>>> >>>> On Mar 3, 2007, at 12:07 PM, Satish Balay wrote: >>>> >>>>> I can't spot any obvious issues here.. Is it possible that the >>>>> #include should go after the 'use statements'? [I don't know >>>>> enough >>>>> about f90 issues here] >>>>> >>>>> Satish >>>>> >>>>> On Sat, 3 Mar 2007, P. Aaron Lott wrote: >>>>> >>>>>> >>>>>> Hi Satish, >>>>>> >>>>>> I'm attaching the file to this e-mail. >>>>>> >>>>>> Thanks, >>>>>> >>>>>> -Aaron >>>>>> >>>>>> >>>>>> >>>>> >>>> >>>> P. Aaron Lott >>>> Ph.D. Candidate >>>> 4239 Computer and Space Sciences Building >>>> University of Maryland >>>> College Park, MD 20742-4015 >>>> >>>> palott at ipst.umd.edu >>>> http://www.lcv.umd.edu/~palott >>>> Office: 301.405.4894 >>>> Fax: 301.314.0827 >>>> >>>> >>> >> >> P. Aaron Lott >> Ph.D. Candidate >> 4239 Computer and Space Sciences Building >> University of Maryland >> College Park, MD 20742-4015 >> >> palott at ipst.umd.edu >> http://www.lcv.umd.edu/~palott >> Office: 301.405.4894 >> Fax: 301.314.0827 >> >> > P. Aaron Lott Ph.D. Candidate 4239 Computer and Space Sciences Building University of Maryland College Park, MD 20742-4015 palott at ipst.umd.edu http://www.lcv.umd.edu/~palott Office: 301.405.4894 Fax: 301.314.0827 From li76pan at yahoo.com Sun Mar 4 03:11:15 2007 From: li76pan at yahoo.com (li pan) Date: Sun, 4 Mar 2007 01:11:15 -0800 (PST) Subject: about Unstructured Meshes In-Reply-To: Message-ID: <999643.61155.qm@web36815.mail.mud.yahoo.com> hi Matt, have you ever evaluated Tetgen? How do you think about the quality of the 3D mesh it generated? see u pan --- Matthew Knepley wrote: > On 3/2/07, Jianing Shi wrote: > > Well, let me rephrase my question. So the mesh > support in PETSc > > already includes the functionality of partitioning > meshes, I guess, > > using ParMetis, is that the case? Something that > an end user need to > > worry about is really how to generate a mesh that > is tailored towards > > his/her application. > > It can use a range of partioners, like Chaco for > instance. > > > I am trying to write a library on top of PETSc > meshes that will > > generate meshes according to some neurophysiology. > I would like to > > know what are the mesh generate softwares out > there that will > > interface nicely with PETSc, or if it makes sense > for me to write my > > own? Just would like to understand more about the > data structure in > > the PETSc ALE::Mesh classes. Is there any > tutorial out there apart > > from looking at the source code in the mesh > directory? > > 1) No, it makes no sense for you to write a mesh > generator > > 2) In 2D, Triangle. In 3d, the only free things are > TetGen and Netgen. I support > TetGen. Hopefully, CMU will release its MG soon. > > There is a tutorial on the website. > > > I am currently using the petsc-2.3.2-p3. Is there > any new > > functionality about meshes in the development > version? > > All the working stuff is in petsc-dev. > > Matt > > > Jianing > -- > One trouble is that despite this system, anyone who > reads journals widely > and critically is forced to realize that there are > scarcely any bars to eventual > publication. There seems to be no study too > fragmented, no hypothesis too > trivial, no literature citation too biased or too > egotistical, no design too > warped, no methodology too bungled, no presentation > of results too > inaccurate, too obscure, and too contradictory, no > analysis too self-serving, > no argument too circular, no conclusions too > trifling or too unjustified, and > no grammar and syntax too offensive for a paper to > end up in print. -- > Drummond Rennie > > ____________________________________________________________________________________ Do you Yahoo!? Everyone is raving about the all-new Yahoo! Mail beta. http://new.mail.yahoo.com From knepley at gmail.com Sun Mar 4 08:38:23 2007 From: knepley at gmail.com (Matthew Knepley) Date: Sun, 4 Mar 2007 08:38:23 -0600 Subject: about Unstructured Meshes In-Reply-To: <999643.61155.qm@web36815.mail.mud.yahoo.com> References: <999643.61155.qm@web36815.mail.mud.yahoo.com> Message-ID: On 3/4/07, li pan wrote: > hi Matt, > have you ever evaluated Tetgen? How do you think about > the quality of the 3D mesh it generated? I have heard anecdotes about failures for complicated geometry, however I have never had a problem. The quality is pretty good. Gary Miller has shown that the number of tets generated is way too high, but I know of no better implementation. Matt > see u > > pan > > > --- Matthew Knepley wrote: > > > On 3/2/07, Jianing Shi wrote: > > > Well, let me rephrase my question. So the mesh > > support in PETSc > > > already includes the functionality of partitioning > > meshes, I guess, > > > using ParMetis, is that the case? Something that > > an end user need to > > > worry about is really how to generate a mesh that > > is tailored towards > > > his/her application. > > > > It can use a range of partioners, like Chaco for > > instance. > > > > > I am trying to write a library on top of PETSc > > meshes that will > > > generate meshes according to some neurophysiology. > > I would like to > > > know what are the mesh generate softwares out > > there that will > > > interface nicely with PETSc, or if it makes sense > > for me to write my > > > own? Just would like to understand more about the > > data structure in > > > the PETSc ALE::Mesh classes. Is there any > > tutorial out there apart > > > from looking at the source code in the mesh > > directory? > > > > 1) No, it makes no sense for you to write a mesh > > generator > > > > 2) In 2D, Triangle. In 3d, the only free things are > > TetGen and Netgen. I support > > TetGen. Hopefully, CMU will release its MG soon. > > > > There is a tutorial on the website. > > > > > I am currently using the petsc-2.3.2-p3. Is there > > any new > > > functionality about meshes in the development > > version? > > > > All the working stuff is in petsc-dev. > > > > Matt > > > > > Jianing > > -- > > One trouble is that despite this system, anyone who > > reads journals widely > > and critically is forced to realize that there are > > scarcely any bars to eventual > > publication. There seems to be no study too > > fragmented, no hypothesis too > > trivial, no literature citation too biased or too > > egotistical, no design too > > warped, no methodology too bungled, no presentation > > of results too > > inaccurate, too obscure, and too contradictory, no > > analysis too self-serving, > > no argument too circular, no conclusions too > > trifling or too unjustified, and > > no grammar and syntax too offensive for a paper to > > end up in print. -- > > Drummond Rennie > > > > > > > > > ____________________________________________________________________________________ > Do you Yahoo!? > Everyone is raving about the all-new Yahoo! Mail beta. > http://new.mail.yahoo.com > > -- One trouble is that despite this system, anyone who reads journals widely and critically is forced to realize that there are scarcely any bars to eventual publication. There seems to be no study too fragmented, no hypothesis too trivial, no literature citation too biased or too egotistical, no design too warped, no methodology too bungled, no presentation of results too inaccurate, too obscure, and too contradictory, no analysis too self-serving, no argument too circular, no conclusions too trifling or too unjustified, and no grammar and syntax too offensive for a paper to end up in print. -- Drummond Rennie From palott at ipst.umd.edu Sun Mar 4 14:25:39 2007 From: palott at ipst.umd.edu (P. Aaron Lott) Date: Sun, 4 Mar 2007 15:25:39 -0500 Subject: matrix-free additive schwarz? Message-ID: Hi, I'm new to petsc, and I've been reading through the petsc manual today trying to figure out if petsc provides a matrix-free implementation of a Newton-Krylov-Schwarz solver. It looks like in terms of the Newton-Krylov part, everything can be done in a matrix- free setting, but I can't tell if there are routines available to allow for a matrix-free Additive Schwarz preconditioner, and if so, what needs to be provided by the user to implement it. I'm very keen on using petsc to provide an additive schwarz preconditioner for my spectral element CFD code, but I need a matrix-free implementation because I don't have access to the global system matrix. Does anyone have experience with applying matrix-free preconditioners in petsc? Thanks, -Aaron P. Aaron Lott Ph.D. Candidate 4239 Computer and Space Sciences Building University of Maryland College Park, MD 20742-4015 palott at ipst.umd.edu http://www.lcv.umd.edu/~palott Office: 301.405.4894 Fax: 301.314.0827 From palott at ipst.umd.edu Sun Mar 4 15:44:18 2007 From: palott at ipst.umd.edu (P. Aaron Lott) Date: Sun, 4 Mar 2007 16:44:18 -0500 Subject: matrix-free additive schwarz? In-Reply-To: References: Message-ID: Hi Matt, I hope I can explain a bit better what it is I'm looking for, and also try to understand what I would need to provide to petsc in order to use its ASM preconditioner. It seems that in essence, for a matrix- free ASM implementation, I would need to provide either the restriction operators for each subdomain, or the assembled system matrix so that petsc's routines could determine the restriction operators. Currently, I have a Newton-Krylov spectral element code for solving the 2D incompressible Navier-Stokes equations. I perform all my matrix-vector products in a matrix-free framework. I now need a preconditioner for the subsidiary Advection-Diffusion equation solve, and I'm wanting to use ASM. I'm hoping to apply this preconditioner in a matrix free setting as well. My code applies matrix-vector products element-wise, and then performs an assembly (gather-scatter) procedure to get the global solution. If all the subdomains for ASM are extensions of the elements from the spectral element discretization, then challenge is to construct the the restriction operators for each element to produce the ASM subdomains. From your message, it seems like petsc isn't able to construct these restriction operators unless I provide an assembled global system matrix. If this is what petsc needs, I could compute the assembled system matrix (although storing this requires a lot of memory), and pass it to petsc. I thought though, that it may be possible to compute the restriction operators without accessing the assembled system, but rather through local-global maps providing indirect addressing. If someone has done this before, in a completely matrix-free setting, the same code could be used, or modified to obtain the restriction operators for each subdomain. I would be particularly interested in the second option if it is available either in petsc, or some package that links with petsc. I suppose the third option is, as you mentioned, providing my own preconditioner with PCSHELL, but my original reason for wanting to use petsc, was for it to provide me with the preconditioner... Thanks, -Aaron On Mar 4, 2007, at 3:41 PM, Matthew Knepley wrote: > On 3/4/07, P. Aaron Lott wrote: >> Hi, >> >> I'm new to petsc, and I've been reading through the petsc manual >> today trying to figure out if petsc provides a matrix-free >> implementation of a Newton-Krylov-Schwarz solver. It looks like in >> terms of the Newton-Krylov part, everything can be done in a matrix- >> free setting, but I can't tell if there are routines available to >> allow for a matrix-free Additive Schwarz preconditioner, and if so, >> what needs to be provided by the user to implement it. I'm very keen >> on using petsc to provide an additive schwarz preconditioner for my >> spectral element CFD code, but I need a matrix-free implementation >> because I don't have access to the global system matrix. Does anyone >> have experience with applying matrix-free preconditioners in petsc? > > Its not quite clear what you want. You can, of course, apply any > preconditioner > you want with PCSHELL. We do have an ASM preconditioner, however > the extra > support this provides is to > > 1) automatically figure out the overlapping domains using the matrix > nonzero structure > > 2) solves the equation restricted to these overlapping partitions > > The first job cannot be done without a structure of some sort, > which we do not > have in the matrix-free case, and the second depends on the first. > > Matt > >> Thanks, >> >> -Aaron >> >> >> >> P. Aaron Lott >> Ph.D. Candidate >> 4239 Computer and Space Sciences Building >> University of Maryland >> College Park, MD 20742-4015 >> >> palott at ipst.umd.edu >> http://www.lcv.umd.edu/~palott >> Office: 301.405.4894 >> Fax: 301.314.0827 >> >> >> > > > -- > One trouble is that despite this system, anyone who reads journals > widely > and critically is forced to realize that there are scarcely any > bars to eventual > publication. There seems to be no study too fragmented, no > hypothesis too > trivial, no literature citation too biased or too egotistical, no > design too > warped, no methodology too bungled, no presentation of results too > inaccurate, too obscure, and too contradictory, no analysis too > self-serving, > no argument too circular, no conclusions too trifling or too > unjustified, and > no grammar and syntax too offensive for a paper to end up in print. -- > Drummond Rennie P. Aaron Lott Ph.D. Candidate 4239 Computer and Space Sciences Building University of Maryland College Park, MD 20742-4015 palott at ipst.umd.edu http://www.lcv.umd.edu/~palott Office: 301.405.4894 Fax: 301.314.0827 From zonexo at gmail.com Sun Mar 4 18:30:35 2007 From: zonexo at gmail.com (Ben Tay) Date: Mon, 5 Mar 2007 08:30:35 +0800 Subject: How to configure Compaq visual fortran for PETSc w/o mpi Message-ID: <804ab5d40703041630r1581749dj345c74320a858c17@mail.gmail.com> Hi, I've managed to use PETSc in Compaq visual fortran using --download-blas-lapack=1 with mpi installed. Now I tried to recompile PETSc with --with-mpi=0. I then tried to remove mpich.lib from the linking options. compiling is ok but during linking, I got these error msg: linking... global.obj : error LNK2001: unresolved external symbol _MPI_DUP_FN petsc_sub.obj : error LNK2001: unresolved external symbol _MPI_DUP_FN global.obj : error LNK2001: unresolved external symbol _MPI_NULL_DELETE_FN petsc_sub.obj : error LNK2001: unresolved external symbol _MPI_NULL_DELETE_FN global.obj : error LNK2001: unresolved external symbol _MPI_NULL_COPY_FN petsc_sub.obj : error LNK2001: unresolved external symbol _MPI_NULL_COPY_FN libpetsc.lib(eventLog.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetsc.lib(dmouse.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetsc.lib(petscvu.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetsc.lib(pdisplay.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetsc.lib(fretrieve.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetsc.lib(mpiu.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetsc.lib(mpiuopen.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetsc.lib(dtri.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetsc.lib(sseenabled.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetsc.lib(random.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetsc.lib(axis.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetsc.lib(mtr.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetsc.lib(sysio.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetsc.lib(psplit.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetsc.lib(mpimesg.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetsc.lib(init.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetsc.lib(pbarrier.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetsc.lib(tagm.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetsc.lib(dupl.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetsc.lib(draw.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetsc.lib(options.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetsc.lib(dscatter.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetsc.lib(mprint.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetsc.lib(view.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetsc.lib(verboseinfo.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetsc.lib(plog.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetsc.lib(lg.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetsc.lib(binv.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetsc.lib(drawv.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetsc.lib(pinit.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetsc.lib(filev.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscvec.lib(comb.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscvec.lib(pf.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetsc.lib(zstart.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetsc.lib(zutils.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscvec.lib(vpscat.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscvec.lib(pdvec.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscvec.lib(pvec2.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscvec.lib(shvec.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscvec.lib(vecstash.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscvec.lib(dlregisvec.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscvec.lib(dvec2.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscvec.lib(vecio.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscvec.lib(isltog.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscvec.lib(pmap.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscvec.lib(pbvec.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscvec.lib(block.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscvec.lib(index.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscvec.lib(general.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscvec.lib(stride.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscvec.lib(iscomp.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscvec.lib(vinv.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscvec.lib(bvec2.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscvec.lib(vscat.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscvec.lib(iscoloring.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscmat.lib(mmsbaij.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscmat.lib(baijov.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscvec.lib(vector.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscvec.lib(rvector.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscmat.lib(mpibaij.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscmat.lib(aijtype.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscmat.lib(maij.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscmat.lib(sbaijov.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscmat.lib(bdiag.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscmat.lib(mpibdiag.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscmat.lib(sbaij.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscmat.lib(mpisbaij.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscmat.lib(partition.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscmat.lib(fdmatrix.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscmat.lib(mscatter.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscmat.lib(mpiadj.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscmat.lib(fdmpiaij.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscmat.lib(mpiov.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscmat.lib(mpicsrperm.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscmat.lib(mmdense.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscmat.lib(baij.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscmat.lib(matstash.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscmat.lib(mpiptap.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscmat.lib(mpimatmatmult.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscmat.lib(mpiaij.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscmat.lib(dense.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscmat.lib(mpidense.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscmat.lib(aij.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscksp.lib(comm.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscksp.lib(gs.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscmat.lib(matrix.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscmat.lib(axpy.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscksp.lib(mg.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscksp.lib(lu.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscksp.lib(bjacobi.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscksp.lib(pbjacobi.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscksp.lib(eige.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscksp.lib(nn.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscksp.lib(redundant.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscksp.lib(asm.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscksp.lib(itcreate.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscksp.lib(xmon.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscksp.lib(itfunc.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetscksp.lib(precon.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP libpetsc.lib(errstop.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Abort libpetsc.lib(fp.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Abort libpetsc.lib(mtr.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Abort libpetsc.lib(mpimesg.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Abort libpetsc.lib(mpiu.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Abort libpetsc.lib(signal.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Abort libpetsc.lib(pinit.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Abort libpetsc.lib(plog.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Abort libpetsc.lib(mprint.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Abort libpetsc.lib(init.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Abort libpetscvec.lib(pvec2.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Abort libpetscvec.lib(comb.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Abort libpetsc.lib(zutils.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Abort libpetsc.lib(err.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Abort libpetscvec.lib(vecstash.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Abort libpetscvec.lib(vecio.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Abort libpetscvec.lib(vpscat.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Abort libpetscvec.lib(pdvec.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Abort libpetscmat.lib(sbaijov.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Abort libpetscmat.lib(baijov.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Abort libpetscvec.lib(iscoloring.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Abort libpetscvec.lib(isltog.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Abort libpetscmat.lib(mmdense.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Abort libpetscmat.lib(mpibdiag.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Abort libpetscmat.lib(mpisbaij.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Abort libpetscmat.lib(mpibaij.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Abort libpetscmat.lib(matstash.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Abort libpetscmat.lib(mpiptap.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Abort libpetscmat.lib(mpimatmatmult.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Abort libpetscmat.lib(mpiov.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Abort libpetscmat.lib(mpiaij.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Abort libpetscmat.lib(mpidense.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Abort libpetscmat.lib(aij.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Abort libpetscmat.lib(baij.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Abort libpetscksp.lib(nn.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Abort libpetscksp.lib(error.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Abort libpetscksp.lib(comm.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Abort libpetscksp.lib(gs.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Abort libpetsc.lib(mpimesg.o) : error LNK2001: unresolved external symbol _MPIUNI_Memcpy libpetsc.lib(sseenabled.o) : error LNK2001: unresolved external symbol _MPIUNI_Memcpy libpetsc.lib(fretrieve.o) : error LNK2001: unresolved external symbol _MPIUNI_Memcpy libpetsc.lib(pinit.o) : error LNK2001: unresolved external symbol _MPIUNI_Memcpy libpetsc.lib(plog.o) : error LNK2001: unresolved external symbol _MPIUNI_Memcpy libpetsc.lib(tagm.o) : error LNK2001: unresolved external symbol _MPIUNI_Memcpy libpetsc.lib(psplit.o) : error LNK2001: unresolved external symbol _MPIUNI_Memcpy libpetscvec.lib(vpscat.o) : error LNK2001: unresolved external symbol _MPIUNI_Memcpy libpetscvec.lib(pdvec.o) : error LNK2001: unresolved external symbol _MPIUNI_Memcpy libpetscvec.lib(pvec2.o) : error LNK2001: unresolved external symbol _MPIUNI_Memcpy libpetscvec.lib(comb.o) : error LNK2001: unresolved external symbol _MPIUNI_Memcpy libpetscvec.lib(pmap.o) : error LNK2001: unresolved external symbol _MPIUNI_Memcpy libpetscvec.lib(pbvec.o) : error LNK2001: unresolved external symbol _MPIUNI_Memcpy libpetscvec.lib(block.o) : error LNK2001: unresolved external symbol _MPIUNI_Memcpy libpetscvec.lib(dvec2.o) : error LNK2001: unresolved external symbol _MPIUNI_Memcpy libpetscvec.lib(general.o) : error LNK2001: unresolved external symbol _MPIUNI_Memcpy libpetscvec.lib(stride.o) : error LNK2001: unresolved external symbol _MPIUNI_Memcpy libpetscvec.lib(iscomp.o) : error LNK2001: unresolved external symbol _MPIUNI_Memcpy libpetscvec.lib(isltog.o) : error LNK2001: unresolved external symbol _MPIUNI_Memcpy libpetscvec.lib(rvector.o) : error LNK2001: unresolved external symbol _MPIUNI_Memcpy libpetscvec.lib(vinv.o) : error LNK2001: unresolved external symbol _MPIUNI_Memcpy libpetscvec.lib(vscat.o) : error LNK2001: unresolved external symbol _MPIUNI_Memcpy libpetscvec.lib(iscoloring.o) : error LNK2001: unresolved external symbol _MPIUNI_Memcpy libpetscmat.lib(mpisbaij.o) : error LNK2001: unresolved external symbol _MPIUNI_Memcpy libpetscmat.lib(mpibaij.o) : error LNK2001: unresolved external symbol _MPIUNI_Memcpy libpetscmat.lib(sbaijov.o) : error LNK2001: unresolved external symbol _MPIUNI_Memcpy libpetscmat.lib(baijov.o) : error LNK2001: unresolved external symbol _MPIUNI_Memcpy libpetscmat.lib(mpiov.o) : error LNK2001: unresolved external symbol _MPIUNI_Memcpy libpetscmat.lib(mmdense.o) : error LNK2001: unresolved external symbol _MPIUNI_Memcpy libpetscmat.lib(mpiadj.o) : error LNK2001: unresolved external symbol _MPIUNI_Memcpy libpetscmat.lib(mpibdiag.o) : error LNK2001: unresolved external symbol _MPIUNI_Memcpy libpetscmat.lib(mpiaij.o) : error LNK2001: unresolved external symbol _MPIUNI_Memcpy libpetscmat.lib(mpidense.o) : error LNK2001: unresolved external symbol _MPIUNI_Memcpy libpetscmat.lib(mpiptap.o) : error LNK2001: unresolved external symbol _MPIUNI_Memcpy libpetscmat.lib(fdmpiaij.o) : error LNK2001: unresolved external symbol _MPIUNI_Memcpy libpetscksp.lib(asm.o) : error LNK2001: unresolved external symbol _MPIUNI_Memcpy libpetscksp.lib(bjacobi.o) : error LNK2001: unresolved external symbol _MPIUNI_Memcpy libpetscksp.lib(comm.o) : error LNK2001: unresolved external symbol _MPIUNI_Memcpy libpetscksp.lib(gs.o) : error LNK2001: unresolved external symbol _MPIUNI_Memcpy libpetsc.lib(filev.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Attr_put libpetsc.lib(tagm.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Attr_put libpetsc.lib(fretrieve.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Attr_put libpetsc.lib(mpiu.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Attr_put libpetscvec.lib(comb.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Attr_put libpetsc.lib(binv.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Attr_put libpetsc.lib(vcreatea.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Attr_put libpetsc.lib(drawv.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Attr_put libpetsc.lib(filev.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Attr_get libpetsc.lib(tagm.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Attr_get libpetsc.lib(fretrieve.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Attr_get libpetsc.lib(mpiu.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Attr_get libpetscvec.lib(comb.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Attr_get libpetsc.lib(binv.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Attr_get libpetsc.lib(vcreatea.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Attr_get libpetsc.lib(drawv.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Attr_get libpetsc.lib(filev.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Keyval_create libpetsc.lib(tagm.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Keyval_create libpetsc.lib(fretrieve.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Keyval_create libpetsc.lib(mpiu.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Keyval_create libpetscvec.lib(comb.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Keyval_create libpetsc.lib(binv.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Keyval_create libpetsc.lib(vcreatea.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Keyval_create libpetsc.lib(drawv.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Keyval_create libpetsc.lib(zstart.o) : error LNK2001: unresolved external symbol _MPI_INIT at 4 libpetsc.lib(zstart.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Initialized libpetsc.lib(pinit.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Initialized libpetsc.lib(pinit.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Finalize libpetsc.lib(init.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Finalize libpetsc.lib(adebug.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Finalize libpetsc.lib(pinit.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Comm_free libpetsc.lib(tagm.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Comm_free libpetsc.lib(mpiu.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Comm_free libpetsc.lib(pinit.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Comm_dup libpetsc.lib(tagm.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Comm_dup libpetsc.lib(mpiu.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Comm_dup libpetsc.lib(tagm.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Attr_delete libpetsc.lib(mpiu.o) : error LNK2001: unresolved external symbol _Petsc_MPI_Attr_delete Debug/ns2d_c.exe : fatal error LNK1120: 15 unresolved externals Error executing link.exe. ns2d_c.exe - 221 error(s), 0 warning(s) How can I resolve these errors? Thank you. -------------- next part -------------- An HTML attachment was scrubbed... URL: From zonexo at gmail.com Sun Mar 4 19:13:53 2007 From: zonexo at gmail.com (Ben Tay) Date: Mon, 5 Mar 2007 09:13:53 +0800 Subject: Using compaq visual fortran with gcc to compile PETSc Message-ID: <804ab5d40703041713p7fd625f8nbc212383cd9348f5@mail.gmail.com> Hi, I'm using compaq visual fortran and I program entirely in fortran. However, to compile PETSc, I need to use visual c++ in the past. Is it possible to use gcc or other c compilers to compile PETSc in cygwin? Thank you. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Sun Mar 4 20:59:14 2007 From: bsmith at mcs.anl.gov (Barry Smith) Date: Sun, 4 Mar 2007 20:59:14 -0600 (CST) Subject: How to configure Compaq visual fortran for PETSc w/o mpi In-Reply-To: <804ab5d40703041630r1581749dj345c74320a858c17@mail.gmail.com> References: <804ab5d40703041630r1581749dj345c74320a858c17@mail.gmail.com> Message-ID: Ben, Sounds like you are using your own makefiles with your own list of link libraries. Instead of removing the mpich.lib from the linking options you must replace it with the mpiuni library (which is a stub library for one process) libmpiuni.lib Barry On Mon, 5 Mar 2007, Ben Tay wrote: > Hi, > > I've managed to use PETSc in Compaq visual fortran using > --download-blas-lapack=1 with mpi installed. > > Now I tried to recompile PETSc with --with-mpi=0. I then tried to remove > mpich.lib from the linking options. compiling is ok but during linking, I > got these error msg: > > > linking... > global.obj : error LNK2001: unresolved external symbol _MPI_DUP_FN > petsc_sub.obj : error LNK2001: unresolved external symbol _MPI_DUP_FN > global.obj : error LNK2001: unresolved external symbol _MPI_NULL_DELETE_FN > petsc_sub.obj : error LNK2001: unresolved external symbol > _MPI_NULL_DELETE_FN > global.obj : error LNK2001: unresolved external symbol _MPI_NULL_COPY_FN > petsc_sub.obj : error LNK2001: unresolved external symbol _MPI_NULL_COPY_FN > libpetsc.lib(eventLog.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetsc.lib(dmouse.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetsc.lib(petscvu.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetsc.lib(pdisplay.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetsc.lib(fretrieve.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetsc.lib(mpiu.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP > libpetsc.lib(mpiuopen.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetsc.lib(dtri.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP > libpetsc.lib(sseenabled.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetsc.lib(random.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetsc.lib(axis.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP > libpetsc.lib(mtr.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP > libpetsc.lib(sysio.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetsc.lib(psplit.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetsc.lib(mpimesg.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetsc.lib(init.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP > libpetsc.lib(pbarrier.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetsc.lib(tagm.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP > libpetsc.lib(dupl.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP > libpetsc.lib(draw.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP > libpetsc.lib(options.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetsc.lib(dscatter.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetsc.lib(mprint.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetsc.lib(view.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP > libpetsc.lib(verboseinfo.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetsc.lib(plog.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP > libpetsc.lib(lg.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP > libpetsc.lib(binv.o) : error LNK2001: unresolved external symbol _MPIUNI_TMP > libpetsc.lib(drawv.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetsc.lib(pinit.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetsc.lib(filev.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscvec.lib(comb.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscvec.lib(pf.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetsc.lib(zstart.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetsc.lib(zutils.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscvec.lib(vpscat.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscvec.lib(pdvec.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscvec.lib(pvec2.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscvec.lib(shvec.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscvec.lib(vecstash.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscvec.lib(dlregisvec.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscvec.lib(dvec2.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscvec.lib(vecio.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscvec.lib(isltog.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscvec.lib(pmap.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscvec.lib(pbvec.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscvec.lib(block.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscvec.lib(index.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscvec.lib(general.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscvec.lib(stride.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscvec.lib(iscomp.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscvec.lib(vinv.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscvec.lib(bvec2.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscvec.lib(vscat.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscvec.lib(iscoloring.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscmat.lib(mmsbaij.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscmat.lib(baijov.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscvec.lib(vector.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscvec.lib(rvector.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscmat.lib(mpibaij.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscmat.lib(aijtype.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscmat.lib(maij.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscmat.lib(sbaijov.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscmat.lib(bdiag.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscmat.lib(mpibdiag.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscmat.lib(sbaij.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscmat.lib(mpisbaij.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscmat.lib(partition.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscmat.lib(fdmatrix.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscmat.lib(mscatter.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscmat.lib(mpiadj.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscmat.lib(fdmpiaij.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscmat.lib(mpiov.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscmat.lib(mpicsrperm.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscmat.lib(mmdense.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscmat.lib(baij.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscmat.lib(matstash.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscmat.lib(mpiptap.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscmat.lib(mpimatmatmult.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscmat.lib(mpiaij.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscmat.lib(dense.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscmat.lib(mpidense.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscmat.lib(aij.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscksp.lib(comm.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscksp.lib(gs.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscmat.lib(matrix.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscmat.lib(axpy.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscksp.lib(mg.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscksp.lib(lu.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscksp.lib(bjacobi.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscksp.lib(pbjacobi.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscksp.lib(eige.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscksp.lib(nn.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscksp.lib(redundant.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscksp.lib(asm.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscksp.lib(itcreate.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscksp.lib(xmon.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscksp.lib(itfunc.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetscksp.lib(precon.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > libpetsc.lib(errstop.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Abort > libpetsc.lib(fp.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Abort > libpetsc.lib(mtr.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Abort > libpetsc.lib(mpimesg.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Abort > libpetsc.lib(mpiu.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Abort > libpetsc.lib(signal.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Abort > libpetsc.lib(pinit.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Abort > libpetsc.lib(plog.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Abort > libpetsc.lib(mprint.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Abort > libpetsc.lib(init.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Abort > libpetscvec.lib(pvec2.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Abort > libpetscvec.lib(comb.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Abort > libpetsc.lib(zutils.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Abort > libpetsc.lib(err.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Abort > libpetscvec.lib(vecstash.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Abort > libpetscvec.lib(vecio.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Abort > libpetscvec.lib(vpscat.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Abort > libpetscvec.lib(pdvec.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Abort > libpetscmat.lib(sbaijov.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Abort > libpetscmat.lib(baijov.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Abort > libpetscvec.lib(iscoloring.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Abort > libpetscvec.lib(isltog.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Abort > libpetscmat.lib(mmdense.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Abort > libpetscmat.lib(mpibdiag.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Abort > libpetscmat.lib(mpisbaij.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Abort > libpetscmat.lib(mpibaij.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Abort > libpetscmat.lib(matstash.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Abort > libpetscmat.lib(mpiptap.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Abort > libpetscmat.lib(mpimatmatmult.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Abort > libpetscmat.lib(mpiov.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Abort > libpetscmat.lib(mpiaij.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Abort > libpetscmat.lib(mpidense.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Abort > libpetscmat.lib(aij.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Abort > libpetscmat.lib(baij.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Abort > libpetscksp.lib(nn.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Abort > libpetscksp.lib(error.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Abort > libpetscksp.lib(comm.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Abort > libpetscksp.lib(gs.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Abort > libpetsc.lib(mpimesg.o) : error LNK2001: unresolved external symbol > _MPIUNI_Memcpy > libpetsc.lib(sseenabled.o) : error LNK2001: unresolved external symbol > _MPIUNI_Memcpy > libpetsc.lib(fretrieve.o) : error LNK2001: unresolved external symbol > _MPIUNI_Memcpy > libpetsc.lib(pinit.o) : error LNK2001: unresolved external symbol > _MPIUNI_Memcpy > libpetsc.lib(plog.o) : error LNK2001: unresolved external symbol > _MPIUNI_Memcpy > libpetsc.lib(tagm.o) : error LNK2001: unresolved external symbol > _MPIUNI_Memcpy > libpetsc.lib(psplit.o) : error LNK2001: unresolved external symbol > _MPIUNI_Memcpy > libpetscvec.lib(vpscat.o) : error LNK2001: unresolved external symbol > _MPIUNI_Memcpy > libpetscvec.lib(pdvec.o) : error LNK2001: unresolved external symbol > _MPIUNI_Memcpy > libpetscvec.lib(pvec2.o) : error LNK2001: unresolved external symbol > _MPIUNI_Memcpy > libpetscvec.lib(comb.o) : error LNK2001: unresolved external symbol > _MPIUNI_Memcpy > libpetscvec.lib(pmap.o) : error LNK2001: unresolved external symbol > _MPIUNI_Memcpy > libpetscvec.lib(pbvec.o) : error LNK2001: unresolved external symbol > _MPIUNI_Memcpy > libpetscvec.lib(block.o) : error LNK2001: unresolved external symbol > _MPIUNI_Memcpy > libpetscvec.lib(dvec2.o) : error LNK2001: unresolved external symbol > _MPIUNI_Memcpy > libpetscvec.lib(general.o) : error LNK2001: unresolved external symbol > _MPIUNI_Memcpy > libpetscvec.lib(stride.o) : error LNK2001: unresolved external symbol > _MPIUNI_Memcpy > libpetscvec.lib(iscomp.o) : error LNK2001: unresolved external symbol > _MPIUNI_Memcpy > libpetscvec.lib(isltog.o) : error LNK2001: unresolved external symbol > _MPIUNI_Memcpy > libpetscvec.lib(rvector.o) : error LNK2001: unresolved external symbol > _MPIUNI_Memcpy > libpetscvec.lib(vinv.o) : error LNK2001: unresolved external symbol > _MPIUNI_Memcpy > libpetscvec.lib(vscat.o) : error LNK2001: unresolved external symbol > _MPIUNI_Memcpy > libpetscvec.lib(iscoloring.o) : error LNK2001: unresolved external symbol > _MPIUNI_Memcpy > libpetscmat.lib(mpisbaij.o) : error LNK2001: unresolved external symbol > _MPIUNI_Memcpy > libpetscmat.lib(mpibaij.o) : error LNK2001: unresolved external symbol > _MPIUNI_Memcpy > libpetscmat.lib(sbaijov.o) : error LNK2001: unresolved external symbol > _MPIUNI_Memcpy > libpetscmat.lib(baijov.o) : error LNK2001: unresolved external symbol > _MPIUNI_Memcpy > libpetscmat.lib(mpiov.o) : error LNK2001: unresolved external symbol > _MPIUNI_Memcpy > libpetscmat.lib(mmdense.o) : error LNK2001: unresolved external symbol > _MPIUNI_Memcpy > libpetscmat.lib(mpiadj.o) : error LNK2001: unresolved external symbol > _MPIUNI_Memcpy > libpetscmat.lib(mpibdiag.o) : error LNK2001: unresolved external symbol > _MPIUNI_Memcpy > libpetscmat.lib(mpiaij.o) : error LNK2001: unresolved external symbol > _MPIUNI_Memcpy > libpetscmat.lib(mpidense.o) : error LNK2001: unresolved external symbol > _MPIUNI_Memcpy > libpetscmat.lib(mpiptap.o) : error LNK2001: unresolved external symbol > _MPIUNI_Memcpy > libpetscmat.lib(fdmpiaij.o) : error LNK2001: unresolved external symbol > _MPIUNI_Memcpy > libpetscksp.lib(asm.o) : error LNK2001: unresolved external symbol > _MPIUNI_Memcpy > libpetscksp.lib(bjacobi.o) : error LNK2001: unresolved external symbol > _MPIUNI_Memcpy > libpetscksp.lib(comm.o) : error LNK2001: unresolved external symbol > _MPIUNI_Memcpy > libpetscksp.lib(gs.o) : error LNK2001: unresolved external symbol > _MPIUNI_Memcpy > libpetsc.lib(filev.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Attr_put > libpetsc.lib(tagm.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Attr_put > libpetsc.lib(fretrieve.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Attr_put > libpetsc.lib(mpiu.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Attr_put > libpetscvec.lib(comb.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Attr_put > libpetsc.lib(binv.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Attr_put > libpetsc.lib(vcreatea.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Attr_put > libpetsc.lib(drawv.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Attr_put > libpetsc.lib(filev.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Attr_get > libpetsc.lib(tagm.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Attr_get > libpetsc.lib(fretrieve.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Attr_get > libpetsc.lib(mpiu.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Attr_get > libpetscvec.lib(comb.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Attr_get > libpetsc.lib(binv.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Attr_get > libpetsc.lib(vcreatea.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Attr_get > libpetsc.lib(drawv.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Attr_get > libpetsc.lib(filev.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Keyval_create > libpetsc.lib(tagm.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Keyval_create > libpetsc.lib(fretrieve.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Keyval_create > libpetsc.lib(mpiu.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Keyval_create > libpetscvec.lib(comb.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Keyval_create > libpetsc.lib(binv.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Keyval_create > libpetsc.lib(vcreatea.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Keyval_create > libpetsc.lib(drawv.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Keyval_create > libpetsc.lib(zstart.o) : error LNK2001: unresolved external symbol > _MPI_INIT at 4 > libpetsc.lib(zstart.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Initialized > libpetsc.lib(pinit.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Initialized > libpetsc.lib(pinit.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Finalize > libpetsc.lib(init.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Finalize > libpetsc.lib(adebug.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Finalize > libpetsc.lib(pinit.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Comm_free > libpetsc.lib(tagm.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Comm_free > libpetsc.lib(mpiu.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Comm_free > libpetsc.lib(pinit.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Comm_dup > libpetsc.lib(tagm.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Comm_dup > libpetsc.lib(mpiu.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Comm_dup > libpetsc.lib(tagm.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Attr_delete > libpetsc.lib(mpiu.o) : error LNK2001: unresolved external symbol > _Petsc_MPI_Attr_delete > Debug/ns2d_c.exe : fatal error LNK1120: 15 unresolved externals > Error executing link.exe. > > ns2d_c.exe - 221 error(s), 0 warning(s) > > How can I resolve these errors? > > Thank you. > From zonexo at gmail.com Sun Mar 4 21:49:26 2007 From: zonexo at gmail.com (Ben Tay) Date: Mon, 5 Mar 2007 11:49:26 +0800 Subject: How to configure Compaq visual fortran for PETSc w/o mpi In-Reply-To: References: <804ab5d40703041630r1581749dj345c74320a858c17@mail.gmail.com> Message-ID: <804ab5d40703041949h5354e195o41c11f6ddfe4cd20@mail.gmail.com> Hi, Thanks alot!. It worked. On 3/5/07, Barry Smith wrote: > > > Ben, > > Sounds like you are using your own makefiles with your own list > of link libraries. Instead of removing the mpich.lib from the linking > options you must replace it with the mpiuni library (which is a stub > library for one process) libmpiuni.lib > > Barry > > On Mon, 5 Mar 2007, Ben Tay wrote: > > > Hi, > > > > I've managed to use PETSc in Compaq visual fortran using > > --download-blas-lapack=1 with mpi installed. > > > > Now I tried to recompile PETSc with --with-mpi=0. I then tried to remove > > mpich.lib from the linking options. compiling is ok but during linking, > I > > got these error msg: > > > > > > linking... > > global.obj : error LNK2001: unresolved external symbol _MPI_DUP_FN > > petsc_sub.obj : error LNK2001: unresolved external symbol _MPI_DUP_FN > > global.obj : error LNK2001: unresolved external symbol > _MPI_NULL_DELETE_FN > > petsc_sub.obj : error LNK2001: unresolved external symbol > > _MPI_NULL_DELETE_FN > > global.obj : error LNK2001: unresolved external symbol _MPI_NULL_COPY_FN > > petsc_sub.obj : error LNK2001: unresolved external symbol > _MPI_NULL_COPY_FN > > libpetsc.lib(eventLog.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetsc.lib(dmouse.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetsc.lib(petscvu.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetsc.lib(pdisplay.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetsc.lib(fretrieve.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetsc.lib(mpiu.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > > libpetsc.lib(mpiuopen.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetsc.lib(dtri.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > > libpetsc.lib(sseenabled.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetsc.lib(random.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetsc.lib(axis.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > > libpetsc.lib(mtr.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > > libpetsc.lib(sysio.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetsc.lib(psplit.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetsc.lib(mpimesg.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetsc.lib(init.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > > libpetsc.lib(pbarrier.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetsc.lib(tagm.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > > libpetsc.lib(dupl.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > > libpetsc.lib(draw.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > > libpetsc.lib(options.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetsc.lib(dscatter.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetsc.lib(mprint.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetsc.lib(view.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > > libpetsc.lib(verboseinfo.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetsc.lib(plog.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > > libpetsc.lib(lg.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > > libpetsc.lib(binv.o) : error LNK2001: unresolved external symbol > _MPIUNI_TMP > > libpetsc.lib(drawv.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetsc.lib(pinit.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetsc.lib(filev.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetscvec.lib(comb.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetscvec.lib(pf.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetsc.lib(zstart.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetsc.lib(zutils.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetscvec.lib(vpscat.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetscvec.lib(pdvec.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetscvec.lib(pvec2.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetscvec.lib(shvec.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetscvec.lib(vecstash.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetscvec.lib(dlregisvec.o) : error LNK2001: unresolved external > symbol > > _MPIUNI_TMP > > libpetscvec.lib(dvec2.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetscvec.lib(vecio.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetscvec.lib(isltog.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetscvec.lib(pmap.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetscvec.lib(pbvec.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetscvec.lib(block.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetscvec.lib(index.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetscvec.lib(general.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetscvec.lib(stride.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetscvec.lib(iscomp.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetscvec.lib(vinv.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetscvec.lib(bvec2.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetscvec.lib(vscat.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetscvec.lib(iscoloring.o) : error LNK2001: unresolved external > symbol > > _MPIUNI_TMP > > libpetscmat.lib(mmsbaij.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetscmat.lib(baijov.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetscvec.lib(vector.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetscvec.lib(rvector.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetscmat.lib(mpibaij.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetscmat.lib(aijtype.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetscmat.lib(maij.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetscmat.lib(sbaijov.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetscmat.lib(bdiag.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetscmat.lib(mpibdiag.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetscmat.lib(sbaij.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetscmat.lib(mpisbaij.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetscmat.lib(partition.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetscmat.lib(fdmatrix.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetscmat.lib(mscatter.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetscmat.lib(mpiadj.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetscmat.lib(fdmpiaij.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetscmat.lib(mpiov.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetscmat.lib(mpicsrperm.o) : error LNK2001: unresolved external > symbol > > _MPIUNI_TMP > > libpetscmat.lib(mmdense.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetscmat.lib(baij.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetscmat.lib(matstash.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetscmat.lib(mpiptap.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetscmat.lib(mpimatmatmult.o) : error LNK2001: unresolved external > symbol > > _MPIUNI_TMP > > libpetscmat.lib(mpiaij.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetscmat.lib(dense.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetscmat.lib(mpidense.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetscmat.lib(aij.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetscksp.lib(comm.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetscksp.lib(gs.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetscmat.lib(matrix.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetscmat.lib(axpy.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetscksp.lib(mg.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetscksp.lib(lu.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetscksp.lib(bjacobi.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetscksp.lib(pbjacobi.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetscksp.lib(eige.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetscksp.lib(nn.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetscksp.lib(redundant.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetscksp.lib(asm.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetscksp.lib(itcreate.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetscksp.lib(xmon.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetscksp.lib(itfunc.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetscksp.lib(precon.o) : error LNK2001: unresolved external symbol > > _MPIUNI_TMP > > libpetsc.lib(errstop.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Abort > > libpetsc.lib(fp.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Abort > > libpetsc.lib(mtr.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Abort > > libpetsc.lib(mpimesg.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Abort > > libpetsc.lib(mpiu.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Abort > > libpetsc.lib(signal.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Abort > > libpetsc.lib(pinit.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Abort > > libpetsc.lib(plog.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Abort > > libpetsc.lib(mprint.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Abort > > libpetsc.lib(init.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Abort > > libpetscvec.lib(pvec2.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Abort > > libpetscvec.lib(comb.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Abort > > libpetsc.lib(zutils.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Abort > > libpetsc.lib(err.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Abort > > libpetscvec.lib(vecstash.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Abort > > libpetscvec.lib(vecio.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Abort > > libpetscvec.lib(vpscat.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Abort > > libpetscvec.lib(pdvec.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Abort > > libpetscmat.lib(sbaijov.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Abort > > libpetscmat.lib(baijov.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Abort > > libpetscvec.lib(iscoloring.o) : error LNK2001: unresolved external > symbol > > _Petsc_MPI_Abort > > libpetscvec.lib(isltog.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Abort > > libpetscmat.lib(mmdense.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Abort > > libpetscmat.lib(mpibdiag.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Abort > > libpetscmat.lib(mpisbaij.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Abort > > libpetscmat.lib(mpibaij.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Abort > > libpetscmat.lib(matstash.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Abort > > libpetscmat.lib(mpiptap.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Abort > > libpetscmat.lib(mpimatmatmult.o) : error LNK2001: unresolved external > symbol > > _Petsc_MPI_Abort > > libpetscmat.lib(mpiov.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Abort > > libpetscmat.lib(mpiaij.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Abort > > libpetscmat.lib(mpidense.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Abort > > libpetscmat.lib(aij.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Abort > > libpetscmat.lib(baij.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Abort > > libpetscksp.lib(nn.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Abort > > libpetscksp.lib(error.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Abort > > libpetscksp.lib(comm.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Abort > > libpetscksp.lib(gs.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Abort > > libpetsc.lib(mpimesg.o) : error LNK2001: unresolved external symbol > > _MPIUNI_Memcpy > > libpetsc.lib(sseenabled.o) : error LNK2001: unresolved external symbol > > _MPIUNI_Memcpy > > libpetsc.lib(fretrieve.o) : error LNK2001: unresolved external symbol > > _MPIUNI_Memcpy > > libpetsc.lib(pinit.o) : error LNK2001: unresolved external symbol > > _MPIUNI_Memcpy > > libpetsc.lib(plog.o) : error LNK2001: unresolved external symbol > > _MPIUNI_Memcpy > > libpetsc.lib(tagm.o) : error LNK2001: unresolved external symbol > > _MPIUNI_Memcpy > > libpetsc.lib(psplit.o) : error LNK2001: unresolved external symbol > > _MPIUNI_Memcpy > > libpetscvec.lib(vpscat.o) : error LNK2001: unresolved external symbol > > _MPIUNI_Memcpy > > libpetscvec.lib(pdvec.o) : error LNK2001: unresolved external symbol > > _MPIUNI_Memcpy > > libpetscvec.lib(pvec2.o) : error LNK2001: unresolved external symbol > > _MPIUNI_Memcpy > > libpetscvec.lib(comb.o) : error LNK2001: unresolved external symbol > > _MPIUNI_Memcpy > > libpetscvec.lib(pmap.o) : error LNK2001: unresolved external symbol > > _MPIUNI_Memcpy > > libpetscvec.lib(pbvec.o) : error LNK2001: unresolved external symbol > > _MPIUNI_Memcpy > > libpetscvec.lib(block.o) : error LNK2001: unresolved external symbol > > _MPIUNI_Memcpy > > libpetscvec.lib(dvec2.o) : error LNK2001: unresolved external symbol > > _MPIUNI_Memcpy > > libpetscvec.lib(general.o) : error LNK2001: unresolved external symbol > > _MPIUNI_Memcpy > > libpetscvec.lib(stride.o) : error LNK2001: unresolved external symbol > > _MPIUNI_Memcpy > > libpetscvec.lib(iscomp.o) : error LNK2001: unresolved external symbol > > _MPIUNI_Memcpy > > libpetscvec.lib(isltog.o) : error LNK2001: unresolved external symbol > > _MPIUNI_Memcpy > > libpetscvec.lib(rvector.o) : error LNK2001: unresolved external symbol > > _MPIUNI_Memcpy > > libpetscvec.lib(vinv.o) : error LNK2001: unresolved external symbol > > _MPIUNI_Memcpy > > libpetscvec.lib(vscat.o) : error LNK2001: unresolved external symbol > > _MPIUNI_Memcpy > > libpetscvec.lib(iscoloring.o) : error LNK2001: unresolved external > symbol > > _MPIUNI_Memcpy > > libpetscmat.lib(mpisbaij.o) : error LNK2001: unresolved external symbol > > _MPIUNI_Memcpy > > libpetscmat.lib(mpibaij.o) : error LNK2001: unresolved external symbol > > _MPIUNI_Memcpy > > libpetscmat.lib(sbaijov.o) : error LNK2001: unresolved external symbol > > _MPIUNI_Memcpy > > libpetscmat.lib(baijov.o) : error LNK2001: unresolved external symbol > > _MPIUNI_Memcpy > > libpetscmat.lib(mpiov.o) : error LNK2001: unresolved external symbol > > _MPIUNI_Memcpy > > libpetscmat.lib(mmdense.o) : error LNK2001: unresolved external symbol > > _MPIUNI_Memcpy > > libpetscmat.lib(mpiadj.o) : error LNK2001: unresolved external symbol > > _MPIUNI_Memcpy > > libpetscmat.lib(mpibdiag.o) : error LNK2001: unresolved external symbol > > _MPIUNI_Memcpy > > libpetscmat.lib(mpiaij.o) : error LNK2001: unresolved external symbol > > _MPIUNI_Memcpy > > libpetscmat.lib(mpidense.o) : error LNK2001: unresolved external symbol > > _MPIUNI_Memcpy > > libpetscmat.lib(mpiptap.o) : error LNK2001: unresolved external symbol > > _MPIUNI_Memcpy > > libpetscmat.lib(fdmpiaij.o) : error LNK2001: unresolved external symbol > > _MPIUNI_Memcpy > > libpetscksp.lib(asm.o) : error LNK2001: unresolved external symbol > > _MPIUNI_Memcpy > > libpetscksp.lib(bjacobi.o) : error LNK2001: unresolved external symbol > > _MPIUNI_Memcpy > > libpetscksp.lib(comm.o) : error LNK2001: unresolved external symbol > > _MPIUNI_Memcpy > > libpetscksp.lib(gs.o) : error LNK2001: unresolved external symbol > > _MPIUNI_Memcpy > > libpetsc.lib(filev.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Attr_put > > libpetsc.lib(tagm.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Attr_put > > libpetsc.lib(fretrieve.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Attr_put > > libpetsc.lib(mpiu.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Attr_put > > libpetscvec.lib(comb.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Attr_put > > libpetsc.lib(binv.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Attr_put > > libpetsc.lib(vcreatea.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Attr_put > > libpetsc.lib(drawv.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Attr_put > > libpetsc.lib(filev.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Attr_get > > libpetsc.lib(tagm.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Attr_get > > libpetsc.lib(fretrieve.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Attr_get > > libpetsc.lib(mpiu.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Attr_get > > libpetscvec.lib(comb.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Attr_get > > libpetsc.lib(binv.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Attr_get > > libpetsc.lib(vcreatea.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Attr_get > > libpetsc.lib(drawv.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Attr_get > > libpetsc.lib(filev.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Keyval_create > > libpetsc.lib(tagm.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Keyval_create > > libpetsc.lib(fretrieve.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Keyval_create > > libpetsc.lib(mpiu.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Keyval_create > > libpetscvec.lib(comb.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Keyval_create > > libpetsc.lib(binv.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Keyval_create > > libpetsc.lib(vcreatea.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Keyval_create > > libpetsc.lib(drawv.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Keyval_create > > libpetsc.lib(zstart.o) : error LNK2001: unresolved external symbol > > _MPI_INIT at 4 > > libpetsc.lib(zstart.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Initialized > > libpetsc.lib(pinit.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Initialized > > libpetsc.lib(pinit.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Finalize > > libpetsc.lib(init.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Finalize > > libpetsc.lib(adebug.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Finalize > > libpetsc.lib(pinit.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Comm_free > > libpetsc.lib(tagm.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Comm_free > > libpetsc.lib(mpiu.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Comm_free > > libpetsc.lib(pinit.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Comm_dup > > libpetsc.lib(tagm.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Comm_dup > > libpetsc.lib(mpiu.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Comm_dup > > libpetsc.lib(tagm.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Attr_delete > > libpetsc.lib(mpiu.o) : error LNK2001: unresolved external symbol > > _Petsc_MPI_Attr_delete > > Debug/ns2d_c.exe : fatal error LNK1120: 15 unresolved externals > > Error executing link.exe. > > > > ns2d_c.exe - 221 error(s), 0 warning(s) > > > > How can I resolve these errors? > > > > Thank you. > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From li76pan at yahoo.com Mon Mar 5 01:29:44 2007 From: li76pan at yahoo.com (li pan) Date: Sun, 4 Mar 2007 23:29:44 -0800 (PST) Subject: about Unstructured Meshes In-Reply-To: Message-ID: <219736.10676.qm@web36808.mail.mud.yahoo.com> I always need complex geometry of different curves and distorted surface. In my case, I found, Netgen generates much better mesh. Just out of my curiousity. How complicated is your geometry? see u pan --- Matthew Knepley wrote: > On 3/4/07, li pan wrote: > > hi Matt, > > have you ever evaluated Tetgen? How do you think > about > > the quality of the 3D mesh it generated? > > I have heard anecdotes about failures for > complicated geometry, > however I have never had a problem. The quality is > pretty good. > Gary Miller has shown that the number of tets > generated is way > too high, but I know of no better implementation. > > Matt > > > see u > > > > pan > > > > > > --- Matthew Knepley wrote: > > > > > On 3/2/07, Jianing Shi > wrote: > > > > Well, let me rephrase my question. So the > mesh > > > support in PETSc > > > > already includes the functionality of > partitioning > > > meshes, I guess, > > > > using ParMetis, is that the case? Something > that > > > an end user need to > > > > worry about is really how to generate a mesh > that > > > is tailored towards > > > > his/her application. > > > > > > It can use a range of partioners, like Chaco for > > > instance. > > > > > > > I am trying to write a library on top of PETSc > > > meshes that will > > > > generate meshes according to some > neurophysiology. > > > I would like to > > > > know what are the mesh generate softwares out > > > there that will > > > > interface nicely with PETSc, or if it makes > sense > > > for me to write my > > > > own? Just would like to understand more about > the > > > data structure in > > > > the PETSc ALE::Mesh classes. Is there any > > > tutorial out there apart > > > > from looking at the source code in the mesh > > > directory? > > > > > > 1) No, it makes no sense for you to write a mesh > > > generator > > > > > > 2) In 2D, Triangle. In 3d, the only free things > are > > > TetGen and Netgen. I support > > > TetGen. Hopefully, CMU will release its MG > soon. > > > > > > There is a tutorial on the website. > > > > > > > I am currently using the petsc-2.3.2-p3. Is > there > > > any new > > > > functionality about meshes in the development > > > version? > > > > > > All the working stuff is in petsc-dev. > > > > > > Matt > > > > > > > Jianing > > > -- > > > One trouble is that despite this system, anyone > who > > > reads journals widely > > > and critically is forced to realize that there > are > > > scarcely any bars to eventual > > > publication. There seems to be no study too > > > fragmented, no hypothesis too > > > trivial, no literature citation too biased or > too > > > egotistical, no design too > > > warped, no methodology too bungled, no > presentation > > > of results too > > > inaccurate, too obscure, and too contradictory, > no > > > analysis too self-serving, > > > no argument too circular, no conclusions too > > > trifling or too unjustified, and > > > no grammar and syntax too offensive for a paper > to > > > end up in print. -- > > > Drummond Rennie > > > > > > > > > > > > > > > > > ____________________________________________________________________________________ > > Do you Yahoo!? > > Everyone is raving about the all-new Yahoo! Mail > beta. > > http://new.mail.yahoo.com > > > > > > > -- > One trouble is that despite this system, anyone who > reads journals widely > and critically is forced to realize that there are > scarcely any bars to eventual > publication. There seems to be no study too > fragmented, no hypothesis too > trivial, no literature citation too biased or too > egotistical, no design too > warped, no methodology too bungled, no presentation > of results too > inaccurate, too obscure, and too contradictory, no > analysis too self-serving, > no argument too circular, no conclusions too > trifling or too unjustified, and > no grammar and syntax too offensive for a paper to > end up in print. -- > Drummond Rennie > > ____________________________________________________________________________________ 8:00? 8:25? 8:40? Find a flick in no time with the Yahoo! Search movie showtime shortcut. http://tools.search.yahoo.com/shortcuts/#news From knepley at gmail.com Mon Mar 5 09:08:38 2007 From: knepley at gmail.com (Matthew Knepley) Date: Mon, 5 Mar 2007 09:08:38 -0600 Subject: about Unstructured Meshes In-Reply-To: <219736.10676.qm@web36808.mail.mud.yahoo.com> References: <219736.10676.qm@web36808.mail.mud.yahoo.com> Message-ID: On 3/5/07, li pan wrote: > I always need complex geometry of different curves and > distorted surface. In my case, I found, Netgen > generates much better mesh. Just out of my curiousity. > How complicated is your geometry? Thats odd. I have never had netgen give me a better mesh. Matt > see u > > pan > > > --- Matthew Knepley wrote: > > > On 3/4/07, li pan wrote: > > > hi Matt, > > > have you ever evaluated Tetgen? How do you think > > about > > > the quality of the 3D mesh it generated? > > > > I have heard anecdotes about failures for > > complicated geometry, > > however I have never had a problem. The quality is > > pretty good. > > Gary Miller has shown that the number of tets > > generated is way > > too high, but I know of no better implementation. > > > > Matt > > > > > see u > > > > > > pan > > > > > > > > > --- Matthew Knepley wrote: > > > > > > > On 3/2/07, Jianing Shi > > wrote: > > > > > Well, let me rephrase my question. So the > > mesh > > > > support in PETSc > > > > > already includes the functionality of > > partitioning > > > > meshes, I guess, > > > > > using ParMetis, is that the case? Something > > that > > > > an end user need to > > > > > worry about is really how to generate a mesh > > that > > > > is tailored towards > > > > > his/her application. > > > > > > > > It can use a range of partioners, like Chaco for > > > > instance. > > > > > > > > > I am trying to write a library on top of PETSc > > > > meshes that will > > > > > generate meshes according to some > > neurophysiology. > > > > I would like to > > > > > know what are the mesh generate softwares out > > > > there that will > > > > > interface nicely with PETSc, or if it makes > > sense > > > > for me to write my > > > > > own? Just would like to understand more about > > the > > > > data structure in > > > > > the PETSc ALE::Mesh classes. Is there any > > > > tutorial out there apart > > > > > from looking at the source code in the mesh > > > > directory? > > > > > > > > 1) No, it makes no sense for you to write a mesh > > > > generator > > > > > > > > 2) In 2D, Triangle. In 3d, the only free things > > are > > > > TetGen and Netgen. I support > > > > TetGen. Hopefully, CMU will release its MG > > soon. > > > > > > > > There is a tutorial on the website. > > > > > > > > > I am currently using the petsc-2.3.2-p3. Is > > there > > > > any new > > > > > functionality about meshes in the development > > > > version? > > > > > > > > All the working stuff is in petsc-dev. > > > > > > > > Matt > > > > > > > > > Jianing > > > > -- > > > > One trouble is that despite this system, anyone > > who > > > > reads journals widely > > > > and critically is forced to realize that there > > are > > > > scarcely any bars to eventual > > > > publication. There seems to be no study too > > > > fragmented, no hypothesis too > > > > trivial, no literature citation too biased or > > too > > > > egotistical, no design too > > > > warped, no methodology too bungled, no > > presentation > > > > of results too > > > > inaccurate, too obscure, and too contradictory, > > no > > > > analysis too self-serving, > > > > no argument too circular, no conclusions too > > > > trifling or too unjustified, and > > > > no grammar and syntax too offensive for a paper > > to > > > > end up in print. -- > > > > Drummond Rennie > > > > > > > > > > > > > > > > > > > > > > > > > > ____________________________________________________________________________________ > > > Do you Yahoo!? > > > Everyone is raving about the all-new Yahoo! Mail > > beta. > > > http://new.mail.yahoo.com > > > > > > > > > > > > -- > > One trouble is that despite this system, anyone who > > reads journals widely > > and critically is forced to realize that there are > > scarcely any bars to eventual > > publication. There seems to be no study too > > fragmented, no hypothesis too > > trivial, no literature citation too biased or too > > egotistical, no design too > > warped, no methodology too bungled, no presentation > > of results too > > inaccurate, too obscure, and too contradictory, no > > analysis too self-serving, > > no argument too circular, no conclusions too > > trifling or too unjustified, and > > no grammar and syntax too offensive for a paper to > > end up in print. -- > > Drummond Rennie > > > > > > > > > ____________________________________________________________________________________ > 8:00? 8:25? 8:40? Find a flick in no time > with the Yahoo! Search movie showtime shortcut. > http://tools.search.yahoo.com/shortcuts/#news > > -- One trouble is that despite this system, anyone who reads journals widely and critically is forced to realize that there are scarcely any bars to eventual publication. There seems to be no study too fragmented, no hypothesis too trivial, no literature citation too biased or too egotistical, no design too warped, no methodology too bungled, no presentation of results too inaccurate, too obscure, and too contradictory, no analysis too self-serving, no argument too circular, no conclusions too trifling or too unjustified, and no grammar and syntax too offensive for a paper to end up in print. -- Drummond Rennie From petsc-maint at mcs.anl.gov Sun Mar 4 21:52:27 2007 From: petsc-maint at mcs.anl.gov (Barry Smith) Date: Sun, 4 Mar 2007 21:52:27 -0600 (CST) Subject: [PETSC #15886] Re: matrix-free additive schwarz? In-Reply-To: References: Message-ID: Aaron, You can use PCASMSetLocalSubdomains(pc,n,is[]) where is[] is an array of one or more IS index sets that define the subdomains (with overlap) for that process. Usually n is 1, which means one subdomain per processor. The routine PCASMCreateSubdomains2D() may be helpful for you in figuring out how to compute the is. You must also call PCASMSetOverlap(pc,0); so that PETSc will not try to increase the overlap from what you already provided. From this information the ASM PC figures out the size of the local vectors and the VecScatters that are needed to map to and from the local vectors. It will also create the appropriate local KSP solvers for the subdomains. BUT, BUT, BUT what operator (Mat) are you going to use on the overlapped region and if you are going to use matrix-free on the subdomain from what will you construct a preconditioner? The way PCASM currently works is it calls MatGetSubMatrices() on the original global matrix to get the Mat that is used as the operator on the subdomain. So what you need to do is for your matrix-free matrix (which I assume you create with MatCreateShell()) do MatShellSetOperation(mat,MAT_GET_SUBMATRICES,(void(*)(void))YourSubMat); Where YourSubMat() is a function that returns your subdomain matrix; which, of course, could be matrix-free or you could assemble it for the subdomain. Of course, if you use matrix-free here then what will you use for a preconditioner for the subdomain?? You could always build some other matrix to use to construct the preconditioner. Let us know if this helps, Barry There is on caveat, the is's you pass in get sorted by the ASM PC so when you provide your local matrix it must be provided in the sorted order. We could possibly fix this limitation if you are working fully with matrix-free. On Sun, 4 Mar 2007, P. Aaron Lott wrote: > > Hi Matt, > > I hope I can explain a bit better what it is I'm looking for, and also try to > understand what I would need to provide to petsc in order to use its ASM > preconditioner. It seems that in essence, for a matrix-free ASM > implementation, I would need to provide either the restriction operators for > each subdomain, or the assembled system matrix so that petsc's routines could > determine the restriction operators. > > Currently, I have a Newton-Krylov spectral element code for solving the 2D > incompressible Navier-Stokes equations. I perform all my matrix-vector > products in a matrix-free framework. I now need a preconditioner for the > subsidiary Advection-Diffusion equation solve, and I'm wanting to use ASM. I'm > hoping to apply this preconditioner in a matrix free setting as well. My code > applies matrix-vector products element-wise, and then performs an assembly > (gather-scatter) procedure to get the global solution. > > If all the subdomains for ASM are extensions of the elements from the spectral > element discretization, then challenge is to construct the the restriction > operators for each element to produce the ASM subdomains. From your message, > it seems like petsc isn't able to construct these restriction operators unless > I provide an assembled global system matrix. If this is what petsc needs, I > could compute the assembled system matrix (although storing this requires a > lot of memory), and pass it to petsc. I thought though, that it may be > possible to compute the restriction operators without accessing the assembled > system, but rather through local-global maps providing indirect addressing. If > someone has done this before, in a completely matrix-free setting, the same > code could be used, or modified to obtain the restriction operators for each > subdomain. I would be particularly interested in the second option if it is > available either in petsc, or some package that links with petsc. I suppose > the third option is, as you mentioned, providing my own preconditioner with > PCSHELL, but my original reason for wanting to use petsc, was for it to > provide me with the preconditioner... > > Thanks, > > -Aaron > > > On Mar 4, 2007, at 3:41 PM, Matthew Knepley wrote: > > > On 3/4/07, P. Aaron Lott wrote: > > > Hi, > > > > > > I'm new to petsc, and I've been reading through the petsc manual > > > today trying to figure out if petsc provides a matrix-free > > > implementation of a Newton-Krylov-Schwarz solver. It looks like in > > > terms of the Newton-Krylov part, everything can be done in a matrix- > > > free setting, but I can't tell if there are routines available to > > > allow for a matrix-free Additive Schwarz preconditioner, and if so, > > > what needs to be provided by the user to implement it. I'm very keen > > > on using petsc to provide an additive schwarz preconditioner for my > > > spectral element CFD code, but I need a matrix-free implementation > > > because I don't have access to the global system matrix. Does anyone > > > have experience with applying matrix-free preconditioners in petsc? > > > > Its not quite clear what you want. You can, of course, apply any > > preconditioner > > you want with PCSHELL. We do have an ASM preconditioner, however the extra > > support this provides is to > > > > 1) automatically figure out the overlapping domains using the matrix > > nonzero structure > > > > 2) solves the equation restricted to these overlapping partitions > > > > The first job cannot be done without a structure of some sort, which we do > > not > > have in the matrix-free case, and the second depends on the first. > > > > Matt > > > > > Thanks, > > > > > > -Aaron > > > > > > > > > > > > P. Aaron Lott > > > Ph.D. Candidate > > > 4239 Computer and Space Sciences Building > > > University of Maryland > > > College Park, MD 20742-4015 > > > > > > palott at ipst.umd.edu > > > http://www.lcv.umd.edu/~palott > > > Office: 301.405.4894 > > > Fax: 301.314.0827 > > > > > > > > > > > > > > > -- > > One trouble is that despite this system, anyone who reads journals widely > > and critically is forced to realize that there are scarcely any bars to > > eventual > > publication. There seems to be no study too fragmented, no hypothesis too > > trivial, no literature citation too biased or too egotistical, no design too > > warped, no methodology too bungled, no presentation of results too > > inaccurate, too obscure, and too contradictory, no analysis too > > self-serving, > > no argument too circular, no conclusions too trifling or too unjustified, > > and > > no grammar and syntax too offensive for a paper to end up in print. -- > > Drummond Rennie > > P. Aaron Lott > Ph.D. Candidate > 4239 Computer and Space Sciences Building > University of Maryland > College Park, MD 20742-4015 > > palott at ipst.umd.edu > http://www.lcv.umd.edu/~palott > Office: 301.405.4894 > Fax: 301.314.0827 > > From balay at mcs.anl.gov Mon Mar 5 11:13:59 2007 From: balay at mcs.anl.gov (Satish Balay) Date: Mon, 5 Mar 2007 11:13:59 -0600 (CST) Subject: Using compaq visual fortran with gcc to compile PETSc In-Reply-To: <804ab5d40703041713p7fd625f8nbc212383cd9348f5@mail.gmail.com> References: <804ab5d40703041713p7fd625f8nbc212383cd9348f5@mail.gmail.com> Message-ID: On Mon, 5 Mar 2007, Ben Tay wrote: > Hi, > > I'm using compaq visual fortran and I program entirely in fortran. However, > to compile PETSc, I need to use visual c++ in the past. Is it possible to > use gcc or other c compilers to compile PETSc in cygwin? Not - if you want to use PETSc from compaq fortran. You can use gcc+g77 [or g95] Satish From jianings at gmail.com Tue Mar 6 14:04:39 2007 From: jianings at gmail.com (Jianing Shi) Date: Tue, 6 Mar 2007 14:04:39 -0600 Subject: PETSc jacobian Message-ID: <63516a2e0703061204j68768e0eyda6ddb876ad49a26@mail.gmail.com> Hi PETSc designers, I have a non-trivial extension to a diffusion problem, that involves talking between neighboring (neighboring band) points. I have a question about the data structures in PETSc, since I am fantasized about elegant designs. I have two options: 1) use structured grid (DA) 2) in some applications, my grid can be unstructured, in fact a directed graph. I am considering the first option now (since I still need to read up onto the PETSc mesh/graph). Suppose I need to represent the connectivity of a point to its neighboring points, I need a weight matrix. I guess I can (A) use a DA structure, setting DOF = total number of points connected to a point (assuming the connectivity range for each point is the same). or (B) use a Jacobian matrix data structure to store these weights. In this case, I guess I will use MATMPIAIJ or MATMPIBAIJ. - What is the difference between these two types of Jacobian matrix? - For distributed sparse matrix like MATMPIAIJ, MATMPIBAIJ, how efficient is it to extract one column of the matrix M(i,:) ? Does PETSc have routines to do this? - How is such a distributed sparse matrix stored on different processors, how is it divided up, and collected? I want to understand this, because if a distributed sparse matrix is an efficient representation of my sparse weight matrix, I will go for this option (B). Otherwise, I will opt for option (A). Thanks! Jianing From aja2111 at columbia.edu Tue Mar 6 15:09:26 2007 From: aja2111 at columbia.edu (Aron Ahmadia) Date: Tue, 6 Mar 2007 16:09:26 -0500 Subject: PETSc jacobian In-Reply-To: <63516a2e0703061204j68768e0eyda6ddb876ad49a26@mail.gmail.com> References: <63516a2e0703061204j68768e0eyda6ddb876ad49a26@mail.gmail.com> Message-ID: <37604ab40703061309q32f456e7g7e345470dcc37da5@mail.gmail.com> Hi Jianing, The way I understand things, since a DA is fundamentally designed to handle spatially discretized differential equations, it is good for distributing your data and automatically handling situations that arise in PDE/ODE-solving, like updating of ghost points as fields change over time. Since a DA is not an operator, it can handle multiple dimensions without difficulty, as well as take the role of a 'weight matrix' defining some sort of interactions between points. The PETSc Mat, on the other hand, is fundamentally a linear operator. It's a way to define a relationship between two vectors in your space. It can easily multiply vectors and perform other 'linear algebra' operations, but it really isn't designed designed as a graph between two vectors. If I were trying to solve this problem, I would probably use DA to define my weight operator, then use direct access to the DA components to extract the columns. I would only use Mat if I was trying to solve for some sort of linear system, like Mx = b. Thanks, ~Aron PS - I fantasize about elegant designs as well On 3/6/07, Jianing Shi wrote: > Hi PETSc designers, > > I have a non-trivial extension to a diffusion problem, that involves > talking between neighboring (neighboring band) points. I have a > question about the data structures in PETSc, since I am fantasized > about elegant designs. > > I have two options: 1) use structured grid (DA) 2) in some > applications, my grid can be unstructured, in fact a directed graph. > I am considering the first option now (since I still need to read up > onto the PETSc mesh/graph). > > Suppose I need to represent the connectivity of a point to its > neighboring points, I need a weight matrix. I guess I can > (A) use a DA structure, setting DOF = total number of points connected > to a point (assuming the connectivity range for each point is the > same). > or > (B) use a Jacobian matrix data structure to store these weights. In > this case, I guess I will use MATMPIAIJ or MATMPIBAIJ. > > - What is the difference between these two types of Jacobian matrix? > - For distributed sparse matrix like MATMPIAIJ, MATMPIBAIJ, how > efficient is it to extract one column of the matrix M(i,:) ? Does > PETSc have routines to do this? > - How is such a distributed sparse matrix stored on different > processors, how is it divided up, and collected? > > I want to understand this, because if a distributed sparse matrix is > an efficient representation of my sparse weight matrix, I will go for > this option (B). Otherwise, I will opt for option (A). > > Thanks! > > Jianing > > From manav at u.washington.edu Tue Mar 6 15:28:37 2007 From: manav at u.washington.edu (Manav Bhatia) Date: Tue, 6 Mar 2007 13:28:37 -0800 (PST) Subject: General Question about Linear Solvers Message-ID: Hi I have an application where a linear system of equations has to be solved for many right hand sides. I am curious to know about how the different solvers compare with each other for such a case? ` So far, I have only used the LU decompositionm, where, once decomposed, the factorization can be use each time. I have been solving small systems so far, hence, cost of factorization has not been my concern. However, I will be moving towards much larger problems, requiring parallel processing. I would greatly appreciate if someone could comment on this. Thanks, Manav Bhatia PhD Candidate Department of Aeronautics and Astronautics Guggenheim Building University of Washington Seattle, WA 98195-2400 Ph (O) (206) 685 8063 e-mail: manav at u.washington.edu From knepley at gmail.com Tue Mar 6 15:44:42 2007 From: knepley at gmail.com (Matthew Knepley) Date: Tue, 6 Mar 2007 15:44:42 -0600 Subject: General Question about Linear Solvers In-Reply-To: References: Message-ID: In general, iterative information cannot be reused, or it is of little use. Until you get truly large, I would use sparse direct packages like MUMPS or SuperLU_dist through PETSc. They are simple (hopefully) with configure. Matt On 3/6/07, Manav Bhatia wrote: > Hi > > I have an application where a linear system of equations has to be > solved for many right hand sides. I am curious to know about how the > different solvers compare with each other for such a case? > ` So far, I have only used the LU decompositionm, where, once > decomposed, > the factorization can be use each time. I have been solving small systems > so far, hence, cost of factorization has not been my concern. However, I > will be moving towards much larger problems, requiring parallel > processing. > I would greatly appreciate if someone could comment on this. > > Thanks, > Manav Bhatia > PhD Candidate > Department of Aeronautics and Astronautics > Guggenheim Building > University of Washington > Seattle, WA 98195-2400 > > Ph (O) (206) 685 8063 > e-mail: manav at u.washington.edu > > -- One trouble is that despite this system, anyone who reads journals widely and critically is forced to realize that there are scarcely any bars to eventual publication. There seems to be no study too fragmented, no hypothesis too trivial, no literature citation too biased or too egotistical, no design too warped, no methodology too bungled, no presentation of results too inaccurate, too obscure, and too contradictory, no analysis too self-serving, no argument too circular, no conclusions too trifling or too unjustified, and no grammar and syntax too offensive for a paper to end up in print. -- Drummond Rennie From knepley at gmail.com Tue Mar 6 16:04:51 2007 From: knepley at gmail.com (Matthew Knepley) Date: Tue, 6 Mar 2007 16:04:51 -0600 Subject: PETSc jacobian In-Reply-To: <63516a2e0703061204j68768e0eyda6ddb876ad49a26@mail.gmail.com> References: <63516a2e0703061204j68768e0eyda6ddb876ad49a26@mail.gmail.com> Message-ID: On 3/6/07, Jianing Shi wrote: > Hi PETSc designers, > > I have a non-trivial extension to a diffusion problem, that involves > talking between neighboring (neighboring band) points. I have a > question about the data structures in PETSc, since I am fantasized > about elegant designs. > > I have two options: 1) use structured grid (DA) 2) in some > applications, my grid can be unstructured, in fact a directed graph. > I am considering the first option now (since I still need to read up > onto the PETSc mesh/graph). > > Suppose I need to represent the connectivity of a point to its > neighboring points, I need a weight matrix. I guess I can > (A) use a DA structure, setting DOF = total number of points connected > to a point (assuming the connectivity range for each point is the > same). > or > (B) use a Jacobian matrix data structure to store these weights. In > this case, I guess I will use MATMPIAIJ or MATMPIBAIJ. > > - What is the difference between these two types of Jacobian matrix? One has a fixed Cartesian topology (connectivity). The other can have an arbitrary connectivity. > - For distributed sparse matrix like MATMPIAIJ, MATMPIBAIJ, how > efficient is it to extract one column of the matrix M(i,:) ? Does > PETSc have routines to do this? It is there, but it is not efficient. Row extraction is efficient. > - How is such a distributed sparse matrix stored on different > processors, how is it divided up, and collected? Row-wise. There is a picture in the manual. > I want to understand this, because if a distributed sparse matrix is > an efficient representation of my sparse weight matrix, I will go for > this option (B). Otherwise, I will opt for option (A). I think you are confusing two issues. The DA is not an operator or a matrix. it is a topology or connectivity. It can create an AIJ matrix with that connectivity automatically. Matt > Thanks! > > Jianing > > -- One trouble is that despite this system, anyone who reads journals widely and critically is forced to realize that there are scarcely any bars to eventual publication. There seems to be no study too fragmented, no hypothesis too trivial, no literature citation too biased or too egotistical, no design too warped, no methodology too bungled, no presentation of results too inaccurate, too obscure, and too contradictory, no analysis too self-serving, no argument too circular, no conclusions too trifling or too unjustified, and no grammar and syntax too offensive for a paper to end up in print. -- Drummond Rennie From knutert at stud.ntnu.no Fri Mar 9 10:12:00 2007 From: knutert at stud.ntnu.no (Knut Erik Teigen) Date: Fri, 09 Mar 2007 17:12:00 +0100 Subject: PETSc CG solver uses more iterations than other CG solver Message-ID: <1173456720.6075.43.camel@iept0415.ivt.ntnu.no> Hello, I am using PETSc to calculate two-phase flow with the level set method, on a structured grid using finite differences. To solve the Navier-Stokes equations, I use an explicit Runge-Kutta method with Chorin's projection method, so a Poisson equation with von Neumann boundary conditions for the pressure has to be solved at every time-step. The equation system is positive definite, so I use the CG solver with the ICC preconditioner. The problem is that the PETSc solver seems to need a lot more iterations to reach the solution than another CG solver I'm using. On a small test problem(a rising bubble) with a 60x40 grid, the PETSc solver needs over 1000 iterations on average, while the other solver needs less than 100. I am using KSPSetInitialGuessNonzero, without this the number of iterations is even higher. I have also tried applying PETSc to a similar problem, solving the Poisson equation with von Neumann boundaries and a forcing function of f=sin(pi * x)+sin(pi *y). For this problem, the number of iterations is almost exactly the same for PETSc and the other solver. Does anyone know what the problem might be? Any help is greatly appreciated. I've included the -ksp_view of one of the time steps and the -log_summary below. Regards, Knut Erik Teigen MSc student Norwegian University of Science and Technology Output from -ksp_view: KSP Object: type: cg maximum iterations=10000 tolerances: relative=1e-06, absolute=1e-50, divergence=10000 left preconditioning PC Object: type: icc ICC: 0 levels of fill ICC: factor fill ratio allocated 1 ICC: factor fill ratio needed 0.601695 Factored matrix follows Matrix Object: type=seqsbaij, rows=2400, cols=2400 total: nonzeros=7100, allocated nonzeros=7100 block size is 1 linear system matrix = precond matrix: Matrix Object: type=seqaij, rows=2400, cols=2400 total: nonzeros=11800, allocated nonzeros=12000 not using I-node routines Poisson converged after 1403 iterations Output from -log_summary ---------------------------------------------- PETSc Performance Summary: ---------------------------------------------- ./run on a gcc-ifc-d named iept0415 with 1 processor, by knutert Fri Mar 9 17:06:05 2007 Using Petsc Release Version 2.3.2, Patch 8, Tue Jan 2 14:33:59 PST 2007 HG revision: ebeddcedcc065e32fc252af32cf1d01ed4fc7a80 Max Max/Min Avg Total Time (sec): 5.425e+02 1.00000 5.425e+02 Objects: 7.000e+02 1.00000 7.000e+02 Flops: 6.744e+10 1.00000 6.744e+10 6.744e+10 Flops/sec: 1.243e+08 1.00000 1.243e+08 1.243e+08 Memory: 4.881e+05 1.00000 4.881e+05 MPI Messages: 0.000e+00 0.00000 0.000e+00 0.000e+00 MPI Message Lengths: 0.000e+00 0.00000 0.000e+00 0.000e+00 MPI Reductions: 1.390e+03 1.00000 Flop counting convention: 1 flop = 1 real number operation of type (multiply/divide/add/subtract) e.g., VecAXPY() for real vectors of length N --> 2N flops and VecAXPY() for complex vectors of length N --> 8N flops Summary of Stages: ----- Time ------ ----- Flops ----- --- Messages --- -- Message Lengths -- -- Reductions -- Avg %Total Avg %Total counts % Total Avg %Total counts %Total 0: Main Stage: 5.4246e+02 100.0% 6.7437e+10 100.0% 0.000e+00 0.0% 0.000e+00 0.0% 1.390e+03 100.0% ------------------------------------------------------------------------------------------------------------------------ See the 'Profiling' chapter of the users' manual for details on interpreting output. Phase summary info: Count: number of times phase was executed Time and Flops/sec: Max - maximum over all processors Ratio - ratio of maximum to minimum over all processors Mess: number of messages sent Avg. len: average message length Reduct: number of global reductions Global: entire computation Stage: stages of a computation. Set stages with PetscLogStagePush() and PetscLogStagePop(). %T - percent time in this phase %F - percent flops in this phase %M - percent messages in this phase %L - percent message lengths in this phase %R - percent reductions in this phase Total Mflop/s: 10e-6 * (sum of flops over all processors)/(max time over all processors) ------------------------------------------------------------------------------------------------------------------------ ########################################################## # # # WARNING!!! # # # # This code was compiled with a debugging option, # # To get timing results run config/configure.py # # using --with-debugging=no, the performance will # # be generally two or three times faster. # # # ########################################################## ########################################################## # # # WARNING!!! # # # # This code was run without the PreLoadBegin() # # macros. To get timing results we always recommend # # preloading. otherwise timing numbers may be # # meaningless. # ########################################################## Event Count Time (sec) Flops/sec --- Global --- --- Stage --- Total Max Ratio Max Ratio Max Ratio Mess Avg len Reduct %T %F %M %L %R %T %F %M %L %R Mflop/s ------------------------------------------------------------------------------------------------------------------------ --- Event Stage 0: Main Stage VecDot 1668318 1.0 1.9186e+01 1.0 4.17e+08 1.0 0.0e+00 0.0e+00 0.0e+00 4 12 0 0 0 4 12 0 0 0 417 VecNorm 835191 1.0 1.0935e+01 1.0 3.67e+08 1.0 0.0e+00 0.0e+00 0.0e+00 2 6 0 0 0 2 6 0 0 0 367 VecCopy 688 1.0 1.6126e-02 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecAXPY 1667630 1.0 2.4141e+01 1.0 3.32e+08 1.0 0.0e+00 0.0e+00 0.0e+00 4 12 0 0 0 4 12 0 0 0 332 VecAYPX 833815 1.0 1.9062e+01 1.0 2.10e+08 1.0 0.0e+00 0.0e+00 0.0e+00 4 6 0 0 0 4 6 0 0 0 210 VecAssemblyBegin 688 1.0 8.5354e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecAssemblyEnd 688 1.0 7.8177e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatMult 834503 1.0 1.5044e+02 1.0 1.18e+08 1.0 0.0e+00 0.0e+00 0.0e+00 28 26 0 0 0 28 26 0 0 0 118 MatSolve 835191 1.0 1.8130e+02 1.0 1.42e+08 1.0 0.0e+00 0.0e+00 0.0e+00 33 38 0 0 0 33 38 0 0 0 142 MatCholFctrNum 1 1.0 1.1630e-03 1.0 2.06e+06 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 2 MatICCFactorSym 1 1.0 2.9588e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 2.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatAssemblyBegin 688 1.0 2.6343e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatAssemblyEnd 688 1.0 1.0964e-01 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatGetOrdering 1 1.0 2.6798e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 2.0e+00 0 0 0 0 0 0 0 0 0 0 0 KSPSetup 1 1.0 2.7585e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 6.0e+00 0 0 0 0 0 0 0 0 0 0 0 KSPSolve 688 1.0 4.2809e+02 1.0 1.58e+08 1.0 0.0e+00 0.0e+00 1.4e+03 79100 0 0100 79100 0 0100 158 PCSetUp 1 1.0 1.7900e-03 1.0 1.34e+06 1.0 0.0e+00 0.0e+00 4.0e+00 0 0 0 0 0 0 0 0 0 0 1 PCApply 835191 1.0 1.8864e+02 1.0 1.36e+08 1.0 0.0e+00 0.0e+00 0.0e+00 35 38 0 0 0 35 38 0 0 0 136 ------------------------------------------------------------------------------------------------------------------------ Memory usage is given in bytes: Object Type Creations Destructions Memory Descendants' Mem. --- Event Stage 0: Main Stage Index Set 2 2 19640 0 Vec 694 693 299376 0 Matrix 2 2 56400 0 Krylov Solver 1 1 36 0 Preconditioner 1 1 108 0 ======================================================================================================================== Average time to get PetscTime(): 2.86102e-07 OptionTable: -ksp_type cg OptionTable: -log_summary -ksp_view OptionTable: -pc_type icc Compiled without FORTRAN kernels Compiled with full precision matrices (default) sizeof(short) 2 sizeof(int) 4 sizeof(long) 4 sizeof(void*) 4 sizeof(PetscScalar) 8 Configure run at: Thu Mar 8 11:54:22 2007 Configure options: --with-cc=gcc --with-fc=ifort --download-f-blas-lapack=1 --download-mpich=1 --with-debugging=1 --with-shared=0 ----------------------------------------- Libraries compiled on Thu Mar 8 12:08:22 CET 2007 on iept0415 Machine characteristics: Linux iept0415 2.6.16.21-0.8-default #1 Mon Jul 3 18:25:39 UTC 2006 i686 i686 i386 GNU/Linux Using PETSc directory: /opt/petsc-2.3.2-p8 Using PETSc arch: gcc-ifc-debug ----------------------------------------- Using C compiler: gcc -fPIC -Wall -Wwrite-strings -Wno-strict-aliasing -g3 Using Fortran compiler: ifort -fPIC -g ----------------------------------------- Using include paths: -I/opt/petsc-2.3.2-p8 -I/opt/petsc-2.3.2-p8/bmake/gcc-ifc-debug -I/opt/petsc-2.3.2-p8/include -I/opt/petsc-2.3.2-p8/externalpackages/mpich2-1.0.4p1/gcc-ifc-debug/include -I/usr/X11R6/include ------------------------------------------ Using C linker: gcc -fPIC -Wall -Wwrite-strings -Wno-strict-aliasing -g3 Using Fortran linker: ifort -fPIC -g Using libraries: -Wl,-rpath,/opt/petsc-2.3.2-p8/lib/gcc-ifc-debug -L/opt/petsc-2.3.2-p8/lib/gcc-ifc-debug -lpetscts -lpetscsnes -lpetscksp -lpetscdm -lpetscmat -lpetscvec -lpetsc -Wl,-rpath,/opt/petsc-2.3.2-p8/externalpackages/mpich2-1.0.4p1/gcc-ifc-debug/lib -L/opt/petsc-2.3.2-p8/externalpackages/mpich2-1.0.4p1/gcc-ifc-debug/lib -lmpich -lnsl -lrt -L/usr/X11R6/lib -lX11 -Wl,-rpath,/opt/petsc-2.3.2-p8/externalpackages/fblaslapack/gcc-ifc-debug -L/opt/petsc-2.3.2-p8/externalpackages/fblaslapack/gcc-ifc-debug -lflapack -Wl,-rpath,/opt/petsc-2.3.2-p8/externalpackages/fblaslapack/gcc-ifc-debug -L/opt/petsc-2.3.2-p8/externalpackages/fblaslapack/gcc-ifc-debug -lfblas -lm -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0 -L/usr/lib/gcc/i586-suse-linux/4.1.0 -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linux/lib -L/usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linux/lib -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../.. -L/usr/lib/gcc/i586-suse-linux/4.1.0/../../.. -ldl -lgcc_s -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0 -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linux/lib -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../.. -Wl,-rpath,/opt/intel/fc/9.1.036/lib -L/opt/intel/fc/9.1.036/lib -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/ -L/usr/lib/gcc/i586-suse-linux/4.1.0/ -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../../ -L/usr/lib/gcc/i586-suse-linux/4.1.0/../../../ -lifport -lifcore -limf -lm -lipgo -lirc -lirc_s -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0 -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linux/lib -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../.. -lm -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0 -L/usr/lib/gcc/i586-suse-linux/4.1.0 -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linux/lib -L/usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linux/lib -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../.. -L/usr/lib/gcc/i586-suse-linux/4.1.0/../../.. -ldl -lgcc_s -ldl From dalcinl at gmail.com Fri Mar 9 11:05:08 2007 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Fri, 9 Mar 2007 14:05:08 -0300 Subject: PETSc CG solver uses more iterations than other CG solver In-Reply-To: <1173456720.6075.43.camel@iept0415.ivt.ntnu.no> References: <1173456720.6075.43.camel@iept0415.ivt.ntnu.no> Message-ID: On 3/9/07, Knut Erik Teigen wrote: > To solve the Navier-Stokes equations, I use an explicit Runge-Kutta > method with Chorin's projection method, > so a Poisson equation with von Neumann boundary conditions for the > pressure has to be solved at every time-step. All boundary conditions are Neumann type? In that case, please try to run your program with the following command line option: -ksp_constant_null_space and let me know if this corrected your problem. The equation system is > positive definite, so I use the CG solver with the ICC preconditioner. > The problem is that the PETSc solver seems to need a lot more iterations > to reach the solution than another CG solver I'm using. On a small test > problem(a rising bubble) with a 60x40 grid, the PETSc solver needs over > 1000 iterations on average, while the other solver needs less than 100. > I am using KSPSetInitialGuessNonzero, without this the number of > iterations is even higher. > I have also tried applying PETSc to a similar problem, solving the > Poisson equation with von Neumann boundaries and a forcing function of > f=sin(pi * x)+sin(pi *y). For this problem, the number of iterations is > almost exactly the same for PETSc and the other solver. > > Does anyone know what the problem might be? Any help is greatly > appreciated. I've included the -ksp_view of one of the time steps > and the -log_summary below. > > Regards, > Knut Erik Teigen > MSc student > Norwegian University of Science and Technology > > Output from -ksp_view: > > KSP Object: > type: cg > maximum iterations=10000 > tolerances: relative=1e-06, absolute=1e-50, divergence=10000 > left preconditioning > PC Object: > type: icc > ICC: 0 levels of fill > ICC: factor fill ratio allocated 1 > ICC: factor fill ratio needed 0.601695 > Factored matrix follows > Matrix Object: > type=seqsbaij, rows=2400, cols=2400 > total: nonzeros=7100, allocated nonzeros=7100 > block size is 1 > linear system matrix = precond matrix: > Matrix Object: > type=seqaij, rows=2400, cols=2400 > total: nonzeros=11800, allocated nonzeros=12000 > not using I-node routines > Poisson converged after 1403 iterations > > Output from -log_summary > > ---------------------------------------------- PETSc Performance > Summary: ---------------------------------------------- > > ./run on a gcc-ifc-d named iept0415 with 1 processor, by knutert Fri Mar > 9 17:06:05 2007 > Using Petsc Release Version 2.3.2, Patch 8, Tue Jan 2 14:33:59 PST 2007 > HG revision: ebeddcedcc065e32fc252af32cf1d01ed4fc7a80 > > Max Max/Min Avg Total > Time (sec): 5.425e+02 1.00000 5.425e+02 > Objects: 7.000e+02 1.00000 7.000e+02 > Flops: 6.744e+10 1.00000 6.744e+10 6.744e+10 > Flops/sec: 1.243e+08 1.00000 1.243e+08 1.243e+08 > Memory: 4.881e+05 1.00000 4.881e+05 > MPI Messages: 0.000e+00 0.00000 0.000e+00 0.000e+00 > MPI Message Lengths: 0.000e+00 0.00000 0.000e+00 0.000e+00 > MPI Reductions: 1.390e+03 1.00000 > > Flop counting convention: 1 flop = 1 real number operation of type > (multiply/divide/add/subtract) > e.g., VecAXPY() for real vectors of length N > --> 2N flops > and VecAXPY() for complex vectors of length > N --> 8N flops > > Summary of Stages: ----- Time ------ ----- Flops ----- --- Messages > --- -- Message Lengths -- -- Reductions -- > Avg %Total Avg %Total counts % > Total Avg %Total counts %Total > 0: Main Stage: 5.4246e+02 100.0% 6.7437e+10 100.0% 0.000e+00 > 0.0% 0.000e+00 0.0% 1.390e+03 100.0% > > ------------------------------------------------------------------------------------------------------------------------ > See the 'Profiling' chapter of the users' manual for details on > interpreting output. > Phase summary info: > Count: number of times phase was executed > Time and Flops/sec: Max - maximum over all processors > Ratio - ratio of maximum to minimum over all > processors > Mess: number of messages sent > Avg. len: average message length > Reduct: number of global reductions > Global: entire computation > Stage: stages of a computation. Set stages with PetscLogStagePush() > and PetscLogStagePop(). > %T - percent time in this phase %F - percent flops in this > phase > %M - percent messages in this phase %L - percent message > lengths in this phase > %R - percent reductions in this phase > Total Mflop/s: 10e-6 * (sum of flops over all processors)/(max time > over all processors) > ------------------------------------------------------------------------------------------------------------------------ > > > ########################################################## > # # > # WARNING!!! # > # # > # This code was compiled with a debugging option, # > # To get timing results run config/configure.py # > # using --with-debugging=no, the performance will # > # be generally two or three times faster. # > # # > ########################################################## > > > > > ########################################################## > # # > # WARNING!!! # > # # > # This code was run without the PreLoadBegin() # > # macros. To get timing results we always recommend # > # preloading. otherwise timing numbers may be # > # meaningless. # > ########################################################## > > > Event Count Time (sec) Flops/sec > --- Global --- --- Stage --- Total > Max Ratio Max Ratio Max Ratio Mess Avg len > Reduct %T %F %M %L %R %T %F %M %L %R Mflop/s > ------------------------------------------------------------------------------------------------------------------------ > > --- Event Stage 0: Main Stage > > VecDot 1668318 1.0 1.9186e+01 1.0 4.17e+08 1.0 0.0e+00 0.0e+00 > 0.0e+00 4 12 0 0 0 4 12 0 0 0 417 > VecNorm 835191 1.0 1.0935e+01 1.0 3.67e+08 1.0 0.0e+00 0.0e+00 > 0.0e+00 2 6 0 0 0 2 6 0 0 0 367 > VecCopy 688 1.0 1.6126e-02 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > VecAXPY 1667630 1.0 2.4141e+01 1.0 3.32e+08 1.0 0.0e+00 0.0e+00 > 0.0e+00 4 12 0 0 0 4 12 0 0 0 332 > VecAYPX 833815 1.0 1.9062e+01 1.0 2.10e+08 1.0 0.0e+00 0.0e+00 > 0.0e+00 4 6 0 0 0 4 6 0 0 0 210 > VecAssemblyBegin 688 1.0 8.5354e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > VecAssemblyEnd 688 1.0 7.8177e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > MatMult 834503 1.0 1.5044e+02 1.0 1.18e+08 1.0 0.0e+00 0.0e+00 > 0.0e+00 28 26 0 0 0 28 26 0 0 0 118 > MatSolve 835191 1.0 1.8130e+02 1.0 1.42e+08 1.0 0.0e+00 0.0e+00 > 0.0e+00 33 38 0 0 0 33 38 0 0 0 142 > MatCholFctrNum 1 1.0 1.1630e-03 1.0 2.06e+06 1.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 2 > MatICCFactorSym 1 1.0 2.9588e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > 2.0e+00 0 0 0 0 0 0 0 0 0 0 0 > MatAssemblyBegin 688 1.0 2.6343e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > MatAssemblyEnd 688 1.0 1.0964e-01 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > MatGetOrdering 1 1.0 2.6798e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > 2.0e+00 0 0 0 0 0 0 0 0 0 0 0 > KSPSetup 1 1.0 2.7585e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > 6.0e+00 0 0 0 0 0 0 0 0 0 0 0 > KSPSolve 688 1.0 4.2809e+02 1.0 1.58e+08 1.0 0.0e+00 0.0e+00 > 1.4e+03 79100 0 0100 79100 0 0100 158 > PCSetUp 1 1.0 1.7900e-03 1.0 1.34e+06 1.0 0.0e+00 0.0e+00 > 4.0e+00 0 0 0 0 0 0 0 0 0 0 1 > PCApply 835191 1.0 1.8864e+02 1.0 1.36e+08 1.0 0.0e+00 0.0e+00 > 0.0e+00 35 38 0 0 0 35 38 0 0 0 136 > ------------------------------------------------------------------------------------------------------------------------ > > Memory usage is given in bytes: > > Object Type Creations Destructions Memory Descendants' > Mem. > > --- Event Stage 0: Main Stage > > Index Set 2 2 19640 0 > Vec 694 693 299376 0 > Matrix 2 2 56400 0 > Krylov Solver 1 1 36 0 > Preconditioner 1 1 108 0 > ======================================================================================================================== > Average time to get PetscTime(): 2.86102e-07 > OptionTable: -ksp_type cg > OptionTable: -log_summary -ksp_view > OptionTable: -pc_type icc > Compiled without FORTRAN kernels > Compiled with full precision matrices (default) > sizeof(short) 2 sizeof(int) 4 sizeof(long) 4 sizeof(void*) 4 > sizeof(PetscScalar) 8 > Configure run at: Thu Mar 8 11:54:22 2007 > Configure options: --with-cc=gcc --with-fc=ifort > --download-f-blas-lapack=1 --download-mpich=1 --with-debugging=1 > --with-shared=0 > ----------------------------------------- > Libraries compiled on Thu Mar 8 12:08:22 CET 2007 on iept0415 > Machine characteristics: Linux iept0415 2.6.16.21-0.8-default #1 Mon Jul > 3 18:25:39 UTC 2006 i686 i686 i386 GNU/Linux > Using PETSc directory: /opt/petsc-2.3.2-p8 > Using PETSc arch: gcc-ifc-debug > ----------------------------------------- > Using C compiler: gcc -fPIC -Wall -Wwrite-strings -Wno-strict-aliasing > -g3 > Using Fortran compiler: ifort -fPIC -g > ----------------------------------------- > Using include paths: -I/opt/petsc-2.3.2-p8 > -I/opt/petsc-2.3.2-p8/bmake/gcc-ifc-debug -I/opt/petsc-2.3.2-p8/include > -I/opt/petsc-2.3.2-p8/externalpackages/mpich2-1.0.4p1/gcc-ifc-debug/include -I/usr/X11R6/include > ------------------------------------------ > Using C linker: gcc -fPIC -Wall -Wwrite-strings -Wno-strict-aliasing -g3 > Using Fortran linker: ifort -fPIC -g > Using libraries: -Wl,-rpath,/opt/petsc-2.3.2-p8/lib/gcc-ifc-debug > -L/opt/petsc-2.3.2-p8/lib/gcc-ifc-debug -lpetscts -lpetscsnes -lpetscksp > -lpetscdm -lpetscmat -lpetscvec -lpetsc > -Wl,-rpath,/opt/petsc-2.3.2-p8/externalpackages/mpich2-1.0.4p1/gcc-ifc-debug/lib -L/opt/petsc-2.3.2-p8/externalpackages/mpich2-1.0.4p1/gcc-ifc-debug/lib -lmpich -lnsl -lrt -L/usr/X11R6/lib -lX11 -Wl,-rpath,/opt/petsc-2.3.2-p8/externalpackages/fblaslapack/gcc-ifc-debug -L/opt/petsc-2.3.2-p8/externalpackages/fblaslapack/gcc-ifc-debug -lflapack -Wl,-rpath,/opt/petsc-2.3.2-p8/externalpackages/fblaslapack/gcc-ifc-debug -L/opt/petsc-2.3.2-p8/externalpackages/fblaslapack/gcc-ifc-debug -lfblas -lm -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0 -L/usr/lib/gcc/i586-suse-linux/4.1.0 -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linux/lib -L/usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linux/lib -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../.. -L/usr/lib/gcc/i586-suse-linux/4.1.0/../../.. -ldl -lgcc_s -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0 -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linux/lib -Wl,-rpath,/usr/lib/gcc/i5! > 86-suse-linux/4.1.0/../../.. -Wl,-rpath,/opt/intel/fc/9.1.036/lib -L/opt/intel/fc/9.1.036/lib -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/ -L/usr/lib/gcc/i586-suse-linux/4.1.0/ -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../../ -L/usr/lib/gcc/i586-suse-linux/4.1.0/../../../ -lifport -lifcore -limf -lm -lipgo -lirc -lirc_s -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0 -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linux/lib -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../.. -lm -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0 -L/usr/lib/gcc/i586-suse-linux/4.1.0 -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linux/lib -L/usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linux/lib -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../.. -L/usr/lib/gcc/i586-suse-linux/4.1.0/../../.. -ldl -lgcc_s -ldl > > > > -- Lisandro Dalc?n --------------- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 From knutert at stud.ntnu.no Sat Mar 10 11:43:16 2007 From: knutert at stud.ntnu.no (knutert at stud.ntnu.no) Date: Sat, 10 Mar 2007 18:43:16 +0100 Subject: PETSc CG solver uses more iterations than other CG solver In-Reply-To: References: <1173456720.6075.43.camel@iept0415.ivt.ntnu.no> Message-ID: <20070310184316.pirkra3plskwoc4k@webmail.ntnu.no> Thank you for your reply. One boundary cell is defined to have constant pressure, since that makes the equation system have a unique solution. I tried your command, and it lowered the number of iterations for most of the time steps, but for some it reached the maximum number of iterations (10000) without converging. I also tried making all the boundaries von Neumann and using your command. That made the number of iterations more constant, instead of varying between 700 and 2000, it stayed on around 1200. But it actually increased the average number of iterations somewhat. Still far from the performance of the other solver. I've also checked the convergence criteria, and it is the same for both solvers. Siterer Lisandro Dalcin : > On 3/9/07, Knut Erik Teigen wrote: >> To solve the Navier-Stokes equations, I use an explicit Runge-Kutta >> method with Chorin's projection method, >> so a Poisson equation with von Neumann boundary conditions for the >> pressure has to be solved at every time-step. > > All boundary conditions are Neumann type? In that case, please try to > run your program with the following command line option: > > -ksp_constant_null_space > > and let me know if this corrected your problem. > > > The equation system is >> positive definite, so I use the CG solver with the ICC preconditioner. >> The problem is that the PETSc solver seems to need a lot more iterations >> to reach the solution than another CG solver I'm using. On a small test >> problem(a rising bubble) with a 60x40 grid, the PETSc solver needs over >> 1000 iterations on average, while the other solver needs less than 100. >> I am using KSPSetInitialGuessNonzero, without this the number of >> iterations is even higher. >> I have also tried applying PETSc to a similar problem, solving the >> Poisson equation with von Neumann boundaries and a forcing function of >> f=sin(pi * x)+sin(pi *y). For this problem, the number of iterations is >> almost exactly the same for PETSc and the other solver. >> >> Does anyone know what the problem might be? Any help is greatly >> appreciated. I've included the -ksp_view of one of the time steps >> and the -log_summary below. >> >> Regards, >> Knut Erik Teigen >> MSc student >> Norwegian University of Science and Technology >> >> Output from -ksp_view: >> >> KSP Object: >> type: cg >> maximum iterations=10000 >> tolerances: relative=1e-06, absolute=1e-50, divergence=10000 >> left preconditioning >> PC Object: >> type: icc >> ICC: 0 levels of fill >> ICC: factor fill ratio allocated 1 >> ICC: factor fill ratio needed 0.601695 >> Factored matrix follows >> Matrix Object: >> type=seqsbaij, rows=2400, cols=2400 >> total: nonzeros=7100, allocated nonzeros=7100 >> block size is 1 >> linear system matrix = precond matrix: >> Matrix Object: >> type=seqaij, rows=2400, cols=2400 >> total: nonzeros=11800, allocated nonzeros=12000 >> not using I-node routines >> Poisson converged after 1403 iterations >> >> Output from -log_summary >> >> ---------------------------------------------- PETSc Performance >> Summary: ---------------------------------------------- >> >> ./run on a gcc-ifc-d named iept0415 with 1 processor, by knutert Fri Mar >> 9 17:06:05 2007 >> Using Petsc Release Version 2.3.2, Patch 8, Tue Jan 2 14:33:59 PST 2007 >> HG revision: ebeddcedcc065e32fc252af32cf1d01ed4fc7a80 >> >> Max Max/Min Avg Total >> Time (sec): 5.425e+02 1.00000 5.425e+02 >> Objects: 7.000e+02 1.00000 7.000e+02 >> Flops: 6.744e+10 1.00000 6.744e+10 6.744e+10 >> Flops/sec: 1.243e+08 1.00000 1.243e+08 1.243e+08 >> Memory: 4.881e+05 1.00000 4.881e+05 >> MPI Messages: 0.000e+00 0.00000 0.000e+00 0.000e+00 >> MPI Message Lengths: 0.000e+00 0.00000 0.000e+00 0.000e+00 >> MPI Reductions: 1.390e+03 1.00000 >> >> Flop counting convention: 1 flop = 1 real number operation of type >> (multiply/divide/add/subtract) >> e.g., VecAXPY() for real vectors of length N >> --> 2N flops >> and VecAXPY() for complex vectors of length >> N --> 8N flops >> >> Summary of Stages: ----- Time ------ ----- Flops ----- --- Messages >> --- -- Message Lengths -- -- Reductions -- >> Avg %Total Avg %Total counts % >> Total Avg %Total counts %Total >> 0: Main Stage: 5.4246e+02 100.0% 6.7437e+10 100.0% 0.000e+00 >> 0.0% 0.000e+00 0.0% 1.390e+03 100.0% >> >> ------------------------------------------------------------------------------------------------------------------------ >> See the 'Profiling' chapter of the users' manual for details on >> interpreting output. >> Phase summary info: >> Count: number of times phase was executed >> Time and Flops/sec: Max - maximum over all processors >> Ratio - ratio of maximum to minimum over all >> processors >> Mess: number of messages sent >> Avg. len: average message length >> Reduct: number of global reductions >> Global: entire computation >> Stage: stages of a computation. Set stages with PetscLogStagePush() >> and PetscLogStagePop(). >> %T - percent time in this phase %F - percent flops in this >> phase >> %M - percent messages in this phase %L - percent message >> lengths in this phase >> %R - percent reductions in this phase >> Total Mflop/s: 10e-6 * (sum of flops over all processors)/(max time >> over all processors) >> ------------------------------------------------------------------------------------------------------------------------ >> >> >> ########################################################## >> # # >> # WARNING!!! # >> # # >> # This code was compiled with a debugging option, # >> # To get timing results run config/configure.py # >> # using --with-debugging=no, the performance will # >> # be generally two or three times faster. # >> # # >> ########################################################## >> >> >> >> >> ########################################################## >> # # >> # WARNING!!! # >> # # >> # This code was run without the PreLoadBegin() # >> # macros. To get timing results we always recommend # >> # preloading. otherwise timing numbers may be # >> # meaningless. # >> ########################################################## >> >> >> Event Count Time (sec) Flops/sec >> --- Global --- --- Stage --- Total >> Max Ratio Max Ratio Max Ratio Mess Avg len >> Reduct %T %F %M %L %R %T %F %M %L %R Mflop/s >> ------------------------------------------------------------------------------------------------------------------------ >> >> --- Event Stage 0: Main Stage >> >> VecDot 1668318 1.0 1.9186e+01 1.0 4.17e+08 1.0 0.0e+00 0.0e+00 >> 0.0e+00 4 12 0 0 0 4 12 0 0 0 417 >> VecNorm 835191 1.0 1.0935e+01 1.0 3.67e+08 1.0 0.0e+00 0.0e+00 >> 0.0e+00 2 6 0 0 0 2 6 0 0 0 367 >> VecCopy 688 1.0 1.6126e-02 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 >> 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 >> VecAXPY 1667630 1.0 2.4141e+01 1.0 3.32e+08 1.0 0.0e+00 0.0e+00 >> 0.0e+00 4 12 0 0 0 4 12 0 0 0 332 >> VecAYPX 833815 1.0 1.9062e+01 1.0 2.10e+08 1.0 0.0e+00 0.0e+00 >> 0.0e+00 4 6 0 0 0 4 6 0 0 0 210 >> VecAssemblyBegin 688 1.0 8.5354e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 >> 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 >> VecAssemblyEnd 688 1.0 7.8177e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 >> 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 >> MatMult 834503 1.0 1.5044e+02 1.0 1.18e+08 1.0 0.0e+00 0.0e+00 >> 0.0e+00 28 26 0 0 0 28 26 0 0 0 118 >> MatSolve 835191 1.0 1.8130e+02 1.0 1.42e+08 1.0 0.0e+00 0.0e+00 >> 0.0e+00 33 38 0 0 0 33 38 0 0 0 142 >> MatCholFctrNum 1 1.0 1.1630e-03 1.0 2.06e+06 1.0 0.0e+00 0.0e+00 >> 0.0e+00 0 0 0 0 0 0 0 0 0 0 2 >> MatICCFactorSym 1 1.0 2.9588e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 >> 2.0e+00 0 0 0 0 0 0 0 0 0 0 0 >> MatAssemblyBegin 688 1.0 2.6343e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 >> 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 >> MatAssemblyEnd 688 1.0 1.0964e-01 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 >> 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 >> MatGetOrdering 1 1.0 2.6798e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 >> 2.0e+00 0 0 0 0 0 0 0 0 0 0 0 >> KSPSetup 1 1.0 2.7585e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 >> 6.0e+00 0 0 0 0 0 0 0 0 0 0 0 >> KSPSolve 688 1.0 4.2809e+02 1.0 1.58e+08 1.0 0.0e+00 0.0e+00 >> 1.4e+03 79100 0 0100 79100 0 0100 158 >> PCSetUp 1 1.0 1.7900e-03 1.0 1.34e+06 1.0 0.0e+00 0.0e+00 >> 4.0e+00 0 0 0 0 0 0 0 0 0 0 1 >> PCApply 835191 1.0 1.8864e+02 1.0 1.36e+08 1.0 0.0e+00 0.0e+00 >> 0.0e+00 35 38 0 0 0 35 38 0 0 0 136 >> ------------------------------------------------------------------------------------------------------------------------ >> >> Memory usage is given in bytes: >> >> Object Type Creations Destructions Memory Descendants' >> Mem. >> >> --- Event Stage 0: Main Stage >> >> Index Set 2 2 19640 0 >> Vec 694 693 299376 0 >> Matrix 2 2 56400 0 >> Krylov Solver 1 1 36 0 >> Preconditioner 1 1 108 0 >> ======================================================================================================================== >> Average time to get PetscTime(): 2.86102e-07 >> OptionTable: -ksp_type cg >> OptionTable: -log_summary -ksp_view >> OptionTable: -pc_type icc >> Compiled without FORTRAN kernels >> Compiled with full precision matrices (default) >> sizeof(short) 2 sizeof(int) 4 sizeof(long) 4 sizeof(void*) 4 >> sizeof(PetscScalar) 8 >> Configure run at: Thu Mar 8 11:54:22 2007 >> Configure options: --with-cc=gcc --with-fc=ifort >> --download-f-blas-lapack=1 --download-mpich=1 --with-debugging=1 >> --with-shared=0 >> ----------------------------------------- >> Libraries compiled on Thu Mar 8 12:08:22 CET 2007 on iept0415 >> Machine characteristics: Linux iept0415 2.6.16.21-0.8-default #1 Mon Jul >> 3 18:25:39 UTC 2006 i686 i686 i386 GNU/Linux >> Using PETSc directory: /opt/petsc-2.3.2-p8 >> Using PETSc arch: gcc-ifc-debug >> ----------------------------------------- >> Using C compiler: gcc -fPIC -Wall -Wwrite-strings -Wno-strict-aliasing >> -g3 >> Using Fortran compiler: ifort -fPIC -g >> ----------------------------------------- >> Using include paths: -I/opt/petsc-2.3.2-p8 >> -I/opt/petsc-2.3.2-p8/bmake/gcc-ifc-debug -I/opt/petsc-2.3.2-p8/include >> -I/opt/petsc-2.3.2-p8/externalpackages/mpich2-1.0.4p1/gcc-ifc-debug/include >> -I/usr/X11R6/include >> ------------------------------------------ >> Using C linker: gcc -fPIC -Wall -Wwrite-strings -Wno-strict-aliasing -g3 >> Using Fortran linker: ifort -fPIC -g >> Using libraries: -Wl,-rpath,/opt/petsc-2.3.2-p8/lib/gcc-ifc-debug >> -L/opt/petsc-2.3.2-p8/lib/gcc-ifc-debug -lpetscts -lpetscsnes -lpetscksp >> -lpetscdm -lpetscmat -lpetscvec -lpetsc >> -Wl,-rpath,/opt/petsc-2.3.2-p8/externalpackages/mpich2-1.0.4p1/gcc-ifc-debug/lib -L/opt/petsc-2.3.2-p8/externalpackages/mpich2-1.0.4p1/gcc-ifc-debug/lib -lmpich -lnsl -lrt -L/usr/X11R6/lib -lX11 -Wl,-rpath,/opt/petsc-2.3.2-p8/externalpackages/fblaslapack/gcc-ifc-debug -L/opt/petsc-2.3.2-p8/externalpackages/fblaslapack/gcc-ifc-debug -lflapack -Wl,-rpath,/opt/petsc-2.3.2-p8/externalpackages/fblaslapack/gcc-ifc-debug -L/opt/petsc-2.3.2-p8/externalpackages/fblaslapack/gcc-ifc-debug -lfblas -lm -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0 -L/usr/lib/gcc/i586-suse-linux/4.1.0 -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linux/lib -L/usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linux/lib -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../.. -L/usr/lib/gcc/i586-suse-linux/4.1.0/../../.. -ldl -lgcc_s -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0 -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linux/lib >> -Wl,-rpath,/usr/lib/gcc/! > i5! >> 86-suse-linux/4.1.0/../../.. -Wl,-rpath,/opt/intel/fc/9.1.036/lib >> -L/opt/intel/fc/9.1.036/lib >> -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/ >> -L/usr/lib/gcc/i586-suse-linux/4.1.0/ >> -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../../ >> -L/usr/lib/gcc/i586-suse-linux/4.1.0/../../../ -lifport -lifcore >> -limf -lm -lipgo -lirc -lirc_s >> -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0 >> -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linux/lib -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../.. -lm -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0 -L/usr/lib/gcc/i586-suse-linux/4.1.0 -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linux/lib -L/usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linux/lib -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../.. -L/usr/lib/gcc/i586-suse-linux/4.1.0/../../.. -ldl -lgcc_s >> -ldl >> >> >> >> > > > -- > Lisandro Dalc?n > --------------- > Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) > Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) > Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) > PTLC - G?emes 3450, (3000) Santa Fe, Argentina > Tel/Fax: +54-(0)342-451.1594 From bsmith at mcs.anl.gov Sat Mar 10 11:54:58 2007 From: bsmith at mcs.anl.gov (Barry Smith) Date: Sat, 10 Mar 2007 11:54:58 -0600 (CST) Subject: PETSc CG solver uses more iterations than other CG solver In-Reply-To: <20070310184316.pirkra3plskwoc4k@webmail.ntnu.no> References: <1173456720.6075.43.camel@iept0415.ivt.ntnu.no> <20070310184316.pirkra3plskwoc4k@webmail.ntnu.no> Message-ID: On Sat, 10 Mar 2007, knutert at stud.ntnu.no wrote: > Thank you for your reply. > > One boundary cell is defined to have constant pressure, since that makes the > equation system have a unique solution. This is really not the best way to do it; it can produce very ill-conditioned matrices. >I tried your command, and it lowered > the number of iterations for most of the time steps, but for some it reached > the maximum number of iterations (10000) without converging. > > I also tried making all the boundaries von Neumann and using your command. > That made the number of iterations more constant, instead of varying between > 700 and 2000, it stayed on around 1200. But it actually increased the average > number of iterations somewhat. Still far from the performance of the other > solver. > I've also checked the convergence criteria, and it is the same for both > solvers. There is something most definitely wrong. Are you sure your matrix is COMPLETELY symmetric? Can you send your "model" code to petsc-maint at mcs.anl.gov so we can check it out? Since you are on a structured grid eventually you'll want to use simply geometric multigrid; it is really the "right" way to solve the problem and will be much better then CG + ICC. With your code we check out how difficult it is to do this. Barry > > > Siterer Lisandro Dalcin : > > > On 3/9/07, Knut Erik Teigen wrote: > > > To solve the Navier-Stokes equations, I use an explicit Runge-Kutta > > > method with Chorin's projection method, > > > so a Poisson equation with von Neumann boundary conditions for the > > > pressure has to be solved at every time-step. > > > > All boundary conditions are Neumann type? In that case, please try to > > run your program with the following command line option: > > > > -ksp_constant_null_space > > > > and let me know if this corrected your problem. > > > > > > The equation system is > > > positive definite, so I use the CG solver with the ICC preconditioner. > > > The problem is that the PETSc solver seems to need a lot more iterations > > > to reach the solution than another CG solver I'm using. On a small test > > > problem(a rising bubble) with a 60x40 grid, the PETSc solver needs over > > > 1000 iterations on average, while the other solver needs less than 100. > > > I am using KSPSetInitialGuessNonzero, without this the number of > > > iterations is even higher. > > > I have also tried applying PETSc to a similar problem, solving the > > > Poisson equation with von Neumann boundaries and a forcing function of > > > f=sin(pi * x)+sin(pi *y). For this problem, the number of iterations is > > > almost exactly the same for PETSc and the other solver. > > > > > > Does anyone know what the problem might be? Any help is greatly > > > appreciated. I've included the -ksp_view of one of the time steps > > > and the -log_summary below. > > > > > > Regards, > > > Knut Erik Teigen > > > MSc student > > > Norwegian University of Science and Technology > > > > > > Output from -ksp_view: > > > > > > KSP Object: > > > type: cg > > > maximum iterations=10000 > > > tolerances: relative=1e-06, absolute=1e-50, divergence=10000 > > > left preconditioning > > > PC Object: > > > type: icc > > > ICC: 0 levels of fill > > > ICC: factor fill ratio allocated 1 > > > ICC: factor fill ratio needed 0.601695 > > > Factored matrix follows > > > Matrix Object: > > > type=seqsbaij, rows=2400, cols=2400 > > > total: nonzeros=7100, allocated nonzeros=7100 > > > block size is 1 > > > linear system matrix = precond matrix: > > > Matrix Object: > > > type=seqaij, rows=2400, cols=2400 > > > total: nonzeros=11800, allocated nonzeros=12000 > > > not using I-node routines > > > Poisson converged after 1403 iterations > > > > > > Output from -log_summary > > > > > > ---------------------------------------------- PETSc Performance > > > Summary: ---------------------------------------------- > > > > > > ./run on a gcc-ifc-d named iept0415 with 1 processor, by knutert Fri Mar > > > 9 17:06:05 2007 > > > Using Petsc Release Version 2.3.2, Patch 8, Tue Jan 2 14:33:59 PST 2007 > > > HG revision: ebeddcedcc065e32fc252af32cf1d01ed4fc7a80 > > > > > > Max Max/Min Avg Total > > > Time (sec): 5.425e+02 1.00000 5.425e+02 > > > Objects: 7.000e+02 1.00000 7.000e+02 > > > Flops: 6.744e+10 1.00000 6.744e+10 6.744e+10 > > > Flops/sec: 1.243e+08 1.00000 1.243e+08 1.243e+08 > > > Memory: 4.881e+05 1.00000 4.881e+05 > > > MPI Messages: 0.000e+00 0.00000 0.000e+00 0.000e+00 > > > MPI Message Lengths: 0.000e+00 0.00000 0.000e+00 0.000e+00 > > > MPI Reductions: 1.390e+03 1.00000 > > > > > > Flop counting convention: 1 flop = 1 real number operation of type > > > (multiply/divide/add/subtract) > > > e.g., VecAXPY() for real vectors of length N > > > --> 2N flops > > > and VecAXPY() for complex vectors of length > > > N --> 8N flops > > > > > > Summary of Stages: ----- Time ------ ----- Flops ----- --- Messages > > > --- -- Message Lengths -- -- Reductions -- > > > Avg %Total Avg %Total counts % > > > Total Avg %Total counts %Total > > > 0: Main Stage: 5.4246e+02 100.0% 6.7437e+10 100.0% 0.000e+00 > > > 0.0% 0.000e+00 0.0% 1.390e+03 100.0% > > > > > > ------------------------------------------------------------------------------------------------------------------------ > > > See the 'Profiling' chapter of the users' manual for details on > > > interpreting output. > > > Phase summary info: > > > Count: number of times phase was executed > > > Time and Flops/sec: Max - maximum over all processors > > > Ratio - ratio of maximum to minimum over all > > > processors > > > Mess: number of messages sent > > > Avg. len: average message length > > > Reduct: number of global reductions > > > Global: entire computation > > > Stage: stages of a computation. Set stages with PetscLogStagePush() > > > and PetscLogStagePop(). > > > %T - percent time in this phase %F - percent flops in this > > > phase > > > %M - percent messages in this phase %L - percent message > > > lengths in this phase > > > %R - percent reductions in this phase > > > Total Mflop/s: 10e-6 * (sum of flops over all processors)/(max time > > > over all processors) > > > ------------------------------------------------------------------------------------------------------------------------ > > > > > > > > > ########################################################## > > > # # > > > # WARNING!!! # > > > # # > > > # This code was compiled with a debugging option, # > > > # To get timing results run config/configure.py # > > > # using --with-debugging=no, the performance will # > > > # be generally two or three times faster. # > > > # # > > > ########################################################## > > > > > > > > > > > > > > > ########################################################## > > > # # > > > # WARNING!!! # > > > # # > > > # This code was run without the PreLoadBegin() # > > > # macros. To get timing results we always recommend # > > > # preloading. otherwise timing numbers may be # > > > # meaningless. # > > > ########################################################## > > > > > > > > > Event Count Time (sec) Flops/sec > > > --- Global --- --- Stage --- Total > > > Max Ratio Max Ratio Max Ratio Mess Avg len > > > Reduct %T %F %M %L %R %T %F %M %L %R Mflop/s > > > ------------------------------------------------------------------------------------------------------------------------ > > > > > > --- Event Stage 0: Main Stage > > > > > > VecDot 1668318 1.0 1.9186e+01 1.0 4.17e+08 1.0 0.0e+00 0.0e+00 > > > 0.0e+00 4 12 0 0 0 4 12 0 0 0 417 > > > VecNorm 835191 1.0 1.0935e+01 1.0 3.67e+08 1.0 0.0e+00 0.0e+00 > > > 0.0e+00 2 6 0 0 0 2 6 0 0 0 367 > > > VecCopy 688 1.0 1.6126e-02 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > > > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > > VecAXPY 1667630 1.0 2.4141e+01 1.0 3.32e+08 1.0 0.0e+00 0.0e+00 > > > 0.0e+00 4 12 0 0 0 4 12 0 0 0 332 > > > VecAYPX 833815 1.0 1.9062e+01 1.0 2.10e+08 1.0 0.0e+00 0.0e+00 > > > 0.0e+00 4 6 0 0 0 4 6 0 0 0 210 > > > VecAssemblyBegin 688 1.0 8.5354e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > > > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > > VecAssemblyEnd 688 1.0 7.8177e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > > > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > > MatMult 834503 1.0 1.5044e+02 1.0 1.18e+08 1.0 0.0e+00 0.0e+00 > > > 0.0e+00 28 26 0 0 0 28 26 0 0 0 118 > > > MatSolve 835191 1.0 1.8130e+02 1.0 1.42e+08 1.0 0.0e+00 0.0e+00 > > > 0.0e+00 33 38 0 0 0 33 38 0 0 0 142 > > > MatCholFctrNum 1 1.0 1.1630e-03 1.0 2.06e+06 1.0 0.0e+00 0.0e+00 > > > 0.0e+00 0 0 0 0 0 0 0 0 0 0 2 > > > MatICCFactorSym 1 1.0 2.9588e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > > > 2.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > > MatAssemblyBegin 688 1.0 2.6343e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > > > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > > MatAssemblyEnd 688 1.0 1.0964e-01 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > > > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > > MatGetOrdering 1 1.0 2.6798e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > > > 2.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > > KSPSetup 1 1.0 2.7585e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > > > 6.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > > KSPSolve 688 1.0 4.2809e+02 1.0 1.58e+08 1.0 0.0e+00 0.0e+00 > > > 1.4e+03 79100 0 0100 79100 0 0100 158 > > > PCSetUp 1 1.0 1.7900e-03 1.0 1.34e+06 1.0 0.0e+00 0.0e+00 > > > 4.0e+00 0 0 0 0 0 0 0 0 0 0 1 > > > PCApply 835191 1.0 1.8864e+02 1.0 1.36e+08 1.0 0.0e+00 0.0e+00 > > > 0.0e+00 35 38 0 0 0 35 38 0 0 0 136 > > > ------------------------------------------------------------------------------------------------------------------------ > > > > > > Memory usage is given in bytes: > > > > > > Object Type Creations Destructions Memory Descendants' > > > Mem. > > > > > > --- Event Stage 0: Main Stage > > > > > > Index Set 2 2 19640 0 > > > Vec 694 693 299376 0 > > > Matrix 2 2 56400 0 > > > Krylov Solver 1 1 36 0 > > > Preconditioner 1 1 108 0 > > > ======================================================================================================================== > > > Average time to get PetscTime(): 2.86102e-07 > > > OptionTable: -ksp_type cg > > > OptionTable: -log_summary -ksp_view > > > OptionTable: -pc_type icc > > > Compiled without FORTRAN kernels > > > Compiled with full precision matrices (default) > > > sizeof(short) 2 sizeof(int) 4 sizeof(long) 4 sizeof(void*) 4 > > > sizeof(PetscScalar) 8 > > > Configure run at: Thu Mar 8 11:54:22 2007 > > > Configure options: --with-cc=gcc --with-fc=ifort > > > --download-f-blas-lapack=1 --download-mpich=1 --with-debugging=1 > > > --with-shared=0 > > > ----------------------------------------- > > > Libraries compiled on Thu Mar 8 12:08:22 CET 2007 on iept0415 > > > Machine characteristics: Linux iept0415 2.6.16.21-0.8-default #1 Mon Jul > > > 3 18:25:39 UTC 2006 i686 i686 i386 GNU/Linux > > > Using PETSc directory: /opt/petsc-2.3.2-p8 > > > Using PETSc arch: gcc-ifc-debug > > > ----------------------------------------- > > > Using C compiler: gcc -fPIC -Wall -Wwrite-strings -Wno-strict-aliasing > > > -g3 > > > Using Fortran compiler: ifort -fPIC -g > > > ----------------------------------------- > > > Using include paths: -I/opt/petsc-2.3.2-p8 > > > -I/opt/petsc-2.3.2-p8/bmake/gcc-ifc-debug -I/opt/petsc-2.3.2-p8/include > > > -I/opt/petsc-2.3.2-p8/externalpackages/mpich2-1.0.4p1/gcc-ifc-debug/include > > > -I/usr/X11R6/include > > > ------------------------------------------ > > > Using C linker: gcc -fPIC -Wall -Wwrite-strings -Wno-strict-aliasing -g3 > > > Using Fortran linker: ifort -fPIC -g > > > Using libraries: -Wl,-rpath,/opt/petsc-2.3.2-p8/lib/gcc-ifc-debug > > > -L/opt/petsc-2.3.2-p8/lib/gcc-ifc-debug -lpetscts -lpetscsnes -lpetscksp > > > -lpetscdm -lpetscmat -lpetscvec -lpetsc > > > -Wl,-rpath,/opt/petsc-2.3.2-p8/externalpackages/mpich2-1.0.4p1/gcc-ifc-debug/lib > > > -L/opt/petsc-2.3.2-p8/externalpackages/mpich2-1.0.4p1/gcc-ifc-debug/lib > > > -lmpich -lnsl -lrt -L/usr/X11R6/lib -lX11 > > > -Wl,-rpath,/opt/petsc-2.3.2-p8/externalpackages/fblaslapack/gcc-ifc-debug > > > -L/opt/petsc-2.3.2-p8/externalpackages/fblaslapack/gcc-ifc-debug -lflapack > > > -Wl,-rpath,/opt/petsc-2.3.2-p8/externalpackages/fblaslapack/gcc-ifc-debug > > > -L/opt/petsc-2.3.2-p8/externalpackages/fblaslapack/gcc-ifc-debug -lfblas > > > -lm -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0 > > > -L/usr/lib/gcc/i586-suse-linux/4.1.0 > > > -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linux/lib > > > -L/usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linux/lib > > > -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../.. > > > -L/usr/lib/gcc/i586-suse-linux/4.1.0/../../.. -ldl -lgcc_s > > > -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0 > > > -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linux/lib > > > -Wl,-rpath,/usr/lib/gcc/! > > i5! > > > 86-suse-linux/4.1.0/../../.. -Wl,-rpath,/opt/intel/fc/9.1.036/lib > > > -L/opt/intel/fc/9.1.036/lib > > > -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/ > > > -L/usr/lib/gcc/i586-suse-linux/4.1.0/ > > > -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../../ > > > -L/usr/lib/gcc/i586-suse-linux/4.1.0/../../../ -lifport -lifcore -limf > > > -lm -lipgo -lirc -lirc_s -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0 > > > -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linux/lib > > > -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../.. -lm > > > -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0 > > > -L/usr/lib/gcc/i586-suse-linux/4.1.0 > > > -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linux/lib > > > -L/usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linux/lib > > > -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../.. > > > -L/usr/lib/gcc/i586-suse-linux/4.1.0/../../.. -ldl -lgcc_s -ldl > > > > > > > > > > > > > > > > > > -- > > Lisandro Dalc?n > > --------------- > > Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) > > Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) > > Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) > > PTLC - G?emes 3450, (3000) Santa Fe, Argentina > > Tel/Fax: +54-(0)342-451.1594 > > > From knepley at gmail.com Sat Mar 10 11:55:15 2007 From: knepley at gmail.com (Matthew Knepley) Date: Sat, 10 Mar 2007 11:55:15 -0600 Subject: PETSc CG solver uses more iterations than other CG solver In-Reply-To: <20070310184316.pirkra3plskwoc4k@webmail.ntnu.no> References: <1173456720.6075.43.camel@iept0415.ivt.ntnu.no> <20070310184316.pirkra3plskwoc4k@webmail.ntnu.no> Message-ID: Something is obviously wrong. CG has a simple definition and so does ICC(0). So I would suggest 1) Make sure you are comparing apples to apples. Usually, the other solver will have something you have not turned on, like scaling. 2) Compare the iterates exactly. I do this all the time, and it is the best way to compare differences. Matt On 3/10/07, knutert at stud.ntnu.no wrote: > Thank you for your reply. > > One boundary cell is defined to have constant pressure, since that > makes the equation system have a unique solution. I tried your > command, and it lowered the number of iterations for most of the time > steps, but for some it reached the maximum number of iterations > (10000) without converging. > > I also tried making all the boundaries von Neumann and using your > command. That made the number of iterations more constant, instead of > varying between 700 and 2000, it stayed on around 1200. But it > actually increased the average number of iterations somewhat. Still > far from the performance of the other solver. > I've also checked the convergence criteria, and it is the same for > both solvers. > > > Siterer Lisandro Dalcin : > > > On 3/9/07, Knut Erik Teigen wrote: > >> To solve the Navier-Stokes equations, I use an explicit Runge-Kutta > >> method with Chorin's projection method, > >> so a Poisson equation with von Neumann boundary conditions for the > >> pressure has to be solved at every time-step. > > > > All boundary conditions are Neumann type? In that case, please try to > > run your program with the following command line option: > > > > -ksp_constant_null_space > > > > and let me know if this corrected your problem. > > > > > > The equation system is > >> positive definite, so I use the CG solver with the ICC preconditioner. > >> The problem is that the PETSc solver seems to need a lot more iterations > >> to reach the solution than another CG solver I'm using. On a small test > >> problem(a rising bubble) with a 60x40 grid, the PETSc solver needs over > >> 1000 iterations on average, while the other solver needs less than 100. > >> I am using KSPSetInitialGuessNonzero, without this the number of > >> iterations is even higher. > >> I have also tried applying PETSc to a similar problem, solving the > >> Poisson equation with von Neumann boundaries and a forcing function of > >> f=sin(pi * x)+sin(pi *y). For this problem, the number of iterations is > >> almost exactly the same for PETSc and the other solver. > >> > >> Does anyone know what the problem might be? Any help is greatly > >> appreciated. I've included the -ksp_view of one of the time steps > >> and the -log_summary below. > >> > >> Regards, > >> Knut Erik Teigen > >> MSc student > >> Norwegian University of Science and Technology > >> > >> Output from -ksp_view: > >> > >> KSP Object: > >> type: cg > >> maximum iterations=10000 > >> tolerances: relative=1e-06, absolute=1e-50, divergence=10000 > >> left preconditioning > >> PC Object: > >> type: icc > >> ICC: 0 levels of fill > >> ICC: factor fill ratio allocated 1 > >> ICC: factor fill ratio needed 0.601695 > >> Factored matrix follows > >> Matrix Object: > >> type=seqsbaij, rows=2400, cols=2400 > >> total: nonzeros=7100, allocated nonzeros=7100 > >> block size is 1 > >> linear system matrix = precond matrix: > >> Matrix Object: > >> type=seqaij, rows=2400, cols=2400 > >> total: nonzeros=11800, allocated nonzeros=12000 > >> not using I-node routines > >> Poisson converged after 1403 iterations > >> > >> Output from -log_summary > >> > >> ---------------------------------------------- PETSc Performance > >> Summary: ---------------------------------------------- > >> > >> ./run on a gcc-ifc-d named iept0415 with 1 processor, by knutert Fri Mar > >> 9 17:06:05 2007 > >> Using Petsc Release Version 2.3.2, Patch 8, Tue Jan 2 14:33:59 PST 2007 > >> HG revision: ebeddcedcc065e32fc252af32cf1d01ed4fc7a80 > >> > >> Max Max/Min Avg Total > >> Time (sec): 5.425e+02 1.00000 5.425e+02 > >> Objects: 7.000e+02 1.00000 7.000e+02 > >> Flops: 6.744e+10 1.00000 6.744e+10 6.744e+10 > >> Flops/sec: 1.243e+08 1.00000 1.243e+08 1.243e+08 > >> Memory: 4.881e+05 1.00000 4.881e+05 > >> MPI Messages: 0.000e+00 0.00000 0.000e+00 0.000e+00 > >> MPI Message Lengths: 0.000e+00 0.00000 0.000e+00 0.000e+00 > >> MPI Reductions: 1.390e+03 1.00000 > >> > >> Flop counting convention: 1 flop = 1 real number operation of type > >> (multiply/divide/add/subtract) > >> e.g., VecAXPY() for real vectors of length N > >> --> 2N flops > >> and VecAXPY() for complex vectors of length > >> N --> 8N flops > >> > >> Summary of Stages: ----- Time ------ ----- Flops ----- --- Messages > >> --- -- Message Lengths -- -- Reductions -- > >> Avg %Total Avg %Total counts % > >> Total Avg %Total counts %Total > >> 0: Main Stage: 5.4246e+02 100.0% 6.7437e+10 100.0% 0.000e+00 > >> 0.0% 0.000e+00 0.0% 1.390e+03 100.0% > >> > >> ------------------------------------------------------------------------------------------------------------------------ > >> See the 'Profiling' chapter of the users' manual for details on > >> interpreting output. > >> Phase summary info: > >> Count: number of times phase was executed > >> Time and Flops/sec: Max - maximum over all processors > >> Ratio - ratio of maximum to minimum over all > >> processors > >> Mess: number of messages sent > >> Avg. len: average message length > >> Reduct: number of global reductions > >> Global: entire computation > >> Stage: stages of a computation. Set stages with PetscLogStagePush() > >> and PetscLogStagePop(). > >> %T - percent time in this phase %F - percent flops in this > >> phase > >> %M - percent messages in this phase %L - percent message > >> lengths in this phase > >> %R - percent reductions in this phase > >> Total Mflop/s: 10e-6 * (sum of flops over all processors)/(max time > >> over all processors) > >> ------------------------------------------------------------------------------------------------------------------------ > >> > >> > >> ########################################################## > >> # # > >> # WARNING!!! # > >> # # > >> # This code was compiled with a debugging option, # > >> # To get timing results run config/configure.py # > >> # using --with-debugging=no, the performance will # > >> # be generally two or three times faster. # > >> # # > >> ########################################################## > >> > >> > >> > >> > >> ########################################################## > >> # # > >> # WARNING!!! # > >> # # > >> # This code was run without the PreLoadBegin() # > >> # macros. To get timing results we always recommend # > >> # preloading. otherwise timing numbers may be # > >> # meaningless. # > >> ########################################################## > >> > >> > >> Event Count Time (sec) Flops/sec > >> --- Global --- --- Stage --- Total > >> Max Ratio Max Ratio Max Ratio Mess Avg len > >> Reduct %T %F %M %L %R %T %F %M %L %R Mflop/s > >> ------------------------------------------------------------------------------------------------------------------------ > >> > >> --- Event Stage 0: Main Stage > >> > >> VecDot 1668318 1.0 1.9186e+01 1.0 4.17e+08 1.0 0.0e+00 0.0e+00 > >> 0.0e+00 4 12 0 0 0 4 12 0 0 0 417 > >> VecNorm 835191 1.0 1.0935e+01 1.0 3.67e+08 1.0 0.0e+00 0.0e+00 > >> 0.0e+00 2 6 0 0 0 2 6 0 0 0 367 > >> VecCopy 688 1.0 1.6126e-02 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > >> 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > >> VecAXPY 1667630 1.0 2.4141e+01 1.0 3.32e+08 1.0 0.0e+00 0.0e+00 > >> 0.0e+00 4 12 0 0 0 4 12 0 0 0 332 > >> VecAYPX 833815 1.0 1.9062e+01 1.0 2.10e+08 1.0 0.0e+00 0.0e+00 > >> 0.0e+00 4 6 0 0 0 4 6 0 0 0 210 > >> VecAssemblyBegin 688 1.0 8.5354e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > >> 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > >> VecAssemblyEnd 688 1.0 7.8177e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > >> 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > >> MatMult 834503 1.0 1.5044e+02 1.0 1.18e+08 1.0 0.0e+00 0.0e+00 > >> 0.0e+00 28 26 0 0 0 28 26 0 0 0 118 > >> MatSolve 835191 1.0 1.8130e+02 1.0 1.42e+08 1.0 0.0e+00 0.0e+00 > >> 0.0e+00 33 38 0 0 0 33 38 0 0 0 142 > >> MatCholFctrNum 1 1.0 1.1630e-03 1.0 2.06e+06 1.0 0.0e+00 0.0e+00 > >> 0.0e+00 0 0 0 0 0 0 0 0 0 0 2 > >> MatICCFactorSym 1 1.0 2.9588e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > >> 2.0e+00 0 0 0 0 0 0 0 0 0 0 0 > >> MatAssemblyBegin 688 1.0 2.6343e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > >> 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > >> MatAssemblyEnd 688 1.0 1.0964e-01 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > >> 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > >> MatGetOrdering 1 1.0 2.6798e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > >> 2.0e+00 0 0 0 0 0 0 0 0 0 0 0 > >> KSPSetup 1 1.0 2.7585e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > >> 6.0e+00 0 0 0 0 0 0 0 0 0 0 0 > >> KSPSolve 688 1.0 4.2809e+02 1.0 1.58e+08 1.0 0.0e+00 0.0e+00 > >> 1.4e+03 79100 0 0100 79100 0 0100 158 > >> PCSetUp 1 1.0 1.7900e-03 1.0 1.34e+06 1.0 0.0e+00 0.0e+00 > >> 4.0e+00 0 0 0 0 0 0 0 0 0 0 1 > >> PCApply 835191 1.0 1.8864e+02 1.0 1.36e+08 1.0 0.0e+00 0.0e+00 > >> 0.0e+00 35 38 0 0 0 35 38 0 0 0 136 > >> ------------------------------------------------------------------------------------------------------------------------ > >> > >> Memory usage is given in bytes: > >> > >> Object Type Creations Destructions Memory Descendants' > >> Mem. > >> > >> --- Event Stage 0: Main Stage > >> > >> Index Set 2 2 19640 0 > >> Vec 694 693 299376 0 > >> Matrix 2 2 56400 0 > >> Krylov Solver 1 1 36 0 > >> Preconditioner 1 1 108 0 > >> ======================================================================================================================== > >> Average time to get PetscTime(): 2.86102e-07 > >> OptionTable: -ksp_type cg > >> OptionTable: -log_summary -ksp_view > >> OptionTable: -pc_type icc > >> Compiled without FORTRAN kernels > >> Compiled with full precision matrices (default) > >> sizeof(short) 2 sizeof(int) 4 sizeof(long) 4 sizeof(void*) 4 > >> sizeof(PetscScalar) 8 > >> Configure run at: Thu Mar 8 11:54:22 2007 > >> Configure options: --with-cc=gcc --with-fc=ifort > >> --download-f-blas-lapack=1 --download-mpich=1 --with-debugging=1 > >> --with-shared=0 > >> ----------------------------------------- > >> Libraries compiled on Thu Mar 8 12:08:22 CET 2007 on iept0415 > >> Machine characteristics: Linux iept0415 2.6.16.21-0.8-default #1 Mon Jul > >> 3 18:25:39 UTC 2006 i686 i686 i386 GNU/Linux > >> Using PETSc directory: /opt/petsc-2.3.2-p8 > >> Using PETSc arch: gcc-ifc-debug > >> ----------------------------------------- > >> Using C compiler: gcc -fPIC -Wall -Wwrite-strings -Wno-strict-aliasing > >> -g3 > >> Using Fortran compiler: ifort -fPIC -g > >> ----------------------------------------- > >> Using include paths: -I/opt/petsc-2.3.2-p8 > >> -I/opt/petsc-2.3.2-p8/bmake/gcc-ifc-debug -I/opt/petsc-2.3.2-p8/include > >> -I/opt/petsc-2.3.2-p8/externalpackages/mpich2-1.0.4p1/gcc-ifc-debug/include > >> -I/usr/X11R6/include > >> ------------------------------------------ > >> Using C linker: gcc -fPIC -Wall -Wwrite-strings -Wno-strict-aliasing -g3 > >> Using Fortran linker: ifort -fPIC -g > >> Using libraries: -Wl,-rpath,/opt/petsc-2.3.2-p8/lib/gcc-ifc-debug > >> -L/opt/petsc-2.3.2-p8/lib/gcc-ifc-debug -lpetscts -lpetscsnes -lpetscksp > >> -lpetscdm -lpetscmat -lpetscvec -lpetsc > >> -Wl,-rpath,/opt/petsc-2.3.2-p8/externalpackages/mpich2-1.0.4p1/gcc-ifc-debug/lib -L/opt/petsc-2.3.2-p8/externalpackages/mpich2-1.0.4p1/gcc-ifc-debug/lib -lmpich -lnsl -lrt -L/usr/X11R6/lib -lX11 -Wl,-rpath,/opt/petsc-2.3.2-p8/externalpackages/fblaslapack/gcc-ifc-debug -L/opt/petsc-2.3.2-p8/externalpackages/fblaslapack/gcc-ifc-debug -lflapack -Wl,-rpath,/opt/petsc-2.3.2-p8/externalpackages/fblaslapack/gcc-ifc-debug -L/opt/petsc-2.3.2-p8/externalpackages/fblaslapack/gcc-ifc-debug -lfblas -lm -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0 -L/usr/lib/gcc/i586-suse-linux/4.1.0 -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linux/lib -L/usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linux/lib -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../.. -L/usr/lib/gcc/i586-suse-linux/4.1.0/../../.. -ldl -lgcc_s -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0 -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linux/lib > >> -Wl,-rpath,/usr/lib/gcc/! > > i5! > >> 86-suse-linux/4.1.0/../../.. -Wl,-rpath,/opt/intel/fc/9.1.036/lib > >> -L/opt/intel/fc/9.1.036/lib > >> -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/ > >> -L/usr/lib/gcc/i586-suse-linux/4.1.0/ > >> -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../../ > >> -L/usr/lib/gcc/i586-suse-linux/4.1.0/../../../ -lifport -lifcore > >> -limf -lm -lipgo -lirc -lirc_s > >> -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0 > >> -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linux/lib -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../.. -lm -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0 -L/usr/lib/gcc/i586-suse-linux/4.1.0 -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linux/lib -L/usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linux/lib -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../.. -L/usr/lib/gcc/i586-suse-linux/4.1.0/../../.. -ldl -lgcc_s > >> -ldl > >> > >> > >> > >> > > > > > > -- > > Lisandro Dalc?n > > --------------- > > Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) > > Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) > > Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) > > PTLC - G?emes 3450, (3000) Santa Fe, Argentina > > Tel/Fax: +54-(0)342-451.1594 > > > > -- One trouble is that despite this system, anyone who reads journals widely and critically is forced to realize that there are scarcely any bars to eventual publication. There seems to be no study too fragmented, no hypothesis too trivial, no literature citation too biased or too egotistical, no design too warped, no methodology too bungled, no presentation of results too inaccurate, too obscure, and too contradictory, no analysis too self-serving, no argument too circular, no conclusions too trifling or too unjustified, and no grammar and syntax too offensive for a paper to end up in print. -- Drummond Rennie From berend at chalmers.se Sat Mar 10 14:39:28 2007 From: berend at chalmers.se (Berend van Wachem) Date: Sat, 10 Mar 2007 21:39:28 +0100 Subject: PETSc CG solver uses more iterations than other CG solver In-Reply-To: <20070310184316.pirkra3plskwoc4k@webmail.ntnu.no> References: <1173456720.6075.43.camel@iept0415.ivt.ntnu.no> <20070310184316.pirkra3plskwoc4k@webmail.ntnu.no> Message-ID: <200703102139.28725.berend@chalmers.se> Hi, I have also used the VOF/level set model in our PETSc based code, and it does require a lot more iterations than a single phase model because of the stencil. I guess you are solving d/dx 1/rho d/dx p = 0 ? Which can result in a terrible coefficient structure. Are you sure the solver/preconditioner you use with PETSc are the same as in your other code? I think for the problem size, you are having an awful lot of iterations! Are you sure the coefficients are put in the right location in the matrix? Berend. > > One boundary cell is defined to have constant pressure, since that > makes the equation system have a unique solution. I tried your > command, and it lowered the number of iterations for most of the time > steps, but for some it reached the maximum number of iterations > (10000) without converging. > > I also tried making all the boundaries von Neumann and using your > command. That made the number of iterations more constant, instead of > varying between 700 and 2000, it stayed on around 1200. But it > actually increased the average number of iterations somewhat. Still > far from the performance of the other solver. > I've also checked the convergence criteria, and it is the same for > both solvers. > > Siterer Lisandro Dalcin : > > On 3/9/07, Knut Erik Teigen wrote: > >> To solve the Navier-Stokes equations, I use an explicit Runge-Kutta > >> method with Chorin's projection method, > >> so a Poisson equation with von Neumann boundary conditions for the > >> pressure has to be solved at every time-step. > > > > All boundary conditions are Neumann type? In that case, please try to > > run your program with the following command line option: > > > > -ksp_constant_null_space > > > > and let me know if this corrected your problem. > > > > > > The equation system is > > > >> positive definite, so I use the CG solver with the ICC preconditioner. > >> The problem is that the PETSc solver seems to need a lot more iterations > >> to reach the solution than another CG solver I'm using. On a small test > >> problem(a rising bubble) with a 60x40 grid, the PETSc solver needs over > >> 1000 iterations on average, while the other solver needs less than 100. > >> I am using KSPSetInitialGuessNonzero, without this the number of > >> iterations is even higher. > >> I have also tried applying PETSc to a similar problem, solving the > >> Poisson equation with von Neumann boundaries and a forcing function of > >> f=sin(pi * x)+sin(pi *y). For this problem, the number of iterations is > >> almost exactly the same for PETSc and the other solver. > >> > >> Does anyone know what the problem might be? Any help is greatly > >> appreciated. I've included the -ksp_view of one of the time steps > >> and the -log_summary below. > >> > >> Regards, > >> Knut Erik Teigen > >> MSc student > >> Norwegian University of Science and Technology > >> > >> Output from -ksp_view: > >> > >> KSP Object: > >> type: cg > >> maximum iterations=10000 > >> tolerances: relative=1e-06, absolute=1e-50, divergence=10000 > >> left preconditioning > >> PC Object: > >> type: icc > >> ICC: 0 levels of fill > >> ICC: factor fill ratio allocated 1 > >> ICC: factor fill ratio needed 0.601695 > >> Factored matrix follows > >> Matrix Object: > >> type=seqsbaij, rows=2400, cols=2400 > >> total: nonzeros=7100, allocated nonzeros=7100 > >> block size is 1 > >> linear system matrix = precond matrix: > >> Matrix Object: > >> type=seqaij, rows=2400, cols=2400 > >> total: nonzeros=11800, allocated nonzeros=12000 > >> not using I-node routines > >> Poisson converged after 1403 iterations > >> > >> Output from -log_summary > >> > >> ---------------------------------------------- PETSc Performance > >> Summary: ---------------------------------------------- > >> > >> ./run on a gcc-ifc-d named iept0415 with 1 processor, by knutert Fri Mar > >> 9 17:06:05 2007 > >> Using Petsc Release Version 2.3.2, Patch 8, Tue Jan 2 14:33:59 PST 2007 > >> HG revision: ebeddcedcc065e32fc252af32cf1d01ed4fc7a80 > >> > >> Max Max/Min Avg Total > >> Time (sec): 5.425e+02 1.00000 5.425e+02 > >> Objects: 7.000e+02 1.00000 7.000e+02 > >> Flops: 6.744e+10 1.00000 6.744e+10 6.744e+10 > >> Flops/sec: 1.243e+08 1.00000 1.243e+08 1.243e+08 > >> Memory: 4.881e+05 1.00000 4.881e+05 > >> MPI Messages: 0.000e+00 0.00000 0.000e+00 0.000e+00 > >> MPI Message Lengths: 0.000e+00 0.00000 0.000e+00 0.000e+00 > >> MPI Reductions: 1.390e+03 1.00000 > >> > >> Flop counting convention: 1 flop = 1 real number operation of type > >> (multiply/divide/add/subtract) > >> e.g., VecAXPY() for real vectors of length N > >> --> 2N flops > >> and VecAXPY() for complex vectors of length > >> N --> 8N flops > >> > >> Summary of Stages: ----- Time ------ ----- Flops ----- --- Messages > >> --- -- Message Lengths -- -- Reductions -- > >> Avg %Total Avg %Total counts % > >> Total Avg %Total counts %Total > >> 0: Main Stage: 5.4246e+02 100.0% 6.7437e+10 100.0% 0.000e+00 > >> 0.0% 0.000e+00 0.0% 1.390e+03 100.0% > >> > >> ------------------------------------------------------------------------ > >>------------------------------------------------ See the 'Profiling' > >> chapter of the users' manual for details on interpreting output. > >> Phase summary info: > >> Count: number of times phase was executed > >> Time and Flops/sec: Max - maximum over all processors > >> Ratio - ratio of maximum to minimum over all > >> processors > >> Mess: number of messages sent > >> Avg. len: average message length > >> Reduct: number of global reductions > >> Global: entire computation > >> Stage: stages of a computation. Set stages with PetscLogStagePush() > >> and PetscLogStagePop(). > >> %T - percent time in this phase %F - percent flops in this > >> phase > >> %M - percent messages in this phase %L - percent message > >> lengths in this phase > >> %R - percent reductions in this phase > >> Total Mflop/s: 10e-6 * (sum of flops over all processors)/(max time > >> over all processors) > >> ------------------------------------------------------------------------ > >>------------------------------------------------ > >> > >> > >> ########################################################## > >> # # > >> # WARNING!!! # > >> # # > >> # This code was compiled with a debugging option, # > >> # To get timing results run config/configure.py # > >> # using --with-debugging=no, the performance will # > >> # be generally two or three times faster. # > >> # # > >> ########################################################## > >> > >> > >> > >> > >> ########################################################## > >> # # > >> # WARNING!!! # > >> # # > >> # This code was run without the PreLoadBegin() # > >> # macros. To get timing results we always recommend # > >> # preloading. otherwise timing numbers may be # > >> # meaningless. # > >> ########################################################## > >> > >> > >> Event Count Time (sec) Flops/sec > >> --- Global --- --- Stage --- Total > >> Max Ratio Max Ratio Max Ratio Mess Avg len > >> Reduct %T %F %M %L %R %T %F %M %L %R Mflop/s > >> ------------------------------------------------------------------------ > >>------------------------------------------------ > >> > >> --- Event Stage 0: Main Stage > >> > >> VecDot 1668318 1.0 1.9186e+01 1.0 4.17e+08 1.0 0.0e+00 0.0e+00 > >> 0.0e+00 4 12 0 0 0 4 12 0 0 0 417 > >> VecNorm 835191 1.0 1.0935e+01 1.0 3.67e+08 1.0 0.0e+00 0.0e+00 > >> 0.0e+00 2 6 0 0 0 2 6 0 0 0 367 > >> VecCopy 688 1.0 1.6126e-02 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > >> 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > >> VecAXPY 1667630 1.0 2.4141e+01 1.0 3.32e+08 1.0 0.0e+00 0.0e+00 > >> 0.0e+00 4 12 0 0 0 4 12 0 0 0 332 > >> VecAYPX 833815 1.0 1.9062e+01 1.0 2.10e+08 1.0 0.0e+00 0.0e+00 > >> 0.0e+00 4 6 0 0 0 4 6 0 0 0 210 > >> VecAssemblyBegin 688 1.0 8.5354e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > >> 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > >> VecAssemblyEnd 688 1.0 7.8177e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > >> 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > >> MatMult 834503 1.0 1.5044e+02 1.0 1.18e+08 1.0 0.0e+00 0.0e+00 > >> 0.0e+00 28 26 0 0 0 28 26 0 0 0 118 > >> MatSolve 835191 1.0 1.8130e+02 1.0 1.42e+08 1.0 0.0e+00 0.0e+00 > >> 0.0e+00 33 38 0 0 0 33 38 0 0 0 142 > >> MatCholFctrNum 1 1.0 1.1630e-03 1.0 2.06e+06 1.0 0.0e+00 0.0e+00 > >> 0.0e+00 0 0 0 0 0 0 0 0 0 0 2 > >> MatICCFactorSym 1 1.0 2.9588e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > >> 2.0e+00 0 0 0 0 0 0 0 0 0 0 0 > >> MatAssemblyBegin 688 1.0 2.6343e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > >> 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > >> MatAssemblyEnd 688 1.0 1.0964e-01 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > >> 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > >> MatGetOrdering 1 1.0 2.6798e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > >> 2.0e+00 0 0 0 0 0 0 0 0 0 0 0 > >> KSPSetup 1 1.0 2.7585e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > >> 6.0e+00 0 0 0 0 0 0 0 0 0 0 0 > >> KSPSolve 688 1.0 4.2809e+02 1.0 1.58e+08 1.0 0.0e+00 0.0e+00 > >> 1.4e+03 79100 0 0100 79100 0 0100 158 > >> PCSetUp 1 1.0 1.7900e-03 1.0 1.34e+06 1.0 0.0e+00 0.0e+00 > >> 4.0e+00 0 0 0 0 0 0 0 0 0 0 1 > >> PCApply 835191 1.0 1.8864e+02 1.0 1.36e+08 1.0 0.0e+00 0.0e+00 > >> 0.0e+00 35 38 0 0 0 35 38 0 0 0 136 > >> ------------------------------------------------------------------------ > >>------------------------------------------------ > >> > >> Memory usage is given in bytes: > >> > >> Object Type Creations Destructions Memory Descendants' > >> Mem. > >> > >> --- Event Stage 0: Main Stage > >> > >> Index Set 2 2 19640 0 > >> Vec 694 693 299376 0 > >> Matrix 2 2 56400 0 > >> Krylov Solver 1 1 36 0 > >> Preconditioner 1 1 108 0 > >> ======================================================================== > >>================================================ Average time to get > >> PetscTime(): 2.86102e-07 > >> OptionTable: -ksp_type cg > >> OptionTable: -log_summary -ksp_view > >> OptionTable: -pc_type icc > >> Compiled without FORTRAN kernels > >> Compiled with full precision matrices (default) > >> sizeof(short) 2 sizeof(int) 4 sizeof(long) 4 sizeof(void*) 4 > >> sizeof(PetscScalar) 8 > >> Configure run at: Thu Mar 8 11:54:22 2007 > >> Configure options: --with-cc=gcc --with-fc=ifort > >> --download-f-blas-lapack=1 --download-mpich=1 --with-debugging=1 > >> --with-shared=0 > >> ----------------------------------------- > >> Libraries compiled on Thu Mar 8 12:08:22 CET 2007 on iept0415 > >> Machine characteristics: Linux iept0415 2.6.16.21-0.8-default #1 Mon Jul > >> 3 18:25:39 UTC 2006 i686 i686 i386 GNU/Linux > >> Using PETSc directory: /opt/petsc-2.3.2-p8 > >> Using PETSc arch: gcc-ifc-debug > >> ----------------------------------------- > >> Using C compiler: gcc -fPIC -Wall -Wwrite-strings -Wno-strict-aliasing > >> -g3 > >> Using Fortran compiler: ifort -fPIC -g > >> ----------------------------------------- > >> Using include paths: -I/opt/petsc-2.3.2-p8 > >> -I/opt/petsc-2.3.2-p8/bmake/gcc-ifc-debug -I/opt/petsc-2.3.2-p8/include > >> -I/opt/petsc-2.3.2-p8/externalpackages/mpich2-1.0.4p1/gcc-ifc-debug/incl > >>ude -I/usr/X11R6/include > >> ------------------------------------------ > >> Using C linker: gcc -fPIC -Wall -Wwrite-strings -Wno-strict-aliasing -g3 > >> Using Fortran linker: ifort -fPIC -g > >> Using libraries: -Wl,-rpath,/opt/petsc-2.3.2-p8/lib/gcc-ifc-debug > >> -L/opt/petsc-2.3.2-p8/lib/gcc-ifc-debug -lpetscts -lpetscsnes -lpetscksp > >> -lpetscdm -lpetscmat -lpetscvec -lpetsc > >> -Wl,-rpath,/opt/petsc-2.3.2-p8/externalpackages/mpich2-1.0.4p1/gcc-ifc-d > >>ebug/lib > >> -L/opt/petsc-2.3.2-p8/externalpackages/mpich2-1.0.4p1/gcc-ifc-debug/lib > >> -lmpich -lnsl -lrt -L/usr/X11R6/lib -lX11 > >> -Wl,-rpath,/opt/petsc-2.3.2-p8/externalpackages/fblaslapack/gcc-ifc-debu > >>g -L/opt/petsc-2.3.2-p8/externalpackages/fblaslapack/gcc-ifc-debug > >> -lflapack > >> -Wl,-rpath,/opt/petsc-2.3.2-p8/externalpackages/fblaslapack/gcc-ifc-debu > >>g -L/opt/petsc-2.3.2-p8/externalpackages/fblaslapack/gcc-ifc-debug > >> -lfblas -lm -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0 > >> -L/usr/lib/gcc/i586-suse-linux/4.1.0 > >> -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linu > >>x/lib > >> -L/usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linux/lib > >> -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../.. > >> -L/usr/lib/gcc/i586-suse-linux/4.1.0/../../.. -ldl -lgcc_s > >> -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0 > >> -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linu > >>x/lib -Wl,-rpath,/usr/lib/gcc/! > > > > i5! > > > >> 86-suse-linux/4.1.0/../../.. -Wl,-rpath,/opt/intel/fc/9.1.036/lib > >> -L/opt/intel/fc/9.1.036/lib > >> -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/ > >> -L/usr/lib/gcc/i586-suse-linux/4.1.0/ > >> -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../../ > >> -L/usr/lib/gcc/i586-suse-linux/4.1.0/../../../ -lifport -lifcore > >> -limf -lm -lipgo -lirc -lirc_s > >> -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0 > >> -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linu > >>x/lib -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../.. -lm > >> -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0 > >> -L/usr/lib/gcc/i586-suse-linux/4.1.0 > >> -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linu > >>x/lib > >> -L/usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linux/lib > >> -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../.. > >> -L/usr/lib/gcc/i586-suse-linux/4.1.0/../../.. -ldl -lgcc_s -ldl > > > > -- > > Lisandro Dalc?n > > --------------- > > Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) > > Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) > > Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) > > PTLC - G?emes 3450, (3000) Santa Fe, Argentina > > Tel/Fax: +54-(0)342-451.1594 From knutert at stud.ntnu.no Mon Mar 12 02:37:25 2007 From: knutert at stud.ntnu.no (Knut Erik Teigen) Date: Mon, 12 Mar 2007 08:37:25 +0100 Subject: PETSc CG solver uses more iterations than other CG solver In-Reply-To: References: <1173456720.6075.43.camel@iept0415.ivt.ntnu.no> <20070310184316.pirkra3plskwoc4k@webmail.ntnu.no> Message-ID: <1173685045.6075.55.camel@iept0415.ivt.ntnu.no> On Sat, 2007-03-10 at 11:54 -0600, Barry Smith wrote: > > > On Sat, 10 Mar 2007, knutert at stud.ntnu.no wrote: > > > Thank you for your reply. > > > > One boundary cell is defined to have constant pressure, since that makes the > > equation system have a unique solution. > > This is really not the best way to do it; it can produce very ill-conditioned > matrices. > So the best way is to use von Neumann on all boundaries, and then use -ksp_constant_nullspace? > >I tried your command, and it lowered > > the number of iterations for most of the time steps, but for some it reached > > the maximum number of iterations (10000) without converging. > > > > I also tried making all the boundaries von Neumann and using your command. > > That made the number of iterations more constant, instead of varying between > > 700 and 2000, it stayed on around 1200. But it actually increased the average > > number of iterations somewhat. Still far from the performance of the other > > solver. > > I've also checked the convergence criteria, and it is the same for both > > solvers. > > There is something most definitely wrong. Are you sure your matrix is > COMPLETELY symmetric? > Yes, I've controlled it several times. > Can you send your "model" code to petsc-maint at mcs.anl.gov so we can check it > out? > Done. Hope you can find the problem. > Since you are on a structured grid eventually you'll want to use simply > geometric multigrid; it is really the "right" way to solve the problem > and will be much better then CG + ICC. With your code we check out how > difficult it is to do this. > I am currently working my way through the tutorial programs on multigrid, so hopefully I'm getting there. Thanks a lot for the help! -Knut Erik- > Barry > > > > > > Siterer Lisandro Dalcin : > > > > > On 3/9/07, Knut Erik Teigen wrote: > > > > To solve the Navier-Stokes equations, I use an explicit Runge-Kutta > > > > method with Chorin's projection method, > > > > so a Poisson equation with von Neumann boundary conditions for the > > > > pressure has to be solved at every time-step. > > > > > > All boundary conditions are Neumann type? In that case, please try to > > > run your program with the following command line option: > > > > > > -ksp_constant_null_space > > > > > > and let me know if this corrected your problem. > > > > > > > > > The equation system is > > > > positive definite, so I use the CG solver with the ICC preconditioner. > > > > The problem is that the PETSc solver seems to need a lot more iterations > > > > to reach the solution than another CG solver I'm using. On a small test > > > > problem(a rising bubble) with a 60x40 grid, the PETSc solver needs over > > > > 1000 iterations on average, while the other solver needs less than 100. > > > > I am using KSPSetInitialGuessNonzero, without this the number of > > > > iterations is even higher. > > > > I have also tried applying PETSc to a similar problem, solving the > > > > Poisson equation with von Neumann boundaries and a forcing function of > > > > f=sin(pi * x)+sin(pi *y). For this problem, the number of iterations is > > > > almost exactly the same for PETSc and the other solver. > > > > > > > > Does anyone know what the problem might be? Any help is greatly > > > > appreciated. I've included the -ksp_view of one of the time steps > > > > and the -log_summary below. > > > > > > > > Regards, > > > > Knut Erik Teigen > > > > MSc student > > > > Norwegian University of Science and Technology > > > > > > > > Output from -ksp_view: > > > > > > > > KSP Object: > > > > type: cg > > > > maximum iterations=10000 > > > > tolerances: relative=1e-06, absolute=1e-50, divergence=10000 > > > > left preconditioning > > > > PC Object: > > > > type: icc > > > > ICC: 0 levels of fill > > > > ICC: factor fill ratio allocated 1 > > > > ICC: factor fill ratio needed 0.601695 > > > > Factored matrix follows > > > > Matrix Object: > > > > type=seqsbaij, rows=2400, cols=2400 > > > > total: nonzeros=7100, allocated nonzeros=7100 > > > > block size is 1 > > > > linear system matrix = precond matrix: > > > > Matrix Object: > > > > type=seqaij, rows=2400, cols=2400 > > > > total: nonzeros=11800, allocated nonzeros=12000 > > > > not using I-node routines > > > > Poisson converged after 1403 iterations > > > > > > > > Output from -log_summary > > > > > > > > ---------------------------------------------- PETSc Performance > > > > Summary: ---------------------------------------------- > > > > > > > > ./run on a gcc-ifc-d named iept0415 with 1 processor, by knutert Fri Mar > > > > 9 17:06:05 2007 > > > > Using Petsc Release Version 2.3.2, Patch 8, Tue Jan 2 14:33:59 PST 2007 > > > > HG revision: ebeddcedcc065e32fc252af32cf1d01ed4fc7a80 > > > > > > > > Max Max/Min Avg Total > > > > Time (sec): 5.425e+02 1.00000 5.425e+02 > > > > Objects: 7.000e+02 1.00000 7.000e+02 > > > > Flops: 6.744e+10 1.00000 6.744e+10 6.744e+10 > > > > Flops/sec: 1.243e+08 1.00000 1.243e+08 1.243e+08 > > > > Memory: 4.881e+05 1.00000 4.881e+05 > > > > MPI Messages: 0.000e+00 0.00000 0.000e+00 0.000e+00 > > > > MPI Message Lengths: 0.000e+00 0.00000 0.000e+00 0.000e+00 > > > > MPI Reductions: 1.390e+03 1.00000 > > > > > > > > Flop counting convention: 1 flop = 1 real number operation of type > > > > (multiply/divide/add/subtract) > > > > e.g., VecAXPY() for real vectors of length N > > > > --> 2N flops > > > > and VecAXPY() for complex vectors of length > > > > N --> 8N flops > > > > > > > > Summary of Stages: ----- Time ------ ----- Flops ----- --- Messages > > > > --- -- Message Lengths -- -- Reductions -- > > > > Avg %Total Avg %Total counts % > > > > Total Avg %Total counts %Total > > > > 0: Main Stage: 5.4246e+02 100.0% 6.7437e+10 100.0% 0.000e+00 > > > > 0.0% 0.000e+00 0.0% 1.390e+03 100.0% > > > > > > > > ------------------------------------------------------------------------------------------------------------------------ > > > > See the 'Profiling' chapter of the users' manual for details on > > > > interpreting output. > > > > Phase summary info: > > > > Count: number of times phase was executed > > > > Time and Flops/sec: Max - maximum over all processors > > > > Ratio - ratio of maximum to minimum over all > > > > processors > > > > Mess: number of messages sent > > > > Avg. len: average message length > > > > Reduct: number of global reductions > > > > Global: entire computation > > > > Stage: stages of a computation. Set stages with PetscLogStagePush() > > > > and PetscLogStagePop(). > > > > %T - percent time in this phase %F - percent flops in this > > > > phase > > > > %M - percent messages in this phase %L - percent message > > > > lengths in this phase > > > > %R - percent reductions in this phase > > > > Total Mflop/s: 10e-6 * (sum of flops over all processors)/(max time > > > > over all processors) > > > > ------------------------------------------------------------------------------------------------------------------------ > > > > > > > > > > > > ########################################################## > > > > # # > > > > # WARNING!!! # > > > > # # > > > > # This code was compiled with a debugging option, # > > > > # To get timing results run config/configure.py # > > > > # using --with-debugging=no, the performance will # > > > > # be generally two or three times faster. # > > > > # # > > > > ########################################################## > > > > > > > > > > > > > > > > > > > > ########################################################## > > > > # # > > > > # WARNING!!! # > > > > # # > > > > # This code was run without the PreLoadBegin() # > > > > # macros. To get timing results we always recommend # > > > > # preloading. otherwise timing numbers may be # > > > > # meaningless. # > > > > ########################################################## > > > > > > > > > > > > Event Count Time (sec) Flops/sec > > > > --- Global --- --- Stage --- Total > > > > Max Ratio Max Ratio Max Ratio Mess Avg len > > > > Reduct %T %F %M %L %R %T %F %M %L %R Mflop/s > > > > ------------------------------------------------------------------------------------------------------------------------ > > > > > > > > --- Event Stage 0: Main Stage > > > > > > > > VecDot 1668318 1.0 1.9186e+01 1.0 4.17e+08 1.0 0.0e+00 0.0e+00 > > > > 0.0e+00 4 12 0 0 0 4 12 0 0 0 417 > > > > VecNorm 835191 1.0 1.0935e+01 1.0 3.67e+08 1.0 0.0e+00 0.0e+00 > > > > 0.0e+00 2 6 0 0 0 2 6 0 0 0 367 > > > > VecCopy 688 1.0 1.6126e-02 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > > > > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > > > VecAXPY 1667630 1.0 2.4141e+01 1.0 3.32e+08 1.0 0.0e+00 0.0e+00 > > > > 0.0e+00 4 12 0 0 0 4 12 0 0 0 332 > > > > VecAYPX 833815 1.0 1.9062e+01 1.0 2.10e+08 1.0 0.0e+00 0.0e+00 > > > > 0.0e+00 4 6 0 0 0 4 6 0 0 0 210 > > > > VecAssemblyBegin 688 1.0 8.5354e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > > > > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > > > VecAssemblyEnd 688 1.0 7.8177e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > > > > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > > > MatMult 834503 1.0 1.5044e+02 1.0 1.18e+08 1.0 0.0e+00 0.0e+00 > > > > 0.0e+00 28 26 0 0 0 28 26 0 0 0 118 > > > > MatSolve 835191 1.0 1.8130e+02 1.0 1.42e+08 1.0 0.0e+00 0.0e+00 > > > > 0.0e+00 33 38 0 0 0 33 38 0 0 0 142 > > > > MatCholFctrNum 1 1.0 1.1630e-03 1.0 2.06e+06 1.0 0.0e+00 0.0e+00 > > > > 0.0e+00 0 0 0 0 0 0 0 0 0 0 2 > > > > MatICCFactorSym 1 1.0 2.9588e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > > > > 2.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > > > MatAssemblyBegin 688 1.0 2.6343e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > > > > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > > > MatAssemblyEnd 688 1.0 1.0964e-01 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > > > > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > > > MatGetOrdering 1 1.0 2.6798e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > > > > 2.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > > > KSPSetup 1 1.0 2.7585e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > > > > 6.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > > > KSPSolve 688 1.0 4.2809e+02 1.0 1.58e+08 1.0 0.0e+00 0.0e+00 > > > > 1.4e+03 79100 0 0100 79100 0 0100 158 > > > > PCSetUp 1 1.0 1.7900e-03 1.0 1.34e+06 1.0 0.0e+00 0.0e+00 > > > > 4.0e+00 0 0 0 0 0 0 0 0 0 0 1 > > > > PCApply 835191 1.0 1.8864e+02 1.0 1.36e+08 1.0 0.0e+00 0.0e+00 > > > > 0.0e+00 35 38 0 0 0 35 38 0 0 0 136 > > > > ------------------------------------------------------------------------------------------------------------------------ > > > > > > > > Memory usage is given in bytes: > > > > > > > > Object Type Creations Destructions Memory Descendants' > > > > Mem. > > > > > > > > --- Event Stage 0: Main Stage > > > > > > > > Index Set 2 2 19640 0 > > > > Vec 694 693 299376 0 > > > > Matrix 2 2 56400 0 > > > > Krylov Solver 1 1 36 0 > > > > Preconditioner 1 1 108 0 > > > > ======================================================================================================================== > > > > Average time to get PetscTime(): 2.86102e-07 > > > > OptionTable: -ksp_type cg > > > > OptionTable: -log_summary -ksp_view > > > > OptionTable: -pc_type icc > > > > Compiled without FORTRAN kernels > > > > Compiled with full precision matrices (default) > > > > sizeof(short) 2 sizeof(int) 4 sizeof(long) 4 sizeof(void*) 4 > > > > sizeof(PetscScalar) 8 > > > > Configure run at: Thu Mar 8 11:54:22 2007 > > > > Configure options: --with-cc=gcc --with-fc=ifort > > > > --download-f-blas-lapack=1 --download-mpich=1 --with-debugging=1 > > > > --with-shared=0 > > > > ----------------------------------------- > > > > Libraries compiled on Thu Mar 8 12:08:22 CET 2007 on iept0415 > > > > Machine characteristics: Linux iept0415 2.6.16.21-0.8-default #1 Mon Jul > > > > 3 18:25:39 UTC 2006 i686 i686 i386 GNU/Linux > > > > Using PETSc directory: /opt/petsc-2.3.2-p8 > > > > Using PETSc arch: gcc-ifc-debug > > > > ----------------------------------------- > > > > Using C compiler: gcc -fPIC -Wall -Wwrite-strings -Wno-strict-aliasing > > > > -g3 > > > > Using Fortran compiler: ifort -fPIC -g > > > > ----------------------------------------- > > > > Using include paths: -I/opt/petsc-2.3.2-p8 > > > > -I/opt/petsc-2.3.2-p8/bmake/gcc-ifc-debug -I/opt/petsc-2.3.2-p8/include > > > > -I/opt/petsc-2.3.2-p8/externalpackages/mpich2-1.0.4p1/gcc-ifc-debug/include > > > > -I/usr/X11R6/include > > > > ------------------------------------------ > > > > Using C linker: gcc -fPIC -Wall -Wwrite-strings -Wno-strict-aliasing -g3 > > > > Using Fortran linker: ifort -fPIC -g > > > > Using libraries: -Wl,-rpath,/opt/petsc-2.3.2-p8/lib/gcc-ifc-debug > > > > -L/opt/petsc-2.3.2-p8/lib/gcc-ifc-debug -lpetscts -lpetscsnes -lpetscksp > > > > -lpetscdm -lpetscmat -lpetscvec -lpetsc > > > > -Wl,-rpath,/opt/petsc-2.3.2-p8/externalpackages/mpich2-1.0.4p1/gcc-ifc-debug/lib > > > > -L/opt/petsc-2.3.2-p8/externalpackages/mpich2-1.0.4p1/gcc-ifc-debug/lib > > > > -lmpich -lnsl -lrt -L/usr/X11R6/lib -lX11 > > > > -Wl,-rpath,/opt/petsc-2.3.2-p8/externalpackages/fblaslapack/gcc-ifc-debug > > > > -L/opt/petsc-2.3.2-p8/externalpackages/fblaslapack/gcc-ifc-debug -lflapack > > > > -Wl,-rpath,/opt/petsc-2.3.2-p8/externalpackages/fblaslapack/gcc-ifc-debug > > > > -L/opt/petsc-2.3.2-p8/externalpackages/fblaslapack/gcc-ifc-debug -lfblas > > > > -lm -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0 > > > > -L/usr/lib/gcc/i586-suse-linux/4.1.0 > > > > -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linux/lib > > > > -L/usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linux/lib > > > > -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../.. > > > > -L/usr/lib/gcc/i586-suse-linux/4.1.0/../../.. -ldl -lgcc_s > > > > -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0 > > > > -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linux/lib > > > > -Wl,-rpath,/usr/lib/gcc/! > > > i5! > > > > 86-suse-linux/4.1.0/../../.. -Wl,-rpath,/opt/intel/fc/9.1.036/lib > > > > -L/opt/intel/fc/9.1.036/lib > > > > -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/ > > > > -L/usr/lib/gcc/i586-suse-linux/4.1.0/ > > > > -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../../ > > > > -L/usr/lib/gcc/i586-suse-linux/4.1.0/../../../ -lifport -lifcore -limf > > > > -lm -lipgo -lirc -lirc_s -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0 > > > > -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linux/lib > > > > -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../.. -lm > > > > -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0 > > > > -L/usr/lib/gcc/i586-suse-linux/4.1.0 > > > > -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linux/lib > > > > -L/usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linux/lib > > > > -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../.. > > > > -L/usr/lib/gcc/i586-suse-linux/4.1.0/../../.. -ldl -lgcc_s -ldl > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > Lisandro Dalc?n > > > --------------- > > > Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) > > > Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) > > > Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) > > > PTLC - G?emes 3450, (3000) Santa Fe, Argentina > > > Tel/Fax: +54-(0)342-451.1594 > > > > > > From knutert at stud.ntnu.no Tue Mar 13 04:04:36 2007 From: knutert at stud.ntnu.no (Knut Erik Teigen) Date: Tue, 13 Mar 2007 10:04:36 +0100 Subject: PETSc CG solver uses more iterations than other CG solver In-Reply-To: <200703102139.28725.berend@chalmers.se> References: <1173456720.6075.43.camel@iept0415.ivt.ntnu.no> <20070310184316.pirkra3plskwoc4k@webmail.ntnu.no> <200703102139.28725.berend@chalmers.se> Message-ID: <1173776676.6075.64.camel@iept0415.ivt.ntnu.no> On Sat, 2007-03-10 at 21:39 +0100, Berend van Wachem wrote: > Hi, > > I have also used the VOF/level set model in our PETSc based code, and it does > require a lot more iterations than a single phase model because of the > stencil. I guess you are solving > > d/dx 1/rho d/dx p = 0 > > ? Which can result in a terrible coefficient structure. That's the equation I'm solving, yes. > > Are you sure the solver/preconditioner you use with PETSc are the same as in > your other code? Both are CG with incomplete cholesky preconditioner. I don't know enough about the solver to compare the source code, so the implementation might be different. But I would think that the PETSc solver is at least as efficient as the other solver. And the performance is equal when solving a simpler Poisson equation. > > I think for the problem size, you are having an awful lot of iterations! Are > you sure the coefficients are put in the right location in the matrix? > I've compared the matrices and right hand sides for the two solvers, and they are equal. -Knut Erik- > Berend. > > > > > One boundary cell is defined to have constant pressure, since that > > makes the equation system have a unique solution. I tried your > > command, and it lowered the number of iterations for most of the time > > steps, but for some it reached the maximum number of iterations > > (10000) without converging. > > > > I also tried making all the boundaries von Neumann and using your > > command. That made the number of iterations more constant, instead of > > varying between 700 and 2000, it stayed on around 1200. But it > > actually increased the average number of iterations somewhat. Still > > far from the performance of the other solver. > > I've also checked the convergence criteria, and it is the same for > > both solvers. > > > > Siterer Lisandro Dalcin : > > > On 3/9/07, Knut Erik Teigen wrote: > > >> To solve the Navier-Stokes equations, I use an explicit Runge-Kutta > > >> method with Chorin's projection method, > > >> so a Poisson equation with von Neumann boundary conditions for the > > >> pressure has to be solved at every time-step. > > > > > > All boundary conditions are Neumann type? In that case, please try to > > > run your program with the following command line option: > > > > > > -ksp_constant_null_space > > > > > > and let me know if this corrected your problem. > > > > > > > > > The equation system is > > > > > >> positive definite, so I use the CG solver with the ICC preconditioner. > > >> The problem is that the PETSc solver seems to need a lot more iterations > > >> to reach the solution than another CG solver I'm using. On a small test > > >> problem(a rising bubble) with a 60x40 grid, the PETSc solver needs over > > >> 1000 iterations on average, while the other solver needs less than 100. > > >> I am using KSPSetInitialGuessNonzero, without this the number of > > >> iterations is even higher. > > >> I have also tried applying PETSc to a similar problem, solving the > > >> Poisson equation with von Neumann boundaries and a forcing function of > > >> f=sin(pi * x)+sin(pi *y). For this problem, the number of iterations is > > >> almost exactly the same for PETSc and the other solver. > > >> > > >> Does anyone know what the problem might be? Any help is greatly > > >> appreciated. I've included the -ksp_view of one of the time steps > > >> and the -log_summary below. > > >> > > >> Regards, > > >> Knut Erik Teigen > > >> MSc student > > >> Norwegian University of Science and Technology > > >> > > >> Output from -ksp_view: > > >> > > >> KSP Object: > > >> type: cg > > >> maximum iterations=10000 > > >> tolerances: relative=1e-06, absolute=1e-50, divergence=10000 > > >> left preconditioning > > >> PC Object: > > >> type: icc > > >> ICC: 0 levels of fill > > >> ICC: factor fill ratio allocated 1 > > >> ICC: factor fill ratio needed 0.601695 > > >> Factored matrix follows > > >> Matrix Object: > > >> type=seqsbaij, rows=2400, cols=2400 > > >> total: nonzeros=7100, allocated nonzeros=7100 > > >> block size is 1 > > >> linear system matrix = precond matrix: > > >> Matrix Object: > > >> type=seqaij, rows=2400, cols=2400 > > >> total: nonzeros=11800, allocated nonzeros=12000 > > >> not using I-node routines > > >> Poisson converged after 1403 iterations > > >> > > >> Output from -log_summary > > >> > > >> ---------------------------------------------- PETSc Performance > > >> Summary: ---------------------------------------------- > > >> > > >> ./run on a gcc-ifc-d named iept0415 with 1 processor, by knutert Fri Mar > > >> 9 17:06:05 2007 > > >> Using Petsc Release Version 2.3.2, Patch 8, Tue Jan 2 14:33:59 PST 2007 > > >> HG revision: ebeddcedcc065e32fc252af32cf1d01ed4fc7a80 > > >> > > >> Max Max/Min Avg Total > > >> Time (sec): 5.425e+02 1.00000 5.425e+02 > > >> Objects: 7.000e+02 1.00000 7.000e+02 > > >> Flops: 6.744e+10 1.00000 6.744e+10 6.744e+10 > > >> Flops/sec: 1.243e+08 1.00000 1.243e+08 1.243e+08 > > >> Memory: 4.881e+05 1.00000 4.881e+05 > > >> MPI Messages: 0.000e+00 0.00000 0.000e+00 0.000e+00 > > >> MPI Message Lengths: 0.000e+00 0.00000 0.000e+00 0.000e+00 > > >> MPI Reductions: 1.390e+03 1.00000 > > >> > > >> Flop counting convention: 1 flop = 1 real number operation of type > > >> (multiply/divide/add/subtract) > > >> e.g., VecAXPY() for real vectors of length N > > >> --> 2N flops > > >> and VecAXPY() for complex vectors of length > > >> N --> 8N flops > > >> > > >> Summary of Stages: ----- Time ------ ----- Flops ----- --- Messages > > >> --- -- Message Lengths -- -- Reductions -- > > >> Avg %Total Avg %Total counts % > > >> Total Avg %Total counts %Total > > >> 0: Main Stage: 5.4246e+02 100.0% 6.7437e+10 100.0% 0.000e+00 > > >> 0.0% 0.000e+00 0.0% 1.390e+03 100.0% > > >> > > >> ------------------------------------------------------------------------ > > >>------------------------------------------------ See the 'Profiling' > > >> chapter of the users' manual for details on interpreting output. > > >> Phase summary info: > > >> Count: number of times phase was executed > > >> Time and Flops/sec: Max - maximum over all processors > > >> Ratio - ratio of maximum to minimum over all > > >> processors > > >> Mess: number of messages sent > > >> Avg. len: average message length > > >> Reduct: number of global reductions > > >> Global: entire computation > > >> Stage: stages of a computation. Set stages with PetscLogStagePush() > > >> and PetscLogStagePop(). > > >> %T - percent time in this phase %F - percent flops in this > > >> phase > > >> %M - percent messages in this phase %L - percent message > > >> lengths in this phase > > >> %R - percent reductions in this phase > > >> Total Mflop/s: 10e-6 * (sum of flops over all processors)/(max time > > >> over all processors) > > >> ------------------------------------------------------------------------ > > >>------------------------------------------------ > > >> > > >> > > >> ########################################################## > > >> # # > > >> # WARNING!!! # > > >> # # > > >> # This code was compiled with a debugging option, # > > >> # To get timing results run config/configure.py # > > >> # using --with-debugging=no, the performance will # > > >> # be generally two or three times faster. # > > >> # # > > >> ########################################################## > > >> > > >> > > >> > > >> > > >> ########################################################## > > >> # # > > >> # WARNING!!! # > > >> # # > > >> # This code was run without the PreLoadBegin() # > > >> # macros. To get timing results we always recommend # > > >> # preloading. otherwise timing numbers may be # > > >> # meaningless. # > > >> ########################################################## > > >> > > >> > > >> Event Count Time (sec) Flops/sec > > >> --- Global --- --- Stage --- Total > > >> Max Ratio Max Ratio Max Ratio Mess Avg len > > >> Reduct %T %F %M %L %R %T %F %M %L %R Mflop/s > > >> ------------------------------------------------------------------------ > > >>------------------------------------------------ > > >> > > >> --- Event Stage 0: Main Stage > > >> > > >> VecDot 1668318 1.0 1.9186e+01 1.0 4.17e+08 1.0 0.0e+00 0.0e+00 > > >> 0.0e+00 4 12 0 0 0 4 12 0 0 0 417 > > >> VecNorm 835191 1.0 1.0935e+01 1.0 3.67e+08 1.0 0.0e+00 0.0e+00 > > >> 0.0e+00 2 6 0 0 0 2 6 0 0 0 367 > > >> VecCopy 688 1.0 1.6126e-02 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > > >> 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > >> VecAXPY 1667630 1.0 2.4141e+01 1.0 3.32e+08 1.0 0.0e+00 0.0e+00 > > >> 0.0e+00 4 12 0 0 0 4 12 0 0 0 332 > > >> VecAYPX 833815 1.0 1.9062e+01 1.0 2.10e+08 1.0 0.0e+00 0.0e+00 > > >> 0.0e+00 4 6 0 0 0 4 6 0 0 0 210 > > >> VecAssemblyBegin 688 1.0 8.5354e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > > >> 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > >> VecAssemblyEnd 688 1.0 7.8177e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > > >> 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > >> MatMult 834503 1.0 1.5044e+02 1.0 1.18e+08 1.0 0.0e+00 0.0e+00 > > >> 0.0e+00 28 26 0 0 0 28 26 0 0 0 118 > > >> MatSolve 835191 1.0 1.8130e+02 1.0 1.42e+08 1.0 0.0e+00 0.0e+00 > > >> 0.0e+00 33 38 0 0 0 33 38 0 0 0 142 > > >> MatCholFctrNum 1 1.0 1.1630e-03 1.0 2.06e+06 1.0 0.0e+00 0.0e+00 > > >> 0.0e+00 0 0 0 0 0 0 0 0 0 0 2 > > >> MatICCFactorSym 1 1.0 2.9588e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > > >> 2.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > >> MatAssemblyBegin 688 1.0 2.6343e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > > >> 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > >> MatAssemblyEnd 688 1.0 1.0964e-01 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > > >> 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > >> MatGetOrdering 1 1.0 2.6798e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > > >> 2.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > >> KSPSetup 1 1.0 2.7585e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > > >> 6.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > >> KSPSolve 688 1.0 4.2809e+02 1.0 1.58e+08 1.0 0.0e+00 0.0e+00 > > >> 1.4e+03 79100 0 0100 79100 0 0100 158 > > >> PCSetUp 1 1.0 1.7900e-03 1.0 1.34e+06 1.0 0.0e+00 0.0e+00 > > >> 4.0e+00 0 0 0 0 0 0 0 0 0 0 1 > > >> PCApply 835191 1.0 1.8864e+02 1.0 1.36e+08 1.0 0.0e+00 0.0e+00 > > >> 0.0e+00 35 38 0 0 0 35 38 0 0 0 136 > > >> ------------------------------------------------------------------------ > > >>------------------------------------------------ > > >> > > >> Memory usage is given in bytes: > > >> > > >> Object Type Creations Destructions Memory Descendants' > > >> Mem. > > >> > > >> --- Event Stage 0: Main Stage > > >> > > >> Index Set 2 2 19640 0 > > >> Vec 694 693 299376 0 > > >> Matrix 2 2 56400 0 > > >> Krylov Solver 1 1 36 0 > > >> Preconditioner 1 1 108 0 > > >> ======================================================================== > > >>================================================ Average time to get > > >> PetscTime(): 2.86102e-07 > > >> OptionTable: -ksp_type cg > > >> OptionTable: -log_summary -ksp_view > > >> OptionTable: -pc_type icc > > >> Compiled without FORTRAN kernels > > >> Compiled with full precision matrices (default) > > >> sizeof(short) 2 sizeof(int) 4 sizeof(long) 4 sizeof(void*) 4 > > >> sizeof(PetscScalar) 8 > > >> Configure run at: Thu Mar 8 11:54:22 2007 > > >> Configure options: --with-cc=gcc --with-fc=ifort > > >> --download-f-blas-lapack=1 --download-mpich=1 --with-debugging=1 > > >> --with-shared=0 > > >> ----------------------------------------- > > >> Libraries compiled on Thu Mar 8 12:08:22 CET 2007 on iept0415 > > >> Machine characteristics: Linux iept0415 2.6.16.21-0.8-default #1 Mon Jul > > >> 3 18:25:39 UTC 2006 i686 i686 i386 GNU/Linux > > >> Using PETSc directory: /opt/petsc-2.3.2-p8 > > >> Using PETSc arch: gcc-ifc-debug > > >> ----------------------------------------- > > >> Using C compiler: gcc -fPIC -Wall -Wwrite-strings -Wno-strict-aliasing > > >> -g3 > > >> Using Fortran compiler: ifort -fPIC -g > > >> ----------------------------------------- > > >> Using include paths: -I/opt/petsc-2.3.2-p8 > > >> -I/opt/petsc-2.3.2-p8/bmake/gcc-ifc-debug -I/opt/petsc-2.3.2-p8/include > > >> -I/opt/petsc-2.3.2-p8/externalpackages/mpich2-1.0.4p1/gcc-ifc-debug/incl > > >>ude -I/usr/X11R6/include > > >> ------------------------------------------ > > >> Using C linker: gcc -fPIC -Wall -Wwrite-strings -Wno-strict-aliasing -g3 > > >> Using Fortran linker: ifort -fPIC -g > > >> Using libraries: -Wl,-rpath,/opt/petsc-2.3.2-p8/lib/gcc-ifc-debug > > >> -L/opt/petsc-2.3.2-p8/lib/gcc-ifc-debug -lpetscts -lpetscsnes -lpetscksp > > >> -lpetscdm -lpetscmat -lpetscvec -lpetsc > > >> -Wl,-rpath,/opt/petsc-2.3.2-p8/externalpackages/mpich2-1.0.4p1/gcc-ifc-d > > >>ebug/lib > > >> -L/opt/petsc-2.3.2-p8/externalpackages/mpich2-1.0.4p1/gcc-ifc-debug/lib > > >> -lmpich -lnsl -lrt -L/usr/X11R6/lib -lX11 > > >> -Wl,-rpath,/opt/petsc-2.3.2-p8/externalpackages/fblaslapack/gcc-ifc-debu > > >>g -L/opt/petsc-2.3.2-p8/externalpackages/fblaslapack/gcc-ifc-debug > > >> -lflapack > > >> -Wl,-rpath,/opt/petsc-2.3.2-p8/externalpackages/fblaslapack/gcc-ifc-debu > > >>g -L/opt/petsc-2.3.2-p8/externalpackages/fblaslapack/gcc-ifc-debug > > >> -lfblas -lm -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0 > > >> -L/usr/lib/gcc/i586-suse-linux/4.1.0 > > >> -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linu > > >>x/lib > > >> -L/usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linux/lib > > >> -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../.. > > >> -L/usr/lib/gcc/i586-suse-linux/4.1.0/../../.. -ldl -lgcc_s > > >> -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0 > > >> -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linu > > >>x/lib -Wl,-rpath,/usr/lib/gcc/! > > > > > > i5! > > > > > >> 86-suse-linux/4.1.0/../../.. -Wl,-rpath,/opt/intel/fc/9.1.036/lib > > >> -L/opt/intel/fc/9.1.036/lib > > >> -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/ > > >> -L/usr/lib/gcc/i586-suse-linux/4.1.0/ > > >> -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../../ > > >> -L/usr/lib/gcc/i586-suse-linux/4.1.0/../../../ -lifport -lifcore > > >> -limf -lm -lipgo -lirc -lirc_s > > >> -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0 > > >> -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linu > > >>x/lib -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../.. -lm > > >> -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0 > > >> -L/usr/lib/gcc/i586-suse-linux/4.1.0 > > >> -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linu > > >>x/lib > > >> -L/usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linux/lib > > >> -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../.. > > >> -L/usr/lib/gcc/i586-suse-linux/4.1.0/../../.. -ldl -lgcc_s -ldl > > > > > > -- > > > Lisandro Dalc?n > > > --------------- > > > Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) > > > Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) > > > Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) > > > PTLC - G?emes 3450, (3000) Santa Fe, Argentina > > > Tel/Fax: +54-(0)342-451.1594 > > From knutert at stud.ntnu.no Tue Mar 13 04:08:13 2007 From: knutert at stud.ntnu.no (Knut Erik Teigen) Date: Tue, 13 Mar 2007 10:08:13 +0100 Subject: PETSc CG solver uses more iterations than other CG solver In-Reply-To: References: <1173456720.6075.43.camel@iept0415.ivt.ntnu.no> <20070310184316.pirkra3plskwoc4k@webmail.ntnu.no> Message-ID: <1173776893.6075.66.camel@iept0415.ivt.ntnu.no> On Sat, 2007-03-10 at 11:55 -0600, Matthew Knepley wrote: > Something is obviously wrong. CG has a simple definition and so does ICC(0). > So I would suggest > > 1) Make sure you are comparing apples to apples. Usually, the other solver > will have something you have not turned on, like scaling. > > 2) Compare the iterates exactly. I do this all the time, and it is > the best way > to compare differences. Is there an easy way to do this with PETSc, or do I have to add print sentences to the source code and recompile? -Knut Erik- > > Matt > > On 3/10/07, knutert at stud.ntnu.no wrote: > > Thank you for your reply. > > > > One boundary cell is defined to have constant pressure, since that > > makes the equation system have a unique solution. I tried your > > command, and it lowered the number of iterations for most of the time > > steps, but for some it reached the maximum number of iterations > > (10000) without converging. > > > > I also tried making all the boundaries von Neumann and using your > > command. That made the number of iterations more constant, instead of > > varying between 700 and 2000, it stayed on around 1200. But it > > actually increased the average number of iterations somewhat. Still > > far from the performance of the other solver. > > I've also checked the convergence criteria, and it is the same for > > both solvers. > > > > > > Siterer Lisandro Dalcin : > > > > > On 3/9/07, Knut Erik Teigen wrote: > > >> To solve the Navier-Stokes equations, I use an explicit Runge-Kutta > > >> method with Chorin's projection method, > > >> so a Poisson equation with von Neumann boundary conditions for the > > >> pressure has to be solved at every time-step. > > > > > > All boundary conditions are Neumann type? In that case, please try to > > > run your program with the following command line option: > > > > > > -ksp_constant_null_space > > > > > > and let me know if this corrected your problem. > > > > > > > > > The equation system is > > >> positive definite, so I use the CG solver with the ICC preconditioner. > > >> The problem is that the PETSc solver seems to need a lot more iterations > > >> to reach the solution than another CG solver I'm using. On a small test > > >> problem(a rising bubble) with a 60x40 grid, the PETSc solver needs over > > >> 1000 iterations on average, while the other solver needs less than 100. > > >> I am using KSPSetInitialGuessNonzero, without this the number of > > >> iterations is even higher. > > >> I have also tried applying PETSc to a similar problem, solving the > > >> Poisson equation with von Neumann boundaries and a forcing function of > > >> f=sin(pi * x)+sin(pi *y). For this problem, the number of iterations is > > >> almost exactly the same for PETSc and the other solver. > > >> > > >> Does anyone know what the problem might be? Any help is greatly > > >> appreciated. I've included the -ksp_view of one of the time steps > > >> and the -log_summary below. > > >> > > >> Regards, > > >> Knut Erik Teigen > > >> MSc student > > >> Norwegian University of Science and Technology > > >> > > >> Output from -ksp_view: > > >> > > >> KSP Object: > > >> type: cg > > >> maximum iterations=10000 > > >> tolerances: relative=1e-06, absolute=1e-50, divergence=10000 > > >> left preconditioning > > >> PC Object: > > >> type: icc > > >> ICC: 0 levels of fill > > >> ICC: factor fill ratio allocated 1 > > >> ICC: factor fill ratio needed 0.601695 > > >> Factored matrix follows > > >> Matrix Object: > > >> type=seqsbaij, rows=2400, cols=2400 > > >> total: nonzeros=7100, allocated nonzeros=7100 > > >> block size is 1 > > >> linear system matrix = precond matrix: > > >> Matrix Object: > > >> type=seqaij, rows=2400, cols=2400 > > >> total: nonzeros=11800, allocated nonzeros=12000 > > >> not using I-node routines > > >> Poisson converged after 1403 iterations > > >> > > >> Output from -log_summary > > >> > > >> ---------------------------------------------- PETSc Performance > > >> Summary: ---------------------------------------------- > > >> > > >> ./run on a gcc-ifc-d named iept0415 with 1 processor, by knutert Fri Mar > > >> 9 17:06:05 2007 > > >> Using Petsc Release Version 2.3.2, Patch 8, Tue Jan 2 14:33:59 PST 2007 > > >> HG revision: ebeddcedcc065e32fc252af32cf1d01ed4fc7a80 > > >> > > >> Max Max/Min Avg Total > > >> Time (sec): 5.425e+02 1.00000 5.425e+02 > > >> Objects: 7.000e+02 1.00000 7.000e+02 > > >> Flops: 6.744e+10 1.00000 6.744e+10 6.744e+10 > > >> Flops/sec: 1.243e+08 1.00000 1.243e+08 1.243e+08 > > >> Memory: 4.881e+05 1.00000 4.881e+05 > > >> MPI Messages: 0.000e+00 0.00000 0.000e+00 0.000e+00 > > >> MPI Message Lengths: 0.000e+00 0.00000 0.000e+00 0.000e+00 > > >> MPI Reductions: 1.390e+03 1.00000 > > >> > > >> Flop counting convention: 1 flop = 1 real number operation of type > > >> (multiply/divide/add/subtract) > > >> e.g., VecAXPY() for real vectors of length N > > >> --> 2N flops > > >> and VecAXPY() for complex vectors of length > > >> N --> 8N flops > > >> > > >> Summary of Stages: ----- Time ------ ----- Flops ----- --- Messages > > >> --- -- Message Lengths -- -- Reductions -- > > >> Avg %Total Avg %Total counts % > > >> Total Avg %Total counts %Total > > >> 0: Main Stage: 5.4246e+02 100.0% 6.7437e+10 100.0% 0.000e+00 > > >> 0.0% 0.000e+00 0.0% 1.390e+03 100.0% > > >> > > >> ------------------------------------------------------------------------------------------------------------------------ > > >> See the 'Profiling' chapter of the users' manual for details on > > >> interpreting output. > > >> Phase summary info: > > >> Count: number of times phase was executed > > >> Time and Flops/sec: Max - maximum over all processors > > >> Ratio - ratio of maximum to minimum over all > > >> processors > > >> Mess: number of messages sent > > >> Avg. len: average message length > > >> Reduct: number of global reductions > > >> Global: entire computation > > >> Stage: stages of a computation. Set stages with PetscLogStagePush() > > >> and PetscLogStagePop(). > > >> %T - percent time in this phase %F - percent flops in this > > >> phase > > >> %M - percent messages in this phase %L - percent message > > >> lengths in this phase > > >> %R - percent reductions in this phase > > >> Total Mflop/s: 10e-6 * (sum of flops over all processors)/(max time > > >> over all processors) > > >> ------------------------------------------------------------------------------------------------------------------------ > > >> > > >> > > >> ########################################################## > > >> # # > > >> # WARNING!!! # > > >> # # > > >> # This code was compiled with a debugging option, # > > >> # To get timing results run config/configure.py # > > >> # using --with-debugging=no, the performance will # > > >> # be generally two or three times faster. # > > >> # # > > >> ########################################################## > > >> > > >> > > >> > > >> > > >> ########################################################## > > >> # # > > >> # WARNING!!! # > > >> # # > > >> # This code was run without the PreLoadBegin() # > > >> # macros. To get timing results we always recommend # > > >> # preloading. otherwise timing numbers may be # > > >> # meaningless. # > > >> ########################################################## > > >> > > >> > > >> Event Count Time (sec) Flops/sec > > >> --- Global --- --- Stage --- Total > > >> Max Ratio Max Ratio Max Ratio Mess Avg len > > >> Reduct %T %F %M %L %R %T %F %M %L %R Mflop/s > > >> ------------------------------------------------------------------------------------------------------------------------ > > >> > > >> --- Event Stage 0: Main Stage > > >> > > >> VecDot 1668318 1.0 1.9186e+01 1.0 4.17e+08 1.0 0.0e+00 0.0e+00 > > >> 0.0e+00 4 12 0 0 0 4 12 0 0 0 417 > > >> VecNorm 835191 1.0 1.0935e+01 1.0 3.67e+08 1.0 0.0e+00 0.0e+00 > > >> 0.0e+00 2 6 0 0 0 2 6 0 0 0 367 > > >> VecCopy 688 1.0 1.6126e-02 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > > >> 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > >> VecAXPY 1667630 1.0 2.4141e+01 1.0 3.32e+08 1.0 0.0e+00 0.0e+00 > > >> 0.0e+00 4 12 0 0 0 4 12 0 0 0 332 > > >> VecAYPX 833815 1.0 1.9062e+01 1.0 2.10e+08 1.0 0.0e+00 0.0e+00 > > >> 0.0e+00 4 6 0 0 0 4 6 0 0 0 210 > > >> VecAssemblyBegin 688 1.0 8.5354e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > > >> 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > >> VecAssemblyEnd 688 1.0 7.8177e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > > >> 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > >> MatMult 834503 1.0 1.5044e+02 1.0 1.18e+08 1.0 0.0e+00 0.0e+00 > > >> 0.0e+00 28 26 0 0 0 28 26 0 0 0 118 > > >> MatSolve 835191 1.0 1.8130e+02 1.0 1.42e+08 1.0 0.0e+00 0.0e+00 > > >> 0.0e+00 33 38 0 0 0 33 38 0 0 0 142 > > >> MatCholFctrNum 1 1.0 1.1630e-03 1.0 2.06e+06 1.0 0.0e+00 0.0e+00 > > >> 0.0e+00 0 0 0 0 0 0 0 0 0 0 2 > > >> MatICCFactorSym 1 1.0 2.9588e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > > >> 2.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > >> MatAssemblyBegin 688 1.0 2.6343e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > > >> 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > >> MatAssemblyEnd 688 1.0 1.0964e-01 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > > >> 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > >> MatGetOrdering 1 1.0 2.6798e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > > >> 2.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > >> KSPSetup 1 1.0 2.7585e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > > >> 6.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > >> KSPSolve 688 1.0 4.2809e+02 1.0 1.58e+08 1.0 0.0e+00 0.0e+00 > > >> 1.4e+03 79100 0 0100 79100 0 0100 158 > > >> PCSetUp 1 1.0 1.7900e-03 1.0 1.34e+06 1.0 0.0e+00 0.0e+00 > > >> 4.0e+00 0 0 0 0 0 0 0 0 0 0 1 > > >> PCApply 835191 1.0 1.8864e+02 1.0 1.36e+08 1.0 0.0e+00 0.0e+00 > > >> 0.0e+00 35 38 0 0 0 35 38 0 0 0 136 > > >> ------------------------------------------------------------------------------------------------------------------------ > > >> > > >> Memory usage is given in bytes: > > >> > > >> Object Type Creations Destructions Memory Descendants' > > >> Mem. > > >> > > >> --- Event Stage 0: Main Stage > > >> > > >> Index Set 2 2 19640 0 > > >> Vec 694 693 299376 0 > > >> Matrix 2 2 56400 0 > > >> Krylov Solver 1 1 36 0 > > >> Preconditioner 1 1 108 0 > > >> ======================================================================================================================== > > >> Average time to get PetscTime(): 2.86102e-07 > > >> OptionTable: -ksp_type cg > > >> OptionTable: -log_summary -ksp_view > > >> OptionTable: -pc_type icc > > >> Compiled without FORTRAN kernels > > >> Compiled with full precision matrices (default) > > >> sizeof(short) 2 sizeof(int) 4 sizeof(long) 4 sizeof(void*) 4 > > >> sizeof(PetscScalar) 8 > > >> Configure run at: Thu Mar 8 11:54:22 2007 > > >> Configure options: --with-cc=gcc --with-fc=ifort > > >> --download-f-blas-lapack=1 --download-mpich=1 --with-debugging=1 > > >> --with-shared=0 > > >> ----------------------------------------- > > >> Libraries compiled on Thu Mar 8 12:08:22 CET 2007 on iept0415 > > >> Machine characteristics: Linux iept0415 2.6.16.21-0.8-default #1 Mon Jul > > >> 3 18:25:39 UTC 2006 i686 i686 i386 GNU/Linux > > >> Using PETSc directory: /opt/petsc-2.3.2-p8 > > >> Using PETSc arch: gcc-ifc-debug > > >> ----------------------------------------- > > >> Using C compiler: gcc -fPIC -Wall -Wwrite-strings -Wno-strict-aliasing > > >> -g3 > > >> Using Fortran compiler: ifort -fPIC -g > > >> ----------------------------------------- > > >> Using include paths: -I/opt/petsc-2.3.2-p8 > > >> -I/opt/petsc-2.3.2-p8/bmake/gcc-ifc-debug -I/opt/petsc-2.3.2-p8/include > > >> -I/opt/petsc-2.3.2-p8/externalpackages/mpich2-1.0.4p1/gcc-ifc-debug/include > > >> -I/usr/X11R6/include > > >> ------------------------------------------ > > >> Using C linker: gcc -fPIC -Wall -Wwrite-strings -Wno-strict-aliasing -g3 > > >> Using Fortran linker: ifort -fPIC -g > > >> Using libraries: -Wl,-rpath,/opt/petsc-2.3.2-p8/lib/gcc-ifc-debug > > >> -L/opt/petsc-2.3.2-p8/lib/gcc-ifc-debug -lpetscts -lpetscsnes -lpetscksp > > >> -lpetscdm -lpetscmat -lpetscvec -lpetsc > > >> -Wl,-rpath,/opt/petsc-2.3.2-p8/externalpackages/mpich2-1.0.4p1/gcc-ifc-debug/lib -L/opt/petsc-2.3.2-p8/externalpackages/mpich2-1.0.4p1/gcc-ifc-debug/lib -lmpich -lnsl -lrt -L/usr/X11R6/lib -lX11 -Wl,-rpath,/opt/petsc-2.3.2-p8/externalpackages/fblaslapack/gcc-ifc-debug -L/opt/petsc-2.3.2-p8/externalpackages/fblaslapack/gcc-ifc-debug -lflapack -Wl,-rpath,/opt/petsc-2.3.2-p8/externalpackages/fblaslapack/gcc-ifc-debug -L/opt/petsc-2.3.2-p8/externalpackages/fblaslapack/gcc-ifc-debug -lfblas -lm -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0 -L/usr/lib/gcc/i586-suse-linux/4.1.0 -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linux/lib -L/usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linux/lib -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../.. -L/usr/lib/gcc/i586-suse-linux/4.1.0/../../.. -ldl -lgcc_s -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0 -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linux/lib > > >> -Wl,-rpath,/usr/lib/gcc/! > > > i5! > > >> 86-suse-linux/4.1.0/../../.. -Wl,-rpath,/opt/intel/fc/9.1.036/lib > > >> -L/opt/intel/fc/9.1.036/lib > > >> -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/ > > >> -L/usr/lib/gcc/i586-suse-linux/4.1.0/ > > >> -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../../ > > >> -L/usr/lib/gcc/i586-suse-linux/4.1.0/../../../ -lifport -lifcore > > >> -limf -lm -lipgo -lirc -lirc_s > > >> -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0 > > >> -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linux/lib -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../.. -lm -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0 -L/usr/lib/gcc/i586-suse-linux/4.1.0 -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linux/lib -L/usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linux/lib -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../.. -L/usr/lib/gcc/i586-suse-linux/4.1.0/../../.. -ldl -lgcc_s > > >> -ldl > > >> > > >> > > >> > > >> > > > > > > > > > -- > > > Lisandro Dalc?n > > > --------------- > > > Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) > > > Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) > > > Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) > > > PTLC - G?emes 3450, (3000) Santa Fe, Argentina > > > Tel/Fax: +54-(0)342-451.1594 > > > > > > > > > > From berend at chalmers.se Tue Mar 13 04:36:17 2007 From: berend at chalmers.se (Berend van Wachem) Date: Tue, 13 Mar 2007 10:36:17 +0100 Subject: PETSc CG solver uses more iterations than other CG solver In-Reply-To: <1173776676.6075.64.camel@iept0415.ivt.ntnu.no> References: <1173456720.6075.43.camel@iept0415.ivt.ntnu.no> <200703102139.28725.berend@chalmers.se> <1173776676.6075.64.camel@iept0415.ivt.ntnu.no> Message-ID: <200703131036.18099.berend@chalmers.se> Hi, > > I have also used the VOF/level set model in our PETSc based code, and it > > does require a lot more iterations than a single phase model because of > > the stencil. I guess you are solving > > > > d/dx 1/rho d/dx p = 0 > > > > ? Which can result in a terrible coefficient structure. > > That's the equation I'm solving, yes. Which for a bubble rising in water means that the density on the water side is 1000 and on the air side is 1. So instead of the -1 2 -1 contribution from the Poisson stencil you have in single phase flow for each direction, you can get contributions like -1.0e-3 1.0 -1.0 which has a much worse condition to solve. But if you use exactly the same solver and preconditioner, it should result in approximately the same amount of iterations. Berend. From zonexo at gmail.com Tue Mar 13 06:51:03 2007 From: zonexo at gmail.com (Ben Tay) Date: Tue, 13 Mar 2007 19:51:03 +0800 Subject: Copying shared lib files Message-ID: <804ab5d40703130451v3b086896ie37772f45338b377@mail.gmail.com> Hi, I've compiled PETSc on my school's server and I wonder if I can simply copy the compiled shared library/include files to my home comp's linux. Both uses redhat. Is there anything else I have to take care? Thank you -------------- next part -------------- An HTML attachment was scrubbed... URL: From aja2111 at columbia.edu Tue Mar 13 07:06:50 2007 From: aja2111 at columbia.edu (Aron Ahmadia) Date: Tue, 13 Mar 2007 08:06:50 -0400 Subject: Copying shared lib files In-Reply-To: <804ab5d40703130451v3b086896ie37772f45338b377@mail.gmail.com> References: <804ab5d40703130451v3b086896ie37772f45338b377@mail.gmail.com> Message-ID: <37604ab40703130506o64f15105wec542552dde3736b@mail.gmail.com> Hi Ben, This is a bad idea. Unless the systems are virtually identical (same versions of gcc, same processor architecture, same kernel, same system layout) just to name a few, you WILL have problems with the libraries after they've been copied over, though less problems are likely to spring up if you linked them statically. As a former Redhat user, I can attest to similar computers having a same-versioned RH distro and still generating incompatible libraries. ~A On 3/13/07, Ben Tay wrote: > Hi, > > I've compiled PETSc on my school's server and I wonder if I can simply copy > the compiled shared library/include files to my home comp's linux. Both uses > redhat. > > Is there anything else I have to take care? > > Thank you > > From knepley at gmail.com Tue Mar 13 07:39:49 2007 From: knepley at gmail.com (Matthew Knepley) Date: Tue, 13 Mar 2007 07:39:49 -0500 Subject: PETSc CG solver uses more iterations than other CG solver In-Reply-To: <1173776893.6075.66.camel@iept0415.ivt.ntnu.no> References: <1173456720.6075.43.camel@iept0415.ivt.ntnu.no> <20070310184316.pirkra3plskwoc4k@webmail.ntnu.no> <1173776893.6075.66.camel@iept0415.ivt.ntnu.no> Message-ID: On 3/13/07, Knut Erik Teigen wrote: > On Sat, 2007-03-10 at 11:55 -0600, Matthew Knepley wrote: > > Something is obviously wrong. CG has a simple definition and so does ICC(0). > > So I would suggest > > > > 1) Make sure you are comparing apples to apples. Usually, the other solver > > will have something you have not turned on, like scaling. > > > > 2) Compare the iterates exactly. I do this all the time, and it is > > the best way > > to compare differences. > > Is there an easy way to do this with PETSc, or do I have to add print > sentences to the source code and recompile? Here is how I might do it: a) Compare the solver configuration (-ksp_view). Are they both using symmetric preconditioning, or is one left, etc. b) Compare the residual at each iterate (-ksp_monitor) c) Compare the iterate itself. You register your own monitor method http://www-unix.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/KSP/KSPSetMonitor.html and then get the current residual http://www-unix.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/KSP/KSPBuildResidual.html and output it with VecView(). Matt > -Knut Erik- > > > > Matt > > > > On 3/10/07, knutert at stud.ntnu.no wrote: > > > Thank you for your reply. > > > > > > One boundary cell is defined to have constant pressure, since that > > > makes the equation system have a unique solution. I tried your > > > command, and it lowered the number of iterations for most of the time > > > steps, but for some it reached the maximum number of iterations > > > (10000) without converging. > > > > > > I also tried making all the boundaries von Neumann and using your > > > command. That made the number of iterations more constant, instead of > > > varying between 700 and 2000, it stayed on around 1200. But it > > > actually increased the average number of iterations somewhat. Still > > > far from the performance of the other solver. > > > I've also checked the convergence criteria, and it is the same for > > > both solvers. > > > > > > > > > Siterer Lisandro Dalcin : > > > > > > > On 3/9/07, Knut Erik Teigen wrote: > > > >> To solve the Navier-Stokes equations, I use an explicit Runge-Kutta > > > >> method with Chorin's projection method, > > > >> so a Poisson equation with von Neumann boundary conditions for the > > > >> pressure has to be solved at every time-step. > > > > > > > > All boundary conditions are Neumann type? In that case, please try to > > > > run your program with the following command line option: > > > > > > > > -ksp_constant_null_space > > > > > > > > and let me know if this corrected your problem. > > > > > > > > > > > > The equation system is > > > >> positive definite, so I use the CG solver with the ICC preconditioner. > > > >> The problem is that the PETSc solver seems to need a lot more iterations > > > >> to reach the solution than another CG solver I'm using. On a small test > > > >> problem(a rising bubble) with a 60x40 grid, the PETSc solver needs over > > > >> 1000 iterations on average, while the other solver needs less than 100. > > > >> I am using KSPSetInitialGuessNonzero, without this the number of > > > >> iterations is even higher. > > > >> I have also tried applying PETSc to a similar problem, solving the > > > >> Poisson equation with von Neumann boundaries and a forcing function of > > > >> f=sin(pi * x)+sin(pi *y). For this problem, the number of iterations is > > > >> almost exactly the same for PETSc and the other solver. > > > >> > > > >> Does anyone know what the problem might be? Any help is greatly > > > >> appreciated. I've included the -ksp_view of one of the time steps > > > >> and the -log_summary below. > > > >> > > > >> Regards, > > > >> Knut Erik Teigen > > > >> MSc student > > > >> Norwegian University of Science and Technology > > > >> > > > >> Output from -ksp_view: > > > >> > > > >> KSP Object: > > > >> type: cg > > > >> maximum iterations=10000 > > > >> tolerances: relative=1e-06, absolute=1e-50, divergence=10000 > > > >> left preconditioning > > > >> PC Object: > > > >> type: icc > > > >> ICC: 0 levels of fill > > > >> ICC: factor fill ratio allocated 1 > > > >> ICC: factor fill ratio needed 0.601695 > > > >> Factored matrix follows > > > >> Matrix Object: > > > >> type=seqsbaij, rows=2400, cols=2400 > > > >> total: nonzeros=7100, allocated nonzeros=7100 > > > >> block size is 1 > > > >> linear system matrix = precond matrix: > > > >> Matrix Object: > > > >> type=seqaij, rows=2400, cols=2400 > > > >> total: nonzeros=11800, allocated nonzeros=12000 > > > >> not using I-node routines > > > >> Poisson converged after 1403 iterations > > > >> > > > >> Output from -log_summary > > > >> > > > >> ---------------------------------------------- PETSc Performance > > > >> Summary: ---------------------------------------------- > > > >> > > > >> ./run on a gcc-ifc-d named iept0415 with 1 processor, by knutert Fri Mar > > > >> 9 17:06:05 2007 > > > >> Using Petsc Release Version 2.3.2, Patch 8, Tue Jan 2 14:33:59 PST 2007 > > > >> HG revision: ebeddcedcc065e32fc252af32cf1d01ed4fc7a80 > > > >> > > > >> Max Max/Min Avg Total > > > >> Time (sec): 5.425e+02 1.00000 5.425e+02 > > > >> Objects: 7.000e+02 1.00000 7.000e+02 > > > >> Flops: 6.744e+10 1.00000 6.744e+10 6.744e+10 > > > >> Flops/sec: 1.243e+08 1.00000 1.243e+08 1.243e+08 > > > >> Memory: 4.881e+05 1.00000 4.881e+05 > > > >> MPI Messages: 0.000e+00 0.00000 0.000e+00 0.000e+00 > > > >> MPI Message Lengths: 0.000e+00 0.00000 0.000e+00 0.000e+00 > > > >> MPI Reductions: 1.390e+03 1.00000 > > > >> > > > >> Flop counting convention: 1 flop = 1 real number operation of type > > > >> (multiply/divide/add/subtract) > > > >> e.g., VecAXPY() for real vectors of length N > > > >> --> 2N flops > > > >> and VecAXPY() for complex vectors of length > > > >> N --> 8N flops > > > >> > > > >> Summary of Stages: ----- Time ------ ----- Flops ----- --- Messages > > > >> --- -- Message Lengths -- -- Reductions -- > > > >> Avg %Total Avg %Total counts % > > > >> Total Avg %Total counts %Total > > > >> 0: Main Stage: 5.4246e+02 100.0% 6.7437e+10 100.0% 0.000e+00 > > > >> 0.0% 0.000e+00 0.0% 1.390e+03 100.0% > > > >> > > > >> ------------------------------------------------------------------------------------------------------------------------ > > > >> See the 'Profiling' chapter of the users' manual for details on > > > >> interpreting output. > > > >> Phase summary info: > > > >> Count: number of times phase was executed > > > >> Time and Flops/sec: Max - maximum over all processors > > > >> Ratio - ratio of maximum to minimum over all > > > >> processors > > > >> Mess: number of messages sent > > > >> Avg. len: average message length > > > >> Reduct: number of global reductions > > > >> Global: entire computation > > > >> Stage: stages of a computation. Set stages with PetscLogStagePush() > > > >> and PetscLogStagePop(). > > > >> %T - percent time in this phase %F - percent flops in this > > > >> phase > > > >> %M - percent messages in this phase %L - percent message > > > >> lengths in this phase > > > >> %R - percent reductions in this phase > > > >> Total Mflop/s: 10e-6 * (sum of flops over all processors)/(max time > > > >> over all processors) > > > >> ------------------------------------------------------------------------------------------------------------------------ > > > >> > > > >> > > > >> ########################################################## > > > >> # # > > > >> # WARNING!!! # > > > >> # # > > > >> # This code was compiled with a debugging option, # > > > >> # To get timing results run config/configure.py # > > > >> # using --with-debugging=no, the performance will # > > > >> # be generally two or three times faster. # > > > >> # # > > > >> ########################################################## > > > >> > > > >> > > > >> > > > >> > > > >> ########################################################## > > > >> # # > > > >> # WARNING!!! # > > > >> # # > > > >> # This code was run without the PreLoadBegin() # > > > >> # macros. To get timing results we always recommend # > > > >> # preloading. otherwise timing numbers may be # > > > >> # meaningless. # > > > >> ########################################################## > > > >> > > > >> > > > >> Event Count Time (sec) Flops/sec > > > >> --- Global --- --- Stage --- Total > > > >> Max Ratio Max Ratio Max Ratio Mess Avg len > > > >> Reduct %T %F %M %L %R %T %F %M %L %R Mflop/s > > > >> ------------------------------------------------------------------------------------------------------------------------ > > > >> > > > >> --- Event Stage 0: Main Stage > > > >> > > > >> VecDot 1668318 1.0 1.9186e+01 1.0 4.17e+08 1.0 0.0e+00 0.0e+00 > > > >> 0.0e+00 4 12 0 0 0 4 12 0 0 0 417 > > > >> VecNorm 835191 1.0 1.0935e+01 1.0 3.67e+08 1.0 0.0e+00 0.0e+00 > > > >> 0.0e+00 2 6 0 0 0 2 6 0 0 0 367 > > > >> VecCopy 688 1.0 1.6126e-02 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > > > >> 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > > >> VecAXPY 1667630 1.0 2.4141e+01 1.0 3.32e+08 1.0 0.0e+00 0.0e+00 > > > >> 0.0e+00 4 12 0 0 0 4 12 0 0 0 332 > > > >> VecAYPX 833815 1.0 1.9062e+01 1.0 2.10e+08 1.0 0.0e+00 0.0e+00 > > > >> 0.0e+00 4 6 0 0 0 4 6 0 0 0 210 > > > >> VecAssemblyBegin 688 1.0 8.5354e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > > > >> 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > > >> VecAssemblyEnd 688 1.0 7.8177e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > > > >> 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > > >> MatMult 834503 1.0 1.5044e+02 1.0 1.18e+08 1.0 0.0e+00 0.0e+00 > > > >> 0.0e+00 28 26 0 0 0 28 26 0 0 0 118 > > > >> MatSolve 835191 1.0 1.8130e+02 1.0 1.42e+08 1.0 0.0e+00 0.0e+00 > > > >> 0.0e+00 33 38 0 0 0 33 38 0 0 0 142 > > > >> MatCholFctrNum 1 1.0 1.1630e-03 1.0 2.06e+06 1.0 0.0e+00 0.0e+00 > > > >> 0.0e+00 0 0 0 0 0 0 0 0 0 0 2 > > > >> MatICCFactorSym 1 1.0 2.9588e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > > > >> 2.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > > >> MatAssemblyBegin 688 1.0 2.6343e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > > > >> 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > > >> MatAssemblyEnd 688 1.0 1.0964e-01 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > > > >> 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > > >> MatGetOrdering 1 1.0 2.6798e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > > > >> 2.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > > >> KSPSetup 1 1.0 2.7585e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > > > >> 6.0e+00 0 0 0 0 0 0 0 0 0 0 0 > > > >> KSPSolve 688 1.0 4.2809e+02 1.0 1.58e+08 1.0 0.0e+00 0.0e+00 > > > >> 1.4e+03 79100 0 0100 79100 0 0100 158 > > > >> PCSetUp 1 1.0 1.7900e-03 1.0 1.34e+06 1.0 0.0e+00 0.0e+00 > > > >> 4.0e+00 0 0 0 0 0 0 0 0 0 0 1 > > > >> PCApply 835191 1.0 1.8864e+02 1.0 1.36e+08 1.0 0.0e+00 0.0e+00 > > > >> 0.0e+00 35 38 0 0 0 35 38 0 0 0 136 > > > >> ------------------------------------------------------------------------------------------------------------------------ > > > >> > > > >> Memory usage is given in bytes: > > > >> > > > >> Object Type Creations Destructions Memory Descendants' > > > >> Mem. > > > >> > > > >> --- Event Stage 0: Main Stage > > > >> > > > >> Index Set 2 2 19640 0 > > > >> Vec 694 693 299376 0 > > > >> Matrix 2 2 56400 0 > > > >> Krylov Solver 1 1 36 0 > > > >> Preconditioner 1 1 108 0 > > > >> ======================================================================================================================== > > > >> Average time to get PetscTime(): 2.86102e-07 > > > >> OptionTable: -ksp_type cg > > > >> OptionTable: -log_summary -ksp_view > > > >> OptionTable: -pc_type icc > > > >> Compiled without FORTRAN kernels > > > >> Compiled with full precision matrices (default) > > > >> sizeof(short) 2 sizeof(int) 4 sizeof(long) 4 sizeof(void*) 4 > > > >> sizeof(PetscScalar) 8 > > > >> Configure run at: Thu Mar 8 11:54:22 2007 > > > >> Configure options: --with-cc=gcc --with-fc=ifort > > > >> --download-f-blas-lapack=1 --download-mpich=1 --with-debugging=1 > > > >> --with-shared=0 > > > >> ----------------------------------------- > > > >> Libraries compiled on Thu Mar 8 12:08:22 CET 2007 on iept0415 > > > >> Machine characteristics: Linux iept0415 2.6.16.21-0.8-default #1 Mon Jul > > > >> 3 18:25:39 UTC 2006 i686 i686 i386 GNU/Linux > > > >> Using PETSc directory: /opt/petsc-2.3.2-p8 > > > >> Using PETSc arch: gcc-ifc-debug > > > >> ----------------------------------------- > > > >> Using C compiler: gcc -fPIC -Wall -Wwrite-strings -Wno-strict-aliasing > > > >> -g3 > > > >> Using Fortran compiler: ifort -fPIC -g > > > >> ----------------------------------------- > > > >> Using include paths: -I/opt/petsc-2.3.2-p8 > > > >> -I/opt/petsc-2.3.2-p8/bmake/gcc-ifc-debug -I/opt/petsc-2.3.2-p8/include > > > >> -I/opt/petsc-2.3.2-p8/externalpackages/mpich2-1.0.4p1/gcc-ifc-debug/include > > > >> -I/usr/X11R6/include > > > >> ------------------------------------------ > > > >> Using C linker: gcc -fPIC -Wall -Wwrite-strings -Wno-strict-aliasing -g3 > > > >> Using Fortran linker: ifort -fPIC -g > > > >> Using libraries: -Wl,-rpath,/opt/petsc-2.3.2-p8/lib/gcc-ifc-debug > > > >> -L/opt/petsc-2.3.2-p8/lib/gcc-ifc-debug -lpetscts -lpetscsnes -lpetscksp > > > >> -lpetscdm -lpetscmat -lpetscvec -lpetsc > > > >> -Wl,-rpath,/opt/petsc-2.3.2-p8/externalpackages/mpich2-1.0.4p1/gcc-ifc-debug/lib -L/opt/petsc-2.3.2-p8/externalpackages/mpich2-1.0.4p1/gcc-ifc-debug/lib -lmpich -lnsl -lrt -L/usr/X11R6/lib -lX11 -Wl,-rpath,/opt/petsc-2.3.2-p8/externalpackages/fblaslapack/gcc-ifc-debug -L/opt/petsc-2.3.2-p8/externalpackages/fblaslapack/gcc-ifc-debug -lflapack -Wl,-rpath,/opt/petsc-2.3.2-p8/externalpackages/fblaslapack/gcc-ifc-debug -L/opt/petsc-2.3.2-p8/externalpackages/fblaslapack/gcc-ifc-debug -lfblas -lm -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0 -L/usr/lib/gcc/i586-suse-linux/4.1.0 -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linux/lib -L/usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linux/lib -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../.. -L/usr/lib/gcc/i586-suse-linux/4.1.0/../../.. -ldl -lgcc_s -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0 -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linux/lib > > > >> -Wl,-rpath,/usr/lib/gcc/! > > > > i5! > > > >> 86-suse-linux/4.1.0/../../.. -Wl,-rpath,/opt/intel/fc/9.1.036/lib > > > >> -L/opt/intel/fc/9.1.036/lib > > > >> -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/ > > > >> -L/usr/lib/gcc/i586-suse-linux/4.1.0/ > > > >> -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../../ > > > >> -L/usr/lib/gcc/i586-suse-linux/4.1.0/../../../ -lifport -lifcore > > > >> -limf -lm -lipgo -lirc -lirc_s > > > >> -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0 > > > >> -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linux/lib -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../.. -lm -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0 -L/usr/lib/gcc/i586-suse-linux/4.1.0 -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linux/lib -L/usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linux/lib -Wl,-rpath,/usr/lib/gcc/i586-suse-linux/4.1.0/../../.. -L/usr/lib/gcc/i586-suse-linux/4.1.0/../../.. -ldl -lgcc_s > > > >> -ldl > > > >> > > > >> > > > >> > > > >> > > > > > > > > > > > > -- > > > > Lisandro Dalc?n > > > > --------------- > > > > Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) > > > > Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) > > > > Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) > > > > PTLC - G?emes 3450, (3000) Santa Fe, Argentina > > > > Tel/Fax: +54-(0)342-451.1594 > > > > > > > > > > > > > > > > > > -- One trouble is that despite this system, anyone who reads journals widely and critically is forced to realize that there are scarcely any bars to eventual publication. There seems to be no study too fragmented, no hypothesis too trivial, no literature citation too biased or too egotistical, no design too warped, no methodology too bungled, no presentation of results too inaccurate, too obscure, and too contradictory, no analysis too self-serving, no argument too circular, no conclusions too trifling or too unjustified, and no grammar and syntax too offensive for a paper to end up in print. -- Drummond Rennie From xubinbin2004 at gmail.com Wed Mar 14 00:56:21 2007 From: xubinbin2004 at gmail.com (Bin) Date: Tue, 13 Mar 2007 23:56:21 -0600 Subject: Help! Installing PETSc? Message-ID: <8f1c3d500703132256i748652cfpab532f99eeba989e@mail.gmail.com> Hello, I am trying to install PETSc on one of the workstations in my university. I use the following commands for configuration: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ config/configure.py \ > > --with-cc=mpicc \ > > --with-fc=mpif90 \ > > --with-mpi-dir=/opt/hpmpi/lib/linux_amd64/ \ > > --with-mpi-shared=yes \ > > LIBS=-lpgf90rtl \ > > --with-debugging=yes \ > > --with-superlu=yes \ > > --with-superlu-dir=/usr/apps/src/superlu/SuperLU_3.0 \ > > --download-superlu=no \ > > --with-superlu_dist=yes \ > > --with-superlu_dist-dir=/usr/apps/src/superlu/SuperLU_DIST_2.0 \ > > --download-superlu_dist=no \ > > --with-batch > > After this I got a message saying that: Since your compute nodes require use of a batch system or mpirun you must: > 1) Submit ./conftest to 1 processor of your batch system or system you are > > cross-compiling for; this will generate the file reconfigure > > 2) Run "python reconfigure" (to complete the configure process). > So I tyoed: mpirun -np 1 conftest and got the following error message: > ************************************************************* > > *mpid: MPI BUG: execvp failed: Cannot execute conftest: No such file or > directory* > > *MPI Application rank 0 exited before MPI_Init() with status 1* > > ************************************************************** > > I found that there is only conftest.o ! > > Can anybody help me? > > Thanks a lot. > > > > > > > > > > > > > > > > > -- > Yours truly, > > Bin Xu > -------------- next part -------------- An HTML attachment was scrubbed... URL: From shma7099 at student.uu.se Wed Mar 14 02:43:47 2007 From: shma7099 at student.uu.se (Shaman Mahmoudi) Date: Wed, 14 Mar 2007 08:43:47 +0100 Subject: Help! Installing PETSc? In-Reply-To: <8f1c3d500703132256i748652cfpab532f99eeba989e@mail.gmail.com> References: <8f1c3d500703132256i748652cfpab532f99eeba989e@mail.gmail.com> Message-ID: Hi, try mpirun -np 1 ./conftest With best regards, Shaman Mahmoudi On Mar 14, 2007, at 6:56 AM, Bin wrote: > mpirun -np 1 conftest -------------- next part -------------- An HTML attachment was scrubbed... URL: From xubinbin2004 at gmail.com Wed Mar 14 11:19:18 2007 From: xubinbin2004 at gmail.com (Bin) Date: Wed, 14 Mar 2007 10:19:18 -0600 Subject: Help! Installing PETSc? In-Reply-To: References: <8f1c3d500703132256i748652cfpab532f99eeba989e@mail.gmail.com> Message-ID: <8f1c3d500703140919vcdee126m2ed54b0644c45d42@mail.gmail.com> Hello, Thank you. I tried: mpirun -np 1 ./conftest and I got the same results. There is no executable file "conftest". Bin On 3/14/07, Shaman Mahmoudi wrote: > > Hi, > > try mpirun -np 1 ./conftest > > > With best regards, Shaman Mahmoudi > > On Mar 14, 2007, at 6:56 AM, Bin wrote: > > mpirun -np 1 conftest > > > > -- Yours truly, Bin Xu -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Wed Mar 14 11:39:06 2007 From: knepley at gmail.com (Matthew Knepley) Date: Wed, 14 Mar 2007 11:39:06 -0500 Subject: Help! Installing PETSc? In-Reply-To: <8f1c3d500703140919vcdee126m2ed54b0644c45d42@mail.gmail.com> References: <8f1c3d500703132256i748652cfpab532f99eeba989e@mail.gmail.com> <8f1c3d500703140919vcdee126m2ed54b0644c45d42@mail.gmail.com> Message-ID: On 3/14/07, Bin wrote: > Hello, > > Thank you. > > I tried: mpirun -np 1 ./conftest and I got the same results. > > There is no executable file "conftest". The configure produces this executable. Is it in $PETSC_DIR? Matt > Bin > > > On 3/14/07, Shaman Mahmoudi wrote: > > > > Hi, > > > > > > try mpirun -np 1 ./conftest > > > > > > With best regards, Shaman Mahmoudi > > > > > > > > On Mar 14, 2007, at 6:56 AM, Bin wrote: > > > > mpirun -np 1 conftest > > > > > > -- > Yours truly, > > Bin Xu -- One trouble is that despite this system, anyone who reads journals widely and critically is forced to realize that there are scarcely any bars to eventual publication. There seems to be no study too fragmented, no hypothesis too trivial, no literature citation too biased or too egotistical, no design too warped, no methodology too bungled, no presentation of results too inaccurate, too obscure, and too contradictory, no analysis too self-serving, no argument too circular, no conclusions too trifling or too unjustified, and no grammar and syntax too offensive for a paper to end up in print. -- Drummond Rennie From aja2111 at columbia.edu Wed Mar 14 11:41:06 2007 From: aja2111 at columbia.edu (Aron Ahmadia) Date: Wed, 14 Mar 2007 12:41:06 -0400 Subject: Help! Installing PETSc? In-Reply-To: <8f1c3d500703140919vcdee126m2ed54b0644c45d42@mail.gmail.com> References: <8f1c3d500703132256i748652cfpab532f99eeba989e@mail.gmail.com> <8f1c3d500703140919vcdee126m2ed54b0644c45d42@mail.gmail.com> Message-ID: <37604ab40703140941v56d7ce6ejb5749689cbe99d5f@mail.gmail.com> Try removing that -batch argument on the command-line for your original configure statement. I don't think it's doing what you want it to do. ~A On 3/14/07, Bin wrote: > Hello, > > Thank you. > > I tried: mpirun -np 1 ./conftest and I got the same results. > > There is no executable file "conftest". > > Bin > > > > On 3/14/07, Shaman Mahmoudi wrote: > > > > Hi, > > > > > > try mpirun -np 1 ./conftest > > > > > > With best regards, Shaman Mahmoudi > > > > > > > > On Mar 14, 2007, at 6:56 AM, Bin wrote: > > > > mpirun -np 1 conftest > > > > > > -- > Yours truly, > > Bin Xu From xubinbin2004 at gmail.com Wed Mar 14 13:30:50 2007 From: xubinbin2004 at gmail.com (Bin) Date: Wed, 14 Mar 2007 12:30:50 -0600 Subject: Help! Installing PETSc? In-Reply-To: <8f1c3d500703141109n429c7981v48c87b1199d117df@mail.gmail.com> References: <8f1c3d500703132256i748652cfpab532f99eeba989e@mail.gmail.com> <8f1c3d500703141109n429c7981v48c87b1199d117df@mail.gmail.com> Message-ID: <8f1c3d500703141130i6d7a38c9kbf4ccd7b859c633@mail.gmail.com> Thanks, it works. But when I try to configure PETSc with --with-shared > options, > > > config/configure.py \ > > --with-cc=mpicc \ > > --with-fc=mpif90 \ > > --with-mpi-dir=/opt/hpmpi/lib/linux_amd64/ \ > > --with-mpi-shared=yes \ > > LIBS=-lpgf90rtl \ > > --with-debugging=yes \ > > --download-superlu=no \ > > --download-superlu_dist=no \ > > --with-shared > > > I got the following errors: > > ********************************************************************************* > > > > UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.logfor details): > > > > --------------------------------------------------------------------------------------- > > C compiler you provided with -with-cc=mpicc does not work > > > > ********************************************************************************* > > Do you know why? > Thank you. > > On 3/14/07, Shaman Mahmoudi wrote: > > > > Hi, > > > > try mpirun -np 1 ./conftest > > > > > > With best regards, Shaman Mahmoudi > > > > On Mar 14, 2007, at 6:56 AM, Bin wrote: > > > > mpirun -np 1 conftest > > > > > > > > > > > > -- > Yours truly, > > Bin Xu > -- Yours truly, Bin Xu -------------- next part -------------- An HTML attachment was scrubbed... URL: From balay at mcs.anl.gov Wed Mar 14 13:39:04 2007 From: balay at mcs.anl.gov (Satish Balay) Date: Wed, 14 Mar 2007 13:39:04 -0500 (CDT) Subject: Help! Installing PETSc? In-Reply-To: <8f1c3d500703141130i6d7a38c9kbf4ccd7b859c633@mail.gmail.com> References: <8f1c3d500703132256i748652cfpab532f99eeba989e@mail.gmail.com> <8f1c3d500703141109n429c7981v48c87b1199d117df@mail.gmail.com> <8f1c3d500703141130i6d7a38c9kbf4ccd7b859c633@mail.gmail.com> Message-ID: this issues is followed up at petsc-maint [with configure.log] Satish On Wed, 14 Mar 2007, Bin wrote: > Thanks, it works. But when I try to configure PETSc with --with-shared > > options, > > > > > config/configure.py \ > > > --with-cc=mpicc \ > > > --with-fc=mpif90 \ > > > --with-mpi-dir=/opt/hpmpi/lib/linux_amd64/ \ > > > --with-mpi-shared=yes \ > > > LIBS=-lpgf90rtl \ > > > --with-debugging=yes \ > > > --download-superlu=no \ > > > --download-superlu_dist=no \ > > > --with-shared > > > > > I got the following errors: > > > > ********************************************************************************* > > > > > > UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.logfor > > details): > > > > > > > > --------------------------------------------------------------------------------------- > > > C compiler you provided with -with-cc=mpicc does not work > > > > > > > > ********************************************************************************* > > > > Do you know why? > > Thank you. > > > > On 3/14/07, Shaman Mahmoudi wrote: > > > > > > Hi, > > > > > > try mpirun -np 1 ./conftest > > > > > > > > > With best regards, Shaman Mahmoudi > > > > > > On Mar 14, 2007, at 6:56 AM, Bin wrote: > > > > > > mpirun -np 1 conftest > > > > > > > > > > > > > > > > > > > > -- > > Yours truly, > > > > Bin Xu > > > > > > From knepley at gmail.com Wed Mar 14 15:03:39 2007 From: knepley at gmail.com (Matthew Knepley) Date: Wed, 14 Mar 2007 15:03:39 -0500 Subject: Help! Installing PETSc? In-Reply-To: <8f1c3d500703141130i6d7a38c9kbf4ccd7b859c633@mail.gmail.com> References: <8f1c3d500703132256i748652cfpab532f99eeba989e@mail.gmail.com> <8f1c3d500703141109n429c7981v48c87b1199d117df@mail.gmail.com> <8f1c3d500703141130i6d7a38c9kbf4ccd7b859c633@mail.gmail.com> Message-ID: 1) Please send these to petsc-maint at mcs.anl.gov 2) We need to see your configure.log in order to understand any problem Thanks, Matt On 3/14/07, Bin wrote: > > > > > > > > Thanks, it works. But when I try to configure PETSc with --with-shared > options, > > > > > > > > config/configure.py \ > > > --with-cc=mpicc \ > > > --with-fc=mpif90 \ > > > --with-mpi-dir=/opt/hpmpi/lib/linux_amd64/ \ > > > --with-mpi-shared=yes \ > > > LIBS=-lpgf90rtl \ > > > --with-debugging=yes \ > > > --download-superlu=no \ > > > --download-superlu_dist=no \ > > > --with-shared > > I got the following errors: > > > > > > > > ********************************************************************************* > > > > > UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log > for details): > > > > --------------------------------------------------------------------------------------- > > > C compiler you provided with -with-cc=mpicc does not work > > > > ********************************************************************************* > > > > Do you know why? > > Thank you. > > > > On 3/14/07, Shaman Mahmoudi < shma7099 at student.uu.se> wrote: > > > > > > > > Hi, > > > > > > > > > try mpirun -np 1 ./conftest > > > > > > > > > With best regards, Shaman Mahmoudi > > > > > > > > > > > > On Mar 14, 2007, at 6:56 AM, Bin wrote: > > > > > > mpirun -np 1 conftest > > > > > > > > > > > -- > > Yours truly, > > > > Bin Xu > > > > > > -- > Yours truly, > > Bin Xu -- One trouble is that despite this system, anyone who reads journals widely and critically is forced to realize that there are scarcely any bars to eventual publication. There seems to be no study too fragmented, no hypothesis too trivial, no literature citation too biased or too egotistical, no design too warped, no methodology too bungled, no presentation of results too inaccurate, too obscure, and too contradictory, no analysis too self-serving, no argument too circular, no conclusions too trifling or too unjustified, and no grammar and syntax too offensive for a paper to end up in print. -- Drummond Rennie From mafunk at nmsu.edu Thu Mar 15 11:56:24 2007 From: mafunk at nmsu.edu (Matt Funk) Date: Thu, 15 Mar 2007 10:56:24 -0600 Subject: PETSC error Message-ID: <200703151056.25331.mafunk@nmsu.edu> Hi, i am getting the following error message in a program that uses PETSC: "Caught signal number 15 "Terminate: Somet process (or the batch system) has told this process to end" Does any body know what that means or what could possibly cause this? thanks mat From aja2111 at columbia.edu Thu Mar 15 12:31:06 2007 From: aja2111 at columbia.edu (Aron Ahmadia) Date: Thu, 15 Mar 2007 13:31:06 -0400 Subject: PETSC error In-Reply-To: <200703151056.25331.mafunk@nmsu.edu> References: <200703151056.25331.mafunk@nmsu.edu> Message-ID: <37604ab40703151031o3b8d5c9flcf33165147c83b71@mail.gmail.com> Hi Matt, SIGTERM (signal 15) is a very generic termination signal that can be sent by kill or killall, but also by a batch system. We'll be able to diagnose this better if you send the entire PETSC and MPI output from your program with the -log_info command. (i.e.) mpirun -np 4 ./my_prog -log_info It's possible you encountered an error earlier that caused one of the processes in the MPI group to send a SIGTERM to the others. Also, as a general note, if you run this in a debugger, it will stop at the place in the program where the signal is received. ~A On 3/15/07, Matt Funk wrote: > Hi, > > i am getting the following error message in a program that uses PETSC: > "Caught signal number 15 "Terminate: Somet process (or the batch system) has > told this process to end" > > Does any body know what that means or what could possibly cause this? > > > thanks > mat > > From shma7099 at student.uu.se Thu Mar 15 12:33:14 2007 From: shma7099 at student.uu.se (Shaman Mahmoudi) Date: Thu, 15 Mar 2007 18:33:14 +0100 Subject: PETSC error In-Reply-To: <200703151056.25331.mafunk@nmsu.edu> References: <200703151056.25331.mafunk@nmsu.edu> Message-ID: Hi, I got that problem when 1) The time you are allowed to be in the system exceeded and batch- system killed your processes. 2) I did not exit the program properly 3) One of the MPI installs afaik was a bit un-steady or not configured properly and caused error messages like this pop up after the program had exited (properly). With best regards, Shaman Mahmoudi On Mar 15, 2007, at 5:56 PM, Matt Funk wrote: > Hi, > > i am getting the following error message in a program that uses PETSC: > "Caught signal number 15 "Terminate: Somet process (or the batch > system) has > told this process to end" > > Does any body know what that means or what could possibly cause this? > > > thanks > mat > From mafunk at nmsu.edu Thu Mar 15 12:50:14 2007 From: mafunk at nmsu.edu (Matt Funk) Date: Thu, 15 Mar 2007 11:50:14 -0600 Subject: PETSC error In-Reply-To: References: <200703151056.25331.mafunk@nmsu.edu> Message-ID: <200703151150.15918.mafunk@nmsu.edu> Thanks for the replies, I'll run the program with -log_info, but that'll take a while. However, i suspect that it is MPI related, because, even though my program is pretty much static in terms of size it is being shuffled around in terms of process distribution. So, this problem only occurs in parallel. However, it seems to start out fine, but then slows down in the parallel communication section. Also, PETSC is not the base package i use. I interface to it, so i am not sure that PETSC is necessarily the problem. anyway, i'll see if the log file gives me anything. thanks mat On Thursday 15 March 2007 11:33, you wrote: > Hi, > > I got that problem when > > 1) The time you are allowed to be in the system exceeded and batch- > system killed your processes. > 2) I did not exit the program properly > 3) One of the MPI installs afaik was a bit un-steady or not > configured properly and caused error messages like this pop up after > the program had exited (properly). > > With best regards, Shaman Mahmoudi > > On Mar 15, 2007, at 5:56 PM, Matt Funk wrote: > > Hi, > > > > i am getting the following error message in a program that uses PETSC: > > "Caught signal number 15 "Terminate: Somet process (or the batch > > system) has > > told this process to end" > > > > Does any body know what that means or what could possibly cause this? > > > > > > thanks > > mat From gaatenek at irisa.fr Fri Mar 16 07:20:13 2007 From: gaatenek at irisa.fr (gaatenek at irisa.fr) Date: Fri, 16 Mar 2007 13:20:13 +0100 (CET) Subject: About LU Factors Message-ID: <1977.195.24.205.197.1174047613.squirrel@mail.irisa.fr> Hi, I am trying to found the condition number of the factors L and U of sparse matrix factorize by SuperLU in Petsc. The problem is that I not able to manipulate the L and U factors in my code. Please help me. Regards From hzhang at mcs.anl.gov Fri Mar 16 09:39:45 2007 From: hzhang at mcs.anl.gov (Hong Zhang) Date: Fri, 16 Mar 2007 09:39:45 -0500 (CDT) Subject: About LU Factors In-Reply-To: <1977.195.24.205.197.1174047613.squirrel@mail.irisa.fr> References: <1977.195.24.205.197.1174047613.squirrel@mail.irisa.fr> Message-ID: You can get condition number by running your code with the option '-mat_superlu_conditionnumber' Hong On Fri, 16 Mar 2007 gaatenek at irisa.fr wrote: > Hi, > I am trying to found the condition number of the factors L and U of sparse > matrix factorize by SuperLU in Petsc. The problem is that I not able to > manipulate the L and U factors in my code. > Please help me. > Regards > > From geenen at gmail.com Fri Mar 16 13:57:09 2007 From: geenen at gmail.com (Thomas Geenen) Date: Fri, 16 Mar 2007 13:57:09 -0500 Subject: building petsc on a POWER5+ machine Message-ID: <8aa042e10703161157p486a2fe2q39422b5b56c6f7d4@mail.gmail.com> dear petsc users, I build petsc on a POWER5+ machine. now i would like to add mumps. if i do that the way it should de done i end up with a version in which mumps hangs during the factorization phase. If i build mumps standalone linking to essl and pessl i get a working version of mumps. I tried telling the petsc configuration to use essl and pessl instead of scalapack and blacs like this --with-scalapack=yes --with-scalapack-lib=/usr/lib/libpessl.a --with-blacs=yes --with-blacs-lib=/usr/lib/libessl.a --with-scalapack-include=/usr/include/ with-blacs-include=/usr/include/ resulting in ================================================================================= TESTING: check from config.libraries(/ptmp/huangwei/petsc-2.3.2-p8/python/BuildSystem/config/libraries.py:108) ********************************************************************************* UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log for details): --------------------------------------------------------------------------------------- --with-blacs-lib=['/usr/lib/libessl.a'] and --with-blacs-include=/usr/include/ did not work ********************************************************************************* so how can i either link my working version of mumps to petsc without having a scalapack or blacs lib or include, or use essl and pessl as substitute for scalapack and blacs and rebuild petsc cheers Thomas complete config option list ./config/configure.py --with-memcmp-ok --with-endian=big --sizeof_char=1 --sizeof_void_p=8 --sizeof_short=2 --sizeof_int=4 --sizeof_long=8 --sizeof_long_long=8 --sizeof_float=4 --sizeof_double=8 --bits_per_byte=8 --sizeof_MPI_Comm=4 --sizeof_MPI_Fint=4 --with-batch --with-shared=0 --with-mpirun=mpirun.lsf --with-mpi-dir=/usr/lpp/ppe.poe/ --with-mpi-shared=yes --with-fc=mpxlf_r --with-cc=mpcc_r --with-hypre=yes --with-mumps=yes --with-hypre-dir=/blhome/geenen/source/petsc-2.3.2-p8/externalpackages/hypre-1.11.1b/src/hypre/ --with-mumps-dir=/blhome/geenen/source/petsc-2.3.2-p8/externalpackages/MUMPS_4.6.3/ --with-scalapack=yes --with-scalapack-lib=/usr/lib/libpessl.a --with-blacs=yes --with-blacs-lib=/usr/lib/libessl.a --with-scalapack-include=/usr/include/ with-blacs-include=/usr/include/ From bsmith at mcs.anl.gov Fri Mar 16 14:00:15 2007 From: bsmith at mcs.anl.gov (Barry Smith) Date: Fri, 16 Mar 2007 14:00:15 -0500 (CDT) Subject: building petsc on a POWER5+ machine In-Reply-To: <8aa042e10703161157p486a2fe2q39422b5b56c6f7d4@mail.gmail.com> References: <8aa042e10703161157p486a2fe2q39422b5b56c6f7d4@mail.gmail.com> Message-ID: Thomas, Please send configure.log to petsc-maint at mcs.anl.gov Thanks Barry On Fri, 16 Mar 2007, Thomas Geenen wrote: > dear petsc users, > > I build petsc on a POWER5+ machine. > now i would like to add mumps. > if i do that the way it should de done i end up with a version in > which mumps hangs during the factorization phase. > If i build mumps standalone linking to essl and pessl i get a working > version of mumps. > > I tried telling the petsc configuration to use essl and pessl instead > of scalapack and blacs like this > --with-scalapack=yes --with-scalapack-lib=/usr/lib/libpessl.a > --with-blacs=yes --with-blacs-lib=/usr/lib/libessl.a > --with-scalapack-include=/usr/include/ > with-blacs-include=/usr/include/ > > resulting in > ================================================================================= > TESTING: check from > config.libraries(/ptmp/huangwei/petsc-2.3.2-p8/python/BuildSystem/config/libraries.py:108) > ********************************************************************************* > UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log > for details): > --------------------------------------------------------------------------------------- > --with-blacs-lib=['/usr/lib/libessl.a'] and > --with-blacs-include=/usr/include/ did not work > ********************************************************************************* > > so how can i either link my working version of mumps to petsc without > having a scalapack or blacs lib or include, > or use essl and pessl as substitute for scalapack and blacs and rebuild petsc > > cheers > Thomas > > complete config option list > ./config/configure.py --with-memcmp-ok --with-endian=big > --sizeof_char=1 --sizeof_void_p=8 --sizeof_short=2 --sizeof_int=4 > --sizeof_long=8 --sizeof_long_long=8 --sizeof_float=4 > --sizeof_double=8 --bits_per_byte=8 --sizeof_MPI_Comm=4 > --sizeof_MPI_Fint=4 --with-batch --with-shared=0 > --with-mpirun=mpirun.lsf --with-mpi-dir=/usr/lpp/ppe.poe/ > --with-mpi-shared=yes --with-fc=mpxlf_r --with-cc=mpcc_r > --with-hypre=yes --with-mumps=yes > --with-hypre-dir=/blhome/geenen/source/petsc-2.3.2-p8/externalpackages/hypre-1.11.1b/src/hypre/ > --with-mumps-dir=/blhome/geenen/source/petsc-2.3.2-p8/externalpackages/MUMPS_4.6.3/ > --with-scalapack=yes --with-scalapack-lib=/usr/lib/libpessl.a > --with-blacs=yes --with-blacs-lib=/usr/lib/libessl.a > --with-scalapack-include=/usr/include/ > with-blacs-include=/usr/include/ > > From balay at mcs.anl.gov Fri Mar 16 17:27:59 2007 From: balay at mcs.anl.gov (Satish Balay) Date: Fri, 16 Mar 2007 17:27:59 -0500 (CDT) Subject: building petsc on a POWER5+ machine In-Reply-To: <8aa042e10703161157p486a2fe2q39422b5b56c6f7d4@mail.gmail.com> References: <8aa042e10703161157p486a2fe2q39422b5b56c6f7d4@mail.gmail.com> Message-ID: A couple of issues here.. - xlf doesn't compile [mumps] f90 code correctly - You'll need the additional compiler option "-qnosave". This can be specified as: --with-fc='mpxlf_r -qnosave' - Alternative would be to use 'mpxlf90' but either blacs or scalapack sources can't be compiled with it due to fixed/free from issue. I guess one could use: --with-fc='mpxlf90 -qnosave' > --with-scalapack-lib=/usr/lib/libpessl.a --with-blacs-lib=/usr/lib/libessl.a I don't think essl has either scalapack or blacs functionality. You should just use: --download-blacs=1 --download-scalapack=1 Satish On Fri, 16 Mar 2007, Thomas Geenen wrote: > dear petsc users, > > I build petsc on a POWER5+ machine. > now i would like to add mumps. > if i do that the way it should de done i end up with a version in > which mumps hangs during the factorization phase. > If i build mumps standalone linking to essl and pessl i get a working > version of mumps. > > I tried telling the petsc configuration to use essl and pessl instead > of scalapack and blacs like this > --with-scalapack=yes --with-scalapack-lib=/usr/lib/libpessl.a > --with-blacs=yes --with-blacs-lib=/usr/lib/libessl.a > --with-scalapack-include=/usr/include/ > with-blacs-include=/usr/include/ > > resulting in > ================================================================================= > TESTING: check from > config.libraries(/ptmp/huangwei/petsc-2.3.2-p8/python/BuildSystem/config/libraries.py:108) > ********************************************************************************* > UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log > for details): > --------------------------------------------------------------------------------------- > --with-blacs-lib=['/usr/lib/libessl.a'] and > --with-blacs-include=/usr/include/ did not work > ********************************************************************************* > > so how can i either link my working version of mumps to petsc without > having a scalapack or blacs lib or include, > or use essl and pessl as substitute for scalapack and blacs and rebuild petsc > > cheers > Thomas > > complete config option list > ./config/configure.py --with-memcmp-ok --with-endian=big > --sizeof_char=1 --sizeof_void_p=8 --sizeof_short=2 --sizeof_int=4 > --sizeof_long=8 --sizeof_long_long=8 --sizeof_float=4 > --sizeof_double=8 --bits_per_byte=8 --sizeof_MPI_Comm=4 > --sizeof_MPI_Fint=4 --with-batch --with-shared=0 > --with-mpirun=mpirun.lsf --with-mpi-dir=/usr/lpp/ppe.poe/ > --with-mpi-shared=yes --with-fc=mpxlf_r --with-cc=mpcc_r > --with-hypre=yes --with-mumps=yes > --with-hypre-dir=/blhome/geenen/source/petsc-2.3.2-p8/externalpackages/hypre-1.11.1b/src/hypre/ > --with-mumps-dir=/blhome/geenen/source/petsc-2.3.2-p8/externalpackages/MUMPS_4.6.3/ > --with-scalapack=yes --with-scalapack-lib=/usr/lib/libpessl.a > --with-blacs=yes --with-blacs-lib=/usr/lib/libessl.a > --with-scalapack-include=/usr/include/ > with-blacs-include=/usr/include/ > > From geenen at gmail.com Fri Mar 16 18:37:45 2007 From: geenen at gmail.com (Thomas Geenen) Date: Fri, 16 Mar 2007 18:37:45 -0500 Subject: building petsc on a POWER5+ machine In-Reply-To: References: <8aa042e10703161157p486a2fe2q39422b5b56c6f7d4@mail.gmail.com> Message-ID: <8aa042e10703161637ie32db62tf70e78c8663e6560@mail.gmail.com> oke maybe i did not explain it clear enough. i already build petsc with support for hypre and mumps the more or less default way. yes you will have to do a lot of tweaking but that went oke. The problem was that in this version Mumps would hang in the factorisation phase. the next thing i did was building mumps standalone linking it to essl and pessl this gave a working standalone version of mumps. So the next thing i tried was building petsc with mumps supprt without blacs or scalapack but with essl and pessl to get a working version of mumps accesible through petsc. this however seems tricky since scalapack does not build with essl and/or pessl and mumps only builds if you build scalapack as well. a worksround would be to link my working version of mumps directly to petsc and maybe rebuilding some part of mumps that deals with the interface? i also tried ./config/configure.py --with-memcmp-ok --with-endian=big --sizeof_char=1 --sizeof_void_p=8 --sizeof_short=2 --sizeof_int=4 --sizeof_long=8 --sizeof_long_long=8 --sizeof_float=4 --sizeof_double=8 --bits_per_byte=8 --sizeof_MPI_Comm=4 --sizeof_MPI_Fint=4 --with-batch --with-shared=0 --with-mpirun=mpirun.lsf --with-mpi-dir=/usr/lpp/ppe.poe/ --with-mpi-shared=yes --with-fc=mpxlf_r --with-cc=mpcc_r --with-hypre=yes --with-mumps=yes --with-hypre-dir=/blhome/geenen/source/petsc-2.3.2-p8/externalpackages/hypre-1.11.1b/src/hypre/ --with-mumps-dir=/blhome/geenen/source/petsc-2.3.2-p8/externalpackages/MUMPS_4.6.3/ --with-scalapack-lib=/usr/lib/libpessl.a --with-blacs-lib=/usr/lib/libessl.a --with-scalapack-include=/usr/include/ with-blacs-include=/usr/include/ ================================================================================= Configuring PETSc to compile on your system ================================================================================= TESTING: check from config.libraries(/ptmp/huangwei/petsc-2.3.2-p8/python/BuildSystem/config/libraries.py:108) ********************************************************************************* UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log for details): --------------------------------------------------------------------------------------- --with-blacs-lib=['/usr/lib/libessl.a'] and --with-blacs-include=/usr/include/ did not work ********************************************************************************* so without the --with-blacs and --with-scalapack but the build system tries to execute this int main() { blacs_pinfo() ; return 0; } and that results of course in error message = {ld: 0711-317 ERROR: Undefined symbol: .blacs_pinfo thomas On 3/16/07, Satish Balay wrote: > A couple of issues here.. > > - xlf doesn't compile [mumps] f90 code correctly - You'll need the > additional compiler option "-qnosave". This can be specified as: > --with-fc='mpxlf_r -qnosave' > > - Alternative would be to use 'mpxlf90' but either blacs or scalapack > sources can't be compiled with it due to fixed/free from issue. I > guess one could use: --with-fc='mpxlf90 -qnosave' > > > --with-scalapack-lib=/usr/lib/libpessl.a --with-blacs-lib=/usr/lib/libessl.a > > I don't think essl has either scalapack or blacs functionality. You should > just use: --download-blacs=1 --download-scalapack=1 > > Satish > > On Fri, 16 Mar 2007, Thomas Geenen wrote: > > > dear petsc users, > > > > I build petsc on a POWER5+ machine. > > now i would like to add mumps. > > if i do that the way it should de done i end up with a version in > > which mumps hangs during the factorization phase. > > If i build mumps standalone linking to essl and pessl i get a working > > version of mumps. > > > > I tried telling the petsc configuration to use essl and pessl instead > > of scalapack and blacs like this > > --with-scalapack=yes --with-scalapack-lib=/usr/lib/libpessl.a > > --with-blacs=yes --with-blacs-lib=/usr/lib/libessl.a > > --with-scalapack-include=/usr/include/ > > with-blacs-include=/usr/include/ > > > > resulting in > > ================================================================================= > > TESTING: check from > > config.libraries(/ptmp/huangwei/petsc-2.3.2-p8/python/BuildSystem/config/libraries.py:108) > > ********************************************************************************* > > UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log > > for details): > > --------------------------------------------------------------------------------------- > > --with-blacs-lib=['/usr/lib/libessl.a'] and > > --with-blacs-include=/usr/include/ did not work > > ********************************************************************************* > > > > so how can i either link my working version of mumps to petsc without > > having a scalapack or blacs lib or include, > > or use essl and pessl as substitute for scalapack and blacs and rebuild petsc > > > > cheers > > Thomas > > > > complete config option list > > ./config/configure.py --with-memcmp-ok --with-endian=big > > --sizeof_char=1 --sizeof_void_p=8 --sizeof_short=2 --sizeof_int=4 > > --sizeof_long=8 --sizeof_long_long=8 --sizeof_float=4 > > --sizeof_double=8 --bits_per_byte=8 --sizeof_MPI_Comm=4 > > --sizeof_MPI_Fint=4 --with-batch --with-shared=0 > > --with-mpirun=mpirun.lsf --with-mpi-dir=/usr/lpp/ppe.poe/ > > --with-mpi-shared=yes --with-fc=mpxlf_r --with-cc=mpcc_r > > --with-hypre=yes --with-mumps=yes > > --with-hypre-dir=/blhome/geenen/source/petsc-2.3.2-p8/externalpackages/hypre-1.11.1b/src/hypre/ > > --with-mumps-dir=/blhome/geenen/source/petsc-2.3.2-p8/externalpackages/MUMPS_4.6.3/ > > --with-scalapack=yes --with-scalapack-lib=/usr/lib/libpessl.a > > --with-blacs=yes --with-blacs-lib=/usr/lib/libessl.a > > --with-scalapack-include=/usr/include/ > > with-blacs-include=/usr/include/ > > > > > > From balay at mcs.anl.gov Fri Mar 16 19:03:20 2007 From: balay at mcs.anl.gov (Satish Balay) Date: Fri, 16 Mar 2007 19:03:20 -0500 (CDT) Subject: building petsc on a POWER5+ machine In-Reply-To: <8aa042e10703161637ie32db62tf70e78c8663e6560@mail.gmail.com> References: <8aa042e10703161157p486a2fe2q39422b5b56c6f7d4@mail.gmail.com> <8aa042e10703161637ie32db62tf70e78c8663e6560@mail.gmail.com> Message-ID: On Fri, 16 Mar 2007, Thomas Geenen wrote: > oke maybe i did not explain it clear enough. > i already build petsc with support for hypre and mumps the more or > less default way. yes you will have to do a lot of tweaking but that > went oke. > The problem was that in this version Mumps would hang in the > factorisation phase. Please try rebuilding with the following - and see if the hang goes away. --with-fc='mpxlf_r -qnosave' --download-mumps=1 --download-blacs=1 --download-scalapack=1 You might have to do 'rm -rf externalpackages' before attempting this. Satish From gdiso at ustc.edu Sat Mar 17 05:28:02 2007 From: gdiso at ustc.edu (Gong Ding) Date: Sat, 17 Mar 2007 18:28:02 +0800 Subject: =?gb2312?B?VHJ1c3QgUmVnaW9uIHByb2JsZW0=?= Message-ID: <7A0AE5E2-4EB8-493E-AF41-14F3CCE3152A@u.washington.edu@> Hi all, I use PETSC's nonelinear solver to do emiconductor simulation. I found a strange problem with trust region solvers. It usually works well if LU method is used for inner linear solver. But when krylov method i.e. GMRES, is employed, it often stops due to pnorm is small, but still remains large error of function norm. How to solve this problem? LU method is much slower than i.e. GMRES. G.D. -- USTC Alumni Email System From knepley at gmail.com Sat Mar 17 08:15:41 2007 From: knepley at gmail.com (Matthew Knepley) Date: Sat, 17 Mar 2007 08:15:41 -0500 Subject: Trust Region problem In-Reply-To: <-6747730121110201900@unknownmsgid> References: <-6747730121110201900@unknownmsgid> Message-ID: On 3/17/07, Gong Ding wrote: > Hi all, > I use PETSC's nonelinear solver to do > emiconductor simulation. I found a strange problem > with trust region solvers. It usually works well > if LU method is used for inner linear solver. But > when krylov method i.e. GMRES, is employed, it > often stops due to pnorm is small, but still > remains large error of function norm. How to solve > this problem? LU method is much slower than i.e. > GMRES. This sounds like GMRES is not actually converging (you can check this using -ksp_monitor). You need a more powerful preconditioner, which unfortunately usually depends a lot on the problem. Matt > G.D. > -- > USTC Alumni Email System > > -- One trouble is that despite this system, anyone who reads journals widely and critically is forced to realize that there are scarcely any bars to eventual publication. There seems to be no study too fragmented, no hypothesis too trivial, no literature citation too biased or too egotistical, no design too warped, no methodology too bungled, no presentation of results too inaccurate, too obscure, and too contradictory, no analysis too self-serving, no argument too circular, no conclusions too trifling or too unjustified, and no grammar and syntax too offensive for a paper to end up in print. -- Drummond Rennie From knutert at stud.ntnu.no Mon Mar 19 08:29:17 2007 From: knutert at stud.ntnu.no (Knut Erik Teigen) Date: Mon, 19 Mar 2007 14:29:17 +0100 Subject: Problems porting code to an IBM power5+ machine In-Reply-To: References: <8aa042e10703161157p486a2fe2q39422b5b56c6f7d4@mail.gmail.com> <8aa042e10703161637ie32db62tf70e78c8663e6560@mail.gmail.com> Message-ID: <1174310957.5365.27.camel@iept0415.ivt.ntnu.no> Hello I have got a working code on my local machine( Pentium 4), but when I try to run the code on a power5+ machine, the equation solver won't converge. It seems like it calculates the wrong right hand side norm. Below is the result with run-time options "-ksp_type cg -pc_type icc -ksp_monitor -ksp_view -info First with the code running on the power5+ machine, then on my local machine. I've also printed the right hand side, jacobian matrix and solution for a small 3x3 grid. Can anyone help me figure out what's wrong? Regards, Knut Erik Teigen Code running on Power5+: rhs: 980 980 980 -0 -0 -0 -980 -980 -980 jacobian: row 0: (0, 20) (1, -10) (3, -10) row 1: (0, -10) (1, 30) (2, -10) (4, -10) row 2: (1, -10) (2, 20) (5, -10) row 3: (0, -10) (3, 30) (4, -10) (6, -10) row 4: (1, -10) (3, -10) (4, 40) (5, -10) (7, -10) row 5: (2, -10) (4, -10) (5, 30) (8, -10) row 6: (3, -10) (6, 20) (7, -10) row 7: (4, -10) (6, -10) (7, 30) (8, -10) row 8: (5, -10) (7, -10) (8, 20) [0] PCSetUpSetting up new PC [0] PetscCommDuplicateDuplicating a communicator 1 4 max tags = 1073741823 [0] PetscCommDuplicateUsing internal PETSc communicator 1 4 [0] PetscCommDuplicateUsing internal PETSc communicator 1 4 0 KSP Residual norm 7.410163701832e+28 1 KSP Residual norm 6.464393707520e+11 [0] KSPDefaultConvergedLinear solver has converged. Residual norm 6.46439e+11 is less than relative tolerance 1e-07 times initial right hand side norm 7.41016e+28 at iteration 1 KSP Object: type: cg maximum iterations=10000, initial guess is zero tolerances: relative=1e-07, absolute=1e-50, divergence=10000 left preconditioning PC Object: type: icc ICC: 0 levels of fill ICC: factor fill ratio allocated 1 ICC: factor fill ratio needed 0.636364 Factored matrix follows Matrix Object: type=seqsbaij, rows=9, cols=9 total: nonzeros=21, allocated nonzeros=21 block size is 1 linear system matrix = precond matrix: Matrix Object: type=seqaij, rows=9, cols=9 total: nonzeros=33, allocated nonzeros=45 not using I-node routines solution: 6.37205e-09 7.13167e-09 7.49911e-09 -2.48277e-09 -4.56885e-10 0 0 0 0 Code running on local machine: rhs: 980 980 980 -0 -0 -0 -980 -980 -980 jacobian: row 0: (0, 20) (1, -10) (3, -10) row 1: (0, -10) (1, 30) (2, -10) (4, -10) row 2: (1, -10) (2, 20) (5, -10) row 3: (0, -10) (3, 30) (4, -10) (6, -10) row 4: (1, -10) (3, -10) (4, 40) (5, -10) (7, -10) row 5: (2, -10) (4, -10) (5, 30) (8, -10) row 6: (3, -10) (6, 20) (7, -10) row 7: (4, -10) (6, -10) (7, 30) (8, -10) row 8: (5, -10) (7, -10) (8, 20) [0] PCSetUp(): Setting up new PC [0] PetscCommDuplicate(): Duplicating a communicator 1140850689 -2080374783 max tags = 2147483647 [0] PetscCommDuplicate(): Using internal PETSc communicator 1140850689 -2080374783 [0] PetscCommDuplicate(): Using internal PETSc communicator 1140850689 -2080374783 0 KSP Residual norm 2.505507810276e+02 1 KSP Residual norm 3.596555656581e+01 2 KSP Residual norm 2.632672485513e+00 3 KSP Residual norm 1.888285055287e-01 4 KSP Residual norm 7.029433008806e-03 5 KSP Residual norm 3.635267067420e-14 [0] KSPDefaultConverged(): Linear solver has converged. Residual norm 3.63527e-14 is less than relative tolerance 1e-07 times initial right hand side norm 250.551 at iteration 5 KSP Object: type: cg maximum iterations=10000, initial guess is zero tolerances: relative=1e-07, absolute=1e-50, divergence=10000 left preconditioning PC Object: type: icc ICC: 0 levels of fill ICC: factor fill ratio allocated 1 ICC: factor fill ratio needed 0.636364 Factored matrix follows Matrix Object: type=seqsbaij, rows=9, cols=9 total: nonzeros=21, allocated nonzeros=21 block size is 1 linear system matrix = precond matrix: Matrix Object: type=seqaij, rows=9, cols=9 total: nonzeros=33, allocated nonzeros=45 not using I-node routines solution: 92.3023 92.3023 92.3023 -5.69767 -5.69767 -5.69767 -103.698 -103.698 -103.698 From knepley at gmail.com Mon Mar 19 08:39:42 2007 From: knepley at gmail.com (Matthew Knepley) Date: Mon, 19 Mar 2007 08:39:42 -0500 Subject: Problems porting code to an IBM power5+ machine In-Reply-To: <1174310957.5365.27.camel@iept0415.ivt.ntnu.no> References: <8aa042e10703161157p486a2fe2q39422b5b56c6f7d4@mail.gmail.com> <8aa042e10703161637ie32db62tf70e78c8663e6560@mail.gmail.com> <1174310957.5365.27.camel@iept0415.ivt.ntnu.no> Message-ID: This smells like a memory overwrite or use of an uninitialized variable. The initial norm is 1e28. 1) Try Jacobi instead of ICC to see if it is localized to the PC 2) Run with valgrind or something similar to check for a memory overwrite 3) Maybe insert CHKMEMQ statements into the code Thanks, Matt On 3/19/07, Knut Erik Teigen wrote: > Hello > > I have got a working code on my local machine( Pentium 4), but when I > try to run the code on a power5+ machine, the equation solver won't > converge. It seems like it calculates the wrong right hand side norm. > Below is the result with run-time options > "-ksp_type cg -pc_type icc -ksp_monitor -ksp_view -info > First with the code running on the power5+ machine, then on my local > machine. I've also printed the right hand side, jacobian matrix and > solution for a small 3x3 grid. > > Can anyone help me figure out what's wrong? > > Regards, > Knut Erik Teigen > > Code running on Power5+: > rhs: > 980 > 980 > 980 > -0 > -0 > -0 > -980 > -980 > -980 > jacobian: > row 0: (0, 20) (1, -10) (3, -10) > row 1: (0, -10) (1, 30) (2, -10) (4, -10) > row 2: (1, -10) (2, 20) (5, -10) > row 3: (0, -10) (3, 30) (4, -10) (6, -10) > row 4: (1, -10) (3, -10) (4, 40) (5, -10) (7, -10) > row 5: (2, -10) (4, -10) (5, 30) (8, -10) > row 6: (3, -10) (6, 20) (7, -10) > row 7: (4, -10) (6, -10) (7, 30) (8, -10) > row 8: (5, -10) (7, -10) (8, 20) > [0] PCSetUpSetting up new PC > [0] PetscCommDuplicateDuplicating a communicator 1 4 max tags = > 1073741823 > [0] PetscCommDuplicateUsing internal PETSc communicator 1 4 > [0] PetscCommDuplicateUsing internal PETSc communicator 1 4 > 0 KSP Residual norm 7.410163701832e+28 > 1 KSP Residual norm 6.464393707520e+11 > [0] KSPDefaultConvergedLinear solver has converged. Residual norm > 6.46439e+11 is less than relative tolerance 1e-07 times initial right > hand side norm 7.41016e+28 at iteration 1 > KSP Object: > type: cg > maximum iterations=10000, initial guess is zero > tolerances: relative=1e-07, absolute=1e-50, divergence=10000 > left preconditioning > PC Object: > type: icc > ICC: 0 levels of fill > ICC: factor fill ratio allocated 1 > ICC: factor fill ratio needed 0.636364 > Factored matrix follows > Matrix Object: > type=seqsbaij, rows=9, cols=9 > total: nonzeros=21, allocated nonzeros=21 > block size is 1 > linear system matrix = precond matrix: > Matrix Object: > type=seqaij, rows=9, cols=9 > total: nonzeros=33, allocated nonzeros=45 > not using I-node routines > solution: > 6.37205e-09 > 7.13167e-09 > 7.49911e-09 > -2.48277e-09 > -4.56885e-10 > 0 > 0 > 0 > 0 > > Code running on local machine: > rhs: > 980 > 980 > 980 > -0 > -0 > -0 > -980 > -980 > -980 > jacobian: > row 0: (0, 20) (1, -10) (3, -10) > row 1: (0, -10) (1, 30) (2, -10) (4, -10) > row 2: (1, -10) (2, 20) (5, -10) > row 3: (0, -10) (3, 30) (4, -10) (6, -10) > row 4: (1, -10) (3, -10) (4, 40) (5, -10) (7, -10) > row 5: (2, -10) (4, -10) (5, 30) (8, -10) > row 6: (3, -10) (6, 20) (7, -10) > row 7: (4, -10) (6, -10) (7, 30) (8, -10) > row 8: (5, -10) (7, -10) (8, 20) > [0] PCSetUp(): Setting up new PC > [0] PetscCommDuplicate(): Duplicating a communicator 1140850689 > -2080374783 max tags = 2147483647 > [0] PetscCommDuplicate(): Using internal PETSc communicator 1140850689 > -2080374783 > [0] PetscCommDuplicate(): Using internal PETSc communicator 1140850689 > -2080374783 > 0 KSP Residual norm 2.505507810276e+02 > 1 KSP Residual norm 3.596555656581e+01 > 2 KSP Residual norm 2.632672485513e+00 > 3 KSP Residual norm 1.888285055287e-01 > 4 KSP Residual norm 7.029433008806e-03 > 5 KSP Residual norm 3.635267067420e-14 > [0] KSPDefaultConverged(): Linear solver has converged. Residual norm > 3.63527e-14 is less than relative tolerance 1e-07 times initial right > hand side norm 250.551 at iteration 5 > KSP Object: > type: cg > maximum iterations=10000, initial guess is zero > tolerances: relative=1e-07, absolute=1e-50, divergence=10000 > left preconditioning > PC Object: > type: icc > ICC: 0 levels of fill > ICC: factor fill ratio allocated 1 > ICC: factor fill ratio needed 0.636364 > Factored matrix follows > Matrix Object: > type=seqsbaij, rows=9, cols=9 > total: nonzeros=21, allocated nonzeros=21 > block size is 1 > linear system matrix = precond matrix: > Matrix Object: > type=seqaij, rows=9, cols=9 > total: nonzeros=33, allocated nonzeros=45 > not using I-node routines > solution: > 92.3023 > 92.3023 > 92.3023 > -5.69767 > -5.69767 > -5.69767 > -103.698 > -103.698 > -103.698 > > > > -- One trouble is that despite this system, anyone who reads journals widely and critically is forced to realize that there are scarcely any bars to eventual publication. There seems to be no study too fragmented, no hypothesis too trivial, no literature citation too biased or too egotistical, no design too warped, no methodology too bungled, no presentation of results too inaccurate, too obscure, and too contradictory, no analysis too self-serving, no argument too circular, no conclusions too trifling or too unjustified, and no grammar and syntax too offensive for a paper to end up in print. -- Drummond Rennie From bsmith at mcs.anl.gov Mon Mar 19 11:22:23 2007 From: bsmith at mcs.anl.gov (Barry Smith) Date: Mon, 19 Mar 2007 11:22:23 -0500 (CDT) Subject: Problems porting code to an IBM power5+ machine In-Reply-To: <1174310957.5365.27.camel@iept0415.ivt.ntnu.no> References: <8aa042e10703161157p486a2fe2q39422b5b56c6f7d4@mail.gmail.com> <8aa042e10703161637ie32db62tf70e78c8663e6560@mail.gmail.com> <1174310957.5365.27.camel@iept0415.ivt.ntnu.no> Message-ID: On the power5 if the libraries are compiled with --with-debugging=0 build another set (different PETSC_ARCH) with --with-debugging=1 and see what happens. IBM's optimizers are very buggy. Barry On Mon, 19 Mar 2007, Knut Erik Teigen wrote: > Hello > > I have got a working code on my local machine( Pentium 4), but when I > try to run the code on a power5+ machine, the equation solver won't > converge. It seems like it calculates the wrong right hand side norm. > Below is the result with run-time options > "-ksp_type cg -pc_type icc -ksp_monitor -ksp_view -info > First with the code running on the power5+ machine, then on my local > machine. I've also printed the right hand side, jacobian matrix and > solution for a small 3x3 grid. > > Can anyone help me figure out what's wrong? > > Regards, > Knut Erik Teigen > > Code running on Power5+: > rhs: > 980 > 980 > 980 > -0 > -0 > -0 > -980 > -980 > -980 > jacobian: > row 0: (0, 20) (1, -10) (3, -10) > row 1: (0, -10) (1, 30) (2, -10) (4, -10) > row 2: (1, -10) (2, 20) (5, -10) > row 3: (0, -10) (3, 30) (4, -10) (6, -10) > row 4: (1, -10) (3, -10) (4, 40) (5, -10) (7, -10) > row 5: (2, -10) (4, -10) (5, 30) (8, -10) > row 6: (3, -10) (6, 20) (7, -10) > row 7: (4, -10) (6, -10) (7, 30) (8, -10) > row 8: (5, -10) (7, -10) (8, 20) > [0] PCSetUpSetting up new PC > [0] PetscCommDuplicateDuplicating a communicator 1 4 max tags = > 1073741823 > [0] PetscCommDuplicateUsing internal PETSc communicator 1 4 > [0] PetscCommDuplicateUsing internal PETSc communicator 1 4 > 0 KSP Residual norm 7.410163701832e+28 > 1 KSP Residual norm 6.464393707520e+11 > [0] KSPDefaultConvergedLinear solver has converged. Residual norm > 6.46439e+11 is less than relative tolerance 1e-07 times initial right > hand side norm 7.41016e+28 at iteration 1 > KSP Object: > type: cg > maximum iterations=10000, initial guess is zero > tolerances: relative=1e-07, absolute=1e-50, divergence=10000 > left preconditioning > PC Object: > type: icc > ICC: 0 levels of fill > ICC: factor fill ratio allocated 1 > ICC: factor fill ratio needed 0.636364 > Factored matrix follows > Matrix Object: > type=seqsbaij, rows=9, cols=9 > total: nonzeros=21, allocated nonzeros=21 > block size is 1 > linear system matrix = precond matrix: > Matrix Object: > type=seqaij, rows=9, cols=9 > total: nonzeros=33, allocated nonzeros=45 > not using I-node routines > solution: > 6.37205e-09 > 7.13167e-09 > 7.49911e-09 > -2.48277e-09 > -4.56885e-10 > 0 > 0 > 0 > 0 > > Code running on local machine: > rhs: > 980 > 980 > 980 > -0 > -0 > -0 > -980 > -980 > -980 > jacobian: > row 0: (0, 20) (1, -10) (3, -10) > row 1: (0, -10) (1, 30) (2, -10) (4, -10) > row 2: (1, -10) (2, 20) (5, -10) > row 3: (0, -10) (3, 30) (4, -10) (6, -10) > row 4: (1, -10) (3, -10) (4, 40) (5, -10) (7, -10) > row 5: (2, -10) (4, -10) (5, 30) (8, -10) > row 6: (3, -10) (6, 20) (7, -10) > row 7: (4, -10) (6, -10) (7, 30) (8, -10) > row 8: (5, -10) (7, -10) (8, 20) > [0] PCSetUp(): Setting up new PC > [0] PetscCommDuplicate(): Duplicating a communicator 1140850689 > -2080374783 max tags = 2147483647 > [0] PetscCommDuplicate(): Using internal PETSc communicator 1140850689 > -2080374783 > [0] PetscCommDuplicate(): Using internal PETSc communicator 1140850689 > -2080374783 > 0 KSP Residual norm 2.505507810276e+02 > 1 KSP Residual norm 3.596555656581e+01 > 2 KSP Residual norm 2.632672485513e+00 > 3 KSP Residual norm 1.888285055287e-01 > 4 KSP Residual norm 7.029433008806e-03 > 5 KSP Residual norm 3.635267067420e-14 > [0] KSPDefaultConverged(): Linear solver has converged. Residual norm > 3.63527e-14 is less than relative tolerance 1e-07 times initial right > hand side norm 250.551 at iteration 5 > KSP Object: > type: cg > maximum iterations=10000, initial guess is zero > tolerances: relative=1e-07, absolute=1e-50, divergence=10000 > left preconditioning > PC Object: > type: icc > ICC: 0 levels of fill > ICC: factor fill ratio allocated 1 > ICC: factor fill ratio needed 0.636364 > Factored matrix follows > Matrix Object: > type=seqsbaij, rows=9, cols=9 > total: nonzeros=21, allocated nonzeros=21 > block size is 1 > linear system matrix = precond matrix: > Matrix Object: > type=seqaij, rows=9, cols=9 > total: nonzeros=33, allocated nonzeros=45 > not using I-node routines > solution: > 92.3023 > 92.3023 > 92.3023 > -5.69767 > -5.69767 > -5.69767 > -103.698 > -103.698 > -103.698 > > > > From knutert at stud.ntnu.no Tue Mar 20 03:55:09 2007 From: knutert at stud.ntnu.no (Knut Erik Teigen) Date: Tue, 20 Mar 2007 09:55:09 +0100 Subject: Problems porting code to an IBM power5+ machine In-Reply-To: References: <8aa042e10703161157p486a2fe2q39422b5b56c6f7d4@mail.gmail.com> <8aa042e10703161637ie32db62tf70e78c8663e6560@mail.gmail.com> <1174310957.5365.27.camel@iept0415.ivt.ntnu.no> Message-ID: <1174380909.5365.38.camel@iept0415.ivt.ntnu.no> On Mon, 2007-03-19 at 08:39 -0500, Matthew Knepley wrote: > This smells like a memory overwrite or use of an uninitialized > variable. The initial norm is 1e28. > > 1) Try Jacobi instead of ICC to see if it is localized to the PC With jacobi I get: [0] PCSetUpSetting up new PC 0 KSP Residual norm 1.466015468749e+13 1 KSP Residual norm 2.720083022075e+23 [0] KSPDefaultConvergedLinear solver is diverging. Initial right hand size norm 1.46602e+13, current residual norm 2.72008e+23 at iteration 1 And without a preconditioner at all: 0 KSP Residual norm 7.719804763794e+00 1 KSP Residual norm 1.137387752533e+01 [0] KSPSolve_CGdiverging due to indefinite or negative definite matrix The matrix definitely isn't indefinite or negative definite, as is clear from the output in my previous post. > 2) Run with valgrind or something similar to check for a memory overwrite > > 3) Maybe insert CHKMEMQ statements into the code I've run with CHKMEMQ statements, and with -malloc_debug, but didn't get any complaints. Valgrind unfortunately isn't installed on the cluster. -Knut Erik- > > Thanks, > > Matt > > On 3/19/07, Knut Erik Teigen wrote: > > Hello > > > > I have got a working code on my local machine( Pentium 4), but when I > > try to run the code on a power5+ machine, the equation solver won't > > converge. It seems like it calculates the wrong right hand side norm. > > Below is the result with run-time options > > "-ksp_type cg -pc_type icc -ksp_monitor -ksp_view -info > > First with the code running on the power5+ machine, then on my local > > machine. I've also printed the right hand side, jacobian matrix and > > solution for a small 3x3 grid. > > > > Can anyone help me figure out what's wrong? > > > > Regards, > > Knut Erik Teigen > > > > Code running on Power5+: > > rhs: > > 980 > > 980 > > 980 > > -0 > > -0 > > -0 > > -980 > > -980 > > -980 > > jacobian: > > row 0: (0, 20) (1, -10) (3, -10) > > row 1: (0, -10) (1, 30) (2, -10) (4, -10) > > row 2: (1, -10) (2, 20) (5, -10) > > row 3: (0, -10) (3, 30) (4, -10) (6, -10) > > row 4: (1, -10) (3, -10) (4, 40) (5, -10) (7, -10) > > row 5: (2, -10) (4, -10) (5, 30) (8, -10) > > row 6: (3, -10) (6, 20) (7, -10) > > row 7: (4, -10) (6, -10) (7, 30) (8, -10) > > row 8: (5, -10) (7, -10) (8, 20) > > [0] PCSetUpSetting up new PC > > [0] PetscCommDuplicateDuplicating a communicator 1 4 max tags = > > 1073741823 > > [0] PetscCommDuplicateUsing internal PETSc communicator 1 4 > > [0] PetscCommDuplicateUsing internal PETSc communicator 1 4 > > 0 KSP Residual norm 7.410163701832e+28 > > 1 KSP Residual norm 6.464393707520e+11 > > [0] KSPDefaultConvergedLinear solver has converged. Residual norm > > 6.46439e+11 is less than relative tolerance 1e-07 times initial right > > hand side norm 7.41016e+28 at iteration 1 > > KSP Object: > > type: cg > > maximum iterations=10000, initial guess is zero > > tolerances: relative=1e-07, absolute=1e-50, divergence=10000 > > left preconditioning > > PC Object: > > type: icc > > ICC: 0 levels of fill > > ICC: factor fill ratio allocated 1 > > ICC: factor fill ratio needed 0.636364 > > Factored matrix follows > > Matrix Object: > > type=seqsbaij, rows=9, cols=9 > > total: nonzeros=21, allocated nonzeros=21 > > block size is 1 > > linear system matrix = precond matrix: > > Matrix Object: > > type=seqaij, rows=9, cols=9 > > total: nonzeros=33, allocated nonzeros=45 > > not using I-node routines > > solution: > > 6.37205e-09 > > 7.13167e-09 > > 7.49911e-09 > > -2.48277e-09 > > -4.56885e-10 > > 0 > > 0 > > 0 > > 0 > > > > Code running on local machine: > > rhs: > > 980 > > 980 > > 980 > > -0 > > -0 > > -0 > > -980 > > -980 > > -980 > > jacobian: > > row 0: (0, 20) (1, -10) (3, -10) > > row 1: (0, -10) (1, 30) (2, -10) (4, -10) > > row 2: (1, -10) (2, 20) (5, -10) > > row 3: (0, -10) (3, 30) (4, -10) (6, -10) > > row 4: (1, -10) (3, -10) (4, 40) (5, -10) (7, -10) > > row 5: (2, -10) (4, -10) (5, 30) (8, -10) > > row 6: (3, -10) (6, 20) (7, -10) > > row 7: (4, -10) (6, -10) (7, 30) (8, -10) > > row 8: (5, -10) (7, -10) (8, 20) > > [0] PCSetUp(): Setting up new PC > > [0] PetscCommDuplicate(): Duplicating a communicator 1140850689 > > -2080374783 max tags = 2147483647 > > [0] PetscCommDuplicate(): Using internal PETSc communicator 1140850689 > > -2080374783 > > [0] PetscCommDuplicate(): Using internal PETSc communicator 1140850689 > > -2080374783 > > 0 KSP Residual norm 2.505507810276e+02 > > 1 KSP Residual norm 3.596555656581e+01 > > 2 KSP Residual norm 2.632672485513e+00 > > 3 KSP Residual norm 1.888285055287e-01 > > 4 KSP Residual norm 7.029433008806e-03 > > 5 KSP Residual norm 3.635267067420e-14 > > [0] KSPDefaultConverged(): Linear solver has converged. Residual norm > > 3.63527e-14 is less than relative tolerance 1e-07 times initial right > > hand side norm 250.551 at iteration 5 > > KSP Object: > > type: cg > > maximum iterations=10000, initial guess is zero > > tolerances: relative=1e-07, absolute=1e-50, divergence=10000 > > left preconditioning > > PC Object: > > type: icc > > ICC: 0 levels of fill > > ICC: factor fill ratio allocated 1 > > ICC: factor fill ratio needed 0.636364 > > Factored matrix follows > > Matrix Object: > > type=seqsbaij, rows=9, cols=9 > > total: nonzeros=21, allocated nonzeros=21 > > block size is 1 > > linear system matrix = precond matrix: > > Matrix Object: > > type=seqaij, rows=9, cols=9 > > total: nonzeros=33, allocated nonzeros=45 > > not using I-node routines > > solution: > > 92.3023 > > 92.3023 > > 92.3023 > > -5.69767 > > -5.69767 > > -5.69767 > > -103.698 > > -103.698 > > -103.698 > > > > > > > > > > From knepley at gmail.com Tue Mar 20 08:03:42 2007 From: knepley at gmail.com (Matthew Knepley) Date: Tue, 20 Mar 2007 08:03:42 -0500 Subject: Problems porting code to an IBM power5+ machine In-Reply-To: <1174380909.5365.38.camel@iept0415.ivt.ntnu.no> References: <8aa042e10703161157p486a2fe2q39422b5b56c6f7d4@mail.gmail.com> <8aa042e10703161637ie32db62tf70e78c8663e6560@mail.gmail.com> <1174310957.5365.27.camel@iept0415.ivt.ntnu.no> <1174380909.5365.38.camel@iept0415.ivt.ntnu.no> Message-ID: On 3/20/07, Knut Erik Teigen wrote: > On Mon, 2007-03-19 at 08:39 -0500, Matthew Knepley wrote: > > This smells like a memory overwrite or use of an uninitialized > > variable. The initial norm is 1e28. > > > > 1) Try Jacobi instead of ICC to see if it is localized to the PC > > With jacobi I get: > [0] PCSetUpSetting up new PC > 0 KSP Residual norm 1.466015468749e+13 > 1 KSP Residual norm 2.720083022075e+23 > [0] KSPDefaultConvergedLinear solver is diverging. Initial right hand > size norm 1.46602e+13, current residual norm 2.72008e+23 at iteration 1 > > And without a preconditioner at all: > 0 KSP Residual norm 7.719804763794e+00 > 1 KSP Residual norm 1.137387752533e+01 > [0] KSPSolve_CGdiverging due to indefinite or negative definite matrix > > The matrix definitely isn't indefinite or negative definite, as is clear > from the output in my previous post. > > > 2) Run with valgrind or something similar to check for a memory overwrite > > > > 3) Maybe insert CHKMEMQ statements into the code > > I've run with CHKMEMQ statements, and with -malloc_debug, but didn't get > any complaints. > Valgrind unfortunately isn't installed on the cluster. Then I am afraid it will be a slow painful process. I am almost positive that you are overwriting memory somewhere. It is different on the two machines because the initial layout is different. The huge norm can be traced back to a huge Vec element, which can be localied. You will just have to go through the code methodically with the debugger to find it. Or install valgrind. Matt > -Knut Erik- > > > > Thanks, > > > > Matt > > > > On 3/19/07, Knut Erik Teigen wrote: > > > Hello > > > > > > I have got a working code on my local machine( Pentium 4), but when I > > > try to run the code on a power5+ machine, the equation solver won't > > > converge. It seems like it calculates the wrong right hand side norm. > > > Below is the result with run-time options > > > "-ksp_type cg -pc_type icc -ksp_monitor -ksp_view -info > > > First with the code running on the power5+ machine, then on my local > > > machine. I've also printed the right hand side, jacobian matrix and > > > solution for a small 3x3 grid. > > > > > > Can anyone help me figure out what's wrong? > > > > > > Regards, > > > Knut Erik Teigen > > > > > > Code running on Power5+: > > > rhs: > > > 980 > > > 980 > > > 980 > > > -0 > > > -0 > > > -0 > > > -980 > > > -980 > > > -980 > > > jacobian: > > > row 0: (0, 20) (1, -10) (3, -10) > > > row 1: (0, -10) (1, 30) (2, -10) (4, -10) > > > row 2: (1, -10) (2, 20) (5, -10) > > > row 3: (0, -10) (3, 30) (4, -10) (6, -10) > > > row 4: (1, -10) (3, -10) (4, 40) (5, -10) (7, -10) > > > row 5: (2, -10) (4, -10) (5, 30) (8, -10) > > > row 6: (3, -10) (6, 20) (7, -10) > > > row 7: (4, -10) (6, -10) (7, 30) (8, -10) > > > row 8: (5, -10) (7, -10) (8, 20) > > > [0] PCSetUpSetting up new PC > > > [0] PetscCommDuplicateDuplicating a communicator 1 4 max tags = > > > 1073741823 > > > [0] PetscCommDuplicateUsing internal PETSc communicator 1 4 > > > [0] PetscCommDuplicateUsing internal PETSc communicator 1 4 > > > 0 KSP Residual norm 7.410163701832e+28 > > > 1 KSP Residual norm 6.464393707520e+11 > > > [0] KSPDefaultConvergedLinear solver has converged. Residual norm > > > 6.46439e+11 is less than relative tolerance 1e-07 times initial right > > > hand side norm 7.41016e+28 at iteration 1 > > > KSP Object: > > > type: cg > > > maximum iterations=10000, initial guess is zero > > > tolerances: relative=1e-07, absolute=1e-50, divergence=10000 > > > left preconditioning > > > PC Object: > > > type: icc > > > ICC: 0 levels of fill > > > ICC: factor fill ratio allocated 1 > > > ICC: factor fill ratio needed 0.636364 > > > Factored matrix follows > > > Matrix Object: > > > type=seqsbaij, rows=9, cols=9 > > > total: nonzeros=21, allocated nonzeros=21 > > > block size is 1 > > > linear system matrix = precond matrix: > > > Matrix Object: > > > type=seqaij, rows=9, cols=9 > > > total: nonzeros=33, allocated nonzeros=45 > > > not using I-node routines > > > solution: > > > 6.37205e-09 > > > 7.13167e-09 > > > 7.49911e-09 > > > -2.48277e-09 > > > -4.56885e-10 > > > 0 > > > 0 > > > 0 > > > 0 > > > > > > Code running on local machine: > > > rhs: > > > 980 > > > 980 > > > 980 > > > -0 > > > -0 > > > -0 > > > -980 > > > -980 > > > -980 > > > jacobian: > > > row 0: (0, 20) (1, -10) (3, -10) > > > row 1: (0, -10) (1, 30) (2, -10) (4, -10) > > > row 2: (1, -10) (2, 20) (5, -10) > > > row 3: (0, -10) (3, 30) (4, -10) (6, -10) > > > row 4: (1, -10) (3, -10) (4, 40) (5, -10) (7, -10) > > > row 5: (2, -10) (4, -10) (5, 30) (8, -10) > > > row 6: (3, -10) (6, 20) (7, -10) > > > row 7: (4, -10) (6, -10) (7, 30) (8, -10) > > > row 8: (5, -10) (7, -10) (8, 20) > > > [0] PCSetUp(): Setting up new PC > > > [0] PetscCommDuplicate(): Duplicating a communicator 1140850689 > > > -2080374783 max tags = 2147483647 > > > [0] PetscCommDuplicate(): Using internal PETSc communicator 1140850689 > > > -2080374783 > > > [0] PetscCommDuplicate(): Using internal PETSc communicator 1140850689 > > > -2080374783 > > > 0 KSP Residual norm 2.505507810276e+02 > > > 1 KSP Residual norm 3.596555656581e+01 > > > 2 KSP Residual norm 2.632672485513e+00 > > > 3 KSP Residual norm 1.888285055287e-01 > > > 4 KSP Residual norm 7.029433008806e-03 > > > 5 KSP Residual norm 3.635267067420e-14 > > > [0] KSPDefaultConverged(): Linear solver has converged. Residual norm > > > 3.63527e-14 is less than relative tolerance 1e-07 times initial right > > > hand side norm 250.551 at iteration 5 > > > KSP Object: > > > type: cg > > > maximum iterations=10000, initial guess is zero > > > tolerances: relative=1e-07, absolute=1e-50, divergence=10000 > > > left preconditioning > > > PC Object: > > > type: icc > > > ICC: 0 levels of fill > > > ICC: factor fill ratio allocated 1 > > > ICC: factor fill ratio needed 0.636364 > > > Factored matrix follows > > > Matrix Object: > > > type=seqsbaij, rows=9, cols=9 > > > total: nonzeros=21, allocated nonzeros=21 > > > block size is 1 > > > linear system matrix = precond matrix: > > > Matrix Object: > > > type=seqaij, rows=9, cols=9 > > > total: nonzeros=33, allocated nonzeros=45 > > > not using I-node routines > > > solution: > > > 92.3023 > > > 92.3023 > > > 92.3023 > > > -5.69767 > > > -5.69767 > > > -5.69767 > > > -103.698 > > > -103.698 > > > -103.698 > > > > > > > > > > > > > > > > > > -- One trouble is that despite this system, anyone who reads journals widely and critically is forced to realize that there are scarcely any bars to eventual publication. There seems to be no study too fragmented, no hypothesis too trivial, no literature citation too biased or too egotistical, no design too warped, no methodology too bungled, no presentation of results too inaccurate, too obscure, and too contradictory, no analysis too self-serving, no argument too circular, no conclusions too trifling or too unjustified, and no grammar and syntax too offensive for a paper to end up in print. -- Drummond Rennie From shma7099 at student.uu.se Tue Mar 20 08:38:41 2007 From: shma7099 at student.uu.se (Shaman Mahmoudi) Date: Tue, 20 Mar 2007 14:38:41 +0100 Subject: Problems porting code to an IBM power5+ machine In-Reply-To: References: <8aa042e10703161157p486a2fe2q39422b5b56c6f7d4@mail.gmail.com> <8aa042e10703161637ie32db62tf70e78c8663e6560@mail.gmail.com> <1174310957.5365.27.camel@iept0415.ivt.ntnu.no> <1174380909.5365.38.camel@iept0415.ivt.ntnu.no> Message-ID: <52A6F5AB-2957-4E7D-8C20-DDF8CE703775@student.uu.se> Hi, This might sound retarded, but this is how I debugged when I couldn't get my debuggers for parallel code working. What I did was that I added a wait call such as getc in C after I had initialized the processors. I ran my program as usual, and of course it halted as it was waiting for input (getc) from stdin. Then I checked my processes ID numbers, and locked the regular serial debugger on one of those processes, in my case the process that CPU 0 was running, and debugged just that one. Finally I pressed enter in my program as it was waiting for stdin and it continued running and cached the error eventually. Better than no debugging at all. With best regards, Shaman Mahmoudi On Mar 20, 2007, at 2:03 PM, Matthew Knepley wrote: > On 3/20/07, Knut Erik Teigen wrote: >> On Mon, 2007-03-19 at 08:39 -0500, Matthew Knepley wrote: >> > This smells like a memory overwrite or use of an uninitialized >> > variable. The initial norm is 1e28. >> > >> > 1) Try Jacobi instead of ICC to see if it is localized to the PC >> >> With jacobi I get: >> [0] PCSetUpSetting up new PC >> 0 KSP Residual norm 1.466015468749e+13 >> 1 KSP Residual norm 2.720083022075e+23 >> [0] KSPDefaultConvergedLinear solver is diverging. Initial right hand >> size norm 1.46602e+13, current residual norm 2.72008e+23 at >> iteration 1 >> >> And without a preconditioner at all: >> 0 KSP Residual norm 7.719804763794e+00 >> 1 KSP Residual norm 1.137387752533e+01 >> [0] KSPSolve_CGdiverging due to indefinite or negative definite >> matrix >> >> The matrix definitely isn't indefinite or negative definite, as is >> clear >> from the output in my previous post. >> >> > 2) Run with valgrind or something similar to check for a memory >> overwrite >> > >> > 3) Maybe insert CHKMEMQ statements into the code >> >> I've run with CHKMEMQ statements, and with -malloc_debug, but >> didn't get >> any complaints. >> Valgrind unfortunately isn't installed on the cluster. > > Then I am afraid it will be a slow painful process. I am almost > positive that you > are overwriting memory somewhere. It is different on the two > machines because > the initial layout is different. The huge norm can be traced back > to a huge Vec > element, which can be localied. You will just have to go through > the code > methodically with the debugger to find it. Or install valgrind. > > Matt > >> -Knut Erik- >> > >> > Thanks, >> > >> > Matt >> > >> > On 3/19/07, Knut Erik Teigen wrote: >> > > Hello >> > > >> > > I have got a working code on my local machine( Pentium 4), but >> when I >> > > try to run the code on a power5+ machine, the equation solver >> won't >> > > converge. It seems like it calculates the wrong right hand >> side norm. >> > > Below is the result with run-time options >> > > "-ksp_type cg -pc_type icc -ksp_monitor -ksp_view -info >> > > First with the code running on the power5+ machine, then on my >> local >> > > machine. I've also printed the right hand side, jacobian >> matrix and >> > > solution for a small 3x3 grid. >> > > >> > > Can anyone help me figure out what's wrong? >> > > >> > > Regards, >> > > Knut Erik Teigen >> > > >> > > Code running on Power5+: >> > > rhs: >> > > 980 >> > > 980 >> > > 980 >> > > -0 >> > > -0 >> > > -0 >> > > -980 >> > > -980 >> > > -980 >> > > jacobian: >> > > row 0: (0, 20) (1, -10) (3, -10) >> > > row 1: (0, -10) (1, 30) (2, -10) (4, -10) >> > > row 2: (1, -10) (2, 20) (5, -10) >> > > row 3: (0, -10) (3, 30) (4, -10) (6, -10) >> > > row 4: (1, -10) (3, -10) (4, 40) (5, -10) (7, -10) >> > > row 5: (2, -10) (4, -10) (5, 30) (8, -10) >> > > row 6: (3, -10) (6, 20) (7, -10) >> > > row 7: (4, -10) (6, -10) (7, 30) (8, -10) >> > > row 8: (5, -10) (7, -10) (8, 20) >> > > [0] PCSetUpSetting up new PC >> > > [0] PetscCommDuplicateDuplicating a communicator 1 4 max tags = >> > > 1073741823 >> > > [0] PetscCommDuplicateUsing internal PETSc communicator 1 4 >> > > [0] PetscCommDuplicateUsing internal PETSc communicator 1 4 >> > > 0 KSP Residual norm 7.410163701832e+28 >> > > 1 KSP Residual norm 6.464393707520e+11 >> > > [0] KSPDefaultConvergedLinear solver has converged. Residual norm >> > > 6.46439e+11 is less than relative tolerance 1e-07 times >> initial right >> > > hand side norm 7.41016e+28 at iteration 1 >> > > KSP Object: >> > > type: cg >> > > maximum iterations=10000, initial guess is zero >> > > tolerances: relative=1e-07, absolute=1e-50, divergence=10000 >> > > left preconditioning >> > > PC Object: >> > > type: icc >> > > ICC: 0 levels of fill >> > > ICC: factor fill ratio allocated 1 >> > > ICC: factor fill ratio needed 0.636364 >> > > Factored matrix follows >> > > Matrix Object: >> > > type=seqsbaij, rows=9, cols=9 >> > > total: nonzeros=21, allocated nonzeros=21 >> > > block size is 1 >> > > linear system matrix = precond matrix: >> > > Matrix Object: >> > > type=seqaij, rows=9, cols=9 >> > > total: nonzeros=33, allocated nonzeros=45 >> > > not using I-node routines >> > > solution: >> > > 6.37205e-09 >> > > 7.13167e-09 >> > > 7.49911e-09 >> > > -2.48277e-09 >> > > -4.56885e-10 >> > > 0 >> > > 0 >> > > 0 >> > > 0 >> > > >> > > Code running on local machine: >> > > rhs: >> > > 980 >> > > 980 >> > > 980 >> > > -0 >> > > -0 >> > > -0 >> > > -980 >> > > -980 >> > > -980 >> > > jacobian: >> > > row 0: (0, 20) (1, -10) (3, -10) >> > > row 1: (0, -10) (1, 30) (2, -10) (4, -10) >> > > row 2: (1, -10) (2, 20) (5, -10) >> > > row 3: (0, -10) (3, 30) (4, -10) (6, -10) >> > > row 4: (1, -10) (3, -10) (4, 40) (5, -10) (7, -10) >> > > row 5: (2, -10) (4, -10) (5, 30) (8, -10) >> > > row 6: (3, -10) (6, 20) (7, -10) >> > > row 7: (4, -10) (6, -10) (7, 30) (8, -10) >> > > row 8: (5, -10) (7, -10) (8, 20) >> > > [0] PCSetUp(): Setting up new PC >> > > [0] PetscCommDuplicate(): Duplicating a communicator 1140850689 >> > > -2080374783 max tags = 2147483647 >> > > [0] PetscCommDuplicate(): Using internal PETSc communicator >> 1140850689 >> > > -2080374783 >> > > [0] PetscCommDuplicate(): Using internal PETSc communicator >> 1140850689 >> > > -2080374783 >> > > 0 KSP Residual norm 2.505507810276e+02 >> > > 1 KSP Residual norm 3.596555656581e+01 >> > > 2 KSP Residual norm 2.632672485513e+00 >> > > 3 KSP Residual norm 1.888285055287e-01 >> > > 4 KSP Residual norm 7.029433008806e-03 >> > > 5 KSP Residual norm 3.635267067420e-14 >> > > [0] KSPDefaultConverged(): Linear solver has converged. >> Residual norm >> > > 3.63527e-14 is less than relative tolerance 1e-07 times >> initial right >> > > hand side norm 250.551 at iteration 5 >> > > KSP Object: >> > > type: cg >> > > maximum iterations=10000, initial guess is zero >> > > tolerances: relative=1e-07, absolute=1e-50, divergence=10000 >> > > left preconditioning >> > > PC Object: >> > > type: icc >> > > ICC: 0 levels of fill >> > > ICC: factor fill ratio allocated 1 >> > > ICC: factor fill ratio needed 0.636364 >> > > Factored matrix follows >> > > Matrix Object: >> > > type=seqsbaij, rows=9, cols=9 >> > > total: nonzeros=21, allocated nonzeros=21 >> > > block size is 1 >> > > linear system matrix = precond matrix: >> > > Matrix Object: >> > > type=seqaij, rows=9, cols=9 >> > > total: nonzeros=33, allocated nonzeros=45 >> > > not using I-node routines >> > > solution: >> > > 92.3023 >> > > 92.3023 >> > > 92.3023 >> > > -5.69767 >> > > -5.69767 >> > > -5.69767 >> > > -103.698 >> > > -103.698 >> > > -103.698 >> > > >> > > >> > > >> > > >> > >> > >> >> > > > -- > One trouble is that despite this system, anyone who reads journals > widely > and critically is forced to realize that there are scarcely any > bars to eventual > publication. There seems to be no study too fragmented, no > hypothesis too > trivial, no literature citation too biased or too egotistical, no > design too > warped, no methodology too bungled, no presentation of results too > inaccurate, too obscure, and too contradictory, no analysis too > self-serving, > no argument too circular, no conclusions too trifling or too > unjustified, and > no grammar and syntax too offensive for a paper to end up in print. -- > Drummond Rennie > From knepley at gmail.com Tue Mar 20 08:45:16 2007 From: knepley at gmail.com (Matthew Knepley) Date: Tue, 20 Mar 2007 08:45:16 -0500 Subject: Problems porting code to an IBM power5+ machine In-Reply-To: <52A6F5AB-2957-4E7D-8C20-DDF8CE703775@student.uu.se> References: <8aa042e10703161157p486a2fe2q39422b5b56c6f7d4@mail.gmail.com> <8aa042e10703161637ie32db62tf70e78c8663e6560@mail.gmail.com> <1174310957.5365.27.camel@iept0415.ivt.ntnu.no> <1174380909.5365.38.camel@iept0415.ivt.ntnu.no> <52A6F5AB-2957-4E7D-8C20-DDF8CE703775@student.uu.se> Message-ID: I would point out that PETSc does this automatically with -start_in_debugger. By default it attaches the debugger to all processes, but you can attach to a subset using -debugger_nodes 0,2,5 or some other choice of processes. Matt On 3/20/07, Shaman Mahmoudi wrote: > Hi, > > This might sound retarded, but this is how I debugged when I couldn't > get my debuggers for parallel code working. > > What I did was that I added a wait call such as getc in C after I had > initialized the processors. I ran my program as usual, and of course > it halted as it was waiting for input (getc) from stdin. Then I > checked my processes ID numbers, and locked the regular serial > debugger on one of those processes, in my case the process that CPU > 0 was running, and debugged just that one. Finally I pressed enter > in my program as it was waiting for stdin and it continued running > and cached the error eventually. > > Better than no debugging at all. > > With best regards, Shaman Mahmoudi > > On Mar 20, 2007, at 2:03 PM, Matthew Knepley wrote: > > > On 3/20/07, Knut Erik Teigen wrote: > >> On Mon, 2007-03-19 at 08:39 -0500, Matthew Knepley wrote: > >> > This smells like a memory overwrite or use of an uninitialized > >> > variable. The initial norm is 1e28. > >> > > >> > 1) Try Jacobi instead of ICC to see if it is localized to the PC > >> > >> With jacobi I get: > >> [0] PCSetUpSetting up new PC > >> 0 KSP Residual norm 1.466015468749e+13 > >> 1 KSP Residual norm 2.720083022075e+23 > >> [0] KSPDefaultConvergedLinear solver is diverging. Initial right hand > >> size norm 1.46602e+13, current residual norm 2.72008e+23 at > >> iteration 1 > >> > >> And without a preconditioner at all: > >> 0 KSP Residual norm 7.719804763794e+00 > >> 1 KSP Residual norm 1.137387752533e+01 > >> [0] KSPSolve_CGdiverging due to indefinite or negative definite > >> matrix > >> > >> The matrix definitely isn't indefinite or negative definite, as is > >> clear > >> from the output in my previous post. > >> > >> > 2) Run with valgrind or something similar to check for a memory > >> overwrite > >> > > >> > 3) Maybe insert CHKMEMQ statements into the code > >> > >> I've run with CHKMEMQ statements, and with -malloc_debug, but > >> didn't get > >> any complaints. > >> Valgrind unfortunately isn't installed on the cluster. > > > > Then I am afraid it will be a slow painful process. I am almost > > positive that you > > are overwriting memory somewhere. It is different on the two > > machines because > > the initial layout is different. The huge norm can be traced back > > to a huge Vec > > element, which can be localied. You will just have to go through > > the code > > methodically with the debugger to find it. Or install valgrind. > > > > Matt > > > >> -Knut Erik- > >> > > >> > Thanks, > >> > > >> > Matt > >> > > >> > On 3/19/07, Knut Erik Teigen wrote: > >> > > Hello > >> > > > >> > > I have got a working code on my local machine( Pentium 4), but > >> when I > >> > > try to run the code on a power5+ machine, the equation solver > >> won't > >> > > converge. It seems like it calculates the wrong right hand > >> side norm. > >> > > Below is the result with run-time options > >> > > "-ksp_type cg -pc_type icc -ksp_monitor -ksp_view -info > >> > > First with the code running on the power5+ machine, then on my > >> local > >> > > machine. I've also printed the right hand side, jacobian > >> matrix and > >> > > solution for a small 3x3 grid. > >> > > > >> > > Can anyone help me figure out what's wrong? > >> > > > >> > > Regards, > >> > > Knut Erik Teigen > >> > > > >> > > Code running on Power5+: > >> > > rhs: > >> > > 980 > >> > > 980 > >> > > 980 > >> > > -0 > >> > > -0 > >> > > -0 > >> > > -980 > >> > > -980 > >> > > -980 > >> > > jacobian: > >> > > row 0: (0, 20) (1, -10) (3, -10) > >> > > row 1: (0, -10) (1, 30) (2, -10) (4, -10) > >> > > row 2: (1, -10) (2, 20) (5, -10) > >> > > row 3: (0, -10) (3, 30) (4, -10) (6, -10) > >> > > row 4: (1, -10) (3, -10) (4, 40) (5, -10) (7, -10) > >> > > row 5: (2, -10) (4, -10) (5, 30) (8, -10) > >> > > row 6: (3, -10) (6, 20) (7, -10) > >> > > row 7: (4, -10) (6, -10) (7, 30) (8, -10) > >> > > row 8: (5, -10) (7, -10) (8, 20) > >> > > [0] PCSetUpSetting up new PC > >> > > [0] PetscCommDuplicateDuplicating a communicator 1 4 max tags = > >> > > 1073741823 > >> > > [0] PetscCommDuplicateUsing internal PETSc communicator 1 4 > >> > > [0] PetscCommDuplicateUsing internal PETSc communicator 1 4 > >> > > 0 KSP Residual norm 7.410163701832e+28 > >> > > 1 KSP Residual norm 6.464393707520e+11 > >> > > [0] KSPDefaultConvergedLinear solver has converged. Residual norm > >> > > 6.46439e+11 is less than relative tolerance 1e-07 times > >> initial right > >> > > hand side norm 7.41016e+28 at iteration 1 > >> > > KSP Object: > >> > > type: cg > >> > > maximum iterations=10000, initial guess is zero > >> > > tolerances: relative=1e-07, absolute=1e-50, divergence=10000 > >> > > left preconditioning > >> > > PC Object: > >> > > type: icc > >> > > ICC: 0 levels of fill > >> > > ICC: factor fill ratio allocated 1 > >> > > ICC: factor fill ratio needed 0.636364 > >> > > Factored matrix follows > >> > > Matrix Object: > >> > > type=seqsbaij, rows=9, cols=9 > >> > > total: nonzeros=21, allocated nonzeros=21 > >> > > block size is 1 > >> > > linear system matrix = precond matrix: > >> > > Matrix Object: > >> > > type=seqaij, rows=9, cols=9 > >> > > total: nonzeros=33, allocated nonzeros=45 > >> > > not using I-node routines > >> > > solution: > >> > > 6.37205e-09 > >> > > 7.13167e-09 > >> > > 7.49911e-09 > >> > > -2.48277e-09 > >> > > -4.56885e-10 > >> > > 0 > >> > > 0 > >> > > 0 > >> > > 0 > >> > > > >> > > Code running on local machine: > >> > > rhs: > >> > > 980 > >> > > 980 > >> > > 980 > >> > > -0 > >> > > -0 > >> > > -0 > >> > > -980 > >> > > -980 > >> > > -980 > >> > > jacobian: > >> > > row 0: (0, 20) (1, -10) (3, -10) > >> > > row 1: (0, -10) (1, 30) (2, -10) (4, -10) > >> > > row 2: (1, -10) (2, 20) (5, -10) > >> > > row 3: (0, -10) (3, 30) (4, -10) (6, -10) > >> > > row 4: (1, -10) (3, -10) (4, 40) (5, -10) (7, -10) > >> > > row 5: (2, -10) (4, -10) (5, 30) (8, -10) > >> > > row 6: (3, -10) (6, 20) (7, -10) > >> > > row 7: (4, -10) (6, -10) (7, 30) (8, -10) > >> > > row 8: (5, -10) (7, -10) (8, 20) > >> > > [0] PCSetUp(): Setting up new PC > >> > > [0] PetscCommDuplicate(): Duplicating a communicator 1140850689 > >> > > -2080374783 max tags = 2147483647 > >> > > [0] PetscCommDuplicate(): Using internal PETSc communicator > >> 1140850689 > >> > > -2080374783 > >> > > [0] PetscCommDuplicate(): Using internal PETSc communicator > >> 1140850689 > >> > > -2080374783 > >> > > 0 KSP Residual norm 2.505507810276e+02 > >> > > 1 KSP Residual norm 3.596555656581e+01 > >> > > 2 KSP Residual norm 2.632672485513e+00 > >> > > 3 KSP Residual norm 1.888285055287e-01 > >> > > 4 KSP Residual norm 7.029433008806e-03 > >> > > 5 KSP Residual norm 3.635267067420e-14 > >> > > [0] KSPDefaultConverged(): Linear solver has converged. > >> Residual norm > >> > > 3.63527e-14 is less than relative tolerance 1e-07 times > >> initial right > >> > > hand side norm 250.551 at iteration 5 > >> > > KSP Object: > >> > > type: cg > >> > > maximum iterations=10000, initial guess is zero > >> > > tolerances: relative=1e-07, absolute=1e-50, divergence=10000 > >> > > left preconditioning > >> > > PC Object: > >> > > type: icc > >> > > ICC: 0 levels of fill > >> > > ICC: factor fill ratio allocated 1 > >> > > ICC: factor fill ratio needed 0.636364 > >> > > Factored matrix follows > >> > > Matrix Object: > >> > > type=seqsbaij, rows=9, cols=9 > >> > > total: nonzeros=21, allocated nonzeros=21 > >> > > block size is 1 > >> > > linear system matrix = precond matrix: > >> > > Matrix Object: > >> > > type=seqaij, rows=9, cols=9 > >> > > total: nonzeros=33, allocated nonzeros=45 > >> > > not using I-node routines > >> > > solution: > >> > > 92.3023 > >> > > 92.3023 > >> > > 92.3023 > >> > > -5.69767 > >> > > -5.69767 > >> > > -5.69767 > >> > > -103.698 > >> > > -103.698 > >> > > -103.698 > >> > > > >> > > > >> > > > >> > > > >> > > >> > > >> > >> > > > > > > -- > > One trouble is that despite this system, anyone who reads journals > > widely > > and critically is forced to realize that there are scarcely any > > bars to eventual > > publication. There seems to be no study too fragmented, no > > hypothesis too > > trivial, no literature citation too biased or too egotistical, no > > design too > > warped, no methodology too bungled, no presentation of results too > > inaccurate, too obscure, and too contradictory, no analysis too > > self-serving, > > no argument too circular, no conclusions too trifling or too > > unjustified, and > > no grammar and syntax too offensive for a paper to end up in print. -- > > Drummond Rennie > > > > -- One trouble is that despite this system, anyone who reads journals widely and critically is forced to realize that there are scarcely any bars to eventual publication. There seems to be no study too fragmented, no hypothesis too trivial, no literature citation too biased or too egotistical, no design too warped, no methodology too bungled, no presentation of results too inaccurate, too obscure, and too contradictory, no analysis too self-serving, no argument too circular, no conclusions too trifling or too unjustified, and no grammar and syntax too offensive for a paper to end up in print. -- Drummond Rennie From bsmith at mcs.anl.gov Sat Mar 24 18:30:08 2007 From: bsmith at mcs.anl.gov (Barry Smith) Date: Sat, 24 Mar 2007 18:30:08 -0500 (CDT) Subject: higher level interfaces to PETSc? Message-ID: PETSc-users, A representative of Interactive Supercomputing has asked us to gauge if there is interest among PETSc users in calling PETSc from higher level systems such as Matlab (and presumably money to pay for it :-). I've attached a portion of the message below. If you are interested please drop a note to Robert McGill . Barry ---------- Forwarded message ---------- Date: Mon, 19 Mar 2007 12:39:16 -0500 From: Robert McGill We have several customers interested in linking their MATLAB routines to PDE solvers such as those offered by PETSc. To date, we have linked to several libraries but not specifically to PETSc. We are working with some of the folks on the Trilinos project and am curious to find out if there are users in the PETSc community who are interested in calling PETSc from a high level language such as MATLAB or R or IDL. Our product, Star-P is a parallel processing platform that is used by VHHL users to not only run their client applications in parallel, but also to make calls to libraries such as PETSc directly from MATLAB and R via the Star-P Connect SDK. Robert McGill Business Alliances Interactive Supercomputing, Inc. 617 694-9451 www.interactivesupercomputing.com From h.a.slim at durham.ac.uk Mon Mar 26 09:57:59 2007 From: h.a.slim at durham.ac.uk (SLIM H.A.) Date: Mon, 26 Mar 2007 15:57:59 +0100 Subject: No subject Message-ID: Dear users I want to build petsc with shared object libraries but get errors when building libpetscmat.so and libpetscksp.so I am using the intel compiler and intel's imkl for lapack/blas Configuration is: ./config/configure.py --with-blas-lapack-dir=/usr/local/Cluster-Apps/intel/mkl/8.0/lib/em64t --with-vendor-compilers=intel --with-gnu-compilers=0 --with-shared The error is: ld: /usr/local/Cluster-Apps/intel/mkl/8.0/lib/em64t/libmkl_lapack.a(dgetrf_o mp.o): relocation R_X86_64_PC32 against `_mkl_lapack_dgetrf_omp_271__par_loop0' can not be used when making a shared object; recompile with -fPIC ld: final link failed: Bad value make[2]: [shared_linux] Error 1 (ignored) Details are in the attached male log file. Thanks for any help Henk -------------- next part -------------- A non-text attachment was scrubbed... Name: make_log_linux-gnu-intel Type: application/octet-stream Size: 53428 bytes Desc: make_log_linux-gnu-intel URL: From dalcinl at gmail.com Mon Mar 26 10:16:52 2007 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Mon, 26 Mar 2007 12:16:52 -0300 Subject: In-Reply-To: References: Message-ID: This seems to be a bug of 'binutils' in 64 bits machines. http://sources.redhat.com/bugzilla/show_bug.cgi?id=584 but apparently it was fixed in Feb 2005 http://sourceware.org/ml/binutils-cvs/2005-02/msg00011.html Can you upgrade your 'binutils' package and try what happens? On 3/26/07, SLIM H.A. wrote: > Dear users > > I want to build petsc with shared object libraries but get errors when > building libpetscmat.so and libpetscksp.so > I am using the intel compiler and intel's imkl for lapack/blas > > Configuration is: > ./config/configure.py > --with-blas-lapack-dir=/usr/local/Cluster-Apps/intel/mkl/8.0/lib/em64t > --with-vendor-compilers=intel --with-gnu-compilers=0 --with-shared > > The error is: > ld: > /usr/local/Cluster-Apps/intel/mkl/8.0/lib/em64t/libmkl_lapack.a(dgetrf_o > mp.o): relocation R_X86_64_PC32 against > `_mkl_lapack_dgetrf_omp_271__par_loop0' can not be used when making a > shared object; recompile with -fPIC > ld: final link failed: Bad value > make[2]: [shared_linux] Error 1 (ignored) > > Details are in the attached male log file. > > Thanks for any help > > Henk > > -- Lisandro Dalc?n --------------- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 From balay at mcs.anl.gov Mon Mar 26 10:28:15 2007 From: balay at mcs.anl.gov (Satish Balay) Date: Mon, 26 Mar 2007 10:28:15 -0500 (CDT) Subject: In-Reply-To: References: Message-ID: Not sure about the binutils stuff - but x86_64 linux complains about using non PIC code in a shared library. Currently PETSc configure is defaulting to .a version of MKL - which are incompatible with sharedlibraries. And I'm not sure what the correct library naems are for the shared version of MLK. You could try using the option: --with-blas-lapack-lib=[/usr/local/Cluster-Apps/intel/mkl/8.0/lib/em64t/libmkl_lapack32.so,libmkl_def.so,libguide.so] or just use --download-f-blas-lapack=1 Satish On Mon, 26 Mar 2007, Lisandro Dalcin wrote: > This seems to be a bug of 'binutils' in 64 bits machines. > > http://sources.redhat.com/bugzilla/show_bug.cgi?id=584 > > but apparently it was fixed in Feb 2005 > > http://sourceware.org/ml/binutils-cvs/2005-02/msg00011.html > > Can you upgrade your 'binutils' package and try what happens? > > > On 3/26/07, SLIM H.A. wrote: > > Dear users > > > > I want to build petsc with shared object libraries but get errors when > > building libpetscmat.so and libpetscksp.so > > I am using the intel compiler and intel's imkl for lapack/blas > > > > Configuration is: > > ./config/configure.py > > --with-blas-lapack-dir=/usr/local/Cluster-Apps/intel/mkl/8.0/lib/em64t > > --with-vendor-compilers=intel --with-gnu-compilers=0 --with-shared > > > > The error is: > > ld: > > /usr/local/Cluster-Apps/intel/mkl/8.0/lib/em64t/libmkl_lapack.a(dgetrf_o > > mp.o): relocation R_X86_64_PC32 against > > `_mkl_lapack_dgetrf_omp_271__par_loop0' can not be used when making a > > shared object; recompile with -fPIC > > ld: final link failed: Bad value > > make[2]: [shared_linux] Error 1 (ignored) > > > > Details are in the attached male log file. > > > > Thanks for any help > > > > Henk > > > > > > > From geenen at gmail.com Mon Mar 26 10:02:08 2007 From: geenen at gmail.com (Thomas Geenen) Date: Mon, 26 Mar 2007 17:02:08 +0200 Subject: In-Reply-To: References: Message-ID: <200703261702.08665.geenen@gmail.com> i think you should. On Monday 26 March 2007 16:57, SLIM H.A. wrote: > recompile with -fPIC cheers Thomas From h.a.slim at durham.ac.uk Mon Mar 26 10:59:47 2007 From: h.a.slim at durham.ac.uk (SLIM H.A.) Date: Mon, 26 Mar 2007 16:59:47 +0100 Subject: References: <200703261702.08665.geenen@gmail.com> Message-ID: That's what I thought, but as you can see from the make log, compilation is done with -fPIC: mpif90 -c -I. -fPIC -g -I/data/hamilton/drk1has/_libs/petsc/petsc-2.3.2-p9 -I/data/hamilton/drk1has/_libs/petsc/petsc-2.3.2-p9/ bmake/linux-gnu-intel -I/data/hamilton/drk1has/_libs/petsc/petsc-2.3.2-p9/include -o somefort.o somefort.F > -----Original Message----- > From: owner-petsc-users at mcs.anl.gov > [mailto:owner-petsc-users at mcs.anl.gov] On Behalf Of Thomas Geenen > Sent: 26 March 2007 16:02 > To: petsc-users at mcs.anl.gov > Subject: Re: > > i think you should. > > > On Monday 26 March 2007 16:57, SLIM H.A. wrote: > > recompile with -fPIC > > cheers > Thomas > > From knepley at gmail.com Mon Mar 26 11:07:41 2007 From: knepley at gmail.com (Matthew Knepley) Date: Mon, 26 Mar 2007 11:07:41 -0500 Subject: In-Reply-To: References: <200703261702.08665.geenen@gmail.com> Message-ID: On 3/26/07, SLIM H.A. wrote: > That's what I thought, but as you can see from the make log, compilation > is done with -fPIC: Yes, but some libraries which PETSc does not build (in this case MKL) were not compiled with -fPIC. Satish's fix should work. Matt > mpif90 -c -I. -fPIC -g > -I/data/hamilton/drk1has/_libs/petsc/petsc-2.3.2-p9 > -I/data/hamilton/drk1has/_libs/petsc/petsc-2.3.2-p9/ > bmake/linux-gnu-intel > -I/data/hamilton/drk1has/_libs/petsc/petsc-2.3.2-p9/include -o > somefort.o somefort.F > > > -----Original Message----- > > From: owner-petsc-users at mcs.anl.gov > > [mailto:owner-petsc-users at mcs.anl.gov] On Behalf Of Thomas Geenen > > Sent: 26 March 2007 16:02 > > To: petsc-users at mcs.anl.gov > > Subject: Re: > > > > i think you should. > > > > > > On Monday 26 March 2007 16:57, SLIM H.A. wrote: > > > recompile with -fPIC > > > > cheers > > Thomas > > > > > > -- One trouble is that despite this system, anyone who reads journals widely and critically is forced to realize that there are scarcely any bars to eventual publication. There seems to be no study too fragmented, no hypothesis too trivial, no literature citation too biased or too egotistical, no design too warped, no methodology too bungled, no presentation of results too inaccurate, too obscure, and too contradictory, no analysis too self-serving, no argument too circular, no conclusions too trifling or too unjustified, and no grammar and syntax too offensive for a paper to end up in print. -- Drummond Rennie From h.a.slim at durham.ac.uk Mon Mar 26 11:10:14 2007 From: h.a.slim at durham.ac.uk (SLIM H.A.) Date: Mon, 26 Mar 2007 17:10:14 +0100 Subject: References: Message-ID: Thanks I tried ./config/configure.py --with-blas-lapack-lib=[/usr/local/Cluster-Apps/intel/mkl/8.0/lib/em64t/ libmkl_lapack64.so,libmkl_def.so,libguide.so] --with-vendor-compilers=intel --with-gnu-compilers=0 which gives: ./config/configure.py: No match. I checked that the files libmkl_lapack64.so,libmkl_def.so,libguide.so exist in /usr/local/Cluster-Apps/intel/mkl/8.0/lib/em64t However I am not familiar with the configure command. Did I use it corrctly? Thanks Henk > -----Original Message----- > From: owner-petsc-users at mcs.anl.gov > [mailto:owner-petsc-users at mcs.anl.gov] On Behalf Of Satish Balay > Sent: 26 March 2007 16:28 > To: petsc-users at mcs.anl.gov > Subject: Re: > > Not sure about the binutils stuff - but x86_64 linux > complains about using non PIC code in a shared library. > > Currently PETSc configure is defaulting to .a version of MKL > - which are incompatible with sharedlibraries. And I'm not > sure what the correct library naems are for the shared > version of MLK. You could try using the option: > > --with-blas-lapack-lib=[/usr/local/Cluster-Apps/intel/mkl/8.0/ lib/em64t/libmkl_lapack32.so,libmkl_def.so,libguide.so] > > or just use --download-f-blas-lapack=1 > > Satish > > On Mon, 26 Mar 2007, Lisandro Dalcin wrote: > > > This seems to be a bug of 'binutils' in 64 bits machines. > > > > http://sources.redhat.com/bugzilla/show_bug.cgi?id=584 > > > > but apparently it was fixed in Feb 2005 > > > > http://sourceware.org/ml/binutils-cvs/2005-02/msg00011.html > > > > Can you upgrade your 'binutils' package and try what happens? > > > > > > On 3/26/07, SLIM H.A. wrote: > > > Dear users > > > > > > I want to build petsc with shared object libraries but get errors > > > when building libpetscmat.so and libpetscksp.so I am > using the intel > > > compiler and intel's imkl for lapack/blas > > > > > > Configuration is: > > > ./config/configure.py > > > > --with-blas-lapack-dir=/usr/local/Cluster-Apps/intel/mkl/8.0/lib/em6 > > > 4t --with-vendor-compilers=intel --with-gnu-compilers=0 > > > --with-shared > > > > > > The error is: > > > ld: > > > > /usr/local/Cluster-Apps/intel/mkl/8.0/lib/em64t/libmkl_lapack.a(dget > > > rf_o > > > mp.o): relocation R_X86_64_PC32 against > > > `_mkl_lapack_dgetrf_omp_271__par_loop0' can not be used > when making > > > a shared object; recompile with -fPIC > > > ld: final link failed: Bad value > > > make[2]: [shared_linux] Error 1 (ignored) > > > > > > Details are in the attached male log file. > > > > > > Thanks for any help > > > > > > Henk > > > > > > > > > > > > > > From balay at mcs.anl.gov Mon Mar 26 11:14:01 2007 From: balay at mcs.anl.gov (Satish Balay) Date: Mon, 26 Mar 2007 11:14:01 -0500 (CDT) Subject: In-Reply-To: References: Message-ID: With tcsh shell - one has to escape the special chars. [with bash the command should work as is]. i.e --with-blas-lapack-lib=\[/usr/local/Cluster-Apps/intel/mkl/8.0/lib/em64t/libmkl_lapack64.so\,libmkl_def.so\,libguide.so\] Satish On Mon, 26 Mar 2007, SLIM H.A. wrote: > Thanks > > I tried > ./config/configure.py > --with-blas-lapack-lib=[/usr/local/Cluster-Apps/intel/mkl/8.0/lib/em64t/ > libmkl_lapack64.so,libmkl_def.so,libguide.so] > --with-vendor-compilers=intel --with-gnu-compilers=0 > > which gives: > ./config/configure.py: No match. > > I checked that the files libmkl_lapack64.so,libmkl_def.so,libguide.so > exist in /usr/local/Cluster-Apps/intel/mkl/8.0/lib/em64t > > However I am not familiar with the configure command. Did I use it > corrctly? > > Thanks > Henk > > > > -----Original Message----- > > From: owner-petsc-users at mcs.anl.gov > > [mailto:owner-petsc-users at mcs.anl.gov] On Behalf Of Satish Balay > > Sent: 26 March 2007 16:28 > > To: petsc-users at mcs.anl.gov > > Subject: Re: > > > > Not sure about the binutils stuff - but x86_64 linux > > complains about using non PIC code in a shared library. > > > > Currently PETSc configure is defaulting to .a version of MKL > > - which are incompatible with sharedlibraries. And I'm not > > sure what the correct library naems are for the shared > > version of MLK. You could try using the option: > > > > --with-blas-lapack-lib=[/usr/local/Cluster-Apps/intel/mkl/8.0/ > lib/em64t/libmkl_lapack32.so,libmkl_def.so,libguide.so] > > > > or just use --download-f-blas-lapack=1 > > > > Satish > > > > On Mon, 26 Mar 2007, Lisandro Dalcin wrote: > > > > > This seems to be a bug of 'binutils' in 64 bits machines. > > > > > > http://sources.redhat.com/bugzilla/show_bug.cgi?id=584 > > > > > > but apparently it was fixed in Feb 2005 > > > > > > http://sourceware.org/ml/binutils-cvs/2005-02/msg00011.html > > > > > > Can you upgrade your 'binutils' package and try what happens? > > > > > > > > > On 3/26/07, SLIM H.A. wrote: > > > > Dear users > > > > > > > > I want to build petsc with shared object libraries but get errors > > > > when building libpetscmat.so and libpetscksp.so I am > > using the intel > > > > compiler and intel's imkl for lapack/blas > > > > > > > > Configuration is: > > > > ./config/configure.py > > > > > > --with-blas-lapack-dir=/usr/local/Cluster-Apps/intel/mkl/8.0/lib/em6 > > > > 4t --with-vendor-compilers=intel --with-gnu-compilers=0 > > > > --with-shared > > > > > > > > The error is: > > > > ld: > > > > > > /usr/local/Cluster-Apps/intel/mkl/8.0/lib/em64t/libmkl_lapack.a(dget > > > > rf_o > > > > mp.o): relocation R_X86_64_PC32 against > > > > `_mkl_lapack_dgetrf_omp_271__par_loop0' can not be used > > when making > > > > a shared object; recompile with -fPIC > > > > ld: final link failed: Bad value > > > > make[2]: [shared_linux] Error 1 (ignored) > > > > > > > > Details are in the attached male log file. > > > > > > > > Thanks for any help > > > > > > > > Henk > > > > > > > > > > > > > > > > > > > > > > > From h.a.slim at durham.ac.uk Mon Mar 26 12:36:15 2007 From: h.a.slim at durham.ac.uk (SLIM H.A.) Date: Mon, 26 Mar 2007 18:36:15 +0100 Subject: Message-ID: Dear Satish This worked now. ./config/configure.py --with-blas-lapack-lib=\[/usr/local/Cluster-Apps/intel/mkl/8.0/lib/em64t /libmkl_lapack64.so,libmkl_def.so,libguide.so\] --with-vendor-compilers=intel --with-gnu-compilers=0 The tests were succesfull with up to two mpi processes. However when I look into the lib directory there are only static libraries: hamilton_dcl0has(296)>ls -la lib/linux-gnu-intel total 27248 drwx------ 2 dcl0has dcl 4096 Mar 26 17:39 ./ drwx------ 4 dcl0has dcl 4096 Mar 23 16:43 ../ -rw------- 1 dcl0has dcl 4394974 Mar 26 17:39 libpetsc.a -rw------- 1 dcl0has dcl 193378 Mar 26 17:39 libpetsccontrib.a -rw------- 1 dcl0has dcl 2612462 Mar 26 17:39 libpetscdm.a -rw------- 1 dcl0has dcl 4525610 Mar 26 17:39 libpetscksp.a -rw------- 1 dcl0has dcl 10905628 Mar 26 17:39 libpetscmat.a -rw------- 1 dcl0has dcl 1476668 Mar 26 17:39 libpetscsnes.a -rw------- 1 dcl0has dcl 685944 Mar 26 17:39 libpetscts.a -rw------- 1 dcl0has dcl 3025700 Mar 26 17:39 libpetscvec.a Regards henk > -----Original Message----- > From: owner-petsc-users at mcs.anl.gov > [mailto:owner-petsc-users at mcs.anl.gov] On Behalf Of Satish Balay > Sent: 26 March 2007 17:14 > To: petsc-users at mcs.anl.gov > Subject: RE: > > With tcsh shell - one has to escape the special chars. [with > bash the command should work as is]. > > i.e > > --with-blas-lapack-lib=\[/usr/local/Cluster-Apps/intel/mkl/8.0 > /lib/em64t/libmkl_lapack64.so\,libmkl_def.so\,libguide.so\] > > Satish > > > On Mon, 26 Mar 2007, SLIM H.A. wrote: > > > Thanks > > > > I tried > > ./config/configure.py > > > --with-blas-lapack-lib=[/usr/local/Cluster-Apps/intel/mkl/8.0/lib/em64 > > t/ libmkl_lapack64.so,libmkl_def.so,libguide.so] > > --with-vendor-compilers=intel --with-gnu-compilers=0 > > > > which gives: > > ./config/configure.py: No match. > > > > I checked that the files > libmkl_lapack64.so,libmkl_def.so,libguide.so > > exist in /usr/local/Cluster-Apps/intel/mkl/8.0/lib/em64t > > > > However I am not familiar with the configure command. Did I use it > > corrctly? > > > > Thanks > > Henk > > > > > > > -----Original Message----- > > > From: owner-petsc-users at mcs.anl.gov > > > [mailto:owner-petsc-users at mcs.anl.gov] On Behalf Of Satish Balay > > > Sent: 26 March 2007 16:28 > > > To: petsc-users at mcs.anl.gov > > > Subject: Re: > > > > > > Not sure about the binutils stuff - but x86_64 linux > complains about > > > using non PIC code in a shared library. > > > > > > Currently PETSc configure is defaulting to .a version of MKL > > > - which are incompatible with sharedlibraries. And I'm not sure > > > what the correct library naems are for the shared version > of MLK. > > > You could try using the option: > > > > > > --with-blas-lapack-lib=[/usr/local/Cluster-Apps/intel/mkl/8.0/ > > lib/em64t/libmkl_lapack32.so,libmkl_def.so,libguide.so] > > > > > > or just use --download-f-blas-lapack=1 > > > > > > Satish > > > > > > On Mon, 26 Mar 2007, Lisandro Dalcin wrote: > > > > > > > This seems to be a bug of 'binutils' in 64 bits machines. > > > > > > > > http://sources.redhat.com/bugzilla/show_bug.cgi?id=584 > > > > > > > > but apparently it was fixed in Feb 2005 > > > > > > > > http://sourceware.org/ml/binutils-cvs/2005-02/msg00011.html > > > > > > > > Can you upgrade your 'binutils' package and try what happens? > > > > > > > > > > > > On 3/26/07, SLIM H.A. wrote: > > > > > Dear users > > > > > > > > > > I want to build petsc with shared object libraries but get > > > > > errors when building libpetscmat.so and libpetscksp.so I am > > > using the intel > > > > > compiler and intel's imkl for lapack/blas > > > > > > > > > > Configuration is: > > > > > ./config/configure.py > > > > > > > > > --with-blas-lapack-dir=/usr/local/Cluster-Apps/intel/mkl/8.0/lib/em6 > > > > > 4t --with-vendor-compilers=intel --with-gnu-compilers=0 > > > > > --with-shared > > > > > > > > > > The error is: > > > > > ld: > > > > > > > > > /usr/local/Cluster-Apps/intel/mkl/8.0/lib/em64t/libmkl_lapack.a(dget > > > > > rf_o > > > > > mp.o): relocation R_X86_64_PC32 against > > > > > `_mkl_lapack_dgetrf_omp_271__par_loop0' can not be used > > > when making > > > > > a shared object; recompile with -fPIC > > > > > ld: final link failed: Bad value > > > > > make[2]: [shared_linux] Error 1 (ignored) > > > > > > > > > > Details are in the attached male log file. > > > > > > > > > > Thanks for any help > > > > > > > > > > Henk > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > From knepley at gmail.com Mon Mar 26 15:46:30 2007 From: knepley at gmail.com (Matthew Knepley) Date: Mon, 26 Mar 2007 15:46:30 -0500 Subject: In-Reply-To: References: Message-ID: If you want shared libraries, you need to use --with-shared and the tests have to pass. This kind of question belongs on petsc-maint at mcs.anl.gov. Thanks, Matt On 3/26/07, SLIM H.A. wrote: > > Dear Satish > > This worked now. > > ./config/configure.py > --with-blas-lapack-lib=\[/usr/local/Cluster-Apps/intel/mkl/8.0/lib/em64t > /libmkl_lapack64.so,libmkl_def.so,libguide.so\] > --with-vendor-compilers=intel --with-gnu-compilers=0 > > The tests were succesfull with up to two mpi processes. However when I > look into the lib directory there are only static libraries: > hamilton_dcl0has(296)>ls -la lib/linux-gnu-intel > total 27248 > drwx------ 2 dcl0has dcl 4096 Mar 26 17:39 ./ > drwx------ 4 dcl0has dcl 4096 Mar 23 16:43 ../ > -rw------- 1 dcl0has dcl 4394974 Mar 26 17:39 libpetsc.a > -rw------- 1 dcl0has dcl 193378 Mar 26 17:39 libpetsccontrib.a > -rw------- 1 dcl0has dcl 2612462 Mar 26 17:39 libpetscdm.a > -rw------- 1 dcl0has dcl 4525610 Mar 26 17:39 libpetscksp.a > -rw------- 1 dcl0has dcl 10905628 Mar 26 17:39 libpetscmat.a > -rw------- 1 dcl0has dcl 1476668 Mar 26 17:39 libpetscsnes.a > -rw------- 1 dcl0has dcl 685944 Mar 26 17:39 libpetscts.a > -rw------- 1 dcl0has dcl 3025700 Mar 26 17:39 libpetscvec.a > > Regards > > henk > > > > -----Original Message----- > > From: owner-petsc-users at mcs.anl.gov > > [mailto:owner-petsc-users at mcs.anl.gov] On Behalf Of Satish Balay > > Sent: 26 March 2007 17:14 > > To: petsc-users at mcs.anl.gov > > Subject: RE: > > > > With tcsh shell - one has to escape the special chars. [with > > bash the command should work as is]. > > > > i.e > > > > --with-blas-lapack-lib=\[/usr/local/Cluster-Apps/intel/mkl/8.0 > > /lib/em64t/libmkl_lapack64.so\,libmkl_def.so\,libguide.so\] > > > > Satish > > > > > > On Mon, 26 Mar 2007, SLIM H.A. wrote: > > > > > Thanks > > > > > > I tried > > > ./config/configure.py > > > > > --with-blas-lapack-lib=[/usr/local/Cluster-Apps/intel/mkl/8.0/lib/em64 > > > t/ libmkl_lapack64.so,libmkl_def.so,libguide.so] > > > --with-vendor-compilers=intel --with-gnu-compilers=0 > > > > > > which gives: > > > ./config/configure.py: No match. > > > > > > I checked that the files > > libmkl_lapack64.so,libmkl_def.so,libguide.so > > > exist in /usr/local/Cluster-Apps/intel/mkl/8.0/lib/em64t > > > > > > However I am not familiar with the configure command. Did I use it > > > corrctly? > > > > > > Thanks > > > Henk > > > > > > > > > > -----Original Message----- > > > > From: owner-petsc-users at mcs.anl.gov > > > > [mailto:owner-petsc-users at mcs.anl.gov] On Behalf Of Satish Balay > > > > Sent: 26 March 2007 16:28 > > > > To: petsc-users at mcs.anl.gov > > > > Subject: Re: > > > > > > > > Not sure about the binutils stuff - but x86_64 linux > > complains about > > > > using non PIC code in a shared library. > > > > > > > > Currently PETSc configure is defaulting to .a version of MKL > > > > - which are incompatible with sharedlibraries. And I'm not sure > > > > what the correct library naems are for the shared version > > of MLK. > > > > You could try using the option: > > > > > > > > --with-blas-lapack-lib=[/usr/local/Cluster-Apps/intel/mkl/8.0/ > > > lib/em64t/libmkl_lapack32.so,libmkl_def.so,libguide.so] > > > > > > > > or just use --download-f-blas-lapack=1 > > > > > > > > Satish > > > > > > > > On Mon, 26 Mar 2007, Lisandro Dalcin wrote: > > > > > > > > > This seems to be a bug of 'binutils' in 64 bits machines. > > > > > > > > > > http://sources.redhat.com/bugzilla/show_bug.cgi?id=584 > > > > > > > > > > but apparently it was fixed in Feb 2005 > > > > > > > > > > http://sourceware.org/ml/binutils-cvs/2005-02/msg00011.html > > > > > > > > > > Can you upgrade your 'binutils' package and try what happens? > > > > > > > > > > > > > > > On 3/26/07, SLIM H.A. wrote: > > > > > > Dear users > > > > > > > > > > > > I want to build petsc with shared object libraries but get > > > > > > errors when building libpetscmat.so and libpetscksp.so I am > > > > using the intel > > > > > > compiler and intel's imkl for lapack/blas > > > > > > > > > > > > Configuration is: > > > > > > ./config/configure.py > > > > > > > > > > > > --with-blas-lapack-dir=/usr/local/Cluster-Apps/intel/mkl/8.0/lib/em6 > > > > > > 4t --with-vendor-compilers=intel --with-gnu-compilers=0 > > > > > > --with-shared > > > > > > > > > > > > The error is: > > > > > > ld: > > > > > > > > > > > > /usr/local/Cluster-Apps/intel/mkl/8.0/lib/em64t/libmkl_lapack.a(dget > > > > > > rf_o > > > > > > mp.o): relocation R_X86_64_PC32 against > > > > > > `_mkl_lapack_dgetrf_omp_271__par_loop0' can not be used > > > > when making > > > > > > a shared object; recompile with -fPIC > > > > > > ld: final link failed: Bad value > > > > > > make[2]: [shared_linux] Error 1 (ignored) > > > > > > > > > > > > Details are in the attached male log file. > > > > > > > > > > > > Thanks for any help > > > > > > > > > > > > Henk > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- One trouble is that despite this system, anyone who reads journals widely and critically is forced to realize that there are scarcely any bars to eventual publication. There seems to be no study too fragmented, no hypothesis too trivial, no literature citation too biased or too egotistical, no design too warped, no methodology too bungled, no presentation of results too inaccurate, too obscure, and too contradictory, no analysis too self-serving, no argument too circular, no conclusions too trifling or too unjustified, and no grammar and syntax too offensive for a paper to end up in print. -- Drummond Rennie From diego.nehab at gmail.com Thu Mar 29 02:42:50 2007 From: diego.nehab at gmail.com (Diego Nehab) Date: Thu, 29 Mar 2007 03:42:50 -0400 Subject: Newbie question about synchronization Message-ID: Hi, I am very new to Petsc, and I am not very knowledgeable in numerical methods. Therefore, please forgive my ignorance. I need to solve sparse (at most 7 non-zero elements per row) overconstrained linear systems, in the least squares sense. The systems are very large (typically 3M x 500k), but have a reasonably strong diagonal component that apparently makes them solvable. In the past I have directly used the C version of C. C. Paige and M. A. Saunders LSQR library, with no problems. My interest in Petsc is the eternal quest for speed. That, I hope, could come in the form of better methods, a better matrix representation and, naturally, parallelism. 1) Am I right to assume the only solver that accepts overconstrained systems is the lsqr? So far, using only one process, everything is simple and works (it took me longer to compile and test MPI and Petsc then to write code that solves my problem :)). When I move to multiple processes, the solver still works, but I couldn't figure out how to select one of the processes to consolidate the solution vector and use it to save a file on disk. I always get an error of the form [0]PETSC ERROR: Argument out of range! [0]PETSC ERROR: Can only get local values, trying xxx! I assume I must bring the solution vector back from the other processes, right? 2) If so, how do I do this? As it turns out, the normal version of the system (A^t A x = A^t b) is still very sparse in my case. So I used MatMatMultTranspose and MatMultTranspose to produce the normal equations and solved the square system. Using ksp_type cg and the default settings for the ilu, the solver converges much faster than using lsqr, which is great. However, MatMatMultTranspose doesn't seem to like the mpiaij matrix format. 3) If I want to try the normal approach in multiple processes, do I have perform the product myself, or is there another function that does this for me? Thanks in advance, Diego. From knutert at stud.ntnu.no Thu Mar 29 03:24:14 2007 From: knutert at stud.ntnu.no (Knut Erik Teigen) Date: Thu, 29 Mar 2007 10:24:14 +0200 Subject: Newbie question about synchronization In-Reply-To: References: Message-ID: <1175156655.920.70.camel@iept0415.ivt.ntnu.no> On Thu, 2007-03-29 at 03:42 -0400, Diego Nehab wrote: > Hi, > > I am very new to Petsc, and I am not very knowledgeable in > numerical methods. Therefore, please forgive my ignorance. > > I need to solve sparse (at most 7 non-zero elements per row) > overconstrained linear systems, in the least squares sense. > The systems are very large (typically 3M x 500k), but have a > reasonably strong diagonal component that apparently makes > them solvable. > > In the past I have directly used the C version of > C. C. Paige and M. A. Saunders LSQR library, with no problems. > My interest in Petsc is the eternal quest for speed. That, > I hope, could come in the form of better methods, a better matrix > representation and, naturally, parallelism. > > 1) Am I right to assume the only solver that accepts > overconstrained systems is the lsqr? > > So far, using only one process, everything is simple and > works (it took me longer to compile and test MPI and Petsc > then to write code that solves my problem :)). > > When I move to multiple processes, the solver still works, but I > couldn't figure out how to select one of the processes to > consolidate the solution vector and use it to save a file on disk. > I always get an error of the form > > [0]PETSC ERROR: Argument out of range! > [0]PETSC ERROR: Can only get local values, trying xxx! > > I assume I must bring the solution vector back from the other > processes, right? > > 2) If so, how do I do this? If you only want to save the results to disk, you can use the VecView function. Just create a viewer, like e.g. PetscViewerAsciiOpen(PETSC_COMM_WORLD,"filename",&viewer) VecView(solution,viewer) You can also output in binary format using BinaryOpen instead. Check the chapter on Viewers in the manual. If you need to gather the results on one processor for further computations, I use standard MPI calls, like this(in Fortran): call VecGetArray(sol,sol_ptr,sol_i,ierr) ! gather solution on process 0 if (rank==0) then do i=low,high !copy local solution to global solution global_sol(i)=sol_ptr(sol_i+i) enddo do p=1,size-1 !recieve local solution from other processes call MPI_Recv(local_sol,loc_n,MPI_REAL,p,1,PETSC_COMM_WORLD,istat,mpierr) do i=1,loc_n !copy local part to correct position in global global_sol(i+high)=local_sol(i) high=high+1 enddo enddo else do j=1,loc_n local_sol(j)=sol_ptr(sol_i+j) enddo call MPI_Send(local_sol,loc_n,MPI_REAL,0,1,PETSC_COMM_WORLD,mpierr) endif call VecRestoreArray(sol,sol_ptr,sol_i,ierr) !copy global solution vector back to grid array if (rank==0) then do j=1,jmax do i=1,imax T(i,j)=global_sol((j-1)*imax+i) end do end do endif This is probably not the recommended way of doing things. I'm quite new at using PETSc myself, so if anyone has a better solution, please enlighten me! I should use PETSc data structures for everything, but I'm trying to integrate PETSc into already existing code, so it's not that easy to do. Regards, Knut Erik Teigen > > As it turns out, the normal version of the system (A^t A x = A^t b) > is still very sparse in my case. So I used MatMatMultTranspose > and MatMultTranspose to produce the normal equations and > solved the square system. > > Using ksp_type cg and the default settings for the ilu, the solver > converges much faster than using lsqr, which is great. However, > MatMatMultTranspose doesn't seem to like the mpiaij matrix format. > > 3) If I want to try the normal approach in multiple processes, do I > have perform > the product myself, or is there another function that does this for me? > > Thanks in advance, > Diego. > > From balay at mcs.anl.gov Thu Mar 29 09:51:24 2007 From: balay at mcs.anl.gov (Satish Balay) Date: Thu, 29 Mar 2007 09:51:24 -0500 (CDT) Subject: Newbie question about synchronization In-Reply-To: <1175156655.920.70.camel@iept0415.ivt.ntnu.no> References: <1175156655.920.70.camel@iept0415.ivt.ntnu.no> Message-ID: On Thu, 29 Mar 2007, Knut Erik Teigen wrote: > On Thu, 2007-03-29 at 03:42 -0400, Diego Nehab wrote: > > So far, using only one process, everything is simple and > > works (it took me longer to compile and test MPI and Petsc > > then to write code that solves my problem :)). > > > > When I move to multiple processes, the solver still works, but I > > couldn't figure out how to select one of the processes to > > consolidate the solution vector and use it to save a file on disk. > > I always get an error of the form > > > > [0]PETSC ERROR: Argument out of range! > > [0]PETSC ERROR: Can only get local values, trying xxx! > > > > I assume I must bring the solution vector back from the other > > processes, right? > > > > 2) If so, how do I do this? > If you only want to save the results to disk, you can use the VecView > function. Just create a viewer, like e.g. > PetscViewerAsciiOpen(PETSC_COMM_WORLD,"filename",&viewer) > VecView(solution,viewer) > You can also output in binary format using BinaryOpen instead. Check the > chapter on Viewers in the manual. > If you need to gather the results on one processor for further > computations, I use standard MPI calls, like this(in Fortran): > > call VecGetArray(sol,sol_ptr,sol_i,ierr) > > ! gather solution on process 0 > if (rank==0) then > do i=low,high !copy local solution to global solution > global_sol(i)=sol_ptr(sol_i+i) > enddo > do p=1,size-1 !recieve local solution from other processes > call > MPI_Recv(local_sol,loc_n,MPI_REAL,p,1,PETSC_COMM_WORLD,istat,mpierr) > do i=1,loc_n !copy local part to correct position in global > global_sol(i+high)=local_sol(i) > high=high+1 > enddo > enddo > else > do j=1,loc_n > local_sol(j)=sol_ptr(sol_i+j) > enddo > call MPI_Send(local_sol,loc_n,MPI_REAL,0,1,PETSC_COMM_WORLD,mpierr) > endif > > call VecRestoreArray(sol,sol_ptr,sol_i,ierr) > > !copy global solution vector back to grid array > if (rank==0) then > do j=1,jmax > do i=1,imax > T(i,j)=global_sol((j-1)*imax+i) > end do > end do > endif > > This is probably not the recommended way of doing things. I'm quite new > at using PETSc myself, so if anyone has a better solution, please > enlighten me! I should use PETSc data structures for everything, but I'm > trying to integrate PETSc into already existing code, so it's not that > easy to do. VecView() is the correct thing to do for dumping the vec into a file. However - if you need all the values of the vec on 1 proc - then the use VecScatterCreateToZero() http://www-unix.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/Vec/VecScatterCreateToZero.html Satish From h.a.slim at durham.ac.uk Thu Mar 29 11:43:32 2007 From: h.a.slim at durham.ac.uk (SLIM H.A.) Date: Thu, 29 Mar 2007 17:43:32 +0100 Subject: make test fails Message-ID: Dear all After building pestc-2.3.0 I get the following errors when runing the tests: >make test Running test examples to verify correct installation --------------Error detected during compile or link!----------------------- See http://www.mcs.anl.gov/petsc/petsc-2/documentation/troubleshooting.html *********************Warning************************* Your PETSC_DIR may not match the directory you are in PETSC_DIR /data/hamilton/drk1has/_libs/petsc/petsc-2.3.0 Current directory /home/stevens/staff/dcl0has/hamilton/_libs/petsc/petsc-2.3.0/src/snes/ex amples/tutorials ****************************************************** mpicc -c -fPIC -g -I/data/hamilton/drk1has/_libs/petsc/petsc-2.3.0 -I/data/hamilton/drk1has/_libs/petsc/petsc-2.3.0/bmake/linux-gnu-intel -I/data/hamilton/drk1has/_libs/petsc/petsc-2.3.0/include -D__SDIR__="src/snes/examples/tutorials/" ex19.c mpicc -fPIC -g -o ex19 ex19.o -Wl,-rpath,/data/hamilton/drk1has/_libs/petsc/petsc-2.3.0/lib/linux-gnu- intel -L/data/hamilton/drk1has/_libs/petsc/petsc-2.3.0/lib/linux-gnu-intel -lpetscsnes -lpetscksp -lpetscdm -lpetscmat -lpetscvec -lpetsc -lnsl -lrt -Wl,-rpath,/usr/local/Cluster-Apps/intel/mkl/8.0/lib/em64t -L/usr/local/Cluster-Apps/intel/mkl/8.0/lib/em64t -lmkl_lapack64 -lmkl_def -lguide -L/usr/local/Cluster-Apps/mpich/ge/intel/64/1.2.7/lib/shared -L/usr/local/Cluster-Apps/mpich/ge/intel/64/1.2.7/lib -lmpichf90 -lmpichfarg -lmpich -lpthread -lrt -Wl,-rpath,"/usr/local/Cluster-Apps/mpich/ge/intel/64/1.2.7/lib/shared" -L"/usr/local/Cluster-Apps/mpich/ge/intel/64/1.2.7/lib/shared" -L"/usr/local/Cluster-Apps/mpich/ge/intel/64/1.2.7/lib" -L/usr/local/Cluster-Apps/intel/fce/9.0/lib -L/usr/lib64/gcc/x86_64-suse-linux/4.0.2/ -L/usr/lib64/gcc/x86_64-suse-linux/4.0.2/../../../../lib64 -lsvml -lifport -lifcore -limf -lm -lirc -lgcc_s -lirc_s -ldl -Wl,-rpath,/usr/local/Cluster-Apps/mpich/ge/intel/64/1.2.7/lib/shared -L/usr/local/Cluster-Apps/mpich/ge/intel/64/1.2.7/lib/shared -L/usr/local/Cluster-Apps/mpich/ge/intel/64/1.2.7/lib -lm /bin/rm -f ex19.o C/C++ example src/snes/examples/tutorials/ex19 run successfully with 1 MPI process Possible error running C/C++ src/snes/examples/tutorials/ex19 with 2 MPI processes See http://www.mcs.anl.gov/petsc/petsc-as/documentation/troubleshooting.html /home/stevens/staff/dcl0has/hamilton/_libs/petsc/petsc-2.3.0/src/snes/ex amples/tutorials/./ex19: error while loading shared libraries: libsvml.so: cannot open shared object file: No such file or directory p0_26282: p4_error: Child process exited while making connection to remote process on node001: 0 p0_26282: (2.242188) net_send: could not write to fd=4, errno = 32 Fortran example src/snes/examples/tutorials/ex5f run successfully with 1 MPI process The library libsvml.so in the line "libsvml.so: cannot open shared object file: No such file or directory" does actually exist as /usr/local/Cluster-Apps/intel/cce/9.0/lib/libsvml.so ex19 is a c file, so I would expect the cc compiler library path in the mpicc linking stage, similar to -L/usr/local/Cluster-Apps/intel/fce/9.0/lib Thanks Henk From diego.nehab at gmail.com Fri Mar 30 00:48:35 2007 From: diego.nehab at gmail.com (Diego Nehab) Date: Fri, 30 Mar 2007 01:48:35 -0400 Subject: Newbie question about synchronization In-Reply-To: References: <1175156655.920.70.camel@iept0415.ivt.ntnu.no> Message-ID: Hi, > VecView() is the correct thing to do for dumping the vec into a file. > However - if you need all the values of the vec on 1 proc - then the > use VecScatterCreateToZero() Thanks. This VecScatterCreateToZero() worked like a charm. Regards, Diego. From knutert at stud.ntnu.no Fri Mar 30 01:18:22 2007 From: knutert at stud.ntnu.no (Knut Erik Teigen) Date: Fri, 30 Mar 2007 08:18:22 +0200 Subject: Newbie question about synchronization In-Reply-To: References: <1175156655.920.70.camel@iept0415.ivt.ntnu.no> Message-ID: <1175235502.920.76.camel@iept0415.ivt.ntnu.no> I knew there had to be a simpler solution than mine. Thanks a lot, Satish! On Thu, 2007-03-29 at 09:51 -0500, Satish Balay wrote: > On Thu, 29 Mar 2007, Knut Erik Teigen wrote: > > > On Thu, 2007-03-29 at 03:42 -0400, Diego Nehab wrote: > > > > So far, using only one process, everything is simple and > > > works (it took me longer to compile and test MPI and Petsc > > > then to write code that solves my problem :)). > > > > > > When I move to multiple processes, the solver still works, but I > > > couldn't figure out how to select one of the processes to > > > consolidate the solution vector and use it to save a file on disk. > > > I always get an error of the form > > > > > > [0]PETSC ERROR: Argument out of range! > > > [0]PETSC ERROR: Can only get local values, trying xxx! > > > > > > I assume I must bring the solution vector back from the other > > > processes, right? > > > > > > 2) If so, how do I do this? > > If you only want to save the results to disk, you can use the VecView > > function. Just create a viewer, like e.g. > > PetscViewerAsciiOpen(PETSC_COMM_WORLD,"filename",&viewer) > > VecView(solution,viewer) > > You can also output in binary format using BinaryOpen instead. Check the > > chapter on Viewers in the manual. > > If you need to gather the results on one processor for further > > computations, I use standard MPI calls, like this(in Fortran): > > > > call VecGetArray(sol,sol_ptr,sol_i,ierr) > > > > ! gather solution on process 0 > > if (rank==0) then > > do i=low,high !copy local solution to global solution > > global_sol(i)=sol_ptr(sol_i+i) > > enddo > > do p=1,size-1 !recieve local solution from other processes > > call > > MPI_Recv(local_sol,loc_n,MPI_REAL,p,1,PETSC_COMM_WORLD,istat,mpierr) > > do i=1,loc_n !copy local part to correct position in global > > global_sol(i+high)=local_sol(i) > > high=high+1 > > enddo > > enddo > > else > > do j=1,loc_n > > local_sol(j)=sol_ptr(sol_i+j) > > enddo > > call MPI_Send(local_sol,loc_n,MPI_REAL,0,1,PETSC_COMM_WORLD,mpierr) > > endif > > > > call VecRestoreArray(sol,sol_ptr,sol_i,ierr) > > > > !copy global solution vector back to grid array > > if (rank==0) then > > do j=1,jmax > > do i=1,imax > > T(i,j)=global_sol((j-1)*imax+i) > > end do > > end do > > endif > > > > This is probably not the recommended way of doing things. I'm quite new > > at using PETSc myself, so if anyone has a better solution, please > > enlighten me! I should use PETSc data structures for everything, but I'm > > trying to integrate PETSc into already existing code, so it's not that > > easy to do. > > VecView() is the correct thing to do for dumping the vec into a file. > However - if you need all the values of the vec on 1 proc - then the > use VecScatterCreateToZero() > > http://www-unix.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/Vec/VecScatterCreateToZero.html > > Satish > > From yaron at oak-research.com Sat Mar 31 01:10:17 2007 From: yaron at oak-research.com (yaron at oak-research.com) Date: Fri, 30 Mar 2007 22:10:17 -0800 Subject: Getting an underlying 3-vector description from a sparse,local, matrix Message-ID: <20070331061017.8134.qmail@s402.sureserver.com> Hi Petsc-users I'd like to get the 3-vector description (NZ values,Indexs,Pointers) from a local sparse matrix- I need those in order to interface a Petsc matrix object with another software package. Any idea on how to do that ? I'm sure there are already some methods that give access to the underlying arrays... Thanks much Yaron -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Sat Mar 31 09:20:19 2007 From: bsmith at mcs.anl.gov (Barry Smith) Date: Sat, 31 Mar 2007 09:20:19 -0500 (CDT) Subject: Getting an underlying 3-vector description from a sparse,local, matrix In-Reply-To: <20070331061017.8134.qmail@s402.sureserver.com> References: <20070331061017.8134.qmail@s402.sureserver.com> Message-ID: MatGetArray() and MatGetRowIJ() together. On Fri, 30 Mar 2007, yaron at oak-research.com wrote: > Hi Petsc-users > I'd like to get the 3-vector description (NZ values,Indexs,Pointers) from > a local sparse matrix- I need those in order to interface a Petsc matrix > object with another software package. > Any idea on how to do that ? I'm sure there are already some methods that > give access to the underlying arrays... > > Thanks much > Yaron > From wsubber at connect.carleton.ca Sat Mar 31 16:27:17 2007 From: wsubber at connect.carleton.ca (Waad Subber) Date: Sat, 31 Mar 2007 17:27:17 -0400 (EDT) Subject: DDM with PETSc Message-ID: <591409.1175376437413.JavaMail.wsubber@connect.carleton.ca> Hello everyone: I am new to PETSC. Reading the tutorials, I understand that PETSC supports domain decomposition with additive Schwartz and iterative substructuring (balancing Neumann-Neumann). I am looking for some example codes involving these domain decomposition methods for 1D and 2D PDE (preferable linear PDE) so that I can get started. Can anyone kindly point me to the right place where I can find them. Thanks :) waad